2024-09-18T10:01:31.544 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-18T10:01:31.564 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-18T10:01:31.658 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/skanta-2024-09-18_09:13:00-rados-wip-bharath11-testing-2024-09-18-1115-quincy-distro-default-smithi/7910845 branch: wip-bharath11-testing-2024-09-18-1115-quincy description: rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/osd-dispatch-delay objectstore/bluestore-bitmap rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} email: skanta@redhat.com first_in_suite: false job_id: '7910845' kernel: kdb: 1 sha1: distro ktype: distro last_in_suite: false machine_type: smithi name: skanta-2024-09-18_09:13:00-rados-wip-bharath11-testing-2024-09-18-1115-quincy-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 10 os_type: ubuntu os_version: '20.04' overrides: admin_socket: branch: wip-bharath11-testing-2024-09-18-1115-quincy ceph: conf: global: mon election default strategy: 1 osd debug inject dispatch delay duration: 0.1 osd debug inject dispatch delay probability: 0.1 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon scrub interval: 300 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true bluestore zero block detection: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd debug reject backfill probability: 0.3 osd debug verify cached snaps: true osd debug verify missing on start: true osd failsafe full ratio: 0.95 osd max backfills: 3 osd max markdown count: 1000 osd mclock override recovery settings: true osd mclock profile: high_recovery_ops osd objectstore: bluestore osd op queue: debug_random osd op queue cut off: debug_random osd scrub max interval: 120 osd scrub min interval: 60 osd shutdown pgref assert: true flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - but it is still running - objects unfound and apparently lost - slow request - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - mons down - mon down - out of quorum sha1: 7061ea6ca50908d104e9f3979dc615e118d9e153 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 7061ea6ca50908d104e9f3979dc615e118d9e153 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-bharath11-testing-2024-09-18-1115-quincy sha1: 7061ea6ca50908d104e9f3979dc615e118d9e153 owner: scheduled_skanta@teuthology priority: 99 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.y - osd.0 - osd.4 - osd.8 - osd.12 - - mon.b - osd.1 - osd.5 - osd.9 - osd.13 - - mon.c - osd.2 - osd.6 - osd.10 - osd.14 - - mgr.x - osd.3 - osd.7 - osd.11 - osd.15 - client.0 seed: 4631 sha1: 7061ea6ca50908d104e9f3979dc615e118d9e153 sleep_before_teardown: 0 subset: 111/120000 suite: rados suite_branch: wip-bharath11-testing-2024-09-18-1115-quincy suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_7061ea6ca50908d104e9f3979dc615e118d9e153/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 7061ea6ca50908d104e9f3979dc615e118d9e153 targets: smithi029.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM= smithi136.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM= smithi154.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM= smithi189.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM= tasks: - install: null - ceph: conf: osd: debug monc: 20 - thrashosds: aggressive_pg_num_changes: false chance_pgnum_grow: 1 chance_pgnum_shrink: 1 chance_pgpnum_fix: 1 min_in: 8 timeout: 1200 - rados: clients: - client.0 ec_pool: true erasure_code_profile: c: 2 crush-failure-domain: osd k: 4 m: 3 name: shecprofile plugin: shec objects: 50 op_weights: append: 100 copy_from: 50 delete: 50 read: 100 rmattr: 25 rollback: 50 setattr: 25 snap_create: 50 snap_remove: 50 write: 0 ops: 400 write_append_excl: false teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-18_09:13:00 tube: smithi user: skanta verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-18T10:01:31.658 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_7061ea6ca50908d104e9f3979dc615e118d9e153/qa; will attempt to use it 2024-09-18T10:01:31.659 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_7061ea6ca50908d104e9f3979dc615e118d9e153/qa/tasks 2024-09-18T10:01:31.659 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-18T10:01:31.661 INFO:teuthology.task.internal:Checking packages... 2024-09-18T10:01:31.686 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '7061ea6ca50908d104e9f3979dc615e118d9e153' 2024-09-18T10:01:31.686 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-18T10:01:31.686 INFO:teuthology.packaging:ref: None 2024-09-18T10:01:31.686 INFO:teuthology.packaging:tag: None 2024-09-18T10:01:31.686 INFO:teuthology.packaging:branch: wip-bharath11-testing-2024-09-18-1115-quincy 2024-09-18T10:01:31.686 INFO:teuthology.packaging:sha1: 7061ea6ca50908d104e9f3979dc615e118d9e153 2024-09-18T10:01:31.686 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&ref=wip-bharath11-testing-2024-09-18-1115-quincy 2024-09-18T10:01:31.827 INFO:teuthology.task.internal:Found packages for ceph version 17.2.7-1645-g7061ea6c-1focal 2024-09-18T10:01:31.829 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-18T10:01:31.836 INFO:teuthology.task.internal:no buildpackages task found 2024-09-18T10:01:31.836 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-18T10:01:31.845 INFO:teuthology.task.internal:Saving configuration 2024-09-18T10:01:31.862 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-18T10:01:31.869 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-18T10:01:31.895 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi029.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/skanta-2024-09-18_09:13:00-rados-wip-bharath11-testing-2024-09-18-1115-quincy-distro-default-smithi/7910845', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-18 09:53:49.719349', 'locked_by': 'scheduled_skanta@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGETOy3xONfbx7kUFCNiUtRLNfkcrcEsWzPdq6Np/6Bw'} 2024-09-18T10:01:31.920 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi136.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/skanta-2024-09-18_09:13:00-rados-wip-bharath11-testing-2024-09-18-1115-quincy-distro-default-smithi/7910845', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-18 09:53:49.722344', 'locked_by': 'scheduled_skanta@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGETOy3xONfbx7kUFCNiUtRLNfkcrcEsWzPdq6Np/6Bw'} 2024-09-18T10:01:31.940 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi154.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/skanta-2024-09-18_09:13:00-rados-wip-bharath11-testing-2024-09-18-1115-quincy-distro-default-smithi/7910845', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-18 09:53:49.721141', 'locked_by': 'scheduled_skanta@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGETOy3xONfbx7kUFCNiUtRLNfkcrcEsWzPdq6Np/6Bw'} 2024-09-18T10:01:31.965 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi189.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/skanta-2024-09-18_09:13:00-rados-wip-bharath11-testing-2024-09-18-1115-quincy-distro-default-smithi/7910845', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-18 09:53:49.717915', 'locked_by': 'scheduled_skanta@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGETOy3xONfbx7kUFCNiUtRLNfkcrcEsWzPdq6Np/6Bw'} 2024-09-18T10:01:31.965 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-18T10:01:31.971 INFO:teuthology.task.internal:roles: ubuntu@smithi029.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'osd.0', 'osd.4', 'osd.8', 'osd.12'] 2024-09-18T10:01:31.971 INFO:teuthology.task.internal:roles: ubuntu@smithi136.front.sepia.ceph.com - ['mon.b', 'osd.1', 'osd.5', 'osd.9', 'osd.13'] 2024-09-18T10:01:31.971 INFO:teuthology.task.internal:roles: ubuntu@smithi154.front.sepia.ceph.com - ['mon.c', 'osd.2', 'osd.6', 'osd.10', 'osd.14'] 2024-09-18T10:01:31.972 INFO:teuthology.task.internal:roles: ubuntu@smithi189.front.sepia.ceph.com - ['mgr.x', 'osd.3', 'osd.7', 'osd.11', 'osd.15', 'client.0'] 2024-09-18T10:01:31.972 INFO:teuthology.run_tasks:Running task console_log... 2024-09-18T10:01:32.097 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fc8e4a93ac0>, signals=[15]) 2024-09-18T10:01:32.098 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-18T10:01:32.104 INFO:teuthology.task.internal:Opening connections... 2024-09-18T10:01:32.104 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi029.front.sepia.ceph.com 2024-09-18T10:01:32.105 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi029.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-18T10:01:32.196 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi136.front.sepia.ceph.com 2024-09-18T10:01:32.197 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-18T10:01:32.285 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi154.front.sepia.ceph.com 2024-09-18T10:01:32.286 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-18T10:01:32.374 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi189.front.sepia.ceph.com 2024-09-18T10:01:32.375 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi189.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-18T10:01:32.477 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-18T10:01:32.484 DEBUG:teuthology.orchestra.run.smithi029:> uname -m 2024-09-18T10:01:32.491 INFO:teuthology.orchestra.run.smithi029.stdout:x86_64 2024-09-18T10:01:32.491 DEBUG:teuthology.orchestra.run.smithi029:> cat /etc/os-release 2024-09-18T10:01:32.544 INFO:teuthology.orchestra.run.smithi029.stdout:NAME="Ubuntu" 2024-09-18T10:01:32.545 INFO:teuthology.orchestra.run.smithi029.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2024-09-18T10:01:32.545 INFO:teuthology.orchestra.run.smithi029.stdout:ID=ubuntu 2024-09-18T10:01:32.545 INFO:teuthology.orchestra.run.smithi029.stdout:ID_LIKE=debian 2024-09-18T10:01:32.545 INFO:teuthology.orchestra.run.smithi029.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2024-09-18T10:01:32.545 INFO:teuthology.orchestra.run.smithi029.stdout:VERSION_ID="20.04" 2024-09-18T10:01:32.545 INFO:teuthology.orchestra.run.smithi029.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-18T10:01:32.545 INFO:teuthology.orchestra.run.smithi029.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-18T10:01:32.545 INFO:teuthology.orchestra.run.smithi029.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-18T10:01:32.545 INFO:teuthology.orchestra.run.smithi029.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-18T10:01:32.545 INFO:teuthology.orchestra.run.smithi029.stdout:VERSION_CODENAME=focal 2024-09-18T10:01:32.545 INFO:teuthology.orchestra.run.smithi029.stdout:UBUNTU_CODENAME=focal 2024-09-18T10:01:32.546 INFO:teuthology.lock.ops:Updating smithi029.front.sepia.ceph.com on lock server 2024-09-18T10:01:32.569 DEBUG:teuthology.orchestra.run.smithi136:> uname -m 2024-09-18T10:01:32.575 INFO:teuthology.orchestra.run.smithi136.stdout:x86_64 2024-09-18T10:01:32.576 DEBUG:teuthology.orchestra.run.smithi136:> cat /etc/os-release 2024-09-18T10:01:32.625 INFO:teuthology.orchestra.run.smithi136.stdout:NAME="Ubuntu" 2024-09-18T10:01:32.626 INFO:teuthology.orchestra.run.smithi136.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2024-09-18T10:01:32.626 INFO:teuthology.orchestra.run.smithi136.stdout:ID=ubuntu 2024-09-18T10:01:32.626 INFO:teuthology.orchestra.run.smithi136.stdout:ID_LIKE=debian 2024-09-18T10:01:32.626 INFO:teuthology.orchestra.run.smithi136.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2024-09-18T10:01:32.626 INFO:teuthology.orchestra.run.smithi136.stdout:VERSION_ID="20.04" 2024-09-18T10:01:32.626 INFO:teuthology.orchestra.run.smithi136.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-18T10:01:32.626 INFO:teuthology.orchestra.run.smithi136.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-18T10:01:32.626 INFO:teuthology.orchestra.run.smithi136.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-18T10:01:32.626 INFO:teuthology.orchestra.run.smithi136.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-18T10:01:32.627 INFO:teuthology.orchestra.run.smithi136.stdout:VERSION_CODENAME=focal 2024-09-18T10:01:32.627 INFO:teuthology.orchestra.run.smithi136.stdout:UBUNTU_CODENAME=focal 2024-09-18T10:01:32.627 INFO:teuthology.lock.ops:Updating smithi136.front.sepia.ceph.com on lock server 2024-09-18T10:01:32.652 DEBUG:teuthology.orchestra.run.smithi154:> uname -m 2024-09-18T10:01:32.661 INFO:teuthology.orchestra.run.smithi154.stdout:x86_64 2024-09-18T10:01:32.661 DEBUG:teuthology.orchestra.run.smithi154:> cat /etc/os-release 2024-09-18T10:01:32.709 INFO:teuthology.orchestra.run.smithi154.stdout:NAME="Ubuntu" 2024-09-18T10:01:32.709 INFO:teuthology.orchestra.run.smithi154.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2024-09-18T10:01:32.710 INFO:teuthology.orchestra.run.smithi154.stdout:ID=ubuntu 2024-09-18T10:01:32.710 INFO:teuthology.orchestra.run.smithi154.stdout:ID_LIKE=debian 2024-09-18T10:01:32.710 INFO:teuthology.orchestra.run.smithi154.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2024-09-18T10:01:32.710 INFO:teuthology.orchestra.run.smithi154.stdout:VERSION_ID="20.04" 2024-09-18T10:01:32.710 INFO:teuthology.orchestra.run.smithi154.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-18T10:01:32.710 INFO:teuthology.orchestra.run.smithi154.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-18T10:01:32.710 INFO:teuthology.orchestra.run.smithi154.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-18T10:01:32.710 INFO:teuthology.orchestra.run.smithi154.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-18T10:01:32.710 INFO:teuthology.orchestra.run.smithi154.stdout:VERSION_CODENAME=focal 2024-09-18T10:01:32.710 INFO:teuthology.orchestra.run.smithi154.stdout:UBUNTU_CODENAME=focal 2024-09-18T10:01:32.710 INFO:teuthology.lock.ops:Updating smithi154.front.sepia.ceph.com on lock server 2024-09-18T10:01:32.735 DEBUG:teuthology.orchestra.run.smithi189:> uname -m 2024-09-18T10:01:32.742 INFO:teuthology.orchestra.run.smithi189.stdout:x86_64 2024-09-18T10:01:32.742 DEBUG:teuthology.orchestra.run.smithi189:> cat /etc/os-release 2024-09-18T10:01:32.791 INFO:teuthology.orchestra.run.smithi189.stdout:NAME="Ubuntu" 2024-09-18T10:01:32.791 INFO:teuthology.orchestra.run.smithi189.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2024-09-18T10:01:32.791 INFO:teuthology.orchestra.run.smithi189.stdout:ID=ubuntu 2024-09-18T10:01:32.791 INFO:teuthology.orchestra.run.smithi189.stdout:ID_LIKE=debian 2024-09-18T10:01:32.791 INFO:teuthology.orchestra.run.smithi189.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2024-09-18T10:01:32.791 INFO:teuthology.orchestra.run.smithi189.stdout:VERSION_ID="20.04" 2024-09-18T10:01:32.791 INFO:teuthology.orchestra.run.smithi189.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-18T10:01:32.791 INFO:teuthology.orchestra.run.smithi189.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-18T10:01:32.791 INFO:teuthology.orchestra.run.smithi189.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-18T10:01:32.791 INFO:teuthology.orchestra.run.smithi189.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-18T10:01:32.792 INFO:teuthology.orchestra.run.smithi189.stdout:VERSION_CODENAME=focal 2024-09-18T10:01:32.792 INFO:teuthology.orchestra.run.smithi189.stdout:UBUNTU_CODENAME=focal 2024-09-18T10:01:32.792 INFO:teuthology.lock.ops:Updating smithi189.front.sepia.ceph.com on lock server 2024-09-18T10:01:32.818 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-18T10:01:32.828 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-18T10:01:32.834 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-18T10:01:32.834 DEBUG:teuthology.orchestra.run.smithi029:> test '!' -e /home/ubuntu/cephtest 2024-09-18T10:01:32.837 DEBUG:teuthology.orchestra.run.smithi136:> test '!' -e /home/ubuntu/cephtest 2024-09-18T10:01:32.839 DEBUG:teuthology.orchestra.run.smithi154:> test '!' -e /home/ubuntu/cephtest 2024-09-18T10:01:32.841 DEBUG:teuthology.orchestra.run.smithi189:> test '!' -e /home/ubuntu/cephtest 2024-09-18T10:01:32.847 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-18T10:01:32.852 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-18T10:01:32.852 DEBUG:teuthology.orchestra.run.smithi029:> test -z $(ls -A /var/lib/ceph) 2024-09-18T10:01:32.854 DEBUG:teuthology.orchestra.run.smithi136:> test -z $(ls -A /var/lib/ceph) 2024-09-18T10:01:32.856 DEBUG:teuthology.orchestra.run.smithi154:> test -z $(ls -A /var/lib/ceph) 2024-09-18T10:01:32.859 DEBUG:teuthology.orchestra.run.smithi189:> test -z $(ls -A /var/lib/ceph) 2024-09-18T10:01:32.879 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-18T10:01:32.976 INFO:teuthology.run_tasks:Running task kernel... 2024-09-18T10:01:32.990 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-18T10:01:32.990 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}, 'mon.c': {'kdb': 1, 'sha1': 'distro'}, 'mgr.x': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-18T10:01:32.991 DEBUG:teuthology.orchestra.run.smithi029:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-18T10:01:32.991 DEBUG:teuthology.orchestra.run.smithi136:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-18T10:01:32.992 DEBUG:teuthology.orchestra.run.smithi154:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-18T10:01:32.992 DEBUG:teuthology.orchestra.run.smithi189:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-18T10:01:32.998 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-18T10:01:32.998 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-18T10:01:32.999 DEBUG:teuthology.orchestra.run.smithi154:> uname -r 2024-09-18T10:01:32.999 DEBUG:teuthology.orchestra.run.smithi136:> uname -r 2024-09-18T10:01:33.000 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-18T10:01:33.000 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-18T10:01:33.001 DEBUG:teuthology.orchestra.run.smithi189:> uname -r 2024-09-18T10:01:33.002 DEBUG:teuthology.orchestra.run.smithi029:> uname -r 2024-09-18T10:01:33.006 INFO:teuthology.orchestra.run.smithi154.stdout:5.4.0-124-generic 2024-09-18T10:01:33.006 INFO:teuthology.task.kernel:Running kernel on smithi154: 5.4.0-124-generic 2024-09-18T10:01:33.007 DEBUG:teuthology.orchestra.run.smithi154:> sudo apt-get clean 2024-09-18T10:01:33.009 INFO:teuthology.orchestra.run.smithi189.stdout:5.4.0-124-generic 2024-09-18T10:01:33.009 INFO:teuthology.orchestra.run.smithi136.stdout:5.4.0-124-generic 2024-09-18T10:01:33.009 INFO:teuthology.task.kernel:Running kernel on smithi189: 5.4.0-124-generic 2024-09-18T10:01:33.009 DEBUG:teuthology.orchestra.run.smithi189:> sudo apt-get clean 2024-09-18T10:01:33.009 INFO:teuthology.task.kernel:Running kernel on smithi136: 5.4.0-124-generic 2024-09-18T10:01:33.010 DEBUG:teuthology.orchestra.run.smithi136:> sudo apt-get clean 2024-09-18T10:01:33.011 INFO:teuthology.orchestra.run.smithi029.stdout:5.4.0-124-generic 2024-09-18T10:01:33.011 INFO:teuthology.task.kernel:Running kernel on smithi029: 5.4.0-124-generic 2024-09-18T10:01:33.011 DEBUG:teuthology.orchestra.run.smithi029:> sudo apt-get clean 2024-09-18T10:01:33.145 DEBUG:teuthology.orchestra.run.smithi136:> sudo apt-get update 2024-09-18T10:01:33.155 DEBUG:teuthology.orchestra.run.smithi154:> sudo apt-get update 2024-09-18T10:01:33.164 DEBUG:teuthology.orchestra.run.smithi189:> sudo apt-get update 2024-09-18T10:01:33.193 DEBUG:teuthology.orchestra.run.smithi029:> sudo apt-get update 2024-09-18T10:01:33.315 INFO:teuthology.orchestra.run.smithi154.stdout:Get:1 http://security.ubuntu.com/ubuntu focal-security InRelease [128 kB] 2024-09-18T10:01:33.351 INFO:teuthology.orchestra.run.smithi029.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2024-09-18T10:01:33.355 INFO:teuthology.orchestra.run.smithi189.stdout:Get:1 http://security.ubuntu.com/ubuntu focal-security InRelease [128 kB] 2024-09-18T10:01:33.367 INFO:teuthology.orchestra.run.smithi029.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease [128 kB] 2024-09-18T10:01:33.422 INFO:teuthology.orchestra.run.smithi154.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal InRelease 2024-09-18T10:01:33.422 INFO:teuthology.orchestra.run.smithi029.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-backports InRelease [128 kB] 2024-09-18T10:01:33.448 INFO:teuthology.orchestra.run.smithi189.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal InRelease 2024-09-18T10:01:33.448 INFO:teuthology.orchestra.run.smithi136.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2024-09-18T10:01:33.448 INFO:teuthology.orchestra.run.smithi136.stdout:Get:2 http://security.ubuntu.com/ubuntu focal-security InRelease [128 kB] 2024-09-18T10:01:33.456 INFO:teuthology.orchestra.run.smithi029.stdout:Get:4 http://security.ubuntu.com/ubuntu focal-security InRelease [128 kB] 2024-09-18T10:01:33.497 INFO:teuthology.orchestra.run.smithi136.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease [128 kB] 2024-09-18T10:01:33.504 INFO:teuthology.orchestra.run.smithi154.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease [128 kB] 2024-09-18T10:01:33.530 INFO:teuthology.orchestra.run.smithi189.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease [128 kB] 2024-09-18T10:01:33.621 INFO:teuthology.orchestra.run.smithi154.stdout:Get:4 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [3,223 kB] 2024-09-18T10:01:33.629 INFO:teuthology.orchestra.run.smithi189.stdout:Get:4 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [817 kB] 2024-09-18T10:01:33.714 INFO:teuthology.orchestra.run.smithi189.stdout:Get:5 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [3,223 kB] 2024-09-18T10:01:33.755 INFO:teuthology.orchestra.run.smithi189.stdout:Get:6 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [475 kB] 2024-09-18T10:01:33.756 INFO:teuthology.orchestra.run.smithi154.stdout:Get:5 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [817 kB] 2024-09-18T10:01:33.759 INFO:teuthology.orchestra.run.smithi189.stdout:Get:7 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [14.3 kB] 2024-09-18T10:01:33.759 INFO:teuthology.orchestra.run.smithi189.stdout:Get:8 http://security.ubuntu.com/ubuntu focal-security/restricted i386 Packages [38.3 kB] 2024-09-18T10:01:33.759 INFO:teuthology.orchestra.run.smithi189.stdout:Get:9 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [3,152 kB] 2024-09-18T10:01:33.769 INFO:teuthology.orchestra.run.smithi154.stdout:Get:6 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [475 kB] 2024-09-18T10:01:33.774 INFO:teuthology.orchestra.run.smithi154.stdout:Get:7 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [14.3 kB] 2024-09-18T10:01:33.774 INFO:teuthology.orchestra.run.smithi154.stdout:Get:8 http://security.ubuntu.com/ubuntu focal-security/restricted i386 Packages [38.3 kB] 2024-09-18T10:01:33.775 INFO:teuthology.orchestra.run.smithi154.stdout:Get:9 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [3,152 kB] 2024-09-18T10:01:33.776 INFO:teuthology.orchestra.run.smithi029.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [1,036 kB] 2024-09-18T10:01:33.779 INFO:teuthology.orchestra.run.smithi189.stdout:Get:10 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [441 kB] 2024-09-18T10:01:33.782 INFO:teuthology.orchestra.run.smithi189.stdout:Get:11 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 c-n-f Metadata [548 B] 2024-09-18T10:01:33.782 INFO:teuthology.orchestra.run.smithi189.stdout:Get:12 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [681 kB] 2024-09-18T10:01:33.787 INFO:teuthology.orchestra.run.smithi189.stdout:Get:13 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [1,012 kB] 2024-09-18T10:01:33.793 INFO:teuthology.orchestra.run.smithi189.stdout:Get:14 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [213 kB] 2024-09-18T10:01:33.795 INFO:teuthology.orchestra.run.smithi189.stdout:Get:15 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [21.4 kB] 2024-09-18T10:01:33.795 INFO:teuthology.orchestra.run.smithi189.stdout:Get:16 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [24.8 kB] 2024-09-18T10:01:33.795 INFO:teuthology.orchestra.run.smithi189.stdout:Get:17 http://security.ubuntu.com/ubuntu focal-security/multiverse i386 Packages [7,204 B] 2024-09-18T10:01:33.796 INFO:teuthology.orchestra.run.smithi189.stdout:Get:18 http://security.ubuntu.com/ubuntu focal-security/multiverse Translation-en [5,968 B] 2024-09-18T10:01:33.796 INFO:teuthology.orchestra.run.smithi154.stdout:Get:10 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [441 kB] 2024-09-18T10:01:33.799 INFO:teuthology.orchestra.run.smithi154.stdout:Get:11 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 c-n-f Metadata [548 B] 2024-09-18T10:01:33.799 INFO:teuthology.orchestra.run.smithi154.stdout:Get:12 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [1,012 kB] 2024-09-18T10:01:33.807 INFO:teuthology.orchestra.run.smithi154.stdout:Get:13 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [681 kB] 2024-09-18T10:01:33.809 INFO:teuthology.orchestra.run.smithi189.stdout:Get:19 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 c-n-f Metadata [540 B] 2024-09-18T10:01:33.811 INFO:teuthology.orchestra.run.smithi154.stdout:Get:14 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [213 kB] 2024-09-18T10:01:33.812 INFO:teuthology.orchestra.run.smithi154.stdout:Get:15 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [21.4 kB] 2024-09-18T10:01:33.813 INFO:teuthology.orchestra.run.smithi154.stdout:Get:16 http://security.ubuntu.com/ubuntu focal-security/multiverse i386 Packages [7,204 B] 2024-09-18T10:01:33.813 INFO:teuthology.orchestra.run.smithi154.stdout:Get:17 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [24.8 kB] 2024-09-18T10:01:33.823 INFO:teuthology.orchestra.run.smithi154.stdout:Get:18 http://security.ubuntu.com/ubuntu focal-security/multiverse Translation-en [5,968 B] 2024-09-18T10:01:33.824 INFO:teuthology.orchestra.run.smithi154.stdout:Get:19 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 c-n-f Metadata [540 B] 2024-09-18T10:01:33.851 INFO:teuthology.orchestra.run.smithi136.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease [128 kB] 2024-09-18T10:01:33.858 INFO:teuthology.orchestra.run.smithi154.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-backports InRelease [128 kB] 2024-09-18T10:01:33.881 INFO:teuthology.orchestra.run.smithi029.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [3,598 kB] 2024-09-18T10:01:33.883 INFO:teuthology.orchestra.run.smithi189.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-backports InRelease [128 kB] 2024-09-18T10:01:33.926 INFO:teuthology.orchestra.run.smithi029.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [553 kB] 2024-09-18T10:01:33.928 INFO:teuthology.orchestra.run.smithi136.stdout:Get:5 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [817 kB] 2024-09-18T10:01:33.932 INFO:teuthology.orchestra.run.smithi029.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [17.7 kB] 2024-09-18T10:01:33.932 INFO:teuthology.orchestra.run.smithi029.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/restricted i386 Packages [39.6 kB] 2024-09-18T10:01:33.947 INFO:teuthology.orchestra.run.smithi029.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [3,279 kB] 2024-09-18T10:01:33.976 INFO:teuthology.orchestra.run.smithi029.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [459 kB] 2024-09-18T10:01:33.979 INFO:teuthology.orchestra.run.smithi029.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 c-n-f Metadata [548 B] 2024-09-18T10:01:33.979 INFO:teuthology.orchestra.run.smithi029.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [808 kB] 2024-09-18T10:01:33.985 INFO:teuthology.orchestra.run.smithi029.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [1,231 kB] 2024-09-18T10:01:33.999 INFO:teuthology.orchestra.run.smithi154.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [3,598 kB] 2024-09-18T10:01:33.999 INFO:teuthology.orchestra.run.smithi029.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [295 kB] 2024-09-18T10:01:34.002 INFO:teuthology.orchestra.run.smithi029.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [28.3 kB] 2024-09-18T10:01:34.003 INFO:teuthology.orchestra.run.smithi029.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [27.1 kB] 2024-09-18T10:01:34.003 INFO:teuthology.orchestra.run.smithi029.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal-updates/multiverse i386 Packages [8,440 B] 2024-09-18T10:01:34.004 INFO:teuthology.orchestra.run.smithi029.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/multiverse Translation-en [7,936 B] 2024-09-18T10:01:34.004 INFO:teuthology.orchestra.run.smithi029.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 c-n-f Metadata [616 B] 2024-09-18T10:01:34.004 INFO:teuthology.orchestra.run.smithi029.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-backports/main i386 Packages [36.1 kB] 2024-09-18T10:01:34.004 INFO:teuthology.orchestra.run.smithi029.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 Packages [45.7 kB] 2024-09-18T10:01:34.021 INFO:teuthology.orchestra.run.smithi029.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 c-n-f Metadata [1,420 B] 2024-09-18T10:01:34.024 INFO:teuthology.orchestra.run.smithi189.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [3,598 kB] 2024-09-18T10:01:34.039 INFO:teuthology.orchestra.run.smithi029.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-backports/universe i386 Packages [13.8 kB] 2024-09-18T10:01:34.039 INFO:teuthology.orchestra.run.smithi029.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [25.0 kB] 2024-09-18T10:01:34.040 INFO:teuthology.orchestra.run.smithi029.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 c-n-f Metadata [880 B] 2024-09-18T10:01:34.069 INFO:teuthology.orchestra.run.smithi136.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [3,598 kB] 2024-09-18T10:01:34.103 INFO:teuthology.orchestra.run.smithi029.stdout:Get:27 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [817 kB] 2024-09-18T10:01:34.311 INFO:teuthology.orchestra.run.smithi136.stdout:Get:7 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [3,223 kB] 2024-09-18T10:01:34.384 INFO:teuthology.orchestra.run.smithi154.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [1,036 kB] 2024-09-18T10:01:34.409 INFO:teuthology.orchestra.run.smithi189.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [1,036 kB] 2024-09-18T10:01:34.412 INFO:teuthology.orchestra.run.smithi154.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [553 kB] 2024-09-18T10:01:34.425 INFO:teuthology.orchestra.run.smithi154.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [17.7 kB] 2024-09-18T10:01:34.425 INFO:teuthology.orchestra.run.smithi154.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/restricted i386 Packages [39.6 kB] 2024-09-18T10:01:34.426 INFO:teuthology.orchestra.run.smithi154.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [3,279 kB] 2024-09-18T10:01:34.437 INFO:teuthology.orchestra.run.smithi189.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [553 kB] 2024-09-18T10:01:34.438 INFO:teuthology.orchestra.run.smithi029.stdout:Get:28 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [3,223 kB] 2024-09-18T10:01:34.449 INFO:teuthology.orchestra.run.smithi189.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [17.7 kB] 2024-09-18T10:01:34.450 INFO:teuthology.orchestra.run.smithi189.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/restricted i386 Packages [39.6 kB] 2024-09-18T10:01:34.451 INFO:teuthology.orchestra.run.smithi189.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [3,279 kB] 2024-09-18T10:01:34.505 INFO:teuthology.orchestra.run.smithi136.stdout:Get:8 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [475 kB] 2024-09-18T10:01:34.519 INFO:teuthology.orchestra.run.smithi136.stdout:Get:9 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [14.3 kB] 2024-09-18T10:01:34.519 INFO:teuthology.orchestra.run.smithi136.stdout:Get:10 http://security.ubuntu.com/ubuntu focal-security/restricted i386 Packages [38.3 kB] 2024-09-18T10:01:34.520 INFO:teuthology.orchestra.run.smithi136.stdout:Get:11 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [3,152 kB] 2024-09-18T10:01:34.525 INFO:teuthology.orchestra.run.smithi154.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [459 kB] 2024-09-18T10:01:34.538 INFO:teuthology.orchestra.run.smithi136.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [1,036 kB] 2024-09-18T10:01:34.543 INFO:teuthology.orchestra.run.smithi154.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 c-n-f Metadata [548 B] 2024-09-18T10:01:34.543 INFO:teuthology.orchestra.run.smithi154.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [1,231 kB] 2024-09-18T10:01:34.550 INFO:teuthology.orchestra.run.smithi189.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [459 kB] 2024-09-18T10:01:34.555 INFO:teuthology.orchestra.run.smithi136.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [553 kB] 2024-09-18T10:01:34.564 INFO:teuthology.orchestra.run.smithi136.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [17.7 kB] 2024-09-18T10:01:34.564 INFO:teuthology.orchestra.run.smithi136.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal-updates/restricted i386 Packages [39.6 kB] 2024-09-18T10:01:34.565 INFO:teuthology.orchestra.run.smithi189.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 c-n-f Metadata [548 B] 2024-09-18T10:01:34.565 INFO:teuthology.orchestra.run.smithi189.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [1,231 kB] 2024-09-18T10:01:34.565 INFO:teuthology.orchestra.run.smithi136.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [3,279 kB] 2024-09-18T10:01:34.575 INFO:teuthology.orchestra.run.smithi154.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [808 kB] 2024-09-18T10:01:34.594 INFO:teuthology.orchestra.run.smithi154.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [295 kB] 2024-09-18T10:01:34.598 INFO:teuthology.orchestra.run.smithi189.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [808 kB] 2024-09-18T10:01:34.606 INFO:teuthology.orchestra.run.smithi154.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [28.3 kB] 2024-09-18T10:01:34.606 INFO:teuthology.orchestra.run.smithi154.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [27.1 kB] 2024-09-18T10:01:34.607 INFO:teuthology.orchestra.run.smithi154.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-updates/multiverse i386 Packages [8,440 B] 2024-09-18T10:01:34.607 INFO:teuthology.orchestra.run.smithi154.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/multiverse Translation-en [7,936 B] 2024-09-18T10:01:34.611 INFO:teuthology.orchestra.run.smithi136.stdout:Get:17 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [441 kB] 2024-09-18T10:01:34.617 INFO:teuthology.orchestra.run.smithi189.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [295 kB] 2024-09-18T10:01:34.624 INFO:teuthology.orchestra.run.smithi189.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [28.3 kB] 2024-09-18T10:01:34.626 INFO:teuthology.orchestra.run.smithi136.stdout:Get:18 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 c-n-f Metadata [548 B] 2024-09-18T10:01:34.626 INFO:teuthology.orchestra.run.smithi136.stdout:Get:19 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [681 kB] 2024-09-18T10:01:34.629 INFO:teuthology.orchestra.run.smithi154.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 c-n-f Metadata [616 B] 2024-09-18T10:01:34.629 INFO:teuthology.orchestra.run.smithi154.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal-backports/main i386 Packages [36.1 kB] 2024-09-18T10:01:34.630 INFO:teuthology.orchestra.run.smithi154.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 Packages [45.7 kB] 2024-09-18T10:01:34.631 INFO:teuthology.orchestra.run.smithi189.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [27.1 kB] 2024-09-18T10:01:34.631 INFO:teuthology.orchestra.run.smithi029.stdout:Get:29 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [475 kB] 2024-09-18T10:01:34.632 INFO:teuthology.orchestra.run.smithi189.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-updates/multiverse i386 Packages [8,440 B] 2024-09-18T10:01:34.632 INFO:teuthology.orchestra.run.smithi189.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/multiverse Translation-en [7,936 B] 2024-09-18T10:01:34.644 INFO:teuthology.orchestra.run.smithi136.stdout:Get:20 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [1,012 kB] 2024-09-18T10:01:34.646 INFO:teuthology.orchestra.run.smithi029.stdout:Get:30 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [14.3 kB] 2024-09-18T10:01:34.646 INFO:teuthology.orchestra.run.smithi029.stdout:Get:31 http://security.ubuntu.com/ubuntu focal-security/restricted i386 Packages [38.3 kB] 2024-09-18T10:01:34.647 INFO:teuthology.orchestra.run.smithi029.stdout:Get:32 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [3,152 kB] 2024-09-18T10:01:34.655 INFO:teuthology.orchestra.run.smithi136.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [459 kB] 2024-09-18T10:01:34.659 INFO:teuthology.orchestra.run.smithi189.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 c-n-f Metadata [616 B] 2024-09-18T10:01:34.659 INFO:teuthology.orchestra.run.smithi189.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal-backports/main i386 Packages [36.1 kB] 2024-09-18T10:01:34.660 INFO:teuthology.orchestra.run.smithi189.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 Packages [45.7 kB] 2024-09-18T10:01:34.663 INFO:teuthology.orchestra.run.smithi136.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 c-n-f Metadata [548 B] 2024-09-18T10:01:34.663 INFO:teuthology.orchestra.run.smithi136.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [808 kB] 2024-09-18T10:01:34.675 INFO:teuthology.orchestra.run.smithi136.stdout:Get:24 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [213 kB] 2024-09-18T10:01:34.685 INFO:teuthology.orchestra.run.smithi136.stdout:Get:25 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [21.4 kB] 2024-09-18T10:01:34.685 INFO:teuthology.orchestra.run.smithi136.stdout:Get:26 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [24.8 kB] 2024-09-18T10:01:34.685 INFO:teuthology.orchestra.run.smithi136.stdout:Get:27 http://security.ubuntu.com/ubuntu focal-security/multiverse i386 Packages [7,204 B] 2024-09-18T10:01:34.686 INFO:teuthology.orchestra.run.smithi136.stdout:Get:28 http://security.ubuntu.com/ubuntu focal-security/multiverse Translation-en [5,968 B] 2024-09-18T10:01:34.711 INFO:teuthology.orchestra.run.smithi154.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 c-n-f Metadata [1,420 B] 2024-09-18T10:01:34.711 INFO:teuthology.orchestra.run.smithi154.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal-backports/universe i386 Packages [13.8 kB] 2024-09-18T10:01:34.712 INFO:teuthology.orchestra.run.smithi154.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [25.0 kB] 2024-09-18T10:01:34.712 INFO:teuthology.orchestra.run.smithi154.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 c-n-f Metadata [880 B] 2024-09-18T10:01:34.713 INFO:teuthology.orchestra.run.smithi136.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [1,231 kB] 2024-09-18T10:01:34.732 INFO:teuthology.orchestra.run.smithi136.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [295 kB] 2024-09-18T10:01:34.738 INFO:teuthology.orchestra.run.smithi136.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [28.3 kB] 2024-09-18T10:01:34.738 INFO:teuthology.orchestra.run.smithi136.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal-updates/multiverse i386 Packages [8,440 B] 2024-09-18T10:01:34.739 INFO:teuthology.orchestra.run.smithi029.stdout:Get:33 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [441 kB] 2024-09-18T10:01:34.739 INFO:teuthology.orchestra.run.smithi136.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [27.1 kB] 2024-09-18T10:01:34.739 INFO:teuthology.orchestra.run.smithi136.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-updates/multiverse Translation-en [7,936 B] 2024-09-18T10:01:34.740 INFO:teuthology.orchestra.run.smithi189.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 c-n-f Metadata [1,420 B] 2024-09-18T10:01:34.740 INFO:teuthology.orchestra.run.smithi189.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal-backports/universe i386 Packages [13.8 kB] 2024-09-18T10:01:34.741 INFO:teuthology.orchestra.run.smithi189.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [25.0 kB] 2024-09-18T10:01:34.742 INFO:teuthology.orchestra.run.smithi189.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 c-n-f Metadata [880 B] 2024-09-18T10:01:34.751 INFO:teuthology.orchestra.run.smithi029.stdout:Get:34 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 c-n-f Metadata [548 B] 2024-09-18T10:01:34.751 INFO:teuthology.orchestra.run.smithi029.stdout:Get:35 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [681 kB] 2024-09-18T10:01:34.755 INFO:teuthology.orchestra.run.smithi136.stdout:Get:35 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 c-n-f Metadata [540 B] 2024-09-18T10:01:34.772 INFO:teuthology.orchestra.run.smithi029.stdout:Get:36 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [1,012 kB] 2024-09-18T10:01:34.801 INFO:teuthology.orchestra.run.smithi136.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 c-n-f Metadata [616 B] 2024-09-18T10:01:34.801 INFO:teuthology.orchestra.run.smithi136.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal-backports/main i386 Packages [36.1 kB] 2024-09-18T10:01:34.801 INFO:teuthology.orchestra.run.smithi136.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 Packages [45.7 kB] 2024-09-18T10:01:34.802 INFO:teuthology.orchestra.run.smithi029.stdout:Get:37 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [213 kB] 2024-09-18T10:01:34.802 INFO:teuthology.orchestra.run.smithi136.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 c-n-f Metadata [1,420 B] 2024-09-18T10:01:34.809 INFO:teuthology.orchestra.run.smithi029.stdout:Get:38 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [21.4 kB] 2024-09-18T10:01:34.809 INFO:teuthology.orchestra.run.smithi029.stdout:Get:39 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [24.8 kB] 2024-09-18T10:01:34.810 INFO:teuthology.orchestra.run.smithi029.stdout:Get:40 http://security.ubuntu.com/ubuntu focal-security/multiverse i386 Packages [7,204 B] 2024-09-18T10:01:34.811 INFO:teuthology.orchestra.run.smithi029.stdout:Get:41 http://security.ubuntu.com/ubuntu focal-security/multiverse Translation-en [5,968 B] 2024-09-18T10:01:34.882 INFO:teuthology.orchestra.run.smithi136.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal-backports/universe i386 Packages [13.8 kB] 2024-09-18T10:01:34.882 INFO:teuthology.orchestra.run.smithi136.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [25.0 kB] 2024-09-18T10:01:34.883 INFO:teuthology.orchestra.run.smithi029.stdout:Get:42 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 c-n-f Metadata [540 B] 2024-09-18T10:01:34.883 INFO:teuthology.orchestra.run.smithi136.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 c-n-f Metadata [880 B] 2024-09-18T10:01:38.148 INFO:teuthology.orchestra.run.smithi154.stdout:Fetched 22.0 MB in 3s (7,344 kB/s) 2024-09-18T10:01:38.180 INFO:teuthology.orchestra.run.smithi189.stdout:Fetched 22.0 MB in 3s (7,484 kB/s) 2024-09-18T10:01:38.664 INFO:teuthology.orchestra.run.smithi029.stdout:Fetched 22.0 MB in 3s (6,897 kB/s) 2024-09-18T10:01:38.793 INFO:teuthology.orchestra.run.smithi136.stdout:Fetched 22.0 MB in 4s (6,048 kB/s) 2024-09-18T10:01:39.647 INFO:teuthology.orchestra.run.smithi154.stdout:Reading package lists... 2024-09-18T10:01:39.675 DEBUG:teuthology.orchestra.run.smithi154:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-18T10:01:39.769 INFO:teuthology.orchestra.run.smithi154.stdout:Reading package lists... 2024-09-18T10:01:39.821 INFO:teuthology.orchestra.run.smithi189.stdout:Reading package lists... 2024-09-18T10:01:39.849 DEBUG:teuthology.orchestra.run.smithi189:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-18T10:01:39.918 INFO:teuthology.orchestra.run.smithi189.stdout:Reading package lists... 2024-09-18T10:01:39.989 INFO:teuthology.orchestra.run.smithi154.stdout:Building dependency tree... 2024-09-18T10:01:39.989 INFO:teuthology.orchestra.run.smithi154.stdout:Reading state information... 2024-09-18T10:01:40.113 INFO:teuthology.orchestra.run.smithi189.stdout:Building dependency tree... 2024-09-18T10:01:40.114 INFO:teuthology.orchestra.run.smithi189.stdout:Reading state information... 2024-09-18T10:01:40.196 INFO:teuthology.orchestra.run.smithi154.stdout:The following additional packages will be installed: 2024-09-18T10:01:40.196 INFO:teuthology.orchestra.run.smithi154.stdout: linux-generic linux-headers-5.4.0-196 linux-headers-5.4.0-196-generic 2024-09-18T10:01:40.196 INFO:teuthology.orchestra.run.smithi154.stdout: linux-headers-generic linux-image-5.4.0-196-generic 2024-09-18T10:01:40.196 INFO:teuthology.orchestra.run.smithi154.stdout: linux-modules-5.4.0-196-generic linux-modules-extra-5.4.0-196-generic 2024-09-18T10:01:40.197 INFO:teuthology.orchestra.run.smithi154.stdout:Suggested packages: 2024-09-18T10:01:40.198 INFO:teuthology.orchestra.run.smithi154.stdout: fdutils linux-doc | linux-source-5.4.0 linux-tools 2024-09-18T10:01:40.198 INFO:teuthology.orchestra.run.smithi154.stdout:Recommended packages: 2024-09-18T10:01:40.198 INFO:teuthology.orchestra.run.smithi154.stdout: thermald 2024-09-18T10:01:40.256 INFO:teuthology.orchestra.run.smithi154.stdout:The following NEW packages will be installed: 2024-09-18T10:01:40.256 INFO:teuthology.orchestra.run.smithi154.stdout: linux-headers-5.4.0-196 linux-headers-5.4.0-196-generic 2024-09-18T10:01:40.256 INFO:teuthology.orchestra.run.smithi154.stdout: linux-image-5.4.0-196-generic linux-modules-5.4.0-196-generic 2024-09-18T10:01:40.256 INFO:teuthology.orchestra.run.smithi154.stdout: linux-modules-extra-5.4.0-196-generic 2024-09-18T10:01:40.257 INFO:teuthology.orchestra.run.smithi154.stdout:The following packages will be upgraded: 2024-09-18T10:01:40.258 INFO:teuthology.orchestra.run.smithi154.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-18T10:01:40.311 INFO:teuthology.orchestra.run.smithi189.stdout:The following additional packages will be installed: 2024-09-18T10:01:40.312 INFO:teuthology.orchestra.run.smithi189.stdout: linux-generic linux-headers-5.4.0-196 linux-headers-5.4.0-196-generic 2024-09-18T10:01:40.312 INFO:teuthology.orchestra.run.smithi189.stdout: linux-headers-generic linux-image-5.4.0-196-generic 2024-09-18T10:01:40.312 INFO:teuthology.orchestra.run.smithi189.stdout: linux-modules-5.4.0-196-generic linux-modules-extra-5.4.0-196-generic 2024-09-18T10:01:40.313 INFO:teuthology.orchestra.run.smithi189.stdout:Suggested packages: 2024-09-18T10:01:40.313 INFO:teuthology.orchestra.run.smithi189.stdout: fdutils linux-doc | linux-source-5.4.0 linux-tools 2024-09-18T10:01:40.313 INFO:teuthology.orchestra.run.smithi189.stdout:Recommended packages: 2024-09-18T10:01:40.313 INFO:teuthology.orchestra.run.smithi189.stdout: thermald 2024-09-18T10:01:40.349 INFO:teuthology.orchestra.run.smithi136.stdout:Reading package lists... 2024-09-18T10:01:40.371 INFO:teuthology.orchestra.run.smithi189.stdout:The following NEW packages will be installed: 2024-09-18T10:01:40.371 INFO:teuthology.orchestra.run.smithi189.stdout: linux-headers-5.4.0-196 linux-headers-5.4.0-196-generic 2024-09-18T10:01:40.371 INFO:teuthology.orchestra.run.smithi189.stdout: linux-image-5.4.0-196-generic linux-modules-5.4.0-196-generic 2024-09-18T10:01:40.371 INFO:teuthology.orchestra.run.smithi189.stdout: linux-modules-extra-5.4.0-196-generic 2024-09-18T10:01:40.372 INFO:teuthology.orchestra.run.smithi189.stdout:The following packages will be upgraded: 2024-09-18T10:01:40.373 INFO:teuthology.orchestra.run.smithi189.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-18T10:01:40.375 DEBUG:teuthology.orchestra.run.smithi136:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-18T10:01:40.427 INFO:teuthology.orchestra.run.smithi189.stdout:3 upgraded, 5 newly installed, 0 to remove and 316 not upgraded. 2024-09-18T10:01:40.427 INFO:teuthology.orchestra.run.smithi189.stdout:Need to get 77.2 MB of archives. 2024-09-18T10:01:40.427 INFO:teuthology.orchestra.run.smithi189.stdout:After this operation, 381 MB of additional disk space will be used. 2024-09-18T10:01:40.427 INFO:teuthology.orchestra.run.smithi189.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-5.4.0-196-generic amd64 5.4.0-196.216 [15.0 MB] 2024-09-18T10:01:40.440 INFO:teuthology.orchestra.run.smithi154.stdout:3 upgraded, 5 newly installed, 0 to remove and 316 not upgraded. 2024-09-18T10:01:40.440 INFO:teuthology.orchestra.run.smithi154.stdout:Need to get 77.2 MB of archives. 2024-09-18T10:01:40.440 INFO:teuthology.orchestra.run.smithi154.stdout:After this operation, 381 MB of additional disk space will be used. 2024-09-18T10:01:40.441 INFO:teuthology.orchestra.run.smithi154.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-5.4.0-196-generic amd64 5.4.0-196.216 [15.0 MB] 2024-09-18T10:01:40.452 INFO:teuthology.orchestra.run.smithi136.stdout:Reading package lists... 2024-09-18T10:01:40.673 INFO:teuthology.orchestra.run.smithi136.stdout:Building dependency tree... 2024-09-18T10:01:40.673 INFO:teuthology.orchestra.run.smithi136.stdout:Reading state information... 2024-09-18T10:01:40.759 INFO:teuthology.orchestra.run.smithi189.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-5.4.0-196-generic amd64 5.4.0-196.216 [10.5 MB] 2024-09-18T10:01:40.851 INFO:teuthology.orchestra.run.smithi189.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-extra-5.4.0-196-generic amd64 5.4.0-196.216 [39.2 MB] 2024-09-18T10:01:40.875 INFO:teuthology.orchestra.run.smithi136.stdout:The following additional packages will be installed: 2024-09-18T10:01:40.875 INFO:teuthology.orchestra.run.smithi136.stdout: linux-generic linux-headers-5.4.0-196 linux-headers-5.4.0-196-generic 2024-09-18T10:01:40.875 INFO:teuthology.orchestra.run.smithi136.stdout: linux-headers-generic linux-image-5.4.0-196-generic 2024-09-18T10:01:40.875 INFO:teuthology.orchestra.run.smithi136.stdout: linux-modules-5.4.0-196-generic linux-modules-extra-5.4.0-196-generic 2024-09-18T10:01:40.876 INFO:teuthology.orchestra.run.smithi136.stdout:Suggested packages: 2024-09-18T10:01:40.876 INFO:teuthology.orchestra.run.smithi136.stdout: fdutils linux-doc | linux-source-5.4.0 linux-tools 2024-09-18T10:01:40.876 INFO:teuthology.orchestra.run.smithi136.stdout:Recommended packages: 2024-09-18T10:01:40.876 INFO:teuthology.orchestra.run.smithi136.stdout: thermald 2024-09-18T10:01:40.931 INFO:teuthology.orchestra.run.smithi136.stdout:The following NEW packages will be installed: 2024-09-18T10:01:40.931 INFO:teuthology.orchestra.run.smithi136.stdout: linux-headers-5.4.0-196 linux-headers-5.4.0-196-generic 2024-09-18T10:01:40.931 INFO:teuthology.orchestra.run.smithi136.stdout: linux-image-5.4.0-196-generic linux-modules-5.4.0-196-generic 2024-09-18T10:01:40.931 INFO:teuthology.orchestra.run.smithi136.stdout: linux-modules-extra-5.4.0-196-generic 2024-09-18T10:01:40.933 INFO:teuthology.orchestra.run.smithi136.stdout:The following packages will be upgraded: 2024-09-18T10:01:40.933 INFO:teuthology.orchestra.run.smithi136.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-18T10:01:41.019 INFO:teuthology.orchestra.run.smithi029.stdout:Reading package lists... 2024-09-18T10:01:41.048 DEBUG:teuthology.orchestra.run.smithi029:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-18T10:01:41.118 INFO:teuthology.orchestra.run.smithi136.stdout:3 upgraded, 5 newly installed, 0 to remove and 316 not upgraded. 2024-09-18T10:01:41.119 INFO:teuthology.orchestra.run.smithi136.stdout:Need to get 77.2 MB of archives. 2024-09-18T10:01:41.119 INFO:teuthology.orchestra.run.smithi136.stdout:After this operation, 381 MB of additional disk space will be used. 2024-09-18T10:01:41.119 INFO:teuthology.orchestra.run.smithi136.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-5.4.0-196-generic amd64 5.4.0-196.216 [15.0 MB] 2024-09-18T10:01:41.163 INFO:teuthology.orchestra.run.smithi029.stdout:Reading package lists... 2024-09-18T10:01:41.305 INFO:teuthology.orchestra.run.smithi189.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-generic amd64 5.4.0.196.194 [1,900 B] 2024-09-18T10:01:41.305 INFO:teuthology.orchestra.run.smithi189.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-generic amd64 5.4.0.196.194 [2,592 B] 2024-09-18T10:01:41.306 INFO:teuthology.orchestra.run.smithi189.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-196 all 5.4.0-196.216 [11.0 MB] 2024-09-18T10:01:41.390 INFO:teuthology.orchestra.run.smithi029.stdout:Building dependency tree... 2024-09-18T10:01:41.390 INFO:teuthology.orchestra.run.smithi029.stdout:Reading state information... 2024-09-18T10:01:41.398 INFO:teuthology.orchestra.run.smithi189.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-196-generic amd64 5.4.0-196.216 [1,386 kB] 2024-09-18T10:01:41.408 INFO:teuthology.orchestra.run.smithi189.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-generic amd64 5.4.0.196.194 [2,460 B] 2024-09-18T10:01:41.607 INFO:teuthology.orchestra.run.smithi029.stdout:The following additional packages will be installed: 2024-09-18T10:01:41.608 INFO:teuthology.orchestra.run.smithi029.stdout: linux-generic linux-headers-5.4.0-196 linux-headers-5.4.0-196-generic 2024-09-18T10:01:41.608 INFO:teuthology.orchestra.run.smithi029.stdout: linux-headers-generic linux-image-5.4.0-196-generic 2024-09-18T10:01:41.608 INFO:teuthology.orchestra.run.smithi029.stdout: linux-modules-5.4.0-196-generic linux-modules-extra-5.4.0-196-generic 2024-09-18T10:01:41.609 INFO:teuthology.orchestra.run.smithi029.stdout:Suggested packages: 2024-09-18T10:01:41.609 INFO:teuthology.orchestra.run.smithi029.stdout: fdutils linux-doc | linux-source-5.4.0 linux-tools 2024-09-18T10:01:41.609 INFO:teuthology.orchestra.run.smithi029.stdout:Recommended packages: 2024-09-18T10:01:41.609 INFO:teuthology.orchestra.run.smithi029.stdout: thermald 2024-09-18T10:01:41.610 INFO:teuthology.orchestra.run.smithi154.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-5.4.0-196-generic amd64 5.4.0-196.216 [10.5 MB] 2024-09-18T10:01:41.675 INFO:teuthology.orchestra.run.smithi029.stdout:The following NEW packages will be installed: 2024-09-18T10:01:41.675 INFO:teuthology.orchestra.run.smithi029.stdout: linux-headers-5.4.0-196 linux-headers-5.4.0-196-generic 2024-09-18T10:01:41.675 INFO:teuthology.orchestra.run.smithi029.stdout: linux-image-5.4.0-196-generic linux-modules-5.4.0-196-generic 2024-09-18T10:01:41.675 INFO:teuthology.orchestra.run.smithi029.stdout: linux-modules-extra-5.4.0-196-generic 2024-09-18T10:01:41.675 INFO:teuthology.orchestra.run.smithi029.stdout:The following packages will be upgraded: 2024-09-18T10:01:41.676 INFO:teuthology.orchestra.run.smithi029.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-18T10:01:41.732 INFO:teuthology.orchestra.run.smithi029.stdout:3 upgraded, 5 newly installed, 0 to remove and 316 not upgraded. 2024-09-18T10:01:41.732 INFO:teuthology.orchestra.run.smithi029.stdout:Need to get 77.2 MB of archives. 2024-09-18T10:01:41.732 INFO:teuthology.orchestra.run.smithi029.stdout:After this operation, 381 MB of additional disk space will be used. 2024-09-18T10:01:41.732 INFO:teuthology.orchestra.run.smithi029.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-5.4.0-196-generic amd64 5.4.0-196.216 [15.0 MB] 2024-09-18T10:01:41.885 INFO:teuthology.orchestra.run.smithi154.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-extra-5.4.0-196-generic amd64 5.4.0-196.216 [39.2 MB] 2024-09-18T10:01:42.081 INFO:teuthology.orchestra.run.smithi029.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-5.4.0-196-generic amd64 5.4.0-196.216 [10.5 MB] 2024-09-18T10:01:42.119 INFO:teuthology.orchestra.run.smithi189.stdout:Fetched 77.2 MB in 1s (75.1 MB/s) 2024-09-18T10:01:42.174 INFO:teuthology.orchestra.run.smithi029.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-extra-5.4.0-196-generic amd64 5.4.0-196.216 [39.2 MB] 2024-09-18T10:01:42.303 INFO:teuthology.orchestra.run.smithi136.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-5.4.0-196-generic amd64 5.4.0-196.216 [10.5 MB] 2024-09-18T10:01:42.318 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package linux-modules-5.4.0-196-generic. 2024-09-18T10:01:42.574 INFO:teuthology.orchestra.run.smithi136.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-extra-5.4.0-196-generic amd64 5.4.0-196.216 [39.2 MB] 2024-09-18T10:01:42.873 INFO:teuthology.orchestra.run.smithi029.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-generic amd64 5.4.0.196.194 [1,900 B] 2024-09-18T10:01:42.873 INFO:teuthology.orchestra.run.smithi029.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-generic amd64 5.4.0.196.194 [2,592 B] 2024-09-18T10:01:42.873 INFO:teuthology.orchestra.run.smithi029.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-196 all 5.4.0-196.216 [11.0 MB] 2024-09-18T10:01:42.969 INFO:teuthology.orchestra.run.smithi029.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-196-generic amd64 5.4.0-196.216 [1,386 kB] 2024-09-18T10:01:42.980 INFO:teuthology.orchestra.run.smithi029.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-generic amd64 5.4.0.196.194 [2,460 B] 2024-09-18T10:01:43.238 INFO:teuthology.orchestra.run.smithi154.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-generic amd64 5.4.0.196.194 [1,900 B] 2024-09-18T10:01:43.238 INFO:teuthology.orchestra.run.smithi154.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-generic amd64 5.4.0.196.194 [2,592 B] 2024-09-18T10:01:43.239 INFO:teuthology.orchestra.run.smithi154.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-196 all 5.4.0-196.216 [11.0 MB] 2024-09-18T10:01:43.521 INFO:teuthology.orchestra.run.smithi154.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-196-generic amd64 5.4.0-196.216 [1,386 kB] 2024-09-18T10:01:43.533 INFO:teuthology.orchestra.run.smithi154.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-generic amd64 5.4.0.196.194 [2,460 B] 2024-09-18T10:01:43.808 INFO:teuthology.orchestra.run.smithi029.stdout:Fetched 77.2 MB in 1s (59.5 MB/s) 2024-09-18T10:01:43.955 INFO:teuthology.orchestra.run.smithi136.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-generic amd64 5.4.0.196.194 [1,900 B] 2024-09-18T10:01:43.955 INFO:teuthology.orchestra.run.smithi136.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-generic amd64 5.4.0.196.194 [2,592 B] 2024-09-18T10:01:43.956 INFO:teuthology.orchestra.run.smithi136.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-196 all 5.4.0-196.216 [11.0 MB] 2024-09-18T10:01:44.007 INFO:teuthology.orchestra.run.smithi189.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 104373 files and directories currently installed.) 2024-09-18T10:01:44.012 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../0-linux-modules-5.4.0-196-generic_5.4.0-196.216_amd64.deb ... 2024-09-18T10:01:44.045 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package linux-modules-5.4.0-196-generic. 2024-09-18T10:01:44.141 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking linux-modules-5.4.0-196-generic (5.4.0-196.216) ... 2024-09-18T10:01:44.234 INFO:teuthology.orchestra.run.smithi154.stdout:Fetched 77.2 MB in 3s (23.6 MB/s) 2024-09-18T10:01:44.236 INFO:teuthology.orchestra.run.smithi136.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-196-generic amd64 5.4.0-196.216 [1,386 kB] 2024-09-18T10:01:44.272 INFO:teuthology.orchestra.run.smithi136.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-generic amd64 5.4.0.196.194 [2,460 B] 2024-09-18T10:01:44.435 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package linux-modules-5.4.0-196-generic. 2024-09-18T10:01:45.007 INFO:teuthology.orchestra.run.smithi136.stdout:Fetched 77.2 MB in 3s (23.2 MB/s) 2024-09-18T10:01:45.187 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package linux-modules-5.4.0-196-generic. 2024-09-18T10:01:46.150 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package linux-image-5.4.0-196-generic. 2024-09-18T10:01:46.165 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../1-linux-image-5.4.0-196-generic_5.4.0-196.216_amd64.deb ... 2024-09-18T10:01:46.181 INFO:teuthology.orchestra.run.smithi154.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 104373 files and directories currently installed.) 2024-09-18T10:01:46.186 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../0-linux-modules-5.4.0-196-generic_5.4.0-196.216_amd64.deb ... 2024-09-18T10:01:46.263 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking linux-image-5.4.0-196-generic (5.4.0-196.216) ... 2024-09-18T10:01:46.282 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking linux-modules-5.4.0-196-generic (5.4.0-196.216) ... 2024-09-18T10:01:46.781 INFO:teuthology.orchestra.run.smithi029.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 104373 files and directories currently installed.) 2024-09-18T10:01:46.786 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../0-linux-modules-5.4.0-196-generic_5.4.0-196.216_amd64.deb ... 2024-09-18T10:01:46.921 INFO:teuthology.orchestra.run.smithi136.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 104373 files and directories currently installed.) 2024-09-18T10:01:46.926 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../0-linux-modules-5.4.0-196-generic_5.4.0-196.216_amd64.deb ... 2024-09-18T10:01:46.959 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking linux-modules-5.4.0-196-generic (5.4.0-196.216) ... 2024-09-18T10:01:47.034 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking linux-modules-5.4.0-196-generic (5.4.0-196.216) ... 2024-09-18T10:01:47.336 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package linux-modules-extra-5.4.0-196-generic. 2024-09-18T10:01:47.352 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../2-linux-modules-extra-5.4.0-196-generic_5.4.0-196.216_amd64.deb ... 2024-09-18T10:01:47.394 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking linux-modules-extra-5.4.0-196-generic (5.4.0-196.216) ... 2024-09-18T10:01:48.191 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package linux-image-5.4.0-196-generic. 2024-09-18T10:01:48.205 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../1-linux-image-5.4.0-196-generic_5.4.0-196.216_amd64.deb ... 2024-09-18T10:01:48.305 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking linux-image-5.4.0-196-generic (5.4.0-196.216) ... 2024-09-18T10:01:49.150 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package linux-image-5.4.0-196-generic. 2024-09-18T10:01:49.163 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../1-linux-image-5.4.0-196-generic_5.4.0-196.216_amd64.deb ... 2024-09-18T10:01:49.255 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package linux-image-5.4.0-196-generic. 2024-09-18T10:01:49.270 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking linux-image-5.4.0-196-generic (5.4.0-196.216) ... 2024-09-18T10:01:49.270 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../1-linux-image-5.4.0-196-generic_5.4.0-196.216_amd64.deb ... 2024-09-18T10:01:49.361 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package linux-modules-extra-5.4.0-196-generic. 2024-09-18T10:01:49.376 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../2-linux-modules-extra-5.4.0-196-generic_5.4.0-196.216_amd64.deb ... 2024-09-18T10:01:49.380 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking linux-image-5.4.0-196-generic (5.4.0-196.216) ... 2024-09-18T10:01:49.419 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking linux-modules-extra-5.4.0-196-generic (5.4.0-196.216) ... 2024-09-18T10:01:50.586 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package linux-modules-extra-5.4.0-196-generic. 2024-09-18T10:01:50.599 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../2-linux-modules-extra-5.4.0-196-generic_5.4.0-196.216_amd64.deb ... 2024-09-18T10:01:50.644 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking linux-modules-extra-5.4.0-196-generic (5.4.0-196.216) ... 2024-09-18T10:01:50.670 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package linux-modules-extra-5.4.0-196-generic. 2024-09-18T10:01:50.684 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../2-linux-modules-extra-5.4.0-196-generic_5.4.0-196.216_amd64.deb ... 2024-09-18T10:01:50.729 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking linux-modules-extra-5.4.0-196-generic (5.4.0-196.216) ... 2024-09-18T10:01:52.341 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../3-linux-generic_5.4.0.196.194_amd64.deb ... 2024-09-18T10:01:52.492 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking linux-generic (5.4.0.196.194) over (5.4.0.124.125) ... 2024-09-18T10:01:52.879 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../4-linux-image-generic_5.4.0.196.194_amd64.deb ... 2024-09-18T10:01:53.004 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking linux-image-generic (5.4.0.196.194) over (5.4.0.124.125) ... 2024-09-18T10:01:53.382 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package linux-headers-5.4.0-196. 2024-09-18T10:01:53.398 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../5-linux-headers-5.4.0-196_5.4.0-196.216_all.deb ... 2024-09-18T10:01:53.440 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking linux-headers-5.4.0-196 (5.4.0-196.216) ... 2024-09-18T10:01:54.193 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../3-linux-generic_5.4.0.196.194_amd64.deb ... 2024-09-18T10:01:54.317 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking linux-generic (5.4.0.196.194) over (5.4.0.124.125) ... 2024-09-18T10:01:54.646 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../4-linux-image-generic_5.4.0.196.194_amd64.deb ... 2024-09-18T10:01:54.771 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking linux-image-generic (5.4.0.196.194) over (5.4.0.124.125) ... 2024-09-18T10:01:55.116 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package linux-headers-5.4.0-196. 2024-09-18T10:01:55.133 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../5-linux-headers-5.4.0-196_5.4.0-196.216_all.deb ... 2024-09-18T10:01:55.174 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking linux-headers-5.4.0-196 (5.4.0-196.216) ... 2024-09-18T10:01:55.425 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../3-linux-generic_5.4.0.196.194_amd64.deb ... 2024-09-18T10:01:55.551 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking linux-generic (5.4.0.196.194) over (5.4.0.124.125) ... 2024-09-18T10:01:55.889 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../3-linux-generic_5.4.0.196.194_amd64.deb ... 2024-09-18T10:01:55.921 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../4-linux-image-generic_5.4.0.196.194_amd64.deb ... 2024-09-18T10:01:56.046 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking linux-image-generic (5.4.0.196.194) over (5.4.0.124.125) ... 2024-09-18T10:01:56.416 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package linux-headers-5.4.0-196. 2024-09-18T10:01:56.430 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../5-linux-headers-5.4.0-196_5.4.0-196.216_all.deb ... 2024-09-18T10:01:56.442 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking linux-generic (5.4.0.196.194) over (5.4.0.124.125) ... 2024-09-18T10:01:56.474 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking linux-headers-5.4.0-196 (5.4.0-196.216) ... 2024-09-18T10:01:56.823 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../4-linux-image-generic_5.4.0.196.194_amd64.deb ... 2024-09-18T10:01:56.950 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking linux-image-generic (5.4.0.196.194) over (5.4.0.124.125) ... 2024-09-18T10:01:57.357 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package linux-headers-5.4.0-196. 2024-09-18T10:01:57.371 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../5-linux-headers-5.4.0-196_5.4.0-196.216_all.deb ... 2024-09-18T10:01:57.432 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking linux-headers-5.4.0-196 (5.4.0-196.216) ... 2024-09-18T10:01:59.212 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package linux-headers-5.4.0-196-generic. 2024-09-18T10:01:59.234 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../6-linux-headers-5.4.0-196-generic_5.4.0-196.216_amd64.deb ... 2024-09-18T10:01:59.278 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking linux-headers-5.4.0-196-generic (5.4.0-196.216) ... 2024-09-18T10:02:00.584 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package linux-headers-5.4.0-196-generic. 2024-09-18T10:02:00.607 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../6-linux-headers-5.4.0-196-generic_5.4.0-196.216_amd64.deb ... 2024-09-18T10:02:00.668 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking linux-headers-5.4.0-196-generic (5.4.0-196.216) ... 2024-09-18T10:02:02.134 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../7-linux-headers-generic_5.4.0.196.194_amd64.deb ... 2024-09-18T10:02:02.286 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking linux-headers-generic (5.4.0.196.194) over (5.4.0.124.125) ... 2024-09-18T10:02:02.295 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package linux-headers-5.4.0-196-generic. 2024-09-18T10:02:02.319 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../6-linux-headers-5.4.0-196-generic_5.4.0-196.216_amd64.deb ... 2024-09-18T10:02:02.370 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking linux-headers-5.4.0-196-generic (5.4.0-196.216) ... 2024-09-18T10:02:02.639 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up linux-headers-5.4.0-196 (5.4.0-196.216) ... 2024-09-18T10:02:02.774 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up linux-headers-5.4.0-196-generic (5.4.0-196.216) ... 2024-09-18T10:02:02.909 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up linux-headers-generic (5.4.0.196.194) ... 2024-09-18T10:02:03.060 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up linux-modules-5.4.0-196-generic (5.4.0-196.216) ... 2024-09-18T10:02:03.385 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../7-linux-headers-generic_5.4.0.196.194_amd64.deb ... 2024-09-18T10:02:03.553 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking linux-headers-generic (5.4.0.196.194) over (5.4.0.124.125) ... 2024-09-18T10:02:03.914 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up linux-headers-5.4.0-196 (5.4.0-196.216) ... 2024-09-18T10:02:03.995 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package linux-headers-5.4.0-196-generic. 2024-09-18T10:02:04.018 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../6-linux-headers-5.4.0-196-generic_5.4.0-196.216_amd64.deb ... 2024-09-18T10:02:04.042 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up linux-headers-5.4.0-196-generic (5.4.0-196.216) ... 2024-09-18T10:02:04.095 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking linux-headers-5.4.0-196-generic (5.4.0-196.216) ... 2024-09-18T10:02:04.176 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up linux-headers-generic (5.4.0.196.194) ... 2024-09-18T10:02:04.245 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up linux-image-5.4.0-196-generic (5.4.0-196.216) ... 2024-09-18T10:02:04.319 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up linux-modules-5.4.0-196-generic (5.4.0-196.216) ... 2024-09-18T10:02:05.034 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../7-linux-headers-generic_5.4.0.196.194_amd64.deb ... 2024-09-18T10:02:05.161 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking linux-headers-generic (5.4.0.196.194) over (5.4.0.124.125) ... 2024-09-18T10:02:05.381 INFO:teuthology.orchestra.run.smithi189.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.4.0-196-generic 2024-09-18T10:02:05.381 INFO:teuthology.orchestra.run.smithi189.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.4.0-196-generic 2024-09-18T10:02:05.523 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up linux-modules-extra-5.4.0-196-generic (5.4.0-196.216) ... 2024-09-18T10:02:05.528 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up linux-headers-5.4.0-196 (5.4.0-196.216) ... 2024-09-18T10:02:05.649 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up linux-headers-5.4.0-196-generic (5.4.0-196.216) ... 2024-09-18T10:02:05.663 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up linux-image-5.4.0-196-generic (5.4.0-196.216) ... 2024-09-18T10:02:05.800 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up linux-headers-generic (5.4.0.196.194) ... 2024-09-18T10:02:05.926 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up linux-modules-5.4.0-196-generic (5.4.0-196.216) ... 2024-09-18T10:02:06.749 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up linux-image-generic (5.4.0.196.194) ... 2024-09-18T10:02:06.900 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up linux-generic (5.4.0.196.194) ... 2024-09-18T10:02:07.052 INFO:teuthology.orchestra.run.smithi189.stdout:Processing triggers for linux-image-5.4.0-196-generic (5.4.0-196.216) ... 2024-09-18T10:02:07.123 INFO:teuthology.orchestra.run.smithi189.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-18T10:02:07.123 INFO:teuthology.orchestra.run.smithi189.stdout:update-initramfs: Generating /boot/initrd.img-5.4.0-196-generic 2024-09-18T10:02:07.311 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up linux-image-5.4.0-196-generic (5.4.0-196.216) ... 2024-09-18T10:02:07.879 INFO:teuthology.orchestra.run.smithi154.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.4.0-196-generic 2024-09-18T10:02:07.879 INFO:teuthology.orchestra.run.smithi154.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.4.0-196-generic 2024-09-18T10:02:08.023 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up linux-modules-extra-5.4.0-196-generic (5.4.0-196.216) ... 2024-09-18T10:02:08.463 INFO:teuthology.orchestra.run.smithi136.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.4.0-196-generic 2024-09-18T10:02:08.464 INFO:teuthology.orchestra.run.smithi136.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.4.0-196-generic 2024-09-18T10:02:08.629 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up linux-modules-extra-5.4.0-196-generic (5.4.0-196.216) ... 2024-09-18T10:02:09.153 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../7-linux-headers-generic_5.4.0.196.194_amd64.deb ... 2024-09-18T10:02:09.224 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up linux-image-generic (5.4.0.196.194) ... 2024-09-18T10:02:09.325 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking linux-headers-generic (5.4.0.196.194) over (5.4.0.124.125) ... 2024-09-18T10:02:09.351 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up linux-generic (5.4.0.196.194) ... 2024-09-18T10:02:09.494 INFO:teuthology.orchestra.run.smithi154.stdout:Processing triggers for linux-image-5.4.0-196-generic (5.4.0-196.216) ... 2024-09-18T10:02:09.579 INFO:teuthology.orchestra.run.smithi154.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-18T10:02:09.580 INFO:teuthology.orchestra.run.smithi154.stdout:update-initramfs: Generating /boot/initrd.img-5.4.0-196-generic 2024-09-18T10:02:09.740 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up linux-headers-5.4.0-196 (5.4.0-196.216) ... 2024-09-18T10:02:09.797 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up linux-image-generic (5.4.0.196.194) ... 2024-09-18T10:02:09.851 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up linux-headers-5.4.0-196-generic (5.4.0-196.216) ... 2024-09-18T10:02:09.948 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up linux-generic (5.4.0.196.194) ... 2024-09-18T10:02:10.022 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up linux-headers-generic (5.4.0.196.194) ... 2024-09-18T10:02:10.075 INFO:teuthology.orchestra.run.smithi136.stdout:Processing triggers for linux-image-5.4.0-196-generic (5.4.0-196.216) ... 2024-09-18T10:02:10.148 INFO:teuthology.orchestra.run.smithi136.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-18T10:02:10.148 INFO:teuthology.orchestra.run.smithi136.stdout:update-initramfs: Generating /boot/initrd.img-5.4.0-196-generic 2024-09-18T10:02:10.148 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up linux-modules-5.4.0-196-generic (5.4.0-196.216) ... 2024-09-18T10:02:11.413 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up linux-image-5.4.0-196-generic (5.4.0-196.216) ... 2024-09-18T10:02:12.668 INFO:teuthology.orchestra.run.smithi029.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.4.0-196-generic 2024-09-18T10:02:12.668 INFO:teuthology.orchestra.run.smithi029.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.4.0-196-generic 2024-09-18T10:02:12.810 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up linux-modules-extra-5.4.0-196-generic (5.4.0-196.216) ... 2024-09-18T10:02:15.597 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up linux-image-generic (5.4.0.196.194) ... 2024-09-18T10:02:15.723 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up linux-generic (5.4.0.196.194) ... 2024-09-18T10:02:15.884 INFO:teuthology.orchestra.run.smithi029.stdout:Processing triggers for linux-image-5.4.0-196-generic (5.4.0-196.216) ... 2024-09-18T10:02:15.948 INFO:teuthology.orchestra.run.smithi029.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-18T10:02:15.948 INFO:teuthology.orchestra.run.smithi029.stdout:update-initramfs: Generating /boot/initrd.img-5.4.0-196-generic 2024-09-18T10:02:29.017 INFO:teuthology.orchestra.run.smithi189.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-18T10:02:29.018 INFO:teuthology.orchestra.run.smithi189.stdout:Sourcing file `/etc/default/grub' 2024-09-18T10:02:29.034 INFO:teuthology.orchestra.run.smithi189.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-18T10:02:29.051 INFO:teuthology.orchestra.run.smithi189.stdout:Generating grub configuration file ... 2024-09-18T10:02:29.382 INFO:teuthology.orchestra.run.smithi189.stdout:Found linux image: /boot/vmlinuz-5.4.0-196-generic 2024-09-18T10:02:29.401 INFO:teuthology.orchestra.run.smithi189.stdout:Found initrd image: /boot/initrd.img-5.4.0-196-generic 2024-09-18T10:02:29.819 INFO:teuthology.orchestra.run.smithi189.stdout:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-09-18T10:02:29.829 INFO:teuthology.orchestra.run.smithi189.stdout:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-09-18T10:02:31.364 INFO:teuthology.orchestra.run.smithi154.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-18T10:02:31.364 INFO:teuthology.orchestra.run.smithi154.stdout:Sourcing file `/etc/default/grub' 2024-09-18T10:02:31.384 INFO:teuthology.orchestra.run.smithi154.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-18T10:02:31.405 INFO:teuthology.orchestra.run.smithi154.stdout:Generating grub configuration file ... 2024-09-18T10:02:31.748 INFO:teuthology.orchestra.run.smithi154.stdout:Found linux image: /boot/vmlinuz-5.4.0-196-generic 2024-09-18T10:02:31.767 INFO:teuthology.orchestra.run.smithi154.stdout:Found initrd image: /boot/initrd.img-5.4.0-196-generic 2024-09-18T10:02:32.134 INFO:teuthology.orchestra.run.smithi154.stdout:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-09-18T10:02:32.144 INFO:teuthology.orchestra.run.smithi154.stdout:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-09-18T10:02:32.244 INFO:teuthology.orchestra.run.smithi189.stdout:done 2024-09-18T10:02:32.388 INFO:teuthology.orchestra.run.smithi136.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-18T10:02:32.388 INFO:teuthology.orchestra.run.smithi136.stdout:Sourcing file `/etc/default/grub' 2024-09-18T10:02:32.393 INFO:teuthology.orchestra.run.smithi136.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-18T10:02:32.405 INFO:teuthology.orchestra.run.smithi136.stdout:Generating grub configuration file ... 2024-09-18T10:02:32.692 DEBUG:teuthology.orchestra.run.smithi189:> dpkg -s linux-image-generic 2024-09-18T10:02:32.750 INFO:teuthology.orchestra.run.smithi189.stdout:Package: linux-image-generic 2024-09-18T10:02:32.751 INFO:teuthology.orchestra.run.smithi189.stdout:Status: install ok installed 2024-09-18T10:02:32.751 INFO:teuthology.orchestra.run.smithi189.stdout:Priority: optional 2024-09-18T10:02:32.751 INFO:teuthology.orchestra.run.smithi189.stdout:Section: kernel 2024-09-18T10:02:32.751 INFO:teuthology.orchestra.run.smithi189.stdout:Installed-Size: 20 2024-09-18T10:02:32.751 INFO:teuthology.orchestra.run.smithi189.stdout:Maintainer: Ubuntu Kernel Team 2024-09-18T10:02:32.751 INFO:teuthology.orchestra.run.smithi189.stdout:Architecture: amd64 2024-09-18T10:02:32.751 INFO:teuthology.orchestra.run.smithi189.stdout:Source: linux-meta 2024-09-18T10:02:32.751 INFO:teuthology.orchestra.run.smithi189.stdout:Version: 5.4.0.196.194 2024-09-18T10:02:32.751 INFO:teuthology.orchestra.run.smithi189.stdout:Provides: virtualbox-guest-modules (= 6.1.50-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.17) 2024-09-18T10:02:32.751 INFO:teuthology.orchestra.run.smithi189.stdout:Depends: linux-image-5.4.0-196-generic, linux-modules-extra-5.4.0-196-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-18T10:02:32.752 INFO:teuthology.orchestra.run.smithi189.stdout:Recommends: thermald 2024-09-18T10:02:32.752 INFO:teuthology.orchestra.run.smithi189.stdout:Description: Generic Linux kernel image 2024-09-18T10:02:32.752 INFO:teuthology.orchestra.run.smithi189.stdout: This package will always depend on the latest generic kernel image 2024-09-18T10:02:32.752 INFO:teuthology.orchestra.run.smithi189.stdout: available. 2024-09-18T10:02:32.752 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.4.0-124-generic Expected: 5.4.0-196-generic 2024-09-18T10:02:32.752 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-18T10:02:32.752 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-18T10:02:32.752 INFO:teuthology.task.kernel:Installing distro kernel on mgr.x... 2024-09-18T10:02:32.753 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi189.front.sepia.ceph.com, path=None, version=distro) 2024-09-18T10:02:32.753 DEBUG:teuthology.orchestra.run.smithi189:> sudo apt-get clean 2024-09-18T10:02:32.753 INFO:teuthology.orchestra.run.smithi136.stdout:Found linux image: /boot/vmlinuz-5.4.0-196-generic 2024-09-18T10:02:32.777 INFO:teuthology.orchestra.run.smithi136.stdout:Found initrd image: /boot/initrd.img-5.4.0-196-generic 2024-09-18T10:02:32.832 DEBUG:teuthology.orchestra.run.smithi189:> sudo apt-get update 2024-09-18T10:02:32.967 INFO:teuthology.orchestra.run.smithi189.stdout:Hit:1 http://security.ubuntu.com/ubuntu focal-security InRelease 2024-09-18T10:02:33.084 INFO:teuthology.orchestra.run.smithi189.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal InRelease 2024-09-18T10:02:33.166 INFO:teuthology.orchestra.run.smithi189.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2024-09-18T10:02:33.225 INFO:teuthology.orchestra.run.smithi136.stdout:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-09-18T10:02:33.237 INFO:teuthology.orchestra.run.smithi136.stdout:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-09-18T10:02:33.253 INFO:teuthology.orchestra.run.smithi189.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2024-09-18T10:02:34.641 INFO:teuthology.orchestra.run.smithi154.stdout:done 2024-09-18T10:02:34.819 INFO:teuthology.orchestra.run.smithi189.stdout:Reading package lists... 2024-09-18T10:02:34.848 DEBUG:teuthology.orchestra.run.smithi189:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-18T10:02:34.921 INFO:teuthology.orchestra.run.smithi189.stdout:Reading package lists... 2024-09-18T10:02:35.056 INFO:teuthology.orchestra.run.smithi136.stdout:done 2024-09-18T10:02:35.116 DEBUG:teuthology.orchestra.run.smithi154:> dpkg -s linux-image-generic 2024-09-18T10:02:35.141 INFO:teuthology.orchestra.run.smithi189.stdout:Building dependency tree... 2024-09-18T10:02:35.142 INFO:teuthology.orchestra.run.smithi189.stdout:Reading state information... 2024-09-18T10:02:35.168 INFO:teuthology.orchestra.run.smithi154.stdout:Package: linux-image-generic 2024-09-18T10:02:35.168 INFO:teuthology.orchestra.run.smithi154.stdout:Status: install ok installed 2024-09-18T10:02:35.168 INFO:teuthology.orchestra.run.smithi154.stdout:Priority: optional 2024-09-18T10:02:35.168 INFO:teuthology.orchestra.run.smithi154.stdout:Section: kernel 2024-09-18T10:02:35.168 INFO:teuthology.orchestra.run.smithi154.stdout:Installed-Size: 20 2024-09-18T10:02:35.168 INFO:teuthology.orchestra.run.smithi154.stdout:Maintainer: Ubuntu Kernel Team 2024-09-18T10:02:35.168 INFO:teuthology.orchestra.run.smithi154.stdout:Architecture: amd64 2024-09-18T10:02:35.169 INFO:teuthology.orchestra.run.smithi154.stdout:Source: linux-meta 2024-09-18T10:02:35.169 INFO:teuthology.orchestra.run.smithi154.stdout:Version: 5.4.0.196.194 2024-09-18T10:02:35.169 INFO:teuthology.orchestra.run.smithi154.stdout:Provides: virtualbox-guest-modules (= 6.1.50-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.17) 2024-09-18T10:02:35.169 INFO:teuthology.orchestra.run.smithi154.stdout:Depends: linux-image-5.4.0-196-generic, linux-modules-extra-5.4.0-196-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-18T10:02:35.169 INFO:teuthology.orchestra.run.smithi154.stdout:Recommends: thermald 2024-09-18T10:02:35.169 INFO:teuthology.orchestra.run.smithi154.stdout:Description: Generic Linux kernel image 2024-09-18T10:02:35.169 INFO:teuthology.orchestra.run.smithi154.stdout: This package will always depend on the latest generic kernel image 2024-09-18T10:02:35.169 INFO:teuthology.orchestra.run.smithi154.stdout: available. 2024-09-18T10:02:35.169 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.4.0-124-generic Expected: 5.4.0-196-generic 2024-09-18T10:02:35.170 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-18T10:02:35.170 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-18T10:02:35.170 INFO:teuthology.task.kernel:Installing distro kernel on mon.c... 2024-09-18T10:02:35.170 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi154.front.sepia.ceph.com, path=None, version=distro) 2024-09-18T10:02:35.170 DEBUG:teuthology.orchestra.run.smithi154:> sudo apt-get clean 2024-09-18T10:02:35.260 DEBUG:teuthology.orchestra.run.smithi154:> sudo apt-get update 2024-09-18T10:02:35.374 INFO:teuthology.orchestra.run.smithi189.stdout:linux-image-generic is already the newest version (5.4.0.196.194). 2024-09-18T10:02:35.374 INFO:teuthology.orchestra.run.smithi189.stdout:0 upgraded, 0 newly installed, 0 to remove and 316 not upgraded. 2024-09-18T10:02:35.376 DEBUG:teuthology.orchestra.run.smithi189:> dpkg -s linux-image-generic 2024-09-18T10:02:35.393 INFO:teuthology.orchestra.run.smithi154.stdout:Hit:1 http://security.ubuntu.com/ubuntu focal-security InRelease 2024-09-18T10:02:35.399 INFO:teuthology.orchestra.run.smithi189.stdout:Package: linux-image-generic 2024-09-18T10:02:35.399 INFO:teuthology.orchestra.run.smithi189.stdout:Status: install ok installed 2024-09-18T10:02:35.399 INFO:teuthology.orchestra.run.smithi189.stdout:Priority: optional 2024-09-18T10:02:35.399 INFO:teuthology.orchestra.run.smithi189.stdout:Section: kernel 2024-09-18T10:02:35.399 INFO:teuthology.orchestra.run.smithi189.stdout:Installed-Size: 20 2024-09-18T10:02:35.400 INFO:teuthology.orchestra.run.smithi189.stdout:Maintainer: Ubuntu Kernel Team 2024-09-18T10:02:35.400 INFO:teuthology.orchestra.run.smithi189.stdout:Architecture: amd64 2024-09-18T10:02:35.400 INFO:teuthology.orchestra.run.smithi189.stdout:Source: linux-meta 2024-09-18T10:02:35.400 INFO:teuthology.orchestra.run.smithi189.stdout:Version: 5.4.0.196.194 2024-09-18T10:02:35.400 INFO:teuthology.orchestra.run.smithi189.stdout:Provides: virtualbox-guest-modules (= 6.1.50-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.17) 2024-09-18T10:02:35.400 INFO:teuthology.orchestra.run.smithi189.stdout:Depends: linux-image-5.4.0-196-generic, linux-modules-extra-5.4.0-196-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-18T10:02:35.400 INFO:teuthology.orchestra.run.smithi189.stdout:Recommends: thermald 2024-09-18T10:02:35.400 INFO:teuthology.orchestra.run.smithi189.stdout:Description: Generic Linux kernel image 2024-09-18T10:02:35.400 INFO:teuthology.orchestra.run.smithi189.stdout: This package will always depend on the latest generic kernel image 2024-09-18T10:02:35.400 INFO:teuthology.orchestra.run.smithi189.stdout: available. 2024-09-18T10:02:35.401 DEBUG:teuthology.orchestra.run.smithi189:> mktemp 2024-09-18T10:02:35.405 INFO:teuthology.orchestra.run.smithi189.stdout:/tmp/tmp.wqeNpc7wJ3 2024-09-18T10:02:35.405 DEBUG:teuthology.orchestra.run.smithi189:> sudo cp /boot/grub/grub.cfg /tmp/tmp.wqeNpc7wJ3 2024-09-18T10:02:35.416 DEBUG:teuthology.orchestra.run.smithi189:> sudo chmod 0666 /tmp/tmp.wqeNpc7wJ3 2024-09-18T10:02:35.495 DEBUG:teuthology.orchestra.remote:smithi189:/tmp/tmp.wqeNpc7wJ3 is 9KB 2024-09-18T10:02:35.504 DEBUG:teuthology.orchestra.run.smithi136:> dpkg -s linux-image-generic 2024-09-18T10:02:35.510 INFO:teuthology.orchestra.run.smithi154.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal InRelease 2024-09-18T10:02:35.512 DEBUG:teuthology.orchestra.run.smithi189:> rm -fr /tmp/tmp.wqeNpc7wJ3 2024-09-18T10:02:35.518 DEBUG:teuthology.orchestra.run.smithi189:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-18T10:02:35.561 INFO:teuthology.orchestra.run.smithi136.stdout:Package: linux-image-generic 2024-09-18T10:02:35.561 INFO:teuthology.orchestra.run.smithi136.stdout:Status: install ok installed 2024-09-18T10:02:35.562 INFO:teuthology.orchestra.run.smithi136.stdout:Priority: optional 2024-09-18T10:02:35.562 INFO:teuthology.orchestra.run.smithi136.stdout:Section: kernel 2024-09-18T10:02:35.562 INFO:teuthology.orchestra.run.smithi136.stdout:Installed-Size: 20 2024-09-18T10:02:35.562 INFO:teuthology.orchestra.run.smithi136.stdout:Maintainer: Ubuntu Kernel Team 2024-09-18T10:02:35.562 INFO:teuthology.orchestra.run.smithi136.stdout:Architecture: amd64 2024-09-18T10:02:35.562 INFO:teuthology.orchestra.run.smithi136.stdout:Source: linux-meta 2024-09-18T10:02:35.562 INFO:teuthology.orchestra.run.smithi136.stdout:Version: 5.4.0.196.194 2024-09-18T10:02:35.562 INFO:teuthology.orchestra.run.smithi136.stdout:Provides: virtualbox-guest-modules (= 6.1.50-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.17) 2024-09-18T10:02:35.562 INFO:teuthology.orchestra.run.smithi136.stdout:Depends: linux-image-5.4.0-196-generic, linux-modules-extra-5.4.0-196-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-18T10:02:35.562 INFO:teuthology.orchestra.run.smithi136.stdout:Recommends: thermald 2024-09-18T10:02:35.562 INFO:teuthology.orchestra.run.smithi136.stdout:Description: Generic Linux kernel image 2024-09-18T10:02:35.562 INFO:teuthology.orchestra.run.smithi136.stdout: This package will always depend on the latest generic kernel image 2024-09-18T10:02:35.563 INFO:teuthology.orchestra.run.smithi136.stdout: available. 2024-09-18T10:02:35.563 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.4.0-124-generic Expected: 5.4.0-196-generic 2024-09-18T10:02:35.563 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-18T10:02:35.563 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-18T10:02:35.563 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-18T10:02:35.563 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi136.front.sepia.ceph.com, path=None, version=distro) 2024-09-18T10:02:35.563 DEBUG:teuthology.orchestra.run.smithi136:> sudo apt-get clean 2024-09-18T10:02:35.577 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-18T10:02:35.577 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-18T10:02:35.577 DEBUG:teuthology.orchestra.run.smithi189:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-18T10:02:35.591 INFO:teuthology.orchestra.run.smithi154.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2024-09-18T10:02:35.609 INFO:teuthology.task.kernel:Distro Kernel Version: 5.4.0-196-generic 2024-09-18T10:02:35.609 DEBUG:teuthology.orchestra.run.smithi189:> sudo update-grub 2024-09-18T10:02:35.661 DEBUG:teuthology.orchestra.run.smithi136:> sudo apt-get update 2024-09-18T10:02:35.673 INFO:teuthology.orchestra.run.smithi154.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2024-09-18T10:02:35.808 INFO:teuthology.orchestra.run.smithi136.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2024-09-18T10:02:35.814 INFO:teuthology.orchestra.run.smithi136.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2024-09-18T10:02:35.832 INFO:teuthology.orchestra.run.smithi136.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2024-09-18T10:02:35.925 INFO:teuthology.orchestra.run.smithi136.stdout:Hit:4 http://security.ubuntu.com/ubuntu focal-security InRelease 2024-09-18T10:02:37.046 INFO:teuthology.orchestra.run.smithi189.stderr:Sourcing file `/etc/default/grub' 2024-09-18T10:02:37.058 INFO:teuthology.orchestra.run.smithi189.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-18T10:02:37.071 INFO:teuthology.orchestra.run.smithi189.stderr:Generating grub configuration file ... 2024-09-18T10:02:37.286 INFO:teuthology.orchestra.run.smithi154.stdout:Reading package lists... 2024-09-18T10:02:37.313 DEBUG:teuthology.orchestra.run.smithi154:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-18T10:02:37.324 INFO:teuthology.orchestra.run.smithi189.stderr:Found linux image: /boot/vmlinuz-5.4.0-196-generic 2024-09-18T10:02:37.335 INFO:teuthology.orchestra.run.smithi189.stderr:Found initrd image: /boot/initrd.img-5.4.0-196-generic 2024-09-18T10:02:37.384 INFO:teuthology.orchestra.run.smithi154.stdout:Reading package lists... 2024-09-18T10:02:37.602 INFO:teuthology.orchestra.run.smithi154.stdout:Building dependency tree... 2024-09-18T10:02:37.603 INFO:teuthology.orchestra.run.smithi154.stdout:Reading state information... 2024-09-18T10:02:37.660 INFO:teuthology.orchestra.run.smithi136.stdout:Reading package lists... 2024-09-18T10:02:37.688 DEBUG:teuthology.orchestra.run.smithi136:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-18T10:02:37.738 INFO:teuthology.orchestra.run.smithi189.stderr:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-09-18T10:02:37.749 INFO:teuthology.orchestra.run.smithi189.stderr:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-09-18T10:02:37.757 INFO:teuthology.orchestra.run.smithi136.stdout:Reading package lists... 2024-09-18T10:02:37.845 INFO:teuthology.orchestra.run.smithi154.stdout:linux-image-generic is already the newest version (5.4.0.196.194). 2024-09-18T10:02:37.845 INFO:teuthology.orchestra.run.smithi154.stdout:0 upgraded, 0 newly installed, 0 to remove and 316 not upgraded. 2024-09-18T10:02:37.847 DEBUG:teuthology.orchestra.run.smithi154:> dpkg -s linux-image-generic 2024-09-18T10:02:37.867 INFO:teuthology.orchestra.run.smithi154.stdout:Package: linux-image-generic 2024-09-18T10:02:37.867 INFO:teuthology.orchestra.run.smithi154.stdout:Status: install ok installed 2024-09-18T10:02:37.867 INFO:teuthology.orchestra.run.smithi154.stdout:Priority: optional 2024-09-18T10:02:37.867 INFO:teuthology.orchestra.run.smithi154.stdout:Section: kernel 2024-09-18T10:02:37.868 INFO:teuthology.orchestra.run.smithi154.stdout:Installed-Size: 20 2024-09-18T10:02:37.868 INFO:teuthology.orchestra.run.smithi154.stdout:Maintainer: Ubuntu Kernel Team 2024-09-18T10:02:37.868 INFO:teuthology.orchestra.run.smithi154.stdout:Architecture: amd64 2024-09-18T10:02:37.868 INFO:teuthology.orchestra.run.smithi154.stdout:Source: linux-meta 2024-09-18T10:02:37.868 INFO:teuthology.orchestra.run.smithi154.stdout:Version: 5.4.0.196.194 2024-09-18T10:02:37.868 INFO:teuthology.orchestra.run.smithi154.stdout:Provides: virtualbox-guest-modules (= 6.1.50-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.17) 2024-09-18T10:02:37.868 INFO:teuthology.orchestra.run.smithi154.stdout:Depends: linux-image-5.4.0-196-generic, linux-modules-extra-5.4.0-196-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-18T10:02:37.868 INFO:teuthology.orchestra.run.smithi154.stdout:Recommends: thermald 2024-09-18T10:02:37.868 INFO:teuthology.orchestra.run.smithi154.stdout:Description: Generic Linux kernel image 2024-09-18T10:02:37.868 INFO:teuthology.orchestra.run.smithi154.stdout: This package will always depend on the latest generic kernel image 2024-09-18T10:02:37.868 INFO:teuthology.orchestra.run.smithi154.stdout: available. 2024-09-18T10:02:37.869 DEBUG:teuthology.orchestra.run.smithi154:> mktemp 2024-09-18T10:02:37.875 INFO:teuthology.orchestra.run.smithi154.stdout:/tmp/tmp.X41xzmgnGh 2024-09-18T10:02:37.875 DEBUG:teuthology.orchestra.run.smithi154:> sudo cp /boot/grub/grub.cfg /tmp/tmp.X41xzmgnGh 2024-09-18T10:02:37.944 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod 0666 /tmp/tmp.X41xzmgnGh 2024-09-18T10:02:37.977 INFO:teuthology.orchestra.run.smithi136.stdout:Building dependency tree... 2024-09-18T10:02:37.977 INFO:teuthology.orchestra.run.smithi136.stdout:Reading state information... 2024-09-18T10:02:37.986 DEBUG:teuthology.orchestra.remote:smithi154:/tmp/tmp.X41xzmgnGh is 9KB 2024-09-18T10:02:38.000 DEBUG:teuthology.orchestra.run.smithi154:> rm -fr /tmp/tmp.X41xzmgnGh 2024-09-18T10:02:38.006 DEBUG:teuthology.orchestra.run.smithi154:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-18T10:02:38.063 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-09-18T10:02:38.063 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-18T10:02:38.063 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-18T10:02:38.096 INFO:teuthology.task.kernel:Distro Kernel Version: 5.4.0-196-generic 2024-09-18T10:02:38.097 DEBUG:teuthology.orchestra.run.smithi154:> sudo update-grub 2024-09-18T10:02:38.214 INFO:teuthology.orchestra.run.smithi136.stdout:linux-image-generic is already the newest version (5.4.0.196.194). 2024-09-18T10:02:38.214 INFO:teuthology.orchestra.run.smithi136.stdout:0 upgraded, 0 newly installed, 0 to remove and 316 not upgraded. 2024-09-18T10:02:38.216 DEBUG:teuthology.orchestra.run.smithi136:> dpkg -s linux-image-generic 2024-09-18T10:02:38.233 INFO:teuthology.orchestra.run.smithi136.stdout:Package: linux-image-generic 2024-09-18T10:02:38.233 INFO:teuthology.orchestra.run.smithi136.stdout:Status: install ok installed 2024-09-18T10:02:38.233 INFO:teuthology.orchestra.run.smithi136.stdout:Priority: optional 2024-09-18T10:02:38.233 INFO:teuthology.orchestra.run.smithi136.stdout:Section: kernel 2024-09-18T10:02:38.233 INFO:teuthology.orchestra.run.smithi136.stdout:Installed-Size: 20 2024-09-18T10:02:38.233 INFO:teuthology.orchestra.run.smithi136.stdout:Maintainer: Ubuntu Kernel Team 2024-09-18T10:02:38.234 INFO:teuthology.orchestra.run.smithi136.stdout:Architecture: amd64 2024-09-18T10:02:38.234 INFO:teuthology.orchestra.run.smithi136.stdout:Source: linux-meta 2024-09-18T10:02:38.234 INFO:teuthology.orchestra.run.smithi136.stdout:Version: 5.4.0.196.194 2024-09-18T10:02:38.234 INFO:teuthology.orchestra.run.smithi136.stdout:Provides: virtualbox-guest-modules (= 6.1.50-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.17) 2024-09-18T10:02:38.234 INFO:teuthology.orchestra.run.smithi136.stdout:Depends: linux-image-5.4.0-196-generic, linux-modules-extra-5.4.0-196-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-18T10:02:38.234 INFO:teuthology.orchestra.run.smithi136.stdout:Recommends: thermald 2024-09-18T10:02:38.234 INFO:teuthology.orchestra.run.smithi136.stdout:Description: Generic Linux kernel image 2024-09-18T10:02:38.234 INFO:teuthology.orchestra.run.smithi136.stdout: This package will always depend on the latest generic kernel image 2024-09-18T10:02:38.234 INFO:teuthology.orchestra.run.smithi136.stdout: available. 2024-09-18T10:02:38.235 DEBUG:teuthology.orchestra.run.smithi136:> mktemp 2024-09-18T10:02:38.239 INFO:teuthology.orchestra.run.smithi136.stdout:/tmp/tmp.0Zn8iudWnk 2024-09-18T10:02:38.239 DEBUG:teuthology.orchestra.run.smithi136:> sudo cp /boot/grub/grub.cfg /tmp/tmp.0Zn8iudWnk 2024-09-18T10:02:38.297 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod 0666 /tmp/tmp.0Zn8iudWnk 2024-09-18T10:02:38.329 DEBUG:teuthology.orchestra.remote:smithi136:/tmp/tmp.0Zn8iudWnk is 9KB 2024-09-18T10:02:38.342 DEBUG:teuthology.orchestra.run.smithi136:> rm -fr /tmp/tmp.0Zn8iudWnk 2024-09-18T10:02:38.349 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-18T10:02:38.405 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-09-18T10:02:38.405 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-18T10:02:38.405 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-18T10:02:38.439 INFO:teuthology.task.kernel:Distro Kernel Version: 5.4.0-196-generic 2024-09-18T10:02:38.440 DEBUG:teuthology.orchestra.run.smithi136:> sudo update-grub 2024-09-18T10:02:39.186 INFO:teuthology.orchestra.run.smithi189.stderr:done 2024-09-18T10:02:39.186 DEBUG:teuthology.orchestra.run.smithi189:> sudo shutdown -r now 2024-09-18T10:02:39.481 INFO:teuthology.orchestra.run.smithi154.stderr:Sourcing file `/etc/default/grub' 2024-09-18T10:02:39.490 INFO:teuthology.orchestra.run.smithi154.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-18T10:02:39.503 INFO:teuthology.orchestra.run.smithi154.stderr:Generating grub configuration file ... 2024-09-18T10:02:39.791 INFO:teuthology.orchestra.run.smithi154.stderr:Found linux image: /boot/vmlinuz-5.4.0-196-generic 2024-09-18T10:02:39.802 INFO:teuthology.orchestra.run.smithi154.stderr:Found initrd image: /boot/initrd.img-5.4.0-196-generic 2024-09-18T10:02:39.830 INFO:teuthology.orchestra.run.smithi136.stderr:Sourcing file `/etc/default/grub' 2024-09-18T10:02:39.837 INFO:teuthology.orchestra.run.smithi136.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-18T10:02:39.846 INFO:teuthology.orchestra.run.smithi136.stderr:Generating grub configuration file ... 2024-09-18T10:02:40.102 INFO:teuthology.orchestra.run.smithi136.stderr:Found linux image: /boot/vmlinuz-5.4.0-196-generic 2024-09-18T10:02:40.113 INFO:teuthology.orchestra.run.smithi136.stderr:Found initrd image: /boot/initrd.img-5.4.0-196-generic 2024-09-18T10:02:40.191 INFO:teuthology.orchestra.run.smithi154.stderr:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-09-18T10:02:40.202 INFO:teuthology.orchestra.run.smithi154.stderr:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-09-18T10:02:40.273 INFO:teuthology.orchestra.run.smithi029.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-18T10:02:40.273 INFO:teuthology.orchestra.run.smithi029.stdout:Sourcing file `/etc/default/grub' 2024-09-18T10:02:40.295 INFO:teuthology.orchestra.run.smithi029.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-18T10:02:40.320 INFO:teuthology.orchestra.run.smithi029.stdout:Generating grub configuration file ... 2024-09-18T10:02:40.488 INFO:teuthology.orchestra.run.smithi136.stderr:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-09-18T10:02:40.499 INFO:teuthology.orchestra.run.smithi136.stderr:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-09-18T10:02:40.651 INFO:teuthology.orchestra.run.smithi029.stdout:Found linux image: /boot/vmlinuz-5.4.0-196-generic 2024-09-18T10:02:40.679 INFO:teuthology.orchestra.run.smithi029.stdout:Found initrd image: /boot/initrd.img-5.4.0-196-generic 2024-09-18T10:02:41.108 INFO:teuthology.orchestra.run.smithi029.stdout:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-09-18T10:02:41.119 INFO:teuthology.orchestra.run.smithi029.stdout:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-09-18T10:02:41.288 INFO:teuthology.orchestra.run.smithi136.stderr:done 2024-09-18T10:02:41.290 DEBUG:teuthology.orchestra.run.smithi136:> sudo shutdown -r now 2024-09-18T10:02:41.611 INFO:teuthology.orchestra.run.smithi154.stderr:done 2024-09-18T10:02:41.613 DEBUG:teuthology.orchestra.run.smithi154:> sudo shutdown -r now 2024-09-18T10:02:43.918 INFO:teuthology.orchestra.run.smithi029.stdout:done 2024-09-18T10:02:44.415 DEBUG:teuthology.orchestra.run.smithi029:> dpkg -s linux-image-generic 2024-09-18T10:02:44.466 INFO:teuthology.orchestra.run.smithi029.stdout:Package: linux-image-generic 2024-09-18T10:02:44.466 INFO:teuthology.orchestra.run.smithi029.stdout:Status: install ok installed 2024-09-18T10:02:44.466 INFO:teuthology.orchestra.run.smithi029.stdout:Priority: optional 2024-09-18T10:02:44.466 INFO:teuthology.orchestra.run.smithi029.stdout:Section: kernel 2024-09-18T10:02:44.466 INFO:teuthology.orchestra.run.smithi029.stdout:Installed-Size: 20 2024-09-18T10:02:44.467 INFO:teuthology.orchestra.run.smithi029.stdout:Maintainer: Ubuntu Kernel Team 2024-09-18T10:02:44.467 INFO:teuthology.orchestra.run.smithi029.stdout:Architecture: amd64 2024-09-18T10:02:44.467 INFO:teuthology.orchestra.run.smithi029.stdout:Source: linux-meta 2024-09-18T10:02:44.467 INFO:teuthology.orchestra.run.smithi029.stdout:Version: 5.4.0.196.194 2024-09-18T10:02:44.467 INFO:teuthology.orchestra.run.smithi029.stdout:Provides: virtualbox-guest-modules (= 6.1.50-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.17) 2024-09-18T10:02:44.467 INFO:teuthology.orchestra.run.smithi029.stdout:Depends: linux-image-5.4.0-196-generic, linux-modules-extra-5.4.0-196-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-18T10:02:44.467 INFO:teuthology.orchestra.run.smithi029.stdout:Recommends: thermald 2024-09-18T10:02:44.467 INFO:teuthology.orchestra.run.smithi029.stdout:Description: Generic Linux kernel image 2024-09-18T10:02:44.467 INFO:teuthology.orchestra.run.smithi029.stdout: This package will always depend on the latest generic kernel image 2024-09-18T10:02:44.467 INFO:teuthology.orchestra.run.smithi029.stdout: available. 2024-09-18T10:02:44.468 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.4.0-124-generic Expected: 5.4.0-196-generic 2024-09-18T10:02:44.468 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-18T10:02:44.468 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-18T10:02:44.468 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-18T10:02:44.468 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi029.front.sepia.ceph.com, path=None, version=distro) 2024-09-18T10:02:44.468 DEBUG:teuthology.orchestra.run.smithi029:> sudo apt-get clean 2024-09-18T10:02:44.576 DEBUG:teuthology.orchestra.run.smithi029:> sudo apt-get update 2024-09-18T10:02:44.716 INFO:teuthology.orchestra.run.smithi029.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2024-09-18T10:02:44.722 INFO:teuthology.orchestra.run.smithi029.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2024-09-18T10:02:44.740 INFO:teuthology.orchestra.run.smithi029.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2024-09-18T10:02:44.830 INFO:teuthology.orchestra.run.smithi029.stdout:Hit:4 http://security.ubuntu.com/ubuntu focal-security InRelease 2024-09-18T10:02:46.677 INFO:teuthology.orchestra.run.smithi029.stdout:Reading package lists... 2024-09-18T10:02:46.705 DEBUG:teuthology.orchestra.run.smithi029:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-18T10:02:46.779 INFO:teuthology.orchestra.run.smithi029.stdout:Reading package lists... 2024-09-18T10:02:46.982 INFO:teuthology.orchestra.run.smithi029.stdout:Building dependency tree... 2024-09-18T10:02:46.982 INFO:teuthology.orchestra.run.smithi029.stdout:Reading state information... 2024-09-18T10:02:47.223 INFO:teuthology.orchestra.run.smithi029.stdout:linux-image-generic is already the newest version (5.4.0.196.194). 2024-09-18T10:02:47.223 INFO:teuthology.orchestra.run.smithi029.stdout:0 upgraded, 0 newly installed, 0 to remove and 316 not upgraded. 2024-09-18T10:02:47.225 DEBUG:teuthology.orchestra.run.smithi029:> dpkg -s linux-image-generic 2024-09-18T10:02:47.248 INFO:teuthology.orchestra.run.smithi029.stdout:Package: linux-image-generic 2024-09-18T10:02:47.248 INFO:teuthology.orchestra.run.smithi029.stdout:Status: install ok installed 2024-09-18T10:02:47.248 INFO:teuthology.orchestra.run.smithi029.stdout:Priority: optional 2024-09-18T10:02:47.248 INFO:teuthology.orchestra.run.smithi029.stdout:Section: kernel 2024-09-18T10:02:47.248 INFO:teuthology.orchestra.run.smithi029.stdout:Installed-Size: 20 2024-09-18T10:02:47.248 INFO:teuthology.orchestra.run.smithi029.stdout:Maintainer: Ubuntu Kernel Team 2024-09-18T10:02:47.248 INFO:teuthology.orchestra.run.smithi029.stdout:Architecture: amd64 2024-09-18T10:02:47.248 INFO:teuthology.orchestra.run.smithi029.stdout:Source: linux-meta 2024-09-18T10:02:47.248 INFO:teuthology.orchestra.run.smithi029.stdout:Version: 5.4.0.196.194 2024-09-18T10:02:47.248 INFO:teuthology.orchestra.run.smithi029.stdout:Provides: virtualbox-guest-modules (= 6.1.50-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.17) 2024-09-18T10:02:47.248 INFO:teuthology.orchestra.run.smithi029.stdout:Depends: linux-image-5.4.0-196-generic, linux-modules-extra-5.4.0-196-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-18T10:02:47.249 INFO:teuthology.orchestra.run.smithi029.stdout:Recommends: thermald 2024-09-18T10:02:47.249 INFO:teuthology.orchestra.run.smithi029.stdout:Description: Generic Linux kernel image 2024-09-18T10:02:47.249 INFO:teuthology.orchestra.run.smithi029.stdout: This package will always depend on the latest generic kernel image 2024-09-18T10:02:47.249 INFO:teuthology.orchestra.run.smithi029.stdout: available. 2024-09-18T10:02:47.249 DEBUG:teuthology.orchestra.run.smithi029:> mktemp 2024-09-18T10:02:47.255 INFO:teuthology.orchestra.run.smithi029.stdout:/tmp/tmp.noBvfMIlHC 2024-09-18T10:02:47.255 DEBUG:teuthology.orchestra.run.smithi029:> sudo cp /boot/grub/grub.cfg /tmp/tmp.noBvfMIlHC 2024-09-18T10:02:47.276 DEBUG:teuthology.orchestra.run.smithi029:> sudo chmod 0666 /tmp/tmp.noBvfMIlHC 2024-09-18T10:02:47.369 DEBUG:teuthology.orchestra.remote:smithi029:/tmp/tmp.noBvfMIlHC is 9KB 2024-09-18T10:02:47.385 DEBUG:teuthology.orchestra.run.smithi029:> rm -fr /tmp/tmp.noBvfMIlHC 2024-09-18T10:02:47.391 DEBUG:teuthology.orchestra.run.smithi029:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-18T10:02:47.450 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-09-18T10:02:47.450 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-18T10:02:47.450 DEBUG:teuthology.orchestra.run.smithi029:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-18T10:02:47.478 INFO:teuthology.task.kernel:Distro Kernel Version: 5.4.0-196-generic 2024-09-18T10:02:47.478 DEBUG:teuthology.orchestra.run.smithi029:> sudo update-grub 2024-09-18T10:02:49.980 INFO:teuthology.orchestra.run.smithi029.stderr:Sourcing file `/etc/default/grub' 2024-09-18T10:02:49.994 INFO:teuthology.orchestra.run.smithi029.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-18T10:02:50.019 INFO:teuthology.orchestra.run.smithi029.stderr:Generating grub configuration file ... 2024-09-18T10:02:50.362 INFO:teuthology.orchestra.run.smithi029.stderr:Found linux image: /boot/vmlinuz-5.4.0-196-generic 2024-09-18T10:02:50.374 INFO:teuthology.orchestra.run.smithi029.stderr:Found initrd image: /boot/initrd.img-5.4.0-196-generic 2024-09-18T10:02:50.783 INFO:teuthology.orchestra.run.smithi029.stderr:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-09-18T10:02:50.795 INFO:teuthology.orchestra.run.smithi029.stderr:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-09-18T10:02:52.250 INFO:teuthology.orchestra.run.smithi029.stderr:done 2024-09-18T10:02:52.251 DEBUG:teuthology.orchestra.run.smithi029:> sudo shutdown -r now 2024-09-18T10:03:09.205 INFO:teuthology.task.kernel:Checking client mgr.x for new kernel version... 2024-09-18T10:03:09.206 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi189.front.sepia.ceph.com' 2024-09-18T10:03:09.207 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi189.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-18T10:03:11.293 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-18T10:03:11.293 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi136.front.sepia.ceph.com' 2024-09-18T10:03:11.294 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-18T10:03:11.617 INFO:teuthology.task.kernel:Checking client mon.c for new kernel version... 2024-09-18T10:03:11.617 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi154.front.sepia.ceph.com' 2024-09-18T10:03:11.617 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-18T10:03:22.263 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-18T10:03:22.264 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi029.front.sepia.ceph.com' 2024-09-18T10:03:22.264 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi029.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-18T10:03:43.805 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.189 2024-09-18T10:03:46.109 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.154 2024-09-18T10:03:52.812 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi189.front.sepia.ceph.com' 2024-09-18T10:03:52.813 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi189.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-18T10:03:55.113 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi154.front.sepia.ceph.com' 2024-09-18T10:03:55.113 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-18T10:04:11.309 DEBUG:teuthology.orchestra.remote:timed out 2024-09-18T10:04:20.317 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi136.front.sepia.ceph.com' 2024-09-18T10:04:20.318 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-18T10:04:20.733 DEBUG:teuthology.orchestra.run.smithi136:> true 2024-09-18T10:04:21.822 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi136.front.sepia.ceph.com' 2024-09-18T10:04:21.822 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.4.0-196-generic"... 2024-09-18T10:04:21.822 DEBUG:teuthology.orchestra.run.smithi136:> uname -r 2024-09-18T10:04:21.829 INFO:teuthology.orchestra.run.smithi136.stdout:5.4.0-196-generic 2024-09-18T10:04:21.829 DEBUG:teuthology.task.kernel:current kernel version is 5.4.0-196-generic vs 5.4.0-196-generic 2024-09-18T10:04:21.829 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-18T10:04:21.830 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-18T10:04:22.266 DEBUG:teuthology.orchestra.remote:timed out 2024-09-18T10:04:22.831 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-18T10:04:22.831 DEBUG:teuthology.orchestra.run.smithi136:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-18T10:04:22.919 INFO:teuthology.orchestra.run.smithi136.stdout:ttyS1 2024-09-18T10:04:22.959 DEBUG:teuthology.parallel:result is None 2024-09-18T10:04:31.267 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi029.front.sepia.ceph.com' 2024-09-18T10:04:31.267 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi029.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-18T10:04:47.347 DEBUG:teuthology.orchestra.run.smithi029:> true 2024-09-18T10:04:48.393 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi029.front.sepia.ceph.com' 2024-09-18T10:04:48.393 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.4.0-196-generic"... 2024-09-18T10:04:48.393 DEBUG:teuthology.orchestra.run.smithi029:> uname -r 2024-09-18T10:04:48.400 INFO:teuthology.orchestra.run.smithi029.stdout:5.4.0-196-generic 2024-09-18T10:04:48.400 DEBUG:teuthology.task.kernel:current kernel version is 5.4.0-196-generic vs 5.4.0-196-generic 2024-09-18T10:04:48.400 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-18T10:04:48.400 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-18T10:04:49.402 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-18T10:04:49.402 DEBUG:teuthology.orchestra.run.smithi029:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-18T10:04:49.512 INFO:teuthology.orchestra.run.smithi029.stdout:ttyS1 2024-09-18T10:04:49.532 DEBUG:teuthology.parallel:result is None 2024-09-18T10:04:52.816 DEBUG:teuthology.orchestra.remote:timed out 2024-09-18T10:04:55.115 DEBUG:teuthology.orchestra.remote:timed out 2024-09-18T10:05:04.817 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi189.front.sepia.ceph.com' 2024-09-18T10:05:04.819 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi189.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-18T10:05:04.984 DEBUG:teuthology.orchestra.run.smithi189:> true 2024-09-18T10:05:05.565 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi189.front.sepia.ceph.com' 2024-09-18T10:05:05.566 INFO:teuthology.task.kernel:Checking kernel version of mgr.x, want "5.4.0-196-generic"... 2024-09-18T10:05:05.566 DEBUG:teuthology.orchestra.run.smithi189:> uname -r 2024-09-18T10:05:05.572 INFO:teuthology.orchestra.run.smithi189.stdout:5.4.0-196-generic 2024-09-18T10:05:05.572 DEBUG:teuthology.task.kernel:current kernel version is 5.4.0-196-generic vs 5.4.0-196-generic 2024-09-18T10:05:05.573 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-18T10:05:05.573 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-18T10:05:06.574 INFO:teuthology.task.kernel:Enabling kdb on mgr.x... 2024-09-18T10:05:06.574 DEBUG:teuthology.orchestra.run.smithi189:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-18T10:05:06.668 INFO:teuthology.orchestra.run.smithi189.stdout:ttyS1 2024-09-18T10:05:06.701 DEBUG:teuthology.parallel:result is None 2024-09-18T10:05:07.117 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi154.front.sepia.ceph.com' 2024-09-18T10:05:07.117 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-18T10:05:07.242 DEBUG:teuthology.orchestra.run.smithi154:> true 2024-09-18T10:05:07.738 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi154.front.sepia.ceph.com' 2024-09-18T10:05:07.738 INFO:teuthology.task.kernel:Checking kernel version of mon.c, want "5.4.0-196-generic"... 2024-09-18T10:05:07.739 DEBUG:teuthology.orchestra.run.smithi154:> uname -r 2024-09-18T10:05:07.745 INFO:teuthology.orchestra.run.smithi154.stdout:5.4.0-196-generic 2024-09-18T10:05:07.745 DEBUG:teuthology.task.kernel:current kernel version is 5.4.0-196-generic vs 5.4.0-196-generic 2024-09-18T10:05:07.745 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-18T10:05:07.745 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-18T10:05:08.746 INFO:teuthology.task.kernel:Enabling kdb on mon.c... 2024-09-18T10:05:08.747 DEBUG:teuthology.orchestra.run.smithi154:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-18T10:05:08.853 INFO:teuthology.orchestra.run.smithi154.stdout:ttyS1 2024-09-18T10:05:08.877 DEBUG:teuthology.parallel:result is None 2024-09-18T10:05:08.877 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-18T10:05:08.884 INFO:teuthology.task.internal:Creating test directory... 2024-09-18T10:05:08.884 DEBUG:teuthology.orchestra.run.smithi029:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-18T10:05:08.887 DEBUG:teuthology.orchestra.run.smithi136:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-18T10:05:08.889 DEBUG:teuthology.orchestra.run.smithi154:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-18T10:05:08.892 DEBUG:teuthology.orchestra.run.smithi189:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-18T10:05:08.899 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-18T10:05:08.906 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-18T10:05:08.916 INFO:teuthology.task.internal:Creating archive directory... 2024-09-18T10:05:08.916 DEBUG:teuthology.orchestra.run.smithi029:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-18T10:05:08.935 DEBUG:teuthology.orchestra.run.smithi136:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-18T10:05:08.938 DEBUG:teuthology.orchestra.run.smithi154:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-18T10:05:08.940 DEBUG:teuthology.orchestra.run.smithi189:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-18T10:05:08.965 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-18T10:05:08.972 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-18T10:05:08.972 DEBUG:teuthology.orchestra.run.smithi029:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-18T10:05:08.974 DEBUG:teuthology.orchestra.run.smithi136:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-18T10:05:08.976 DEBUG:teuthology.orchestra.run.smithi154:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-18T10:05:08.979 DEBUG:teuthology.orchestra.run.smithi189:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-18T10:05:08.994 INFO:teuthology.orchestra.run.smithi154.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-18T10:05:08.996 INFO:teuthology.orchestra.run.smithi136.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-18T10:05:09.001 INFO:teuthology.orchestra.run.smithi029.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-18T10:05:09.005 INFO:teuthology.orchestra.run.smithi154.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-18T10:05:09.007 INFO:teuthology.orchestra.run.smithi136.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-18T10:05:09.012 INFO:teuthology.orchestra.run.smithi029.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-18T10:05:09.020 INFO:teuthology.orchestra.run.smithi189.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-18T10:05:09.029 INFO:teuthology.orchestra.run.smithi189.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-18T10:05:09.030 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-18T10:05:09.040 INFO:teuthology.task.internal:Configuring sudo... 2024-09-18T10:05:09.040 DEBUG:teuthology.orchestra.run.smithi029:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-18T10:05:09.060 DEBUG:teuthology.orchestra.run.smithi136:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-18T10:05:09.062 DEBUG:teuthology.orchestra.run.smithi154:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-18T10:05:09.065 DEBUG:teuthology.orchestra.run.smithi189:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-18T10:05:09.080 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-18T10:05:09.089 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-18T10:05:09.089 DEBUG:teuthology.orchestra.run.smithi029:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-18T10:05:09.092 DEBUG:teuthology.orchestra.run.smithi136:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-18T10:05:09.094 DEBUG:teuthology.orchestra.run.smithi154:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-18T10:05:09.096 DEBUG:teuthology.orchestra.run.smithi189:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-18T10:05:09.103 DEBUG:teuthology.orchestra.run.smithi029:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-18T10:05:09.109 DEBUG:teuthology.orchestra.run.smithi029:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-18T10:05:09.157 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-09-18T10:05:09.157 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-18T10:05:09.179 DEBUG:teuthology.orchestra.run.smithi136:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-18T10:05:09.185 DEBUG:teuthology.orchestra.run.smithi136:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-18T10:05:09.235 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-09-18T10:05:09.235 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-18T10:05:09.255 DEBUG:teuthology.orchestra.run.smithi154:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-18T10:05:09.263 DEBUG:teuthology.orchestra.run.smithi154:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-18T10:05:09.313 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-09-18T10:05:09.313 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-18T10:05:09.334 DEBUG:teuthology.orchestra.run.smithi189:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-18T10:05:09.341 DEBUG:teuthology.orchestra.run.smithi189:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-18T10:05:09.348 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-18T10:05:09.348 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-18T10:05:09.376 DEBUG:teuthology.orchestra.run.smithi029:> sudo service rsyslog restart 2024-09-18T10:05:09.379 DEBUG:teuthology.orchestra.run.smithi136:> sudo service rsyslog restart 2024-09-18T10:05:09.381 DEBUG:teuthology.orchestra.run.smithi154:> sudo service rsyslog restart 2024-09-18T10:05:09.383 DEBUG:teuthology.orchestra.run.smithi189:> sudo service rsyslog restart 2024-09-18T10:05:09.466 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-18T10:05:09.474 INFO:teuthology.task.internal:Starting timer... 2024-09-18T10:05:09.474 INFO:teuthology.run_tasks:Running task pcp... 2024-09-18T10:05:09.489 INFO:teuthology.run_tasks:Running task selinux... 2024-09-18T10:05:09.498 DEBUG:teuthology.task.selinux:Excluding smithi029: OS 'ubuntu' does not support SELinux 2024-09-18T10:05:09.498 DEBUG:teuthology.task.selinux:Excluding smithi136: OS 'ubuntu' does not support SELinux 2024-09-18T10:05:09.499 DEBUG:teuthology.task.selinux:Excluding smithi154: OS 'ubuntu' does not support SELinux 2024-09-18T10:05:09.499 DEBUG:teuthology.task.selinux:Excluding smithi189: OS 'ubuntu' does not support SELinux 2024-09-18T10:05:09.499 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-18T10:05:09.499 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-09-18T10:05:09.499 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-18T10:05:09.499 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-18T10:05:09.509 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-18T10:05:09.522 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-18T10:05:09.524 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi029.front.sepia.ceph.com,smithi136.front.sepia.ceph.com,smithi154.front.sepia.ceph.com,smithi189.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-18T10:10:55.643 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi029.front.sepia.ceph.com'), Remote(name='ubuntu@smithi136.front.sepia.ceph.com'), Remote(name='ubuntu@smithi154.front.sepia.ceph.com'), Remote(name='ubuntu@smithi189.front.sepia.ceph.com')] 2024-09-18T10:10:55.645 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi029.front.sepia.ceph.com' 2024-09-18T10:10:55.646 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi029.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-18T10:10:55.751 DEBUG:teuthology.orchestra.run.smithi029:> true 2024-09-18T10:10:55.874 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi029.front.sepia.ceph.com' 2024-09-18T10:10:55.875 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi136.front.sepia.ceph.com' 2024-09-18T10:10:55.875 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-18T10:10:55.960 DEBUG:teuthology.orchestra.run.smithi136:> true 2024-09-18T10:10:56.086 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi136.front.sepia.ceph.com' 2024-09-18T10:10:56.087 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi154.front.sepia.ceph.com' 2024-09-18T10:10:56.087 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-18T10:10:56.173 DEBUG:teuthology.orchestra.run.smithi154:> true 2024-09-18T10:10:56.294 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi154.front.sepia.ceph.com' 2024-09-18T10:10:56.294 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi189.front.sepia.ceph.com' 2024-09-18T10:10:56.295 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi189.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-18T10:10:56.383 DEBUG:teuthology.orchestra.run.smithi189:> true 2024-09-18T10:10:56.510 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi189.front.sepia.ceph.com' 2024-09-18T10:10:56.511 INFO:teuthology.run_tasks:Running task clock... 2024-09-18T10:10:56.521 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-18T10:10:56.521 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-18T10:10:56.522 DEBUG:teuthology.orchestra.run.smithi029:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-18T10:10:56.524 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-18T10:10:56.524 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-18T10:10:56.526 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-18T10:10:56.527 DEBUG:teuthology.orchestra.run.smithi154:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-18T10:10:56.529 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-18T10:10:56.529 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-18T10:10:56.556 INFO:teuthology.orchestra.run.smithi029.stdout:18 Sep 10:10:56 ntpd[13725]: ntpd 4.2.8p12@1.3728-o (1): Starting 2024-09-18T10:10:56.556 INFO:teuthology.orchestra.run.smithi029.stdout:18 Sep 10:10:56 ntpd[13725]: Command line: ntpd -gq 2024-09-18T10:10:56.557 INFO:teuthology.orchestra.run.smithi029.stdout:18 Sep 10:10:56 ntpd[13725]: proto: precision = 0.120 usec (-23) 2024-09-18T10:10:56.558 INFO:teuthology.orchestra.run.smithi029.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-18T10:10:56.558 INFO:teuthology.orchestra.run.smithi029.stdout:18 Sep 10:10:56 ntpd[13725]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-18T10:10:56.558 INFO:teuthology.orchestra.run.smithi029.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-18T10:10:56.559 INFO:teuthology.orchestra.run.smithi029.stdout:18 Sep 10:10:56 ntpd[13725]: restrict ::: KOD does nothing without LIMITED. 2024-09-18T10:10:56.559 INFO:teuthology.orchestra.run.smithi029.stdout:18 Sep 10:10:56 ntpd[13725]: Listen and drop on 0 v6wildcard [::]:123 2024-09-18T10:10:56.560 INFO:teuthology.orchestra.run.smithi029.stdout:18 Sep 10:10:56 ntpd[13725]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-18T10:10:56.560 INFO:teuthology.orchestra.run.smithi029.stdout:18 Sep 10:10:56 ntpd[13725]: Listen normally on 2 lo 127.0.0.1:123 2024-09-18T10:10:56.560 INFO:teuthology.orchestra.run.smithi029.stdout:18 Sep 10:10:56 ntpd[13725]: Listen normally on 3 ens1f0 172.21.15.29:123 2024-09-18T10:10:56.560 INFO:teuthology.orchestra.run.smithi029.stdout:18 Sep 10:10:56 ntpd[13725]: Listen normally on 4 lo [::1]:123 2024-09-18T10:10:56.560 INFO:teuthology.orchestra.run.smithi029.stdout:18 Sep 10:10:56 ntpd[13725]: Listen normally on 5 ens1f0 [fe80::ec4:7aff:febd:125a%4]:123 2024-09-18T10:10:56.560 INFO:teuthology.orchestra.run.smithi029.stdout:18 Sep 10:10:56 ntpd[13725]: Listening on routing socket on fd #22 for interface updates 2024-09-18T10:10:56.561 INFO:teuthology.orchestra.run.smithi136.stdout:18 Sep 10:10:56 ntpd[13764]: ntpd 4.2.8p12@1.3728-o (1): Starting 2024-09-18T10:10:56.561 INFO:teuthology.orchestra.run.smithi136.stdout:18 Sep 10:10:56 ntpd[13764]: Command line: ntpd -gq 2024-09-18T10:10:56.561 INFO:teuthology.orchestra.run.smithi136.stdout:18 Sep 10:10:56 ntpd[13764]: proto: precision = 0.293 usec (-22) 2024-09-18T10:10:56.561 INFO:teuthology.orchestra.run.smithi136.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-18T10:10:56.561 INFO:teuthology.orchestra.run.smithi189.stdout:18 Sep 10:10:56 ntpd[13753]: ntpd 4.2.8p12@1.3728-o (1): Starting 2024-09-18T10:10:56.562 INFO:teuthology.orchestra.run.smithi189.stdout:18 Sep 10:10:56 ntpd[13753]: Command line: ntpd -gq 2024-09-18T10:10:56.562 INFO:teuthology.orchestra.run.smithi189.stdout:18 Sep 10:10:56 ntpd[13753]: proto: precision = 0.108 usec (-23) 2024-09-18T10:10:56.562 INFO:teuthology.orchestra.run.smithi189.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-18T10:10:56.562 INFO:teuthology.orchestra.run.smithi136.stdout:18 Sep 10:10:56 ntpd[13764]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-18T10:10:56.563 INFO:teuthology.orchestra.run.smithi136.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-18T10:10:56.563 INFO:teuthology.orchestra.run.smithi189.stdout:18 Sep 10:10:56 ntpd[13753]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-18T10:10:56.563 INFO:teuthology.orchestra.run.smithi189.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-18T10:10:56.565 INFO:teuthology.orchestra.run.smithi136.stdout:18 Sep 10:10:56 ntpd[13764]: restrict ::: KOD does nothing without LIMITED. 2024-09-18T10:10:56.565 INFO:teuthology.orchestra.run.smithi136.stdout:18 Sep 10:10:56 ntpd[13764]: Listen and drop on 0 v6wildcard [::]:123 2024-09-18T10:10:56.565 INFO:teuthology.orchestra.run.smithi136.stdout:18 Sep 10:10:56 ntpd[13764]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-18T10:10:56.565 INFO:teuthology.orchestra.run.smithi136.stdout:18 Sep 10:10:56 ntpd[13764]: Listen normally on 2 lo 127.0.0.1:123 2024-09-18T10:10:56.565 INFO:teuthology.orchestra.run.smithi136.stdout:18 Sep 10:10:56 ntpd[13764]: Listen normally on 3 enp3s0f1 172.21.15.136:123 2024-09-18T10:10:56.565 INFO:teuthology.orchestra.run.smithi136.stdout:18 Sep 10:10:56 ntpd[13764]: Listen normally on 4 lo [::1]:123 2024-09-18T10:10:56.565 INFO:teuthology.orchestra.run.smithi136.stdout:18 Sep 10:10:56 ntpd[13764]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:6e43%5]:123 2024-09-18T10:10:56.565 INFO:teuthology.orchestra.run.smithi136.stdout:18 Sep 10:10:56 ntpd[13764]: Listening on routing socket on fd #22 for interface updates 2024-09-18T10:10:56.566 INFO:teuthology.orchestra.run.smithi189.stdout:18 Sep 10:10:56 ntpd[13753]: restrict ::: KOD does nothing without LIMITED. 2024-09-18T10:10:56.566 INFO:teuthology.orchestra.run.smithi189.stdout:18 Sep 10:10:56 ntpd[13753]: Listen and drop on 0 v6wildcard [::]:123 2024-09-18T10:10:56.566 INFO:teuthology.orchestra.run.smithi189.stdout:18 Sep 10:10:56 ntpd[13753]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-18T10:10:56.566 INFO:teuthology.orchestra.run.smithi189.stdout:18 Sep 10:10:56 ntpd[13753]: Listen normally on 2 lo 127.0.0.1:123 2024-09-18T10:10:56.566 INFO:teuthology.orchestra.run.smithi189.stdout:18 Sep 10:10:56 ntpd[13753]: Listen normally on 3 enp3s0f1 172.21.15.189:123 2024-09-18T10:10:56.566 INFO:teuthology.orchestra.run.smithi189.stdout:18 Sep 10:10:56 ntpd[13753]: Listen normally on 4 lo [::1]:123 2024-09-18T10:10:56.566 INFO:teuthology.orchestra.run.smithi189.stdout:18 Sep 10:10:56 ntpd[13753]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:6df1%5]:123 2024-09-18T10:10:56.566 INFO:teuthology.orchestra.run.smithi189.stdout:18 Sep 10:10:56 ntpd[13753]: Listening on routing socket on fd #22 for interface updates 2024-09-18T10:10:56.567 INFO:teuthology.orchestra.run.smithi154.stdout:18 Sep 10:10:56 ntpd[13766]: ntpd 4.2.8p12@1.3728-o (1): Starting 2024-09-18T10:10:56.567 INFO:teuthology.orchestra.run.smithi154.stdout:18 Sep 10:10:56 ntpd[13766]: Command line: ntpd -gq 2024-09-18T10:10:56.567 INFO:teuthology.orchestra.run.smithi154.stdout:18 Sep 10:10:56 ntpd[13766]: proto: precision = 0.087 usec (-23) 2024-09-18T10:10:56.567 INFO:teuthology.orchestra.run.smithi154.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-18T10:10:56.567 INFO:teuthology.orchestra.run.smithi154.stdout:18 Sep 10:10:56 ntpd[13766]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-18T10:10:56.568 INFO:teuthology.orchestra.run.smithi154.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-18T10:10:56.569 INFO:teuthology.orchestra.run.smithi154.stdout:18 Sep 10:10:56 ntpd[13766]: restrict ::: KOD does nothing without LIMITED. 2024-09-18T10:10:56.569 INFO:teuthology.orchestra.run.smithi154.stdout:18 Sep 10:10:56 ntpd[13766]: Listen and drop on 0 v6wildcard [::]:123 2024-09-18T10:10:56.569 INFO:teuthology.orchestra.run.smithi154.stdout:18 Sep 10:10:56 ntpd[13766]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-18T10:10:56.569 INFO:teuthology.orchestra.run.smithi154.stdout:18 Sep 10:10:56 ntpd[13766]: Listen normally on 2 lo 127.0.0.1:123 2024-09-18T10:10:56.569 INFO:teuthology.orchestra.run.smithi154.stdout:18 Sep 10:10:56 ntpd[13766]: Listen normally on 3 enp3s0f1 172.21.15.154:123 2024-09-18T10:10:56.569 INFO:teuthology.orchestra.run.smithi154.stdout:18 Sep 10:10:56 ntpd[13766]: Listen normally on 4 lo [::1]:123 2024-09-18T10:10:56.569 INFO:teuthology.orchestra.run.smithi154.stdout:18 Sep 10:10:56 ntpd[13766]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:7761%5]:123 2024-09-18T10:10:56.569 INFO:teuthology.orchestra.run.smithi154.stdout:18 Sep 10:10:56 ntpd[13766]: Listening on routing socket on fd #22 for interface updates 2024-09-18T10:10:57.557 INFO:teuthology.orchestra.run.smithi029.stderr:18 Sep 10:10:57 ntpd[13725]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T10:10:57.557 INFO:teuthology.orchestra.run.smithi029.stderr:18 Sep 10:10:57 ntpd[13725]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T10:10:57.557 INFO:teuthology.orchestra.run.smithi029.stderr:18 Sep 10:10:57 ntpd[13725]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T10:10:57.557 INFO:teuthology.orchestra.run.smithi029.stderr:18 Sep 10:10:57 ntpd[13725]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T10:10:57.558 INFO:teuthology.orchestra.run.smithi029.stderr:18 Sep 10:10:57 ntpd[13725]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T10:10:57.558 INFO:teuthology.orchestra.run.smithi029.stderr:18 Sep 10:10:57 ntpd[13725]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T10:10:57.558 INFO:teuthology.orchestra.run.smithi029.stderr:18 Sep 10:10:57 ntpd[13725]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-18T10:10:57.559 INFO:teuthology.orchestra.run.smithi029.stderr:18 Sep 10:10:57 ntpd[13725]: can't open /var/log/ntpstats/peerstats.20240918: Permission denied 2024-09-18T10:10:57.559 INFO:teuthology.orchestra.run.smithi029.stderr:18 Sep 10:10:57 ntpd[13725]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T10:10:57.559 INFO:teuthology.orchestra.run.smithi029.stderr:18 Sep 10:10:57 ntpd[13725]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T10:10:57.559 INFO:teuthology.orchestra.run.smithi029.stderr:18 Sep 10:10:57 ntpd[13725]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-18T10:10:57.559 INFO:teuthology.orchestra.run.smithi029.stderr:18 Sep 10:10:57 ntpd[13725]: can't open /var/log/ntpstats/peerstats.20240918: Permission denied 2024-09-18T10:10:57.559 INFO:teuthology.orchestra.run.smithi189.stderr:18 Sep 10:10:57 ntpd[13753]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T10:10:57.560 INFO:teuthology.orchestra.run.smithi189.stderr:18 Sep 10:10:57 ntpd[13753]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T10:10:57.561 INFO:teuthology.orchestra.run.smithi189.stderr:18 Sep 10:10:57 ntpd[13753]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T10:10:57.561 INFO:teuthology.orchestra.run.smithi189.stderr:18 Sep 10:10:57 ntpd[13753]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T10:10:57.561 INFO:teuthology.orchestra.run.smithi189.stderr:18 Sep 10:10:57 ntpd[13753]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-18T10:10:57.561 INFO:teuthology.orchestra.run.smithi189.stderr:18 Sep 10:10:57 ntpd[13753]: can't open /var/log/ntpstats/peerstats.20240918: Permission denied 2024-09-18T10:10:57.562 INFO:teuthology.orchestra.run.smithi136.stderr:18 Sep 10:10:57 ntpd[13764]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T10:10:57.562 INFO:teuthology.orchestra.run.smithi136.stderr:18 Sep 10:10:57 ntpd[13764]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T10:10:57.562 INFO:teuthology.orchestra.run.smithi136.stderr:18 Sep 10:10:57 ntpd[13764]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T10:10:57.563 INFO:teuthology.orchestra.run.smithi136.stderr:18 Sep 10:10:57 ntpd[13764]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T10:10:57.563 INFO:teuthology.orchestra.run.smithi136.stderr:18 Sep 10:10:57 ntpd[13764]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T10:10:57.563 INFO:teuthology.orchestra.run.smithi136.stderr:18 Sep 10:10:57 ntpd[13764]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T10:10:57.563 INFO:teuthology.orchestra.run.smithi136.stderr:18 Sep 10:10:57 ntpd[13764]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T10:10:57.563 INFO:teuthology.orchestra.run.smithi136.stderr:18 Sep 10:10:57 ntpd[13764]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T10:10:57.563 INFO:teuthology.orchestra.run.smithi136.stderr:18 Sep 10:10:57 ntpd[13764]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-18T10:10:57.563 INFO:teuthology.orchestra.run.smithi136.stderr:18 Sep 10:10:57 ntpd[13764]: can't open /var/log/ntpstats/peerstats.20240918: Permission denied 2024-09-18T10:10:57.563 INFO:teuthology.orchestra.run.smithi136.stderr:18 Sep 10:10:57 ntpd[13764]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T10:10:57.563 INFO:teuthology.orchestra.run.smithi136.stderr:18 Sep 10:10:57 ntpd[13764]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T10:10:57.563 INFO:teuthology.orchestra.run.smithi136.stderr:18 Sep 10:10:57 ntpd[13764]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-18T10:10:57.563 INFO:teuthology.orchestra.run.smithi136.stderr:18 Sep 10:10:57 ntpd[13764]: can't open /var/log/ntpstats/peerstats.20240918: Permission denied 2024-09-18T10:10:57.563 INFO:teuthology.orchestra.run.smithi136.stderr:18 Sep 10:10:57 ntpd[13764]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T10:10:57.563 INFO:teuthology.orchestra.run.smithi136.stderr:18 Sep 10:10:57 ntpd[13764]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T10:10:57.564 INFO:teuthology.orchestra.run.smithi136.stderr:18 Sep 10:10:57 ntpd[13764]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-18T10:10:57.564 INFO:teuthology.orchestra.run.smithi136.stderr:18 Sep 10:10:57 ntpd[13764]: can't open /var/log/ntpstats/peerstats.20240918: Permission denied 2024-09-18T10:10:57.565 INFO:teuthology.orchestra.run.smithi154.stderr:18 Sep 10:10:57 ntpd[13766]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T10:10:57.565 INFO:teuthology.orchestra.run.smithi154.stderr:18 Sep 10:10:57 ntpd[13766]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T10:10:57.565 INFO:teuthology.orchestra.run.smithi154.stderr:18 Sep 10:10:57 ntpd[13766]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T10:10:57.565 INFO:teuthology.orchestra.run.smithi154.stderr:18 Sep 10:10:57 ntpd[13766]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T10:10:57.565 INFO:teuthology.orchestra.run.smithi154.stderr:18 Sep 10:10:57 ntpd[13766]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T10:10:57.565 INFO:teuthology.orchestra.run.smithi154.stderr:18 Sep 10:10:57 ntpd[13766]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T10:10:57.565 INFO:teuthology.orchestra.run.smithi154.stderr:18 Sep 10:10:57 ntpd[13766]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-18T10:10:57.565 INFO:teuthology.orchestra.run.smithi154.stderr:18 Sep 10:10:57 ntpd[13766]: can't open /var/log/ntpstats/peerstats.20240918: Permission denied 2024-09-18T10:10:58.560 INFO:teuthology.orchestra.run.smithi189.stderr:18 Sep 10:10:58 ntpd[13753]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T10:10:58.560 INFO:teuthology.orchestra.run.smithi189.stderr:18 Sep 10:10:58 ntpd[13753]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T10:10:58.560 INFO:teuthology.orchestra.run.smithi189.stderr:18 Sep 10:10:58 ntpd[13753]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T10:10:58.560 INFO:teuthology.orchestra.run.smithi189.stderr:18 Sep 10:10:58 ntpd[13753]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T10:10:58.561 INFO:teuthology.orchestra.run.smithi189.stderr:18 Sep 10:10:58 ntpd[13753]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-18T10:10:58.561 INFO:teuthology.orchestra.run.smithi189.stderr:18 Sep 10:10:58 ntpd[13753]: can't open /var/log/ntpstats/peerstats.20240918: Permission denied 2024-09-18T10:10:59.558 INFO:teuthology.orchestra.run.smithi029.stderr:18 Sep 10:10:59 ntpd[13725]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T10:10:59.558 INFO:teuthology.orchestra.run.smithi029.stderr:18 Sep 10:10:59 ntpd[13725]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T10:10:59.558 INFO:teuthology.orchestra.run.smithi029.stderr:18 Sep 10:10:59 ntpd[13725]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T10:10:59.558 INFO:teuthology.orchestra.run.smithi029.stderr:18 Sep 10:10:59 ntpd[13725]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T10:10:59.558 INFO:teuthology.orchestra.run.smithi029.stderr:18 Sep 10:10:59 ntpd[13725]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T10:10:59.558 INFO:teuthology.orchestra.run.smithi029.stderr:18 Sep 10:10:59 ntpd[13725]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T10:10:59.559 INFO:teuthology.orchestra.run.smithi029.stderr:18 Sep 10:10:59 ntpd[13725]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T10:10:59.559 INFO:teuthology.orchestra.run.smithi029.stderr:18 Sep 10:10:59 ntpd[13725]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T10:10:59.559 INFO:teuthology.orchestra.run.smithi029.stderr:18 Sep 10:10:59 ntpd[13725]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-18T10:10:59.559 INFO:teuthology.orchestra.run.smithi029.stderr:18 Sep 10:10:59 ntpd[13725]: can't open /var/log/ntpstats/peerstats.20240918: Permission denied 2024-09-18T10:10:59.559 INFO:teuthology.orchestra.run.smithi029.stderr:18 Sep 10:10:59 ntpd[13725]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T10:10:59.559 INFO:teuthology.orchestra.run.smithi029.stderr:18 Sep 10:10:59 ntpd[13725]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T10:10:59.559 INFO:teuthology.orchestra.run.smithi029.stderr:18 Sep 10:10:59 ntpd[13725]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-18T10:10:59.559 INFO:teuthology.orchestra.run.smithi029.stderr:18 Sep 10:10:59 ntpd[13725]: can't open /var/log/ntpstats/peerstats.20240918: Permission denied 2024-09-18T10:10:59.559 INFO:teuthology.orchestra.run.smithi029.stderr:18 Sep 10:10:59 ntpd[13725]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T10:10:59.559 INFO:teuthology.orchestra.run.smithi029.stderr:18 Sep 10:10:59 ntpd[13725]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T10:10:59.559 INFO:teuthology.orchestra.run.smithi029.stderr:18 Sep 10:10:59 ntpd[13725]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-18T10:10:59.559 INFO:teuthology.orchestra.run.smithi029.stderr:18 Sep 10:10:59 ntpd[13725]: can't open /var/log/ntpstats/peerstats.20240918: Permission denied 2024-09-18T10:10:59.561 INFO:teuthology.orchestra.run.smithi189.stderr:18 Sep 10:10:59 ntpd[13753]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T10:10:59.561 INFO:teuthology.orchestra.run.smithi189.stderr:18 Sep 10:10:59 ntpd[13753]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T10:10:59.561 INFO:teuthology.orchestra.run.smithi189.stderr:18 Sep 10:10:59 ntpd[13753]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T10:10:59.561 INFO:teuthology.orchestra.run.smithi189.stderr:18 Sep 10:10:59 ntpd[13753]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T10:10:59.561 INFO:teuthology.orchestra.run.smithi189.stderr:18 Sep 10:10:59 ntpd[13753]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-18T10:10:59.561 INFO:teuthology.orchestra.run.smithi189.stderr:18 Sep 10:10:59 ntpd[13753]: can't open /var/log/ntpstats/peerstats.20240918: Permission denied 2024-09-18T10:10:59.562 INFO:teuthology.orchestra.run.smithi136.stderr:18 Sep 10:10:59 ntpd[13764]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T10:10:59.563 INFO:teuthology.orchestra.run.smithi136.stderr:18 Sep 10:10:59 ntpd[13764]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T10:10:59.563 INFO:teuthology.orchestra.run.smithi136.stderr:18 Sep 10:10:59 ntpd[13764]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T10:10:59.563 INFO:teuthology.orchestra.run.smithi136.stderr:18 Sep 10:10:59 ntpd[13764]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T10:10:59.563 INFO:teuthology.orchestra.run.smithi136.stderr:18 Sep 10:10:59 ntpd[13764]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T10:10:59.563 INFO:teuthology.orchestra.run.smithi136.stderr:18 Sep 10:10:59 ntpd[13764]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T10:10:59.563 INFO:teuthology.orchestra.run.smithi136.stderr:18 Sep 10:10:59 ntpd[13764]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T10:10:59.563 INFO:teuthology.orchestra.run.smithi136.stderr:18 Sep 10:10:59 ntpd[13764]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T10:10:59.563 INFO:teuthology.orchestra.run.smithi136.stderr:18 Sep 10:10:59 ntpd[13764]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-18T10:10:59.563 INFO:teuthology.orchestra.run.smithi136.stderr:18 Sep 10:10:59 ntpd[13764]: can't open /var/log/ntpstats/peerstats.20240918: Permission denied 2024-09-18T10:10:59.563 INFO:teuthology.orchestra.run.smithi136.stderr:18 Sep 10:10:59 ntpd[13764]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T10:10:59.563 INFO:teuthology.orchestra.run.smithi136.stderr:18 Sep 10:10:59 ntpd[13764]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T10:10:59.563 INFO:teuthology.orchestra.run.smithi136.stderr:18 Sep 10:10:59 ntpd[13764]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-18T10:10:59.563 INFO:teuthology.orchestra.run.smithi136.stderr:18 Sep 10:10:59 ntpd[13764]: can't open /var/log/ntpstats/peerstats.20240918: Permission denied 2024-09-18T10:10:59.564 INFO:teuthology.orchestra.run.smithi136.stderr:18 Sep 10:10:59 ntpd[13764]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T10:10:59.564 INFO:teuthology.orchestra.run.smithi136.stderr:18 Sep 10:10:59 ntpd[13764]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T10:10:59.564 INFO:teuthology.orchestra.run.smithi136.stderr:18 Sep 10:10:59 ntpd[13764]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-18T10:10:59.564 INFO:teuthology.orchestra.run.smithi136.stderr:18 Sep 10:10:59 ntpd[13764]: can't open /var/log/ntpstats/peerstats.20240918: Permission denied 2024-09-18T10:10:59.565 INFO:teuthology.orchestra.run.smithi154.stderr:18 Sep 10:10:59 ntpd[13766]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T10:10:59.566 INFO:teuthology.orchestra.run.smithi154.stderr:18 Sep 10:10:59 ntpd[13766]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T10:10:59.566 INFO:teuthology.orchestra.run.smithi154.stderr:18 Sep 10:10:59 ntpd[13766]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T10:10:59.566 INFO:teuthology.orchestra.run.smithi154.stderr:18 Sep 10:10:59 ntpd[13766]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T10:10:59.566 INFO:teuthology.orchestra.run.smithi154.stderr:18 Sep 10:10:59 ntpd[13766]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T10:10:59.566 INFO:teuthology.orchestra.run.smithi154.stderr:18 Sep 10:10:59 ntpd[13766]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T10:10:59.566 INFO:teuthology.orchestra.run.smithi154.stderr:18 Sep 10:10:59 ntpd[13766]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T10:10:59.566 INFO:teuthology.orchestra.run.smithi154.stderr:18 Sep 10:10:59 ntpd[13766]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T10:10:59.566 INFO:teuthology.orchestra.run.smithi154.stderr:18 Sep 10:10:59 ntpd[13766]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-18T10:10:59.566 INFO:teuthology.orchestra.run.smithi154.stderr:18 Sep 10:10:59 ntpd[13766]: can't open /var/log/ntpstats/peerstats.20240918: Permission denied 2024-09-18T10:10:59.566 INFO:teuthology.orchestra.run.smithi154.stderr:18 Sep 10:10:59 ntpd[13766]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T10:10:59.566 INFO:teuthology.orchestra.run.smithi154.stderr:18 Sep 10:10:59 ntpd[13766]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T10:10:59.566 INFO:teuthology.orchestra.run.smithi154.stderr:18 Sep 10:10:59 ntpd[13766]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-18T10:10:59.566 INFO:teuthology.orchestra.run.smithi154.stderr:18 Sep 10:10:59 ntpd[13766]: can't open /var/log/ntpstats/peerstats.20240918: Permission denied 2024-09-18T10:10:59.567 INFO:teuthology.orchestra.run.smithi154.stderr:18 Sep 10:10:59 ntpd[13766]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T10:10:59.567 INFO:teuthology.orchestra.run.smithi154.stderr:18 Sep 10:10:59 ntpd[13766]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T10:10:59.567 INFO:teuthology.orchestra.run.smithi154.stderr:18 Sep 10:10:59 ntpd[13766]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-18T10:10:59.567 INFO:teuthology.orchestra.run.smithi154.stderr:18 Sep 10:10:59 ntpd[13766]: can't open /var/log/ntpstats/peerstats.20240918: Permission denied 2024-09-18T10:11:00.560 INFO:teuthology.orchestra.run.smithi189.stderr:18 Sep 10:11:00 ntpd[13753]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T10:11:00.561 INFO:teuthology.orchestra.run.smithi189.stderr:18 Sep 10:11:00 ntpd[13753]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T10:11:00.561 INFO:teuthology.orchestra.run.smithi189.stderr:18 Sep 10:11:00 ntpd[13753]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T10:11:00.561 INFO:teuthology.orchestra.run.smithi189.stderr:18 Sep 10:11:00 ntpd[13753]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T10:11:00.561 INFO:teuthology.orchestra.run.smithi189.stderr:18 Sep 10:11:00 ntpd[13753]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T10:11:00.561 INFO:teuthology.orchestra.run.smithi189.stderr:18 Sep 10:11:00 ntpd[13753]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T10:11:00.561 INFO:teuthology.orchestra.run.smithi189.stderr:18 Sep 10:11:00 ntpd[13753]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-18T10:11:00.561 INFO:teuthology.orchestra.run.smithi189.stderr:18 Sep 10:11:00 ntpd[13753]: can't open /var/log/ntpstats/peerstats.20240918: Permission denied 2024-09-18T10:11:01.559 INFO:teuthology.orchestra.run.smithi029.stderr:18 Sep 10:11:01 ntpd[13725]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T10:11:01.559 INFO:teuthology.orchestra.run.smithi029.stderr:18 Sep 10:11:01 ntpd[13725]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T10:11:01.559 INFO:teuthology.orchestra.run.smithi029.stderr:18 Sep 10:11:01 ntpd[13725]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T10:11:01.559 INFO:teuthology.orchestra.run.smithi029.stderr:18 Sep 10:11:01 ntpd[13725]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T10:11:01.559 INFO:teuthology.orchestra.run.smithi029.stderr:18 Sep 10:11:01 ntpd[13725]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T10:11:01.559 INFO:teuthology.orchestra.run.smithi029.stderr:18 Sep 10:11:01 ntpd[13725]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T10:11:01.559 INFO:teuthology.orchestra.run.smithi029.stderr:18 Sep 10:11:01 ntpd[13725]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T10:11:01.559 INFO:teuthology.orchestra.run.smithi029.stderr:18 Sep 10:11:01 ntpd[13725]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T10:11:01.559 INFO:teuthology.orchestra.run.smithi029.stderr:18 Sep 10:11:01 ntpd[13725]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T10:11:01.559 INFO:teuthology.orchestra.run.smithi029.stderr:18 Sep 10:11:01 ntpd[13725]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T10:11:01.560 INFO:teuthology.orchestra.run.smithi029.stderr:18 Sep 10:11:01 ntpd[13725]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-18T10:11:01.560 INFO:teuthology.orchestra.run.smithi029.stderr:18 Sep 10:11:01 ntpd[13725]: can't open /var/log/ntpstats/peerstats.20240918: Permission denied 2024-09-18T10:11:01.560 INFO:teuthology.orchestra.run.smithi029.stderr:18 Sep 10:11:01 ntpd[13725]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T10:11:01.560 INFO:teuthology.orchestra.run.smithi029.stderr:18 Sep 10:11:01 ntpd[13725]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T10:11:01.560 INFO:teuthology.orchestra.run.smithi029.stderr:18 Sep 10:11:01 ntpd[13725]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-18T10:11:01.560 INFO:teuthology.orchestra.run.smithi029.stderr:18 Sep 10:11:01 ntpd[13725]: can't open /var/log/ntpstats/peerstats.20240918: Permission denied 2024-09-18T10:11:01.560 INFO:teuthology.orchestra.run.smithi029.stderr:18 Sep 10:11:01 ntpd[13725]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T10:11:01.560 INFO:teuthology.orchestra.run.smithi029.stderr:18 Sep 10:11:01 ntpd[13725]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T10:11:01.560 INFO:teuthology.orchestra.run.smithi029.stderr:18 Sep 10:11:01 ntpd[13725]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-18T10:11:01.560 INFO:teuthology.orchestra.run.smithi029.stderr:18 Sep 10:11:01 ntpd[13725]: can't open /var/log/ntpstats/peerstats.20240918: Permission denied 2024-09-18T10:11:01.563 INFO:teuthology.orchestra.run.smithi189.stderr:18 Sep 10:11:01 ntpd[13753]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T10:11:01.564 INFO:teuthology.orchestra.run.smithi189.stderr:18 Sep 10:11:01 ntpd[13753]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T10:11:01.564 INFO:teuthology.orchestra.run.smithi189.stderr:18 Sep 10:11:01 ntpd[13753]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T10:11:01.564 INFO:teuthology.orchestra.run.smithi189.stderr:18 Sep 10:11:01 ntpd[13753]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T10:11:01.564 INFO:teuthology.orchestra.run.smithi189.stderr:18 Sep 10:11:01 ntpd[13753]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T10:11:01.564 INFO:teuthology.orchestra.run.smithi189.stderr:18 Sep 10:11:01 ntpd[13753]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T10:11:01.564 INFO:teuthology.orchestra.run.smithi189.stderr:18 Sep 10:11:01 ntpd[13753]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-18T10:11:01.564 INFO:teuthology.orchestra.run.smithi189.stderr:18 Sep 10:11:01 ntpd[13753]: can't open /var/log/ntpstats/peerstats.20240918: Permission denied 2024-09-18T10:11:01.564 INFO:teuthology.orchestra.run.smithi189.stderr:18 Sep 10:11:01 ntpd[13753]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T10:11:01.564 INFO:teuthology.orchestra.run.smithi189.stderr:18 Sep 10:11:01 ntpd[13753]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T10:11:01.564 INFO:teuthology.orchestra.run.smithi189.stderr:18 Sep 10:11:01 ntpd[13753]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-18T10:11:01.564 INFO:teuthology.orchestra.run.smithi189.stderr:18 Sep 10:11:01 ntpd[13753]: can't open /var/log/ntpstats/peerstats.20240918: Permission denied 2024-09-18T10:11:01.565 INFO:teuthology.orchestra.run.smithi136.stderr:18 Sep 10:11:01 ntpd[13764]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T10:11:01.566 INFO:teuthology.orchestra.run.smithi136.stderr:18 Sep 10:11:01 ntpd[13764]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T10:11:01.566 INFO:teuthology.orchestra.run.smithi136.stderr:18 Sep 10:11:01 ntpd[13764]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T10:11:01.566 INFO:teuthology.orchestra.run.smithi136.stderr:18 Sep 10:11:01 ntpd[13764]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T10:11:01.566 INFO:teuthology.orchestra.run.smithi136.stderr:18 Sep 10:11:01 ntpd[13764]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T10:11:01.566 INFO:teuthology.orchestra.run.smithi136.stderr:18 Sep 10:11:01 ntpd[13764]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T10:11:01.566 INFO:teuthology.orchestra.run.smithi136.stderr:18 Sep 10:11:01 ntpd[13764]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T10:11:01.566 INFO:teuthology.orchestra.run.smithi136.stderr:18 Sep 10:11:01 ntpd[13764]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T10:11:01.566 INFO:teuthology.orchestra.run.smithi136.stderr:18 Sep 10:11:01 ntpd[13764]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T10:11:01.566 INFO:teuthology.orchestra.run.smithi136.stderr:18 Sep 10:11:01 ntpd[13764]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T10:11:01.566 INFO:teuthology.orchestra.run.smithi136.stderr:18 Sep 10:11:01 ntpd[13764]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-18T10:11:01.566 INFO:teuthology.orchestra.run.smithi136.stderr:18 Sep 10:11:01 ntpd[13764]: can't open /var/log/ntpstats/peerstats.20240918: Permission denied 2024-09-18T10:11:01.566 INFO:teuthology.orchestra.run.smithi136.stderr:18 Sep 10:11:01 ntpd[13764]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T10:11:01.567 INFO:teuthology.orchestra.run.smithi136.stderr:18 Sep 10:11:01 ntpd[13764]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T10:11:01.567 INFO:teuthology.orchestra.run.smithi136.stderr:18 Sep 10:11:01 ntpd[13764]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-18T10:11:01.567 INFO:teuthology.orchestra.run.smithi136.stderr:18 Sep 10:11:01 ntpd[13764]: can't open /var/log/ntpstats/peerstats.20240918: Permission denied 2024-09-18T10:11:01.567 INFO:teuthology.orchestra.run.smithi136.stderr:18 Sep 10:11:01 ntpd[13764]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T10:11:01.567 INFO:teuthology.orchestra.run.smithi136.stderr:18 Sep 10:11:01 ntpd[13764]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T10:11:01.567 INFO:teuthology.orchestra.run.smithi136.stderr:18 Sep 10:11:01 ntpd[13764]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-18T10:11:01.567 INFO:teuthology.orchestra.run.smithi136.stderr:18 Sep 10:11:01 ntpd[13764]: can't open /var/log/ntpstats/peerstats.20240918: Permission denied 2024-09-18T10:11:01.569 INFO:teuthology.orchestra.run.smithi154.stderr:18 Sep 10:11:01 ntpd[13766]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T10:11:01.569 INFO:teuthology.orchestra.run.smithi154.stderr:18 Sep 10:11:01 ntpd[13766]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T10:11:01.569 INFO:teuthology.orchestra.run.smithi154.stderr:18 Sep 10:11:01 ntpd[13766]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T10:11:01.569 INFO:teuthology.orchestra.run.smithi154.stderr:18 Sep 10:11:01 ntpd[13766]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T10:11:01.569 INFO:teuthology.orchestra.run.smithi154.stderr:18 Sep 10:11:01 ntpd[13766]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T10:11:01.569 INFO:teuthology.orchestra.run.smithi154.stderr:18 Sep 10:11:01 ntpd[13766]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T10:11:01.569 INFO:teuthology.orchestra.run.smithi154.stderr:18 Sep 10:11:01 ntpd[13766]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T10:11:01.569 INFO:teuthology.orchestra.run.smithi154.stderr:18 Sep 10:11:01 ntpd[13766]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T10:11:01.569 INFO:teuthology.orchestra.run.smithi154.stderr:18 Sep 10:11:01 ntpd[13766]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-18T10:11:01.569 INFO:teuthology.orchestra.run.smithi154.stderr:18 Sep 10:11:01 ntpd[13766]: can't open /var/log/ntpstats/peerstats.20240918: Permission denied 2024-09-18T10:11:01.570 INFO:teuthology.orchestra.run.smithi154.stderr:18 Sep 10:11:01 ntpd[13766]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T10:11:01.570 INFO:teuthology.orchestra.run.smithi154.stderr:18 Sep 10:11:01 ntpd[13766]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T10:11:01.570 INFO:teuthology.orchestra.run.smithi154.stderr:18 Sep 10:11:01 ntpd[13766]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-18T10:11:01.570 INFO:teuthology.orchestra.run.smithi154.stderr:18 Sep 10:11:01 ntpd[13766]: can't open /var/log/ntpstats/peerstats.20240918: Permission denied 2024-09-18T10:11:01.570 INFO:teuthology.orchestra.run.smithi154.stderr:18 Sep 10:11:01 ntpd[13766]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T10:11:01.570 INFO:teuthology.orchestra.run.smithi154.stderr:18 Sep 10:11:01 ntpd[13766]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T10:11:01.570 INFO:teuthology.orchestra.run.smithi154.stderr:18 Sep 10:11:01 ntpd[13766]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-18T10:11:01.570 INFO:teuthology.orchestra.run.smithi154.stderr:18 Sep 10:11:01 ntpd[13766]: can't open /var/log/ntpstats/peerstats.20240918: Permission denied 2024-09-18T10:11:02.560 INFO:teuthology.orchestra.run.smithi189.stderr:18 Sep 10:11:02 ntpd[13753]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T10:11:02.560 INFO:teuthology.orchestra.run.smithi189.stderr:18 Sep 10:11:02 ntpd[13753]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T10:11:02.560 INFO:teuthology.orchestra.run.smithi189.stderr:18 Sep 10:11:02 ntpd[13753]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T10:11:02.560 INFO:teuthology.orchestra.run.smithi189.stderr:18 Sep 10:11:02 ntpd[13753]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T10:11:02.561 INFO:teuthology.orchestra.run.smithi189.stderr:18 Sep 10:11:02 ntpd[13753]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-18T10:11:02.561 INFO:teuthology.orchestra.run.smithi189.stderr:18 Sep 10:11:02 ntpd[13753]: can't open /var/log/ntpstats/peerstats.20240918: Permission denied 2024-09-18T10:11:03.557 INFO:teuthology.orchestra.run.smithi029.stderr:18 Sep 10:11:03 ntpd[13725]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T10:11:03.558 INFO:teuthology.orchestra.run.smithi029.stderr:18 Sep 10:11:03 ntpd[13725]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T10:11:03.558 INFO:teuthology.orchestra.run.smithi029.stderr:18 Sep 10:11:03 ntpd[13725]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T10:11:03.558 INFO:teuthology.orchestra.run.smithi029.stderr:18 Sep 10:11:03 ntpd[13725]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T10:11:03.558 INFO:teuthology.orchestra.run.smithi029.stderr:18 Sep 10:11:03 ntpd[13725]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T10:11:03.558 INFO:teuthology.orchestra.run.smithi029.stderr:18 Sep 10:11:03 ntpd[13725]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T10:11:03.558 INFO:teuthology.orchestra.run.smithi029.stderr:18 Sep 10:11:03 ntpd[13725]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T10:11:03.558 INFO:teuthology.orchestra.run.smithi029.stderr:18 Sep 10:11:03 ntpd[13725]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T10:11:03.558 INFO:teuthology.orchestra.run.smithi029.stderr:18 Sep 10:11:03 ntpd[13725]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-18T10:11:03.558 INFO:teuthology.orchestra.run.smithi029.stderr:18 Sep 10:11:03 ntpd[13725]: can't open /var/log/ntpstats/peerstats.20240918: Permission denied 2024-09-18T10:11:03.559 INFO:teuthology.orchestra.run.smithi029.stdout:18 Sep 10:11:03 ntpd[13725]: ntpd: time slew +0.001198 s 2024-09-18T10:11:03.559 INFO:teuthology.orchestra.run.smithi029.stdout:ntpd: time slew +0.001198s 2024-09-18T10:11:03.559 INFO:teuthology.orchestra.run.smithi029.stderr:18 Sep 10:11:03 ntpd[13725]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-18T10:11:03.559 INFO:teuthology.orchestra.run.smithi029.stderr:18 Sep 10:11:03 ntpd[13725]: can't open /var/log/ntpstats/loopstats.20240918: Permission denied 2024-09-18T10:11:03.562 INFO:teuthology.orchestra.run.smithi189.stderr:18 Sep 10:11:03 ntpd[13753]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T10:11:03.562 INFO:teuthology.orchestra.run.smithi189.stderr:18 Sep 10:11:03 ntpd[13753]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T10:11:03.562 INFO:teuthology.orchestra.run.smithi189.stderr:18 Sep 10:11:03 ntpd[13753]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T10:11:03.562 INFO:teuthology.orchestra.run.smithi189.stderr:18 Sep 10:11:03 ntpd[13753]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T10:11:03.562 INFO:teuthology.orchestra.run.smithi189.stderr:18 Sep 10:11:03 ntpd[13753]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T10:11:03.562 INFO:teuthology.orchestra.run.smithi189.stderr:18 Sep 10:11:03 ntpd[13753]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T10:11:03.562 INFO:teuthology.orchestra.run.smithi189.stderr:18 Sep 10:11:03 ntpd[13753]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-18T10:11:03.562 INFO:teuthology.orchestra.run.smithi189.stderr:18 Sep 10:11:03 ntpd[13753]: can't open /var/log/ntpstats/peerstats.20240918: Permission denied 2024-09-18T10:11:03.563 INFO:teuthology.orchestra.run.smithi136.stderr:18 Sep 10:11:03 ntpd[13764]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T10:11:03.563 INFO:teuthology.orchestra.run.smithi136.stderr:18 Sep 10:11:03 ntpd[13764]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T10:11:03.563 INFO:teuthology.orchestra.run.smithi136.stderr:18 Sep 10:11:03 ntpd[13764]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T10:11:03.563 INFO:teuthology.orchestra.run.smithi136.stderr:18 Sep 10:11:03 ntpd[13764]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T10:11:03.563 INFO:teuthology.orchestra.run.smithi136.stderr:18 Sep 10:11:03 ntpd[13764]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T10:11:03.563 INFO:teuthology.orchestra.run.smithi136.stderr:18 Sep 10:11:03 ntpd[13764]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T10:11:03.563 INFO:teuthology.orchestra.run.smithi136.stderr:18 Sep 10:11:03 ntpd[13764]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T10:11:03.563 INFO:teuthology.orchestra.run.smithi136.stderr:18 Sep 10:11:03 ntpd[13764]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T10:11:03.563 INFO:teuthology.orchestra.run.smithi136.stderr:18 Sep 10:11:03 ntpd[13764]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-18T10:11:03.563 INFO:teuthology.orchestra.run.smithi136.stderr:18 Sep 10:11:03 ntpd[13764]: can't open /var/log/ntpstats/peerstats.20240918: Permission denied 2024-09-18T10:11:03.564 INFO:teuthology.orchestra.run.smithi189.stdout:18 Sep 10:11:03 ntpd[13753]: ntpd: time slew +0.000627 s 2024-09-18T10:11:03.564 INFO:teuthology.orchestra.run.smithi189.stdout:ntpd: time slew +0.000627s 2024-09-18T10:11:03.564 INFO:teuthology.orchestra.run.smithi136.stdout:18 Sep 10:11:03 ntpd[13764]: ntpd: time slew +0.000374 s 2024-09-18T10:11:03.564 INFO:teuthology.orchestra.run.smithi136.stdout:ntpd: time slew +0.000374s 2024-09-18T10:11:03.565 INFO:teuthology.orchestra.run.smithi189.stderr:18 Sep 10:11:03 ntpd[13753]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-18T10:11:03.565 INFO:teuthology.orchestra.run.smithi189.stderr:18 Sep 10:11:03 ntpd[13753]: can't open /var/log/ntpstats/loopstats.20240918: Permission denied 2024-09-18T10:11:03.565 INFO:teuthology.orchestra.run.smithi136.stderr:18 Sep 10:11:03 ntpd[13764]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-18T10:11:03.565 INFO:teuthology.orchestra.run.smithi136.stderr:18 Sep 10:11:03 ntpd[13764]: can't open /var/log/ntpstats/loopstats.20240918: Permission denied 2024-09-18T10:11:03.567 INFO:teuthology.orchestra.run.smithi154.stderr:18 Sep 10:11:03 ntpd[13766]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T10:11:03.567 INFO:teuthology.orchestra.run.smithi154.stderr:18 Sep 10:11:03 ntpd[13766]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T10:11:03.567 INFO:teuthology.orchestra.run.smithi154.stderr:18 Sep 10:11:03 ntpd[13766]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T10:11:03.567 INFO:teuthology.orchestra.run.smithi154.stderr:18 Sep 10:11:03 ntpd[13766]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T10:11:03.567 INFO:teuthology.orchestra.run.smithi154.stderr:18 Sep 10:11:03 ntpd[13766]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T10:11:03.567 INFO:teuthology.orchestra.run.smithi154.stderr:18 Sep 10:11:03 ntpd[13766]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T10:11:03.567 INFO:teuthology.orchestra.run.smithi154.stderr:18 Sep 10:11:03 ntpd[13766]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T10:11:03.567 INFO:teuthology.orchestra.run.smithi154.stderr:18 Sep 10:11:03 ntpd[13766]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T10:11:03.568 INFO:teuthology.orchestra.run.smithi154.stderr:18 Sep 10:11:03 ntpd[13766]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-18T10:11:03.568 INFO:teuthology.orchestra.run.smithi154.stderr:18 Sep 10:11:03 ntpd[13766]: can't open /var/log/ntpstats/peerstats.20240918: Permission denied 2024-09-18T10:11:03.568 INFO:teuthology.orchestra.run.smithi154.stderr:18 Sep 10:11:03 ntpd[13766]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T10:11:03.568 INFO:teuthology.orchestra.run.smithi154.stderr:18 Sep 10:11:03 ntpd[13766]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T10:11:03.568 INFO:teuthology.orchestra.run.smithi154.stderr:18 Sep 10:11:03 ntpd[13766]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-18T10:11:03.568 INFO:teuthology.orchestra.run.smithi154.stderr:18 Sep 10:11:03 ntpd[13766]: can't open /var/log/ntpstats/peerstats.20240918: Permission denied 2024-09-18T10:11:03.568 INFO:teuthology.orchestra.run.smithi154.stderr:18 Sep 10:11:03 ntpd[13766]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T10:11:03.568 INFO:teuthology.orchestra.run.smithi154.stderr:18 Sep 10:11:03 ntpd[13766]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T10:11:03.568 INFO:teuthology.orchestra.run.smithi154.stderr:18 Sep 10:11:03 ntpd[13766]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-18T10:11:03.568 INFO:teuthology.orchestra.run.smithi154.stderr:18 Sep 10:11:03 ntpd[13766]: can't open /var/log/ntpstats/peerstats.20240918: Permission denied 2024-09-18T10:11:03.568 INFO:teuthology.orchestra.run.smithi154.stdout:18 Sep 10:11:03 ntpd[13766]: ntpd: time slew -0.002134 s 2024-09-18T10:11:03.569 INFO:teuthology.orchestra.run.smithi154.stdout:ntpd: time slew -0.002134s 2024-09-18T10:11:03.569 INFO:teuthology.orchestra.run.smithi154.stderr:18 Sep 10:11:03 ntpd[13766]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-18T10:11:03.569 INFO:teuthology.orchestra.run.smithi154.stderr:18 Sep 10:11:03 ntpd[13766]: can't open /var/log/ntpstats/loopstats.20240918: Permission denied 2024-09-18T10:11:03.627 INFO:teuthology.orchestra.run.smithi136.stdout: remote refid st t when poll reach delay offset jitter 2024-09-18T10:11:03.627 INFO:teuthology.orchestra.run.smithi136.stdout:============================================================================== 2024-09-18T10:11:03.627 INFO:teuthology.orchestra.run.smithi136.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-18T10:11:03.627 INFO:teuthology.orchestra.run.smithi136.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-18T10:11:03.627 INFO:teuthology.orchestra.run.smithi136.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-18T10:11:03.627 INFO:teuthology.orchestra.run.smithi136.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-18T10:11:03.631 INFO:teuthology.orchestra.run.smithi029.stdout: remote refid st t when poll reach delay offset jitter 2024-09-18T10:11:03.631 INFO:teuthology.orchestra.run.smithi029.stdout:============================================================================== 2024-09-18T10:11:03.631 INFO:teuthology.orchestra.run.smithi029.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-18T10:11:03.631 INFO:teuthology.orchestra.run.smithi029.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-18T10:11:03.631 INFO:teuthology.orchestra.run.smithi029.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-18T10:11:03.631 INFO:teuthology.orchestra.run.smithi029.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-18T10:11:03.633 INFO:teuthology.orchestra.run.smithi154.stdout: remote refid st t when poll reach delay offset jitter 2024-09-18T10:11:03.633 INFO:teuthology.orchestra.run.smithi154.stdout:============================================================================== 2024-09-18T10:11:03.633 INFO:teuthology.orchestra.run.smithi154.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-18T10:11:03.633 INFO:teuthology.orchestra.run.smithi154.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-18T10:11:03.633 INFO:teuthology.orchestra.run.smithi154.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-18T10:11:03.633 INFO:teuthology.orchestra.run.smithi154.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-18T10:11:03.638 INFO:teuthology.orchestra.run.smithi189.stdout: remote refid st t when poll reach delay offset jitter 2024-09-18T10:11:03.638 INFO:teuthology.orchestra.run.smithi189.stdout:============================================================================== 2024-09-18T10:11:03.638 INFO:teuthology.orchestra.run.smithi189.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-18T10:11:03.638 INFO:teuthology.orchestra.run.smithi189.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-18T10:11:03.638 INFO:teuthology.orchestra.run.smithi189.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-18T10:11:03.638 INFO:teuthology.orchestra.run.smithi189.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-18T10:11:03.640 INFO:teuthology.run_tasks:Running task install... 2024-09-18T10:11:03.650 DEBUG:teuthology.task.install:project ceph 2024-09-18T10:11:03.650 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '7061ea6ca50908d104e9f3979dc615e118d9e153'}} 2024-09-18T10:11:03.651 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '7061ea6ca50908d104e9f3979dc615e118d9e153'} 2024-09-18T10:11:03.651 INFO:teuthology.task.install:Using flavor: default 2024-09-18T10:11:03.656 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-18T10:11:03.656 INFO:teuthology.task.install:extra packages: [] 2024-09-18T10:11:03.657 DEBUG:teuthology.orchestra.run.smithi029:> sudo apt-key list | grep Ceph 2024-09-18T10:11:03.657 DEBUG:teuthology.orchestra.run.smithi136:> sudo apt-key list | grep Ceph 2024-09-18T10:11:03.658 DEBUG:teuthology.orchestra.run.smithi154:> sudo apt-key list | grep Ceph 2024-09-18T10:11:03.658 DEBUG:teuthology.orchestra.run.smithi189:> sudo apt-key list | grep Ceph 2024-09-18T10:11:03.774 INFO:teuthology.orchestra.run.smithi136.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2024-09-18T10:11:03.777 INFO:teuthology.orchestra.run.smithi154.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2024-09-18T10:11:03.780 INFO:teuthology.orchestra.run.smithi029.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2024-09-18T10:11:03.789 INFO:teuthology.orchestra.run.smithi189.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2024-09-18T10:11:03.835 INFO:teuthology.orchestra.run.smithi136.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-18T10:11:03.835 INFO:teuthology.orchestra.run.smithi136.stdout:uid [ unknown] Ceph.com (release key) 2024-09-18T10:11:03.836 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-09-18T10:11:03.837 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=7061ea6ca50908d104e9f3979dc615e118d9e153 2024-09-18T10:11:03.842 INFO:teuthology.orchestra.run.smithi154.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-18T10:11:03.842 INFO:teuthology.orchestra.run.smithi154.stdout:uid [ unknown] Ceph.com (release key) 2024-09-18T10:11:03.843 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-09-18T10:11:03.843 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=7061ea6ca50908d104e9f3979dc615e118d9e153 2024-09-18T10:11:03.847 INFO:teuthology.orchestra.run.smithi029.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-18T10:11:03.848 INFO:teuthology.orchestra.run.smithi029.stdout:uid [ unknown] Ceph.com (release key) 2024-09-18T10:11:03.848 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-09-18T10:11:03.848 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=7061ea6ca50908d104e9f3979dc615e118d9e153 2024-09-18T10:11:03.853 INFO:teuthology.orchestra.run.smithi189.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-18T10:11:03.853 INFO:teuthology.orchestra.run.smithi189.stdout:uid [ unknown] Ceph.com (release key) 2024-09-18T10:11:03.853 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-09-18T10:11:03.854 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=7061ea6ca50908d104e9f3979dc615e118d9e153 2024-09-18T10:11:03.985 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default/ 2024-09-18T10:11:03.986 INFO:teuthology.task.install.deb:Package version is 17.2.7-1645-g7061ea6c-1focal 2024-09-18T10:11:03.993 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default/ 2024-09-18T10:11:03.993 INFO:teuthology.task.install.deb:Package version is 17.2.7-1645-g7061ea6c-1focal 2024-09-18T10:11:03.997 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default/ 2024-09-18T10:11:03.997 INFO:teuthology.task.install.deb:Package version is 17.2.7-1645-g7061ea6c-1focal 2024-09-18T10:11:04.005 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default/ 2024-09-18T10:11:04.005 INFO:teuthology.task.install.deb:Package version is 17.2.7-1645-g7061ea6c-1focal 2024-09-18T10:11:04.099 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-09-18T10:11:04.099 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-18T10:11:04.105 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-09-18T10:11:04.105 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-18T10:11:04.116 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-18T10:11:04.116 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-18T10:11:04.119 DEBUG:teuthology.orchestra.run.smithi154:> sudo apt-get update 2024-09-18T10:11:04.120 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-09-18T10:11:04.120 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-18T10:11:04.121 DEBUG:teuthology.orchestra.run.smithi136:> sudo apt-get update 2024-09-18T10:11:04.137 DEBUG:teuthology.orchestra.run.smithi189:> sudo apt-get update 2024-09-18T10:11:04.138 DEBUG:teuthology.orchestra.run.smithi029:> sudo apt-get update 2024-09-18T10:11:04.262 INFO:teuthology.orchestra.run.smithi136.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2024-09-18T10:11:04.268 INFO:teuthology.orchestra.run.smithi136.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2024-09-18T10:11:04.286 INFO:teuthology.orchestra.run.smithi136.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2024-09-18T10:11:04.309 INFO:teuthology.orchestra.run.smithi154.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2024-09-18T10:11:04.315 INFO:teuthology.orchestra.run.smithi154.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2024-09-18T10:11:04.318 INFO:teuthology.orchestra.run.smithi136.stdout:Ign:4 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal InRelease 2024-09-18T10:11:04.326 INFO:teuthology.orchestra.run.smithi189.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2024-09-18T10:11:04.330 INFO:teuthology.orchestra.run.smithi029.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2024-09-18T10:11:04.330 INFO:teuthology.orchestra.run.smithi029.stdout:Hit:2 http://security.ubuntu.com/ubuntu focal-security InRelease 2024-09-18T10:11:04.332 INFO:teuthology.orchestra.run.smithi189.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2024-09-18T10:11:04.332 INFO:teuthology.orchestra.run.smithi154.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2024-09-18T10:11:04.335 INFO:teuthology.orchestra.run.smithi029.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2024-09-18T10:11:04.340 INFO:teuthology.orchestra.run.smithi136.stdout:Get:5 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal Release [7,662 B] 2024-09-18T10:11:04.349 INFO:teuthology.orchestra.run.smithi189.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2024-09-18T10:11:04.353 INFO:teuthology.orchestra.run.smithi029.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2024-09-18T10:11:04.363 INFO:teuthology.orchestra.run.smithi154.stdout:Ign:4 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal InRelease 2024-09-18T10:11:04.364 INFO:teuthology.orchestra.run.smithi136.stdout:Ign:6 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal Release.gpg 2024-09-18T10:11:04.380 INFO:teuthology.orchestra.run.smithi136.stdout:Hit:7 http://security.ubuntu.com/ubuntu focal-security InRelease 2024-09-18T10:11:04.380 INFO:teuthology.orchestra.run.smithi189.stdout:Ign:4 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal InRelease 2024-09-18T10:11:04.385 INFO:teuthology.orchestra.run.smithi029.stdout:Ign:5 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal InRelease 2024-09-18T10:11:04.385 INFO:teuthology.orchestra.run.smithi154.stdout:Get:5 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal Release [7,662 B] 2024-09-18T10:11:04.386 INFO:teuthology.orchestra.run.smithi136.stdout:Get:8 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 Packages [17.4 kB] 2024-09-18T10:11:04.401 INFO:teuthology.orchestra.run.smithi189.stdout:Get:5 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal Release [7,662 B] 2024-09-18T10:11:04.406 INFO:teuthology.orchestra.run.smithi029.stdout:Get:6 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal Release [7,662 B] 2024-09-18T10:11:04.408 INFO:teuthology.orchestra.run.smithi154.stdout:Ign:6 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal Release.gpg 2024-09-18T10:11:04.418 INFO:teuthology.orchestra.run.smithi136.stdout:Get:9 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main i386 Packages [3,390 B] 2024-09-18T10:11:04.425 INFO:teuthology.orchestra.run.smithi189.stdout:Ign:6 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal Release.gpg 2024-09-18T10:11:04.426 INFO:teuthology.orchestra.run.smithi154.stdout:Hit:7 http://security.ubuntu.com/ubuntu focal-security InRelease 2024-09-18T10:11:04.429 INFO:teuthology.orchestra.run.smithi154.stdout:Get:8 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 Packages [17.4 kB] 2024-09-18T10:11:04.430 INFO:teuthology.orchestra.run.smithi029.stdout:Ign:7 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal Release.gpg 2024-09-18T10:11:04.443 INFO:teuthology.orchestra.run.smithi189.stdout:Hit:7 http://security.ubuntu.com/ubuntu focal-security InRelease 2024-09-18T10:11:04.446 INFO:teuthology.orchestra.run.smithi189.stdout:Get:8 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 Packages [17.4 kB] 2024-09-18T10:11:04.451 INFO:teuthology.orchestra.run.smithi029.stdout:Get:8 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 Packages [17.4 kB] 2024-09-18T10:11:04.462 INFO:teuthology.orchestra.run.smithi154.stdout:Get:9 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main i386 Packages [3,390 B] 2024-09-18T10:11:04.480 INFO:teuthology.orchestra.run.smithi189.stdout:Get:9 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main i386 Packages [3,390 B] 2024-09-18T10:11:04.485 INFO:teuthology.orchestra.run.smithi029.stdout:Get:9 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main i386 Packages [3,390 B] 2024-09-18T10:11:04.916 INFO:teuthology.orchestra.run.smithi136.stdout:Fetched 28.4 kB in 1s (45.6 kB/s) 2024-09-18T10:11:04.967 INFO:teuthology.orchestra.run.smithi154.stdout:Fetched 28.4 kB in 1s (44.9 kB/s) 2024-09-18T10:11:04.976 INFO:teuthology.orchestra.run.smithi189.stdout:Fetched 28.4 kB in 1s (45.4 kB/s) 2024-09-18T10:11:05.014 INFO:teuthology.orchestra.run.smithi029.stdout:Fetched 28.4 kB in 1s (43.1 kB/s) 2024-09-18T10:11:06.123 INFO:teuthology.orchestra.run.smithi154.stdout:Reading package lists... 2024-09-18T10:11:06.132 INFO:teuthology.orchestra.run.smithi189.stdout:Reading package lists... 2024-09-18T10:11:06.149 DEBUG:teuthology.orchestra.run.smithi154:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=17.2.7-1645-g7061ea6c-1focal cephadm=17.2.7-1645-g7061ea6c-1focal ceph-mds=17.2.7-1645-g7061ea6c-1focal ceph-mgr=17.2.7-1645-g7061ea6c-1focal ceph-common=17.2.7-1645-g7061ea6c-1focal ceph-fuse=17.2.7-1645-g7061ea6c-1focal ceph-test=17.2.7-1645-g7061ea6c-1focal ceph-volume=17.2.7-1645-g7061ea6c-1focal radosgw=17.2.7-1645-g7061ea6c-1focal python3-rados=17.2.7-1645-g7061ea6c-1focal python3-rgw=17.2.7-1645-g7061ea6c-1focal python3-cephfs=17.2.7-1645-g7061ea6c-1focal python3-rbd=17.2.7-1645-g7061ea6c-1focal libcephfs2=17.2.7-1645-g7061ea6c-1focal libcephfs-dev=17.2.7-1645-g7061ea6c-1focal librados2=17.2.7-1645-g7061ea6c-1focal librbd1=17.2.7-1645-g7061ea6c-1focal rbd-fuse=17.2.7-1645-g7061ea6c-1focal 2024-09-18T10:11:06.159 DEBUG:teuthology.orchestra.run.smithi189:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=17.2.7-1645-g7061ea6c-1focal cephadm=17.2.7-1645-g7061ea6c-1focal ceph-mds=17.2.7-1645-g7061ea6c-1focal ceph-mgr=17.2.7-1645-g7061ea6c-1focal ceph-common=17.2.7-1645-g7061ea6c-1focal ceph-fuse=17.2.7-1645-g7061ea6c-1focal ceph-test=17.2.7-1645-g7061ea6c-1focal ceph-volume=17.2.7-1645-g7061ea6c-1focal radosgw=17.2.7-1645-g7061ea6c-1focal python3-rados=17.2.7-1645-g7061ea6c-1focal python3-rgw=17.2.7-1645-g7061ea6c-1focal python3-cephfs=17.2.7-1645-g7061ea6c-1focal python3-rbd=17.2.7-1645-g7061ea6c-1focal libcephfs2=17.2.7-1645-g7061ea6c-1focal libcephfs-dev=17.2.7-1645-g7061ea6c-1focal librados2=17.2.7-1645-g7061ea6c-1focal librbd1=17.2.7-1645-g7061ea6c-1focal rbd-fuse=17.2.7-1645-g7061ea6c-1focal 2024-09-18T10:11:06.183 INFO:teuthology.orchestra.run.smithi136.stdout:Reading package lists... 2024-09-18T10:11:06.211 DEBUG:teuthology.orchestra.run.smithi136:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=17.2.7-1645-g7061ea6c-1focal cephadm=17.2.7-1645-g7061ea6c-1focal ceph-mds=17.2.7-1645-g7061ea6c-1focal ceph-mgr=17.2.7-1645-g7061ea6c-1focal ceph-common=17.2.7-1645-g7061ea6c-1focal ceph-fuse=17.2.7-1645-g7061ea6c-1focal ceph-test=17.2.7-1645-g7061ea6c-1focal ceph-volume=17.2.7-1645-g7061ea6c-1focal radosgw=17.2.7-1645-g7061ea6c-1focal python3-rados=17.2.7-1645-g7061ea6c-1focal python3-rgw=17.2.7-1645-g7061ea6c-1focal python3-cephfs=17.2.7-1645-g7061ea6c-1focal python3-rbd=17.2.7-1645-g7061ea6c-1focal libcephfs2=17.2.7-1645-g7061ea6c-1focal libcephfs-dev=17.2.7-1645-g7061ea6c-1focal librados2=17.2.7-1645-g7061ea6c-1focal librbd1=17.2.7-1645-g7061ea6c-1focal rbd-fuse=17.2.7-1645-g7061ea6c-1focal 2024-09-18T10:11:06.215 INFO:teuthology.orchestra.run.smithi029.stdout:Reading package lists... 2024-09-18T10:11:06.219 INFO:teuthology.orchestra.run.smithi154.stdout:Reading package lists... 2024-09-18T10:11:06.231 INFO:teuthology.orchestra.run.smithi189.stdout:Reading package lists... 2024-09-18T10:11:06.241 DEBUG:teuthology.orchestra.run.smithi029:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=17.2.7-1645-g7061ea6c-1focal cephadm=17.2.7-1645-g7061ea6c-1focal ceph-mds=17.2.7-1645-g7061ea6c-1focal ceph-mgr=17.2.7-1645-g7061ea6c-1focal ceph-common=17.2.7-1645-g7061ea6c-1focal ceph-fuse=17.2.7-1645-g7061ea6c-1focal ceph-test=17.2.7-1645-g7061ea6c-1focal ceph-volume=17.2.7-1645-g7061ea6c-1focal radosgw=17.2.7-1645-g7061ea6c-1focal python3-rados=17.2.7-1645-g7061ea6c-1focal python3-rgw=17.2.7-1645-g7061ea6c-1focal python3-cephfs=17.2.7-1645-g7061ea6c-1focal python3-rbd=17.2.7-1645-g7061ea6c-1focal libcephfs2=17.2.7-1645-g7061ea6c-1focal libcephfs-dev=17.2.7-1645-g7061ea6c-1focal librados2=17.2.7-1645-g7061ea6c-1focal librbd1=17.2.7-1645-g7061ea6c-1focal rbd-fuse=17.2.7-1645-g7061ea6c-1focal 2024-09-18T10:11:06.282 INFO:teuthology.orchestra.run.smithi136.stdout:Reading package lists... 2024-09-18T10:11:06.316 INFO:teuthology.orchestra.run.smithi029.stdout:Reading package lists... 2024-09-18T10:11:06.424 INFO:teuthology.orchestra.run.smithi189.stdout:Building dependency tree... 2024-09-18T10:11:06.425 INFO:teuthology.orchestra.run.smithi189.stdout:Reading state information... 2024-09-18T10:11:06.438 INFO:teuthology.orchestra.run.smithi154.stdout:Building dependency tree... 2024-09-18T10:11:06.439 INFO:teuthology.orchestra.run.smithi154.stdout:Reading state information... 2024-09-18T10:11:06.499 INFO:teuthology.orchestra.run.smithi136.stdout:Building dependency tree... 2024-09-18T10:11:06.499 INFO:teuthology.orchestra.run.smithi136.stdout:Reading state information... 2024-09-18T10:11:06.517 INFO:teuthology.orchestra.run.smithi029.stdout:Building dependency tree... 2024-09-18T10:11:06.518 INFO:teuthology.orchestra.run.smithi029.stdout:Reading state information... 2024-09-18T10:11:06.615 INFO:teuthology.orchestra.run.smithi189.stdout:The following packages were automatically installed and are no longer required: 2024-09-18T10:11:06.616 INFO:teuthology.orchestra.run.smithi189.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2024-09-18T10:11:06.616 INFO:teuthology.orchestra.run.smithi189.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-18T10:11:06.617 INFO:teuthology.orchestra.run.smithi189.stdout:The following additional packages will be installed: 2024-09-18T10:11:06.617 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-18T10:11:06.617 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd fonts-mathjax jq 2024-09-18T10:11:06.617 INFO:teuthology.orchestra.run.smithi189.stdout: libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc libjs-underscore 2024-09-18T10:11:06.617 INFO:teuthology.orchestra.run.smithi189.stdout: liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 libpcre2-16-0 2024-09-18T10:11:06.617 INFO:teuthology.orchestra.run.smithi189.stdout: libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 libradosstriper1 2024-09-18T10:11:06.618 INFO:teuthology.orchestra.run.smithi189.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli 2024-09-18T10:11:06.618 INFO:teuthology.orchestra.run.smithi189.stdout: python-natsort-doc python-pastedeploy-tpl python3-atomicwrites python3-attr 2024-09-18T10:11:06.618 INFO:teuthology.orchestra.run.smithi189.stdout: python3-bcrypt python3-blinker python3-bs4 python3-cachetools 2024-09-18T10:11:06.618 INFO:teuthology.orchestra.run.smithi189.stdout: python3-ceph-argparse python3-ceph-common python3-cherrypy3 2024-09-18T10:11:06.618 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cryptography python3-decorator python3-google-auth python3-html5lib 2024-09-18T10:11:06.618 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jinja2 python3-joblib python3-jwt python3-kubernetes 2024-09-18T10:11:06.618 INFO:teuthology.orchestra.run.smithi189.stdout: python3-logutils python3-lxml python3-mako python3-markupsafe 2024-09-18T10:11:06.618 INFO:teuthology.orchestra.run.smithi189.stdout: python3-natsort python3-oauthlib python3-openssl python3-packaging 2024-09-18T10:11:06.619 INFO:teuthology.orchestra.run.smithi189.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-09-18T10:11:06.619 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pluggy python3-prettytable python3-psutil python3-py python3-pyasn1 2024-09-18T10:11:06.619 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pyasn1-modules python3-pyinotify python3-pytest python3-repoze.lru 2024-09-18T10:11:06.619 INFO:teuthology.orchestra.run.smithi189.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-scipy 2024-09-18T10:11:06.619 INFO:teuthology.orchestra.run.smithi189.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-18T10:11:06.619 INFO:teuthology.orchestra.run.smithi189.stdout: python3-sklearn python3-sklearn-lib python3-soupsieve python3-tempita 2024-09-18T10:11:06.619 INFO:teuthology.orchestra.run.smithi189.stdout: python3-waitress python3-wcwidth python3-webencodings python3-webob 2024-09-18T10:11:06.619 INFO:teuthology.orchestra.run.smithi189.stdout: python3-websocket python3-webtest python3-werkzeug qttranslations5-l10n 2024-09-18T10:11:06.619 INFO:teuthology.orchestra.run.smithi189.stdout: socat xmlstarlet 2024-09-18T10:11:06.620 INFO:teuthology.orchestra.run.smithi189.stdout:Suggested packages: 2024-09-18T10:11:06.620 INFO:teuthology.orchestra.run.smithi189.stdout: python3-influxdb fonts-mathjax-extras fonts-stix libjs-mathjax-doc 2024-09-18T10:11:06.620 INFO:teuthology.orchestra.run.smithi189.stdout: python-attr-doc python-blinker-doc python-cryptography-doc 2024-09-18T10:11:06.620 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2024-09-18T10:11:06.620 INFO:teuthology.orchestra.run.smithi189.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2024-09-18T10:11:06.620 INFO:teuthology.orchestra.run.smithi189.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2024-09-18T10:11:06.620 INFO:teuthology.orchestra.run.smithi189.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2024-09-18T10:11:06.620 INFO:teuthology.orchestra.run.smithi189.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2024-09-18T10:11:06.620 INFO:teuthology.orchestra.run.smithi189.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-18T10:11:06.621 INFO:teuthology.orchestra.run.smithi189.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2024-09-18T10:11:06.621 INFO:teuthology.orchestra.run.smithi189.stdout:Recommended packages: 2024-09-18T10:11:06.621 INFO:teuthology.orchestra.run.smithi189.stdout: btrfs-tools 2024-09-18T10:11:06.642 INFO:teuthology.orchestra.run.smithi154.stdout:The following packages were automatically installed and are no longer required: 2024-09-18T10:11:06.644 INFO:teuthology.orchestra.run.smithi154.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2024-09-18T10:11:06.644 INFO:teuthology.orchestra.run.smithi154.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-18T10:11:06.644 INFO:teuthology.orchestra.run.smithi154.stdout:The following additional packages will be installed: 2024-09-18T10:11:06.644 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-18T10:11:06.644 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd fonts-mathjax jq 2024-09-18T10:11:06.644 INFO:teuthology.orchestra.run.smithi154.stdout: libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc libjs-underscore 2024-09-18T10:11:06.644 INFO:teuthology.orchestra.run.smithi154.stdout: liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 libpcre2-16-0 2024-09-18T10:11:06.644 INFO:teuthology.orchestra.run.smithi154.stdout: libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 libradosstriper1 2024-09-18T10:11:06.645 INFO:teuthology.orchestra.run.smithi154.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli 2024-09-18T10:11:06.645 INFO:teuthology.orchestra.run.smithi154.stdout: python-natsort-doc python-pastedeploy-tpl python3-atomicwrites python3-attr 2024-09-18T10:11:06.645 INFO:teuthology.orchestra.run.smithi154.stdout: python3-bcrypt python3-blinker python3-bs4 python3-cachetools 2024-09-18T10:11:06.645 INFO:teuthology.orchestra.run.smithi154.stdout: python3-ceph-argparse python3-ceph-common python3-cherrypy3 2024-09-18T10:11:06.645 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cryptography python3-decorator python3-google-auth python3-html5lib 2024-09-18T10:11:06.646 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jinja2 python3-joblib python3-jwt python3-kubernetes 2024-09-18T10:11:06.646 INFO:teuthology.orchestra.run.smithi154.stdout: python3-logutils python3-lxml python3-mako python3-markupsafe 2024-09-18T10:11:06.646 INFO:teuthology.orchestra.run.smithi154.stdout: python3-natsort python3-oauthlib python3-openssl python3-packaging 2024-09-18T10:11:06.646 INFO:teuthology.orchestra.run.smithi154.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-09-18T10:11:06.646 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pluggy python3-prettytable python3-psutil python3-py python3-pyasn1 2024-09-18T10:11:06.646 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pyasn1-modules python3-pyinotify python3-pytest python3-repoze.lru 2024-09-18T10:11:06.646 INFO:teuthology.orchestra.run.smithi154.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-scipy 2024-09-18T10:11:06.646 INFO:teuthology.orchestra.run.smithi154.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-18T10:11:06.646 INFO:teuthology.orchestra.run.smithi154.stdout: python3-sklearn python3-sklearn-lib python3-soupsieve python3-tempita 2024-09-18T10:11:06.646 INFO:teuthology.orchestra.run.smithi154.stdout: python3-waitress python3-wcwidth python3-webencodings python3-webob 2024-09-18T10:11:06.646 INFO:teuthology.orchestra.run.smithi154.stdout: python3-websocket python3-webtest python3-werkzeug qttranslations5-l10n 2024-09-18T10:11:06.646 INFO:teuthology.orchestra.run.smithi154.stdout: socat xmlstarlet 2024-09-18T10:11:06.647 INFO:teuthology.orchestra.run.smithi154.stdout:Suggested packages: 2024-09-18T10:11:06.647 INFO:teuthology.orchestra.run.smithi154.stdout: python3-influxdb fonts-mathjax-extras fonts-stix libjs-mathjax-doc 2024-09-18T10:11:06.647 INFO:teuthology.orchestra.run.smithi154.stdout: python-attr-doc python-blinker-doc python-cryptography-doc 2024-09-18T10:11:06.647 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2024-09-18T10:11:06.647 INFO:teuthology.orchestra.run.smithi154.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2024-09-18T10:11:06.647 INFO:teuthology.orchestra.run.smithi154.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2024-09-18T10:11:06.647 INFO:teuthology.orchestra.run.smithi154.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2024-09-18T10:11:06.647 INFO:teuthology.orchestra.run.smithi154.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2024-09-18T10:11:06.647 INFO:teuthology.orchestra.run.smithi154.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-18T10:11:06.647 INFO:teuthology.orchestra.run.smithi154.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2024-09-18T10:11:06.647 INFO:teuthology.orchestra.run.smithi154.stdout:Recommended packages: 2024-09-18T10:11:06.647 INFO:teuthology.orchestra.run.smithi154.stdout: btrfs-tools 2024-09-18T10:11:06.694 INFO:teuthology.orchestra.run.smithi136.stdout:The following packages were automatically installed and are no longer required: 2024-09-18T10:11:06.695 INFO:teuthology.orchestra.run.smithi136.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2024-09-18T10:11:06.695 INFO:teuthology.orchestra.run.smithi136.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-18T10:11:06.695 INFO:teuthology.orchestra.run.smithi136.stdout:The following additional packages will be installed: 2024-09-18T10:11:06.695 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-18T10:11:06.695 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd fonts-mathjax jq 2024-09-18T10:11:06.696 INFO:teuthology.orchestra.run.smithi136.stdout: libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc libjs-underscore 2024-09-18T10:11:06.696 INFO:teuthology.orchestra.run.smithi136.stdout: liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 libpcre2-16-0 2024-09-18T10:11:06.696 INFO:teuthology.orchestra.run.smithi136.stdout: libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 libradosstriper1 2024-09-18T10:11:06.697 INFO:teuthology.orchestra.run.smithi136.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli 2024-09-18T10:11:06.697 INFO:teuthology.orchestra.run.smithi136.stdout: python-natsort-doc python-pastedeploy-tpl python3-atomicwrites python3-attr 2024-09-18T10:11:06.697 INFO:teuthology.orchestra.run.smithi136.stdout: python3-bcrypt python3-blinker python3-bs4 python3-cachetools 2024-09-18T10:11:06.697 INFO:teuthology.orchestra.run.smithi136.stdout: python3-ceph-argparse python3-ceph-common python3-cherrypy3 2024-09-18T10:11:06.697 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cryptography python3-decorator python3-google-auth python3-html5lib 2024-09-18T10:11:06.697 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jinja2 python3-joblib python3-jwt python3-kubernetes 2024-09-18T10:11:06.697 INFO:teuthology.orchestra.run.smithi136.stdout: python3-logutils python3-lxml python3-mako python3-markupsafe 2024-09-18T10:11:06.697 INFO:teuthology.orchestra.run.smithi136.stdout: python3-natsort python3-oauthlib python3-openssl python3-packaging 2024-09-18T10:11:06.697 INFO:teuthology.orchestra.run.smithi136.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-09-18T10:11:06.697 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pluggy python3-prettytable python3-psutil python3-py python3-pyasn1 2024-09-18T10:11:06.698 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pyasn1-modules python3-pyinotify python3-pytest python3-repoze.lru 2024-09-18T10:11:06.698 INFO:teuthology.orchestra.run.smithi136.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-scipy 2024-09-18T10:11:06.698 INFO:teuthology.orchestra.run.smithi136.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-18T10:11:06.698 INFO:teuthology.orchestra.run.smithi136.stdout: python3-sklearn python3-sklearn-lib python3-soupsieve python3-tempita 2024-09-18T10:11:06.698 INFO:teuthology.orchestra.run.smithi136.stdout: python3-waitress python3-wcwidth python3-webencodings python3-webob 2024-09-18T10:11:06.698 INFO:teuthology.orchestra.run.smithi136.stdout: python3-websocket python3-webtest python3-werkzeug qttranslations5-l10n 2024-09-18T10:11:06.698 INFO:teuthology.orchestra.run.smithi136.stdout: socat xmlstarlet 2024-09-18T10:11:06.698 INFO:teuthology.orchestra.run.smithi136.stdout:Suggested packages: 2024-09-18T10:11:06.698 INFO:teuthology.orchestra.run.smithi136.stdout: python3-influxdb fonts-mathjax-extras fonts-stix libjs-mathjax-doc 2024-09-18T10:11:06.698 INFO:teuthology.orchestra.run.smithi136.stdout: python-attr-doc python-blinker-doc python-cryptography-doc 2024-09-18T10:11:06.699 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2024-09-18T10:11:06.699 INFO:teuthology.orchestra.run.smithi136.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2024-09-18T10:11:06.699 INFO:teuthology.orchestra.run.smithi136.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2024-09-18T10:11:06.699 INFO:teuthology.orchestra.run.smithi136.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2024-09-18T10:11:06.699 INFO:teuthology.orchestra.run.smithi136.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2024-09-18T10:11:06.699 INFO:teuthology.orchestra.run.smithi136.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-18T10:11:06.699 INFO:teuthology.orchestra.run.smithi136.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2024-09-18T10:11:06.699 INFO:teuthology.orchestra.run.smithi136.stdout:Recommended packages: 2024-09-18T10:11:06.699 INFO:teuthology.orchestra.run.smithi136.stdout: btrfs-tools 2024-09-18T10:11:06.726 INFO:teuthology.orchestra.run.smithi029.stdout:The following packages were automatically installed and are no longer required: 2024-09-18T10:11:06.727 INFO:teuthology.orchestra.run.smithi029.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2024-09-18T10:11:06.727 INFO:teuthology.orchestra.run.smithi029.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-18T10:11:06.727 INFO:teuthology.orchestra.run.smithi029.stdout:The following additional packages will be installed: 2024-09-18T10:11:06.727 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-18T10:11:06.727 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd fonts-mathjax jq 2024-09-18T10:11:06.728 INFO:teuthology.orchestra.run.smithi029.stdout: libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc libjs-underscore 2024-09-18T10:11:06.728 INFO:teuthology.orchestra.run.smithi029.stdout: liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 libpcre2-16-0 2024-09-18T10:11:06.728 INFO:teuthology.orchestra.run.smithi029.stdout: libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 libradosstriper1 2024-09-18T10:11:06.729 INFO:teuthology.orchestra.run.smithi029.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli 2024-09-18T10:11:06.729 INFO:teuthology.orchestra.run.smithi029.stdout: python-natsort-doc python-pastedeploy-tpl python3-atomicwrites python3-attr 2024-09-18T10:11:06.729 INFO:teuthology.orchestra.run.smithi029.stdout: python3-bcrypt python3-blinker python3-bs4 python3-cachetools 2024-09-18T10:11:06.729 INFO:teuthology.orchestra.run.smithi029.stdout: python3-ceph-argparse python3-ceph-common python3-cherrypy3 2024-09-18T10:11:06.729 INFO:teuthology.orchestra.run.smithi029.stdout: python3-cryptography python3-decorator python3-google-auth python3-html5lib 2024-09-18T10:11:06.729 INFO:teuthology.orchestra.run.smithi029.stdout: python3-jinja2 python3-joblib python3-jwt python3-kubernetes 2024-09-18T10:11:06.729 INFO:teuthology.orchestra.run.smithi029.stdout: python3-logutils python3-lxml python3-mako python3-markupsafe 2024-09-18T10:11:06.729 INFO:teuthology.orchestra.run.smithi029.stdout: python3-natsort python3-oauthlib python3-openssl python3-packaging 2024-09-18T10:11:06.729 INFO:teuthology.orchestra.run.smithi029.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-09-18T10:11:06.729 INFO:teuthology.orchestra.run.smithi029.stdout: python3-pluggy python3-prettytable python3-psutil python3-py python3-pyasn1 2024-09-18T10:11:06.729 INFO:teuthology.orchestra.run.smithi029.stdout: python3-pyasn1-modules python3-pyinotify python3-pytest python3-repoze.lru 2024-09-18T10:11:06.729 INFO:teuthology.orchestra.run.smithi029.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-scipy 2024-09-18T10:11:06.729 INFO:teuthology.orchestra.run.smithi029.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-18T10:11:06.730 INFO:teuthology.orchestra.run.smithi029.stdout: python3-sklearn python3-sklearn-lib python3-soupsieve python3-tempita 2024-09-18T10:11:06.730 INFO:teuthology.orchestra.run.smithi029.stdout: python3-waitress python3-wcwidth python3-webencodings python3-webob 2024-09-18T10:11:06.730 INFO:teuthology.orchestra.run.smithi029.stdout: python3-websocket python3-webtest python3-werkzeug qttranslations5-l10n 2024-09-18T10:11:06.730 INFO:teuthology.orchestra.run.smithi029.stdout: socat xmlstarlet 2024-09-18T10:11:06.730 INFO:teuthology.orchestra.run.smithi029.stdout:Suggested packages: 2024-09-18T10:11:06.730 INFO:teuthology.orchestra.run.smithi029.stdout: python3-influxdb fonts-mathjax-extras fonts-stix libjs-mathjax-doc 2024-09-18T10:11:06.730 INFO:teuthology.orchestra.run.smithi029.stdout: python-attr-doc python-blinker-doc python-cryptography-doc 2024-09-18T10:11:06.730 INFO:teuthology.orchestra.run.smithi029.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2024-09-18T10:11:06.730 INFO:teuthology.orchestra.run.smithi029.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2024-09-18T10:11:06.731 INFO:teuthology.orchestra.run.smithi029.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2024-09-18T10:11:06.731 INFO:teuthology.orchestra.run.smithi029.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2024-09-18T10:11:06.731 INFO:teuthology.orchestra.run.smithi029.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2024-09-18T10:11:06.731 INFO:teuthology.orchestra.run.smithi029.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-18T10:11:06.731 INFO:teuthology.orchestra.run.smithi029.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2024-09-18T10:11:06.731 INFO:teuthology.orchestra.run.smithi029.stdout:Recommended packages: 2024-09-18T10:11:06.731 INFO:teuthology.orchestra.run.smithi029.stdout: btrfs-tools 2024-09-18T10:11:06.738 INFO:teuthology.orchestra.run.smithi189.stdout:The following NEW packages will be installed: 2024-09-18T10:11:06.738 INFO:teuthology.orchestra.run.smithi189.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-18T10:11:06.738 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-18T10:11:06.738 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-09-18T10:11:06.739 INFO:teuthology.orchestra.run.smithi189.stdout: fonts-mathjax jq libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 2024-09-18T10:11:06.739 INFO:teuthology.orchestra.run.smithi189.stdout: libjs-mathjax libjs-sphinxdoc libjs-underscore liblbfgsb0 liblttng-ust0 2024-09-18T10:11:06.739 INFO:teuthology.orchestra.run.smithi189.stdout: liblua5.3-0 liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 2024-09-18T10:11:06.739 INFO:teuthology.orchestra.run.smithi189.stdout: libqt5network5 librabbitmq4 libradosstriper1 librdkafka1 librgw2 2024-09-18T10:11:06.740 INFO:teuthology.orchestra.run.smithi189.stdout: libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli python-natsort-doc 2024-09-18T10:11:06.740 INFO:teuthology.orchestra.run.smithi189.stdout: python-pastedeploy-tpl python3-atomicwrites python3-attr python3-bcrypt 2024-09-18T10:11:06.740 INFO:teuthology.orchestra.run.smithi189.stdout: python3-blinker python3-bs4 python3-cachetools python3-ceph-argparse 2024-09-18T10:11:06.740 INFO:teuthology.orchestra.run.smithi189.stdout: python3-ceph-common python3-cephfs python3-cherrypy3 python3-cryptography 2024-09-18T10:11:06.740 INFO:teuthology.orchestra.run.smithi189.stdout: python3-decorator python3-google-auth python3-html5lib python3-jinja2 2024-09-18T10:11:06.740 INFO:teuthology.orchestra.run.smithi189.stdout: python3-joblib python3-jwt python3-kubernetes python3-logutils python3-lxml 2024-09-18T10:11:06.741 INFO:teuthology.orchestra.run.smithi189.stdout: python3-mako python3-markupsafe python3-natsort python3-oauthlib 2024-09-18T10:11:06.741 INFO:teuthology.orchestra.run.smithi189.stdout: python3-openssl python3-packaging python3-paste python3-pastedeploy 2024-09-18T10:11:06.741 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pastescript python3-pecan python3-pluggy python3-prettytable 2024-09-18T10:11:06.741 INFO:teuthology.orchestra.run.smithi189.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-18T10:11:06.741 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pyinotify python3-pytest python3-rados python3-rbd 2024-09-18T10:11:06.741 INFO:teuthology.orchestra.run.smithi189.stdout: python3-repoze.lru python3-requests-oauthlib python3-rgw python3-routes 2024-09-18T10:11:06.741 INFO:teuthology.orchestra.run.smithi189.stdout: python3-rsa python3-scipy python3-simplegeneric python3-simplejson 2024-09-18T10:11:06.741 INFO:teuthology.orchestra.run.smithi189.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2024-09-18T10:11:06.741 INFO:teuthology.orchestra.run.smithi189.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2024-09-18T10:11:06.741 INFO:teuthology.orchestra.run.smithi189.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2024-09-18T10:11:06.741 INFO:teuthology.orchestra.run.smithi189.stdout: qttranslations5-l10n radosgw rbd-fuse socat xmlstarlet 2024-09-18T10:11:06.742 INFO:teuthology.orchestra.run.smithi189.stdout:The following packages will be upgraded: 2024-09-18T10:11:06.742 INFO:teuthology.orchestra.run.smithi189.stdout: librados2 librbd1 2024-09-18T10:11:06.764 INFO:teuthology.orchestra.run.smithi154.stdout:The following NEW packages will be installed: 2024-09-18T10:11:06.765 INFO:teuthology.orchestra.run.smithi154.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-18T10:11:06.765 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-18T10:11:06.765 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-09-18T10:11:06.766 INFO:teuthology.orchestra.run.smithi154.stdout: fonts-mathjax jq libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 2024-09-18T10:11:06.766 INFO:teuthology.orchestra.run.smithi154.stdout: libjs-mathjax libjs-sphinxdoc libjs-underscore liblbfgsb0 liblttng-ust0 2024-09-18T10:11:06.766 INFO:teuthology.orchestra.run.smithi154.stdout: liblua5.3-0 liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 2024-09-18T10:11:06.766 INFO:teuthology.orchestra.run.smithi154.stdout: libqt5network5 librabbitmq4 libradosstriper1 librdkafka1 librgw2 2024-09-18T10:11:06.766 INFO:teuthology.orchestra.run.smithi154.stdout: libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli python-natsort-doc 2024-09-18T10:11:06.766 INFO:teuthology.orchestra.run.smithi154.stdout: python-pastedeploy-tpl python3-atomicwrites python3-attr python3-bcrypt 2024-09-18T10:11:06.766 INFO:teuthology.orchestra.run.smithi154.stdout: python3-blinker python3-bs4 python3-cachetools python3-ceph-argparse 2024-09-18T10:11:06.766 INFO:teuthology.orchestra.run.smithi154.stdout: python3-ceph-common python3-cephfs python3-cherrypy3 python3-cryptography 2024-09-18T10:11:06.766 INFO:teuthology.orchestra.run.smithi154.stdout: python3-decorator python3-google-auth python3-html5lib python3-jinja2 2024-09-18T10:11:06.766 INFO:teuthology.orchestra.run.smithi154.stdout: python3-joblib python3-jwt python3-kubernetes python3-logutils python3-lxml 2024-09-18T10:11:06.767 INFO:teuthology.orchestra.run.smithi154.stdout: python3-mako python3-markupsafe python3-natsort python3-oauthlib 2024-09-18T10:11:06.767 INFO:teuthology.orchestra.run.smithi154.stdout: python3-openssl python3-packaging python3-paste python3-pastedeploy 2024-09-18T10:11:06.767 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pastescript python3-pecan python3-pluggy python3-prettytable 2024-09-18T10:11:06.767 INFO:teuthology.orchestra.run.smithi154.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-18T10:11:06.767 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pyinotify python3-pytest python3-rados python3-rbd 2024-09-18T10:11:06.767 INFO:teuthology.orchestra.run.smithi154.stdout: python3-repoze.lru python3-requests-oauthlib python3-rgw python3-routes 2024-09-18T10:11:06.767 INFO:teuthology.orchestra.run.smithi154.stdout: python3-rsa python3-scipy python3-simplegeneric python3-simplejson 2024-09-18T10:11:06.767 INFO:teuthology.orchestra.run.smithi154.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2024-09-18T10:11:06.767 INFO:teuthology.orchestra.run.smithi154.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2024-09-18T10:11:06.767 INFO:teuthology.orchestra.run.smithi154.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2024-09-18T10:11:06.767 INFO:teuthology.orchestra.run.smithi154.stdout: qttranslations5-l10n radosgw rbd-fuse socat xmlstarlet 2024-09-18T10:11:06.768 INFO:teuthology.orchestra.run.smithi154.stdout:The following packages will be upgraded: 2024-09-18T10:11:06.768 INFO:teuthology.orchestra.run.smithi154.stdout: librados2 librbd1 2024-09-18T10:11:06.806 INFO:teuthology.orchestra.run.smithi189.stdout:2 upgraded, 108 newly installed, 0 to remove and 295 not upgraded. 2024-09-18T10:11:06.806 INFO:teuthology.orchestra.run.smithi189.stdout:Need to get 201 MB of archives. 2024-09-18T10:11:06.806 INFO:teuthology.orchestra.run.smithi189.stdout:After this operation, 938 MB of additional disk space will be used. 2024-09-18T10:11:06.806 INFO:teuthology.orchestra.run.smithi189.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2024-09-18T10:11:06.816 INFO:teuthology.orchestra.run.smithi136.stdout:The following NEW packages will be installed: 2024-09-18T10:11:06.816 INFO:teuthology.orchestra.run.smithi136.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-18T10:11:06.816 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-18T10:11:06.816 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-09-18T10:11:06.817 INFO:teuthology.orchestra.run.smithi136.stdout: fonts-mathjax jq libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 2024-09-18T10:11:06.817 INFO:teuthology.orchestra.run.smithi136.stdout: libjs-mathjax libjs-sphinxdoc libjs-underscore liblbfgsb0 liblttng-ust0 2024-09-18T10:11:06.817 INFO:teuthology.orchestra.run.smithi136.stdout: liblua5.3-0 liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 2024-09-18T10:11:06.817 INFO:teuthology.orchestra.run.smithi136.stdout: libqt5network5 librabbitmq4 libradosstriper1 librdkafka1 librgw2 2024-09-18T10:11:06.817 INFO:teuthology.orchestra.run.smithi136.stdout: libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli python-natsort-doc 2024-09-18T10:11:06.817 INFO:teuthology.orchestra.run.smithi136.stdout: python-pastedeploy-tpl python3-atomicwrites python3-attr python3-bcrypt 2024-09-18T10:11:06.818 INFO:teuthology.orchestra.run.smithi136.stdout: python3-blinker python3-bs4 python3-cachetools python3-ceph-argparse 2024-09-18T10:11:06.818 INFO:teuthology.orchestra.run.smithi136.stdout: python3-ceph-common python3-cephfs python3-cherrypy3 python3-cryptography 2024-09-18T10:11:06.818 INFO:teuthology.orchestra.run.smithi136.stdout: python3-decorator python3-google-auth python3-html5lib python3-jinja2 2024-09-18T10:11:06.818 INFO:teuthology.orchestra.run.smithi136.stdout: python3-joblib python3-jwt python3-kubernetes python3-logutils python3-lxml 2024-09-18T10:11:06.818 INFO:teuthology.orchestra.run.smithi136.stdout: python3-mako python3-markupsafe python3-natsort python3-oauthlib 2024-09-18T10:11:06.818 INFO:teuthology.orchestra.run.smithi136.stdout: python3-openssl python3-packaging python3-paste python3-pastedeploy 2024-09-18T10:11:06.818 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pastescript python3-pecan python3-pluggy python3-prettytable 2024-09-18T10:11:06.818 INFO:teuthology.orchestra.run.smithi136.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-18T10:11:06.818 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pyinotify python3-pytest python3-rados python3-rbd 2024-09-18T10:11:06.818 INFO:teuthology.orchestra.run.smithi136.stdout: python3-repoze.lru python3-requests-oauthlib python3-rgw python3-routes 2024-09-18T10:11:06.818 INFO:teuthology.orchestra.run.smithi136.stdout: python3-rsa python3-scipy python3-simplegeneric python3-simplejson 2024-09-18T10:11:06.818 INFO:teuthology.orchestra.run.smithi136.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2024-09-18T10:11:06.818 INFO:teuthology.orchestra.run.smithi136.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2024-09-18T10:11:06.819 INFO:teuthology.orchestra.run.smithi136.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2024-09-18T10:11:06.819 INFO:teuthology.orchestra.run.smithi136.stdout: qttranslations5-l10n radosgw rbd-fuse socat xmlstarlet 2024-09-18T10:11:06.819 INFO:teuthology.orchestra.run.smithi136.stdout:The following packages will be upgraded: 2024-09-18T10:11:06.819 INFO:teuthology.orchestra.run.smithi136.stdout: librados2 librbd1 2024-09-18T10:11:06.833 INFO:teuthology.orchestra.run.smithi154.stdout:2 upgraded, 108 newly installed, 0 to remove and 295 not upgraded. 2024-09-18T10:11:06.834 INFO:teuthology.orchestra.run.smithi154.stdout:Need to get 201 MB of archives. 2024-09-18T10:11:06.834 INFO:teuthology.orchestra.run.smithi154.stdout:After this operation, 938 MB of additional disk space will be used. 2024-09-18T10:11:06.834 INFO:teuthology.orchestra.run.smithi154.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2024-09-18T10:11:06.855 INFO:teuthology.orchestra.run.smithi029.stdout:The following NEW packages will be installed: 2024-09-18T10:11:06.855 INFO:teuthology.orchestra.run.smithi029.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-18T10:11:06.855 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-18T10:11:06.856 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-09-18T10:11:06.856 INFO:teuthology.orchestra.run.smithi029.stdout: fonts-mathjax jq libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 2024-09-18T10:11:06.856 INFO:teuthology.orchestra.run.smithi029.stdout: libjs-mathjax libjs-sphinxdoc libjs-underscore liblbfgsb0 liblttng-ust0 2024-09-18T10:11:06.856 INFO:teuthology.orchestra.run.smithi029.stdout: liblua5.3-0 liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 2024-09-18T10:11:06.856 INFO:teuthology.orchestra.run.smithi029.stdout: libqt5network5 librabbitmq4 libradosstriper1 librdkafka1 librgw2 2024-09-18T10:11:06.857 INFO:teuthology.orchestra.run.smithi029.stdout: libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli python-natsort-doc 2024-09-18T10:11:06.857 INFO:teuthology.orchestra.run.smithi029.stdout: python-pastedeploy-tpl python3-atomicwrites python3-attr python3-bcrypt 2024-09-18T10:11:06.857 INFO:teuthology.orchestra.run.smithi029.stdout: python3-blinker python3-bs4 python3-cachetools python3-ceph-argparse 2024-09-18T10:11:06.857 INFO:teuthology.orchestra.run.smithi029.stdout: python3-ceph-common python3-cephfs python3-cherrypy3 python3-cryptography 2024-09-18T10:11:06.857 INFO:teuthology.orchestra.run.smithi029.stdout: python3-decorator python3-google-auth python3-html5lib python3-jinja2 2024-09-18T10:11:06.857 INFO:teuthology.orchestra.run.smithi029.stdout: python3-joblib python3-jwt python3-kubernetes python3-logutils python3-lxml 2024-09-18T10:11:06.858 INFO:teuthology.orchestra.run.smithi029.stdout: python3-mako python3-markupsafe python3-natsort python3-oauthlib 2024-09-18T10:11:06.858 INFO:teuthology.orchestra.run.smithi029.stdout: python3-openssl python3-packaging python3-paste python3-pastedeploy 2024-09-18T10:11:06.858 INFO:teuthology.orchestra.run.smithi029.stdout: python3-pastescript python3-pecan python3-pluggy python3-prettytable 2024-09-18T10:11:06.858 INFO:teuthology.orchestra.run.smithi029.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-18T10:11:06.858 INFO:teuthology.orchestra.run.smithi029.stdout: python3-pyinotify python3-pytest python3-rados python3-rbd 2024-09-18T10:11:06.858 INFO:teuthology.orchestra.run.smithi029.stdout: python3-repoze.lru python3-requests-oauthlib python3-rgw python3-routes 2024-09-18T10:11:06.858 INFO:teuthology.orchestra.run.smithi029.stdout: python3-rsa python3-scipy python3-simplegeneric python3-simplejson 2024-09-18T10:11:06.858 INFO:teuthology.orchestra.run.smithi029.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2024-09-18T10:11:06.858 INFO:teuthology.orchestra.run.smithi029.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2024-09-18T10:11:06.858 INFO:teuthology.orchestra.run.smithi029.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2024-09-18T10:11:06.858 INFO:teuthology.orchestra.run.smithi029.stdout: qttranslations5-l10n radosgw rbd-fuse socat xmlstarlet 2024-09-18T10:11:06.858 INFO:teuthology.orchestra.run.smithi029.stdout:The following packages will be upgraded: 2024-09-18T10:11:06.859 INFO:teuthology.orchestra.run.smithi029.stdout: librados2 librbd1 2024-09-18T10:11:06.884 INFO:teuthology.orchestra.run.smithi136.stdout:2 upgraded, 108 newly installed, 0 to remove and 295 not upgraded. 2024-09-18T10:11:06.884 INFO:teuthology.orchestra.run.smithi136.stdout:Need to get 201 MB of archives. 2024-09-18T10:11:06.884 INFO:teuthology.orchestra.run.smithi136.stdout:After this operation, 938 MB of additional disk space will be used. 2024-09-18T10:11:06.884 INFO:teuthology.orchestra.run.smithi136.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2024-09-18T10:11:06.889 INFO:teuthology.orchestra.run.smithi189.stdout:Get:2 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 17.2.7-1645-g7061ea6c-1focal [3,507 kB] 2024-09-18T10:11:06.890 INFO:teuthology.orchestra.run.smithi189.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2024-09-18T10:11:06.891 INFO:teuthology.orchestra.run.smithi189.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 liblua5.3-0 amd64 5.3.3-1.1ubuntu2 [116 kB] 2024-09-18T10:11:06.901 INFO:teuthology.orchestra.run.smithi189.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2024-09-18T10:11:06.907 INFO:teuthology.orchestra.run.smithi189.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2024-09-18T10:11:06.916 INFO:teuthology.orchestra.run.smithi154.stdout:Get:2 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 17.2.7-1645-g7061ea6c-1focal [3,507 kB] 2024-09-18T10:11:06.918 INFO:teuthology.orchestra.run.smithi154.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2024-09-18T10:11:06.919 INFO:teuthology.orchestra.run.smithi154.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 liblua5.3-0 amd64 5.3.3-1.1ubuntu2 [116 kB] 2024-09-18T10:11:06.925 INFO:teuthology.orchestra.run.smithi029.stdout:2 upgraded, 108 newly installed, 0 to remove and 295 not upgraded. 2024-09-18T10:11:06.925 INFO:teuthology.orchestra.run.smithi029.stdout:Need to get 201 MB of archives. 2024-09-18T10:11:06.925 INFO:teuthology.orchestra.run.smithi029.stdout:After this operation, 938 MB of additional disk space will be used. 2024-09-18T10:11:06.925 INFO:teuthology.orchestra.run.smithi029.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2024-09-18T10:11:06.929 INFO:teuthology.orchestra.run.smithi154.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2024-09-18T10:11:06.931 INFO:teuthology.orchestra.run.smithi189.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/universe amd64 libdouble-conversion3 amd64 3.1.5-4ubuntu1 [37.9 kB] 2024-09-18T10:11:06.931 INFO:teuthology.orchestra.run.smithi189.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libpcre2-16-0 amd64 10.34-7ubuntu0.1 [181 kB] 2024-09-18T10:11:06.935 INFO:teuthology.orchestra.run.smithi154.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2024-09-18T10:11:06.937 INFO:teuthology.orchestra.run.smithi189.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5core5a amd64 5.12.8+dfsg-0ubuntu2.1 [2,006 kB] 2024-09-18T10:11:06.959 INFO:teuthology.orchestra.run.smithi154.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/universe amd64 libdouble-conversion3 amd64 3.1.5-4ubuntu1 [37.9 kB] 2024-09-18T10:11:06.960 INFO:teuthology.orchestra.run.smithi154.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libpcre2-16-0 amd64 10.34-7ubuntu0.1 [181 kB] 2024-09-18T10:11:06.966 INFO:teuthology.orchestra.run.smithi154.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5core5a amd64 5.12.8+dfsg-0ubuntu2.1 [2,006 kB] 2024-09-18T10:11:06.967 INFO:teuthology.orchestra.run.smithi136.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2024-09-18T10:11:06.967 INFO:teuthology.orchestra.run.smithi136.stdout:Get:3 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 17.2.7-1645-g7061ea6c-1focal [3,507 kB] 2024-09-18T10:11:06.968 INFO:teuthology.orchestra.run.smithi136.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 liblua5.3-0 amd64 5.3.3-1.1ubuntu2 [116 kB] 2024-09-18T10:11:06.969 INFO:teuthology.orchestra.run.smithi189.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5dbus5 amd64 5.12.8+dfsg-0ubuntu2.1 [208 kB] 2024-09-18T10:11:06.971 INFO:teuthology.orchestra.run.smithi189.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5network5 amd64 5.12.8+dfsg-0ubuntu2.1 [673 kB] 2024-09-18T10:11:06.977 INFO:teuthology.orchestra.run.smithi189.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal/universe amd64 libthrift-0.13.0 amd64 0.13.0-2build2 [249 kB] 2024-09-18T10:11:06.977 INFO:teuthology.orchestra.run.smithi136.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2024-09-18T10:11:06.978 INFO:teuthology.orchestra.run.smithi189.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2024-09-18T10:11:06.979 INFO:teuthology.orchestra.run.smithi189.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.3 [211 kB] 2024-09-18T10:11:06.980 INFO:teuthology.orchestra.run.smithi136.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2024-09-18T10:11:06.981 INFO:teuthology.orchestra.run.smithi189.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2024-09-18T10:11:06.981 INFO:teuthology.orchestra.run.smithi189.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2024-09-18T10:11:06.982 INFO:teuthology.orchestra.run.smithi189.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2024-09-18T10:11:06.982 INFO:teuthology.orchestra.run.smithi189.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2024-09-18T10:11:06.987 INFO:teuthology.orchestra.run.smithi189.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-natsort all 7.0.1-1 [29.2 kB] 2024-09-18T10:11:06.998 INFO:teuthology.orchestra.run.smithi154.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5dbus5 amd64 5.12.8+dfsg-0ubuntu2.1 [208 kB] 2024-09-18T10:11:07.000 INFO:teuthology.orchestra.run.smithi154.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5network5 amd64 5.12.8+dfsg-0ubuntu2.1 [673 kB] 2024-09-18T10:11:07.001 INFO:teuthology.orchestra.run.smithi189.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jwt all 1.7.1-2ubuntu2.1 [18.0 kB] 2024-09-18T10:11:07.001 INFO:teuthology.orchestra.run.smithi189.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2024-09-18T10:11:07.002 INFO:teuthology.orchestra.run.smithi189.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2024-09-18T10:11:07.002 INFO:teuthology.orchestra.run.smithi189.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2.1 [59.2 kB] 2024-09-18T10:11:07.003 INFO:teuthology.orchestra.run.smithi189.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-18T10:11:07.003 INFO:teuthology.orchestra.run.smithi189.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2024-09-18T10:11:07.004 INFO:teuthology.orchestra.run.smithi189.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-webob all 1:1.8.5-2ubuntu0.1 [84.5 kB] 2024-09-18T10:11:07.005 INFO:teuthology.orchestra.run.smithi189.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2024-09-18T10:11:07.006 INFO:teuthology.orchestra.run.smithi154.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal/universe amd64 libthrift-0.13.0 amd64 0.13.0-2build2 [249 kB] 2024-09-18T10:11:07.006 INFO:teuthology.orchestra.run.smithi029.stdout:Get:2 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 17.2.7-1645-g7061ea6c-1focal [3,507 kB] 2024-09-18T10:11:07.006 INFO:teuthology.orchestra.run.smithi189.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2024-09-18T10:11:07.007 INFO:teuthology.orchestra.run.smithi154.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2024-09-18T10:11:07.008 INFO:teuthology.orchestra.run.smithi154.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.3 [211 kB] 2024-09-18T10:11:07.008 INFO:teuthology.orchestra.run.smithi136.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/universe amd64 libdouble-conversion3 amd64 3.1.5-4ubuntu1 [37.9 kB] 2024-09-18T10:11:07.009 INFO:teuthology.orchestra.run.smithi136.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libpcre2-16-0 amd64 10.34-7ubuntu0.1 [181 kB] 2024-09-18T10:11:07.011 INFO:teuthology.orchestra.run.smithi029.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2024-09-18T10:11:07.011 INFO:teuthology.orchestra.run.smithi154.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2024-09-18T10:11:07.011 INFO:teuthology.orchestra.run.smithi154.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2024-09-18T10:11:07.011 INFO:teuthology.orchestra.run.smithi154.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2024-09-18T10:11:07.012 INFO:teuthology.orchestra.run.smithi154.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2024-09-18T10:11:07.012 INFO:teuthology.orchestra.run.smithi029.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 liblua5.3-0 amd64 5.3.3-1.1ubuntu2 [116 kB] 2024-09-18T10:11:07.015 INFO:teuthology.orchestra.run.smithi136.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5core5a amd64 5.12.8+dfsg-0ubuntu2.1 [2,006 kB] 2024-09-18T10:11:07.016 INFO:teuthology.orchestra.run.smithi154.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-natsort all 7.0.1-1 [29.2 kB] 2024-09-18T10:11:07.021 INFO:teuthology.orchestra.run.smithi029.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2024-09-18T10:11:07.021 INFO:teuthology.orchestra.run.smithi189.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-waitress all 1.4.1-1ubuntu0.1 [81.4 kB] 2024-09-18T10:11:07.023 INFO:teuthology.orchestra.run.smithi189.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2024-09-18T10:11:07.023 INFO:teuthology.orchestra.run.smithi189.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2024-09-18T10:11:07.027 INFO:teuthology.orchestra.run.smithi029.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2024-09-18T10:11:07.029 INFO:teuthology.orchestra.run.smithi189.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2024-09-18T10:11:07.029 INFO:teuthology.orchestra.run.smithi189.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2024-09-18T10:11:07.030 INFO:teuthology.orchestra.run.smithi154.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jwt all 1.7.1-2ubuntu2.1 [18.0 kB] 2024-09-18T10:11:07.031 INFO:teuthology.orchestra.run.smithi154.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2024-09-18T10:11:07.032 INFO:teuthology.orchestra.run.smithi154.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2024-09-18T10:11:07.032 INFO:teuthology.orchestra.run.smithi154.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2.1 [59.2 kB] 2024-09-18T10:11:07.033 INFO:teuthology.orchestra.run.smithi154.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-18T10:11:07.034 INFO:teuthology.orchestra.run.smithi154.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2024-09-18T10:11:07.034 INFO:teuthology.orchestra.run.smithi154.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-webob all 1:1.8.5-2ubuntu0.1 [84.5 kB] 2024-09-18T10:11:07.036 INFO:teuthology.orchestra.run.smithi154.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2024-09-18T10:11:07.037 INFO:teuthology.orchestra.run.smithi154.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2024-09-18T10:11:07.039 INFO:teuthology.orchestra.run.smithi189.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2024-09-18T10:11:07.040 INFO:teuthology.orchestra.run.smithi189.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2024-09-18T10:11:07.041 INFO:teuthology.orchestra.run.smithi189.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-werkzeug all 0.16.1+dfsg1-2ubuntu0.2 [183 kB] 2024-09-18T10:11:07.044 INFO:teuthology.orchestra.run.smithi189.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jinja2 all 2.10.1-2ubuntu0.3 [96.0 kB] 2024-09-18T10:11:07.046 INFO:teuthology.orchestra.run.smithi136.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5dbus5 amd64 5.12.8+dfsg-0ubuntu2.1 [208 kB] 2024-09-18T10:11:07.047 INFO:teuthology.orchestra.run.smithi189.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-18T10:11:07.048 INFO:teuthology.orchestra.run.smithi136.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5network5 amd64 5.12.8+dfsg-0ubuntu2.1 [673 kB] 2024-09-18T10:11:07.050 INFO:teuthology.orchestra.run.smithi154.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-waitress all 1.4.1-1ubuntu0.1 [81.4 kB] 2024-09-18T10:11:07.051 INFO:teuthology.orchestra.run.smithi029.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/universe amd64 libdouble-conversion3 amd64 3.1.5-4ubuntu1 [37.9 kB] 2024-09-18T10:11:07.052 INFO:teuthology.orchestra.run.smithi154.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2024-09-18T10:11:07.052 INFO:teuthology.orchestra.run.smithi029.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libpcre2-16-0 amd64 10.34-7ubuntu0.1 [181 kB] 2024-09-18T10:11:07.053 INFO:teuthology.orchestra.run.smithi154.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2024-09-18T10:11:07.054 INFO:teuthology.orchestra.run.smithi136.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal/universe amd64 libthrift-0.13.0 amd64 0.13.0-2build2 [249 kB] 2024-09-18T10:11:07.056 INFO:teuthology.orchestra.run.smithi136.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2024-09-18T10:11:07.056 INFO:teuthology.orchestra.run.smithi136.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.3 [211 kB] 2024-09-18T10:11:07.057 INFO:teuthology.orchestra.run.smithi189.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2024-09-18T10:11:07.058 INFO:teuthology.orchestra.run.smithi136.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2024-09-18T10:11:07.059 INFO:teuthology.orchestra.run.smithi189.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2024-09-18T10:11:07.059 INFO:teuthology.orchestra.run.smithi136.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2024-09-18T10:11:07.059 INFO:teuthology.orchestra.run.smithi136.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2024-09-18T10:11:07.060 INFO:teuthology.orchestra.run.smithi029.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5core5a amd64 5.12.8+dfsg-0ubuntu2.1 [2,006 kB] 2024-09-18T10:11:07.060 INFO:teuthology.orchestra.run.smithi189.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2024-09-18T10:11:07.060 INFO:teuthology.orchestra.run.smithi136.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2024-09-18T10:11:07.060 INFO:teuthology.orchestra.run.smithi189.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2024-09-18T10:11:07.061 INFO:teuthology.orchestra.run.smithi189.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2024-09-18T10:11:07.064 INFO:teuthology.orchestra.run.smithi136.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-natsort all 7.0.1-1 [29.2 kB] 2024-09-18T10:11:07.068 INFO:teuthology.orchestra.run.smithi154.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2024-09-18T10:11:07.068 INFO:teuthology.orchestra.run.smithi154.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2024-09-18T10:11:07.069 INFO:teuthology.orchestra.run.smithi154.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2024-09-18T10:11:07.069 INFO:teuthology.orchestra.run.smithi154.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2024-09-18T10:11:07.070 INFO:teuthology.orchestra.run.smithi154.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-werkzeug all 0.16.1+dfsg1-2ubuntu0.2 [183 kB] 2024-09-18T10:11:07.072 INFO:teuthology.orchestra.run.smithi154.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jinja2 all 2.10.1-2ubuntu0.3 [96.0 kB] 2024-09-18T10:11:07.074 INFO:teuthology.orchestra.run.smithi189.stdout:Get:44 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 17.2.7-1645-g7061ea6c-1focal [3,562 kB] 2024-09-18T10:11:07.074 INFO:teuthology.orchestra.run.smithi154.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-18T10:11:07.074 INFO:teuthology.orchestra.run.smithi189.stdout:Get:45 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2024-09-18T10:11:07.079 INFO:teuthology.orchestra.run.smithi136.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jwt all 1.7.1-2ubuntu2.1 [18.0 kB] 2024-09-18T10:11:07.079 INFO:teuthology.orchestra.run.smithi136.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2024-09-18T10:11:07.079 INFO:teuthology.orchestra.run.smithi136.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2024-09-18T10:11:07.080 INFO:teuthology.orchestra.run.smithi136.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2.1 [59.2 kB] 2024-09-18T10:11:07.081 INFO:teuthology.orchestra.run.smithi136.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-18T10:11:07.082 INFO:teuthology.orchestra.run.smithi136.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2024-09-18T10:11:07.082 INFO:teuthology.orchestra.run.smithi136.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-webob all 1:1.8.5-2ubuntu0.1 [84.5 kB] 2024-09-18T10:11:07.084 INFO:teuthology.orchestra.run.smithi136.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2024-09-18T10:11:07.084 INFO:teuthology.orchestra.run.smithi136.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2024-09-18T10:11:07.088 INFO:teuthology.orchestra.run.smithi154.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2024-09-18T10:11:07.088 INFO:teuthology.orchestra.run.smithi154.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2024-09-18T10:11:07.090 INFO:teuthology.orchestra.run.smithi154.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2024-09-18T10:11:07.091 INFO:teuthology.orchestra.run.smithi029.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5dbus5 amd64 5.12.8+dfsg-0ubuntu2.1 [208 kB] 2024-09-18T10:11:07.093 INFO:teuthology.orchestra.run.smithi029.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5network5 amd64 5.12.8+dfsg-0ubuntu2.1 [673 kB] 2024-09-18T10:11:07.099 INFO:teuthology.orchestra.run.smithi029.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal/universe amd64 libthrift-0.13.0 amd64 0.13.0-2build2 [249 kB] 2024-09-18T10:11:07.101 INFO:teuthology.orchestra.run.smithi136.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-waitress all 1.4.1-1ubuntu0.1 [81.4 kB] 2024-09-18T10:11:07.101 INFO:teuthology.orchestra.run.smithi029.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2024-09-18T10:11:07.102 INFO:teuthology.orchestra.run.smithi029.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.3 [211 kB] 2024-09-18T10:11:07.103 INFO:teuthology.orchestra.run.smithi136.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2024-09-18T10:11:07.104 INFO:teuthology.orchestra.run.smithi136.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2024-09-18T10:11:07.104 INFO:teuthology.orchestra.run.smithi029.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2024-09-18T10:11:07.104 INFO:teuthology.orchestra.run.smithi029.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2024-09-18T10:11:07.105 INFO:teuthology.orchestra.run.smithi029.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2024-09-18T10:11:07.105 INFO:teuthology.orchestra.run.smithi029.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2024-09-18T10:11:07.106 INFO:teuthology.orchestra.run.smithi154.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2024-09-18T10:11:07.106 INFO:teuthology.orchestra.run.smithi154.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2024-09-18T10:11:07.106 INFO:teuthology.orchestra.run.smithi154.stdout:Get:44 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2024-09-18T10:11:07.111 INFO:teuthology.orchestra.run.smithi029.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-natsort all 7.0.1-1 [29.2 kB] 2024-09-18T10:11:07.112 INFO:teuthology.orchestra.run.smithi136.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2024-09-18T10:11:07.120 INFO:teuthology.orchestra.run.smithi136.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2024-09-18T10:11:07.120 INFO:teuthology.orchestra.run.smithi136.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2024-09-18T10:11:07.121 INFO:teuthology.orchestra.run.smithi136.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2024-09-18T10:11:07.122 INFO:teuthology.orchestra.run.smithi136.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-werkzeug all 0.16.1+dfsg1-2ubuntu0.2 [183 kB] 2024-09-18T10:11:07.123 INFO:teuthology.orchestra.run.smithi136.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jinja2 all 2.10.1-2ubuntu0.3 [96.0 kB] 2024-09-18T10:11:07.125 INFO:teuthology.orchestra.run.smithi136.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-18T10:11:07.127 INFO:teuthology.orchestra.run.smithi029.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jwt all 1.7.1-2ubuntu2.1 [18.0 kB] 2024-09-18T10:11:07.128 INFO:teuthology.orchestra.run.smithi029.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2024-09-18T10:11:07.128 INFO:teuthology.orchestra.run.smithi029.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2024-09-18T10:11:07.128 INFO:teuthology.orchestra.run.smithi029.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2.1 [59.2 kB] 2024-09-18T10:11:07.130 INFO:teuthology.orchestra.run.smithi029.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-18T10:11:07.130 INFO:teuthology.orchestra.run.smithi029.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2024-09-18T10:11:07.131 INFO:teuthology.orchestra.run.smithi029.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-webob all 1:1.8.5-2ubuntu0.1 [84.5 kB] 2024-09-18T10:11:07.133 INFO:teuthology.orchestra.run.smithi029.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2024-09-18T10:11:07.133 INFO:teuthology.orchestra.run.smithi029.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2024-09-18T10:11:07.140 INFO:teuthology.orchestra.run.smithi136.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2024-09-18T10:11:07.141 INFO:teuthology.orchestra.run.smithi136.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2024-09-18T10:11:07.143 INFO:teuthology.orchestra.run.smithi136.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2024-09-18T10:11:07.144 INFO:teuthology.orchestra.run.smithi136.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2024-09-18T10:11:07.151 INFO:teuthology.orchestra.run.smithi029.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-waitress all 1.4.1-1ubuntu0.1 [81.4 kB] 2024-09-18T10:11:07.153 INFO:teuthology.orchestra.run.smithi029.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2024-09-18T10:11:07.154 INFO:teuthology.orchestra.run.smithi029.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2024-09-18T10:11:07.158 INFO:teuthology.orchestra.run.smithi136.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2024-09-18T10:11:07.159 INFO:teuthology.orchestra.run.smithi136.stdout:Get:44 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2024-09-18T10:11:07.160 INFO:teuthology.orchestra.run.smithi029.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2024-09-18T10:11:07.160 INFO:teuthology.orchestra.run.smithi029.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2024-09-18T10:11:07.161 INFO:teuthology.orchestra.run.smithi189.stdout:Get:46 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2024-09-18T10:11:07.161 INFO:teuthology.orchestra.run.smithi029.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2024-09-18T10:11:07.162 INFO:teuthology.orchestra.run.smithi029.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2024-09-18T10:11:07.173 INFO:teuthology.orchestra.run.smithi029.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-werkzeug all 0.16.1+dfsg1-2ubuntu0.2 [183 kB] 2024-09-18T10:11:07.175 INFO:teuthology.orchestra.run.smithi029.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jinja2 all 2.10.1-2ubuntu0.3 [96.0 kB] 2024-09-18T10:11:07.177 INFO:teuthology.orchestra.run.smithi029.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-18T10:11:07.197 INFO:teuthology.orchestra.run.smithi029.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2024-09-18T10:11:07.199 INFO:teuthology.orchestra.run.smithi029.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2024-09-18T10:11:07.201 INFO:teuthology.orchestra.run.smithi029.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2024-09-18T10:11:07.203 INFO:teuthology.orchestra.run.smithi029.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2024-09-18T10:11:07.204 INFO:teuthology.orchestra.run.smithi029.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2024-09-18T10:11:07.205 INFO:teuthology.orchestra.run.smithi029.stdout:Get:44 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2024-09-18T10:11:07.219 INFO:teuthology.orchestra.run.smithi154.stdout:Get:45 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 17.2.7-1645-g7061ea6c-1focal [3,562 kB] 2024-09-18T10:11:07.253 INFO:teuthology.orchestra.run.smithi189.stdout:Get:47 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2024-09-18T10:11:07.254 INFO:teuthology.orchestra.run.smithi189.stdout:Get:48 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 17.2.7-1645-g7061ea6c-1focal [761 kB] 2024-09-18T10:11:07.256 INFO:teuthology.orchestra.run.smithi189.stdout:Get:49 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2024-09-18T10:11:07.266 INFO:teuthology.orchestra.run.smithi189.stdout:Get:50 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 17.2.7-1645-g7061ea6c-1focal [414 kB] 2024-09-18T10:11:07.270 INFO:teuthology.orchestra.run.smithi189.stdout:Get:51 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 17.2.7-1645-g7061ea6c-1focal [22.6 kB] 2024-09-18T10:11:07.270 INFO:teuthology.orchestra.run.smithi189.stdout:Get:52 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 17.2.7-1645-g7061ea6c-1focal [200 kB] 2024-09-18T10:11:07.272 INFO:teuthology.orchestra.run.smithi189.stdout:Get:53 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 17.2.7-1645-g7061ea6c-1focal [49.1 kB] 2024-09-18T10:11:07.273 INFO:teuthology.orchestra.run.smithi189.stdout:Get:54 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2024-09-18T10:11:07.273 INFO:teuthology.orchestra.run.smithi189.stdout:Get:55 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2024-09-18T10:11:07.273 INFO:teuthology.orchestra.run.smithi154.stdout:Get:46 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2024-09-18T10:11:07.274 INFO:teuthology.orchestra.run.smithi189.stdout:Get:56 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 17.2.7-1645-g7061ea6c-1focal [390 kB] 2024-09-18T10:11:07.274 INFO:teuthology.orchestra.run.smithi189.stdout:Get:57 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2024-09-18T10:11:07.274 INFO:teuthology.orchestra.run.smithi189.stdout:Get:58 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2024-09-18T10:11:07.274 INFO:teuthology.orchestra.run.smithi189.stdout:Get:59 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2024-09-18T10:11:07.275 INFO:teuthology.orchestra.run.smithi189.stdout:Get:60 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2024-09-18T10:11:07.275 INFO:teuthology.orchestra.run.smithi189.stdout:Get:61 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2024-09-18T10:11:07.275 INFO:teuthology.orchestra.run.smithi136.stdout:Get:45 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2024-09-18T10:11:07.275 INFO:teuthology.orchestra.run.smithi189.stdout:Get:62 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2024-09-18T10:11:07.289 INFO:teuthology.orchestra.run.smithi029.stdout:Get:45 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2024-09-18T10:11:07.292 INFO:teuthology.orchestra.run.smithi136.stdout:Get:46 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2024-09-18T10:11:07.292 INFO:teuthology.orchestra.run.smithi189.stdout:Get:63 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2024-09-18T10:11:07.296 INFO:teuthology.orchestra.run.smithi154.stdout:Get:47 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2024-09-18T10:11:07.297 INFO:teuthology.orchestra.run.smithi189.stdout:Get:64 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2024-09-18T10:11:07.297 INFO:teuthology.orchestra.run.smithi189.stdout:Get:65 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 17.2.7-1645-g7061ea6c-1focal [5,637 kB] 2024-09-18T10:11:07.297 INFO:teuthology.orchestra.run.smithi154.stdout:Get:48 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2024-09-18T10:11:07.301 INFO:teuthology.orchestra.run.smithi189.stdout:Get:66 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2024-09-18T10:11:07.303 INFO:teuthology.orchestra.run.smithi189.stdout:Get:67 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2024-09-18T10:11:07.305 INFO:teuthology.orchestra.run.smithi029.stdout:Get:46 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2024-09-18T10:11:07.306 INFO:teuthology.orchestra.run.smithi029.stdout:Get:47 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2024-09-18T10:11:07.310 INFO:teuthology.orchestra.run.smithi154.stdout:Get:49 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2024-09-18T10:11:07.311 INFO:teuthology.orchestra.run.smithi154.stdout:Get:50 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2024-09-18T10:11:07.311 INFO:teuthology.orchestra.run.smithi154.stdout:Get:51 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2024-09-18T10:11:07.312 INFO:teuthology.orchestra.run.smithi189.stdout:Get:68 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2024-09-18T10:11:07.312 INFO:teuthology.orchestra.run.smithi189.stdout:Get:69 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2024-09-18T10:11:07.312 INFO:teuthology.orchestra.run.smithi154.stdout:Get:52 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2024-09-18T10:11:07.312 INFO:teuthology.orchestra.run.smithi154.stdout:Get:53 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2024-09-18T10:11:07.312 INFO:teuthology.orchestra.run.smithi154.stdout:Get:54 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2024-09-18T10:11:07.312 INFO:teuthology.orchestra.run.smithi154.stdout:Get:55 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2024-09-18T10:11:07.314 INFO:teuthology.orchestra.run.smithi189.stdout:Get:70 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2024-09-18T10:11:07.316 INFO:teuthology.orchestra.run.smithi154.stdout:Get:56 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2024-09-18T10:11:07.317 INFO:teuthology.orchestra.run.smithi189.stdout:Get:71 http://archive.ubuntu.com/ubuntu focal/universe amd64 fonts-mathjax all 2.7.4+dfsg-1 [2,208 kB] 2024-09-18T10:11:07.320 INFO:teuthology.orchestra.run.smithi029.stdout:Get:48 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2024-09-18T10:11:07.320 INFO:teuthology.orchestra.run.smithi029.stdout:Get:49 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2024-09-18T10:11:07.332 INFO:teuthology.orchestra.run.smithi189.stdout:Get:72 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.2 [283 kB] 2024-09-18T10:11:07.332 INFO:teuthology.orchestra.run.smithi154.stdout:Get:57 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2024-09-18T10:11:07.333 INFO:teuthology.orchestra.run.smithi154.stdout:Get:58 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2024-09-18T10:11:07.334 INFO:teuthology.orchestra.run.smithi189.stdout:Get:73 http://archive.ubuntu.com/ubuntu focal/universe amd64 libjs-mathjax all 2.7.4+dfsg-1 [5,654 kB] 2024-09-18T10:11:07.337 INFO:teuthology.orchestra.run.smithi154.stdout:Get:59 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2024-09-18T10:11:07.339 INFO:teuthology.orchestra.run.smithi154.stdout:Get:60 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2024-09-18T10:11:07.342 INFO:teuthology.orchestra.run.smithi154.stdout:Get:61 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2024-09-18T10:11:07.343 INFO:teuthology.orchestra.run.smithi154.stdout:Get:62 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2024-09-18T10:11:07.347 INFO:teuthology.orchestra.run.smithi154.stdout:Get:63 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2024-09-18T10:11:07.350 INFO:teuthology.orchestra.run.smithi154.stdout:Get:64 http://archive.ubuntu.com/ubuntu focal/universe amd64 fonts-mathjax all 2.7.4+dfsg-1 [2,208 kB] 2024-09-18T10:11:07.353 INFO:teuthology.orchestra.run.smithi154.stdout:Get:65 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 17.2.7-1645-g7061ea6c-1focal [761 kB] 2024-09-18T10:11:07.364 INFO:teuthology.orchestra.run.smithi154.stdout:Get:66 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.2 [283 kB] 2024-09-18T10:11:07.366 INFO:teuthology.orchestra.run.smithi136.stdout:Get:47 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2024-09-18T10:11:07.367 INFO:teuthology.orchestra.run.smithi154.stdout:Get:67 http://archive.ubuntu.com/ubuntu focal/universe amd64 libjs-mathjax all 2.7.4+dfsg-1 [5,654 kB] 2024-09-18T10:11:07.373 INFO:teuthology.orchestra.run.smithi189.stdout:Get:74 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-natsort-doc all 7.0.1-1 [103 kB] 2024-09-18T10:11:07.375 INFO:teuthology.orchestra.run.smithi189.stdout:Get:75 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2024-09-18T10:11:07.375 INFO:teuthology.orchestra.run.smithi189.stdout:Get:76 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2024-09-18T10:11:07.375 INFO:teuthology.orchestra.run.smithi189.stdout:Get:77 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2024-09-18T10:11:07.375 INFO:teuthology.orchestra.run.smithi189.stdout:Get:78 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2024-09-18T10:11:07.376 INFO:teuthology.orchestra.run.smithi189.stdout:Get:79 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.5 [1,384 kB] 2024-09-18T10:11:07.376 INFO:teuthology.orchestra.run.smithi154.stdout:Get:68 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 17.2.7-1645-g7061ea6c-1focal [414 kB] 2024-09-18T10:11:07.383 INFO:teuthology.orchestra.run.smithi154.stdout:Get:69 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 17.2.7-1645-g7061ea6c-1focal [22.6 kB] 2024-09-18T10:11:07.383 INFO:teuthology.orchestra.run.smithi154.stdout:Get:70 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 17.2.7-1645-g7061ea6c-1focal [200 kB] 2024-09-18T10:11:07.385 INFO:teuthology.orchestra.run.smithi029.stdout:Get:50 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2024-09-18T10:11:07.386 INFO:teuthology.orchestra.run.smithi154.stdout:Get:71 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 17.2.7-1645-g7061ea6c-1focal [49.1 kB] 2024-09-18T10:11:07.386 INFO:teuthology.orchestra.run.smithi029.stdout:Get:51 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2024-09-18T10:11:07.386 INFO:teuthology.orchestra.run.smithi029.stdout:Get:52 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2024-09-18T10:11:07.387 INFO:teuthology.orchestra.run.smithi154.stdout:Get:72 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 17.2.7-1645-g7061ea6c-1focal [390 kB] 2024-09-18T10:11:07.387 INFO:teuthology.orchestra.run.smithi029.stdout:Get:53 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2024-09-18T10:11:07.387 INFO:teuthology.orchestra.run.smithi029.stdout:Get:54 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2024-09-18T10:11:07.388 INFO:teuthology.orchestra.run.smithi136.stdout:Get:48 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2024-09-18T10:11:07.388 INFO:teuthology.orchestra.run.smithi136.stdout:Get:49 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2024-09-18T10:11:07.388 INFO:teuthology.orchestra.run.smithi136.stdout:Get:50 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2024-09-18T10:11:07.388 INFO:teuthology.orchestra.run.smithi136.stdout:Get:51 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2024-09-18T10:11:07.388 INFO:teuthology.orchestra.run.smithi136.stdout:Get:52 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2024-09-18T10:11:07.389 INFO:teuthology.orchestra.run.smithi136.stdout:Get:53 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2024-09-18T10:11:07.389 INFO:teuthology.orchestra.run.smithi136.stdout:Get:54 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2024-09-18T10:11:07.390 INFO:teuthology.orchestra.run.smithi136.stdout:Get:55 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2024-09-18T10:11:07.390 INFO:teuthology.orchestra.run.smithi136.stdout:Get:56 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2024-09-18T10:11:07.390 INFO:teuthology.orchestra.run.smithi029.stdout:Get:55 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2024-09-18T10:11:07.390 INFO:teuthology.orchestra.run.smithi029.stdout:Get:56 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2024-09-18T10:11:07.390 INFO:teuthology.orchestra.run.smithi189.stdout:Get:80 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-18T10:11:07.391 INFO:teuthology.orchestra.run.smithi189.stdout:Get:81 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2024-09-18T10:11:07.391 INFO:teuthology.orchestra.run.smithi189.stdout:Get:82 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2024-09-18T10:11:07.392 INFO:teuthology.orchestra.run.smithi189.stdout:Get:83 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2024-09-18T10:11:07.395 INFO:teuthology.orchestra.run.smithi136.stdout:Get:57 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 17.2.7-1645-g7061ea6c-1focal [3,562 kB] 2024-09-18T10:11:07.403 INFO:teuthology.orchestra.run.smithi029.stdout:Get:57 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2024-09-18T10:11:07.406 INFO:teuthology.orchestra.run.smithi154.stdout:Get:73 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-natsort-doc all 7.0.1-1 [103 kB] 2024-09-18T10:11:07.407 INFO:teuthology.orchestra.run.smithi154.stdout:Get:74 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2024-09-18T10:11:07.407 INFO:teuthology.orchestra.run.smithi154.stdout:Get:75 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2024-09-18T10:11:07.408 INFO:teuthology.orchestra.run.smithi029.stdout:Get:58 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2024-09-18T10:11:07.408 INFO:teuthology.orchestra.run.smithi154.stdout:Get:76 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2024-09-18T10:11:07.408 INFO:teuthology.orchestra.run.smithi154.stdout:Get:77 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2024-09-18T10:11:07.408 INFO:teuthology.orchestra.run.smithi136.stdout:Get:58 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2024-09-18T10:11:07.409 INFO:teuthology.orchestra.run.smithi154.stdout:Get:78 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.5 [1,384 kB] 2024-09-18T10:11:07.411 INFO:teuthology.orchestra.run.smithi136.stdout:Get:59 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2024-09-18T10:11:07.411 INFO:teuthology.orchestra.run.smithi189.stdout:Get:84 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2024-09-18T10:11:07.412 INFO:teuthology.orchestra.run.smithi189.stdout:Get:85 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2024-09-18T10:11:07.412 INFO:teuthology.orchestra.run.smithi189.stdout:Get:86 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2024-09-18T10:11:07.415 INFO:teuthology.orchestra.run.smithi189.stdout:Get:87 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2024-09-18T10:11:07.416 INFO:teuthology.orchestra.run.smithi189.stdout:Get:88 http://archive.ubuntu.com/ubuntu focal/universe amd64 qttranslations5-l10n all 5.12.8-0ubuntu1 [1,486 kB] 2024-09-18T10:11:07.417 INFO:teuthology.orchestra.run.smithi154.stdout:Get:79 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 17.2.7-1645-g7061ea6c-1focal [5,637 kB] 2024-09-18T10:11:07.420 INFO:teuthology.orchestra.run.smithi154.stdout:Get:80 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-18T10:11:07.421 INFO:teuthology.orchestra.run.smithi154.stdout:Get:81 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2024-09-18T10:11:07.422 INFO:teuthology.orchestra.run.smithi029.stdout:Get:59 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2024-09-18T10:11:07.423 INFO:teuthology.orchestra.run.smithi029.stdout:Get:60 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2024-09-18T10:11:07.423 INFO:teuthology.orchestra.run.smithi136.stdout:Get:60 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2024-09-18T10:11:07.424 INFO:teuthology.orchestra.run.smithi029.stdout:Get:61 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2024-09-18T10:11:07.425 INFO:teuthology.orchestra.run.smithi136.stdout:Get:61 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2024-09-18T10:11:07.426 INFO:teuthology.orchestra.run.smithi136.stdout:Get:62 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2024-09-18T10:11:07.428 INFO:teuthology.orchestra.run.smithi029.stdout:Get:62 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2024-09-18T10:11:07.430 INFO:teuthology.orchestra.run.smithi136.stdout:Get:63 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2024-09-18T10:11:07.432 INFO:teuthology.orchestra.run.smithi029.stdout:Get:63 http://archive.ubuntu.com/ubuntu focal/universe amd64 fonts-mathjax all 2.7.4+dfsg-1 [2,208 kB] 2024-09-18T10:11:07.433 INFO:teuthology.orchestra.run.smithi136.stdout:Get:64 http://archive.ubuntu.com/ubuntu focal/universe amd64 fonts-mathjax all 2.7.4+dfsg-1 [2,208 kB] 2024-09-18T10:11:07.447 INFO:teuthology.orchestra.run.smithi154.stdout:Get:82 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2024-09-18T10:11:07.448 INFO:teuthology.orchestra.run.smithi029.stdout:Get:64 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.2 [283 kB] 2024-09-18T10:11:07.449 INFO:teuthology.orchestra.run.smithi154.stdout:Get:83 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2024-09-18T10:11:07.451 INFO:teuthology.orchestra.run.smithi029.stdout:Get:65 http://archive.ubuntu.com/ubuntu focal/universe amd64 libjs-mathjax all 2.7.4+dfsg-1 [5,654 kB] 2024-09-18T10:11:07.451 INFO:teuthology.orchestra.run.smithi154.stdout:Get:84 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2024-09-18T10:11:07.452 INFO:teuthology.orchestra.run.smithi154.stdout:Get:85 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2024-09-18T10:11:07.452 INFO:teuthology.orchestra.run.smithi154.stdout:Get:86 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2024-09-18T10:11:07.455 INFO:teuthology.orchestra.run.smithi154.stdout:Get:87 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2024-09-18T10:11:07.456 INFO:teuthology.orchestra.run.smithi154.stdout:Get:88 http://archive.ubuntu.com/ubuntu focal/universe amd64 qttranslations5-l10n all 5.12.8-0ubuntu1 [1,486 kB] 2024-09-18T10:11:07.468 INFO:teuthology.orchestra.run.smithi136.stdout:Get:65 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.2 [283 kB] 2024-09-18T10:11:07.471 INFO:teuthology.orchestra.run.smithi136.stdout:Get:66 http://archive.ubuntu.com/ubuntu focal/universe amd64 libjs-mathjax all 2.7.4+dfsg-1 [5,654 kB] 2024-09-18T10:11:07.493 INFO:teuthology.orchestra.run.smithi029.stdout:Get:66 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-natsort-doc all 7.0.1-1 [103 kB] 2024-09-18T10:11:07.494 INFO:teuthology.orchestra.run.smithi029.stdout:Get:67 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2024-09-18T10:11:07.494 INFO:teuthology.orchestra.run.smithi029.stdout:Get:68 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2024-09-18T10:11:07.495 INFO:teuthology.orchestra.run.smithi029.stdout:Get:69 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2024-09-18T10:11:07.495 INFO:teuthology.orchestra.run.smithi029.stdout:Get:70 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2024-09-18T10:11:07.496 INFO:teuthology.orchestra.run.smithi029.stdout:Get:71 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.5 [1,384 kB] 2024-09-18T10:11:07.507 INFO:teuthology.orchestra.run.smithi029.stdout:Get:72 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-18T10:11:07.518 INFO:teuthology.orchestra.run.smithi136.stdout:Get:67 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-natsort-doc all 7.0.1-1 [103 kB] 2024-09-18T10:11:07.519 INFO:teuthology.orchestra.run.smithi136.stdout:Get:68 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2024-09-18T10:11:07.519 INFO:teuthology.orchestra.run.smithi136.stdout:Get:69 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2024-09-18T10:11:07.520 INFO:teuthology.orchestra.run.smithi136.stdout:Get:70 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2024-09-18T10:11:07.520 INFO:teuthology.orchestra.run.smithi136.stdout:Get:71 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2024-09-18T10:11:07.521 INFO:teuthology.orchestra.run.smithi136.stdout:Get:72 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.5 [1,384 kB] 2024-09-18T10:11:07.532 INFO:teuthology.orchestra.run.smithi136.stdout:Get:73 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-18T10:11:07.532 INFO:teuthology.orchestra.run.smithi136.stdout:Get:74 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2024-09-18T10:11:07.532 INFO:teuthology.orchestra.run.smithi136.stdout:Get:75 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2024-09-18T10:11:07.536 INFO:teuthology.orchestra.run.smithi136.stdout:Get:76 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2024-09-18T10:11:07.555 INFO:teuthology.orchestra.run.smithi136.stdout:Get:77 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2024-09-18T10:11:07.557 INFO:teuthology.orchestra.run.smithi136.stdout:Get:78 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2024-09-18T10:11:07.557 INFO:teuthology.orchestra.run.smithi136.stdout:Get:79 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2024-09-18T10:11:07.558 INFO:teuthology.orchestra.run.smithi189.stdout:Get:89 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 17.2.7-1645-g7061ea6c-1focal [116 kB] 2024-09-18T10:11:07.560 INFO:teuthology.orchestra.run.smithi136.stdout:Get:80 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2024-09-18T10:11:07.561 INFO:teuthology.orchestra.run.smithi189.stdout:Get:90 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 17.2.7-1645-g7061ea6c-1focal [479 kB] 2024-09-18T10:11:07.561 INFO:teuthology.orchestra.run.smithi136.stdout:Get:81 http://archive.ubuntu.com/ubuntu focal/universe amd64 qttranslations5-l10n all 5.12.8-0ubuntu1 [1,486 kB] 2024-09-18T10:11:07.594 INFO:teuthology.orchestra.run.smithi029.stdout:Get:73 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2024-09-18T10:11:07.594 INFO:teuthology.orchestra.run.smithi029.stdout:Get:74 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2024-09-18T10:11:07.597 INFO:teuthology.orchestra.run.smithi029.stdout:Get:75 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2024-09-18T10:11:07.598 INFO:teuthology.orchestra.run.smithi029.stdout:Get:76 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2024-09-18T10:11:07.599 INFO:teuthology.orchestra.run.smithi029.stdout:Get:77 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2024-09-18T10:11:07.599 INFO:teuthology.orchestra.run.smithi029.stdout:Get:78 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2024-09-18T10:11:07.600 INFO:teuthology.orchestra.run.smithi189.stdout:Get:91 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 17.2.7-1645-g7061ea6c-1focal [27.0 MB] 2024-09-18T10:11:07.602 INFO:teuthology.orchestra.run.smithi029.stdout:Get:79 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2024-09-18T10:11:07.603 INFO:teuthology.orchestra.run.smithi029.stdout:Get:80 http://archive.ubuntu.com/ubuntu focal/universe amd64 qttranslations5-l10n all 5.12.8-0ubuntu1 [1,486 kB] 2024-09-18T10:11:07.606 INFO:teuthology.orchestra.run.smithi136.stdout:Get:82 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 17.2.7-1645-g7061ea6c-1focal [761 kB] 2024-09-18T10:11:07.626 INFO:teuthology.orchestra.run.smithi154.stdout:Get:89 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 17.2.7-1645-g7061ea6c-1focal [116 kB] 2024-09-18T10:11:07.628 INFO:teuthology.orchestra.run.smithi154.stdout:Get:90 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 17.2.7-1645-g7061ea6c-1focal [479 kB] 2024-09-18T10:11:07.635 INFO:teuthology.orchestra.run.smithi154.stdout:Get:91 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 17.2.7-1645-g7061ea6c-1focal [27.0 MB] 2024-09-18T10:11:07.653 INFO:teuthology.orchestra.run.smithi136.stdout:Get:83 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 17.2.7-1645-g7061ea6c-1focal [414 kB] 2024-09-18T10:11:07.661 INFO:teuthology.orchestra.run.smithi136.stdout:Get:84 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 17.2.7-1645-g7061ea6c-1focal [22.6 kB] 2024-09-18T10:11:07.662 INFO:teuthology.orchestra.run.smithi136.stdout:Get:85 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 17.2.7-1645-g7061ea6c-1focal [200 kB] 2024-09-18T10:11:07.664 INFO:teuthology.orchestra.run.smithi136.stdout:Get:86 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 17.2.7-1645-g7061ea6c-1focal [49.1 kB] 2024-09-18T10:11:07.700 INFO:teuthology.orchestra.run.smithi136.stdout:Get:87 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 17.2.7-1645-g7061ea6c-1focal [390 kB] 2024-09-18T10:11:07.708 INFO:teuthology.orchestra.run.smithi136.stdout:Get:88 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 17.2.7-1645-g7061ea6c-1focal [5,637 kB] 2024-09-18T10:11:08.045 INFO:teuthology.orchestra.run.smithi136.stdout:Get:89 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 17.2.7-1645-g7061ea6c-1focal [116 kB] 2024-09-18T10:11:08.047 INFO:teuthology.orchestra.run.smithi136.stdout:Get:90 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 17.2.7-1645-g7061ea6c-1focal [479 kB] 2024-09-18T10:11:08.055 INFO:teuthology.orchestra.run.smithi136.stdout:Get:91 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 17.2.7-1645-g7061ea6c-1focal [27.0 MB] 2024-09-18T10:11:08.065 INFO:teuthology.orchestra.run.smithi029.stdout:Get:81 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 17.2.7-1645-g7061ea6c-1focal [3,562 kB] 2024-09-18T10:11:08.686 INFO:teuthology.orchestra.run.smithi154.stdout:Get:92 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 17.2.7-1645-g7061ea6c-1focal [5,177 kB] 2024-09-18T10:11:08.783 INFO:teuthology.orchestra.run.smithi154.stdout:Get:93 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 17.2.7-1645-g7061ea6c-1focal [223 kB] 2024-09-18T10:11:08.801 INFO:teuthology.orchestra.run.smithi154.stdout:Get:94 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 libsqlite3-mod-ceph amd64 17.2.7-1645-g7061ea6c-1focal [157 kB] 2024-09-18T10:11:08.805 INFO:teuthology.orchestra.run.smithi154.stdout:Get:95 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 17.2.7-1645-g7061ea6c-1focal [1,594 kB] 2024-09-18T10:11:08.850 INFO:teuthology.orchestra.run.smithi029.stdout:Get:82 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 17.2.7-1645-g7061ea6c-1focal [761 kB] 2024-09-18T10:11:08.862 INFO:teuthology.orchestra.run.smithi154.stdout:Get:96 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 17.2.7-1645-g7061ea6c-1focal [4,688 kB] 2024-09-18T10:11:09.021 INFO:teuthology.orchestra.run.smithi154.stdout:Get:97 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 17.2.7-1645-g7061ea6c-1focal [23.1 MB] 2024-09-18T10:11:09.033 INFO:teuthology.orchestra.run.smithi029.stdout:Get:83 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 17.2.7-1645-g7061ea6c-1focal [414 kB] 2024-09-18T10:11:09.099 INFO:teuthology.orchestra.run.smithi029.stdout:Get:84 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 17.2.7-1645-g7061ea6c-1focal [22.6 kB] 2024-09-18T10:11:09.103 INFO:teuthology.orchestra.run.smithi029.stdout:Get:85 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 17.2.7-1645-g7061ea6c-1focal [200 kB] 2024-09-18T10:11:09.161 INFO:teuthology.orchestra.run.smithi029.stdout:Get:86 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 17.2.7-1645-g7061ea6c-1focal [49.1 kB] 2024-09-18T10:11:09.162 INFO:teuthology.orchestra.run.smithi029.stdout:Get:87 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 17.2.7-1645-g7061ea6c-1focal [390 kB] 2024-09-18T10:11:09.284 INFO:teuthology.orchestra.run.smithi029.stdout:Get:88 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 17.2.7-1645-g7061ea6c-1focal [5,637 kB] 2024-09-18T10:11:09.587 INFO:teuthology.orchestra.run.smithi136.stdout:Get:92 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 17.2.7-1645-g7061ea6c-1focal [5,177 kB] 2024-09-18T10:11:09.803 INFO:teuthology.orchestra.run.smithi136.stdout:Get:93 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 17.2.7-1645-g7061ea6c-1focal [223 kB] 2024-09-18T10:11:09.805 INFO:teuthology.orchestra.run.smithi136.stdout:Get:94 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 libsqlite3-mod-ceph amd64 17.2.7-1645-g7061ea6c-1focal [157 kB] 2024-09-18T10:11:09.808 INFO:teuthology.orchestra.run.smithi136.stdout:Get:95 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 17.2.7-1645-g7061ea6c-1focal [1,594 kB] 2024-09-18T10:11:09.893 INFO:teuthology.orchestra.run.smithi136.stdout:Get:96 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 17.2.7-1645-g7061ea6c-1focal [4,688 kB] 2024-09-18T10:11:09.920 INFO:teuthology.orchestra.run.smithi154.stdout:Get:98 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 17.2.7-1645-g7061ea6c-1focal [3,936 B] 2024-09-18T10:11:09.920 INFO:teuthology.orchestra.run.smithi154.stdout:Get:99 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 17.2.7-1645-g7061ea6c-1focal [872 kB] 2024-09-18T10:11:09.933 INFO:teuthology.orchestra.run.smithi154.stdout:Get:100 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 17.2.7-1645-g7061ea6c-1focal [2,416 kB] 2024-09-18T10:11:09.950 INFO:teuthology.orchestra.run.smithi154.stdout:Get:101 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 17.2.7-1645-g7061ea6c-1focal [81.4 kB] 2024-09-18T10:11:09.951 INFO:teuthology.orchestra.run.smithi154.stdout:Get:102 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 17.2.7-1645-g7061ea6c-1focal [120 kB] 2024-09-18T10:11:09.951 INFO:teuthology.orchestra.run.smithi154.stdout:Get:103 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 17.2.7-1645-g7061ea6c-1focal [6,653 kB] 2024-09-18T10:11:10.130 INFO:teuthology.orchestra.run.smithi136.stdout:Get:97 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 17.2.7-1645-g7061ea6c-1focal [23.1 MB] 2024-09-18T10:11:10.185 INFO:teuthology.orchestra.run.smithi154.stdout:Get:104 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 17.2.7-1645-g7061ea6c-1focal [7,884 kB] 2024-09-18T10:11:10.447 INFO:teuthology.orchestra.run.smithi154.stdout:Get:105 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 17.2.7-1645-g7061ea6c-1focal [4,056 B] 2024-09-18T10:11:10.448 INFO:teuthology.orchestra.run.smithi154.stdout:Get:106 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 17.2.7-1645-g7061ea6c-1focal [53.4 MB] 2024-09-18T10:11:10.482 INFO:teuthology.orchestra.run.smithi029.stdout:Get:89 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 17.2.7-1645-g7061ea6c-1focal [116 kB] 2024-09-18T10:11:10.485 INFO:teuthology.orchestra.run.smithi029.stdout:Get:90 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 17.2.7-1645-g7061ea6c-1focal [479 kB] 2024-09-18T10:11:10.554 INFO:teuthology.orchestra.run.smithi189.stdout:Get:92 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 17.2.7-1645-g7061ea6c-1focal [5,177 kB] 2024-09-18T10:11:10.611 INFO:teuthology.orchestra.run.smithi029.stdout:Get:91 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 17.2.7-1645-g7061ea6c-1focal [27.0 MB] 2024-09-18T10:11:11.004 INFO:teuthology.orchestra.run.smithi189.stdout:Get:93 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 17.2.7-1645-g7061ea6c-1focal [223 kB] 2024-09-18T10:11:11.009 INFO:teuthology.orchestra.run.smithi189.stdout:Get:94 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 libsqlite3-mod-ceph amd64 17.2.7-1645-g7061ea6c-1focal [157 kB] 2024-09-18T10:11:11.057 INFO:teuthology.orchestra.run.smithi189.stdout:Get:95 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 17.2.7-1645-g7061ea6c-1focal [1,594 kB] 2024-09-18T10:11:11.224 INFO:teuthology.orchestra.run.smithi189.stdout:Get:96 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 17.2.7-1645-g7061ea6c-1focal [4,688 kB] 2024-09-18T10:11:11.468 INFO:teuthology.orchestra.run.smithi136.stdout:Get:98 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 17.2.7-1645-g7061ea6c-1focal [3,936 B] 2024-09-18T10:11:11.468 INFO:teuthology.orchestra.run.smithi136.stdout:Get:99 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 17.2.7-1645-g7061ea6c-1focal [872 kB] 2024-09-18T10:11:11.482 INFO:teuthology.orchestra.run.smithi136.stdout:Get:100 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 17.2.7-1645-g7061ea6c-1focal [2,416 kB] 2024-09-18T10:11:11.547 INFO:teuthology.orchestra.run.smithi136.stdout:Get:101 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 17.2.7-1645-g7061ea6c-1focal [81.4 kB] 2024-09-18T10:11:11.547 INFO:teuthology.orchestra.run.smithi136.stdout:Get:102 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 17.2.7-1645-g7061ea6c-1focal [120 kB] 2024-09-18T10:11:11.577 INFO:teuthology.orchestra.run.smithi136.stdout:Get:103 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 17.2.7-1645-g7061ea6c-1focal [6,653 kB] 2024-09-18T10:11:11.744 INFO:teuthology.orchestra.run.smithi189.stdout:Get:97 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 17.2.7-1645-g7061ea6c-1focal [23.1 MB] 2024-09-18T10:11:11.922 INFO:teuthology.orchestra.run.smithi136.stdout:Get:104 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 17.2.7-1645-g7061ea6c-1focal [7,884 kB] 2024-09-18T10:11:12.345 INFO:teuthology.orchestra.run.smithi136.stdout:Get:105 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 17.2.7-1645-g7061ea6c-1focal [4,056 B] 2024-09-18T10:11:12.345 INFO:teuthology.orchestra.run.smithi136.stdout:Get:106 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 17.2.7-1645-g7061ea6c-1focal [53.4 MB] 2024-09-18T10:11:12.535 INFO:teuthology.orchestra.run.smithi154.stdout:Get:107 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 ceph-volume all 17.2.7-1645-g7061ea6c-1focal [231 kB] 2024-09-18T10:11:12.539 INFO:teuthology.orchestra.run.smithi154.stdout:Get:108 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 17.2.7-1645-g7061ea6c-1focal [21.9 kB] 2024-09-18T10:11:12.539 INFO:teuthology.orchestra.run.smithi154.stdout:Get:109 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 17.2.7-1645-g7061ea6c-1focal [17.2 MB] 2024-09-18T10:11:13.118 INFO:teuthology.orchestra.run.smithi154.stdout:Get:110 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 17.2.7-1645-g7061ea6c-1focal [80.6 kB] 2024-09-18T10:11:13.775 INFO:teuthology.orchestra.run.smithi154.stdout:Fetched 201 MB in 6s (31.7 MB/s) 2024-09-18T10:11:13.831 INFO:teuthology.orchestra.run.smithi189.stdout:Get:98 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 17.2.7-1645-g7061ea6c-1focal [3,936 B] 2024-09-18T10:11:13.832 INFO:teuthology.orchestra.run.smithi189.stdout:Get:99 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 17.2.7-1645-g7061ea6c-1focal [872 kB] 2024-09-18T10:11:13.846 INFO:teuthology.orchestra.run.smithi189.stdout:Get:100 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 17.2.7-1645-g7061ea6c-1focal [2,416 kB] 2024-09-18T10:11:13.912 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2024-09-18T10:11:13.970 INFO:teuthology.orchestra.run.smithi154.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 141720 files and directories currently installed.) 2024-09-18T10:11:13.974 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../000-liblttng-ust0_2.11.0-1_amd64.deb ... 2024-09-18T10:11:13.975 INFO:teuthology.orchestra.run.smithi189.stdout:Get:101 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 17.2.7-1645-g7061ea6c-1focal [81.4 kB] 2024-09-18T10:11:13.991 INFO:teuthology.orchestra.run.smithi189.stdout:Get:102 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 17.2.7-1645-g7061ea6c-1focal [120 kB] 2024-09-18T10:11:13.994 INFO:teuthology.orchestra.run.smithi189.stdout:Get:103 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 17.2.7-1645-g7061ea6c-1focal [6,653 kB] 2024-09-18T10:11:14.052 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2024-09-18T10:11:14.286 INFO:teuthology.orchestra.run.smithi189.stdout:Get:104 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 17.2.7-1645-g7061ea6c-1focal [7,884 kB] 2024-09-18T10:11:14.314 INFO:teuthology.orchestra.run.smithi136.stdout:Get:107 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 ceph-volume all 17.2.7-1645-g7061ea6c-1focal [231 kB] 2024-09-18T10:11:14.318 INFO:teuthology.orchestra.run.smithi136.stdout:Get:108 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 17.2.7-1645-g7061ea6c-1focal [21.9 kB] 2024-09-18T10:11:14.319 INFO:teuthology.orchestra.run.smithi136.stdout:Get:109 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 17.2.7-1645-g7061ea6c-1focal [17.2 MB] 2024-09-18T10:11:14.339 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../001-librbd1_17.2.7-1645-g7061ea6c-1focal_amd64.deb ... 2024-09-18T10:11:14.464 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking librbd1 (17.2.7-1645-g7061ea6c-1focal) over (15.2.17-0ubuntu0.20.04.6) ... 2024-09-18T10:11:14.644 INFO:teuthology.orchestra.run.smithi029.stdout:Get:92 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 17.2.7-1645-g7061ea6c-1focal [5,177 kB] 2024-09-18T10:11:14.704 INFO:teuthology.orchestra.run.smithi189.stdout:Get:105 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 17.2.7-1645-g7061ea6c-1focal [4,056 B] 2024-09-18T10:11:14.704 INFO:teuthology.orchestra.run.smithi189.stdout:Get:106 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 17.2.7-1645-g7061ea6c-1focal [53.4 MB] 2024-09-18T10:11:14.781 INFO:teuthology.orchestra.run.smithi136.stdout:Get:110 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 17.2.7-1645-g7061ea6c-1focal [80.6 kB] 2024-09-18T10:11:14.935 INFO:teuthology.orchestra.run.smithi029.stdout:Get:93 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 17.2.7-1645-g7061ea6c-1focal [223 kB] 2024-09-18T10:11:14.942 INFO:teuthology.orchestra.run.smithi029.stdout:Get:94 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 libsqlite3-mod-ceph amd64 17.2.7-1645-g7061ea6c-1focal [157 kB] 2024-09-18T10:11:14.955 INFO:teuthology.orchestra.run.smithi029.stdout:Get:95 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 17.2.7-1645-g7061ea6c-1focal [1,594 kB] 2024-09-18T10:11:15.045 INFO:teuthology.orchestra.run.smithi029.stdout:Get:96 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 17.2.7-1645-g7061ea6c-1focal [4,688 kB] 2024-09-18T10:11:15.159 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../002-librados2_17.2.7-1645-g7061ea6c-1focal_amd64.deb ... 2024-09-18T10:11:15.285 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking librados2 (17.2.7-1645-g7061ea6c-1focal) over (15.2.17-0ubuntu0.20.04.6) ... 2024-09-18T10:11:15.314 INFO:teuthology.orchestra.run.smithi029.stdout:Get:97 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 17.2.7-1645-g7061ea6c-1focal [23.1 MB] 2024-09-18T10:11:15.431 INFO:teuthology.orchestra.run.smithi136.stdout:Fetched 201 MB in 8s (25.3 MB/s) 2024-09-18T10:11:15.604 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2024-09-18T10:11:15.660 INFO:teuthology.orchestra.run.smithi136.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 141720 files and directories currently installed.) 2024-09-18T10:11:15.665 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../000-liblttng-ust0_2.11.0-1_amd64.deb ... 2024-09-18T10:11:15.744 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2024-09-18T10:11:15.881 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libcephfs2. 2024-09-18T10:11:15.897 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../003-libcephfs2_17.2.7-1645-g7061ea6c-1focal_amd64.deb ... 2024-09-18T10:11:15.939 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libcephfs2 (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:11:16.115 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../001-librbd1_17.2.7-1645-g7061ea6c-1focal_amd64.deb ... 2024-09-18T10:11:16.240 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking librbd1 (17.2.7-1645-g7061ea6c-1focal) over (15.2.17-0ubuntu0.20.04.6) ... 2024-09-18T10:11:16.275 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-rados. 2024-09-18T10:11:16.291 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../004-python3-rados_17.2.7-1645-g7061ea6c-1focal_amd64.deb ... 2024-09-18T10:11:16.333 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-rados (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:11:16.640 INFO:teuthology.orchestra.run.smithi189.stdout:Get:107 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 ceph-volume all 17.2.7-1645-g7061ea6c-1focal [231 kB] 2024-09-18T10:11:16.644 INFO:teuthology.orchestra.run.smithi189.stdout:Get:108 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 17.2.7-1645-g7061ea6c-1focal [21.9 kB] 2024-09-18T10:11:16.645 INFO:teuthology.orchestra.run.smithi189.stdout:Get:109 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 17.2.7-1645-g7061ea6c-1focal [17.2 MB] 2024-09-18T10:11:16.694 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-18T10:11:16.712 INFO:teuthology.orchestra.run.smithi029.stdout:Get:98 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 17.2.7-1645-g7061ea6c-1focal [3,936 B] 2024-09-18T10:11:16.713 INFO:teuthology.orchestra.run.smithi029.stdout:Get:99 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 17.2.7-1645-g7061ea6c-1focal [872 kB] 2024-09-18T10:11:16.713 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../005-python3-ceph-argparse_17.2.7-1645-g7061ea6c-1focal_all.deb ... 2024-09-18T10:11:16.733 INFO:teuthology.orchestra.run.smithi029.stdout:Get:100 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 17.2.7-1645-g7061ea6c-1focal [2,416 kB] 2024-09-18T10:11:16.752 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-ceph-argparse (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:11:16.858 INFO:teuthology.orchestra.run.smithi029.stdout:Get:101 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 17.2.7-1645-g7061ea6c-1focal [81.4 kB] 2024-09-18T10:11:16.859 INFO:teuthology.orchestra.run.smithi029.stdout:Get:102 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 17.2.7-1645-g7061ea6c-1focal [120 kB] 2024-09-18T10:11:16.862 INFO:teuthology.orchestra.run.smithi029.stdout:Get:103 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 17.2.7-1645-g7061ea6c-1focal [6,653 kB] 2024-09-18T10:11:16.980 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-cephfs. 2024-09-18T10:11:16.987 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../002-librados2_17.2.7-1645-g7061ea6c-1focal_amd64.deb ... 2024-09-18T10:11:16.998 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../006-python3-cephfs_17.2.7-1645-g7061ea6c-1focal_amd64.deb ... 2024-09-18T10:11:17.029 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-cephfs (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:11:17.103 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking librados2 (17.2.7-1645-g7061ea6c-1focal) over (15.2.17-0ubuntu0.20.04.6) ... 2024-09-18T10:11:17.181 INFO:teuthology.orchestra.run.smithi189.stdout:Get:110 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 17.2.7-1645-g7061ea6c-1focal [80.6 kB] 2024-09-18T10:11:17.197 INFO:teuthology.orchestra.run.smithi029.stdout:Get:104 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 17.2.7-1645-g7061ea6c-1focal [7,884 kB] 2024-09-18T10:11:17.324 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-18T10:11:17.344 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../007-python3-ceph-common_17.2.7-1645-g7061ea6c-1focal_all.deb ... 2024-09-18T10:11:17.382 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-ceph-common (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:11:17.591 INFO:teuthology.orchestra.run.smithi029.stdout:Get:105 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 17.2.7-1645-g7061ea6c-1focal [4,056 B] 2024-09-18T10:11:17.591 INFO:teuthology.orchestra.run.smithi029.stdout:Get:106 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 17.2.7-1645-g7061ea6c-1focal [53.4 MB] 2024-09-18T10:11:17.743 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-prettytable. 2024-09-18T10:11:17.759 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../008-python3-prettytable_0.7.2-5_all.deb ... 2024-09-18T10:11:17.766 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package libcephfs2. 2024-09-18T10:11:17.781 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../003-libcephfs2_17.2.7-1645-g7061ea6c-1focal_amd64.deb ... 2024-09-18T10:11:17.801 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2024-09-18T10:11:17.824 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking libcephfs2 (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:11:17.832 INFO:teuthology.orchestra.run.smithi189.stdout:Fetched 201 MB in 10s (19.3 MB/s) 2024-09-18T10:11:17.993 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2024-09-18T10:11:18.051 INFO:teuthology.orchestra.run.smithi189.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 141720 files and directories currently installed.) 2024-09-18T10:11:18.054 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-rbd. 2024-09-18T10:11:18.055 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../000-liblttng-ust0_2.11.0-1_amd64.deb ... 2024-09-18T10:11:18.073 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../009-python3-rbd_17.2.7-1645-g7061ea6c-1focal_amd64.deb ... 2024-09-18T10:11:18.113 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-rbd (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:11:18.192 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2024-09-18T10:11:18.193 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-rados. 2024-09-18T10:11:18.209 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../004-python3-rados_17.2.7-1645-g7061ea6c-1focal_amd64.deb ... 2024-09-18T10:11:18.251 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-rados (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:11:18.457 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package liblua5.3-0:amd64. 2024-09-18T10:11:18.476 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../010-liblua5.3-0_5.3.3-1.1ubuntu2_amd64.deb ... 2024-09-18T10:11:18.515 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2024-09-18T10:11:18.538 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../001-librbd1_17.2.7-1645-g7061ea6c-1focal_amd64.deb ... 2024-09-18T10:11:18.596 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-18T10:11:18.612 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../005-python3-ceph-argparse_17.2.7-1645-g7061ea6c-1focal_all.deb ... 2024-09-18T10:11:18.663 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-ceph-argparse (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:11:18.713 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking librbd1 (17.2.7-1645-g7061ea6c-1focal) over (15.2.17-0ubuntu0.20.04.6) ... 2024-09-18T10:11:18.885 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package librabbitmq4:amd64. 2024-09-18T10:11:18.902 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../011-librabbitmq4_0.10.0-1_amd64.deb ... 2024-09-18T10:11:18.924 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-cephfs. 2024-09-18T10:11:18.942 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../006-python3-cephfs_17.2.7-1645-g7061ea6c-1focal_amd64.deb ... 2024-09-18T10:11:18.943 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2024-09-18T10:11:18.982 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-cephfs (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:11:19.271 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-18T10:11:19.289 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../012-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2024-09-18T10:11:19.310 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-18T10:11:19.325 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../007-python3-ceph-common_17.2.7-1645-g7061ea6c-1focal_all.deb ... 2024-09-18T10:11:19.329 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2024-09-18T10:11:19.368 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-ceph-common (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:11:19.369 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../002-librados2_17.2.7-1645-g7061ea6c-1focal_amd64.deb ... 2024-09-18T10:11:19.520 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking librados2 (17.2.7-1645-g7061ea6c-1focal) over (15.2.17-0ubuntu0.20.04.6) ... 2024-09-18T10:11:19.682 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-18T10:11:19.701 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../013-libdouble-conversion3_3.1.5-4ubuntu1_amd64.deb ... 2024-09-18T10:11:19.730 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-prettytable. 2024-09-18T10:11:19.740 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2024-09-18T10:11:19.745 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../008-python3-prettytable_0.7.2-5_all.deb ... 2024-09-18T10:11:19.796 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2024-09-18T10:11:20.032 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-rbd. 2024-09-18T10:11:20.050 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../009-python3-rbd_17.2.7-1645-g7061ea6c-1focal_amd64.deb ... 2024-09-18T10:11:20.082 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-rbd (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:11:20.101 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-18T10:11:20.118 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../014-libpcre2-16-0_10.34-7ubuntu0.1_amd64.deb ... 2024-09-18T10:11:20.160 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2024-09-18T10:11:20.206 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package libcephfs2. 2024-09-18T10:11:20.224 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../003-libcephfs2_17.2.7-1645-g7061ea6c-1focal_amd64.deb ... 2024-09-18T10:11:20.256 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking libcephfs2 (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:11:20.418 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package liblua5.3-0:amd64. 2024-09-18T10:11:20.425 INFO:teuthology.orchestra.run.smithi029.stdout:Get:107 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 ceph-volume all 17.2.7-1645-g7061ea6c-1focal [231 kB] 2024-09-18T10:11:20.429 INFO:teuthology.orchestra.run.smithi029.stdout:Get:108 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 17.2.7-1645-g7061ea6c-1focal [21.9 kB] 2024-09-18T10:11:20.430 INFO:teuthology.orchestra.run.smithi029.stdout:Get:109 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 17.2.7-1645-g7061ea6c-1focal [17.2 MB] 2024-09-18T10:11:20.435 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../010-liblua5.3-0_5.3.3-1.1ubuntu2_amd64.deb ... 2024-09-18T10:11:20.477 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2024-09-18T10:11:20.559 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-rados. 2024-09-18T10:11:20.575 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../004-python3-rados_17.2.7-1645-g7061ea6c-1focal_amd64.deb ... 2024-09-18T10:11:20.608 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-rados (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:11:20.621 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-18T10:11:20.641 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../015-libqt5core5a_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-18T10:11:20.690 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-18T10:11:20.855 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package librabbitmq4:amd64. 2024-09-18T10:11:20.872 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../011-librabbitmq4_0.10.0-1_amd64.deb ... 2024-09-18T10:11:20.913 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2024-09-18T10:11:20.986 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-18T10:11:21.003 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../005-python3-ceph-argparse_17.2.7-1645-g7061ea6c-1focal_all.deb ... 2024-09-18T10:11:21.044 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-ceph-argparse (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:11:21.079 INFO:teuthology.orchestra.run.smithi029.stdout:Get:110 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 17.2.7-1645-g7061ea6c-1focal [80.6 kB] 2024-09-18T10:11:21.199 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-18T10:11:21.216 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-18T10:11:21.218 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../012-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2024-09-18T10:11:21.236 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../016-libqt5dbus5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-18T10:11:21.291 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-18T10:11:21.322 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-cephfs. 2024-09-18T10:11:21.341 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../006-python3-cephfs_17.2.7-1645-g7061ea6c-1focal_amd64.deb ... 2024-09-18T10:11:21.349 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2024-09-18T10:11:21.380 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-cephfs (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:11:21.652 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-18T10:11:21.667 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../017-libqt5network5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-18T10:11:21.691 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-18T10:11:21.707 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../007-python3-ceph-common_17.2.7-1645-g7061ea6c-1focal_all.deb ... 2024-09-18T10:11:21.710 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-18T10:11:21.743 INFO:teuthology.orchestra.run.smithi029.stdout:Fetched 201 MB in 14s (14.1 MB/s) 2024-09-18T10:11:21.774 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-ceph-common (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:11:21.794 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-18T10:11:21.811 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../013-libdouble-conversion3_3.1.5-4ubuntu1_amd64.deb ... 2024-09-18T10:11:21.853 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2024-09-18T10:11:21.879 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2024-09-18T10:11:21.939 INFO:teuthology.orchestra.run.smithi029.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 141720 files and directories currently installed.) 2024-09-18T10:11:21.943 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../000-liblttng-ust0_2.11.0-1_amd64.deb ... 2024-09-18T10:11:22.028 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2024-09-18T10:11:22.052 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-prettytable. 2024-09-18T10:11:22.067 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../008-python3-prettytable_0.7.2-5_all.deb ... 2024-09-18T10:11:22.102 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2024-09-18T10:11:22.213 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libthrift-0.13.0. 2024-09-18T10:11:22.214 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-18T10:11:22.230 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../014-libpcre2-16-0_10.34-7ubuntu0.1_amd64.deb ... 2024-09-18T10:11:22.233 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../018-libthrift-0.13.0_0.13.0-2build2_amd64.deb ... 2024-09-18T10:11:22.271 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libthrift-0.13.0 (0.13.0-2build2) ... 2024-09-18T10:11:22.272 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2024-09-18T10:11:22.371 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-rbd. 2024-09-18T10:11:22.381 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../001-librbd1_17.2.7-1645-g7061ea6c-1focal_amd64.deb ... 2024-09-18T10:11:22.387 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../009-python3-rbd_17.2.7-1645-g7061ea6c-1focal_amd64.deb ... 2024-09-18T10:11:22.429 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-rbd (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:11:22.512 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking librbd1 (17.2.7-1645-g7061ea6c-1focal) over (15.2.17-0ubuntu0.20.04.6) ... 2024-09-18T10:11:22.624 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package librgw2. 2024-09-18T10:11:22.642 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../019-librgw2_17.2.7-1645-g7061ea6c-1focal_amd64.deb ... 2024-09-18T10:11:22.668 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-18T10:11:22.682 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking librgw2 (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:11:22.684 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../015-libqt5core5a_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-18T10:11:22.728 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-18T10:11:22.991 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package liblua5.3-0:amd64. 2024-09-18T10:11:23.009 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../010-liblua5.3-0_5.3.3-1.1ubuntu2_amd64.deb ... 2024-09-18T10:11:23.049 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2024-09-18T10:11:23.229 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-18T10:11:23.250 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../016-libqt5dbus5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-18T10:11:23.287 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-18T10:11:23.386 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../002-librados2_17.2.7-1645-g7061ea6c-1focal_amd64.deb ... 2024-09-18T10:11:23.393 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package librabbitmq4:amd64. 2024-09-18T10:11:23.410 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../011-librabbitmq4_0.10.0-1_amd64.deb ... 2024-09-18T10:11:23.451 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2024-09-18T10:11:23.460 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-rgw. 2024-09-18T10:11:23.475 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../020-python3-rgw_17.2.7-1645-g7061ea6c-1focal_amd64.deb ... 2024-09-18T10:11:23.518 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-rgw (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:11:23.588 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking librados2 (17.2.7-1645-g7061ea6c-1focal) over (15.2.17-0ubuntu0.20.04.6) ... 2024-09-18T10:11:23.674 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-18T10:11:23.691 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../017-libqt5network5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-18T10:11:23.732 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-18T10:11:23.821 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package liboath0. 2024-09-18T10:11:23.821 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-18T10:11:23.838 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../012-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2024-09-18T10:11:23.838 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../021-liboath0_2.6.1-1.3_amd64.deb ... 2024-09-18T10:11:23.871 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2024-09-18T10:11:23.879 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2024-09-18T10:11:24.069 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package libthrift-0.13.0. 2024-09-18T10:11:24.089 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../018-libthrift-0.13.0_0.13.0-2build2_amd64.deb ... 2024-09-18T10:11:24.127 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking libthrift-0.13.0 (0.13.0-2build2) ... 2024-09-18T10:11:24.190 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libradosstriper1. 2024-09-18T10:11:24.207 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../022-libradosstriper1_17.2.7-1645-g7061ea6c-1focal_amd64.deb ... 2024-09-18T10:11:24.248 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libradosstriper1 (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:11:24.282 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-18T10:11:24.299 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../013-libdouble-conversion3_3.1.5-4ubuntu1_amd64.deb ... 2024-09-18T10:11:24.340 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2024-09-18T10:11:24.363 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package libcephfs2. 2024-09-18T10:11:24.381 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../003-libcephfs2_17.2.7-1645-g7061ea6c-1focal_amd64.deb ... 2024-09-18T10:11:24.422 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking libcephfs2 (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:11:24.497 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package librgw2. 2024-09-18T10:11:24.517 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../019-librgw2_17.2.7-1645-g7061ea6c-1focal_amd64.deb ... 2024-09-18T10:11:24.555 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking librgw2 (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:11:24.635 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-18T10:11:24.651 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../014-libpcre2-16-0_10.34-7ubuntu0.1_amd64.deb ... 2024-09-18T10:11:24.684 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2024-09-18T10:11:24.693 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package ceph-common. 2024-09-18T10:11:24.713 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../023-ceph-common_17.2.7-1645-g7061ea6c-1focal_amd64.deb ... 2024-09-18T10:11:24.751 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking ceph-common (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:11:24.759 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-rados. 2024-09-18T10:11:24.777 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../004-python3-rados_17.2.7-1645-g7061ea6c-1focal_amd64.deb ... 2024-09-18T10:11:24.826 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-rados (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:11:25.196 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-18T10:11:25.199 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-18T10:11:25.212 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../015-libqt5core5a_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-18T10:11:25.215 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../005-python3-ceph-argparse_17.2.7-1645-g7061ea6c-1focal_all.deb ... 2024-09-18T10:11:25.258 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-ceph-argparse (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:11:25.265 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-18T10:11:25.341 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-rgw. 2024-09-18T10:11:25.360 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../020-python3-rgw_17.2.7-1645-g7061ea6c-1focal_amd64.deb ... 2024-09-18T10:11:25.399 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-rgw (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:11:25.504 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-cephfs. 2024-09-18T10:11:25.522 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../006-python3-cephfs_17.2.7-1645-g7061ea6c-1focal_amd64.deb ... 2024-09-18T10:11:25.562 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-cephfs (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:11:25.736 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package liboath0. 2024-09-18T10:11:25.755 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../021-liboath0_2.6.1-1.3_amd64.deb ... 2024-09-18T10:11:25.786 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2024-09-18T10:11:25.791 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-18T10:11:25.808 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../016-libqt5dbus5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-18T10:11:25.849 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-18T10:11:25.910 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-18T10:11:25.927 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../007-python3-ceph-common_17.2.7-1645-g7061ea6c-1focal_all.deb ... 2024-09-18T10:11:25.969 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-ceph-common (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:11:26.097 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package libradosstriper1. 2024-09-18T10:11:26.113 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../022-libradosstriper1_17.2.7-1645-g7061ea6c-1focal_amd64.deb ... 2024-09-18T10:11:26.152 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-18T10:11:26.155 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking libradosstriper1 (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:11:26.170 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../017-libqt5network5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-18T10:11:26.220 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-18T10:11:26.317 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-prettytable. 2024-09-18T10:11:26.337 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../008-python3-prettytable_0.7.2-5_all.deb ... 2024-09-18T10:11:26.376 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2024-09-18T10:11:26.613 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package libthrift-0.13.0. 2024-09-18T10:11:26.617 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package ceph-common. 2024-09-18T10:11:26.622 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-rbd. 2024-09-18T10:11:26.630 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../018-libthrift-0.13.0_0.13.0-2build2_amd64.deb ... 2024-09-18T10:11:26.635 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../023-ceph-common_17.2.7-1645-g7061ea6c-1focal_amd64.deb ... 2024-09-18T10:11:26.638 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../009-python3-rbd_17.2.7-1645-g7061ea6c-1focal_amd64.deb ... 2024-09-18T10:11:26.675 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking ceph-common (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:11:26.680 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-rbd (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:11:26.696 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking libthrift-0.13.0 (0.13.0-2build2) ... 2024-09-18T10:11:27.045 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package liblua5.3-0:amd64. 2024-09-18T10:11:27.049 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package librgw2. 2024-09-18T10:11:27.065 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../010-liblua5.3-0_5.3.3-1.1ubuntu2_amd64.deb ... 2024-09-18T10:11:27.066 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../019-librgw2_17.2.7-1645-g7061ea6c-1focal_amd64.deb ... 2024-09-18T10:11:27.104 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2024-09-18T10:11:27.124 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking librgw2 (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:11:27.207 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package ceph-base. 2024-09-18T10:11:27.223 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../024-ceph-base_17.2.7-1645-g7061ea6c-1focal_amd64.deb ... 2024-09-18T10:11:27.292 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking ceph-base (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:11:27.459 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package librabbitmq4:amd64. 2024-09-18T10:11:27.475 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../011-librabbitmq4_0.10.0-1_amd64.deb ... 2024-09-18T10:11:27.519 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2024-09-18T10:11:27.866 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-18T10:11:27.883 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../012-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2024-09-18T10:11:27.924 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2024-09-18T10:11:27.943 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-rgw. 2024-09-18T10:11:27.960 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../020-python3-rgw_17.2.7-1645-g7061ea6c-1focal_amd64.deb ... 2024-09-18T10:11:28.001 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-rgw (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:11:28.136 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-cryptography. 2024-09-18T10:11:28.151 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../025-python3-cryptography_2.8-3ubuntu0.3_amd64.deb ... 2024-09-18T10:11:28.194 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.3) ... 2024-09-18T10:11:28.314 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-18T10:11:28.331 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../013-libdouble-conversion3_3.1.5-4ubuntu1_amd64.deb ... 2024-09-18T10:11:28.373 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2024-09-18T10:11:28.581 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-openssl. 2024-09-18T10:11:28.596 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package liboath0. 2024-09-18T10:11:28.600 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../026-python3-openssl_19.0.0-1build1_all.deb ... 2024-09-18T10:11:28.614 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../021-liboath0_2.6.1-1.3_amd64.deb ... 2024-09-18T10:11:28.639 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2024-09-18T10:11:28.654 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2024-09-18T10:11:28.703 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-18T10:11:28.719 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../014-libpcre2-16-0_10.34-7ubuntu0.1_amd64.deb ... 2024-09-18T10:11:28.755 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2024-09-18T10:11:28.950 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-packaging. 2024-09-18T10:11:28.966 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../027-python3-packaging_20.3-1_all.deb ... 2024-09-18T10:11:28.982 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package libradosstriper1. 2024-09-18T10:11:29.001 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../022-libradosstriper1_17.2.7-1645-g7061ea6c-1focal_amd64.deb ... 2024-09-18T10:11:29.008 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-packaging (20.3-1) ... 2024-09-18T10:11:29.040 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking libradosstriper1 (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:11:29.089 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package ceph-base. 2024-09-18T10:11:29.107 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../024-ceph-base_17.2.7-1645-g7061ea6c-1focal_amd64.deb ... 2024-09-18T10:11:29.158 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking ceph-base (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:11:29.261 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-18T10:11:29.269 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-18T10:11:29.277 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../028-ceph-mgr-modules-core_17.2.7-1645-g7061ea6c-1focal_all.deb ... 2024-09-18T10:11:29.286 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../015-libqt5core5a_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-18T10:11:29.319 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking ceph-mgr-modules-core (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:11:29.338 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-18T10:11:29.501 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package ceph-common. 2024-09-18T10:11:29.521 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../023-ceph-common_17.2.7-1645-g7061ea6c-1focal_amd64.deb ... 2024-09-18T10:11:29.559 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking ceph-common (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:11:29.699 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-18T10:11:29.715 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../029-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2024-09-18T10:11:29.757 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2024-09-18T10:11:29.868 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-18T10:11:29.889 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../016-libqt5dbus5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-18T10:11:29.927 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-18T10:11:30.002 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-cryptography. 2024-09-18T10:11:30.022 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../025-python3-cryptography_2.8-3ubuntu0.3_amd64.deb ... 2024-09-18T10:11:30.060 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-18T10:11:30.060 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.3) ... 2024-09-18T10:11:30.078 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../030-python3-cherrypy3_8.9.1-6_all.deb ... 2024-09-18T10:11:30.118 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2024-09-18T10:11:30.317 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-18T10:11:30.333 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../017-libqt5network5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-18T10:11:30.375 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-18T10:11:30.466 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-openssl. 2024-09-18T10:11:30.483 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../026-python3-openssl_19.0.0-1build1_all.deb ... 2024-09-18T10:11:30.488 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-natsort. 2024-09-18T10:11:30.506 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../031-python3-natsort_7.0.1-1_all.deb ... 2024-09-18T10:11:30.546 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2024-09-18T10:11:30.547 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-natsort (7.0.1-1) ... 2024-09-18T10:11:30.781 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package libthrift-0.13.0. 2024-09-18T10:11:30.797 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../018-libthrift-0.13.0_0.13.0-2build2_amd64.deb ... 2024-09-18T10:11:30.833 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-jwt. 2024-09-18T10:11:30.835 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-packaging. 2024-09-18T10:11:30.839 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking libthrift-0.13.0 (0.13.0-2build2) ... 2024-09-18T10:11:30.852 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../027-python3-packaging_20.3-1_all.deb ... 2024-09-18T10:11:30.852 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../032-python3-jwt_1.7.1-2ubuntu2.1_all.deb ... 2024-09-18T10:11:30.891 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2.1) ... 2024-09-18T10:11:30.894 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-packaging (20.3-1) ... 2024-09-18T10:11:31.172 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-18T10:11:31.188 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../028-ceph-mgr-modules-core_17.2.7-1645-g7061ea6c-1focal_all.deb ... 2024-09-18T10:11:31.196 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package librgw2. 2024-09-18T10:11:31.202 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-logutils. 2024-09-18T10:11:31.212 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../019-librgw2_17.2.7-1645-g7061ea6c-1focal_amd64.deb ... 2024-09-18T10:11:31.218 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../033-python3-logutils_0.3.3-7_all.deb ... 2024-09-18T10:11:31.229 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking ceph-mgr-modules-core (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:11:31.255 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking librgw2 (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:11:31.260 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-logutils (0.3.3-7) ... 2024-09-18T10:11:31.580 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-18T10:11:31.585 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-18T10:11:31.600 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../034-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2024-09-18T10:11:31.601 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../029-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2024-09-18T10:11:31.638 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2024-09-18T10:11:31.643 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2024-09-18T10:11:31.941 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-mako. 2024-09-18T10:11:31.946 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-18T10:11:31.958 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../035-python3-mako_1.1.0+ds1-1ubuntu2.1_all.deb ... 2024-09-18T10:11:31.963 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../030-python3-cherrypy3_8.9.1-6_all.deb ... 2024-09-18T10:11:31.999 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2024-09-18T10:11:32.004 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2024-09-18T10:11:32.115 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package ceph-base. 2024-09-18T10:11:32.131 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../024-ceph-base_17.2.7-1645-g7061ea6c-1focal_amd64.deb ... 2024-09-18T10:11:32.183 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking ceph-base (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:11:32.276 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-rgw. 2024-09-18T10:11:32.294 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../020-python3-rgw_17.2.7-1645-g7061ea6c-1focal_amd64.deb ... 2024-09-18T10:11:32.311 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-18T10:11:32.329 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../036-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-18T10:11:32.344 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-rgw (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:11:32.369 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-18T10:11:32.375 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-natsort. 2024-09-18T10:11:32.392 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../031-python3-natsort_7.0.1-1_all.deb ... 2024-09-18T10:11:32.433 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-natsort (7.0.1-1) ... 2024-09-18T10:11:32.697 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-18T10:11:32.707 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package liboath0. 2024-09-18T10:11:32.716 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../037-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2024-09-18T10:11:32.724 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../021-liboath0_2.6.1-1.3_amd64.deb ... 2024-09-18T10:11:32.753 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-jwt. 2024-09-18T10:11:32.755 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2024-09-18T10:11:32.766 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2024-09-18T10:11:32.772 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../032-python3-jwt_1.7.1-2ubuntu2.1_all.deb ... 2024-09-18T10:11:32.811 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2.1) ... 2024-09-18T10:11:33.041 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-webob. 2024-09-18T10:11:33.060 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../038-python3-webob_1%3a1.8.5-2ubuntu0.1_all.deb ... 2024-09-18T10:11:33.077 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-cryptography. 2024-09-18T10:11:33.094 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../025-python3-cryptography_2.8-3ubuntu0.3_amd64.deb ... 2024-09-18T10:11:33.097 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package libradosstriper1. 2024-09-18T10:11:33.099 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-webob (1:1.8.5-2ubuntu0.1) ... 2024-09-18T10:11:33.114 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../022-libradosstriper1_17.2.7-1645-g7061ea6c-1focal_amd64.deb ... 2024-09-18T10:11:33.144 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.3) ... 2024-09-18T10:11:33.156 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking libradosstriper1 (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:11:33.156 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-logutils. 2024-09-18T10:11:33.175 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../033-python3-logutils_0.3.3-7_all.deb ... 2024-09-18T10:11:33.215 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-logutils (0.3.3-7) ... 2024-09-18T10:11:33.427 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-soupsieve. 2024-09-18T10:11:33.443 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../039-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2024-09-18T10:11:33.485 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2024-09-18T10:11:33.493 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-18T10:11:33.510 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../034-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2024-09-18T10:11:33.551 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2024-09-18T10:11:33.564 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-openssl. 2024-09-18T10:11:33.584 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../026-python3-openssl_19.0.0-1build1_all.deb ... 2024-09-18T10:11:33.622 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2024-09-18T10:11:33.720 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package ceph-common. 2024-09-18T10:11:33.738 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../023-ceph-common_17.2.7-1645-g7061ea6c-1focal_amd64.deb ... 2024-09-18T10:11:33.787 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking ceph-common (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:11:33.813 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-bs4. 2024-09-18T10:11:33.829 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../040-python3-bs4_4.8.2-1_all.deb ... 2024-09-18T10:11:33.871 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-mako. 2024-09-18T10:11:33.871 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2024-09-18T10:11:33.890 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../035-python3-mako_1.1.0+ds1-1ubuntu2.1_all.deb ... 2024-09-18T10:11:33.929 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2024-09-18T10:11:33.975 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-packaging. 2024-09-18T10:11:33.991 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../027-python3-packaging_20.3-1_all.deb ... 2024-09-18T10:11:34.033 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-packaging (20.3-1) ... 2024-09-18T10:11:34.249 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-18T10:11:34.265 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../036-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-18T10:11:34.283 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-waitress. 2024-09-18T10:11:34.301 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../041-python3-waitress_1.4.1-1ubuntu0.1_all.deb ... 2024-09-18T10:11:34.311 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-18T10:11:34.327 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../028-ceph-mgr-modules-core_17.2.7-1645-g7061ea6c-1focal_all.deb ... 2024-09-18T10:11:34.332 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-18T10:11:34.344 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-waitress (1.4.1-1ubuntu0.1) ... 2024-09-18T10:11:34.369 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking ceph-mgr-modules-core (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:11:34.619 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-18T10:11:34.638 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../037-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2024-09-18T10:11:34.676 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2024-09-18T10:11:34.694 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-tempita. 2024-09-18T10:11:34.710 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../042-python3-tempita_0.5.2-6_all.deb ... 2024-09-18T10:11:34.752 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-tempita (0.5.2-6) ... 2024-09-18T10:11:34.773 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-18T10:11:34.791 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../029-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2024-09-18T10:11:34.832 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2024-09-18T10:11:34.996 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-webob. 2024-09-18T10:11:35.015 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../038-python3-webob_1%3a1.8.5-2ubuntu0.1_all.deb ... 2024-09-18T10:11:35.052 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-paste. 2024-09-18T10:11:35.054 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-webob (1:1.8.5-2ubuntu0.1) ... 2024-09-18T10:11:35.070 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../043-python3-paste_3.4.0+dfsg1-1_all.deb ... 2024-09-18T10:11:35.111 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2024-09-18T10:11:35.151 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-18T10:11:35.170 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../030-python3-cherrypy3_8.9.1-6_all.deb ... 2024-09-18T10:11:35.209 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2024-09-18T10:11:35.408 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-soupsieve. 2024-09-18T10:11:35.424 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../039-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2024-09-18T10:11:35.431 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-18T10:11:35.449 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../044-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2024-09-18T10:11:35.466 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2024-09-18T10:11:35.489 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2024-09-18T10:11:35.554 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-natsort. 2024-09-18T10:11:35.573 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../031-python3-natsort_7.0.1-1_all.deb ... 2024-09-18T10:11:35.612 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-natsort (7.0.1-1) ... 2024-09-18T10:11:35.810 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-bs4. 2024-09-18T10:11:35.817 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-18T10:11:35.827 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../040-python3-bs4_4.8.2-1_all.deb ... 2024-09-18T10:11:35.832 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../045-python3-pastedeploy_2.1.0-1_all.deb ... 2024-09-18T10:11:35.869 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2024-09-18T10:11:35.875 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2024-09-18T10:11:35.924 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-jwt. 2024-09-18T10:11:35.940 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../032-python3-jwt_1.7.1-2ubuntu2.1_all.deb ... 2024-09-18T10:11:35.998 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2.1) ... 2024-09-18T10:11:36.161 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-webtest. 2024-09-18T10:11:36.181 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../046-python3-webtest_2.0.32-2_all.deb ... 2024-09-18T10:11:36.219 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-webtest (2.0.32-2) ... 2024-09-18T10:11:36.231 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-waitress. 2024-09-18T10:11:36.248 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../041-python3-waitress_1.4.1-1ubuntu0.1_all.deb ... 2024-09-18T10:11:36.268 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-logutils. 2024-09-18T10:11:36.287 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../033-python3-logutils_0.3.3-7_all.deb ... 2024-09-18T10:11:36.292 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-waitress (1.4.1-1ubuntu0.1) ... 2024-09-18T10:11:36.326 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-logutils (0.3.3-7) ... 2024-09-18T10:11:36.483 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package ceph-base. 2024-09-18T10:11:36.499 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../024-ceph-base_17.2.7-1645-g7061ea6c-1focal_amd64.deb ... 2024-09-18T10:11:36.505 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-pecan. 2024-09-18T10:11:36.525 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../047-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2024-09-18T10:11:36.552 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking ceph-base (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:11:36.563 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2024-09-18T10:11:36.596 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-18T10:11:36.611 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../034-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2024-09-18T10:11:36.634 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-tempita. 2024-09-18T10:11:36.651 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../042-python3-tempita_0.5.2-6_all.deb ... 2024-09-18T10:11:36.654 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2024-09-18T10:11:36.692 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-tempita (0.5.2-6) ... 2024-09-18T10:11:36.935 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-18T10:11:36.951 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../048-python3-werkzeug_0.16.1+dfsg1-2ubuntu0.2_all.deb ... 2024-09-18T10:11:36.973 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-mako. 2024-09-18T10:11:36.993 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../035-python3-mako_1.1.0+ds1-1ubuntu2.1_all.deb ... 2024-09-18T10:11:36.993 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2ubuntu0.2) ... 2024-09-18T10:11:37.031 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2024-09-18T10:11:37.322 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-18T10:11:37.341 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../049-libsqlite3-mod-ceph_17.2.7-1645-g7061ea6c-1focal_amd64.deb ... 2024-09-18T10:11:37.376 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-18T10:11:37.380 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libsqlite3-mod-ceph (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:11:37.394 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../036-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-18T10:11:37.434 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-18T10:11:37.527 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-cryptography. 2024-09-18T10:11:37.543 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../025-python3-cryptography_2.8-3ubuntu0.3_amd64.deb ... 2024-09-18T10:11:37.585 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.3) ... 2024-09-18T10:11:37.774 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-18T10:11:37.779 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-paste. 2024-09-18T10:11:37.791 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../037-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2024-09-18T10:11:37.795 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../043-python3-paste_3.4.0+dfsg1-1_all.deb ... 2024-09-18T10:11:37.833 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2024-09-18T10:11:37.837 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2024-09-18T10:11:38.008 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-openssl. 2024-09-18T10:11:38.024 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../026-python3-openssl_19.0.0-1build1_all.deb ... 2024-09-18T10:11:38.068 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2024-09-18T10:11:38.136 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-webob. 2024-09-18T10:11:38.154 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../038-python3-webob_1%3a1.8.5-2ubuntu0.1_all.deb ... 2024-09-18T10:11:38.165 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-18T10:11:38.184 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../044-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2024-09-18T10:11:38.219 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-webob (1:1.8.5-2ubuntu0.1) ... 2024-09-18T10:11:38.224 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2024-09-18T10:11:38.381 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-packaging. 2024-09-18T10:11:38.398 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../027-python3-packaging_20.3-1_all.deb ... 2024-09-18T10:11:38.440 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-packaging (20.3-1) ... 2024-09-18T10:11:38.512 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package ceph-mgr. 2024-09-18T10:11:38.532 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../050-ceph-mgr_17.2.7-1645-g7061ea6c-1focal_amd64.deb ... 2024-09-18T10:11:38.535 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-18T10:11:38.552 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../045-python3-pastedeploy_2.1.0-1_all.deb ... 2024-09-18T10:11:38.570 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking ceph-mgr (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:11:38.593 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2024-09-18T10:11:38.721 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-18T10:11:38.741 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../028-ceph-mgr-modules-core_17.2.7-1645-g7061ea6c-1focal_all.deb ... 2024-09-18T10:11:38.779 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking ceph-mgr-modules-core (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:11:38.897 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-webtest. 2024-09-18T10:11:38.916 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../046-python3-webtest_2.0.32-2_all.deb ... 2024-09-18T10:11:38.955 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-webtest (2.0.32-2) ... 2024-09-18T10:11:39.082 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package ceph-mon. 2024-09-18T10:11:39.099 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../051-ceph-mon_17.2.7-1645-g7061ea6c-1focal_amd64.deb ... 2024-09-18T10:11:39.140 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking ceph-mon (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:11:39.317 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-pecan. 2024-09-18T10:11:39.333 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../047-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2024-09-18T10:11:39.375 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2024-09-18T10:11:39.464 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-soupsieve. 2024-09-18T10:11:39.480 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../039-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2024-09-18T10:11:39.522 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2024-09-18T10:11:39.768 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-18T10:11:39.784 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../048-python3-werkzeug_0.16.1+dfsg1-2ubuntu0.2_all.deb ... 2024-09-18T10:11:39.808 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-bs4. 2024-09-18T10:11:39.824 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../040-python3-bs4_4.8.2-1_all.deb ... 2024-09-18T10:11:39.825 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2ubuntu0.2) ... 2024-09-18T10:11:39.851 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package ceph-osd. 2024-09-18T10:11:39.866 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2024-09-18T10:11:39.869 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../052-ceph-osd_17.2.7-1645-g7061ea6c-1focal_amd64.deb ... 2024-09-18T10:11:39.900 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking ceph-osd (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:11:40.113 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-18T10:11:40.130 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../049-libsqlite3-mod-ceph_17.2.7-1645-g7061ea6c-1focal_amd64.deb ... 2024-09-18T10:11:40.170 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking libsqlite3-mod-ceph (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:11:40.228 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-waitress. 2024-09-18T10:11:40.246 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../041-python3-waitress_1.4.1-1ubuntu0.1_all.deb ... 2024-09-18T10:11:40.280 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-waitress (1.4.1-1ubuntu0.1) ... 2024-09-18T10:11:40.548 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package ceph-mgr. 2024-09-18T10:11:40.564 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../050-ceph-mgr_17.2.7-1645-g7061ea6c-1focal_amd64.deb ... 2024-09-18T10:11:40.606 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking ceph-mgr (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:11:40.622 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-tempita. 2024-09-18T10:11:40.641 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../042-python3-tempita_0.5.2-6_all.deb ... 2024-09-18T10:11:40.680 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-tempita (0.5.2-6) ... 2024-09-18T10:11:40.782 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-18T10:11:40.799 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../029-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2024-09-18T10:11:40.840 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2024-09-18T10:11:40.975 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-paste. 2024-09-18T10:11:40.991 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../043-python3-paste_3.4.0+dfsg1-1_all.deb ... 2024-09-18T10:11:41.050 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2024-09-18T10:11:41.135 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package ceph-mon. 2024-09-18T10:11:41.151 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../051-ceph-mon_17.2.7-1645-g7061ea6c-1focal_amd64.deb ... 2024-09-18T10:11:41.153 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-18T10:11:41.171 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../030-python3-cherrypy3_8.9.1-6_all.deb ... 2024-09-18T10:11:41.193 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking ceph-mon (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:11:41.220 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2024-09-18T10:11:41.370 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-18T10:11:41.389 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../044-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2024-09-18T10:11:41.428 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2024-09-18T10:11:41.569 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-natsort. 2024-09-18T10:11:41.587 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../031-python3-natsort_7.0.1-1_all.deb ... 2024-09-18T10:11:41.636 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-natsort (7.0.1-1) ... 2024-09-18T10:11:41.751 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-18T10:11:41.768 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../045-python3-pastedeploy_2.1.0-1_all.deb ... 2024-09-18T10:11:41.809 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2024-09-18T10:11:41.952 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-jwt. 2024-09-18T10:11:41.968 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../032-python3-jwt_1.7.1-2ubuntu2.1_all.deb ... 2024-09-18T10:11:41.988 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package ceph-osd. 2024-09-18T10:11:42.004 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../052-ceph-osd_17.2.7-1645-g7061ea6c-1focal_amd64.deb ... 2024-09-18T10:11:42.010 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2.1) ... 2024-09-18T10:11:42.038 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking ceph-osd (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:11:42.095 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package ceph. 2024-09-18T10:11:42.113 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../053-ceph_17.2.7-1645-g7061ea6c-1focal_amd64.deb ... 2024-09-18T10:11:42.153 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking ceph (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:11:42.154 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-webtest. 2024-09-18T10:11:42.173 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../046-python3-webtest_2.0.32-2_all.deb ... 2024-09-18T10:11:42.212 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-webtest (2.0.32-2) ... 2024-09-18T10:11:42.334 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-logutils. 2024-09-18T10:11:42.353 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../033-python3-logutils_0.3.3-7_all.deb ... 2024-09-18T10:11:42.401 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-logutils (0.3.3-7) ... 2024-09-18T10:11:42.481 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package ceph-fuse. 2024-09-18T10:11:42.498 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../054-ceph-fuse_17.2.7-1645-g7061ea6c-1focal_amd64.deb ... 2024-09-18T10:11:42.539 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking ceph-fuse (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:11:42.563 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-pecan. 2024-09-18T10:11:42.582 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../047-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2024-09-18T10:11:42.622 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2024-09-18T10:11:42.714 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-18T10:11:42.731 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../034-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2024-09-18T10:11:42.772 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2024-09-18T10:11:42.909 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package ceph-mds. 2024-09-18T10:11:42.929 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../055-ceph-mds_17.2.7-1645-g7061ea6c-1focal_amd64.deb ... 2024-09-18T10:11:42.960 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-18T10:11:42.968 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking ceph-mds (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:11:42.976 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../048-python3-werkzeug_0.16.1+dfsg1-2ubuntu0.2_all.deb ... 2024-09-18T10:11:43.018 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2ubuntu0.2) ... 2024-09-18T10:11:43.096 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-mako. 2024-09-18T10:11:43.112 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../035-python3-mako_1.1.0+ds1-1ubuntu2.1_all.deb ... 2024-09-18T10:11:43.154 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2024-09-18T10:11:43.405 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-18T10:11:43.420 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../049-libsqlite3-mod-ceph_17.2.7-1645-g7061ea6c-1focal_amd64.deb ... 2024-09-18T10:11:43.463 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking libsqlite3-mod-ceph (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:11:43.487 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package cephadm. 2024-09-18T10:11:43.503 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../056-cephadm_17.2.7-1645-g7061ea6c-1focal_amd64.deb ... 2024-09-18T10:11:43.518 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-18T10:11:43.535 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../036-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-18T10:11:43.545 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking cephadm (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:11:43.578 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-18T10:11:43.791 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package ceph-mgr. 2024-09-18T10:11:43.809 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../050-ceph-mgr_17.2.7-1645-g7061ea6c-1focal_amd64.deb ... 2024-09-18T10:11:43.841 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking ceph-mgr (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:11:43.865 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-jinja2. 2024-09-18T10:11:43.882 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../057-python3-jinja2_2.10.1-2ubuntu0.3_all.deb ... 2024-09-18T10:11:43.923 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-jinja2 (2.10.1-2ubuntu0.3) ... 2024-09-18T10:11:43.925 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-18T10:11:43.943 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../037-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2024-09-18T10:11:43.991 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2024-09-18T10:11:44.084 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package ceph. 2024-09-18T10:11:44.102 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../053-ceph_17.2.7-1645-g7061ea6c-1focal_amd64.deb ... 2024-09-18T10:11:44.142 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking ceph (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:11:44.268 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-18T10:11:44.283 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../058-ceph-mgr-cephadm_17.2.7-1645-g7061ea6c-1focal_all.deb ... 2024-09-18T10:11:44.290 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-webob. 2024-09-18T10:11:44.307 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../038-python3-webob_1%3a1.8.5-2ubuntu0.1_all.deb ... 2024-09-18T10:11:44.326 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking ceph-mgr-cephadm (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:11:44.357 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-webob (1:1.8.5-2ubuntu0.1) ... 2024-09-18T10:11:44.360 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package ceph-mon. 2024-09-18T10:11:44.376 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../051-ceph-mon_17.2.7-1645-g7061ea6c-1focal_amd64.deb ... 2024-09-18T10:11:44.418 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking ceph-mon (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:11:44.495 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package ceph-fuse. 2024-09-18T10:11:44.511 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../054-ceph-fuse_17.2.7-1645-g7061ea6c-1focal_amd64.deb ... 2024-09-18T10:11:44.553 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking ceph-fuse (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:11:44.687 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-18T10:11:44.703 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-soupsieve. 2024-09-18T10:11:44.706 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../059-python3-repoze.lru_0.7-2_all.deb ... 2024-09-18T10:11:44.722 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../039-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2024-09-18T10:11:44.745 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-18T10:11:44.770 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2024-09-18T10:11:44.981 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package ceph-mds. 2024-09-18T10:11:44.997 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../055-ceph-mds_17.2.7-1645-g7061ea6c-1focal_amd64.deb ... 2024-09-18T10:11:44.998 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libjs-underscore. 2024-09-18T10:11:45.017 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../060-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2024-09-18T10:11:45.039 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking ceph-mds (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:11:45.056 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2024-09-18T10:11:45.077 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-bs4. 2024-09-18T10:11:45.097 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../040-python3-bs4_4.8.2-1_all.deb ... 2024-09-18T10:11:45.143 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2024-09-18T10:11:45.239 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package ceph-osd. 2024-09-18T10:11:45.256 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../052-ceph-osd_17.2.7-1645-g7061ea6c-1focal_amd64.deb ... 2024-09-18T10:11:45.297 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking ceph-osd (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:11:45.334 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libjs-sphinxdoc. 2024-09-18T10:11:45.351 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../061-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2024-09-18T10:11:45.384 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2024-09-18T10:11:45.501 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-waitress. 2024-09-18T10:11:45.521 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../041-python3-waitress_1.4.1-1ubuntu0.1_all.deb ... 2024-09-18T10:11:45.570 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-waitress (1.4.1-1ubuntu0.1) ... 2024-09-18T10:11:45.584 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package cephadm. 2024-09-18T10:11:45.600 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../056-cephadm_17.2.7-1645-g7061ea6c-1focal_amd64.deb ... 2024-09-18T10:11:45.643 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking cephadm (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:11:45.720 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-routes. 2024-09-18T10:11:45.735 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../062-python3-routes_2.4.1-2_all.deb ... 2024-09-18T10:11:45.778 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-routes (2.4.1-2) ... 2024-09-18T10:11:45.906 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-tempita. 2024-09-18T10:11:45.923 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../042-python3-tempita_0.5.2-6_all.deb ... 2024-09-18T10:11:45.962 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-jinja2. 2024-09-18T10:11:45.965 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-tempita (0.5.2-6) ... 2024-09-18T10:11:45.979 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../057-python3-jinja2_2.10.1-2ubuntu0.3_all.deb ... 2024-09-18T10:11:46.037 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-jinja2 (2.10.1-2ubuntu0.3) ... 2024-09-18T10:11:46.081 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-18T10:11:46.097 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../063-ceph-mgr-dashboard_17.2.7-1645-g7061ea6c-1focal_all.deb ... 2024-09-18T10:11:46.139 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking ceph-mgr-dashboard (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:11:46.340 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-paste. 2024-09-18T10:11:46.357 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-18T10:11:46.358 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../043-python3-paste_3.4.0+dfsg1-1_all.deb ... 2024-09-18T10:11:46.373 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../058-ceph-mgr-cephadm_17.2.7-1645-g7061ea6c-1focal_all.deb ... 2024-09-18T10:11:46.399 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2024-09-18T10:11:46.415 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking ceph-mgr-cephadm (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:11:46.723 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-18T10:11:46.742 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../044-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2024-09-18T10:11:46.781 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2024-09-18T10:11:46.785 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-18T10:11:46.804 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../059-python3-repoze.lru_0.7-2_all.deb ... 2024-09-18T10:11:46.843 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-18T10:11:47.078 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-18T10:11:47.095 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../045-python3-pastedeploy_2.1.0-1_all.deb ... 2024-09-18T10:11:47.096 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package libjs-underscore. 2024-09-18T10:11:47.114 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../060-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2024-09-18T10:11:47.136 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2024-09-18T10:11:47.179 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2024-09-18T10:11:47.435 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-webtest. 2024-09-18T10:11:47.448 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package libjs-sphinxdoc. 2024-09-18T10:11:47.450 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package ceph. 2024-09-18T10:11:47.453 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../046-python3-webtest_2.0.32-2_all.deb ... 2024-09-18T10:11:47.466 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../061-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2024-09-18T10:11:47.467 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../053-ceph_17.2.7-1645-g7061ea6c-1focal_amd64.deb ... 2024-09-18T10:11:47.501 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-webtest (2.0.32-2) ... 2024-09-18T10:11:47.503 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-decorator. 2024-09-18T10:11:47.506 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2024-09-18T10:11:47.508 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking ceph (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:11:47.519 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../064-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2024-09-18T10:11:47.561 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2024-09-18T10:11:47.798 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-pecan. 2024-09-18T10:11:47.817 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../047-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2024-09-18T10:11:47.832 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2024-09-18T10:11:47.844 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package ceph-fuse. 2024-09-18T10:11:47.848 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../065-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2024-09-18T10:11:47.858 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2024-09-18T10:11:47.862 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../054-ceph-fuse_17.2.7-1645-g7061ea6c-1focal_amd64.deb ... 2024-09-18T10:11:47.882 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2024-09-18T10:11:47.884 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-routes. 2024-09-18T10:11:47.902 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking ceph-fuse (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:11:47.903 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../062-python3-routes_2.4.1-2_all.deb ... 2024-09-18T10:11:47.943 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-routes (2.4.1-2) ... 2024-09-18T10:11:48.185 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-scipy. 2024-09-18T10:11:48.202 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../066-python3-scipy_1.3.3-3build1_amd64.deb ... 2024-09-18T10:11:48.210 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-18T10:11:48.227 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../048-python3-werkzeug_0.16.1+dfsg1-2ubuntu0.2_all.deb ... 2024-09-18T10:11:48.243 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2024-09-18T10:11:48.254 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-18T10:11:48.268 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2ubuntu0.2) ... 2024-09-18T10:11:48.274 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../063-ceph-mgr-dashboard_17.2.7-1645-g7061ea6c-1focal_all.deb ... 2024-09-18T10:11:48.312 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking ceph-mgr-dashboard (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:11:48.380 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package ceph-mds. 2024-09-18T10:11:48.400 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../055-ceph-mds_17.2.7-1645-g7061ea6c-1focal_amd64.deb ... 2024-09-18T10:11:48.438 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking ceph-mds (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:11:48.650 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-18T10:11:48.667 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../049-libsqlite3-mod-ceph_17.2.7-1645-g7061ea6c-1focal_amd64.deb ... 2024-09-18T10:11:48.708 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking libsqlite3-mod-ceph (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:11:48.958 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package cephadm. 2024-09-18T10:11:48.974 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../056-cephadm_17.2.7-1645-g7061ea6c-1focal_amd64.deb ... 2024-09-18T10:11:49.016 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking cephadm (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:11:49.115 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package ceph-mgr. 2024-09-18T10:11:49.134 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../050-ceph-mgr_17.2.7-1645-g7061ea6c-1focal_amd64.deb ... 2024-09-18T10:11:49.182 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking ceph-mgr (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:11:49.410 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-jinja2. 2024-09-18T10:11:49.427 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../057-python3-jinja2_2.10.1-2ubuntu0.3_all.deb ... 2024-09-18T10:11:49.468 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-jinja2 (2.10.1-2ubuntu0.3) ... 2024-09-18T10:11:49.587 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-sklearn-lib. 2024-09-18T10:11:49.604 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../067-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2024-09-18T10:11:49.645 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2024-09-18T10:11:49.645 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-decorator. 2024-09-18T10:11:49.661 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../064-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2024-09-18T10:11:49.703 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2024-09-18T10:11:49.746 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package ceph-mon. 2024-09-18T10:11:49.763 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../051-ceph-mon_17.2.7-1645-g7061ea6c-1focal_amd64.deb ... 2024-09-18T10:11:49.806 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking ceph-mon (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:11:49.830 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-18T10:11:49.847 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../058-ceph-mgr-cephadm_17.2.7-1645-g7061ea6c-1focal_all.deb ... 2024-09-18T10:11:49.888 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking ceph-mgr-cephadm (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:11:50.039 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2024-09-18T10:11:50.056 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../065-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2024-09-18T10:11:50.097 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2024-09-18T10:11:50.165 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-joblib. 2024-09-18T10:11:50.185 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../068-python3-joblib_0.14.0-3_all.deb ... 2024-09-18T10:11:50.223 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-joblib (0.14.0-3) ... 2024-09-18T10:11:50.249 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-18T10:11:50.269 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../059-python3-repoze.lru_0.7-2_all.deb ... 2024-09-18T10:11:50.307 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-18T10:11:50.425 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-scipy. 2024-09-18T10:11:50.441 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../066-python3-scipy_1.3.3-3build1_amd64.deb ... 2024-09-18T10:11:50.517 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2024-09-18T10:11:50.585 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package libjs-underscore. 2024-09-18T10:11:50.604 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../060-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2024-09-18T10:11:50.610 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-sklearn. 2024-09-18T10:11:50.627 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../069-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2024-09-18T10:11:50.643 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2024-09-18T10:11:50.668 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2024-09-18T10:11:50.737 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package ceph-osd. 2024-09-18T10:11:50.753 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../052-ceph-osd_17.2.7-1645-g7061ea6c-1focal_amd64.deb ... 2024-09-18T10:11:50.795 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking ceph-osd (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:11:50.945 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package libjs-sphinxdoc. 2024-09-18T10:11:50.965 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../061-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2024-09-18T10:11:50.995 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2024-09-18T10:11:51.291 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-routes. 2024-09-18T10:11:51.309 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../062-python3-routes_2.4.1-2_all.deb ... 2024-09-18T10:11:51.368 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-routes (2.4.1-2) ... 2024-09-18T10:11:51.377 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-18T10:11:51.396 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../070-ceph-mgr-diskprediction-local_17.2.7-1645-g7061ea6c-1focal_all.deb ... 2024-09-18T10:11:51.434 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking ceph-mgr-diskprediction-local (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:11:51.685 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-18T10:11:51.702 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../063-ceph-mgr-dashboard_17.2.7-1645-g7061ea6c-1focal_all.deb ... 2024-09-18T10:11:51.743 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking ceph-mgr-dashboard (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:11:51.776 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-sklearn-lib. 2024-09-18T10:11:51.793 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../067-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2024-09-18T10:11:51.859 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2024-09-18T10:11:52.379 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-joblib. 2024-09-18T10:11:52.396 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../068-python3-joblib_0.14.0-3_all.deb ... 2024-09-18T10:11:52.437 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-joblib (0.14.0-3) ... 2024-09-18T10:11:52.504 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-cachetools. 2024-09-18T10:11:52.522 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../071-python3-cachetools_4.0.0-1_all.deb ... 2024-09-18T10:11:52.562 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2024-09-18T10:11:52.833 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-sklearn. 2024-09-18T10:11:52.851 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../069-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2024-09-18T10:11:52.881 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-18T10:11:52.891 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2024-09-18T10:11:52.900 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../072-python3-pyasn1_0.4.2-3build1_all.deb ... 2024-09-18T10:11:52.939 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2024-09-18T10:11:53.132 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-decorator. 2024-09-18T10:11:53.136 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package ceph. 2024-09-18T10:11:53.148 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../064-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2024-09-18T10:11:53.154 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../053-ceph_17.2.7-1645-g7061ea6c-1focal_amd64.deb ... 2024-09-18T10:11:53.202 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking ceph (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:11:53.223 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2024-09-18T10:11:53.276 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-18T10:11:53.292 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../073-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2024-09-18T10:11:53.334 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2024-09-18T10:11:53.509 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2024-09-18T10:11:53.515 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package ceph-fuse. 2024-09-18T10:11:53.529 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../065-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2024-09-18T10:11:53.534 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../054-ceph-fuse_17.2.7-1645-g7061ea6c-1focal_amd64.deb ... 2024-09-18T10:11:53.567 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2024-09-18T10:11:53.582 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking ceph-fuse (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:11:53.886 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-scipy. 2024-09-18T10:11:53.887 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-rsa. 2024-09-18T10:11:53.903 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../066-python3-scipy_1.3.3-3build1_amd64.deb ... 2024-09-18T10:11:53.905 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../074-python3-rsa_4.0-3_all.deb ... 2024-09-18T10:11:53.944 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2024-09-18T10:11:53.985 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-18T10:11:54.003 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../070-ceph-mgr-diskprediction-local_17.2.7-1645-g7061ea6c-1focal_all.deb ... 2024-09-18T10:11:54.039 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package ceph-mds. 2024-09-18T10:11:54.041 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking ceph-mgr-diskprediction-local (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:11:54.056 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../055-ceph-mds_17.2.7-1645-g7061ea6c-1focal_amd64.deb ... 2024-09-18T10:11:54.069 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-rsa (4.0-3) ... 2024-09-18T10:11:54.097 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking ceph-mds (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:11:54.381 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-google-auth. 2024-09-18T10:11:54.398 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../075-python3-google-auth_1.5.1-2_all.deb ... 2024-09-18T10:11:54.439 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2024-09-18T10:11:54.747 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package cephadm. 2024-09-18T10:11:54.759 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-blinker. 2024-09-18T10:11:54.765 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../056-cephadm_17.2.7-1645-g7061ea6c-1focal_amd64.deb ... 2024-09-18T10:11:54.777 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../076-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2024-09-18T10:11:54.806 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking cephadm (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:11:54.850 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2024-09-18T10:11:55.119 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-jinja2. 2024-09-18T10:11:55.136 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-oauthlib. 2024-09-18T10:11:55.138 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../057-python3-jinja2_2.10.1-2ubuntu0.3_all.deb ... 2024-09-18T10:11:55.156 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../077-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2024-09-18T10:11:55.201 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2024-09-18T10:11:55.202 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-jinja2 (2.10.1-2ubuntu0.3) ... 2024-09-18T10:11:55.216 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-cachetools. 2024-09-18T10:11:55.233 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../071-python3-cachetools_4.0.0-1_all.deb ... 2024-09-18T10:11:55.277 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2024-09-18T10:11:55.345 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-sklearn-lib. 2024-09-18T10:11:55.363 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../067-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2024-09-18T10:11:55.403 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2024-09-18T10:11:55.533 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-18T10:11:55.534 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-18T10:11:55.550 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../078-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2024-09-18T10:11:55.551 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../058-ceph-mgr-cephadm_17.2.7-1645-g7061ea6c-1focal_all.deb ... 2024-09-18T10:11:55.580 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-18T10:11:55.591 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2024-09-18T10:11:55.592 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking ceph-mgr-cephadm (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:11:55.596 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../072-python3-pyasn1_0.4.2-3build1_all.deb ... 2024-09-18T10:11:55.638 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2024-09-18T10:11:55.919 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-websocket. 2024-09-18T10:11:55.935 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../079-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2024-09-18T10:11:55.965 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-joblib. 2024-09-18T10:11:55.976 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2024-09-18T10:11:55.981 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../068-python3-joblib_0.14.0-3_all.deb ... 2024-09-18T10:11:56.017 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-18T10:11:56.023 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-joblib (0.14.0-3) ... 2024-09-18T10:11:56.035 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../073-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2024-09-18T10:11:56.075 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2024-09-18T10:11:56.272 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-18T10:11:56.290 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../080-python3-kubernetes_7.0.0~a1-2_all.deb ... 2024-09-18T10:11:56.329 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2024-09-18T10:11:56.387 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-rsa. 2024-09-18T10:11:56.401 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-18T10:11:56.404 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../074-python3-rsa_4.0-3_all.deb ... 2024-09-18T10:11:56.410 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-sklearn. 2024-09-18T10:11:56.420 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../059-python3-repoze.lru_0.7-2_all.deb ... 2024-09-18T10:11:56.427 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../069-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2024-09-18T10:11:56.444 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-rsa (4.0-3) ... 2024-09-18T10:11:56.459 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-18T10:11:56.467 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2024-09-18T10:11:56.722 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package libjs-underscore. 2024-09-18T10:11:56.739 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../060-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2024-09-18T10:11:56.766 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-google-auth. 2024-09-18T10:11:56.780 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2024-09-18T10:11:56.782 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../075-python3-google-auth_1.5.1-2_all.deb ... 2024-09-18T10:11:56.824 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2024-09-18T10:11:56.962 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-18T10:11:56.979 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../081-ceph-mgr-k8sevents_17.2.7-1645-g7061ea6c-1focal_all.deb ... 2024-09-18T10:11:57.020 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking ceph-mgr-k8sevents (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:11:57.094 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package libjs-sphinxdoc. 2024-09-18T10:11:57.111 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../061-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2024-09-18T10:11:57.126 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-18T10:11:57.144 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-blinker. 2024-09-18T10:11:57.145 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../070-ceph-mgr-diskprediction-local_17.2.7-1645-g7061ea6c-1focal_all.deb ... 2024-09-18T10:11:57.152 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2024-09-18T10:11:57.163 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../076-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2024-09-18T10:11:57.183 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking ceph-mgr-diskprediction-local (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:11:57.227 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2024-09-18T10:11:57.340 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-18T10:11:57.356 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../082-libonig5_6.9.4-1_amd64.deb ... 2024-09-18T10:11:57.398 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2024-09-18T10:11:57.532 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-oauthlib. 2024-09-18T10:11:57.549 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-routes. 2024-09-18T10:11:57.550 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../077-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2024-09-18T10:11:57.569 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../062-python3-routes_2.4.1-2_all.deb ... 2024-09-18T10:11:57.588 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2024-09-18T10:11:57.616 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-routes (2.4.1-2) ... 2024-09-18T10:11:57.746 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-18T10:11:57.763 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../083-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2024-09-18T10:11:57.805 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2024-09-18T10:11:57.951 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-18T10:11:57.956 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-18T10:11:57.968 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../078-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2024-09-18T10:11:57.974 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../063-ceph-mgr-dashboard_17.2.7-1645-g7061ea6c-1focal_all.deb ... 2024-09-18T10:11:58.009 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2024-09-18T10:11:58.022 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking ceph-mgr-dashboard (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:11:58.099 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package jq. 2024-09-18T10:11:58.116 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../084-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2024-09-18T10:11:58.157 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2024-09-18T10:11:58.320 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-cachetools. 2024-09-18T10:11:58.320 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-websocket. 2024-09-18T10:11:58.336 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../071-python3-cachetools_4.0.0-1_all.deb ... 2024-09-18T10:11:58.339 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../079-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2024-09-18T10:11:58.377 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2024-09-18T10:11:58.378 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2024-09-18T10:11:58.410 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package socat. 2024-09-18T10:11:58.427 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../085-socat_1.7.3.3-2_amd64.deb ... 2024-09-18T10:11:58.468 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking socat (1.7.3.3-2) ... 2024-09-18T10:11:58.700 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-18T10:11:58.715 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-18T10:11:58.719 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../072-python3-pyasn1_0.4.2-3build1_all.deb ... 2024-09-18T10:11:58.732 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../080-python3-kubernetes_7.0.0~a1-2_all.deb ... 2024-09-18T10:11:58.749 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2024-09-18T10:11:58.773 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2024-09-18T10:11:58.788 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package xmlstarlet. 2024-09-18T10:11:58.806 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../086-xmlstarlet_1.6.1-2build1_amd64.deb ... 2024-09-18T10:11:58.846 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2024-09-18T10:11:59.094 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-18T10:11:59.111 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../073-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2024-09-18T10:11:59.152 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2024-09-18T10:11:59.210 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package ceph-test. 2024-09-18T10:11:59.227 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../087-ceph-test_17.2.7-1645-g7061ea6c-1focal_amd64.deb ... 2024-09-18T10:11:59.268 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking ceph-test (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:11:59.401 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-18T10:11:59.417 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../081-ceph-mgr-k8sevents_17.2.7-1645-g7061ea6c-1focal_all.deb ... 2024-09-18T10:11:59.459 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking ceph-mgr-k8sevents (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:11:59.475 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-rsa. 2024-09-18T10:11:59.494 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../074-python3-rsa_4.0-3_all.deb ... 2024-09-18T10:11:59.679 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-rsa (4.0-3) ... 2024-09-18T10:11:59.709 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-decorator. 2024-09-18T10:11:59.725 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../064-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2024-09-18T10:11:59.746 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-18T10:11:59.762 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../082-libonig5_6.9.4-1_amd64.deb ... 2024-09-18T10:11:59.767 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2024-09-18T10:11:59.804 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2024-09-18T10:12:00.082 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2024-09-18T10:12:00.099 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../065-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2024-09-18T10:12:00.149 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2024-09-18T10:12:00.150 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-google-auth. 2024-09-18T10:12:00.169 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../075-python3-google-auth_1.5.1-2_all.deb ... 2024-09-18T10:12:00.199 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-18T10:12:00.207 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2024-09-18T10:12:00.216 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../083-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2024-09-18T10:12:00.257 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2024-09-18T10:12:00.535 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package jq. 2024-09-18T10:12:00.537 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-scipy. 2024-09-18T10:12:00.553 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../084-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2024-09-18T10:12:00.553 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-blinker. 2024-09-18T10:12:00.554 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../066-python3-scipy_1.3.3-3build1_amd64.deb ... 2024-09-18T10:12:00.573 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../076-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2024-09-18T10:12:00.595 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2024-09-18T10:12:00.611 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2024-09-18T10:12:00.618 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2024-09-18T10:12:00.855 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package socat. 2024-09-18T10:12:00.872 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../085-socat_1.7.3.3-2_amd64.deb ... 2024-09-18T10:12:00.905 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-oauthlib. 2024-09-18T10:12:00.913 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking socat (1.7.3.3-2) ... 2024-09-18T10:12:00.925 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../077-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2024-09-18T10:12:00.963 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2024-09-18T10:12:01.250 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package xmlstarlet. 2024-09-18T10:12:01.333 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-18T10:12:01.333 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../078-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2024-09-18T10:12:01.334 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../086-xmlstarlet_1.6.1-2build1_amd64.deb ... 2024-09-18T10:12:01.334 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2024-09-18T10:12:01.351 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2024-09-18T10:12:01.660 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package ceph-test. 2024-09-18T10:12:01.664 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-websocket. 2024-09-18T10:12:01.679 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../087-ceph-test_17.2.7-1645-g7061ea6c-1focal_amd64.deb ... 2024-09-18T10:12:01.680 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../079-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2024-09-18T10:12:01.718 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking ceph-test (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:01.748 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2024-09-18T10:12:01.952 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-sklearn-lib. 2024-09-18T10:12:01.969 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../067-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2024-09-18T10:12:02.011 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2024-09-18T10:12:02.059 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-18T10:12:02.075 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../080-python3-kubernetes_7.0.0~a1-2_all.deb ... 2024-09-18T10:12:02.117 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2024-09-18T10:12:02.592 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-joblib. 2024-09-18T10:12:02.609 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../068-python3-joblib_0.14.0-3_all.deb ... 2024-09-18T10:12:02.653 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-joblib (0.14.0-3) ... 2024-09-18T10:12:02.758 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-18T10:12:02.775 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../081-ceph-mgr-k8sevents_17.2.7-1645-g7061ea6c-1focal_all.deb ... 2024-09-18T10:12:02.816 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking ceph-mgr-k8sevents (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:03.050 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-sklearn. 2024-09-18T10:12:03.068 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../069-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2024-09-18T10:12:03.108 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2024-09-18T10:12:03.144 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-18T10:12:03.162 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../082-libonig5_6.9.4-1_amd64.deb ... 2024-09-18T10:12:03.202 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2024-09-18T10:12:03.547 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-18T10:12:03.566 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../083-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2024-09-18T10:12:03.596 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2024-09-18T10:12:03.747 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package ceph-volume. 2024-09-18T10:12:03.763 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../088-ceph-volume_17.2.7-1645-g7061ea6c-1focal_all.deb ... 2024-09-18T10:12:03.805 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking ceph-volume (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:03.809 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-18T10:12:03.827 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../070-ceph-mgr-diskprediction-local_17.2.7-1645-g7061ea6c-1focal_all.deb ... 2024-09-18T10:12:03.868 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking ceph-mgr-diskprediction-local (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:03.881 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package jq. 2024-09-18T10:12:03.895 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../084-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2024-09-18T10:12:03.960 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2024-09-18T10:12:04.122 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package fonts-mathjax. 2024-09-18T10:12:04.140 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../089-fonts-mathjax_2.7.4+dfsg-1_all.deb ... 2024-09-18T10:12:04.180 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking fonts-mathjax (2.7.4+dfsg-1) ... 2024-09-18T10:12:04.213 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package socat. 2024-09-18T10:12:04.231 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../085-socat_1.7.3.3-2_amd64.deb ... 2024-09-18T10:12:04.271 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking socat (1.7.3.3-2) ... 2024-09-18T10:12:04.627 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package xmlstarlet. 2024-09-18T10:12:04.644 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../086-xmlstarlet_1.6.1-2build1_amd64.deb ... 2024-09-18T10:12:04.685 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2024-09-18T10:12:04.734 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-18T10:12:04.751 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../090-libcephfs-dev_17.2.7-1645-g7061ea6c-1focal_amd64.deb ... 2024-09-18T10:12:04.792 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libcephfs-dev (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:05.073 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package ceph-test. 2024-09-18T10:12:05.090 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../087-ceph-test_17.2.7-1645-g7061ea6c-1focal_amd64.deb ... 2024-09-18T10:12:05.131 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking ceph-test (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:05.164 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package nvme-cli. 2024-09-18T10:12:05.185 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../091-nvme-cli_1.9-1ubuntu0.2_amd64.deb ... 2024-09-18T10:12:05.240 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking nvme-cli (1.9-1ubuntu0.2) ... 2024-09-18T10:12:05.475 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-cachetools. 2024-09-18T10:12:05.492 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../071-python3-cachetools_4.0.0-1_all.deb ... 2024-09-18T10:12:05.533 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2024-09-18T10:12:05.585 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libjs-mathjax. 2024-09-18T10:12:05.602 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../092-libjs-mathjax_2.7.4+dfsg-1_all.deb ... 2024-09-18T10:12:05.643 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libjs-mathjax (2.7.4+dfsg-1) ... 2024-09-18T10:12:05.879 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-18T10:12:05.896 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../072-python3-pyasn1_0.4.2-3build1_all.deb ... 2024-09-18T10:12:05.979 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2024-09-18T10:12:06.312 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-18T10:12:06.328 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../073-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2024-09-18T10:12:06.370 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2024-09-18T10:12:06.407 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package ceph-volume. 2024-09-18T10:12:06.424 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../088-ceph-volume_17.2.7-1645-g7061ea6c-1focal_all.deb ... 2024-09-18T10:12:06.463 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking ceph-volume (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:06.692 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-rsa. 2024-09-18T10:12:06.708 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../074-python3-rsa_4.0-3_all.deb ... 2024-09-18T10:12:06.743 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-rsa (4.0-3) ... 2024-09-18T10:12:06.811 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package fonts-mathjax. 2024-09-18T10:12:06.828 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../089-fonts-mathjax_2.7.4+dfsg-1_all.deb ... 2024-09-18T10:12:06.869 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking fonts-mathjax (2.7.4+dfsg-1) ... 2024-09-18T10:12:07.099 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-google-auth. 2024-09-18T10:12:07.118 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../075-python3-google-auth_1.5.1-2_all.deb ... 2024-09-18T10:12:07.161 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python-natsort-doc. 2024-09-18T10:12:07.165 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2024-09-18T10:12:07.180 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../093-python-natsort-doc_7.0.1-1_all.deb ... 2024-09-18T10:12:07.219 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python-natsort-doc (7.0.1-1) ... 2024-09-18T10:12:07.406 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-18T10:12:07.422 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../090-libcephfs-dev_17.2.7-1645-g7061ea6c-1focal_amd64.deb ... 2024-09-18T10:12:07.464 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking libcephfs-dev (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:07.498 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-blinker. 2024-09-18T10:12:07.518 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../076-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2024-09-18T10:12:07.547 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-atomicwrites. 2024-09-18T10:12:07.556 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2024-09-18T10:12:07.564 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../094-python3-atomicwrites_1.1.5-2build1_all.deb ... 2024-09-18T10:12:07.630 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2024-09-18T10:12:07.809 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package nvme-cli. 2024-09-18T10:12:07.827 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../091-nvme-cli_1.9-1ubuntu0.2_amd64.deb ... 2024-09-18T10:12:07.876 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking nvme-cli (1.9-1ubuntu0.2) ... 2024-09-18T10:12:07.894 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-oauthlib. 2024-09-18T10:12:07.911 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../077-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2024-09-18T10:12:07.916 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-attr. 2024-09-18T10:12:07.937 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../095-python3-attr_19.3.0-2_all.deb ... 2024-09-18T10:12:07.952 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2024-09-18T10:12:07.974 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-attr (19.3.0-2) ... 2024-09-18T10:12:08.213 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package libjs-mathjax. 2024-09-18T10:12:08.230 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../092-libjs-mathjax_2.7.4+dfsg-1_all.deb ... 2024-09-18T10:12:08.270 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-18T10:12:08.271 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking libjs-mathjax (2.7.4+dfsg-1) ... 2024-09-18T10:12:08.290 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../078-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2024-09-18T10:12:08.319 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-webencodings. 2024-09-18T10:12:08.328 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2024-09-18T10:12:08.338 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../096-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2024-09-18T10:12:08.377 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2024-09-18T10:12:08.643 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-websocket. 2024-09-18T10:12:08.660 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../079-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2024-09-18T10:12:08.705 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-html5lib. 2024-09-18T10:12:08.709 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2024-09-18T10:12:08.723 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../097-python3-html5lib_1.0.1-2_all.deb ... 2024-09-18T10:12:08.763 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2024-09-18T10:12:09.083 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-18T10:12:09.099 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-lxml:amd64. 2024-09-18T10:12:09.102 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../080-python3-kubernetes_7.0.0~a1-2_all.deb ... 2024-09-18T10:12:09.116 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../098-python3-lxml_4.5.0-1ubuntu0.5_amd64.deb ... 2024-09-18T10:12:09.141 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2024-09-18T10:12:09.157 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2024-09-18T10:12:09.611 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-pastescript. 2024-09-18T10:12:09.627 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../099-python3-pastescript_2.0.2-4_all.deb ... 2024-09-18T10:12:09.668 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-18T10:12:09.735 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package ceph-volume. 2024-09-18T10:12:09.751 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../088-ceph-volume_17.2.7-1645-g7061ea6c-1focal_all.deb ... 2024-09-18T10:12:09.774 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-18T10:12:09.792 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../081-ceph-mgr-k8sevents_17.2.7-1645-g7061ea6c-1focal_all.deb ... 2024-09-18T10:12:09.793 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking ceph-volume (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:09.832 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking ceph-mgr-k8sevents (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:09.946 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python-natsort-doc. 2024-09-18T10:12:09.963 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../093-python-natsort-doc_7.0.1-1_all.deb ... 2024-09-18T10:12:10.020 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python-natsort-doc (7.0.1-1) ... 2024-09-18T10:12:10.131 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package fonts-mathjax. 2024-09-18T10:12:10.148 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../089-fonts-mathjax_2.7.4+dfsg-1_all.deb ... 2024-09-18T10:12:10.155 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-18T10:12:10.174 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../082-libonig5_6.9.4-1_amd64.deb ... 2024-09-18T10:12:10.180 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-pluggy. 2024-09-18T10:12:10.189 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking fonts-mathjax (2.7.4+dfsg-1) ... 2024-09-18T10:12:10.197 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../100-python3-pluggy_0.13.0-2_all.deb ... 2024-09-18T10:12:10.214 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2024-09-18T10:12:10.238 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2024-09-18T10:12:10.365 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-atomicwrites. 2024-09-18T10:12:10.384 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../094-python3-atomicwrites_1.1.5-2build1_all.deb ... 2024-09-18T10:12:10.424 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2024-09-18T10:12:10.566 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-psutil. 2024-09-18T10:12:10.586 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../101-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2024-09-18T10:12:10.594 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-18T10:12:10.612 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../083-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2024-09-18T10:12:10.624 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2024-09-18T10:12:10.661 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2024-09-18T10:12:10.709 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-18T10:12:10.727 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../090-libcephfs-dev_17.2.7-1645-g7061ea6c-1focal_amd64.deb ... 2024-09-18T10:12:10.742 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-attr. 2024-09-18T10:12:10.760 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../095-python3-attr_19.3.0-2_all.deb ... 2024-09-18T10:12:10.767 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking libcephfs-dev (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:10.800 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-attr (19.3.0-2) ... 2024-09-18T10:12:10.951 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package jq. 2024-09-18T10:12:10.968 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../084-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2024-09-18T10:12:10.969 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-py. 2024-09-18T10:12:10.989 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../102-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2024-09-18T10:12:11.009 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2024-09-18T10:12:11.027 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2024-09-18T10:12:11.111 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-webencodings. 2024-09-18T10:12:11.131 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../096-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2024-09-18T10:12:11.170 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2024-09-18T10:12:11.179 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package nvme-cli. 2024-09-18T10:12:11.195 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../091-nvme-cli_1.9-1ubuntu0.2_amd64.deb ... 2024-09-18T10:12:11.266 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package socat. 2024-09-18T10:12:11.279 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking nvme-cli (1.9-1ubuntu0.2) ... 2024-09-18T10:12:11.286 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../085-socat_1.7.3.3-2_amd64.deb ... 2024-09-18T10:12:11.324 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking socat (1.7.3.3-2) ... 2024-09-18T10:12:11.347 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-18T10:12:11.364 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../103-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2024-09-18T10:12:11.405 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2024-09-18T10:12:11.490 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-html5lib. 2024-09-18T10:12:11.508 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../097-python3-html5lib_1.0.1-2_all.deb ... 2024-09-18T10:12:11.548 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2024-09-18T10:12:11.608 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package libjs-mathjax. 2024-09-18T10:12:11.629 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../092-libjs-mathjax_2.7.4+dfsg-1_all.deb ... 2024-09-18T10:12:11.667 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking libjs-mathjax (2.7.4+dfsg-1) ... 2024-09-18T10:12:11.681 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package xmlstarlet. 2024-09-18T10:12:11.697 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../086-xmlstarlet_1.6.1-2build1_amd64.deb ... 2024-09-18T10:12:11.716 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-18T10:12:11.737 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../104-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2024-09-18T10:12:11.739 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2024-09-18T10:12:11.790 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2024-09-18T10:12:11.858 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-lxml:amd64. 2024-09-18T10:12:11.879 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../098-python3-lxml_4.5.0-1ubuntu0.5_amd64.deb ... 2024-09-18T10:12:11.916 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2024-09-18T10:12:12.101 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package ceph-test. 2024-09-18T10:12:12.118 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-pytest. 2024-09-18T10:12:12.119 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../087-ceph-test_17.2.7-1645-g7061ea6c-1focal_amd64.deb ... 2024-09-18T10:12:12.136 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../105-python3-pytest_4.6.9-1_all.deb ... 2024-09-18T10:12:12.168 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking ceph-test (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:12.176 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-pytest (4.6.9-1) ... 2024-09-18T10:12:12.387 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-pastescript. 2024-09-18T10:12:12.405 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../099-python3-pastescript_2.0.2-4_all.deb ... 2024-09-18T10:12:12.444 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-18T10:12:12.530 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-simplejson. 2024-09-18T10:12:12.547 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../106-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2024-09-18T10:12:12.588 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2024-09-18T10:12:12.798 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-pluggy. 2024-09-18T10:12:12.815 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../100-python3-pluggy_0.13.0-2_all.deb ... 2024-09-18T10:12:12.856 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2024-09-18T10:12:12.899 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-18T10:12:12.918 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../107-qttranslations5-l10n_5.12.8-0ubuntu1_all.deb ... 2024-09-18T10:12:12.957 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking qttranslations5-l10n (5.12.8-0ubuntu1) ... 2024-09-18T10:12:13.150 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-psutil. 2024-09-18T10:12:13.171 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../101-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2024-09-18T10:12:13.208 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2024-09-18T10:12:13.357 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python-natsort-doc. 2024-09-18T10:12:13.375 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../093-python-natsort-doc_7.0.1-1_all.deb ... 2024-09-18T10:12:13.415 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python-natsort-doc (7.0.1-1) ... 2024-09-18T10:12:13.569 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-py. 2024-09-18T10:12:13.586 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../102-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2024-09-18T10:12:13.636 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2024-09-18T10:12:13.678 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package radosgw. 2024-09-18T10:12:13.695 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../108-radosgw_17.2.7-1645-g7061ea6c-1focal_amd64.deb ... 2024-09-18T10:12:13.736 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking radosgw (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:13.768 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-atomicwrites. 2024-09-18T10:12:13.785 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../094-python3-atomicwrites_1.1.5-2build1_all.deb ... 2024-09-18T10:12:13.826 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2024-09-18T10:12:13.972 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-18T10:12:13.990 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../103-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2024-09-18T10:12:14.047 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2024-09-18T10:12:14.146 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-attr. 2024-09-18T10:12:14.164 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../095-python3-attr_19.3.0-2_all.deb ... 2024-09-18T10:12:14.195 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-attr (19.3.0-2) ... 2024-09-18T10:12:14.358 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-18T10:12:14.376 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../104-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2024-09-18T10:12:14.416 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2024-09-18T10:12:14.548 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-webencodings. 2024-09-18T10:12:14.567 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../096-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2024-09-18T10:12:14.606 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2024-09-18T10:12:14.719 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-pytest. 2024-09-18T10:12:14.737 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../105-python3-pytest_4.6.9-1_all.deb ... 2024-09-18T10:12:14.777 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-pytest (4.6.9-1) ... 2024-09-18T10:12:14.909 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-html5lib. 2024-09-18T10:12:14.925 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../097-python3-html5lib_1.0.1-2_all.deb ... 2024-09-18T10:12:14.999 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2024-09-18T10:12:15.114 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-simplejson. 2024-09-18T10:12:15.131 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../106-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2024-09-18T10:12:15.172 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2024-09-18T10:12:15.353 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-lxml:amd64. 2024-09-18T10:12:15.371 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../098-python3-lxml_4.5.0-1ubuntu0.5_amd64.deb ... 2024-09-18T10:12:15.411 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2024-09-18T10:12:15.452 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-18T10:12:15.456 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package rbd-fuse. 2024-09-18T10:12:15.471 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../107-qttranslations5-l10n_5.12.8-0ubuntu1_all.deb ... 2024-09-18T10:12:15.473 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../109-rbd-fuse_17.2.7-1645-g7061ea6c-1focal_amd64.deb ... 2024-09-18T10:12:15.510 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking qttranslations5-l10n (5.12.8-0ubuntu1) ... 2024-09-18T10:12:15.514 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking rbd-fuse (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:15.838 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-attr (19.3.0-2) ... 2024-09-18T10:12:15.906 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-pastescript. 2024-09-18T10:12:15.926 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../099-python3-pastescript_2.0.2-4_all.deb ... 2024-09-18T10:12:15.964 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-18T10:12:16.138 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2024-09-18T10:12:16.264 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package radosgw. 2024-09-18T10:12:16.265 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2024-09-18T10:12:16.281 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../108-radosgw_17.2.7-1645-g7061ea6c-1focal_amd64.deb ... 2024-09-18T10:12:16.301 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-pluggy. 2024-09-18T10:12:16.317 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../100-python3-pluggy_0.13.0-2_all.deb ... 2024-09-18T10:12:16.330 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking radosgw (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:16.376 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2024-09-18T10:12:16.390 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up nvme-cli (1.9-1ubuntu0.2) ... 2024-09-18T10:12:16.592 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up cephadm (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:16.662 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-psutil. 2024-09-18T10:12:16.681 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../101-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2024-09-18T10:12:16.720 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2024-09-18T10:12:17.089 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-py. 2024-09-18T10:12:17.106 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../102-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2024-09-18T10:12:17.139 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2024-09-18T10:12:17.168 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package ceph-volume. 2024-09-18T10:12:17.187 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../088-ceph-volume_17.2.7-1645-g7061ea6c-1focal_all.deb ... 2024-09-18T10:12:17.205 INFO:teuthology.orchestra.run.smithi154.stdout:Adding system user cephadm....done 2024-09-18T10:12:17.269 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking ceph-volume (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:17.367 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up fonts-mathjax (2.7.4+dfsg-1) ... 2024-09-18T10:12:17.475 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-18T10:12:17.492 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../103-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2024-09-18T10:12:17.504 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-waitress (1.4.1-1ubuntu0.1) ... 2024-09-18T10:12:17.534 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2024-09-18T10:12:17.622 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package fonts-mathjax. 2024-09-18T10:12:17.639 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../089-fonts-mathjax_2.7.4+dfsg-1_all.deb ... 2024-09-18T10:12:17.680 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking fonts-mathjax (2.7.4+dfsg-1) ... 2024-09-18T10:12:17.855 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-18T10:12:17.864 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-18T10:12:17.881 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../104-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2024-09-18T10:12:17.922 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2024-09-18T10:12:18.092 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package rbd-fuse. 2024-09-18T10:12:18.112 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../109-rbd-fuse_17.2.7-1645-g7061ea6c-1focal_amd64.deb ... 2024-09-18T10:12:18.150 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking rbd-fuse (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:18.159 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-18T10:12:18.173 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libjs-mathjax (2.7.4+dfsg-1) ... 2024-09-18T10:12:18.176 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../090-libcephfs-dev_17.2.7-1645-g7061ea6c-1focal_amd64.deb ... 2024-09-18T10:12:18.219 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking libcephfs-dev (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:18.267 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-pytest. 2024-09-18T10:12:18.284 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../105-python3-pytest_4.6.9-1_all.deb ... 2024-09-18T10:12:18.299 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up liboath0 (2.6.1-1.3) ... 2024-09-18T10:12:18.324 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-pytest (4.6.9-1) ... 2024-09-18T10:12:18.425 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2024-09-18T10:12:18.444 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-attr (19.3.0-2) ... 2024-09-18T10:12:18.643 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package nvme-cli. 2024-09-18T10:12:18.663 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../091-nvme-cli_1.9-1ubuntu0.2_amd64.deb ... 2024-09-18T10:12:18.687 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-simplejson. 2024-09-18T10:12:18.706 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../106-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2024-09-18T10:12:18.728 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking nvme-cli (1.9-1ubuntu0.2) ... 2024-09-18T10:12:18.744 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2024-09-18T10:12:18.752 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-joblib (0.14.0-3) ... 2024-09-18T10:12:18.764 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2024-09-18T10:12:18.881 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2024-09-18T10:12:19.009 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up nvme-cli (1.9-1ubuntu0.2) ... 2024-09-18T10:12:19.022 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-18T10:12:19.039 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../107-qttranslations5-l10n_5.12.8-0ubuntu1_all.deb ... 2024-09-18T10:12:19.054 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package libjs-mathjax. 2024-09-18T10:12:19.072 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../092-libjs-mathjax_2.7.4+dfsg-1_all.deb ... 2024-09-18T10:12:19.088 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking qttranslations5-l10n (5.12.8-0ubuntu1) ... 2024-09-18T10:12:19.112 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking libjs-mathjax (2.7.4+dfsg-1) ... 2024-09-18T10:12:19.202 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up cephadm (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:19.212 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-jwt (1.7.1-2ubuntu2.1) ... 2024-09-18T10:12:19.505 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-cachetools (4.0.0-1) ... 2024-09-18T10:12:19.783 INFO:teuthology.orchestra.run.smithi136.stdout:Adding system user cephadm....done 2024-09-18T10:12:19.790 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2024-09-18T10:12:19.842 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package radosgw. 2024-09-18T10:12:19.860 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../108-radosgw_17.2.7-1645-g7061ea6c-1focal_amd64.deb ... 2024-09-18T10:12:19.909 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking radosgw (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:19.916 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up fonts-mathjax (2.7.4+dfsg-1) ... 2024-09-18T10:12:20.064 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-waitress (1.4.1-1ubuntu0.1) ... 2024-09-18T10:12:20.084 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2024-09-18T10:12:20.368 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-ceph-argparse (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:20.418 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-18T10:12:20.678 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2024-09-18T10:12:20.733 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up libjs-mathjax (2.7.4+dfsg-1) ... 2024-09-18T10:12:20.804 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2024-09-18T10:12:20.863 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up liboath0 (2.6.1-1.3) ... 2024-09-18T10:12:20.899 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python-natsort-doc. 2024-09-18T10:12:20.918 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../093-python-natsort-doc_7.0.1-1_all.deb ... 2024-09-18T10:12:20.957 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python-natsort-doc (7.0.1-1) ... 2024-09-18T10:12:20.973 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2024-09-18T10:12:21.096 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2024-09-18T10:12:21.289 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-atomicwrites. 2024-09-18T10:12:21.299 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-joblib (0.14.0-3) ... 2024-09-18T10:12:21.310 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../094-python3-atomicwrites_1.1.5-2build1_all.deb ... 2024-09-18T10:12:21.356 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2024-09-18T10:12:21.381 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2024-09-18T10:12:21.507 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2024-09-18T10:12:21.687 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-attr. 2024-09-18T10:12:21.706 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../095-python3-attr_19.3.0-2_all.deb ... 2024-09-18T10:12:21.714 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package rbd-fuse. 2024-09-18T10:12:21.720 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-jwt (1.7.1-2ubuntu2.1) ... 2024-09-18T10:12:21.732 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../109-rbd-fuse_17.2.7-1645-g7061ea6c-1focal_amd64.deb ... 2024-09-18T10:12:21.745 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-attr (19.3.0-2) ... 2024-09-18T10:12:21.772 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking rbd-fuse (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:21.891 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-natsort (7.0.1-1) ... 2024-09-18T10:12:22.002 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-cachetools (4.0.0-1) ... 2024-09-18T10:12:22.059 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-webencodings. 2024-09-18T10:12:22.079 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../096-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2024-09-18T10:12:22.107 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-attr (19.3.0-2) ... 2024-09-18T10:12:22.125 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2024-09-18T10:12:22.193 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2024-09-18T10:12:22.303 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2024-09-18T10:12:22.386 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2024-09-18T10:12:22.457 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-html5lib. 2024-09-18T10:12:22.478 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../097-python3-html5lib_1.0.1-2_all.deb ... 2024-09-18T10:12:22.485 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2024-09-18T10:12:22.495 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2024-09-18T10:12:22.523 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2024-09-18T10:12:22.579 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2024-09-18T10:12:22.621 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up nvme-cli (1.9-1ubuntu0.2) ... 2024-09-18T10:12:22.820 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2024-09-18T10:12:22.847 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up cephadm (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:22.872 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-ceph-argparse (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:22.904 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-lxml:amd64. 2024-09-18T10:12:22.923 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../098-python3-lxml_4.5.0-1ubuntu0.5_amd64.deb ... 2024-09-18T10:12:22.963 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2024-09-18T10:12:23.104 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-jinja2 (2.10.1-2ubuntu0.3) ... 2024-09-18T10:12:23.165 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2024-09-18T10:12:23.290 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2024-09-18T10:12:23.443 INFO:teuthology.orchestra.run.smithi189.stdout:Adding system user cephadm....done 2024-09-18T10:12:23.453 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-pastescript. 2024-09-18T10:12:23.464 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-tempita (0.5.2-6) ... 2024-09-18T10:12:23.474 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../099-python3-pastescript_2.0.2-4_all.deb ... 2024-09-18T10:12:23.519 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-18T10:12:23.575 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2024-09-18T10:12:23.604 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up fonts-mathjax (2.7.4+dfsg-1) ... 2024-09-18T10:12:23.748 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-packaging (20.3-1) ... 2024-09-18T10:12:23.771 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-waitress (1.4.1-1ubuntu0.1) ... 2024-09-18T10:12:23.818 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-pluggy. 2024-09-18T10:12:23.837 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../100-python3-pluggy_0.13.0-2_all.deb ... 2024-09-18T10:12:23.885 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2024-09-18T10:12:24.041 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2024-09-18T10:12:24.059 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2024-09-18T10:12:24.165 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-18T10:12:24.167 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up qttranslations5-l10n (5.12.8-0ubuntu1) ... 2024-09-18T10:12:24.201 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2024-09-18T10:12:24.232 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-psutil. 2024-09-18T10:12:24.252 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../101-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2024-09-18T10:12:24.292 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2024-09-18T10:12:24.293 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2024-09-18T10:12:24.449 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up libjs-mathjax (2.7.4+dfsg-1) ... 2024-09-18T10:12:24.577 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2024-09-18T10:12:24.578 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-natsort (7.0.1-1) ... 2024-09-18T10:12:24.608 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up liboath0 (2.6.1-1.3) ... 2024-09-18T10:12:24.672 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-py. 2024-09-18T10:12:24.692 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../102-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2024-09-18T10:12:24.726 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2024-09-18T10:12:24.738 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2024-09-18T10:12:24.862 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2024-09-18T10:12:25.061 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-joblib (0.14.0-3) ... 2024-09-18T10:12:25.079 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-18T10:12:25.097 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../103-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2024-09-18T10:12:25.137 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2024-09-18T10:12:25.146 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2024-09-18T10:12:25.379 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2ubuntu0.2) ... 2024-09-18T10:12:25.459 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-18T10:12:25.477 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../104-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2024-09-18T10:12:25.506 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2024-09-18T10:12:25.525 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2024-09-18T10:12:25.588 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-jwt (1.7.1-2ubuntu2.1) ... 2024-09-18T10:12:25.739 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-cryptography (2.8-3ubuntu0.3) ... 2024-09-18T10:12:25.790 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-jinja2 (2.10.1-2ubuntu0.3) ... 2024-09-18T10:12:25.889 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-cachetools (4.0.0-1) ... 2024-09-18T10:12:25.890 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-pytest. 2024-09-18T10:12:25.908 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../105-python3-pytest_4.6.9-1_all.deb ... 2024-09-18T10:12:25.948 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-pytest (4.6.9-1) ... 2024-09-18T10:12:26.099 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up socat (1.7.3.3-2) ... 2024-09-18T10:12:26.141 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-tempita (0.5.2-6) ... 2024-09-18T10:12:26.207 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2024-09-18T10:12:26.225 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-ceph-common (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:26.315 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package python3-simplejson. 2024-09-18T10:12:26.332 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../106-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2024-09-18T10:12:26.381 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2024-09-18T10:12:26.425 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-packaging (20.3-1) ... 2024-09-18T10:12:26.500 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2024-09-18T10:12:26.534 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-18T10:12:26.694 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-html5lib (1.0.1-2) ... 2024-09-18T10:12:26.702 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2024-09-18T10:12:26.708 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-18T10:12:26.727 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../107-qttranslations5-l10n_5.12.8-0ubuntu1_all.deb ... 2024-09-18T10:12:26.818 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-ceph-argparse (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:26.853 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up qttranslations5-l10n (5.12.8-0ubuntu1) ... 2024-09-18T10:12:26.978 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2024-09-18T10:12:27.062 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2024-09-18T10:12:27.135 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2024-09-18T10:12:27.188 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2024-09-18T10:12:27.236 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2024-09-18T10:12:27.263 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2024-09-18T10:12:27.314 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-pluggy (0.13.0-2) ... 2024-09-18T10:12:27.329 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking qttranslations5-l10n (5.12.8-0ubuntu1) ... 2024-09-18T10:12:27.496 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2024-09-18T10:12:27.599 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2024-09-18T10:12:27.664 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2ubuntu0.2) ... 2024-09-18T10:12:27.830 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2024-09-18T10:12:27.917 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-18T10:12:27.956 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2024-09-18T10:12:28.016 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-cryptography (2.8-3ubuntu0.3) ... 2024-09-18T10:12:28.018 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2024-09-18T10:12:28.086 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package radosgw. 2024-09-18T10:12:28.104 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../108-radosgw_17.2.7-1645-g7061ea6c-1focal_amd64.deb ... 2024-09-18T10:12:28.144 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2024-09-18T10:12:28.144 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking radosgw (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:28.340 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-natsort (7.0.1-1) ... 2024-09-18T10:12:28.376 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up socat (1.7.3.3-2) ... 2024-09-18T10:12:28.485 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-ceph-common (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:28.487 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2024-09-18T10:12:28.641 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2024-09-18T10:12:28.780 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-logutils (0.3.3-7) ... 2024-09-18T10:12:28.887 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-18T10:12:28.936 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2024-09-18T10:12:29.090 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2024-09-18T10:12:29.137 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-html5lib (1.0.1-2) ... 2024-09-18T10:12:29.216 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-18T10:12:29.279 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2024-09-18T10:12:29.475 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-prettytable (0.7.2-5) ... 2024-09-18T10:12:29.497 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2024-09-18T10:12:29.605 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-jinja2 (2.10.1-2ubuntu0.3) ... 2024-09-18T10:12:29.631 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2024-09-18T10:12:29.734 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2024-09-18T10:12:29.757 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-pluggy (0.13.0-2) ... 2024-09-18T10:12:29.931 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-tempita (0.5.2-6) ... 2024-09-18T10:12:30.066 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2024-09-18T10:12:30.102 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2024-09-18T10:12:30.202 INFO:teuthology.orchestra.run.smithi029.stdout:Selecting previously unselected package rbd-fuse. 2024-09-18T10:12:30.223 INFO:teuthology.orchestra.run.smithi029.stdout:Preparing to unpack .../109-rbd-fuse_17.2.7-1645-g7061ea6c-1focal_amd64.deb ... 2024-09-18T10:12:30.249 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-packaging (20.3-1) ... 2024-09-18T10:12:30.272 INFO:teuthology.orchestra.run.smithi029.stdout:Unpacking rbd-fuse (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:30.392 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-18T10:12:30.518 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2024-09-18T10:12:30.550 INFO:teuthology.orchestra.run.smithi154.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2024-09-18T10:12:30.608 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-attr (19.3.0-2) ... 2024-09-18T10:12:30.644 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2024-09-18T10:12:30.789 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2024-09-18T10:12:30.884 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2024-09-18T10:12:30.920 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2024-09-18T10:12:30.940 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2024-09-18T10:12:30.971 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2024-09-18T10:12:31.035 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up qttranslations5-l10n (5.12.8-0ubuntu1) ... 2024-09-18T10:12:31.047 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2024-09-18T10:12:31.144 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2024-09-18T10:12:31.199 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up nvme-cli (1.9-1ubuntu0.2) ... 2024-09-18T10:12:31.247 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-logutils (0.3.3-7) ... 2024-09-18T10:12:31.274 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2024-09-18T10:12:31.437 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up cephadm (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:31.437 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2024-09-18T10:12:31.548 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2024-09-18T10:12:31.609 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2024-09-18T10:12:31.673 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-18T10:12:31.718 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2024-09-18T10:12:31.853 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-webob (1:1.8.5-2ubuntu0.1) ... 2024-09-18T10:12:31.864 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2ubuntu0.2) ... 2024-09-18T10:12:31.961 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-prettytable (0.7.2-5) ... 2024-09-18T10:12:32.054 INFO:teuthology.orchestra.run.smithi029.stdout:Adding system user cephadm....done 2024-09-18T10:12:32.204 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2024-09-18T10:12:32.217 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up fonts-mathjax (2.7.4+dfsg-1) ... 2024-09-18T10:12:32.232 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-cryptography (2.8-3ubuntu0.3) ... 2024-09-18T10:12:32.242 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2024-09-18T10:12:32.348 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-waitress (1.4.1-1ubuntu0.1) ... 2024-09-18T10:12:32.497 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up librados2 (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:32.535 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2024-09-18T10:12:32.623 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2024-09-18T10:12:32.634 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up socat (1.7.3.3-2) ... 2024-09-18T10:12:32.678 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-18T10:12:32.793 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-ceph-common (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:32.907 INFO:teuthology.orchestra.run.smithi136.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2024-09-18T10:12:32.963 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up libjs-mathjax (2.7.4+dfsg-1) ... 2024-09-18T10:12:33.091 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up liboath0 (2.6.1-1.3) ... 2024-09-18T10:12:33.111 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-18T10:12:33.179 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2024-09-18T10:12:33.210 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2024-09-18T10:12:33.262 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-html5lib (1.0.1-2) ... 2024-09-18T10:12:33.305 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2024-09-18T10:12:33.546 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-joblib (0.14.0-3) ... 2024-09-18T10:12:33.623 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2024-09-18T10:12:33.630 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2024-09-18T10:12:33.757 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2024-09-18T10:12:33.882 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-pluggy (0.13.0-2) ... 2024-09-18T10:12:33.952 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-jwt (1.7.1-2ubuntu2.1) ... 2024-09-18T10:12:33.974 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2024-09-18T10:12:34.095 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libsqlite3-mod-ceph (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:34.099 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2024-09-18T10:12:34.175 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2024-09-18T10:12:34.221 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2024-09-18T10:12:34.246 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-cachetools (4.0.0-1) ... 2024-09-18T10:12:34.250 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-webob (1:1.8.5-2ubuntu0.1) ... 2024-09-18T10:12:34.347 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-pytest (4.6.9-1) ... 2024-09-18T10:12:34.527 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-18T10:12:34.540 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2024-09-18T10:12:34.585 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2024-09-18T10:12:34.661 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2024-09-18T10:12:34.724 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2024-09-18T10:12:34.787 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2024-09-18T10:12:34.821 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2024-09-18T10:12:34.886 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up librados2 (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:34.992 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2024-09-18T10:12:35.012 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2024-09-18T10:12:35.097 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2024-09-18T10:12:35.115 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-ceph-argparse (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:35.293 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-18T10:12:35.373 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-logutils (0.3.3-7) ... 2024-09-18T10:12:35.411 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-bs4 (4.8.2-1) ... 2024-09-18T10:12:35.417 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2024-09-18T10:12:35.544 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2024-09-18T10:12:35.691 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2024-09-18T10:12:35.754 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libcephfs2 (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:35.817 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-18T10:12:35.830 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2024-09-18T10:12:35.880 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2024-09-18T10:12:36.109 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-prettytable (0.7.2-5) ... 2024-09-18T10:12:36.116 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2024-09-18T10:12:36.207 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libradosstriper1 (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:36.243 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2024-09-18T10:12:36.325 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2024-09-18T10:12:36.394 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2024-09-18T10:12:36.469 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up libsqlite3-mod-ceph (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:36.595 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2024-09-18T10:12:36.637 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-natsort (7.0.1-1) ... 2024-09-18T10:12:36.642 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2024-09-18T10:12:36.687 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2024-09-18T10:12:36.721 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-pytest (4.6.9-1) ... 2024-09-18T10:12:36.982 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2024-09-18T10:12:37.098 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2024-09-18T10:12:37.120 INFO:teuthology.orchestra.run.smithi189.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2024-09-18T10:12:37.267 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2024-09-18T10:12:37.348 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2024-09-18T10:12:37.391 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2024-09-18T10:12:37.474 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2024-09-18T10:12:37.611 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2024-09-18T10:12:37.665 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up librbd1 (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:37.675 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-18T10:12:37.783 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2024-09-18T10:12:37.791 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2024-09-18T10:12:37.801 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-bs4 (4.8.2-1) ... 2024-09-18T10:12:37.892 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python-natsort-doc (7.0.1-1) ... 2024-09-18T10:12:37.905 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-jinja2 (2.10.1-2ubuntu0.3) ... 2024-09-18T10:12:38.018 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2024-09-18T10:12:38.126 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2024-09-18T10:12:38.127 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up libcephfs2 (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:38.144 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-rsa (4.0-3) ... 2024-09-18T10:12:38.236 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2024-09-18T10:12:38.242 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-tempita (0.5.2-6) ... 2024-09-18T10:12:38.278 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2024-09-18T10:12:38.363 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-webob (1:1.8.5-2ubuntu0.1) ... 2024-09-18T10:12:38.487 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libthrift-0.13.0 (0.13.0-2build2) ... 2024-09-18T10:12:38.536 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-packaging (20.3-1) ... 2024-09-18T10:12:38.605 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up libradosstriper1 (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:38.613 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-webtest (2.0.32-2) ... 2024-09-18T10:12:38.680 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2024-09-18T10:12:38.723 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2024-09-18T10:12:38.863 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2024-09-18T10:12:38.914 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up ceph-mgr-modules-core (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:38.989 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up librados2 (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:38.997 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up qttranslations5-l10n (5.12.8-0ubuntu1) ... 2024-09-18T10:12:39.032 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2024-09-18T10:12:39.040 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up ceph-fuse (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:39.118 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2024-09-18T10:12:39.123 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2024-09-18T10:12:39.259 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-18T10:12:39.259 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-18T10:12:39.411 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2024-09-18T10:12:39.716 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libcephfs-dev (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:39.864 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2ubuntu0.2) ... 2024-09-18T10:12:39.928 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2024-09-18T10:12:40.063 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up librbd1 (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:40.189 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2024-09-18T10:12:40.235 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-cryptography (2.8-3ubuntu0.3) ... 2024-09-18T10:12:40.237 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-18T10:12:40.331 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python-natsort-doc (7.0.1-1) ... 2024-09-18T10:12:40.457 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2024-09-18T10:12:40.556 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2024-09-18T10:12:40.580 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up libsqlite3-mod-ceph (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:40.583 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-rsa (4.0-3) ... 2024-09-18T10:12:40.614 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up socat (1.7.3.3-2) ... 2024-09-18T10:12:40.705 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2024-09-18T10:12:40.740 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-ceph-common (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:40.857 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-pytest (4.6.9-1) ... 2024-09-18T10:12:40.868 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up libthrift-0.13.0 (0.13.0-2build2) ... 2024-09-18T10:12:40.908 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-rados (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:41.019 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-webtest (2.0.32-2) ... 2024-09-18T10:12:41.034 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up librgw2 (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:41.070 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-18T10:12:41.160 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-rbd (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:41.230 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-html5lib (1.0.1-2) ... 2024-09-18T10:12:41.258 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2024-09-18T10:12:41.287 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up rbd-fuse (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:41.320 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up ceph-mgr-modules-core (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:41.438 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-routes (2.4.1-2) ... 2024-09-18T10:12:41.446 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up ceph-fuse (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:41.551 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2024-09-18T10:12:41.574 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2024-09-18T10:12:41.672 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-18T10:12:41.672 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-18T10:12:41.701 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2024-09-18T10:12:41.739 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-google-auth (1.5.1-2) ... 2024-09-18T10:12:41.828 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-18T10:12:41.829 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-pluggy (0.13.0-2) ... 2024-09-18T10:12:41.945 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-bs4 (4.8.2-1) ... 2024-09-18T10:12:42.041 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-rgw (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:42.123 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2024-09-18T10:12:42.131 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up libcephfs-dev (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:42.166 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-cephfs (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:42.288 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up libcephfs2 (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:42.292 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2024-09-18T10:12:42.359 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2024-09-18T10:12:42.414 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2024-09-18T10:12:42.442 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-18T10:12:42.638 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2024-09-18T10:12:42.660 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-18T10:12:42.733 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up libradosstriper1 (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:42.764 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2024-09-18T10:12:42.859 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2024-09-18T10:12:42.987 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2024-09-18T10:12:43.084 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2024-09-18T10:12:43.177 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2024-09-18T10:12:43.221 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up ceph-common (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:43.348 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-rados (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:43.377 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-logutils (0.3.3-7) ... 2024-09-18T10:12:43.474 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up librgw2 (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:43.625 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-rbd (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:43.671 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2024-09-18T10:12:43.751 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up rbd-fuse (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:43.790 INFO:teuthology.orchestra.run.smithi154.stdout:Adding group ceph....done 2024-09-18T10:12:43.816 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-18T10:12:43.877 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-routes (2.4.1-2) ... 2024-09-18T10:12:44.102 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-prettytable (0.7.2-5) ... 2024-09-18T10:12:44.170 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-google-auth (1.5.1-2) ... 2024-09-18T10:12:44.175 INFO:teuthology.orchestra.run.smithi154.stdout:Adding system user ceph....done 2024-09-18T10:12:44.199 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up librbd1 (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:44.325 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2024-09-18T10:12:44.353 INFO:teuthology.orchestra.run.smithi154.stdout:Setting system user ceph properties....done 2024-09-18T10:12:44.377 INFO:teuthology.orchestra.run.smithi154.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-18T10:12:44.404 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2024-09-18T10:12:44.468 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python-natsort-doc (7.0.1-1) ... 2024-09-18T10:12:44.488 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-rgw (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:44.532 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-18T10:12:44.586 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2024-09-18T10:12:44.631 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-cephfs (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:44.695 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-rsa (4.0-3) ... 2024-09-18T10:12:44.722 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2024-09-18T10:12:44.749 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2024-09-18T10:12:44.864 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-18T10:12:44.988 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up libthrift-0.13.0 (0.13.0-2build2) ... 2024-09-18T10:12:45.114 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-webtest (2.0.32-2) ... 2024-09-18T10:12:45.130 INFO:teuthology.orchestra.run.smithi029.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2024-09-18T10:12:45.314 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up ceph-test (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:45.415 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up ceph-mgr-modules-core (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:45.429 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2024-09-18T10:12:45.449 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up radosgw (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:45.541 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up ceph-fuse (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:45.572 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2024-09-18T10:12:45.652 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up ceph-common (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:45.721 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-18T10:12:45.721 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-18T10:12:45.770 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-18T10:12:45.770 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-18T10:12:45.908 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2024-09-18T10:12:46.187 INFO:teuthology.orchestra.run.smithi136.stdout:Adding group ceph....done 2024-09-18T10:12:46.192 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up libcephfs-dev (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:46.270 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2024-09-18T10:12:46.350 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up ceph-base (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:46.396 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2024-09-18T10:12:46.405 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2024-09-18T10:12:46.549 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-webob (1:1.8.5-2ubuntu0.1) ... 2024-09-18T10:12:46.606 INFO:teuthology.orchestra.run.smithi136.stdout:Adding system user ceph....done 2024-09-18T10:12:46.689 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-18T10:12:46.727 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-18T10:12:46.791 INFO:teuthology.orchestra.run.smithi136.stdout:Setting system user ceph properties....done 2024-09-18T10:12:46.816 INFO:teuthology.orchestra.run.smithi136.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-18T10:12:46.893 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2024-09-18T10:12:46.962 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-18T10:12:46.998 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2024-09-18T10:12:47.169 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up ceph-mds (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:47.179 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up librados2 (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:47.272 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-18T10:12:47.306 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2024-09-18T10:12:47.376 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-rados (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:47.401 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-18T10:12:47.402 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-18T10:12:47.527 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up librgw2 (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:47.653 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-rbd (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:47.720 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up ceph-test (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:47.779 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up rbd-fuse (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:47.846 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up radosgw (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:47.854 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up ceph-mgr (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:47.905 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-routes (2.4.1-2) ... 2024-09-18T10:12:48.088 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-18T10:12:48.088 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-18T10:12:48.114 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-18T10:12:48.115 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-18T10:12:48.231 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-google-auth (1.5.1-2) ... 2024-09-18T10:12:48.530 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up ceph-osd (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:48.541 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-rgw (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:48.642 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-cephfs (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:48.743 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2024-09-18T10:12:48.765 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up ceph-base (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:48.860 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up libsqlite3-mod-ceph (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:48.987 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2024-09-18T10:12:49.003 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-18T10:12:49.003 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-18T10:12:49.113 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-18T10:12:49.115 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-pytest (4.6.9-1) ... 2024-09-18T10:12:49.440 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up ceph-mgr-k8sevents (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:49.526 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2024-09-18T10:12:49.566 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up ceph-mgr-diskprediction-local (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:49.634 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up ceph-mds (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:49.688 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up ceph-common (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:49.717 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up ceph-mon (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:49.837 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2024-09-18T10:12:49.888 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-18T10:12:49.888 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-18T10:12:49.944 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-18T10:12:49.945 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-18T10:12:50.132 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-18T10:12:50.283 INFO:teuthology.orchestra.run.smithi189.stdout:Adding group ceph....done 2024-09-18T10:12:50.284 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-bs4 (4.8.2-1) ... 2024-09-18T10:12:50.344 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up ceph-mgr (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:50.402 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up ceph-mgr-cephadm (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:50.573 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-18T10:12:50.573 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-18T10:12:50.628 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up libcephfs2 (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:50.679 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up ceph (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:50.685 INFO:teuthology.orchestra.run.smithi189.stdout:Adding system user ceph....done 2024-09-18T10:12:50.755 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2024-09-18T10:12:50.805 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up ceph-mgr-dashboard (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:50.861 INFO:teuthology.orchestra.run.smithi189.stdout:Setting system user ceph properties....done 2024-09-18T10:12:50.886 INFO:teuthology.orchestra.run.smithi189.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-18T10:12:50.965 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up ceph-volume (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:51.037 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up ceph-osd (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:51.037 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-18T10:12:51.092 INFO:teuthology.orchestra.run.smithi154.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2024-09-18T10:12:51.093 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up libradosstriper1 (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:51.220 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2024-09-18T10:12:51.243 INFO:teuthology.orchestra.run.smithi154.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2024-09-18T10:12:51.341 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-18T10:12:51.528 INFO:teuthology.orchestra.run.smithi154.stdout:Processing triggers for man-db (2.9.1-1) ... 2024-09-18T10:12:51.528 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-18T10:12:51.528 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-18T10:12:51.556 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2024-09-18T10:12:51.766 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up ceph-test (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:51.891 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up radosgw (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:51.997 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up ceph-mgr-k8sevents (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:52.122 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up ceph-mgr-diskprediction-local (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:52.197 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-18T10:12:52.197 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-18T10:12:52.265 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up ceph-mon (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:52.530 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-18T10:12:52.531 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-18T10:12:52.651 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up librbd1 (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:52.787 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2024-09-18T10:12:52.802 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up ceph-base (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:52.914 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python-natsort-doc (7.0.1-1) ... 2024-09-18T10:12:53.016 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up ceph-mgr-cephadm (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:53.049 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2024-09-18T10:12:53.159 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up ceph (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:53.201 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-rsa (4.0-3) ... 2024-09-18T10:12:53.214 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-18T10:12:53.285 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up ceph-mgr-dashboard (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:53.420 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up ceph-volume (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:53.513 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up libthrift-0.13.0 (0.13.0-2build2) ... 2024-09-18T10:12:53.547 INFO:teuthology.orchestra.run.smithi136.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2024-09-18T10:12:53.620 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up ceph-mds (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:53.640 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-webtest (2.0.32-2) ... 2024-09-18T10:12:53.723 INFO:teuthology.orchestra.run.smithi136.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2024-09-18T10:12:53.841 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-18T10:12:53.841 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-18T10:12:53.950 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up ceph-mgr-modules-core (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:54.025 INFO:teuthology.orchestra.run.smithi136.stdout:Processing triggers for man-db (2.9.1-1) ... 2024-09-18T10:12:54.094 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up ceph-fuse (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:54.264 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up ceph-mgr (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:54.324 INFO:teuthology.orchestra.run.smithi029.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-18T10:12:54.324 INFO:teuthology.orchestra.run.smithi029.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-18T10:12:54.355 INFO:teuthology.orchestra.run.smithi154.stdout:Processing triggers for fontconfig (2.13.1-2ubuntu3) ... 2024-09-18T10:12:54.493 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-18T10:12:54.493 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-18T10:12:54.740 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up libcephfs-dev (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:54.906 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up ceph-osd (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:54.954 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2024-09-18T10:12:55.015 INFO:teuthology.orchestra.run.smithi154.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-18T10:12:55.022 DEBUG:teuthology.parallel:result is None 2024-09-18T10:12:55.241 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-18T10:12:55.387 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-18T10:12:55.387 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-18T10:12:55.561 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2024-09-18T10:12:55.808 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up ceph-mgr-k8sevents (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:55.925 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-rados (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:55.934 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up ceph-mgr-diskprediction-local (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:56.044 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up librgw2 (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:56.102 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up ceph-mon (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:56.170 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-rbd (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:56.289 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up rbd-fuse (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:56.317 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-18T10:12:56.317 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-18T10:12:56.416 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-routes (2.4.1-2) ... 2024-09-18T10:12:56.720 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-google-auth (1.5.1-2) ... 2024-09-18T10:12:56.737 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up ceph-mgr-cephadm (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:56.823 INFO:teuthology.orchestra.run.smithi136.stdout:Processing triggers for fontconfig (2.13.1-2ubuntu3) ... 2024-09-18T10:12:56.880 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up ceph (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:57.023 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up ceph-mgr-dashboard (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:57.031 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-rgw (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:57.157 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up ceph-volume (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:57.158 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-cephfs (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:57.276 INFO:teuthology.orchestra.run.smithi189.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2024-09-18T10:12:57.285 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2024-09-18T10:12:57.452 INFO:teuthology.orchestra.run.smithi136.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-18T10:12:57.452 INFO:teuthology.orchestra.run.smithi189.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2024-09-18T10:12:57.456 DEBUG:teuthology.parallel:result is None 2024-09-18T10:12:57.729 INFO:teuthology.orchestra.run.smithi189.stdout:Processing triggers for man-db (2.9.1-1) ... 2024-09-18T10:12:58.329 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up ceph-common (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:12:58.877 INFO:teuthology.orchestra.run.smithi029.stdout:Adding group ceph....done 2024-09-18T10:12:59.316 INFO:teuthology.orchestra.run.smithi029.stdout:Adding system user ceph....done 2024-09-18T10:12:59.560 INFO:teuthology.orchestra.run.smithi029.stdout:Setting system user ceph properties....done 2024-09-18T10:12:59.582 INFO:teuthology.orchestra.run.smithi029.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-18T10:12:59.741 INFO:teuthology.orchestra.run.smithi029.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-18T10:13:00.027 INFO:teuthology.orchestra.run.smithi029.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-18T10:13:00.573 INFO:teuthology.orchestra.run.smithi189.stdout:Processing triggers for fontconfig (2.13.1-2ubuntu3) ... 2024-09-18T10:13:01.202 INFO:teuthology.orchestra.run.smithi189.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-18T10:13:01.207 DEBUG:teuthology.parallel:result is None 2024-09-18T10:13:01.243 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up ceph-test (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:13:01.370 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up radosgw (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:13:01.649 INFO:teuthology.orchestra.run.smithi029.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-18T10:13:01.649 INFO:teuthology.orchestra.run.smithi029.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-18T10:13:02.324 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up ceph-base (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:13:02.718 INFO:teuthology.orchestra.run.smithi029.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-18T10:13:03.137 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up ceph-mds (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:13:03.381 INFO:teuthology.orchestra.run.smithi029.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-18T10:13:03.382 INFO:teuthology.orchestra.run.smithi029.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-18T10:13:03.807 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up ceph-mgr (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:13:04.045 INFO:teuthology.orchestra.run.smithi029.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-18T10:13:04.045 INFO:teuthology.orchestra.run.smithi029.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-18T10:13:04.486 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up ceph-osd (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:13:04.981 INFO:teuthology.orchestra.run.smithi029.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-18T10:13:04.981 INFO:teuthology.orchestra.run.smithi029.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-18T10:13:05.406 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up ceph-mgr-k8sevents (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:13:05.532 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up ceph-mgr-diskprediction-local (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:13:05.684 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up ceph-mon (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:13:05.931 INFO:teuthology.orchestra.run.smithi029.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-18T10:13:05.932 INFO:teuthology.orchestra.run.smithi029.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-18T10:13:06.380 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up ceph-mgr-cephadm (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:13:06.532 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up ceph (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:13:06.659 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up ceph-mgr-dashboard (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:13:06.802 INFO:teuthology.orchestra.run.smithi029.stdout:Setting up ceph-volume (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:13:06.932 INFO:teuthology.orchestra.run.smithi029.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2024-09-18T10:13:07.101 INFO:teuthology.orchestra.run.smithi029.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2024-09-18T10:13:07.370 INFO:teuthology.orchestra.run.smithi029.stdout:Processing triggers for man-db (2.9.1-1) ... 2024-09-18T10:13:10.270 INFO:teuthology.orchestra.run.smithi029.stdout:Processing triggers for fontconfig (2.13.1-2ubuntu3) ... 2024-09-18T10:13:10.882 INFO:teuthology.orchestra.run.smithi029.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-18T10:13:10.886 DEBUG:teuthology.parallel:result is None 2024-09-18T10:13:10.887 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=7061ea6ca50908d104e9f3979dc615e118d9e153 2024-09-18T10:13:11.034 DEBUG:teuthology.orchestra.run.smithi029:> dpkg-query -W -f '${Version}' ceph 2024-09-18T10:13:11.057 INFO:teuthology.orchestra.run.smithi029.stdout:17.2.7-1645-g7061ea6c-1focal 2024-09-18T10:13:11.058 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-1645-g7061ea6c-1focal 2024-09-18T10:13:11.058 INFO:teuthology.task.install:The correct ceph version 17.2.7-1645-g7061ea6c-1focal is installed. 2024-09-18T10:13:11.060 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=7061ea6ca50908d104e9f3979dc615e118d9e153 2024-09-18T10:13:11.204 DEBUG:teuthology.orchestra.run.smithi136:> dpkg-query -W -f '${Version}' ceph 2024-09-18T10:13:11.226 INFO:teuthology.orchestra.run.smithi136.stdout:17.2.7-1645-g7061ea6c-1focal 2024-09-18T10:13:11.227 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-1645-g7061ea6c-1focal 2024-09-18T10:13:11.227 INFO:teuthology.task.install:The correct ceph version 17.2.7-1645-g7061ea6c-1focal is installed. 2024-09-18T10:13:11.228 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=7061ea6ca50908d104e9f3979dc615e118d9e153 2024-09-18T10:13:11.368 DEBUG:teuthology.orchestra.run.smithi154:> dpkg-query -W -f '${Version}' ceph 2024-09-18T10:13:11.392 INFO:teuthology.orchestra.run.smithi154.stdout:17.2.7-1645-g7061ea6c-1focal 2024-09-18T10:13:11.392 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-1645-g7061ea6c-1focal 2024-09-18T10:13:11.392 INFO:teuthology.task.install:The correct ceph version 17.2.7-1645-g7061ea6c-1focal is installed. 2024-09-18T10:13:11.394 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=7061ea6ca50908d104e9f3979dc615e118d9e153 2024-09-18T10:13:11.536 DEBUG:teuthology.orchestra.run.smithi189:> dpkg-query -W -f '${Version}' ceph 2024-09-18T10:13:11.560 INFO:teuthology.orchestra.run.smithi189.stdout:17.2.7-1645-g7061ea6c-1focal 2024-09-18T10:13:11.561 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-1645-g7061ea6c-1focal 2024-09-18T10:13:11.561 INFO:teuthology.task.install:The correct ceph version 17.2.7-1645-g7061ea6c-1focal is installed. 2024-09-18T10:13:11.563 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-18T10:13:11.563 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-09-18T10:13:11.563 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-18T10:13:11.580 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-09-18T10:13:11.580 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-18T10:13:11.595 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-09-18T10:13:11.595 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-18T10:13:11.612 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-18T10:13:11.612 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-18T10:13:11.629 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-18T10:13:11.629 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-09-18T10:13:11.629 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd of=/usr/bin/daemon-helper 2024-09-18T10:13:11.644 DEBUG:teuthology.orchestra.run.smithi029:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-18T10:13:11.660 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-09-18T10:13:11.660 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/usr/bin/daemon-helper 2024-09-18T10:13:11.676 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-18T10:13:11.734 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-09-18T10:13:11.735 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/usr/bin/daemon-helper 2024-09-18T10:13:11.751 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-18T10:13:11.765 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-18T10:13:11.765 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/usr/bin/daemon-helper 2024-09-18T10:13:11.782 DEBUG:teuthology.orchestra.run.smithi189:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-18T10:13:11.791 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-18T10:13:11.791 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-09-18T10:13:11.791 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-18T10:13:11.807 DEBUG:teuthology.orchestra.run.smithi029:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-18T10:13:11.818 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-09-18T10:13:11.819 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-18T10:13:11.835 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-18T10:13:11.846 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-09-18T10:13:11.846 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-18T10:13:11.861 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-18T10:13:11.918 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-18T10:13:11.918 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-18T10:13:11.934 DEBUG:teuthology.orchestra.run.smithi189:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-18T10:13:11.990 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-18T10:13:11.990 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-09-18T10:13:11.990 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd of=/usr/bin/stdin-killer 2024-09-18T10:13:12.006 DEBUG:teuthology.orchestra.run.smithi029:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-18T10:13:12.064 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-09-18T10:13:12.064 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/usr/bin/stdin-killer 2024-09-18T10:13:12.080 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-18T10:13:12.089 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-09-18T10:13:12.089 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/usr/bin/stdin-killer 2024-09-18T10:13:12.105 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-18T10:13:12.116 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-18T10:13:12.116 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/usr/bin/stdin-killer 2024-09-18T10:13:12.133 DEBUG:teuthology.orchestra.run.smithi189:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-18T10:13:12.147 INFO:teuthology.run_tasks:Running task ceph... 2024-09-18T10:13:12.224 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-18T10:13:12.224 DEBUG:teuthology.orchestra.run.smithi029:> sudo chmod 777 /var/log/ceph 2024-09-18T10:13:12.226 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod 777 /var/log/ceph 2024-09-18T10:13:12.228 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod 777 /var/log/ceph 2024-09-18T10:13:12.231 DEBUG:teuthology.orchestra.run.smithi189:> sudo chmod 777 /var/log/ceph 2024-09-18T10:13:12.246 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-18T10:13:12.246 DEBUG:teuthology.orchestra.run.smithi029:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-18T10:13:12.248 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-18T10:13:12.286 DEBUG:teuthology.orchestra.run.smithi154:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-18T10:13:12.288 DEBUG:teuthology.orchestra.run.smithi189:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-18T10:13:12.303 INFO:tasks.ceph:Creating extra log directories... 2024-09-18T10:13:12.303 DEBUG:teuthology.orchestra.run.smithi029:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-18T10:13:12.305 DEBUG:teuthology.orchestra.run.smithi136:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-18T10:13:12.307 DEBUG:teuthology.orchestra.run.smithi154:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-18T10:13:12.309 DEBUG:teuthology.orchestra.run.smithi189:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-18T10:13:12.332 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-18T10:13:12.332 INFO:tasks.ceph:config {'conf': {'osd': {'debug monc': 20, 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug reject backfill probability': 0.3, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True}, 'global': {'mon election default strategy': 1, 'osd debug inject dispatch delay duration': 0.1, 'osd debug inject dispatch delay probability': 0.1}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'but it is still running', 'objects unfound and apparently lost', 'slow request', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'mons down', 'mon down', 'out of quorum'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-18T10:13:12.333 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/skanta-2024-09-18_09:13:00-rados-wip-bharath11-testing-2024-09-18-1115-quincy-distro-default-smithi/7910845', 'branch': 'wip-bharath11-testing-2024-09-18-1115-quincy', 'description': 'rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/osd-dispatch-delay objectstore/bluestore-bitmap rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2}', 'email': 'skanta@redhat.com', 'first_in_suite': False, 'job_id': '7910845', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'skanta-2024-09-18_09:13:00-rados-wip-bharath11-testing-2024-09-18-1115-quincy-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 10}}], 'os_type': 'ubuntu', 'os_version': '20.04', 'overrides': {'admin_socket': {'branch': 'wip-bharath11-testing-2024-09-18-1115-quincy'}, 'ceph': {'conf': {'global': {'mon election default strategy': 1, 'osd debug inject dispatch delay duration': 0.1, 'osd debug inject dispatch delay probability': 0.1}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug reject backfill probability': 0.3, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'but it is still running', 'objects unfound and apparently lost', 'slow request', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'mons down', 'mon down', 'out of quorum'], 'sha1': '7061ea6ca50908d104e9f3979dc615e118d9e153'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '7061ea6ca50908d104e9f3979dc615e118d9e153'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-bharath11-testing-2024-09-18-1115-quincy', 'sha1': '7061ea6ca50908d104e9f3979dc615e118d9e153'}}, 'owner': 'scheduled_skanta@teuthology', 'priority': 99, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'osd.0', 'osd.4', 'osd.8', 'osd.12'], ['mon.b', 'osd.1', 'osd.5', 'osd.9', 'osd.13'], ['mon.c', 'osd.2', 'osd.6', 'osd.10', 'osd.14'], ['mgr.x', 'osd.3', 'osd.7', 'osd.11', 'osd.15', 'client.0']], 'seed': 4631, 'sha1': '7061ea6ca50908d104e9f3979dc615e118d9e153', 'sleep_before_teardown': 0, 'subset': '111/120000', 'suite': 'rados', 'suite_branch': 'wip-bharath11-testing-2024-09-18-1115-quincy', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_7061ea6ca50908d104e9f3979dc615e118d9e153/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': '7061ea6ca50908d104e9f3979dc615e118d9e153', 'targets': {'smithi029.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM=', 'smithi136.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM=', 'smithi154.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM=', 'smithi189.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': {'conf': {'osd': {'debug monc': 20, 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug reject backfill probability': 0.3, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True}, 'global': {'mon election default strategy': 1, 'osd debug inject dispatch delay duration': 0.1, 'osd debug inject dispatch delay probability': 0.1}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'but it is still running', 'objects unfound and apparently lost', 'slow request', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'mons down', 'mon down', 'out of quorum'], 'sha1': '7061ea6ca50908d104e9f3979dc615e118d9e153', 'cluster': 'ceph'}}, {'thrashosds': {'aggressive_pg_num_changes': False, 'chance_pgnum_grow': 1, 'chance_pgnum_shrink': 1, 'chance_pgpnum_fix': 1, 'min_in': 8, 'timeout': 1200}}, {'rados': {'clients': ['client.0'], 'ec_pool': True, 'erasure_code_profile': {'c': 2, 'crush-failure-domain': 'osd', 'k': 4, 'm': 3, 'name': 'shecprofile', 'plugin': 'shec'}, 'objects': 50, 'op_weights': {'append': 100, 'copy_from': 50, 'delete': 50, 'read': 100, 'rmattr': 25, 'rollback': 50, 'setattr': 25, 'snap_create': 50, 'snap_remove': 50, 'write': 0}, 'ops': 400, 'write_append_excl': False}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-18_09:13:00', 'tube': 'smithi', 'user': 'skanta', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-18T10:13:12.333 DEBUG:teuthology.orchestra.run.smithi029:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-18T10:13:12.335 DEBUG:teuthology.orchestra.run.smithi136:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-18T10:13:12.337 DEBUG:teuthology.orchestra.run.smithi154:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-18T10:13:12.364 DEBUG:teuthology.orchestra.run.smithi189:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-18T10:13:12.371 DEBUG:teuthology.orchestra.run.smithi029:> sudo install -d -m0777 -- /var/run/ceph 2024-09-18T10:13:12.385 DEBUG:teuthology.orchestra.run.smithi136:> sudo install -d -m0777 -- /var/run/ceph 2024-09-18T10:13:12.388 DEBUG:teuthology.orchestra.run.smithi154:> sudo install -d -m0777 -- /var/run/ceph 2024-09-18T10:13:12.390 DEBUG:teuthology.orchestra.run.smithi189:> sudo install -d -m0777 -- /var/run/ceph 2024-09-18T10:13:12.406 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-09-18T10:13:12.406 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/scratch_devs of=/dev/stdout 2024-09-18T10:13:12.412 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-18T10:13:12.412 DEBUG:teuthology.orchestra.run.smithi029:> stat /dev/vg_nvme/lv_1 2024-09-18T10:13:12.459 INFO:teuthology.orchestra.run.smithi029.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-18T10:13:12.459 INFO:teuthology.orchestra.run.smithi029.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-18T10:13:12.459 INFO:teuthology.orchestra.run.smithi029.stdout:Device: 6h/6d Inode: 360 Links: 1 2024-09-18T10:13:12.459 INFO:teuthology.orchestra.run.smithi029.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-18T10:13:12.459 INFO:teuthology.orchestra.run.smithi029.stdout:Access: 2024-09-18 10:09:27.837141464 +0000 2024-09-18T10:13:12.459 INFO:teuthology.orchestra.run.smithi029.stdout:Modify: 2024-09-18 10:09:27.621146441 +0000 2024-09-18T10:13:12.459 INFO:teuthology.orchestra.run.smithi029.stdout:Change: 2024-09-18 10:09:27.621146441 +0000 2024-09-18T10:13:12.459 INFO:teuthology.orchestra.run.smithi029.stdout: Birth: - 2024-09-18T10:13:12.460 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-18T10:13:12.472 INFO:teuthology.orchestra.run.smithi029.stderr:1+0 records in 2024-09-18T10:13:12.472 INFO:teuthology.orchestra.run.smithi029.stderr:1+0 records out 2024-09-18T10:13:12.472 INFO:teuthology.orchestra.run.smithi029.stderr:512 bytes copied, 0.000308563 s, 1.7 MB/s 2024-09-18T10:13:12.473 DEBUG:teuthology.orchestra.run.smithi029:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-18T10:13:12.481 DEBUG:teuthology.orchestra.run.smithi029:> stat /dev/vg_nvme/lv_2 2024-09-18T10:13:12.531 INFO:teuthology.orchestra.run.smithi029.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-18T10:13:12.531 INFO:teuthology.orchestra.run.smithi029.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-18T10:13:12.531 INFO:teuthology.orchestra.run.smithi029.stdout:Device: 6h/6d Inode: 364 Links: 1 2024-09-18T10:13:12.531 INFO:teuthology.orchestra.run.smithi029.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-18T10:13:12.531 INFO:teuthology.orchestra.run.smithi029.stdout:Access: 2024-09-18 10:09:28.253131874 +0000 2024-09-18T10:13:12.531 INFO:teuthology.orchestra.run.smithi029.stdout:Modify: 2024-09-18 10:09:28.037136853 +0000 2024-09-18T10:13:12.531 INFO:teuthology.orchestra.run.smithi029.stdout:Change: 2024-09-18 10:09:28.037136853 +0000 2024-09-18T10:13:12.532 INFO:teuthology.orchestra.run.smithi029.stdout: Birth: - 2024-09-18T10:13:12.532 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-18T10:13:12.546 INFO:teuthology.orchestra.run.smithi029.stderr:1+0 records in 2024-09-18T10:13:12.546 INFO:teuthology.orchestra.run.smithi029.stderr:1+0 records out 2024-09-18T10:13:12.547 INFO:teuthology.orchestra.run.smithi029.stderr:512 bytes copied, 0.000295384 s, 1.7 MB/s 2024-09-18T10:13:12.547 DEBUG:teuthology.orchestra.run.smithi029:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-18T10:13:12.554 DEBUG:teuthology.orchestra.run.smithi029:> stat /dev/vg_nvme/lv_3 2024-09-18T10:13:12.603 INFO:teuthology.orchestra.run.smithi029.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-18T10:13:12.603 INFO:teuthology.orchestra.run.smithi029.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-18T10:13:12.603 INFO:teuthology.orchestra.run.smithi029.stdout:Device: 6h/6d Inode: 371 Links: 1 2024-09-18T10:13:12.603 INFO:teuthology.orchestra.run.smithi029.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-18T10:13:12.603 INFO:teuthology.orchestra.run.smithi029.stdout:Access: 2024-09-18 10:09:28.449127357 +0000 2024-09-18T10:13:12.603 INFO:teuthology.orchestra.run.smithi029.stdout:Modify: 2024-09-18 10:09:28.445127448 +0000 2024-09-18T10:13:12.603 INFO:teuthology.orchestra.run.smithi029.stdout:Change: 2024-09-18 10:09:28.445127448 +0000 2024-09-18T10:13:12.603 INFO:teuthology.orchestra.run.smithi029.stdout: Birth: - 2024-09-18T10:13:12.603 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-18T10:13:12.616 INFO:teuthology.orchestra.run.smithi029.stderr:1+0 records in 2024-09-18T10:13:12.616 INFO:teuthology.orchestra.run.smithi029.stderr:1+0 records out 2024-09-18T10:13:12.616 INFO:teuthology.orchestra.run.smithi029.stderr:512 bytes copied, 0.000203201 s, 2.5 MB/s 2024-09-18T10:13:12.617 DEBUG:teuthology.orchestra.run.smithi029:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-18T10:13:12.624 DEBUG:teuthology.orchestra.run.smithi029:> stat /dev/vg_nvme/lv_4 2024-09-18T10:13:12.675 INFO:teuthology.orchestra.run.smithi029.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-18T10:13:12.675 INFO:teuthology.orchestra.run.smithi029.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-18T10:13:12.675 INFO:teuthology.orchestra.run.smithi029.stdout:Device: 6h/6d Inode: 377 Links: 1 2024-09-18T10:13:12.675 INFO:teuthology.orchestra.run.smithi029.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-18T10:13:12.675 INFO:teuthology.orchestra.run.smithi029.stdout:Access: 2024-09-18 10:09:29.077112881 +0000 2024-09-18T10:13:12.675 INFO:teuthology.orchestra.run.smithi029.stdout:Modify: 2024-09-18 10:09:28.873117582 +0000 2024-09-18T10:13:12.675 INFO:teuthology.orchestra.run.smithi029.stdout:Change: 2024-09-18 10:09:28.873117582 +0000 2024-09-18T10:13:12.675 INFO:teuthology.orchestra.run.smithi029.stdout: Birth: - 2024-09-18T10:13:12.676 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-18T10:13:12.690 INFO:teuthology.orchestra.run.smithi029.stderr:1+0 records in 2024-09-18T10:13:12.690 INFO:teuthology.orchestra.run.smithi029.stderr:1+0 records out 2024-09-18T10:13:12.690 INFO:teuthology.orchestra.run.smithi029.stderr:512 bytes copied, 0.000202493 s, 2.5 MB/s 2024-09-18T10:13:12.691 DEBUG:teuthology.orchestra.run.smithi029:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-18T10:13:12.698 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.8': '/dev/vg_nvme/lv_3', 'osd.12': '/dev/vg_nvme/lv_4'} 2024-09-18T10:13:12.698 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-09-18T10:13:12.698 DEBUG:teuthology.orchestra.run.smithi136:> dd if=/scratch_devs of=/dev/stdout 2024-09-18T10:13:12.705 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-18T10:13:12.705 DEBUG:teuthology.orchestra.run.smithi136:> stat /dev/vg_nvme/lv_1 2024-09-18T10:13:12.755 INFO:teuthology.orchestra.run.smithi136.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-18T10:13:12.755 INFO:teuthology.orchestra.run.smithi136.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-18T10:13:12.756 INFO:teuthology.orchestra.run.smithi136.stdout:Device: 6h/6d Inode: 359 Links: 1 2024-09-18T10:13:12.756 INFO:teuthology.orchestra.run.smithi136.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-18T10:13:12.756 INFO:teuthology.orchestra.run.smithi136.stdout:Access: 2024-09-18 10:09:16.530507664 +0000 2024-09-18T10:13:12.756 INFO:teuthology.orchestra.run.smithi136.stdout:Modify: 2024-09-18 10:09:16.526507758 +0000 2024-09-18T10:13:12.756 INFO:teuthology.orchestra.run.smithi136.stdout:Change: 2024-09-18 10:09:16.526507758 +0000 2024-09-18T10:13:12.756 INFO:teuthology.orchestra.run.smithi136.stdout: Birth: - 2024-09-18T10:13:12.756 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-18T10:13:12.772 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records in 2024-09-18T10:13:12.772 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records out 2024-09-18T10:13:12.772 INFO:teuthology.orchestra.run.smithi136.stderr:512 bytes copied, 0.00043984 s, 1.2 MB/s 2024-09-18T10:13:12.773 DEBUG:teuthology.orchestra.run.smithi136:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-18T10:13:12.782 DEBUG:teuthology.orchestra.run.smithi136:> stat /dev/vg_nvme/lv_2 2024-09-18T10:13:12.789 INFO:teuthology.orchestra.run.smithi136.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-18T10:13:12.789 INFO:teuthology.orchestra.run.smithi136.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-18T10:13:12.789 INFO:teuthology.orchestra.run.smithi136.stdout:Device: 6h/6d Inode: 364 Links: 1 2024-09-18T10:13:12.789 INFO:teuthology.orchestra.run.smithi136.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-18T10:13:12.789 INFO:teuthology.orchestra.run.smithi136.stdout:Access: 2024-09-18 10:09:17.178492721 +0000 2024-09-18T10:13:12.789 INFO:teuthology.orchestra.run.smithi136.stdout:Modify: 2024-09-18 10:09:16.954497887 +0000 2024-09-18T10:13:12.789 INFO:teuthology.orchestra.run.smithi136.stdout:Change: 2024-09-18 10:09:16.954497887 +0000 2024-09-18T10:13:12.789 INFO:teuthology.orchestra.run.smithi136.stdout: Birth: - 2024-09-18T10:13:12.790 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-18T10:13:12.846 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records in 2024-09-18T10:13:12.846 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records out 2024-09-18T10:13:12.846 INFO:teuthology.orchestra.run.smithi136.stderr:512 bytes copied, 0.000403634 s, 1.3 MB/s 2024-09-18T10:13:12.847 DEBUG:teuthology.orchestra.run.smithi136:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-18T10:13:12.856 DEBUG:teuthology.orchestra.run.smithi136:> stat /dev/vg_nvme/lv_3 2024-09-18T10:13:12.861 INFO:teuthology.orchestra.run.smithi136.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-18T10:13:12.861 INFO:teuthology.orchestra.run.smithi136.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-18T10:13:12.861 INFO:teuthology.orchestra.run.smithi136.stdout:Device: 6h/6d Inode: 372 Links: 1 2024-09-18T10:13:12.861 INFO:teuthology.orchestra.run.smithi136.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-18T10:13:12.862 INFO:teuthology.orchestra.run.smithi136.stdout:Access: 2024-09-18 10:09:17.578483497 +0000 2024-09-18T10:13:12.862 INFO:teuthology.orchestra.run.smithi136.stdout:Modify: 2024-09-18 10:09:17.374488202 +0000 2024-09-18T10:13:12.862 INFO:teuthology.orchestra.run.smithi136.stdout:Change: 2024-09-18 10:09:17.374488202 +0000 2024-09-18T10:13:12.862 INFO:teuthology.orchestra.run.smithi136.stdout: Birth: - 2024-09-18T10:13:12.862 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-18T10:13:12.916 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records in 2024-09-18T10:13:12.917 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records out 2024-09-18T10:13:12.917 INFO:teuthology.orchestra.run.smithi136.stderr:512 bytes copied, 0.000391878 s, 1.3 MB/s 2024-09-18T10:13:12.918 DEBUG:teuthology.orchestra.run.smithi136:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-18T10:13:12.928 DEBUG:teuthology.orchestra.run.smithi136:> stat /dev/vg_nvme/lv_4 2024-09-18T10:13:12.933 INFO:teuthology.orchestra.run.smithi136.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-18T10:13:12.934 INFO:teuthology.orchestra.run.smithi136.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-18T10:13:12.934 INFO:teuthology.orchestra.run.smithi136.stdout:Device: 6h/6d Inode: 379 Links: 1 2024-09-18T10:13:12.934 INFO:teuthology.orchestra.run.smithi136.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-18T10:13:12.934 INFO:teuthology.orchestra.run.smithi136.stdout:Access: 2024-09-18 10:09:17.990473995 +0000 2024-09-18T10:13:12.934 INFO:teuthology.orchestra.run.smithi136.stdout:Modify: 2024-09-18 10:09:17.782478792 +0000 2024-09-18T10:13:12.934 INFO:teuthology.orchestra.run.smithi136.stdout:Change: 2024-09-18 10:09:17.782478792 +0000 2024-09-18T10:13:12.934 INFO:teuthology.orchestra.run.smithi136.stdout: Birth: - 2024-09-18T10:13:12.934 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-18T10:13:12.990 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records in 2024-09-18T10:13:12.991 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records out 2024-09-18T10:13:12.991 INFO:teuthology.orchestra.run.smithi136.stderr:512 bytes copied, 0.000393835 s, 1.3 MB/s 2024-09-18T10:13:12.992 DEBUG:teuthology.orchestra.run.smithi136:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-18T10:13:13.000 INFO:tasks.ceph:osd dev map: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.9': '/dev/vg_nvme/lv_3', 'osd.13': '/dev/vg_nvme/lv_4'} 2024-09-18T10:13:13.001 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-09-18T10:13:13.001 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/scratch_devs of=/dev/stdout 2024-09-18T10:13:13.008 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-18T10:13:13.008 DEBUG:teuthology.orchestra.run.smithi154:> stat /dev/vg_nvme/lv_1 2024-09-18T10:13:13.016 INFO:teuthology.orchestra.run.smithi154.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-18T10:13:13.016 INFO:teuthology.orchestra.run.smithi154.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-18T10:13:13.016 INFO:teuthology.orchestra.run.smithi154.stdout:Device: 6h/6d Inode: 361 Links: 1 2024-09-18T10:13:13.016 INFO:teuthology.orchestra.run.smithi154.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-18T10:13:13.016 INFO:teuthology.orchestra.run.smithi154.stdout:Access: 2024-09-18 10:09:18.251242570 +0000 2024-09-18T10:13:13.016 INFO:teuthology.orchestra.run.smithi154.stdout:Modify: 2024-09-18 10:09:18.051247174 +0000 2024-09-18T10:13:13.017 INFO:teuthology.orchestra.run.smithi154.stdout:Change: 2024-09-18 10:09:18.051247174 +0000 2024-09-18T10:13:13.017 INFO:teuthology.orchestra.run.smithi154.stdout: Birth: - 2024-09-18T10:13:13.017 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-18T10:13:13.029 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records in 2024-09-18T10:13:13.029 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records out 2024-09-18T10:13:13.029 INFO:teuthology.orchestra.run.smithi154.stderr:512 bytes copied, 0.000398562 s, 1.3 MB/s 2024-09-18T10:13:13.030 DEBUG:teuthology.orchestra.run.smithi154:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-18T10:13:13.083 DEBUG:teuthology.orchestra.run.smithi154:> stat /dev/vg_nvme/lv_2 2024-09-18T10:13:13.089 INFO:teuthology.orchestra.run.smithi154.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-18T10:13:13.089 INFO:teuthology.orchestra.run.smithi154.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-18T10:13:13.089 INFO:teuthology.orchestra.run.smithi154.stdout:Device: 6h/6d Inode: 366 Links: 1 2024-09-18T10:13:13.090 INFO:teuthology.orchestra.run.smithi154.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-18T10:13:13.090 INFO:teuthology.orchestra.run.smithi154.stdout:Access: 2024-09-18 10:09:18.707232074 +0000 2024-09-18T10:13:13.090 INFO:teuthology.orchestra.run.smithi154.stdout:Modify: 2024-09-18 10:09:18.471237506 +0000 2024-09-18T10:13:13.090 INFO:teuthology.orchestra.run.smithi154.stdout:Change: 2024-09-18 10:09:18.471237506 +0000 2024-09-18T10:13:13.090 INFO:teuthology.orchestra.run.smithi154.stdout: Birth: - 2024-09-18T10:13:13.091 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-18T10:13:13.102 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records in 2024-09-18T10:13:13.103 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records out 2024-09-18T10:13:13.103 INFO:teuthology.orchestra.run.smithi154.stderr:512 bytes copied, 0.000451484 s, 1.1 MB/s 2024-09-18T10:13:13.103 DEBUG:teuthology.orchestra.run.smithi154:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-18T10:13:13.155 DEBUG:teuthology.orchestra.run.smithi154:> stat /dev/vg_nvme/lv_3 2024-09-18T10:13:13.162 INFO:teuthology.orchestra.run.smithi154.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-18T10:13:13.163 INFO:teuthology.orchestra.run.smithi154.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-18T10:13:13.163 INFO:teuthology.orchestra.run.smithi154.stdout:Device: 6h/6d Inode: 374 Links: 1 2024-09-18T10:13:13.163 INFO:teuthology.orchestra.run.smithi154.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-18T10:13:13.163 INFO:teuthology.orchestra.run.smithi154.stdout:Access: 2024-09-18 10:09:19.155221762 +0000 2024-09-18T10:13:13.163 INFO:teuthology.orchestra.run.smithi154.stdout:Modify: 2024-09-18 10:09:18.943226641 +0000 2024-09-18T10:13:13.163 INFO:teuthology.orchestra.run.smithi154.stdout:Change: 2024-09-18 10:09:18.943226641 +0000 2024-09-18T10:13:13.163 INFO:teuthology.orchestra.run.smithi154.stdout: Birth: - 2024-09-18T10:13:13.163 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-18T10:13:13.220 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records in 2024-09-18T10:13:13.221 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records out 2024-09-18T10:13:13.221 INFO:teuthology.orchestra.run.smithi154.stderr:512 bytes copied, 0.000403762 s, 1.3 MB/s 2024-09-18T10:13:13.222 DEBUG:teuthology.orchestra.run.smithi154:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-18T10:13:13.230 DEBUG:teuthology.orchestra.run.smithi154:> stat /dev/vg_nvme/lv_4 2024-09-18T10:13:13.235 INFO:teuthology.orchestra.run.smithi154.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-18T10:13:13.236 INFO:teuthology.orchestra.run.smithi154.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-18T10:13:13.236 INFO:teuthology.orchestra.run.smithi154.stdout:Device: 6h/6d Inode: 379 Links: 1 2024-09-18T10:13:13.236 INFO:teuthology.orchestra.run.smithi154.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-18T10:13:13.236 INFO:teuthology.orchestra.run.smithi154.stdout:Access: 2024-09-18 10:09:19.371216789 +0000 2024-09-18T10:13:13.236 INFO:teuthology.orchestra.run.smithi154.stdout:Modify: 2024-09-18 10:09:19.367216882 +0000 2024-09-18T10:13:13.236 INFO:teuthology.orchestra.run.smithi154.stdout:Change: 2024-09-18 10:09:19.367216882 +0000 2024-09-18T10:13:13.236 INFO:teuthology.orchestra.run.smithi154.stdout: Birth: - 2024-09-18T10:13:13.236 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-18T10:13:13.291 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records in 2024-09-18T10:13:13.291 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records out 2024-09-18T10:13:13.291 INFO:teuthology.orchestra.run.smithi154.stderr:512 bytes copied, 0.000335797 s, 1.5 MB/s 2024-09-18T10:13:13.292 DEBUG:teuthology.orchestra.run.smithi154:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-18T10:13:13.300 INFO:tasks.ceph:osd dev map: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.6': '/dev/vg_nvme/lv_2', 'osd.10': '/dev/vg_nvme/lv_3', 'osd.14': '/dev/vg_nvme/lv_4'} 2024-09-18T10:13:13.300 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-18T10:13:13.300 DEBUG:teuthology.orchestra.run.smithi189:> dd if=/scratch_devs of=/dev/stdout 2024-09-18T10:13:13.307 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-18T10:13:13.308 DEBUG:teuthology.orchestra.run.smithi189:> stat /dev/vg_nvme/lv_1 2024-09-18T10:13:13.358 INFO:teuthology.orchestra.run.smithi189.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-18T10:13:13.359 INFO:teuthology.orchestra.run.smithi189.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-18T10:13:13.359 INFO:teuthology.orchestra.run.smithi189.stdout:Device: 6h/6d Inode: 368 Links: 1 2024-09-18T10:13:13.359 INFO:teuthology.orchestra.run.smithi189.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-18T10:13:13.359 INFO:teuthology.orchestra.run.smithi189.stdout:Access: 2024-09-18 10:09:19.579075602 +0000 2024-09-18T10:13:13.359 INFO:teuthology.orchestra.run.smithi189.stdout:Modify: 2024-09-18 10:09:19.575075693 +0000 2024-09-18T10:13:13.359 INFO:teuthology.orchestra.run.smithi189.stdout:Change: 2024-09-18 10:09:19.575075693 +0000 2024-09-18T10:13:13.359 INFO:teuthology.orchestra.run.smithi189.stdout: Birth: - 2024-09-18T10:13:13.359 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-18T10:13:13.371 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records in 2024-09-18T10:13:13.371 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records out 2024-09-18T10:13:13.371 INFO:teuthology.orchestra.run.smithi189.stderr:512 bytes copied, 0.000425927 s, 1.2 MB/s 2024-09-18T10:13:13.372 DEBUG:teuthology.orchestra.run.smithi189:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-18T10:13:13.380 DEBUG:teuthology.orchestra.run.smithi189:> stat /dev/vg_nvme/lv_2 2024-09-18T10:13:13.386 INFO:teuthology.orchestra.run.smithi189.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-18T10:13:13.386 INFO:teuthology.orchestra.run.smithi189.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-18T10:13:13.386 INFO:teuthology.orchestra.run.smithi189.stdout:Device: 6h/6d Inode: 375 Links: 1 2024-09-18T10:13:13.386 INFO:teuthology.orchestra.run.smithi189.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-18T10:13:13.386 INFO:teuthology.orchestra.run.smithi189.stdout:Access: 2024-09-18 10:09:20.207061126 +0000 2024-09-18T10:13:13.386 INFO:teuthology.orchestra.run.smithi189.stdout:Modify: 2024-09-18 10:09:20.007065736 +0000 2024-09-18T10:13:13.386 INFO:teuthology.orchestra.run.smithi189.stdout:Change: 2024-09-18 10:09:20.007065736 +0000 2024-09-18T10:13:13.386 INFO:teuthology.orchestra.run.smithi189.stdout: Birth: - 2024-09-18T10:13:13.387 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-18T10:13:13.398 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records in 2024-09-18T10:13:13.398 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records out 2024-09-18T10:13:13.398 INFO:teuthology.orchestra.run.smithi189.stderr:512 bytes copied, 0.00040688 s, 1.3 MB/s 2024-09-18T10:13:13.399 DEBUG:teuthology.orchestra.run.smithi189:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-18T10:13:13.452 DEBUG:teuthology.orchestra.run.smithi189:> stat /dev/vg_nvme/lv_3 2024-09-18T10:13:13.459 INFO:teuthology.orchestra.run.smithi189.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-18T10:13:13.459 INFO:teuthology.orchestra.run.smithi189.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-18T10:13:13.459 INFO:teuthology.orchestra.run.smithi189.stdout:Device: 6h/6d Inode: 381 Links: 1 2024-09-18T10:13:13.459 INFO:teuthology.orchestra.run.smithi189.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-18T10:13:13.459 INFO:teuthology.orchestra.run.smithi189.stdout:Access: 2024-09-18 10:09:20.627051445 +0000 2024-09-18T10:13:13.460 INFO:teuthology.orchestra.run.smithi189.stdout:Modify: 2024-09-18 10:09:20.435055871 +0000 2024-09-18T10:13:13.460 INFO:teuthology.orchestra.run.smithi189.stdout:Change: 2024-09-18 10:09:20.435055871 +0000 2024-09-18T10:13:13.460 INFO:teuthology.orchestra.run.smithi189.stdout: Birth: - 2024-09-18T10:13:13.460 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-18T10:13:13.470 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records in 2024-09-18T10:13:13.471 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records out 2024-09-18T10:13:13.471 INFO:teuthology.orchestra.run.smithi189.stderr:512 bytes copied, 0.000414277 s, 1.2 MB/s 2024-09-18T10:13:13.472 DEBUG:teuthology.orchestra.run.smithi189:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-18T10:13:13.524 DEBUG:teuthology.orchestra.run.smithi189:> stat /dev/vg_nvme/lv_4 2024-09-18T10:13:13.531 INFO:teuthology.orchestra.run.smithi189.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-18T10:13:13.531 INFO:teuthology.orchestra.run.smithi189.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-18T10:13:13.531 INFO:teuthology.orchestra.run.smithi189.stdout:Device: 6h/6d Inode: 388 Links: 1 2024-09-18T10:13:13.531 INFO:teuthology.orchestra.run.smithi189.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-18T10:13:13.531 INFO:teuthology.orchestra.run.smithi189.stdout:Access: 2024-09-18 10:09:21.075041119 +0000 2024-09-18T10:13:13.531 INFO:teuthology.orchestra.run.smithi189.stdout:Modify: 2024-09-18 10:09:20.859046098 +0000 2024-09-18T10:13:13.531 INFO:teuthology.orchestra.run.smithi189.stdout:Change: 2024-09-18 10:09:20.859046098 +0000 2024-09-18T10:13:13.532 INFO:teuthology.orchestra.run.smithi189.stdout: Birth: - 2024-09-18T10:13:13.532 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-18T10:13:13.545 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records in 2024-09-18T10:13:13.545 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records out 2024-09-18T10:13:13.545 INFO:teuthology.orchestra.run.smithi189.stderr:512 bytes copied, 0.000363102 s, 1.4 MB/s 2024-09-18T10:13:13.546 DEBUG:teuthology.orchestra.run.smithi189:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-18T10:13:13.596 INFO:tasks.ceph:osd dev map: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.7': '/dev/vg_nvme/lv_2', 'osd.11': '/dev/vg_nvme/lv_3', 'osd.15': '/dev/vg_nvme/lv_4'} 2024-09-18T10:13:13.596 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi029.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.8': '/dev/vg_nvme/lv_3', 'osd.12': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi136.front.sepia.ceph.com'): {'osd.1': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.9': '/dev/vg_nvme/lv_3', 'osd.13': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi154.front.sepia.ceph.com'): {'osd.2': '/dev/vg_nvme/lv_1', 'osd.6': '/dev/vg_nvme/lv_2', 'osd.10': '/dev/vg_nvme/lv_3', 'osd.14': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi189.front.sepia.ceph.com'): {'osd.3': '/dev/vg_nvme/lv_1', 'osd.7': '/dev/vg_nvme/lv_2', 'osd.11': '/dev/vg_nvme/lv_3', 'osd.15': '/dev/vg_nvme/lv_4'}} 2024-09-18T10:13:13.596 INFO:tasks.ceph:Generating config... 2024-09-18T10:13:13.598 INFO:tasks.ceph:[osd] debug monc = 20 2024-09-18T10:13:13.598 INFO:tasks.ceph:[osd] bdev async discard = True 2024-09-18T10:13:13.598 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-09-18T10:13:13.598 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-09-18T10:13:13.598 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-18T10:13:13.598 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-18T10:13:13.598 INFO:tasks.ceph:[osd] bluestore zero block detection = True 2024-09-18T10:13:13.598 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-09-18T10:13:13.598 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-09-18T10:13:13.598 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-18T10:13:13.599 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-18T10:13:13.599 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-09-18T10:13:13.599 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-18T10:13:13.599 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-18T10:13:13.599 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-18T10:13:13.599 INFO:tasks.ceph:[osd] osd debug reject backfill probability = 0.3 2024-09-18T10:13:13.599 INFO:tasks.ceph:[osd] osd debug verify cached snaps = True 2024-09-18T10:13:13.599 INFO:tasks.ceph:[osd] osd debug verify missing on start = True 2024-09-18T10:13:13.599 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-18T10:13:13.599 INFO:tasks.ceph:[osd] osd max backfills = 3 2024-09-18T10:13:13.599 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2024-09-18T10:13:13.600 INFO:tasks.ceph:[osd] osd mclock override recovery settings = True 2024-09-18T10:13:13.600 INFO:tasks.ceph:[osd] osd mclock profile = high_recovery_ops 2024-09-18T10:13:13.600 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-18T10:13:13.600 INFO:tasks.ceph:[osd] osd op queue = debug_random 2024-09-18T10:13:13.600 INFO:tasks.ceph:[osd] osd op queue cut off = debug_random 2024-09-18T10:13:13.600 INFO:tasks.ceph:[osd] osd scrub max interval = 120 2024-09-18T10:13:13.600 INFO:tasks.ceph:[osd] osd scrub min interval = 60 2024-09-18T10:13:13.600 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2024-09-18T10:13:13.600 INFO:tasks.ceph:[global] mon election default strategy = 1 2024-09-18T10:13:13.600 INFO:tasks.ceph:[global] osd debug inject dispatch delay duration = 0.1 2024-09-18T10:13:13.600 INFO:tasks.ceph:[global] osd debug inject dispatch delay probability = 0.1 2024-09-18T10:13:13.600 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-18T10:13:13.600 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-18T10:13:13.600 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-18T10:13:13.601 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-18T10:13:13.601 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-18T10:13:13.601 INFO:tasks.ceph:[mon] mon scrub interval = 300 2024-09-18T10:13:13.601 INFO:tasks.ceph:Setting up mon.a... 2024-09-18T10:13:13.601 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-18T10:13:13.636 INFO:teuthology.orchestra.run.smithi029.stdout:creating /etc/ceph/ceph.keyring 2024-09-18T10:13:13.639 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-18T10:13:13.672 DEBUG:teuthology.orchestra.run.smithi029:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-18T10:13:13.732 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.29'), ('mon.b', '172.21.15.136'), ('mon.c', '172.21.15.154')] 2024-09-18T10:13:13.732 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.29,172.21.15.136,172.21.15.154', 'mon election default strategy': 1, 'osd debug inject dispatch delay duration': 0.1, 'osd debug inject dispatch delay probability': 0.1}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': 120, 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'debug monc': 20, 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug reject backfill probability': 0.3, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd mclock override recovery settings': True, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub min interval': 60, 'osd shutdown pgref assert': True}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon scrub interval': 300}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok'}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}} 2024-09-18T10:13:13.733 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-09-18T10:13:13.733 DEBUG:teuthology.orchestra.run.smithi029:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-18T10:13:13.738 DEBUG:teuthology.orchestra.run.smithi029:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.29 --add b 172.21.15.136 --add c 172.21.15.154 --print /home/ubuntu/cephtest/ceph.monmap 2024-09-18T10:13:13.821 INFO:teuthology.orchestra.run.smithi029.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-18T10:13:13.821 INFO:teuthology.orchestra.run.smithi029.stdout:monmaptool: generated fsid ab6d6beb-0627-4b7f-8cdb-4e0a44970f10 2024-09-18T10:13:13.821 INFO:teuthology.orchestra.run.smithi029.stdout:setting min_mon_release = octopus 2024-09-18T10:13:13.821 INFO:teuthology.orchestra.run.smithi029.stdout:epoch 0 2024-09-18T10:13:13.821 INFO:teuthology.orchestra.run.smithi029.stdout:fsid ab6d6beb-0627-4b7f-8cdb-4e0a44970f10 2024-09-18T10:13:13.821 INFO:teuthology.orchestra.run.smithi029.stdout:last_changed 2024-09-18T10:13:13.821269+0000 2024-09-18T10:13:13.821 INFO:teuthology.orchestra.run.smithi029.stdout:created 2024-09-18T10:13:13.821269+0000 2024-09-18T10:13:13.821 INFO:teuthology.orchestra.run.smithi029.stdout:min_mon_release 15 (octopus) 2024-09-18T10:13:13.822 INFO:teuthology.orchestra.run.smithi029.stdout:election_strategy: 1 2024-09-18T10:13:13.822 INFO:teuthology.orchestra.run.smithi029.stdout:0: [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] mon.a 2024-09-18T10:13:13.822 INFO:teuthology.orchestra.run.smithi029.stdout:1: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.b 2024-09-18T10:13:13.822 INFO:teuthology.orchestra.run.smithi029.stdout:2: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.c 2024-09-18T10:13:13.822 INFO:teuthology.orchestra.run.smithi029.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-18T10:13:13.823 DEBUG:teuthology.orchestra.run.smithi029:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-18T10:13:13.827 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID ab6d6beb-0627-4b7f-8cdb-4e0a44970f10... 2024-09-18T10:13:13.828 DEBUG:teuthology.orchestra.run.smithi029:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-18T10:13:13.830 DEBUG:teuthology.orchestra.run.smithi136:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-18T10:13:13.832 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-18T10:13:13.835 DEBUG:teuthology.orchestra.run.smithi189:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-18T10:13:13.865 INFO:teuthology.orchestra.run.smithi029.stdout:[global] 2024-09-18T10:13:13.865 INFO:teuthology.orchestra.run.smithi029.stdout: chdir = "" 2024-09-18T10:13:13.865 INFO:teuthology.orchestra.run.smithi029.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-18T10:13:13.865 INFO:teuthology.orchestra.run.smithi029.stdout: auth supported = cephx 2024-09-18T10:13:13.865 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-18T10:13:13.865 INFO:teuthology.orchestra.run.smithi029.stdout: filestore xattr use omap = true 2024-09-18T10:13:13.865 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-18T10:13:13.865 INFO:teuthology.orchestra.run.smithi029.stdout: mon clock drift allowed = 1.000 2024-09-18T10:13:13.866 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-18T10:13:13.866 INFO:teuthology.orchestra.run.smithi029.stdout: osd crush chooseleaf type = 0 2024-09-18T10:13:13.866 INFO:teuthology.orchestra.run.smithi029.stdout: auth debug = true 2024-09-18T10:13:13.866 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-18T10:13:13.866 INFO:teuthology.orchestra.run.smithi029.stdout: ms die on old message = true 2024-09-18T10:13:13.866 INFO:teuthology.orchestra.run.smithi029.stdout: ms die on bug = true 2024-09-18T10:13:13.866 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-18T10:13:13.866 INFO:teuthology.orchestra.run.smithi029.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-18T10:13:13.866 INFO:teuthology.orchestra.run.smithi029.stdout: mon pg warn max object skew = 0 2024-09-18T10:13:13.866 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-18T10:13:13.866 INFO:teuthology.orchestra.run.smithi029.stdout: # disable pg_autoscaler by default for new pools 2024-09-18T10:13:13.866 INFO:teuthology.orchestra.run.smithi029.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-18T10:13:13.866 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-18T10:13:13.867 INFO:teuthology.orchestra.run.smithi029.stdout: osd pool default size = 2 2024-09-18T10:13:13.867 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-18T10:13:13.867 INFO:teuthology.orchestra.run.smithi029.stdout: mon osd allow primary affinity = true 2024-09-18T10:13:13.867 INFO:teuthology.orchestra.run.smithi029.stdout: mon osd allow pg remap = true 2024-09-18T10:13:13.867 INFO:teuthology.orchestra.run.smithi029.stdout: mon warn on legacy crush tunables = false 2024-09-18T10:13:13.867 INFO:teuthology.orchestra.run.smithi029.stdout: mon warn on crush straw calc version zero = false 2024-09-18T10:13:13.867 INFO:teuthology.orchestra.run.smithi029.stdout: mon warn on no sortbitwise = false 2024-09-18T10:13:13.867 INFO:teuthology.orchestra.run.smithi029.stdout: mon warn on osd down out interval zero = false 2024-09-18T10:13:13.867 INFO:teuthology.orchestra.run.smithi029.stdout: mon warn on too few osds = false 2024-09-18T10:13:13.867 INFO:teuthology.orchestra.run.smithi029.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-18T10:13:13.867 INFO:teuthology.orchestra.run.smithi029.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-18T10:13:13.867 INFO:teuthology.orchestra.run.smithi029.stdout: mon_allow_pool_size_one = true 2024-09-18T10:13:13.867 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-18T10:13:13.867 INFO:teuthology.orchestra.run.smithi029.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-18T10:13:13.868 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-18T10:13:13.868 INFO:teuthology.orchestra.run.smithi029.stdout: osd default data pool replay window = 5 2024-09-18T10:13:13.868 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-18T10:13:13.868 INFO:teuthology.orchestra.run.smithi029.stdout: mon allow pool delete = true 2024-09-18T10:13:13.868 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-18T10:13:13.868 INFO:teuthology.orchestra.run.smithi029.stdout: mon cluster log file level = debug 2024-09-18T10:13:13.868 INFO:teuthology.orchestra.run.smithi029.stdout: debug asserts on shutdown = true 2024-09-18T10:13:13.868 INFO:teuthology.orchestra.run.smithi029.stdout: mon health detail to clog = false 2024-09-18T10:13:13.868 INFO:teuthology.orchestra.run.smithi029.stdout: mon host = "172.21.15.29,172.21.15.136,172.21.15.154" 2024-09-18T10:13:13.868 INFO:teuthology.orchestra.run.smithi029.stdout: mon election default strategy = 1 2024-09-18T10:13:13.868 INFO:teuthology.orchestra.run.smithi029.stdout: osd debug inject dispatch delay duration = 0.1 2024-09-18T10:13:13.868 INFO:teuthology.orchestra.run.smithi029.stdout: osd debug inject dispatch delay probability = 0.1 2024-09-18T10:13:13.868 INFO:teuthology.orchestra.run.smithi029.stdout: fsid = ab6d6beb-0627-4b7f-8cdb-4e0a44970f10 2024-09-18T10:13:13.869 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-18T10:13:13.869 INFO:teuthology.orchestra.run.smithi029.stdout:[osd] 2024-09-18T10:13:13.869 INFO:teuthology.orchestra.run.smithi029.stdout: osd journal size = 100 2024-09-18T10:13:13.869 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-18T10:13:13.869 INFO:teuthology.orchestra.run.smithi029.stdout: osd scrub load threshold = 5.0 2024-09-18T10:13:13.869 INFO:teuthology.orchestra.run.smithi029.stdout: osd scrub max interval = 120 2024-09-18T10:13:13.869 INFO:teuthology.orchestra.run.smithi029.stdout: osd mclock profile = high_recovery_ops 2024-09-18T10:13:13.869 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-18T10:13:13.869 INFO:teuthology.orchestra.run.smithi029.stdout: osd recover clone overlap = true 2024-09-18T10:13:13.869 INFO:teuthology.orchestra.run.smithi029.stdout: osd recovery max chunk = 1048576 2024-09-18T10:13:13.869 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-18T10:13:13.869 INFO:teuthology.orchestra.run.smithi029.stdout: osd debug shutdown = true 2024-09-18T10:13:13.869 INFO:teuthology.orchestra.run.smithi029.stdout: osd debug op order = true 2024-09-18T10:13:13.869 INFO:teuthology.orchestra.run.smithi029.stdout: osd debug verify stray on activate = true 2024-09-18T10:13:13.870 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-18T10:13:13.870 INFO:teuthology.orchestra.run.smithi029.stdout: osd open classes on start = true 2024-09-18T10:13:13.870 INFO:teuthology.orchestra.run.smithi029.stdout: osd debug pg log writeout = true 2024-09-18T10:13:13.870 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-18T10:13:13.870 INFO:teuthology.orchestra.run.smithi029.stdout: osd deep scrub update digest min age = 30 2024-09-18T10:13:13.870 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-18T10:13:13.870 INFO:teuthology.orchestra.run.smithi029.stdout: osd map max advance = 10 2024-09-18T10:13:13.870 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-18T10:13:13.870 INFO:teuthology.orchestra.run.smithi029.stdout: journal zero on create = true 2024-09-18T10:13:13.870 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-18T10:13:13.871 INFO:teuthology.orchestra.run.smithi029.stdout: filestore ondisk finisher threads = 3 2024-09-18T10:13:13.871 INFO:teuthology.orchestra.run.smithi029.stdout: filestore apply finisher threads = 3 2024-09-18T10:13:13.871 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-18T10:13:13.871 INFO:teuthology.orchestra.run.smithi029.stdout: bdev debug aio = true 2024-09-18T10:13:13.871 INFO:teuthology.orchestra.run.smithi029.stdout: osd debug misdirected ops = true 2024-09-18T10:13:13.871 INFO:teuthology.orchestra.run.smithi029.stdout: debug monc = 20 2024-09-18T10:13:13.871 INFO:teuthology.orchestra.run.smithi029.stdout: bdev async discard = True 2024-09-18T10:13:13.871 INFO:teuthology.orchestra.run.smithi029.stdout: bdev enable discard = True 2024-09-18T10:13:13.871 INFO:teuthology.orchestra.run.smithi029.stdout: bluestore allocator = bitmap 2024-09-18T10:13:13.872 INFO:teuthology.orchestra.run.smithi029.stdout: bluestore block size = 96636764160 2024-09-18T10:13:13.872 INFO:teuthology.orchestra.run.smithi029.stdout: bluestore fsck on mount = True 2024-09-18T10:13:13.872 INFO:teuthology.orchestra.run.smithi029.stdout: bluestore zero block detection = True 2024-09-18T10:13:13.872 INFO:teuthology.orchestra.run.smithi029.stdout: debug bluefs = 20 2024-09-18T10:13:13.872 INFO:teuthology.orchestra.run.smithi029.stdout: debug bluestore = 20 2024-09-18T10:13:13.872 INFO:teuthology.orchestra.run.smithi029.stdout: debug ms = 1 2024-09-18T10:13:13.872 INFO:teuthology.orchestra.run.smithi029.stdout: debug osd = 20 2024-09-18T10:13:13.872 INFO:teuthology.orchestra.run.smithi029.stdout: debug rocksdb = 10 2024-09-18T10:13:13.872 INFO:teuthology.orchestra.run.smithi029.stdout: mon osd backfillfull_ratio = 0.85 2024-09-18T10:13:13.872 INFO:teuthology.orchestra.run.smithi029.stdout: mon osd full ratio = 0.9 2024-09-18T10:13:13.872 INFO:teuthology.orchestra.run.smithi029.stdout: mon osd nearfull ratio = 0.8 2024-09-18T10:13:13.872 INFO:teuthology.orchestra.run.smithi029.stdout: osd debug reject backfill probability = 0.3 2024-09-18T10:13:13.872 INFO:teuthology.orchestra.run.smithi029.stdout: osd debug verify cached snaps = True 2024-09-18T10:13:13.873 INFO:teuthology.orchestra.run.smithi029.stdout: osd debug verify missing on start = True 2024-09-18T10:13:13.873 INFO:teuthology.orchestra.run.smithi029.stdout: osd failsafe full ratio = 0.95 2024-09-18T10:13:13.873 INFO:teuthology.orchestra.run.smithi029.stdout: osd max backfills = 3 2024-09-18T10:13:13.873 INFO:teuthology.orchestra.run.smithi029.stdout: osd max markdown count = 1000 2024-09-18T10:13:13.873 INFO:teuthology.orchestra.run.smithi029.stdout: osd mclock override recovery settings = True 2024-09-18T10:13:13.873 INFO:teuthology.orchestra.run.smithi029.stdout: osd objectstore = bluestore 2024-09-18T10:13:13.873 INFO:teuthology.orchestra.run.smithi029.stdout: osd op queue = debug_random 2024-09-18T10:13:13.873 INFO:teuthology.orchestra.run.smithi029.stdout: osd op queue cut off = debug_random 2024-09-18T10:13:13.873 INFO:teuthology.orchestra.run.smithi029.stdout: osd scrub min interval = 60 2024-09-18T10:13:13.873 INFO:teuthology.orchestra.run.smithi029.stdout: osd shutdown pgref assert = True 2024-09-18T10:13:13.873 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-18T10:13:13.873 INFO:teuthology.orchestra.run.smithi029.stdout:[mgr] 2024-09-18T10:13:13.873 INFO:teuthology.orchestra.run.smithi029.stdout: debug ms = 1 2024-09-18T10:13:13.873 INFO:teuthology.orchestra.run.smithi029.stdout: debug mgr = 20 2024-09-18T10:13:13.874 INFO:teuthology.orchestra.run.smithi029.stdout: debug mon = 20 2024-09-18T10:13:13.874 INFO:teuthology.orchestra.run.smithi029.stdout: debug auth = 20 2024-09-18T10:13:13.874 INFO:teuthology.orchestra.run.smithi029.stdout: mon reweight min pgs per osd = 4 2024-09-18T10:13:13.874 INFO:teuthology.orchestra.run.smithi029.stdout: mon reweight min bytes per osd = 10 2024-09-18T10:13:13.874 INFO:teuthology.orchestra.run.smithi029.stdout: mgr/telemetry/nag = false 2024-09-18T10:13:13.874 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-18T10:13:13.874 INFO:teuthology.orchestra.run.smithi029.stdout:[mon] 2024-09-18T10:13:13.874 INFO:teuthology.orchestra.run.smithi029.stdout: debug ms = 1 2024-09-18T10:13:13.874 INFO:teuthology.orchestra.run.smithi029.stdout: debug mon = 20 2024-09-18T10:13:13.874 INFO:teuthology.orchestra.run.smithi029.stdout: debug paxos = 20 2024-09-18T10:13:13.874 INFO:teuthology.orchestra.run.smithi029.stdout: debug auth = 20 2024-09-18T10:13:13.874 INFO:teuthology.orchestra.run.smithi029.stdout: mon data avail warn = 5 2024-09-18T10:13:13.874 INFO:teuthology.orchestra.run.smithi029.stdout: mon mgr mkfs grace = 240 2024-09-18T10:13:13.874 INFO:teuthology.orchestra.run.smithi029.stdout: mon reweight min pgs per osd = 4 2024-09-18T10:13:13.875 INFO:teuthology.orchestra.run.smithi029.stdout: mon osd reporter subtree level = osd 2024-09-18T10:13:13.875 INFO:teuthology.orchestra.run.smithi029.stdout: mon osd prime pg temp = true 2024-09-18T10:13:13.875 INFO:teuthology.orchestra.run.smithi029.stdout: mon reweight min bytes per osd = 10 2024-09-18T10:13:13.875 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-18T10:13:13.875 INFO:teuthology.orchestra.run.smithi029.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-18T10:13:13.875 INFO:teuthology.orchestra.run.smithi029.stdout: auth mon ticket ttl = 660 # 11m 2024-09-18T10:13:13.875 INFO:teuthology.orchestra.run.smithi029.stdout: auth service ticket ttl = 240 # 4m 2024-09-18T10:13:13.875 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-18T10:13:13.875 INFO:teuthology.orchestra.run.smithi029.stdout: # don't complain about insecure global_id in the test suite 2024-09-18T10:13:13.875 INFO:teuthology.orchestra.run.smithi029.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-18T10:13:13.875 INFO:teuthology.orchestra.run.smithi029.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-18T10:13:13.875 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-18T10:13:13.875 INFO:teuthology.orchestra.run.smithi029.stdout: # 1m isn't quite enough 2024-09-18T10:13:13.875 INFO:teuthology.orchestra.run.smithi029.stdout: mon_down_mkfs_grace = 2m 2024-09-18T10:13:13.876 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-18T10:13:13.876 INFO:teuthology.orchestra.run.smithi029.stdout: mon_warn_on_filestore_osds = false 2024-09-18T10:13:13.876 INFO:teuthology.orchestra.run.smithi029.stdout: mon scrub interval = 300 2024-09-18T10:13:13.876 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-18T10:13:13.876 INFO:teuthology.orchestra.run.smithi029.stdout:[client] 2024-09-18T10:13:13.876 INFO:teuthology.orchestra.run.smithi029.stdout: rgw cache enabled = true 2024-09-18T10:13:13.876 INFO:teuthology.orchestra.run.smithi029.stdout: rgw enable ops log = true 2024-09-18T10:13:13.876 INFO:teuthology.orchestra.run.smithi029.stdout: rgw enable usage log = true 2024-09-18T10:13:13.876 INFO:teuthology.orchestra.run.smithi029.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-18T10:13:13.876 INFO:teuthology.orchestra.run.smithi029.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-18T10:13:13.876 INFO:teuthology.orchestra.run.smithi029.stdout:[mon.a] 2024-09-18T10:13:13.876 INFO:teuthology.orchestra.run.smithi029.stdout:[mon.b] 2024-09-18T10:13:13.876 INFO:teuthology.orchestra.run.smithi029.stdout:[mon.c] 2024-09-18T10:13:13.878 INFO:teuthology.orchestra.run.smithi136.stdout:[global] 2024-09-18T10:13:13.878 INFO:teuthology.orchestra.run.smithi136.stdout: chdir = "" 2024-09-18T10:13:13.878 INFO:teuthology.orchestra.run.smithi136.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-18T10:13:13.878 INFO:teuthology.orchestra.run.smithi136.stdout: auth supported = cephx 2024-09-18T10:13:13.878 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-18T10:13:13.878 INFO:teuthology.orchestra.run.smithi136.stdout: filestore xattr use omap = true 2024-09-18T10:13:13.878 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-18T10:13:13.878 INFO:teuthology.orchestra.run.smithi136.stdout: mon clock drift allowed = 1.000 2024-09-18T10:13:13.878 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-18T10:13:13.878 INFO:teuthology.orchestra.run.smithi136.stdout: osd crush chooseleaf type = 0 2024-09-18T10:13:13.878 INFO:teuthology.orchestra.run.smithi136.stdout: auth debug = true 2024-09-18T10:13:13.878 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-18T10:13:13.878 INFO:teuthology.orchestra.run.smithi136.stdout: ms die on old message = true 2024-09-18T10:13:13.879 INFO:teuthology.orchestra.run.smithi136.stdout: ms die on bug = true 2024-09-18T10:13:13.879 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-18T10:13:13.879 INFO:teuthology.orchestra.run.smithi136.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-18T10:13:13.879 INFO:teuthology.orchestra.run.smithi136.stdout: mon pg warn max object skew = 0 2024-09-18T10:13:13.879 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-18T10:13:13.879 INFO:teuthology.orchestra.run.smithi136.stdout: # disable pg_autoscaler by default for new pools 2024-09-18T10:13:13.879 INFO:teuthology.orchestra.run.smithi136.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-18T10:13:13.879 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-18T10:13:13.879 INFO:teuthology.orchestra.run.smithi136.stdout: osd pool default size = 2 2024-09-18T10:13:13.879 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-18T10:13:13.879 INFO:teuthology.orchestra.run.smithi136.stdout: mon osd allow primary affinity = true 2024-09-18T10:13:13.879 INFO:teuthology.orchestra.run.smithi136.stdout: mon osd allow pg remap = true 2024-09-18T10:13:13.879 INFO:teuthology.orchestra.run.smithi136.stdout: mon warn on legacy crush tunables = false 2024-09-18T10:13:13.880 INFO:teuthology.orchestra.run.smithi136.stdout: mon warn on crush straw calc version zero = false 2024-09-18T10:13:13.880 INFO:teuthology.orchestra.run.smithi136.stdout: mon warn on no sortbitwise = false 2024-09-18T10:13:13.880 INFO:teuthology.orchestra.run.smithi136.stdout: mon warn on osd down out interval zero = false 2024-09-18T10:13:13.880 INFO:teuthology.orchestra.run.smithi136.stdout: mon warn on too few osds = false 2024-09-18T10:13:13.880 INFO:teuthology.orchestra.run.smithi136.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-18T10:13:13.880 INFO:teuthology.orchestra.run.smithi136.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-18T10:13:13.880 INFO:teuthology.orchestra.run.smithi136.stdout: mon_allow_pool_size_one = true 2024-09-18T10:13:13.880 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-18T10:13:13.880 INFO:teuthology.orchestra.run.smithi136.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-18T10:13:13.880 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-18T10:13:13.880 INFO:teuthology.orchestra.run.smithi136.stdout: osd default data pool replay window = 5 2024-09-18T10:13:13.880 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-18T10:13:13.880 INFO:teuthology.orchestra.run.smithi136.stdout: mon allow pool delete = true 2024-09-18T10:13:13.880 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-18T10:13:13.881 INFO:teuthology.orchestra.run.smithi136.stdout: mon cluster log file level = debug 2024-09-18T10:13:13.881 INFO:teuthology.orchestra.run.smithi136.stdout: debug asserts on shutdown = true 2024-09-18T10:13:13.881 INFO:teuthology.orchestra.run.smithi136.stdout: mon health detail to clog = false 2024-09-18T10:13:13.881 INFO:teuthology.orchestra.run.smithi136.stdout: mon host = "172.21.15.29,172.21.15.136,172.21.15.154" 2024-09-18T10:13:13.881 INFO:teuthology.orchestra.run.smithi136.stdout: mon election default strategy = 1 2024-09-18T10:13:13.881 INFO:teuthology.orchestra.run.smithi136.stdout: osd debug inject dispatch delay duration = 0.1 2024-09-18T10:13:13.881 INFO:teuthology.orchestra.run.smithi136.stdout: osd debug inject dispatch delay probability = 0.1 2024-09-18T10:13:13.881 INFO:teuthology.orchestra.run.smithi136.stdout: fsid = ab6d6beb-0627-4b7f-8cdb-4e0a44970f10 2024-09-18T10:13:13.881 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-18T10:13:13.881 INFO:teuthology.orchestra.run.smithi136.stdout:[osd] 2024-09-18T10:13:13.881 INFO:teuthology.orchestra.run.smithi136.stdout: osd journal size = 100 2024-09-18T10:13:13.882 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-18T10:13:13.882 INFO:teuthology.orchestra.run.smithi136.stdout: osd scrub load threshold = 5.0 2024-09-18T10:13:13.882 INFO:teuthology.orchestra.run.smithi136.stdout: osd scrub max interval = 120 2024-09-18T10:13:13.882 INFO:teuthology.orchestra.run.smithi136.stdout: osd mclock profile = high_recovery_ops 2024-09-18T10:13:13.882 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-18T10:13:13.882 INFO:teuthology.orchestra.run.smithi136.stdout: osd recover clone overlap = true 2024-09-18T10:13:13.882 INFO:teuthology.orchestra.run.smithi136.stdout: osd recovery max chunk = 1048576 2024-09-18T10:13:13.882 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-18T10:13:13.882 INFO:teuthology.orchestra.run.smithi136.stdout: osd debug shutdown = true 2024-09-18T10:13:13.882 INFO:teuthology.orchestra.run.smithi136.stdout: osd debug op order = true 2024-09-18T10:13:13.882 INFO:teuthology.orchestra.run.smithi136.stdout: osd debug verify stray on activate = true 2024-09-18T10:13:13.882 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-18T10:13:13.882 INFO:teuthology.orchestra.run.smithi136.stdout: osd open classes on start = true 2024-09-18T10:13:13.882 INFO:teuthology.orchestra.run.smithi136.stdout: osd debug pg log writeout = true 2024-09-18T10:13:13.883 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-18T10:13:13.883 INFO:teuthology.orchestra.run.smithi136.stdout: osd deep scrub update digest min age = 30 2024-09-18T10:13:13.883 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-18T10:13:13.883 INFO:teuthology.orchestra.run.smithi136.stdout: osd map max advance = 10 2024-09-18T10:13:13.883 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-18T10:13:13.883 INFO:teuthology.orchestra.run.smithi136.stdout: journal zero on create = true 2024-09-18T10:13:13.883 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-18T10:13:13.883 INFO:teuthology.orchestra.run.smithi136.stdout: filestore ondisk finisher threads = 3 2024-09-18T10:13:13.883 INFO:teuthology.orchestra.run.smithi136.stdout: filestore apply finisher threads = 3 2024-09-18T10:13:13.883 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-18T10:13:13.883 INFO:teuthology.orchestra.run.smithi136.stdout: bdev debug aio = true 2024-09-18T10:13:13.883 INFO:teuthology.orchestra.run.smithi136.stdout: osd debug misdirected ops = true 2024-09-18T10:13:13.883 INFO:teuthology.orchestra.run.smithi136.stdout: debug monc = 20 2024-09-18T10:13:13.883 INFO:teuthology.orchestra.run.smithi136.stdout: bdev async discard = True 2024-09-18T10:13:13.884 INFO:teuthology.orchestra.run.smithi136.stdout: bdev enable discard = True 2024-09-18T10:13:13.884 INFO:teuthology.orchestra.run.smithi136.stdout: bluestore allocator = bitmap 2024-09-18T10:13:13.884 INFO:teuthology.orchestra.run.smithi136.stdout: bluestore block size = 96636764160 2024-09-18T10:13:13.884 INFO:teuthology.orchestra.run.smithi136.stdout: bluestore fsck on mount = True 2024-09-18T10:13:13.884 INFO:teuthology.orchestra.run.smithi136.stdout: bluestore zero block detection = True 2024-09-18T10:13:13.884 INFO:teuthology.orchestra.run.smithi136.stdout: debug bluefs = 20 2024-09-18T10:13:13.884 INFO:teuthology.orchestra.run.smithi136.stdout: debug bluestore = 20 2024-09-18T10:13:13.884 INFO:teuthology.orchestra.run.smithi136.stdout: debug ms = 1 2024-09-18T10:13:13.884 INFO:teuthology.orchestra.run.smithi136.stdout: debug osd = 20 2024-09-18T10:13:13.884 INFO:teuthology.orchestra.run.smithi136.stdout: debug rocksdb = 10 2024-09-18T10:13:13.884 INFO:teuthology.orchestra.run.smithi136.stdout: mon osd backfillfull_ratio = 0.85 2024-09-18T10:13:13.884 INFO:teuthology.orchestra.run.smithi136.stdout: mon osd full ratio = 0.9 2024-09-18T10:13:13.884 INFO:teuthology.orchestra.run.smithi136.stdout: mon osd nearfull ratio = 0.8 2024-09-18T10:13:13.884 INFO:teuthology.orchestra.run.smithi136.stdout: osd debug reject backfill probability = 0.3 2024-09-18T10:13:13.885 INFO:teuthology.orchestra.run.smithi136.stdout: osd debug verify cached snaps = True 2024-09-18T10:13:13.885 INFO:teuthology.orchestra.run.smithi136.stdout: osd debug verify missing on start = True 2024-09-18T10:13:13.885 INFO:teuthology.orchestra.run.smithi136.stdout: osd failsafe full ratio = 0.95 2024-09-18T10:13:13.885 INFO:teuthology.orchestra.run.smithi136.stdout: osd max backfills = 3 2024-09-18T10:13:13.885 INFO:teuthology.orchestra.run.smithi136.stdout: osd max markdown count = 1000 2024-09-18T10:13:13.885 INFO:teuthology.orchestra.run.smithi136.stdout: osd mclock override recovery settings = True 2024-09-18T10:13:13.885 INFO:teuthology.orchestra.run.smithi136.stdout: osd objectstore = bluestore 2024-09-18T10:13:13.885 INFO:teuthology.orchestra.run.smithi136.stdout: osd op queue = debug_random 2024-09-18T10:13:13.885 INFO:teuthology.orchestra.run.smithi136.stdout: osd op queue cut off = debug_random 2024-09-18T10:13:13.885 INFO:teuthology.orchestra.run.smithi136.stdout: osd scrub min interval = 60 2024-09-18T10:13:13.885 INFO:teuthology.orchestra.run.smithi136.stdout: osd shutdown pgref assert = True 2024-09-18T10:13:13.885 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-18T10:13:13.885 INFO:teuthology.orchestra.run.smithi136.stdout:[mgr] 2024-09-18T10:13:13.886 INFO:teuthology.orchestra.run.smithi136.stdout: debug ms = 1 2024-09-18T10:13:13.886 INFO:teuthology.orchestra.run.smithi136.stdout: debug mgr = 20 2024-09-18T10:13:13.886 INFO:teuthology.orchestra.run.smithi136.stdout: debug mon = 20 2024-09-18T10:13:13.886 INFO:teuthology.orchestra.run.smithi136.stdout: debug auth = 20 2024-09-18T10:13:13.886 INFO:teuthology.orchestra.run.smithi136.stdout: mon reweight min pgs per osd = 4 2024-09-18T10:13:13.886 INFO:teuthology.orchestra.run.smithi136.stdout: mon reweight min bytes per osd = 10 2024-09-18T10:13:13.886 INFO:teuthology.orchestra.run.smithi136.stdout: mgr/telemetry/nag = false 2024-09-18T10:13:13.886 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-18T10:13:13.886 INFO:teuthology.orchestra.run.smithi136.stdout:[mon] 2024-09-18T10:13:13.886 INFO:teuthology.orchestra.run.smithi136.stdout: debug ms = 1 2024-09-18T10:13:13.886 INFO:teuthology.orchestra.run.smithi136.stdout: debug mon = 20 2024-09-18T10:13:13.886 INFO:teuthology.orchestra.run.smithi136.stdout: debug paxos = 20 2024-09-18T10:13:13.886 INFO:teuthology.orchestra.run.smithi136.stdout: debug auth = 20 2024-09-18T10:13:13.886 INFO:teuthology.orchestra.run.smithi136.stdout: mon data avail warn = 5 2024-09-18T10:13:13.887 INFO:teuthology.orchestra.run.smithi136.stdout: mon mgr mkfs grace = 240 2024-09-18T10:13:13.887 INFO:teuthology.orchestra.run.smithi136.stdout: mon reweight min pgs per osd = 4 2024-09-18T10:13:13.887 INFO:teuthology.orchestra.run.smithi136.stdout: mon osd reporter subtree level = osd 2024-09-18T10:13:13.887 INFO:teuthology.orchestra.run.smithi136.stdout: mon osd prime pg temp = true 2024-09-18T10:13:13.887 INFO:teuthology.orchestra.run.smithi136.stdout: mon reweight min bytes per osd = 10 2024-09-18T10:13:13.887 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-18T10:13:13.887 INFO:teuthology.orchestra.run.smithi136.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-18T10:13:13.887 INFO:teuthology.orchestra.run.smithi136.stdout: auth mon ticket ttl = 660 # 11m 2024-09-18T10:13:13.887 INFO:teuthology.orchestra.run.smithi136.stdout: auth service ticket ttl = 240 # 4m 2024-09-18T10:13:13.887 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-18T10:13:13.887 INFO:teuthology.orchestra.run.smithi136.stdout: # don't complain about insecure global_id in the test suite 2024-09-18T10:13:13.887 INFO:teuthology.orchestra.run.smithi136.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-18T10:13:13.887 INFO:teuthology.orchestra.run.smithi136.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-18T10:13:13.887 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-18T10:13:13.888 INFO:teuthology.orchestra.run.smithi136.stdout: # 1m isn't quite enough 2024-09-18T10:13:13.888 INFO:teuthology.orchestra.run.smithi136.stdout: mon_down_mkfs_grace = 2m 2024-09-18T10:13:13.888 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-18T10:13:13.888 INFO:teuthology.orchestra.run.smithi136.stdout: mon_warn_on_filestore_osds = false 2024-09-18T10:13:13.888 INFO:teuthology.orchestra.run.smithi136.stdout: mon scrub interval = 300 2024-09-18T10:13:13.888 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-18T10:13:13.888 INFO:teuthology.orchestra.run.smithi136.stdout:[client] 2024-09-18T10:13:13.888 INFO:teuthology.orchestra.run.smithi136.stdout: rgw cache enabled = true 2024-09-18T10:13:13.888 INFO:teuthology.orchestra.run.smithi136.stdout: rgw enable ops log = true 2024-09-18T10:13:13.888 INFO:teuthology.orchestra.run.smithi136.stdout: rgw enable usage log = true 2024-09-18T10:13:13.888 INFO:teuthology.orchestra.run.smithi136.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-18T10:13:13.888 INFO:teuthology.orchestra.run.smithi136.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-18T10:13:13.888 INFO:teuthology.orchestra.run.smithi136.stdout:[mon.a] 2024-09-18T10:13:13.889 INFO:teuthology.orchestra.run.smithi136.stdout:[mon.b] 2024-09-18T10:13:13.889 INFO:teuthology.orchestra.run.smithi136.stdout:[mon.c] 2024-09-18T10:13:13.890 INFO:teuthology.orchestra.run.smithi189.stdout:[global] 2024-09-18T10:13:13.890 INFO:teuthology.orchestra.run.smithi189.stdout: chdir = "" 2024-09-18T10:13:13.890 INFO:teuthology.orchestra.run.smithi189.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-18T10:13:13.890 INFO:teuthology.orchestra.run.smithi189.stdout: auth supported = cephx 2024-09-18T10:13:13.890 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-18T10:13:13.890 INFO:teuthology.orchestra.run.smithi189.stdout: filestore xattr use omap = true 2024-09-18T10:13:13.890 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-18T10:13:13.890 INFO:teuthology.orchestra.run.smithi189.stdout: mon clock drift allowed = 1.000 2024-09-18T10:13:13.890 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-18T10:13:13.890 INFO:teuthology.orchestra.run.smithi189.stdout: osd crush chooseleaf type = 0 2024-09-18T10:13:13.891 INFO:teuthology.orchestra.run.smithi189.stdout: auth debug = true 2024-09-18T10:13:13.891 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-18T10:13:13.891 INFO:teuthology.orchestra.run.smithi189.stdout: ms die on old message = true 2024-09-18T10:13:13.891 INFO:teuthology.orchestra.run.smithi189.stdout: ms die on bug = true 2024-09-18T10:13:13.891 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-18T10:13:13.891 INFO:teuthology.orchestra.run.smithi189.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-18T10:13:13.891 INFO:teuthology.orchestra.run.smithi189.stdout: mon pg warn max object skew = 0 2024-09-18T10:13:13.891 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-18T10:13:13.891 INFO:teuthology.orchestra.run.smithi189.stdout: # disable pg_autoscaler by default for new pools 2024-09-18T10:13:13.891 INFO:teuthology.orchestra.run.smithi189.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-18T10:13:13.891 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-18T10:13:13.891 INFO:teuthology.orchestra.run.smithi189.stdout: osd pool default size = 2 2024-09-18T10:13:13.891 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-18T10:13:13.891 INFO:teuthology.orchestra.run.smithi189.stdout: mon osd allow primary affinity = true 2024-09-18T10:13:13.892 INFO:teuthology.orchestra.run.smithi189.stdout: mon osd allow pg remap = true 2024-09-18T10:13:13.892 INFO:teuthology.orchestra.run.smithi189.stdout: mon warn on legacy crush tunables = false 2024-09-18T10:13:13.892 INFO:teuthology.orchestra.run.smithi189.stdout: mon warn on crush straw calc version zero = false 2024-09-18T10:13:13.892 INFO:teuthology.orchestra.run.smithi189.stdout: mon warn on no sortbitwise = false 2024-09-18T10:13:13.892 INFO:teuthology.orchestra.run.smithi189.stdout: mon warn on osd down out interval zero = false 2024-09-18T10:13:13.892 INFO:teuthology.orchestra.run.smithi189.stdout: mon warn on too few osds = false 2024-09-18T10:13:13.892 INFO:teuthology.orchestra.run.smithi189.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-18T10:13:13.892 INFO:teuthology.orchestra.run.smithi189.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-18T10:13:13.892 INFO:teuthology.orchestra.run.smithi189.stdout: mon_allow_pool_size_one = true 2024-09-18T10:13:13.892 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-18T10:13:13.892 INFO:teuthology.orchestra.run.smithi189.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-18T10:13:13.892 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-18T10:13:13.892 INFO:teuthology.orchestra.run.smithi189.stdout: osd default data pool replay window = 5 2024-09-18T10:13:13.893 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-18T10:13:13.893 INFO:teuthology.orchestra.run.smithi189.stdout: mon allow pool delete = true 2024-09-18T10:13:13.893 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-18T10:13:13.893 INFO:teuthology.orchestra.run.smithi189.stdout: mon cluster log file level = debug 2024-09-18T10:13:13.893 INFO:teuthology.orchestra.run.smithi189.stdout: debug asserts on shutdown = true 2024-09-18T10:13:13.893 INFO:teuthology.orchestra.run.smithi189.stdout: mon health detail to clog = false 2024-09-18T10:13:13.893 INFO:teuthology.orchestra.run.smithi189.stdout: mon host = "172.21.15.29,172.21.15.136,172.21.15.154" 2024-09-18T10:13:13.893 INFO:teuthology.orchestra.run.smithi189.stdout: mon election default strategy = 1 2024-09-18T10:13:13.893 INFO:teuthology.orchestra.run.smithi189.stdout: osd debug inject dispatch delay duration = 0.1 2024-09-18T10:13:13.893 INFO:teuthology.orchestra.run.smithi189.stdout: osd debug inject dispatch delay probability = 0.1 2024-09-18T10:13:13.893 INFO:teuthology.orchestra.run.smithi189.stdout: fsid = ab6d6beb-0627-4b7f-8cdb-4e0a44970f10 2024-09-18T10:13:13.893 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-18T10:13:13.893 INFO:teuthology.orchestra.run.smithi189.stdout:[osd] 2024-09-18T10:13:13.893 INFO:teuthology.orchestra.run.smithi189.stdout: osd journal size = 100 2024-09-18T10:13:13.894 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-18T10:13:13.894 INFO:teuthology.orchestra.run.smithi189.stdout: osd scrub load threshold = 5.0 2024-09-18T10:13:13.894 INFO:teuthology.orchestra.run.smithi189.stdout: osd scrub max interval = 120 2024-09-18T10:13:13.894 INFO:teuthology.orchestra.run.smithi189.stdout: osd mclock profile = high_recovery_ops 2024-09-18T10:13:13.894 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-18T10:13:13.894 INFO:teuthology.orchestra.run.smithi189.stdout: osd recover clone overlap = true 2024-09-18T10:13:13.894 INFO:teuthology.orchestra.run.smithi189.stdout: osd recovery max chunk = 1048576 2024-09-18T10:13:13.894 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-18T10:13:13.894 INFO:teuthology.orchestra.run.smithi189.stdout: osd debug shutdown = true 2024-09-18T10:13:13.894 INFO:teuthology.orchestra.run.smithi189.stdout: osd debug op order = true 2024-09-18T10:13:13.894 INFO:teuthology.orchestra.run.smithi189.stdout: osd debug verify stray on activate = true 2024-09-18T10:13:13.894 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-18T10:13:13.894 INFO:teuthology.orchestra.run.smithi189.stdout: osd open classes on start = true 2024-09-18T10:13:13.894 INFO:teuthology.orchestra.run.smithi189.stdout: osd debug pg log writeout = true 2024-09-18T10:13:13.895 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-18T10:13:13.895 INFO:teuthology.orchestra.run.smithi189.stdout: osd deep scrub update digest min age = 30 2024-09-18T10:13:13.895 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-18T10:13:13.895 INFO:teuthology.orchestra.run.smithi189.stdout: osd map max advance = 10 2024-09-18T10:13:13.895 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-18T10:13:13.895 INFO:teuthology.orchestra.run.smithi189.stdout: journal zero on create = true 2024-09-18T10:13:13.895 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-18T10:13:13.895 INFO:teuthology.orchestra.run.smithi189.stdout: filestore ondisk finisher threads = 3 2024-09-18T10:13:13.895 INFO:teuthology.orchestra.run.smithi189.stdout: filestore apply finisher threads = 3 2024-09-18T10:13:13.895 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-18T10:13:13.895 INFO:teuthology.orchestra.run.smithi189.stdout: bdev debug aio = true 2024-09-18T10:13:13.895 INFO:teuthology.orchestra.run.smithi189.stdout: osd debug misdirected ops = true 2024-09-18T10:13:13.895 INFO:teuthology.orchestra.run.smithi189.stdout: debug monc = 20 2024-09-18T10:13:13.895 INFO:teuthology.orchestra.run.smithi189.stdout: bdev async discard = True 2024-09-18T10:13:13.895 INFO:teuthology.orchestra.run.smithi189.stdout: bdev enable discard = True 2024-09-18T10:13:13.896 INFO:teuthology.orchestra.run.smithi189.stdout: bluestore allocator = bitmap 2024-09-18T10:13:13.896 INFO:teuthology.orchestra.run.smithi189.stdout: bluestore block size = 96636764160 2024-09-18T10:13:13.896 INFO:teuthology.orchestra.run.smithi189.stdout: bluestore fsck on mount = True 2024-09-18T10:13:13.896 INFO:teuthology.orchestra.run.smithi189.stdout: bluestore zero block detection = True 2024-09-18T10:13:13.896 INFO:teuthology.orchestra.run.smithi189.stdout: debug bluefs = 20 2024-09-18T10:13:13.896 INFO:teuthology.orchestra.run.smithi189.stdout: debug bluestore = 20 2024-09-18T10:13:13.896 INFO:teuthology.orchestra.run.smithi189.stdout: debug ms = 1 2024-09-18T10:13:13.896 INFO:teuthology.orchestra.run.smithi189.stdout: debug osd = 20 2024-09-18T10:13:13.896 INFO:teuthology.orchestra.run.smithi189.stdout: debug rocksdb = 10 2024-09-18T10:13:13.896 INFO:teuthology.orchestra.run.smithi189.stdout: mon osd backfillfull_ratio = 0.85 2024-09-18T10:13:13.896 INFO:teuthology.orchestra.run.smithi189.stdout: mon osd full ratio = 0.9 2024-09-18T10:13:13.896 INFO:teuthology.orchestra.run.smithi189.stdout: mon osd nearfull ratio = 0.8 2024-09-18T10:13:13.896 INFO:teuthology.orchestra.run.smithi189.stdout: osd debug reject backfill probability = 0.3 2024-09-18T10:13:13.896 INFO:teuthology.orchestra.run.smithi189.stdout: osd debug verify cached snaps = True 2024-09-18T10:13:13.897 INFO:teuthology.orchestra.run.smithi189.stdout: osd debug verify missing on start = True 2024-09-18T10:13:13.897 INFO:teuthology.orchestra.run.smithi189.stdout: osd failsafe full ratio = 0.95 2024-09-18T10:13:13.897 INFO:teuthology.orchestra.run.smithi189.stdout: osd max backfills = 3 2024-09-18T10:13:13.897 INFO:teuthology.orchestra.run.smithi189.stdout: osd max markdown count = 1000 2024-09-18T10:13:13.897 INFO:teuthology.orchestra.run.smithi189.stdout: osd mclock override recovery settings = True 2024-09-18T10:13:13.897 INFO:teuthology.orchestra.run.smithi189.stdout: osd objectstore = bluestore 2024-09-18T10:13:13.897 INFO:teuthology.orchestra.run.smithi189.stdout: osd op queue = debug_random 2024-09-18T10:13:13.897 INFO:teuthology.orchestra.run.smithi189.stdout: osd op queue cut off = debug_random 2024-09-18T10:13:13.897 INFO:teuthology.orchestra.run.smithi189.stdout: osd scrub min interval = 60 2024-09-18T10:13:13.897 INFO:teuthology.orchestra.run.smithi189.stdout: osd shutdown pgref assert = True 2024-09-18T10:13:13.897 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-18T10:13:13.897 INFO:teuthology.orchestra.run.smithi189.stdout:[mgr] 2024-09-18T10:13:13.897 INFO:teuthology.orchestra.run.smithi189.stdout: debug ms = 1 2024-09-18T10:13:13.897 INFO:teuthology.orchestra.run.smithi189.stdout: debug mgr = 20 2024-09-18T10:13:13.898 INFO:teuthology.orchestra.run.smithi189.stdout: debug mon = 20 2024-09-18T10:13:13.898 INFO:teuthology.orchestra.run.smithi189.stdout: debug auth = 20 2024-09-18T10:13:13.898 INFO:teuthology.orchestra.run.smithi189.stdout: mon reweight min pgs per osd = 4 2024-09-18T10:13:13.898 INFO:teuthology.orchestra.run.smithi189.stdout: mon reweight min bytes per osd = 10 2024-09-18T10:13:13.898 INFO:teuthology.orchestra.run.smithi189.stdout: mgr/telemetry/nag = false 2024-09-18T10:13:13.898 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-18T10:13:13.898 INFO:teuthology.orchestra.run.smithi189.stdout:[mon] 2024-09-18T10:13:13.898 INFO:teuthology.orchestra.run.smithi189.stdout: debug ms = 1 2024-09-18T10:13:13.898 INFO:teuthology.orchestra.run.smithi189.stdout: debug mon = 20 2024-09-18T10:13:13.898 INFO:teuthology.orchestra.run.smithi189.stdout: debug paxos = 20 2024-09-18T10:13:13.898 INFO:teuthology.orchestra.run.smithi189.stdout: debug auth = 20 2024-09-18T10:13:13.898 INFO:teuthology.orchestra.run.smithi189.stdout: mon data avail warn = 5 2024-09-18T10:13:13.898 INFO:teuthology.orchestra.run.smithi189.stdout: mon mgr mkfs grace = 240 2024-09-18T10:13:13.898 INFO:teuthology.orchestra.run.smithi189.stdout: mon reweight min pgs per osd = 4 2024-09-18T10:13:13.898 INFO:teuthology.orchestra.run.smithi189.stdout: mon osd reporter subtree level = osd 2024-09-18T10:13:13.899 INFO:teuthology.orchestra.run.smithi189.stdout: mon osd prime pg temp = true 2024-09-18T10:13:13.899 INFO:teuthology.orchestra.run.smithi189.stdout: mon reweight min bytes per osd = 10 2024-09-18T10:13:13.899 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-18T10:13:13.899 INFO:teuthology.orchestra.run.smithi189.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-18T10:13:13.899 INFO:teuthology.orchestra.run.smithi189.stdout: auth mon ticket ttl = 660 # 11m 2024-09-18T10:13:13.899 INFO:teuthology.orchestra.run.smithi189.stdout: auth service ticket ttl = 240 # 4m 2024-09-18T10:13:13.899 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-18T10:13:13.899 INFO:teuthology.orchestra.run.smithi189.stdout: # don't complain about insecure global_id in the test suite 2024-09-18T10:13:13.899 INFO:teuthology.orchestra.run.smithi189.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-18T10:13:13.899 INFO:teuthology.orchestra.run.smithi189.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-18T10:13:13.899 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-18T10:13:13.899 INFO:teuthology.orchestra.run.smithi189.stdout: # 1m isn't quite enough 2024-09-18T10:13:13.899 INFO:teuthology.orchestra.run.smithi189.stdout: mon_down_mkfs_grace = 2m 2024-09-18T10:13:13.899 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-18T10:13:13.900 INFO:teuthology.orchestra.run.smithi189.stdout: mon_warn_on_filestore_osds = false 2024-09-18T10:13:13.900 INFO:teuthology.orchestra.run.smithi189.stdout: mon scrub interval = 300 2024-09-18T10:13:13.900 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-18T10:13:13.900 INFO:teuthology.orchestra.run.smithi189.stdout:[client] 2024-09-18T10:13:13.900 INFO:teuthology.orchestra.run.smithi189.stdout: rgw cache enabled = true 2024-09-18T10:13:13.900 INFO:teuthology.orchestra.run.smithi189.stdout: rgw enable ops log = true 2024-09-18T10:13:13.900 INFO:teuthology.orchestra.run.smithi189.stdout: rgw enable usage log = true 2024-09-18T10:13:13.900 INFO:teuthology.orchestra.run.smithi189.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-18T10:13:13.900 INFO:teuthology.orchestra.run.smithi189.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-18T10:13:13.900 INFO:teuthology.orchestra.run.smithi189.stdout:[mon.a] 2024-09-18T10:13:13.900 INFO:teuthology.orchestra.run.smithi189.stdout:[mon.b] 2024-09-18T10:13:13.900 INFO:teuthology.orchestra.run.smithi189.stdout:[mon.c] 2024-09-18T10:13:13.900 INFO:teuthology.orchestra.run.smithi154.stdout:[global] 2024-09-18T10:13:13.901 INFO:teuthology.orchestra.run.smithi154.stdout: chdir = "" 2024-09-18T10:13:13.901 INFO:teuthology.orchestra.run.smithi154.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-18T10:13:13.901 INFO:teuthology.orchestra.run.smithi154.stdout: auth supported = cephx 2024-09-18T10:13:13.901 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-09-18T10:13:13.901 INFO:teuthology.orchestra.run.smithi154.stdout: filestore xattr use omap = true 2024-09-18T10:13:13.901 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-09-18T10:13:13.901 INFO:teuthology.orchestra.run.smithi154.stdout: mon clock drift allowed = 1.000 2024-09-18T10:13:13.901 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-09-18T10:13:13.901 INFO:teuthology.orchestra.run.smithi154.stdout: osd crush chooseleaf type = 0 2024-09-18T10:13:13.901 INFO:teuthology.orchestra.run.smithi154.stdout: auth debug = true 2024-09-18T10:13:13.901 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-09-18T10:13:13.901 INFO:teuthology.orchestra.run.smithi154.stdout: ms die on old message = true 2024-09-18T10:13:13.901 INFO:teuthology.orchestra.run.smithi154.stdout: ms die on bug = true 2024-09-18T10:13:13.901 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-09-18T10:13:13.902 INFO:teuthology.orchestra.run.smithi154.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-18T10:13:13.902 INFO:teuthology.orchestra.run.smithi154.stdout: mon pg warn max object skew = 0 2024-09-18T10:13:13.902 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-09-18T10:13:13.902 INFO:teuthology.orchestra.run.smithi154.stdout: # disable pg_autoscaler by default for new pools 2024-09-18T10:13:13.902 INFO:teuthology.orchestra.run.smithi154.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-18T10:13:13.902 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-09-18T10:13:13.902 INFO:teuthology.orchestra.run.smithi154.stdout: osd pool default size = 2 2024-09-18T10:13:13.902 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-09-18T10:13:13.902 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd allow primary affinity = true 2024-09-18T10:13:13.902 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd allow pg remap = true 2024-09-18T10:13:13.902 INFO:teuthology.orchestra.run.smithi154.stdout: mon warn on legacy crush tunables = false 2024-09-18T10:13:13.902 INFO:teuthology.orchestra.run.smithi154.stdout: mon warn on crush straw calc version zero = false 2024-09-18T10:13:13.902 INFO:teuthology.orchestra.run.smithi154.stdout: mon warn on no sortbitwise = false 2024-09-18T10:13:13.902 INFO:teuthology.orchestra.run.smithi154.stdout: mon warn on osd down out interval zero = false 2024-09-18T10:13:13.903 INFO:teuthology.orchestra.run.smithi154.stdout: mon warn on too few osds = false 2024-09-18T10:13:13.903 INFO:teuthology.orchestra.run.smithi154.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-18T10:13:13.903 INFO:teuthology.orchestra.run.smithi154.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-18T10:13:13.903 INFO:teuthology.orchestra.run.smithi154.stdout: mon_allow_pool_size_one = true 2024-09-18T10:13:13.903 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-09-18T10:13:13.903 INFO:teuthology.orchestra.run.smithi154.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-18T10:13:13.903 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-09-18T10:13:13.903 INFO:teuthology.orchestra.run.smithi154.stdout: osd default data pool replay window = 5 2024-09-18T10:13:13.903 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-09-18T10:13:13.903 INFO:teuthology.orchestra.run.smithi154.stdout: mon allow pool delete = true 2024-09-18T10:13:13.903 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-09-18T10:13:13.903 INFO:teuthology.orchestra.run.smithi154.stdout: mon cluster log file level = debug 2024-09-18T10:13:13.903 INFO:teuthology.orchestra.run.smithi154.stdout: debug asserts on shutdown = true 2024-09-18T10:13:13.903 INFO:teuthology.orchestra.run.smithi154.stdout: mon health detail to clog = false 2024-09-18T10:13:13.904 INFO:teuthology.orchestra.run.smithi154.stdout: mon host = "172.21.15.29,172.21.15.136,172.21.15.154" 2024-09-18T10:13:13.904 INFO:teuthology.orchestra.run.smithi154.stdout: mon election default strategy = 1 2024-09-18T10:13:13.904 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug inject dispatch delay duration = 0.1 2024-09-18T10:13:13.904 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug inject dispatch delay probability = 0.1 2024-09-18T10:13:13.904 INFO:teuthology.orchestra.run.smithi154.stdout: fsid = ab6d6beb-0627-4b7f-8cdb-4e0a44970f10 2024-09-18T10:13:13.904 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-09-18T10:13:13.904 INFO:teuthology.orchestra.run.smithi154.stdout:[osd] 2024-09-18T10:13:13.904 INFO:teuthology.orchestra.run.smithi154.stdout: osd journal size = 100 2024-09-18T10:13:13.904 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-09-18T10:13:13.904 INFO:teuthology.orchestra.run.smithi154.stdout: osd scrub load threshold = 5.0 2024-09-18T10:13:13.904 INFO:teuthology.orchestra.run.smithi154.stdout: osd scrub max interval = 120 2024-09-18T10:13:13.904 INFO:teuthology.orchestra.run.smithi154.stdout: osd mclock profile = high_recovery_ops 2024-09-18T10:13:13.904 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-09-18T10:13:13.904 INFO:teuthology.orchestra.run.smithi154.stdout: osd recover clone overlap = true 2024-09-18T10:13:13.905 INFO:teuthology.orchestra.run.smithi154.stdout: osd recovery max chunk = 1048576 2024-09-18T10:13:13.905 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-09-18T10:13:13.905 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug shutdown = true 2024-09-18T10:13:13.905 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug op order = true 2024-09-18T10:13:13.905 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug verify stray on activate = true 2024-09-18T10:13:13.905 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-09-18T10:13:13.905 INFO:teuthology.orchestra.run.smithi154.stdout: osd open classes on start = true 2024-09-18T10:13:13.905 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug pg log writeout = true 2024-09-18T10:13:13.905 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-09-18T10:13:13.905 INFO:teuthology.orchestra.run.smithi154.stdout: osd deep scrub update digest min age = 30 2024-09-18T10:13:13.905 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-09-18T10:13:13.905 INFO:teuthology.orchestra.run.smithi154.stdout: osd map max advance = 10 2024-09-18T10:13:13.905 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-09-18T10:13:13.905 INFO:teuthology.orchestra.run.smithi154.stdout: journal zero on create = true 2024-09-18T10:13:13.906 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-09-18T10:13:13.906 INFO:teuthology.orchestra.run.smithi154.stdout: filestore ondisk finisher threads = 3 2024-09-18T10:13:13.906 INFO:teuthology.orchestra.run.smithi154.stdout: filestore apply finisher threads = 3 2024-09-18T10:13:13.906 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-09-18T10:13:13.906 INFO:teuthology.orchestra.run.smithi154.stdout: bdev debug aio = true 2024-09-18T10:13:13.906 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug misdirected ops = true 2024-09-18T10:13:13.906 INFO:teuthology.orchestra.run.smithi154.stdout: debug monc = 20 2024-09-18T10:13:13.906 INFO:teuthology.orchestra.run.smithi154.stdout: bdev async discard = True 2024-09-18T10:13:13.906 INFO:teuthology.orchestra.run.smithi154.stdout: bdev enable discard = True 2024-09-18T10:13:13.906 INFO:teuthology.orchestra.run.smithi154.stdout: bluestore allocator = bitmap 2024-09-18T10:13:13.906 INFO:teuthology.orchestra.run.smithi154.stdout: bluestore block size = 96636764160 2024-09-18T10:13:13.906 INFO:teuthology.orchestra.run.smithi154.stdout: bluestore fsck on mount = True 2024-09-18T10:13:13.906 INFO:teuthology.orchestra.run.smithi154.stdout: bluestore zero block detection = True 2024-09-18T10:13:13.906 INFO:teuthology.orchestra.run.smithi154.stdout: debug bluefs = 20 2024-09-18T10:13:13.907 INFO:teuthology.orchestra.run.smithi154.stdout: debug bluestore = 20 2024-09-18T10:13:13.907 INFO:teuthology.orchestra.run.smithi154.stdout: debug ms = 1 2024-09-18T10:13:13.907 INFO:teuthology.orchestra.run.smithi154.stdout: debug osd = 20 2024-09-18T10:13:13.907 INFO:teuthology.orchestra.run.smithi154.stdout: debug rocksdb = 10 2024-09-18T10:13:13.907 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd backfillfull_ratio = 0.85 2024-09-18T10:13:13.907 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd full ratio = 0.9 2024-09-18T10:13:13.907 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd nearfull ratio = 0.8 2024-09-18T10:13:13.907 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug reject backfill probability = 0.3 2024-09-18T10:13:13.907 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug verify cached snaps = True 2024-09-18T10:13:13.907 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug verify missing on start = True 2024-09-18T10:13:13.907 INFO:teuthology.orchestra.run.smithi154.stdout: osd failsafe full ratio = 0.95 2024-09-18T10:13:13.907 INFO:teuthology.orchestra.run.smithi154.stdout: osd max backfills = 3 2024-09-18T10:13:13.907 INFO:teuthology.orchestra.run.smithi154.stdout: osd max markdown count = 1000 2024-09-18T10:13:13.907 INFO:teuthology.orchestra.run.smithi154.stdout: osd mclock override recovery settings = True 2024-09-18T10:13:13.908 INFO:teuthology.orchestra.run.smithi154.stdout: osd objectstore = bluestore 2024-09-18T10:13:13.908 INFO:teuthology.orchestra.run.smithi154.stdout: osd op queue = debug_random 2024-09-18T10:13:13.908 INFO:teuthology.orchestra.run.smithi154.stdout: osd op queue cut off = debug_random 2024-09-18T10:13:13.908 INFO:teuthology.orchestra.run.smithi154.stdout: osd scrub min interval = 60 2024-09-18T10:13:13.908 INFO:teuthology.orchestra.run.smithi154.stdout: osd shutdown pgref assert = True 2024-09-18T10:13:13.908 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-09-18T10:13:13.908 INFO:teuthology.orchestra.run.smithi154.stdout:[mgr] 2024-09-18T10:13:13.908 INFO:teuthology.orchestra.run.smithi154.stdout: debug ms = 1 2024-09-18T10:13:13.908 INFO:teuthology.orchestra.run.smithi154.stdout: debug mgr = 20 2024-09-18T10:13:13.908 INFO:teuthology.orchestra.run.smithi154.stdout: debug mon = 20 2024-09-18T10:13:13.908 INFO:teuthology.orchestra.run.smithi154.stdout: debug auth = 20 2024-09-18T10:13:13.908 INFO:teuthology.orchestra.run.smithi154.stdout: mon reweight min pgs per osd = 4 2024-09-18T10:13:13.908 INFO:teuthology.orchestra.run.smithi154.stdout: mon reweight min bytes per osd = 10 2024-09-18T10:13:13.909 INFO:teuthology.orchestra.run.smithi154.stdout: mgr/telemetry/nag = false 2024-09-18T10:13:13.909 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-09-18T10:13:13.909 INFO:teuthology.orchestra.run.smithi154.stdout:[mon] 2024-09-18T10:13:13.909 INFO:teuthology.orchestra.run.smithi154.stdout: debug ms = 1 2024-09-18T10:13:13.909 INFO:teuthology.orchestra.run.smithi154.stdout: debug mon = 20 2024-09-18T10:13:13.909 INFO:teuthology.orchestra.run.smithi154.stdout: debug paxos = 20 2024-09-18T10:13:13.909 INFO:teuthology.orchestra.run.smithi154.stdout: debug auth = 20 2024-09-18T10:13:13.909 INFO:teuthology.orchestra.run.smithi154.stdout: mon data avail warn = 5 2024-09-18T10:13:13.909 INFO:teuthology.orchestra.run.smithi154.stdout: mon mgr mkfs grace = 240 2024-09-18T10:13:13.909 INFO:teuthology.orchestra.run.smithi154.stdout: mon reweight min pgs per osd = 4 2024-09-18T10:13:13.909 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd reporter subtree level = osd 2024-09-18T10:13:13.909 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd prime pg temp = true 2024-09-18T10:13:13.909 INFO:teuthology.orchestra.run.smithi154.stdout: mon reweight min bytes per osd = 10 2024-09-18T10:13:13.909 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-09-18T10:13:13.910 INFO:teuthology.orchestra.run.smithi154.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-18T10:13:13.910 INFO:teuthology.orchestra.run.smithi154.stdout: auth mon ticket ttl = 660 # 11m 2024-09-18T10:13:13.910 INFO:teuthology.orchestra.run.smithi154.stdout: auth service ticket ttl = 240 # 4m 2024-09-18T10:13:13.910 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-09-18T10:13:13.910 INFO:teuthology.orchestra.run.smithi154.stdout: # don't complain about insecure global_id in the test suite 2024-09-18T10:13:13.910 INFO:teuthology.orchestra.run.smithi154.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-18T10:13:13.910 INFO:teuthology.orchestra.run.smithi154.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-18T10:13:13.910 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-09-18T10:13:13.910 INFO:teuthology.orchestra.run.smithi154.stdout: # 1m isn't quite enough 2024-09-18T10:13:13.910 INFO:teuthology.orchestra.run.smithi154.stdout: mon_down_mkfs_grace = 2m 2024-09-18T10:13:13.910 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-09-18T10:13:13.910 INFO:teuthology.orchestra.run.smithi154.stdout: mon_warn_on_filestore_osds = false 2024-09-18T10:13:13.910 INFO:teuthology.orchestra.run.smithi154.stdout: mon scrub interval = 300 2024-09-18T10:13:13.910 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-09-18T10:13:13.911 INFO:teuthology.orchestra.run.smithi154.stdout:[client] 2024-09-18T10:13:13.911 INFO:teuthology.orchestra.run.smithi154.stdout: rgw cache enabled = true 2024-09-18T10:13:13.911 INFO:teuthology.orchestra.run.smithi154.stdout: rgw enable ops log = true 2024-09-18T10:13:13.911 INFO:teuthology.orchestra.run.smithi154.stdout: rgw enable usage log = true 2024-09-18T10:13:13.911 INFO:teuthology.orchestra.run.smithi154.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-18T10:13:13.911 INFO:teuthology.orchestra.run.smithi154.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-18T10:13:13.911 INFO:teuthology.orchestra.run.smithi154.stdout:[mon.a] 2024-09-18T10:13:13.911 INFO:teuthology.orchestra.run.smithi154.stdout:[mon.b] 2024-09-18T10:13:13.911 INFO:teuthology.orchestra.run.smithi154.stdout:[mon.c] 2024-09-18T10:13:13.912 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-18T10:13:13.912 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-18T10:13:13.982 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-18T10:13:13.982 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-09-18T10:13:13.982 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-18T10:13:13.989 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-09-18T10:13:13.989 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-18T10:13:13.994 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi029.front.sepia.ceph.com 2024-09-18T10:13:13.994 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-09-18T10:13:13.994 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-18T10:13:13.994 DEBUG:teuthology.orchestra.run.smithi029:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-18T10:13:14.016 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-09-18T10:13:14.016 DEBUG:teuthology.orchestra.run.smithi029:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-18T10:13:14.023 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi136.front.sepia.ceph.com 2024-09-18T10:13:14.023 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-09-18T10:13:14.023 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-18T10:13:14.023 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-18T10:13:14.049 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-09-18T10:13:14.049 DEBUG:teuthology.orchestra.run.smithi136:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-18T10:13:14.056 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi154.front.sepia.ceph.com 2024-09-18T10:13:14.056 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-09-18T10:13:14.056 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-18T10:13:14.056 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-18T10:13:14.083 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-09-18T10:13:14.083 DEBUG:teuthology.orchestra.run.smithi154:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-18T10:13:14.087 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi189.front.sepia.ceph.com 2024-09-18T10:13:14.088 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-18T10:13:14.088 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-18T10:13:14.088 DEBUG:teuthology.orchestra.run.smithi189:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-18T10:13:14.116 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-18T10:13:14.116 DEBUG:teuthology.orchestra.run.smithi189:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-18T10:13:14.166 INFO:tasks.ceph:Setting up mon nodes... 2024-09-18T10:13:14.166 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-18T10:13:14.166 DEBUG:teuthology.orchestra.run.smithi029:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-09-18T10:13:14.212 INFO:teuthology.orchestra.run.smithi029.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-09-18T10:13:14.215 DEBUG:teuthology.orchestra.run.smithi189:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-18T10:13:14.260 INFO:teuthology.orchestra.run.smithi189.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-18T10:13:14.263 INFO:tasks.ceph:Setting up mds nodes... 2024-09-18T10:13:14.263 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-18T10:13:14.263 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-18T10:13:14.296 INFO:teuthology.orchestra.run.smithi189.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-18T10:13:14.309 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-18T10:13:14.310 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi029.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.8': '/dev/vg_nvme/lv_3', 'osd.12': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi136.front.sepia.ceph.com'): {'osd.1': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.9': '/dev/vg_nvme/lv_3', 'osd.13': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi154.front.sepia.ceph.com'): {'osd.2': '/dev/vg_nvme/lv_1', 'osd.6': '/dev/vg_nvme/lv_2', 'osd.10': '/dev/vg_nvme/lv_3', 'osd.14': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi189.front.sepia.ceph.com'): {'osd.3': '/dev/vg_nvme/lv_1', 'osd.7': '/dev/vg_nvme/lv_2', 'osd.11': '/dev/vg_nvme/lv_3', 'osd.15': '/dev/vg_nvme/lv_4'}} 2024-09-18T10:13:14.310 DEBUG:teuthology.orchestra.run.smithi029:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-18T10:13:14.327 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.8': '/dev/vg_nvme/lv_3', 'osd.12': '/dev/vg_nvme/lv_4'} 2024-09-18T10:13:14.327 INFO:tasks.ceph:role: osd.0 2024-09-18T10:13:14.327 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi029.front.sepia.ceph.com 2024-09-18T10:13:14.328 DEBUG:teuthology.orchestra.run.smithi029:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-18T10:13:14.371 INFO:teuthology.orchestra.run.smithi029.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-18T10:13:14.371 INFO:teuthology.orchestra.run.smithi029.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-18T10:13:14.371 INFO:teuthology.orchestra.run.smithi029.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-18T10:13:14.371 INFO:teuthology.orchestra.run.smithi029.stdout: = reflink=1 2024-09-18T10:13:14.371 INFO:teuthology.orchestra.run.smithi029.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-18T10:13:14.372 INFO:teuthology.orchestra.run.smithi029.stdout: = sunit=0 swidth=0 blks 2024-09-18T10:13:14.372 INFO:teuthology.orchestra.run.smithi029.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-18T10:13:14.372 INFO:teuthology.orchestra.run.smithi029.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-18T10:13:14.372 INFO:teuthology.orchestra.run.smithi029.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-18T10:13:14.372 INFO:teuthology.orchestra.run.smithi029.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-18T10:13:14.373 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi029.front.sepia.ceph.com -o noatime 2024-09-18T10:13:14.373 DEBUG:teuthology.orchestra.run.smithi029:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-18T10:13:14.393 DEBUG:teuthology.orchestra.run.smithi029:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-18T10:13:14.403 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-18T10:13:14.403 INFO:teuthology.orchestra.run.smithi029.stderr:sudo: /sbin/restorecon: command not found 2024-09-18T10:13:14.403 DEBUG:teuthology.orchestra.run.smithi029:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-18T10:13:14.413 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.8': '/dev/vg_nvme/lv_3', 'osd.12': '/dev/vg_nvme/lv_4'} 2024-09-18T10:13:14.413 INFO:tasks.ceph:role: osd.4 2024-09-18T10:13:14.413 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi029.front.sepia.ceph.com 2024-09-18T10:13:14.413 DEBUG:teuthology.orchestra.run.smithi029:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-18T10:13:14.452 INFO:teuthology.orchestra.run.smithi029.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-18T10:13:14.452 INFO:teuthology.orchestra.run.smithi029.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-18T10:13:14.452 INFO:teuthology.orchestra.run.smithi029.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-18T10:13:14.452 INFO:teuthology.orchestra.run.smithi029.stdout: = reflink=1 2024-09-18T10:13:14.452 INFO:teuthology.orchestra.run.smithi029.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-18T10:13:14.452 INFO:teuthology.orchestra.run.smithi029.stdout: = sunit=0 swidth=0 blks 2024-09-18T10:13:14.453 INFO:teuthology.orchestra.run.smithi029.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-18T10:13:14.453 INFO:teuthology.orchestra.run.smithi029.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-18T10:13:14.453 INFO:teuthology.orchestra.run.smithi029.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-18T10:13:14.453 INFO:teuthology.orchestra.run.smithi029.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-18T10:13:14.454 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi029.front.sepia.ceph.com -o noatime 2024-09-18T10:13:14.454 DEBUG:teuthology.orchestra.run.smithi029:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-4 2024-09-18T10:13:14.473 DEBUG:teuthology.orchestra.run.smithi029:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-18T10:13:14.528 INFO:teuthology.orchestra.run.smithi029.stderr:sudo: /sbin/restorecon: command not found 2024-09-18T10:13:14.529 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-18T10:13:14.529 DEBUG:teuthology.orchestra.run.smithi029:> sudo mkdir -p /var/lib/ceph/osd/ceph-8 2024-09-18T10:13:14.545 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.8': '/dev/vg_nvme/lv_3', 'osd.12': '/dev/vg_nvme/lv_4'} 2024-09-18T10:13:14.545 INFO:tasks.ceph:role: osd.8 2024-09-18T10:13:14.545 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi029.front.sepia.ceph.com 2024-09-18T10:13:14.545 DEBUG:teuthology.orchestra.run.smithi029:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-18T10:13:14.588 INFO:teuthology.orchestra.run.smithi029.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-18T10:13:14.588 INFO:teuthology.orchestra.run.smithi029.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-18T10:13:14.588 INFO:teuthology.orchestra.run.smithi029.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-18T10:13:14.588 INFO:teuthology.orchestra.run.smithi029.stdout: = reflink=1 2024-09-18T10:13:14.588 INFO:teuthology.orchestra.run.smithi029.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-18T10:13:14.588 INFO:teuthology.orchestra.run.smithi029.stdout: = sunit=0 swidth=0 blks 2024-09-18T10:13:14.589 INFO:teuthology.orchestra.run.smithi029.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-18T10:13:14.589 INFO:teuthology.orchestra.run.smithi029.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-18T10:13:14.589 INFO:teuthology.orchestra.run.smithi029.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-18T10:13:14.589 INFO:teuthology.orchestra.run.smithi029.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-18T10:13:14.590 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi029.front.sepia.ceph.com -o noatime 2024-09-18T10:13:14.590 DEBUG:teuthology.orchestra.run.smithi029:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-8 2024-09-18T10:13:14.612 DEBUG:teuthology.orchestra.run.smithi029:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-8 2024-09-18T10:13:14.624 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-18T10:13:14.624 INFO:teuthology.orchestra.run.smithi029.stderr:sudo: /sbin/restorecon: command not found 2024-09-18T10:13:14.625 DEBUG:teuthology.orchestra.run.smithi029:> sudo mkdir -p /var/lib/ceph/osd/ceph-12 2024-09-18T10:13:14.684 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.8': '/dev/vg_nvme/lv_3', 'osd.12': '/dev/vg_nvme/lv_4'} 2024-09-18T10:13:14.684 INFO:tasks.ceph:role: osd.12 2024-09-18T10:13:14.684 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi029.front.sepia.ceph.com 2024-09-18T10:13:14.684 DEBUG:teuthology.orchestra.run.smithi029:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-18T10:13:14.727 INFO:teuthology.orchestra.run.smithi029.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-18T10:13:14.727 INFO:teuthology.orchestra.run.smithi029.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-18T10:13:14.727 INFO:teuthology.orchestra.run.smithi029.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-18T10:13:14.727 INFO:teuthology.orchestra.run.smithi029.stdout: = reflink=1 2024-09-18T10:13:14.727 INFO:teuthology.orchestra.run.smithi029.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-18T10:13:14.727 INFO:teuthology.orchestra.run.smithi029.stdout: = sunit=0 swidth=0 blks 2024-09-18T10:13:14.727 INFO:teuthology.orchestra.run.smithi029.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-18T10:13:14.727 INFO:teuthology.orchestra.run.smithi029.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-18T10:13:14.727 INFO:teuthology.orchestra.run.smithi029.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-18T10:13:14.728 INFO:teuthology.orchestra.run.smithi029.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-18T10:13:14.729 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi029.front.sepia.ceph.com -o noatime 2024-09-18T10:13:14.729 DEBUG:teuthology.orchestra.run.smithi029:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-12 2024-09-18T10:13:14.748 DEBUG:teuthology.orchestra.run.smithi029:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-12 2024-09-18T10:13:14.804 INFO:teuthology.orchestra.run.smithi029.stderr:sudo: /sbin/restorecon: command not found 2024-09-18T10:13:14.805 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-18T10:13:14.805 DEBUG:teuthology.orchestra.run.smithi029:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-18T10:13:14.885 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-18T10:13:14.881+0000 7fde7fc25540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-18T10:13:14.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-18T10:13:14.881+0000 7fde7fc25540 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-18T10:13:14.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-18T10:13:14.881+0000 7fde7fc25540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-18T10:13:14.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-18T10:13:14.881+0000 7fde7fc25540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-18T10:13:14.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-18T10:13:14.881+0000 7fde7fc25540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-18T10:13:14.886 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-18T10:13:14.881+0000 7fde7fc25540 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-18T10:13:15.438 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-18T10:13:15.433+0000 7fde7fc25540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-18T10:13:16.220 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-18T10:13:16.213+0000 7fde7fc25540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-18T10:13:17.001 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-18T10:13:16.997+0000 7fde7fc25540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-18T10:13:17.869 DEBUG:teuthology.orchestra.run.smithi029:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-18T10:13:17.885 DEBUG:teuthology.orchestra.run.smithi029:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-18T10:13:17.971 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-18T10:13:17.965+0000 7fddb7ace540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-18T10:13:17.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-18T10:13:17.965+0000 7fddb7ace540 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-18T10:13:17.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-18T10:13:17.965+0000 7fddb7ace540 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-09-18T10:13:17.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-18T10:13:17.965+0000 7fddb7ace540 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-09-18T10:13:17.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-18T10:13:17.965+0000 7fddb7ace540 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-09-18T10:13:17.972 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-18T10:13:17.965+0000 7fddb7ace540 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-18T10:13:18.534 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-18T10:13:18.529+0000 7fddb7ace540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-18T10:13:19.325 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-18T10:13:19.321+0000 7fddb7ace540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-18T10:13:20.121 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-18T10:13:20.116+0000 7fddb7ace540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-18T10:13:20.996 DEBUG:teuthology.orchestra.run.smithi029:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-18T10:13:21.012 DEBUG:teuthology.orchestra.run.smithi029:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 8 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-18T10:13:21.054 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-18T10:13:21.048+0000 7f1282b25540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-8/keyring: can't open /var/lib/ceph/osd/ceph-8/keyring: (2) No such file or directory 2024-09-18T10:13:21.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-18T10:13:21.048+0000 7f1282b25540 -1 created new key in keyring /var/lib/ceph/osd/ceph-8/keyring 2024-09-18T10:13:21.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-18T10:13:21.048+0000 7f1282b25540 -1 bluestore(/var/lib/ceph/osd/ceph-8/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-8/block: (2) No such file or directory 2024-09-18T10:13:21.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-18T10:13:21.048+0000 7f1282b25540 -1 bluestore(/var/lib/ceph/osd/ceph-8/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-8/block: (2) No such file or directory 2024-09-18T10:13:21.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-18T10:13:21.048+0000 7f1282b25540 -1 bluestore(/var/lib/ceph/osd/ceph-8/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-8/block: (2) No such file or directory 2024-09-18T10:13:21.055 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-18T10:13:21.048+0000 7f1282b25540 -1 bluestore(/var/lib/ceph/osd/ceph-8) _read_fsid unparsable uuid 2024-09-18T10:13:21.619 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-18T10:13:21.612+0000 7f1282b25540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-18T10:13:22.409 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-18T10:13:22.404+0000 7f1282b25540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-18T10:13:23.190 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-18T10:13:23.184+0000 7f1282b25540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-18T10:13:24.048 DEBUG:teuthology.orchestra.run.smithi029:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-8 2024-09-18T10:13:24.104 DEBUG:teuthology.orchestra.run.smithi029:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 12 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-18T10:13:24.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-18T10:13:24.140+0000 7fda0f8de540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-12/keyring: can't open /var/lib/ceph/osd/ceph-12/keyring: (2) No such file or directory 2024-09-18T10:13:24.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-18T10:13:24.140+0000 7fda0f8de540 -1 created new key in keyring /var/lib/ceph/osd/ceph-12/keyring 2024-09-18T10:13:24.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-18T10:13:24.140+0000 7fda0f8de540 -1 bluestore(/var/lib/ceph/osd/ceph-12/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-12/block: (2) No such file or directory 2024-09-18T10:13:24.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-18T10:13:24.140+0000 7fda0f8de540 -1 bluestore(/var/lib/ceph/osd/ceph-12/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-12/block: (2) No such file or directory 2024-09-18T10:13:24.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-18T10:13:24.140+0000 7fda0f8de540 -1 bluestore(/var/lib/ceph/osd/ceph-12/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-12/block: (2) No such file or directory 2024-09-18T10:13:24.146 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-18T10:13:24.140+0000 7fda0f8de540 -1 bluestore(/var/lib/ceph/osd/ceph-12) _read_fsid unparsable uuid 2024-09-18T10:13:24.715 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-18T10:13:24.708+0000 7fda0f8de540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-18T10:13:25.506 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-18T10:13:25.500+0000 7fda0f8de540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-18T10:13:26.286 INFO:teuthology.orchestra.run.smithi029.stderr:2024-09-18T10:13:26.280+0000 7fda0f8de540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-18T10:13:27.135 DEBUG:teuthology.orchestra.run.smithi029:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-12 2024-09-18T10:13:27.151 DEBUG:teuthology.orchestra.run.smithi136:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-18T10:13:27.168 INFO:tasks.ceph:roles_to_devs: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.9': '/dev/vg_nvme/lv_3', 'osd.13': '/dev/vg_nvme/lv_4'} 2024-09-18T10:13:27.168 INFO:tasks.ceph:role: osd.1 2024-09-18T10:13:27.168 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi136.front.sepia.ceph.com 2024-09-18T10:13:27.168 DEBUG:teuthology.orchestra.run.smithi136:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-18T10:13:27.821 INFO:teuthology.orchestra.run.smithi136.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-18T10:13:27.821 INFO:teuthology.orchestra.run.smithi136.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-18T10:13:27.821 INFO:teuthology.orchestra.run.smithi136.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-18T10:13:27.822 INFO:teuthology.orchestra.run.smithi136.stdout: = reflink=1 2024-09-18T10:13:27.822 INFO:teuthology.orchestra.run.smithi136.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-18T10:13:27.822 INFO:teuthology.orchestra.run.smithi136.stdout: = sunit=0 swidth=0 blks 2024-09-18T10:13:27.822 INFO:teuthology.orchestra.run.smithi136.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-18T10:13:27.822 INFO:teuthology.orchestra.run.smithi136.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-18T10:13:27.822 INFO:teuthology.orchestra.run.smithi136.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-18T10:13:27.822 INFO:teuthology.orchestra.run.smithi136.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-18T10:13:27.824 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi136.front.sepia.ceph.com -o noatime 2024-09-18T10:13:27.824 DEBUG:teuthology.orchestra.run.smithi136:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-1 2024-09-18T10:13:27.894 DEBUG:teuthology.orchestra.run.smithi136:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-18T10:13:27.904 INFO:teuthology.orchestra.run.smithi136.stderr:sudo: /sbin/restorecon: command not found 2024-09-18T10:13:27.904 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-18T10:13:27.905 DEBUG:teuthology.orchestra.run.smithi136:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-18T10:13:27.964 INFO:tasks.ceph:roles_to_devs: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.9': '/dev/vg_nvme/lv_3', 'osd.13': '/dev/vg_nvme/lv_4'} 2024-09-18T10:13:27.964 INFO:tasks.ceph:role: osd.5 2024-09-18T10:13:27.964 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi136.front.sepia.ceph.com 2024-09-18T10:13:27.964 DEBUG:teuthology.orchestra.run.smithi136:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-18T10:13:28.420 INFO:teuthology.orchestra.run.smithi136.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-18T10:13:28.420 INFO:teuthology.orchestra.run.smithi136.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-18T10:13:28.420 INFO:teuthology.orchestra.run.smithi136.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-18T10:13:28.421 INFO:teuthology.orchestra.run.smithi136.stdout: = reflink=1 2024-09-18T10:13:28.421 INFO:teuthology.orchestra.run.smithi136.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-18T10:13:28.421 INFO:teuthology.orchestra.run.smithi136.stdout: = sunit=0 swidth=0 blks 2024-09-18T10:13:28.421 INFO:teuthology.orchestra.run.smithi136.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-18T10:13:28.421 INFO:teuthology.orchestra.run.smithi136.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-18T10:13:28.422 INFO:teuthology.orchestra.run.smithi136.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-18T10:13:28.422 INFO:teuthology.orchestra.run.smithi136.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-18T10:13:28.423 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi136.front.sepia.ceph.com -o noatime 2024-09-18T10:13:28.424 DEBUG:teuthology.orchestra.run.smithi136:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-09-18T10:13:28.449 DEBUG:teuthology.orchestra.run.smithi136:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-18T10:13:28.504 INFO:teuthology.orchestra.run.smithi136.stderr:sudo: /sbin/restorecon: command not found 2024-09-18T10:13:28.505 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-18T10:13:28.506 DEBUG:teuthology.orchestra.run.smithi136:> sudo mkdir -p /var/lib/ceph/osd/ceph-9 2024-09-18T10:13:28.517 INFO:tasks.ceph:roles_to_devs: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.9': '/dev/vg_nvme/lv_3', 'osd.13': '/dev/vg_nvme/lv_4'} 2024-09-18T10:13:28.517 INFO:tasks.ceph:role: osd.9 2024-09-18T10:13:28.517 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi136.front.sepia.ceph.com 2024-09-18T10:13:28.517 DEBUG:teuthology.orchestra.run.smithi136:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-18T10:13:29.136 INFO:teuthology.orchestra.run.smithi136.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-18T10:13:29.136 INFO:teuthology.orchestra.run.smithi136.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-18T10:13:29.136 INFO:teuthology.orchestra.run.smithi136.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-18T10:13:29.136 INFO:teuthology.orchestra.run.smithi136.stdout: = reflink=1 2024-09-18T10:13:29.137 INFO:teuthology.orchestra.run.smithi136.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-18T10:13:29.137 INFO:teuthology.orchestra.run.smithi136.stdout: = sunit=0 swidth=0 blks 2024-09-18T10:13:29.137 INFO:teuthology.orchestra.run.smithi136.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-18T10:13:29.137 INFO:teuthology.orchestra.run.smithi136.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-18T10:13:29.137 INFO:teuthology.orchestra.run.smithi136.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-18T10:13:29.137 INFO:teuthology.orchestra.run.smithi136.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-18T10:13:29.138 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi136.front.sepia.ceph.com -o noatime 2024-09-18T10:13:29.139 DEBUG:teuthology.orchestra.run.smithi136:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-9 2024-09-18T10:13:29.204 DEBUG:teuthology.orchestra.run.smithi136:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-9 2024-09-18T10:13:29.215 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-18T10:13:29.216 INFO:teuthology.orchestra.run.smithi136.stderr:sudo: /sbin/restorecon: command not found 2024-09-18T10:13:29.216 DEBUG:teuthology.orchestra.run.smithi136:> sudo mkdir -p /var/lib/ceph/osd/ceph-13 2024-09-18T10:13:29.275 INFO:tasks.ceph:roles_to_devs: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.9': '/dev/vg_nvme/lv_3', 'osd.13': '/dev/vg_nvme/lv_4'} 2024-09-18T10:13:29.276 INFO:tasks.ceph:role: osd.13 2024-09-18T10:13:29.276 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi136.front.sepia.ceph.com 2024-09-18T10:13:29.276 DEBUG:teuthology.orchestra.run.smithi136:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-18T10:13:29.875 INFO:teuthology.orchestra.run.smithi136.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-18T10:13:29.875 INFO:teuthology.orchestra.run.smithi136.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-18T10:13:29.875 INFO:teuthology.orchestra.run.smithi136.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-18T10:13:29.875 INFO:teuthology.orchestra.run.smithi136.stdout: = reflink=1 2024-09-18T10:13:29.875 INFO:teuthology.orchestra.run.smithi136.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-18T10:13:29.875 INFO:teuthology.orchestra.run.smithi136.stdout: = sunit=0 swidth=0 blks 2024-09-18T10:13:29.875 INFO:teuthology.orchestra.run.smithi136.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-18T10:13:29.875 INFO:teuthology.orchestra.run.smithi136.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-18T10:13:29.875 INFO:teuthology.orchestra.run.smithi136.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-18T10:13:29.875 INFO:teuthology.orchestra.run.smithi136.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-18T10:13:29.877 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi136.front.sepia.ceph.com -o noatime 2024-09-18T10:13:29.877 DEBUG:teuthology.orchestra.run.smithi136:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-13 2024-09-18T10:13:29.939 DEBUG:teuthology.orchestra.run.smithi136:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-13 2024-09-18T10:13:29.950 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-18T10:13:29.950 INFO:teuthology.orchestra.run.smithi136.stderr:sudo: /sbin/restorecon: command not found 2024-09-18T10:13:29.951 DEBUG:teuthology.orchestra.run.smithi136:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-18T10:13:30.070 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-18T10:13:30.068+0000 7f4a6575a540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-18T10:13:30.073 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-18T10:13:30.072+0000 7f4a6575a540 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-18T10:13:30.073 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-18T10:13:30.072+0000 7f4a6575a540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-18T10:13:30.073 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-18T10:13:30.072+0000 7f4a6575a540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-18T10:13:30.073 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-18T10:13:30.072+0000 7f4a6575a540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-18T10:13:30.074 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-18T10:13:30.072+0000 7f4a6575a540 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-18T10:13:30.640 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-18T10:13:30.636+0000 7f4a6575a540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-18T10:13:31.419 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-18T10:13:31.416+0000 7f4a6575a540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-18T10:13:32.202 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-18T10:13:32.200+0000 7f4a6575a540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-18T10:13:33.048 DEBUG:teuthology.orchestra.run.smithi136:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-18T10:13:33.065 DEBUG:teuthology.orchestra.run.smithi136:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-18T10:13:33.103 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-18T10:13:33.100+0000 7f8eac780540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-18T10:13:33.104 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-18T10:13:33.100+0000 7f8eac780540 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-18T10:13:33.104 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-18T10:13:33.100+0000 7f8eac780540 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-09-18T10:13:33.104 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-18T10:13:33.100+0000 7f8eac780540 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-09-18T10:13:33.104 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-18T10:13:33.100+0000 7f8eac780540 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-09-18T10:13:33.104 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-18T10:13:33.104+0000 7f8eac780540 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-18T10:13:33.658 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-18T10:13:33.656+0000 7f8eac780540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-18T10:13:34.443 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-18T10:13:34.440+0000 7f8eac780540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-18T10:13:35.222 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-18T10:13:35.220+0000 7f8eac780540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-18T10:13:36.085 DEBUG:teuthology.orchestra.run.smithi136:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-18T10:13:36.101 DEBUG:teuthology.orchestra.run.smithi136:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 9 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-18T10:13:36.184 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-18T10:13:36.184+0000 7fc1cb572540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-9/keyring: can't open /var/lib/ceph/osd/ceph-9/keyring: (2) No such file or directory 2024-09-18T10:13:36.185 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-18T10:13:36.184+0000 7fc1cb572540 -1 created new key in keyring /var/lib/ceph/osd/ceph-9/keyring 2024-09-18T10:13:36.185 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-18T10:13:36.184+0000 7fc1cb572540 -1 bluestore(/var/lib/ceph/osd/ceph-9/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-9/block: (2) No such file or directory 2024-09-18T10:13:36.185 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-18T10:13:36.184+0000 7fc1cb572540 -1 bluestore(/var/lib/ceph/osd/ceph-9/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-9/block: (2) No such file or directory 2024-09-18T10:13:36.185 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-18T10:13:36.184+0000 7fc1cb572540 -1 bluestore(/var/lib/ceph/osd/ceph-9/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-9/block: (2) No such file or directory 2024-09-18T10:13:36.186 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-18T10:13:36.184+0000 7fc1cb572540 -1 bluestore(/var/lib/ceph/osd/ceph-9) _read_fsid unparsable uuid 2024-09-18T10:13:36.744 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-18T10:13:36.740+0000 7fc1cb572540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-18T10:13:37.533 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-18T10:13:37.532+0000 7fc1cb572540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-18T10:13:38.315 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-18T10:13:38.312+0000 7fc1cb572540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-18T10:13:39.160 DEBUG:teuthology.orchestra.run.smithi136:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-9 2024-09-18T10:13:39.175 DEBUG:teuthology.orchestra.run.smithi136:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 13 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-18T10:13:39.211 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-18T10:13:39.208+0000 7f1ad3373540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-13/keyring: can't open /var/lib/ceph/osd/ceph-13/keyring: (2) No such file or directory 2024-09-18T10:13:39.212 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-18T10:13:39.208+0000 7f1ad3373540 -1 created new key in keyring /var/lib/ceph/osd/ceph-13/keyring 2024-09-18T10:13:39.212 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-18T10:13:39.208+0000 7f1ad3373540 -1 bluestore(/var/lib/ceph/osd/ceph-13/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-13/block: (2) No such file or directory 2024-09-18T10:13:39.212 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-18T10:13:39.208+0000 7f1ad3373540 -1 bluestore(/var/lib/ceph/osd/ceph-13/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-13/block: (2) No such file or directory 2024-09-18T10:13:39.212 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-18T10:13:39.208+0000 7f1ad3373540 -1 bluestore(/var/lib/ceph/osd/ceph-13/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-13/block: (2) No such file or directory 2024-09-18T10:13:39.212 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-18T10:13:39.212+0000 7f1ad3373540 -1 bluestore(/var/lib/ceph/osd/ceph-13) _read_fsid unparsable uuid 2024-09-18T10:13:39.769 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-18T10:13:39.768+0000 7f1ad3373540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-18T10:13:40.550 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-18T10:13:40.548+0000 7f1ad3373540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-18T10:13:41.331 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-18T10:13:41.328+0000 7f1ad3373540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-18T10:13:42.188 DEBUG:teuthology.orchestra.run.smithi136:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-13 2024-09-18T10:13:42.205 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-18T10:13:42.221 INFO:tasks.ceph:roles_to_devs: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.6': '/dev/vg_nvme/lv_2', 'osd.10': '/dev/vg_nvme/lv_3', 'osd.14': '/dev/vg_nvme/lv_4'} 2024-09-18T10:13:42.221 INFO:tasks.ceph:role: osd.2 2024-09-18T10:13:42.222 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi154.front.sepia.ceph.com 2024-09-18T10:13:42.222 DEBUG:teuthology.orchestra.run.smithi154:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-18T10:13:42.735 INFO:teuthology.orchestra.run.smithi154.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-18T10:13:42.735 INFO:teuthology.orchestra.run.smithi154.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-18T10:13:42.735 INFO:teuthology.orchestra.run.smithi154.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-18T10:13:42.735 INFO:teuthology.orchestra.run.smithi154.stdout: = reflink=1 2024-09-18T10:13:42.735 INFO:teuthology.orchestra.run.smithi154.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-18T10:13:42.735 INFO:teuthology.orchestra.run.smithi154.stdout: = sunit=0 swidth=0 blks 2024-09-18T10:13:42.735 INFO:teuthology.orchestra.run.smithi154.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-18T10:13:42.735 INFO:teuthology.orchestra.run.smithi154.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-18T10:13:42.735 INFO:teuthology.orchestra.run.smithi154.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-18T10:13:42.736 INFO:teuthology.orchestra.run.smithi154.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-18T10:13:42.737 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi154.front.sepia.ceph.com -o noatime 2024-09-18T10:13:42.737 DEBUG:teuthology.orchestra.run.smithi154:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-2 2024-09-18T10:13:42.820 DEBUG:teuthology.orchestra.run.smithi154:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-18T10:13:42.833 INFO:teuthology.orchestra.run.smithi154.stderr:sudo: /sbin/restorecon: command not found 2024-09-18T10:13:42.833 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-18T10:13:42.834 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-09-18T10:13:42.843 INFO:tasks.ceph:roles_to_devs: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.6': '/dev/vg_nvme/lv_2', 'osd.10': '/dev/vg_nvme/lv_3', 'osd.14': '/dev/vg_nvme/lv_4'} 2024-09-18T10:13:42.844 INFO:tasks.ceph:role: osd.6 2024-09-18T10:13:42.844 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi154.front.sepia.ceph.com 2024-09-18T10:13:42.844 DEBUG:teuthology.orchestra.run.smithi154:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-18T10:13:43.373 INFO:teuthology.orchestra.run.smithi154.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-18T10:13:43.374 INFO:teuthology.orchestra.run.smithi154.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-18T10:13:43.374 INFO:teuthology.orchestra.run.smithi154.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-18T10:13:43.374 INFO:teuthology.orchestra.run.smithi154.stdout: = reflink=1 2024-09-18T10:13:43.374 INFO:teuthology.orchestra.run.smithi154.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-18T10:13:43.374 INFO:teuthology.orchestra.run.smithi154.stdout: = sunit=0 swidth=0 blks 2024-09-18T10:13:43.374 INFO:teuthology.orchestra.run.smithi154.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-18T10:13:43.374 INFO:teuthology.orchestra.run.smithi154.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-18T10:13:43.374 INFO:teuthology.orchestra.run.smithi154.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-18T10:13:43.374 INFO:teuthology.orchestra.run.smithi154.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-18T10:13:43.375 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi154.front.sepia.ceph.com -o noatime 2024-09-18T10:13:43.376 DEBUG:teuthology.orchestra.run.smithi154:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-6 2024-09-18T10:13:43.437 DEBUG:teuthology.orchestra.run.smithi154:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-09-18T10:13:43.447 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-18T10:13:43.447 INFO:teuthology.orchestra.run.smithi154.stderr:sudo: /sbin/restorecon: command not found 2024-09-18T10:13:43.448 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /var/lib/ceph/osd/ceph-10 2024-09-18T10:13:43.457 INFO:tasks.ceph:roles_to_devs: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.6': '/dev/vg_nvme/lv_2', 'osd.10': '/dev/vg_nvme/lv_3', 'osd.14': '/dev/vg_nvme/lv_4'} 2024-09-18T10:13:43.457 INFO:tasks.ceph:role: osd.10 2024-09-18T10:13:43.457 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi154.front.sepia.ceph.com 2024-09-18T10:13:43.457 DEBUG:teuthology.orchestra.run.smithi154:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-18T10:13:43.914 INFO:teuthology.orchestra.run.smithi154.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-18T10:13:43.914 INFO:teuthology.orchestra.run.smithi154.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-18T10:13:43.914 INFO:teuthology.orchestra.run.smithi154.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-18T10:13:43.914 INFO:teuthology.orchestra.run.smithi154.stdout: = reflink=1 2024-09-18T10:13:43.914 INFO:teuthology.orchestra.run.smithi154.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-18T10:13:43.914 INFO:teuthology.orchestra.run.smithi154.stdout: = sunit=0 swidth=0 blks 2024-09-18T10:13:43.914 INFO:teuthology.orchestra.run.smithi154.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-18T10:13:43.915 INFO:teuthology.orchestra.run.smithi154.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-18T10:13:43.915 INFO:teuthology.orchestra.run.smithi154.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-18T10:13:43.915 INFO:teuthology.orchestra.run.smithi154.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-18T10:13:43.916 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi154.front.sepia.ceph.com -o noatime 2024-09-18T10:13:43.916 DEBUG:teuthology.orchestra.run.smithi154:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-10 2024-09-18T10:13:43.961 DEBUG:teuthology.orchestra.run.smithi154:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-10 2024-09-18T10:13:43.971 INFO:teuthology.orchestra.run.smithi154.stderr:sudo: /sbin/restorecon: command not found 2024-09-18T10:13:43.971 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-18T10:13:43.972 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /var/lib/ceph/osd/ceph-14 2024-09-18T10:13:43.983 INFO:tasks.ceph:roles_to_devs: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.6': '/dev/vg_nvme/lv_2', 'osd.10': '/dev/vg_nvme/lv_3', 'osd.14': '/dev/vg_nvme/lv_4'} 2024-09-18T10:13:43.983 INFO:tasks.ceph:role: osd.14 2024-09-18T10:13:43.984 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi154.front.sepia.ceph.com 2024-09-18T10:13:43.984 DEBUG:teuthology.orchestra.run.smithi154:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-18T10:13:44.508 INFO:teuthology.orchestra.run.smithi154.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-18T10:13:44.509 INFO:teuthology.orchestra.run.smithi154.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-18T10:13:44.509 INFO:teuthology.orchestra.run.smithi154.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-18T10:13:44.509 INFO:teuthology.orchestra.run.smithi154.stdout: = reflink=1 2024-09-18T10:13:44.509 INFO:teuthology.orchestra.run.smithi154.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-18T10:13:44.509 INFO:teuthology.orchestra.run.smithi154.stdout: = sunit=0 swidth=0 blks 2024-09-18T10:13:44.509 INFO:teuthology.orchestra.run.smithi154.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-18T10:13:44.510 INFO:teuthology.orchestra.run.smithi154.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-18T10:13:44.510 INFO:teuthology.orchestra.run.smithi154.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-18T10:13:44.510 INFO:teuthology.orchestra.run.smithi154.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-18T10:13:44.511 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi154.front.sepia.ceph.com -o noatime 2024-09-18T10:13:44.512 DEBUG:teuthology.orchestra.run.smithi154:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-14 2024-09-18T10:13:44.564 DEBUG:teuthology.orchestra.run.smithi154:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-14 2024-09-18T10:13:44.576 INFO:teuthology.orchestra.run.smithi154.stderr:sudo: /sbin/restorecon: command not found 2024-09-18T10:13:44.577 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-18T10:13:44.578 DEBUG:teuthology.orchestra.run.smithi154:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-18T10:13:44.656 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-18T10:13:44.654+0000 7f8f5bd80540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-18T10:13:44.660 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-18T10:13:44.658+0000 7f8f5bd80540 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-18T10:13:44.660 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-18T10:13:44.658+0000 7f8f5bd80540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-18T10:13:44.660 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-18T10:13:44.658+0000 7f8f5bd80540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-18T10:13:44.660 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-18T10:13:44.658+0000 7f8f5bd80540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-18T10:13:44.661 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-18T10:13:44.658+0000 7f8f5bd80540 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-18T10:13:45.223 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-18T10:13:45.222+0000 7f8f5bd80540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-18T10:13:45.998 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-18T10:13:45.994+0000 7f8f5bd80540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-18T10:13:46.788 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-18T10:13:46.786+0000 7f8f5bd80540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-18T10:13:47.630 DEBUG:teuthology.orchestra.run.smithi154:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-18T10:13:47.647 DEBUG:teuthology.orchestra.run.smithi154:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-18T10:13:47.685 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-18T10:13:47.682+0000 7f78ef95c540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-09-18T10:13:47.685 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-18T10:13:47.682+0000 7f78ef95c540 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-09-18T10:13:47.685 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-18T10:13:47.682+0000 7f78ef95c540 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-09-18T10:13:47.686 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-18T10:13:47.682+0000 7f78ef95c540 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-09-18T10:13:47.686 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-18T10:13:47.682+0000 7f78ef95c540 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-09-18T10:13:47.686 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-18T10:13:47.682+0000 7f78ef95c540 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-09-18T10:13:48.241 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-18T10:13:48.238+0000 7f78ef95c540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-18T10:13:49.022 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-18T10:13:49.018+0000 7f78ef95c540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-18T10:13:49.804 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-18T10:13:49.802+0000 7f78ef95c540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-18T10:13:50.646 DEBUG:teuthology.orchestra.run.smithi154:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-09-18T10:13:50.706 DEBUG:teuthology.orchestra.run.smithi154:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 10 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-18T10:13:50.742 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-18T10:13:50.742+0000 7f240dd52540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-10/keyring: can't open /var/lib/ceph/osd/ceph-10/keyring: (2) No such file or directory 2024-09-18T10:13:50.743 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-18T10:13:50.742+0000 7f240dd52540 -1 created new key in keyring /var/lib/ceph/osd/ceph-10/keyring 2024-09-18T10:13:50.743 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-18T10:13:50.742+0000 7f240dd52540 -1 bluestore(/var/lib/ceph/osd/ceph-10/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-10/block: (2) No such file or directory 2024-09-18T10:13:50.743 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-18T10:13:50.742+0000 7f240dd52540 -1 bluestore(/var/lib/ceph/osd/ceph-10/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-10/block: (2) No such file or directory 2024-09-18T10:13:50.743 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-18T10:13:50.742+0000 7f240dd52540 -1 bluestore(/var/lib/ceph/osd/ceph-10/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-10/block: (2) No such file or directory 2024-09-18T10:13:50.744 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-18T10:13:50.742+0000 7f240dd52540 -1 bluestore(/var/lib/ceph/osd/ceph-10) _read_fsid unparsable uuid 2024-09-18T10:13:51.294 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-18T10:13:51.290+0000 7f240dd52540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-18T10:13:52.079 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-18T10:13:52.078+0000 7f240dd52540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-18T10:13:52.860 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-18T10:13:52.858+0000 7f240dd52540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-18T10:13:53.710 DEBUG:teuthology.orchestra.run.smithi154:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-10 2024-09-18T10:13:53.727 DEBUG:teuthology.orchestra.run.smithi154:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 14 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-18T10:13:53.810 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-18T10:13:53.806+0000 7fc132784540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-14/keyring: can't open /var/lib/ceph/osd/ceph-14/keyring: (2) No such file or directory 2024-09-18T10:13:53.810 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-18T10:13:53.806+0000 7fc132784540 -1 created new key in keyring /var/lib/ceph/osd/ceph-14/keyring 2024-09-18T10:13:53.810 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-18T10:13:53.806+0000 7fc132784540 -1 bluestore(/var/lib/ceph/osd/ceph-14/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-14/block: (2) No such file or directory 2024-09-18T10:13:53.810 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-18T10:13:53.806+0000 7fc132784540 -1 bluestore(/var/lib/ceph/osd/ceph-14/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-14/block: (2) No such file or directory 2024-09-18T10:13:53.811 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-18T10:13:53.806+0000 7fc132784540 -1 bluestore(/var/lib/ceph/osd/ceph-14/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-14/block: (2) No such file or directory 2024-09-18T10:13:53.811 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-18T10:13:53.810+0000 7fc132784540 -1 bluestore(/var/lib/ceph/osd/ceph-14) _read_fsid unparsable uuid 2024-09-18T10:13:54.367 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-18T10:13:54.366+0000 7fc132784540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-18T10:13:55.151 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-18T10:13:55.150+0000 7fc132784540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-18T10:13:55.936 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-18T10:13:55.934+0000 7fc132784540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-18T10:13:56.786 DEBUG:teuthology.orchestra.run.smithi154:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-14 2024-09-18T10:13:56.802 DEBUG:teuthology.orchestra.run.smithi189:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-18T10:13:56.819 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.7': '/dev/vg_nvme/lv_2', 'osd.11': '/dev/vg_nvme/lv_3', 'osd.15': '/dev/vg_nvme/lv_4'} 2024-09-18T10:13:56.819 INFO:tasks.ceph:role: osd.3 2024-09-18T10:13:56.819 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi189.front.sepia.ceph.com 2024-09-18T10:13:56.820 DEBUG:teuthology.orchestra.run.smithi189:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-18T10:13:57.345 INFO:teuthology.orchestra.run.smithi189.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-18T10:13:57.345 INFO:teuthology.orchestra.run.smithi189.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-18T10:13:57.346 INFO:teuthology.orchestra.run.smithi189.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-18T10:13:57.346 INFO:teuthology.orchestra.run.smithi189.stdout: = reflink=1 2024-09-18T10:13:57.346 INFO:teuthology.orchestra.run.smithi189.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-18T10:13:57.346 INFO:teuthology.orchestra.run.smithi189.stdout: = sunit=0 swidth=0 blks 2024-09-18T10:13:57.346 INFO:teuthology.orchestra.run.smithi189.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-18T10:13:57.346 INFO:teuthology.orchestra.run.smithi189.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-18T10:13:57.346 INFO:teuthology.orchestra.run.smithi189.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-18T10:13:57.346 INFO:teuthology.orchestra.run.smithi189.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-18T10:13:57.347 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi189.front.sepia.ceph.com -o noatime 2024-09-18T10:13:57.347 DEBUG:teuthology.orchestra.run.smithi189:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-3 2024-09-18T10:13:57.439 DEBUG:teuthology.orchestra.run.smithi189:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-18T10:13:57.452 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-18T10:13:57.452 INFO:teuthology.orchestra.run.smithi189.stderr:sudo: /sbin/restorecon: command not found 2024-09-18T10:13:57.453 DEBUG:teuthology.orchestra.run.smithi189:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-09-18T10:13:57.467 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.7': '/dev/vg_nvme/lv_2', 'osd.11': '/dev/vg_nvme/lv_3', 'osd.15': '/dev/vg_nvme/lv_4'} 2024-09-18T10:13:57.467 INFO:tasks.ceph:role: osd.7 2024-09-18T10:13:57.467 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi189.front.sepia.ceph.com 2024-09-18T10:13:57.467 DEBUG:teuthology.orchestra.run.smithi189:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-18T10:13:58.028 INFO:teuthology.orchestra.run.smithi189.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-18T10:13:58.028 INFO:teuthology.orchestra.run.smithi189.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-18T10:13:58.028 INFO:teuthology.orchestra.run.smithi189.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-18T10:13:58.029 INFO:teuthology.orchestra.run.smithi189.stdout: = reflink=1 2024-09-18T10:13:58.029 INFO:teuthology.orchestra.run.smithi189.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-18T10:13:58.029 INFO:teuthology.orchestra.run.smithi189.stdout: = sunit=0 swidth=0 blks 2024-09-18T10:13:58.029 INFO:teuthology.orchestra.run.smithi189.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-18T10:13:58.029 INFO:teuthology.orchestra.run.smithi189.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-18T10:13:58.029 INFO:teuthology.orchestra.run.smithi189.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-18T10:13:58.029 INFO:teuthology.orchestra.run.smithi189.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-18T10:13:58.030 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi189.front.sepia.ceph.com -o noatime 2024-09-18T10:13:58.030 DEBUG:teuthology.orchestra.run.smithi189:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-7 2024-09-18T10:13:58.065 DEBUG:teuthology.orchestra.run.smithi189:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-09-18T10:13:58.076 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-18T10:13:58.077 INFO:teuthology.orchestra.run.smithi189.stderr:sudo: /sbin/restorecon: command not found 2024-09-18T10:13:58.077 DEBUG:teuthology.orchestra.run.smithi189:> sudo mkdir -p /var/lib/ceph/osd/ceph-11 2024-09-18T10:13:58.088 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.7': '/dev/vg_nvme/lv_2', 'osd.11': '/dev/vg_nvme/lv_3', 'osd.15': '/dev/vg_nvme/lv_4'} 2024-09-18T10:13:58.088 INFO:tasks.ceph:role: osd.11 2024-09-18T10:13:58.088 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi189.front.sepia.ceph.com 2024-09-18T10:13:58.089 DEBUG:teuthology.orchestra.run.smithi189:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-18T10:13:58.540 INFO:teuthology.orchestra.run.smithi189.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-18T10:13:58.541 INFO:teuthology.orchestra.run.smithi189.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-18T10:13:58.541 INFO:teuthology.orchestra.run.smithi189.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-18T10:13:58.541 INFO:teuthology.orchestra.run.smithi189.stdout: = reflink=1 2024-09-18T10:13:58.541 INFO:teuthology.orchestra.run.smithi189.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-18T10:13:58.541 INFO:teuthology.orchestra.run.smithi189.stdout: = sunit=0 swidth=0 blks 2024-09-18T10:13:58.541 INFO:teuthology.orchestra.run.smithi189.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-18T10:13:58.541 INFO:teuthology.orchestra.run.smithi189.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-18T10:13:58.541 INFO:teuthology.orchestra.run.smithi189.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-18T10:13:58.541 INFO:teuthology.orchestra.run.smithi189.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-18T10:13:58.542 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi189.front.sepia.ceph.com -o noatime 2024-09-18T10:13:58.542 DEBUG:teuthology.orchestra.run.smithi189:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-11 2024-09-18T10:13:58.595 DEBUG:teuthology.orchestra.run.smithi189:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-11 2024-09-18T10:13:58.605 INFO:teuthology.orchestra.run.smithi189.stderr:sudo: /sbin/restorecon: command not found 2024-09-18T10:13:58.606 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-18T10:13:58.606 DEBUG:teuthology.orchestra.run.smithi189:> sudo mkdir -p /var/lib/ceph/osd/ceph-15 2024-09-18T10:13:58.616 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.7': '/dev/vg_nvme/lv_2', 'osd.11': '/dev/vg_nvme/lv_3', 'osd.15': '/dev/vg_nvme/lv_4'} 2024-09-18T10:13:58.617 INFO:tasks.ceph:role: osd.15 2024-09-18T10:13:58.617 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi189.front.sepia.ceph.com 2024-09-18T10:13:58.617 DEBUG:teuthology.orchestra.run.smithi189:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-18T10:13:59.126 INFO:teuthology.orchestra.run.smithi189.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-18T10:13:59.126 INFO:teuthology.orchestra.run.smithi189.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-18T10:13:59.126 INFO:teuthology.orchestra.run.smithi189.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-18T10:13:59.126 INFO:teuthology.orchestra.run.smithi189.stdout: = reflink=1 2024-09-18T10:13:59.126 INFO:teuthology.orchestra.run.smithi189.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-18T10:13:59.126 INFO:teuthology.orchestra.run.smithi189.stdout: = sunit=0 swidth=0 blks 2024-09-18T10:13:59.126 INFO:teuthology.orchestra.run.smithi189.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-18T10:13:59.126 INFO:teuthology.orchestra.run.smithi189.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-18T10:13:59.126 INFO:teuthology.orchestra.run.smithi189.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-18T10:13:59.126 INFO:teuthology.orchestra.run.smithi189.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-18T10:13:59.127 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi189.front.sepia.ceph.com -o noatime 2024-09-18T10:13:59.127 DEBUG:teuthology.orchestra.run.smithi189:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-15 2024-09-18T10:13:59.191 DEBUG:teuthology.orchestra.run.smithi189:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-15 2024-09-18T10:13:59.200 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-18T10:13:59.201 INFO:teuthology.orchestra.run.smithi189.stderr:sudo: /sbin/restorecon: command not found 2024-09-18T10:13:59.201 DEBUG:teuthology.orchestra.run.smithi189:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-18T10:13:59.274 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-18T10:13:59.269+0000 7f92796b9540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-18T10:13:59.274 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-18T10:13:59.269+0000 7f92796b9540 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-18T10:13:59.274 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-18T10:13:59.269+0000 7f92796b9540 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-18T10:13:59.274 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-18T10:13:59.269+0000 7f92796b9540 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-18T10:13:59.274 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-18T10:13:59.269+0000 7f92796b9540 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-18T10:13:59.275 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-18T10:13:59.269+0000 7f92796b9540 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-18T10:13:59.849 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-18T10:13:59.845+0000 7f92796b9540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-18T10:14:00.631 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-18T10:14:00.625+0000 7f92796b9540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-18T10:14:01.416 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-18T10:14:01.409+0000 7f92796b9540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-18T10:14:02.280 DEBUG:teuthology.orchestra.run.smithi189:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-18T10:14:02.296 DEBUG:teuthology.orchestra.run.smithi189:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-18T10:14:02.334 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-18T10:14:02.329+0000 7fd657b39540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-09-18T10:14:02.335 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-18T10:14:02.329+0000 7fd657b39540 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-09-18T10:14:02.335 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-18T10:14:02.329+0000 7fd657b39540 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-09-18T10:14:02.335 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-18T10:14:02.329+0000 7fd657b39540 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-09-18T10:14:02.335 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-18T10:14:02.329+0000 7fd657b39540 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-09-18T10:14:02.335 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-18T10:14:02.329+0000 7fd657b39540 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-09-18T10:14:02.886 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-18T10:14:02.881+0000 7fd657b39540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-18T10:14:03.672 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-18T10:14:03.665+0000 7fd657b39540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-18T10:14:04.465 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-18T10:14:04.461+0000 7fd657b39540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-18T10:14:05.316 DEBUG:teuthology.orchestra.run.smithi189:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-09-18T10:14:05.333 DEBUG:teuthology.orchestra.run.smithi189:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 11 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-18T10:14:05.418 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-18T10:14:05.413+0000 7f042606d540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-11/keyring: can't open /var/lib/ceph/osd/ceph-11/keyring: (2) No such file or directory 2024-09-18T10:14:05.418 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-18T10:14:05.413+0000 7f042606d540 -1 created new key in keyring /var/lib/ceph/osd/ceph-11/keyring 2024-09-18T10:14:05.418 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-18T10:14:05.413+0000 7f042606d540 -1 bluestore(/var/lib/ceph/osd/ceph-11/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-11/block: (2) No such file or directory 2024-09-18T10:14:05.418 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-18T10:14:05.413+0000 7f042606d540 -1 bluestore(/var/lib/ceph/osd/ceph-11/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-11/block: (2) No such file or directory 2024-09-18T10:14:05.418 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-18T10:14:05.413+0000 7f042606d540 -1 bluestore(/var/lib/ceph/osd/ceph-11/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-11/block: (2) No such file or directory 2024-09-18T10:14:05.419 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-18T10:14:05.413+0000 7f042606d540 -1 bluestore(/var/lib/ceph/osd/ceph-11) _read_fsid unparsable uuid 2024-09-18T10:14:05.975 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-18T10:14:05.969+0000 7f042606d540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-18T10:14:06.754 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-18T10:14:06.749+0000 7f042606d540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-18T10:14:07.535 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-18T10:14:07.529+0000 7f042606d540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-18T10:14:08.387 DEBUG:teuthology.orchestra.run.smithi189:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-11 2024-09-18T10:14:08.404 DEBUG:teuthology.orchestra.run.smithi189:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 15 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-18T10:14:08.444 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-18T10:14:08.437+0000 7f7403d38540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-15/keyring: can't open /var/lib/ceph/osd/ceph-15/keyring: (2) No such file or directory 2024-09-18T10:14:08.444 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-18T10:14:08.437+0000 7f7403d38540 -1 created new key in keyring /var/lib/ceph/osd/ceph-15/keyring 2024-09-18T10:14:08.444 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-18T10:14:08.437+0000 7f7403d38540 -1 bluestore(/var/lib/ceph/osd/ceph-15/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-15/block: (2) No such file or directory 2024-09-18T10:14:08.444 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-18T10:14:08.437+0000 7f7403d38540 -1 bluestore(/var/lib/ceph/osd/ceph-15/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-15/block: (2) No such file or directory 2024-09-18T10:14:08.444 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-18T10:14:08.437+0000 7f7403d38540 -1 bluestore(/var/lib/ceph/osd/ceph-15/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-15/block: (2) No such file or directory 2024-09-18T10:14:08.444 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-18T10:14:08.441+0000 7f7403d38540 -1 bluestore(/var/lib/ceph/osd/ceph-15) _read_fsid unparsable uuid 2024-09-18T10:14:09.001 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-18T10:14:08.997+0000 7f7403d38540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-18T10:14:09.789 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-18T10:14:09.785+0000 7f7403d38540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-18T10:14:10.579 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-18T10:14:10.572+0000 7f7403d38540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-18T10:14:11.427 DEBUG:teuthology.orchestra.run.smithi189:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-15 2024-09-18T10:14:11.482 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-18T10:14:11.482 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-09-18T10:14:11.482 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-09-18T10:14:11.499 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-09-18T10:14:11.499 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-18T10:14:11.510 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-09-18T10:14:11.510 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-18T10:14:11.568 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-09-18T10:14:11.568 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd if=/var/lib/ceph/osd/ceph-8/keyring of=/dev/stdout 2024-09-18T10:14:11.578 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-09-18T10:14:11.578 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd if=/var/lib/ceph/osd/ceph-12/keyring of=/dev/stdout 2024-09-18T10:14:11.592 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-09-18T10:14:11.592 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-18T10:14:11.609 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-09-18T10:14:11.609 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-18T10:14:11.624 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-09-18T10:14:11.624 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/var/lib/ceph/osd/ceph-9/keyring of=/dev/stdout 2024-09-18T10:14:11.636 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-09-18T10:14:11.636 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/var/lib/ceph/osd/ceph-13/keyring of=/dev/stdout 2024-09-18T10:14:11.647 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-09-18T10:14:11.647 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-18T10:14:11.664 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-09-18T10:14:11.664 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-09-18T10:14:11.722 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-09-18T10:14:11.722 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/osd/ceph-10/keyring of=/dev/stdout 2024-09-18T10:14:11.736 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-09-18T10:14:11.736 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/osd/ceph-14/keyring of=/dev/stdout 2024-09-18T10:14:11.747 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-18T10:14:11.747 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-18T10:14:11.763 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-18T10:14:11.763 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-18T10:14:11.774 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-18T10:14:11.774 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-09-18T10:14:11.832 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-18T10:14:11.832 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd if=/var/lib/ceph/osd/ceph-11/keyring of=/dev/stdout 2024-09-18T10:14:11.842 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-18T10:14:11.842 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd if=/var/lib/ceph/osd/ceph-15/keyring of=/dev/stdout 2024-09-18T10:14:11.854 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-18T10:14:11.854 DEBUG:teuthology.orchestra.run.smithi189:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-18T10:14:11.859 INFO:tasks.ceph:Adding keys to all mons... 2024-09-18T10:14:11.859 DEBUG:teuthology.orchestra.run.smithi029:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-18T10:14:11.862 DEBUG:teuthology.orchestra.run.smithi136:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-18T10:14:11.864 DEBUG:teuthology.orchestra.run.smithi154:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-18T10:14:11.874 INFO:teuthology.orchestra.run.smithi029.stdout:[mgr.y] 2024-09-18T10:14:11.874 INFO:teuthology.orchestra.run.smithi029.stdout: key = AQC6p+pm2fmnDBAAO7+1sptOrgPofbE2ZGF+Dg== 2024-09-18T10:14:11.875 INFO:teuthology.orchestra.run.smithi029.stdout:[osd.0] 2024-09-18T10:14:11.875 INFO:teuthology.orchestra.run.smithi029.stdout: key = AQC6p+pmtc7HNBAAwof3WvHStJVrkAJfol3tfA== 2024-09-18T10:14:11.875 INFO:teuthology.orchestra.run.smithi029.stdout:[osd.4] 2024-09-18T10:14:11.875 INFO:teuthology.orchestra.run.smithi029.stdout: key = AQC9p+pmjsHoORAAd3slIY05kMLCS7hRpVDOgA== 2024-09-18T10:14:11.875 INFO:teuthology.orchestra.run.smithi029.stdout:[osd.8] 2024-09-18T10:14:11.875 INFO:teuthology.orchestra.run.smithi029.stdout: key = AQDBp+pm4CtFAxAAl1tE6oP20gxH9xNC5oJXrQ== 2024-09-18T10:14:11.875 INFO:teuthology.orchestra.run.smithi029.stdout:[osd.12] 2024-09-18T10:14:11.875 INFO:teuthology.orchestra.run.smithi029.stdout: key = AQDEp+pm6n6yCBAAJgwzMykSnIQD5cJiKZNylA== 2024-09-18T10:14:11.875 INFO:teuthology.orchestra.run.smithi029.stdout:[osd.1] 2024-09-18T10:14:11.875 INFO:teuthology.orchestra.run.smithi029.stdout: key = AQDKp+pmtPs5BBAAbCUxe16mm9nPwZqaapo5Tg== 2024-09-18T10:14:11.876 INFO:teuthology.orchestra.run.smithi029.stdout:[osd.5] 2024-09-18T10:14:11.876 INFO:teuthology.orchestra.run.smithi029.stdout: key = AQDNp+pmDwU8BhAAs+cjYKHXE+KTpTfg1NbPGQ== 2024-09-18T10:14:11.876 INFO:teuthology.orchestra.run.smithi029.stdout:[osd.9] 2024-09-18T10:14:11.876 INFO:teuthology.orchestra.run.smithi029.stdout: key = AQDQp+pmNm8UCxAA6p+gF9gbc591X2xIYRrZmQ== 2024-09-18T10:14:11.876 INFO:teuthology.orchestra.run.smithi029.stdout:[osd.13] 2024-09-18T10:14:11.876 INFO:teuthology.orchestra.run.smithi029.stdout: key = AQDTp+pmhiSsDBAAFiUsrvuJCABSWxvJRf2PEg== 2024-09-18T10:14:11.876 INFO:teuthology.orchestra.run.smithi029.stdout:[osd.2] 2024-09-18T10:14:11.876 INFO:teuthology.orchestra.run.smithi029.stdout: key = AQDYp+pmLiwqJxAAPIa4Kf1q4QhfqDAQ4/zzcQ== 2024-09-18T10:14:11.876 INFO:teuthology.orchestra.run.smithi029.stdout:[osd.6] 2024-09-18T10:14:11.876 INFO:teuthology.orchestra.run.smithi029.stdout: key = AQDbp+pmTJ7iKBAATayb6oN7sq7dJfp1dbFKxg== 2024-09-18T10:14:11.877 INFO:teuthology.orchestra.run.smithi029.stdout:[osd.10] 2024-09-18T10:14:11.877 INFO:teuthology.orchestra.run.smithi029.stdout: key = AQDep+pmdtpSLBAAvBjj5NJPBWiN2EwGtarK4Q== 2024-09-18T10:14:11.877 INFO:teuthology.orchestra.run.smithi029.stdout:[osd.14] 2024-09-18T10:14:11.877 INFO:teuthology.orchestra.run.smithi029.stdout: key = AQDhp+pmU9BGMBAAdeqf79MuzmhveGEsxuT/LA== 2024-09-18T10:14:11.877 INFO:teuthology.orchestra.run.smithi029.stdout:[mgr.x] 2024-09-18T10:14:11.877 INFO:teuthology.orchestra.run.smithi029.stdout: key = AQC6p+pmfzmUDxAAdfDK62Otomi62QEBUyJIog== 2024-09-18T10:14:11.877 INFO:teuthology.orchestra.run.smithi029.stdout:[osd.3] 2024-09-18T10:14:11.877 INFO:teuthology.orchestra.run.smithi029.stdout: key = AQDnp+pmZxJWEBAA0yVtiiCCQ9/BeljGzEdhLg== 2024-09-18T10:14:11.877 INFO:teuthology.orchestra.run.smithi029.stdout:[osd.7] 2024-09-18T10:14:11.877 INFO:teuthology.orchestra.run.smithi029.stdout: key = AQDqp+pmeTL1ExAAa6ObikFdb9w3fkgL1/4N6w== 2024-09-18T10:14:11.878 INFO:teuthology.orchestra.run.smithi029.stdout:[osd.11] 2024-09-18T10:14:11.878 INFO:teuthology.orchestra.run.smithi029.stdout: key = AQDtp+pm2nnxGBAAvv+avVtQjEAJYcJzGxieIw== 2024-09-18T10:14:11.878 INFO:teuthology.orchestra.run.smithi029.stdout:[osd.15] 2024-09-18T10:14:11.878 INFO:teuthology.orchestra.run.smithi029.stdout: key = AQDwp+pmRd53GhAAyqvpa2ed9Yr5XrGOSKgB6Q== 2024-09-18T10:14:11.878 INFO:teuthology.orchestra.run.smithi029.stdout:[client.0] 2024-09-18T10:14:11.878 INFO:teuthology.orchestra.run.smithi029.stdout: key = AQC6p+pmh7e3ERAAstMXEbKNvqVMOI0wUZcq9A== 2024-09-18T10:14:11.880 INFO:teuthology.orchestra.run.smithi136.stdout:[mgr.y] 2024-09-18T10:14:11.880 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQC6p+pm2fmnDBAAO7+1sptOrgPofbE2ZGF+Dg== 2024-09-18T10:14:11.881 INFO:teuthology.orchestra.run.smithi136.stdout:[osd.0] 2024-09-18T10:14:11.881 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQC6p+pmtc7HNBAAwof3WvHStJVrkAJfol3tfA== 2024-09-18T10:14:11.881 INFO:teuthology.orchestra.run.smithi136.stdout:[osd.4] 2024-09-18T10:14:11.881 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQC9p+pmjsHoORAAd3slIY05kMLCS7hRpVDOgA== 2024-09-18T10:14:11.881 INFO:teuthology.orchestra.run.smithi136.stdout:[osd.8] 2024-09-18T10:14:11.881 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQDBp+pm4CtFAxAAl1tE6oP20gxH9xNC5oJXrQ== 2024-09-18T10:14:11.881 INFO:teuthology.orchestra.run.smithi136.stdout:[osd.12] 2024-09-18T10:14:11.881 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQDEp+pm6n6yCBAAJgwzMykSnIQD5cJiKZNylA== 2024-09-18T10:14:11.881 INFO:teuthology.orchestra.run.smithi136.stdout:[osd.1] 2024-09-18T10:14:11.881 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQDKp+pmtPs5BBAAbCUxe16mm9nPwZqaapo5Tg== 2024-09-18T10:14:11.881 INFO:teuthology.orchestra.run.smithi136.stdout:[osd.5] 2024-09-18T10:14:11.882 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQDNp+pmDwU8BhAAs+cjYKHXE+KTpTfg1NbPGQ== 2024-09-18T10:14:11.882 INFO:teuthology.orchestra.run.smithi136.stdout:[osd.9] 2024-09-18T10:14:11.882 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQDQp+pmNm8UCxAA6p+gF9gbc591X2xIYRrZmQ== 2024-09-18T10:14:11.882 INFO:teuthology.orchestra.run.smithi136.stdout:[osd.13] 2024-09-18T10:14:11.882 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQDTp+pmhiSsDBAAFiUsrvuJCABSWxvJRf2PEg== 2024-09-18T10:14:11.882 INFO:teuthology.orchestra.run.smithi136.stdout:[osd.2] 2024-09-18T10:14:11.882 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQDYp+pmLiwqJxAAPIa4Kf1q4QhfqDAQ4/zzcQ== 2024-09-18T10:14:11.882 INFO:teuthology.orchestra.run.smithi136.stdout:[osd.6] 2024-09-18T10:14:11.882 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQDbp+pmTJ7iKBAATayb6oN7sq7dJfp1dbFKxg== 2024-09-18T10:14:11.882 INFO:teuthology.orchestra.run.smithi136.stdout:[osd.10] 2024-09-18T10:14:11.882 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQDep+pmdtpSLBAAvBjj5NJPBWiN2EwGtarK4Q== 2024-09-18T10:14:11.883 INFO:teuthology.orchestra.run.smithi136.stdout:[osd.14] 2024-09-18T10:14:11.883 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQDhp+pmU9BGMBAAdeqf79MuzmhveGEsxuT/LA== 2024-09-18T10:14:11.883 INFO:teuthology.orchestra.run.smithi136.stdout:[mgr.x] 2024-09-18T10:14:11.883 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQC6p+pmfzmUDxAAdfDK62Otomi62QEBUyJIog== 2024-09-18T10:14:11.883 INFO:teuthology.orchestra.run.smithi136.stdout:[osd.3] 2024-09-18T10:14:11.883 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQDnp+pmZxJWEBAA0yVtiiCCQ9/BeljGzEdhLg== 2024-09-18T10:14:11.883 INFO:teuthology.orchestra.run.smithi136.stdout:[osd.7] 2024-09-18T10:14:11.883 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQDqp+pmeTL1ExAAa6ObikFdb9w3fkgL1/4N6w== 2024-09-18T10:14:11.883 INFO:teuthology.orchestra.run.smithi136.stdout:[osd.11] 2024-09-18T10:14:11.883 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQDtp+pm2nnxGBAAvv+avVtQjEAJYcJzGxieIw== 2024-09-18T10:14:11.883 INFO:teuthology.orchestra.run.smithi136.stdout:[osd.15] 2024-09-18T10:14:11.884 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQDwp+pmRd53GhAAyqvpa2ed9Yr5XrGOSKgB6Q== 2024-09-18T10:14:11.884 INFO:teuthology.orchestra.run.smithi136.stdout:[client.0] 2024-09-18T10:14:11.884 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQC6p+pmh7e3ERAAstMXEbKNvqVMOI0wUZcq9A== 2024-09-18T10:14:11.884 INFO:teuthology.orchestra.run.smithi154.stdout:[mgr.y] 2024-09-18T10:14:11.884 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQC6p+pm2fmnDBAAO7+1sptOrgPofbE2ZGF+Dg== 2024-09-18T10:14:11.884 INFO:teuthology.orchestra.run.smithi154.stdout:[osd.0] 2024-09-18T10:14:11.884 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQC6p+pmtc7HNBAAwof3WvHStJVrkAJfol3tfA== 2024-09-18T10:14:11.884 INFO:teuthology.orchestra.run.smithi154.stdout:[osd.4] 2024-09-18T10:14:11.884 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQC9p+pmjsHoORAAd3slIY05kMLCS7hRpVDOgA== 2024-09-18T10:14:11.884 INFO:teuthology.orchestra.run.smithi154.stdout:[osd.8] 2024-09-18T10:14:11.884 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQDBp+pm4CtFAxAAl1tE6oP20gxH9xNC5oJXrQ== 2024-09-18T10:14:11.885 INFO:teuthology.orchestra.run.smithi154.stdout:[osd.12] 2024-09-18T10:14:11.885 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQDEp+pm6n6yCBAAJgwzMykSnIQD5cJiKZNylA== 2024-09-18T10:14:11.885 INFO:teuthology.orchestra.run.smithi154.stdout:[osd.1] 2024-09-18T10:14:11.885 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQDKp+pmtPs5BBAAbCUxe16mm9nPwZqaapo5Tg== 2024-09-18T10:14:11.885 INFO:teuthology.orchestra.run.smithi154.stdout:[osd.5] 2024-09-18T10:14:11.885 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQDNp+pmDwU8BhAAs+cjYKHXE+KTpTfg1NbPGQ== 2024-09-18T10:14:11.885 INFO:teuthology.orchestra.run.smithi154.stdout:[osd.9] 2024-09-18T10:14:11.885 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQDQp+pmNm8UCxAA6p+gF9gbc591X2xIYRrZmQ== 2024-09-18T10:14:11.885 INFO:teuthology.orchestra.run.smithi154.stdout:[osd.13] 2024-09-18T10:14:11.885 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQDTp+pmhiSsDBAAFiUsrvuJCABSWxvJRf2PEg== 2024-09-18T10:14:11.885 INFO:teuthology.orchestra.run.smithi154.stdout:[osd.2] 2024-09-18T10:14:11.886 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQDYp+pmLiwqJxAAPIa4Kf1q4QhfqDAQ4/zzcQ== 2024-09-18T10:14:11.886 INFO:teuthology.orchestra.run.smithi154.stdout:[osd.6] 2024-09-18T10:14:11.886 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQDbp+pmTJ7iKBAATayb6oN7sq7dJfp1dbFKxg== 2024-09-18T10:14:11.886 INFO:teuthology.orchestra.run.smithi154.stdout:[osd.10] 2024-09-18T10:14:11.886 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQDep+pmdtpSLBAAvBjj5NJPBWiN2EwGtarK4Q== 2024-09-18T10:14:11.886 INFO:teuthology.orchestra.run.smithi154.stdout:[osd.14] 2024-09-18T10:14:11.886 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQDhp+pmU9BGMBAAdeqf79MuzmhveGEsxuT/LA== 2024-09-18T10:14:11.886 INFO:teuthology.orchestra.run.smithi154.stdout:[mgr.x] 2024-09-18T10:14:11.886 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQC6p+pmfzmUDxAAdfDK62Otomi62QEBUyJIog== 2024-09-18T10:14:11.886 INFO:teuthology.orchestra.run.smithi154.stdout:[osd.3] 2024-09-18T10:14:11.886 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQDnp+pmZxJWEBAA0yVtiiCCQ9/BeljGzEdhLg== 2024-09-18T10:14:11.886 INFO:teuthology.orchestra.run.smithi154.stdout:[osd.7] 2024-09-18T10:14:11.887 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQDqp+pmeTL1ExAAa6ObikFdb9w3fkgL1/4N6w== 2024-09-18T10:14:11.887 INFO:teuthology.orchestra.run.smithi154.stdout:[osd.11] 2024-09-18T10:14:11.887 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQDtp+pm2nnxGBAAvv+avVtQjEAJYcJzGxieIw== 2024-09-18T10:14:11.887 INFO:teuthology.orchestra.run.smithi154.stdout:[osd.15] 2024-09-18T10:14:11.887 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQDwp+pmRd53GhAAyqvpa2ed9Yr5XrGOSKgB6Q== 2024-09-18T10:14:11.887 INFO:teuthology.orchestra.run.smithi154.stdout:[client.0] 2024-09-18T10:14:11.887 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQC6p+pmh7e3ERAAstMXEbKNvqVMOI0wUZcq9A== 2024-09-18T10:14:11.888 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-18T10:14:11.890 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-18T10:14:11.892 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-18T10:14:11.969 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T10:14:11.974 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T10:14:11.978 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T10:14:12.020 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T10:14:12.022 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T10:14:12.025 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T10:14:12.066 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.8 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T10:14:12.068 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.8 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T10:14:12.070 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.8 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T10:14:12.154 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.12 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T10:14:12.157 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.12 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T10:14:12.159 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.12 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T10:14:12.204 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T10:14:12.206 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T10:14:12.208 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T10:14:12.251 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T10:14:12.253 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T10:14:12.256 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T10:14:12.340 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.9 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T10:14:12.342 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.9 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T10:14:12.344 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.9 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T10:14:12.390 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.13 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T10:14:12.392 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.13 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T10:14:12.394 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.13 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T10:14:12.440 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T10:14:12.442 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T10:14:12.444 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T10:14:12.489 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T10:14:12.492 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T10:14:12.530 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T10:14:12.573 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.10 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T10:14:12.576 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.10 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T10:14:12.578 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.10 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T10:14:12.622 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.14 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T10:14:12.662 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.14 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T10:14:12.665 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.14 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T10:14:12.714 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-18T10:14:12.717 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-18T10:14:12.754 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-18T10:14:12.799 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T10:14:12.801 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T10:14:12.803 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T10:14:12.848 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T10:14:12.851 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T10:14:12.853 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T10:14:12.935 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.11 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T10:14:12.937 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.11 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T10:14:12.939 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.11 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T10:14:12.984 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.15 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T10:14:12.987 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.15 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T10:14:12.989 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.15 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T10:14:13.037 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-18T10:14:13.075 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-18T10:14:13.077 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-18T10:14:13.125 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-18T10:14:13.126 DEBUG:teuthology.orchestra.run.smithi029:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-18T10:14:13.141 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-18T10:14:13.239 DEBUG:teuthology.orchestra.run.smithi029:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-18T10:14:13.252 DEBUG:teuthology.orchestra.run.smithi136:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-18T10:14:13.268 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-18T10:14:13.323 DEBUG:teuthology.orchestra.run.smithi136:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-18T10:14:13.336 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-18T10:14:13.354 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-18T10:14:13.411 DEBUG:teuthology.orchestra.run.smithi154:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-18T10:14:13.424 DEBUG:teuthology.orchestra.run.smithi029:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-18T10:14:13.426 DEBUG:teuthology.orchestra.run.smithi136:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-18T10:14:13.429 DEBUG:teuthology.orchestra.run.smithi154:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-18T10:14:13.434 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-18T10:14:13.435 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-18T10:14:13.435 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-18T10:14:13.437 INFO:tasks.ceph.mon.a:Started 2024-09-18T10:14:13.438 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-18T10:14:13.438 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-18T10:14:13.440 INFO:tasks.ceph.mon.b:Started 2024-09-18T10:14:13.440 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-18T10:14:13.441 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-18T10:14:13.443 INFO:tasks.ceph.mon.c:Started 2024-09-18T10:14:13.443 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-18T10:14:13.443 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-09-18T10:14:13.443 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-09-18T10:14:13.445 INFO:tasks.ceph.mgr.y:Started 2024-09-18T10:14:13.445 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-18T10:14:13.445 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-18T10:14:13.447 INFO:tasks.ceph.mgr.x:Started 2024-09-18T10:14:13.448 INFO:tasks.ceph:Setting crush tunables to default 2024-09-18T10:14:13.448 DEBUG:teuthology.orchestra.run.smithi029:> sudo ceph --cluster ceph osd crush tunables default 2024-09-18T10:14:18.950 INFO:teuthology.orchestra.run.smithi029.stderr:adjusted tunables profile to default 2024-09-18T10:14:18.961 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-18T10:14:18.961 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-09-18T10:14:18.961 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-18T10:14:18.972 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-09-18T10:14:18.972 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-18T10:14:18.984 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-09-18T10:14:18.985 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd if=/var/lib/ceph/osd/ceph-8/fsid of=/dev/stdout 2024-09-18T10:14:19.041 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2024-09-18T10:14:19.042 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd if=/var/lib/ceph/osd/ceph-12/fsid of=/dev/stdout 2024-09-18T10:14:19.053 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-09-18T10:14:19.053 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-18T10:14:19.070 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-09-18T10:14:19.071 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-18T10:14:19.091 INFO:tasks.ceph.mgr.x.smithi189.stderr:2024-09-18T10:14:19.084+0000 7fd41855b000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-18T10:14:19.097 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-09-18T10:14:19.091+0000 7f8cb8456000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-18T10:14:19.128 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-09-18T10:14:19.128 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/var/lib/ceph/osd/ceph-9/fsid of=/dev/stdout 2024-09-18T10:14:19.139 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-09-18T10:14:19.139 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/var/lib/ceph/osd/ceph-13/fsid of=/dev/stdout 2024-09-18T10:14:19.199 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-09-18T10:14:19.200 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-18T10:14:19.217 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-09-18T10:14:19.218 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-09-18T10:14:19.273 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-09-18T10:14:19.273 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/osd/ceph-10/fsid of=/dev/stdout 2024-09-18T10:14:19.289 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-09-18T10:14:19.289 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/osd/ceph-14/fsid of=/dev/stdout 2024-09-18T10:14:19.298 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-18T10:14:19.298 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-18T10:14:19.315 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-18T10:14:19.315 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-09-18T10:14:19.324 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-18T10:14:19.325 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd if=/var/lib/ceph/osd/ceph-11/fsid of=/dev/stdout 2024-09-18T10:14:19.384 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-18T10:14:19.384 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd if=/var/lib/ceph/osd/ceph-15/fsid of=/dev/stdout 2024-09-18T10:14:19.395 DEBUG:teuthology.orchestra.run.smithi189:> sudo ceph --cluster ceph osd new 39cf0511-0009-433f-92f2-e5686f33d713 0 2024-09-18T10:14:19.641 INFO:teuthology.orchestra.run.smithi189.stdout:0 2024-09-18T10:14:19.655 DEBUG:teuthology.orchestra.run.smithi189:> sudo ceph --cluster ceph osd new a0bf06ab-b98e-4130-91d2-7d4283540c08 1 2024-09-18T10:14:19.788 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-09-18T10:14:19.783+0000 7f8cb8456000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-18T10:14:19.823 INFO:tasks.ceph.mgr.x.smithi189.stderr:2024-09-18T10:14:19.816+0000 7fd41855b000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-18T10:14:19.846 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-09-18T10:14:19.843+0000 7f8cb8456000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-18T10:14:19.861 INFO:teuthology.orchestra.run.smithi189.stdout:1 2024-09-18T10:14:19.876 DEBUG:teuthology.orchestra.run.smithi189:> sudo ceph --cluster ceph osd new 742a786e-6129-4fbc-bb5b-e2e3c03c2d77 2 2024-09-18T10:14:19.881 INFO:tasks.ceph.mgr.x.smithi189.stderr:2024-09-18T10:14:19.876+0000 7fd41855b000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-18T10:14:19.966 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-09-18T10:14:19.963+0000 7f8cb8456000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-18T10:14:20.007 INFO:tasks.ceph.mgr.x.smithi189.stderr:2024-09-18T10:14:20.000+0000 7fd41855b000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-18T10:14:20.091 INFO:teuthology.orchestra.run.smithi189.stdout:2 2024-09-18T10:14:20.105 DEBUG:teuthology.orchestra.run.smithi189:> sudo ceph --cluster ceph osd new 3aa77114-24ed-49ab-86ff-a0600794cbf8 3 2024-09-18T10:14:20.211 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-09-18T10:14:20.207+0000 7f8cb8456000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-18T10:14:20.260 INFO:tasks.ceph.mgr.x.smithi189.stderr:2024-09-18T10:14:20.252+0000 7fd41855b000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-18T10:14:20.264 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-09-18T10:14:20.259+0000 7f8cb8456000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-18T10:14:20.309 INFO:tasks.ceph.mgr.x.smithi189.stderr:2024-09-18T10:14:20.304+0000 7fd41855b000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-18T10:14:20.350 INFO:teuthology.orchestra.run.smithi189.stdout:3 2024-09-18T10:14:20.364 DEBUG:teuthology.orchestra.run.smithi189:> sudo ceph --cluster ceph osd new 426567ad-b0be-41d4-afcf-dc56ec3974cd 4 2024-09-18T10:14:20.459 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-09-18T10:14:20.455+0000 7f8cb8456000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-18T10:14:20.482 INFO:tasks.ceph.mgr.x.smithi189.stderr:2024-09-18T10:14:20.476+0000 7fd41855b000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-18T10:14:20.526 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-09-18T10:14:20.523+0000 7f8cb8456000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-18T10:14:20.544 INFO:tasks.ceph.mgr.x.smithi189.stderr:2024-09-18T10:14:20.536+0000 7fd41855b000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-18T10:14:20.566 INFO:teuthology.orchestra.run.smithi189.stdout:4 2024-09-18T10:14:20.579 DEBUG:teuthology.orchestra.run.smithi189:> sudo ceph --cluster ceph osd new d9559e42-1f4e-4373-8392-a536a0afd871 5 2024-09-18T10:14:20.594 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-09-18T10:14:20.591+0000 7f8cb8456000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-18T10:14:20.606 INFO:tasks.ceph.mgr.x.smithi189.stderr:2024-09-18T10:14:20.600+0000 7fd41855b000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-18T10:14:20.772 INFO:teuthology.orchestra.run.smithi189.stdout:5 2024-09-18T10:14:20.786 DEBUG:teuthology.orchestra.run.smithi189:> sudo ceph --cluster ceph osd new d50ef1e2-8bc5-4a90-9be3-365a0db2be6f 6 2024-09-18T10:14:20.988 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-09-18T10:14:20.983+0000 7f8cb8456000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-18T10:14:20.992 INFO:tasks.ceph.mgr.x.smithi189.stderr:2024-09-18T10:14:20.988+0000 7fd41855b000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-18T10:14:21.039 INFO:teuthology.orchestra.run.smithi189.stdout:6 2024-09-18T10:14:21.052 DEBUG:teuthology.orchestra.run.smithi189:> sudo ceph --cluster ceph osd new bd0b85e4-cfee-4d19-90a5-79854794b48b 7 2024-09-18T10:14:21.103 INFO:tasks.ceph.mgr.x.smithi189.stderr:2024-09-18T10:14:21.096+0000 7fd41855b000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-18T10:14:21.114 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-09-18T10:14:21.111+0000 7f8cb8456000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-18T10:14:21.242 INFO:teuthology.orchestra.run.smithi189.stdout:7 2024-09-18T10:14:21.256 DEBUG:teuthology.orchestra.run.smithi189:> sudo ceph --cluster ceph osd new cd333f56-c165-45d7-8c8b-48d34bc4ddc5 8 2024-09-18T10:14:21.398 INFO:tasks.ceph.mgr.x.smithi189.stderr:2024-09-18T10:14:21.392+0000 7fd41855b000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-18T10:14:21.443 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-09-18T10:14:21.439+0000 7f8cb8456000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-18T10:14:21.454 INFO:teuthology.orchestra.run.smithi189.stdout:8 2024-09-18T10:14:21.467 DEBUG:teuthology.orchestra.run.smithi189:> sudo ceph --cluster ceph osd new 1e079e4c-8801-4754-9d0d-d0b68695d32e 9 2024-09-18T10:14:21.565 INFO:tasks.ceph.mgr.x.smithi189.stderr:2024-09-18T10:14:21.560+0000 7fd41855b000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-18T10:14:21.619 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-09-18T10:14:21.615+0000 7f8cb8456000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-18T10:14:21.633 INFO:tasks.ceph.mgr.x.smithi189.stderr:2024-09-18T10:14:21.628+0000 7fd41855b000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-18T10:14:21.663 INFO:teuthology.orchestra.run.smithi189.stdout:9 2024-09-18T10:14:21.675 DEBUG:teuthology.orchestra.run.smithi189:> sudo ceph --cluster ceph osd new 92ee68d4-371c-4849-930f-a8fa18d64b99 10 2024-09-18T10:14:21.689 INFO:tasks.ceph.mgr.x.smithi189.stderr:2024-09-18T10:14:21.684+0000 7fd41855b000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-18T10:14:21.690 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-09-18T10:14:21.687+0000 7f8cb8456000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-18T10:14:21.743 INFO:tasks.ceph.mgr.x.smithi189.stderr:2024-09-18T10:14:21.736+0000 7fd41855b000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-18T10:14:21.745 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-09-18T10:14:21.743+0000 7f8cb8456000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-18T10:14:21.804 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-09-18T10:14:21.799+0000 7f8cb8456000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-18T10:14:21.862 INFO:teuthology.orchestra.run.smithi189.stdout:10 2024-09-18T10:14:21.876 DEBUG:teuthology.orchestra.run.smithi189:> sudo ceph --cluster ceph osd new 634ddaf5-f7b8-4b27-a46a-04e21b73fcf0 11 2024-09-18T10:14:21.961 INFO:tasks.ceph.mgr.x.smithi189.stderr:2024-09-18T10:14:21.956+0000 7fd41855b000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-18T10:14:22.021 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-09-18T10:14:22.019+0000 7f8cb8456000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-18T10:14:22.077 INFO:teuthology.orchestra.run.smithi189.stdout:11 2024-09-18T10:14:22.090 DEBUG:teuthology.orchestra.run.smithi189:> sudo ceph --cluster ceph osd new bc58137b-5948-400d-b377-193b15824d8d 12 2024-09-18T10:14:22.131 INFO:tasks.ceph.mgr.x.smithi189.stderr:2024-09-18T10:14:22.124+0000 7fd41855b000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-18T10:14:22.188 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-09-18T10:14:22.183+0000 7f8cb8456000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-18T10:14:22.199 INFO:tasks.ceph.mgr.x.smithi189.stderr:2024-09-18T10:14:22.192+0000 7fd41855b000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-18T10:14:22.256 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-09-18T10:14:22.251+0000 7f8cb8456000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-18T10:14:22.284 INFO:teuthology.orchestra.run.smithi189.stdout:12 2024-09-18T10:14:22.296 DEBUG:teuthology.orchestra.run.smithi189:> sudo ceph --cluster ceph osd new a8b753a7-ab69-4853-ac36-66d9b85c2e20 13 2024-09-18T10:14:22.361 INFO:tasks.ceph.mgr.x.smithi189.stderr:2024-09-18T10:14:22.356+0000 7fd41855b000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-18T10:14:22.422 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-09-18T10:14:22.419+0000 7f8cb8456000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-18T10:14:22.425 INFO:tasks.ceph.mgr.x.smithi189.stderr:2024-09-18T10:14:22.420+0000 7fd41855b000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-18T10:14:22.494 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-09-18T10:14:22.491+0000 7f8cb8456000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-18T10:14:22.499 INFO:teuthology.orchestra.run.smithi189.stdout:13 2024-09-18T10:14:22.512 DEBUG:teuthology.orchestra.run.smithi189:> sudo ceph --cluster ceph osd new 8ad0848f-0de4-4a75-9c63-8c4a4a7b8524 14 2024-09-18T10:14:22.716 INFO:teuthology.orchestra.run.smithi189.stdout:14 2024-09-18T10:14:22.730 DEBUG:teuthology.orchestra.run.smithi189:> sudo ceph --cluster ceph osd new 069099b3-ec1a-483b-b967-98cddbf2f678 15 2024-09-18T10:14:22.932 INFO:teuthology.orchestra.run.smithi189.stdout:15 2024-09-18T10:14:22.946 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-18T10:14:22.946 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-18T10:14:22.949 INFO:tasks.ceph.osd.0:Started 2024-09-18T10:14:22.949 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-18T10:14:22.949 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-18T10:14:22.951 INFO:tasks.ceph.osd.4:Started 2024-09-18T10:14:22.951 INFO:tasks.ceph.osd.8:Restarting daemon 2024-09-18T10:14:22.952 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 8 2024-09-18T10:14:22.954 INFO:tasks.ceph.osd.8:Started 2024-09-18T10:14:22.954 INFO:tasks.ceph.osd.12:Restarting daemon 2024-09-18T10:14:22.954 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 12 2024-09-18T10:14:22.956 INFO:tasks.ceph.osd.12:Started 2024-09-18T10:14:22.956 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-18T10:14:22.957 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-18T10:14:22.959 INFO:tasks.ceph.osd.1:Started 2024-09-18T10:14:22.960 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-18T10:14:22.960 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-18T10:14:22.962 INFO:tasks.ceph.osd.5:Started 2024-09-18T10:14:22.962 INFO:tasks.ceph.osd.9:Restarting daemon 2024-09-18T10:14:22.963 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 9 2024-09-18T10:14:22.965 INFO:tasks.ceph.osd.9:Started 2024-09-18T10:14:22.965 INFO:tasks.ceph.osd.13:Restarting daemon 2024-09-18T10:14:22.965 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 13 2024-09-18T10:14:22.968 INFO:tasks.ceph.osd.13:Started 2024-09-18T10:14:22.968 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-18T10:14:22.969 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-18T10:14:22.971 INFO:tasks.ceph.osd.2:Started 2024-09-18T10:14:22.972 INFO:tasks.ceph.osd.6:Restarting daemon 2024-09-18T10:14:22.972 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-09-18T10:14:22.974 INFO:tasks.ceph.osd.6:Started 2024-09-18T10:14:22.975 INFO:tasks.ceph.osd.10:Restarting daemon 2024-09-18T10:14:22.975 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 10 2024-09-18T10:14:22.978 INFO:tasks.ceph.osd.10:Started 2024-09-18T10:14:22.978 INFO:tasks.ceph.osd.14:Restarting daemon 2024-09-18T10:14:22.978 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 14 2024-09-18T10:14:22.980 INFO:tasks.ceph.osd.14:Started 2024-09-18T10:14:22.981 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-18T10:14:22.981 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-18T10:14:22.983 INFO:tasks.ceph.osd.3:Started 2024-09-18T10:14:22.983 INFO:tasks.ceph.osd.7:Restarting daemon 2024-09-18T10:14:22.983 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-09-18T10:14:22.985 INFO:tasks.ceph.osd.7:Started 2024-09-18T10:14:22.985 INFO:tasks.ceph.osd.11:Restarting daemon 2024-09-18T10:14:22.986 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 11 2024-09-18T10:14:22.988 INFO:tasks.ceph.osd.11:Started 2024-09-18T10:14:22.988 INFO:tasks.ceph.osd.15:Restarting daemon 2024-09-18T10:14:22.988 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 15 2024-09-18T10:14:22.990 INFO:tasks.ceph.osd.15:Started 2024-09-18T10:14:22.991 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-18T10:14:23.163 INFO:tasks.ceph.mgr.x.smithi189.stderr:2024-09-18T10:14:23.156+0000 7fd41855b000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-18T10:14:23.174 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-18T10:14:23.174 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":18,"fsid":"ab6d6beb-0627-4b7f-8cdb-4e0a44970f10","created":"2024-09-18T10:14:18.827644+0000","modified":"2024-09-18T10:14:22.927003+0000","last_up_change":"0.000000","last_in_change":"2024-09-18T10:14:22.927003+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"39cf0511-0009-433f-92f2-e5686f33d713","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"a0bf06ab-b98e-4130-91d2-7d4283540c08","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"742a786e-6129-4fbc-bb5b-e2e3c03c2d77","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"3aa77114-24ed-49ab-86ff-a0600794cbf8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"426567ad-b0be-41d4-afcf-dc56ec3974cd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"d9559e42-1f4e-4373-8392-a536a0afd871","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"d50ef1e2-8bc5-4a90-9be3-365a0db2be6f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"bd0b85e4-cfee-4d19-90a5-79854794b48b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":8,"uuid":"cd333f56-c165-45d7-8c8b-48d34bc4ddc5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":9,"uuid":"1e079e4c-8801-4754-9d0d-d0b68695d32e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":10,"uuid":"92ee68d4-371c-4849-930f-a8fa18d64b99","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":11,"uuid":"634ddaf5-f7b8-4b27-a46a-04e21b73fcf0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":12,"uuid":"bc58137b-5948-400d-b377-193b15824d8d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":13,"uuid":"a8b753a7-ab69-4853-ac36-66d9b85c2e20","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":14,"uuid":"8ad0848f-0de4-4a75-9c63-8c4a4a7b8524","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":15,"uuid":"069099b3-ec1a-483b-b967-98cddbf2f678","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-18T10:14:23.186 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-18T10:14:23.186 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-18T10:14:23.285 INFO:tasks.ceph.mgr.y.smithi029.stderr:2024-09-18T10:14:23.279+0000 7f8cb8456000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-18T10:14:23.323 INFO:tasks.ceph.osd.4.smithi029.stderr:2024-09-18T10:14:23.319+0000 7f55107aa540 -1 Falling back to public interface 2024-09-18T10:14:23.326 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-18T10:14:23.323+0000 7f3032aff540 -1 Falling back to public interface 2024-09-18T10:14:23.334 INFO:tasks.ceph.osd.12.smithi029.stderr:2024-09-18T10:14:23.331+0000 7f8e51e63540 -1 Falling back to public interface 2024-09-18T10:14:23.362 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-18T10:14:23.359+0000 7f0007ee4540 -1 Falling back to public interface 2024-09-18T10:14:23.574 INFO:tasks.ceph.osd.8.smithi029.stderr:2024-09-18T10:14:23.571+0000 7f2559463540 -1 Falling back to public interface 2024-09-18T10:14:23.583 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-18T10:14:23.579+0000 7f19850af540 -1 Falling back to public interface 2024-09-18T10:14:23.586 INFO:tasks.ceph.osd.3.smithi189.stderr:2024-09-18T10:14:23.580+0000 7f2444803540 -1 Falling back to public interface 2024-09-18T10:14:23.592 INFO:tasks.ceph.osd.6.smithi154.stderr:2024-09-18T10:14:23.590+0000 7f0ac021d540 -1 Falling back to public interface 2024-09-18T10:14:23.595 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-18T10:14:23.591+0000 7f578fd6b540 -1 Falling back to public interface 2024-09-18T10:14:23.596 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-18T10:14:23.591+0000 7f14ca256540 -1 Falling back to public interface 2024-09-18T10:14:23.596 INFO:tasks.ceph.osd.10.smithi154.stderr:2024-09-18T10:14:23.594+0000 7fb735a56540 -1 Falling back to public interface 2024-09-18T10:14:23.596 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-18T10:14:23.595+0000 7f578ed34700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6816/0: (98) Address already in use 2024-09-18T10:14:23.603 INFO:tasks.ceph.osd.14.smithi154.stderr:2024-09-18T10:14:23.602+0000 7f75a5223540 -1 Falling back to public interface 2024-09-18T10:14:23.605 INFO:tasks.ceph.osd.15.smithi189.stderr:2024-09-18T10:14:23.600+0000 7fd0eaa9a540 -1 Falling back to public interface 2024-09-18T10:14:23.611 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-09-18T10:14:23.610+0000 7fdfabed4540 -1 Falling back to public interface 2024-09-18T10:14:23.617 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-18T10:14:23.612+0000 7f14e0497540 -1 Falling back to public interface 2024-09-18T10:14:23.625 INFO:tasks.ceph.osd.11.smithi189.stderr:2024-09-18T10:14:23.620+0000 7f718ee46540 -1 Falling back to public interface 2024-09-18T10:14:24.205 INFO:tasks.ceph.mgr.x.smithi189.stderr:2024-09-18T10:14:24.200+0000 7fd4015d2700 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-18T10:14:24.206 INFO:tasks.ceph.mgr.x.smithi189.stderr:2024-09-18T10:14:24.200+0000 7fd4015d2700 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-18T10:14:24.793 DEBUG:teuthology.orchestra.run.smithi029:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-18T10:14:25.084 INFO:teuthology.misc.health.smithi029.stdout: 2024-09-18T10:14:25.085 INFO:teuthology.misc.health.smithi029.stdout:{"epoch":18,"fsid":"ab6d6beb-0627-4b7f-8cdb-4e0a44970f10","created":"2024-09-18T10:14:18.827644+0000","modified":"2024-09-18T10:14:22.927003+0000","last_up_change":"0.000000","last_in_change":"2024-09-18T10:14:22.927003+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"39cf0511-0009-433f-92f2-e5686f33d713","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"a0bf06ab-b98e-4130-91d2-7d4283540c08","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"742a786e-6129-4fbc-bb5b-e2e3c03c2d77","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"3aa77114-24ed-49ab-86ff-a0600794cbf8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"426567ad-b0be-41d4-afcf-dc56ec3974cd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"d9559e42-1f4e-4373-8392-a536a0afd871","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"d50ef1e2-8bc5-4a90-9be3-365a0db2be6f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"bd0b85e4-cfee-4d19-90a5-79854794b48b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":8,"uuid":"cd333f56-c165-45d7-8c8b-48d34bc4ddc5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":9,"uuid":"1e079e4c-8801-4754-9d0d-d0b68695d32e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":10,"uuid":"92ee68d4-371c-4849-930f-a8fa18d64b99","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":11,"uuid":"634ddaf5-f7b8-4b27-a46a-04e21b73fcf0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":12,"uuid":"bc58137b-5948-400d-b377-193b15824d8d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":13,"uuid":"a8b753a7-ab69-4853-ac36-66d9b85c2e20","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":14,"uuid":"8ad0848f-0de4-4a75-9c63-8c4a4a7b8524","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":15,"uuid":"069099b3-ec1a-483b-b967-98cddbf2f678","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-18T10:14:25.096 DEBUG:teuthology.misc:0 of 16 OSDs are up 2024-09-18T10:14:27.058 INFO:tasks.ceph.osd.4.smithi029.stderr:2024-09-18T10:14:27.055+0000 7f55107aa540 -1 osd.4 0 log_to_monitors true 2024-09-18T10:14:27.137 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-18T10:14:27.135+0000 7f0007ee4540 -1 osd.9 0 log_to_monitors true 2024-09-18T10:14:27.166 INFO:tasks.ceph.osd.11.smithi189.stderr:2024-09-18T10:14:27.164+0000 7f718ee46540 -1 osd.11 0 log_to_monitors true 2024-09-18T10:14:27.168 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-18T10:14:27.163+0000 7f3032aff540 -1 osd.0 0 log_to_monitors true 2024-09-18T10:14:27.191 INFO:tasks.ceph.osd.12.smithi029.stderr:2024-09-18T10:14:27.187+0000 7f8e51e63540 -1 osd.12 0 log_to_monitors true 2024-09-18T10:14:27.258 INFO:tasks.ceph.osd.8.smithi029.stderr:2024-09-18T10:14:27.255+0000 7f2559463540 -1 osd.8 0 log_to_monitors true 2024-09-18T10:14:27.278 INFO:tasks.ceph.osd.3.smithi189.stderr:2024-09-18T10:14:27.276+0000 7f2444803540 -1 osd.3 0 log_to_monitors true 2024-09-18T10:14:27.347 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-18T10:14:27.343+0000 7f19850af540 -1 osd.1 0 log_to_monitors true 2024-09-18T10:14:27.356 INFO:tasks.ceph.osd.15.smithi189.stderr:2024-09-18T10:14:27.352+0000 7fd0eaa9a540 -1 osd.15 0 log_to_monitors true 2024-09-18T10:14:27.358 INFO:tasks.ceph.osd.10.smithi154.stderr:2024-09-18T10:14:27.357+0000 7fb735a56540 -1 osd.10 0 log_to_monitors true 2024-09-18T10:14:27.413 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-18T10:14:27.412+0000 7f14e0497540 -1 osd.7 0 log_to_monitors true 2024-09-18T10:14:27.426 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-18T10:14:27.423+0000 7f578fd6b540 -1 osd.5 0 log_to_monitors true 2024-09-18T10:14:27.440 INFO:tasks.ceph.osd.6.smithi154.stderr:2024-09-18T10:14:27.437+0000 7f0ac021d540 -1 osd.6 0 log_to_monitors true 2024-09-18T10:14:27.488 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-18T10:14:27.483+0000 7f14ca256540 -1 osd.13 0 log_to_monitors true 2024-09-18T10:14:27.490 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-09-18T10:14:27.489+0000 7fdfabed4540 -1 osd.2 0 log_to_monitors true 2024-09-18T10:14:27.513 INFO:tasks.ceph.osd.14.smithi154.stderr:2024-09-18T10:14:27.509+0000 7f75a5223540 -1 osd.14 0 log_to_monitors true 2024-09-18T10:14:28.229 INFO:tasks.ceph.osd.4.smithi029.stderr:2024-09-18T10:14:28.227+0000 7f550bf4a700 -1 osd.4 0 waiting for initial osdmap 2024-09-18T10:14:28.319 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-18T10:14:28.315+0000 7f0003684700 -1 osd.9 0 waiting for initial osdmap 2024-09-18T10:14:28.319 INFO:tasks.ceph.osd.12.smithi029.stderr:2024-09-18T10:14:28.315+0000 7f8e4d603700 -1 osd.12 0 waiting for initial osdmap 2024-09-18T10:14:28.320 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-18T10:14:28.315+0000 7f302fac4700 -1 osd.0 0 waiting for initial osdmap 2024-09-18T10:14:28.320 INFO:tasks.ceph.osd.11.smithi189.stderr:2024-09-18T10:14:28.316+0000 7f718a5e6700 -1 osd.11 0 waiting for initial osdmap 2024-09-18T10:14:28.329 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-18T10:14:28.327+0000 7efffe42b700 -1 osd.9 20 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-18T10:14:28.329 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-18T10:14:28.327+0000 7f3029046700 -1 osd.0 20 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-18T10:14:28.331 INFO:tasks.ceph.osd.12.smithi029.stderr:2024-09-18T10:14:28.327+0000 7f8e483aa700 -1 osd.12 20 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-18T10:14:28.332 INFO:tasks.ceph.osd.11.smithi189.stderr:2024-09-18T10:14:28.328+0000 7f718538d700 -1 osd.11 20 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-18T10:14:28.341 INFO:tasks.ceph.osd.4.smithi029.stderr:2024-09-18T10:14:28.335+0000 7f55074f2700 -1 osd.4 20 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-18T10:14:29.332 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-18T10:14:29.327+0000 7f578b50b700 -1 osd.5 0 waiting for initial osdmap 2024-09-18T10:14:29.332 INFO:tasks.ceph.osd.14.smithi154.stderr:2024-09-18T10:14:29.329+0000 7f75a09c3700 -1 osd.14 0 waiting for initial osdmap 2024-09-18T10:14:29.333 INFO:tasks.ceph.osd.3.smithi189.stderr:2024-09-18T10:14:29.332+0000 7f24417c8700 -1 osd.3 0 waiting for initial osdmap 2024-09-18T10:14:29.333 INFO:tasks.ceph.osd.8.smithi029.stderr:2024-09-18T10:14:29.327+0000 7f2556428700 -1 osd.8 0 waiting for initial osdmap 2024-09-18T10:14:29.333 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-18T10:14:29.332+0000 7f14dbc37700 -1 osd.7 0 waiting for initial osdmap 2024-09-18T10:14:29.334 INFO:tasks.ceph.osd.10.smithi154.stderr:2024-09-18T10:14:29.329+0000 7fb7311f6700 -1 osd.10 0 waiting for initial osdmap 2024-09-18T10:14:29.334 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-18T10:14:29.331+0000 7f1982074700 -1 osd.1 0 waiting for initial osdmap 2024-09-18T10:14:29.334 INFO:tasks.ceph.osd.6.smithi154.stderr:2024-09-18T10:14:29.329+0000 7f0abb9bd700 -1 osd.6 0 waiting for initial osdmap 2024-09-18T10:14:29.335 INFO:tasks.ceph.osd.15.smithi189.stderr:2024-09-18T10:14:29.332+0000 7fd0e623a700 -1 osd.15 0 waiting for initial osdmap 2024-09-18T10:14:29.335 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-18T10:14:29.331+0000 7f14c721b700 -1 osd.13 0 waiting for initial osdmap 2024-09-18T10:14:29.335 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-09-18T10:14:29.333+0000 7fdfa8e99700 -1 osd.2 0 waiting for initial osdmap 2024-09-18T10:14:29.344 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-18T10:14:29.339+0000 7f197b5f6700 -1 osd.1 21 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-18T10:14:29.344 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-18T10:14:29.339+0000 7f57862b2700 -1 osd.5 21 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-18T10:14:29.345 INFO:tasks.ceph.osd.6.smithi154.stderr:2024-09-18T10:14:29.341+0000 7f0ab6764700 -1 osd.6 21 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-18T10:14:29.346 INFO:tasks.ceph.osd.8.smithi029.stderr:2024-09-18T10:14:29.343+0000 7f254f9aa700 -1 osd.8 21 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-18T10:14:29.346 INFO:tasks.ceph.osd.10.smithi154.stderr:2024-09-18T10:14:29.341+0000 7fb72bf9d700 -1 osd.10 21 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-18T10:14:29.346 INFO:tasks.ceph.osd.14.smithi154.stderr:2024-09-18T10:14:29.341+0000 7f759af69700 -1 osd.14 21 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-18T10:14:29.347 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-09-18T10:14:29.341+0000 7fdfa1c1a700 -1 osd.2 21 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-18T10:14:29.348 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-18T10:14:29.344+0000 7f14d69de700 -1 osd.7 21 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-18T10:14:29.348 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-18T10:14:29.343+0000 7f14bff9c700 -1 osd.13 21 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-18T10:14:29.348 INFO:tasks.ceph.osd.15.smithi189.stderr:2024-09-18T10:14:29.344+0000 7fd0e0fe1700 -1 osd.15 21 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-18T10:14:29.349 INFO:tasks.ceph.osd.3.smithi189.stderr:2024-09-18T10:14:29.344+0000 7f243ad4a700 -1 osd.3 21 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-18T10:14:32.709 DEBUG:teuthology.orchestra.run.smithi029:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-18T10:14:33.004 INFO:teuthology.misc.health.smithi029.stdout: 2024-09-18T10:14:33.004 INFO:teuthology.misc.health.smithi029.stdout:{"epoch":24,"fsid":"ab6d6beb-0627-4b7f-8cdb-4e0a44970f10","created":"2024-09-18T10:14:18.827644+0000","modified":"2024-09-18T10:14:32.277817+0000","last_up_change":"2024-09-18T10:14:30.228591+0000","last_in_change":"2024-09-18T10:14:22.927003+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-18T10:14:31.266113+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"39cf0511-0009-433f-92f2-e5686f33d713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":22197},{"type":"v1","addr":"172.21.15.29:6809","nonce":22197}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":22197},{"type":"v1","addr":"172.21.15.29:6811","nonce":22197}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":22197},{"type":"v1","addr":"172.21.15.29:6815","nonce":22197}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":22197},{"type":"v1","addr":"172.21.15.29:6813","nonce":22197}]},"public_addr":"172.21.15.29:6809/22197","cluster_addr":"172.21.15.29:6811/22197","heartbeat_back_addr":"172.21.15.29:6815/22197","heartbeat_front_addr":"172.21.15.29:6813/22197","state":["exists","up"]},{"osd":1,"uuid":"a0bf06ab-b98e-4130-91d2-7d4283540c08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":22140},{"type":"v1","addr":"172.21.15.136:6809","nonce":22140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":22140},{"type":"v1","addr":"172.21.15.136:6811","nonce":22140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":22140},{"type":"v1","addr":"172.21.15.136:6815","nonce":22140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":22140},{"type":"v1","addr":"172.21.15.136:6813","nonce":22140}]},"public_addr":"172.21.15.136:6809/22140","cluster_addr":"172.21.15.136:6811/22140","heartbeat_back_addr":"172.21.15.136:6815/22140","heartbeat_front_addr":"172.21.15.136:6813/22140","state":["exists","up"]},{"osd":2,"uuid":"742a786e-6129-4fbc-bb5b-e2e3c03c2d77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":22095},{"type":"v1","addr":"172.21.15.154:6825","nonce":22095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":22095},{"type":"v1","addr":"172.21.15.154:6827","nonce":22095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":22095},{"type":"v1","addr":"172.21.15.154:6831","nonce":22095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":22095},{"type":"v1","addr":"172.21.15.154:6829","nonce":22095}]},"public_addr":"172.21.15.154:6825/22095","cluster_addr":"172.21.15.154:6827/22095","heartbeat_back_addr":"172.21.15.154:6831/22095","heartbeat_front_addr":"172.21.15.154:6829/22095","state":["exists","up"]},{"osd":3,"uuid":"3aa77114-24ed-49ab-86ff-a0600794cbf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":22404},{"type":"v1","addr":"172.21.15.189:6803","nonce":22404}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":22404},{"type":"v1","addr":"172.21.15.189:6805","nonce":22404}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":22404},{"type":"v1","addr":"172.21.15.189:6809","nonce":22404}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":22404},{"type":"v1","addr":"172.21.15.189:6807","nonce":22404}]},"public_addr":"172.21.15.189:6803/22404","cluster_addr":"172.21.15.189:6805/22404","heartbeat_back_addr":"172.21.15.189:6809/22404","heartbeat_front_addr":"172.21.15.189:6807/22404","state":["exists","up"]},{"osd":4,"uuid":"426567ad-b0be-41d4-afcf-dc56ec3974cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":22195},{"type":"v1","addr":"172.21.15.29:6801","nonce":22195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":22195},{"type":"v1","addr":"172.21.15.29:6803","nonce":22195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":22195},{"type":"v1","addr":"172.21.15.29:6807","nonce":22195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":22195},{"type":"v1","addr":"172.21.15.29:6805","nonce":22195}]},"public_addr":"172.21.15.29:6801/22195","cluster_addr":"172.21.15.29:6803/22195","heartbeat_back_addr":"172.21.15.29:6807/22195","heartbeat_front_addr":"172.21.15.29:6805/22195","state":["exists","up"]},{"osd":5,"uuid":"d9559e42-1f4e-4373-8392-a536a0afd871","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6817","nonce":22141},{"type":"v1","addr":"172.21.15.136:6819","nonce":22141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6821","nonce":22141},{"type":"v1","addr":"172.21.15.136:6823","nonce":22141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6829","nonce":22141},{"type":"v1","addr":"172.21.15.136:6831","nonce":22141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6825","nonce":22141},{"type":"v1","addr":"172.21.15.136:6827","nonce":22141}]},"public_addr":"172.21.15.136:6819/22141","cluster_addr":"172.21.15.136:6823/22141","heartbeat_back_addr":"172.21.15.136:6831/22141","heartbeat_front_addr":"172.21.15.136:6827/22141","state":["exists","up"]},{"osd":6,"uuid":"d50ef1e2-8bc5-4a90-9be3-365a0db2be6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":22096},{"type":"v1","addr":"172.21.15.154:6801","nonce":22096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":22096},{"type":"v1","addr":"172.21.15.154:6803","nonce":22096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":22096},{"type":"v1","addr":"172.21.15.154:6807","nonce":22096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":22096},{"type":"v1","addr":"172.21.15.154:6805","nonce":22096}]},"public_addr":"172.21.15.154:6801/22096","cluster_addr":"172.21.15.154:6803/22096","heartbeat_back_addr":"172.21.15.154:6807/22096","heartbeat_front_addr":"172.21.15.154:6805/22096","state":["exists","up"]},{"osd":7,"uuid":"bd0b85e4-cfee-4d19-90a5-79854794b48b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":22405},{"type":"v1","addr":"172.21.15.189:6819","nonce":22405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":22405},{"type":"v1","addr":"172.21.15.189:6821","nonce":22405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":22405},{"type":"v1","addr":"172.21.15.189:6825","nonce":22405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":22405},{"type":"v1","addr":"172.21.15.189:6823","nonce":22405}]},"public_addr":"172.21.15.189:6819/22405","cluster_addr":"172.21.15.189:6821/22405","heartbeat_back_addr":"172.21.15.189:6825/22405","heartbeat_front_addr":"172.21.15.189:6823/22405","state":["exists","up"]},{"osd":8,"uuid":"cd333f56-c165-45d7-8c8b-48d34bc4ddc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":22200},{"type":"v1","addr":"172.21.15.29:6825","nonce":22200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":22200},{"type":"v1","addr":"172.21.15.29:6827","nonce":22200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":22200},{"type":"v1","addr":"172.21.15.29:6831","nonce":22200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":22200},{"type":"v1","addr":"172.21.15.29:6829","nonce":22200}]},"public_addr":"172.21.15.29:6825/22200","cluster_addr":"172.21.15.29:6827/22200","heartbeat_back_addr":"172.21.15.29:6831/22200","heartbeat_front_addr":"172.21.15.29:6829/22200","state":["exists","up"]},{"osd":9,"uuid":"1e079e4c-8801-4754-9d0d-d0b68695d32e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":22142},{"type":"v1","addr":"172.21.15.136:6801","nonce":22142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":22142},{"type":"v1","addr":"172.21.15.136:6803","nonce":22142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":22142},{"type":"v1","addr":"172.21.15.136:6807","nonce":22142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":22142},{"type":"v1","addr":"172.21.15.136:6805","nonce":22142}]},"public_addr":"172.21.15.136:6801/22142","cluster_addr":"172.21.15.136:6803/22142","heartbeat_back_addr":"172.21.15.136:6807/22142","heartbeat_front_addr":"172.21.15.136:6805/22142","state":["exists","up"]},{"osd":10,"uuid":"92ee68d4-371c-4849-930f-a8fa18d64b99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":22097},{"type":"v1","addr":"172.21.15.154:6809","nonce":22097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":22097},{"type":"v1","addr":"172.21.15.154:6811","nonce":22097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":22097},{"type":"v1","addr":"172.21.15.154:6815","nonce":22097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":22097},{"type":"v1","addr":"172.21.15.154:6813","nonce":22097}]},"public_addr":"172.21.15.154:6809/22097","cluster_addr":"172.21.15.154:6811/22097","heartbeat_back_addr":"172.21.15.154:6815/22097","heartbeat_front_addr":"172.21.15.154:6813/22097","state":["exists","up"]},{"osd":11,"uuid":"634ddaf5-f7b8-4b27-a46a-04e21b73fcf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":22416},{"type":"v1","addr":"172.21.15.189:6827","nonce":22416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":22416},{"type":"v1","addr":"172.21.15.189:6829","nonce":22416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":22416},{"type":"v1","addr":"172.21.15.189:6833","nonce":22416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":22416},{"type":"v1","addr":"172.21.15.189:6831","nonce":22416}]},"public_addr":"172.21.15.189:6827/22416","cluster_addr":"172.21.15.189:6829/22416","heartbeat_back_addr":"172.21.15.189:6833/22416","heartbeat_front_addr":"172.21.15.189:6831/22416","state":["exists","up"]},{"osd":12,"uuid":"bc58137b-5948-400d-b377-193b15824d8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":22210},{"type":"v1","addr":"172.21.15.29:6817","nonce":22210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":22210},{"type":"v1","addr":"172.21.15.29:6819","nonce":22210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":22210},{"type":"v1","addr":"172.21.15.29:6823","nonce":22210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":22210},{"type":"v1","addr":"172.21.15.29:6821","nonce":22210}]},"public_addr":"172.21.15.29:6817/22210","cluster_addr":"172.21.15.29:6819/22210","heartbeat_back_addr":"172.21.15.29:6823/22210","heartbeat_front_addr":"172.21.15.29:6821/22210","state":["exists","up"]},{"osd":13,"uuid":"a8b753a7-ab69-4853-ac36-66d9b85c2e20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":22143},{"type":"v1","addr":"172.21.15.136:6818","nonce":22143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":22143},{"type":"v1","addr":"172.21.15.136:6822","nonce":22143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":22143},{"type":"v1","addr":"172.21.15.136:6830","nonce":22143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":22143},{"type":"v1","addr":"172.21.15.136:6826","nonce":22143}]},"public_addr":"172.21.15.136:6818/22143","cluster_addr":"172.21.15.136:6822/22143","heartbeat_back_addr":"172.21.15.136:6830/22143","heartbeat_front_addr":"172.21.15.136:6826/22143","state":["exists","up"]},{"osd":14,"uuid":"8ad0848f-0de4-4a75-9c63-8c4a4a7b8524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":22098},{"type":"v1","addr":"172.21.15.154:6817","nonce":22098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":22098},{"type":"v1","addr":"172.21.15.154:6819","nonce":22098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":22098},{"type":"v1","addr":"172.21.15.154:6823","nonce":22098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":22098},{"type":"v1","addr":"172.21.15.154:6821","nonce":22098}]},"public_addr":"172.21.15.154:6817/22098","cluster_addr":"172.21.15.154:6819/22098","heartbeat_back_addr":"172.21.15.154:6823/22098","heartbeat_front_addr":"172.21.15.154:6821/22098","state":["exists","up"]},{"osd":15,"uuid":"069099b3-ec1a-483b-b967-98cddbf2f678","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":22415},{"type":"v1","addr":"172.21.15.189:6811","nonce":22415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":22415},{"type":"v1","addr":"172.21.15.189:6813","nonce":22415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":22415},{"type":"v1","addr":"172.21.15.189:6817","nonce":22415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":22415},{"type":"v1","addr":"172.21.15.189:6815","nonce":22415}]},"public_addr":"172.21.15.189:6811/22415","cluster_addr":"172.21.15.189:6813/22415","heartbeat_back_addr":"172.21.15.189:6817/22415","heartbeat_front_addr":"172.21.15.189:6815/22415","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:14:28.132367+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:14:28.342192+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:14:28.486853+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:14:28.299130+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:14:28.027740+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:14:28.450120+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:14:28.412452+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:14:28.368142+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:14:28.238445+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:14:28.096605+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:14:28.345864+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:14:28.131654+0000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:14:28.311441+0000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:14:28.483117+0000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:14:28.510528+0000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:14:28.398477+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-18T10:14:33.016 DEBUG:teuthology.misc:16 of 16 OSDs are up 2024-09-18T10:14:33.017 INFO:tasks.ceph:Creating RBD pool 2024-09-18T10:14:33.017 DEBUG:teuthology.orchestra.run.smithi029:> sudo ceph --cluster ceph osd pool create rbd 8 2024-09-18T10:14:34.312 INFO:teuthology.orchestra.run.smithi029.stderr:pool 'rbd' created 2024-09-18T10:14:34.328 DEBUG:teuthology.orchestra.run.smithi029:> rbd --cluster ceph pool init rbd 2024-09-18T10:14:37.437 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-18T10:14:37.438 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-18T10:14:37.438 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-18T10:14:37.438 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-18T10:14:37.439 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-18T10:14:37.782 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-18T10:14:37.795 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":5,"flags":0,"active_gid":4110,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":21929},{"type":"v1","addr":"172.21.15.189:6801","nonce":21929}]},"active_addr":"172.21.15.189:6801/21929","active_change":"2024-09-18T10:14:23.181466+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[{"gid":4102,"name":"y","mgr_features":4540138320759226367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":false,"error_string":"loading asyncssh library:No module named 'asyncssh'","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":false,"error_string":"loading asyncssh library:No module named 'asyncssh'","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.189:0","nonce":3493739733}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.189:0","nonce":3092077334}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.189:0","nonce":1316029260}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.189:0","nonce":1437033923}]}]} 2024-09-18T10:14:37.805 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-18T10:14:37.805 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-18T10:14:37.806 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-18T10:14:38.107 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-18T10:14:38.107 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":29,"fsid":"ab6d6beb-0627-4b7f-8cdb-4e0a44970f10","created":"2024-09-18T10:14:18.827644+0000","modified":"2024-09-18T10:14:37.318779+0000","last_up_change":"2024-09-18T10:14:30.228591+0000","last_in_change":"2024-09-18T10:14:22.927003+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-18T10:14:31.266113+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-18T10:14:33.358485+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":29,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"39cf0511-0009-433f-92f2-e5686f33d713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":22197},{"type":"v1","addr":"172.21.15.29:6809","nonce":22197}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":22197},{"type":"v1","addr":"172.21.15.29:6811","nonce":22197}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":22197},{"type":"v1","addr":"172.21.15.29:6815","nonce":22197}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":22197},{"type":"v1","addr":"172.21.15.29:6813","nonce":22197}]},"public_addr":"172.21.15.29:6809/22197","cluster_addr":"172.21.15.29:6811/22197","heartbeat_back_addr":"172.21.15.29:6815/22197","heartbeat_front_addr":"172.21.15.29:6813/22197","state":["exists","up"]},{"osd":1,"uuid":"a0bf06ab-b98e-4130-91d2-7d4283540c08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":22140},{"type":"v1","addr":"172.21.15.136:6809","nonce":22140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":22140},{"type":"v1","addr":"172.21.15.136:6811","nonce":22140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":22140},{"type":"v1","addr":"172.21.15.136:6815","nonce":22140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":22140},{"type":"v1","addr":"172.21.15.136:6813","nonce":22140}]},"public_addr":"172.21.15.136:6809/22140","cluster_addr":"172.21.15.136:6811/22140","heartbeat_back_addr":"172.21.15.136:6815/22140","heartbeat_front_addr":"172.21.15.136:6813/22140","state":["exists","up"]},{"osd":2,"uuid":"742a786e-6129-4fbc-bb5b-e2e3c03c2d77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":22095},{"type":"v1","addr":"172.21.15.154:6825","nonce":22095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":22095},{"type":"v1","addr":"172.21.15.154:6827","nonce":22095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":22095},{"type":"v1","addr":"172.21.15.154:6831","nonce":22095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":22095},{"type":"v1","addr":"172.21.15.154:6829","nonce":22095}]},"public_addr":"172.21.15.154:6825/22095","cluster_addr":"172.21.15.154:6827/22095","heartbeat_back_addr":"172.21.15.154:6831/22095","heartbeat_front_addr":"172.21.15.154:6829/22095","state":["exists","up"]},{"osd":3,"uuid":"3aa77114-24ed-49ab-86ff-a0600794cbf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":22404},{"type":"v1","addr":"172.21.15.189:6803","nonce":22404}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":22404},{"type":"v1","addr":"172.21.15.189:6805","nonce":22404}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":22404},{"type":"v1","addr":"172.21.15.189:6809","nonce":22404}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":22404},{"type":"v1","addr":"172.21.15.189:6807","nonce":22404}]},"public_addr":"172.21.15.189:6803/22404","cluster_addr":"172.21.15.189:6805/22404","heartbeat_back_addr":"172.21.15.189:6809/22404","heartbeat_front_addr":"172.21.15.189:6807/22404","state":["exists","up"]},{"osd":4,"uuid":"426567ad-b0be-41d4-afcf-dc56ec3974cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":22195},{"type":"v1","addr":"172.21.15.29:6801","nonce":22195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":22195},{"type":"v1","addr":"172.21.15.29:6803","nonce":22195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":22195},{"type":"v1","addr":"172.21.15.29:6807","nonce":22195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":22195},{"type":"v1","addr":"172.21.15.29:6805","nonce":22195}]},"public_addr":"172.21.15.29:6801/22195","cluster_addr":"172.21.15.29:6803/22195","heartbeat_back_addr":"172.21.15.29:6807/22195","heartbeat_front_addr":"172.21.15.29:6805/22195","state":["exists","up"]},{"osd":5,"uuid":"d9559e42-1f4e-4373-8392-a536a0afd871","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6817","nonce":22141},{"type":"v1","addr":"172.21.15.136:6819","nonce":22141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6821","nonce":22141},{"type":"v1","addr":"172.21.15.136:6823","nonce":22141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6829","nonce":22141},{"type":"v1","addr":"172.21.15.136:6831","nonce":22141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6825","nonce":22141},{"type":"v1","addr":"172.21.15.136:6827","nonce":22141}]},"public_addr":"172.21.15.136:6819/22141","cluster_addr":"172.21.15.136:6823/22141","heartbeat_back_addr":"172.21.15.136:6831/22141","heartbeat_front_addr":"172.21.15.136:6827/22141","state":["exists","up"]},{"osd":6,"uuid":"d50ef1e2-8bc5-4a90-9be3-365a0db2be6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":22096},{"type":"v1","addr":"172.21.15.154:6801","nonce":22096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":22096},{"type":"v1","addr":"172.21.15.154:6803","nonce":22096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":22096},{"type":"v1","addr":"172.21.15.154:6807","nonce":22096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":22096},{"type":"v1","addr":"172.21.15.154:6805","nonce":22096}]},"public_addr":"172.21.15.154:6801/22096","cluster_addr":"172.21.15.154:6803/22096","heartbeat_back_addr":"172.21.15.154:6807/22096","heartbeat_front_addr":"172.21.15.154:6805/22096","state":["exists","up"]},{"osd":7,"uuid":"bd0b85e4-cfee-4d19-90a5-79854794b48b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":22405},{"type":"v1","addr":"172.21.15.189:6819","nonce":22405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":22405},{"type":"v1","addr":"172.21.15.189:6821","nonce":22405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":22405},{"type":"v1","addr":"172.21.15.189:6825","nonce":22405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":22405},{"type":"v1","addr":"172.21.15.189:6823","nonce":22405}]},"public_addr":"172.21.15.189:6819/22405","cluster_addr":"172.21.15.189:6821/22405","heartbeat_back_addr":"172.21.15.189:6825/22405","heartbeat_front_addr":"172.21.15.189:6823/22405","state":["exists","up"]},{"osd":8,"uuid":"cd333f56-c165-45d7-8c8b-48d34bc4ddc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":22200},{"type":"v1","addr":"172.21.15.29:6825","nonce":22200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":22200},{"type":"v1","addr":"172.21.15.29:6827","nonce":22200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":22200},{"type":"v1","addr":"172.21.15.29:6831","nonce":22200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":22200},{"type":"v1","addr":"172.21.15.29:6829","nonce":22200}]},"public_addr":"172.21.15.29:6825/22200","cluster_addr":"172.21.15.29:6827/22200","heartbeat_back_addr":"172.21.15.29:6831/22200","heartbeat_front_addr":"172.21.15.29:6829/22200","state":["exists","up"]},{"osd":9,"uuid":"1e079e4c-8801-4754-9d0d-d0b68695d32e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":22142},{"type":"v1","addr":"172.21.15.136:6801","nonce":22142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":22142},{"type":"v1","addr":"172.21.15.136:6803","nonce":22142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":22142},{"type":"v1","addr":"172.21.15.136:6807","nonce":22142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":22142},{"type":"v1","addr":"172.21.15.136:6805","nonce":22142}]},"public_addr":"172.21.15.136:6801/22142","cluster_addr":"172.21.15.136:6803/22142","heartbeat_back_addr":"172.21.15.136:6807/22142","heartbeat_front_addr":"172.21.15.136:6805/22142","state":["exists","up"]},{"osd":10,"uuid":"92ee68d4-371c-4849-930f-a8fa18d64b99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":22097},{"type":"v1","addr":"172.21.15.154:6809","nonce":22097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":22097},{"type":"v1","addr":"172.21.15.154:6811","nonce":22097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":22097},{"type":"v1","addr":"172.21.15.154:6815","nonce":22097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":22097},{"type":"v1","addr":"172.21.15.154:6813","nonce":22097}]},"public_addr":"172.21.15.154:6809/22097","cluster_addr":"172.21.15.154:6811/22097","heartbeat_back_addr":"172.21.15.154:6815/22097","heartbeat_front_addr":"172.21.15.154:6813/22097","state":["exists","up"]},{"osd":11,"uuid":"634ddaf5-f7b8-4b27-a46a-04e21b73fcf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":22416},{"type":"v1","addr":"172.21.15.189:6827","nonce":22416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":22416},{"type":"v1","addr":"172.21.15.189:6829","nonce":22416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":22416},{"type":"v1","addr":"172.21.15.189:6833","nonce":22416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":22416},{"type":"v1","addr":"172.21.15.189:6831","nonce":22416}]},"public_addr":"172.21.15.189:6827/22416","cluster_addr":"172.21.15.189:6829/22416","heartbeat_back_addr":"172.21.15.189:6833/22416","heartbeat_front_addr":"172.21.15.189:6831/22416","state":["exists","up"]},{"osd":12,"uuid":"bc58137b-5948-400d-b377-193b15824d8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":22210},{"type":"v1","addr":"172.21.15.29:6817","nonce":22210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":22210},{"type":"v1","addr":"172.21.15.29:6819","nonce":22210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":22210},{"type":"v1","addr":"172.21.15.29:6823","nonce":22210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":22210},{"type":"v1","addr":"172.21.15.29:6821","nonce":22210}]},"public_addr":"172.21.15.29:6817/22210","cluster_addr":"172.21.15.29:6819/22210","heartbeat_back_addr":"172.21.15.29:6823/22210","heartbeat_front_addr":"172.21.15.29:6821/22210","state":["exists","up"]},{"osd":13,"uuid":"a8b753a7-ab69-4853-ac36-66d9b85c2e20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":22143},{"type":"v1","addr":"172.21.15.136:6818","nonce":22143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":22143},{"type":"v1","addr":"172.21.15.136:6822","nonce":22143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":22143},{"type":"v1","addr":"172.21.15.136:6830","nonce":22143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":22143},{"type":"v1","addr":"172.21.15.136:6826","nonce":22143}]},"public_addr":"172.21.15.136:6818/22143","cluster_addr":"172.21.15.136:6822/22143","heartbeat_back_addr":"172.21.15.136:6830/22143","heartbeat_front_addr":"172.21.15.136:6826/22143","state":["exists","up"]},{"osd":14,"uuid":"8ad0848f-0de4-4a75-9c63-8c4a4a7b8524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":22098},{"type":"v1","addr":"172.21.15.154:6817","nonce":22098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":22098},{"type":"v1","addr":"172.21.15.154:6819","nonce":22098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":22098},{"type":"v1","addr":"172.21.15.154:6823","nonce":22098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":22098},{"type":"v1","addr":"172.21.15.154:6821","nonce":22098}]},"public_addr":"172.21.15.154:6817/22098","cluster_addr":"172.21.15.154:6819/22098","heartbeat_back_addr":"172.21.15.154:6823/22098","heartbeat_front_addr":"172.21.15.154:6821/22098","state":["exists","up"]},{"osd":15,"uuid":"069099b3-ec1a-483b-b967-98cddbf2f678","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":22415},{"type":"v1","addr":"172.21.15.189:6811","nonce":22415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":22415},{"type":"v1","addr":"172.21.15.189:6813","nonce":22415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":22415},{"type":"v1","addr":"172.21.15.189:6817","nonce":22415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":22415},{"type":"v1","addr":"172.21.15.189:6815","nonce":22415}]},"public_addr":"172.21.15.189:6811/22415","cluster_addr":"172.21.15.189:6813/22415","heartbeat_back_addr":"172.21.15.189:6817/22415","heartbeat_front_addr":"172.21.15.189:6815/22415","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:14:28.132367+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:14:28.342192+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:14:28.486853+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:14:28.299130+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:14:28.027740+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:14:28.450120+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:14:28.412452+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:14:28.368142+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:14:28.238445+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:14:28.096605+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:14:28.345864+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:14:28.131654+0000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:14:28.311441+0000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:14:28.483117+0000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:14:28.510528+0000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:14:28.398477+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-18T10:14:38.124 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-18T10:14:38.124 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-18T10:14:38.415 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-18T10:14:38.415 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":29,"fsid":"ab6d6beb-0627-4b7f-8cdb-4e0a44970f10","created":"2024-09-18T10:14:18.827644+0000","modified":"2024-09-18T10:14:37.318779+0000","last_up_change":"2024-09-18T10:14:30.228591+0000","last_in_change":"2024-09-18T10:14:22.927003+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-18T10:14:31.266113+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-18T10:14:33.358485+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":29,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"39cf0511-0009-433f-92f2-e5686f33d713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":22197},{"type":"v1","addr":"172.21.15.29:6809","nonce":22197}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":22197},{"type":"v1","addr":"172.21.15.29:6811","nonce":22197}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":22197},{"type":"v1","addr":"172.21.15.29:6815","nonce":22197}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":22197},{"type":"v1","addr":"172.21.15.29:6813","nonce":22197}]},"public_addr":"172.21.15.29:6809/22197","cluster_addr":"172.21.15.29:6811/22197","heartbeat_back_addr":"172.21.15.29:6815/22197","heartbeat_front_addr":"172.21.15.29:6813/22197","state":["exists","up"]},{"osd":1,"uuid":"a0bf06ab-b98e-4130-91d2-7d4283540c08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":22140},{"type":"v1","addr":"172.21.15.136:6809","nonce":22140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":22140},{"type":"v1","addr":"172.21.15.136:6811","nonce":22140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":22140},{"type":"v1","addr":"172.21.15.136:6815","nonce":22140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":22140},{"type":"v1","addr":"172.21.15.136:6813","nonce":22140}]},"public_addr":"172.21.15.136:6809/22140","cluster_addr":"172.21.15.136:6811/22140","heartbeat_back_addr":"172.21.15.136:6815/22140","heartbeat_front_addr":"172.21.15.136:6813/22140","state":["exists","up"]},{"osd":2,"uuid":"742a786e-6129-4fbc-bb5b-e2e3c03c2d77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":22095},{"type":"v1","addr":"172.21.15.154:6825","nonce":22095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":22095},{"type":"v1","addr":"172.21.15.154:6827","nonce":22095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":22095},{"type":"v1","addr":"172.21.15.154:6831","nonce":22095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":22095},{"type":"v1","addr":"172.21.15.154:6829","nonce":22095}]},"public_addr":"172.21.15.154:6825/22095","cluster_addr":"172.21.15.154:6827/22095","heartbeat_back_addr":"172.21.15.154:6831/22095","heartbeat_front_addr":"172.21.15.154:6829/22095","state":["exists","up"]},{"osd":3,"uuid":"3aa77114-24ed-49ab-86ff-a0600794cbf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":22404},{"type":"v1","addr":"172.21.15.189:6803","nonce":22404}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":22404},{"type":"v1","addr":"172.21.15.189:6805","nonce":22404}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":22404},{"type":"v1","addr":"172.21.15.189:6809","nonce":22404}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":22404},{"type":"v1","addr":"172.21.15.189:6807","nonce":22404}]},"public_addr":"172.21.15.189:6803/22404","cluster_addr":"172.21.15.189:6805/22404","heartbeat_back_addr":"172.21.15.189:6809/22404","heartbeat_front_addr":"172.21.15.189:6807/22404","state":["exists","up"]},{"osd":4,"uuid":"426567ad-b0be-41d4-afcf-dc56ec3974cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":22195},{"type":"v1","addr":"172.21.15.29:6801","nonce":22195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":22195},{"type":"v1","addr":"172.21.15.29:6803","nonce":22195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":22195},{"type":"v1","addr":"172.21.15.29:6807","nonce":22195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":22195},{"type":"v1","addr":"172.21.15.29:6805","nonce":22195}]},"public_addr":"172.21.15.29:6801/22195","cluster_addr":"172.21.15.29:6803/22195","heartbeat_back_addr":"172.21.15.29:6807/22195","heartbeat_front_addr":"172.21.15.29:6805/22195","state":["exists","up"]},{"osd":5,"uuid":"d9559e42-1f4e-4373-8392-a536a0afd871","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6817","nonce":22141},{"type":"v1","addr":"172.21.15.136:6819","nonce":22141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6821","nonce":22141},{"type":"v1","addr":"172.21.15.136:6823","nonce":22141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6829","nonce":22141},{"type":"v1","addr":"172.21.15.136:6831","nonce":22141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6825","nonce":22141},{"type":"v1","addr":"172.21.15.136:6827","nonce":22141}]},"public_addr":"172.21.15.136:6819/22141","cluster_addr":"172.21.15.136:6823/22141","heartbeat_back_addr":"172.21.15.136:6831/22141","heartbeat_front_addr":"172.21.15.136:6827/22141","state":["exists","up"]},{"osd":6,"uuid":"d50ef1e2-8bc5-4a90-9be3-365a0db2be6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":22096},{"type":"v1","addr":"172.21.15.154:6801","nonce":22096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":22096},{"type":"v1","addr":"172.21.15.154:6803","nonce":22096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":22096},{"type":"v1","addr":"172.21.15.154:6807","nonce":22096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":22096},{"type":"v1","addr":"172.21.15.154:6805","nonce":22096}]},"public_addr":"172.21.15.154:6801/22096","cluster_addr":"172.21.15.154:6803/22096","heartbeat_back_addr":"172.21.15.154:6807/22096","heartbeat_front_addr":"172.21.15.154:6805/22096","state":["exists","up"]},{"osd":7,"uuid":"bd0b85e4-cfee-4d19-90a5-79854794b48b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":22405},{"type":"v1","addr":"172.21.15.189:6819","nonce":22405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":22405},{"type":"v1","addr":"172.21.15.189:6821","nonce":22405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":22405},{"type":"v1","addr":"172.21.15.189:6825","nonce":22405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":22405},{"type":"v1","addr":"172.21.15.189:6823","nonce":22405}]},"public_addr":"172.21.15.189:6819/22405","cluster_addr":"172.21.15.189:6821/22405","heartbeat_back_addr":"172.21.15.189:6825/22405","heartbeat_front_addr":"172.21.15.189:6823/22405","state":["exists","up"]},{"osd":8,"uuid":"cd333f56-c165-45d7-8c8b-48d34bc4ddc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":22200},{"type":"v1","addr":"172.21.15.29:6825","nonce":22200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":22200},{"type":"v1","addr":"172.21.15.29:6827","nonce":22200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":22200},{"type":"v1","addr":"172.21.15.29:6831","nonce":22200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":22200},{"type":"v1","addr":"172.21.15.29:6829","nonce":22200}]},"public_addr":"172.21.15.29:6825/22200","cluster_addr":"172.21.15.29:6827/22200","heartbeat_back_addr":"172.21.15.29:6831/22200","heartbeat_front_addr":"172.21.15.29:6829/22200","state":["exists","up"]},{"osd":9,"uuid":"1e079e4c-8801-4754-9d0d-d0b68695d32e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":22142},{"type":"v1","addr":"172.21.15.136:6801","nonce":22142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":22142},{"type":"v1","addr":"172.21.15.136:6803","nonce":22142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":22142},{"type":"v1","addr":"172.21.15.136:6807","nonce":22142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":22142},{"type":"v1","addr":"172.21.15.136:6805","nonce":22142}]},"public_addr":"172.21.15.136:6801/22142","cluster_addr":"172.21.15.136:6803/22142","heartbeat_back_addr":"172.21.15.136:6807/22142","heartbeat_front_addr":"172.21.15.136:6805/22142","state":["exists","up"]},{"osd":10,"uuid":"92ee68d4-371c-4849-930f-a8fa18d64b99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":22097},{"type":"v1","addr":"172.21.15.154:6809","nonce":22097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":22097},{"type":"v1","addr":"172.21.15.154:6811","nonce":22097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":22097},{"type":"v1","addr":"172.21.15.154:6815","nonce":22097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":22097},{"type":"v1","addr":"172.21.15.154:6813","nonce":22097}]},"public_addr":"172.21.15.154:6809/22097","cluster_addr":"172.21.15.154:6811/22097","heartbeat_back_addr":"172.21.15.154:6815/22097","heartbeat_front_addr":"172.21.15.154:6813/22097","state":["exists","up"]},{"osd":11,"uuid":"634ddaf5-f7b8-4b27-a46a-04e21b73fcf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":22416},{"type":"v1","addr":"172.21.15.189:6827","nonce":22416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":22416},{"type":"v1","addr":"172.21.15.189:6829","nonce":22416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":22416},{"type":"v1","addr":"172.21.15.189:6833","nonce":22416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":22416},{"type":"v1","addr":"172.21.15.189:6831","nonce":22416}]},"public_addr":"172.21.15.189:6827/22416","cluster_addr":"172.21.15.189:6829/22416","heartbeat_back_addr":"172.21.15.189:6833/22416","heartbeat_front_addr":"172.21.15.189:6831/22416","state":["exists","up"]},{"osd":12,"uuid":"bc58137b-5948-400d-b377-193b15824d8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":22210},{"type":"v1","addr":"172.21.15.29:6817","nonce":22210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":22210},{"type":"v1","addr":"172.21.15.29:6819","nonce":22210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":22210},{"type":"v1","addr":"172.21.15.29:6823","nonce":22210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":22210},{"type":"v1","addr":"172.21.15.29:6821","nonce":22210}]},"public_addr":"172.21.15.29:6817/22210","cluster_addr":"172.21.15.29:6819/22210","heartbeat_back_addr":"172.21.15.29:6823/22210","heartbeat_front_addr":"172.21.15.29:6821/22210","state":["exists","up"]},{"osd":13,"uuid":"a8b753a7-ab69-4853-ac36-66d9b85c2e20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":22143},{"type":"v1","addr":"172.21.15.136:6818","nonce":22143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":22143},{"type":"v1","addr":"172.21.15.136:6822","nonce":22143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":22143},{"type":"v1","addr":"172.21.15.136:6830","nonce":22143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":22143},{"type":"v1","addr":"172.21.15.136:6826","nonce":22143}]},"public_addr":"172.21.15.136:6818/22143","cluster_addr":"172.21.15.136:6822/22143","heartbeat_back_addr":"172.21.15.136:6830/22143","heartbeat_front_addr":"172.21.15.136:6826/22143","state":["exists","up"]},{"osd":14,"uuid":"8ad0848f-0de4-4a75-9c63-8c4a4a7b8524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":22098},{"type":"v1","addr":"172.21.15.154:6817","nonce":22098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":22098},{"type":"v1","addr":"172.21.15.154:6819","nonce":22098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":22098},{"type":"v1","addr":"172.21.15.154:6823","nonce":22098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":22098},{"type":"v1","addr":"172.21.15.154:6821","nonce":22098}]},"public_addr":"172.21.15.154:6817/22098","cluster_addr":"172.21.15.154:6819/22098","heartbeat_back_addr":"172.21.15.154:6823/22098","heartbeat_front_addr":"172.21.15.154:6821/22098","state":["exists","up"]},{"osd":15,"uuid":"069099b3-ec1a-483b-b967-98cddbf2f678","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":22415},{"type":"v1","addr":"172.21.15.189:6811","nonce":22415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":22415},{"type":"v1","addr":"172.21.15.189:6813","nonce":22415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":22415},{"type":"v1","addr":"172.21.15.189:6817","nonce":22415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":22415},{"type":"v1","addr":"172.21.15.189:6815","nonce":22415}]},"public_addr":"172.21.15.189:6811/22415","cluster_addr":"172.21.15.189:6813/22415","heartbeat_back_addr":"172.21.15.189:6817/22415","heartbeat_front_addr":"172.21.15.189:6815/22415","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:14:28.132367+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:14:28.342192+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:14:28.486853+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:14:28.299130+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:14:28.027740+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:14:28.450120+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:14:28.412452+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:14:28.368142+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:14:28.238445+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:14:28.096605+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:14:28.345864+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:14:28.131654+0000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:14:28.311441+0000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:14:28.483117+0000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:14:28.510528+0000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:14:28.398477+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-18T10:14:38.430 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-18T10:14:38.430 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-18T10:14:38.431 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-18T10:14:38.431 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-18T10:14:38.431 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-18T10:14:38.432 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-18T10:14:38.432 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-18T10:14:38.433 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-18T10:14:38.433 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.8 flush_pg_stats 2024-09-18T10:14:38.434 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.9 flush_pg_stats 2024-09-18T10:14:38.434 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.10 flush_pg_stats 2024-09-18T10:14:38.434 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.11 flush_pg_stats 2024-09-18T10:14:38.435 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.12 flush_pg_stats 2024-09-18T10:14:38.435 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.13 flush_pg_stats 2024-09-18T10:14:38.436 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.14 flush_pg_stats 2024-09-18T10:14:38.436 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.15 flush_pg_stats 2024-09-18T10:14:38.753 INFO:teuthology.orchestra.run.smithi029.stdout:90194313221 2024-09-18T10:14:38.753 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-18T10:14:38.754 INFO:teuthology.orchestra.run.smithi029.stdout:90194313219 2024-09-18T10:14:38.755 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-18T10:14:38.762 INFO:teuthology.orchestra.run.smithi029.stdout:94489280516 2024-09-18T10:14:38.763 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.13 2024-09-18T10:14:38.771 INFO:teuthology.orchestra.run.smithi029.stdout:94489280515 2024-09-18T10:14:38.771 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-18T10:14:38.773 INFO:teuthology.orchestra.run.smithi029.stdout:94489280515 2024-09-18T10:14:38.773 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-18T10:14:38.792 INFO:teuthology.orchestra.run.smithi029.stdout:94489280515 2024-09-18T10:14:38.792 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-18T10:14:38.825 INFO:teuthology.orchestra.run.smithi029.stdout:90194313219 2024-09-18T10:14:38.825 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.9 2024-09-18T10:14:38.845 INFO:teuthology.orchestra.run.smithi029.stdout:94489280515 2024-09-18T10:14:38.846 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.8 2024-09-18T10:14:38.850 INFO:teuthology.orchestra.run.smithi029.stdout:90194313220 2024-09-18T10:14:38.851 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.12 2024-09-18T10:14:38.868 INFO:teuthology.orchestra.run.smithi029.stdout:94489280515 2024-09-18T10:14:38.868 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-18T10:14:38.873 INFO:teuthology.orchestra.run.smithi029.stdout:94489280515 2024-09-18T10:14:38.874 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-18T10:14:38.891 INFO:teuthology.orchestra.run.smithi029.stdout:90194313219 2024-09-18T10:14:38.892 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.11 2024-09-18T10:14:38.893 INFO:teuthology.orchestra.run.smithi029.stdout:94489280515 2024-09-18T10:14:38.893 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-18T10:14:38.905 INFO:teuthology.orchestra.run.smithi029.stdout:94489280515 2024-09-18T10:14:38.906 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.15 2024-09-18T10:14:38.910 INFO:teuthology.orchestra.run.smithi029.stdout:94489280515 2024-09-18T10:14:38.910 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.14 2024-09-18T10:14:38.914 INFO:teuthology.orchestra.run.smithi029.stdout:94489280516 2024-09-18T10:14:38.914 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.10 2024-09-18T10:14:39.445 INFO:teuthology.orchestra.run.smithi029.stdout:90194313221 2024-09-18T10:14:39.486 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313221 got 90194313221 for osd.4 2024-09-18T10:14:39.487 DEBUG:teuthology.parallel:result is None 2024-09-18T10:14:39.545 INFO:teuthology.orchestra.run.smithi029.stdout:90194313219 2024-09-18T10:14:39.578 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313219 got 90194313219 for osd.0 2024-09-18T10:14:39.579 DEBUG:teuthology.parallel:result is None 2024-09-18T10:14:39.625 INFO:teuthology.orchestra.run.smithi029.stdout:94489280516 2024-09-18T10:14:39.663 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280516 got 94489280516 for osd.13 2024-09-18T10:14:39.663 DEBUG:teuthology.parallel:result is None 2024-09-18T10:14:39.683 INFO:teuthology.orchestra.run.smithi029.stdout:94489280515 2024-09-18T10:14:39.689 INFO:teuthology.orchestra.run.smithi029.stdout:94489280515 2024-09-18T10:14:39.707 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280515 got 94489280515 for osd.1 2024-09-18T10:14:39.707 DEBUG:teuthology.parallel:result is None 2024-09-18T10:14:39.709 INFO:teuthology.orchestra.run.smithi029.stdout:94489280515 2024-09-18T10:14:39.713 INFO:teuthology.orchestra.run.smithi029.stdout:90194313220 2024-09-18T10:14:39.717 INFO:teuthology.orchestra.run.smithi029.stdout:94489280515 2024-09-18T10:14:39.722 INFO:teuthology.orchestra.run.smithi029.stdout:94489280516 2024-09-18T10:14:39.735 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280515 got 94489280515 for osd.7 2024-09-18T10:14:39.735 DEBUG:teuthology.parallel:result is None 2024-09-18T10:14:39.737 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313220 got 90194313220 for osd.12 2024-09-18T10:14:39.737 DEBUG:teuthology.parallel:result is None 2024-09-18T10:14:39.743 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280515 got 94489280515 for osd.8 2024-09-18T10:14:39.743 DEBUG:teuthology.parallel:result is None 2024-09-18T10:14:39.750 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280515 got 94489280515 for osd.3 2024-09-18T10:14:39.751 DEBUG:teuthology.parallel:result is None 2024-09-18T10:14:39.757 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280516 got 94489280516 for osd.10 2024-09-18T10:14:39.757 DEBUG:teuthology.parallel:result is None 2024-09-18T10:14:39.777 INFO:teuthology.orchestra.run.smithi029.stdout:94489280515 2024-09-18T10:14:39.794 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280515 got 94489280515 for osd.5 2024-09-18T10:14:39.794 DEBUG:teuthology.parallel:result is None 2024-09-18T10:14:39.797 INFO:teuthology.orchestra.run.smithi029.stdout:94489280515 2024-09-18T10:14:39.802 INFO:teuthology.orchestra.run.smithi029.stdout:90194313219 2024-09-18T10:14:39.814 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280515 got 94489280515 for osd.6 2024-09-18T10:14:39.814 DEBUG:teuthology.parallel:result is None 2024-09-18T10:14:39.819 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313219 got 90194313219 for osd.9 2024-09-18T10:14:39.819 DEBUG:teuthology.parallel:result is None 2024-09-18T10:14:39.821 INFO:teuthology.orchestra.run.smithi029.stdout:94489280515 2024-09-18T10:14:39.821 INFO:teuthology.orchestra.run.smithi029.stdout:94489280515 2024-09-18T10:14:39.826 INFO:teuthology.orchestra.run.smithi029.stdout:90194313219 2024-09-18T10:14:39.836 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280515 got 94489280515 for osd.15 2024-09-18T10:14:39.836 DEBUG:teuthology.parallel:result is None 2024-09-18T10:14:39.836 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280515 got 94489280515 for osd.2 2024-09-18T10:14:39.837 DEBUG:teuthology.parallel:result is None 2024-09-18T10:14:39.843 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313219 got 90194313219 for osd.11 2024-09-18T10:14:39.843 DEBUG:teuthology.parallel:result is None 2024-09-18T10:14:39.886 INFO:teuthology.orchestra.run.smithi029.stdout:94489280515 2024-09-18T10:14:39.899 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280515 got 94489280515 for osd.14 2024-09-18T10:14:39.900 DEBUG:teuthology.parallel:result is None 2024-09-18T10:14:39.900 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-18T10:14:39.900 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T10:14:40.193 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-18T10:14:40.196 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-09-18T10:14:40.208 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":19,"stamp":"2024-09-18T10:14:39.193873+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":64,"num_read_kb":53,"num_write":95,"num_write_kb":1435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":68,"ondisk_log_size":68,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":9,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":14,"kb":1509949440,"kb_used":4763524,"kb_used_data":5808,"kb_used_omap":26,"kb_used_meta":4757477,"kb_avail":1505185916,"statfs":{"total":1546188226560,"available":1541310377984,"internally_reserved":0,"allocated":5947392,"data_stored":2983132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":27264,"internal_metadata":4871656832},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,2,0,6,5],"upper_bound":1024},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":131111,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":20,"num_read_kb":18,"num_write":46,"num_write_kb":981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.881645"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-18T10:14:37.427288+0000","last_change":"2024-09-18T10:14:37.427722+0000","last_active":"2024-09-18T10:14:37.427288+0000","last_peered":"2024-09-18T10:14:37.427288+0000","last_clean":"2024-09-18T10:14:37.427288+0000","last_became_active":"2024-09-18T10:14:35.348943+0000","last_became_peered":"2024-09-18T10:14:35.348943+0000","last_unstale":"2024-09-18T10:14:37.427288+0000","last_undegraded":"2024-09-18T10:14:37.427288+0000","last_fullsized":"2024-09-18T10:14:37.427288+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:14:34.288596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:03.161727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060442999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,10],"acting":[13,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-18T10:14:35.531736+0000","last_change":"2024-09-18T10:14:35.530856+0000","last_active":"2024-09-18T10:14:35.531736+0000","last_peered":"2024-09-18T10:14:35.531736+0000","last_clean":"2024-09-18T10:14:35.531736+0000","last_became_active":"2024-09-18T10:14:35.429859+0000","last_became_peered":"2024-09-18T10:14:35.429859+0000","last_unstale":"2024-09-18T10:14:35.531736+0000","last_undegraded":"2024-09-18T10:14:35.531736+0000","last_fullsized":"2024-09-18T10:14:35.531736+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:14:34.288596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:15:40.196292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11],"acting":[8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":13,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-18T10:14:37.327453+0000","last_change":"2024-09-18T10:14:37.327877+0000","last_active":"2024-09-18T10:14:37.327453+0000","last_peered":"2024-09-18T10:14:37.327453+0000","last_clean":"2024-09-18T10:14:37.327453+0000","last_became_active":"2024-09-18T10:14:35.329198+0000","last_became_peered":"2024-09-18T10:14:35.329198+0000","last_unstale":"2024-09-18T10:14:37.327453+0000","last_undegraded":"2024-09-18T10:14:37.327453+0000","last_fullsized":"2024-09-18T10:14:37.327453+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:14:34.288596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:15:44.196046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062415699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-18T10:14:37.325628+0000","last_change":"2024-09-18T10:14:37.426192+0000","last_active":"2024-09-18T10:14:37.325628+0000","last_peered":"2024-09-18T10:14:37.325628+0000","last_clean":"2024-09-18T10:14:37.325628+0000","last_became_active":"2024-09-18T10:14:35.524102+0000","last_became_peered":"2024-09-18T10:14:35.524102+0000","last_unstale":"2024-09-18T10:14:37.325628+0000","last_undegraded":"2024-09-18T10:14:37.325628+0000","last_fullsized":"2024-09-18T10:14:37.325628+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:14:34.288596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:15:59.190171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.10077985,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"29'2","reported_seq":16,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-18T10:14:37.429583+0000","last_change":"2024-09-18T10:14:37.429583+0000","last_active":"2024-09-18T10:14:37.429583+0000","last_peered":"2024-09-18T10:14:37.429583+0000","last_clean":"2024-09-18T10:14:37.429583+0000","last_became_active":"2024-09-18T10:14:35.715374+0000","last_became_peered":"2024-09-18T10:14:35.715374+0000","last_unstale":"2024-09-18T10:14:37.429583+0000","last_undegraded":"2024-09-18T10:14:37.429583+0000","last_fullsized":"2024-09-18T10:14:37.429583+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:14:34.288596+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:03.520789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002421224,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,12],"acting":[13,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-18T10:14:37.600658+0000","last_change":"2024-09-18T10:14:37.601046+0000","last_active":"2024-09-18T10:14:37.600658+0000","last_peered":"2024-09-18T10:14:37.600658+0000","last_clean":"2024-09-18T10:14:37.600658+0000","last_became_active":"2024-09-18T10:14:35.348262+0000","last_became_peered":"2024-09-18T10:14:35.348262+0000","last_unstale":"2024-09-18T10:14:37.600658+0000","last_undegraded":"2024-09-18T10:14:37.600658+0000","last_fullsized":"2024-09-18T10:14:37.600658+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:14:34.288596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:03.520789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063430400000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,0],"acting":[14,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-18T10:14:37.600564+0000","last_change":"2024-09-18T10:14:37.600953+0000","last_active":"2024-09-18T10:14:37.600564+0000","last_peered":"2024-09-18T10:14:37.600564+0000","last_clean":"2024-09-18T10:14:37.600564+0000","last_became_active":"2024-09-18T10:14:35.349203+0000","last_became_peered":"2024-09-18T10:14:35.349203+0000","last_unstale":"2024-09-18T10:14:37.600564+0000","last_undegraded":"2024-09-18T10:14:37.600564+0000","last_fullsized":"2024-09-18T10:14:37.600564+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:14:34.288596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:15:37.868112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060130600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3],"acting":[9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"27'1","reported_seq":15,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-18T10:14:37.700725+0000","last_change":"2024-09-18T10:14:37.801252+0000","last_active":"2024-09-18T10:14:37.700725+0000","last_peered":"2024-09-18T10:14:37.700725+0000","last_clean":"2024-09-18T10:14:37.700725+0000","last_became_active":"2024-09-18T10:14:35.487746+0000","last_became_peered":"2024-09-18T10:14:35.487746+0000","last_unstale":"2024-09-18T10:14:37.700725+0000","last_undegraded":"2024-09-18T10:14:37.700725+0000","last_fullsized":"2024-09-18T10:14:37.700725+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:14:34.288596+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:03.161727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.10072764100000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,5],"acting":[14,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"29'65","reported_seq":93,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-18T10:14:38.614000+0000","last_change":"2024-09-18T10:14:32.509942+0000","last_active":"2024-09-18T10:14:38.614000+0000","last_peered":"2024-09-18T10:14:38.614000+0000","last_clean":"2024-09-18T10:14:38.614000+0000","last_became_active":"2024-09-18T10:14:32.508037+0000","last_became_peered":"2024-09-18T10:14:32.508037+0000","last_unstale":"2024-09-18T10:14:38.614000+0000","last_undegraded":"2024-09-18T10:14:38.614000+0000","last_fullsized":"2024-09-18T10:14:38.614000+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:31.267741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:31.267741+0000","last_clean_scrub_stamp":"2024-09-18T10:14:31.267741+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:15:59.262610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":64,"num_read_kb":53,"num_write":93,"num_write_kb":1433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,15],"acting":[6,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":64,"num_read_kb":53,"num_write":93,"num_write_kb":1433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":942080,"data_stored":934976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":65,"ondisk_log_size":65,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":15,"up_from":22,"seq":94489280515,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298120,"kb_used_data":768,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073720,"statfs":{"total":96636764160,"available":96331489280,"internally_reserved":0,"allocated":786432,"data_stored":597283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6,10,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98599999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.123}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.417}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90500000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0660000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.24}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.7170000000000001}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7130000000000001}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.5259999999999998}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.661}]}]},{"osd":14,"up_from":22,"seq":94489280515,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297664,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074176,"statfs":{"total":96636764160,"available":96331956224,"internally_reserved":0,"allocated":315392,"data_stored":129795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.048}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.27}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93799999999999994}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.097}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3480000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.583}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.323}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6339999999999999}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.877}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.526}]}]},{"osd":13,"up_from":22,"seq":94489280516,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297668,"kb_used_data":312,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074172,"statfs":{"total":96636764160,"available":96331952128,"internally_reserved":0,"allocated":319488,"data_stored":129814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93899999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.23}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0509999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.548}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.976}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.768}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.7879999999999998}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.0659999999999998}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.319}]}]},{"osd":0,"up_from":21,"seq":90194313219,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297668,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074172,"statfs":{"total":96636764160,"available":96331952128,"internally_reserved":0,"allocated":315392,"data_stored":129795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,9,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.341}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91900000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.091}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.298}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.401}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8400000000000001}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7889999999999999}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9079999999999999}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":23.806999999999999}]}]},{"osd":4,"up_from":21,"seq":90194313221,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297668,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074172,"statfs":{"total":96636764160,"available":96331952128,"internally_reserved":0,"allocated":315392,"data_stored":129795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,5,6,7,9,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68300000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79400000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.202}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.254}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4139999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.169}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3799999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7150000000000001}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.766}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9259999999999999}]}]},{"osd":9,"up_from":21,"seq":90194313219,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297668,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074172,"statfs":{"total":96636764160,"available":96331952128,"internally_reserved":0,"allocated":315392,"data_stored":129795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,3,4,8,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0509999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.008}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4850000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2589999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.377}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1060000000000001}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2889999999999999}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.157}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3520000000000001}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5700000000000001}]}]},{"osd":11,"up_from":21,"seq":90194313219,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297668,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074172,"statfs":{"total":96636764160,"available":96331952128,"internally_reserved":0,"allocated":315392,"data_stored":129795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,4,9,12],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57499999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83599999999999997}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89700000000000002}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94499999999999995}]}]},{"osd":12,"up_from":21,"seq":90194313220,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297672,"kb_used_data":312,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074168,"statfs":{"total":96636764160,"available":96331948032,"internally_reserved":0,"allocated":319488,"data_stored":129814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,9,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97199999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83899999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77300000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65200000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4419999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1339999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.335}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5269999999999999}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3720000000000001}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2070000000000001}]}]},{"osd":1,"up_from":22,"seq":94489280515,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297668,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074172,"statfs":{"total":96636764160,"available":96331952128,"internally_reserved":0,"allocated":315392,"data_stored":129795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7,8,10,12],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78200000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.083}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.585}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1659999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.55}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":84.939999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.339}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9510000000000001}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.27}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.4950000000000001}]}]},{"osd":2,"up_from":22,"seq":94489280515,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297664,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074176,"statfs":{"total":96636764160,"available":96331956224,"internally_reserved":0,"allocated":315392,"data_stored":129795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,3,4,5,6,7,8,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0429999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96899999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92500000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.089}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4610000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5009999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.536}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6699999999999999}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.716}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.752}]}]},{"osd":3,"up_from":22,"seq":94489280515,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297668,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074172,"statfs":{"total":96636764160,"available":96331952128,"internally_reserved":0,"allocated":315392,"data_stored":129795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,4,5,6,7,8,9,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86699999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.974}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3049999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4219999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.744}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1949999999999998}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8220000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9019999999999999}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.6859999999999999}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.012}]}]},{"osd":5,"up_from":22,"seq":94489280515,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297648,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074192,"statfs":{"total":96636764160,"available":96331972608,"internally_reserved":0,"allocated":299008,"data_stored":120261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[2,3,4,6,7,8,9,10,11,12],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81999999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.024}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2160000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.7960000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1269999999999998}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.347}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.833}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.9239999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.7829999999999999}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.708}]}]},{"osd":6,"up_from":22,"seq":94489280515,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298120,"kb_used_data":768,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073720,"statfs":{"total":96636764160,"available":96331489280,"internally_reserved":0,"allocated":786432,"data_stored":597283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[5,7,8,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,2,0,6,5],"upper_bound":1024},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83699999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89300000000000002}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":54.777999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":54.682000000000002}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2809999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.351}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3180000000000001}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.246}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":54.639000000000003}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5189999999999999}]}]},{"osd":7,"up_from":22,"seq":94489280515,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297664,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074176,"statfs":{"total":96636764160,"available":96331956224,"internally_reserved":0,"allocated":315392,"data_stored":129795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,6,8,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.278}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0710000000000002}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.778}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.861}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4219999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5529999999999999}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.6629999999999998}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.8450000000000002}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.5209999999999999}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.048}]}]},{"osd":8,"up_from":22,"seq":94489280515,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297648,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074192,"statfs":{"total":96636764160,"available":96331972608,"internally_reserved":0,"allocated":299008,"data_stored":120261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,7,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96199999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.073}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5049999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.879}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.4649999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9730000000000001}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0350000000000001}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9139999999999999}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.391}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2330000000000001}]}]},{"osd":10,"up_from":22,"seq":94489280516,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297648,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074192,"statfs":{"total":96636764160,"available":96331972608,"internally_reserved":0,"allocated":299008,"data_stored":120261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,7,9,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.335}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3799999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.056}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.446}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1879999999999999}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.8500000000000001}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1539999999999999}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3050000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":471040,"data_stored":467488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":471040,"data_stored":467488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T10:14:40.210 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T10:14:40.493 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-18T10:14:40.495 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-09-18T10:14:40.507 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":19,"stamp":"2024-09-18T10:14:39.193873+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":64,"num_read_kb":53,"num_write":95,"num_write_kb":1435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":68,"ondisk_log_size":68,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":9,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":14,"kb":1509949440,"kb_used":4763524,"kb_used_data":5808,"kb_used_omap":26,"kb_used_meta":4757477,"kb_avail":1505185916,"statfs":{"total":1546188226560,"available":1541310377984,"internally_reserved":0,"allocated":5947392,"data_stored":2983132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":27264,"internal_metadata":4871656832},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,2,0,6,5],"upper_bound":1024},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":131111,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":20,"num_read_kb":18,"num_write":46,"num_write_kb":981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.881645"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-18T10:14:37.427288+0000","last_change":"2024-09-18T10:14:37.427722+0000","last_active":"2024-09-18T10:14:37.427288+0000","last_peered":"2024-09-18T10:14:37.427288+0000","last_clean":"2024-09-18T10:14:37.427288+0000","last_became_active":"2024-09-18T10:14:35.348943+0000","last_became_peered":"2024-09-18T10:14:35.348943+0000","last_unstale":"2024-09-18T10:14:37.427288+0000","last_undegraded":"2024-09-18T10:14:37.427288+0000","last_fullsized":"2024-09-18T10:14:37.427288+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:14:34.288596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:03.161727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060442999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,10],"acting":[13,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-18T10:14:35.531736+0000","last_change":"2024-09-18T10:14:35.530856+0000","last_active":"2024-09-18T10:14:35.531736+0000","last_peered":"2024-09-18T10:14:35.531736+0000","last_clean":"2024-09-18T10:14:35.531736+0000","last_became_active":"2024-09-18T10:14:35.429859+0000","last_became_peered":"2024-09-18T10:14:35.429859+0000","last_unstale":"2024-09-18T10:14:35.531736+0000","last_undegraded":"2024-09-18T10:14:35.531736+0000","last_fullsized":"2024-09-18T10:14:35.531736+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:14:34.288596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:15:40.196292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11],"acting":[8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":13,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-18T10:14:37.327453+0000","last_change":"2024-09-18T10:14:37.327877+0000","last_active":"2024-09-18T10:14:37.327453+0000","last_peered":"2024-09-18T10:14:37.327453+0000","last_clean":"2024-09-18T10:14:37.327453+0000","last_became_active":"2024-09-18T10:14:35.329198+0000","last_became_peered":"2024-09-18T10:14:35.329198+0000","last_unstale":"2024-09-18T10:14:37.327453+0000","last_undegraded":"2024-09-18T10:14:37.327453+0000","last_fullsized":"2024-09-18T10:14:37.327453+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:14:34.288596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:15:44.196046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062415699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-18T10:14:37.325628+0000","last_change":"2024-09-18T10:14:37.426192+0000","last_active":"2024-09-18T10:14:37.325628+0000","last_peered":"2024-09-18T10:14:37.325628+0000","last_clean":"2024-09-18T10:14:37.325628+0000","last_became_active":"2024-09-18T10:14:35.524102+0000","last_became_peered":"2024-09-18T10:14:35.524102+0000","last_unstale":"2024-09-18T10:14:37.325628+0000","last_undegraded":"2024-09-18T10:14:37.325628+0000","last_fullsized":"2024-09-18T10:14:37.325628+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:14:34.288596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:15:59.190171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.10077985,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"29'2","reported_seq":16,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-18T10:14:37.429583+0000","last_change":"2024-09-18T10:14:37.429583+0000","last_active":"2024-09-18T10:14:37.429583+0000","last_peered":"2024-09-18T10:14:37.429583+0000","last_clean":"2024-09-18T10:14:37.429583+0000","last_became_active":"2024-09-18T10:14:35.715374+0000","last_became_peered":"2024-09-18T10:14:35.715374+0000","last_unstale":"2024-09-18T10:14:37.429583+0000","last_undegraded":"2024-09-18T10:14:37.429583+0000","last_fullsized":"2024-09-18T10:14:37.429583+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:14:34.288596+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:03.520789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002421224,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,12],"acting":[13,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-18T10:14:37.600658+0000","last_change":"2024-09-18T10:14:37.601046+0000","last_active":"2024-09-18T10:14:37.600658+0000","last_peered":"2024-09-18T10:14:37.600658+0000","last_clean":"2024-09-18T10:14:37.600658+0000","last_became_active":"2024-09-18T10:14:35.348262+0000","last_became_peered":"2024-09-18T10:14:35.348262+0000","last_unstale":"2024-09-18T10:14:37.600658+0000","last_undegraded":"2024-09-18T10:14:37.600658+0000","last_fullsized":"2024-09-18T10:14:37.600658+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:14:34.288596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:03.520789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063430400000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,0],"acting":[14,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-18T10:14:37.600564+0000","last_change":"2024-09-18T10:14:37.600953+0000","last_active":"2024-09-18T10:14:37.600564+0000","last_peered":"2024-09-18T10:14:37.600564+0000","last_clean":"2024-09-18T10:14:37.600564+0000","last_became_active":"2024-09-18T10:14:35.349203+0000","last_became_peered":"2024-09-18T10:14:35.349203+0000","last_unstale":"2024-09-18T10:14:37.600564+0000","last_undegraded":"2024-09-18T10:14:37.600564+0000","last_fullsized":"2024-09-18T10:14:37.600564+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:14:34.288596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:15:37.868112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060130600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3],"acting":[9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"27'1","reported_seq":15,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-18T10:14:37.700725+0000","last_change":"2024-09-18T10:14:37.801252+0000","last_active":"2024-09-18T10:14:37.700725+0000","last_peered":"2024-09-18T10:14:37.700725+0000","last_clean":"2024-09-18T10:14:37.700725+0000","last_became_active":"2024-09-18T10:14:35.487746+0000","last_became_peered":"2024-09-18T10:14:35.487746+0000","last_unstale":"2024-09-18T10:14:37.700725+0000","last_undegraded":"2024-09-18T10:14:37.700725+0000","last_fullsized":"2024-09-18T10:14:37.700725+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:14:34.288596+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:03.161727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.10072764100000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,5],"acting":[14,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"29'65","reported_seq":93,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-18T10:14:38.614000+0000","last_change":"2024-09-18T10:14:32.509942+0000","last_active":"2024-09-18T10:14:38.614000+0000","last_peered":"2024-09-18T10:14:38.614000+0000","last_clean":"2024-09-18T10:14:38.614000+0000","last_became_active":"2024-09-18T10:14:32.508037+0000","last_became_peered":"2024-09-18T10:14:32.508037+0000","last_unstale":"2024-09-18T10:14:38.614000+0000","last_undegraded":"2024-09-18T10:14:38.614000+0000","last_fullsized":"2024-09-18T10:14:38.614000+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:31.267741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:31.267741+0000","last_clean_scrub_stamp":"2024-09-18T10:14:31.267741+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:15:59.262610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":64,"num_read_kb":53,"num_write":93,"num_write_kb":1433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,15],"acting":[6,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":64,"num_read_kb":53,"num_write":93,"num_write_kb":1433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":942080,"data_stored":934976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":65,"ondisk_log_size":65,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":15,"up_from":22,"seq":94489280515,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298120,"kb_used_data":768,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073720,"statfs":{"total":96636764160,"available":96331489280,"internally_reserved":0,"allocated":786432,"data_stored":597283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6,10,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98599999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.123}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.417}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90500000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0660000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.24}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.7170000000000001}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7130000000000001}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.5259999999999998}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.661}]}]},{"osd":14,"up_from":22,"seq":94489280515,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297664,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074176,"statfs":{"total":96636764160,"available":96331956224,"internally_reserved":0,"allocated":315392,"data_stored":129795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.048}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.27}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93799999999999994}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.097}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3480000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.583}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.323}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6339999999999999}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.877}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.526}]}]},{"osd":13,"up_from":22,"seq":94489280516,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297668,"kb_used_data":312,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074172,"statfs":{"total":96636764160,"available":96331952128,"internally_reserved":0,"allocated":319488,"data_stored":129814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93899999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.23}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0509999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.548}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.976}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.768}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.7879999999999998}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.0659999999999998}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.319}]}]},{"osd":0,"up_from":21,"seq":90194313219,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297668,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074172,"statfs":{"total":96636764160,"available":96331952128,"internally_reserved":0,"allocated":315392,"data_stored":129795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,9,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.341}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91900000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.091}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.298}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.401}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8400000000000001}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7889999999999999}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9079999999999999}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":23.806999999999999}]}]},{"osd":4,"up_from":21,"seq":90194313221,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297668,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074172,"statfs":{"total":96636764160,"available":96331952128,"internally_reserved":0,"allocated":315392,"data_stored":129795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,5,6,7,9,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68300000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79400000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.202}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.254}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4139999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.169}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3799999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7150000000000001}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.766}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9259999999999999}]}]},{"osd":9,"up_from":21,"seq":90194313219,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297668,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074172,"statfs":{"total":96636764160,"available":96331952128,"internally_reserved":0,"allocated":315392,"data_stored":129795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,3,4,8,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0509999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.008}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4850000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2589999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.377}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1060000000000001}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2889999999999999}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.157}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3520000000000001}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5700000000000001}]}]},{"osd":11,"up_from":21,"seq":90194313219,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297668,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074172,"statfs":{"total":96636764160,"available":96331952128,"internally_reserved":0,"allocated":315392,"data_stored":129795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,4,9,12],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57499999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83599999999999997}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89700000000000002}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94499999999999995}]}]},{"osd":12,"up_from":21,"seq":90194313220,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297672,"kb_used_data":312,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074168,"statfs":{"total":96636764160,"available":96331948032,"internally_reserved":0,"allocated":319488,"data_stored":129814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,9,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97199999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83899999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77300000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65200000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4419999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1339999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.335}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5269999999999999}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3720000000000001}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2070000000000001}]}]},{"osd":1,"up_from":22,"seq":94489280515,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297668,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074172,"statfs":{"total":96636764160,"available":96331952128,"internally_reserved":0,"allocated":315392,"data_stored":129795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7,8,10,12],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78200000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.083}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.585}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1659999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.55}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":84.939999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.339}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9510000000000001}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.27}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.4950000000000001}]}]},{"osd":2,"up_from":22,"seq":94489280515,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297664,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074176,"statfs":{"total":96636764160,"available":96331956224,"internally_reserved":0,"allocated":315392,"data_stored":129795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,3,4,5,6,7,8,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0429999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96899999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92500000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.089}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4610000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5009999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.536}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6699999999999999}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.716}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.752}]}]},{"osd":3,"up_from":22,"seq":94489280515,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297668,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074172,"statfs":{"total":96636764160,"available":96331952128,"internally_reserved":0,"allocated":315392,"data_stored":129795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,4,5,6,7,8,9,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86699999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.974}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3049999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4219999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.744}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1949999999999998}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8220000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9019999999999999}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.6859999999999999}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.012}]}]},{"osd":5,"up_from":22,"seq":94489280515,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297648,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074192,"statfs":{"total":96636764160,"available":96331972608,"internally_reserved":0,"allocated":299008,"data_stored":120261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[2,3,4,6,7,8,9,10,11,12],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81999999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.024}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2160000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.7960000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1269999999999998}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.347}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.833}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.9239999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.7829999999999999}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.708}]}]},{"osd":6,"up_from":22,"seq":94489280515,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298120,"kb_used_data":768,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073720,"statfs":{"total":96636764160,"available":96331489280,"internally_reserved":0,"allocated":786432,"data_stored":597283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[5,7,8,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,2,0,6,5],"upper_bound":1024},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83699999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89300000000000002}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":54.777999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":54.682000000000002}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2809999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.351}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3180000000000001}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.246}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":54.639000000000003}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5189999999999999}]}]},{"osd":7,"up_from":22,"seq":94489280515,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297664,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074176,"statfs":{"total":96636764160,"available":96331956224,"internally_reserved":0,"allocated":315392,"data_stored":129795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,6,8,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.278}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0710000000000002}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.778}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.861}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4219999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5529999999999999}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.6629999999999998}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.8450000000000002}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.5209999999999999}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.048}]}]},{"osd":8,"up_from":22,"seq":94489280515,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297648,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074192,"statfs":{"total":96636764160,"available":96331972608,"internally_reserved":0,"allocated":299008,"data_stored":120261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,7,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96199999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.073}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5049999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.879}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.4649999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9730000000000001}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0350000000000001}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9139999999999999}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.391}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2330000000000001}]}]},{"osd":10,"up_from":22,"seq":94489280516,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297648,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074192,"statfs":{"total":96636764160,"available":96331972608,"internally_reserved":0,"allocated":299008,"data_stored":120261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,7,9,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.335}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3799999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.056}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.446}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1879999999999999}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.8500000000000001}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1539999999999999}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3050000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":471040,"data_stored":467488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":471040,"data_stored":467488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T10:14:40.509 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-18T10:14:40.509 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-18T10:14:40.509 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-18T10:14:40.509 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-18T10:14:40.825 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-18T10:14:40.825 INFO:teuthology.orchestra.run.smithi029.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-18T10:14:40.838 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-18T10:14:40.838 INFO:teuthology.run_tasks:Running task thrashosds... 2024-09-18T10:14:40.850 INFO:tasks.thrashosds:config is {'aggressive_pg_num_changes': False, 'chance_pgnum_grow': 1, 'chance_pgnum_shrink': 1, 'chance_pgpnum_fix': 1, 'min_in': 8, 'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2024-09-18T10:14:40.850 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'wip-bharath11-testing-2024-09-18-1115-quincy'}, 'ceph': {'conf': {'global': {'mon election default strategy': 1, 'osd debug inject dispatch delay duration': 0.1, 'osd debug inject dispatch delay probability': 0.1}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug reject backfill probability': 0.3, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'but it is still running', 'objects unfound and apparently lost', 'slow request', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'mons down', 'mon down', 'out of quorum'], 'sha1': '7061ea6ca50908d104e9f3979dc615e118d9e153'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '7061ea6ca50908d104e9f3979dc615e118d9e153'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-bharath11-testing-2024-09-18-1115-quincy', 'sha1': '7061ea6ca50908d104e9f3979dc615e118d9e153'}} 2024-09-18T10:14:40.850 INFO:tasks.thrashosds:config is {'aggressive_pg_num_changes': False, 'chance_pgnum_grow': 1, 'chance_pgnum_shrink': 1, 'chance_pgpnum_fix': 1, 'min_in': 8, 'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0, 'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5} 2024-09-18T10:14:40.850 INFO:tasks.thrashosds:Beginning thrashosds... 2024-09-18T10:14:40.850 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-18T10:14:40.851 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T10:14:41.128 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-18T10:14:41.131 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-09-18T10:14:41.143 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":19,"stamp":"2024-09-18T10:14:39.193873+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":64,"num_read_kb":53,"num_write":95,"num_write_kb":1435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":68,"ondisk_log_size":68,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":9,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":14,"kb":1509949440,"kb_used":4763524,"kb_used_data":5808,"kb_used_omap":26,"kb_used_meta":4757477,"kb_avail":1505185916,"statfs":{"total":1546188226560,"available":1541310377984,"internally_reserved":0,"allocated":5947392,"data_stored":2983132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":27264,"internal_metadata":4871656832},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,2,0,6,5],"upper_bound":1024},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":131111,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":20,"num_read_kb":18,"num_write":46,"num_write_kb":981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.881645"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-18T10:14:37.427288+0000","last_change":"2024-09-18T10:14:37.427722+0000","last_active":"2024-09-18T10:14:37.427288+0000","last_peered":"2024-09-18T10:14:37.427288+0000","last_clean":"2024-09-18T10:14:37.427288+0000","last_became_active":"2024-09-18T10:14:35.348943+0000","last_became_peered":"2024-09-18T10:14:35.348943+0000","last_unstale":"2024-09-18T10:14:37.427288+0000","last_undegraded":"2024-09-18T10:14:37.427288+0000","last_fullsized":"2024-09-18T10:14:37.427288+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:14:34.288596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:03.161727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060442999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,10],"acting":[13,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-18T10:14:35.531736+0000","last_change":"2024-09-18T10:14:35.530856+0000","last_active":"2024-09-18T10:14:35.531736+0000","last_peered":"2024-09-18T10:14:35.531736+0000","last_clean":"2024-09-18T10:14:35.531736+0000","last_became_active":"2024-09-18T10:14:35.429859+0000","last_became_peered":"2024-09-18T10:14:35.429859+0000","last_unstale":"2024-09-18T10:14:35.531736+0000","last_undegraded":"2024-09-18T10:14:35.531736+0000","last_fullsized":"2024-09-18T10:14:35.531736+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:14:34.288596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:15:40.196292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11],"acting":[8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":13,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-18T10:14:37.327453+0000","last_change":"2024-09-18T10:14:37.327877+0000","last_active":"2024-09-18T10:14:37.327453+0000","last_peered":"2024-09-18T10:14:37.327453+0000","last_clean":"2024-09-18T10:14:37.327453+0000","last_became_active":"2024-09-18T10:14:35.329198+0000","last_became_peered":"2024-09-18T10:14:35.329198+0000","last_unstale":"2024-09-18T10:14:37.327453+0000","last_undegraded":"2024-09-18T10:14:37.327453+0000","last_fullsized":"2024-09-18T10:14:37.327453+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:14:34.288596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:15:44.196046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062415699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-18T10:14:37.325628+0000","last_change":"2024-09-18T10:14:37.426192+0000","last_active":"2024-09-18T10:14:37.325628+0000","last_peered":"2024-09-18T10:14:37.325628+0000","last_clean":"2024-09-18T10:14:37.325628+0000","last_became_active":"2024-09-18T10:14:35.524102+0000","last_became_peered":"2024-09-18T10:14:35.524102+0000","last_unstale":"2024-09-18T10:14:37.325628+0000","last_undegraded":"2024-09-18T10:14:37.325628+0000","last_fullsized":"2024-09-18T10:14:37.325628+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:14:34.288596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:15:59.190171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.10077985,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"29'2","reported_seq":16,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-18T10:14:37.429583+0000","last_change":"2024-09-18T10:14:37.429583+0000","last_active":"2024-09-18T10:14:37.429583+0000","last_peered":"2024-09-18T10:14:37.429583+0000","last_clean":"2024-09-18T10:14:37.429583+0000","last_became_active":"2024-09-18T10:14:35.715374+0000","last_became_peered":"2024-09-18T10:14:35.715374+0000","last_unstale":"2024-09-18T10:14:37.429583+0000","last_undegraded":"2024-09-18T10:14:37.429583+0000","last_fullsized":"2024-09-18T10:14:37.429583+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:14:34.288596+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:03.520789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002421224,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,12],"acting":[13,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-18T10:14:37.600658+0000","last_change":"2024-09-18T10:14:37.601046+0000","last_active":"2024-09-18T10:14:37.600658+0000","last_peered":"2024-09-18T10:14:37.600658+0000","last_clean":"2024-09-18T10:14:37.600658+0000","last_became_active":"2024-09-18T10:14:35.348262+0000","last_became_peered":"2024-09-18T10:14:35.348262+0000","last_unstale":"2024-09-18T10:14:37.600658+0000","last_undegraded":"2024-09-18T10:14:37.600658+0000","last_fullsized":"2024-09-18T10:14:37.600658+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:14:34.288596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:03.520789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063430400000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,0],"acting":[14,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-18T10:14:37.600564+0000","last_change":"2024-09-18T10:14:37.600953+0000","last_active":"2024-09-18T10:14:37.600564+0000","last_peered":"2024-09-18T10:14:37.600564+0000","last_clean":"2024-09-18T10:14:37.600564+0000","last_became_active":"2024-09-18T10:14:35.349203+0000","last_became_peered":"2024-09-18T10:14:35.349203+0000","last_unstale":"2024-09-18T10:14:37.600564+0000","last_undegraded":"2024-09-18T10:14:37.600564+0000","last_fullsized":"2024-09-18T10:14:37.600564+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:14:34.288596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:15:37.868112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060130600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3],"acting":[9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"27'1","reported_seq":15,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-18T10:14:37.700725+0000","last_change":"2024-09-18T10:14:37.801252+0000","last_active":"2024-09-18T10:14:37.700725+0000","last_peered":"2024-09-18T10:14:37.700725+0000","last_clean":"2024-09-18T10:14:37.700725+0000","last_became_active":"2024-09-18T10:14:35.487746+0000","last_became_peered":"2024-09-18T10:14:35.487746+0000","last_unstale":"2024-09-18T10:14:37.700725+0000","last_undegraded":"2024-09-18T10:14:37.700725+0000","last_fullsized":"2024-09-18T10:14:37.700725+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:14:34.288596+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:03.161727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.10072764100000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,5],"acting":[14,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"29'65","reported_seq":93,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-18T10:14:38.614000+0000","last_change":"2024-09-18T10:14:32.509942+0000","last_active":"2024-09-18T10:14:38.614000+0000","last_peered":"2024-09-18T10:14:38.614000+0000","last_clean":"2024-09-18T10:14:38.614000+0000","last_became_active":"2024-09-18T10:14:32.508037+0000","last_became_peered":"2024-09-18T10:14:32.508037+0000","last_unstale":"2024-09-18T10:14:38.614000+0000","last_undegraded":"2024-09-18T10:14:38.614000+0000","last_fullsized":"2024-09-18T10:14:38.614000+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:31.267741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:31.267741+0000","last_clean_scrub_stamp":"2024-09-18T10:14:31.267741+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:15:59.262610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":64,"num_read_kb":53,"num_write":93,"num_write_kb":1433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,15],"acting":[6,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":64,"num_read_kb":53,"num_write":93,"num_write_kb":1433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":942080,"data_stored":934976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":65,"ondisk_log_size":65,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":15,"up_from":22,"seq":94489280515,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298120,"kb_used_data":768,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073720,"statfs":{"total":96636764160,"available":96331489280,"internally_reserved":0,"allocated":786432,"data_stored":597283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6,10,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98599999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.123}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.417}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90500000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0660000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.24}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.7170000000000001}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7130000000000001}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.5259999999999998}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.661}]}]},{"osd":14,"up_from":22,"seq":94489280515,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297664,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074176,"statfs":{"total":96636764160,"available":96331956224,"internally_reserved":0,"allocated":315392,"data_stored":129795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.048}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.27}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93799999999999994}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.097}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3480000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.583}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.323}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6339999999999999}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.877}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.526}]}]},{"osd":13,"up_from":22,"seq":94489280516,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297668,"kb_used_data":312,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074172,"statfs":{"total":96636764160,"available":96331952128,"internally_reserved":0,"allocated":319488,"data_stored":129814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93899999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.23}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0509999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.548}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.976}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.768}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.7879999999999998}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.0659999999999998}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.319}]}]},{"osd":0,"up_from":21,"seq":90194313219,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297668,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074172,"statfs":{"total":96636764160,"available":96331952128,"internally_reserved":0,"allocated":315392,"data_stored":129795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,9,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.341}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91900000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.091}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.298}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.401}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8400000000000001}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7889999999999999}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9079999999999999}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":23.806999999999999}]}]},{"osd":4,"up_from":21,"seq":90194313221,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297668,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074172,"statfs":{"total":96636764160,"available":96331952128,"internally_reserved":0,"allocated":315392,"data_stored":129795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,5,6,7,9,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68300000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79400000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.202}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.254}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4139999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.169}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3799999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7150000000000001}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.766}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9259999999999999}]}]},{"osd":9,"up_from":21,"seq":90194313219,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297668,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074172,"statfs":{"total":96636764160,"available":96331952128,"internally_reserved":0,"allocated":315392,"data_stored":129795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,3,4,8,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0509999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.008}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4850000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2589999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.377}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1060000000000001}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2889999999999999}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.157}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3520000000000001}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5700000000000001}]}]},{"osd":11,"up_from":21,"seq":90194313219,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297668,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074172,"statfs":{"total":96636764160,"available":96331952128,"internally_reserved":0,"allocated":315392,"data_stored":129795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,4,9,12],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57499999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83599999999999997}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89700000000000002}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94499999999999995}]}]},{"osd":12,"up_from":21,"seq":90194313220,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297672,"kb_used_data":312,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074168,"statfs":{"total":96636764160,"available":96331948032,"internally_reserved":0,"allocated":319488,"data_stored":129814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,9,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97199999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83899999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77300000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65200000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4419999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1339999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.335}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5269999999999999}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3720000000000001}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2070000000000001}]}]},{"osd":1,"up_from":22,"seq":94489280515,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297668,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074172,"statfs":{"total":96636764160,"available":96331952128,"internally_reserved":0,"allocated":315392,"data_stored":129795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7,8,10,12],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78200000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.083}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.585}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1659999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.55}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":84.939999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.339}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9510000000000001}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.27}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.4950000000000001}]}]},{"osd":2,"up_from":22,"seq":94489280515,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297664,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074176,"statfs":{"total":96636764160,"available":96331956224,"internally_reserved":0,"allocated":315392,"data_stored":129795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,3,4,5,6,7,8,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0429999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96899999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92500000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.089}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4610000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5009999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.536}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6699999999999999}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.716}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.752}]}]},{"osd":3,"up_from":22,"seq":94489280515,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297668,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074172,"statfs":{"total":96636764160,"available":96331952128,"internally_reserved":0,"allocated":315392,"data_stored":129795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,4,5,6,7,8,9,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86699999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.974}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3049999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4219999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.744}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1949999999999998}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8220000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9019999999999999}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.6859999999999999}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.012}]}]},{"osd":5,"up_from":22,"seq":94489280515,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297648,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074192,"statfs":{"total":96636764160,"available":96331972608,"internally_reserved":0,"allocated":299008,"data_stored":120261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[2,3,4,6,7,8,9,10,11,12],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81999999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.024}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2160000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.7960000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1269999999999998}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.347}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.833}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.9239999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.7829999999999999}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.708}]}]},{"osd":6,"up_from":22,"seq":94489280515,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298120,"kb_used_data":768,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073720,"statfs":{"total":96636764160,"available":96331489280,"internally_reserved":0,"allocated":786432,"data_stored":597283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[5,7,8,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,2,0,6,5],"upper_bound":1024},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83699999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89300000000000002}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":54.777999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":54.682000000000002}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2809999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.351}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3180000000000001}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.246}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":54.639000000000003}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5189999999999999}]}]},{"osd":7,"up_from":22,"seq":94489280515,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297664,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074176,"statfs":{"total":96636764160,"available":96331956224,"internally_reserved":0,"allocated":315392,"data_stored":129795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,6,8,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.278}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0710000000000002}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.778}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.861}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4219999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5529999999999999}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.6629999999999998}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.8450000000000002}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.5209999999999999}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.048}]}]},{"osd":8,"up_from":22,"seq":94489280515,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297648,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074192,"statfs":{"total":96636764160,"available":96331972608,"internally_reserved":0,"allocated":299008,"data_stored":120261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,7,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96199999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.073}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5049999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.879}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.4649999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9730000000000001}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0350000000000001}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9139999999999999}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.391}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2330000000000001}]}]},{"osd":10,"up_from":22,"seq":94489280516,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297648,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074192,"statfs":{"total":96636764160,"available":96331972608,"internally_reserved":0,"allocated":299008,"data_stored":120261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,7,9,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.335}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3799999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.056}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.446}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1879999999999999}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.8500000000000001}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1539999999999999}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3050000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":471040,"data_stored":467488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":471040,"data_stored":467488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T10:14:41.145 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T10:14:41.425 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-18T10:14:41.427 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-09-18T10:14:41.442 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2024-09-18T10:14:41.194640+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":64,"num_read_kb":53,"num_write":108,"num_write_kb":1889,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":81,"ondisk_log_size":81,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":14,"kb":1509949440,"kb_used":4764080,"kb_used_data":5856,"kb_used_omap":26,"kb_used_meta":4757477,"kb_avail":1505185360,"statfs":{"total":1546188226560,"available":1541309808640,"internally_reserved":0,"allocated":5996544,"data_stored":3011734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":27264,"internal_metadata":4871656832},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,0,0,0,0,1],"upper_bound":1024},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":131111,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":20,"num_read_kb":18,"num_write":59,"num_write_kb":1435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.882412"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-18T10:14:37.427288+0000","last_change":"2024-09-18T10:14:37.427722+0000","last_active":"2024-09-18T10:14:37.427288+0000","last_peered":"2024-09-18T10:14:37.427288+0000","last_clean":"2024-09-18T10:14:37.427288+0000","last_became_active":"2024-09-18T10:14:35.348943+0000","last_became_peered":"2024-09-18T10:14:35.348943+0000","last_unstale":"2024-09-18T10:14:37.427288+0000","last_undegraded":"2024-09-18T10:14:37.427288+0000","last_fullsized":"2024-09-18T10:14:37.427288+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:14:34.288596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:03.161727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060442999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,10],"acting":[13,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-18T10:14:38.963061+0000","last_change":"2024-09-18T10:14:38.963300+0000","last_active":"2024-09-18T10:14:38.963061+0000","last_peered":"2024-09-18T10:14:38.963061+0000","last_clean":"2024-09-18T10:14:38.963061+0000","last_became_active":"2024-09-18T10:14:35.429859+0000","last_became_peered":"2024-09-18T10:14:35.429859+0000","last_unstale":"2024-09-18T10:14:38.963061+0000","last_undegraded":"2024-09-18T10:14:38.963061+0000","last_fullsized":"2024-09-18T10:14:38.963061+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:14:34.288596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:15:40.196292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037634499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11],"acting":[8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":13,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-18T10:14:37.327453+0000","last_change":"2024-09-18T10:14:37.327877+0000","last_active":"2024-09-18T10:14:37.327453+0000","last_peered":"2024-09-18T10:14:37.327453+0000","last_clean":"2024-09-18T10:14:37.327453+0000","last_became_active":"2024-09-18T10:14:35.329198+0000","last_became_peered":"2024-09-18T10:14:35.329198+0000","last_unstale":"2024-09-18T10:14:37.327453+0000","last_undegraded":"2024-09-18T10:14:37.327453+0000","last_fullsized":"2024-09-18T10:14:37.327453+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:14:34.288596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:15:44.196046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062415699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-18T10:14:37.325628+0000","last_change":"2024-09-18T10:14:37.426192+0000","last_active":"2024-09-18T10:14:37.325628+0000","last_peered":"2024-09-18T10:14:37.325628+0000","last_clean":"2024-09-18T10:14:37.325628+0000","last_became_active":"2024-09-18T10:14:35.524102+0000","last_became_peered":"2024-09-18T10:14:35.524102+0000","last_unstale":"2024-09-18T10:14:37.325628+0000","last_undegraded":"2024-09-18T10:14:37.325628+0000","last_fullsized":"2024-09-18T10:14:37.325628+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:14:34.288596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:15:59.190171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.10077985,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"29'2","reported_seq":16,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-18T10:14:37.429583+0000","last_change":"2024-09-18T10:14:37.429583+0000","last_active":"2024-09-18T10:14:37.429583+0000","last_peered":"2024-09-18T10:14:37.429583+0000","last_clean":"2024-09-18T10:14:37.429583+0000","last_became_active":"2024-09-18T10:14:35.715374+0000","last_became_peered":"2024-09-18T10:14:35.715374+0000","last_unstale":"2024-09-18T10:14:37.429583+0000","last_undegraded":"2024-09-18T10:14:37.429583+0000","last_fullsized":"2024-09-18T10:14:37.429583+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:14:34.288596+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:03.520789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002421224,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,12],"acting":[13,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-18T10:14:37.600658+0000","last_change":"2024-09-18T10:14:37.601046+0000","last_active":"2024-09-18T10:14:37.600658+0000","last_peered":"2024-09-18T10:14:37.600658+0000","last_clean":"2024-09-18T10:14:37.600658+0000","last_became_active":"2024-09-18T10:14:35.348262+0000","last_became_peered":"2024-09-18T10:14:35.348262+0000","last_unstale":"2024-09-18T10:14:37.600658+0000","last_undegraded":"2024-09-18T10:14:37.600658+0000","last_fullsized":"2024-09-18T10:14:37.600658+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:14:34.288596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:03.520789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063430400000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,0],"acting":[14,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-18T10:14:37.600564+0000","last_change":"2024-09-18T10:14:37.600953+0000","last_active":"2024-09-18T10:14:37.600564+0000","last_peered":"2024-09-18T10:14:37.600564+0000","last_clean":"2024-09-18T10:14:37.600564+0000","last_became_active":"2024-09-18T10:14:35.349203+0000","last_became_peered":"2024-09-18T10:14:35.349203+0000","last_unstale":"2024-09-18T10:14:37.600564+0000","last_undegraded":"2024-09-18T10:14:37.600564+0000","last_fullsized":"2024-09-18T10:14:37.600564+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:14:34.288596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:15:37.868112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060130600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3],"acting":[9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"27'1","reported_seq":15,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-18T10:14:37.700725+0000","last_change":"2024-09-18T10:14:37.801252+0000","last_active":"2024-09-18T10:14:37.700725+0000","last_peered":"2024-09-18T10:14:37.700725+0000","last_clean":"2024-09-18T10:14:37.700725+0000","last_became_active":"2024-09-18T10:14:35.487746+0000","last_became_peered":"2024-09-18T10:14:35.487746+0000","last_unstale":"2024-09-18T10:14:37.700725+0000","last_undegraded":"2024-09-18T10:14:37.700725+0000","last_fullsized":"2024-09-18T10:14:37.700725+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:14:34.288596+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:03.161727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.10072764100000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,5],"acting":[14,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"29'78","reported_seq":109,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-18T10:14:40.238622+0000","last_change":"2024-09-18T10:14:32.509942+0000","last_active":"2024-09-18T10:14:40.238622+0000","last_peered":"2024-09-18T10:14:40.238622+0000","last_clean":"2024-09-18T10:14:40.238622+0000","last_became_active":"2024-09-18T10:14:32.508037+0000","last_became_peered":"2024-09-18T10:14:32.508037+0000","last_unstale":"2024-09-18T10:14:40.238622+0000","last_undegraded":"2024-09-18T10:14:40.238622+0000","last_fullsized":"2024-09-18T10:14:40.238622+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:31.267741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:31.267741+0000","last_clean_scrub_stamp":"2024-09-18T10:14:31.267741+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:15:59.262610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":64,"num_read_kb":53,"num_write":106,"num_write_kb":1887,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,15],"acting":[6,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":64,"num_read_kb":53,"num_write":106,"num_write_kb":1887,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":942080,"data_stored":934976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":78,"ondisk_log_size":78,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":15,"up_from":22,"seq":94489280516,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298496,"kb_used_data":768,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073344,"statfs":{"total":96636764160,"available":96331104256,"internally_reserved":0,"allocated":786432,"data_stored":597283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6,10,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76400000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0149999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.123}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.407}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3120000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.224}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.454}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.379}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.25}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6339999999999999}]}]},{"osd":14,"up_from":22,"seq":94489280516,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297664,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074176,"statfs":{"total":96636764160,"available":96331956224,"internally_reserved":0,"allocated":315392,"data_stored":129795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.048}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.27}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93799999999999994}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.097}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3480000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.583}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.323}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6339999999999999}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.877}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.526}]}]},{"osd":13,"up_from":22,"seq":94489280517,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297668,"kb_used_data":312,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074172,"statfs":{"total":96636764160,"available":96331952128,"internally_reserved":0,"allocated":319488,"data_stored":129814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,10,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84099999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.23}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88400000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.274}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.369}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.137}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":140.02199999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6479999999999999}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.228}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5149999999999999}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6850000000000001}]}]},{"osd":0,"up_from":21,"seq":90194313220,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297668,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074172,"statfs":{"total":96636764160,"available":96331952128,"internally_reserved":0,"allocated":315392,"data_stored":129795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,9,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.341}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91900000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.091}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.298}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.401}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8400000000000001}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7889999999999999}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9079999999999999}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":23.806999999999999}]}]},{"osd":4,"up_from":21,"seq":90194313222,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297668,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074172,"statfs":{"total":96636764160,"available":96331952128,"internally_reserved":0,"allocated":315392,"data_stored":129795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,5,6,7,9,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68300000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79400000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.202}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.254}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4139999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.169}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3799999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7150000000000001}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.766}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9259999999999999}]}]},{"osd":9,"up_from":21,"seq":90194313220,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297668,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074172,"statfs":{"total":96636764160,"available":96331952128,"internally_reserved":0,"allocated":315392,"data_stored":129795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,3,4,8,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0509999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.008}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4850000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2589999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.377}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1060000000000001}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2889999999999999}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.157}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3520000000000001}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5700000000000001}]}]},{"osd":11,"up_from":21,"seq":90194313220,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297668,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074172,"statfs":{"total":96636764160,"available":96331952128,"internally_reserved":0,"allocated":315392,"data_stored":129795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,4,9,12],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57499999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83599999999999997}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89700000000000002}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94499999999999995}]}]},{"osd":12,"up_from":21,"seq":90194313221,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297672,"kb_used_data":312,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074168,"statfs":{"total":96636764160,"available":96331948032,"internally_reserved":0,"allocated":319488,"data_stored":129814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,9,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97199999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83899999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77300000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65200000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4419999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1339999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.335}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5269999999999999}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3720000000000001}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2070000000000001}]}]},{"osd":1,"up_from":22,"seq":94489280516,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297668,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074172,"statfs":{"total":96636764160,"available":96331952128,"internally_reserved":0,"allocated":315392,"data_stored":129795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7,8,10,12],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.629}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1259999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0009999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91800000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88400000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.345}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.208}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2709999999999999}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0589999999999999}]}]},{"osd":2,"up_from":22,"seq":94489280516,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297664,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074176,"statfs":{"total":96636764160,"available":96331956224,"internally_reserved":0,"allocated":315392,"data_stored":129795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,3,4,5,6,7,8,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83599999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0760000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.745}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.04}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.246}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3680000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.383}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2789999999999999}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4710000000000001}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4179999999999999}]}]},{"osd":3,"up_from":22,"seq":94489280516,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297668,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074172,"statfs":{"total":96636764160,"available":96331952128,"internally_reserved":0,"allocated":315392,"data_stored":129795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,4,5,6,7,8,9,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0049999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71099999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0940000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91600000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74399999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81200000000000006}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1699999999999999}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2649999999999999}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3240000000000001}]}]},{"osd":5,"up_from":22,"seq":94489280516,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297668,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074172,"statfs":{"total":96636764160,"available":96331952128,"internally_reserved":0,"allocated":315392,"data_stored":129795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[2,3,4,6,7,8,9,10,11,12],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79500000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0429999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84999999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.024}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2829999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3089999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4530000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4259999999999999}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.407}]}]},{"osd":6,"up_from":22,"seq":94489280516,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298240,"kb_used_data":768,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073600,"statfs":{"total":96636764160,"available":96331366400,"internally_reserved":0,"allocated":786432,"data_stored":597283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[5,7,8,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,0,0,0,0,1],"upper_bound":1024},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84199999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0169999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.874}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.278}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3109999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4059999999999999}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.25}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.347}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6220000000000001}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6499999999999999}]}]},{"osd":7,"up_from":22,"seq":94489280516,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297664,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074176,"statfs":{"total":96636764160,"available":96331956224,"internally_reserved":0,"allocated":315392,"data_stored":129795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,4,6,8,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.008}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76800000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.304}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72899999999999998}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.861}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2549999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.377}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4470000000000001}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5469999999999999}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.419}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5860000000000001}]}]},{"osd":8,"up_from":22,"seq":94489280516,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297668,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074172,"statfs":{"total":96636764160,"available":96331952128,"internally_reserved":0,"allocated":315392,"data_stored":129795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,7,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70599999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78900000000000003}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89400000000000002}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95299999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2889999999999999}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0980000000000001}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.115}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.522}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1399999999999999}]}]},{"osd":10,"up_from":22,"seq":94489280517,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297668,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074172,"statfs":{"total":96636764160,"available":96331952128,"internally_reserved":0,"allocated":315392,"data_stored":129795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,7,9,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68100000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78200000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.224}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93300000000000005}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.254}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1100000000000001}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.161}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.369}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4910000000000001}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.286}]}]}],"pool_statfs":[{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":471040,"data_stored":467488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":471040,"data_stored":467488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T10:14:41.444 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-18T10:14:41.444 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2024-09-18T10:14:41.717 INFO:teuthology.orchestra.run.smithi029.stdout:epoch 29 2024-09-18T10:14:41.718 INFO:teuthology.orchestra.run.smithi029.stdout:fsid ab6d6beb-0627-4b7f-8cdb-4e0a44970f10 2024-09-18T10:14:41.718 INFO:teuthology.orchestra.run.smithi029.stdout:created 2024-09-18T10:14:18.827644+0000 2024-09-18T10:14:41.718 INFO:teuthology.orchestra.run.smithi029.stdout:modified 2024-09-18T10:14:37.318779+0000 2024-09-18T10:14:41.718 INFO:teuthology.orchestra.run.smithi029.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-18T10:14:41.718 INFO:teuthology.orchestra.run.smithi029.stdout:crush_version 5 2024-09-18T10:14:41.718 INFO:teuthology.orchestra.run.smithi029.stdout:full_ratio 0.95 2024-09-18T10:14:41.718 INFO:teuthology.orchestra.run.smithi029.stdout:backfillfull_ratio 0.9 2024-09-18T10:14:41.719 INFO:teuthology.orchestra.run.smithi029.stdout:nearfull_ratio 0.85 2024-09-18T10:14:41.719 INFO:teuthology.orchestra.run.smithi029.stdout:require_min_compat_client luminous 2024-09-18T10:14:41.719 INFO:teuthology.orchestra.run.smithi029.stdout:min_compat_client jewel 2024-09-18T10:14:41.719 INFO:teuthology.orchestra.run.smithi029.stdout:require_osd_release quincy 2024-09-18T10:14:41.719 INFO:teuthology.orchestra.run.smithi029.stdout:stretch_mode_enabled false 2024-09-18T10:14:41.719 INFO:teuthology.orchestra.run.smithi029.stdout:pool 1 '.mgr' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 25 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr 2024-09-18T10:14:41.719 INFO:teuthology.orchestra.run.smithi029.stdout:pool 2 'rbd' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 29 flags hashpspool,selfmanaged_snaps stripe_width 0 application rbd 2024-09-18T10:14:41.719 INFO:teuthology.orchestra.run.smithi029.stdout: removed_snaps_queue [2~1] 2024-09-18T10:14:41.719 INFO:teuthology.orchestra.run.smithi029.stdout:max_osd 16 2024-09-18T10:14:41.719 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0 up in weight 1 up_from 21 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.29:6808/22197,v1:172.21.15.29:6809/22197] [v2:172.21.15.29:6810/22197,v1:172.21.15.29:6811/22197] exists,up 39cf0511-0009-433f-92f2-e5686f33d713 2024-09-18T10:14:41.720 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1 up in weight 1 up_from 22 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.136:6808/22140,v1:172.21.15.136:6809/22140] [v2:172.21.15.136:6810/22140,v1:172.21.15.136:6811/22140] exists,up a0bf06ab-b98e-4130-91d2-7d4283540c08 2024-09-18T10:14:41.720 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2 up in weight 1 up_from 22 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.154:6824/22095,v1:172.21.15.154:6825/22095] [v2:172.21.15.154:6826/22095,v1:172.21.15.154:6827/22095] exists,up 742a786e-6129-4fbc-bb5b-e2e3c03c2d77 2024-09-18T10:14:41.720 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3 up in weight 1 up_from 22 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.189:6802/22404,v1:172.21.15.189:6803/22404] [v2:172.21.15.189:6804/22404,v1:172.21.15.189:6805/22404] exists,up 3aa77114-24ed-49ab-86ff-a0600794cbf8 2024-09-18T10:14:41.720 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4 up in weight 1 up_from 21 up_thru 26 down_at 0 last_clean_interval [0,0) [v2:172.21.15.29:6800/22195,v1:172.21.15.29:6801/22195] [v2:172.21.15.29:6802/22195,v1:172.21.15.29:6803/22195] exists,up 426567ad-b0be-41d4-afcf-dc56ec3974cd 2024-09-18T10:14:41.720 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5 up in weight 1 up_from 22 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.136:6817/22141,v1:172.21.15.136:6819/22141] [v2:172.21.15.136:6821/22141,v1:172.21.15.136:6823/22141] exists,up d9559e42-1f4e-4373-8392-a536a0afd871 2024-09-18T10:14:41.720 INFO:teuthology.orchestra.run.smithi029.stdout:osd.6 up in weight 1 up_from 22 up_thru 23 down_at 0 last_clean_interval [0,0) [v2:172.21.15.154:6800/22096,v1:172.21.15.154:6801/22096] [v2:172.21.15.154:6802/22096,v1:172.21.15.154:6803/22096] exists,up d50ef1e2-8bc5-4a90-9be3-365a0db2be6f 2024-09-18T10:14:41.720 INFO:teuthology.orchestra.run.smithi029.stdout:osd.7 up in weight 1 up_from 22 up_thru 26 down_at 0 last_clean_interval [0,0) [v2:172.21.15.189:6818/22405,v1:172.21.15.189:6819/22405] [v2:172.21.15.189:6820/22405,v1:172.21.15.189:6821/22405] exists,up bd0b85e4-cfee-4d19-90a5-79854794b48b 2024-09-18T10:14:41.720 INFO:teuthology.orchestra.run.smithi029.stdout:osd.8 up in weight 1 up_from 22 up_thru 26 down_at 0 last_clean_interval [0,0) [v2:172.21.15.29:6824/22200,v1:172.21.15.29:6825/22200] [v2:172.21.15.29:6826/22200,v1:172.21.15.29:6827/22200] exists,up cd333f56-c165-45d7-8c8b-48d34bc4ddc5 2024-09-18T10:14:41.720 INFO:teuthology.orchestra.run.smithi029.stdout:osd.9 up in weight 1 up_from 21 up_thru 26 down_at 0 last_clean_interval [0,0) [v2:172.21.15.136:6800/22142,v1:172.21.15.136:6801/22142] [v2:172.21.15.136:6802/22142,v1:172.21.15.136:6803/22142] exists,up 1e079e4c-8801-4754-9d0d-d0b68695d32e 2024-09-18T10:14:41.720 INFO:teuthology.orchestra.run.smithi029.stdout:osd.10 up in weight 1 up_from 22 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.154:6808/22097,v1:172.21.15.154:6809/22097] [v2:172.21.15.154:6810/22097,v1:172.21.15.154:6811/22097] exists,up 92ee68d4-371c-4849-930f-a8fa18d64b99 2024-09-18T10:14:41.720 INFO:teuthology.orchestra.run.smithi029.stdout:osd.11 up in weight 1 up_from 21 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.189:6826/22416,v1:172.21.15.189:6827/22416] [v2:172.21.15.189:6828/22416,v1:172.21.15.189:6829/22416] exists,up 634ddaf5-f7b8-4b27-a46a-04e21b73fcf0 2024-09-18T10:14:41.721 INFO:teuthology.orchestra.run.smithi029.stdout:osd.12 up in weight 1 up_from 21 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.29:6816/22210,v1:172.21.15.29:6817/22210] [v2:172.21.15.29:6818/22210,v1:172.21.15.29:6819/22210] exists,up bc58137b-5948-400d-b377-193b15824d8d 2024-09-18T10:14:41.721 INFO:teuthology.orchestra.run.smithi029.stdout:osd.13 up in weight 1 up_from 22 up_thru 26 down_at 0 last_clean_interval [0,0) [v2:172.21.15.136:6816/22143,v1:172.21.15.136:6818/22143] [v2:172.21.15.136:6820/22143,v1:172.21.15.136:6822/22143] exists,up a8b753a7-ab69-4853-ac36-66d9b85c2e20 2024-09-18T10:14:41.721 INFO:teuthology.orchestra.run.smithi029.stdout:osd.14 up in weight 1 up_from 22 up_thru 26 down_at 0 last_clean_interval [0,0) [v2:172.21.15.154:6816/22098,v1:172.21.15.154:6817/22098] [v2:172.21.15.154:6818/22098,v1:172.21.15.154:6819/22098] exists,up 8ad0848f-0de4-4a75-9c63-8c4a4a7b8524 2024-09-18T10:14:41.721 INFO:teuthology.orchestra.run.smithi029.stdout:osd.15 up in weight 1 up_from 22 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.189:6810/22415,v1:172.21.15.189:6811/22415] [v2:172.21.15.189:6812/22415,v1:172.21.15.189:6813/22415] exists,up 069099b3-ec1a-483b-b967-98cddbf2f678 2024-09-18T10:14:41.729 INFO:tasks.ceph.ceph_manager.ceph:['osd.0 up in weight 1 up_from 21 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.29:6808/22197,v1:172.21.15.29:6809/22197] [v2:172.21.15.29:6810/22197,v1:172.21.15.29:6811/22197] exists,up 39cf0511-0009-433f-92f2-e5686f33d713', 'osd.1 up in weight 1 up_from 22 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.136:6808/22140,v1:172.21.15.136:6809/22140] [v2:172.21.15.136:6810/22140,v1:172.21.15.136:6811/22140] exists,up a0bf06ab-b98e-4130-91d2-7d4283540c08', 'osd.2 up in weight 1 up_from 22 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.154:6824/22095,v1:172.21.15.154:6825/22095] [v2:172.21.15.154:6826/22095,v1:172.21.15.154:6827/22095] exists,up 742a786e-6129-4fbc-bb5b-e2e3c03c2d77', 'osd.3 up in weight 1 up_from 22 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.189:6802/22404,v1:172.21.15.189:6803/22404] [v2:172.21.15.189:6804/22404,v1:172.21.15.189:6805/22404] exists,up 3aa77114-24ed-49ab-86ff-a0600794cbf8', 'osd.4 up in weight 1 up_from 21 up_thru 26 down_at 0 last_clean_interval [0,0) [v2:172.21.15.29:6800/22195,v1:172.21.15.29:6801/22195] [v2:172.21.15.29:6802/22195,v1:172.21.15.29:6803/22195] exists,up 426567ad-b0be-41d4-afcf-dc56ec3974cd', 'osd.5 up in weight 1 up_from 22 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.136:6817/22141,v1:172.21.15.136:6819/22141] [v2:172.21.15.136:6821/22141,v1:172.21.15.136:6823/22141] exists,up d9559e42-1f4e-4373-8392-a536a0afd871', 'osd.6 up in weight 1 up_from 22 up_thru 23 down_at 0 last_clean_interval [0,0) [v2:172.21.15.154:6800/22096,v1:172.21.15.154:6801/22096] [v2:172.21.15.154:6802/22096,v1:172.21.15.154:6803/22096] exists,up d50ef1e2-8bc5-4a90-9be3-365a0db2be6f', 'osd.7 up in weight 1 up_from 22 up_thru 26 down_at 0 last_clean_interval [0,0) [v2:172.21.15.189:6818/22405,v1:172.21.15.189:6819/22405] [v2:172.21.15.189:6820/22405,v1:172.21.15.189:6821/22405] exists,up bd0b85e4-cfee-4d19-90a5-79854794b48b', 'osd.8 up in weight 1 up_from 22 up_thru 26 down_at 0 last_clean_interval [0,0) [v2:172.21.15.29:6824/22200,v1:172.21.15.29:6825/22200] [v2:172.21.15.29:6826/22200,v1:172.21.15.29:6827/22200] exists,up cd333f56-c165-45d7-8c8b-48d34bc4ddc5', 'osd.9 up in weight 1 up_from 21 up_thru 26 down_at 0 last_clean_interval [0,0) [v2:172.21.15.136:6800/22142,v1:172.21.15.136:6801/22142] [v2:172.21.15.136:6802/22142,v1:172.21.15.136:6803/22142] exists,up 1e079e4c-8801-4754-9d0d-d0b68695d32e', 'osd.10 up in weight 1 up_from 22 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.154:6808/22097,v1:172.21.15.154:6809/22097] [v2:172.21.15.154:6810/22097,v1:172.21.15.154:6811/22097] exists,up 92ee68d4-371c-4849-930f-a8fa18d64b99', 'osd.11 up in weight 1 up_from 21 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.189:6826/22416,v1:172.21.15.189:6827/22416] [v2:172.21.15.189:6828/22416,v1:172.21.15.189:6829/22416] exists,up 634ddaf5-f7b8-4b27-a46a-04e21b73fcf0', 'osd.12 up in weight 1 up_from 21 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.29:6816/22210,v1:172.21.15.29:6817/22210] [v2:172.21.15.29:6818/22210,v1:172.21.15.29:6819/22210] exists,up bc58137b-5948-400d-b377-193b15824d8d', 'osd.13 up in weight 1 up_from 22 up_thru 26 down_at 0 last_clean_interval [0,0) [v2:172.21.15.136:6816/22143,v1:172.21.15.136:6818/22143] [v2:172.21.15.136:6820/22143,v1:172.21.15.136:6822/22143] exists,up a8b753a7-ab69-4853-ac36-66d9b85c2e20', 'osd.14 up in weight 1 up_from 22 up_thru 26 down_at 0 last_clean_interval [0,0) [v2:172.21.15.154:6816/22098,v1:172.21.15.154:6817/22098] [v2:172.21.15.154:6818/22098,v1:172.21.15.154:6819/22098] exists,up 8ad0848f-0de4-4a75-9c63-8c4a4a7b8524', 'osd.15 up in weight 1 up_from 22 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.189:6810/22415,v1:172.21.15.189:6811/22415] [v2:172.21.15.189:6812/22415,v1:172.21.15.189:6813/22415] exists,up 069099b3-ec1a-483b-b967-98cddbf2f678'] 2024-09-18T10:14:41.730 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok config show 2024-09-18T10:14:41.824 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-18T10:14:41.825 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "mon.a", 2024-09-18T10:14:41.825 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster": "ceph", 2024-09-18T10:14:41.825 INFO:teuthology.orchestra.run.smithi029.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2024-09-18T10:14:41.825 INFO:teuthology.orchestra.run.smithi029.stdout: "admin_socket_mode": "", 2024-09-18T10:14:41.825 INFO:teuthology.orchestra.run.smithi029.stdout: "auth_allow_insecure_global_id_reclaim": "true", 2024-09-18T10:14:41.825 INFO:teuthology.orchestra.run.smithi029.stdout: "auth_client_required": "cephx, none", 2024-09-18T10:14:41.826 INFO:teuthology.orchestra.run.smithi029.stdout: "auth_cluster_required": "cephx", 2024-09-18T10:14:41.826 INFO:teuthology.orchestra.run.smithi029.stdout: "auth_debug": "true", 2024-09-18T10:14:41.826 INFO:teuthology.orchestra.run.smithi029.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2024-09-18T10:14:41.826 INFO:teuthology.orchestra.run.smithi029.stdout: "auth_mon_ticket_ttl": "660.000000", 2024-09-18T10:14:41.826 INFO:teuthology.orchestra.run.smithi029.stdout: "auth_service_required": "cephx", 2024-09-18T10:14:41.826 INFO:teuthology.orchestra.run.smithi029.stdout: "auth_service_ticket_ttl": "240.000000", 2024-09-18T10:14:41.826 INFO:teuthology.orchestra.run.smithi029.stdout: "auth_supported": "cephx", 2024-09-18T10:14:41.826 INFO:teuthology.orchestra.run.smithi029.stdout: "bdev_aio": "true", 2024-09-18T10:14:41.826 INFO:teuthology.orchestra.run.smithi029.stdout: "bdev_aio_max_queue_depth": "1024", 2024-09-18T10:14:41.826 INFO:teuthology.orchestra.run.smithi029.stdout: "bdev_aio_poll_ms": "250", 2024-09-18T10:14:41.826 INFO:teuthology.orchestra.run.smithi029.stdout: "bdev_aio_reap_max": "16", 2024-09-18T10:14:41.826 INFO:teuthology.orchestra.run.smithi029.stdout: "bdev_async_discard": "false", 2024-09-18T10:14:41.826 INFO:teuthology.orchestra.run.smithi029.stdout: "bdev_block_size": "4096", 2024-09-18T10:14:41.827 INFO:teuthology.orchestra.run.smithi029.stdout: "bdev_debug_aio": "false", 2024-09-18T10:14:41.827 INFO:teuthology.orchestra.run.smithi029.stdout: "bdev_debug_aio_log_age": "5.000000", 2024-09-18T10:14:41.827 INFO:teuthology.orchestra.run.smithi029.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2024-09-18T10:14:41.827 INFO:teuthology.orchestra.run.smithi029.stdout: "bdev_debug_inflight_ios": "false", 2024-09-18T10:14:41.827 INFO:teuthology.orchestra.run.smithi029.stdout: "bdev_enable_discard": "false", 2024-09-18T10:14:41.827 INFO:teuthology.orchestra.run.smithi029.stdout: "bdev_flock_retry": "3", 2024-09-18T10:14:41.827 INFO:teuthology.orchestra.run.smithi029.stdout: "bdev_flock_retry_interval": "0.100000", 2024-09-18T10:14:41.827 INFO:teuthology.orchestra.run.smithi029.stdout: "bdev_inject_crash": "0", 2024-09-18T10:14:41.827 INFO:teuthology.orchestra.run.smithi029.stdout: "bdev_inject_crash_flush_delay": "2", 2024-09-18T10:14:41.828 INFO:teuthology.orchestra.run.smithi029.stdout: "bdev_ioring": "false", 2024-09-18T10:14:41.828 INFO:teuthology.orchestra.run.smithi029.stdout: "bdev_ioring_hipri": "false", 2024-09-18T10:14:41.828 INFO:teuthology.orchestra.run.smithi029.stdout: "bdev_ioring_sqthread_poll": "false", 2024-09-18T10:14:41.828 INFO:teuthology.orchestra.run.smithi029.stdout: "bdev_nvme_unbind_from_kernel": "false", 2024-09-18T10:14:41.828 INFO:teuthology.orchestra.run.smithi029.stdout: "bdev_read_buffer_alignment": "4096", 2024-09-18T10:14:41.828 INFO:teuthology.orchestra.run.smithi029.stdout: "bdev_read_preallocated_huge_buffers": "", 2024-09-18T10:14:41.828 INFO:teuthology.orchestra.run.smithi029.stdout: "bdev_type": "", 2024-09-18T10:14:41.828 INFO:teuthology.orchestra.run.smithi029.stdout: "bluefs_alloc_size": "1048576", 2024-09-18T10:14:41.828 INFO:teuthology.orchestra.run.smithi029.stdout: "bluefs_allocator": "hybrid", 2024-09-18T10:14:41.828 INFO:teuthology.orchestra.run.smithi029.stdout: "bluefs_buffered_io": "true", 2024-09-18T10:14:41.829 INFO:teuthology.orchestra.run.smithi029.stdout: "bluefs_check_for_zeros": "false", 2024-09-18T10:14:41.829 INFO:teuthology.orchestra.run.smithi029.stdout: "bluefs_check_volume_selector_often": "false", 2024-09-18T10:14:41.829 INFO:teuthology.orchestra.run.smithi029.stdout: "bluefs_check_volume_selector_on_umount": "false", 2024-09-18T10:14:41.829 INFO:teuthology.orchestra.run.smithi029.stdout: "bluefs_compact_log_sync": "false", 2024-09-18T10:14:41.829 INFO:teuthology.orchestra.run.smithi029.stdout: "bluefs_failed_shared_alloc_cooldown": "600.000000", 2024-09-18T10:14:41.829 INFO:teuthology.orchestra.run.smithi029.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2024-09-18T10:14:41.829 INFO:teuthology.orchestra.run.smithi029.stdout: "bluefs_log_compact_min_size": "16777216", 2024-09-18T10:14:41.829 INFO:teuthology.orchestra.run.smithi029.stdout: "bluefs_log_replay_check_allocations": "true", 2024-09-18T10:14:41.829 INFO:teuthology.orchestra.run.smithi029.stdout: "bluefs_max_log_runway": "4194304", 2024-09-18T10:14:41.829 INFO:teuthology.orchestra.run.smithi029.stdout: "bluefs_max_prefetch": "1048576", 2024-09-18T10:14:41.829 INFO:teuthology.orchestra.run.smithi029.stdout: "bluefs_min_flush_size": "524288", 2024-09-18T10:14:41.829 INFO:teuthology.orchestra.run.smithi029.stdout: "bluefs_min_log_runway": "1048576", 2024-09-18T10:14:41.830 INFO:teuthology.orchestra.run.smithi029.stdout: "bluefs_replay_recovery": "false", 2024-09-18T10:14:41.830 INFO:teuthology.orchestra.run.smithi029.stdout: "bluefs_replay_recovery_disable_compact": "false", 2024-09-18T10:14:41.830 INFO:teuthology.orchestra.run.smithi029.stdout: "bluefs_shared_alloc_size": "65536", 2024-09-18T10:14:41.830 INFO:teuthology.orchestra.run.smithi029.stdout: "bluefs_sync_write": "false", 2024-09-18T10:14:41.830 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2024-09-18T10:14:41.830 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2024-09-18T10:14:41.830 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2024-09-18T10:14:41.830 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_allocation_from_file": "true", 2024-09-18T10:14:41.830 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_allocator": "hybrid", 2024-09-18T10:14:41.830 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_async_db_compaction": "true", 2024-09-18T10:14:41.830 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2024-09-18T10:14:41.830 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2024-09-18T10:14:41.831 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2024-09-18T10:14:41.831 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2024-09-18T10:14:41.831 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2024-09-18T10:14:41.831 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_bitmapallocator_span_size": "1024", 2024-09-18T10:14:41.831 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_blobid_prealloc": "10240", 2024-09-18T10:14:41.831 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_block_create": "true", 2024-09-18T10:14:41.831 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_block_db_create": "false", 2024-09-18T10:14:41.831 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_block_db_path": "", 2024-09-18T10:14:41.831 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_block_db_size": "0", 2024-09-18T10:14:41.831 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_block_path": "", 2024-09-18T10:14:41.832 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_block_preallocate_file": "false", 2024-09-18T10:14:41.832 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_block_size": "107374182400", 2024-09-18T10:14:41.832 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_block_wal_create": "false", 2024-09-18T10:14:41.832 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_block_wal_path": "", 2024-09-18T10:14:41.832 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_block_wal_size": "100663296", 2024-09-18T10:14:41.832 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_bluefs": "true", 2024-09-18T10:14:41.832 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2024-09-18T10:14:41.832 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_bluefs_env_mirror": "false", 2024-09-18T10:14:41.832 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_bluefs_max_free": "10737418240", 2024-09-18T10:14:41.833 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_age_bin_interval": "1.000000", 2024-09-18T10:14:41.833 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_age_bins_data": "1 2 6 24 120 720 0 0 0 0", 2024-09-18T10:14:41.833 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_age_bins_kv": "1 2 6 24 120 720 0 0 0 0", 2024-09-18T10:14:41.833 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_age_bins_kv_onode": "0 0 0 0 0 0 0 0 0 720", 2024-09-18T10:14:41.833 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_age_bins_meta": "1 2 6 24 120 720 0 0 0 0", 2024-09-18T10:14:41.833 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_autotune": "true", 2024-09-18T10:14:41.833 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_autotune_interval": "5.000000", 2024-09-18T10:14:41.833 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2024-09-18T10:14:41.833 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_kv_ratio": "0.450000", 2024-09-18T10:14:41.833 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_meta_ratio": "0.450000", 2024-09-18T10:14:41.833 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_size": "0", 2024-09-18T10:14:41.834 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_size_hdd": "1073741824", 2024-09-18T10:14:41.834 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_size_ssd": "3221225472", 2024-09-18T10:14:41.834 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_trim_interval": "0.050000", 2024-09-18T10:14:41.834 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2024-09-18T10:14:41.834 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_type": "2q", 2024-09-18T10:14:41.834 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cleaner_sleep_interval": "5.000000", 2024-09-18T10:14:41.834 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_clone_cow": "true", 2024-09-18T10:14:41.834 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_compression_algorithm": "snappy", 2024-09-18T10:14:41.834 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_compression_max_blob_size": "0", 2024-09-18T10:14:41.835 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2024-09-18T10:14:41.835 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2024-09-18T10:14:41.835 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_compression_min_blob_size": "0", 2024-09-18T10:14:41.835 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2024-09-18T10:14:41.835 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_compression_min_blob_size_ssd": "65536", 2024-09-18T10:14:41.835 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_compression_mode": "none", 2024-09-18T10:14:41.835 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_compression_required_ratio": "0.875000", 2024-09-18T10:14:41.835 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_csum_type": "crc32c", 2024-09-18T10:14:41.835 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_debug_enforce_settings": "default", 2024-09-18T10:14:41.836 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_debug_freelist": "false", 2024-09-18T10:14:41.836 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_debug_fsck_abort": "false", 2024-09-18T10:14:41.836 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_debug_inject_allocation_from_file_failure": "0.000000", 2024-09-18T10:14:41.836 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_debug_inject_bug21040": "false", 2024-09-18T10:14:41.836 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2024-09-18T10:14:41.836 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_debug_inject_read_err": "false", 2024-09-18T10:14:41.837 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_debug_legacy_omap": "false", 2024-09-18T10:14:41.837 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_debug_no_reuse_blocks": "false", 2024-09-18T10:14:41.837 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_debug_omit_block_device_write": "false", 2024-09-18T10:14:41.837 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_debug_omit_kv_commit": "false", 2024-09-18T10:14:41.837 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2024-09-18T10:14:41.837 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_debug_prefill": "0.000000", 2024-09-18T10:14:41.838 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_debug_prefragment_max": "1048576", 2024-09-18T10:14:41.838 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_debug_random_read_err": "0.000000", 2024-09-18T10:14:41.838 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2024-09-18T10:14:41.838 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_debug_small_allocations": "0", 2024-09-18T10:14:41.838 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2024-09-18T10:14:41.838 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_default_buffered_read": "true", 2024-09-18T10:14:41.838 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_default_buffered_write": "false", 2024-09-18T10:14:41.838 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_deferred_batch_ops": "0", 2024-09-18T10:14:41.838 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2024-09-18T10:14:41.838 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2024-09-18T10:14:41.839 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2024-09-18T10:14:41.839 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_extent_map_shard_max_size": "1200", 2024-09-18T10:14:41.839 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_extent_map_shard_min_size": "150", 2024-09-18T10:14:41.839 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_extent_map_shard_target_size": "500", 2024-09-18T10:14:41.839 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2024-09-18T10:14:41.839 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_fail_eio": "false", 2024-09-18T10:14:41.839 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_freelist_blocks_per_key": "128", 2024-09-18T10:14:41.839 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2024-09-18T10:14:41.839 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2024-09-18T10:14:41.839 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2024-09-18T10:14:41.840 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_fsck_on_mkfs": "true", 2024-09-18T10:14:41.840 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2024-09-18T10:14:41.840 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_fsck_on_mount": "false", 2024-09-18T10:14:41.840 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_fsck_on_mount_deep": "false", 2024-09-18T10:14:41.840 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_fsck_on_umount": "false", 2024-09-18T10:14:41.840 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_fsck_on_umount_deep": "false", 2024-09-18T10:14:41.840 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2024-09-18T10:14:41.840 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_fsck_quick_fix_threads": "2", 2024-09-18T10:14:41.840 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2024-09-18T10:14:41.840 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2024-09-18T10:14:41.840 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_gc_enable_blob_threshold": "0", 2024-09-18T10:14:41.840 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_gc_enable_total_threshold": "0", 2024-09-18T10:14:41.840 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2024-09-18T10:14:41.841 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_ignore_data_csum": "false", 2024-09-18T10:14:41.841 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2024-09-18T10:14:41.841 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_kvbackend": "rocksdb", 2024-09-18T10:14:41.841 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_log_collection_list_age": "60.000000", 2024-09-18T10:14:41.841 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2024-09-18T10:14:41.841 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_log_op_age": "5.000000", 2024-09-18T10:14:41.841 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_max_alloc_size": "0", 2024-09-18T10:14:41.841 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_max_blob_size": "0", 2024-09-18T10:14:41.841 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_max_blob_size_hdd": "65536", 2024-09-18T10:14:41.841 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_max_blob_size_ssd": "65536", 2024-09-18T10:14:41.841 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_max_defer_interval": "3.000000", 2024-09-18T10:14:41.842 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_max_deferred_txc": "32", 2024-09-18T10:14:41.842 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_min_alloc_size": "0", 2024-09-18T10:14:41.842 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_min_alloc_size_hdd": "4096", 2024-09-18T10:14:41.842 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_min_alloc_size_ssd": "4096", 2024-09-18T10:14:41.842 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_nid_prealloc": "1024", 2024-09-18T10:14:41.842 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_prefer_deferred_size": "0", 2024-09-18T10:14:41.842 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2024-09-18T10:14:41.842 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2024-09-18T10:14:41.842 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_qfsck_on_mount": "true", 2024-09-18T10:14:41.842 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_retry_disk_reads": "3", 2024-09-18T10:14:41.843 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_rocksdb_cf": "true", 2024-09-18T10:14:41.843 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L P", 2024-09-18T10:14:41.843 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_rocksdb_options": "compression=kNoCompression,max_write_buffer_number=4,min_write_buffer_number_to_merge=1,recycle_log_file_num=4,write_buffer_size=268435456,writable_file_max_buffer_size=0,compaction_readahead_size=2097152,max_background_compactions=2,max_total_wal_size=1073741824", 2024-09-18T10:14:41.843 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_rocksdb_options_annex": "", 2024-09-18T10:14:41.843 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_spdk_coremask": "0x1", 2024-09-18T10:14:41.844 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_spdk_io_sleep": "5", 2024-09-18T10:14:41.844 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_spdk_max_io_completion": "0", 2024-09-18T10:14:41.844 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_spdk_mem": "512", 2024-09-18T10:14:41.844 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_sync_submit_transaction": "false", 2024-09-18T10:14:41.844 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_throttle_bytes": "67108864", 2024-09-18T10:14:41.844 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_throttle_cost_per_io": "0", 2024-09-18T10:14:41.845 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2024-09-18T10:14:41.845 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2024-09-18T10:14:41.845 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2024-09-18T10:14:41.845 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_throttle_trace_rate": "0.000000", 2024-09-18T10:14:41.845 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_tracing": "false", 2024-09-18T10:14:41.845 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_use_optimal_io_size_for_min_alloc_size": "false", 2024-09-18T10:14:41.846 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2024-09-18T10:14:41.846 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_volume_selection_reserved": "0", 2024-09-18T10:14:41.846 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2024-09-18T10:14:41.846 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2024-09-18T10:14:41.846 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_warn_on_legacy_statfs": "true", 2024-09-18T10:14:41.846 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2024-09-18T10:14:41.847 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2024-09-18T10:14:41.847 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2024-09-18T10:14:41.847 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_zero_block_detection": "false", 2024-09-18T10:14:41.847 INFO:teuthology.orchestra.run.smithi029.stdout: "cephadm_path": "/usr/sbin/cephadm", 2024-09-18T10:14:41.847 INFO:teuthology.orchestra.run.smithi029.stdout: "cephfs_mirror_action_update_interval": "2", 2024-09-18T10:14:41.847 INFO:teuthology.orchestra.run.smithi029.stdout: "cephfs_mirror_directory_scan_interval": "10", 2024-09-18T10:14:41.848 INFO:teuthology.orchestra.run.smithi029.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2024-09-18T10:14:41.848 INFO:teuthology.orchestra.run.smithi029.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2024-09-18T10:14:41.848 INFO:teuthology.orchestra.run.smithi029.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2024-09-18T10:14:41.848 INFO:teuthology.orchestra.run.smithi029.stdout: "cephfs_mirror_mount_timeout": "10", 2024-09-18T10:14:41.848 INFO:teuthology.orchestra.run.smithi029.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2024-09-18T10:14:41.848 INFO:teuthology.orchestra.run.smithi029.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2024-09-18T10:14:41.848 INFO:teuthology.orchestra.run.smithi029.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2024-09-18T10:14:41.848 INFO:teuthology.orchestra.run.smithi029.stdout: "cephsqlite_blocklist_dead_locker": "true", 2024-09-18T10:14:41.848 INFO:teuthology.orchestra.run.smithi029.stdout: "cephsqlite_lock_renewal_interval": "2000", 2024-09-18T10:14:41.848 INFO:teuthology.orchestra.run.smithi029.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2024-09-18T10:14:41.849 INFO:teuthology.orchestra.run.smithi029.stdout: "cephx_cluster_require_signatures": "false", 2024-09-18T10:14:41.849 INFO:teuthology.orchestra.run.smithi029.stdout: "cephx_cluster_require_version": "2", 2024-09-18T10:14:41.849 INFO:teuthology.orchestra.run.smithi029.stdout: "cephx_require_signatures": "false", 2024-09-18T10:14:41.849 INFO:teuthology.orchestra.run.smithi029.stdout: "cephx_require_version": "2", 2024-09-18T10:14:41.849 INFO:teuthology.orchestra.run.smithi029.stdout: "cephx_service_require_signatures": "false", 2024-09-18T10:14:41.849 INFO:teuthology.orchestra.run.smithi029.stdout: "cephx_service_require_version": "2", 2024-09-18T10:14:41.849 INFO:teuthology.orchestra.run.smithi029.stdout: "cephx_sign_messages": "true", 2024-09-18T10:14:41.849 INFO:teuthology.orchestra.run.smithi029.stdout: "chdir": "", 2024-09-18T10:14:41.849 INFO:teuthology.orchestra.run.smithi029.stdout: "client_acl_type": "", 2024-09-18T10:14:41.850 INFO:teuthology.orchestra.run.smithi029.stdout: "client_asio_thread_count": "2", 2024-09-18T10:14:41.850 INFO:teuthology.orchestra.run.smithi029.stdout: "client_cache_mid": "0.750000", 2024-09-18T10:14:41.850 INFO:teuthology.orchestra.run.smithi029.stdout: "client_cache_size": "16384", 2024-09-18T10:14:41.850 INFO:teuthology.orchestra.run.smithi029.stdout: "client_caps_release_delay": "5", 2024-09-18T10:14:41.850 INFO:teuthology.orchestra.run.smithi029.stdout: "client_check_pool_perm": "true", 2024-09-18T10:14:41.850 INFO:teuthology.orchestra.run.smithi029.stdout: "client_collect_and_send_global_metrics": "false", 2024-09-18T10:14:41.850 INFO:teuthology.orchestra.run.smithi029.stdout: "client_debug_force_sync_read": "false", 2024-09-18T10:14:41.850 INFO:teuthology.orchestra.run.smithi029.stdout: "client_debug_getattr_caps": "false", 2024-09-18T10:14:41.850 INFO:teuthology.orchestra.run.smithi029.stdout: "client_debug_inject_features": "", 2024-09-18T10:14:41.850 INFO:teuthology.orchestra.run.smithi029.stdout: "client_debug_inject_tick_delay": "0", 2024-09-18T10:14:41.850 INFO:teuthology.orchestra.run.smithi029.stdout: "client_die_on_failed_dentry_invalidate": "true", 2024-09-18T10:14:41.850 INFO:teuthology.orchestra.run.smithi029.stdout: "client_die_on_failed_remount": "false", 2024-09-18T10:14:41.850 INFO:teuthology.orchestra.run.smithi029.stdout: "client_dirsize_rbytes": "true", 2024-09-18T10:14:41.850 INFO:teuthology.orchestra.run.smithi029.stdout: "client_force_lazyio": "false", 2024-09-18T10:14:41.851 INFO:teuthology.orchestra.run.smithi029.stdout: "client_fs": "", 2024-09-18T10:14:41.851 INFO:teuthology.orchestra.run.smithi029.stdout: "client_inject_fixed_oldest_tid": "false", 2024-09-18T10:14:41.851 INFO:teuthology.orchestra.run.smithi029.stdout: "client_inject_release_failure": "false", 2024-09-18T10:14:41.851 INFO:teuthology.orchestra.run.smithi029.stdout: "client_max_inline_size": "4096", 2024-09-18T10:14:41.851 INFO:teuthology.orchestra.run.smithi029.stdout: "client_max_retries_on_remount_failure": "5", 2024-09-18T10:14:41.851 INFO:teuthology.orchestra.run.smithi029.stdout: "client_mds_namespace": "", 2024-09-18T10:14:41.851 INFO:teuthology.orchestra.run.smithi029.stdout: "client_metadata": "", 2024-09-18T10:14:41.851 INFO:teuthology.orchestra.run.smithi029.stdout: "client_mount_gid": "-1", 2024-09-18T10:14:41.851 INFO:teuthology.orchestra.run.smithi029.stdout: "client_mount_timeout": "300", 2024-09-18T10:14:41.851 INFO:teuthology.orchestra.run.smithi029.stdout: "client_mount_uid": "-1", 2024-09-18T10:14:41.851 INFO:teuthology.orchestra.run.smithi029.stdout: "client_mountpoint": "/", 2024-09-18T10:14:41.851 INFO:teuthology.orchestra.run.smithi029.stdout: "client_notify_timeout": "10", 2024-09-18T10:14:41.852 INFO:teuthology.orchestra.run.smithi029.stdout: "client_oc": "true", 2024-09-18T10:14:41.852 INFO:teuthology.orchestra.run.smithi029.stdout: "client_oc_max_dirty": "104857600", 2024-09-18T10:14:41.852 INFO:teuthology.orchestra.run.smithi029.stdout: "client_oc_max_dirty_age": "5.000000", 2024-09-18T10:14:41.852 INFO:teuthology.orchestra.run.smithi029.stdout: "client_oc_max_objects": "1000", 2024-09-18T10:14:41.852 INFO:teuthology.orchestra.run.smithi029.stdout: "client_oc_size": "209715200", 2024-09-18T10:14:41.852 INFO:teuthology.orchestra.run.smithi029.stdout: "client_oc_target_dirty": "8388608", 2024-09-18T10:14:41.852 INFO:teuthology.orchestra.run.smithi029.stdout: "client_permissions": "true", 2024-09-18T10:14:41.852 INFO:teuthology.orchestra.run.smithi029.stdout: "client_quota": "true", 2024-09-18T10:14:41.852 INFO:teuthology.orchestra.run.smithi029.stdout: "client_quota_df": "true", 2024-09-18T10:14:41.852 INFO:teuthology.orchestra.run.smithi029.stdout: "client_readahead_max_bytes": "0", 2024-09-18T10:14:41.852 INFO:teuthology.orchestra.run.smithi029.stdout: "client_readahead_max_periods": "4", 2024-09-18T10:14:41.852 INFO:teuthology.orchestra.run.smithi029.stdout: "client_readahead_min": "131072", 2024-09-18T10:14:41.853 INFO:teuthology.orchestra.run.smithi029.stdout: "client_reconnect_stale": "false", 2024-09-18T10:14:41.853 INFO:teuthology.orchestra.run.smithi029.stdout: "client_shutdown_timeout": "30", 2024-09-18T10:14:41.853 INFO:teuthology.orchestra.run.smithi029.stdout: "client_snapdir": ".snap", 2024-09-18T10:14:41.853 INFO:teuthology.orchestra.run.smithi029.stdout: "client_tick_interval": "1", 2024-09-18T10:14:41.853 INFO:teuthology.orchestra.run.smithi029.stdout: "client_trace": "", 2024-09-18T10:14:41.853 INFO:teuthology.orchestra.run.smithi029.stdout: "client_try_dentry_invalidate": "false", 2024-09-18T10:14:41.853 INFO:teuthology.orchestra.run.smithi029.stdout: "client_use_faked_inos": "false", 2024-09-18T10:14:41.853 INFO:teuthology.orchestra.run.smithi029.stdout: "client_use_random_mds": "false", 2024-09-18T10:14:41.853 INFO:teuthology.orchestra.run.smithi029.stdout: "clog_to_graylog": "false", 2024-09-18T10:14:41.853 INFO:teuthology.orchestra.run.smithi029.stdout: "clog_to_graylog_host": "127.0.0.1", 2024-09-18T10:14:41.853 INFO:teuthology.orchestra.run.smithi029.stdout: "clog_to_graylog_port": "12201", 2024-09-18T10:14:41.853 INFO:teuthology.orchestra.run.smithi029.stdout: "clog_to_monitors": "default=true", 2024-09-18T10:14:41.853 INFO:teuthology.orchestra.run.smithi029.stdout: "clog_to_syslog": "false", 2024-09-18T10:14:41.854 INFO:teuthology.orchestra.run.smithi029.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2024-09-18T10:14:41.854 INFO:teuthology.orchestra.run.smithi029.stdout: "clog_to_syslog_level": "info", 2024-09-18T10:14:41.854 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addr": "-", 2024-09-18T10:14:41.854 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_network": "", 2024-09-18T10:14:41.854 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_network_interface": "", 2024-09-18T10:14:41.854 INFO:teuthology.orchestra.run.smithi029.stdout: "compressor_zlib_isal": "false", 2024-09-18T10:14:41.854 INFO:teuthology.orchestra.run.smithi029.stdout: "compressor_zlib_level": "5", 2024-09-18T10:14:41.854 INFO:teuthology.orchestra.run.smithi029.stdout: "compressor_zlib_winsize": "-15", 2024-09-18T10:14:41.854 INFO:teuthology.orchestra.run.smithi029.stdout: "compressor_zstd_level": "1", 2024-09-18T10:14:41.854 INFO:teuthology.orchestra.run.smithi029.stdout: "container_image": "docker.io/ceph/daemon-base:latest-master-devel", 2024-09-18T10:14:41.855 INFO:teuthology.orchestra.run.smithi029.stdout: "crash_dir": "/var/lib/ceph/crash", 2024-09-18T10:14:41.855 INFO:teuthology.orchestra.run.smithi029.stdout: "crimson_alien_op_num_threads": "6", 2024-09-18T10:14:41.855 INFO:teuthology.orchestra.run.smithi029.stdout: "crimson_alien_thread_cpu_cores": "", 2024-09-18T10:14:41.855 INFO:teuthology.orchestra.run.smithi029.stdout: "crimson_osd_obc_lru_size": "10", 2024-09-18T10:14:41.855 INFO:teuthology.orchestra.run.smithi029.stdout: "crimson_osd_scheduler_concurrency": "0", 2024-09-18T10:14:41.855 INFO:teuthology.orchestra.run.smithi029.stdout: "crush_location": "", 2024-09-18T10:14:41.855 INFO:teuthology.orchestra.run.smithi029.stdout: "crush_location_hook": "", 2024-09-18T10:14:41.855 INFO:teuthology.orchestra.run.smithi029.stdout: "crush_location_hook_timeout": "10", 2024-09-18T10:14:41.855 INFO:teuthology.orchestra.run.smithi029.stdout: "daemonize": "false", 2024-09-18T10:14:41.855 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_alienstore": "0/5", 2024-09-18T10:14:41.855 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_allow_any_pool_priority": "false", 2024-09-18T10:14:41.855 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_asok": "1/5", 2024-09-18T10:14:41.855 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_asok_assert_abort": "false", 2024-09-18T10:14:41.855 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_asserts_on_shutdown": "true", 2024-09-18T10:14:41.856 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_auth": "20/20", 2024-09-18T10:14:41.856 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_bdev": "1/3", 2024-09-18T10:14:41.856 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_bluefs": "1/5", 2024-09-18T10:14:41.856 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_bluestore": "1/5", 2024-09-18T10:14:41.856 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_buffer": "0/1", 2024-09-18T10:14:41.856 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_ceph_exporter": "1/5", 2024-09-18T10:14:41.856 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_cephfs_mirror": "0/5", 2024-09-18T10:14:41.856 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_cephsqlite": "0/5", 2024-09-18T10:14:41.856 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_civetweb": "1/10", 2024-09-18T10:14:41.856 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_client": "0/5", 2024-09-18T10:14:41.856 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_compressor": "1/5", 2024-09-18T10:14:41.856 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_context": "0/1", 2024-09-18T10:14:41.856 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_crush": "1/1", 2024-09-18T10:14:41.857 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_crypto": "1/5", 2024-09-18T10:14:41.857 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_deliberately_leak_memory": "false", 2024-09-18T10:14:41.857 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_disable_randomized_ping": "false", 2024-09-18T10:14:41.857 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_dpdk": "1/5", 2024-09-18T10:14:41.857 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_eventtrace": "1/5", 2024-09-18T10:14:41.857 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_filer": "0/1", 2024-09-18T10:14:41.857 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_filestore": "1/3", 2024-09-18T10:14:41.857 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_finisher": "1/1", 2024-09-18T10:14:41.857 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_fuse": "1/5", 2024-09-18T10:14:41.857 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_heartbeat_testing_span": "0", 2024-09-18T10:14:41.857 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_heartbeatmap": "1/5", 2024-09-18T10:14:41.857 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_immutable_obj_cache": "0/5", 2024-09-18T10:14:41.857 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_javaclient": "1/5", 2024-09-18T10:14:41.858 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_journal": "1/3", 2024-09-18T10:14:41.858 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_journaler": "0/5", 2024-09-18T10:14:41.858 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_kstore": "1/5", 2024-09-18T10:14:41.858 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_leveldb": "4/5", 2024-09-18T10:14:41.858 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_lockdep": "0/1", 2024-09-18T10:14:41.858 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_mclock": "1/5", 2024-09-18T10:14:41.858 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_mds": "1/5", 2024-09-18T10:14:41.858 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_mds_balancer": "1/5", 2024-09-18T10:14:41.858 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_mds_locker": "1/5", 2024-09-18T10:14:41.858 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_mds_log": "1/5", 2024-09-18T10:14:41.858 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_mds_log_expire": "1/5", 2024-09-18T10:14:41.859 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_mds_migrator": "1/5", 2024-09-18T10:14:41.859 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_memdb": "4/5", 2024-09-18T10:14:41.859 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_mgr": "2/5", 2024-09-18T10:14:41.859 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_mgrc": "1/5", 2024-09-18T10:14:41.859 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_mon": "20/20", 2024-09-18T10:14:41.859 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_monc": "0/10", 2024-09-18T10:14:41.859 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_ms": "1/1", 2024-09-18T10:14:41.859 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_none": "0/5", 2024-09-18T10:14:41.859 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_objclass": "0/5", 2024-09-18T10:14:41.859 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_objectcacher": "0/5", 2024-09-18T10:14:41.859 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_objecter": "0/1", 2024-09-18T10:14:41.859 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_optracker": "0/5", 2024-09-18T10:14:41.859 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_osd": "1/5", 2024-09-18T10:14:41.860 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_paxos": "20/20", 2024-09-18T10:14:41.860 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_perfcounter": "1/5", 2024-09-18T10:14:41.860 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_prioritycache": "1/5", 2024-09-18T10:14:41.860 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_rados": "0/5", 2024-09-18T10:14:41.860 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_rbd": "0/5", 2024-09-18T10:14:41.860 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_rbd_mirror": "0/5", 2024-09-18T10:14:41.860 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_rbd_pwl": "0/5", 2024-09-18T10:14:41.860 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_rbd_replay": "0/5", 2024-09-18T10:14:41.860 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_refs": "0/0", 2024-09-18T10:14:41.861 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_reserver": "1/1", 2024-09-18T10:14:41.861 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_rgw": "1/5", 2024-09-18T10:14:41.861 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_rgw_datacache": "1/5", 2024-09-18T10:14:41.861 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_rgw_sync": "1/5", 2024-09-18T10:14:41.861 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_rocksdb": "4/5", 2024-09-18T10:14:41.861 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_seastore": "0/5", 2024-09-18T10:14:41.861 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_seastore_cache": "0/5", 2024-09-18T10:14:41.861 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_seastore_cleaner": "0/5", 2024-09-18T10:14:41.861 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_seastore_device": "0/5", 2024-09-18T10:14:41.861 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_seastore_journal": "0/5", 2024-09-18T10:14:41.861 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_seastore_lba": "0/5", 2024-09-18T10:14:41.862 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_seastore_odata": "0/5", 2024-09-18T10:14:41.862 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_seastore_omap": "0/5", 2024-09-18T10:14:41.862 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_seastore_onode": "0/5", 2024-09-18T10:14:41.862 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_seastore_tm": "0/5", 2024-09-18T10:14:41.862 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_striper": "0/1", 2024-09-18T10:14:41.862 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_test": "0/5", 2024-09-18T10:14:41.862 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_throttle": "1/1", 2024-09-18T10:14:41.862 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_timer": "0/1", 2024-09-18T10:14:41.862 INFO:teuthology.orchestra.run.smithi029.stdout: "debug_tp": "0/5", 2024-09-18T10:14:41.862 INFO:teuthology.orchestra.run.smithi029.stdout: "defer_client_eviction_on_laggy_osds": "false", 2024-09-18T10:14:41.863 INFO:teuthology.orchestra.run.smithi029.stdout: "device_failure_prediction_mode": "none", 2024-09-18T10:14:41.863 INFO:teuthology.orchestra.run.smithi029.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "", 2024-09-18T10:14:41.863 INFO:teuthology.orchestra.run.smithi029.stdout: "erasure_code_dir": "/usr/lib/ceph/erasure-code", 2024-09-18T10:14:41.863 INFO:teuthology.orchestra.run.smithi029.stdout: "err_to_graylog": "false", 2024-09-18T10:14:41.863 INFO:teuthology.orchestra.run.smithi029.stdout: "err_to_journald": "false", 2024-09-18T10:14:41.863 INFO:teuthology.orchestra.run.smithi029.stdout: "err_to_stderr": "true", 2024-09-18T10:14:41.863 INFO:teuthology.orchestra.run.smithi029.stdout: "err_to_syslog": "false", 2024-09-18T10:14:41.863 INFO:teuthology.orchestra.run.smithi029.stdout: "event_tracing": "false", 2024-09-18T10:14:41.864 INFO:teuthology.orchestra.run.smithi029.stdout: "exporter_addr": "0.0.0.0", 2024-09-18T10:14:41.864 INFO:teuthology.orchestra.run.smithi029.stdout: "exporter_http_port": "9926", 2024-09-18T10:14:41.864 INFO:teuthology.orchestra.run.smithi029.stdout: "exporter_prio_limit": "5", 2024-09-18T10:14:41.864 INFO:teuthology.orchestra.run.smithi029.stdout: "exporter_sock_dir": "/var/run/ceph/", 2024-09-18T10:14:41.864 INFO:teuthology.orchestra.run.smithi029.stdout: "exporter_sort_metrics": "true", 2024-09-18T10:14:41.864 INFO:teuthology.orchestra.run.smithi029.stdout: "exporter_stats_period": "5", 2024-09-18T10:14:41.864 INFO:teuthology.orchestra.run.smithi029.stdout: "fake_statfs_for_testing": "0", 2024-09-18T10:14:41.865 INFO:teuthology.orchestra.run.smithi029.stdout: "fatal_signal_handlers": "true", 2024-09-18T10:14:41.865 INFO:teuthology.orchestra.run.smithi029.stdout: "filer_max_purge_ops": "10", 2024-09-18T10:14:41.865 INFO:teuthology.orchestra.run.smithi029.stdout: "filer_max_truncate_ops": "128", 2024-09-18T10:14:41.865 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_apply_finisher_threads": "1", 2024-09-18T10:14:41.865 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_blackhole": "false", 2024-09-18T10:14:41.866 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_btrfs_clone_range": "true", 2024-09-18T10:14:41.866 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_btrfs_snap": "true", 2024-09-18T10:14:41.866 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_caller_concurrency": "10", 2024-09-18T10:14:41.866 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_collect_device_partition_information": "true", 2024-09-18T10:14:41.866 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_commit_timeout": "600.000000", 2024-09-18T10:14:41.866 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_debug_inject_read_err": "false", 2024-09-18T10:14:41.866 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_debug_omap_check": "false", 2024-09-18T10:14:41.866 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_debug_random_read_err": "0.000000", 2024-09-18T10:14:41.866 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_debug_verify_split": "false", 2024-09-18T10:14:41.867 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_dump_file": "", 2024-09-18T10:14:41.867 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2024-09-18T10:14:41.867 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_expected_throughput_ops": "200.000000", 2024-09-18T10:14:41.867 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_fadvise": "true", 2024-09-18T10:14:41.867 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_fail_eio": "true", 2024-09-18T10:14:41.867 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_fd_cache_shards": "16", 2024-09-18T10:14:41.867 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_fd_cache_size": "128", 2024-09-18T10:14:41.867 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_fiemap": "false", 2024-09-18T10:14:41.868 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_fiemap_threshold": "4096", 2024-09-18T10:14:41.868 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_fsync_flushes_journal_data": "false", 2024-09-18T10:14:41.868 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_index_retry_probability": "0.000000", 2024-09-18T10:14:41.868 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_inject_stall": "0", 2024-09-18T10:14:41.868 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_journal_parallel": "false", 2024-09-18T10:14:41.868 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_journal_trailing": "false", 2024-09-18T10:14:41.868 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_journal_writeahead": "false", 2024-09-18T10:14:41.868 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_kill_at": "0", 2024-09-18T10:14:41.869 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_max_alloc_hint_size": "1048576", 2024-09-18T10:14:41.869 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_max_inline_xattr_size": "0", 2024-09-18T10:14:41.869 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2024-09-18T10:14:41.869 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_max_inline_xattr_size_other": "512", 2024-09-18T10:14:41.869 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2024-09-18T10:14:41.869 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_max_inline_xattrs": "0", 2024-09-18T10:14:41.869 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2024-09-18T10:14:41.870 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_max_inline_xattrs_other": "2", 2024-09-18T10:14:41.870 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_max_inline_xattrs_xfs": "10", 2024-09-18T10:14:41.870 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_max_sync_interval": "5.000000", 2024-09-18T10:14:41.870 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_max_xattr_value_size": "0", 2024-09-18T10:14:41.870 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2024-09-18T10:14:41.870 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_max_xattr_value_size_other": "1024", 2024-09-18T10:14:41.870 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2024-09-18T10:14:41.870 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_merge_threshold": "-10", 2024-09-18T10:14:41.871 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_min_sync_interval": "0.010000", 2024-09-18T10:14:41.871 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_odsync_write": "false", 2024-09-18T10:14:41.871 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_omap_backend": "rocksdb", 2024-09-18T10:14:41.871 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_omap_backend_path": "", 2024-09-18T10:14:41.871 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_omap_header_cache_size": "1024", 2024-09-18T10:14:41.871 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_ondisk_finisher_threads": "1", 2024-09-18T10:14:41.871 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_op_thread_suicide_timeout": "180", 2024-09-18T10:14:41.871 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_op_thread_timeout": "60", 2024-09-18T10:14:41.871 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_op_threads": "2", 2024-09-18T10:14:41.871 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_punch_hole": "false", 2024-09-18T10:14:41.872 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2024-09-18T10:14:41.872 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2024-09-18T10:14:41.872 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2024-09-18T10:14:41.872 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_queue_high_threshhold": "0.900000", 2024-09-18T10:14:41.872 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_queue_low_threshhold": "0.300000", 2024-09-18T10:14:41.872 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_queue_max_bytes": "104857600", 2024-09-18T10:14:41.872 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2024-09-18T10:14:41.872 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2024-09-18T10:14:41.872 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2024-09-18T10:14:41.872 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_queue_max_ops": "50", 2024-09-18T10:14:41.872 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2024-09-18T10:14:41.873 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_seek_data_hole": "false", 2024-09-18T10:14:41.873 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_sloppy_crc": "false", 2024-09-18T10:14:41.873 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_sloppy_crc_block_size": "65536", 2024-09-18T10:14:41.873 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_splice": "false", 2024-09-18T10:14:41.873 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_split_multiple": "2", 2024-09-18T10:14:41.873 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_split_rand_factor": "20", 2024-09-18T10:14:41.873 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_update_to": "1000", 2024-09-18T10:14:41.873 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2024-09-18T10:14:41.873 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2024-09-18T10:14:41.873 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2024-09-18T10:14:41.874 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2024-09-18T10:14:41.874 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2024-09-18T10:14:41.874 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2024-09-18T10:14:41.874 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_wbthrottle_enable": "true", 2024-09-18T10:14:41.874 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2024-09-18T10:14:41.874 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2024-09-18T10:14:41.874 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2024-09-18T10:14:41.874 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2024-09-18T10:14:41.874 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2024-09-18T10:14:41.874 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2024-09-18T10:14:41.874 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_xfs_extsize": "false", 2024-09-18T10:14:41.875 INFO:teuthology.orchestra.run.smithi029.stdout: "filestore_zfs_snap": "false", 2024-09-18T10:14:41.875 INFO:teuthology.orchestra.run.smithi029.stdout: "fio_dir": "/tmp/fio", 2024-09-18T10:14:41.875 INFO:teuthology.orchestra.run.smithi029.stdout: "fsid": "ab6d6beb-0627-4b7f-8cdb-4e0a44970f10", 2024-09-18T10:14:41.875 INFO:teuthology.orchestra.run.smithi029.stdout: "fuse_allow_other": "true", 2024-09-18T10:14:41.875 INFO:teuthology.orchestra.run.smithi029.stdout: "fuse_atomic_o_trunc": "true", 2024-09-18T10:14:41.875 INFO:teuthology.orchestra.run.smithi029.stdout: "fuse_big_writes": "true", 2024-09-18T10:14:41.875 INFO:teuthology.orchestra.run.smithi029.stdout: "fuse_debug": "false", 2024-09-18T10:14:41.875 INFO:teuthology.orchestra.run.smithi029.stdout: "fuse_default_permissions": "false", 2024-09-18T10:14:41.875 INFO:teuthology.orchestra.run.smithi029.stdout: "fuse_disable_pagecache": "false", 2024-09-18T10:14:41.876 INFO:teuthology.orchestra.run.smithi029.stdout: "fuse_max_write": "0", 2024-09-18T10:14:41.876 INFO:teuthology.orchestra.run.smithi029.stdout: "fuse_multithreaded": "true", 2024-09-18T10:14:41.876 INFO:teuthology.orchestra.run.smithi029.stdout: "fuse_require_active_mds": "true", 2024-09-18T10:14:41.876 INFO:teuthology.orchestra.run.smithi029.stdout: "fuse_set_user_groups": "true", 2024-09-18T10:14:41.876 INFO:teuthology.orchestra.run.smithi029.stdout: "fuse_splice_move": "true", 2024-09-18T10:14:41.876 INFO:teuthology.orchestra.run.smithi029.stdout: "fuse_splice_read": "true", 2024-09-18T10:14:41.876 INFO:teuthology.orchestra.run.smithi029.stdout: "fuse_splice_write": "true", 2024-09-18T10:14:41.876 INFO:teuthology.orchestra.run.smithi029.stdout: "fuse_syncfs_on_mksnap": "true", 2024-09-18T10:14:41.876 INFO:teuthology.orchestra.run.smithi029.stdout: "fuse_use_invalidate_cb": "true", 2024-09-18T10:14:41.876 INFO:teuthology.orchestra.run.smithi029.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2024-09-18T10:14:41.876 INFO:teuthology.orchestra.run.smithi029.stdout: "gss_target_name": "ceph", 2024-09-18T10:14:41.877 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_file": "", 2024-09-18T10:14:41.877 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_inject_failure": "0", 2024-09-18T10:14:41.877 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_interval": "5", 2024-09-18T10:14:41.877 INFO:teuthology.orchestra.run.smithi029.stdout: "host": "smithi029", 2024-09-18T10:14:41.877 INFO:teuthology.orchestra.run.smithi029.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2024-09-18T10:14:41.877 INFO:teuthology.orchestra.run.smithi029.stdout: "immutable_object_cache_max_inflight_ops": "128", 2024-09-18T10:14:41.877 INFO:teuthology.orchestra.run.smithi029.stdout: "immutable_object_cache_max_size": "1073741824", 2024-09-18T10:14:41.877 INFO:teuthology.orchestra.run.smithi029.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2024-09-18T10:14:41.877 INFO:teuthology.orchestra.run.smithi029.stdout: "immutable_object_cache_qos_bps_burst": "0", 2024-09-18T10:14:41.877 INFO:teuthology.orchestra.run.smithi029.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2024-09-18T10:14:41.877 INFO:teuthology.orchestra.run.smithi029.stdout: "immutable_object_cache_qos_bps_limit": "0", 2024-09-18T10:14:41.877 INFO:teuthology.orchestra.run.smithi029.stdout: "immutable_object_cache_qos_iops_burst": "0", 2024-09-18T10:14:41.878 INFO:teuthology.orchestra.run.smithi029.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2024-09-18T10:14:41.878 INFO:teuthology.orchestra.run.smithi029.stdout: "immutable_object_cache_qos_iops_limit": "0", 2024-09-18T10:14:41.878 INFO:teuthology.orchestra.run.smithi029.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2024-09-18T10:14:41.878 INFO:teuthology.orchestra.run.smithi029.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2024-09-18T10:14:41.878 INFO:teuthology.orchestra.run.smithi029.stdout: "immutable_object_cache_watermark": "0.900000", 2024-09-18T10:14:41.878 INFO:teuthology.orchestra.run.smithi029.stdout: "inject_early_sigterm": "false", 2024-09-18T10:14:41.878 INFO:teuthology.orchestra.run.smithi029.stdout: "jaeger_tracing_enable": "false", 2024-09-18T10:14:41.878 INFO:teuthology.orchestra.run.smithi029.stdout: "journal_aio": "true", 2024-09-18T10:14:41.878 INFO:teuthology.orchestra.run.smithi029.stdout: "journal_align_min_size": "65536", 2024-09-18T10:14:41.879 INFO:teuthology.orchestra.run.smithi029.stdout: "journal_block_align": "true", 2024-09-18T10:14:41.879 INFO:teuthology.orchestra.run.smithi029.stdout: "journal_block_size": "4096", 2024-09-18T10:14:41.879 INFO:teuthology.orchestra.run.smithi029.stdout: "journal_dio": "true", 2024-09-18T10:14:41.879 INFO:teuthology.orchestra.run.smithi029.stdout: "journal_discard": "false", 2024-09-18T10:14:41.879 INFO:teuthology.orchestra.run.smithi029.stdout: "journal_force_aio": "false", 2024-09-18T10:14:41.879 INFO:teuthology.orchestra.run.smithi029.stdout: "journal_ignore_corruption": "false", 2024-09-18T10:14:41.880 INFO:teuthology.orchestra.run.smithi029.stdout: "journal_max_write_bytes": "10485760", 2024-09-18T10:14:41.880 INFO:teuthology.orchestra.run.smithi029.stdout: "journal_max_write_entries": "100", 2024-09-18T10:14:41.880 INFO:teuthology.orchestra.run.smithi029.stdout: "journal_replay_from": "0", 2024-09-18T10:14:41.880 INFO:teuthology.orchestra.run.smithi029.stdout: "journal_throttle_high_multiple": "0.000000", 2024-09-18T10:14:41.880 INFO:teuthology.orchestra.run.smithi029.stdout: "journal_throttle_high_threshhold": "0.900000", 2024-09-18T10:14:41.881 INFO:teuthology.orchestra.run.smithi029.stdout: "journal_throttle_low_threshhold": "0.600000", 2024-09-18T10:14:41.881 INFO:teuthology.orchestra.run.smithi029.stdout: "journal_throttle_max_multiple": "0.000000", 2024-09-18T10:14:41.881 INFO:teuthology.orchestra.run.smithi029.stdout: "journal_write_header_frequency": "0", 2024-09-18T10:14:41.881 INFO:teuthology.orchestra.run.smithi029.stdout: "journal_zero_on_create": "false", 2024-09-18T10:14:41.881 INFO:teuthology.orchestra.run.smithi029.stdout: "journaler_prefetch_periods": "10", 2024-09-18T10:14:41.881 INFO:teuthology.orchestra.run.smithi029.stdout: "journaler_prezero_periods": "5", 2024-09-18T10:14:41.881 INFO:teuthology.orchestra.run.smithi029.stdout: "journaler_write_head_interval": "15", 2024-09-18T10:14:41.882 INFO:teuthology.orchestra.run.smithi029.stdout: "key": "", 2024-09-18T10:14:41.882 INFO:teuthology.orchestra.run.smithi029.stdout: "keyfile": "", 2024-09-18T10:14:41.882 INFO:teuthology.orchestra.run.smithi029.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2024-09-18T10:14:41.882 INFO:teuthology.orchestra.run.smithi029.stdout: "kstore_backend": "rocksdb", 2024-09-18T10:14:41.882 INFO:teuthology.orchestra.run.smithi029.stdout: "kstore_default_stripe_size": "65536", 2024-09-18T10:14:41.882 INFO:teuthology.orchestra.run.smithi029.stdout: "kstore_fsck_on_mount": "false", 2024-09-18T10:14:41.883 INFO:teuthology.orchestra.run.smithi029.stdout: "kstore_fsck_on_mount_deep": "true", 2024-09-18T10:14:41.883 INFO:teuthology.orchestra.run.smithi029.stdout: "kstore_max_bytes": "67108864", 2024-09-18T10:14:41.883 INFO:teuthology.orchestra.run.smithi029.stdout: "kstore_max_ops": "512", 2024-09-18T10:14:41.883 INFO:teuthology.orchestra.run.smithi029.stdout: "kstore_nid_prealloc": "1024", 2024-09-18T10:14:41.883 INFO:teuthology.orchestra.run.smithi029.stdout: "kstore_onode_map_size": "1024", 2024-09-18T10:14:41.883 INFO:teuthology.orchestra.run.smithi029.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2024-09-18T10:14:41.883 INFO:teuthology.orchestra.run.smithi029.stdout: "kstore_sync_submit_transaction": "false", 2024-09-18T10:14:41.884 INFO:teuthology.orchestra.run.smithi029.stdout: "kstore_sync_transaction": "false", 2024-09-18T10:14:41.884 INFO:teuthology.orchestra.run.smithi029.stdout: "leveldb_block_size": "65536", 2024-09-18T10:14:41.884 INFO:teuthology.orchestra.run.smithi029.stdout: "leveldb_bloom_size": "0", 2024-09-18T10:14:41.884 INFO:teuthology.orchestra.run.smithi029.stdout: "leveldb_cache_size": "536870912", 2024-09-18T10:14:41.884 INFO:teuthology.orchestra.run.smithi029.stdout: "leveldb_compact_on_mount": "false", 2024-09-18T10:14:41.884 INFO:teuthology.orchestra.run.smithi029.stdout: "leveldb_compression": "false", 2024-09-18T10:14:41.884 INFO:teuthology.orchestra.run.smithi029.stdout: "leveldb_log": "", 2024-09-18T10:14:41.885 INFO:teuthology.orchestra.run.smithi029.stdout: "leveldb_log_to_ceph_log": "true", 2024-09-18T10:14:41.885 INFO:teuthology.orchestra.run.smithi029.stdout: "leveldb_max_open_files": "0", 2024-09-18T10:14:41.885 INFO:teuthology.orchestra.run.smithi029.stdout: "leveldb_paranoid": "false", 2024-09-18T10:14:41.885 INFO:teuthology.orchestra.run.smithi029.stdout: "leveldb_write_buffer_size": "33554432", 2024-09-18T10:14:41.885 INFO:teuthology.orchestra.run.smithi029.stdout: "librados_thread_count": "2", 2024-09-18T10:14:41.885 INFO:teuthology.orchestra.run.smithi029.stdout: "lockdep": "false", 2024-09-18T10:14:41.886 INFO:teuthology.orchestra.run.smithi029.stdout: "lockdep_force_backtrace": "false", 2024-09-18T10:14:41.886 INFO:teuthology.orchestra.run.smithi029.stdout: "log_coarse_timestamps": "true", 2024-09-18T10:14:41.886 INFO:teuthology.orchestra.run.smithi029.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2024-09-18T10:14:41.886 INFO:teuthology.orchestra.run.smithi029.stdout: "log_flush_on_exit": "false", 2024-09-18T10:14:41.886 INFO:teuthology.orchestra.run.smithi029.stdout: "log_graylog_host": "127.0.0.1", 2024-09-18T10:14:41.886 INFO:teuthology.orchestra.run.smithi029.stdout: "log_graylog_port": "12201", 2024-09-18T10:14:41.886 INFO:teuthology.orchestra.run.smithi029.stdout: "log_max_new": "1000", 2024-09-18T10:14:41.887 INFO:teuthology.orchestra.run.smithi029.stdout: "log_max_recent": "10000", 2024-09-18T10:14:41.887 INFO:teuthology.orchestra.run.smithi029.stdout: "log_stderr_prefix": "", 2024-09-18T10:14:41.887 INFO:teuthology.orchestra.run.smithi029.stdout: "log_stop_at_utilization": "0.970000", 2024-09-18T10:14:41.887 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_file": "true", 2024-09-18T10:14:41.887 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_graylog": "false", 2024-09-18T10:14:41.887 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_journald": "false", 2024-09-18T10:14:41.888 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_stderr": "false", 2024-09-18T10:14:41.888 INFO:teuthology.orchestra.run.smithi029.stdout: "log_to_syslog": "false", 2024-09-18T10:14:41.888 INFO:teuthology.orchestra.run.smithi029.stdout: "max_rotating_auth_attempts": "10", 2024-09-18T10:14:41.888 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_abort_on_newly_corrupt_dentry": "true", 2024-09-18T10:14:41.888 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_action_on_write_error": "1", 2024-09-18T10:14:41.888 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_alternate_name_max": "8192", 2024-09-18T10:14:41.889 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_asio_thread_count": "2", 2024-09-18T10:14:41.889 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_bal_export_pin": "true", 2024-09-18T10:14:41.889 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_bal_fragment_dirs": "true", 2024-09-18T10:14:41.892 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2024-09-18T10:14:41.893 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_bal_fragment_interval": "5", 2024-09-18T10:14:41.893 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_bal_fragment_size_max": "100000", 2024-09-18T10:14:41.893 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_bal_idle_threshold": "0.000000", 2024-09-18T10:14:41.893 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_bal_interval": "10", 2024-09-18T10:14:41.893 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_bal_max": "-1", 2024-09-18T10:14:41.893 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_bal_max_until": "-1", 2024-09-18T10:14:41.893 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_bal_merge_size": "50", 2024-09-18T10:14:41.893 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_bal_midchunk": "0.300000", 2024-09-18T10:14:41.894 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_bal_min_rebalance": "0.100000", 2024-09-18T10:14:41.894 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_bal_min_start": "0.200000", 2024-09-18T10:14:41.894 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_bal_minchunk": "0.001000", 2024-09-18T10:14:41.894 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_bal_mode": "0", 2024-09-18T10:14:41.894 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_bal_need_max": "1.200000", 2024-09-18T10:14:41.894 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_bal_need_min": "0.800000", 2024-09-18T10:14:41.894 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_bal_replicate_threshold": "8000.000000", 2024-09-18T10:14:41.894 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_bal_sample_interval": "3.000000", 2024-09-18T10:14:41.894 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_bal_split_bits": "3", 2024-09-18T10:14:41.894 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_bal_split_rd": "25000.000000", 2024-09-18T10:14:41.894 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_bal_split_size": "10000", 2024-09-18T10:14:41.894 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_bal_split_wr": "10000.000000", 2024-09-18T10:14:41.895 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_bal_target_decay": "10.000000", 2024-09-18T10:14:41.895 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2024-09-18T10:14:41.895 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_beacon_grace": "15.000000", 2024-09-18T10:14:41.895 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_beacon_interval": "4.000000", 2024-09-18T10:14:41.895 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_beacon_mon_down_grace": "60", 2024-09-18T10:14:41.895 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_cache_memory_limit": "4294967296", 2024-09-18T10:14:41.895 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_cache_mid": "0.700000", 2024-09-18T10:14:41.895 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_cache_release_free_interval": "10", 2024-09-18T10:14:41.895 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_cache_reservation": "0.050000", 2024-09-18T10:14:41.895 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_cache_trim_decay_rate": "1.000000", 2024-09-18T10:14:41.895 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_cache_trim_interval": "1", 2024-09-18T10:14:41.896 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_cache_trim_threshold": "262144", 2024-09-18T10:14:41.896 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2024-09-18T10:14:41.896 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2024-09-18T10:14:41.896 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_client_delegate_inos_pct": "50", 2024-09-18T10:14:41.896 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_client_prealloc_inos": "1000", 2024-09-18T10:14:41.896 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2024-09-18T10:14:41.897 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_connect_bootstrapping": "false", 2024-09-18T10:14:41.897 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_damage_table_max_entries": "10000", 2024-09-18T10:14:41.897 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2024-09-18T10:14:41.897 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_debug_auth_pins": "false", 2024-09-18T10:14:41.897 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_debug_frag": "false", 2024-09-18T10:14:41.897 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_debug_scatterstat": "false", 2024-09-18T10:14:41.898 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_debug_subtrees": "false", 2024-09-18T10:14:41.898 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_decay_halflife": "5.000000", 2024-09-18T10:14:41.898 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_default_dir_hash": "2", 2024-09-18T10:14:41.898 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_defer_session_stale": "true", 2024-09-18T10:14:41.898 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_deny_all_reconnect": "false", 2024-09-18T10:14:41.898 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_dir_keys_per_op": "16384", 2024-09-18T10:14:41.898 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_dir_max_commit_size": "10", 2024-09-18T10:14:41.898 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_dir_max_entries": "0", 2024-09-18T10:14:41.898 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_dirstat_min_interval": "1.000000", 2024-09-18T10:14:41.898 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_dump_cache_after_rejoin": "false", 2024-09-18T10:14:41.898 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_dump_cache_on_map": "false", 2024-09-18T10:14:41.898 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_dump_cache_threshold_file": "0", 2024-09-18T10:14:41.899 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2024-09-18T10:14:41.899 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_early_reply": "true", 2024-09-18T10:14:41.899 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_enable_op_tracker": "true", 2024-09-18T10:14:41.899 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_enforce_unique_name": "true", 2024-09-18T10:14:41.899 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_export_ephemeral_distributed": "true", 2024-09-18T10:14:41.899 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2024-09-18T10:14:41.899 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_export_ephemeral_random": "true", 2024-09-18T10:14:41.900 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_export_ephemeral_random_max": "0.010000", 2024-09-18T10:14:41.900 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_forward_all_requests_to_auth": "false", 2024-09-18T10:14:41.900 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_freeze_tree_timeout": "30.000000", 2024-09-18T10:14:41.900 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_go_bad_corrupt_dentry": "true", 2024-09-18T10:14:41.900 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2024-09-18T10:14:41.901 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_health_cache_threshold": "1.500000", 2024-09-18T10:14:41.901 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_health_summarize_threshold": "10", 2024-09-18T10:14:41.901 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_heartbeat_grace": "15.000000", 2024-09-18T10:14:41.901 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_heartbeat_reset_grace": "1000", 2024-09-18T10:14:41.901 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_inject_health_dummy": "false", 2024-09-18T10:14:41.901 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_inject_journal_corrupt_dentry_first": "0.000000", 2024-09-18T10:14:41.902 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_inject_migrator_session_race": "false", 2024-09-18T10:14:41.902 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_inject_rename_corrupt_dentry_first": "0.000000", 2024-09-18T10:14:41.902 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_inject_skip_replaying_inotable": "false", 2024-09-18T10:14:41.902 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2024-09-18T10:14:41.902 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_join_fs": "", 2024-09-18T10:14:41.902 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_journal_format": "1", 2024-09-18T10:14:41.902 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_kill_create_at": "0", 2024-09-18T10:14:41.902 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_kill_export_at": "0", 2024-09-18T10:14:41.902 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_kill_import_at": "0", 2024-09-18T10:14:41.902 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_kill_journal_at": "0", 2024-09-18T10:14:41.903 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_kill_journal_expire_at": "0", 2024-09-18T10:14:41.903 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_kill_journal_replay_at": "0", 2024-09-18T10:14:41.903 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_kill_link_at": "0", 2024-09-18T10:14:41.903 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_kill_mdstable_at": "0", 2024-09-18T10:14:41.903 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_kill_openc_at": "0", 2024-09-18T10:14:41.903 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_kill_rename_at": "0", 2024-09-18T10:14:41.903 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_kill_skip_replaying_inotable": "false", 2024-09-18T10:14:41.904 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_log_events_per_segment": "1024", 2024-09-18T10:14:41.904 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_log_max_events": "-1", 2024-09-18T10:14:41.904 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_log_max_segments": "128", 2024-09-18T10:14:41.904 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_log_pause": "false", 2024-09-18T10:14:41.904 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_log_segment_size": "0", 2024-09-18T10:14:41.904 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_log_skip_corrupt_events": "false", 2024-09-18T10:14:41.904 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_log_warn_factor": "2.000000", 2024-09-18T10:14:41.904 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_max_caps_per_client": "1048576", 2024-09-18T10:14:41.904 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_max_completed_flushes": "100000", 2024-09-18T10:14:41.905 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_max_completed_requests": "100000", 2024-09-18T10:14:41.905 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_max_export_size": "20971520", 2024-09-18T10:14:41.905 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_max_file_recover": "32", 2024-09-18T10:14:41.905 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_max_purge_files": "64", 2024-09-18T10:14:41.905 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_max_purge_ops": "8192", 2024-09-18T10:14:41.905 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2024-09-18T10:14:41.905 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_max_scrub_ops_in_progress": "5", 2024-09-18T10:14:41.905 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_max_snaps_per_dir": "100", 2024-09-18T10:14:41.905 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_max_xattr_pairs_size": "65536", 2024-09-18T10:14:41.905 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_metrics_update_interval": "2", 2024-09-18T10:14:41.905 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_min_caps_per_client": "100", 2024-09-18T10:14:41.905 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_min_caps_working_set": "10000", 2024-09-18T10:14:41.905 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_mon_shutdown_timeout": "5.000000", 2024-09-18T10:14:41.906 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_numa_node": "-1", 2024-09-18T10:14:41.906 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_oft_prefetch_dirfrags": "false", 2024-09-18T10:14:41.906 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_op_complaint_time": "30.000000", 2024-09-18T10:14:41.906 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_op_history_duration": "600", 2024-09-18T10:14:41.906 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_op_history_size": "20", 2024-09-18T10:14:41.906 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_op_history_slow_op_size": "20", 2024-09-18T10:14:41.906 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_op_history_slow_op_threshold": "10", 2024-09-18T10:14:41.906 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_op_log_threshold": "5", 2024-09-18T10:14:41.907 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_ping_grace": "15", 2024-09-18T10:14:41.907 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_ping_interval": "5", 2024-09-18T10:14:41.908 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2024-09-18T10:14:41.908 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_recall_global_max_decay_threshold": "131072", 2024-09-18T10:14:41.908 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_recall_max_caps": "30000", 2024-09-18T10:14:41.908 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_recall_max_decay_rate": "1.500000", 2024-09-18T10:14:41.908 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_recall_max_decay_threshold": "131072", 2024-09-18T10:14:41.909 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_recall_warning_decay_rate": "60.000000", 2024-09-18T10:14:41.909 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_recall_warning_threshold": "262144", 2024-09-18T10:14:41.909 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_reconnect_timeout": "45.000000", 2024-09-18T10:14:41.909 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_replay_interval": "1.000000", 2024-09-18T10:14:41.909 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_replay_unsafe_with_closed_session": "false", 2024-09-18T10:14:41.910 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_request_load_average_decay_rate": "60.000000", 2024-09-18T10:14:41.910 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_root_ino_gid": "0", 2024-09-18T10:14:41.910 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_root_ino_uid": "0", 2024-09-18T10:14:41.910 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_scatter_nudge_interval": "5.000000", 2024-09-18T10:14:41.910 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_session_blocklist_on_evict": "true", 2024-09-18T10:14:41.910 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_session_blocklist_on_timeout": "true", 2024-09-18T10:14:41.911 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2024-09-18T10:14:41.911 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_session_cache_liveness_magnitude": "10", 2024-09-18T10:14:41.911 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_session_cap_acquisition_decay_rate": "30.000000", 2024-09-18T10:14:41.911 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_session_cap_acquisition_throttle": "100000", 2024-09-18T10:14:41.911 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2024-09-18T10:14:41.911 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_session_metadata_threshold": "16777216", 2024-09-18T10:14:41.911 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_sessionmap_keys_per_op": "1024", 2024-09-18T10:14:41.912 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_shutdown_check": "0", 2024-09-18T10:14:41.912 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_skip_ino": "0", 2024-09-18T10:14:41.912 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_sleep_rank_change": "0.000000", 2024-09-18T10:14:41.912 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_snap_max_uid": "4294967294", 2024-09-18T10:14:41.912 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_snap_min_uid": "0", 2024-09-18T10:14:41.912 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_snap_rstat": "false", 2024-09-18T10:14:41.912 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_standby_replay_damaged": "false", 2024-09-18T10:14:41.913 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_symlink_recovery": "true", 2024-09-18T10:14:41.913 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_task_status_update_interval": "2.000000", 2024-09-18T10:14:41.913 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_thrash_exports": "0", 2024-09-18T10:14:41.913 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_thrash_fragments": "0", 2024-09-18T10:14:41.913 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_tick_interval": "5.000000", 2024-09-18T10:14:41.913 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_valgrind_exit": "false", 2024-09-18T10:14:41.914 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_verify_backtrace": "1", 2024-09-18T10:14:41.914 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_verify_scatter": "false", 2024-09-18T10:14:41.914 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_wipe_ino_prealloc": "false", 2024-09-18T10:14:41.914 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_wipe_sessions": "false", 2024-09-18T10:14:41.914 INFO:teuthology.orchestra.run.smithi029.stdout: "mempool_debug": "false", 2024-09-18T10:14:41.914 INFO:teuthology.orchestra.run.smithi029.stdout: "memstore_debug_omit_block_device_write": "false", 2024-09-18T10:14:41.915 INFO:teuthology.orchestra.run.smithi029.stdout: "memstore_device_bytes": "1073741824", 2024-09-18T10:14:41.915 INFO:teuthology.orchestra.run.smithi029.stdout: "memstore_page_set": "false", 2024-09-18T10:14:41.915 INFO:teuthology.orchestra.run.smithi029.stdout: "memstore_page_size": "65536", 2024-09-18T10:14:41.915 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr_client_bytes": "134217728", 2024-09-18T10:14:41.915 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr_client_messages": "512", 2024-09-18T10:14:41.915 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2024-09-18T10:14:41.916 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr_connect_retry_interval": "1.000000", 2024-09-18T10:14:41.916 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2024-09-18T10:14:41.916 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2024-09-18T10:14:41.916 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr_disabled_modules": "diskprediction_local", 2024-09-18T10:14:41.916 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr_initial_modules": "restful iostat nfs", 2024-09-18T10:14:41.916 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr_max_pg_num_change": "128", 2024-09-18T10:14:41.916 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr_mds_bytes": "134217728", 2024-09-18T10:14:41.916 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr_mds_messages": "128", 2024-09-18T10:14:41.916 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2024-09-18T10:14:41.916 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr_mon_bytes": "134217728", 2024-09-18T10:14:41.917 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr_mon_messages": "128", 2024-09-18T10:14:41.917 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr_osd_bytes": "536870912", 2024-09-18T10:14:41.917 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr_osd_messages": "8192", 2024-09-18T10:14:41.917 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr_pool": "true", 2024-09-18T10:14:41.917 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr_service_beacon_grace": "60.000000", 2024-09-18T10:14:41.917 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr_standby_modules": "true", 2024-09-18T10:14:41.918 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr_stats_period": "5", 2024-09-18T10:14:41.918 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr_stats_threshold": "5", 2024-09-18T10:14:41.918 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr_tick_period": "2", 2024-09-18T10:14:41.918 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr_ttl_cache_expire_seconds": "0", 2024-09-18T10:14:41.918 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_accept_timeout_factor": "2.000000", 2024-09-18T10:14:41.918 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_allow_pool_delete": "true", 2024-09-18T10:14:41.918 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_allow_pool_size_one": "true", 2024-09-18T10:14:41.918 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_auth_validate_all_caps": "true", 2024-09-18T10:14:41.918 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2024-09-18T10:14:41.918 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2024-09-18T10:14:41.918 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_client_bytes": "104857600", 2024-09-18T10:14:41.919 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_client_directed_command_retry": "2", 2024-09-18T10:14:41.919 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_client_hunt_interval": "3.000000", 2024-09-18T10:14:41.919 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2024-09-18T10:14:41.919 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2024-09-18T10:14:41.919 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2024-09-18T10:14:41.919 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_client_hunt_parallel": "3", 2024-09-18T10:14:41.920 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_client_log_interval": "1.000000", 2024-09-18T10:14:41.920 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_client_max_log_entries_per_message": "1000", 2024-09-18T10:14:41.920 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_client_ping_interval": "10.000000", 2024-09-18T10:14:41.920 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_client_ping_timeout": "30.000000", 2024-09-18T10:14:41.920 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_clock_drift_allowed": "1.000000", 2024-09-18T10:14:41.920 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2024-09-18T10:14:41.921 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2024-09-18T10:14:41.921 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_cluster_log_level": "debug", 2024-09-18T10:14:41.921 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_cluster_log_to_file": "true", 2024-09-18T10:14:41.921 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_cluster_log_to_graylog": "false", 2024-09-18T10:14:41.921 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2024-09-18T10:14:41.921 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_cluster_log_to_graylog_port": "12201", 2024-09-18T10:14:41.921 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_cluster_log_to_journald": "false", 2024-09-18T10:14:41.921 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_cluster_log_to_stderr": "false", 2024-09-18T10:14:41.921 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_cluster_log_to_syslog": "default=false", 2024-09-18T10:14:41.921 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2024-09-18T10:14:41.921 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_compact_on_bootstrap": "false", 2024-09-18T10:14:41.921 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_compact_on_start": "false", 2024-09-18T10:14:41.922 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_compact_on_trim": "true", 2024-09-18T10:14:41.922 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_con_tracker_persist_interval": "10", 2024-09-18T10:14:41.922 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_con_tracker_score_halflife": "43200", 2024-09-18T10:14:41.922 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_config_key_max_entry_size": "65536", 2024-09-18T10:14:41.922 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_cpu_threads": "4", 2024-09-18T10:14:41.922 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_crush_min_required_version": "hammer", 2024-09-18T10:14:41.922 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_daemon_bytes": "419430400", 2024-09-18T10:14:41.922 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2024-09-18T10:14:41.922 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_data_avail_crit": "5", 2024-09-18T10:14:41.923 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_data_avail_warn": "5", 2024-09-18T10:14:41.923 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_data_size_warn": "16106127360", 2024-09-18T10:14:41.923 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_debug_block_osdmap_trim": "false", 2024-09-18T10:14:41.923 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_debug_deprecated_as_obsolete": "false", 2024-09-18T10:14:41.923 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_debug_dump_json": "false", 2024-09-18T10:14:41.923 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2024-09-18T10:14:41.923 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_debug_dump_transactions": "false", 2024-09-18T10:14:41.923 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_debug_extra_checks": "false", 2024-09-18T10:14:41.923 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_debug_no_initial_persistent_features": "false", 2024-09-18T10:14:41.924 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2024-09-18T10:14:41.924 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_debug_no_require_pacific": "false", 2024-09-18T10:14:41.924 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_debug_no_require_quincy": "false", 2024-09-18T10:14:41.924 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2024-09-18T10:14:41.924 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_delta_reset_interval": "10.000000", 2024-09-18T10:14:41.924 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_dns_srv_name": "ceph-mon", 2024-09-18T10:14:41.924 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_down_mkfs_grace": "120", 2024-09-18T10:14:41.924 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_election_default_strategy": "1", 2024-09-18T10:14:41.924 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_election_timeout": "5.000000", 2024-09-18T10:14:41.924 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2024-09-18T10:14:41.924 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_elector_ping_divisor": "2", 2024-09-18T10:14:41.924 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_elector_ping_timeout": "2.000000", 2024-09-18T10:14:41.925 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_enable_op_tracker": "true", 2024-09-18T10:14:41.925 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_fake_pool_delete": "false", 2024-09-18T10:14:41.925 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_force_quorum_join": "false", 2024-09-18T10:14:41.925 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_globalid_prealloc": "10000", 2024-09-18T10:14:41.925 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_health_detail_to_clog": "false", 2024-09-18T10:14:41.925 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_health_log_update_period": "5", 2024-09-18T10:14:41.925 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_health_max_detail": "50", 2024-09-18T10:14:41.925 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_health_to_clog": "true", 2024-09-18T10:14:41.925 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_health_to_clog_interval": "600", 2024-09-18T10:14:41.925 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2024-09-18T10:14:41.925 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_host": "172.21.15.29,172.21.15.136,172.21.15.154", 2024-09-18T10:14:41.925 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_host_override": "", 2024-09-18T10:14:41.926 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_initial_members": "", 2024-09-18T10:14:41.926 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2024-09-18T10:14:41.926 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2024-09-18T10:14:41.926 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_inject_transaction_delay_max": "10.000000", 2024-09-18T10:14:41.926 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2024-09-18T10:14:41.926 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_keyvaluedb": "rocksdb", 2024-09-18T10:14:41.926 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_lease": "5.000000", 2024-09-18T10:14:41.926 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2024-09-18T10:14:41.926 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_lease_renew_interval_factor": "0.600000", 2024-09-18T10:14:41.926 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_log_full_interval": "50", 2024-09-18T10:14:41.926 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_log_max": "10000", 2024-09-18T10:14:41.926 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_log_max_summary": "50", 2024-09-18T10:14:41.926 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_max_log_entries_per_event": "4096", 2024-09-18T10:14:41.927 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_max_log_epochs": "500", 2024-09-18T10:14:41.927 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_max_mdsmap_epochs": "500", 2024-09-18T10:14:41.927 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_max_mgrmap_epochs": "500", 2024-09-18T10:14:41.927 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_max_osd": "10000", 2024-09-18T10:14:41.927 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_max_pg_per_osd": "10000", 2024-09-18T10:14:41.927 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_max_pool_pg_num": "65536", 2024-09-18T10:14:41.927 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_max_snap_prune_per_epoch": "100", 2024-09-18T10:14:41.927 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_mds_blocklist_interval": "86400.000000", 2024-09-18T10:14:41.927 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_mds_force_trim_to": "0", 2024-09-18T10:14:41.928 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_mds_skip_sanity": "false", 2024-09-18T10:14:41.928 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_memory_autotune": "true", 2024-09-18T10:14:41.928 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_memory_target": "2147483648", 2024-09-18T10:14:41.928 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_mgr_beacon_grace": "30", 2024-09-18T10:14:41.928 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2024-09-18T10:14:41.928 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_mgr_digest_period": "5", 2024-09-18T10:14:41.928 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_mgr_inactive_grace": "60", 2024-09-18T10:14:41.928 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_mgr_mkfs_grace": "240", 2024-09-18T10:14:41.928 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2024-09-18T10:14:41.928 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_min_osdmap_epochs": "500", 2024-09-18T10:14:41.929 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_op_complaint_time": "30", 2024-09-18T10:14:41.929 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_op_history_duration": "600", 2024-09-18T10:14:41.929 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_op_history_size": "20", 2024-09-18T10:14:41.929 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_op_history_slow_op_size": "20", 2024-09-18T10:14:41.929 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_op_history_slow_op_threshold": "10", 2024-09-18T10:14:41.929 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_op_log_threshold": "5", 2024-09-18T10:14:41.929 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_osd_adjust_down_out_interval": "true", 2024-09-18T10:14:41.929 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2024-09-18T10:14:41.929 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2024-09-18T10:14:41.929 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_osd_auto_mark_in": "false", 2024-09-18T10:14:41.929 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_osd_auto_mark_new_in": "true", 2024-09-18T10:14:41.930 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2024-09-18T10:14:41.930 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2024-09-18T10:14:41.930 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_osd_cache_size": "500", 2024-09-18T10:14:41.930 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_osd_cache_size_min": "134217728", 2024-09-18T10:14:41.930 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_osd_crush_smoke_test": "true", 2024-09-18T10:14:41.930 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_osd_destroyed_out_interval": "600", 2024-09-18T10:14:41.930 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_osd_down_out_interval": "600", 2024-09-18T10:14:41.930 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_osd_down_out_subtree_limit": "rack", 2024-09-18T10:14:41.930 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2024-09-18T10:14:41.930 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_osd_force_trim_to": "0", 2024-09-18T10:14:41.930 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_osd_full_ratio": "0.950000", 2024-09-18T10:14:41.930 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2024-09-18T10:14:41.930 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_osd_laggy_halflife": "3600", 2024-09-18T10:14:41.931 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_osd_laggy_max_interval": "300", 2024-09-18T10:14:41.931 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_osd_laggy_weight": "0.300000", 2024-09-18T10:14:41.931 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2024-09-18T10:14:41.931 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_osd_max_creating_pgs": "1024", 2024-09-18T10:14:41.931 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_osd_max_initial_pgs": "1024", 2024-09-18T10:14:41.931 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_osd_min_down_reporters": "2", 2024-09-18T10:14:41.931 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_osd_min_in_ratio": "0.750000", 2024-09-18T10:14:41.931 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_osd_min_up_ratio": "0.300000", 2024-09-18T10:14:41.931 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_osd_nearfull_ratio": "0.850000", 2024-09-18T10:14:41.931 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_osd_prime_pg_temp": "true", 2024-09-18T10:14:41.931 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2024-09-18T10:14:41.931 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2024-09-18T10:14:41.931 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_osd_report_timeout": "900", 2024-09-18T10:14:41.932 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_osd_reporter_subtree_level": "osd", 2024-09-18T10:14:41.932 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2024-09-18T10:14:41.932 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_osd_warn_num_repaired": "10", 2024-09-18T10:14:41.932 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_osd_warn_op_age": "32.000000", 2024-09-18T10:14:41.932 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_osdmap_full_prune_enabled": "true", 2024-09-18T10:14:41.932 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_osdmap_full_prune_interval": "10", 2024-09-18T10:14:41.932 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_osdmap_full_prune_min": "10000", 2024-09-18T10:14:41.932 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_osdmap_full_prune_txsize": "100", 2024-09-18T10:14:41.932 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2024-09-18T10:14:41.932 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_pg_stuck_threshold": "60", 2024-09-18T10:14:41.932 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2024-09-18T10:14:41.932 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_pg_warn_min_objects": "10000", 2024-09-18T10:14:41.933 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_pg_warn_min_per_osd": "0", 2024-09-18T10:14:41.933 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_pg_warn_min_pool_objects": "1000", 2024-09-18T10:14:41.933 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_pool_quota_crit_threshold": "0", 2024-09-18T10:14:41.933 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_pool_quota_warn_threshold": "0", 2024-09-18T10:14:41.933 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_probe_timeout": "2.000000", 2024-09-18T10:14:41.933 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_reweight_max_change": "0.050000", 2024-09-18T10:14:41.933 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_reweight_max_osds": "4", 2024-09-18T10:14:41.933 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_reweight_min_bytes_per_osd": "10", 2024-09-18T10:14:41.933 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_reweight_min_pgs_per_osd": "4", 2024-09-18T10:14:41.933 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2024-09-18T10:14:41.933 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2024-09-18T10:14:41.933 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2024-09-18T10:14:41.934 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_scrub_interval": "300", 2024-09-18T10:14:41.934 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_scrub_max_keys": "100", 2024-09-18T10:14:41.934 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_scrub_timeout": "300", 2024-09-18T10:14:41.934 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_session_timeout": "300", 2024-09-18T10:14:41.934 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_smart_report_timeout": "5", 2024-09-18T10:14:41.934 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_stat_smooth_intervals": "6", 2024-09-18T10:14:41.934 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2024-09-18T10:14:41.935 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_stretch_pool_min_size": "2", 2024-09-18T10:14:41.935 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_stretch_pool_size": "4", 2024-09-18T10:14:41.935 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2024-09-18T10:14:41.935 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_subscribe_interval": "86400.000000", 2024-09-18T10:14:41.935 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_sync_debug": "false", 2024-09-18T10:14:41.935 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_sync_max_payload_keys": "2000", 2024-09-18T10:14:41.935 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_sync_max_payload_size": "1048576", 2024-09-18T10:14:41.935 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_sync_provider_kill_at": "0", 2024-09-18T10:14:41.935 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_sync_requester_kill_at": "0", 2024-09-18T10:14:41.935 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_sync_timeout": "60.000000", 2024-09-18T10:14:41.935 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_target_pg_per_osd": "100", 2024-09-18T10:14:41.935 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_tick_interval": "5", 2024-09-18T10:14:41.936 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_timecheck_interval": "300.000000", 2024-09-18T10:14:41.936 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_timecheck_skew_interval": "30.000000", 2024-09-18T10:14:41.936 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_warn_older_version_delay": "604800", 2024-09-18T10:14:41.936 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2024-09-18T10:14:41.936 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2024-09-18T10:14:41.936 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2024-09-18T10:14:41.936 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_warn_on_filestore_osds": "false", 2024-09-18T10:14:41.936 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2024-09-18T10:14:41.936 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2024-09-18T10:14:41.936 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2024-09-18T10:14:41.936 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_warn_on_misplaced": "false", 2024-09-18T10:14:41.936 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2024-09-18T10:14:41.937 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_warn_on_older_version": "true", 2024-09-18T10:14:41.937 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2024-09-18T10:14:41.937 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_warn_on_pool_no_app": "true", 2024-09-18T10:14:41.937 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_warn_on_pool_no_redundancy": "false", 2024-09-18T10:14:41.937 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2024-09-18T10:14:41.937 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2024-09-18T10:14:41.937 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2024-09-18T10:14:41.937 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_warn_on_too_few_osds": "false", 2024-09-18T10:14:41.937 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2024-09-18T10:14:41.937 INFO:teuthology.orchestra.run.smithi029.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2024-09-18T10:14:41.938 INFO:teuthology.orchestra.run.smithi029.stdout: "monmap": "", 2024-09-18T10:14:41.938 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_async_op_threads": "3", 2024-09-18T10:14:41.938 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_async_rdma_buffer_size": "131072", 2024-09-18T10:14:41.938 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_async_rdma_cm": "false", 2024-09-18T10:14:41.938 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_async_rdma_device_name": "", 2024-09-18T10:14:41.938 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_async_rdma_dscp": "96", 2024-09-18T10:14:41.938 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_async_rdma_enable_hugepage": "false", 2024-09-18T10:14:41.938 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_async_rdma_gid_idx": "0", 2024-09-18T10:14:41.939 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_async_rdma_local_gid": "", 2024-09-18T10:14:41.939 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_async_rdma_polling_us": "1000", 2024-09-18T10:14:41.939 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_async_rdma_port_num": "1", 2024-09-18T10:14:41.939 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_async_rdma_receive_buffers": "32768", 2024-09-18T10:14:41.939 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_async_rdma_receive_queue_len": "4096", 2024-09-18T10:14:41.939 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_async_rdma_roce_ver": "1", 2024-09-18T10:14:41.939 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_async_rdma_send_buffers": "1024", 2024-09-18T10:14:41.939 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_async_rdma_sl": "3", 2024-09-18T10:14:41.939 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_async_rdma_support_srq": "true", 2024-09-18T10:14:41.939 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_async_rdma_type": "ib", 2024-09-18T10:14:41.939 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_async_reap_threshold": "5", 2024-09-18T10:14:41.939 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_bind_before_connect": "false", 2024-09-18T10:14:41.939 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_bind_ipv4": "true", 2024-09-18T10:14:41.940 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_bind_ipv6": "false", 2024-09-18T10:14:41.940 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_bind_msgr1": "true", 2024-09-18T10:14:41.940 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_bind_msgr2": "true", 2024-09-18T10:14:41.940 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_bind_port_max": "7568", 2024-09-18T10:14:41.940 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_bind_port_min": "6800", 2024-09-18T10:14:41.940 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_bind_prefer_ipv4": "false", 2024-09-18T10:14:41.940 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_bind_retry_count": "3", 2024-09-18T10:14:41.940 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_bind_retry_delay": "5", 2024-09-18T10:14:41.940 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_blackhole_client": "false", 2024-09-18T10:14:41.940 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_blackhole_mds": "false", 2024-09-18T10:14:41.940 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_blackhole_mgr": "false", 2024-09-18T10:14:41.940 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_blackhole_mon": "false", 2024-09-18T10:14:41.940 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_blackhole_osd": "false", 2024-09-18T10:14:41.941 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_client_mode": "crc secure", 2024-09-18T10:14:41.941 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_cluster_mode": "crc secure", 2024-09-18T10:14:41.941 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_cluster_type": "", 2024-09-18T10:14:41.941 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_compress_secure": "false", 2024-09-18T10:14:41.941 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_connection_idle_timeout": "900", 2024-09-18T10:14:41.941 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_connection_ready_timeout": "10", 2024-09-18T10:14:41.941 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_crc_data": "true", 2024-09-18T10:14:41.941 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_crc_header": "true", 2024-09-18T10:14:41.941 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_die_on_bad_msg": "false", 2024-09-18T10:14:41.941 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_die_on_bug": "true", 2024-09-18T10:14:41.941 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_die_on_old_message": "true", 2024-09-18T10:14:41.941 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_die_on_skipped_message": "false", 2024-09-18T10:14:41.942 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_die_on_unhandled_msg": "false", 2024-09-18T10:14:41.942 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_dispatch_throttle_bytes": "104857600", 2024-09-18T10:14:41.942 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_dpdk_coremask": "0xF", 2024-09-18T10:14:41.942 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_dpdk_debug_allow_loopback": "false", 2024-09-18T10:14:41.942 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_dpdk_devs_allowlist": "", 2024-09-18T10:14:41.942 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_dpdk_enable_tso": "true", 2024-09-18T10:14:41.942 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2024-09-18T10:14:41.942 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_dpdk_host_ipv4_addr": "", 2024-09-18T10:14:41.942 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_dpdk_hugepages": "", 2024-09-18T10:14:41.942 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_dpdk_hw_flow_control": "true", 2024-09-18T10:14:41.942 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2024-09-18T10:14:41.942 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_dpdk_lro": "true", 2024-09-18T10:14:41.942 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_dpdk_memory_channel": "4", 2024-09-18T10:14:41.943 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2024-09-18T10:14:41.943 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_dpdk_pmd": "", 2024-09-18T10:14:41.943 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_dpdk_port_id": "0", 2024-09-18T10:14:41.943 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2024-09-18T10:14:41.943 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_dump_corrupt_message_level": "1", 2024-09-18T10:14:41.943 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_dump_on_send": "false", 2024-09-18T10:14:41.943 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_initial_backoff": "0.200000", 2024-09-18T10:14:41.943 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_inject_delay_max": "1.000000", 2024-09-18T10:14:41.943 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_inject_delay_probability": "0.000000", 2024-09-18T10:14:41.943 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_inject_delay_type": "", 2024-09-18T10:14:41.943 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_inject_internal_delays": "0.000000", 2024-09-18T10:14:41.943 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_inject_socket_failures": "0", 2024-09-18T10:14:41.943 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_learn_addr_from_peer": "true", 2024-09-18T10:14:41.944 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_max_accept_failures": "4", 2024-09-18T10:14:41.944 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_max_backoff": "15.000000", 2024-09-18T10:14:41.944 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_mon_client_mode": "secure crc", 2024-09-18T10:14:41.944 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_mon_cluster_mode": "secure crc", 2024-09-18T10:14:41.944 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_mon_service_mode": "secure crc", 2024-09-18T10:14:41.944 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_osd_compress_min_size": "1024", 2024-09-18T10:14:41.944 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_osd_compress_mode": "none", 2024-09-18T10:14:41.944 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_osd_compression_algorithm": "snappy zlib zstd lz4", 2024-09-18T10:14:41.944 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2024-09-18T10:14:41.945 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_pq_min_cost": "65536", 2024-09-18T10:14:41.945 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_public_type": "", 2024-09-18T10:14:41.945 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_service_mode": "crc secure", 2024-09-18T10:14:41.945 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_tcp_listen_backlog": "512", 2024-09-18T10:14:41.945 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_tcp_nodelay": "true", 2024-09-18T10:14:41.945 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_tcp_prefetch_max_size": "4096", 2024-09-18T10:14:41.945 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_tcp_rcvbuf": "0", 2024-09-18T10:14:41.946 INFO:teuthology.orchestra.run.smithi029.stdout: "ms_type": "async+posix", 2024-09-18T10:14:41.946 INFO:teuthology.orchestra.run.smithi029.stdout: "no_config_file": "false", 2024-09-18T10:14:41.946 INFO:teuthology.orchestra.run.smithi029.stdout: "objecter_completion_locks_per_session": "32", 2024-09-18T10:14:41.946 INFO:teuthology.orchestra.run.smithi029.stdout: "objecter_debug_inject_relock_delay": "false", 2024-09-18T10:14:41.946 INFO:teuthology.orchestra.run.smithi029.stdout: "objecter_inflight_op_bytes": "104857600", 2024-09-18T10:14:41.946 INFO:teuthology.orchestra.run.smithi029.stdout: "objecter_inflight_ops": "1024", 2024-09-18T10:14:41.946 INFO:teuthology.orchestra.run.smithi029.stdout: "objecter_inject_no_watch_ping": "false", 2024-09-18T10:14:41.946 INFO:teuthology.orchestra.run.smithi029.stdout: "objecter_retry_writes_after_first_reply": "false", 2024-09-18T10:14:41.946 INFO:teuthology.orchestra.run.smithi029.stdout: "objecter_tick_interval": "5.000000", 2024-09-18T10:14:41.946 INFO:teuthology.orchestra.run.smithi029.stdout: "objecter_timeout": "10.000000", 2024-09-18T10:14:41.946 INFO:teuthology.orchestra.run.smithi029.stdout: "objectstore_blackhole": "false", 2024-09-18T10:14:41.946 INFO:teuthology.orchestra.run.smithi029.stdout: "openssl_engine_opts": "", 2024-09-18T10:14:41.947 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_agent_delay_time": "5.000000", 2024-09-18T10:14:41.947 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_agent_hist_halflife": "1000", 2024-09-18T10:14:41.947 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_agent_max_low_ops": "2", 2024-09-18T10:14:41.947 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_agent_max_ops": "4", 2024-09-18T10:14:41.947 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_agent_min_evict_effort": "0.100000", 2024-09-18T10:14:41.947 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_agent_quantize_effort": "0.100000", 2024-09-18T10:14:41.947 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_agent_slop": "0.020000", 2024-09-18T10:14:41.947 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_aggregated_slow_ops_logging": "true", 2024-09-18T10:14:41.947 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_allow_recovery_below_min_size": "true", 2024-09-18T10:14:41.947 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_asio_thread_count": "2", 2024-09-18T10:14:41.947 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_async_recovery_min_cost": "100", 2024-09-18T10:14:41.947 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_auto_mark_unfound_lost": "false", 2024-09-18T10:14:41.947 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_backfill_retry_interval": "30.000000", 2024-09-18T10:14:41.948 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_backfill_scan_max": "512", 2024-09-18T10:14:41.948 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_backfill_scan_min": "64", 2024-09-18T10:14:41.948 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_backoff_on_degraded": "false", 2024-09-18T10:14:41.948 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_backoff_on_peering": "false", 2024-09-18T10:14:41.948 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_backoff_on_unfound": "true", 2024-09-18T10:14:41.948 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_beacon_report_interval": "300", 2024-09-18T10:14:41.948 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_bench_duration": "30", 2024-09-18T10:14:41.948 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_bench_large_size_max_throughput": "104857600", 2024-09-18T10:14:41.948 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_bench_max_block_size": "67108864", 2024-09-18T10:14:41.948 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_bench_small_size_max_iops": "100", 2024-09-18T10:14:41.948 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_blkin_trace_all": "false", 2024-09-18T10:14:41.948 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2024-09-18T10:14:41.949 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2024-09-18T10:14:41.949 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_check_for_log_corruption": "false", 2024-09-18T10:14:41.949 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_check_max_object_name_len_on_startup": "true", 2024-09-18T10:14:41.949 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-18T10:14:41.949 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_class_dir": "/usr/lib/rados-classes", 2024-09-18T10:14:41.949 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-18T10:14:41.949 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_class_update_on_start": "true", 2024-09-18T10:14:41.949 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_client_message_cap": "256", 2024-09-18T10:14:41.949 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_client_message_size_cap": "524288000", 2024-09-18T10:14:41.949 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_client_op_priority": "63", 2024-09-18T10:14:41.949 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_client_watch_timeout": "30", 2024-09-18T10:14:41.949 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_command_max_records": "256", 2024-09-18T10:14:41.949 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_compact_on_start": "false", 2024-09-18T10:14:41.950 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_copyfrom_max_chunk": "8388608", 2024-09-18T10:14:41.950 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_crush_chooseleaf_type": "0", 2024-09-18T10:14:41.950 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_crush_initial_weight": "-1.000000", 2024-09-18T10:14:41.950 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_crush_update_on_start": "true", 2024-09-18T10:14:41.950 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_crush_update_weight_set": "true", 2024-09-18T10:14:41.950 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2024-09-18T10:14:41.950 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2024-09-18T10:14:41.950 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2024-09-18T10:14:41.950 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_debug_drop_ping_duration": "0", 2024-09-18T10:14:41.950 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_debug_drop_ping_probability": "0.000000", 2024-09-18T10:14:41.950 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_debug_feed_pullee": "-1", 2024-09-18T10:14:41.950 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_debug_inject_copyfrom_error": "false", 2024-09-18T10:14:41.950 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2024-09-18T10:14:41.951 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_debug_inject_dispatch_delay_probability": "0.100000", 2024-09-18T10:14:41.951 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_debug_misdirected_ops": "false", 2024-09-18T10:14:41.951 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_debug_no_acting_change": "false", 2024-09-18T10:14:41.951 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_debug_no_purge_strays": "false", 2024-09-18T10:14:41.951 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_debug_op_order": "false", 2024-09-18T10:14:41.951 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_debug_pg_log_writeout": "false", 2024-09-18T10:14:41.951 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_debug_pretend_recovery_active": "false", 2024-09-18T10:14:41.951 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_debug_random_push_read_error": "0.000000", 2024-09-18T10:14:41.951 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2024-09-18T10:14:41.951 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_debug_shutdown": "false", 2024-09-18T10:14:41.951 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2024-09-18T10:14:41.951 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2024-09-18T10:14:41.951 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_debug_verify_cached_snaps": "false", 2024-09-18T10:14:41.952 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_debug_verify_missing_on_start": "false", 2024-09-18T10:14:41.952 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_debug_verify_snaps": "false", 2024-09-18T10:14:41.952 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_debug_verify_stray_on_activate": "false", 2024-09-18T10:14:41.952 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_deep_scrub_interval": "604800.000000", 2024-09-18T10:14:41.952 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_deep_scrub_keys": "1024", 2024-09-18T10:14:41.952 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2024-09-18T10:14:41.952 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2024-09-18T10:14:41.952 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2024-09-18T10:14:41.952 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_deep_scrub_stride": "524288", 2024-09-18T10:14:41.952 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2024-09-18T10:14:41.952 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_default_data_pool_replay_window": "5", 2024-09-18T10:14:41.952 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_default_notify_timeout": "30", 2024-09-18T10:14:41.952 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_delete_sleep": "0.000000", 2024-09-18T10:14:41.953 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_delete_sleep_hdd": "5.000000", 2024-09-18T10:14:41.953 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_delete_sleep_hybrid": "1.000000", 2024-09-18T10:14:41.953 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_delete_sleep_ssd": "1.000000", 2024-09-18T10:14:41.953 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_discard_disconnected_ops": "true", 2024-09-18T10:14:41.953 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_enable_op_tracker": "true", 2024-09-18T10:14:41.953 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_erasure_code_plugins": "jerasure lrc", 2024-09-18T10:14:41.953 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_failsafe_full_ratio": "0.970000", 2024-09-18T10:14:41.953 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_fast_fail_on_connection_refused": "true", 2024-09-18T10:14:41.953 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_fast_info": "true", 2024-09-18T10:14:41.953 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_fast_shutdown": "true", 2024-09-18T10:14:41.953 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_fast_shutdown_notify_mon": "true", 2024-09-18T10:14:41.953 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_fast_shutdown_timeout": "15", 2024-09-18T10:14:41.953 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_find_best_info_ignore_history_les": "false", 2024-09-18T10:14:41.954 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_force_auth_primary_missing_objects": "100", 2024-09-18T10:14:41.954 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2024-09-18T10:14:41.954 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_function_tracing": "false", 2024-09-18T10:14:41.954 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_heartbeat_grace": "20", 2024-09-18T10:14:41.954 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_heartbeat_interval": "6", 2024-09-18T10:14:41.954 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2024-09-18T10:14:41.954 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_heartbeat_min_peers": "10", 2024-09-18T10:14:41.954 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_heartbeat_min_size": "2000", 2024-09-18T10:14:41.954 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_heartbeat_stale": "600", 2024-09-18T10:14:41.954 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_heartbeat_use_min_delay_socket": "false", 2024-09-18T10:14:41.955 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_hit_set_max_size": "100000", 2024-09-18T10:14:41.955 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_hit_set_min_size": "1000", 2024-09-18T10:14:41.955 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_hit_set_namespace": ".ceph-internal", 2024-09-18T10:14:41.955 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_ignore_stale_divergent_priors": "false", 2024-09-18T10:14:41.955 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_inject_bad_map_crc_probability": "0.000000", 2024-09-18T10:14:41.955 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_inject_failure_on_pg_removal": "false", 2024-09-18T10:14:41.955 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_journal": "/var/lib/ceph/osd/ceph-a/journal", 2024-09-18T10:14:41.955 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_journal_flush_on_shutdown": "true", 2024-09-18T10:14:41.955 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_journal_size": "5120", 2024-09-18T10:14:41.955 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_kill_backfill_at": "0", 2024-09-18T10:14:41.955 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_loop_before_reset_tphandle": "64", 2024-09-18T10:14:41.955 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_map_cache_size": "50", 2024-09-18T10:14:41.955 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_map_dedup": "true", 2024-09-18T10:14:41.955 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_map_message_max": "40", 2024-09-18T10:14:41.956 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_map_message_max_bytes": "10485760", 2024-09-18T10:14:41.956 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_map_share_max_epochs": "40", 2024-09-18T10:14:41.956 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_max_attr_name_len": "100", 2024-09-18T10:14:41.956 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_max_attr_size": "0", 2024-09-18T10:14:41.956 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_max_backfills": "1", 2024-09-18T10:14:41.956 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_max_markdown_count": "5", 2024-09-18T10:14:41.956 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_max_markdown_period": "600", 2024-09-18T10:14:41.956 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_max_object_name_len": "2048", 2024-09-18T10:14:41.956 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_max_object_namespace_len": "256", 2024-09-18T10:14:41.956 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_max_object_size": "134217728", 2024-09-18T10:14:41.956 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_max_omap_bytes_per_request": "1073741824", 2024-09-18T10:14:41.956 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_max_omap_entries_per_request": "1024", 2024-09-18T10:14:41.956 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_max_pg_blocked_by": "16", 2024-09-18T10:14:41.957 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_max_pg_log_entries": "10000", 2024-09-18T10:14:41.957 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_max_pg_per_osd_hard_ratio": "3.000000", 2024-09-18T10:14:41.957 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_max_pgls": "1024", 2024-09-18T10:14:41.957 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_max_push_cost": "8388608", 2024-09-18T10:14:41.957 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_max_push_objects": "10", 2024-09-18T10:14:41.957 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_max_scrubs": "3", 2024-09-18T10:14:41.957 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_max_snap_prune_intervals_per_epoch": "512", 2024-09-18T10:14:41.957 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_max_trimming_pgs": "2", 2024-09-18T10:14:41.957 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_max_write_op_reply_len": "64", 2024-09-18T10:14:41.957 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_max_write_size": "90", 2024-09-18T10:14:41.957 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_mclock_force_run_benchmark_on_init": "false", 2024-09-18T10:14:41.957 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_mclock_iops_capacity_threshold_hdd": "500.000000", 2024-09-18T10:14:41.957 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_mclock_iops_capacity_threshold_ssd": "80000.000000", 2024-09-18T10:14:41.958 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_mclock_max_capacity_iops_hdd": "315.000000", 2024-09-18T10:14:41.958 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_mclock_max_capacity_iops_ssd": "21500.000000", 2024-09-18T10:14:41.958 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_mclock_max_sequential_bandwidth_hdd": "157286400", 2024-09-18T10:14:41.958 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_mclock_max_sequential_bandwidth_ssd": "1258291200", 2024-09-18T10:14:41.958 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_mclock_override_recovery_settings": "false", 2024-09-18T10:14:41.958 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_mclock_profile": "balanced", 2024-09-18T10:14:41.958 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_mclock_scheduler_anticipation_timeout": "0.000000", 2024-09-18T10:14:41.958 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_mclock_scheduler_background_best_effort_lim": "0.000000", 2024-09-18T10:14:41.958 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_mclock_scheduler_background_best_effort_res": "0.000000", 2024-09-18T10:14:41.958 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_mclock_scheduler_background_best_effort_wgt": "1", 2024-09-18T10:14:41.958 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_mclock_scheduler_background_recovery_lim": "0.000000", 2024-09-18T10:14:41.958 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_mclock_scheduler_background_recovery_res": "0.000000", 2024-09-18T10:14:41.958 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_mclock_scheduler_background_recovery_wgt": "1", 2024-09-18T10:14:41.959 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_mclock_scheduler_client_lim": "0.000000", 2024-09-18T10:14:41.959 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_mclock_scheduler_client_res": "0.000000", 2024-09-18T10:14:41.959 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_mclock_scheduler_client_wgt": "1", 2024-09-18T10:14:41.959 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_mclock_skip_benchmark": "false", 2024-09-18T10:14:41.959 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_memory_base": "805306368", 2024-09-18T10:14:41.959 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_memory_cache_min": "134217728", 2024-09-18T10:14:41.959 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_memory_cache_resize_interval": "1.000000", 2024-09-18T10:14:41.959 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_memory_expected_fragmentation": "0.150000", 2024-09-18T10:14:41.959 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_memory_target": "4294967296", 2024-09-18T10:14:41.959 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_memory_target_autotune": "false", 2024-09-18T10:14:41.959 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_memory_target_cgroup_limit_ratio": "0.800000", 2024-09-18T10:14:41.959 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_min_pg_log_entries": "250", 2024-09-18T10:14:41.959 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_min_recovery_priority": "0", 2024-09-18T10:14:41.960 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_mon_heartbeat_interval": "30", 2024-09-18T10:14:41.960 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_mon_heartbeat_stat_stale": "3600", 2024-09-18T10:14:41.960 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_mon_report_interval": "5", 2024-09-18T10:14:41.960 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_mon_report_max_in_flight": "2", 2024-09-18T10:14:41.960 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_mon_shutdown_timeout": "5.000000", 2024-09-18T10:14:41.960 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_num_cache_shards": "32", 2024-09-18T10:14:41.960 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_num_op_tracker_shard": "32", 2024-09-18T10:14:41.960 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_numa_auto_affinity": "true", 2024-09-18T10:14:41.960 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_numa_node": "-1", 2024-09-18T10:14:41.960 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_numa_prefer_iface": "true", 2024-09-18T10:14:41.960 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_object_clean_region_max_num_intervals": "10", 2024-09-18T10:14:41.960 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_objecter_finishers": "1", 2024-09-18T10:14:41.961 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_objectstore": "bluestore", 2024-09-18T10:14:41.961 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_objectstore_fuse": "false", 2024-09-18T10:14:41.961 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_objectstore_tracing": "false", 2024-09-18T10:14:41.961 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_op_complaint_time": "30.000000", 2024-09-18T10:14:41.961 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_op_history_duration": "600", 2024-09-18T10:14:41.961 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_op_history_size": "20", 2024-09-18T10:14:41.961 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_op_history_slow_op_size": "20", 2024-09-18T10:14:41.961 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_op_history_slow_op_threshold": "10.000000", 2024-09-18T10:14:41.961 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_op_log_threshold": "5", 2024-09-18T10:14:41.962 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_op_num_shards": "0", 2024-09-18T10:14:41.962 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_op_num_shards_hdd": "5", 2024-09-18T10:14:41.962 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_op_num_shards_ssd": "8", 2024-09-18T10:14:41.962 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_op_num_threads_per_shard": "0", 2024-09-18T10:14:41.962 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_op_num_threads_per_shard_hdd": "1", 2024-09-18T10:14:41.962 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_op_num_threads_per_shard_ssd": "2", 2024-09-18T10:14:41.962 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_op_pq_max_tokens_per_priority": "4194304", 2024-09-18T10:14:41.962 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_op_pq_min_cost": "65536", 2024-09-18T10:14:41.962 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_op_queue": "mclock_scheduler", 2024-09-18T10:14:41.962 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_op_queue_cut_off": "high", 2024-09-18T10:14:41.962 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_op_thread_suicide_timeout": "150", 2024-09-18T10:14:41.962 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_op_thread_timeout": "15", 2024-09-18T10:14:41.962 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_open_classes_on_start": "true", 2024-09-18T10:14:41.962 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_os_flags": "0", 2024-09-18T10:14:41.963 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_peering_op_priority": "255", 2024-09-18T10:14:41.963 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_pg_delete_cost": "1048576", 2024-09-18T10:14:41.963 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_pg_delete_priority": "5", 2024-09-18T10:14:41.963 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_pg_epoch_max_lag_factor": "2.000000", 2024-09-18T10:14:41.963 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_pg_epoch_persisted_max_stale": "40", 2024-09-18T10:14:41.963 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_pg_log_dups_tracked": "3000", 2024-09-18T10:14:41.963 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_pg_log_trim_max": "10000", 2024-09-18T10:14:41.963 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_pg_log_trim_min": "100", 2024-09-18T10:14:41.963 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_pg_max_concurrent_snap_trims": "2", 2024-09-18T10:14:41.963 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_pg_object_context_cache_count": "64", 2024-09-18T10:14:41.963 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_pg_stat_report_interval_max": "500", 2024-09-18T10:14:41.964 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_pool_default_cache_max_evict_check_size": "10", 2024-09-18T10:14:41.964 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_pool_default_cache_min_evict_age": "0", 2024-09-18T10:14:41.964 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_pool_default_cache_min_flush_age": "0", 2024-09-18T10:14:41.964 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_pool_default_cache_target_dirty_high_ratio": "0.600000", 2024-09-18T10:14:41.964 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_pool_default_cache_target_dirty_ratio": "0.400000", 2024-09-18T10:14:41.964 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_pool_default_cache_target_full_ratio": "0.800000", 2024-09-18T10:14:41.964 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_pool_default_crush_rule": "-1", 2024-09-18T10:14:41.964 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_pool_default_ec_fast_read": "false", 2024-09-18T10:14:41.965 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_pool_default_erasure_code_profile": "plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd", 2024-09-18T10:14:41.965 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_pool_default_flag_bulk": "false", 2024-09-18T10:14:41.965 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_pool_default_flag_hashpspool": "true", 2024-09-18T10:14:41.965 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_pool_default_flag_nodelete": "false", 2024-09-18T10:14:41.965 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_pool_default_flag_nopgchange": "false", 2024-09-18T10:14:41.965 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_pool_default_flag_nosizechange": "false", 2024-09-18T10:14:41.965 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_pool_default_flags": "0", 2024-09-18T10:14:41.966 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_pool_default_hit_set_bloom_fpp": "0.050000", 2024-09-18T10:14:41.966 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_pool_default_min_size": "0", 2024-09-18T10:14:41.966 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_pool_default_pg_autoscale_mode": "off", 2024-09-18T10:14:41.966 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_pool_default_pg_num": "32", 2024-09-18T10:14:41.966 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_pool_default_pgp_num": "0", 2024-09-18T10:14:41.966 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_pool_default_read_lease_ratio": "0.800000", 2024-09-18T10:14:41.966 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_pool_default_size": "2", 2024-09-18T10:14:41.966 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_pool_default_type": "replicated", 2024-09-18T10:14:41.966 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_pool_erasure_code_stripe_unit": "4096", 2024-09-18T10:14:41.966 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_pool_use_gmt_hitset": "true", 2024-09-18T10:14:41.966 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_push_per_object_cost": "1000", 2024-09-18T10:14:41.966 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_read_ec_check_for_errors": "false", 2024-09-18T10:14:41.966 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_recover_clone_overlap": "true", 2024-09-18T10:14:41.967 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_recover_clone_overlap_limit": "10", 2024-09-18T10:14:41.967 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_recovery_cost": "20971520", 2024-09-18T10:14:41.967 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_recovery_delay_start": "0.000000", 2024-09-18T10:14:41.967 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_recovery_max_active": "0", 2024-09-18T10:14:41.967 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_recovery_max_active_hdd": "3", 2024-09-18T10:14:41.967 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_recovery_max_active_ssd": "10", 2024-09-18T10:14:41.967 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_recovery_max_chunk": "8388608", 2024-09-18T10:14:41.967 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_recovery_max_omap_entries_per_chunk": "8096", 2024-09-18T10:14:41.967 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_recovery_max_single_start": "1", 2024-09-18T10:14:41.967 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_recovery_op_priority": "3", 2024-09-18T10:14:41.967 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_recovery_op_warn_multiple": "16", 2024-09-18T10:14:41.967 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_recovery_priority": "5", 2024-09-18T10:14:41.967 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_recovery_retry_interval": "30.000000", 2024-09-18T10:14:41.968 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_recovery_sleep": "0.000000", 2024-09-18T10:14:41.968 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_recovery_sleep_hdd": "0.100000", 2024-09-18T10:14:41.968 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_recovery_sleep_hybrid": "0.025000", 2024-09-18T10:14:41.968 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_recovery_sleep_ssd": "0.000000", 2024-09-18T10:14:41.968 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_repair_during_recovery": "false", 2024-09-18T10:14:41.968 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_requested_scrub_priority": "120", 2024-09-18T10:14:41.968 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_rocksdb_iterator_bounds_enabled": "true", 2024-09-18T10:14:41.968 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_rollback_to_cluster_snap": "", 2024-09-18T10:14:41.968 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_scrub_auto_repair": "false", 2024-09-18T10:14:41.968 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_scrub_auto_repair_num_errors": "5", 2024-09-18T10:14:41.968 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_scrub_backoff_ratio": "0.660000", 2024-09-18T10:14:41.968 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_scrub_begin_hour": "0", 2024-09-18T10:14:41.968 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_scrub_begin_week_day": "0", 2024-09-18T10:14:41.969 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_scrub_chunk_max": "25", 2024-09-18T10:14:41.969 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_scrub_chunk_min": "5", 2024-09-18T10:14:41.969 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_scrub_cost": "52428800", 2024-09-18T10:14:41.969 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_scrub_during_recovery": "false", 2024-09-18T10:14:41.969 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_scrub_end_hour": "0", 2024-09-18T10:14:41.969 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_scrub_end_week_day": "0", 2024-09-18T10:14:41.969 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_scrub_event_cost": "4096", 2024-09-18T10:14:41.969 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_scrub_extended_sleep": "0.000000", 2024-09-18T10:14:41.969 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_scrub_interval_randomize_ratio": "0.500000", 2024-09-18T10:14:41.969 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_scrub_invalid_stats": "true", 2024-09-18T10:14:41.969 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_scrub_load_threshold": "0.500000", 2024-09-18T10:14:41.969 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_scrub_max_interval": "604800.000000", 2024-09-18T10:14:41.969 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_scrub_max_preemptions": "5", 2024-09-18T10:14:41.970 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_scrub_min_interval": "86400.000000", 2024-09-18T10:14:41.970 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_scrub_priority": "5", 2024-09-18T10:14:41.970 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_scrub_sleep": "0.000000", 2024-09-18T10:14:41.970 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_shutdown_pgref_assert": "false", 2024-09-18T10:14:41.970 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_skip_data_digest": "false", 2024-09-18T10:14:41.970 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_smart_report_timeout": "5", 2024-09-18T10:14:41.970 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_snap_trim_cost": "1048576", 2024-09-18T10:14:41.970 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_snap_trim_priority": "5", 2024-09-18T10:14:41.970 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_snap_trim_sleep": "0.000000", 2024-09-18T10:14:41.970 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_snap_trim_sleep_hdd": "5.000000", 2024-09-18T10:14:41.970 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_snap_trim_sleep_hybrid": "2.000000", 2024-09-18T10:14:41.970 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_snap_trim_sleep_ssd": "0.000000", 2024-09-18T10:14:41.970 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_target_pg_log_entries_per_osd": "300000", 2024-09-18T10:14:41.971 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_target_transaction_size": "30", 2024-09-18T10:14:41.971 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_tier_default_cache_hit_set_count": "4", 2024-09-18T10:14:41.971 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_tier_default_cache_hit_set_grade_decay_rate": "20", 2024-09-18T10:14:41.971 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_tier_default_cache_hit_set_period": "1200", 2024-09-18T10:14:41.971 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_tier_default_cache_hit_set_search_last_n": "1", 2024-09-18T10:14:41.971 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_tier_default_cache_hit_set_type": "bloom", 2024-09-18T10:14:41.971 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_tier_default_cache_min_read_recency_for_promote": "1", 2024-09-18T10:14:41.971 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_tier_default_cache_min_write_recency_for_promote": "1", 2024-09-18T10:14:41.971 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_tier_default_cache_mode": "writeback", 2024-09-18T10:14:41.971 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_tier_promote_max_bytes_sec": "5242880", 2024-09-18T10:14:41.971 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_tier_promote_max_objects_sec": "25", 2024-09-18T10:14:41.971 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_tracing": "false", 2024-09-18T10:14:41.971 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_use_stale_snap": "false", 2024-09-18T10:14:41.972 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_uuid": "00000000-0000-0000-0000-000000000000", 2024-09-18T10:14:41.972 INFO:teuthology.orchestra.run.smithi029.stdout: "osdc_blkin_trace_all": "false", 2024-09-18T10:14:41.972 INFO:teuthology.orchestra.run.smithi029.stdout: "paxos_kill_at": "0", 2024-09-18T10:14:41.972 INFO:teuthology.orchestra.run.smithi029.stdout: "paxos_max_join_drift": "10", 2024-09-18T10:14:41.972 INFO:teuthology.orchestra.run.smithi029.stdout: "paxos_min": "500", 2024-09-18T10:14:41.972 INFO:teuthology.orchestra.run.smithi029.stdout: "paxos_min_wait": "0.050000", 2024-09-18T10:14:41.972 INFO:teuthology.orchestra.run.smithi029.stdout: "paxos_propose_interval": "1.000000", 2024-09-18T10:14:41.972 INFO:teuthology.orchestra.run.smithi029.stdout: "paxos_service_trim_max": "500", 2024-09-18T10:14:41.972 INFO:teuthology.orchestra.run.smithi029.stdout: "paxos_service_trim_max_multiplier": "20", 2024-09-18T10:14:41.972 INFO:teuthology.orchestra.run.smithi029.stdout: "paxos_service_trim_min": "250", 2024-09-18T10:14:41.972 INFO:teuthology.orchestra.run.smithi029.stdout: "paxos_stash_full_interval": "25", 2024-09-18T10:14:41.972 INFO:teuthology.orchestra.run.smithi029.stdout: "paxos_trim_max": "500", 2024-09-18T10:14:41.972 INFO:teuthology.orchestra.run.smithi029.stdout: "paxos_trim_min": "250", 2024-09-18T10:14:41.973 INFO:teuthology.orchestra.run.smithi029.stdout: "perf": "true", 2024-09-18T10:14:41.973 INFO:teuthology.orchestra.run.smithi029.stdout: "pid_file": "/var/run/ceph/ceph-mon.a.pid", 2024-09-18T10:14:41.973 INFO:teuthology.orchestra.run.smithi029.stdout: "plugin_crypto_accelerator": "crypto_isal", 2024-09-18T10:14:41.973 INFO:teuthology.orchestra.run.smithi029.stdout: "plugin_dir": "/usr/lib/ceph", 2024-09-18T10:14:41.973 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addr": "-", 2024-09-18T10:14:41.973 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addrv": "", 2024-09-18T10:14:41.973 INFO:teuthology.orchestra.run.smithi029.stdout: "public_bind_addr": "-", 2024-09-18T10:14:41.973 INFO:teuthology.orchestra.run.smithi029.stdout: "public_network": "", 2024-09-18T10:14:41.973 INFO:teuthology.orchestra.run.smithi029.stdout: "public_network_interface": "", 2024-09-18T10:14:41.973 INFO:teuthology.orchestra.run.smithi029.stdout: "qat_compressor_enabled": "false", 2024-09-18T10:14:41.973 INFO:teuthology.orchestra.run.smithi029.stdout: "rados_mon_op_timeout": "0", 2024-09-18T10:14:41.973 INFO:teuthology.orchestra.run.smithi029.stdout: "rados_osd_op_timeout": "0", 2024-09-18T10:14:41.973 INFO:teuthology.orchestra.run.smithi029.stdout: "rados_tracing": "false", 2024-09-18T10:14:41.973 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_atime_update_interval": "60", 2024-09-18T10:14:41.974 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_auto_exclusive_lock_until_manual_request": "true", 2024-09-18T10:14:41.974 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_balance_parent_reads": "false", 2024-09-18T10:14:41.974 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_balance_snap_reads": "false", 2024-09-18T10:14:41.974 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_blkin_trace_all": "false", 2024-09-18T10:14:41.974 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_blocklist_expire_seconds": "0", 2024-09-18T10:14:41.974 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_blocklist_on_break_lock": "true", 2024-09-18T10:14:41.974 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_cache": "true", 2024-09-18T10:14:41.974 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_cache_block_writes_upfront": "false", 2024-09-18T10:14:41.974 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_cache_max_dirty": "25165824", 2024-09-18T10:14:41.974 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_cache_max_dirty_age": "1.000000", 2024-09-18T10:14:41.974 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_cache_max_dirty_object": "0", 2024-09-18T10:14:41.974 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_cache_policy": "writearound", 2024-09-18T10:14:41.974 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_cache_size": "33554432", 2024-09-18T10:14:41.975 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_cache_target_dirty": "16777216", 2024-09-18T10:14:41.975 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_cache_writethrough_until_flush": "true", 2024-09-18T10:14:41.975 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_clone_copy_on_read": "false", 2024-09-18T10:14:41.975 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_compression_hint": "none", 2024-09-18T10:14:41.975 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_concurrent_management_ops": "10", 2024-09-18T10:14:41.975 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_config_pool_override_update_timestamp": "0", 2024-09-18T10:14:41.975 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_default_clone_format": "auto", 2024-09-18T10:14:41.975 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_default_data_pool": "", 2024-09-18T10:14:41.975 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_default_features": "61", 2024-09-18T10:14:41.975 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_default_format": "2", 2024-09-18T10:14:41.975 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_default_map_options": "", 2024-09-18T10:14:41.976 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_default_order": "22", 2024-09-18T10:14:41.976 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_default_pool": "rbd", 2024-09-18T10:14:41.976 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_default_snapshot_quiesce_mode": "required", 2024-09-18T10:14:41.976 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_default_stripe_count": "0", 2024-09-18T10:14:41.976 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_default_stripe_unit": "0", 2024-09-18T10:14:41.976 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_disable_zero_copy_writes": "true", 2024-09-18T10:14:41.976 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_discard_granularity_bytes": "65536", 2024-09-18T10:14:41.976 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_discard_on_zeroed_write_same": "true", 2024-09-18T10:14:41.976 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_enable_alloc_hint": "true", 2024-09-18T10:14:41.976 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_invalidate_object_map_on_timeout": "true", 2024-09-18T10:14:41.976 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_io_scheduler": "simple", 2024-09-18T10:14:41.976 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_io_scheduler_simple_max_delay": "0", 2024-09-18T10:14:41.977 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_journal_commit_age": "5.000000", 2024-09-18T10:14:41.977 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_journal_max_concurrent_object_sets": "0", 2024-09-18T10:14:41.977 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_journal_max_payload_bytes": "16384", 2024-09-18T10:14:41.977 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_journal_object_flush_age": "0.000000", 2024-09-18T10:14:41.977 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_journal_object_flush_bytes": "1048576", 2024-09-18T10:14:41.977 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_journal_object_flush_interval": "0", 2024-09-18T10:14:41.977 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_journal_object_max_in_flight_appends": "0", 2024-09-18T10:14:41.977 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_journal_object_writethrough_until_flush": "true", 2024-09-18T10:14:41.977 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_journal_order": "24", 2024-09-18T10:14:41.977 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_journal_pool": "", 2024-09-18T10:14:41.977 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_journal_splay_width": "4", 2024-09-18T10:14:41.977 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_localize_parent_reads": "false", 2024-09-18T10:14:41.977 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_localize_snap_reads": "false", 2024-09-18T10:14:41.977 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_mirror_concurrent_image_deletions": "1", 2024-09-18T10:14:41.978 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_mirror_concurrent_image_syncs": "5", 2024-09-18T10:14:41.978 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_mirror_delete_retry_interval": "30.000000", 2024-09-18T10:14:41.978 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_mirror_image_perf_stats_prio": "5", 2024-09-18T10:14:41.978 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_mirror_image_policy_migration_throttle": "300", 2024-09-18T10:14:41.978 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_mirror_image_policy_rebalance_timeout": "0.000000", 2024-09-18T10:14:41.978 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_mirror_image_policy_type": "simple", 2024-09-18T10:14:41.978 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_mirror_image_policy_update_throttle_interval": "1.000000", 2024-09-18T10:14:41.978 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_mirror_image_state_check_interval": "30", 2024-09-18T10:14:41.978 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_mirror_journal_commit_age": "5.000000", 2024-09-18T10:14:41.978 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_mirror_journal_poll_age": "5.000000", 2024-09-18T10:14:41.978 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_mirror_leader_heartbeat_interval": "5", 2024-09-18T10:14:41.978 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_mirror_leader_max_acquire_attempts_before_break": "3", 2024-09-18T10:14:41.979 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_mirror_leader_max_missed_heartbeats": "2", 2024-09-18T10:14:41.979 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_mirror_memory_autotune": "true", 2024-09-18T10:14:41.979 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_mirror_memory_base": "805306368", 2024-09-18T10:14:41.979 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_mirror_memory_cache_autotune_interval": "30.000000", 2024-09-18T10:14:41.979 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_mirror_memory_cache_min": "134217728", 2024-09-18T10:14:41.979 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_mirror_memory_cache_resize_interval": "5.000000", 2024-09-18T10:14:41.979 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_mirror_memory_expected_fragmentation": "0.150000", 2024-09-18T10:14:41.979 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_mirror_memory_target": "4294967296", 2024-09-18T10:14:41.979 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_mirror_perf_stats_prio": "5", 2024-09-18T10:14:41.979 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_mirror_pool_replayers_refresh_interval": "30", 2024-09-18T10:14:41.979 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_mirror_sync_point_update_age": "30.000000", 2024-09-18T10:14:41.979 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_mirroring_delete_delay": "0", 2024-09-18T10:14:41.979 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_mirroring_max_mirroring_snapshots": "5", 2024-09-18T10:14:41.980 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_mirroring_replay_delay": "0", 2024-09-18T10:14:41.980 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_mirroring_resync_after_disconnect": "false", 2024-09-18T10:14:41.980 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_move_parent_to_trash_on_remove": "false", 2024-09-18T10:14:41.980 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_move_to_trash_on_remove": "false", 2024-09-18T10:14:41.980 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_move_to_trash_on_remove_expire_seconds": "0", 2024-09-18T10:14:41.980 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_mtime_update_interval": "60", 2024-09-18T10:14:41.980 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_non_blocking_aio": "true", 2024-09-18T10:14:41.980 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_op_thread_timeout": "60", 2024-09-18T10:14:41.981 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_op_threads": "1", 2024-09-18T10:14:41.981 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_parent_cache_enabled": "false", 2024-09-18T10:14:41.981 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_persistent_cache_mode": "disabled", 2024-09-18T10:14:41.981 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_persistent_cache_path": "/tmp", 2024-09-18T10:14:41.981 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_persistent_cache_size": "1073741824", 2024-09-18T10:14:41.981 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_plugins": "", 2024-09-18T10:14:41.981 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_qos_bps_burst": "0", 2024-09-18T10:14:41.981 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_qos_bps_burst_seconds": "1", 2024-09-18T10:14:41.981 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_qos_bps_limit": "0", 2024-09-18T10:14:41.981 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_qos_exclude_ops": "0", 2024-09-18T10:14:41.981 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_qos_iops_burst": "0", 2024-09-18T10:14:41.981 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_qos_iops_burst_seconds": "1", 2024-09-18T10:14:41.981 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_qos_iops_limit": "0", 2024-09-18T10:14:41.981 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_qos_read_bps_burst": "0", 2024-09-18T10:14:41.982 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_qos_read_bps_burst_seconds": "1", 2024-09-18T10:14:41.982 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_qos_read_bps_limit": "0", 2024-09-18T10:14:41.982 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_qos_read_iops_burst": "0", 2024-09-18T10:14:41.982 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_qos_read_iops_burst_seconds": "1", 2024-09-18T10:14:41.982 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_qos_read_iops_limit": "0", 2024-09-18T10:14:41.982 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_qos_schedule_tick_min": "50", 2024-09-18T10:14:41.982 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_qos_write_bps_burst": "0", 2024-09-18T10:14:41.982 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_qos_write_bps_burst_seconds": "1", 2024-09-18T10:14:41.982 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_qos_write_bps_limit": "0", 2024-09-18T10:14:41.982 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_qos_write_iops_burst": "0", 2024-09-18T10:14:41.982 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_qos_write_iops_burst_seconds": "1", 2024-09-18T10:14:41.982 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_qos_write_iops_limit": "0", 2024-09-18T10:14:41.982 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_quiesce_notification_attempts": "10", 2024-09-18T10:14:41.983 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_read_from_replica_policy": "default", 2024-09-18T10:14:41.983 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_readahead_disable_after_bytes": "52428800", 2024-09-18T10:14:41.983 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_readahead_max_bytes": "524288", 2024-09-18T10:14:41.983 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_readahead_trigger_requests": "10", 2024-09-18T10:14:41.983 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_request_timed_out_seconds": "30", 2024-09-18T10:14:41.983 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_skip_partial_discard": "true", 2024-09-18T10:14:41.983 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_sparse_read_threshold_bytes": "65536", 2024-09-18T10:14:41.983 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_tracing": "false", 2024-09-18T10:14:41.984 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_validate_names": "true", 2024-09-18T10:14:41.984 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd_validate_pool": "true", 2024-09-18T10:14:41.984 INFO:teuthology.orchestra.run.smithi029.stdout: "restapi_base_url": "", 2024-09-18T10:14:41.984 INFO:teuthology.orchestra.run.smithi029.stdout: "restapi_log_level": "", 2024-09-18T10:14:41.984 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_acl_grants_max_num": "100", 2024-09-18T10:14:41.984 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_admin_entry": "admin", 2024-09-18T10:14:41.984 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_backend_store": "rados", 2024-09-18T10:14:41.984 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_barbican_url": "", 2024-09-18T10:14:41.984 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_beast_enable_async": "true", 2024-09-18T10:14:41.984 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_bucket_default_quota_max_objects": "-1", 2024-09-18T10:14:41.984 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_bucket_default_quota_max_size": "-1", 2024-09-18T10:14:41.985 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_bucket_index_max_aio": "128", 2024-09-18T10:14:41.985 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_bucket_quota_cache_size": "10000", 2024-09-18T10:14:41.985 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_bucket_quota_ttl": "600", 2024-09-18T10:14:41.985 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_cache_enabled": "true", 2024-09-18T10:14:41.985 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_cache_expiry_interval": "900", 2024-09-18T10:14:41.985 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_cache_lru_size": "10000", 2024-09-18T10:14:41.985 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_content_length_compat": "false", 2024-09-18T10:14:41.985 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_copy_obj_progress": "true", 2024-09-18T10:14:41.985 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_copy_obj_progress_every_bytes": "1048576", 2024-09-18T10:14:41.985 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_cors_rules_max_num": "100", 2024-09-18T10:14:41.986 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_cross_domain_policy": "", 2024-09-18T10:14:41.986 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_crypt_default_encryption_key": "", 2024-09-18T10:14:41.986 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_crypt_kmip_addr": "", 2024-09-18T10:14:41.986 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_crypt_kmip_ca_path": "", 2024-09-18T10:14:41.986 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_crypt_kmip_client_cert": "", 2024-09-18T10:14:41.986 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_crypt_kmip_client_key": "", 2024-09-18T10:14:41.986 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_crypt_kmip_kms_key_template": "", 2024-09-18T10:14:41.986 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_crypt_kmip_password": "", 2024-09-18T10:14:41.986 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_crypt_kmip_s3_key_template": "$keyid", 2024-09-18T10:14:41.986 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_crypt_kmip_username": "", 2024-09-18T10:14:41.986 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_crypt_require_ssl": "true", 2024-09-18T10:14:41.986 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_crypt_s3_kms_backend": "barbican", 2024-09-18T10:14:41.986 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_crypt_s3_kms_encryption_keys": "", 2024-09-18T10:14:41.987 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_crypt_sse_s3_backend": "vault", 2024-09-18T10:14:41.987 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_crypt_sse_s3_key_template": "%bucket_id", 2024-09-18T10:14:41.987 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_crypt_sse_s3_vault_addr": "", 2024-09-18T10:14:41.987 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_crypt_sse_s3_vault_auth": "token", 2024-09-18T10:14:41.987 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_crypt_sse_s3_vault_namespace": "", 2024-09-18T10:14:41.987 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_crypt_sse_s3_vault_prefix": "", 2024-09-18T10:14:41.987 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_crypt_sse_s3_vault_secret_engine": "transit", 2024-09-18T10:14:41.987 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_crypt_sse_s3_vault_ssl_cacert": "", 2024-09-18T10:14:41.987 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_crypt_sse_s3_vault_ssl_clientcert": "", 2024-09-18T10:14:41.987 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_crypt_sse_s3_vault_ssl_clientkey": "", 2024-09-18T10:14:41.987 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_crypt_sse_s3_vault_token_file": "", 2024-09-18T10:14:41.987 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_crypt_sse_s3_vault_verify_ssl": "true", 2024-09-18T10:14:41.988 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_crypt_suppress_logs": "true", 2024-09-18T10:14:41.988 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_crypt_vault_addr": "", 2024-09-18T10:14:41.988 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_crypt_vault_auth": "token", 2024-09-18T10:14:41.988 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_crypt_vault_namespace": "", 2024-09-18T10:14:41.988 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_crypt_vault_prefix": "", 2024-09-18T10:14:41.988 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_crypt_vault_secret_engine": "transit", 2024-09-18T10:14:41.988 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_crypt_vault_ssl_cacert": "", 2024-09-18T10:14:41.988 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_crypt_vault_ssl_clientcert": "", 2024-09-18T10:14:41.988 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_crypt_vault_ssl_clientkey": "", 2024-09-18T10:14:41.988 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_crypt_vault_token_file": "", 2024-09-18T10:14:41.988 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_crypt_vault_verify_ssl": "true", 2024-09-18T10:14:41.988 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_curl_buffersize": "524288", 2024-09-18T10:14:41.989 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_curl_low_speed_limit": "1024", 2024-09-18T10:14:41.989 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_curl_low_speed_time": "300", 2024-09-18T10:14:41.989 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_curl_wait_timeout_ms": "1000", 2024-09-18T10:14:41.989 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_d3n_l1_datacache_persistent_path": "/tmp/rgw_datacache/", 2024-09-18T10:14:41.989 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_d3n_l1_datacache_size": "1073741824", 2024-09-18T10:14:41.989 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_d3n_l1_evict_cache_on_start": "true", 2024-09-18T10:14:41.989 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_d3n_l1_eviction_policy": "lru", 2024-09-18T10:14:41.989 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_d3n_l1_fadvise": "4", 2024-09-18T10:14:41.989 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_d3n_l1_local_datacache_enabled": "false", 2024-09-18T10:14:41.989 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_d3n_libaio_aio_num": "64", 2024-09-18T10:14:41.989 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_d3n_libaio_aio_threads": "20", 2024-09-18T10:14:41.990 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_data": "/var/lib/ceph/radosgw/ceph-a", 2024-09-18T10:14:41.990 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_data_log_changes_size": "1000", 2024-09-18T10:14:41.990 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_data_log_num_shards": "128", 2024-09-18T10:14:41.990 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_data_log_obj_prefix": "data_log", 2024-09-18T10:14:41.990 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_data_log_window": "30", 2024-09-18T10:14:41.990 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_data_notify_interval_msec": "200", 2024-09-18T10:14:41.990 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_debug_inject_latency_bi_unlink": "0", 2024-09-18T10:14:41.990 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_debug_inject_olh_cancel_modification_err": "false", 2024-09-18T10:14:41.990 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_debug_inject_set_olh_err": "0", 2024-09-18T10:14:41.990 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_default_data_log_backing": "fifo", 2024-09-18T10:14:41.990 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_default_realm_info_oid": "default.realm", 2024-09-18T10:14:41.990 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_default_region_info_oid": "default.region", 2024-09-18T10:14:41.990 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_default_zone_info_oid": "default.zone", 2024-09-18T10:14:41.991 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_default_zonegroup_info_oid": "default.zonegroup", 2024-09-18T10:14:41.991 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_defer_to_bucket_acls": "", 2024-09-18T10:14:41.991 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_delete_multi_obj_max_num": "1000", 2024-09-18T10:14:41.991 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_dmclock_admin_lim": "0.000000", 2024-09-18T10:14:41.991 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_dmclock_admin_res": "100.000000", 2024-09-18T10:14:41.991 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_dmclock_admin_wgt": "100.000000", 2024-09-18T10:14:41.991 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_dmclock_auth_lim": "0.000000", 2024-09-18T10:14:41.991 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_dmclock_auth_res": "200.000000", 2024-09-18T10:14:41.991 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_dmclock_auth_wgt": "100.000000", 2024-09-18T10:14:41.992 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_dmclock_data_lim": "0.000000", 2024-09-18T10:14:41.992 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_dmclock_data_res": "500.000000", 2024-09-18T10:14:41.992 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_dmclock_data_wgt": "500.000000", 2024-09-18T10:14:41.992 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_dmclock_metadata_lim": "0.000000", 2024-09-18T10:14:41.992 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_dmclock_metadata_res": "500.000000", 2024-09-18T10:14:41.992 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_dmclock_metadata_wgt": "500.000000", 2024-09-18T10:14:41.992 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_dns_name": "", 2024-09-18T10:14:41.992 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_dns_s3website_name": "", 2024-09-18T10:14:41.992 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_dynamic_resharding": "true", 2024-09-18T10:14:41.992 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_enable_apis": "s3, s3website, swift, swift_auth, admin, sts, iam, notifications", 2024-09-18T10:14:41.992 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_enable_gc_threads": "true", 2024-09-18T10:14:41.992 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_enable_lc_threads": "true", 2024-09-18T10:14:41.993 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_enable_ops_log": "false", 2024-09-18T10:14:41.993 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_enable_quota_threads": "true", 2024-09-18T10:14:41.993 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_enable_static_website": "false", 2024-09-18T10:14:41.993 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_enable_usage_log": "false", 2024-09-18T10:14:41.993 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_enforce_swift_acls": "true", 2024-09-18T10:14:41.993 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_exit_timeout_secs": "120", 2024-09-18T10:14:41.993 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_expose_bucket": "false", 2024-09-18T10:14:41.993 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_extended_http_attrs": "", 2024-09-18T10:14:41.994 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_frontend_defaults": "beast ssl_certificate=config://rgw/cert/$realm/$zone.crt ssl_private_key=config://rgw/cert/$realm/$zone.key", 2024-09-18T10:14:41.994 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_frontends": "beast port=7480", 2024-09-18T10:14:41.994 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_gc_max_concurrent_io": "10", 2024-09-18T10:14:41.994 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_gc_max_deferred": "50", 2024-09-18T10:14:41.994 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_gc_max_deferred_entries_size": "3072", 2024-09-18T10:14:41.994 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_gc_max_objs": "32", 2024-09-18T10:14:41.994 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_gc_max_queue_size": "134213632", 2024-09-18T10:14:41.994 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_gc_max_trim_chunk": "16", 2024-09-18T10:14:41.994 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_gc_obj_min_wait": "7200", 2024-09-18T10:14:41.994 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_gc_processor_max_time": "3600", 2024-09-18T10:14:41.995 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_gc_processor_period": "3600", 2024-09-18T10:14:41.995 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_get_obj_max_req_size": "4194304", 2024-09-18T10:14:41.995 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_get_obj_window_size": "16777216", 2024-09-18T10:14:41.995 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_healthcheck_disabling_path": "", 2024-09-18T10:14:41.995 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_ignore_get_invalid_range": "false", 2024-09-18T10:14:41.995 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_init_timeout": "300", 2024-09-18T10:14:41.995 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_inject_notify_timeout_probability": "0.000000", 2024-09-18T10:14:41.995 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_kafka_connection_idle": "300", 2024-09-18T10:14:41.995 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_kafka_message_timeout": "5000", 2024-09-18T10:14:41.995 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_keystone_accepted_admin_roles": "", 2024-09-18T10:14:41.995 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_keystone_accepted_roles": "Member, admin", 2024-09-18T10:14:41.996 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_keystone_admin_domain": "", 2024-09-18T10:14:41.996 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_keystone_admin_password": "", 2024-09-18T10:14:41.996 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_keystone_admin_password_path": "", 2024-09-18T10:14:41.996 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_keystone_admin_project": "", 2024-09-18T10:14:41.996 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_keystone_admin_tenant": "", 2024-09-18T10:14:41.996 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_keystone_admin_token": "", 2024-09-18T10:14:41.996 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_keystone_admin_token_path": "", 2024-09-18T10:14:41.996 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_keystone_admin_user": "", 2024-09-18T10:14:41.996 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_keystone_api_version": "2", 2024-09-18T10:14:41.996 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_keystone_barbican_domain": "", 2024-09-18T10:14:41.996 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_keystone_barbican_password": "", 2024-09-18T10:14:41.996 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_keystone_barbican_project": "", 2024-09-18T10:14:41.997 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_keystone_barbican_tenant": "", 2024-09-18T10:14:41.997 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_keystone_barbican_user": "", 2024-09-18T10:14:41.997 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_keystone_expired_token_cache_expiration": "3600", 2024-09-18T10:14:41.997 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_keystone_implicit_tenants": "false", 2024-09-18T10:14:41.997 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_keystone_service_token_accepted_roles": "admin", 2024-09-18T10:14:41.997 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_keystone_service_token_enabled": "false", 2024-09-18T10:14:41.997 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_keystone_token_cache_size": "10000", 2024-09-18T10:14:41.997 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_keystone_url": "", 2024-09-18T10:14:41.997 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_keystone_verify_ssl": "true", 2024-09-18T10:14:41.997 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_lc_debug_interval": "-1", 2024-09-18T10:14:41.997 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_lc_lock_max_time": "90", 2024-09-18T10:14:41.998 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_lc_max_objs": "32", 2024-09-18T10:14:41.998 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_lc_max_rules": "1000", 2024-09-18T10:14:41.998 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_lc_max_worker": "3", 2024-09-18T10:14:41.998 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_lc_max_wp_worker": "3", 2024-09-18T10:14:41.998 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_lc_thread_delay": "0", 2024-09-18T10:14:41.998 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_ldap_binddn": "uid=admin,cn=users,dc=example,dc=com", 2024-09-18T10:14:41.998 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_ldap_dnattr": "uid", 2024-09-18T10:14:41.998 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_ldap_searchdn": "cn=users,cn=accounts,dc=example,dc=com", 2024-09-18T10:14:41.998 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_ldap_searchfilter": "", 2024-09-18T10:14:41.998 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_ldap_secret": "/etc/openldap/secret", 2024-09-18T10:14:41.999 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_ldap_uri": "", 2024-09-18T10:14:41.999 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_lifecycle_work_time": "00:00-06:00", 2024-09-18T10:14:41.999 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_list_bucket_min_readahead": "1000", 2024-09-18T10:14:41.999 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_list_buckets_max_chunk": "1000", 2024-09-18T10:14:41.999 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_log_http_headers": "", 2024-09-18T10:14:41.999 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_log_nonexistent_bucket": "false", 2024-09-18T10:14:41.999 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_log_object_name": "%Y-%m-%d-%H-%i-%n", 2024-09-18T10:14:41.999 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_log_object_name_utc": "false", 2024-09-18T10:14:41.999 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_luarocks_location": "/tmp/luarocks", 2024-09-18T10:14:41.999 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_max_attr_name_len": "0", 2024-09-18T10:14:41.999 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_max_attr_size": "0", 2024-09-18T10:14:41.999 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_max_attrs_num_in_req": "0", 2024-09-18T10:14:42.000 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_max_chunk_size": "4194304", 2024-09-18T10:14:42.000 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_max_concurrent_requests": "1024", 2024-09-18T10:14:42.000 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_max_dynamic_shards": "1999", 2024-09-18T10:14:42.000 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_max_listing_results": "1000", 2024-09-18T10:14:42.000 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_max_notify_retries": "10", 2024-09-18T10:14:42.000 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_max_objs_per_shard": "100000", 2024-09-18T10:14:42.000 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_max_put_param_size": "1048576", 2024-09-18T10:14:42.000 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_max_put_size": "5368709120", 2024-09-18T10:14:42.000 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_max_slo_entries": "1000", 2024-09-18T10:14:42.000 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_md_log_max_shards": "64", 2024-09-18T10:14:42.000 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_md_notify_interval_msec": "200", 2024-09-18T10:14:42.000 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_mime_types_file": "/etc/mime.types", 2024-09-18T10:14:42.000 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_mp_lock_max_time": "600", 2024-09-18T10:14:42.001 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_multi_obj_del_max_aio": "16", 2024-09-18T10:14:42.001 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_multipart_min_part_size": "5242880", 2024-09-18T10:14:42.001 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_multipart_part_upload_limit": "10000", 2024-09-18T10:14:42.001 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_nfs_fhcache_partitions": "3", 2024-09-18T10:14:42.001 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_nfs_fhcache_size": "2017", 2024-09-18T10:14:42.001 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_nfs_lru_lane_hiwat": "911", 2024-09-18T10:14:42.001 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_nfs_lru_lanes": "5", 2024-09-18T10:14:42.001 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_nfs_max_gc": "300", 2024-09-18T10:14:42.001 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_nfs_namespace_expire_secs": "300", 2024-09-18T10:14:42.001 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_nfs_run_gc_threads": "false", 2024-09-18T10:14:42.001 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_nfs_run_lc_threads": "false", 2024-09-18T10:14:42.001 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_nfs_run_quota_threads": "false", 2024-09-18T10:14:42.002 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_nfs_run_sync_thread": "false", 2024-09-18T10:14:42.002 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_nfs_s3_fast_attrs": "false", 2024-09-18T10:14:42.002 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_nfs_write_completion_interval_s": "10", 2024-09-18T10:14:42.002 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_num_async_rados_threads": "32", 2024-09-18T10:14:42.002 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_num_control_oids": "8", 2024-09-18T10:14:42.002 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_numa_node": "-1", 2024-09-18T10:14:42.002 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_obj_stripe_size": "4194304", 2024-09-18T10:14:42.002 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_obj_tombstone_cache_size": "1000", 2024-09-18T10:14:42.002 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_objexp_chunk_size": "100", 2024-09-18T10:14:42.002 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_objexp_gc_interval": "600", 2024-09-18T10:14:42.002 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_objexp_hints_num_shards": "127", 2024-09-18T10:14:42.003 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_olh_pending_timeout_sec": "3600", 2024-09-18T10:14:42.003 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_op_thread_suicide_timeout": "0", 2024-09-18T10:14:42.003 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_op_thread_timeout": "600", 2024-09-18T10:14:42.003 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_op_tracing": "false", 2024-09-18T10:14:42.003 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_opa_token": "", 2024-09-18T10:14:42.003 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_opa_url": "", 2024-09-18T10:14:42.003 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_opa_verify_ssl": "true", 2024-09-18T10:14:42.003 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_ops_log_data_backlog": "5242880", 2024-09-18T10:14:42.003 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_ops_log_file_path": "", 2024-09-18T10:14:42.003 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_ops_log_rados": "true", 2024-09-18T10:14:42.003 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_ops_log_socket_path": "", 2024-09-18T10:14:42.003 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_override_bucket_index_max_shards": "0", 2024-09-18T10:14:42.004 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_period_latest_epoch_info_oid": ".latest_epoch", 2024-09-18T10:14:42.004 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_period_push_interval": "2.000000", 2024-09-18T10:14:42.004 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_period_push_interval_max": "30.000000", 2024-09-18T10:14:42.004 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_period_root_pool": ".rgw.root", 2024-09-18T10:14:42.004 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_print_continue": "true", 2024-09-18T10:14:42.004 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_print_prohibited_content_length": "false", 2024-09-18T10:14:42.004 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_put_obj_max_window_size": "67108864", 2024-09-18T10:14:42.004 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_put_obj_min_window_size": "16777216", 2024-09-18T10:14:42.004 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_rados_pool_autoscale_bias": "4.000000", 2024-09-18T10:14:42.004 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_rados_pool_recovery_priority": "5", 2024-09-18T10:14:42.004 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_rados_tracing": "false", 2024-09-18T10:14:42.004 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_realm": "", 2024-09-18T10:14:42.005 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_realm_id": "", 2024-09-18T10:14:42.005 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_realm_root_pool": ".rgw.root", 2024-09-18T10:14:42.005 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_region": "", 2024-09-18T10:14:42.005 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_region_root_pool": ".rgw.root", 2024-09-18T10:14:42.005 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_relaxed_region_enforcement": "false", 2024-09-18T10:14:42.005 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_relaxed_s3_bucket_names": "false", 2024-09-18T10:14:42.005 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_remote_addr_param": "REMOTE_ADDR", 2024-09-18T10:14:42.005 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_request_uri": "", 2024-09-18T10:14:42.005 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_reshard_batch_size": "64", 2024-09-18T10:14:42.005 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_reshard_bucket_lock_duration": "360", 2024-09-18T10:14:42.005 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_reshard_max_aio": "128", 2024-09-18T10:14:42.005 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_reshard_num_logs": "16", 2024-09-18T10:14:42.005 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_reshard_thread_interval": "600", 2024-09-18T10:14:42.006 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_resolve_cname": "false", 2024-09-18T10:14:42.006 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_rest_getusage_op_compat": "false", 2024-09-18T10:14:42.006 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_run_sync_thread": "true", 2024-09-18T10:14:42.006 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_s3_auth_disable_signature_url": "false", 2024-09-18T10:14:42.006 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_s3_auth_order": "sts, external, local", 2024-09-18T10:14:42.006 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_s3_auth_use_keystone": "false", 2024-09-18T10:14:42.006 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_s3_auth_use_ldap": "false", 2024-09-18T10:14:42.006 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_s3_auth_use_rados": "true", 2024-09-18T10:14:42.006 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_s3_auth_use_sts": "false", 2024-09-18T10:14:42.006 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_s3_client_max_sig_ver": "-1", 2024-09-18T10:14:42.006 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_s3_success_create_obj_status": "0", 2024-09-18T10:14:42.006 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_safe_max_objects_per_shard": "102400", 2024-09-18T10:14:42.007 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_scheduler_type": "throttler", 2024-09-18T10:14:42.007 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_script_uri": "", 2024-09-18T10:14:42.007 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_service_provider_name": "", 2024-09-18T10:14:42.007 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_shard_warning_threshold": "90.000000", 2024-09-18T10:14:42.007 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_sts_client_id": "", 2024-09-18T10:14:42.007 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_sts_client_secret": "", 2024-09-18T10:14:42.007 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_sts_entry": "sts", 2024-09-18T10:14:42.008 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_sts_key": "sts", 2024-09-18T10:14:42.008 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_sts_max_session_duration": "43200", 2024-09-18T10:14:42.008 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_sts_min_session_duration": "900", 2024-09-18T10:14:42.008 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_sts_token_introspection_url": "", 2024-09-18T10:14:42.008 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_swift_account_in_url": "false", 2024-09-18T10:14:42.008 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_swift_auth_entry": "auth", 2024-09-18T10:14:42.008 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_swift_auth_url": "", 2024-09-18T10:14:42.008 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_swift_custom_header": "", 2024-09-18T10:14:42.008 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_swift_enforce_content_length": "false", 2024-09-18T10:14:42.008 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_swift_need_stats": "true", 2024-09-18T10:14:42.008 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_swift_tenant_name": "", 2024-09-18T10:14:42.008 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_swift_token_expiration": "86400", 2024-09-18T10:14:42.009 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_swift_url": "", 2024-09-18T10:14:42.009 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_swift_url_prefix": "swift", 2024-09-18T10:14:42.009 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_swift_versioning_enabled": "false", 2024-09-18T10:14:42.009 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_sync_data_inject_err_probability": "0.000000", 2024-09-18T10:14:42.009 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_sync_lease_period": "120", 2024-09-18T10:14:42.009 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_sync_log_trim_concurrent_buckets": "4", 2024-09-18T10:14:42.009 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_sync_log_trim_interval": "1200", 2024-09-18T10:14:42.009 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_sync_log_trim_max_buckets": "16", 2024-09-18T10:14:42.009 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_sync_log_trim_min_cold_buckets": "4", 2024-09-18T10:14:42.010 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_sync_meta_inject_err_probability": "0.000000", 2024-09-18T10:14:42.010 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_sync_obj_etag_verify": "false", 2024-09-18T10:14:42.010 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_sync_trace_history_size": "4096", 2024-09-18T10:14:42.010 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_sync_trace_per_node_log_size": "32", 2024-09-18T10:14:42.010 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_sync_trace_servicemap_update_interval": "10", 2024-09-18T10:14:42.010 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_thread_pool_size": "512", 2024-09-18T10:14:42.010 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_torrent_comment": "", 2024-09-18T10:14:42.010 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_torrent_createby": "", 2024-09-18T10:14:42.010 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_torrent_encoding": "", 2024-09-18T10:14:42.010 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_torrent_flag": "false", 2024-09-18T10:14:42.011 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_torrent_origin": "", 2024-09-18T10:14:42.011 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_torrent_sha_unit": "524288", 2024-09-18T10:14:42.011 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_torrent_tracker": "", 2024-09-18T10:14:42.011 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_trust_forwarded_https": "false", 2024-09-18T10:14:42.011 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_usage_log_flush_threshold": "1024", 2024-09-18T10:14:42.011 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_usage_log_tick_interval": "30", 2024-09-18T10:14:42.011 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_usage_max_shards": "32", 2024-09-18T10:14:42.011 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_usage_max_user_shards": "1", 2024-09-18T10:14:42.011 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_use_opa_authz": "false", 2024-09-18T10:14:42.011 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_user_default_quota_max_objects": "-1", 2024-09-18T10:14:42.011 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_user_default_quota_max_size": "-1", 2024-09-18T10:14:42.012 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_user_max_buckets": "1000", 2024-09-18T10:14:42.012 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_user_quota_bucket_sync_interval": "180", 2024-09-18T10:14:42.012 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_user_quota_sync_idle_users": "false", 2024-09-18T10:14:42.012 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_user_quota_sync_interval": "86400", 2024-09-18T10:14:42.012 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_user_quota_sync_wait_time": "86400", 2024-09-18T10:14:42.012 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_user_unique_email": "true", 2024-09-18T10:14:42.012 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_verify_ssl": "true", 2024-09-18T10:14:42.013 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_website_routing_rules_max_num": "50", 2024-09-18T10:14:42.013 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_zone": "", 2024-09-18T10:14:42.013 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_zone_id": "", 2024-09-18T10:14:42.013 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_zone_root_pool": ".rgw.root", 2024-09-18T10:14:42.013 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_zonegroup": "", 2024-09-18T10:14:42.013 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_zonegroup_id": "", 2024-09-18T10:14:42.013 INFO:teuthology.orchestra.run.smithi029.stdout: "rgw_zonegroup_root_pool": ".rgw.root", 2024-09-18T10:14:42.013 INFO:teuthology.orchestra.run.smithi029.stdout: "rocksdb_block_size": "4096", 2024-09-18T10:14:42.013 INFO:teuthology.orchestra.run.smithi029.stdout: "rocksdb_bloom_bits_per_key": "20", 2024-09-18T10:14:42.013 INFO:teuthology.orchestra.run.smithi029.stdout: "rocksdb_cache_index_and_filter_blocks": "true", 2024-09-18T10:14:42.013 INFO:teuthology.orchestra.run.smithi029.stdout: "rocksdb_cache_index_and_filter_blocks_with_high_priority": "false", 2024-09-18T10:14:42.014 INFO:teuthology.orchestra.run.smithi029.stdout: "rocksdb_cache_row_ratio": "0.000000", 2024-09-18T10:14:42.014 INFO:teuthology.orchestra.run.smithi029.stdout: "rocksdb_cache_shard_bits": "4", 2024-09-18T10:14:42.014 INFO:teuthology.orchestra.run.smithi029.stdout: "rocksdb_cache_size": "536870912", 2024-09-18T10:14:42.014 INFO:teuthology.orchestra.run.smithi029.stdout: "rocksdb_cache_type": "binned_lru", 2024-09-18T10:14:42.014 INFO:teuthology.orchestra.run.smithi029.stdout: "rocksdb_cf_compact_on_deletion": "true", 2024-09-18T10:14:42.014 INFO:teuthology.orchestra.run.smithi029.stdout: "rocksdb_cf_compact_on_deletion_sliding_window": "32768", 2024-09-18T10:14:42.014 INFO:teuthology.orchestra.run.smithi029.stdout: "rocksdb_cf_compact_on_deletion_trigger": "16384", 2024-09-18T10:14:42.014 INFO:teuthology.orchestra.run.smithi029.stdout: "rocksdb_collect_compaction_stats": "false", 2024-09-18T10:14:42.014 INFO:teuthology.orchestra.run.smithi029.stdout: "rocksdb_collect_extended_stats": "false", 2024-09-18T10:14:42.014 INFO:teuthology.orchestra.run.smithi029.stdout: "rocksdb_collect_memory_stats": "false", 2024-09-18T10:14:42.014 INFO:teuthology.orchestra.run.smithi029.stdout: "rocksdb_delete_range_threshold": "1048576", 2024-09-18T10:14:42.015 INFO:teuthology.orchestra.run.smithi029.stdout: "rocksdb_index_type": "binary_search", 2024-09-18T10:14:42.015 INFO:teuthology.orchestra.run.smithi029.stdout: "rocksdb_log_to_ceph_log": "true", 2024-09-18T10:14:42.015 INFO:teuthology.orchestra.run.smithi029.stdout: "rocksdb_metadata_block_size": "4096", 2024-09-18T10:14:42.015 INFO:teuthology.orchestra.run.smithi029.stdout: "rocksdb_partition_filters": "false", 2024-09-18T10:14:42.015 INFO:teuthology.orchestra.run.smithi029.stdout: "rocksdb_perf": "false", 2024-09-18T10:14:42.015 INFO:teuthology.orchestra.run.smithi029.stdout: "rocksdb_pin_l0_filter_and_index_blocks_in_cache": "false", 2024-09-18T10:14:42.015 INFO:teuthology.orchestra.run.smithi029.stdout: "rotating_keys_bootstrap_timeout": "30", 2024-09-18T10:14:42.015 INFO:teuthology.orchestra.run.smithi029.stdout: "rotating_keys_renewal_timeout": "10", 2024-09-18T10:14:42.015 INFO:teuthology.orchestra.run.smithi029.stdout: "run_dir": "/var/run/ceph", 2024-09-18T10:14:42.015 INFO:teuthology.orchestra.run.smithi029.stdout: "seastore_block_create": "true", 2024-09-18T10:14:42.015 INFO:teuthology.orchestra.run.smithi029.stdout: "seastore_cache_lru_size": "67108864", 2024-09-18T10:14:42.015 INFO:teuthology.orchestra.run.smithi029.stdout: "seastore_default_max_object_size": "16777216", 2024-09-18T10:14:42.015 INFO:teuthology.orchestra.run.smithi029.stdout: "seastore_default_object_metadata_reservation": "16777216", 2024-09-18T10:14:42.016 INFO:teuthology.orchestra.run.smithi029.stdout: "seastore_device_size": "107374182400", 2024-09-18T10:14:42.016 INFO:teuthology.orchestra.run.smithi029.stdout: "seastore_init_rewrite_segments_num_per_device": "6", 2024-09-18T10:14:42.016 INFO:teuthology.orchestra.run.smithi029.stdout: "seastore_journal_batch_capacity": "16", 2024-09-18T10:14:42.016 INFO:teuthology.orchestra.run.smithi029.stdout: "seastore_journal_batch_flush_size": "16777216", 2024-09-18T10:14:42.016 INFO:teuthology.orchestra.run.smithi029.stdout: "seastore_journal_batch_preferred_fullness": "0.950000", 2024-09-18T10:14:42.016 INFO:teuthology.orchestra.run.smithi029.stdout: "seastore_journal_iodepth_limit": "5", 2024-09-18T10:14:42.016 INFO:teuthology.orchestra.run.smithi029.stdout: "seastore_segment_size": "67108864", 2024-09-18T10:14:42.016 INFO:teuthology.orchestra.run.smithi029.stdout: "setgroup": "", 2024-09-18T10:14:42.016 INFO:teuthology.orchestra.run.smithi029.stdout: "setuser": "", 2024-09-18T10:14:42.016 INFO:teuthology.orchestra.run.smithi029.stdout: "setuser_match_path": "", 2024-09-18T10:14:42.017 INFO:teuthology.orchestra.run.smithi029.stdout: "target_max_misplaced_ratio": "0.050000", 2024-09-18T10:14:42.017 INFO:teuthology.orchestra.run.smithi029.stdout: "thp": "false", 2024-09-18T10:14:42.017 INFO:teuthology.orchestra.run.smithi029.stdout: "threadpool_default_timeout": "60", 2024-09-18T10:14:42.017 INFO:teuthology.orchestra.run.smithi029.stdout: "threadpool_empty_queue_max_wait": "2", 2024-09-18T10:14:42.017 INFO:teuthology.orchestra.run.smithi029.stdout: "throttler_perf_counter": "true" 2024-09-18T10:14:42.017 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-18T10:14:42.020 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=0 2024-09-18T10:14:42.150 INFO:teuthology.orchestra.run.smithi029.stdout:mon.a: {} 2024-09-18T10:14:42.165 INFO:teuthology.orchestra.run.smithi029.stdout:mon.b: {} 2024-09-18T10:14:42.180 INFO:teuthology.orchestra.run.smithi029.stdout:mon.c: {} 2024-09-18T10:14:42.182 INFO:teuthology.orchestra.run.smithi029.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-18T10:14:42.182 INFO:teuthology.orchestra.run.smithi029.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-18T10:14:42.183 INFO:teuthology.orchestra.run.smithi029.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-18T10:14:42.194 DEBUG:teuthology.orchestra.run.smithi154:> type ceph-objectstore-tool 2024-09-18T10:14:42.200 INFO:teuthology.orchestra.run.smithi154.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-18T10:14:42.200 DEBUG:teuthology.orchestra.run.smithi029:> type ceph-objectstore-tool 2024-09-18T10:14:42.206 INFO:teuthology.orchestra.run.smithi029.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-18T10:14:42.206 DEBUG:teuthology.orchestra.run.smithi136:> type ceph-objectstore-tool 2024-09-18T10:14:42.211 INFO:teuthology.orchestra.run.smithi136.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-18T10:14:42.212 DEBUG:teuthology.orchestra.run.smithi189:> type ceph-objectstore-tool 2024-09-18T10:14:42.217 INFO:teuthology.orchestra.run.smithi189.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-18T10:14:42.218 INFO:teuthology.run_tasks:Running task rados... 2024-09-18T10:14:42.229 INFO:tasks.rados:Beginning rados... 2024-09-18T10:14:42.229 INFO:tasks.rados:config is {'clients': ['client.0'], 'ec_pool': True, 'erasure_code_profile': {'c': 2, 'crush-failure-domain': 'osd', 'k': 4, 'm': 3, 'name': 'shecprofile', 'plugin': 'shec'}, 'objects': 50, 'op_weights': {'append': 100, 'copy_from': 50, 'delete': 50, 'read': 100, 'rmattr': 25, 'rollback': 50, 'setattr': 25, 'snap_create': 50, 'snap_remove': 50, 'write': 0}, 'ops': 400, 'write_append_excl': False} 2024-09-18T10:14:42.229 INFO:tasks.rados:overrides is {'admin_socket': {'branch': 'wip-bharath11-testing-2024-09-18-1115-quincy'}, 'ceph': {'conf': {'global': {'mon election default strategy': 1, 'osd debug inject dispatch delay duration': 0.1, 'osd debug inject dispatch delay probability': 0.1}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug reject backfill probability': 0.3, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'but it is still running', 'objects unfound and apparently lost', 'slow request', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'mons down', 'mon down', 'out of quorum'], 'sha1': '7061ea6ca50908d104e9f3979dc615e118d9e153'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '7061ea6ca50908d104e9f3979dc615e118d9e153'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-bharath11-testing-2024-09-18-1115-quincy', 'sha1': '7061ea6ca50908d104e9f3979dc615e118d9e153'}} 2024-09-18T10:14:42.229 INFO:tasks.rados:config is {'clients': ['client.0'], 'ec_pool': True, 'erasure_code_profile': {'c': 2, 'crush-failure-domain': 'osd', 'k': 4, 'm': 3, 'name': 'shecprofile', 'plugin': 'shec'}, 'objects': 50, 'op_weights': {'append': 100, 'copy_from': 50, 'delete': 50, 'read': 100, 'rmattr': 25, 'rollback': 50, 'setattr': 25, 'snap_create': 50, 'snap_remove': 50, 'write': 0}, 'ops': 400, 'write_append_excl': False} 2024-09-18T10:14:42.229 DEBUG:teuthology.run_tasks:Unwinding manager rados 2024-09-18T10:14:42.242 INFO:tasks.rados:joining rados 2024-09-18T10:14:42.242 INFO:tasks.thrashosds.thrasher:starting do_thrash 2024-09-18T10:14:42.242 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15] out_osds: [] dead_osds: [] live_osds: [0, 4, 8, 12, 1, 5, 9, 13, 2, 6, 10, 14, 3, 7, 11, 15] 2024-09-18T10:14:42.243 INFO:tasks.thrashosds.thrasher:choose_action: min_in 8 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T10:14:42.243 INFO:tasks.thrashosds.thrasher:inject_pause on osd.11 2024-09-18T10:14:42.243 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-18T10:14:42.243 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-18T10:14:42.243 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok config set filestore_inject_stall 3 2024-09-18T10:14:42.244 INFO:tasks.thrashosds.thrasher:starting do_sighup with a delay of 0.1 2024-09-18T10:14:42.244 INFO:tasks.thrashosds.thrasher:starting do_optrack_toggle with a delay of 2.0 2024-09-18T10:14:42.244 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T10:14:42.245 INFO:tasks.thrashosds.thrasher:starting do_dump_ops 2024-09-18T10:14:42.245 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T10:14:42.245 INFO:tasks.thrashosds.thrasher:starting do_noscrub_toggle with a delay of 2.0 2024-09-18T10:14:42.245 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-18T10:14:42.246 INFO:tasks.rados:clients are ['client.0'] 2024-09-18T10:14:42.246 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set shecprofile c=2 crush-failure-domain=osd k=4 m=3 name=shecprofile plugin=shec 2024-09-18T10:14:42.253 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-18T10:14:42.250+0000 7f1981873700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22132) UID: 0 2024-09-18T10:14:42.346 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-18T10:14:42.346+0000 7f1981873700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22132) UID: 0 2024-09-18T10:14:42.381 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-18T10:14:42.382 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:14:42.382 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:14:42.391 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T10:14:42.413 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-18T10:14:42.416 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-18T10:14:42.416 INFO:teuthology.orchestra.run.smithi189.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-18T10:14:42.416 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-18T10:14:42.440 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-18T10:14:42.447 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-18T10:14:42.444+0000 7f14dcc5b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22398) UID: 0 2024-09-18T10:14:42.464 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-18T10:14:42.485 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-18T10:14:42.510 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-18T10:14:42.525 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:14:42.525 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:14:42.526 INFO:teuthology.orchestra.run.smithi029.stdout:osd.6: {} 2024-09-18T10:14:42.538 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T10:14:42.548 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-18T10:14:42.542+0000 7f1981873700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22132) UID: 0 2024-09-18T10:14:42.549 INFO:teuthology.orchestra.run.smithi029.stdout:osd.7: {} 2024-09-18T10:14:42.571 INFO:teuthology.orchestra.run.smithi029.stdout:osd.8: {} 2024-09-18T10:14:42.591 INFO:teuthology.orchestra.run.smithi029.stdout:osd.9: {} 2024-09-18T10:14:42.612 INFO:teuthology.orchestra.run.smithi029.stdout:osd.10: {} 2024-09-18T10:14:42.633 INFO:teuthology.orchestra.run.smithi029.stdout:osd.11: {} 2024-09-18T10:14:42.639 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:14:42.639 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:14:42.648 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T10:14:42.649 INFO:tasks.ceph.osd.6.smithi154.stderr:2024-09-18T10:14:42.644+0000 7f0abc9e1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22089) UID: 0 2024-09-18T10:14:42.656 INFO:teuthology.orchestra.run.smithi029.stdout:osd.12: {} 2024-09-18T10:14:42.679 INFO:teuthology.orchestra.run.smithi029.stdout:osd.13: {} 2024-09-18T10:14:42.739 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:14:42.739 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:14:42.749 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T10:14:42.750 INFO:tasks.ceph.osd.11.smithi189.stderr:2024-09-18T10:14:42.747+0000 7f718b60a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 22400) UID: 0 2024-09-18T10:14:42.761 INFO:teuthology.orchestra.run.smithi029.stdout:osd.14: {} 2024-09-18T10:14:42.849 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:14:42.849 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:14:42.851 INFO:tasks.ceph.osd.8.smithi029.stderr:2024-09-18T10:14:42.847+0000 7f2555c27700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 22192) UID: 0 2024-09-18T10:14:42.860 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T10:14:42.873 INFO:teuthology.orchestra.run.smithi029.stdout:osd.15: {} 2024-09-18T10:14:42.876 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T10:14:42.876 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T10:14:42.876 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T10:14:42.876 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T10:14:42.876 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T10:14:42.876 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T10:14:42.876 INFO:teuthology.orchestra.run.smithi029.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T10:14:42.876 INFO:teuthology.orchestra.run.smithi029.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T10:14:42.877 INFO:teuthology.orchestra.run.smithi029.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-18T10:14:42.877 INFO:teuthology.orchestra.run.smithi029.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-18T10:14:42.877 INFO:teuthology.orchestra.run.smithi029.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-18T10:14:42.877 INFO:teuthology.orchestra.run.smithi029.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-18T10:14:42.877 INFO:teuthology.orchestra.run.smithi029.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-18T10:14:42.877 INFO:teuthology.orchestra.run.smithi029.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-18T10:14:42.877 INFO:teuthology.orchestra.run.smithi029.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-18T10:14:42.877 INFO:teuthology.orchestra.run.smithi029.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-18T10:14:42.952 INFO:tasks.ceph.osd.6.smithi154.stderr:2024-09-18T10:14:42.948+0000 7f0abc9e1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22089) UID: 0 2024-09-18T10:14:42.972 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:14:42.972 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:14:42.983 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-09-18T10:14:43.053 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-09-18T10:14:43.052+0000 7fdfa8698700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22087) UID: 0 2024-09-18T10:14:43.083 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:14:43.083 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:14:43.092 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-09-18T10:14:43.154 INFO:tasks.ceph.osd.12.smithi029.stderr:2024-09-18T10:14:43.151+0000 7f8e4e627700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 22190) UID: 0 2024-09-18T10:14:43.197 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:14:43.198 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:14:43.207 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-09-18T10:14:43.255 INFO:tasks.ceph.osd.15.smithi189.stderr:2024-09-18T10:14:43.251+0000 7fd0e725e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 22401) UID: 0 2024-09-18T10:14:43.306 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:14:43.306 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:14:43.314 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_ops_in_flight 2024-09-18T10:14:43.356 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-18T10:14:43.350+0000 7f1981873700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22132) UID: 0 2024-09-18T10:14:43.417 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:14:43.417 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:14:43.427 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_blocked_ops 2024-09-18T10:14:43.457 INFO:tasks.ceph.osd.10.smithi154.stderr:2024-09-18T10:14:43.452+0000 7fb73221a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 22091) UID: 0 2024-09-18T10:14:43.532 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:14:43.533 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:14:43.544 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_historic_ops 2024-09-18T10:14:43.545 INFO:teuthology.orchestra.run.smithi029.stderr:noscrub is set 2024-09-18T10:14:43.545 INFO:teuthology.orchestra.run.smithi029.stderr:load: shec 2024-09-18T10:14:43.557 INFO:tasks.rados:starting run 0 out of 1 2024-09-18T10:14:43.557 INFO:tasks.ceph.ceph_manager.ceph:creating pool_name unique_pool_0 2024-09-18T10:14:43.557 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create unique_pool_0 16 16 erasure shecprofile 2024-09-18T10:14:43.560 INFO:tasks.ceph.osd.3.smithi189.stderr:2024-09-18T10:14:43.555+0000 7f2440fc7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22396) UID: 0 2024-09-18T10:14:43.643 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:14:43.643 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:14:43.653 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T10:14:43.658 INFO:tasks.ceph.osd.8.smithi029.stderr:2024-09-18T10:14:43.655+0000 7f2555c27700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 22192) UID: 0 2024-09-18T10:14:43.759 INFO:tasks.ceph.osd.15.smithi189.stderr:2024-09-18T10:14:43.755+0000 7fd0e725e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 22401) UID: 0 2024-09-18T10:14:43.766 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-18T10:14:43.767 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:14:43.776 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T10:14:43.860 INFO:tasks.ceph.osd.10.smithi154.stderr:2024-09-18T10:14:43.856+0000 7fb73221a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 22091) UID: 0 2024-09-18T10:14:43.878 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-18T10:14:43.878 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:14:43.887 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T10:14:43.961 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-18T10:14:43.958+0000 7f578c52f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22134) UID: 0 2024-09-18T10:14:43.986 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-18T10:14:43.986 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:14:43.995 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T10:14:44.063 INFO:tasks.ceph.osd.8.smithi029.stderr:2024-09-18T10:14:44.059+0000 7f2555c27700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 22192) UID: 0 2024-09-18T10:14:44.093 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-18T10:14:44.094 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:14:44.103 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T10:14:44.163 INFO:tasks.ceph.osd.11.smithi189.stderr:2024-09-18T10:14:44.159+0000 7f718b60a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 22400) UID: 0 2024-09-18T10:14:44.198 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-18T10:14:44.198 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:14:44.207 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T10:14:44.263 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-09-18T10:14:44.260+0000 7fdfa8698700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22087) UID: 0 2024-09-18T10:14:44.304 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-18T10:14:44.304 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:14:44.314 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-09-18T10:14:44.365 INFO:tasks.ceph.osd.10.smithi154.stderr:2024-09-18T10:14:44.360+0000 7fb73221a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 22091) UID: 0 2024-09-18T10:14:44.411 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-18T10:14:44.412 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:14:44.421 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-09-18T10:14:44.466 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-18T10:14:44.462+0000 7f578c52f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22134) UID: 0 2024-09-18T10:14:44.517 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-18T10:14:44.517 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:14:44.526 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-09-18T10:14:44.566 INFO:tasks.ceph.osd.6.smithi154.stderr:2024-09-18T10:14:44.560+0000 7f0abc9e1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22089) UID: 0 2024-09-18T10:14:44.622 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-18T10:14:44.622 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:14:44.631 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_ops_in_flight 2024-09-18T10:14:44.667 INFO:tasks.ceph.osd.14.smithi154.stderr:2024-09-18T10:14:44.664+0000 7f75a19e7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 22093) UID: 0 2024-09-18T10:14:44.724 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-18T10:14:44.724 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:14:44.733 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_blocked_ops 2024-09-18T10:14:44.768 INFO:tasks.ceph.osd.6.smithi154.stderr:2024-09-18T10:14:44.764+0000 7f0abc9e1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22089) UID: 0 2024-09-18T10:14:44.829 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-18T10:14:44.830 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:14:44.839 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_historic_ops 2024-09-18T10:14:44.869 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-09-18T10:14:44.864+0000 7fdfa8698700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22087) UID: 0 2024-09-18T10:14:44.887 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T10:14:44.939 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-18T10:14:44.940 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:14:44.949 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T10:14:44.995 INFO:tasks.ceph.osd.14.smithi154.stderr:2024-09-18T10:14:44.992+0000 7f75a19e7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 22093) UID: 0 2024-09-18T10:14:45.038 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-18T10:14:45.059 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-09-18T10:14:45.059 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:14:45.063 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-18T10:14:45.067 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T10:14:45.070 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-18T10:14:45.066+0000 7f14c6a1a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 22138) UID: 0 2024-09-18T10:14:45.088 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-18T10:14:45.113 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-18T10:14:45.134 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-18T10:14:45.158 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-18T10:14:45.164 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-09-18T10:14:45.164 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-09-18T10:14:45.164 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-09-18T10:14:45.165 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-09-18T10:14:45.165 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-09-18T10:14:45.171 INFO:tasks.ceph.osd.12.smithi029.stderr:2024-09-18T10:14:45.167+0000 7f8e4e627700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 22190) UID: 0 2024-09-18T10:14:45.173 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T10:14:45.179 INFO:teuthology.orchestra.run.smithi029.stdout:osd.6: {} 2024-09-18T10:14:45.200 INFO:teuthology.orchestra.run.smithi029.stdout:osd.7: {} 2024-09-18T10:14:45.221 INFO:teuthology.orchestra.run.smithi029.stdout:osd.8: {} 2024-09-18T10:14:45.245 INFO:teuthology.orchestra.run.smithi029.stdout:osd.9: {} 2024-09-18T10:14:45.270 INFO:teuthology.orchestra.run.smithi029.stdout:osd.10: {} 2024-09-18T10:14:45.272 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-09-18T10:14:45.272 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-09-18T10:14:45.272 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-09-18T10:14:45.272 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [] 2024-09-18T10:14:45.272 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-09-18T10:14:45.273 INFO:tasks.ceph.osd.15.smithi189.stderr:2024-09-18T10:14:45.271+0000 7fd0e725e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 22401) UID: 0 2024-09-18T10:14:45.281 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T10:14:45.290 INFO:teuthology.orchestra.run.smithi029.stdout:osd.11: {} 2024-09-18T10:14:45.313 INFO:teuthology.orchestra.run.smithi029.stdout:osd.12: {} 2024-09-18T10:14:45.336 INFO:teuthology.orchestra.run.smithi029.stdout:osd.13: {} 2024-09-18T10:14:45.359 INFO:teuthology.orchestra.run.smithi029.stdout:osd.14: {} 2024-09-18T10:14:45.373 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-18T10:14:45.370+0000 7f1981873700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22132) UID: 0 2024-09-18T10:14:45.377 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-09-18T10:14:45.377 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-09-18T10:14:45.377 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-09-18T10:14:45.377 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-09-18T10:14:45.383 INFO:teuthology.orchestra.run.smithi029.stdout:osd.15: {} 2024-09-18T10:14:45.386 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T10:14:45.386 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T10:14:45.387 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T10:14:45.387 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T10:14:45.387 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T10:14:45.387 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T10:14:45.387 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T10:14:45.387 INFO:teuthology.orchestra.run.smithi029.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T10:14:45.387 INFO:teuthology.orchestra.run.smithi029.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T10:14:45.387 INFO:teuthology.orchestra.run.smithi029.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-18T10:14:45.387 INFO:teuthology.orchestra.run.smithi029.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-18T10:14:45.387 INFO:teuthology.orchestra.run.smithi029.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-18T10:14:45.387 INFO:teuthology.orchestra.run.smithi029.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-18T10:14:45.387 INFO:teuthology.orchestra.run.smithi029.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-18T10:14:45.387 INFO:teuthology.orchestra.run.smithi029.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-18T10:14:45.388 INFO:teuthology.orchestra.run.smithi029.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-18T10:14:45.388 INFO:teuthology.orchestra.run.smithi029.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-18T10:14:45.474 INFO:tasks.ceph.osd.3.smithi189.stderr:2024-09-18T10:14:45.471+0000 7f2440fc7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22396) UID: 0 2024-09-18T10:14:45.486 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-09-18T10:14:45.486 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-09-18T10:14:45.486 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-09-18T10:14:45.486 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-09-18T10:14:45.486 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-09-18T10:14:45.495 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T10:14:45.557 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-18T10:14:45.576 INFO:tasks.ceph.osd.15.smithi189.stderr:2024-09-18T10:14:45.571+0000 7fd0e725e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 22401) UID: 0 2024-09-18T10:14:45.589 INFO:teuthology.orchestra.run.smithi029.stderr:pool 'unique_pool_0' created 2024-09-18T10:14:45.607 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-09-18T10:14:45.607 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-09-18T10:14:45.607 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-09-18T10:14:45.607 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-09-18T10:14:45.607 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.607 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:45 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-18T10:14:45.607 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:35.093050+0000", 2024-09-18T10:14:45.607 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 10.506747998, 2024-09-18T10:14:45.607 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.50958515100000001, 2024-09-18T10:14:45.608 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:45.608 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:14:45.608 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:14:45.608 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:14:45.608 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:14:45.608 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 45 2024-09-18T10:14:45.608 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.609 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:45.609 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.609 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:45.609 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.093050+0000", 2024-09-18T10:14:45.609 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:45.609 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.609 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.609 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:45.609 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.093050+0000", 2024-09-18T10:14:45.609 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:45.609 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.610 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.610 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:45.610 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.093049+0000", 2024-09-18T10:14:45.610 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:14:45.610 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.610 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.610 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:45.610 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.093064+0000", 2024-09-18T10:14:45.610 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.5492999999999999e-05 2024-09-18T10:14:45.610 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.610 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.610 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:45.611 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.093076+0000", 2024-09-18T10:14:45.611 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1112e-05 2024-09-18T10:14:45.611 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.611 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.611 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:45.611 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.093081+0000", 2024-09-18T10:14:45.611 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5929999999999996e-06 2024-09-18T10:14:45.611 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.611 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.611 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:45.612 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.198774+0000", 2024-09-18T10:14:45.612 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.105692676 2024-09-18T10:14:45.612 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.612 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.612 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:14:45.612 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.198906+0000", 2024-09-18T10:14:45.612 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00013173300000000001 2024-09-18T10:14:45.612 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.612 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.612 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:14:45.613 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.199104+0000", 2024-09-18T10:14:45.613 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00019817500000000001 2024-09-18T10:14:45.613 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.613 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.613 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:14:45.613 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.203616+0000", 2024-09-18T10:14:45.613 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0045118529999999997 2024-09-18T10:14:45.613 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.613 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.613 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:14:45.613 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.602486+0000", 2024-09-18T10:14:45.614 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.39887087500000001 2024-09-18T10:14:45.614 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.614 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.614 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:14:45.614 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.602575+0000", 2024-09-18T10:14:45.614 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.8290999999999999e-05 2024-09-18T10:14:45.614 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.614 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.614 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:45.614 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.602635+0000", 2024-09-18T10:14:45.615 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.0170000000000002e-05 2024-09-18T10:14:45.615 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:45.615 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:45.615 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:45.615 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.615 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.615 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-18T10:14:45.615 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:35.093089+0000", 2024-09-18T10:14:45.615 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 10.506709184, 2024-09-18T10:14:45.615 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.50974395900000002, 2024-09-18T10:14:45.615 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:45.616 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:14:45.616 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:14:45.616 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:14:45.616 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:14:45.616 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 46 2024-09-18T10:14:45.616 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.616 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:45.616 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.616 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:45.616 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.093089+0000", 2024-09-18T10:14:45.616 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:45.617 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.617 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.617 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:45.617 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.093089+0000", 2024-09-18T10:14:45.617 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:45.617 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.617 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.617 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:45.617 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.093088+0000", 2024-09-18T10:14:45.617 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999995 2024-09-18T10:14:45.617 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.618 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.618 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:45.618 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.093093+0000", 2024-09-18T10:14:45.618 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.0440000000000003e-06 2024-09-18T10:14:45.618 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.618 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.618 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:45.618 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.093101+0000", 2024-09-18T10:14:45.618 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.5490000000000006e-06 2024-09-18T10:14:45.618 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.618 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.619 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:45.619 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.093109+0000", 2024-09-18T10:14:45.619 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.1330000000000001e-06 2024-09-18T10:14:45.619 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.619 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.619 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:45.619 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.199767+0000", 2024-09-18T10:14:45.619 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.106658379 2024-09-18T10:14:45.619 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.619 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.619 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:14:45.620 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.199922+0000", 2024-09-18T10:14:45.620 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000154797 2024-09-18T10:14:45.620 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.620 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.620 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:14:45.620 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.200138+0000", 2024-09-18T10:14:45.620 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00021600100000000001 2024-09-18T10:14:45.620 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.620 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.620 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:14:45.620 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.203626+0000", 2024-09-18T10:14:45.620 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0034878470000000001 2024-09-18T10:14:45.621 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.621 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.621 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:14:45.621 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.602720+0000", 2024-09-18T10:14:45.621 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.399094646 2024-09-18T10:14:45.621 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.621 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.621 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:14:45.621 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.602787+0000", 2024-09-18T10:14:45.621 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.6878999999999994e-05 2024-09-18T10:14:45.621 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.622 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.622 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:45.622 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.602833+0000", 2024-09-18T10:14:45.622 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.5393999999999999e-05 2024-09-18T10:14:45.622 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:45.622 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:45.622 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:45.622 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.622 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.622 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-18T10:14:45.623 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:35.093116+0000", 2024-09-18T10:14:45.623 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 10.506681815, 2024-09-18T10:14:45.623 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.51000002600000005, 2024-09-18T10:14:45.623 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:45.623 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:14:45.623 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:14:45.623 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:14:45.623 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:14:45.624 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 47 2024-09-18T10:14:45.624 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.624 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:45.624 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.624 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:45.624 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.093116+0000", 2024-09-18T10:14:45.624 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:45.624 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.624 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.624 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:45.624 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.093116+0000", 2024-09-18T10:14:45.625 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:45.625 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.625 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.625 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:45.625 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.093115+0000", 2024-09-18T10:14:45.625 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:14:45.625 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.625 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.625 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:45.625 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.093198+0000", 2024-09-18T10:14:45.625 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.2395e-05 2024-09-18T10:14:45.626 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.626 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.626 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:45.626 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.093208+0000", 2024-09-18T10:14:45.626 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.9960000000000005e-06 2024-09-18T10:14:45.626 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.626 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.626 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:45.626 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.093214+0000", 2024-09-18T10:14:45.626 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.1929999999999998e-06 2024-09-18T10:14:45.626 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.627 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.627 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:45.627 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.200843+0000", 2024-09-18T10:14:45.627 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.107629271 2024-09-18T10:14:45.627 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.627 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.627 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:14:45.627 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.200996+0000", 2024-09-18T10:14:45.627 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00015333 2024-09-18T10:14:45.627 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.628 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.628 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:14:45.628 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.201196+0000", 2024-09-18T10:14:45.628 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00019938900000000001 2024-09-18T10:14:45.628 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.628 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.628 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:14:45.628 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.204200+0000", 2024-09-18T10:14:45.628 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030046399999999998 2024-09-18T10:14:45.628 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.629 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.629 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:14:45.629 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.602968+0000", 2024-09-18T10:14:45.629 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.39876762100000002 2024-09-18T10:14:45.629 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.629 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.629 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:14:45.629 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.603056+0000", 2024-09-18T10:14:45.629 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.8158000000000005e-05 2024-09-18T10:14:45.629 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.629 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.630 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:45.630 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.603116+0000", 2024-09-18T10:14:45.630 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.9753000000000002e-05 2024-09-18T10:14:45.630 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:45.630 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:45.630 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:45.630 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.630 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.630 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-18T10:14:45.630 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:35.093224+0000", 2024-09-18T10:14:45.630 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 10.506573482, 2024-09-18T10:14:45.631 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.51036105200000004, 2024-09-18T10:14:45.631 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:45.631 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:14:45.631 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:14:45.631 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:14:45.631 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:14:45.631 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 48 2024-09-18T10:14:45.631 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.631 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:45.631 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.631 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:45.632 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.093224+0000", 2024-09-18T10:14:45.632 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:45.632 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.632 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.632 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:45.632 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.093224+0000", 2024-09-18T10:14:45.632 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:45.632 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.632 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.632 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:45.632 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.093223+0000", 2024-09-18T10:14:45.633 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:14:45.633 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.633 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.633 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:45.633 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.093229+0000", 2024-09-18T10:14:45.633 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.3229999999999997e-06 2024-09-18T10:14:45.633 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.633 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.633 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:45.633 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.093239+0000", 2024-09-18T10:14:45.633 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0063e-05 2024-09-18T10:14:45.634 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.634 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.634 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:45.634 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.193381+0000", 2024-09-18T10:14:45.634 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100142232 2024-09-18T10:14:45.634 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.634 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.634 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:45.634 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.293719+0000", 2024-09-18T10:14:45.634 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100337421 2024-09-18T10:14:45.634 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.635 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.635 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:14:45.635 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.293866+0000", 2024-09-18T10:14:45.635 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00014781199999999999 2024-09-18T10:14:45.635 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.635 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.635 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:14:45.635 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.294072+0000", 2024-09-18T10:14:45.635 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0002052 2024-09-18T10:14:45.635 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.636 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.636 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:14:45.636 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.300276+0000", 2024-09-18T10:14:45.636 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0062046940000000002 2024-09-18T10:14:45.636 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.636 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.636 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:14:45.636 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.603433+0000", 2024-09-18T10:14:45.636 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.30315636200000001 2024-09-18T10:14:45.636 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.636 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.637 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:14:45.637 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.603525+0000", 2024-09-18T10:14:45.637 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.2394999999999999e-05 2024-09-18T10:14:45.637 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.637 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.637 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:45.637 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.603585+0000", 2024-09-18T10:14:45.637 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.0343000000000001e-05 2024-09-18T10:14:45.637 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:45.637 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:45.638 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:45.638 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.638 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.638 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-18T10:14:45.638 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:35.293815+0000", 2024-09-18T10:14:45.638 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 10.305982965, 2024-09-18T10:14:45.638 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.513288525, 2024-09-18T10:14:45.638 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:45.638 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:14:45.638 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:14:45.638 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:14:45.639 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:14:45.639 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 52 2024-09-18T10:14:45.639 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.639 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:45.639 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.639 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:45.639 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.293815+0000", 2024-09-18T10:14:45.639 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:45.639 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.639 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.639 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:45.640 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.293815+0000", 2024-09-18T10:14:45.640 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:45.640 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.640 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.640 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:45.640 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.293814+0000", 2024-09-18T10:14:45.640 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:14:45.640 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.640 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.640 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:45.641 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.293822+0000", 2024-09-18T10:14:45.641 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.8089999999999993e-06 2024-09-18T10:14:45.641 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.641 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.641 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:45.641 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.293833+0000", 2024-09-18T10:14:45.641 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1389000000000001e-05 2024-09-18T10:14:45.641 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.641 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.641 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:45.641 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.393937+0000", 2024-09-18T10:14:45.641 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10010415 2024-09-18T10:14:45.642 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.642 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.642 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:45.642 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.704491+0000", 2024-09-18T10:14:45.642 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.310554202 2024-09-18T10:14:45.642 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.642 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.642 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:14:45.642 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.704647+0000", 2024-09-18T10:14:45.642 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00015524399999999999 2024-09-18T10:14:45.642 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.643 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.643 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:14:45.643 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.704859+0000", 2024-09-18T10:14:45.643 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000212664 2024-09-18T10:14:45.643 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.643 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.643 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:14:45.643 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.705935+0000", 2024-09-18T10:14:45.643 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001075505 2024-09-18T10:14:45.643 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.643 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.644 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:14:45.644 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.806952+0000", 2024-09-18T10:14:45.644 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10101700299999999 2024-09-18T10:14:45.644 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.644 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.644 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:14:45.644 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.807041+0000", 2024-09-18T10:14:45.644 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.8811000000000001e-05 2024-09-18T10:14:45.644 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.644 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.645 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:45.645 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.807103+0000", 2024-09-18T10:14:45.645 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.2596999999999997e-05 2024-09-18T10:14:45.645 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:45.645 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:45.645 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:45.645 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.645 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.645 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:70 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T10:14:45.645 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:36.786853+0000", 2024-09-18T10:14:45.645 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 8.8129450049999996, 2024-09-18T10:14:45.646 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.51178262299999999, 2024-09-18T10:14:45.646 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:45.646 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:14:45.646 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:14:45.646 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:14:45.646 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:14:45.646 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 70 2024-09-18T10:14:45.646 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.646 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:45.646 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.646 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:45.647 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.786853+0000", 2024-09-18T10:14:45.647 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:45.647 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.647 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.647 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:45.647 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.786853+0000", 2024-09-18T10:14:45.647 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:45.647 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.647 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.647 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:45.647 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.786851+0000", 2024-09-18T10:14:45.648 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:14:45.648 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.648 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.648 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:45.648 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.786861+0000", 2024-09-18T10:14:45.648 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.8700000000000004e-06 2024-09-18T10:14:45.648 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.648 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.648 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:45.648 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.786877+0000", 2024-09-18T10:14:45.648 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.5569e-05 2024-09-18T10:14:45.649 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.649 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.649 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:45.649 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.786887+0000", 2024-09-18T10:14:45.649 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0526000000000001e-05 2024-09-18T10:14:45.649 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.649 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.649 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:45.649 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.292790+0000", 2024-09-18T10:14:45.649 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.50590225600000005 2024-09-18T10:14:45.649 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.650 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.650 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:14:45.650 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.292929+0000", 2024-09-18T10:14:45.650 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00013913200000000001 2024-09-18T10:14:45.650 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.650 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.650 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:14:45.650 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.293137+0000", 2024-09-18T10:14:45.650 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00020826199999999999 2024-09-18T10:14:45.650 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.650 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.650 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:14:45.651 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.297461+0000", 2024-09-18T10:14:45.651 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0043237299999999996 2024-09-18T10:14:45.651 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.651 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.651 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:14:45.651 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.298490+0000", 2024-09-18T10:14:45.651 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0010289769999999999 2024-09-18T10:14:45.651 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.651 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.651 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:14:45.651 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.298572+0000", 2024-09-18T10:14:45.651 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.2151999999999995e-05 2024-09-18T10:14:45.652 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.652 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.652 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:45.652 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.298635+0000", 2024-09-18T10:14:45.652 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.3468999999999998e-05 2024-09-18T10:14:45.652 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:45.652 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:45.652 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:45.652 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.652 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.652 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:71 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T10:14:45.653 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:36.786902+0000", 2024-09-18T10:14:45.653 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 8.8128962010000009, 2024-09-18T10:14:45.653 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.51197160200000003, 2024-09-18T10:14:45.653 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:45.653 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:14:45.653 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:14:45.653 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:14:45.653 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:14:45.653 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 71 2024-09-18T10:14:45.653 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.653 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:45.653 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.654 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:45.654 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.786902+0000", 2024-09-18T10:14:45.654 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:45.654 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.654 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.654 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:45.654 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.786902+0000", 2024-09-18T10:14:45.654 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:45.654 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.654 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.654 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:45.654 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.786900+0000", 2024-09-18T10:14:45.655 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:14:45.655 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.655 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.655 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:45.655 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.786909+0000", 2024-09-18T10:14:45.655 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.9430000000000006e-06 2024-09-18T10:14:45.655 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.655 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.655 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:45.655 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.786921+0000", 2024-09-18T10:14:45.655 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1789e-05 2024-09-18T10:14:45.655 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.656 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.656 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:45.656 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.786927+0000", 2024-09-18T10:14:45.656 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5770000000000001e-06 2024-09-18T10:14:45.656 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.656 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.656 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:45.656 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.293768+0000", 2024-09-18T10:14:45.656 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.50684175499999995 2024-09-18T10:14:45.656 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.656 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.657 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:14:45.657 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.293894+0000", 2024-09-18T10:14:45.657 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00012588899999999999 2024-09-18T10:14:45.657 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.657 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.657 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:14:45.657 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.294094+0000", 2024-09-18T10:14:45.657 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000199896 2024-09-18T10:14:45.657 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.657 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.657 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:14:45.657 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.298660+0000", 2024-09-18T10:14:45.658 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.004566082 2024-09-18T10:14:45.658 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.658 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.658 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:14:45.658 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.298739+0000", 2024-09-18T10:14:45.658 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.8269000000000006e-05 2024-09-18T10:14:45.658 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.658 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.658 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:14:45.658 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.298814+0000", 2024-09-18T10:14:45.658 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.5582999999999998e-05 2024-09-18T10:14:45.658 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.658 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.659 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:45.659 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.298873+0000", 2024-09-18T10:14:45.659 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.8989000000000001e-05 2024-09-18T10:14:45.659 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:45.659 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:45.659 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:45.659 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.659 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.659 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T10:14:45.659 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:36.786936+0000", 2024-09-18T10:14:45.659 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 8.8128620029999993, 2024-09-18T10:14:45.659 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.71130157999999999, 2024-09-18T10:14:45.660 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:45.660 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:14:45.660 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:14:45.660 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:14:45.660 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:14:45.660 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 72 2024-09-18T10:14:45.660 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.660 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:45.660 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.660 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:45.660 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.786936+0000", 2024-09-18T10:14:45.660 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:45.661 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.661 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.661 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:45.661 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.786936+0000", 2024-09-18T10:14:45.661 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:45.661 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.661 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.661 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:45.661 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.786935+0000", 2024-09-18T10:14:45.661 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999995 2024-09-18T10:14:45.661 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.662 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.662 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:45.662 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787018+0000", 2024-09-18T10:14:45.662 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.2511999999999999e-05 2024-09-18T10:14:45.662 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.662 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.662 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:45.662 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787028+0000", 2024-09-18T10:14:45.662 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0709e-05 2024-09-18T10:14:45.662 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.663 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.663 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:45.663 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787034+0000", 2024-09-18T10:14:45.663 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.7769999999999999e-06 2024-09-18T10:14:45.663 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.663 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.663 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:45.663 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.295095+0000", 2024-09-18T10:14:45.663 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.50806109099999996 2024-09-18T10:14:45.663 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.663 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.664 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:14:45.664 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.295221+0000", 2024-09-18T10:14:45.664 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00012563599999999999 2024-09-18T10:14:45.664 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.664 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.664 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:14:45.664 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.295453+0000", 2024-09-18T10:14:45.664 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000232387 2024-09-18T10:14:45.664 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.664 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.665 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:14:45.665 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.301373+0000", 2024-09-18T10:14:45.665 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0059202279999999996 2024-09-18T10:14:45.665 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.665 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.665 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:14:45.665 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.498044+0000", 2024-09-18T10:14:45.665 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.19667104699999999 2024-09-18T10:14:45.665 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.665 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.665 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:14:45.665 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.498133+0000", 2024-09-18T10:14:45.666 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.9044999999999999e-05 2024-09-18T10:14:45.666 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.666 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.666 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:45.666 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.498237+0000", 2024-09-18T10:14:45.666 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00010385100000000001 2024-09-18T10:14:45.666 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:45.666 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:45.666 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:45.666 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.667 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.667 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T10:14:45.667 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:36.787042+0000", 2024-09-18T10:14:45.667 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 8.8127552960000006, 2024-09-18T10:14:45.667 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.71152325100000002, 2024-09-18T10:14:45.667 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:45.667 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:14:45.667 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:14:45.667 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:14:45.667 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:14:45.668 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 73 2024-09-18T10:14:45.668 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.668 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:45.668 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.668 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:45.668 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787042+0000", 2024-09-18T10:14:45.668 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:45.668 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.668 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.668 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:45.669 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787042+0000", 2024-09-18T10:14:45.669 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:45.669 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.669 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.669 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:45.669 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787042+0000", 2024-09-18T10:14:45.669 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:14:45.669 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.669 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.669 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:45.670 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787048+0000", 2024-09-18T10:14:45.670 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.8900000000000001e-06 2024-09-18T10:14:45.670 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.670 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.670 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:45.670 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787057+0000", 2024-09-18T10:14:45.670 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.4430000000000003e-06 2024-09-18T10:14:45.670 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.670 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.670 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:45.670 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787063+0000", 2024-09-18T10:14:45.670 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.6459999999999998e-06 2024-09-18T10:14:45.671 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.671 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.671 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:45.671 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.297548+0000", 2024-09-18T10:14:45.671 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.51048589499999997 2024-09-18T10:14:45.671 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.671 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.671 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:14:45.671 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.297668+0000", 2024-09-18T10:14:45.671 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00011917 2024-09-18T10:14:45.671 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.671 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.671 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:14:45.672 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.297891+0000", 2024-09-18T10:14:45.672 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00022301999999999999 2024-09-18T10:14:45.672 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.672 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.672 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:14:45.672 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.301383+0000", 2024-09-18T10:14:45.672 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.003492762 2024-09-18T10:14:45.672 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.672 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.672 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:14:45.672 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.498374+0000", 2024-09-18T10:14:45.672 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.19699053899999999 2024-09-18T10:14:45.673 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.673 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.673 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:14:45.673 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.498506+0000", 2024-09-18T10:14:45.673 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00013246899999999999 2024-09-18T10:14:45.673 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.673 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.673 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:45.673 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.498566+0000", 2024-09-18T10:14:45.673 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.9274000000000002e-05 2024-09-18T10:14:45.673 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:45.673 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:45.674 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:45.674 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.674 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.674 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T10:14:45.674 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:36.787128+0000", 2024-09-18T10:14:45.674 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 8.8126699879999997, 2024-09-18T10:14:45.674 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.71169541800000002, 2024-09-18T10:14:45.674 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:45.674 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:14:45.674 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:14:45.674 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:14:45.674 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:14:45.675 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 74 2024-09-18T10:14:45.675 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.675 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:45.675 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.675 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:45.675 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787128+0000", 2024-09-18T10:14:45.675 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:45.675 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.675 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.675 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:45.675 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787128+0000", 2024-09-18T10:14:45.675 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:45.676 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.676 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.676 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:45.676 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787127+0000", 2024-09-18T10:14:45.676 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:14:45.676 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.676 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.676 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:45.676 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787133+0000", 2024-09-18T10:14:45.676 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.6930000000000004e-06 2024-09-18T10:14:45.676 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.677 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.677 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:45.677 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787142+0000", 2024-09-18T10:14:45.677 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.1959999999999997e-06 2024-09-18T10:14:45.677 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.677 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.677 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:45.677 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787150+0000", 2024-09-18T10:14:45.677 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.6399999999999997e-06 2024-09-18T10:14:45.677 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.677 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.678 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:45.678 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.298925+0000", 2024-09-18T10:14:45.678 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.51177523599999997 2024-09-18T10:14:45.678 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.678 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.678 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:14:45.678 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.299064+0000", 2024-09-18T10:14:45.678 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000139622 2024-09-18T10:14:45.678 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.678 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.679 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:14:45.679 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.299287+0000", 2024-09-18T10:14:45.679 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00022309700000000001 2024-09-18T10:14:45.679 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.679 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.679 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:14:45.679 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.301389+0000", 2024-09-18T10:14:45.679 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0021018759999999999 2024-09-18T10:14:45.679 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.679 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.679 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:14:45.680 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.498648+0000", 2024-09-18T10:14:45.680 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.19725873999999999 2024-09-18T10:14:45.680 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.680 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.680 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:14:45.680 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.498731+0000", 2024-09-18T10:14:45.680 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.2429000000000006e-05 2024-09-18T10:14:45.680 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.680 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.680 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:45.681 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.498823+0000", 2024-09-18T10:14:45.681 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.2655e-05 2024-09-18T10:14:45.681 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:45.681 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:45.681 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:45.681 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.681 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.681 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T10:14:45.681 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:36.787157+0000", 2024-09-18T10:14:45.681 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 8.8126407629999992, 2024-09-18T10:14:45.681 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.81204889999999996, 2024-09-18T10:14:45.682 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:45.682 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:14:45.682 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:14:45.682 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:14:45.682 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:14:45.682 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 75 2024-09-18T10:14:45.682 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.682 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:45.682 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.682 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:45.682 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787157+0000", 2024-09-18T10:14:45.683 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:45.683 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.683 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.683 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:45.683 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787157+0000", 2024-09-18T10:14:45.683 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:45.683 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.683 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.683 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:45.683 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787156+0000", 2024-09-18T10:14:45.683 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:14:45.684 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.684 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.684 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:45.684 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787236+0000", 2024-09-18T10:14:45.684 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.0175999999999999e-05 2024-09-18T10:14:45.684 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.684 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.684 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:45.684 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787247+0000", 2024-09-18T10:14:45.684 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0448999999999999e-05 2024-09-18T10:14:45.684 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.685 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.685 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:45.685 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787253+0000", 2024-09-18T10:14:45.685 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.2469999999999997e-06 2024-09-18T10:14:45.685 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.685 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.685 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:45.685 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.299974+0000", 2024-09-18T10:14:45.685 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.51272114499999999 2024-09-18T10:14:45.685 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.685 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.686 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:14:45.686 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.300115+0000", 2024-09-18T10:14:45.686 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000141452 2024-09-18T10:14:45.686 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.686 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.686 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:14:45.686 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.300349+0000", 2024-09-18T10:14:45.686 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00023363999999999999 2024-09-18T10:14:45.686 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.686 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.686 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:14:45.686 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.301947+0000", 2024-09-18T10:14:45.687 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0015976510000000001 2024-09-18T10:14:45.687 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.687 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.687 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:14:45.687 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.599027+0000", 2024-09-18T10:14:45.687 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.29708051600000002 2024-09-18T10:14:45.687 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.687 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.687 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:14:45.687 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.599106+0000", 2024-09-18T10:14:45.688 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.9033e-05 2024-09-18T10:14:45.688 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.688 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.688 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:45.688 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.599206+0000", 2024-09-18T10:14:45.688 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.9561000000000004e-05 2024-09-18T10:14:45.688 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:45.688 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:45.688 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:45.688 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.688 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.689 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T10:14:45.689 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:36.787261+0000", 2024-09-18T10:14:45.689 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 8.8125371119999993, 2024-09-18T10:14:45.689 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.81223649499999995, 2024-09-18T10:14:45.689 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:45.689 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:14:45.689 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:14:45.689 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:14:45.690 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:14:45.690 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 76 2024-09-18T10:14:45.690 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.690 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:45.690 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.690 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:45.690 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787261+0000", 2024-09-18T10:14:45.690 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:45.690 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.690 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.690 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:45.690 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787261+0000", 2024-09-18T10:14:45.691 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:45.691 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.691 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.691 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:45.691 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787260+0000", 2024-09-18T10:14:45.691 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:14:45.691 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.691 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.691 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:45.691 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787265+0000", 2024-09-18T10:14:45.691 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1590000000000001e-06 2024-09-18T10:14:45.691 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.692 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.692 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:45.692 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787273+0000", 2024-09-18T10:14:45.692 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.4100000000000008e-06 2024-09-18T10:14:45.692 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.692 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.692 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:45.692 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787279+0000", 2024-09-18T10:14:45.692 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.3630000000000002e-06 2024-09-18T10:14:45.692 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.692 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.692 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:45.693 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.496955+0000", 2024-09-18T10:14:45.693 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.70967585099999997 2024-09-18T10:14:45.693 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.693 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.693 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:14:45.693 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.497107+0000", 2024-09-18T10:14:45.693 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000152044 2024-09-18T10:14:45.693 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.693 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.693 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:14:45.693 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.497345+0000", 2024-09-18T10:14:45.694 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000238693 2024-09-18T10:14:45.694 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.694 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.694 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:14:45.694 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.599248+0000", 2024-09-18T10:14:45.694 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.101902836 2024-09-18T10:14:45.694 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.694 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.695 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:14:45.695 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.599329+0000", 2024-09-18T10:14:45.695 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.1064999999999997e-05 2024-09-18T10:14:45.695 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.695 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.695 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:14:45.695 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.599441+0000", 2024-09-18T10:14:45.695 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000111797 2024-09-18T10:14:45.695 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.695 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.696 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:45.696 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.599497+0000", 2024-09-18T10:14:45.696 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.6119999999999998e-05 2024-09-18T10:14:45.696 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:45.696 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:45.696 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:45.696 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.696 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.696 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:93 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T10:14:45.696 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:38.616472+0000", 2024-09-18T10:14:45.696 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 6.983325829, 2024-09-18T10:14:45.697 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.60940747100000003, 2024-09-18T10:14:45.697 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:45.697 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:14:45.697 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:14:45.697 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:14:45.697 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:14:45.697 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 93 2024-09-18T10:14:45.697 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.697 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:45.697 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.697 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:45.698 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.616472+0000", 2024-09-18T10:14:45.698 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:45.698 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.698 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.698 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:45.698 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.616472+0000", 2024-09-18T10:14:45.698 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:45.698 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.698 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.698 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:45.699 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.616470+0000", 2024-09-18T10:14:45.699 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:14:45.699 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.699 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.699 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:45.699 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.616644+0000", 2024-09-18T10:14:45.699 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00017372000000000001 2024-09-18T10:14:45.699 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.699 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.699 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:45.699 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.616695+0000", 2024-09-18T10:14:45.699 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.0991e-05 2024-09-18T10:14:45.700 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.700 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.700 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:45.700 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.616709+0000", 2024-09-18T10:14:45.700 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4008999999999999e-05 2024-09-18T10:14:45.700 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.700 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.700 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:45.700 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.720960+0000", 2024-09-18T10:14:45.700 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.104250624 2024-09-18T10:14:45.700 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.700 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.701 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:14:45.701 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.721068+0000", 2024-09-18T10:14:45.701 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00010814700000000001 2024-09-18T10:14:45.701 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.701 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.701 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:14:45.701 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.721270+0000", 2024-09-18T10:14:45.701 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00020184200000000001 2024-09-18T10:14:45.701 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.701 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.701 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:14:45.702 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.024691+0000", 2024-09-18T10:14:45.702 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.30342105600000002 2024-09-18T10:14:45.702 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.702 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.702 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:14:45.702 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.225725+0000", 2024-09-18T10:14:45.702 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20103395099999999 2024-09-18T10:14:45.702 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.702 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.702 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:14:45.702 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.225816+0000", 2024-09-18T10:14:45.702 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.1928999999999993e-05 2024-09-18T10:14:45.703 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.703 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.703 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:45.703 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.225879+0000", 2024-09-18T10:14:45.703 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.2918999999999998e-05 2024-09-18T10:14:45.703 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:45.703 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:45.703 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:45.703 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.703 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.703 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:94 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T10:14:45.703 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:38.616729+0000", 2024-09-18T10:14:45.703 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 6.9830688040000002, 2024-09-18T10:14:45.704 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.60943170999999996, 2024-09-18T10:14:45.704 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:45.704 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:14:45.704 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:14:45.704 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:14:45.704 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:14:45.704 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 94 2024-09-18T10:14:45.704 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.704 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:45.704 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.705 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:45.705 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.616729+0000", 2024-09-18T10:14:45.705 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:45.705 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.705 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.705 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:45.705 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.616729+0000", 2024-09-18T10:14:45.705 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:45.705 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.705 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.705 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:45.705 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.616727+0000", 2024-09-18T10:14:45.706 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:14:45.706 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.706 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.706 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:45.706 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.616738+0000", 2024-09-18T10:14:45.706 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0776e-05 2024-09-18T10:14:45.706 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.706 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.706 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:45.706 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.616756+0000", 2024-09-18T10:14:45.706 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8079000000000001e-05 2024-09-18T10:14:45.706 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.706 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.707 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:45.707 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.716882+0000", 2024-09-18T10:14:45.707 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10012612999999999 2024-09-18T10:14:45.707 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.707 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.707 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:45.707 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.822315+0000", 2024-09-18T10:14:45.707 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.105433162 2024-09-18T10:14:45.707 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.707 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.707 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:14:45.707 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.822453+0000", 2024-09-18T10:14:45.708 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000137396 2024-09-18T10:14:45.708 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.708 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.708 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:14:45.708 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.822674+0000", 2024-09-18T10:14:45.708 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00022154400000000001 2024-09-18T10:14:45.708 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.708 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.708 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:14:45.708 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.024888+0000", 2024-09-18T10:14:45.708 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20221397199999999 2024-09-18T10:14:45.709 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.709 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.709 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:14:45.709 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.226011+0000", 2024-09-18T10:14:45.709 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20112316299999999 2024-09-18T10:14:45.709 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.709 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.709 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:14:45.709 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.226094+0000", 2024-09-18T10:14:45.709 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.2155000000000002e-05 2024-09-18T10:14:45.710 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.710 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.710 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:45.710 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.226161+0000", 2024-09-18T10:14:45.710 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.7138999999999995e-05 2024-09-18T10:14:45.710 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:45.710 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:45.710 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:45.710 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.710 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.710 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:95 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T10:14:45.711 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:38.716909+0000", 2024-09-18T10:14:45.711 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 6.8828890899999999, 2024-09-18T10:14:45.711 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.60960763600000001, 2024-09-18T10:14:45.711 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:45.711 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:14:45.711 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:14:45.711 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:14:45.711 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:14:45.711 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 95 2024-09-18T10:14:45.712 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.712 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:45.712 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.712 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:45.712 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.716909+0000", 2024-09-18T10:14:45.712 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:45.712 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.712 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.712 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:45.712 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.716909+0000", 2024-09-18T10:14:45.712 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:45.712 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.713 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.713 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:45.713 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.716907+0000", 2024-09-18T10:14:45.713 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:14:45.713 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.713 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.713 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:45.713 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.716921+0000", 2024-09-18T10:14:45.713 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4572e-05 2024-09-18T10:14:45.714 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.714 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.714 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:45.714 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.716939+0000", 2024-09-18T10:14:45.714 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7149000000000001e-05 2024-09-18T10:14:45.714 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.714 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.714 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:45.714 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.716948+0000", 2024-09-18T10:14:45.714 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.7000000000000003e-06 2024-09-18T10:14:45.715 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.715 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.715 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:45.715 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.923430+0000", 2024-09-18T10:14:45.715 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20648133299999999 2024-09-18T10:14:45.715 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.715 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.715 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:14:45.715 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.923566+0000", 2024-09-18T10:14:45.715 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00013657599999999999 2024-09-18T10:14:45.716 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.716 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.716 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:14:45.716 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.923782+0000", 2024-09-18T10:14:45.716 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00021599400000000001 2024-09-18T10:14:45.716 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.716 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.716 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:14:45.716 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.024899+0000", 2024-09-18T10:14:45.716 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10111666800000001 2024-09-18T10:14:45.717 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.717 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.717 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:14:45.717 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.326379+0000", 2024-09-18T10:14:45.717 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.30148048799999999 2024-09-18T10:14:45.717 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.717 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.717 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:14:45.717 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.326454+0000", 2024-09-18T10:14:45.717 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.4988999999999996e-05 2024-09-18T10:14:45.718 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.718 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.718 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:45.718 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.326516+0000", 2024-09-18T10:14:45.718 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.1902999999999993e-05 2024-09-18T10:14:45.718 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:45.718 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:45.718 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:45.718 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.718 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.718 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:96 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T10:14:45.719 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:38.716972+0000", 2024-09-18T10:14:45.719 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 6.8828260999999999, 2024-09-18T10:14:45.719 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.91539460800000005, 2024-09-18T10:14:45.719 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:45.719 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:14:45.719 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:14:45.719 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:14:45.719 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:14:45.719 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 96 2024-09-18T10:14:45.719 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.719 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:45.719 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.720 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:45.720 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.716972+0000", 2024-09-18T10:14:45.720 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:45.720 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.720 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.720 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:45.720 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.716972+0000", 2024-09-18T10:14:45.720 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:45.720 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.720 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.721 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:45.721 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.716971+0000", 2024-09-18T10:14:45.721 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:14:45.721 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.721 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.721 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:45.721 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717061+0000", 2024-09-18T10:14:45.721 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.0744999999999999e-05 2024-09-18T10:14:45.721 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.721 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.721 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:45.721 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717073+0000", 2024-09-18T10:14:45.722 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1943000000000001e-05 2024-09-18T10:14:45.722 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.722 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.722 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:45.722 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717080+0000", 2024-09-18T10:14:45.722 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0060000000000003e-06 2024-09-18T10:14:45.722 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.722 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.722 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:45.722 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.426708+0000", 2024-09-18T10:14:45.722 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.70962730399999996 2024-09-18T10:14:45.722 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.722 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.722 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:14:45.723 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.426854+0000", 2024-09-18T10:14:45.723 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00014654199999999999 2024-09-18T10:14:45.723 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.723 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.723 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:14:45.723 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.427106+0000", 2024-09-18T10:14:45.723 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00025226600000000001 2024-09-18T10:14:45.723 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.723 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.723 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:14:45.723 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.433087+0000", 2024-09-18T10:14:45.724 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0059804860000000001 2024-09-18T10:14:45.724 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.724 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.724 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:14:45.724 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.632096+0000", 2024-09-18T10:14:45.724 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.19900922099999999 2024-09-18T10:14:45.724 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.724 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.724 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:14:45.724 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.632257+0000", 2024-09-18T10:14:45.724 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00016058399999999999 2024-09-18T10:14:45.725 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.725 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.725 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:45.725 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.632366+0000", 2024-09-18T10:14:45.725 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000109601 2024-09-18T10:14:45.725 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:45.725 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:45.725 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:45.725 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.725 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.725 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:97 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T10:14:45.726 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:38.717093+0000", 2024-09-18T10:14:45.726 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 6.8827042899999995, 2024-09-18T10:14:45.726 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.91561460900000002, 2024-09-18T10:14:45.726 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:45.726 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:14:45.726 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:14:45.726 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:14:45.726 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:14:45.726 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 97 2024-09-18T10:14:45.726 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.726 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:45.726 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.727 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:45.727 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717093+0000", 2024-09-18T10:14:45.727 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:45.727 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.727 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.727 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:45.727 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717093+0000", 2024-09-18T10:14:45.727 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:45.727 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.727 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.728 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:45.728 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717092+0000", 2024-09-18T10:14:45.728 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:14:45.728 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.728 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.728 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:45.728 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717099+0000", 2024-09-18T10:14:45.728 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.246e-06 2024-09-18T10:14:45.728 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.728 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.728 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:45.729 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717109+0000", 2024-09-18T10:14:45.729 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0499999999999999e-05 2024-09-18T10:14:45.729 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.729 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.729 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:45.729 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717115+0000", 2024-09-18T10:14:45.729 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.7690000000000002e-06 2024-09-18T10:14:45.729 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.729 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.729 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:45.730 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.428509+0000", 2024-09-18T10:14:45.730 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.71139444299999999 2024-09-18T10:14:45.730 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.730 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.730 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:14:45.730 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.428621+0000", 2024-09-18T10:14:45.730 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00011156 2024-09-18T10:14:45.730 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.730 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.730 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:14:45.730 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.428805+0000", 2024-09-18T10:14:45.730 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000183597 2024-09-18T10:14:45.731 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.731 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.731 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:14:45.731 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.433106+0000", 2024-09-18T10:14:45.731 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0043015019999999996 2024-09-18T10:14:45.731 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.731 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.731 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:14:45.731 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.632474+0000", 2024-09-18T10:14:45.731 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.199368092 2024-09-18T10:14:45.731 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.732 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.732 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:14:45.732 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.632605+0000", 2024-09-18T10:14:45.732 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00013110899999999999 2024-09-18T10:14:45.732 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.732 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.732 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:45.732 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.632708+0000", 2024-09-18T10:14:45.732 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000102748 2024-09-18T10:14:45.733 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:45.733 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:45.733 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:45.733 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.733 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.733 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:98 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T10:14:45.733 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:38.717126+0000", 2024-09-18T10:14:45.733 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 6.882672028, 2024-09-18T10:14:45.733 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.015084141, 2024-09-18T10:14:45.733 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:45.733 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:14:45.734 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:14:45.734 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:14:45.734 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:14:45.734 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 98 2024-09-18T10:14:45.734 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.734 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:45.734 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.734 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:45.734 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717126+0000", 2024-09-18T10:14:45.734 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:45.735 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.735 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.735 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:45.735 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717126+0000", 2024-09-18T10:14:45.735 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:45.735 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.735 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.735 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:45.735 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717125+0000", 2024-09-18T10:14:45.735 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:14:45.736 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.736 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.736 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:45.736 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717130+0000", 2024-09-18T10:14:45.736 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.4160000000000003e-06 2024-09-18T10:14:45.736 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.736 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.736 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:45.736 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717139+0000", 2024-09-18T10:14:45.736 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.6230000000000003e-06 2024-09-18T10:14:45.736 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.737 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.737 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:45.737 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717144+0000", 2024-09-18T10:14:45.737 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5330000000000002e-06 2024-09-18T10:14:45.737 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.737 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.737 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:45.737 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.429491+0000", 2024-09-18T10:14:45.737 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.71234629500000002 2024-09-18T10:14:45.737 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.738 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.738 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:14:45.738 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.429741+0000", 2024-09-18T10:14:45.738 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000250516 2024-09-18T10:14:45.738 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.738 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.738 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:14:45.738 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.429989+0000", 2024-09-18T10:14:45.738 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00024811599999999999 2024-09-18T10:14:45.738 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.738 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.739 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:14:45.739 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.435056+0000", 2024-09-18T10:14:45.739 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0050669670000000003 2024-09-18T10:14:45.739 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.739 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.739 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:14:45.739 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.732068+0000", 2024-09-18T10:14:45.739 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.29701180900000002 2024-09-18T10:14:45.739 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.739 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.739 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:14:45.740 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.732145+0000", 2024-09-18T10:14:45.740 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.7152999999999998e-05 2024-09-18T10:14:45.740 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.740 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.740 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:45.740 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.732210+0000", 2024-09-18T10:14:45.740 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.4503000000000002e-05 2024-09-18T10:14:45.740 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:45.740 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:45.740 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:45.740 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.741 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.741 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:99 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T10:14:45.741 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:38.717230+0000", 2024-09-18T10:14:45.741 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 6.8825674679999995, 2024-09-18T10:14:45.741 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0157905819999999, 2024-09-18T10:14:45.741 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:45.741 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:14:45.741 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:14:45.741 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:14:45.741 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:14:45.741 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 99 2024-09-18T10:14:45.742 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.742 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:45.742 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.742 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:45.742 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717230+0000", 2024-09-18T10:14:45.742 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:45.742 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.742 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.742 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:45.742 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717230+0000", 2024-09-18T10:14:45.742 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:45.742 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.743 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.743 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:45.743 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717229+0000", 2024-09-18T10:14:45.743 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:14:45.743 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.743 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.743 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:45.743 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717318+0000", 2024-09-18T10:14:45.743 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.8197999999999998e-05 2024-09-18T10:14:45.743 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.744 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.744 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:45.744 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717342+0000", 2024-09-18T10:14:45.744 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4445000000000001e-05 2024-09-18T10:14:45.744 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.744 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.744 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:45.744 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717360+0000", 2024-09-18T10:14:45.744 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7782999999999999e-05 2024-09-18T10:14:45.744 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.744 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.744 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:45.745 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.430650+0000", 2024-09-18T10:14:45.745 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.71329020200000004 2024-09-18T10:14:45.745 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.745 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.745 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:14:45.745 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.430805+0000", 2024-09-18T10:14:45.745 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00015474900000000001 2024-09-18T10:14:45.745 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.745 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.745 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:14:45.745 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.431021+0000", 2024-09-18T10:14:45.745 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000215911 2024-09-18T10:14:45.746 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.746 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.746 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:14:45.746 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.435099+0000", 2024-09-18T10:14:45.746 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0040787439999999996 2024-09-18T10:14:45.746 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.746 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.746 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:14:45.746 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.732758+0000", 2024-09-18T10:14:45.746 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.297658637 2024-09-18T10:14:45.746 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.747 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.747 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:14:45.747 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.732909+0000", 2024-09-18T10:14:45.747 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00015069899999999999 2024-09-18T10:14:45.747 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.747 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.747 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:45.747 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.733021+0000", 2024-09-18T10:14:45.747 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00011207 2024-09-18T10:14:45.747 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:45.747 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:45.748 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:45.748 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.748 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.748 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:100 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T10:14:45.748 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:38.717370+0000", 2024-09-18T10:14:45.748 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 6.8824279060000002, 2024-09-18T10:14:45.748 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1158324850000001, 2024-09-18T10:14:45.748 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:45.748 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:14:45.748 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:14:45.748 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:14:45.749 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:14:45.749 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 100 2024-09-18T10:14:45.749 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.749 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:45.749 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.749 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:45.749 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717370+0000", 2024-09-18T10:14:45.749 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:45.749 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.749 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.749 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:45.750 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717370+0000", 2024-09-18T10:14:45.750 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:45.750 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.750 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.750 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:45.750 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717369+0000", 2024-09-18T10:14:45.750 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:14:45.750 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.750 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.750 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:45.750 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717378+0000", 2024-09-18T10:14:45.750 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.3559999999999999e-06 2024-09-18T10:14:45.751 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.751 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.751 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:45.751 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717398+0000", 2024-09-18T10:14:45.751 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.9962e-05 2024-09-18T10:14:45.751 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.751 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.751 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:45.751 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717407+0000", 2024-09-18T10:14:45.751 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.2699999999999993e-06 2024-09-18T10:14:45.751 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.751 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.752 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:45.752 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.432184+0000", 2024-09-18T10:14:45.752 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.71477630299999995 2024-09-18T10:14:45.752 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.752 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.752 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:14:45.752 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.432316+0000", 2024-09-18T10:14:45.752 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000132082 2024-09-18T10:14:45.752 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.752 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.752 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:14:45.753 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.432526+0000", 2024-09-18T10:14:45.753 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000210659 2024-09-18T10:14:45.753 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.753 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.753 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:14:45.753 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.435651+0000", 2024-09-18T10:14:45.753 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.003124169 2024-09-18T10:14:45.753 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.753 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.753 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:14:45.753 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.832931+0000", 2024-09-18T10:14:45.753 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.39728084899999999 2024-09-18T10:14:45.754 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.754 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.754 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:14:45.754 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.833058+0000", 2024-09-18T10:14:45.754 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000126969 2024-09-18T10:14:45.754 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:45.754 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:45.754 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:45.754 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.833202+0000", 2024-09-18T10:14:45.754 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000143922 2024-09-18T10:14:45.754 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:45.754 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:45.755 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:45.755 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:45.755 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:45.755 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-09-18T10:14:45.757 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool application enable unique_pool_0 rados --yes-i-really-mean-it || true 2024-09-18T10:14:45.758 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-09-18T10:14:45.758 INFO:tasks.ceph.osd.6.smithi154.stderr:2024-09-18T10:14:45.752+0000 7f0abc9e1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22089) UID: 0 2024-09-18T10:14:45.858 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-18T10:14:45.854+0000 7f14c6a1a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 22138) UID: 0 2024-09-18T10:14:45.868 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-09-18T10:14:45.868 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-09-18T10:14:45.868 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-09-18T10:14:45.869 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-09-18T10:14:45.878 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-09-18T10:14:45.959 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-18T10:14:45.958+0000 7f1981873700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22132) UID: 0 2024-09-18T10:14:45.973 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-09-18T10:14:45.973 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-09-18T10:14:45.973 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-09-18T10:14:45.973 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-09-18T10:14:45.973 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-09-18T10:14:45.982 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-09-18T10:14:46.059 INFO:tasks.ceph.osd.14.smithi154.stderr:2024-09-18T10:14:46.056+0000 7f75a19e7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 22093) UID: 0 2024-09-18T10:14:46.078 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-09-18T10:14:46.078 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-09-18T10:14:46.078 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-09-18T10:14:46.078 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [] 2024-09-18T10:14:46.078 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-09-18T10:14:46.087 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_ops_in_flight 2024-09-18T10:14:46.161 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-18T10:14:46.158+0000 7f14c6a1a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 22138) UID: 0 2024-09-18T10:14:46.188 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-09-18T10:14:46.188 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-09-18T10:14:46.189 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-09-18T10:14:46.189 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-09-18T10:14:46.198 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_blocked_ops 2024-09-18T10:14:46.262 INFO:tasks.ceph.osd.4.smithi029.stderr:2024-09-18T10:14:46.259+0000 7f550cf6e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22189) UID: 0 2024-09-18T10:14:46.296 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-09-18T10:14:46.296 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-09-18T10:14:46.296 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-09-18T10:14:46.296 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-09-18T10:14:46.296 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-09-18T10:14:46.305 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_historic_ops 2024-09-18T10:14:46.362 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-18T10:14:46.359+0000 7f302f2c3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22187) UID: 0 2024-09-18T10:14:46.404 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-09-18T10:14:46.405 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-09-18T10:14:46.405 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-09-18T10:14:46.405 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-09-18T10:14:46.405 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:46.405 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4341.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-18T10:14:46.405 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:35.324351+0000", 2024-09-18T10:14:46.405 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 11.079939389, 2024-09-18T10:14:46.405 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.36640713699999999, 2024-09-18T10:14:46.405 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:46.405 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:14:46.405 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:14:46.405 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4341", 2024-09-18T10:14:46.406 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.29:0/3001591505", 2024-09-18T10:14:46.406 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 1 2024-09-18T10:14:46.406 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:46.406 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:46.406 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:46.406 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:46.406 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.324351+0000", 2024-09-18T10:14:46.406 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:46.406 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:46.406 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:46.406 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:46.406 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.324351+0000", 2024-09-18T10:14:46.406 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:46.407 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:46.407 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:46.407 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:46.407 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.324348+0000", 2024-09-18T10:14:46.407 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999976 2024-09-18T10:14:46.407 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:46.407 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:46.407 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:46.407 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.324359+0000", 2024-09-18T10:14:46.407 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1048e-05 2024-09-18T10:14:46.407 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:46.407 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:46.407 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:46.407 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.324383+0000", 2024-09-18T10:14:46.408 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.3813000000000001e-05 2024-09-18T10:14:46.408 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:46.408 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:46.408 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:46.408 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.324421+0000", 2024-09-18T10:14:46.408 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8012000000000001e-05 2024-09-18T10:14:46.408 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:46.408 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:46.408 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:46.408 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.324548+0000", 2024-09-18T10:14:46.408 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00012702199999999999 2024-09-18T10:14:46.408 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:46.408 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:46.409 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for peered", 2024-09-18T10:14:46.409 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.324565+0000", 2024-09-18T10:14:46.409 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.6325e-05 2024-09-18T10:14:46.409 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:46.409 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:46.409 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:46.409 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.588380+0000", 2024-09-18T10:14:46.409 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.263815458 2024-09-18T10:14:46.409 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:46.409 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:46.409 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:14:46.409 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.588626+0000", 2024-09-18T10:14:46.409 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00024633499999999998 2024-09-18T10:14:46.409 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:46.410 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:46.410 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 5", 2024-09-18T10:14:46.410 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.588803+0000", 2024-09-18T10:14:46.410 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00017640599999999999 2024-09-18T10:14:46.410 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:46.410 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:46.410 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:14:46.410 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.589943+0000", 2024-09-18T10:14:46.410 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0011400049999999999 2024-09-18T10:14:46.410 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:46.410 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:46.410 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:14:46.410 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.690623+0000", 2024-09-18T10:14:46.410 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100680032 2024-09-18T10:14:46.411 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:46.411 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:46.411 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:14:46.411 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.690702+0000", 2024-09-18T10:14:46.411 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.9342e-05 2024-09-18T10:14:46.411 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:46.411 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:46.411 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:46.411 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.690758+0000", 2024-09-18T10:14:46.411 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5639e-05 2024-09-18T10:14:46.411 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:46.411 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:46.411 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:46.411 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:46.412 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:46.412 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop_reply(client.4341.0:1 2.3 e27/26 ondisk, result = 0)", 2024-09-18T10:14:46.412 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:35.690433+0000", 2024-09-18T10:14:46.412 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 10.713857355, 2024-09-18T10:14:46.412 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00033331300000000001, 2024-09-18T10:14:46.412 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:46.412 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "started", 2024-09-18T10:14:46.412 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:46.412 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:46.412 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:46.412 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.690433+0000", 2024-09-18T10:14:46.412 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:46.412 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:46.413 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:46.413 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:46.413 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.690433+0000", 2024-09-18T10:14:46.413 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:46.413 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:46.413 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:46.413 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:46.413 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.690431+0000", 2024-09-18T10:14:46.413 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:14:46.413 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:46.413 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:46.413 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:46.413 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.690442+0000", 2024-09-18T10:14:46.413 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1178999999999999e-05 2024-09-18T10:14:46.414 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:46.414 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:46.414 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:46.414 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.690472+0000", 2024-09-18T10:14:46.414 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9649e-05 2024-09-18T10:14:46.414 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:46.414 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:46.414 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:46.414 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.690485+0000", 2024-09-18T10:14:46.414 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2719e-05 2024-09-18T10:14:46.414 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:46.414 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:46.414 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:46.414 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.690588+0000", 2024-09-18T10:14:46.415 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00010368400000000001 2024-09-18T10:14:46.415 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:46.415 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:46.415 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:14:46.415 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.690612+0000", 2024-09-18T10:14:46.415 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.3308000000000001e-05 2024-09-18T10:14:46.415 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:46.415 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:46.415 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:46.415 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.690766+0000", 2024-09-18T10:14:46.415 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00015436099999999999 2024-09-18T10:14:46.415 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:46.415 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:46.415 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:46.416 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:46.416 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:46.416 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-09-18T10:14:46.417 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T10:14:46.463 INFO:tasks.ceph.osd.4.smithi029.stderr:2024-09-18T10:14:46.459+0000 7f550cf6e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22189) UID: 0 2024-09-18T10:14:46.526 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-18T10:14:46.526 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-18T10:14:46.526 INFO:teuthology.orchestra.run.smithi189.stdout: "num_ops": 0 2024-09-18T10:14:46.526 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-18T10:14:46.536 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T10:14:46.564 INFO:tasks.ceph.osd.10.smithi154.stderr:2024-09-18T10:14:46.560+0000 7fb73221a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 22091) UID: 0 2024-09-18T10:14:46.600 INFO:teuthology.orchestra.run.smithi029.stderr:nodeep-scrub is set 2024-09-18T10:14:46.603 INFO:teuthology.orchestra.run.smithi029.stderr:enabled application 'rados' on pool 'unique_pool_0' 2024-09-18T10:14:46.642 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-18T10:14:46.642 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-18T10:14:46.642 INFO:teuthology.orchestra.run.smithi189.stdout: "complaint_time": 30, 2024-09-18T10:14:46.642 INFO:teuthology.orchestra.run.smithi189.stdout: "num_blocked_ops": 0 2024-09-18T10:14:46.642 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-18T10:14:46.651 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T10:14:46.665 INFO:tasks.ceph.osd.10.smithi154.stderr:2024-09-18T10:14:46.660+0000 7fb73221a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 22091) UID: 0 2024-09-18T10:14:46.745 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-18T10:14:46.745 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 20, 2024-09-18T10:14:46.745 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 600, 2024-09-18T10:14:46.745 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [] 2024-09-18T10:14:46.745 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-18T10:14:46.754 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T10:14:46.765 INFO:tasks.ceph.osd.14.smithi154.stderr:2024-09-18T10:14:46.760+0000 7f75a19e7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 22093) UID: 0 2024-09-18T10:14:46.851 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-18T10:14:46.851 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-18T10:14:46.851 INFO:teuthology.orchestra.run.smithi189.stdout: "num_ops": 0 2024-09-18T10:14:46.852 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-18T10:14:46.860 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T10:14:46.866 INFO:tasks.ceph.osd.12.smithi029.stderr:2024-09-18T10:14:46.863+0000 7f8e4e627700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 22190) UID: 0 2024-09-18T10:14:46.957 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-18T10:14:46.958 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-18T10:14:46.958 INFO:teuthology.orchestra.run.smithi189.stdout: "complaint_time": 30, 2024-09-18T10:14:46.958 INFO:teuthology.orchestra.run.smithi189.stdout: "num_blocked_ops": 0 2024-09-18T10:14:46.958 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-18T10:14:46.967 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T10:14:46.967 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-09-18T10:14:46.960+0000 7fdfa8698700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22087) UID: 0 2024-09-18T10:14:47.063 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-18T10:14:47.063 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 20, 2024-09-18T10:14:47.063 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 600, 2024-09-18T10:14:47.063 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [] 2024-09-18T10:14:47.063 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-18T10:14:47.067 INFO:tasks.ceph.osd.4.smithi029.stderr:2024-09-18T10:14:47.062+0000 7f550cf6e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22189) UID: 0 2024-09-18T10:14:47.072 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-09-18T10:14:47.167 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-18T10:14:47.167 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-18T10:14:47.167 INFO:teuthology.orchestra.run.smithi189.stdout: "num_ops": 0 2024-09-18T10:14:47.167 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-18T10:14:47.168 INFO:tasks.ceph.osd.15.smithi189.stderr:2024-09-18T10:14:47.167+0000 7fd0e725e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 22401) UID: 0 2024-09-18T10:14:47.176 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-09-18T10:14:47.269 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-18T10:14:47.266+0000 7f302f2c3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22187) UID: 0 2024-09-18T10:14:47.275 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-18T10:14:47.275 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-18T10:14:47.275 INFO:teuthology.orchestra.run.smithi189.stdout: "complaint_time": 30, 2024-09-18T10:14:47.275 INFO:teuthology.orchestra.run.smithi189.stdout: "num_blocked_ops": 0 2024-09-18T10:14:47.275 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-18T10:14:47.284 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-09-18T10:14:47.371 INFO:tasks.ceph.osd.14.smithi154.stderr:2024-09-18T10:14:47.368+0000 7f75a19e7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 22093) UID: 0 2024-09-18T10:14:47.385 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-18T10:14:47.385 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 20, 2024-09-18T10:14:47.386 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 600, 2024-09-18T10:14:47.386 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [] 2024-09-18T10:14:47.386 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-18T10:14:47.395 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_ops_in_flight 2024-09-18T10:14:47.398 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T10:14:47.426 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15] out_osds: [] dead_osds: [] live_osds: [0, 4, 8, 12, 1, 5, 9, 13, 2, 6, 10, 14, 3, 7, 11, 15] 2024-09-18T10:14:47.427 INFO:tasks.thrashosds.thrasher:choose_action: min_in 8 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T10:14:47.427 INFO:tasks.thrashosds.thrasher:inject_pause on osd.3 2024-09-18T10:14:47.427 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-18T10:14:47.427 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-18T10:14:47.427 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok config set filestore_inject_stall 3 2024-09-18T10:14:47.472 INFO:tasks.ceph.osd.4.smithi029.stderr:2024-09-18T10:14:47.466+0000 7f550cf6e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22189) UID: 0 2024-09-18T10:14:47.493 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-18T10:14:47.494 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-18T10:14:47.494 INFO:teuthology.orchestra.run.smithi189.stdout: "num_ops": 0 2024-09-18T10:14:47.494 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-18T10:14:47.503 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_blocked_ops 2024-09-18T10:14:47.528 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-18T10:14:47.528 INFO:teuthology.orchestra.run.smithi189.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-18T10:14:47.528 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-18T10:14:47.544 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-18T10:14:47.568 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-18T10:14:47.580 INFO:tasks.ceph.osd.11.smithi189.stderr:2024-09-18T10:14:47.579+0000 7f718b60a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 22400) UID: 0 2024-09-18T10:14:47.596 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-18T10:14:47.598 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-18T10:14:47.598 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-18T10:14:47.598 INFO:teuthology.orchestra.run.smithi189.stdout: "complaint_time": 30, 2024-09-18T10:14:47.598 INFO:teuthology.orchestra.run.smithi189.stdout: "num_blocked_ops": 0 2024-09-18T10:14:47.598 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-18T10:14:47.607 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_historic_ops 2024-09-18T10:14:47.616 DEBUG:teuthology.orchestra.run.smithi189:> CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --no-omap --ec-pool --max-ops 400 --objects 50 --max-in-flight 16 --size 4000000 --min-stride-size 400000 --max-stride-size 800000 --max-seconds 0 --op read 100 --op write 0 --op delete 50 --op snap_create 50 --op snap_remove 50 --op rollback 50 --op setattr 25 --op rmattr 25 --op copy_from 50 --op append 100 --pool unique_pool_0 2024-09-18T10:14:47.618 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-18T10:14:47.635 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-18T10:14:47.636 INFO:tasks.rados.rados.0.smithi189.stdout:adding op weight read -> 100 2024-09-18T10:14:47.636 INFO:tasks.rados.rados.0.smithi189.stdout:adding op weight delete -> 50 2024-09-18T10:14:47.636 INFO:tasks.rados.rados.0.smithi189.stdout:adding op weight snap_create -> 50 2024-09-18T10:14:47.636 INFO:tasks.rados.rados.0.smithi189.stdout:adding op weight snap_remove -> 50 2024-09-18T10:14:47.636 INFO:tasks.rados.rados.0.smithi189.stdout:adding op weight rollback -> 50 2024-09-18T10:14:47.636 INFO:tasks.rados.rados.0.smithi189.stdout:adding op weight setattr -> 25 2024-09-18T10:14:47.636 INFO:tasks.rados.rados.0.smithi189.stdout:adding op weight rmattr -> 25 2024-09-18T10:14:47.637 INFO:tasks.rados.rados.0.smithi189.stdout:adding op weight copy_from -> 50 2024-09-18T10:14:47.637 INFO:tasks.rados.rados.0.smithi189.stdout:adding op weight append -> 100 2024-09-18T10:14:47.637 INFO:tasks.rados.rados.0.smithi189.stdout:ceph version 17.2.7-1645-g7061ea6c (7061ea6ca50908d104e9f3979dc615e118d9e153) quincy (stable) 2024-09-18T10:14:47.637 INFO:tasks.rados.rados.0.smithi189.stdout:Configuration: 2024-09-18T10:14:47.637 INFO:tasks.rados.rados.0.smithi189.stdout: Number of operations: 400 2024-09-18T10:14:47.637 INFO:tasks.rados.rados.0.smithi189.stdout: Number of objects: 50 2024-09-18T10:14:47.637 INFO:tasks.rados.rados.0.smithi189.stdout: Max in flight operations: 16 2024-09-18T10:14:47.637 INFO:tasks.rados.rados.0.smithi189.stdout: Object size (in bytes): 4000000 2024-09-18T10:14:47.637 INFO:tasks.rados.rados.0.smithi189.stdout: Write stride min: 400000 2024-09-18T10:14:47.637 INFO:tasks.rados.rados.0.smithi189.stdout: Write stride max: 800000 2024-09-18T10:14:47.658 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-18T10:14:47.674 INFO:tasks.ceph.osd.15.smithi189.stderr:2024-09-18T10:14:47.671+0000 7fd0e725e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 22401) UID: 0 2024-09-18T10:14:47.682 INFO:teuthology.orchestra.run.smithi029.stdout:osd.6: {} 2024-09-18T10:14:47.705 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-18T10:14:47.705 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 20, 2024-09-18T10:14:47.705 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 600, 2024-09-18T10:14:47.705 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [ 2024-09-18T10:14:47.705 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.706 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4257.0:19 1.0 e24/23 1:30306672:devicehealth::main.db.0000000000000000:head v 24'9, mlcod=24'7)", 2024-09-18T10:14:47.706 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:33.346192+0000", 2024-09-18T10:14:47.706 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 14.356360092999999, 2024-09-18T10:14:47.706 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20178458699999999, 2024-09-18T10:14:47.706 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:14:47.706 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:14:47.706 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:14:47.706 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.706 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:14:47.706 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:33.346192+0000", 2024-09-18T10:14:47.706 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:47.706 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.707 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.707 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:14:47.707 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:33.346192+0000", 2024-09-18T10:14:47.707 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:47.707 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.707 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.707 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:14:47.707 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:33.346190+0000", 2024-09-18T10:14:47.707 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:14:47.707 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.707 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.707 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:14:47.707 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:33.346206+0000", 2024-09-18T10:14:47.707 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6257e-05 2024-09-18T10:14:47.708 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.708 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.708 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:14:47.708 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:33.346237+0000", 2024-09-18T10:14:47.708 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0471999999999999e-05 2024-09-18T10:14:47.708 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.708 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.708 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:14:47.708 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:33.446377+0000", 2024-09-18T10:14:47.708 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10014044499999999 2024-09-18T10:14:47.708 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.708 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.708 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:14:47.709 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:33.546722+0000", 2024-09-18T10:14:47.709 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100344593 2024-09-18T10:14:47.709 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.709 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.709 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:14:47.709 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:33.546787+0000", 2024-09-18T10:14:47.709 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.5347999999999998e-05 2024-09-18T10:14:47.709 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.709 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.709 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:14:47.709 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:33.547891+0000", 2024-09-18T10:14:47.709 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001104379 2024-09-18T10:14:47.709 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.710 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.710 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:14:47.710 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:33.547976+0000", 2024-09-18T10:14:47.710 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.5023000000000002e-05 2024-09-18T10:14:47.710 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:47.710 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:14:47.710 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:47.710 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.710 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.710 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4257.0:20 1.0 e25/23 1:30306672:devicehealth::main.db.0000000000000000:head v 25'10, mlcod=24'8)", 2024-09-18T10:14:47.710 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:33.750589+0000", 2024-09-18T10:14:47.710 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 13.951962801000001, 2024-09-18T10:14:47.710 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10494690499999999, 2024-09-18T10:14:47.710 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:14:47.711 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:14:47.711 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:14:47.711 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.711 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:14:47.711 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:33.750589+0000", 2024-09-18T10:14:47.711 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:47.711 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.711 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.711 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:14:47.711 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:33.750589+0000", 2024-09-18T10:14:47.711 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:47.711 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.711 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.712 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:14:47.712 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:33.750587+0000", 2024-09-18T10:14:47.712 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:14:47.712 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.712 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.712 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:14:47.712 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:33.750796+0000", 2024-09-18T10:14:47.712 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00020836499999999999 2024-09-18T10:14:47.712 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.712 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.712 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:14:47.712 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:33.750828+0000", 2024-09-18T10:14:47.712 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2225999999999999e-05 2024-09-18T10:14:47.713 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.713 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.713 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:14:47.713 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:33.750846+0000", 2024-09-18T10:14:47.713 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7612e-05 2024-09-18T10:14:47.713 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.713 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.713 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:14:47.713 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:33.851062+0000", 2024-09-18T10:14:47.713 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100216765 2024-09-18T10:14:47.713 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.713 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.714 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:14:47.714 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:33.851099+0000", 2024-09-18T10:14:47.714 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7088000000000001e-05 2024-09-18T10:14:47.714 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.714 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.714 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:14:47.714 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:33.855442+0000", 2024-09-18T10:14:47.714 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0043423419999999999 2024-09-18T10:14:47.714 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.714 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.714 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:14:47.714 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:33.855536+0000", 2024-09-18T10:14:47.714 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.4300000000000002e-05 2024-09-18T10:14:47.715 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:47.715 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:14:47.715 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:47.715 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.715 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.715 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4257.0:21 1.0 e25/23 1:30306672:devicehealth::main.db.0000000000000000:head v 25'11, mlcod=24'8)", 2024-09-18T10:14:47.715 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:33.751653+0000", 2024-09-18T10:14:47.715 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 13.950898697, 2024-09-18T10:14:47.715 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10398861700000001, 2024-09-18T10:14:47.715 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:14:47.715 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:14:47.715 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:14:47.715 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.716 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:14:47.716 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:33.751653+0000", 2024-09-18T10:14:47.716 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:47.716 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.716 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.716 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:14:47.716 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:33.751653+0000", 2024-09-18T10:14:47.716 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:47.716 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.716 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.716 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:14:47.716 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:33.751651+0000", 2024-09-18T10:14:47.716 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:14:47.717 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.717 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.717 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:14:47.717 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:33.751811+0000", 2024-09-18T10:14:47.717 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00015969500000000001 2024-09-18T10:14:47.717 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.717 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.717 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:14:47.717 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:33.751844+0000", 2024-09-18T10:14:47.717 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2462000000000002e-05 2024-09-18T10:14:47.717 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.717 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.717 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:14:47.718 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:33.751860+0000", 2024-09-18T10:14:47.718 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6396e-05 2024-09-18T10:14:47.718 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.718 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.718 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:14:47.718 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:33.851799+0000", 2024-09-18T10:14:47.718 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.099938674000000005 2024-09-18T10:14:47.718 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.718 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.718 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:14:47.718 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:33.851829+0000", 2024-09-18T10:14:47.718 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0185999999999999e-05 2024-09-18T10:14:47.718 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.719 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.719 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:14:47.719 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:33.855554+0000", 2024-09-18T10:14:47.719 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0037252309999999999 2024-09-18T10:14:47.719 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.719 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.719 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:14:47.719 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:33.855642+0000", 2024-09-18T10:14:47.719 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.7756999999999993e-05 2024-09-18T10:14:47.719 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:47.719 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:14:47.719 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:47.719 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.719 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.720 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4257.0:42 1.0 e26/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 26'24, mlcod=25'20)", 2024-09-18T10:14:47.720 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:35.195493+0000", 2024-09-18T10:14:47.720 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 12.507058993999999, 2024-09-18T10:14:47.720 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.104409683, 2024-09-18T10:14:47.720 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:14:47.720 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:14:47.720 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:14:47.720 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.720 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:14:47.720 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:35.195493+0000", 2024-09-18T10:14:47.720 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:47.720 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.721 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.721 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:14:47.721 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:35.195493+0000", 2024-09-18T10:14:47.721 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:47.721 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.721 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.721 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:14:47.721 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:35.195491+0000", 2024-09-18T10:14:47.721 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:14:47.721 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.721 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.721 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:14:47.721 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:35.195509+0000", 2024-09-18T10:14:47.722 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.8046e-05 2024-09-18T10:14:47.722 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.722 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.722 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:14:47.722 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:35.195537+0000", 2024-09-18T10:14:47.722 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7665999999999999e-05 2024-09-18T10:14:47.722 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.722 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.722 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:14:47.722 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:35.195552+0000", 2024-09-18T10:14:47.722 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5176e-05 2024-09-18T10:14:47.722 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.722 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.723 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:14:47.723 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:35.195670+0000", 2024-09-18T10:14:47.723 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000118095 2024-09-18T10:14:47.723 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.723 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.723 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:14:47.723 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:35.195703+0000", 2024-09-18T10:14:47.723 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3408999999999998e-05 2024-09-18T10:14:47.723 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.723 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.723 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:14:47.723 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:35.299837+0000", 2024-09-18T10:14:47.723 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.104133116 2024-09-18T10:14:47.724 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.724 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.724 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:14:47.724 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:35.299903+0000", 2024-09-18T10:14:47.724 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.5994999999999994e-05 2024-09-18T10:14:47.724 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:47.724 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:14:47.724 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:47.724 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.725 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.725 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4257.0:46 1.0 e26/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 26'28, mlcod=26'22)", 2024-09-18T10:14:47.725 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:35.200118+0000", 2024-09-18T10:14:47.725 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 12.502434002999999, 2024-09-18T10:14:47.725 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.104914961, 2024-09-18T10:14:47.725 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:14:47.725 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:14:47.725 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:14:47.725 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.725 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:14:47.725 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:35.200118+0000", 2024-09-18T10:14:47.725 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:47.725 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.725 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.726 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:14:47.726 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:35.200118+0000", 2024-09-18T10:14:47.726 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:47.726 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.726 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.726 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:14:47.726 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:35.200116+0000", 2024-09-18T10:14:47.726 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:14:47.726 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.726 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.726 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:14:47.726 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:35.200130+0000", 2024-09-18T10:14:47.726 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3716e-05 2024-09-18T10:14:47.727 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.727 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.727 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:14:47.727 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:35.200163+0000", 2024-09-18T10:14:47.727 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2909e-05 2024-09-18T10:14:47.727 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.727 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.727 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:14:47.727 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:35.200181+0000", 2024-09-18T10:14:47.727 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7768999999999999e-05 2024-09-18T10:14:47.727 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.727 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.727 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:14:47.728 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:35.299419+0000", 2024-09-18T10:14:47.728 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.099238449000000006 2024-09-18T10:14:47.728 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.728 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.728 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:14:47.728 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:35.299454+0000", 2024-09-18T10:14:47.728 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.5315000000000003e-05 2024-09-18T10:14:47.728 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.728 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.728 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:14:47.728 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:35.304962+0000", 2024-09-18T10:14:47.728 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.005507835 2024-09-18T10:14:47.729 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.729 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.729 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:14:47.729 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:35.305033+0000", 2024-09-18T10:14:47.729 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.0561000000000004e-05 2024-09-18T10:14:47.729 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:47.729 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:14:47.729 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:47.729 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.729 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.729 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4257.0:48 1.0 e26/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 26'30, mlcod=26'22)", 2024-09-18T10:14:47.729 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:35.294144+0000", 2024-09-18T10:14:47.730 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 12.408407950000001, 2024-09-18T10:14:47.730 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20346468400000001, 2024-09-18T10:14:47.730 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:14:47.730 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:14:47.730 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:14:47.730 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.730 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:14:47.730 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:35.294144+0000", 2024-09-18T10:14:47.730 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:47.730 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.730 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.730 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:14:47.730 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:35.294144+0000", 2024-09-18T10:14:47.731 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:47.731 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.731 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.731 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:14:47.731 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:35.294142+0000", 2024-09-18T10:14:47.731 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:14:47.731 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.731 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.731 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:14:47.731 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:35.294156+0000", 2024-09-18T10:14:47.731 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4059e-05 2024-09-18T10:14:47.731 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.731 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.731 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:14:47.732 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:35.294209+0000", 2024-09-18T10:14:47.732 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.2164999999999999e-05 2024-09-18T10:14:47.732 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.732 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.732 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:14:47.732 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:35.394305+0000", 2024-09-18T10:14:47.732 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10009605000000001 2024-09-18T10:14:47.732 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.732 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.732 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:14:47.732 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:35.394422+0000", 2024-09-18T10:14:47.732 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000117093 2024-09-18T10:14:47.733 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.733 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.733 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:14:47.733 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:35.394460+0000", 2024-09-18T10:14:47.733 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8037999999999997e-05 2024-09-18T10:14:47.733 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.733 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.733 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:14:47.733 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:35.497437+0000", 2024-09-18T10:14:47.733 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.102977206 2024-09-18T10:14:47.733 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.733 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.733 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:14:47.734 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:35.497609+0000", 2024-09-18T10:14:47.734 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00017165299999999999 2024-09-18T10:14:47.734 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:47.734 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:14:47.734 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:47.734 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.734 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.734 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4257.0:53 1.0 e27/23 1:30306672:devicehealth::main.db.0000000000000000:head v 27'35, mlcod=26'33)", 2024-09-18T10:14:47.734 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:35.908908+0000", 2024-09-18T10:14:47.734 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 11.793643641999999, 2024-09-18T10:14:47.734 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.104649954, 2024-09-18T10:14:47.734 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:14:47.734 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:14:47.734 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:14:47.735 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.735 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:14:47.735 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:35.908908+0000", 2024-09-18T10:14:47.735 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:47.735 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.735 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.735 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:14:47.735 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:35.908908+0000", 2024-09-18T10:14:47.735 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:47.735 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.735 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.735 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:14:47.735 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:35.908907+0000", 2024-09-18T10:14:47.736 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:14:47.736 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.736 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.736 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:14:47.736 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:35.909026+0000", 2024-09-18T10:14:47.736 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00011937799999999999 2024-09-18T10:14:47.736 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.736 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.736 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:14:47.736 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:35.909059+0000", 2024-09-18T10:14:47.736 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3225999999999997e-05 2024-09-18T10:14:47.736 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.737 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.737 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:14:47.737 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:35.909071+0000", 2024-09-18T10:14:47.737 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1513e-05 2024-09-18T10:14:47.737 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.737 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.737 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:14:47.737 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:36.010557+0000", 2024-09-18T10:14:47.737 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.101485779 2024-09-18T10:14:47.737 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.737 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.737 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:14:47.737 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:36.010598+0000", 2024-09-18T10:14:47.738 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1174999999999998e-05 2024-09-18T10:14:47.738 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.738 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.738 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:14:47.738 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:36.013484+0000", 2024-09-18T10:14:47.738 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.002886142 2024-09-18T10:14:47.738 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.738 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.738 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:14:47.738 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:36.013558+0000", 2024-09-18T10:14:47.738 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.4376999999999996e-05 2024-09-18T10:14:47.738 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:47.738 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:14:47.739 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:47.739 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.739 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.739 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4257.0:78 1.0 e28/23 1:30306672:devicehealth::main.db.0000000000000000:head v 28'56, mlcod=28'53)", 2024-09-18T10:14:47.739 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:37.602756+0000", 2024-09-18T10:14:47.739 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 10.099795507, 2024-09-18T10:14:47.739 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10501774799999999, 2024-09-18T10:14:47.739 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:14:47.739 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:14:47.739 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:14:47.739 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.739 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:14:47.740 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:37.602756+0000", 2024-09-18T10:14:47.740 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:47.740 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.740 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.740 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:14:47.740 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:37.602756+0000", 2024-09-18T10:14:47.740 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:47.740 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.740 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.740 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:14:47.740 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:37.602755+0000", 2024-09-18T10:14:47.740 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:14:47.741 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.741 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.741 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:14:47.741 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:37.602876+0000", 2024-09-18T10:14:47.741 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000120826 2024-09-18T10:14:47.741 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.741 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.741 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:14:47.741 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:37.602900+0000", 2024-09-18T10:14:47.741 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4283000000000001e-05 2024-09-18T10:14:47.741 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.741 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.741 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:14:47.742 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:37.602910+0000", 2024-09-18T10:14:47.742 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.6530000000000006e-06 2024-09-18T10:14:47.742 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.742 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.742 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:14:47.742 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:37.705673+0000", 2024-09-18T10:14:47.742 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10276343 2024-09-18T10:14:47.742 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.742 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.742 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:14:47.742 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:37.705707+0000", 2024-09-18T10:14:47.742 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3847999999999998e-05 2024-09-18T10:14:47.742 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.743 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.743 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:14:47.743 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:37.707673+0000", 2024-09-18T10:14:47.743 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001966059 2024-09-18T10:14:47.743 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.743 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.743 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:14:47.743 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:37.707774+0000", 2024-09-18T10:14:47.743 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000100986 2024-09-18T10:14:47.743 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:47.743 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:14:47.743 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:47.743 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.743 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.744 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4257.0:96 1.0 e29/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 29'70, mlcod=29'68)", 2024-09-18T10:14:47.744 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:39.427232+0000", 2024-09-18T10:14:47.744 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 8.2753198260000005, 2024-09-18T10:14:47.744 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20408627300000001, 2024-09-18T10:14:47.744 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:14:47.744 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:14:47.744 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:14:47.744 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.744 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:14:47.744 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.427232+0000", 2024-09-18T10:14:47.744 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:47.744 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.744 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.745 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:14:47.745 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.427232+0000", 2024-09-18T10:14:47.745 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:47.745 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.745 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.745 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:14:47.745 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.427230+0000", 2024-09-18T10:14:47.745 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:14:47.745 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.745 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.745 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:14:47.745 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.427399+0000", 2024-09-18T10:14:47.745 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000168787 2024-09-18T10:14:47.745 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.746 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.746 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:14:47.746 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.427431+0000", 2024-09-18T10:14:47.746 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2372000000000001e-05 2024-09-18T10:14:47.746 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.746 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.746 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:14:47.746 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.427444+0000", 2024-09-18T10:14:47.746 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2303000000000001e-05 2024-09-18T10:14:47.746 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.746 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.746 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:14:47.746 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.427580+0000", 2024-09-18T10:14:47.747 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000136019 2024-09-18T10:14:47.747 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.747 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.747 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:14:47.747 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.427622+0000", 2024-09-18T10:14:47.747 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.2551e-05 2024-09-18T10:14:47.747 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.747 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.747 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:14:47.747 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.631238+0000", 2024-09-18T10:14:47.747 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20361580100000001 2024-09-18T10:14:47.747 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.747 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.747 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:14:47.748 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.631318+0000", 2024-09-18T10:14:47.748 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.0370000000000005e-05 2024-09-18T10:14:47.748 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:47.748 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:14:47.748 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:47.748 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.748 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.748 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4257.0:97 1.0 e29/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 29'71, mlcod=29'68)", 2024-09-18T10:14:47.748 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:39.428800+0000", 2024-09-18T10:14:47.748 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 8.2737518839999993, 2024-09-18T10:14:47.748 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.202566529, 2024-09-18T10:14:47.748 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:14:47.748 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:14:47.749 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:14:47.749 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.749 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:14:47.749 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.428800+0000", 2024-09-18T10:14:47.749 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:47.749 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.749 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.749 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:14:47.749 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.428800+0000", 2024-09-18T10:14:47.749 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:47.749 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.749 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.749 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:14:47.749 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.428798+0000", 2024-09-18T10:14:47.750 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:14:47.750 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.750 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.750 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:14:47.750 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.428814+0000", 2024-09-18T10:14:47.750 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5262999999999999e-05 2024-09-18T10:14:47.750 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.750 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.750 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:14:47.750 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.428836+0000", 2024-09-18T10:14:47.750 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.2096e-05 2024-09-18T10:14:47.750 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.750 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.750 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:14:47.751 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.428847+0000", 2024-09-18T10:14:47.751 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1272e-05 2024-09-18T10:14:47.751 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.751 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.751 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:14:47.751 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.529081+0000", 2024-09-18T10:14:47.751 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100233853 2024-09-18T10:14:47.751 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.751 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.751 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:14:47.751 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.529115+0000", 2024-09-18T10:14:47.751 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3862000000000002e-05 2024-09-18T10:14:47.751 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.752 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.752 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:14:47.752 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.631329+0000", 2024-09-18T10:14:47.752 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.102214465 2024-09-18T10:14:47.752 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.752 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.752 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:14:47.752 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.631367+0000", 2024-09-18T10:14:47.752 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7450999999999998e-05 2024-09-18T10:14:47.752 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:47.752 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:14:47.752 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:47.752 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.753 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.753 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4257.0:98 1.0 e29/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 29'72, mlcod=29'68)", 2024-09-18T10:14:47.753 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:39.429995+0000", 2024-09-18T10:14:47.753 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 8.2725566609999994, 2024-09-18T10:14:47.753 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20141884500000001, 2024-09-18T10:14:47.753 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:14:47.753 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:14:47.753 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:14:47.753 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.753 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:14:47.753 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.429995+0000", 2024-09-18T10:14:47.753 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:47.753 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.754 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.754 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:14:47.754 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.429995+0000", 2024-09-18T10:14:47.754 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:47.754 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.754 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.754 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:14:47.754 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.429994+0000", 2024-09-18T10:14:47.754 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:14:47.754 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.754 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.754 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:14:47.754 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.430008+0000", 2024-09-18T10:14:47.754 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4185999999999999e-05 2024-09-18T10:14:47.755 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.755 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.755 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:14:47.755 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.430036+0000", 2024-09-18T10:14:47.755 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7878999999999999e-05 2024-09-18T10:14:47.755 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.755 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.755 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:14:47.755 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.430048+0000", 2024-09-18T10:14:47.755 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1725999999999999e-05 2024-09-18T10:14:47.755 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.755 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.755 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:14:47.755 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.629731+0000", 2024-09-18T10:14:47.756 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.19968341100000001 2024-09-18T10:14:47.756 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.756 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.756 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:14:47.756 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.629764+0000", 2024-09-18T10:14:47.756 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3572999999999999e-05 2024-09-18T10:14:47.756 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.756 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.756 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:14:47.756 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.631372+0000", 2024-09-18T10:14:47.756 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0016074279999999999 2024-09-18T10:14:47.756 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.756 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.757 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:14:47.757 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.631414+0000", 2024-09-18T10:14:47.757 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.2191999999999999e-05 2024-09-18T10:14:47.757 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:47.757 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:14:47.757 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:47.757 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.757 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.757 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4257.0:99 1.0 e29/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 29'73, mlcod=29'68)", 2024-09-18T10:14:47.757 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:39.431146+0000", 2024-09-18T10:14:47.757 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 8.2714059370000008, 2024-09-18T10:14:47.757 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.200928938, 2024-09-18T10:14:47.757 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:14:47.758 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:14:47.758 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:14:47.758 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.758 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:14:47.758 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.431146+0000", 2024-09-18T10:14:47.758 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:47.758 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.758 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.758 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:14:47.758 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.431146+0000", 2024-09-18T10:14:47.758 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:47.758 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.758 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.758 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:14:47.759 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.431144+0000", 2024-09-18T10:14:47.759 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:14:47.759 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.759 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.759 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:14:47.759 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.431218+0000", 2024-09-18T10:14:47.759 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.3577000000000004e-05 2024-09-18T10:14:47.759 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.759 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.759 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:14:47.759 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.431250+0000", 2024-09-18T10:14:47.759 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2268999999999999e-05 2024-09-18T10:14:47.759 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.760 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.760 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:14:47.760 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.431262+0000", 2024-09-18T10:14:47.760 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2053e-05 2024-09-18T10:14:47.760 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.760 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.760 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:14:47.760 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.630328+0000", 2024-09-18T10:14:47.760 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.19906532900000001 2024-09-18T10:14:47.760 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.760 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.760 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:14:47.760 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.630359+0000", 2024-09-18T10:14:47.761 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1606000000000002e-05 2024-09-18T10:14:47.761 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.761 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.761 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:14:47.761 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.632015+0000", 2024-09-18T10:14:47.761 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0016557900000000001 2024-09-18T10:14:47.761 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.761 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.761 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:14:47.761 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.632075+0000", 2024-09-18T10:14:47.761 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9904000000000001e-05 2024-09-18T10:14:47.761 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:47.761 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:14:47.762 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:47.762 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.762 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.762 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4257.0:100 1.0 e29/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 29'74, mlcod=29'68)", 2024-09-18T10:14:47.762 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:39.432631+0000", 2024-09-18T10:14:47.762 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 8.2699208479999999, 2024-09-18T10:14:47.762 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20014804899999999, 2024-09-18T10:14:47.762 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:14:47.762 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:14:47.762 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:14:47.762 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.762 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:14:47.762 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.432631+0000", 2024-09-18T10:14:47.762 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:47.763 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.763 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.763 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:14:47.763 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.432631+0000", 2024-09-18T10:14:47.763 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:47.763 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.763 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.763 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:14:47.763 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.432630+0000", 2024-09-18T10:14:47.763 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:14:47.763 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.763 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.763 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:14:47.763 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.432643+0000", 2024-09-18T10:14:47.764 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3525e-05 2024-09-18T10:14:47.764 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.764 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.764 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:14:47.764 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.432674+0000", 2024-09-18T10:14:47.764 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1183000000000003e-05 2024-09-18T10:14:47.764 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.764 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.764 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:14:47.764 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.432690+0000", 2024-09-18T10:14:47.764 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5665999999999999e-05 2024-09-18T10:14:47.764 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.764 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.765 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:14:47.765 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.631457+0000", 2024-09-18T10:14:47.765 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.19876732899999999 2024-09-18T10:14:47.765 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.765 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.765 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:14:47.765 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.631488+0000", 2024-09-18T10:14:47.765 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1059000000000002e-05 2024-09-18T10:14:47.765 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.766 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.766 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:14:47.766 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.632717+0000", 2024-09-18T10:14:47.766 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0012285219999999999 2024-09-18T10:14:47.766 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.766 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.766 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:14:47.766 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.632779+0000", 2024-09-18T10:14:47.766 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.2328000000000004e-05 2024-09-18T10:14:47.766 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:47.766 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:14:47.766 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:47.766 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.766 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.767 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4257.0:103 1.0 e29/23 1:30306672:devicehealth::main.db.0000000000000000:head v 29'77, mlcod=29'73)", 2024-09-18T10:14:47.767 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:40.036738+0000", 2024-09-18T10:14:47.767 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 7.6658138779999998, 2024-09-18T10:14:47.767 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20132596899999999, 2024-09-18T10:14:47.767 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:14:47.767 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:14:47.767 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:14:47.767 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.767 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:14:47.767 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.036738+0000", 2024-09-18T10:14:47.767 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:47.767 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.767 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.768 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:14:47.768 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.036738+0000", 2024-09-18T10:14:47.768 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:47.768 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.768 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.768 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:14:47.768 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.036737+0000", 2024-09-18T10:14:47.768 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:14:47.768 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.768 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.768 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:14:47.768 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.036804+0000", 2024-09-18T10:14:47.768 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.7178e-05 2024-09-18T10:14:47.769 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.769 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.769 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:14:47.769 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.036821+0000", 2024-09-18T10:14:47.769 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7153e-05 2024-09-18T10:14:47.769 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.769 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.769 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:14:47.769 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.136954+0000", 2024-09-18T10:14:47.769 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100132572 2024-09-18T10:14:47.769 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.769 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.769 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:14:47.770 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.137090+0000", 2024-09-18T10:14:47.770 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000136332 2024-09-18T10:14:47.770 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.770 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.770 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:14:47.770 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.137124+0000", 2024-09-18T10:14:47.770 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3458999999999999e-05 2024-09-18T10:14:47.770 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.770 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.770 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:14:47.770 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.237999+0000", 2024-09-18T10:14:47.770 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100875385 2024-09-18T10:14:47.770 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.770 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.771 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:14:47.771 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.238064+0000", 2024-09-18T10:14:47.771 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.5086999999999995e-05 2024-09-18T10:14:47.771 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:47.771 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:14:47.771 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:47.771 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.771 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.771 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4257.0:113 1.0 e29/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 29'83, mlcod=29'80)", 2024-09-18T10:14:47.771 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:40.801786+0000", 2024-09-18T10:14:47.771 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 6.9007663739999998, 2024-09-18T10:14:47.772 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.203408961, 2024-09-18T10:14:47.772 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:14:47.772 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:14:47.772 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:14:47.772 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.772 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:14:47.772 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.801786+0000", 2024-09-18T10:14:47.772 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:47.772 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.772 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.773 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:14:47.773 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.801786+0000", 2024-09-18T10:14:47.773 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:47.773 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.773 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.773 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:14:47.773 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.801784+0000", 2024-09-18T10:14:47.773 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:14:47.773 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.773 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.774 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:14:47.774 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.801801+0000", 2024-09-18T10:14:47.774 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7223000000000001e-05 2024-09-18T10:14:47.774 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.774 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.774 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:14:47.774 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.801823+0000", 2024-09-18T10:14:47.774 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1469000000000001e-05 2024-09-18T10:14:47.774 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.774 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.774 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:14:47.774 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.801835+0000", 2024-09-18T10:14:47.774 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2033e-05 2024-09-18T10:14:47.775 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.775 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.775 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:14:47.775 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.902049+0000", 2024-09-18T10:14:47.775 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100213659 2024-09-18T10:14:47.775 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.775 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.775 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:14:47.775 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.902082+0000", 2024-09-18T10:14:47.775 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3668999999999999e-05 2024-09-18T10:14:47.775 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.775 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.776 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:14:47.776 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.005131+0000", 2024-09-18T10:14:47.776 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.103048379 2024-09-18T10:14:47.776 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.776 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.776 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:14:47.776 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.005195+0000", 2024-09-18T10:14:47.776 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.3805000000000003e-05 2024-09-18T10:14:47.776 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:47.776 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:14:47.777 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:47.777 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.777 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.777 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4257.0:114 1.0 e29/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 29'84, mlcod=29'80)", 2024-09-18T10:14:47.777 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:40.802657+0000", 2024-09-18T10:14:47.777 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 6.8998952400000002, 2024-09-18T10:14:47.777 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20256802600000001, 2024-09-18T10:14:47.777 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:14:47.777 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:14:47.777 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:14:47.777 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.777 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:14:47.777 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.802657+0000", 2024-09-18T10:14:47.777 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:47.778 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.778 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.778 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:14:47.778 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.802657+0000", 2024-09-18T10:14:47.778 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:47.778 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.778 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.778 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:14:47.778 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.802656+0000", 2024-09-18T10:14:47.778 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:14:47.778 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.778 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.778 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:14:47.779 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.802824+0000", 2024-09-18T10:14:47.779 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000168784 2024-09-18T10:14:47.779 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.779 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.779 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:14:47.779 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.802844+0000", 2024-09-18T10:14:47.779 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0058999999999999e-05 2024-09-18T10:14:47.779 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.779 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.779 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:14:47.779 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.802857+0000", 2024-09-18T10:14:47.779 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2163e-05 2024-09-18T10:14:47.780 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.780 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.780 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:14:47.780 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.902531+0000", 2024-09-18T10:14:47.780 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.099674591000000007 2024-09-18T10:14:47.780 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.780 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.780 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:14:47.780 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.902559+0000", 2024-09-18T10:14:47.780 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7483000000000001e-05 2024-09-18T10:14:47.780 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.780 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.780 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:14:47.780 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.005198+0000", 2024-09-18T10:14:47.781 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.102638853 2024-09-18T10:14:47.781 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.781 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.781 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:14:47.781 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.005225+0000", 2024-09-18T10:14:47.781 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7206e-05 2024-09-18T10:14:47.781 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:47.781 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:14:47.781 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:47.781 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.781 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.781 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4257.0:115 1.0 e29/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 29'85, mlcod=29'80)", 2024-09-18T10:14:47.782 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:40.803806+0000", 2024-09-18T10:14:47.782 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 6.8987458530000003, 2024-09-18T10:14:47.782 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.201443711, 2024-09-18T10:14:47.782 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:14:47.782 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:14:47.782 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:14:47.782 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.782 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:14:47.782 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.803806+0000", 2024-09-18T10:14:47.782 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:47.782 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.782 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.782 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:14:47.783 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.803806+0000", 2024-09-18T10:14:47.783 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:47.783 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.783 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.783 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:14:47.783 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.803804+0000", 2024-09-18T10:14:47.783 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:14:47.783 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.783 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.783 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:14:47.783 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.803818+0000", 2024-09-18T10:14:47.783 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3477e-05 2024-09-18T10:14:47.783 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.783 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.784 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:14:47.784 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.803851+0000", 2024-09-18T10:14:47.784 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3086000000000001e-05 2024-09-18T10:14:47.784 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.784 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.784 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:14:47.784 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.803867+0000", 2024-09-18T10:14:47.784 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5885000000000001e-05 2024-09-18T10:14:47.784 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.784 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.784 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:14:47.784 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.903206+0000", 2024-09-18T10:14:47.784 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.099338990000000002 2024-09-18T10:14:47.785 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.785 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.785 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:14:47.785 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.903253+0000", 2024-09-18T10:14:47.785 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.7419000000000001e-05 2024-09-18T10:14:47.785 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.785 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.785 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:14:47.785 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.005227+0000", 2024-09-18T10:14:47.785 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.101973199 2024-09-18T10:14:47.785 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.785 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.785 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:14:47.786 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.005250+0000", 2024-09-18T10:14:47.786 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3261999999999999e-05 2024-09-18T10:14:47.786 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:47.786 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:14:47.786 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:47.786 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.786 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.786 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4257.0:116 1.0 e29/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 29'86, mlcod=29'80)", 2024-09-18T10:14:47.786 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:40.804730+0000", 2024-09-18T10:14:47.786 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 6.8978218739999999, 2024-09-18T10:14:47.786 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20068456000000001, 2024-09-18T10:14:47.786 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:14:47.786 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:14:47.786 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:14:47.787 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.787 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:14:47.787 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.804730+0000", 2024-09-18T10:14:47.787 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:47.787 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.787 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.787 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:14:47.787 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.804730+0000", 2024-09-18T10:14:47.787 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:47.787 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.787 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.787 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:14:47.787 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.804729+0000", 2024-09-18T10:14:47.788 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:14:47.788 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.788 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.788 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:14:47.788 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.804739+0000", 2024-09-18T10:14:47.788 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0669000000000001e-05 2024-09-18T10:14:47.788 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.788 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.788 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:14:47.788 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.804762+0000", 2024-09-18T10:14:47.788 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3062999999999999e-05 2024-09-18T10:14:47.788 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.788 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.788 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:14:47.789 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.804776+0000", 2024-09-18T10:14:47.789 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3273e-05 2024-09-18T10:14:47.789 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.789 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.789 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:14:47.789 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.003889+0000", 2024-09-18T10:14:47.789 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.19911304099999999 2024-09-18T10:14:47.789 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.789 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.789 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:14:47.789 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.003922+0000", 2024-09-18T10:14:47.789 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3621999999999998e-05 2024-09-18T10:14:47.789 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.790 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.790 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:14:47.790 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.005347+0000", 2024-09-18T10:14:47.790 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0014251050000000001 2024-09-18T10:14:47.790 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.790 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.790 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:14:47.790 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.005415+0000", 2024-09-18T10:14:47.790 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.7154000000000001e-05 2024-09-18T10:14:47.790 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:47.790 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:14:47.790 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:47.790 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.790 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.791 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4257.0:117 1.0 e29/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 29'87, mlcod=29'80)", 2024-09-18T10:14:47.791 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:40.805727+0000", 2024-09-18T10:14:47.791 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 6.8968246149999999, 2024-09-18T10:14:47.791 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.200946191, 2024-09-18T10:14:47.791 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:14:47.791 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:14:47.791 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:14:47.791 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.791 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:14:47.791 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.805727+0000", 2024-09-18T10:14:47.791 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:47.791 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.791 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.792 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:14:47.792 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.805727+0000", 2024-09-18T10:14:47.792 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:47.792 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.792 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.792 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:14:47.792 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.805726+0000", 2024-09-18T10:14:47.792 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:14:47.792 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.792 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.792 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:14:47.792 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.805783+0000", 2024-09-18T10:14:47.792 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.7068000000000003e-05 2024-09-18T10:14:47.792 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.793 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.793 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:14:47.793 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.805817+0000", 2024-09-18T10:14:47.793 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4471999999999998e-05 2024-09-18T10:14:47.793 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.793 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.793 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:14:47.793 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.805833+0000", 2024-09-18T10:14:47.793 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6313000000000001e-05 2024-09-18T10:14:47.793 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.793 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.793 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:14:47.793 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.004481+0000", 2024-09-18T10:14:47.794 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.198647823 2024-09-18T10:14:47.794 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.794 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.794 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:14:47.794 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.004520+0000", 2024-09-18T10:14:47.794 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8992000000000002e-05 2024-09-18T10:14:47.794 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.794 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.794 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:14:47.794 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.006585+0000", 2024-09-18T10:14:47.794 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0020647389999999999 2024-09-18T10:14:47.794 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.794 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.794 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:14:47.795 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.006673+0000", 2024-09-18T10:14:47.795 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.8443999999999995e-05 2024-09-18T10:14:47.795 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:47.795 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:14:47.795 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:47.795 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.795 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.795 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4257.0:128 1.0 e29/23 1:30306672:devicehealth::main.db.0000000000000000:head v 29'98, mlcod=29'95)", 2024-09-18T10:14:47.795 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:41.222806+0000", 2024-09-18T10:14:47.795 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 6.4797459740000001, 2024-09-18T10:14:47.795 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20200015099999999, 2024-09-18T10:14:47.795 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:14:47.795 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:14:47.795 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:14:47.796 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.796 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:14:47.796 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.222806+0000", 2024-09-18T10:14:47.796 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:47.796 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.796 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.796 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:14:47.796 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.222806+0000", 2024-09-18T10:14:47.796 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:47.796 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.796 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.796 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:14:47.797 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.222804+0000", 2024-09-18T10:14:47.797 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:14:47.797 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.797 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.797 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:14:47.797 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.222885+0000", 2024-09-18T10:14:47.797 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.0140000000000002e-05 2024-09-18T10:14:47.797 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.797 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.797 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:14:47.797 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.222915+0000", 2024-09-18T10:14:47.797 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0168999999999999e-05 2024-09-18T10:14:47.797 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.797 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.798 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:14:47.798 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.323060+0000", 2024-09-18T10:14:47.798 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100145475 2024-09-18T10:14:47.798 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.798 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.798 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:14:47.798 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.423304+0000", 2024-09-18T10:14:47.798 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100243924 2024-09-18T10:14:47.798 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.798 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.798 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:14:47.798 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.423338+0000", 2024-09-18T10:14:47.798 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3819000000000003e-05 2024-09-18T10:14:47.798 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.799 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.799 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:14:47.799 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.424724+0000", 2024-09-18T10:14:47.799 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0013862690000000001 2024-09-18T10:14:47.799 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:47.799 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:47.799 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:14:47.799 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.424806+0000", 2024-09-18T10:14:47.799 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.1901000000000001e-05 2024-09-18T10:14:47.799 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:47.799 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:14:47.799 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:47.799 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:47.799 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:14:47.800 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-18T10:14:47.801 INFO:teuthology.orchestra.run.smithi029.stdout:osd.7: {} 2024-09-18T10:14:47.802 INFO:teuthology.orchestra.run.smithi029.stdout:osd.8: {} 2024-09-18T10:14:47.802 INFO:teuthology.orchestra.run.smithi029.stdout:osd.9: {} 2024-09-18T10:14:47.802 INFO:teuthology.orchestra.run.smithi029.stdout:osd.10: {} 2024-09-18T10:14:47.802 INFO:teuthology.orchestra.run.smithi029.stdout:osd.11: {} 2024-09-18T10:14:47.802 INFO:tasks.ceph.osd.12.smithi029.stderr:2024-09-18T10:14:47.798+0000 7f8e4e627700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 22190) UID: 0 2024-09-18T10:14:47.802 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T10:14:47.818 INFO:teuthology.orchestra.run.smithi029.stdout:osd.12: {} 2024-09-18T10:14:47.840 INFO:teuthology.orchestra.run.smithi029.stdout:osd.13: {} 2024-09-18T10:14:47.861 INFO:teuthology.orchestra.run.smithi029.stdout:osd.14: {} 2024-09-18T10:14:47.884 INFO:teuthology.orchestra.run.smithi029.stdout:osd.15: {} 2024-09-18T10:14:47.887 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T10:14:47.887 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T10:14:47.887 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T10:14:47.887 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T10:14:47.888 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T10:14:47.888 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T10:14:47.888 INFO:teuthology.orchestra.run.smithi029.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T10:14:47.888 INFO:teuthology.orchestra.run.smithi029.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T10:14:47.888 INFO:teuthology.orchestra.run.smithi029.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-18T10:14:47.888 INFO:teuthology.orchestra.run.smithi029.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-18T10:14:47.888 INFO:teuthology.orchestra.run.smithi029.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-18T10:14:47.888 INFO:teuthology.orchestra.run.smithi029.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-18T10:14:47.888 INFO:teuthology.orchestra.run.smithi029.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-18T10:14:47.888 INFO:teuthology.orchestra.run.smithi029.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-18T10:14:47.888 INFO:teuthology.orchestra.run.smithi029.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-18T10:14:47.888 INFO:teuthology.orchestra.run.smithi029.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-18T10:14:47.902 INFO:tasks.ceph.osd.14.smithi154.stderr:2024-09-18T10:14:47.896+0000 7f75a19e7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 22093) UID: 0 2024-09-18T10:14:47.903 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:14:47.903 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:14:47.912 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T10:14:48.003 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-09-18T10:14:48.000+0000 7fdfa8698700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22087) UID: 0 2024-09-18T10:14:48.007 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:14:48.007 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:14:48.016 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T10:14:48.104 INFO:tasks.ceph.osd.10.smithi154.stderr:2024-09-18T10:14:48.100+0000 7fb73221a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 22091) UID: 0 2024-09-18T10:14:48.115 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:14:48.116 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:14:48.125 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T10:14:48.205 INFO:tasks.ceph.osd.3.smithi189.stderr:2024-09-18T10:14:48.203+0000 7f2440fc7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22396) UID: 0 2024-09-18T10:14:48.225 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:14:48.225 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:14:48.234 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T10:14:48.306 INFO:tasks.ceph.osd.11.smithi189.stderr:2024-09-18T10:14:48.303+0000 7f718b60a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 22400) UID: 0 2024-09-18T10:14:48.335 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:14:48.336 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:14:48.345 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T10:14:48.407 INFO:tasks.ceph.osd.3.smithi189.stderr:2024-09-18T10:14:48.403+0000 7f2440fc7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22396) UID: 0 2024-09-18T10:14:48.449 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:14:48.450 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:14:48.459 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-09-18T10:14:48.507 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-09-18T10:14:48.504+0000 7fdfa8698700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22087) UID: 0 2024-09-18T10:14:48.559 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:14:48.559 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:14:48.568 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-09-18T10:14:48.609 INFO:tasks.ceph.osd.11.smithi189.stderr:2024-09-18T10:14:48.607+0000 7f718b60a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 22400) UID: 0 2024-09-18T10:14:48.613 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-18T10:14:48.657 INFO:tasks.rados.rados.0.smithi189.stdout:1: write initial oid 1 2024-09-18T10:14:48.657 INFO:tasks.rados.rados.0.smithi189.stdout:1: seq_num 0 ranges {0=32768} 2024-09-18T10:14:48.658 INFO:tasks.rados.rados.0.smithi189.stdout:1: writing smithi18925896-1 from 0 to 32768 tid 1 2024-09-18T10:14:48.658 INFO:tasks.rados.rados.0.smithi189.stdout:2: write initial oid 2 2024-09-18T10:14:48.658 INFO:tasks.rados.rados.0.smithi189.stdout:2: seq_num 1 ranges {0=32768} 2024-09-18T10:14:48.659 INFO:tasks.rados.rados.0.smithi189.stdout:2: writing smithi18925896-2 from 0 to 32768 tid 1 2024-09-18T10:14:48.659 INFO:tasks.rados.rados.0.smithi189.stdout:3: write initial oid 3 2024-09-18T10:14:48.659 INFO:tasks.rados.rados.0.smithi189.stdout:3: seq_num 2 ranges {0=49152} 2024-09-18T10:14:48.659 INFO:tasks.rados.rados.0.smithi189.stdout:3: writing smithi18925896-3 from 0 to 49152 tid 1 2024-09-18T10:14:48.659 INFO:tasks.rados.rados.0.smithi189.stdout:4: write initial oid 4 2024-09-18T10:14:48.660 INFO:tasks.rados.rados.0.smithi189.stdout:4: seq_num 3 ranges {0=65536} 2024-09-18T10:14:48.661 INFO:tasks.rados.rados.0.smithi189.stdout:4: writing smithi18925896-4 from 0 to 65536 tid 1 2024-09-18T10:14:48.661 INFO:tasks.rados.rados.0.smithi189.stdout:5: write initial oid 5 2024-09-18T10:14:48.661 INFO:tasks.rados.rados.0.smithi189.stdout:5: seq_num 4 ranges {0=81920} 2024-09-18T10:14:48.663 INFO:tasks.rados.rados.0.smithi189.stdout:5: writing smithi18925896-5 from 0 to 81920 tid 1 2024-09-18T10:14:48.663 INFO:tasks.rados.rados.0.smithi189.stdout:6: write initial oid 6 2024-09-18T10:14:48.663 INFO:tasks.rados.rados.0.smithi189.stdout:6: seq_num 5 ranges {0=98304} 2024-09-18T10:14:48.664 INFO:tasks.rados.rados.0.smithi189.stdout:6: writing smithi18925896-6 from 0 to 98304 tid 1 2024-09-18T10:14:48.665 INFO:tasks.rados.rados.0.smithi189.stdout:7: write initial oid 7 2024-09-18T10:14:48.665 INFO:tasks.rados.rados.0.smithi189.stdout:7: seq_num 6 ranges {0=114688} 2024-09-18T10:14:48.667 INFO:tasks.rados.rados.0.smithi189.stdout:7: writing smithi18925896-7 from 0 to 114688 tid 1 2024-09-18T10:14:48.667 INFO:tasks.rados.rados.0.smithi189.stdout:8: write initial oid 8 2024-09-18T10:14:48.668 INFO:tasks.rados.rados.0.smithi189.stdout:8: seq_num 7 ranges {0=131072} 2024-09-18T10:14:48.670 INFO:tasks.rados.rados.0.smithi189.stdout:8: writing smithi18925896-8 from 0 to 131072 tid 1 2024-09-18T10:14:48.670 INFO:tasks.rados.rados.0.smithi189.stdout:9: write initial oid 9 2024-09-18T10:14:48.670 INFO:tasks.rados.rados.0.smithi189.stdout:9: seq_num 8 ranges {0=147456} 2024-09-18T10:14:48.673 INFO:tasks.rados.rados.0.smithi189.stdout:9: writing smithi18925896-9 from 0 to 147456 tid 1 2024-09-18T10:14:48.673 INFO:tasks.rados.rados.0.smithi189.stdout:10: write initial oid 10 2024-09-18T10:14:48.673 INFO:tasks.rados.rados.0.smithi189.stdout:10: seq_num 9 ranges {0=163840} 2024-09-18T10:14:48.674 INFO:tasks.rados.rados.0.smithi189.stdout:10: writing smithi18925896-10 from 0 to 163840 tid 1 2024-09-18T10:14:48.674 INFO:tasks.rados.rados.0.smithi189.stdout:11: write initial oid 11 2024-09-18T10:14:48.674 INFO:tasks.rados.rados.0.smithi189.stdout:11: seq_num 10 ranges {0=180224} 2024-09-18T10:14:48.675 INFO:tasks.rados.rados.0.smithi189.stdout:11: writing smithi18925896-11 from 0 to 180224 tid 1 2024-09-18T10:14:48.675 INFO:tasks.rados.rados.0.smithi189.stdout:12: write initial oid 12 2024-09-18T10:14:48.675 INFO:tasks.rados.rados.0.smithi189.stdout:12: seq_num 11 ranges {0=196608} 2024-09-18T10:14:48.676 INFO:tasks.rados.rados.0.smithi189.stdout:12: writing smithi18925896-12 from 0 to 196608 tid 1 2024-09-18T10:14:48.676 INFO:tasks.rados.rados.0.smithi189.stdout:13: write initial oid 13 2024-09-18T10:14:48.676 INFO:tasks.rados.rados.0.smithi189.stdout:13: seq_num 12 ranges {0=212992} 2024-09-18T10:14:48.677 INFO:tasks.rados.rados.0.smithi189.stdout:13: writing smithi18925896-13 from 0 to 212992 tid 1 2024-09-18T10:14:48.678 INFO:tasks.rados.rados.0.smithi189.stdout:14: write initial oid 14 2024-09-18T10:14:48.678 INFO:tasks.rados.rados.0.smithi189.stdout:14: seq_num 13 ranges {0=229376} 2024-09-18T10:14:48.679 INFO:tasks.rados.rados.0.smithi189.stdout:14: writing smithi18925896-14 from 0 to 229376 tid 1 2024-09-18T10:14:48.679 INFO:tasks.rados.rados.0.smithi189.stdout:15: write initial oid 15 2024-09-18T10:14:48.679 INFO:tasks.rados.rados.0.smithi189.stdout:15: seq_num 14 ranges {0=245760} 2024-09-18T10:14:48.681 INFO:tasks.rados.rados.0.smithi189.stdout:15: writing smithi18925896-15 from 0 to 245760 tid 1 2024-09-18T10:14:48.681 INFO:tasks.rados.rados.0.smithi189.stdout:16: write initial oid 16 2024-09-18T10:14:48.681 INFO:tasks.rados.rados.0.smithi189.stdout:16: seq_num 15 ranges {0=262144} 2024-09-18T10:14:48.681 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:14:48.681 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:14:48.682 INFO:tasks.rados.rados.0.smithi189.stdout:16: writing smithi18925896-16 from 0 to 262144 tid 1 2024-09-18T10:14:48.682 INFO:tasks.rados.rados.0.smithi189.stdout: waiting on 16 2024-09-18T10:14:48.691 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-09-18T10:14:48.710 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-18T10:14:48.706+0000 7f00046a8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 22136) UID: 0 2024-09-18T10:14:48.790 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:14:48.790 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:14:48.800 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_ops_in_flight 2024-09-18T10:14:48.810 INFO:tasks.ceph.osd.10.smithi154.stderr:2024-09-18T10:14:48.804+0000 7fb73221a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 22091) UID: 0 2024-09-18T10:14:48.867 INFO:tasks.rados.rados.0.smithi189.stdout:3: finishing write tid 1 to smithi18925896-3 2024-09-18T10:14:48.901 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:14:48.901 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:14:48.910 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_blocked_ops 2024-09-18T10:14:48.912 INFO:tasks.ceph.osd.3.smithi189.stderr:2024-09-18T10:14:48.907+0000 7f2440fc7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22396) UID: 0 2024-09-18T10:14:48.964 INFO:tasks.rados.rados.0.smithi189.stdout:1: finishing write tid 1 to smithi18925896-1 2024-09-18T10:14:48.965 INFO:tasks.rados.rados.0.smithi189.stdout:1: finishing write tid 2 to smithi18925896-1 2024-09-18T10:14:48.971 INFO:tasks.rados.rados.0.smithi189.stdout:3: finishing write tid 2 to smithi18925896-3 2024-09-18T10:14:49.013 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-09-18T10:14:49.012+0000 7fdfa8698700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22087) UID: 0 2024-09-18T10:14:49.014 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:14:49.014 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:14:49.023 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_historic_ops 2024-09-18T10:14:49.062 INFO:tasks.rados.rados.0.smithi189.stdout:4: finishing write tid 1 to smithi18925896-4 2024-09-18T10:14:49.062 INFO:tasks.rados.rados.0.smithi189.stdout:6: finishing write tid 1 to smithi18925896-6 2024-09-18T10:14:49.062 INFO:tasks.rados.rados.0.smithi189.stdout:4: finishing write tid 2 to smithi18925896-4 2024-09-18T10:14:49.080 INFO:tasks.rados.rados.0.smithi189.stdout:5: finishing write tid 1 to smithi18925896-5 2024-09-18T10:14:49.114 INFO:tasks.ceph.osd.3.smithi189.stderr:2024-09-18T10:14:49.111+0000 7f2440fc7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22396) UID: 0 2024-09-18T10:14:49.120 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:14:49.120 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:14:49.130 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T10:14:49.162 INFO:tasks.rados.rados.0.smithi189.stdout:2: finishing write tid 1 to smithi18925896-2 2024-09-18T10:14:49.162 INFO:tasks.rados.rados.0.smithi189.stdout:2: finishing write tid 2 to smithi18925896-2 2024-09-18T10:14:49.163 INFO:tasks.rados.rados.0.smithi189.stdout:6: finishing write tid 2 to smithi18925896-6 2024-09-18T10:14:49.168 INFO:tasks.rados.rados.0.smithi189.stdout:1: finishing write tid 3 to smithi18925896-1 2024-09-18T10:14:49.168 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 1 v 2 (ObjNum 0 snap 0 seq_num 0) dirty exists 2024-09-18T10:14:49.168 INFO:tasks.rados.rados.0.smithi189.stdout:1: left oid 1 (ObjNum 0 snap 0 seq_num 0) 2024-09-18T10:14:49.168 INFO:tasks.rados.rados.0.smithi189.stdout:1: done (15 left) 2024-09-18T10:14:49.168 INFO:tasks.rados.rados.0.smithi189.stdout:17: write initial oid 17 2024-09-18T10:14:49.168 INFO:tasks.rados.rados.0.smithi189.stdout:17: seq_num 16 ranges {0=278528} 2024-09-18T10:14:49.173 INFO:tasks.rados.rados.0.smithi189.stdout:17: writing smithi18925896-17 from 0 to 278528 tid 1 2024-09-18T10:14:49.173 INFO:tasks.rados.rados.0.smithi189.stdout: waiting on 16 2024-09-18T10:14:49.173 INFO:tasks.rados.rados.0.smithi189.stdout:7: finishing write tid 1 to smithi18925896-7 2024-09-18T10:14:49.215 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-18T10:14:49.211+0000 7f14dcc5b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22398) UID: 0 2024-09-18T10:14:49.239 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-18T10:14:49.240 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:14:49.249 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T10:14:49.274 INFO:tasks.rados.rados.0.smithi189.stdout:7: finishing write tid 2 to smithi18925896-7 2024-09-18T10:14:49.274 INFO:tasks.rados.rados.0.smithi189.stdout:10: finishing write tid 1 to smithi18925896-10 2024-09-18T10:14:49.274 INFO:tasks.rados.rados.0.smithi189.stdout:10: finishing write tid 2 to smithi18925896-10 2024-09-18T10:14:49.275 INFO:tasks.rados.rados.0.smithi189.stdout:3: finishing write tid 3 to smithi18925896-3 2024-09-18T10:14:49.275 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 3 v 2 (ObjNum 2 snap 0 seq_num 2) dirty exists 2024-09-18T10:14:49.275 INFO:tasks.rados.rados.0.smithi189.stdout:3: left oid 3 (ObjNum 2 snap 0 seq_num 2) 2024-09-18T10:14:49.275 INFO:tasks.rados.rados.0.smithi189.stdout:3: done (15 left) 2024-09-18T10:14:49.275 INFO:tasks.rados.rados.0.smithi189.stdout:18: write initial oid 18 2024-09-18T10:14:49.275 INFO:tasks.rados.rados.0.smithi189.stdout:18: seq_num 17 ranges {0=294912} 2024-09-18T10:14:49.281 INFO:tasks.rados.rados.0.smithi189.stdout:18: writing smithi18925896-18 from 0 to 294912 tid 1 2024-09-18T10:14:49.281 INFO:tasks.rados.rados.0.smithi189.stdout: waiting on 16 2024-09-18T10:14:49.281 INFO:tasks.rados.rados.0.smithi189.stdout:14: finishing write tid 1 to smithi18925896-14 2024-09-18T10:14:49.281 INFO:tasks.rados.rados.0.smithi189.stdout:15: finishing write tid 1 to smithi18925896-15 2024-09-18T10:14:49.281 INFO:tasks.rados.rados.0.smithi189.stdout:14: finishing write tid 2 to smithi18925896-14 2024-09-18T10:14:49.281 INFO:tasks.rados.rados.0.smithi189.stdout:9: finishing write tid 1 to smithi18925896-9 2024-09-18T10:14:49.282 INFO:tasks.rados.rados.0.smithi189.stdout:5: finishing write tid 2 to smithi18925896-5 2024-09-18T10:14:49.317 INFO:tasks.ceph.osd.6.smithi154.stderr:2024-09-18T10:14:49.312+0000 7f0abc9e1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22089) UID: 0 2024-09-18T10:14:49.345 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-18T10:14:49.346 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:14:49.355 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T10:14:49.362 INFO:tasks.rados.rados.0.smithi189.stdout:16: finishing write tid 1 to smithi18925896-16 2024-09-18T10:14:49.365 INFO:tasks.rados.rados.0.smithi189.stdout:4: finishing write tid 3 to smithi18925896-4 2024-09-18T10:14:49.365 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 4 v 2 (ObjNum 3 snap 0 seq_num 3) dirty exists 2024-09-18T10:14:49.365 INFO:tasks.rados.rados.0.smithi189.stdout:4: left oid 4 (ObjNum 3 snap 0 seq_num 3) 2024-09-18T10:14:49.365 INFO:tasks.rados.rados.0.smithi189.stdout:4: done (15 left) 2024-09-18T10:14:49.365 INFO:tasks.rados.rados.0.smithi189.stdout:19: write initial oid 19 2024-09-18T10:14:49.365 INFO:tasks.rados.rados.0.smithi189.stdout:19: seq_num 18 ranges {0=311296} 2024-09-18T10:14:49.369 INFO:tasks.rados.rados.0.smithi189.stdout:19: writing smithi18925896-19 from 0 to 311296 tid 1 2024-09-18T10:14:49.369 INFO:tasks.rados.rados.0.smithi189.stdout: waiting on 16 2024-09-18T10:14:49.379 INFO:tasks.rados.rados.0.smithi189.stdout:8: finishing write tid 1 to smithi18925896-8 2024-09-18T10:14:49.379 INFO:tasks.rados.rados.0.smithi189.stdout:2: finishing write tid 3 to smithi18925896-2 2024-09-18T10:14:49.379 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 2 v 2 (ObjNum 1 snap 0 seq_num 1) dirty exists 2024-09-18T10:14:49.379 INFO:tasks.rados.rados.0.smithi189.stdout:2: left oid 2 (ObjNum 1 snap 0 seq_num 1) 2024-09-18T10:14:49.379 INFO:tasks.rados.rados.0.smithi189.stdout:2: done (15 left) 2024-09-18T10:14:49.379 INFO:tasks.rados.rados.0.smithi189.stdout:20: write initial oid 20 2024-09-18T10:14:49.379 INFO:tasks.rados.rados.0.smithi189.stdout:20: seq_num 19 ranges {0=327680} 2024-09-18T10:14:49.385 INFO:tasks.rados.rados.0.smithi189.stdout:20: writing smithi18925896-20 from 0 to 327680 tid 1 2024-09-18T10:14:49.385 INFO:tasks.rados.rados.0.smithi189.stdout: waiting on 16 2024-09-18T10:14:49.385 INFO:tasks.rados.rados.0.smithi189.stdout:8: finishing write tid 2 to smithi18925896-8 2024-09-18T10:14:49.385 INFO:tasks.rados.rados.0.smithi189.stdout:12: finishing write tid 1 to smithi18925896-12 2024-09-18T10:14:49.385 INFO:tasks.rados.rados.0.smithi189.stdout:12: finishing write tid 2 to smithi18925896-12 2024-09-18T10:14:49.385 INFO:tasks.rados.rados.0.smithi189.stdout:11: finishing write tid 1 to smithi18925896-11 2024-09-18T10:14:49.385 INFO:tasks.rados.rados.0.smithi189.stdout:10: finishing write tid 3 to smithi18925896-10 2024-09-18T10:14:49.385 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 10 v 2 (ObjNum 9 snap 0 seq_num 9) dirty exists 2024-09-18T10:14:49.386 INFO:tasks.rados.rados.0.smithi189.stdout:10: left oid 10 (ObjNum 9 snap 0 seq_num 9) 2024-09-18T10:14:49.386 INFO:tasks.rados.rados.0.smithi189.stdout:9: finishing write tid 2 to smithi18925896-9 2024-09-18T10:14:49.386 INFO:tasks.rados.rados.0.smithi189.stdout:10: done (15 left) 2024-09-18T10:14:49.386 INFO:tasks.rados.rados.0.smithi189.stdout:21: write initial oid 21 2024-09-18T10:14:49.386 INFO:tasks.rados.rados.0.smithi189.stdout:21: seq_num 20 ranges {0=344064} 2024-09-18T10:14:49.390 INFO:tasks.rados.rados.0.smithi189.stdout:21: writing smithi18925896-21 from 0 to 344064 tid 1 2024-09-18T10:14:49.390 INFO:tasks.rados.rados.0.smithi189.stdout: waiting on 16 2024-09-18T10:14:49.390 INFO:tasks.rados.rados.0.smithi189.stdout:13: finishing write tid 1 to smithi18925896-13 2024-09-18T10:14:49.418 INFO:tasks.ceph.osd.8.smithi029.stderr:2024-09-18T10:14:49.414+0000 7f2555c27700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 22192) UID: 0 2024-09-18T10:14:49.455 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-18T10:14:49.455 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:14:49.462 INFO:tasks.rados.rados.0.smithi189.stdout:16: finishing write tid 2 to smithi18925896-16 2024-09-18T10:14:49.465 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T10:14:49.476 INFO:tasks.rados.rados.0.smithi189.stdout:15: finishing write tid 2 to smithi18925896-15 2024-09-18T10:14:49.485 INFO:tasks.rados.rados.0.smithi189.stdout:6: finishing write tid 3 to smithi18925896-6 2024-09-18T10:14:49.485 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 6 v 2 (ObjNum 5 snap 0 seq_num 5) dirty exists 2024-09-18T10:14:49.485 INFO:tasks.rados.rados.0.smithi189.stdout:6: left oid 6 (ObjNum 5 snap 0 seq_num 5) 2024-09-18T10:14:49.485 INFO:tasks.rados.rados.0.smithi189.stdout:6: done (15 left) 2024-09-18T10:14:49.485 INFO:tasks.rados.rados.0.smithi189.stdout:22: write initial oid 22 2024-09-18T10:14:49.485 INFO:tasks.rados.rados.0.smithi189.stdout:22: seq_num 21 ranges {0=360448} 2024-09-18T10:14:49.491 INFO:tasks.rados.rados.0.smithi189.stdout:22: writing smithi18925896-22 from 0 to 360448 tid 1 2024-09-18T10:14:49.491 INFO:tasks.rados.rados.0.smithi189.stdout: waiting on 16 2024-09-18T10:14:49.495 INFO:tasks.rados.rados.0.smithi189.stdout:11: finishing write tid 2 to smithi18925896-11 2024-09-18T10:14:49.518 INFO:tasks.ceph.osd.3.smithi189.stderr:2024-09-18T10:14:49.515+0000 7f2440fc7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22396) UID: 0 2024-09-18T10:14:49.561 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-18T10:14:49.562 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:14:49.571 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T10:14:49.584 INFO:tasks.rados.rados.0.smithi189.stdout:13: finishing write tid 2 to smithi18925896-13 2024-09-18T10:14:49.586 INFO:tasks.rados.rados.0.smithi189.stdout:9: finishing write tid 3 to smithi18925896-9 2024-09-18T10:14:49.586 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 9 v 2 (ObjNum 8 snap 0 seq_num 8) dirty exists 2024-09-18T10:14:49.586 INFO:tasks.rados.rados.0.smithi189.stdout:9: left oid 9 (ObjNum 8 snap 0 seq_num 8) 2024-09-18T10:14:49.586 INFO:tasks.rados.rados.0.smithi189.stdout:9: done (15 left) 2024-09-18T10:14:49.586 INFO:tasks.rados.rados.0.smithi189.stdout:23: write initial oid 23 2024-09-18T10:14:49.586 INFO:tasks.rados.rados.0.smithi189.stdout:23: seq_num 22 ranges {0=376832} 2024-09-18T10:14:49.592 INFO:tasks.rados.rados.0.smithi189.stdout:23: writing smithi18925896-23 from 0 to 376832 tid 1 2024-09-18T10:14:49.592 INFO:tasks.rados.rados.0.smithi189.stdout: waiting on 16 2024-09-18T10:14:49.593 INFO:tasks.rados.rados.0.smithi189.stdout:7: finishing write tid 3 to smithi18925896-7 2024-09-18T10:14:49.593 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 7 v 2 (ObjNum 6 snap 0 seq_num 6) dirty exists 2024-09-18T10:14:49.593 INFO:tasks.rados.rados.0.smithi189.stdout:7: left oid 7 (ObjNum 6 snap 0 seq_num 6) 2024-09-18T10:14:49.593 INFO:tasks.rados.rados.0.smithi189.stdout:16: finishing write tid 3 to smithi18925896-16 2024-09-18T10:14:49.593 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 16 v 2 (ObjNum 15 snap 0 seq_num 15) dirty exists 2024-09-18T10:14:49.593 INFO:tasks.rados.rados.0.smithi189.stdout:16: left oid 16 (ObjNum 15 snap 0 seq_num 15) 2024-09-18T10:14:49.593 INFO:tasks.rados.rados.0.smithi189.stdout:7: done (15 left) 2024-09-18T10:14:49.593 INFO:tasks.rados.rados.0.smithi189.stdout:16: done (14 left) 2024-09-18T10:14:49.593 INFO:tasks.rados.rados.0.smithi189.stdout:24: write initial oid 24 2024-09-18T10:14:49.593 INFO:tasks.rados.rados.0.smithi189.stdout:24: seq_num 23 ranges {0=393216} 2024-09-18T10:14:49.598 INFO:tasks.rados.rados.0.smithi189.stdout:24: writing smithi18925896-24 from 0 to 393216 tid 1 2024-09-18T10:14:49.598 INFO:tasks.rados.rados.0.smithi189.stdout:25: write initial oid 25 2024-09-18T10:14:49.598 INFO:tasks.rados.rados.0.smithi189.stdout:25: seq_num 24 ranges {0=409600} 2024-09-18T10:14:49.602 INFO:tasks.rados.rados.0.smithi189.stdout:25: writing smithi18925896-25 from 0 to 409600 tid 1 2024-09-18T10:14:49.602 INFO:tasks.rados.rados.0.smithi189.stdout: waiting on 16 2024-09-18T10:14:49.602 INFO:tasks.rados.rados.0.smithi189.stdout:20: finishing write tid 1 to smithi18925896-20 2024-09-18T10:14:49.618 INFO:tasks.ceph.osd.4.smithi029.stderr:2024-09-18T10:14:49.614+0000 7f550cf6e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22189) UID: 0 2024-09-18T10:14:49.669 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-18T10:14:49.669 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:14:49.672 INFO:teuthology.orchestra.run.smithi029.stderr:noscrub is unset 2024-09-18T10:14:49.678 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T10:14:49.679 INFO:tasks.rados.rados.0.smithi189.stdout:15: finishing write tid 3 to smithi18925896-15 2024-09-18T10:14:49.679 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 15 v 2 (ObjNum 14 snap 0 seq_num 14) dirty exists 2024-09-18T10:14:49.679 INFO:tasks.rados.rados.0.smithi189.stdout:15: left oid 15 (ObjNum 14 snap 0 seq_num 14) 2024-09-18T10:14:49.679 INFO:tasks.rados.rados.0.smithi189.stdout:15: done (15 left) 2024-09-18T10:14:49.680 INFO:tasks.rados.rados.0.smithi189.stdout:26: write initial oid 26 2024-09-18T10:14:49.680 INFO:tasks.rados.rados.0.smithi189.stdout:26: seq_num 25 ranges {0=425984} 2024-09-18T10:14:49.686 INFO:tasks.rados.rados.0.smithi189.stdout:26: writing smithi18925896-26 from 0 to 425984 tid 1 2024-09-18T10:14:49.687 INFO:tasks.rados.rados.0.smithi189.stdout: waiting on 16 2024-09-18T10:14:49.692 INFO:tasks.rados.rados.0.smithi189.stdout:5: finishing write tid 3 to smithi18925896-5 2024-09-18T10:14:49.693 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 5 v 2 (ObjNum 4 snap 0 seq_num 4) dirty exists 2024-09-18T10:14:49.693 INFO:tasks.rados.rados.0.smithi189.stdout:5: left oid 5 (ObjNum 4 snap 0 seq_num 4) 2024-09-18T10:14:49.693 INFO:tasks.rados.rados.0.smithi189.stdout:5: done (15 left) 2024-09-18T10:14:49.693 INFO:tasks.rados.rados.0.smithi189.stdout:27: write initial oid 27 2024-09-18T10:14:49.693 INFO:tasks.rados.rados.0.smithi189.stdout:27: seq_num 26 ranges {0=442368} 2024-09-18T10:14:49.698 INFO:tasks.rados.rados.0.smithi189.stdout:27: writing smithi18925896-27 from 0 to 442368 tid 1 2024-09-18T10:14:49.698 INFO:tasks.rados.rados.0.smithi189.stdout: waiting on 16 2024-09-18T10:14:49.698 INFO:tasks.rados.rados.0.smithi189.stdout:20: finishing write tid 2 to smithi18925896-20 2024-09-18T10:14:49.698 INFO:tasks.rados.rados.0.smithi189.stdout:17: finishing write tid 1 to smithi18925896-17 2024-09-18T10:14:49.719 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-18T10:14:49.714+0000 7f1981873700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22132) UID: 0 2024-09-18T10:14:49.769 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-18T10:14:49.769 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:14:49.778 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-09-18T10:14:49.788 INFO:tasks.rados.rados.0.smithi189.stdout:13: finishing write tid 3 to smithi18925896-13 2024-09-18T10:14:49.788 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 13 v 4 (ObjNum 12 snap 0 seq_num 12) dirty exists 2024-09-18T10:14:49.788 INFO:tasks.rados.rados.0.smithi189.stdout:13: left oid 13 (ObjNum 12 snap 0 seq_num 12) 2024-09-18T10:14:49.789 INFO:tasks.rados.rados.0.smithi189.stdout:13: done (15 left) 2024-09-18T10:14:49.789 INFO:tasks.rados.rados.0.smithi189.stdout:28: write initial oid 28 2024-09-18T10:14:49.789 INFO:tasks.rados.rados.0.smithi189.stdout:28: seq_num 27 ranges {0=458752} 2024-09-18T10:14:49.797 INFO:tasks.rados.rados.0.smithi189.stdout:28: writing smithi18925896-28 from 0 to 458752 tid 1 2024-09-18T10:14:49.797 INFO:tasks.rados.rados.0.smithi189.stdout: waiting on 16 2024-09-18T10:14:49.797 INFO:tasks.rados.rados.0.smithi189.stdout:14: finishing write tid 3 to smithi18925896-14 2024-09-18T10:14:49.797 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 14 v 4 (ObjNum 13 snap 0 seq_num 13) dirty exists 2024-09-18T10:14:49.797 INFO:tasks.rados.rados.0.smithi189.stdout:14: left oid 14 (ObjNum 13 snap 0 seq_num 13) 2024-09-18T10:14:49.797 INFO:tasks.rados.rados.0.smithi189.stdout:14: done (15 left) 2024-09-18T10:14:49.797 INFO:tasks.rados.rados.0.smithi189.stdout:29: write initial oid 29 2024-09-18T10:14:49.797 INFO:tasks.rados.rados.0.smithi189.stdout:29: seq_num 28 ranges {0=475136} 2024-09-18T10:14:49.803 INFO:tasks.rados.rados.0.smithi189.stdout:29: writing smithi18925896-29 from 0 to 475136 tid 1 2024-09-18T10:14:49.803 INFO:tasks.rados.rados.0.smithi189.stdout: waiting on 16 2024-09-18T10:14:49.803 INFO:tasks.rados.rados.0.smithi189.stdout:17: finishing write tid 2 to smithi18925896-17 2024-09-18T10:14:49.820 INFO:tasks.ceph.osd.3.smithi189.stderr:2024-09-18T10:14:49.815+0000 7f2440fc7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22396) UID: 0 2024-09-18T10:14:49.893 INFO:tasks.rados.rados.0.smithi189.stdout:18: finishing write tid 1 to smithi18925896-18 2024-09-18T10:14:49.893 INFO:tasks.rados.rados.0.smithi189.stdout:18: finishing write tid 2 to smithi18925896-18 2024-09-18T10:14:49.898 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T10:14:49.902 INFO:tasks.rados.rados.0.smithi189.stdout:11: finishing write tid 3 to smithi18925896-11 2024-09-18T10:14:49.902 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 11 v 4 (ObjNum 10 snap 0 seq_num 10) dirty exists 2024-09-18T10:14:49.902 INFO:tasks.rados.rados.0.smithi189.stdout:11: left oid 11 (ObjNum 10 snap 0 seq_num 10) 2024-09-18T10:14:49.902 INFO:tasks.rados.rados.0.smithi189.stdout:11: done (15 left) 2024-09-18T10:14:49.902 INFO:tasks.rados.rados.0.smithi189.stdout:30: write initial oid 30 2024-09-18T10:14:49.902 INFO:tasks.rados.rados.0.smithi189.stdout:30: seq_num 29 ranges {0=491520} 2024-09-18T10:14:49.908 INFO:tasks.rados.rados.0.smithi189.stdout:30: writing smithi18925896-30 from 0 to 491520 tid 1 2024-09-18T10:14:49.908 INFO:tasks.rados.rados.0.smithi189.stdout: waiting on 16 2024-09-18T10:14:49.920 INFO:tasks.ceph.osd.3.smithi189.stderr:2024-09-18T10:14:49.919+0000 7f2440fc7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22396) UID: 0 2024-09-18T10:14:49.927 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-18T10:14:49.927 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:14:49.937 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-09-18T10:14:49.984 INFO:tasks.rados.rados.0.smithi189.stdout:8: finishing write tid 3 to smithi18925896-8 2024-09-18T10:14:49.985 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 8 v 2 (ObjNum 7 snap 0 seq_num 7) dirty exists 2024-09-18T10:14:49.985 INFO:tasks.rados.rados.0.smithi189.stdout:8: left oid 8 (ObjNum 7 snap 0 seq_num 7) 2024-09-18T10:14:49.985 INFO:tasks.rados.rados.0.smithi189.stdout:8: done (15 left) 2024-09-18T10:14:49.985 INFO:tasks.rados.rados.0.smithi189.stdout:31: write initial oid 31 2024-09-18T10:14:49.985 INFO:tasks.rados.rados.0.smithi189.stdout:31: seq_num 30 ranges {0=507904} 2024-09-18T10:14:49.988 INFO:tasks.rados.rados.0.smithi189.stdout:31: writing smithi18925896-31 from 0 to 507904 tid 1 2024-09-18T10:14:49.988 INFO:tasks.rados.rados.0.smithi189.stdout: waiting on 16 2024-09-18T10:14:49.989 INFO:tasks.rados.rados.0.smithi189.stdout:28: finishing write tid 1 to smithi18925896-28 2024-09-18T10:14:49.989 INFO:tasks.rados.rados.0.smithi189.stdout:28: finishing write tid 2 to smithi18925896-28 2024-09-18T10:14:49.994 INFO:tasks.rados.rados.0.smithi189.stdout:19: finishing write tid 1 to smithi18925896-19 2024-09-18T10:14:49.995 INFO:tasks.rados.rados.0.smithi189.stdout:19: finishing write tid 2 to smithi18925896-19 2024-09-18T10:14:50.021 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-18T10:14:50.018+0000 7f1981873700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22132) UID: 0 2024-09-18T10:14:50.037 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-18T10:14:50.037 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:14:50.046 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-09-18T10:14:50.084 INFO:tasks.rados.rados.0.smithi189.stdout:12: finishing write tid 3 to smithi18925896-12 2024-09-18T10:14:50.085 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 12 v 4 (ObjNum 11 snap 0 seq_num 11) dirty exists 2024-09-18T10:14:50.085 INFO:tasks.rados.rados.0.smithi189.stdout:12: left oid 12 (ObjNum 11 snap 0 seq_num 11) 2024-09-18T10:14:50.085 INFO:tasks.rados.rados.0.smithi189.stdout:12: done (15 left) 2024-09-18T10:14:50.085 INFO:tasks.rados.rados.0.smithi189.stdout:32: write initial oid 32 2024-09-18T10:14:50.085 INFO:tasks.rados.rados.0.smithi189.stdout:32: seq_num 31 ranges {0=524288} 2024-09-18T10:14:50.093 INFO:tasks.rados.rados.0.smithi189.stdout:32: writing smithi18925896-32 from 0 to 524288 tid 1 2024-09-18T10:14:50.093 INFO:tasks.rados.rados.0.smithi189.stdout: waiting on 16 2024-09-18T10:14:50.093 INFO:tasks.rados.rados.0.smithi189.stdout:22: finishing write tid 1 to smithi18925896-22 2024-09-18T10:14:50.093 INFO:tasks.rados.rados.0.smithi189.stdout:21: finishing write tid 1 to smithi18925896-21 2024-09-18T10:14:50.094 INFO:tasks.rados.rados.0.smithi189.stdout:28: finishing write tid 3 to smithi18925896-28 2024-09-18T10:14:50.094 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 28 v 6 (ObjNum 27 snap 0 seq_num 27) dirty exists 2024-09-18T10:14:50.094 INFO:tasks.rados.rados.0.smithi189.stdout:28: left oid 28 (ObjNum 27 snap 0 seq_num 27) 2024-09-18T10:14:50.094 INFO:tasks.rados.rados.0.smithi189.stdout:28: done (15 left) 2024-09-18T10:14:50.094 INFO:tasks.rados.rados.0.smithi189.stdout:33: write initial oid 33 2024-09-18T10:14:50.094 INFO:tasks.rados.rados.0.smithi189.stdout:33: seq_num 32 ranges {0=540672} 2024-09-18T10:14:50.099 INFO:tasks.rados.rados.0.smithi189.stdout:33: writing smithi18925896-33 from 0 to 540672 tid 1 2024-09-18T10:14:50.099 INFO:tasks.rados.rados.0.smithi189.stdout: waiting on 16 2024-09-18T10:14:50.099 INFO:tasks.rados.rados.0.smithi189.stdout:20: finishing write tid 3 to smithi18925896-20 2024-09-18T10:14:50.100 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 20 v 2 (ObjNum 19 snap 0 seq_num 19) dirty exists 2024-09-18T10:14:50.100 INFO:tasks.rados.rados.0.smithi189.stdout:20: left oid 20 (ObjNum 19 snap 0 seq_num 19) 2024-09-18T10:14:50.100 INFO:tasks.rados.rados.0.smithi189.stdout:22: finishing write tid 2 to smithi18925896-22 2024-09-18T10:14:50.100 INFO:tasks.rados.rados.0.smithi189.stdout:20: done (15 left) 2024-09-18T10:14:50.100 INFO:tasks.rados.rados.0.smithi189.stdout:34: write initial oid 34 2024-09-18T10:14:50.100 INFO:tasks.rados.rados.0.smithi189.stdout:34: seq_num 33 ranges {0=557056} 2024-09-18T10:14:50.109 INFO:tasks.rados.rados.0.smithi189.stdout:34: writing smithi18925896-34 from 0 to 557056 tid 1 2024-09-18T10:14:50.109 INFO:tasks.rados.rados.0.smithi189.stdout: waiting on 16 2024-09-18T10:14:50.109 INFO:tasks.rados.rados.0.smithi189.stdout:23: finishing write tid 1 to smithi18925896-23 2024-09-18T10:14:50.109 INFO:tasks.rados.rados.0.smithi189.stdout:23: finishing write tid 2 to smithi18925896-23 2024-09-18T10:14:50.122 INFO:tasks.ceph.osd.12.smithi029.stderr:2024-09-18T10:14:50.118+0000 7f8e4e627700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 22190) UID: 0 2024-09-18T10:14:50.186 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-18T10:14:50.186 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:14:50.195 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_ops_in_flight 2024-09-18T10:14:50.222 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-18T10:14:50.223 INFO:tasks.ceph.osd.3.smithi189.stderr:2024-09-18T10:14:50.219+0000 7f2440fc7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22396) UID: 0 2024-09-18T10:14:50.244 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-18T10:14:50.267 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-18T10:14:50.289 INFO:tasks.rados.rados.0.smithi189.stdout:21: finishing write tid 2 to smithi18925896-21 2024-09-18T10:14:50.290 INFO:tasks.rados.rados.0.smithi189.stdout:24: finishing write tid 1 to smithi18925896-24 2024-09-18T10:14:50.290 INFO:tasks.rados.rados.0.smithi189.stdout:24: finishing write tid 2 to smithi18925896-24 2024-09-18T10:14:50.290 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-18T10:14:50.292 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-18T10:14:50.292 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:14:50.294 INFO:tasks.rados.rados.0.smithi189.stdout:25: finishing write tid 1 to smithi18925896-25 2024-09-18T10:14:50.294 INFO:tasks.rados.rados.0.smithi189.stdout:25: finishing write tid 2 to smithi18925896-25 2024-09-18T10:14:50.295 INFO:tasks.rados.rados.0.smithi189.stdout:26: finishing write tid 1 to smithi18925896-26 2024-09-18T10:14:50.302 INFO:tasks.rados.rados.0.smithi189.stdout:32: finishing write tid 1 to smithi18925896-32 2024-09-18T10:14:50.303 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_blocked_ops 2024-09-18T10:14:50.304 INFO:tasks.rados.rados.0.smithi189.stdout:17: finishing write tid 3 to smithi18925896-17 2024-09-18T10:14:50.304 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 17 v 6 (ObjNum 16 snap 0 seq_num 16) dirty exists 2024-09-18T10:14:50.304 INFO:tasks.rados.rados.0.smithi189.stdout:17: left oid 17 (ObjNum 16 snap 0 seq_num 16) 2024-09-18T10:14:50.304 INFO:tasks.rados.rados.0.smithi189.stdout:17: done (15 left) 2024-09-18T10:14:50.304 INFO:tasks.rados.rados.0.smithi189.stdout:35: write initial oid 35 2024-09-18T10:14:50.304 INFO:tasks.rados.rados.0.smithi189.stdout:35: seq_num 34 ranges {0=573440} 2024-09-18T10:14:50.313 INFO:tasks.rados.rados.0.smithi189.stdout:35: writing smithi18925896-35 from 0 to 573440 tid 1 2024-09-18T10:14:50.313 INFO:tasks.rados.rados.0.smithi189.stdout: waiting on 16 2024-09-18T10:14:50.324 INFO:tasks.ceph.osd.12.smithi029.stderr:2024-09-18T10:14:50.318+0000 7f8e4e627700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 22190) UID: 0 2024-09-18T10:14:50.396 INFO:tasks.rados.rados.0.smithi189.stdout:26: finishing write tid 2 to smithi18925896-26 2024-09-18T10:14:50.397 INFO:tasks.rados.rados.0.smithi189.stdout:29: finishing write tid 1 to smithi18925896-29 2024-09-18T10:14:50.401 INFO:tasks.rados.rados.0.smithi189.stdout:23: finishing write tid 3 to smithi18925896-23 2024-09-18T10:14:50.401 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 23 v 4 (ObjNum 22 snap 0 seq_num 22) dirty exists 2024-09-18T10:14:50.401 INFO:tasks.rados.rados.0.smithi189.stdout:23: left oid 23 (ObjNum 22 snap 0 seq_num 22) 2024-09-18T10:14:50.402 INFO:tasks.rados.rados.0.smithi189.stdout:23: done (15 left) 2024-09-18T10:14:50.402 INFO:tasks.rados.rados.0.smithi189.stdout:36: write initial oid 36 2024-09-18T10:14:50.402 INFO:tasks.rados.rados.0.smithi189.stdout:36: seq_num 35 ranges {0=589824} 2024-09-18T10:14:50.403 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-18T10:14:50.403 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:14:50.411 INFO:tasks.rados.rados.0.smithi189.stdout:36: writing smithi18925896-36 from 0 to 589824 tid 1 2024-09-18T10:14:50.412 INFO:tasks.rados.rados.0.smithi189.stdout: waiting on 16 2024-09-18T10:14:50.413 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_historic_ops 2024-09-18T10:14:50.417 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-18T10:14:50.424 INFO:tasks.ceph.osd.3.smithi189.stderr:2024-09-18T10:14:50.419+0000 7f2440fc7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22396) UID: 0 2024-09-18T10:14:50.436 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-18T10:14:50.492 INFO:tasks.rados.rados.0.smithi189.stdout:33: finishing write tid 1 to smithi18925896-33 2024-09-18T10:14:50.496 INFO:tasks.rados.rados.0.smithi189.stdout:29: finishing write tid 2 to smithi18925896-29 2024-09-18T10:14:50.500 INFO:tasks.rados.rados.0.smithi189.stdout:32: finishing write tid 2 to smithi18925896-32 2024-09-18T10:14:50.504 INFO:tasks.rados.rados.0.smithi189.stdout:31: finishing write tid 1 to smithi18925896-31 2024-09-18T10:14:50.504 INFO:tasks.rados.rados.0.smithi189.stdout:31: finishing write tid 2 to smithi18925896-31 2024-09-18T10:14:50.515 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-18T10:14:50.516 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:14:50.524 INFO:teuthology.orchestra.run.smithi029.stdout:osd.6: {} 2024-09-18T10:14:50.525 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-09-18T10:14:50.520+0000 7fdfa8698700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22087) UID: 0 2024-09-18T10:14:50.526 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T10:14:50.548 INFO:teuthology.orchestra.run.smithi029.stdout:osd.7: {} 2024-09-18T10:14:50.599 INFO:tasks.rados.rados.0.smithi189.stdout:21: finishing write tid 3 to smithi18925896-21 2024-09-18T10:14:50.599 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 21 v 4 (ObjNum 20 snap 0 seq_num 20) dirty exists 2024-09-18T10:14:50.599 INFO:tasks.rados.rados.0.smithi189.stdout:21: left oid 21 (ObjNum 20 snap 0 seq_num 20) 2024-09-18T10:14:50.599 INFO:tasks.rados.rados.0.smithi189.stdout:21: done (15 left) 2024-09-18T10:14:50.599 INFO:tasks.rados.rados.0.smithi189.stdout:37: write initial oid 37 2024-09-18T10:14:50.599 INFO:tasks.rados.rados.0.smithi189.stdout:37: seq_num 36 ranges {0=606208} 2024-09-18T10:14:50.609 INFO:tasks.rados.rados.0.smithi189.stdout:37: writing smithi18925896-37 from 0 to 606208 tid 1 2024-09-18T10:14:50.610 INFO:tasks.rados.rados.0.smithi189.stdout: waiting on 16 2024-09-18T10:14:50.610 INFO:tasks.rados.rados.0.smithi189.stdout:27: finishing write tid 1 to smithi18925896-27 2024-09-18T10:14:50.610 INFO:tasks.rados.rados.0.smithi189.stdout:33: finishing write tid 2 to smithi18925896-33 2024-09-18T10:14:50.610 INFO:tasks.rados.rados.0.smithi189.stdout:25: finishing write tid 3 to smithi18925896-25 2024-09-18T10:14:50.610 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 25 v 2 (ObjNum 24 snap 0 seq_num 24) dirty exists 2024-09-18T10:14:50.610 INFO:tasks.rados.rados.0.smithi189.stdout:25: left oid 25 (ObjNum 24 snap 0 seq_num 24) 2024-09-18T10:14:50.610 INFO:tasks.rados.rados.0.smithi189.stdout:18: finishing write tid 3 to smithi18925896-18 2024-09-18T10:14:50.610 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 18 v 4 (ObjNum 17 snap 0 seq_num 17) dirty exists 2024-09-18T10:14:50.611 INFO:tasks.rados.rados.0.smithi189.stdout:18: left oid 18 (ObjNum 17 snap 0 seq_num 17) 2024-09-18T10:14:50.611 INFO:tasks.rados.rados.0.smithi189.stdout:22: finishing write tid 3 to smithi18925896-22 2024-09-18T10:14:50.611 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 22 v 4 (ObjNum 21 snap 0 seq_num 21) dirty exists 2024-09-18T10:14:50.611 INFO:tasks.rados.rados.0.smithi189.stdout:22: left oid 22 (ObjNum 21 snap 0 seq_num 21) 2024-09-18T10:14:50.611 INFO:tasks.rados.rados.0.smithi189.stdout:19: finishing write tid 3 to smithi18925896-19 2024-09-18T10:14:50.611 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 19 v 6 (ObjNum 18 snap 0 seq_num 18) dirty exists 2024-09-18T10:14:50.611 INFO:tasks.rados.rados.0.smithi189.stdout:19: left oid 19 (ObjNum 18 snap 0 seq_num 18) 2024-09-18T10:14:50.611 INFO:tasks.rados.rados.0.smithi189.stdout:18: done (15 left) 2024-09-18T10:14:50.611 INFO:tasks.rados.rados.0.smithi189.stdout:19: done (14 left) 2024-09-18T10:14:50.611 INFO:tasks.rados.rados.0.smithi189.stdout:22: done (13 left) 2024-09-18T10:14:50.611 INFO:tasks.rados.rados.0.smithi189.stdout:25: done (12 left) 2024-09-18T10:14:50.611 INFO:tasks.rados.rados.0.smithi189.stdout:38: write initial oid 38 2024-09-18T10:14:50.611 INFO:tasks.rados.rados.0.smithi189.stdout:38: seq_num 37 ranges {0=622592} 2024-09-18T10:14:50.615 INFO:tasks.rados.rados.0.smithi189.stdout:38: writing smithi18925896-38 from 0 to 622592 tid 1 2024-09-18T10:14:50.615 INFO:tasks.rados.rados.0.smithi189.stdout:39: write initial oid 39 2024-09-18T10:14:50.615 INFO:tasks.rados.rados.0.smithi189.stdout:39: seq_num 38 ranges {0=638976} 2024-09-18T10:14:50.618 INFO:tasks.rados.rados.0.smithi189.stdout:39: writing smithi18925896-39 from 0 to 638976 tid 1 2024-09-18T10:14:50.619 INFO:tasks.rados.rados.0.smithi189.stdout:40: write initial oid 40 2024-09-18T10:14:50.619 INFO:tasks.rados.rados.0.smithi189.stdout:40: seq_num 39 ranges {0=671744} 2024-09-18T10:14:50.622 INFO:tasks.rados.rados.0.smithi189.stdout:40: writing smithi18925896-40 from 0 to 671744 tid 1 2024-09-18T10:14:50.622 INFO:tasks.rados.rados.0.smithi189.stdout:41: write initial oid 41 2024-09-18T10:14:50.622 INFO:tasks.rados.rados.0.smithi189.stdout:41: seq_num 40 ranges {0=688128} 2024-09-18T10:14:50.626 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-18T10:14:50.622+0000 7f00046a8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 22136) UID: 0 2024-09-18T10:14:50.627 INFO:tasks.rados.rados.0.smithi189.stdout:41: writing smithi18925896-41 from 0 to 688128 tid 1 2024-09-18T10:14:50.627 INFO:tasks.rados.rados.0.smithi189.stdout: waiting on 16 2024-09-18T10:14:50.635 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-09-18T10:14:50.635 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-09-18T10:14:50.635 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-09-18T10:14:50.635 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-09-18T10:14:50.645 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T10:14:50.699 INFO:tasks.rados.rados.0.smithi189.stdout:35: finishing write tid 1 to smithi18925896-35 2024-09-18T10:14:50.699 INFO:tasks.rados.rados.0.smithi189.stdout:24: finishing write tid 3 to smithi18925896-24 2024-09-18T10:14:50.700 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 24 v 6 (ObjNum 23 snap 0 seq_num 23) dirty exists 2024-09-18T10:14:50.700 INFO:tasks.rados.rados.0.smithi189.stdout:24: left oid 24 (ObjNum 23 snap 0 seq_num 23) 2024-09-18T10:14:50.700 INFO:tasks.rados.rados.0.smithi189.stdout:24: done (15 left) 2024-09-18T10:14:50.700 INFO:tasks.rados.rados.0.smithi189.stdout:42: write initial oid 42 2024-09-18T10:14:50.700 INFO:tasks.rados.rados.0.smithi189.stdout:42: seq_num 41 ranges {0=704512} 2024-09-18T10:14:50.705 INFO:tasks.rados.rados.0.smithi189.stdout:42: writing smithi18925896-42 from 0 to 704512 tid 1 2024-09-18T10:14:50.705 INFO:tasks.rados.rados.0.smithi189.stdout: waiting on 16 2024-09-18T10:14:50.705 INFO:tasks.rados.rados.0.smithi189.stdout:35: finishing write tid 2 to smithi18925896-35 2024-09-18T10:14:50.705 INFO:tasks.rados.rados.0.smithi189.stdout:33: finishing write tid 3 to smithi18925896-33 2024-09-18T10:14:50.705 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 33 v 6 (ObjNum 32 snap 0 seq_num 32) dirty exists 2024-09-18T10:14:50.705 INFO:tasks.rados.rados.0.smithi189.stdout:33: left oid 33 (ObjNum 32 snap 0 seq_num 32) 2024-09-18T10:14:50.706 INFO:tasks.rados.rados.0.smithi189.stdout:33: done (15 left) 2024-09-18T10:14:50.706 INFO:tasks.rados.rados.0.smithi189.stdout:43: write initial oid 43 2024-09-18T10:14:50.706 INFO:tasks.rados.rados.0.smithi189.stdout:43: seq_num 42 ranges {0=720896} 2024-09-18T10:14:50.713 INFO:tasks.rados.rados.0.smithi189.stdout:43: writing smithi18925896-43 from 0 to 720896 tid 1 2024-09-18T10:14:50.713 INFO:tasks.rados.rados.0.smithi189.stdout: waiting on 16 2024-09-18T10:14:50.713 INFO:tasks.rados.rados.0.smithi189.stdout:30: finishing write tid 1 to smithi18925896-30 2024-09-18T10:14:50.713 INFO:tasks.rados.rados.0.smithi189.stdout:30: finishing write tid 2 to smithi18925896-30 2024-09-18T10:14:50.713 INFO:tasks.rados.rados.0.smithi189.stdout:31: finishing write tid 3 to smithi18925896-31 2024-09-18T10:14:50.713 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 31 v 6 (ObjNum 30 snap 0 seq_num 30) dirty exists 2024-09-18T10:14:50.713 INFO:tasks.rados.rados.0.smithi189.stdout:31: left oid 31 (ObjNum 30 snap 0 seq_num 30) 2024-09-18T10:14:50.713 INFO:tasks.rados.rados.0.smithi189.stdout:31: done (15 left) 2024-09-18T10:14:50.713 INFO:tasks.rados.rados.0.smithi189.stdout:44: write initial oid 44 2024-09-18T10:14:50.713 INFO:tasks.rados.rados.0.smithi189.stdout:44: seq_num 43 ranges {0=737280} 2024-09-18T10:14:50.717 INFO:tasks.rados.rados.0.smithi189.stdout:44: writing smithi18925896-44 from 0 to 737280 tid 1 2024-09-18T10:14:50.717 INFO:tasks.rados.rados.0.smithi189.stdout: waiting on 16 2024-09-18T10:14:50.727 INFO:tasks.ceph.osd.12.smithi029.stderr:2024-09-18T10:14:50.722+0000 7f8e4e627700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 22190) UID: 0 2024-09-18T10:14:50.741 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-09-18T10:14:50.741 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-09-18T10:14:50.741 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-09-18T10:14:50.741 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-09-18T10:14:50.742 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-09-18T10:14:50.751 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T10:14:50.822 INFO:teuthology.orchestra.run.smithi029.stdout:osd.8: {} 2024-09-18T10:14:50.828 INFO:tasks.ceph.osd.8.smithi029.stderr:2024-09-18T10:14:50.822+0000 7f2555c27700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 22192) UID: 0 2024-09-18T10:14:50.850 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-09-18T10:14:50.850 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-09-18T10:14:50.851 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-09-18T10:14:50.851 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-09-18T10:14:50.851 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:50.851 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.fs4 36/32 ECSubWrite(tid=17, reqid=client.4632.0:103, at_version=36'1, trim_to=0'0, roll_forward_to=0'0))", 2024-09-18T10:14:50.851 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:50.416941+0000", 2024-09-18T10:14:50.851 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 0.43213966500000001, 2024-09-18T10:14:50.851 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0026131800000000001, 2024-09-18T10:14:50.851 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:50.851 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:14:50.851 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:50.851 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:50.851 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:50.851 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.416941+0000", 2024-09-18T10:14:50.852 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:50.852 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:50.852 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:50.852 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:50.852 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.416941+0000", 2024-09-18T10:14:50.852 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:50.852 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:50.852 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:50.852 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:50.852 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.416939+0000", 2024-09-18T10:14:50.852 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:14:50.852 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:50.852 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:50.853 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:50.853 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.417391+0000", 2024-09-18T10:14:50.853 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00045155 2024-09-18T10:14:50.853 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:50.853 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:50.853 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:50.853 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.417426+0000", 2024-09-18T10:14:50.853 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.4619000000000002e-05 2024-09-18T10:14:50.853 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:50.853 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:50.853 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:50.853 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.417460+0000", 2024-09-18T10:14:50.853 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.4731999999999999e-05 2024-09-18T10:14:50.854 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:50.854 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:50.854 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:50.854 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.417609+0000", 2024-09-18T10:14:50.854 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000148336 2024-09-18T10:14:50.854 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:50.854 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:50.854 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:14:50.854 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.417634+0000", 2024-09-18T10:14:50.854 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4972000000000001e-05 2024-09-18T10:14:50.854 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:50.854 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:50.854 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:14:50.854 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.419505+0000", 2024-09-18T10:14:50.855 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001871073 2024-09-18T10:14:50.855 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:50.855 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:50.855 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:50.855 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.419554+0000", 2024-09-18T10:14:50.855 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9588e-05 2024-09-18T10:14:50.855 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:50.855 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:50.855 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:50.855 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:50.855 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:50.855 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.fs4 36/32 ECSubWrite(tid=18, reqid=client.4632.0:104, at_version=36'2, trim_to=0'0, roll_forward_to=0'0))", 2024-09-18T10:14:50.855 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:50.419743+0000", 2024-09-18T10:14:50.856 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 0.429337356, 2024-09-18T10:14:50.856 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001340136, 2024-09-18T10:14:50.856 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:50.856 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:14:50.856 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:50.856 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:50.856 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:50.856 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.419743+0000", 2024-09-18T10:14:50.856 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:50.856 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:50.856 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:50.856 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:50.856 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.419743+0000", 2024-09-18T10:14:50.857 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:50.857 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:50.857 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:50.857 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:50.857 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.419742+0000", 2024-09-18T10:14:50.857 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:14:50.857 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:50.857 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:50.857 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:50.857 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.419773+0000", 2024-09-18T10:14:50.857 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1018e-05 2024-09-18T10:14:50.857 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:50.858 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:50.858 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:50.858 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.419799+0000", 2024-09-18T10:14:50.858 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6122000000000002e-05 2024-09-18T10:14:50.858 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:50.858 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:50.858 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:50.858 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.419810+0000", 2024-09-18T10:14:50.858 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.099e-05 2024-09-18T10:14:50.858 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:50.858 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:50.858 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:50.858 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.419908+0000", 2024-09-18T10:14:50.858 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.7504999999999996e-05 2024-09-18T10:14:50.859 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:50.859 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:50.859 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:14:50.859 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.419937+0000", 2024-09-18T10:14:50.859 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9666e-05 2024-09-18T10:14:50.859 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:50.859 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:50.859 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:14:50.859 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.421013+0000", 2024-09-18T10:14:50.859 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001075782 2024-09-18T10:14:50.859 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:50.859 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:50.859 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:50.859 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.421084+0000", 2024-09-18T10:14:50.860 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0233000000000003e-05 2024-09-18T10:14:50.860 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:50.860 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:50.860 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:50.860 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:50.860 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:50.860 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.4s3 36/32 ECSubWrite(tid=6, reqid=client.4632.0:95, at_version=0'0, trim_to=0'0, roll_forward_to=36'4))", 2024-09-18T10:14:50.860 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:50.501142+0000", 2024-09-18T10:14:50.860 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 0.34793913399999998, 2024-09-18T10:14:50.860 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0020030009999999999, 2024-09-18T10:14:50.860 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:50.860 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:14:50.860 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:50.861 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:50.861 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:50.861 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.501142+0000", 2024-09-18T10:14:50.861 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:50.861 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:50.861 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:50.861 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:50.861 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.501142+0000", 2024-09-18T10:14:50.861 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:50.861 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:50.861 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:50.861 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:50.861 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.501140+0000", 2024-09-18T10:14:50.861 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:14:50.862 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:50.862 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:50.862 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:50.862 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.501174+0000", 2024-09-18T10:14:50.862 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.3884999999999998e-05 2024-09-18T10:14:50.862 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:50.862 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:50.862 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:50.862 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.501231+0000", 2024-09-18T10:14:50.862 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.7207000000000003e-05 2024-09-18T10:14:50.862 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:50.862 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:50.862 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:50.862 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.501249+0000", 2024-09-18T10:14:50.863 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7883000000000001e-05 2024-09-18T10:14:50.863 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:50.863 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:50.863 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:50.863 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.501413+0000", 2024-09-18T10:14:50.863 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000164709 2024-09-18T10:14:50.863 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:50.863 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:50.863 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:14:50.863 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.501438+0000", 2024-09-18T10:14:50.863 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4406e-05 2024-09-18T10:14:50.863 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:50.863 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:50.863 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:14:50.864 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.503098+0000", 2024-09-18T10:14:50.864 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0016597490000000001 2024-09-18T10:14:50.864 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:50.864 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:50.864 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:50.864 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.503145+0000", 2024-09-18T10:14:50.864 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.7074999999999999e-05 2024-09-18T10:14:50.864 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:50.864 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:50.864 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:50.864 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:50.864 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:50.864 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpRead(3.4s3 36/32 ECSubRead(tid=7, to_read={3:213f864e:::smithi18925896-32:head=0,4096,0}, subchunks={3:213f864e:::smithi18925896-32:head=[0,1]}, attrs_to_read=))", 2024-09-18T10:14:50.865 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:50.502372+0000", 2024-09-18T10:14:50.865 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 0.34670857599999999, 2024-09-18T10:14:50.865 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000783065, 2024-09-18T10:14:50.865 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:50.865 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:14:50.865 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:50.865 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:50.865 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:50.865 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.502372+0000", 2024-09-18T10:14:50.865 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:50.865 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:50.865 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:50.865 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:50.866 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.502372+0000", 2024-09-18T10:14:50.866 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:50.866 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:50.866 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:50.866 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:50.866 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.502371+0000", 2024-09-18T10:14:50.866 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:14:50.866 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:50.866 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:50.866 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:50.866 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.502393+0000", 2024-09-18T10:14:50.866 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.2245000000000001e-05 2024-09-18T10:14:50.866 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:50.867 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:50.867 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:50.867 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.502443+0000", 2024-09-18T10:14:50.867 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9768000000000002e-05 2024-09-18T10:14:50.867 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:50.867 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:50.867 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:50.867 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.502462+0000", 2024-09-18T10:14:50.867 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.9782000000000001e-05 2024-09-18T10:14:50.867 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:50.867 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:50.867 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:50.867 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.502604+0000", 2024-09-18T10:14:50.867 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000141464 2024-09-18T10:14:50.868 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:50.868 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:50.868 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:50.868 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.503155+0000", 2024-09-18T10:14:50.868 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00055132499999999997 2024-09-18T10:14:50.868 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:50.868 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:50.868 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:50.868 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:50.868 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:50.868 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.6s3 36/32 ECSubWrite(tid=12, reqid=client.4632.0:79, at_version=0'0, trim_to=0'0, roll_forward_to=36'7))", 2024-09-18T10:14:50.868 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:50.601317+0000", 2024-09-18T10:14:50.868 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 0.247763539, 2024-09-18T10:14:50.869 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0013476339999999999, 2024-09-18T10:14:50.869 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:50.869 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:14:50.869 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:50.869 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:50.869 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:50.869 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.601317+0000", 2024-09-18T10:14:50.869 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:50.869 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:50.869 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:50.869 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:50.869 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.601317+0000", 2024-09-18T10:14:50.869 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:50.870 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:50.870 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:50.870 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:50.870 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.601315+0000", 2024-09-18T10:14:50.870 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:14:50.870 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:50.870 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:50.870 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:50.870 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.601342+0000", 2024-09-18T10:14:50.870 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6859000000000002e-05 2024-09-18T10:14:50.870 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:50.870 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:50.870 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:50.870 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.601375+0000", 2024-09-18T10:14:50.871 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2827000000000003e-05 2024-09-18T10:14:50.871 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:50.871 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:50.871 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:50.871 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.601390+0000", 2024-09-18T10:14:50.871 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.5342e-05 2024-09-18T10:14:50.871 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:50.871 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:50.871 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:50.871 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.601580+0000", 2024-09-18T10:14:50.871 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00018969399999999999 2024-09-18T10:14:50.871 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:50.872 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:50.872 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:14:50.872 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.601608+0000", 2024-09-18T10:14:50.872 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.7945000000000001e-05 2024-09-18T10:14:50.872 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:50.872 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:50.872 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:14:50.872 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.602634+0000", 2024-09-18T10:14:50.872 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0010257689999999999 2024-09-18T10:14:50.872 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:50.872 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:50.872 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:50.872 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.602665+0000", 2024-09-18T10:14:50.872 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0975000000000001e-05 2024-09-18T10:14:50.873 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:50.873 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:50.873 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:50.873 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:50.873 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:50.873 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.1s6 36/32 ECSubWrite(tid=23, reqid=client.4632.0:98, at_version=0'0, trim_to=0'0, roll_forward_to=36'6))", 2024-09-18T10:14:50.873 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:50.602726+0000", 2024-09-18T10:14:50.873 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 0.24635452499999999, 2024-09-18T10:14:50.873 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00095330399999999998, 2024-09-18T10:14:50.873 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:50.873 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:14:50.873 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:50.873 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:50.874 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:50.874 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.602726+0000", 2024-09-18T10:14:50.874 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:50.874 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:50.874 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:50.874 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:50.874 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.602726+0000", 2024-09-18T10:14:50.874 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:50.874 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:50.874 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:50.874 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:50.874 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.602725+0000", 2024-09-18T10:14:50.874 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:14:50.875 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:50.875 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:50.875 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:50.875 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.602751+0000", 2024-09-18T10:14:50.875 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5919e-05 2024-09-18T10:14:50.875 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:50.875 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:50.875 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:50.875 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.602783+0000", 2024-09-18T10:14:50.875 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2421e-05 2024-09-18T10:14:50.875 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:50.875 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:50.875 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:50.875 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.602826+0000", 2024-09-18T10:14:50.876 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.2873000000000001e-05 2024-09-18T10:14:50.876 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:50.876 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:50.876 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:50.876 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.602917+0000", 2024-09-18T10:14:50.876 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.0907000000000002e-05 2024-09-18T10:14:50.876 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:50.876 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:50.876 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:14:50.876 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.602931+0000", 2024-09-18T10:14:50.876 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3454e-05 2024-09-18T10:14:50.876 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:50.876 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:50.877 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:14:50.877 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.603594+0000", 2024-09-18T10:14:50.877 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00066337600000000003 2024-09-18T10:14:50.877 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:50.877 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:50.877 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:50.877 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.603680+0000", 2024-09-18T10:14:50.877 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.5622000000000001e-05 2024-09-18T10:14:50.877 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:50.877 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:50.877 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:50.877 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:50.877 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:50.877 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.fs4 36/32 ECSubWrite(tid=19, reqid=client.4632.0:104, at_version=0'0, trim_to=0'0, roll_forward_to=36'2))", 2024-09-18T10:14:50.878 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:50.700704+0000", 2024-09-18T10:14:50.878 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 0.14837678900000001, 2024-09-18T10:14:50.878 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0012567800000000001, 2024-09-18T10:14:50.878 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:50.878 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:14:50.878 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:50.878 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:50.878 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:50.878 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.700704+0000", 2024-09-18T10:14:50.878 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:50.878 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:50.878 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:50.878 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:50.878 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.700704+0000", 2024-09-18T10:14:50.878 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:50.879 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:50.879 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:50.879 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:50.879 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.700702+0000", 2024-09-18T10:14:50.879 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:14:50.879 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:50.879 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:50.879 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:50.879 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.700734+0000", 2024-09-18T10:14:50.879 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2285e-05 2024-09-18T10:14:50.879 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:50.879 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:50.879 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:50.880 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.700770+0000", 2024-09-18T10:14:50.880 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5377999999999999e-05 2024-09-18T10:14:50.880 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:50.880 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:50.880 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:50.880 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.700786+0000", 2024-09-18T10:14:50.880 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.6155999999999999e-05 2024-09-18T10:14:50.880 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:50.880 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:50.880 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:50.880 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.700916+0000", 2024-09-18T10:14:50.880 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00013062100000000001 2024-09-18T10:14:50.880 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:50.880 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:50.881 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:14:50.881 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.700932+0000", 2024-09-18T10:14:50.881 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.5483000000000001e-05 2024-09-18T10:14:50.881 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:50.881 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:50.881 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:14:50.881 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.701903+0000", 2024-09-18T10:14:50.881 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00097119100000000005 2024-09-18T10:14:50.881 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:50.881 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:50.881 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:50.881 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.701961+0000", 2024-09-18T10:14:50.881 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.7612000000000003e-05 2024-09-18T10:14:50.882 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:50.882 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:50.882 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:50.882 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:50.882 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:50.882 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.0s3 36/32 ECSubWrite(tid=6, reqid=client.4632.0:89, at_version=0'0, trim_to=0'0, roll_forward_to=36'4))", 2024-09-18T10:14:50.882 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:50.707911+0000", 2024-09-18T10:14:50.882 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 0.14116952199999999, 2024-09-18T10:14:50.882 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00168305, 2024-09-18T10:14:50.882 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:50.882 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:14:50.882 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:50.882 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:50.883 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:50.883 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.707911+0000", 2024-09-18T10:14:50.883 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:50.883 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:50.883 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:50.883 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:50.883 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.707911+0000", 2024-09-18T10:14:50.883 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:50.883 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:50.884 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:50.884 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:50.884 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.707910+0000", 2024-09-18T10:14:50.884 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:14:50.884 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:50.884 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:50.884 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:50.884 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.707954+0000", 2024-09-18T10:14:50.884 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4601000000000002e-05 2024-09-18T10:14:50.884 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:50.884 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:50.884 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:50.884 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.708002+0000", 2024-09-18T10:14:50.884 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.8105000000000001e-05 2024-09-18T10:14:50.885 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:50.885 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:50.885 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:50.885 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.708052+0000", 2024-09-18T10:14:50.885 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9481000000000003e-05 2024-09-18T10:14:50.885 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:50.885 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:50.885 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:50.885 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.708186+0000", 2024-09-18T10:14:50.885 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00013385400000000001 2024-09-18T10:14:50.885 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:50.885 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:50.886 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:14:50.886 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.708201+0000", 2024-09-18T10:14:50.886 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.5519000000000002e-05 2024-09-18T10:14:50.886 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:50.886 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:50.886 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:14:50.886 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.709546+0000", 2024-09-18T10:14:50.886 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0013452539999999999 2024-09-18T10:14:50.886 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:50.886 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:50.886 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:50.886 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.709594+0000", 2024-09-18T10:14:50.886 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.7868999999999999e-05 2024-09-18T10:14:50.887 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:50.887 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:50.887 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:50.887 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:50.887 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:50.887 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpRead(3.0s3 36/32 ECSubRead(tid=7, to_read={3:02f544f0:::smithi18925896-30:head=0,4096,0}, subchunks={3:02f544f0:::smithi18925896-30:head=[0,1]}, attrs_to_read=))", 2024-09-18T10:14:50.887 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:50.708935+0000", 2024-09-18T10:14:50.887 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 0.14014565400000001, 2024-09-18T10:14:50.887 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00060509599999999995, 2024-09-18T10:14:50.887 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:50.887 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:14:50.887 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:50.887 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:50.887 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:50.888 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.708935+0000", 2024-09-18T10:14:50.888 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:50.888 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:50.888 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:50.888 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:50.888 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.708935+0000", 2024-09-18T10:14:50.888 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:50.888 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:50.888 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:50.888 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:50.888 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.708933+0000", 2024-09-18T10:14:50.888 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:14:50.888 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:50.889 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:50.889 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:50.889 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.708958+0000", 2024-09-18T10:14:50.889 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5065000000000001e-05 2024-09-18T10:14:50.889 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:50.889 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:50.889 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:50.889 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.709004+0000", 2024-09-18T10:14:50.889 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.5058000000000001e-05 2024-09-18T10:14:50.889 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:50.889 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:50.889 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:50.889 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.709021+0000", 2024-09-18T10:14:50.889 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7859e-05 2024-09-18T10:14:50.890 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:50.890 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:50.890 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:50.890 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.709105+0000", 2024-09-18T10:14:50.890 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.3639000000000003e-05 2024-09-18T10:14:50.890 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:50.890 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:50.890 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:50.890 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.709540+0000", 2024-09-18T10:14:50.890 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00043512500000000002 2024-09-18T10:14:50.890 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:50.890 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:50.890 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:50.890 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:50.891 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:50.891 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-09-18T10:14:50.892 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T10:14:50.900 INFO:tasks.rados.rados.0.smithi189.stdout:26: finishing write tid 3 to smithi18925896-26 2024-09-18T10:14:50.900 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 26 v 4 (ObjNum 25 snap 0 seq_num 25) dirty exists 2024-09-18T10:14:50.901 INFO:tasks.rados.rados.0.smithi189.stdout:26: left oid 26 (ObjNum 25 snap 0 seq_num 25) 2024-09-18T10:14:50.901 INFO:tasks.rados.rados.0.smithi189.stdout:26: done (15 left) 2024-09-18T10:14:50.901 INFO:tasks.rados.rados.0.smithi189.stdout:45: write initial oid 45 2024-09-18T10:14:50.901 INFO:tasks.rados.rados.0.smithi189.stdout:45: seq_num 44 ranges {0=753664} 2024-09-18T10:14:50.914 INFO:tasks.rados.rados.0.smithi189.stdout:45: writing smithi18925896-45 from 0 to 753664 tid 1 2024-09-18T10:14:50.914 INFO:tasks.rados.rados.0.smithi189.stdout: waiting on 16 2024-09-18T10:14:50.914 INFO:tasks.rados.rados.0.smithi189.stdout:29: finishing write tid 3 to smithi18925896-29 2024-09-18T10:14:50.914 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 29 v 6 (ObjNum 28 snap 0 seq_num 28) dirty exists 2024-09-18T10:14:50.914 INFO:tasks.rados.rados.0.smithi189.stdout:29: left oid 29 (ObjNum 28 snap 0 seq_num 28) 2024-09-18T10:14:50.914 INFO:tasks.rados.rados.0.smithi189.stdout:29: done (15 left) 2024-09-18T10:14:50.914 INFO:tasks.rados.rados.0.smithi189.stdout:46: write initial oid 46 2024-09-18T10:14:50.914 INFO:tasks.rados.rados.0.smithi189.stdout:46: seq_num 45 ranges {0=770048} 2024-09-18T10:14:50.920 INFO:tasks.rados.rados.0.smithi189.stdout:46: writing smithi18925896-46 from 0 to 770048 tid 1 2024-09-18T10:14:50.920 INFO:tasks.rados.rados.0.smithi189.stdout: waiting on 16 2024-09-18T10:14:50.921 INFO:tasks.rados.rados.0.smithi189.stdout:34: finishing write tid 1 to smithi18925896-34 2024-09-18T10:14:50.921 INFO:tasks.rados.rados.0.smithi189.stdout:34: finishing write tid 2 to smithi18925896-34 2024-09-18T10:14:50.921 INFO:tasks.rados.rados.0.smithi189.stdout:27: finishing write tid 2 to smithi18925896-27 2024-09-18T10:14:50.921 INFO:tasks.rados.rados.0.smithi189.stdout:32: finishing write tid 3 to smithi18925896-32 2024-09-18T10:14:50.921 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 32 v 4 (ObjNum 31 snap 0 seq_num 31) dirty exists 2024-09-18T10:14:50.921 INFO:tasks.rados.rados.0.smithi189.stdout:32: left oid 32 (ObjNum 31 snap 0 seq_num 31) 2024-09-18T10:14:50.921 INFO:tasks.rados.rados.0.smithi189.stdout:30: finishing write tid 3 to smithi18925896-30 2024-09-18T10:14:50.921 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 30 v 4 (ObjNum 29 snap 0 seq_num 29) dirty exists 2024-09-18T10:14:50.921 INFO:tasks.rados.rados.0.smithi189.stdout:30: left oid 30 (ObjNum 29 snap 0 seq_num 29) 2024-09-18T10:14:50.921 INFO:tasks.rados.rados.0.smithi189.stdout:30: done (15 left) 2024-09-18T10:14:50.921 INFO:tasks.rados.rados.0.smithi189.stdout:32: done (14 left) 2024-09-18T10:14:50.921 INFO:tasks.rados.rados.0.smithi189.stdout:47: write initial oid 47 2024-09-18T10:14:50.921 INFO:tasks.rados.rados.0.smithi189.stdout:47: seq_num 46 ranges {0=786432} 2024-09-18T10:14:50.926 INFO:tasks.rados.rados.0.smithi189.stdout:47: writing smithi18925896-47 from 0 to 786432 tid 1 2024-09-18T10:14:50.927 INFO:tasks.rados.rados.0.smithi189.stdout:48: write initial oid 48 2024-09-18T10:14:50.927 INFO:tasks.rados.rados.0.smithi189.stdout:48: seq_num 47 ranges {0=425984,425984=376832} 2024-09-18T10:14:50.929 INFO:tasks.ceph.osd.3.smithi189.stderr:2024-09-18T10:14:50.927+0000 7f2440fc7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22396) UID: 0 2024-09-18T10:14:50.929 INFO:tasks.rados.rados.0.smithi189.stdout:48: writing smithi18925896-48 from 0 to 425984 tid 1 2024-09-18T10:14:50.932 INFO:tasks.rados.rados.0.smithi189.stdout:48: writing smithi18925896-48 from 425984 to 802816 tid 2 2024-09-18T10:14:50.932 INFO:tasks.rados.rados.0.smithi189.stdout: waiting on 16 2024-09-18T10:14:50.932 INFO:tasks.rados.rados.0.smithi189.stdout:39: finishing write tid 1 to smithi18925896-39 2024-09-18T10:14:50.932 INFO:tasks.rados.rados.0.smithi189.stdout:38: finishing write tid 1 to smithi18925896-38 2024-09-18T10:14:50.932 INFO:tasks.rados.rados.0.smithi189.stdout:38: finishing write tid 2 to smithi18925896-38 2024-09-18T10:14:50.933 INFO:teuthology.orchestra.run.smithi029.stdout:osd.9: {} 2024-09-18T10:14:50.954 INFO:teuthology.orchestra.run.smithi029.stdout:osd.10: {} 2024-09-18T10:14:50.991 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-09-18T10:14:50.992 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-09-18T10:14:50.992 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:50.992 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:130 3.8s0 3:1162852a:::smithi18925896-44:head [append 0~737280 in=737280b,stat out=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:14:50.992 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:50.718178+0000", 2024-09-18T10:14:50.992 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 0.272979998, 2024-09-18T10:14:50.992 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.27300745999999998, 2024-09-18T10:14:50.992 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:50.992 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "started", 2024-09-18T10:14:50.992 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:14:50.992 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:14:50.992 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:14:50.993 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 130 2024-09-18T10:14:50.993 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:50.993 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:50.993 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:50.993 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:50.993 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.718178+0000", 2024-09-18T10:14:50.993 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:50.993 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:50.993 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:50.993 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:50.993 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.718178+0000", 2024-09-18T10:14:50.993 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:50.993 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:50.993 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:50.994 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:50.994 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.718176+0000", 2024-09-18T10:14:50.994 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999976 2024-09-18T10:14:50.994 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:50.994 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:50.994 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:50.994 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.719726+0000", 2024-09-18T10:14:50.994 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001549901 2024-09-18T10:14:50.994 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:50.994 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:50.994 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:50.994 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.719756+0000", 2024-09-18T10:14:50.994 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0583000000000002e-05 2024-09-18T10:14:50.994 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:50.994 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:50.995 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:50.995 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.719769+0000", 2024-09-18T10:14:50.995 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2969999999999999e-05 2024-09-18T10:14:50.995 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:50.995 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:50.995 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:50.995 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.719876+0000", 2024-09-18T10:14:50.995 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000107031 2024-09-18T10:14:50.995 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:50.995 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:50.995 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:14:50.995 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.720125+0000", 2024-09-18T10:14:50.995 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00024875899999999998 2024-09-18T10:14:50.995 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:50.996 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:50.996 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:14:50.996 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.721998+0000", 2024-09-18T10:14:50.996 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018732359999999999 2024-09-18T10:14:50.996 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:50.996 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:50.996 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:14:50.996 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.724353+0000", 2024-09-18T10:14:50.996 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0023550120000000001 2024-09-18T10:14:50.996 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:50.996 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:50.996 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:50.996 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:50.996 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:50.997 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:131 3.8s0 3:1162852a:::smithi18925896-44:head [setxattr _header (56) in=63b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:14:50.997 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:50.719794+0000", 2024-09-18T10:14:50.997 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 0.27136486700000001, 2024-09-18T10:14:50.997 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.27144372999999999, 2024-09-18T10:14:50.997 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:50.997 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "started", 2024-09-18T10:14:50.997 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:14:50.997 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:14:50.997 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:14:50.997 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 131 2024-09-18T10:14:50.997 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:50.997 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:50.997 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:50.998 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:50.998 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.719794+0000", 2024-09-18T10:14:50.998 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:50.998 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:50.998 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:50.998 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:50.998 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.719794+0000", 2024-09-18T10:14:50.998 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:50.998 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:50.998 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:50.998 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:50.998 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.719793+0000", 2024-09-18T10:14:50.998 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:14:50.999 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:50.999 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:50.999 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:50.999 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.719798+0000", 2024-09-18T10:14:50.999 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.8719999999999999e-06 2024-09-18T10:14:50.999 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:50.999 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:50.999 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:50.999 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.719807+0000", 2024-09-18T10:14:50.999 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.2619999999999998e-06 2024-09-18T10:14:50.999 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:50.999 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:50.999 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:50.999 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.819904+0000", 2024-09-18T10:14:51.000 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10009697400000001 2024-09-18T10:14:51.000 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.000 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.000 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:51.000 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.907660+0000", 2024-09-18T10:14:51.000 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.087756539999999994 2024-09-18T10:14:51.000 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.000 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.000 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:14:51.000 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.907748+0000", 2024-09-18T10:14:51.000 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.7373999999999994e-05 2024-09-18T10:14:51.000 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.000 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.001 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:14:51.001 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.908399+0000", 2024-09-18T10:14:51.001 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00065141899999999998 2024-09-18T10:14:51.001 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:51.001 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:51.001 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:51.001 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.001 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.001 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:132 3.8s0 3:1162852a:::smithi18925896-44:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:14:51.001 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:50.819929+0000", 2024-09-18T10:14:51.001 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 0.17122943299999999, 2024-09-18T10:14:51.001 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.171347837, 2024-09-18T10:14:51.001 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:51.001 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "waiting for rw locks", 2024-09-18T10:14:51.002 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:14:51.002 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:14:51.002 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:14:51.002 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 132 2024-09-18T10:14:51.002 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.002 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:51.002 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.002 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:51.002 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.819929+0000", 2024-09-18T10:14:51.002 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:51.002 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.002 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.002 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:51.002 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.819929+0000", 2024-09-18T10:14:51.003 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:51.003 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.003 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.003 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:51.003 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.819927+0000", 2024-09-18T10:14:51.003 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:14:51.003 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.003 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.003 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:51.003 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.819938+0000", 2024-09-18T10:14:51.003 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0502999999999999e-05 2024-09-18T10:14:51.003 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.003 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.004 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:51.004 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.819962+0000", 2024-09-18T10:14:51.004 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4241999999999999e-05 2024-09-18T10:14:51.004 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.004 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.004 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:51.004 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.819973+0000", 2024-09-18T10:14:51.004 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0730000000000001e-05 2024-09-18T10:14:51.004 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.004 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.004 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:51.004 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.908665+0000", 2024-09-18T10:14:51.005 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.088691995999999995 2024-09-18T10:14:51.005 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.005 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.005 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for rw locks", 2024-09-18T10:14:51.005 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.908807+0000", 2024-09-18T10:14:51.005 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00014202400000000001 2024-09-18T10:14:51.005 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:51.005 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:51.005 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:51.005 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.005 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.005 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWriteReply(3.8s0 36/32 ECSubWriteReply(tid=119, last_complete=36'5, committed=1, applied=1))", 2024-09-18T10:14:51.006 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:50.826752+0000", 2024-09-18T10:14:51.006 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 0.16440665600000001, 2024-09-18T10:14:51.006 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.16455950699999999, 2024-09-18T10:14:51.006 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:51.006 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "queued for pg", 2024-09-18T10:14:51.006 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:51.006 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.006 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:51.006 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.826752+0000", 2024-09-18T10:14:51.006 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:51.006 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.006 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.006 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:51.006 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.826752+0000", 2024-09-18T10:14:51.007 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:51.007 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.007 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.007 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:51.007 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.826750+0000", 2024-09-18T10:14:51.007 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:14:51.007 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.007 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.007 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:51.007 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.826760+0000", 2024-09-18T10:14:51.007 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.1430000000000004e-06 2024-09-18T10:14:51.007 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.007 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.008 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:51.008 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.826784+0000", 2024-09-18T10:14:51.008 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4173e-05 2024-09-18T10:14:51.008 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.008 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.008 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:51.008 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.926879+0000", 2024-09-18T10:14:51.008 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100095641 2024-09-18T10:14:51.008 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:51.008 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:51.008 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:51.008 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.008 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.009 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWriteReply(3.8s0 36/32 ECSubWriteReply(tid=120, last_complete=36'6, committed=1, applied=1))", 2024-09-18T10:14:51.009 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:50.912381+0000", 2024-09-18T10:14:51.009 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 0.078777611999999997, 2024-09-18T10:14:51.009 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.078955077999999998, 2024-09-18T10:14:51.009 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:51.009 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "queued for pg", 2024-09-18T10:14:51.009 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:51.009 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.009 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:51.009 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.912381+0000", 2024-09-18T10:14:51.009 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:51.009 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.009 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.009 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:51.010 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.912381+0000", 2024-09-18T10:14:51.010 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:51.010 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.010 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.010 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:51.010 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.912379+0000", 2024-09-18T10:14:51.010 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:14:51.010 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.010 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.010 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:51.010 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.912390+0000", 2024-09-18T10:14:51.010 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0316000000000001e-05 2024-09-18T10:14:51.010 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.010 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.011 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:51.011 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.912419+0000", 2024-09-18T10:14:51.011 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9349e-05 2024-09-18T10:14:51.011 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.011 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.011 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:51.011 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.912436+0000", 2024-09-18T10:14:51.011 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.6592e-05 2024-09-18T10:14:51.011 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:51.011 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:51.011 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:51.011 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.011 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.011 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.4s4 36/32 ECSubWrite(tid=8, reqid=client.4632.0:133, at_version=36'5, trim_to=0'0, roll_forward_to=36'4))", 2024-09-18T10:14:51.012 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:50.917667+0000", 2024-09-18T10:14:51.012 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 0.073491933999999995, 2024-09-18T10:14:51.012 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.073704795000000004, 2024-09-18T10:14:51.012 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:51.012 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "no flag points reached", 2024-09-18T10:14:51.012 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:51.012 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.012 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:51.012 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.917667+0000", 2024-09-18T10:14:51.012 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:51.012 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.012 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.012 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:51.013 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.917667+0000", 2024-09-18T10:14:51.013 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:51.013 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.013 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.013 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:51.013 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.917666+0000", 2024-09-18T10:14:51.013 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:14:51.013 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.013 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.013 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:51.013 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.918322+0000", 2024-09-18T10:14:51.013 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00065669700000000003 2024-09-18T10:14:51.013 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.013 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.014 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:51.014 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.918338+0000", 2024-09-18T10:14:51.014 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.562e-05 2024-09-18T10:14:51.014 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:51.014 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:51.014 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:51.014 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.014 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.014 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWriteReply(3.8s0 36/32 ECSubWriteReply(tid=119, last_complete=36'5, committed=1, applied=1))", 2024-09-18T10:14:51.014 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:50.926964+0000", 2024-09-18T10:14:51.014 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 0.064194904999999997, 2024-09-18T10:14:51.014 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.064429138999999996, 2024-09-18T10:14:51.014 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:51.015 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "queued for pg", 2024-09-18T10:14:51.015 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:51.015 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.015 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:51.015 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.926964+0000", 2024-09-18T10:14:51.015 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:51.015 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.015 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.015 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:51.015 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.926964+0000", 2024-09-18T10:14:51.015 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:51.015 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.015 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.015 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:51.016 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.926963+0000", 2024-09-18T10:14:51.016 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999995 2024-09-18T10:14:51.016 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.016 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.016 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:51.016 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.926968+0000", 2024-09-18T10:14:51.016 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.4820000000000002e-06 2024-09-18T10:14:51.016 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.016 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.016 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:51.016 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.926982+0000", 2024-09-18T10:14:51.016 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3715e-05 2024-09-18T10:14:51.016 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.017 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.017 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:51.017 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.926994+0000", 2024-09-18T10:14:51.017 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1755e-05 2024-09-18T10:14:51.017 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:51.017 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:51.017 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:51.017 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.017 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.017 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWriteReply(3.8s0 36/32 ECSubWriteReply(tid=120, last_complete=36'6, committed=1, applied=1))", 2024-09-18T10:14:51.017 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:50.928034+0000", 2024-09-18T10:14:51.017 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 0.063124069000000005, 2024-09-18T10:14:51.017 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.063381538000000001, 2024-09-18T10:14:51.017 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:51.018 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "queued for pg", 2024-09-18T10:14:51.018 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:51.018 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.018 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:51.018 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.928034+0000", 2024-09-18T10:14:51.018 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:51.018 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.018 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.018 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:51.018 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.928034+0000", 2024-09-18T10:14:51.018 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:51.018 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.018 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.018 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:51.019 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.928034+0000", 2024-09-18T10:14:51.019 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:14:51.019 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.019 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.019 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:51.019 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.928039+0000", 2024-09-18T10:14:51.019 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9300000000000002e-06 2024-09-18T10:14:51.019 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.019 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.019 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:51.019 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.928051+0000", 2024-09-18T10:14:51.019 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2887999999999999e-05 2024-09-18T10:14:51.019 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.019 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.019 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:51.020 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.928057+0000", 2024-09-18T10:14:51.020 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.6710000000000004e-06 2024-09-18T10:14:51.020 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:51.020 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:51.020 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:51.020 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.020 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.020 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWriteReply(3.8s0 36/32 ECSubWriteReply(tid=120, last_complete=36'6, committed=1, applied=1))", 2024-09-18T10:14:51.020 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:50.928427+0000", 2024-09-18T10:14:51.020 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 0.062731037000000003, 2024-09-18T10:14:51.020 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.063011249000000005, 2024-09-18T10:14:51.020 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:51.020 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "queued for pg", 2024-09-18T10:14:51.020 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:51.021 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.021 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:51.021 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.928427+0000", 2024-09-18T10:14:51.021 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:51.021 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.021 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.021 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:51.021 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.928427+0000", 2024-09-18T10:14:51.021 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:51.021 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.021 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.021 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:51.021 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.928427+0000", 2024-09-18T10:14:51.022 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999995 2024-09-18T10:14:51.022 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.022 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.022 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:51.022 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.928431+0000", 2024-09-18T10:14:51.022 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.692e-06 2024-09-18T10:14:51.022 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.022 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.022 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:51.022 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.928441+0000", 2024-09-18T10:14:51.022 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0504e-05 2024-09-18T10:14:51.022 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.022 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.022 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:51.023 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.928446+0000", 2024-09-18T10:14:51.023 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.5419999999999999e-06 2024-09-18T10:14:51.023 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:51.023 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:51.023 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:51.023 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:51.023 INFO:teuthology.orchestra.run.smithi154.stdout: ], 2024-09-18T10:14:51.023 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 9 2024-09-18T10:14:51.023 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-09-18T10:14:51.024 INFO:tasks.rados.rados.0.smithi189.stdout:41: finishing write tid 1 to smithi18925896-41 2024-09-18T10:14:51.024 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T10:14:51.030 INFO:tasks.ceph.osd.10.smithi154.stderr:2024-09-18T10:14:51.024+0000 7fb73221a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 22091) UID: 0 2024-09-18T10:14:51.053 INFO:teuthology.orchestra.run.smithi029.stdout:osd.11: {} 2024-09-18T10:14:51.076 INFO:teuthology.orchestra.run.smithi029.stdout:osd.12: {} 2024-09-18T10:14:51.105 INFO:tasks.rados.rados.0.smithi189.stdout:40: finishing write tid 1 to smithi18925896-40 2024-09-18T10:14:51.107 INFO:teuthology.orchestra.run.smithi029.stdout:osd.13: {} 2024-09-18T10:14:51.108 INFO:tasks.rados.rados.0.smithi189.stdout:41: finishing write tid 2 to smithi18925896-41 2024-09-18T10:14:51.110 INFO:tasks.rados.rados.0.smithi189.stdout:34: finishing write tid 3 to smithi18925896-34 2024-09-18T10:14:51.110 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 34 v 6 (ObjNum 33 snap 0 seq_num 33) dirty exists 2024-09-18T10:14:51.110 INFO:tasks.rados.rados.0.smithi189.stdout:34: left oid 34 (ObjNum 33 snap 0 seq_num 33) 2024-09-18T10:14:51.110 INFO:tasks.rados.rados.0.smithi189.stdout:34: done (15 left) 2024-09-18T10:14:51.110 INFO:tasks.rados.rados.0.smithi189.stdout:49: write initial oid 49 2024-09-18T10:14:51.111 INFO:tasks.rados.rados.0.smithi189.stdout:49: seq_num 48 ranges {0=442368,442368=376832} 2024-09-18T10:14:51.117 INFO:tasks.rados.rados.0.smithi189.stdout:49: writing smithi18925896-49 from 0 to 442368 tid 1 2024-09-18T10:14:51.122 INFO:tasks.rados.rados.0.smithi189.stdout:49: writing smithi18925896-49 from 442368 to 819200 tid 2 2024-09-18T10:14:51.122 INFO:tasks.rados.rados.0.smithi189.stdout: waiting on 16 2024-09-18T10:14:51.122 INFO:tasks.rados.rados.0.smithi189.stdout:36: finishing write tid 1 to smithi18925896-36 2024-09-18T10:14:51.122 INFO:tasks.rados.rados.0.smithi189.stdout:36: finishing write tid 2 to smithi18925896-36 2024-09-18T10:14:51.122 INFO:tasks.rados.rados.0.smithi189.stdout:42: finishing write tid 1 to smithi18925896-42 2024-09-18T10:14:51.122 INFO:tasks.rados.rados.0.smithi189.stdout:40: finishing write tid 2 to smithi18925896-40 2024-09-18T10:14:51.122 INFO:tasks.rados.rados.0.smithi189.stdout:27: finishing write tid 3 to smithi18925896-27 2024-09-18T10:14:51.122 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 27 v 8 (ObjNum 26 snap 0 seq_num 26) dirty exists 2024-09-18T10:14:51.123 INFO:tasks.rados.rados.0.smithi189.stdout:27: left oid 27 (ObjNum 26 snap 0 seq_num 26) 2024-09-18T10:14:51.123 INFO:tasks.rados.rados.0.smithi189.stdout:42: finishing write tid 2 to smithi18925896-42 2024-09-18T10:14:51.123 INFO:tasks.rados.rados.0.smithi189.stdout:43: finishing write tid 1 to smithi18925896-43 2024-09-18T10:14:51.123 INFO:tasks.rados.rados.0.smithi189.stdout:44: finishing write tid 1 to smithi18925896-44 2024-09-18T10:14:51.123 INFO:tasks.rados.rados.0.smithi189.stdout:27: done (15 left) 2024-09-18T10:14:51.123 INFO:tasks.rados.rados.0.smithi189.stdout:50: write initial oid 50 2024-09-18T10:14:51.123 INFO:tasks.rados.rados.0.smithi189.stdout:50: seq_num 49 ranges {0=458752,458752=376832} 2024-09-18T10:14:51.124 INFO:tasks.rados.rados.0.smithi189.stdout:50: writing smithi18925896-50 from 0 to 458752 tid 1 2024-09-18T10:14:51.127 INFO:tasks.rados.rados.0.smithi189.stdout:50: writing smithi18925896-50 from 458752 to 835584 tid 2 2024-09-18T10:14:51.127 INFO:tasks.rados.rados.0.smithi189.stdout: waiting on 16 2024-09-18T10:14:51.127 INFO:tasks.rados.rados.0.smithi189.stdout:45: finishing write tid 1 to smithi18925896-45 2024-09-18T10:14:51.127 INFO:tasks.rados.rados.0.smithi189.stdout:39: finishing write tid 2 to smithi18925896-39 2024-09-18T10:14:51.127 INFO:tasks.rados.rados.0.smithi189.stdout:46: finishing write tid 1 to smithi18925896-46 2024-09-18T10:14:51.130 INFO:tasks.ceph.osd.12.smithi029.stderr:2024-09-18T10:14:51.126+0000 7f8e4e627700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 22190) UID: 0 2024-09-18T10:14:51.137 INFO:teuthology.orchestra.run.smithi029.stdout:osd.14: {} 2024-09-18T10:14:51.138 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-09-18T10:14:51.139 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-09-18T10:14:51.139 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-09-18T10:14:51.139 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-09-18T10:14:51.139 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-09-18T10:14:51.148 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T10:14:51.158 INFO:teuthology.orchestra.run.smithi029.stdout:osd.15: {} 2024-09-18T10:14:51.161 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T10:14:51.161 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T10:14:51.161 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T10:14:51.162 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T10:14:51.162 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T10:14:51.162 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T10:14:51.162 INFO:teuthology.orchestra.run.smithi029.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T10:14:51.162 INFO:teuthology.orchestra.run.smithi029.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T10:14:51.162 INFO:teuthology.orchestra.run.smithi029.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-18T10:14:51.162 INFO:teuthology.orchestra.run.smithi029.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-18T10:14:51.162 INFO:teuthology.orchestra.run.smithi029.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-18T10:14:51.162 INFO:teuthology.orchestra.run.smithi029.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-18T10:14:51.162 INFO:teuthology.orchestra.run.smithi029.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-18T10:14:51.162 INFO:teuthology.orchestra.run.smithi029.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-18T10:14:51.163 INFO:teuthology.orchestra.run.smithi029.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-18T10:14:51.163 INFO:teuthology.orchestra.run.smithi029.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-18T10:14:51.211 INFO:tasks.rados.rados.0.smithi189.stdout:41: finishing write tid 3 to smithi18925896-41 2024-09-18T10:14:51.211 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 41 v 10 (ObjNum 40 snap 0 seq_num 40) dirty exists 2024-09-18T10:14:51.211 INFO:tasks.rados.rados.0.smithi189.stdout:41: left oid 41 (ObjNum 40 snap 0 seq_num 40) 2024-09-18T10:14:51.211 INFO:tasks.rados.rados.0.smithi189.stdout:41: done (15 left) 2024-09-18T10:14:51.211 INFO:tasks.rados.rados.0.smithi189.stdout:51: copy_from oid 2 from oid 11 current snap is 0 2024-09-18T10:14:51.211 INFO:tasks.rados.rados.0.smithi189.stdout: waiting on 16 2024-09-18T10:14:51.213 INFO:tasks.rados.rados.0.smithi189.stdout:44: finishing write tid 2 to smithi18925896-44 2024-09-18T10:14:51.215 INFO:tasks.rados.rados.0.smithi189.stdout:40: finishing write tid 3 to smithi18925896-40 2024-09-18T10:14:51.215 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 40 v 4 (ObjNum 39 snap 0 seq_num 39) dirty exists 2024-09-18T10:14:51.215 INFO:tasks.rados.rados.0.smithi189.stdout:40: left oid 40 (ObjNum 39 snap 0 seq_num 39) 2024-09-18T10:14:51.215 INFO:tasks.rados.rados.0.smithi189.stdout:40: done (15 left) 2024-09-18T10:14:51.215 INFO:tasks.rados.rados.0.smithi189.stdout:append oid 25 current snap is 0 2024-09-18T10:14:51.215 INFO:tasks.rados.rados.0.smithi189.stdout:52: seq_num 50 ranges {409600=475136,884736=376832} 2024-09-18T10:14:51.224 INFO:tasks.rados.rados.0.smithi189.stdout:52: writing smithi18925896-25 from 409600 to 884736 tid 1 2024-09-18T10:14:51.227 INFO:tasks.rados.rados.0.smithi189.stdout:52: writing smithi18925896-25 from 884736 to 1261568 tid 2 2024-09-18T10:14:51.227 INFO:tasks.rados.rados.0.smithi189.stdout: waiting on 16 2024-09-18T10:14:51.227 INFO:tasks.rados.rados.0.smithi189.stdout:43: finishing write tid 2 to smithi18925896-43 2024-09-18T10:14:51.227 INFO:tasks.rados.rados.0.smithi189.stdout:35: finishing write tid 3 to smithi18925896-35 2024-09-18T10:14:51.227 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 35 v 2 (ObjNum 34 snap 0 seq_num 34) dirty exists 2024-09-18T10:14:51.227 INFO:tasks.rados.rados.0.smithi189.stdout:35: left oid 35 (ObjNum 34 snap 0 seq_num 34) 2024-09-18T10:14:51.228 INFO:tasks.rados.rados.0.smithi189.stdout:35: done (15 left) 2024-09-18T10:14:51.228 INFO:tasks.rados.rados.0.smithi189.stdout:53: rmattr oid 30 current snap is 0 2024-09-18T10:14:51.228 INFO:tasks.rados.rados.0.smithi189.stdout:53: done (15 left) 2024-09-18T10:14:51.228 INFO:tasks.rados.rados.0.smithi189.stdout:54: delete oid 3 current snap is 0 2024-09-18T10:14:51.231 INFO:tasks.ceph.osd.6.smithi154.stderr:2024-09-18T10:14:51.228+0000 7f0abc9e1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22089) UID: 0 2024-09-18T10:14:51.232 INFO:tasks.rados.rados.0.smithi189.stdout:36: finishing write tid 3 to smithi18925896-36 2024-09-18T10:14:51.232 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 36 v 8 (ObjNum 35 snap 0 seq_num 35) dirty exists 2024-09-18T10:14:51.232 INFO:tasks.rados.rados.0.smithi189.stdout:36: left oid 36 (ObjNum 35 snap 0 seq_num 35) 2024-09-18T10:14:51.234 INFO:tasks.rados.rados.0.smithi189.stdout:46: finishing write tid 2 to smithi18925896-46 2024-09-18T10:14:51.235 INFO:tasks.rados.rados.0.smithi189.stdout:38: finishing write tid 3 to smithi18925896-38 2024-09-18T10:14:51.235 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 38 v 8 (ObjNum 37 snap 0 seq_num 37) dirty exists 2024-09-18T10:14:51.235 INFO:tasks.rados.rados.0.smithi189.stdout:38: left oid 38 (ObjNum 37 snap 0 seq_num 37) 2024-09-18T10:14:51.251 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-09-18T10:14:51.251 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-09-18T10:14:51.251 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-09-18T10:14:51.251 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-09-18T10:14:51.251 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.251 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:45 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-18T10:14:51.251 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:35.093050+0000", 2024-09-18T10:14:51.251 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 16.154668605000001, 2024-09-18T10:14:51.251 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.50958515100000001, 2024-09-18T10:14:51.252 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:51.252 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:14:51.252 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:14:51.252 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:14:51.252 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:14:51.252 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 45 2024-09-18T10:14:51.252 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.252 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:51.252 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.252 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:51.252 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.093050+0000", 2024-09-18T10:14:51.253 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:51.253 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.253 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.253 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:51.253 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.093050+0000", 2024-09-18T10:14:51.253 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:51.253 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.253 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.253 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:51.253 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.093049+0000", 2024-09-18T10:14:51.253 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:14:51.253 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.253 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.253 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:51.254 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.093064+0000", 2024-09-18T10:14:51.254 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.5492999999999999e-05 2024-09-18T10:14:51.254 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.254 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.254 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:51.254 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.093076+0000", 2024-09-18T10:14:51.254 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1112e-05 2024-09-18T10:14:51.254 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.254 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.254 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:51.254 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.093081+0000", 2024-09-18T10:14:51.254 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5929999999999996e-06 2024-09-18T10:14:51.254 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.254 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.255 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:51.255 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.198774+0000", 2024-09-18T10:14:51.255 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.105692676 2024-09-18T10:14:51.255 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.255 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.255 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:14:51.255 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.198906+0000", 2024-09-18T10:14:51.255 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00013173300000000001 2024-09-18T10:14:51.255 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.255 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.255 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:14:51.255 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.199104+0000", 2024-09-18T10:14:51.255 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00019817500000000001 2024-09-18T10:14:51.255 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.256 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.256 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:14:51.256 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.203616+0000", 2024-09-18T10:14:51.256 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0045118529999999997 2024-09-18T10:14:51.256 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.256 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.256 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:14:51.256 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.602486+0000", 2024-09-18T10:14:51.256 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.39887087500000001 2024-09-18T10:14:51.256 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.256 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.256 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:14:51.256 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.602575+0000", 2024-09-18T10:14:51.257 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.8290999999999999e-05 2024-09-18T10:14:51.257 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.257 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.257 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:51.257 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.602635+0000", 2024-09-18T10:14:51.257 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.0170000000000002e-05 2024-09-18T10:14:51.257 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:51.257 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:51.257 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:51.257 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.257 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.257 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-18T10:14:51.257 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:35.093089+0000", 2024-09-18T10:14:51.257 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 16.154629791000001, 2024-09-18T10:14:51.258 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.50974395900000002, 2024-09-18T10:14:51.258 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:51.258 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:14:51.258 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:14:51.258 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:14:51.258 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:14:51.258 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 46 2024-09-18T10:14:51.258 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.258 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:51.258 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.258 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:51.258 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.093089+0000", 2024-09-18T10:14:51.259 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:51.259 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.259 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.259 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:51.259 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.093089+0000", 2024-09-18T10:14:51.259 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:51.259 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.259 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.259 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:51.259 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.093088+0000", 2024-09-18T10:14:51.259 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999995 2024-09-18T10:14:51.259 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.259 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.260 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:51.260 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.093093+0000", 2024-09-18T10:14:51.260 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.0440000000000003e-06 2024-09-18T10:14:51.260 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.260 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.260 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:51.260 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.093101+0000", 2024-09-18T10:14:51.260 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.5490000000000006e-06 2024-09-18T10:14:51.260 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.260 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.260 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:51.260 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.093109+0000", 2024-09-18T10:14:51.260 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.1330000000000001e-06 2024-09-18T10:14:51.261 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.261 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.261 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:51.261 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.199767+0000", 2024-09-18T10:14:51.261 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.106658379 2024-09-18T10:14:51.261 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.261 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.261 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:14:51.261 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.199922+0000", 2024-09-18T10:14:51.261 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000154797 2024-09-18T10:14:51.261 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.261 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.261 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:14:51.262 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.200138+0000", 2024-09-18T10:14:51.262 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00021600100000000001 2024-09-18T10:14:51.262 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.262 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.262 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:14:51.262 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.203626+0000", 2024-09-18T10:14:51.262 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0034878470000000001 2024-09-18T10:14:51.262 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.262 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.262 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:14:51.262 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.602720+0000", 2024-09-18T10:14:51.262 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.399094646 2024-09-18T10:14:51.262 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.262 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.262 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:14:51.263 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.602787+0000", 2024-09-18T10:14:51.263 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.6878999999999994e-05 2024-09-18T10:14:51.263 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.263 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.263 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:51.263 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.602833+0000", 2024-09-18T10:14:51.263 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.5393999999999999e-05 2024-09-18T10:14:51.263 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:51.263 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:51.263 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:51.263 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.263 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.263 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-18T10:14:51.263 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:35.093116+0000", 2024-09-18T10:14:51.263 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 16.154602422, 2024-09-18T10:14:51.263 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.51000002600000005, 2024-09-18T10:14:51.264 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:51.264 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:14:51.264 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:14:51.264 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:14:51.264 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:14:51.264 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 47 2024-09-18T10:14:51.264 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.264 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:51.264 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.265 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:51.265 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.093116+0000", 2024-09-18T10:14:51.265 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:51.265 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.265 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.265 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:51.265 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.093116+0000", 2024-09-18T10:14:51.265 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:51.265 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.265 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.265 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:51.266 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.093115+0000", 2024-09-18T10:14:51.266 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:14:51.266 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.266 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.266 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:51.266 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.093198+0000", 2024-09-18T10:14:51.266 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.2395e-05 2024-09-18T10:14:51.266 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.266 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.266 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:51.266 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.093208+0000", 2024-09-18T10:14:51.266 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.9960000000000005e-06 2024-09-18T10:14:51.267 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.267 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.267 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:51.267 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.093214+0000", 2024-09-18T10:14:51.267 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.1929999999999998e-06 2024-09-18T10:14:51.268 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.268 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.268 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:51.268 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.200843+0000", 2024-09-18T10:14:51.268 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.107629271 2024-09-18T10:14:51.269 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.269 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.269 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:14:51.269 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.200996+0000", 2024-09-18T10:14:51.269 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00015333 2024-09-18T10:14:51.270 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.270 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.270 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:14:51.270 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.201196+0000", 2024-09-18T10:14:51.270 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00019938900000000001 2024-09-18T10:14:51.270 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.270 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.270 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:14:51.270 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.204200+0000", 2024-09-18T10:14:51.270 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030046399999999998 2024-09-18T10:14:51.270 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.270 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.271 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:14:51.271 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.602968+0000", 2024-09-18T10:14:51.271 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.39876762100000002 2024-09-18T10:14:51.271 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.271 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.272 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:14:51.272 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.603056+0000", 2024-09-18T10:14:51.272 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.8158000000000005e-05 2024-09-18T10:14:51.272 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.272 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.272 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:51.272 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.603116+0000", 2024-09-18T10:14:51.272 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.9753000000000002e-05 2024-09-18T10:14:51.272 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:51.272 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:51.272 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:51.272 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.273 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.273 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-18T10:14:51.273 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:35.093224+0000", 2024-09-18T10:14:51.273 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 16.154494089, 2024-09-18T10:14:51.273 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.51036105200000004, 2024-09-18T10:14:51.273 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:51.273 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:14:51.273 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:14:51.273 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:14:51.273 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:14:51.273 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 48 2024-09-18T10:14:51.273 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.273 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:51.274 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.274 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:51.274 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.093224+0000", 2024-09-18T10:14:51.274 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:51.274 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.274 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.274 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:51.274 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.093224+0000", 2024-09-18T10:14:51.274 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:51.274 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.274 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.274 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:51.274 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.093223+0000", 2024-09-18T10:14:51.275 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:14:51.275 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.275 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.275 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:51.275 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.093229+0000", 2024-09-18T10:14:51.275 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.3229999999999997e-06 2024-09-18T10:14:51.275 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.275 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.275 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:51.275 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.093239+0000", 2024-09-18T10:14:51.275 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0063e-05 2024-09-18T10:14:51.275 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.275 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.275 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:51.276 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.193381+0000", 2024-09-18T10:14:51.276 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100142232 2024-09-18T10:14:51.276 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.276 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.276 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:51.276 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.293719+0000", 2024-09-18T10:14:51.276 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100337421 2024-09-18T10:14:51.276 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.276 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.276 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:14:51.276 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.293866+0000", 2024-09-18T10:14:51.276 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00014781199999999999 2024-09-18T10:14:51.277 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.277 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.277 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:14:51.277 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.294072+0000", 2024-09-18T10:14:51.277 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0002052 2024-09-18T10:14:51.277 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.277 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.277 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:14:51.277 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.300276+0000", 2024-09-18T10:14:51.277 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0062046940000000002 2024-09-18T10:14:51.278 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.278 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.278 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:14:51.278 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.603433+0000", 2024-09-18T10:14:51.278 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.30315636200000001 2024-09-18T10:14:51.278 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.278 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.278 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:14:51.278 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.603525+0000", 2024-09-18T10:14:51.278 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.2394999999999999e-05 2024-09-18T10:14:51.278 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.278 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.279 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:51.279 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.603585+0000", 2024-09-18T10:14:51.279 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.0343000000000001e-05 2024-09-18T10:14:51.279 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:51.279 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:51.279 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:51.279 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.279 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.279 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-18T10:14:51.279 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:35.293815+0000", 2024-09-18T10:14:51.279 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 15.953903572, 2024-09-18T10:14:51.279 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.513288525, 2024-09-18T10:14:51.279 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:51.279 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:14:51.280 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:14:51.280 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:14:51.280 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:14:51.280 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 52 2024-09-18T10:14:51.280 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.280 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:51.280 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.280 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:51.280 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.293815+0000", 2024-09-18T10:14:51.280 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:51.281 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.281 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.281 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:51.281 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.293815+0000", 2024-09-18T10:14:51.281 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:51.281 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.281 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.281 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:51.281 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.293814+0000", 2024-09-18T10:14:51.281 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:14:51.281 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.281 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.281 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:51.282 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.293822+0000", 2024-09-18T10:14:51.282 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.8089999999999993e-06 2024-09-18T10:14:51.282 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.282 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.282 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:51.282 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.293833+0000", 2024-09-18T10:14:51.282 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1389000000000001e-05 2024-09-18T10:14:51.282 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.282 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.282 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:51.282 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.393937+0000", 2024-09-18T10:14:51.282 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10010415 2024-09-18T10:14:51.282 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.283 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.283 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:51.283 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.704491+0000", 2024-09-18T10:14:51.283 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.310554202 2024-09-18T10:14:51.283 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.283 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.283 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:14:51.283 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.704647+0000", 2024-09-18T10:14:51.283 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00015524399999999999 2024-09-18T10:14:51.283 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.283 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.283 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:14:51.283 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.704859+0000", 2024-09-18T10:14:51.283 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000212664 2024-09-18T10:14:51.284 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.284 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.284 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:14:51.284 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.705935+0000", 2024-09-18T10:14:51.284 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001075505 2024-09-18T10:14:51.284 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.284 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.284 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:14:51.284 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.806952+0000", 2024-09-18T10:14:51.284 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10101700299999999 2024-09-18T10:14:51.284 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.284 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.285 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:14:51.285 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.807041+0000", 2024-09-18T10:14:51.285 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.8811000000000001e-05 2024-09-18T10:14:51.285 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.285 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.285 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:51.285 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.807103+0000", 2024-09-18T10:14:51.285 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.2596999999999997e-05 2024-09-18T10:14:51.285 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:51.285 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:51.285 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:51.285 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.285 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.286 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:70 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T10:14:51.286 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:36.786853+0000", 2024-09-18T10:14:51.286 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 14.460865611999999, 2024-09-18T10:14:51.286 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.51178262299999999, 2024-09-18T10:14:51.286 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:51.286 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:14:51.286 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:14:51.286 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:14:51.286 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:14:51.286 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 70 2024-09-18T10:14:51.286 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.286 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:51.286 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.286 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:51.287 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.786853+0000", 2024-09-18T10:14:51.287 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:51.287 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.287 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.287 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:51.287 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.786853+0000", 2024-09-18T10:14:51.287 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:51.287 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.287 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.287 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:51.287 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.786851+0000", 2024-09-18T10:14:51.287 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:14:51.287 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.287 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.287 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:51.288 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.786861+0000", 2024-09-18T10:14:51.288 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.8700000000000004e-06 2024-09-18T10:14:51.288 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.288 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.288 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:51.288 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.786877+0000", 2024-09-18T10:14:51.288 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.5569e-05 2024-09-18T10:14:51.288 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.288 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.288 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:51.288 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.786887+0000", 2024-09-18T10:14:51.288 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0526000000000001e-05 2024-09-18T10:14:51.288 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.289 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.289 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:51.289 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.292790+0000", 2024-09-18T10:14:51.289 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.50590225600000005 2024-09-18T10:14:51.289 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.289 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.289 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:14:51.289 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.292929+0000", 2024-09-18T10:14:51.289 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00013913200000000001 2024-09-18T10:14:51.289 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.289 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.289 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:14:51.289 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.293137+0000", 2024-09-18T10:14:51.290 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00020826199999999999 2024-09-18T10:14:51.290 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.290 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.290 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:14:51.290 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.297461+0000", 2024-09-18T10:14:51.290 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0043237299999999996 2024-09-18T10:14:51.290 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.290 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.290 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:14:51.290 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.298490+0000", 2024-09-18T10:14:51.290 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0010289769999999999 2024-09-18T10:14:51.290 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.290 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.291 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:14:51.291 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.298572+0000", 2024-09-18T10:14:51.291 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.2151999999999995e-05 2024-09-18T10:14:51.291 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.291 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.291 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:51.291 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.298635+0000", 2024-09-18T10:14:51.291 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.3468999999999998e-05 2024-09-18T10:14:51.291 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:51.291 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:51.291 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:51.292 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.292 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.292 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:71 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T10:14:51.292 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:36.786902+0000", 2024-09-18T10:14:51.292 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 14.460816808000001, 2024-09-18T10:14:51.292 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.51197160200000003, 2024-09-18T10:14:51.292 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:51.292 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:14:51.292 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:14:51.292 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:14:51.292 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:14:51.292 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 71 2024-09-18T10:14:51.292 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.293 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:51.293 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.293 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:51.293 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.786902+0000", 2024-09-18T10:14:51.293 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:51.293 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.293 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.293 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:51.293 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.786902+0000", 2024-09-18T10:14:51.293 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:51.293 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.293 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.293 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:51.293 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.786900+0000", 2024-09-18T10:14:51.294 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:14:51.294 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.294 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.294 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:51.294 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.786909+0000", 2024-09-18T10:14:51.294 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.9430000000000006e-06 2024-09-18T10:14:51.294 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.294 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.294 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:51.294 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.786921+0000", 2024-09-18T10:14:51.294 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1789e-05 2024-09-18T10:14:51.294 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.294 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.295 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:51.295 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.786927+0000", 2024-09-18T10:14:51.295 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5770000000000001e-06 2024-09-18T10:14:51.295 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.295 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.295 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:51.295 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.293768+0000", 2024-09-18T10:14:51.295 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.50684175499999995 2024-09-18T10:14:51.295 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.295 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.295 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:14:51.295 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.293894+0000", 2024-09-18T10:14:51.295 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00012588899999999999 2024-09-18T10:14:51.295 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.295 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.296 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:14:51.296 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.294094+0000", 2024-09-18T10:14:51.296 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000199896 2024-09-18T10:14:51.296 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.296 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.296 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:14:51.296 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.298660+0000", 2024-09-18T10:14:51.296 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.004566082 2024-09-18T10:14:51.296 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.296 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.296 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:14:51.296 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.298739+0000", 2024-09-18T10:14:51.296 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.8269000000000006e-05 2024-09-18T10:14:51.297 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.297 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.297 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:14:51.297 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.298814+0000", 2024-09-18T10:14:51.297 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.5582999999999998e-05 2024-09-18T10:14:51.297 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.297 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.297 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:51.297 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.298873+0000", 2024-09-18T10:14:51.297 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.8989000000000001e-05 2024-09-18T10:14:51.297 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:51.297 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:51.297 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:51.297 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.298 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.298 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T10:14:51.298 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:36.786936+0000", 2024-09-18T10:14:51.298 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 14.460782610000001, 2024-09-18T10:14:51.298 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.71130157999999999, 2024-09-18T10:14:51.298 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:51.298 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:14:51.298 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:14:51.298 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:14:51.298 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:14:51.298 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 72 2024-09-18T10:14:51.298 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.298 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:51.299 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.299 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:51.299 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.786936+0000", 2024-09-18T10:14:51.299 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:51.299 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.299 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.299 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:51.299 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.786936+0000", 2024-09-18T10:14:51.299 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:51.299 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.299 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.299 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:51.299 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.786935+0000", 2024-09-18T10:14:51.300 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999995 2024-09-18T10:14:51.300 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.300 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.300 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:51.300 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787018+0000", 2024-09-18T10:14:51.300 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.2511999999999999e-05 2024-09-18T10:14:51.300 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.300 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.300 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:51.300 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787028+0000", 2024-09-18T10:14:51.300 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0709e-05 2024-09-18T10:14:51.300 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.301 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.301 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:51.301 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787034+0000", 2024-09-18T10:14:51.301 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.7769999999999999e-06 2024-09-18T10:14:51.301 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.301 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.301 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:51.301 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.295095+0000", 2024-09-18T10:14:51.301 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.50806109099999996 2024-09-18T10:14:51.301 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.301 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.301 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:14:51.301 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.295221+0000", 2024-09-18T10:14:51.302 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00012563599999999999 2024-09-18T10:14:51.302 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.302 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.302 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:14:51.302 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.295453+0000", 2024-09-18T10:14:51.302 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000232387 2024-09-18T10:14:51.302 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.302 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.302 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:14:51.302 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.301373+0000", 2024-09-18T10:14:51.302 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0059202279999999996 2024-09-18T10:14:51.303 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.303 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.303 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:14:51.303 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.498044+0000", 2024-09-18T10:14:51.303 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.19667104699999999 2024-09-18T10:14:51.303 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.303 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.303 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:14:51.303 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.498133+0000", 2024-09-18T10:14:51.304 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.9044999999999999e-05 2024-09-18T10:14:51.304 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.304 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.304 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:51.304 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.498237+0000", 2024-09-18T10:14:51.304 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00010385100000000001 2024-09-18T10:14:51.304 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:51.304 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:51.304 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:51.304 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.304 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.304 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T10:14:51.304 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:36.787042+0000", 2024-09-18T10:14:51.305 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 14.460675903, 2024-09-18T10:14:51.305 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.71152325100000002, 2024-09-18T10:14:51.305 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:51.305 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:14:51.305 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:14:51.305 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:14:51.305 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:14:51.305 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 73 2024-09-18T10:14:51.305 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.305 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:51.305 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.305 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:51.306 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787042+0000", 2024-09-18T10:14:51.306 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:51.306 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.306 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.306 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:51.306 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787042+0000", 2024-09-18T10:14:51.306 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:51.306 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.306 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.306 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:51.306 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787042+0000", 2024-09-18T10:14:51.306 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:14:51.306 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.307 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.307 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:51.307 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787048+0000", 2024-09-18T10:14:51.307 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.8900000000000001e-06 2024-09-18T10:14:51.307 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.307 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.307 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:51.307 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787057+0000", 2024-09-18T10:14:51.307 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.4430000000000003e-06 2024-09-18T10:14:51.307 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.307 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.307 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:51.307 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787063+0000", 2024-09-18T10:14:51.308 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.6459999999999998e-06 2024-09-18T10:14:51.308 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.308 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.308 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:51.308 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.297548+0000", 2024-09-18T10:14:51.308 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.51048589499999997 2024-09-18T10:14:51.308 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.308 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.308 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:14:51.308 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.297668+0000", 2024-09-18T10:14:51.308 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00011917 2024-09-18T10:14:51.308 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.308 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.308 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:14:51.309 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.297891+0000", 2024-09-18T10:14:51.309 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00022301999999999999 2024-09-18T10:14:51.309 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.309 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.309 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:14:51.309 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.301383+0000", 2024-09-18T10:14:51.309 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.003492762 2024-09-18T10:14:51.309 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.309 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.309 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:14:51.309 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.498374+0000", 2024-09-18T10:14:51.309 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.19699053899999999 2024-09-18T10:14:51.309 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.309 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.310 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:14:51.310 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.498506+0000", 2024-09-18T10:14:51.310 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00013246899999999999 2024-09-18T10:14:51.310 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.310 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.310 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:51.310 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.498566+0000", 2024-09-18T10:14:51.310 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.9274000000000002e-05 2024-09-18T10:14:51.310 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:51.310 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:51.310 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:51.310 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.311 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.311 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T10:14:51.311 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:36.787128+0000", 2024-09-18T10:14:51.311 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 14.460590594999999, 2024-09-18T10:14:51.311 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.71169541800000002, 2024-09-18T10:14:51.311 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:51.311 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:14:51.311 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:14:51.311 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:14:51.311 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:14:51.311 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 74 2024-09-18T10:14:51.312 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.312 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:51.312 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.312 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:51.312 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787128+0000", 2024-09-18T10:14:51.312 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:51.312 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.312 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.312 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:51.312 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787128+0000", 2024-09-18T10:14:51.312 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:51.312 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.312 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.313 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:51.313 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787127+0000", 2024-09-18T10:14:51.313 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:14:51.313 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.313 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.313 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:51.313 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787133+0000", 2024-09-18T10:14:51.313 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.6930000000000004e-06 2024-09-18T10:14:51.314 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.314 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.314 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:51.314 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787142+0000", 2024-09-18T10:14:51.314 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.1959999999999997e-06 2024-09-18T10:14:51.314 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.315 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.316 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:51.316 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787150+0000", 2024-09-18T10:14:51.316 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.6399999999999997e-06 2024-09-18T10:14:51.316 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.316 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.316 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:51.316 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.298925+0000", 2024-09-18T10:14:51.316 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.51177523599999997 2024-09-18T10:14:51.317 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.317 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.317 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:14:51.317 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.299064+0000", 2024-09-18T10:14:51.317 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000139622 2024-09-18T10:14:51.317 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.317 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.317 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:14:51.317 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.299287+0000", 2024-09-18T10:14:51.317 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00022309700000000001 2024-09-18T10:14:51.317 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.317 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.318 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:14:51.318 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.301389+0000", 2024-09-18T10:14:51.318 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0021018759999999999 2024-09-18T10:14:51.318 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.318 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.318 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:14:51.318 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.498648+0000", 2024-09-18T10:14:51.318 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.19725873999999999 2024-09-18T10:14:51.318 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.318 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.318 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:14:51.318 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.498731+0000", 2024-09-18T10:14:51.318 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.2429000000000006e-05 2024-09-18T10:14:51.318 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.318 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.319 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:51.319 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.498823+0000", 2024-09-18T10:14:51.319 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.2655e-05 2024-09-18T10:14:51.319 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:51.319 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:51.319 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:51.319 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.319 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.319 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T10:14:51.319 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:36.787157+0000", 2024-09-18T10:14:51.319 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 14.460561370000001, 2024-09-18T10:14:51.319 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.81204889999999996, 2024-09-18T10:14:51.319 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:51.320 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:14:51.320 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:14:51.320 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:14:51.320 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:14:51.320 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 75 2024-09-18T10:14:51.320 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.320 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:51.320 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.320 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:51.320 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787157+0000", 2024-09-18T10:14:51.320 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:51.320 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.320 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.320 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:51.321 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787157+0000", 2024-09-18T10:14:51.321 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:51.321 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.321 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.321 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:51.321 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787156+0000", 2024-09-18T10:14:51.321 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:14:51.321 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.321 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.321 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:51.321 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787236+0000", 2024-09-18T10:14:51.321 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.0175999999999999e-05 2024-09-18T10:14:51.321 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.321 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.322 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:51.322 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787247+0000", 2024-09-18T10:14:51.322 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0448999999999999e-05 2024-09-18T10:14:51.322 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.322 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.322 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:51.322 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787253+0000", 2024-09-18T10:14:51.322 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.2469999999999997e-06 2024-09-18T10:14:51.322 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.322 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.322 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:51.322 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.299974+0000", 2024-09-18T10:14:51.322 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.51272114499999999 2024-09-18T10:14:51.322 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.323 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.323 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:14:51.323 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.300115+0000", 2024-09-18T10:14:51.323 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000141452 2024-09-18T10:14:51.323 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.323 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.323 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:14:51.323 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.300349+0000", 2024-09-18T10:14:51.323 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00023363999999999999 2024-09-18T10:14:51.323 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.323 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.323 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:14:51.323 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.301947+0000", 2024-09-18T10:14:51.324 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0015976510000000001 2024-09-18T10:14:51.324 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.324 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.324 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:14:51.324 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.599027+0000", 2024-09-18T10:14:51.324 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.29708051600000002 2024-09-18T10:14:51.324 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.324 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.324 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:14:51.324 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.599106+0000", 2024-09-18T10:14:51.324 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.9033e-05 2024-09-18T10:14:51.324 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.324 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.324 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:51.325 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.599206+0000", 2024-09-18T10:14:51.325 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.9561000000000004e-05 2024-09-18T10:14:51.325 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:51.325 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:51.325 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:51.325 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.325 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.325 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T10:14:51.325 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:36.787261+0000", 2024-09-18T10:14:51.325 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 14.460457719000001, 2024-09-18T10:14:51.325 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.81223649499999995, 2024-09-18T10:14:51.325 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:51.325 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:14:51.325 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:14:51.326 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:14:51.326 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:14:51.326 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 76 2024-09-18T10:14:51.326 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.326 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:51.326 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.326 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:51.326 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787261+0000", 2024-09-18T10:14:51.326 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:51.326 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.326 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.326 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:51.326 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787261+0000", 2024-09-18T10:14:51.327 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:51.327 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.327 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.327 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:51.327 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787260+0000", 2024-09-18T10:14:51.327 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:14:51.327 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.327 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.327 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:51.327 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787265+0000", 2024-09-18T10:14:51.327 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1590000000000001e-06 2024-09-18T10:14:51.327 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.327 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.327 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:51.327 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787273+0000", 2024-09-18T10:14:51.328 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.4100000000000008e-06 2024-09-18T10:14:51.328 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.328 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.328 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:51.328 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787279+0000", 2024-09-18T10:14:51.328 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.3630000000000002e-06 2024-09-18T10:14:51.328 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.328 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.328 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:51.328 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.496955+0000", 2024-09-18T10:14:51.328 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.70967585099999997 2024-09-18T10:14:51.328 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.328 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.328 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:14:51.328 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.497107+0000", 2024-09-18T10:14:51.329 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000152044 2024-09-18T10:14:51.329 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.329 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.329 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:14:51.329 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.497345+0000", 2024-09-18T10:14:51.329 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000238693 2024-09-18T10:14:51.329 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.329 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.329 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:14:51.329 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.599248+0000", 2024-09-18T10:14:51.329 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.101902836 2024-09-18T10:14:51.329 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.329 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.330 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:14:51.330 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.599329+0000", 2024-09-18T10:14:51.330 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.1064999999999997e-05 2024-09-18T10:14:51.330 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.330 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.330 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:14:51.330 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.599441+0000", 2024-09-18T10:14:51.330 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000111797 2024-09-18T10:14:51.330 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.330 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.330 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:51.330 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.599497+0000", 2024-09-18T10:14:51.330 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.6119999999999998e-05 2024-09-18T10:14:51.331 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:51.331 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:51.331 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:51.331 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.331 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.331 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:93 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T10:14:51.331 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:38.616472+0000", 2024-09-18T10:14:51.331 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 12.631246436, 2024-09-18T10:14:51.331 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.60940747100000003, 2024-09-18T10:14:51.331 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:51.331 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:14:51.331 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:14:51.331 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:14:51.332 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:14:51.332 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 93 2024-09-18T10:14:51.332 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.332 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:51.332 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.332 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:51.332 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.616472+0000", 2024-09-18T10:14:51.332 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:51.332 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.332 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.332 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:51.332 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.616472+0000", 2024-09-18T10:14:51.332 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:51.333 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.333 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.333 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:51.333 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.616470+0000", 2024-09-18T10:14:51.333 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:14:51.333 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.333 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.333 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:51.333 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.616644+0000", 2024-09-18T10:14:51.333 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00017372000000000001 2024-09-18T10:14:51.334 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.334 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.334 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:51.334 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.616695+0000", 2024-09-18T10:14:51.334 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.0991e-05 2024-09-18T10:14:51.334 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.334 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.334 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:51.334 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.616709+0000", 2024-09-18T10:14:51.334 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4008999999999999e-05 2024-09-18T10:14:51.334 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.334 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.334 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:51.335 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.720960+0000", 2024-09-18T10:14:51.335 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.104250624 2024-09-18T10:14:51.335 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.335 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.335 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:14:51.335 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.721068+0000", 2024-09-18T10:14:51.335 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00010814700000000001 2024-09-18T10:14:51.335 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.335 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.335 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:14:51.335 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.721270+0000", 2024-09-18T10:14:51.335 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00020184200000000001 2024-09-18T10:14:51.336 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.336 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.336 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:14:51.336 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.024691+0000", 2024-09-18T10:14:51.336 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.30342105600000002 2024-09-18T10:14:51.336 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.336 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.336 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:14:51.336 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.225725+0000", 2024-09-18T10:14:51.336 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20103395099999999 2024-09-18T10:14:51.336 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.336 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.336 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:14:51.337 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.225816+0000", 2024-09-18T10:14:51.337 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.1928999999999993e-05 2024-09-18T10:14:51.337 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.337 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.337 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:51.337 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.225879+0000", 2024-09-18T10:14:51.337 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.2918999999999998e-05 2024-09-18T10:14:51.337 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:51.337 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:51.337 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:51.337 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.337 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.338 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:94 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T10:14:51.338 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:38.616729+0000", 2024-09-18T10:14:51.338 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 12.630989411, 2024-09-18T10:14:51.338 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.60943170999999996, 2024-09-18T10:14:51.338 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:51.338 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:14:51.338 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:14:51.338 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:14:51.338 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:14:51.338 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 94 2024-09-18T10:14:51.338 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.338 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:51.339 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.339 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:51.339 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.616729+0000", 2024-09-18T10:14:51.339 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:51.339 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.339 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.339 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:51.339 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.616729+0000", 2024-09-18T10:14:51.339 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:51.339 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.339 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.339 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:51.339 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.616727+0000", 2024-09-18T10:14:51.340 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:14:51.340 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.340 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.340 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:51.340 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.616738+0000", 2024-09-18T10:14:51.340 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0776e-05 2024-09-18T10:14:51.340 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.340 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.340 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:51.340 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.616756+0000", 2024-09-18T10:14:51.340 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8079000000000001e-05 2024-09-18T10:14:51.340 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.340 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.340 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:51.341 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.716882+0000", 2024-09-18T10:14:51.341 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10012612999999999 2024-09-18T10:14:51.341 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.341 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.341 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:51.341 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.822315+0000", 2024-09-18T10:14:51.341 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.105433162 2024-09-18T10:14:51.341 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.341 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.341 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:14:51.341 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.822453+0000", 2024-09-18T10:14:51.341 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000137396 2024-09-18T10:14:51.341 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.341 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.342 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:14:51.342 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.822674+0000", 2024-09-18T10:14:51.342 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00022154400000000001 2024-09-18T10:14:51.342 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.342 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.342 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:14:51.342 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.024888+0000", 2024-09-18T10:14:51.342 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20221397199999999 2024-09-18T10:14:51.342 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.342 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.342 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:14:51.342 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.226011+0000", 2024-09-18T10:14:51.342 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20112316299999999 2024-09-18T10:14:51.343 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.343 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.343 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:14:51.343 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.226094+0000", 2024-09-18T10:14:51.343 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.2155000000000002e-05 2024-09-18T10:14:51.343 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.343 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.343 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:51.343 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.226161+0000", 2024-09-18T10:14:51.343 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.7138999999999995e-05 2024-09-18T10:14:51.343 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:51.343 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:51.343 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:51.344 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.344 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.344 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:95 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T10:14:51.344 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:38.716909+0000", 2024-09-18T10:14:51.344 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 12.530809697, 2024-09-18T10:14:51.344 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.60960763600000001, 2024-09-18T10:14:51.344 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:51.344 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:14:51.344 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:14:51.344 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:14:51.344 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:14:51.344 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 95 2024-09-18T10:14:51.344 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.345 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:51.345 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.345 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:51.345 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.716909+0000", 2024-09-18T10:14:51.345 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:51.345 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.345 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.345 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:51.345 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.716909+0000", 2024-09-18T10:14:51.345 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:51.345 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.345 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.346 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:51.346 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.716907+0000", 2024-09-18T10:14:51.346 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:14:51.346 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.346 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.346 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:51.346 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.716921+0000", 2024-09-18T10:14:51.346 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4572e-05 2024-09-18T10:14:51.346 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.346 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.346 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:51.346 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.716939+0000", 2024-09-18T10:14:51.347 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7149000000000001e-05 2024-09-18T10:14:51.347 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.347 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.347 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:51.347 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.716948+0000", 2024-09-18T10:14:51.347 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.7000000000000003e-06 2024-09-18T10:14:51.347 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.347 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.347 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:51.347 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.923430+0000", 2024-09-18T10:14:51.347 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20648133299999999 2024-09-18T10:14:51.347 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.347 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.348 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:14:51.348 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.923566+0000", 2024-09-18T10:14:51.348 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00013657599999999999 2024-09-18T10:14:51.348 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.348 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.348 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:14:51.348 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.923782+0000", 2024-09-18T10:14:51.348 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00021599400000000001 2024-09-18T10:14:51.348 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.348 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.348 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:14:51.348 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.024899+0000", 2024-09-18T10:14:51.348 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10111666800000001 2024-09-18T10:14:51.349 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.349 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.349 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:14:51.349 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.326379+0000", 2024-09-18T10:14:51.349 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.30148048799999999 2024-09-18T10:14:51.349 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.349 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.349 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:14:51.349 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.326454+0000", 2024-09-18T10:14:51.349 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.4988999999999996e-05 2024-09-18T10:14:51.349 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.349 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.349 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:51.350 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.326516+0000", 2024-09-18T10:14:51.350 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.1902999999999993e-05 2024-09-18T10:14:51.350 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:51.350 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:51.350 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:51.350 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.350 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.350 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:96 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T10:14:51.350 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:38.716972+0000", 2024-09-18T10:14:51.350 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 12.530746707, 2024-09-18T10:14:51.350 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.91539460800000005, 2024-09-18T10:14:51.350 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:51.350 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:14:51.351 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:14:51.351 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:14:51.351 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:14:51.351 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 96 2024-09-18T10:14:51.351 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.351 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:51.351 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.351 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:51.351 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.716972+0000", 2024-09-18T10:14:51.351 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:51.351 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.351 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.351 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:51.351 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.716972+0000", 2024-09-18T10:14:51.352 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:51.352 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.352 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.352 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:51.352 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.716971+0000", 2024-09-18T10:14:51.352 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:14:51.352 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.352 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.352 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:51.352 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717061+0000", 2024-09-18T10:14:51.352 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.0744999999999999e-05 2024-09-18T10:14:51.352 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.352 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.353 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:51.353 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717073+0000", 2024-09-18T10:14:51.353 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1943000000000001e-05 2024-09-18T10:14:51.353 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.353 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.353 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:51.353 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717080+0000", 2024-09-18T10:14:51.353 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0060000000000003e-06 2024-09-18T10:14:51.353 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.353 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.353 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:51.353 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.426708+0000", 2024-09-18T10:14:51.353 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.70962730399999996 2024-09-18T10:14:51.353 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.354 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.354 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:14:51.354 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.426854+0000", 2024-09-18T10:14:51.354 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00014654199999999999 2024-09-18T10:14:51.354 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.354 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.354 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:14:51.354 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.427106+0000", 2024-09-18T10:14:51.354 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00025226600000000001 2024-09-18T10:14:51.354 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.354 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.354 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:14:51.354 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.433087+0000", 2024-09-18T10:14:51.354 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0059804860000000001 2024-09-18T10:14:51.355 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.355 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.355 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:14:51.355 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.632096+0000", 2024-09-18T10:14:51.355 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.19900922099999999 2024-09-18T10:14:51.355 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.355 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.355 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:14:51.355 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.632257+0000", 2024-09-18T10:14:51.355 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00016058399999999999 2024-09-18T10:14:51.355 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.355 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.355 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:51.356 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.632366+0000", 2024-09-18T10:14:51.356 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000109601 2024-09-18T10:14:51.356 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:51.356 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:51.356 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:51.356 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.356 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.356 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:97 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T10:14:51.356 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:38.717093+0000", 2024-09-18T10:14:51.356 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 12.530624896999999, 2024-09-18T10:14:51.356 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.91561460900000002, 2024-09-18T10:14:51.356 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:51.356 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:14:51.356 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:14:51.357 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:14:51.357 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:14:51.357 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 97 2024-09-18T10:14:51.357 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.357 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:51.357 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.357 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:51.357 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717093+0000", 2024-09-18T10:14:51.357 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:51.357 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.357 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.357 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:51.357 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717093+0000", 2024-09-18T10:14:51.357 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:51.358 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.358 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.358 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:51.358 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717092+0000", 2024-09-18T10:14:51.358 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:14:51.358 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.358 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.358 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:51.358 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717099+0000", 2024-09-18T10:14:51.358 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.246e-06 2024-09-18T10:14:51.358 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.358 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.358 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:51.359 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717109+0000", 2024-09-18T10:14:51.359 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0499999999999999e-05 2024-09-18T10:14:51.359 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.359 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.359 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:51.359 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717115+0000", 2024-09-18T10:14:51.359 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.7690000000000002e-06 2024-09-18T10:14:51.359 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.359 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.359 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:51.359 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.428509+0000", 2024-09-18T10:14:51.359 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.71139444299999999 2024-09-18T10:14:51.360 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.360 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.360 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:14:51.360 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.428621+0000", 2024-09-18T10:14:51.360 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00011156 2024-09-18T10:14:51.360 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.360 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.360 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:14:51.360 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.428805+0000", 2024-09-18T10:14:51.360 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000183597 2024-09-18T10:14:51.360 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.360 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.360 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:14:51.360 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.433106+0000", 2024-09-18T10:14:51.360 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0043015019999999996 2024-09-18T10:14:51.361 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.361 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.361 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:14:51.361 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.632474+0000", 2024-09-18T10:14:51.361 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.199368092 2024-09-18T10:14:51.361 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.361 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.361 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:14:51.361 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.632605+0000", 2024-09-18T10:14:51.361 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00013110899999999999 2024-09-18T10:14:51.361 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.361 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.361 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:51.362 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.632708+0000", 2024-09-18T10:14:51.362 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000102748 2024-09-18T10:14:51.362 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:51.362 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:51.362 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:51.362 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.362 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.362 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:98 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T10:14:51.362 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:38.717126+0000", 2024-09-18T10:14:51.362 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 12.530592635, 2024-09-18T10:14:51.362 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.015084141, 2024-09-18T10:14:51.362 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:51.362 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:14:51.363 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:14:51.363 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:14:51.363 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:14:51.363 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 98 2024-09-18T10:14:51.363 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.363 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:51.363 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.363 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:51.363 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717126+0000", 2024-09-18T10:14:51.363 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:51.363 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.363 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.363 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:51.364 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717126+0000", 2024-09-18T10:14:51.364 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:51.364 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.364 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.364 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:51.364 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717125+0000", 2024-09-18T10:14:51.364 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:14:51.364 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.364 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.364 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:51.364 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717130+0000", 2024-09-18T10:14:51.364 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.4160000000000003e-06 2024-09-18T10:14:51.364 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.364 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.365 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:51.365 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717139+0000", 2024-09-18T10:14:51.365 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.6230000000000003e-06 2024-09-18T10:14:51.365 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.365 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.365 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:51.365 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717144+0000", 2024-09-18T10:14:51.365 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5330000000000002e-06 2024-09-18T10:14:51.365 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.365 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.365 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:51.365 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.429491+0000", 2024-09-18T10:14:51.365 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.71234629500000002 2024-09-18T10:14:51.366 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.366 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.366 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:14:51.366 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.429741+0000", 2024-09-18T10:14:51.366 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000250516 2024-09-18T10:14:51.366 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.366 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.366 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:14:51.366 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.429989+0000", 2024-09-18T10:14:51.366 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00024811599999999999 2024-09-18T10:14:51.366 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.366 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.366 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:14:51.366 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.435056+0000", 2024-09-18T10:14:51.367 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0050669670000000003 2024-09-18T10:14:51.367 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.367 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.367 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:14:51.367 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.732068+0000", 2024-09-18T10:14:51.367 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.29701180900000002 2024-09-18T10:14:51.367 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.367 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.367 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:14:51.367 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.732145+0000", 2024-09-18T10:14:51.367 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.7152999999999998e-05 2024-09-18T10:14:51.367 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.367 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.367 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:51.367 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.732210+0000", 2024-09-18T10:14:51.367 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.4503000000000002e-05 2024-09-18T10:14:51.368 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:51.368 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:51.368 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:51.368 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.368 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.368 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:99 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T10:14:51.368 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:38.717230+0000", 2024-09-18T10:14:51.368 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 12.530488074999999, 2024-09-18T10:14:51.368 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0157905819999999, 2024-09-18T10:14:51.368 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:51.368 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:14:51.368 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:14:51.368 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:14:51.369 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:14:51.369 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 99 2024-09-18T10:14:51.369 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.369 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:51.369 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.369 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:51.369 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717230+0000", 2024-09-18T10:14:51.369 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:51.369 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.369 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.369 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:51.369 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717230+0000", 2024-09-18T10:14:51.369 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:51.369 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.369 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.370 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:51.370 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717229+0000", 2024-09-18T10:14:51.370 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:14:51.370 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.370 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.370 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:51.370 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717318+0000", 2024-09-18T10:14:51.370 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.8197999999999998e-05 2024-09-18T10:14:51.370 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.370 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.370 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:51.370 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717342+0000", 2024-09-18T10:14:51.370 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4445000000000001e-05 2024-09-18T10:14:51.370 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.370 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.371 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:51.371 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717360+0000", 2024-09-18T10:14:51.371 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7782999999999999e-05 2024-09-18T10:14:51.371 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.371 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.371 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:51.371 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.430650+0000", 2024-09-18T10:14:51.371 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.71329020200000004 2024-09-18T10:14:51.371 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.371 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.371 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:14:51.371 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.430805+0000", 2024-09-18T10:14:51.371 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00015474900000000001 2024-09-18T10:14:51.371 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.371 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.372 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:14:51.372 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.431021+0000", 2024-09-18T10:14:51.372 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000215911 2024-09-18T10:14:51.372 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.372 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.372 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:14:51.372 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.435099+0000", 2024-09-18T10:14:51.372 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0040787439999999996 2024-09-18T10:14:51.372 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.372 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.372 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:14:51.372 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.732758+0000", 2024-09-18T10:14:51.372 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.297658637 2024-09-18T10:14:51.373 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.373 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.373 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:14:51.373 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.732909+0000", 2024-09-18T10:14:51.373 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00015069899999999999 2024-09-18T10:14:51.373 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.373 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.373 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:51.373 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.733021+0000", 2024-09-18T10:14:51.373 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00011207 2024-09-18T10:14:51.373 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:51.373 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:51.373 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:51.374 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.374 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.374 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:100 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T10:14:51.374 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:38.717370+0000", 2024-09-18T10:14:51.374 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 12.530348513, 2024-09-18T10:14:51.374 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1158324850000001, 2024-09-18T10:14:51.374 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:51.374 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:14:51.374 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:14:51.374 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:14:51.374 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:14:51.374 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 100 2024-09-18T10:14:51.374 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.375 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:51.375 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.375 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:51.375 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717370+0000", 2024-09-18T10:14:51.375 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:51.375 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.375 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.375 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:51.375 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717370+0000", 2024-09-18T10:14:51.375 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:51.375 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.375 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.375 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:51.375 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717369+0000", 2024-09-18T10:14:51.376 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:14:51.376 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.376 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.376 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:51.376 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717378+0000", 2024-09-18T10:14:51.376 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.3559999999999999e-06 2024-09-18T10:14:51.376 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.376 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.376 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:51.376 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717398+0000", 2024-09-18T10:14:51.376 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.9962e-05 2024-09-18T10:14:51.376 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.376 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.376 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:51.377 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717407+0000", 2024-09-18T10:14:51.377 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.2699999999999993e-06 2024-09-18T10:14:51.377 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.377 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.377 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:51.377 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.432184+0000", 2024-09-18T10:14:51.377 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.71477630299999995 2024-09-18T10:14:51.377 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.377 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.377 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:14:51.377 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.432316+0000", 2024-09-18T10:14:51.377 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000132082 2024-09-18T10:14:51.377 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.377 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.377 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:14:51.377 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.432526+0000", 2024-09-18T10:14:51.378 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000210659 2024-09-18T10:14:51.378 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.378 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.378 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:14:51.378 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.435651+0000", 2024-09-18T10:14:51.378 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.003124169 2024-09-18T10:14:51.378 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.378 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.378 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:14:51.378 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.832931+0000", 2024-09-18T10:14:51.378 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.39728084899999999 2024-09-18T10:14:51.378 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.378 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.378 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:14:51.379 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.833058+0000", 2024-09-18T10:14:51.379 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000126969 2024-09-18T10:14:51.379 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.379 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.379 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:51.379 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.833202+0000", 2024-09-18T10:14:51.379 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000143922 2024-09-18T10:14:51.379 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:51.379 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:51.379 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:51.379 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:51.379 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:51.379 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-09-18T10:14:51.381 INFO:tasks.rados.rados.0.smithi189.stdout:42: finishing write tid 3 to smithi18925896-42 2024-09-18T10:14:51.381 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 42 v 8 (ObjNum 41 snap 0 seq_num 41) dirty exists 2024-09-18T10:14:51.381 INFO:tasks.rados.rados.0.smithi189.stdout:42: left oid 42 (ObjNum 41 snap 0 seq_num 41) 2024-09-18T10:14:51.382 INFO:tasks.rados.rados.0.smithi189.stdout:37: finishing write tid 1 to smithi18925896-37 2024-09-18T10:14:51.382 INFO:tasks.rados.rados.0.smithi189.stdout:37: finishing write tid 2 to smithi18925896-37 2024-09-18T10:14:51.382 INFO:tasks.rados.rados.0.smithi189.stdout:45: finishing write tid 2 to smithi18925896-45 2024-09-18T10:14:51.382 INFO:tasks.rados.rados.0.smithi189.stdout:48: finishing write tid 1 to smithi18925896-48 2024-09-18T10:14:51.382 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-09-18T10:14:51.383 INFO:tasks.ceph.osd.8.smithi029.stderr:2024-09-18T10:14:51.378+0000 7f2555c27700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 22192) UID: 0 2024-09-18T10:14:51.418 INFO:tasks.rados.rados.0.smithi189.stdout:44: finishing write tid 3 to smithi18925896-44 2024-09-18T10:14:51.418 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 44 v 6 (ObjNum 43 snap 0 seq_num 43) dirty exists 2024-09-18T10:14:51.418 INFO:tasks.rados.rados.0.smithi189.stdout:44: left oid 44 (ObjNum 43 snap 0 seq_num 43) 2024-09-18T10:14:51.426 INFO:tasks.rados.rados.0.smithi189.stdout:37: finishing write tid 3 to smithi18925896-37 2024-09-18T10:14:51.426 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 37 v 4 (ObjNum 36 snap 0 seq_num 36) dirty exists 2024-09-18T10:14:51.426 INFO:tasks.rados.rados.0.smithi189.stdout:37: left oid 37 (ObjNum 36 snap 0 seq_num 36) 2024-09-18T10:14:51.482 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-18T10:14:51.482+0000 7f1981873700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22132) UID: 0 2024-09-18T10:14:51.528 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-09-18T10:14:51.529 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-09-18T10:14:51.529 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-09-18T10:14:51.529 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-09-18T10:14:51.532 INFO:tasks.rados.rados.0.smithi189.stdout:36: done (15 left) 2024-09-18T10:14:51.532 INFO:tasks.rados.rados.0.smithi189.stdout:37: done (14 left) 2024-09-18T10:14:51.532 INFO:tasks.rados.rados.0.smithi189.stdout:38: done (13 left) 2024-09-18T10:14:51.532 INFO:tasks.rados.rados.0.smithi189.stdout:42: done (12 left) 2024-09-18T10:14:51.532 INFO:tasks.rados.rados.0.smithi189.stdout:44: done (11 left) 2024-09-18T10:14:51.532 INFO:tasks.rados.rados.0.smithi189.stdout:54: done (10 left) 2024-09-18T10:14:51.533 INFO:tasks.rados.rados.0.smithi189.stdout:55: read oid 36 snap -1 2024-09-18T10:14:51.533 INFO:tasks.rados.rados.0.smithi189.stdout:55: expect (ObjNum 35 snap 0 seq_num 35) 2024-09-18T10:14:51.533 INFO:tasks.rados.rados.0.smithi189.stdout:56: rollback oid 20 current snap is 0 2024-09-18T10:14:51.533 INFO:tasks.rados.rados.0.smithi189.stdout:56: done (11 left) 2024-09-18T10:14:51.533 INFO:tasks.rados.rados.0.smithi189.stdout:57: read oid 8 snap -1 2024-09-18T10:14:51.533 INFO:tasks.rados.rados.0.smithi189.stdout:57: expect (ObjNum 7 snap 0 seq_num 7) 2024-09-18T10:14:51.533 INFO:tasks.rados.rados.0.smithi189.stdout:58: copy_from oid 19 from oid 13 current snap is 0 2024-09-18T10:14:51.533 INFO:tasks.rados.rados.0.smithi189.stdout:59: rollback oid 16 current snap is 0 2024-09-18T10:14:51.533 INFO:tasks.rados.rados.0.smithi189.stdout:59: done (13 left) 2024-09-18T10:14:51.533 INFO:tasks.rados.rados.0.smithi189.stdout:60: copy_from oid 26 from oid 34 current snap is 0 2024-09-18T10:14:51.533 INFO:tasks.rados.rados.0.smithi189.stdout:61: delete oid 9 current snap is 0 2024-09-18T10:14:51.534 INFO:tasks.rados.rados.0.smithi189.stdout:48: finishing write tid 2 to smithi18925896-48 2024-09-18T10:14:51.534 INFO:tasks.rados.rados.0.smithi189.stdout:50: finishing write tid 1 to smithi18925896-50 2024-09-18T10:14:51.534 INFO:tasks.rados.rados.0.smithi189.stdout:43: finishing write tid 3 to smithi18925896-43 2024-09-18T10:14:51.534 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 43 v 8 (ObjNum 42 snap 0 seq_num 42) dirty exists 2024-09-18T10:14:51.534 INFO:tasks.rados.rados.0.smithi189.stdout:43: left oid 43 (ObjNum 42 snap 0 seq_num 42) 2024-09-18T10:14:51.535 INFO:tasks.rados.rados.0.smithi189.stdout:50: finishing write tid 2 to smithi18925896-50 2024-09-18T10:14:51.535 INFO:tasks.rados.rados.0.smithi189.stdout:46: finishing write tid 3 to smithi18925896-46 2024-09-18T10:14:51.535 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 46 v 10 (ObjNum 45 snap 0 seq_num 45) dirty exists 2024-09-18T10:14:51.535 INFO:tasks.rados.rados.0.smithi189.stdout:46: left oid 46 (ObjNum 45 snap 0 seq_num 45) 2024-09-18T10:14:51.535 INFO:tasks.rados.rados.0.smithi189.stdout:48: finishing write tid 3 to smithi18925896-48 2024-09-18T10:14:51.538 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-09-18T10:14:51.583 INFO:tasks.ceph.osd.4.smithi029.stderr:2024-09-18T10:14:51.578+0000 7f550cf6e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22189) UID: 0 2024-09-18T10:14:51.619 INFO:tasks.rados.rados.0.smithi189.stdout:47: finishing write tid 1 to smithi18925896-47 2024-09-18T10:14:51.619 INFO:tasks.rados.rados.0.smithi189.stdout:47: finishing write tid 2 to smithi18925896-47 2024-09-18T10:14:51.631 INFO:tasks.rados.rados.0.smithi189.stdout:52: finishing write tid 1 to smithi18925896-25 2024-09-18T10:14:51.632 INFO:tasks.rados.rados.0.smithi189.stdout:52: finishing write tid 2 to smithi18925896-25 2024-09-18T10:14:51.632 INFO:tasks.rados.rados.0.smithi189.stdout:52: finishing write tid 3 to smithi18925896-25 2024-09-18T10:14:51.636 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-09-18T10:14:51.636 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-09-18T10:14:51.637 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-09-18T10:14:51.637 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-09-18T10:14:51.637 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-09-18T10:14:51.646 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-09-18T10:14:51.684 INFO:tasks.ceph.osd.11.smithi189.stderr:2024-09-18T10:14:51.679+0000 7f718b60a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 22400) UID: 0 2024-09-18T10:14:51.686 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-18T10:14:51.722 INFO:tasks.rados.rados.0.smithi189.stdout:47: finishing write tid 3 to smithi18925896-47 2024-09-18T10:14:51.722 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 47 v 8 (ObjNum 46 snap 0 seq_num 46) dirty exists 2024-09-18T10:14:51.722 INFO:tasks.rados.rados.0.smithi189.stdout:47: left oid 47 (ObjNum 46 snap 0 seq_num 46) 2024-09-18T10:14:51.728 INFO:tasks.rados.rados.0.smithi189.stdout:45: finishing write tid 3 to smithi18925896-45 2024-09-18T10:14:51.728 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 45 v 6 (ObjNum 44 snap 0 seq_num 44) dirty exists 2024-09-18T10:14:51.728 INFO:tasks.rados.rados.0.smithi189.stdout:45: left oid 45 (ObjNum 44 snap 0 seq_num 44) 2024-09-18T10:14:51.731 INFO:tasks.rados.rados.0.smithi189.stdout:49: finishing write tid 1 to smithi18925896-49 2024-09-18T10:14:51.732 INFO:tasks.rados.rados.0.smithi189.stdout:49: finishing write tid 2 to smithi18925896-49 2024-09-18T10:14:51.737 INFO:tasks.rados.rados.0.smithi189.stdout:48: finishing write tid 4 to smithi18925896-48 2024-09-18T10:14:51.737 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 48 v 13 (ObjNum 47 snap 0 seq_num 47) dirty exists 2024-09-18T10:14:51.737 INFO:tasks.rados.rados.0.smithi189.stdout:48: left oid 48 (ObjNum 47 snap 0 seq_num 47) 2024-09-18T10:14:51.745 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-09-18T10:14:51.745 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-09-18T10:14:51.745 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-09-18T10:14:51.745 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-09-18T10:14:51.745 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.745 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.7s5 36/32 ECSubWrite(tid=28, reqid=client.4632.0:128, at_version=36'8, trim_to=0'0, roll_forward_to=36'6))", 2024-09-18T10:14:51.745 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:51.003923+0000", 2024-09-18T10:14:51.745 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 0.73788179300000001, 2024-09-18T10:14:51.745 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0024713159999999999, 2024-09-18T10:14:51.745 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:51.745 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:14:51.746 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:51.746 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.746 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:51.746 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.003923+0000", 2024-09-18T10:14:51.746 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:51.746 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.746 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.746 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:51.746 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.003923+0000", 2024-09-18T10:14:51.746 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:51.746 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.746 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.746 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:51.747 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.003921+0000", 2024-09-18T10:14:51.747 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:14:51.747 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.747 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.747 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:51.747 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.003965+0000", 2024-09-18T10:14:51.747 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.3170999999999997e-05 2024-09-18T10:14:51.747 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.747 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.747 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:51.747 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.003984+0000", 2024-09-18T10:14:51.747 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.9763e-05 2024-09-18T10:14:51.747 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.748 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.748 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:51.748 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.003996+0000", 2024-09-18T10:14:51.748 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1493e-05 2024-09-18T10:14:51.748 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.748 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.748 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:51.748 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.005077+0000", 2024-09-18T10:14:51.748 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0010807379999999999 2024-09-18T10:14:51.748 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.748 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.748 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:14:51.748 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.005091+0000", 2024-09-18T10:14:51.749 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4358999999999999e-05 2024-09-18T10:14:51.749 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.749 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.749 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:14:51.749 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.006373+0000", 2024-09-18T10:14:51.749 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001282219 2024-09-18T10:14:51.749 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.749 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.749 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:51.749 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.006394+0000", 2024-09-18T10:14:51.749 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.1268999999999999e-05 2024-09-18T10:14:51.749 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:51.749 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:51.750 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:51.750 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.750 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.750 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.9s2 36/32 ECSubWrite(tid=12, reqid=client.4632.0:136, at_version=36'9, trim_to=0'0, roll_forward_to=35'6))", 2024-09-18T10:14:51.750 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:51.004061+0000", 2024-09-18T10:14:51.750 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 0.73774367500000004, 2024-09-18T10:14:51.750 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.102012215, 2024-09-18T10:14:51.750 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:51.750 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:14:51.750 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:51.750 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.750 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:51.750 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.004061+0000", 2024-09-18T10:14:51.750 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:51.750 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.751 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.751 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:51.751 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.004061+0000", 2024-09-18T10:14:51.751 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:51.751 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.751 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.751 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:51.751 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.004060+0000", 2024-09-18T10:14:51.751 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:14:51.751 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.751 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.751 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:51.751 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.004355+0000", 2024-09-18T10:14:51.751 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00029493199999999997 2024-09-18T10:14:51.752 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.752 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.752 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:51.752 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.004365+0000", 2024-09-18T10:14:51.752 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.4530000000000008e-06 2024-09-18T10:14:51.752 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.752 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.752 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:51.752 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.004369+0000", 2024-09-18T10:14:51.752 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.6120000000000001e-06 2024-09-18T10:14:51.752 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.752 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.752 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:51.752 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.004454+0000", 2024-09-18T10:14:51.753 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.5050999999999996e-05 2024-09-18T10:14:51.753 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.753 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.753 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:14:51.753 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.004468+0000", 2024-09-18T10:14:51.753 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3521e-05 2024-09-18T10:14:51.753 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.753 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.753 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:14:51.753 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.105999+0000", 2024-09-18T10:14:51.753 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.101530651 2024-09-18T10:14:51.753 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.753 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.753 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:51.754 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.106073+0000", 2024-09-18T10:14:51.754 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.4964999999999998e-05 2024-09-18T10:14:51.754 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:51.754 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:51.754 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:51.754 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.754 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.754 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.9s2 36/32 ECSubWrite(tid=13, reqid=client.4632.0:137, at_version=36'10, trim_to=0'0, roll_forward_to=35'6))", 2024-09-18T10:14:51.754 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:51.004383+0000", 2024-09-18T10:14:51.754 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 0.73742154900000001, 2024-09-18T10:14:51.754 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.101895054, 2024-09-18T10:14:51.754 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:51.754 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:14:51.754 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:51.755 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.755 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:51.755 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.004383+0000", 2024-09-18T10:14:51.755 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:51.755 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.755 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.755 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:51.755 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.004383+0000", 2024-09-18T10:14:51.755 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:51.756 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.756 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.756 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:51.756 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.004383+0000", 2024-09-18T10:14:51.756 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999995 2024-09-18T10:14:51.756 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.756 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.756 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:51.756 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.004400+0000", 2024-09-18T10:14:51.756 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7142999999999998e-05 2024-09-18T10:14:51.756 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.756 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.756 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:51.756 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.004412+0000", 2024-09-18T10:14:51.757 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1970999999999999e-05 2024-09-18T10:14:51.757 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.757 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.757 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:51.757 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.004419+0000", 2024-09-18T10:14:51.757 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.6030000000000001e-06 2024-09-18T10:14:51.757 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.757 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.757 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:51.757 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.105180+0000", 2024-09-18T10:14:51.757 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100760893 2024-09-18T10:14:51.757 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.757 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.757 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:14:51.758 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.105197+0000", 2024-09-18T10:14:51.758 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7872000000000001e-05 2024-09-18T10:14:51.758 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.758 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.758 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:14:51.758 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.106206+0000", 2024-09-18T10:14:51.758 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0010085000000000001 2024-09-18T10:14:51.758 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.758 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.758 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:51.758 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.106278+0000", 2024-09-18T10:14:51.758 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.2478999999999995e-05 2024-09-18T10:14:51.758 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:51.758 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:51.759 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:51.759 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.759 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.759 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpRead(3.9s2 36/32 ECSubRead(tid=14, to_read={3:9a1a774c:::smithi18925896-38:head=0,4096,0}, subchunks={3:9a1a774c:::smithi18925896-38:head=[0,1]}, attrs_to_read=))", 2024-09-18T10:14:51.759 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:51.004463+0000", 2024-09-18T10:14:51.759 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 0.73734144899999998, 2024-09-18T10:14:51.759 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.101527407, 2024-09-18T10:14:51.759 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:51.759 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:14:51.759 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:51.759 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.759 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:51.759 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.004463+0000", 2024-09-18T10:14:51.759 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:51.760 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.760 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.760 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:51.760 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.004463+0000", 2024-09-18T10:14:51.760 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:51.760 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.760 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.760 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:51.760 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.004463+0000", 2024-09-18T10:14:51.760 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999995 2024-09-18T10:14:51.760 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.760 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.760 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:51.761 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.004473+0000", 2024-09-18T10:14:51.761 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.029e-05 2024-09-18T10:14:51.761 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.761 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.761 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:51.761 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.004486+0000", 2024-09-18T10:14:51.761 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3026e-05 2024-09-18T10:14:51.761 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.761 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.761 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:51.761 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.004494+0000", 2024-09-18T10:14:51.761 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.9540000000000004e-06 2024-09-18T10:14:51.761 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.761 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.762 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:51.762 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.105558+0000", 2024-09-18T10:14:51.762 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10106336 2024-09-18T10:14:51.762 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.762 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.762 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:51.762 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.105991+0000", 2024-09-18T10:14:51.762 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00043323400000000001 2024-09-18T10:14:51.762 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:51.762 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:51.762 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:51.762 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.762 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.763 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.2s5 36/32 ECSubWrite(tid=15, reqid=client.4632.0:125, at_version=0'0, trim_to=0'0, roll_forward_to=36'8))", 2024-09-18T10:14:51.763 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:51.112365+0000", 2024-09-18T10:14:51.763 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 0.62943985899999999, 2024-09-18T10:14:51.763 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00051452900000000001, 2024-09-18T10:14:51.763 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:51.763 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:14:51.763 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:51.763 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.763 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:51.763 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.112365+0000", 2024-09-18T10:14:51.763 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:51.763 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.763 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.763 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:51.764 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.112365+0000", 2024-09-18T10:14:51.764 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:51.764 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.764 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.764 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:51.764 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.112364+0000", 2024-09-18T10:14:51.764 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:14:51.764 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.764 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.764 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:51.764 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.112377+0000", 2024-09-18T10:14:51.764 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2676e-05 2024-09-18T10:14:51.764 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.765 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.765 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:51.765 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.112391+0000", 2024-09-18T10:14:51.765 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3991999999999999e-05 2024-09-18T10:14:51.765 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.765 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.765 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:51.765 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.112396+0000", 2024-09-18T10:14:51.765 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.9410000000000004e-06 2024-09-18T10:14:51.765 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.765 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.765 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:51.765 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.112445+0000", 2024-09-18T10:14:51.766 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.8161999999999997e-05 2024-09-18T10:14:51.766 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.766 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.766 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:14:51.766 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.112451+0000", 2024-09-18T10:14:51.766 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.5270000000000004e-06 2024-09-18T10:14:51.766 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.766 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.766 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:14:51.766 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.112849+0000", 2024-09-18T10:14:51.766 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00039820600000000002 2024-09-18T10:14:51.766 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.766 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.766 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:51.767 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.112880+0000", 2024-09-18T10:14:51.767 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0266000000000001e-05 2024-09-18T10:14:51.767 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:51.767 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:51.767 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:51.767 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.767 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.767 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.bs2 36/32 ECSubWrite(tid=29, reqid=client.4632.0:119, at_version=0'0, trim_to=0'0, roll_forward_to=36'4))", 2024-09-18T10:14:51.767 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:51.117719+0000", 2024-09-18T10:14:51.767 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 0.624085848, 2024-09-18T10:14:51.767 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00077433299999999999, 2024-09-18T10:14:51.767 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:51.767 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:14:51.767 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:51.768 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.768 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:51.768 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.117719+0000", 2024-09-18T10:14:51.768 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:51.768 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.768 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.768 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:51.768 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.117719+0000", 2024-09-18T10:14:51.768 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:51.768 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.768 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.768 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:51.768 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.117717+0000", 2024-09-18T10:14:51.768 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:14:51.769 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.769 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.769 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:51.769 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.117748+0000", 2024-09-18T10:14:51.769 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0508e-05 2024-09-18T10:14:51.769 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.769 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.769 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:51.769 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.117788+0000", 2024-09-18T10:14:51.769 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.9925000000000001e-05 2024-09-18T10:14:51.769 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.769 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.769 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:51.770 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.117807+0000", 2024-09-18T10:14:51.770 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8856e-05 2024-09-18T10:14:51.770 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.770 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.770 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:51.770 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.117900+0000", 2024-09-18T10:14:51.770 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.3088e-05 2024-09-18T10:14:51.770 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.770 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.770 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:14:51.770 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.117909+0000", 2024-09-18T10:14:51.770 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.7669999999999992e-06 2024-09-18T10:14:51.770 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.770 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.771 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:14:51.771 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.118469+0000", 2024-09-18T10:14:51.771 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00056012599999999998 2024-09-18T10:14:51.771 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.771 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.771 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:51.771 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.118493+0000", 2024-09-18T10:14:51.771 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4796000000000001e-05 2024-09-18T10:14:51.771 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:51.771 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:51.771 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:51.771 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.771 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.771 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.9s2 36/32 ECSubWrite(tid=15, reqid=client.4632.0:142, at_version=36'11, trim_to=0'0, roll_forward_to=36'9))", 2024-09-18T10:14:51.772 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:51.129967+0000", 2024-09-18T10:14:51.772 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 0.61183808299999998, 2024-09-18T10:14:51.772 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0021650390000000001, 2024-09-18T10:14:51.772 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:51.772 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:14:51.772 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:51.772 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.772 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:51.772 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.129967+0000", 2024-09-18T10:14:51.772 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:51.772 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.772 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.772 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:51.772 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.129967+0000", 2024-09-18T10:14:51.773 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:51.773 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.773 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.773 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:51.773 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.129965+0000", 2024-09-18T10:14:51.773 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:14:51.773 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.773 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.773 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:51.773 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.130152+0000", 2024-09-18T10:14:51.773 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00018660899999999999 2024-09-18T10:14:51.773 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.773 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.773 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:51.774 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.130194+0000", 2024-09-18T10:14:51.774 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.2083999999999999e-05 2024-09-18T10:14:51.774 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.774 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.774 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:51.774 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.130213+0000", 2024-09-18T10:14:51.774 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.9156000000000001e-05 2024-09-18T10:14:51.774 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.774 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.774 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:51.774 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.130340+0000", 2024-09-18T10:14:51.774 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000127257 2024-09-18T10:14:51.774 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.775 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.775 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:14:51.775 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.130374+0000", 2024-09-18T10:14:51.775 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.3213000000000002e-05 2024-09-18T10:14:51.775 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.775 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.775 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:14:51.775 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.132103+0000", 2024-09-18T10:14:51.775 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001729724 2024-09-18T10:14:51.775 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.775 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.775 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:51.775 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.132132+0000", 2024-09-18T10:14:51.776 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8484999999999999e-05 2024-09-18T10:14:51.776 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:51.776 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:51.776 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:51.776 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.776 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.776 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.9s2 36/32 ECSubWrite(tid=16, reqid=client.4632.0:143, at_version=36'12, trim_to=0'0, roll_forward_to=36'9))", 2024-09-18T10:14:51.776 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:51.131292+0000", 2024-09-18T10:14:51.776 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 0.61051255699999996, 2024-09-18T10:14:51.776 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001348932, 2024-09-18T10:14:51.776 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:51.776 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:14:51.776 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:51.776 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.777 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:51.777 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.131292+0000", 2024-09-18T10:14:51.777 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:51.777 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.777 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.777 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:51.777 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.131292+0000", 2024-09-18T10:14:51.777 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:51.777 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.777 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.777 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:51.777 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.131292+0000", 2024-09-18T10:14:51.778 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:14:51.778 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.778 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.778 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:51.778 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.131412+0000", 2024-09-18T10:14:51.778 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000120882 2024-09-18T10:14:51.778 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.778 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.778 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:51.778 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.131426+0000", 2024-09-18T10:14:51.778 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3783e-05 2024-09-18T10:14:51.778 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.778 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.778 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:51.779 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.131435+0000", 2024-09-18T10:14:51.779 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.721e-06 2024-09-18T10:14:51.779 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.779 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.779 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:51.779 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.131513+0000", 2024-09-18T10:14:51.779 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.8127000000000006e-05 2024-09-18T10:14:51.779 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.779 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.779 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:14:51.779 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.131526+0000", 2024-09-18T10:14:51.779 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3317e-05 2024-09-18T10:14:51.779 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.779 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.780 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:14:51.780 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.132605+0000", 2024-09-18T10:14:51.780 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001078823 2024-09-18T10:14:51.780 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.780 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.780 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:51.780 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.132641+0000", 2024-09-18T10:14:51.780 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.6072999999999998e-05 2024-09-18T10:14:51.780 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:51.780 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:51.780 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:51.780 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.780 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.780 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.9s2 36/32 ECSubWrite(tid=17, reqid=client.4632.0:144, at_version=36'13, trim_to=0'0, roll_forward_to=36'9))", 2024-09-18T10:14:51.781 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:51.131979+0000", 2024-09-18T10:14:51.781 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 0.60982550999999996, 2024-09-18T10:14:51.781 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0010591190000000001, 2024-09-18T10:14:51.781 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:51.781 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:14:51.781 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:51.781 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.781 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:51.781 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.131979+0000", 2024-09-18T10:14:51.781 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:51.781 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.781 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.782 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:51.782 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.131979+0000", 2024-09-18T10:14:51.782 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:51.782 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.782 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.782 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:51.782 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.131979+0000", 2024-09-18T10:14:51.782 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:14:51.782 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.782 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.782 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:51.782 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.131998+0000", 2024-09-18T10:14:51.782 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.9694000000000002e-05 2024-09-18T10:14:51.782 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.783 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.783 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:51.783 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.132015+0000", 2024-09-18T10:14:51.783 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.6617e-05 2024-09-18T10:14:51.783 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.783 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.783 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:51.783 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.132024+0000", 2024-09-18T10:14:51.783 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.4760000000000004e-06 2024-09-18T10:14:51.783 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.783 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.783 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:51.783 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.132202+0000", 2024-09-18T10:14:51.783 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00017709400000000001 2024-09-18T10:14:51.784 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.784 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.784 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:14:51.784 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.132220+0000", 2024-09-18T10:14:51.784 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.804e-05 2024-09-18T10:14:51.784 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.784 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.784 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:14:51.784 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.133011+0000", 2024-09-18T10:14:51.784 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00079155399999999998 2024-09-18T10:14:51.784 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.784 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.784 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:51.784 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.133039+0000", 2024-09-18T10:14:51.785 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.7396e-05 2024-09-18T10:14:51.785 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:51.785 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:51.785 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:51.785 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.785 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.785 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.7s5 36/32 ECSubWrite(tid=31, reqid=client.4632.0:128, at_version=0'0, trim_to=0'0, roll_forward_to=36'8))", 2024-09-18T10:14:51.785 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:51.219177+0000", 2024-09-18T10:14:51.785 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 0.52262746400000004, 2024-09-18T10:14:51.785 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.202758309, 2024-09-18T10:14:51.785 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:51.785 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:14:51.785 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:51.785 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.786 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:51.786 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.219177+0000", 2024-09-18T10:14:51.786 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:51.786 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.786 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.786 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:51.786 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.219177+0000", 2024-09-18T10:14:51.786 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:51.786 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.786 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.786 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:51.786 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.219175+0000", 2024-09-18T10:14:51.786 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999976 2024-09-18T10:14:51.786 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.787 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.787 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:51.787 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.219207+0000", 2024-09-18T10:14:51.787 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1285000000000003e-05 2024-09-18T10:14:51.787 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.787 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.787 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:51.787 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.219244+0000", 2024-09-18T10:14:51.787 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.7032000000000001e-05 2024-09-18T10:14:51.787 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.787 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.787 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:51.787 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.219266+0000", 2024-09-18T10:14:51.787 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.2112000000000001e-05 2024-09-18T10:14:51.788 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.788 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.788 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:51.788 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.319431+0000", 2024-09-18T10:14:51.788 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100164869 2024-09-18T10:14:51.788 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.788 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.788 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:14:51.788 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.319446+0000", 2024-09-18T10:14:51.788 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.5945999999999999e-05 2024-09-18T10:14:51.788 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.788 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.788 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:14:51.789 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.421877+0000", 2024-09-18T10:14:51.789 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10243026600000001 2024-09-18T10:14:51.789 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.789 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.789 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:51.789 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.421936+0000", 2024-09-18T10:14:51.789 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.9024999999999998e-05 2024-09-18T10:14:51.789 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:51.789 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:51.789 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:51.789 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.789 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.789 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.7s5 36/32 ECSubWrite(tid=33, reqid=client.4632.0:156, at_version=36'9, trim_to=0'0, roll_forward_to=36'8))", 2024-09-18T10:14:51.790 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:51.227199+0000", 2024-09-18T10:14:51.790 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 0.51460602799999999, 2024-09-18T10:14:51.790 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.19476125799999999, 2024-09-18T10:14:51.790 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:51.790 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:14:51.790 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:51.790 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.790 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:51.790 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.227199+0000", 2024-09-18T10:14:51.790 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:51.790 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.790 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.790 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:51.790 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.227199+0000", 2024-09-18T10:14:51.790 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:51.791 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.791 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.791 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:51.791 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.227198+0000", 2024-09-18T10:14:51.791 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:14:51.791 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.791 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.791 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:51.791 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.227309+0000", 2024-09-18T10:14:51.791 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00011174 2024-09-18T10:14:51.791 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.791 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.791 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:51.792 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.227324+0000", 2024-09-18T10:14:51.792 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4422e-05 2024-09-18T10:14:51.792 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.792 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.792 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:51.792 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.227331+0000", 2024-09-18T10:14:51.792 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.8950000000000001e-06 2024-09-18T10:14:51.792 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.792 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.792 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:51.792 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.419850+0000", 2024-09-18T10:14:51.792 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.19251963599999999 2024-09-18T10:14:51.792 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.792 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.793 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:14:51.793 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.419867+0000", 2024-09-18T10:14:51.793 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.6520000000000001e-05 2024-09-18T10:14:51.793 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.793 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.793 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:14:51.793 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.421943+0000", 2024-09-18T10:14:51.793 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0020762189999999998 2024-09-18T10:14:51.793 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.793 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.793 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:51.793 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.421960+0000", 2024-09-18T10:14:51.793 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7042999999999999e-05 2024-09-18T10:14:51.793 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:51.794 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:51.794 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:51.794 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.794 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.794 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.7s5 36/32 ECSubWrite(tid=34, reqid=client.4632.0:157, at_version=36'10, trim_to=0'0, roll_forward_to=36'8))", 2024-09-18T10:14:51.794 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:51.229493+0000", 2024-09-18T10:14:51.794 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 0.51231229599999994, 2024-09-18T10:14:51.794 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.193027271, 2024-09-18T10:14:51.794 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:51.794 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:14:51.794 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:51.794 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.794 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:51.795 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.229493+0000", 2024-09-18T10:14:51.795 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:51.795 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.795 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.795 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:51.795 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.229493+0000", 2024-09-18T10:14:51.795 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:51.795 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.795 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.795 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:51.795 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.229492+0000", 2024-09-18T10:14:51.795 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999995 2024-09-18T10:14:51.795 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.795 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.795 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:51.796 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.229647+0000", 2024-09-18T10:14:51.796 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00015455800000000001 2024-09-18T10:14:51.796 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.796 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.796 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:51.796 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.229660+0000", 2024-09-18T10:14:51.796 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3567000000000001e-05 2024-09-18T10:14:51.796 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.796 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.796 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:51.796 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.229667+0000", 2024-09-18T10:14:51.796 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.8419999999999999e-06 2024-09-18T10:14:51.796 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.797 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.797 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:51.797 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.420781+0000", 2024-09-18T10:14:51.797 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.19111434199999999 2024-09-18T10:14:51.797 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.797 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.797 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:14:51.797 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.420798+0000", 2024-09-18T10:14:51.797 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7065999999999999e-05 2024-09-18T10:14:51.797 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.797 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.797 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:14:51.797 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.422472+0000", 2024-09-18T10:14:51.797 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001674032 2024-09-18T10:14:51.798 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.798 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.798 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:51.798 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.422520+0000", 2024-09-18T10:14:51.798 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.7543999999999997e-05 2024-09-18T10:14:51.798 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:51.798 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:51.798 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:51.798 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.798 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.798 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.bs2 36/32 ECSubWrite(tid=35, reqid=client.4632.0:160, at_version=36'5, trim_to=0'0, roll_forward_to=36'4))", 2024-09-18T10:14:51.798 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:51.315316+0000", 2024-09-18T10:14:51.798 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 0.42648896800000002, 2024-09-18T10:14:51.799 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.201678933, 2024-09-18T10:14:51.799 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:51.799 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:14:51.799 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:51.799 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.799 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:51.799 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.315316+0000", 2024-09-18T10:14:51.799 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:51.799 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.799 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.799 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:51.799 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.315316+0000", 2024-09-18T10:14:51.799 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:51.800 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.800 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.800 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:51.800 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.315314+0000", 2024-09-18T10:14:51.800 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:14:51.800 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.800 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.800 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:51.800 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.315385+0000", 2024-09-18T10:14:51.800 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.1210000000000004e-05 2024-09-18T10:14:51.800 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.800 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.800 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:51.800 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.315422+0000", 2024-09-18T10:14:51.801 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.7157999999999999e-05 2024-09-18T10:14:51.801 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.801 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.801 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:51.801 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.315441+0000", 2024-09-18T10:14:51.801 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8589000000000001e-05 2024-09-18T10:14:51.801 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.801 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.801 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:51.801 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.515780+0000", 2024-09-18T10:14:51.801 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20033880100000001 2024-09-18T10:14:51.801 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.801 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.801 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:14:51.802 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.515798+0000", 2024-09-18T10:14:51.802 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7762e-05 2024-09-18T10:14:51.802 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.802 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.802 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:14:51.802 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.516927+0000", 2024-09-18T10:14:51.802 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0011289760000000001 2024-09-18T10:14:51.802 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.802 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.802 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:51.802 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.516995+0000", 2024-09-18T10:14:51.802 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.8319999999999996e-05 2024-09-18T10:14:51.802 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:51.803 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:51.803 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:51.803 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.803 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.803 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.4s5 36/32 ECSubWrite(tid=10, reqid=client.4632.0:134, at_version=0'0, trim_to=0'0, roll_forward_to=36'6))", 2024-09-18T10:14:51.803 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:51.324772+0000", 2024-09-18T10:14:51.803 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 0.41703321199999999, 2024-09-18T10:14:51.803 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00097309999999999996, 2024-09-18T10:14:51.803 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:51.803 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:14:51.803 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:51.803 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.803 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:51.803 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.324772+0000", 2024-09-18T10:14:51.804 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:51.804 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.804 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.804 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:51.804 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.324772+0000", 2024-09-18T10:14:51.804 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:51.804 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.804 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.804 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:51.804 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.324770+0000", 2024-09-18T10:14:51.804 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:14:51.804 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.804 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.804 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:51.805 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.324803+0000", 2024-09-18T10:14:51.805 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.3015999999999997e-05 2024-09-18T10:14:51.805 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.805 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.805 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:51.805 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.324848+0000", 2024-09-18T10:14:51.805 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.5011e-05 2024-09-18T10:14:51.805 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.805 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.805 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:51.805 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.324868+0000", 2024-09-18T10:14:51.806 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.0599e-05 2024-09-18T10:14:51.806 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.806 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.806 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:51.806 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.325033+0000", 2024-09-18T10:14:51.806 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00016495600000000001 2024-09-18T10:14:51.806 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.806 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.806 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:14:51.806 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.325058+0000", 2024-09-18T10:14:51.806 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5017000000000001e-05 2024-09-18T10:14:51.806 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.806 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.806 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:14:51.806 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.325697+0000", 2024-09-18T10:14:51.807 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00063912699999999997 2024-09-18T10:14:51.807 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.807 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.807 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:51.807 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.325745+0000", 2024-09-18T10:14:51.807 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.7394000000000001e-05 2024-09-18T10:14:51.807 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:51.807 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:51.807 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:51.807 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.807 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.807 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.7s5 36/32 ECSubWrite(tid=36, reqid=client.4632.0:158, at_version=36'11, trim_to=0'0, roll_forward_to=36'8))", 2024-09-18T10:14:51.807 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:51.331437+0000", 2024-09-18T10:14:51.807 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 0.41036759, 2024-09-18T10:14:51.807 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.091597098000000002, 2024-09-18T10:14:51.808 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:51.808 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:14:51.808 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:51.808 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.808 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:51.808 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.331437+0000", 2024-09-18T10:14:51.808 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:51.808 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.808 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.808 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:51.808 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.331437+0000", 2024-09-18T10:14:51.808 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:51.808 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.808 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.809 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:51.809 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.331436+0000", 2024-09-18T10:14:51.809 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:14:51.809 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.809 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.809 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:51.809 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.331462+0000", 2024-09-18T10:14:51.809 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5358999999999999e-05 2024-09-18T10:14:51.809 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.809 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.809 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:51.809 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.331481+0000", 2024-09-18T10:14:51.809 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8961e-05 2024-09-18T10:14:51.809 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.810 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.810 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:51.810 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.331489+0000", 2024-09-18T10:14:51.810 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.7059999999999993e-06 2024-09-18T10:14:51.810 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.810 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.810 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:51.810 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.421591+0000", 2024-09-18T10:14:51.810 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.090102064999999995 2024-09-18T10:14:51.810 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.810 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.810 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:14:51.810 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.421616+0000", 2024-09-18T10:14:51.811 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4125e-05 2024-09-18T10:14:51.811 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.811 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.811 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:14:51.811 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.422984+0000", 2024-09-18T10:14:51.811 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001368789 2024-09-18T10:14:51.811 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.811 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.811 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:51.811 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.423034+0000", 2024-09-18T10:14:51.811 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.0136999999999998e-05 2024-09-18T10:14:51.811 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:51.811 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:51.811 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:51.812 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.812 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.812 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.9s2 36/32 ECSubWrite(tid=19, reqid=client.4632.0:144, at_version=0'0, trim_to=0'0, roll_forward_to=36'13))", 2024-09-18T10:14:51.812 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:51.534914+0000", 2024-09-18T10:14:51.812 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 0.20689122400000001, 2024-09-18T10:14:51.812 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001102327, 2024-09-18T10:14:51.812 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:51.812 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:14:51.812 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:51.812 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.812 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:51.812 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.534914+0000", 2024-09-18T10:14:51.812 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:51.812 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.813 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.813 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:51.813 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.534914+0000", 2024-09-18T10:14:51.813 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:51.813 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.813 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.813 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:51.813 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.534911+0000", 2024-09-18T10:14:51.813 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999971 2024-09-18T10:14:51.813 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.813 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.813 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:51.813 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.534946+0000", 2024-09-18T10:14:51.813 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.4801000000000001e-05 2024-09-18T10:14:51.814 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.814 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.814 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:51.814 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.534990+0000", 2024-09-18T10:14:51.814 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.3724999999999999e-05 2024-09-18T10:14:51.814 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.814 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.814 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:51.814 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.535008+0000", 2024-09-18T10:14:51.814 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8016000000000002e-05 2024-09-18T10:14:51.814 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.814 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.814 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:51.814 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.535100+0000", 2024-09-18T10:14:51.814 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.2168000000000003e-05 2024-09-18T10:14:51.815 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.815 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.815 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:14:51.815 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.535114+0000", 2024-09-18T10:14:51.815 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3747999999999999e-05 2024-09-18T10:14:51.815 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.815 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.815 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:14:51.815 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.535954+0000", 2024-09-18T10:14:51.815 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00084093799999999995 2024-09-18T10:14:51.815 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.815 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.815 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:51.815 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.536016+0000", 2024-09-18T10:14:51.815 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.1597e-05 2024-09-18T10:14:51.816 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:51.816 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:51.816 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:51.816 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.816 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.816 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpRead(3.9s2 36/32 ECSubRead(tid=20, to_read={3:9221fff5:::smithi18925896-48:head=0,4096,0}, subchunks={3:9221fff5:::smithi18925896-48:head=[0,1]}, attrs_to_read=))", 2024-09-18T10:14:51.816 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:51.536513+0000", 2024-09-18T10:14:51.816 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 0.205291642, 2024-09-18T10:14:51.816 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100528059, 2024-09-18T10:14:51.816 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:51.816 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:14:51.816 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:51.816 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.816 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:51.817 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.536513+0000", 2024-09-18T10:14:51.817 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:51.817 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.817 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.817 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:51.817 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.536513+0000", 2024-09-18T10:14:51.817 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:51.817 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.817 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.817 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:51.817 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.536512+0000", 2024-09-18T10:14:51.817 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:14:51.817 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.818 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.818 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:51.818 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.536531+0000", 2024-09-18T10:14:51.818 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8868999999999998e-05 2024-09-18T10:14:51.818 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.818 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.818 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:51.818 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.536568+0000", 2024-09-18T10:14:51.818 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.7715000000000001e-05 2024-09-18T10:14:51.818 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.818 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.818 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:51.818 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.536585+0000", 2024-09-18T10:14:51.819 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.6569e-05 2024-09-18T10:14:51.819 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.819 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.819 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:51.819 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.636724+0000", 2024-09-18T10:14:51.819 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10013936399999999 2024-09-18T10:14:51.819 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.819 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.819 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:51.819 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.637041+0000", 2024-09-18T10:14:51.819 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00031717200000000001 2024-09-18T10:14:51.819 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:51.819 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:51.819 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:51.820 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.820 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.820 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.bs2 36/32 ECSubWrite(tid=37, reqid=client.4632.0:160, at_version=0'0, trim_to=0'0, roll_forward_to=36'5))", 2024-09-18T10:14:51.820 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:51.631838+0000", 2024-09-18T10:14:51.820 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 0.109967343, 2024-09-18T10:14:51.820 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001173077, 2024-09-18T10:14:51.820 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:51.820 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:14:51.820 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:51.820 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.820 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:51.820 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.631838+0000", 2024-09-18T10:14:51.820 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:51.820 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.821 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.821 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:51.821 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.631838+0000", 2024-09-18T10:14:51.821 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:51.821 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.821 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.821 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:51.821 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.631836+0000", 2024-09-18T10:14:51.821 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:14:51.821 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.821 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.821 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:51.821 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.631864+0000", 2024-09-18T10:14:51.821 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8087999999999999e-05 2024-09-18T10:14:51.822 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.822 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.822 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:51.822 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.631895+0000", 2024-09-18T10:14:51.822 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0942000000000003e-05 2024-09-18T10:14:51.822 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.822 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.822 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:51.822 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.631907+0000", 2024-09-18T10:14:51.822 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2269999999999999e-05 2024-09-18T10:14:51.822 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.822 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.822 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:51.822 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.631978+0000", 2024-09-18T10:14:51.822 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0901000000000004e-05 2024-09-18T10:14:51.823 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.823 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.823 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:14:51.823 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.631990+0000", 2024-09-18T10:14:51.823 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2177e-05 2024-09-18T10:14:51.823 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.823 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.823 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:14:51.823 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.632947+0000", 2024-09-18T10:14:51.823 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000956115 2024-09-18T10:14:51.823 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.823 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.823 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:51.824 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.633011+0000", 2024-09-18T10:14:51.824 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.4064000000000002e-05 2024-09-18T10:14:51.824 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:51.824 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:51.824 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:51.824 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.824 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.824 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpRead(3.1s3 36/32 ECSubRead(tid=38, to_read={3:82ca2386:::smithi18925896-8:head=0,32768,0}, subchunks={3:82ca2386:::smithi18925896-8:head=[0,1]}, attrs_to_read=))", 2024-09-18T10:14:51.824 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:51.631938+0000", 2024-09-18T10:14:51.824 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 0.109867162, 2024-09-18T10:14:51.824 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0055843569999999999, 2024-09-18T10:14:51.824 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:51.824 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:14:51.824 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:51.825 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.825 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:51.825 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.631938+0000", 2024-09-18T10:14:51.825 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:51.825 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.825 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.825 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:51.825 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.631938+0000", 2024-09-18T10:14:51.825 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:51.825 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.825 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.825 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:51.825 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.631936+0000", 2024-09-18T10:14:51.826 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:14:51.826 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.826 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.826 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:51.826 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.631953+0000", 2024-09-18T10:14:51.826 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.6840000000000001e-05 2024-09-18T10:14:51.826 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.826 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.826 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:51.826 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.631978+0000", 2024-09-18T10:14:51.826 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4681999999999998e-05 2024-09-18T10:14:51.826 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.826 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.826 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:51.827 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.631991+0000", 2024-09-18T10:14:51.827 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3179e-05 2024-09-18T10:14:51.827 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.827 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.827 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:51.827 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.637100+0000", 2024-09-18T10:14:51.827 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0051085810000000001 2024-09-18T10:14:51.827 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.827 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.827 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:51.827 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.637522+0000", 2024-09-18T10:14:51.827 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000422378 2024-09-18T10:14:51.827 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:51.827 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:51.828 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:51.828 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.828 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.828 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpRead(3.1s3 36/32 ECSubRead(tid=39, to_read={3:82ca2386:::smithi18925896-8:head=0,32768,0}, subchunks={3:82ca2386:::smithi18925896-8:head=[0,1]}, attrs_to_read=))", 2024-09-18T10:14:51.828 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:51.632006+0000", 2024-09-18T10:14:51.828 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 0.109798751, 2024-09-18T10:14:51.828 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00572223, 2024-09-18T10:14:51.828 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:51.828 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:14:51.828 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:51.828 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.828 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:51.828 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.632006+0000", 2024-09-18T10:14:51.828 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:51.828 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.829 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.829 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:51.829 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.632006+0000", 2024-09-18T10:14:51.829 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:51.829 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.829 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.829 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:51.829 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.632006+0000", 2024-09-18T10:14:51.829 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999995 2024-09-18T10:14:51.829 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.829 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.829 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:51.829 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.632032+0000", 2024-09-18T10:14:51.830 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6332000000000001e-05 2024-09-18T10:14:51.830 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.830 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.830 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:51.830 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.632055+0000", 2024-09-18T10:14:51.830 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.3226000000000001e-05 2024-09-18T10:14:51.830 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.830 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.830 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:51.830 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.632069+0000", 2024-09-18T10:14:51.830 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3579e-05 2024-09-18T10:14:51.830 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.830 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.830 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:51.830 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.637572+0000", 2024-09-18T10:14:51.830 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0055034000000000003 2024-09-18T10:14:51.831 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.831 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.831 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:51.831 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.637728+0000", 2024-09-18T10:14:51.831 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00015622300000000001 2024-09-18T10:14:51.831 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:51.831 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:51.831 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:51.831 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:51.831 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:51.831 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-09-18T10:14:51.833 INFO:tasks.ceph.osd.12.smithi029.stderr:2024-09-18T10:14:51.826+0000 7f8e4e627700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 22190) UID: 0 2024-09-18T10:14:51.833 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_ops_in_flight 2024-09-18T10:14:51.836 INFO:tasks.rados.rados.0.smithi189.stdout:43: done (14 left) 2024-09-18T10:14:51.836 INFO:tasks.rados.rados.0.smithi189.stdout:45: done (13 left) 2024-09-18T10:14:51.836 INFO:tasks.rados.rados.0.smithi189.stdout:46: done (12 left) 2024-09-18T10:14:51.836 INFO:tasks.rados.rados.0.smithi189.stdout:47: done (11 left) 2024-09-18T10:14:51.836 INFO:tasks.rados.rados.0.smithi189.stdout:48: done (10 left) 2024-09-18T10:14:51.836 INFO:tasks.rados.rados.0.smithi189.stdout:61: done (9 left) 2024-09-18T10:14:51.836 INFO:tasks.rados.rados.0.smithi189.stdout:62: read oid 17 snap -1 2024-09-18T10:14:51.836 INFO:tasks.rados.rados.0.smithi189.stdout:62: expect (ObjNum 16 snap 0 seq_num 16) 2024-09-18T10:14:51.837 INFO:tasks.rados.rados.0.smithi189.stdout:63: snap_create 2024-09-18T10:14:51.841 INFO:tasks.rados.rados.0.smithi189.stdout:58: finishing copy_from racing read to smithi18925896-19 2024-09-18T10:14:51.842 INFO:tasks.rados.rados.0.smithi189.stdout:51: finishing copy_from racing read to smithi18925896-2 2024-09-18T10:14:51.932 INFO:tasks.rados.rados.0.smithi189.stdout:50: finishing write tid 3 to smithi18925896-50 2024-09-18T10:14:51.933 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-18T10:14:51.926+0000 7f1981873700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22132) UID: 0 2024-09-18T10:14:51.936 INFO:tasks.rados.rados.0.smithi189.stdout:57: expect (ObjNum 7 snap 0 seq_num 7) 2024-09-18T10:14:51.941 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-09-18T10:14:51.942 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-09-18T10:14:51.942 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.942 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.6s1 36/32 ECSubWrite(tid=19, reqid=client.4632.0:167, at_version=36'11, trim_to=0'0, roll_forward_to=36'10))", 2024-09-18T10:14:51.942 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:51.840327+0000", 2024-09-18T10:14:51.942 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 0.101175047, 2024-09-18T10:14:51.942 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10121361500000001, 2024-09-18T10:14:51.942 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:51.942 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:14:51.942 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:51.942 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.942 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:51.942 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.840327+0000", 2024-09-18T10:14:51.943 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:51.943 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.943 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.943 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:51.943 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.840327+0000", 2024-09-18T10:14:51.943 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:51.943 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.943 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.943 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:51.943 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.840326+0000", 2024-09-18T10:14:51.943 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:14:51.943 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.943 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.943 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:51.944 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.840534+0000", 2024-09-18T10:14:51.944 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000208168 2024-09-18T10:14:51.944 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.944 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.944 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:51.944 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.840575+0000", 2024-09-18T10:14:51.944 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.0497999999999997e-05 2024-09-18T10:14:51.944 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.944 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.944 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:51.944 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.840593+0000", 2024-09-18T10:14:51.944 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8834999999999998e-05 2024-09-18T10:14:51.944 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.944 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.945 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:51.945 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.940848+0000", 2024-09-18T10:14:51.945 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10025437500000001 2024-09-18T10:14:51.945 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:51.945 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:51.945 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:14:51.945 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.940871+0000", 2024-09-18T10:14:51.945 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.3635999999999999e-05 2024-09-18T10:14:51.945 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:51.945 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:51.945 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:51.945 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:51.945 INFO:teuthology.orchestra.run.smithi154.stdout: ], 2024-09-18T10:14:51.945 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 1 2024-09-18T10:14:51.946 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-09-18T10:14:51.946 INFO:tasks.rados.rados.0.smithi189.stdout:51: finishing copy_from to smithi18925896-2 2024-09-18T10:14:51.946 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 2 v 4 (ObjNum 10 snap 0 seq_num 10) dirty exists 2024-09-18T10:14:51.946 INFO:tasks.rados.rados.0.smithi189.stdout:58: finishing copy_from to smithi18925896-19 2024-09-18T10:14:51.946 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 19 v 4 (ObjNum 12 snap 0 seq_num 12) dirty exists 2024-09-18T10:14:51.952 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_blocked_ops 2024-09-18T10:14:52.034 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-18T10:14:52.030+0000 7f578c52f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22134) UID: 0 2024-09-18T10:14:52.037 INFO:tasks.rados.rados.0.smithi189.stdout:49: finishing write tid 3 to smithi18925896-49 2024-09-18T10:14:52.039 INFO:tasks.rados.rados.0.smithi189.stdout:39: finishing write tid 3 to smithi18925896-39 2024-09-18T10:14:52.039 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 39 v 2 (ObjNum 38 snap 0 seq_num 38) dirty exists 2024-09-18T10:14:52.040 INFO:tasks.rados.rados.0.smithi189.stdout:39: left oid 39 (ObjNum 38 snap 0 seq_num 38) 2024-09-18T10:14:52.041 INFO:tasks.rados.rados.0.smithi189.stdout:60: finishing copy_from racing read to smithi18925896-26 2024-09-18T10:14:52.042 INFO:tasks.rados.rados.0.smithi189.stdout:52: finishing write tid 4 to smithi18925896-25 2024-09-18T10:14:52.042 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 25 v 11 (ObjNum 50 snap 0 seq_num 50) dirty exists 2024-09-18T10:14:52.042 INFO:tasks.rados.rados.0.smithi189.stdout:52: left oid 25 (ObjNum 50 snap 0 seq_num 50) 2024-09-18T10:14:52.051 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-09-18T10:14:52.051 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-09-18T10:14:52.052 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-09-18T10:14:52.052 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-09-18T10:14:52.052 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-09-18T10:14:52.061 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_historic_ops 2024-09-18T10:14:52.138 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-18T10:14:52.135+0000 7f14dcc5b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22398) UID: 0 2024-09-18T10:14:52.140 INFO:tasks.rados.rados.0.smithi189.stdout:49: finishing write tid 4 to smithi18925896-49 2024-09-18T10:14:52.140 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 49 v 5 (ObjNum 48 snap 0 seq_num 48) dirty exists 2024-09-18T10:14:52.140 INFO:tasks.rados.rados.0.smithi189.stdout:49: left oid 49 (ObjNum 48 snap 0 seq_num 48) 2024-09-18T10:14:52.141 INFO:tasks.rados.rados.0.smithi189.stdout:60: finishing copy_from to smithi18925896-26 2024-09-18T10:14:52.142 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 26 v 6 (ObjNum 33 snap 0 seq_num 33) dirty exists 2024-09-18T10:14:52.157 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-09-18T10:14:52.157 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-09-18T10:14:52.157 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-09-18T10:14:52.157 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-09-18T10:14:52.157 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.157 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4341.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-18T10:14:52.158 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:35.324351+0000", 2024-09-18T10:14:52.158 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 16.831481994000001, 2024-09-18T10:14:52.158 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.36640713699999999, 2024-09-18T10:14:52.158 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:52.158 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:14:52.158 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:14:52.158 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4341", 2024-09-18T10:14:52.158 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.29:0/3001591505", 2024-09-18T10:14:52.158 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 1 2024-09-18T10:14:52.158 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.158 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:52.158 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.159 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:52.159 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.324351+0000", 2024-09-18T10:14:52.159 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:52.159 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.159 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.159 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:52.159 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.324351+0000", 2024-09-18T10:14:52.159 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:52.159 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.159 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.159 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:52.159 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.324348+0000", 2024-09-18T10:14:52.160 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999976 2024-09-18T10:14:52.160 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.160 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.160 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:52.160 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.324359+0000", 2024-09-18T10:14:52.160 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1048e-05 2024-09-18T10:14:52.160 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.160 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.160 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:52.160 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.324383+0000", 2024-09-18T10:14:52.160 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.3813000000000001e-05 2024-09-18T10:14:52.160 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.161 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.161 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:52.161 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.324421+0000", 2024-09-18T10:14:52.161 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8012000000000001e-05 2024-09-18T10:14:52.161 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.161 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.161 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:52.161 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.324548+0000", 2024-09-18T10:14:52.161 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00012702199999999999 2024-09-18T10:14:52.161 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.161 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.161 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for peered", 2024-09-18T10:14:52.161 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.324565+0000", 2024-09-18T10:14:52.162 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.6325e-05 2024-09-18T10:14:52.162 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.162 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.162 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:52.162 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.588380+0000", 2024-09-18T10:14:52.162 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.263815458 2024-09-18T10:14:52.162 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.162 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.162 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:14:52.162 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.588626+0000", 2024-09-18T10:14:52.162 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00024633499999999998 2024-09-18T10:14:52.162 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.163 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.163 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 5", 2024-09-18T10:14:52.163 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.588803+0000", 2024-09-18T10:14:52.163 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00017640599999999999 2024-09-18T10:14:52.163 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.163 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.163 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:14:52.163 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.589943+0000", 2024-09-18T10:14:52.163 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0011400049999999999 2024-09-18T10:14:52.163 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.163 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.163 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:14:52.163 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.690623+0000", 2024-09-18T10:14:52.164 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100680032 2024-09-18T10:14:52.164 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.164 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.164 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:14:52.164 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.690702+0000", 2024-09-18T10:14:52.164 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.9342e-05 2024-09-18T10:14:52.164 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.164 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.164 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:52.164 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.690758+0000", 2024-09-18T10:14:52.164 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5639e-05 2024-09-18T10:14:52.164 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:52.164 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:52.165 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:52.165 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.165 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.165 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWriteReply(3.2s0 36/32 ECSubWriteReply(tid=15, last_complete=36'8, committed=1, applied=1))", 2024-09-18T10:14:52.165 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:51.214237+0000", 2024-09-18T10:14:52.165 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 0.94159600700000001, 2024-09-18T10:14:52.165 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.017069002, 2024-09-18T10:14:52.165 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:52.165 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:14:52.165 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:52.165 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.165 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:52.165 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.214237+0000", 2024-09-18T10:14:52.166 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:52.166 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.166 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.166 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:52.166 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.214237+0000", 2024-09-18T10:14:52.166 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:52.166 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.166 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.166 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:52.166 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.214235+0000", 2024-09-18T10:14:52.166 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:14:52.166 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.166 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.167 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:52.167 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.214245+0000", 2024-09-18T10:14:52.167 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.6560000000000004e-06 2024-09-18T10:14:52.167 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.167 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.167 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:52.167 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.214296+0000", 2024-09-18T10:14:52.167 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1134999999999997e-05 2024-09-18T10:14:52.167 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.167 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.167 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:52.167 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.214314+0000", 2024-09-18T10:14:52.167 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8179e-05 2024-09-18T10:14:52.167 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.168 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.168 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:52.168 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.231270+0000", 2024-09-18T10:14:52.168 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.016955846 2024-09-18T10:14:52.168 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.168 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.168 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:52.168 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.231306+0000", 2024-09-18T10:14:52.168 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5623000000000001e-05 2024-09-18T10:14:52.168 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:52.168 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:52.168 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:52.168 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.168 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.169 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.cs6 36/32 ECSubWrite(tid=20, reqid=client.4632.0:150, at_version=36'9, trim_to=0'0, roll_forward_to=36'8))", 2024-09-18T10:14:52.169 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:51.214349+0000", 2024-09-18T10:14:52.169 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 0.94148323899999997, 2024-09-18T10:14:52.169 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.102450213, 2024-09-18T10:14:52.169 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:52.169 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:14:52.169 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:52.169 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.169 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:52.169 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.214349+0000", 2024-09-18T10:14:52.169 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:52.169 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.169 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.169 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:52.170 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.214349+0000", 2024-09-18T10:14:52.170 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:52.170 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.170 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.170 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:52.170 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.214349+0000", 2024-09-18T10:14:52.170 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:14:52.170 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.170 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.170 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:52.170 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.214497+0000", 2024-09-18T10:14:52.170 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00014846 2024-09-18T10:14:52.170 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.170 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.171 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:52.171 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.214524+0000", 2024-09-18T10:14:52.171 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6778999999999999e-05 2024-09-18T10:14:52.171 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.171 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.171 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:52.171 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.214533+0000", 2024-09-18T10:14:52.171 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.1589999999999999e-06 2024-09-18T10:14:52.171 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.171 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.171 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:52.171 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.214667+0000", 2024-09-18T10:14:52.171 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00013415700000000001 2024-09-18T10:14:52.171 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.172 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.172 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:14:52.172 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.214692+0000", 2024-09-18T10:14:52.172 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4986000000000001e-05 2024-09-18T10:14:52.172 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.172 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.172 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:14:52.172 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.316728+0000", 2024-09-18T10:14:52.172 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.102035442 2024-09-18T10:14:52.172 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.172 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.172 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:52.172 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.316800+0000", 2024-09-18T10:14:52.172 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.1993e-05 2024-09-18T10:14:52.173 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:52.173 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:52.173 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:52.173 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.173 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.173 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.cs6 36/32 ECSubWrite(tid=21, reqid=client.4632.0:151, at_version=36'10, trim_to=0'0, roll_forward_to=36'8))", 2024-09-18T10:14:52.173 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:51.214556+0000", 2024-09-18T10:14:52.173 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 0.94127638899999999, 2024-09-18T10:14:52.173 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10287521099999999, 2024-09-18T10:14:52.173 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:52.173 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:14:52.173 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:52.173 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.173 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:52.174 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.214556+0000", 2024-09-18T10:14:52.174 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:52.174 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.174 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.174 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:52.174 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.214556+0000", 2024-09-18T10:14:52.174 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:52.174 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.174 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.174 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:52.174 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.214556+0000", 2024-09-18T10:14:52.174 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999995 2024-09-18T10:14:52.174 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.174 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.175 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:52.175 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.214657+0000", 2024-09-18T10:14:52.175 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000101139 2024-09-18T10:14:52.175 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.175 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.175 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:52.175 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.214669+0000", 2024-09-18T10:14:52.175 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2243e-05 2024-09-18T10:14:52.175 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.175 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.175 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:52.175 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.214676+0000", 2024-09-18T10:14:52.175 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.413e-06 2024-09-18T10:14:52.176 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.176 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.176 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:52.176 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.315767+0000", 2024-09-18T10:14:52.176 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10109061699999999 2024-09-18T10:14:52.176 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.176 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.176 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:14:52.176 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.315782+0000", 2024-09-18T10:14:52.176 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.537e-05 2024-09-18T10:14:52.176 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.176 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.176 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:14:52.176 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.317393+0000", 2024-09-18T10:14:52.177 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001610708 2024-09-18T10:14:52.177 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.177 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.177 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:52.177 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.317431+0000", 2024-09-18T10:14:52.177 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8393999999999999e-05 2024-09-18T10:14:52.177 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:52.177 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:52.177 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:52.177 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.177 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.177 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.es1 36/32 ECSubWrite(tid=4, reqid=client.4632.0:146, at_version=36'3, trim_to=0'0, roll_forward_to=36'2))", 2024-09-18T10:14:52.177 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:51.227104+0000", 2024-09-18T10:14:52.177 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 0.92872878699999994, 2024-09-18T10:14:52.178 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001534175, 2024-09-18T10:14:52.178 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:52.178 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:14:52.178 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:52.178 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.178 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:52.178 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.227104+0000", 2024-09-18T10:14:52.178 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:52.178 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.178 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.178 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:52.178 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.227104+0000", 2024-09-18T10:14:52.178 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:52.178 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.179 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.179 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:52.179 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.227103+0000", 2024-09-18T10:14:52.179 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999995 2024-09-18T10:14:52.179 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.179 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.179 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:52.179 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.227195+0000", 2024-09-18T10:14:52.179 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.1984999999999994e-05 2024-09-18T10:14:52.179 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.179 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.179 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:52.179 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.227215+0000", 2024-09-18T10:14:52.179 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.9466e-05 2024-09-18T10:14:52.180 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.180 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.180 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:52.180 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.227223+0000", 2024-09-18T10:14:52.180 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.2130000000000008e-06 2024-09-18T10:14:52.180 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.180 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.180 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:52.180 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.227672+0000", 2024-09-18T10:14:52.180 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00044864500000000001 2024-09-18T10:14:52.180 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.180 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.180 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:14:52.181 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.227679+0000", 2024-09-18T10:14:52.181 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.3579999999999997e-06 2024-09-18T10:14:52.181 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.181 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.181 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:14:52.181 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.228571+0000", 2024-09-18T10:14:52.181 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00089176000000000001 2024-09-18T10:14:52.181 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.181 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.181 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:52.181 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.228638+0000", 2024-09-18T10:14:52.181 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.7211000000000004e-05 2024-09-18T10:14:52.181 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:52.182 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:52.182 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:52.182 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.182 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.182 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.7s4 36/32 ECSubWrite(tid=33, reqid=client.4632.0:156, at_version=36'9, trim_to=0'0, roll_forward_to=36'8))", 2024-09-18T10:14:52.182 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:51.227449+0000", 2024-09-18T10:14:52.182 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 0.92838331200000002, 2024-09-18T10:14:52.182 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.002016314, 2024-09-18T10:14:52.182 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:52.182 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:14:52.182 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:52.182 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.182 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:52.182 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.227449+0000", 2024-09-18T10:14:52.183 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:52.183 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.183 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.183 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:52.183 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.227449+0000", 2024-09-18T10:14:52.183 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:52.183 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.183 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.183 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:52.183 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.227448+0000", 2024-09-18T10:14:52.183 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:14:52.183 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.183 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.183 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:52.184 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.227747+0000", 2024-09-18T10:14:52.184 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000298876 2024-09-18T10:14:52.184 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.184 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.184 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:52.184 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.227758+0000", 2024-09-18T10:14:52.184 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1083e-05 2024-09-18T10:14:52.184 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.184 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.184 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:52.184 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.227766+0000", 2024-09-18T10:14:52.184 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.3100000000000001e-06 2024-09-18T10:14:52.184 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.185 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.185 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:52.185 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.227837+0000", 2024-09-18T10:14:52.185 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0307000000000003e-05 2024-09-18T10:14:52.185 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.185 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.185 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:14:52.185 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.227850+0000", 2024-09-18T10:14:52.185 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3356000000000001e-05 2024-09-18T10:14:52.185 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.185 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.185 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:14:52.185 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.229438+0000", 2024-09-18T10:14:52.185 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001588059 2024-09-18T10:14:52.186 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.186 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.186 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:52.186 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.229466+0000", 2024-09-18T10:14:52.186 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.7379999999999999e-05 2024-09-18T10:14:52.186 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:52.186 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:52.186 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:52.186 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.186 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.186 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpReadReply(3.2s0 36/32 ECSubReadReply(tid=16, attrs_read=0))", 2024-09-18T10:14:52.186 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:51.231167+0000", 2024-09-18T10:14:52.187 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 0.92466606299999998, 2024-09-18T10:14:52.187 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.083329293999999998, 2024-09-18T10:14:52.187 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:52.187 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:14:52.187 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:52.187 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.187 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:52.187 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.231167+0000", 2024-09-18T10:14:52.187 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:52.187 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.187 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.187 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:52.188 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.231167+0000", 2024-09-18T10:14:52.188 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:52.188 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.188 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.188 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:52.188 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.231166+0000", 2024-09-18T10:14:52.188 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999995 2024-09-18T10:14:52.188 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.188 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.188 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:52.188 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.231189+0000", 2024-09-18T10:14:52.188 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.3161999999999999e-05 2024-09-18T10:14:52.188 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.189 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.189 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:52.189 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.231219+0000", 2024-09-18T10:14:52.189 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0176e-05 2024-09-18T10:14:52.189 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.189 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.189 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:52.189 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.231232+0000", 2024-09-18T10:14:52.189 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2439000000000001e-05 2024-09-18T10:14:52.189 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.189 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.189 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:52.190 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.314408+0000", 2024-09-18T10:14:52.190 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.083176343 2024-09-18T10:14:52.190 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.190 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.190 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:52.190 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.314496+0000", 2024-09-18T10:14:52.190 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.7806999999999995e-05 2024-09-18T10:14:52.190 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:52.190 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:52.190 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:52.190 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.190 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.190 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.bs6 36/32 ECSubWrite(tid=35, reqid=client.4632.0:160, at_version=36'5, trim_to=0'0, roll_forward_to=36'4))", 2024-09-18T10:14:52.191 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:51.315444+0000", 2024-09-18T10:14:52.191 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 0.84038834299999998, 2024-09-18T10:14:52.191 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0015330389999999999, 2024-09-18T10:14:52.191 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:52.191 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:14:52.191 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:52.191 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.191 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:52.191 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.315444+0000", 2024-09-18T10:14:52.191 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:52.191 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.191 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.191 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:52.192 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.315444+0000", 2024-09-18T10:14:52.192 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:52.192 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.192 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.192 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:52.192 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.315443+0000", 2024-09-18T10:14:52.192 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:14:52.192 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.192 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.192 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:52.192 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.315482+0000", 2024-09-18T10:14:52.192 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.9100999999999997e-05 2024-09-18T10:14:52.193 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.193 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.193 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:52.193 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.315510+0000", 2024-09-18T10:14:52.193 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.7727999999999999e-05 2024-09-18T10:14:52.193 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.193 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.193 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:52.193 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.315527+0000", 2024-09-18T10:14:52.193 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7615999999999999e-05 2024-09-18T10:14:52.193 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.193 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.194 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:52.194 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.315642+0000", 2024-09-18T10:14:52.194 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00011452599999999999 2024-09-18T10:14:52.194 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.194 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.194 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:14:52.194 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.315663+0000", 2024-09-18T10:14:52.194 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.0806e-05 2024-09-18T10:14:52.194 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.194 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.194 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:14:52.194 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.316913+0000", 2024-09-18T10:14:52.194 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0012498170000000001 2024-09-18T10:14:52.194 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.195 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.195 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:52.195 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.316977+0000", 2024-09-18T10:14:52.195 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.4653999999999995e-05 2024-09-18T10:14:52.195 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:52.195 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:52.195 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:52.195 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.195 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.195 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.fs3 36/32 ECSubWrite(tid=23, reqid=client.4632.0:110, at_version=0'0, trim_to=0'0, roll_forward_to=36'4))", 2024-09-18T10:14:52.195 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:51.323454+0000", 2024-09-18T10:14:52.195 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 0.83237878099999996, 2024-09-18T10:14:52.196 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0016473, 2024-09-18T10:14:52.196 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:52.196 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:14:52.196 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:52.196 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.196 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:52.196 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.323454+0000", 2024-09-18T10:14:52.196 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:52.196 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.196 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.197 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:52.197 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.323454+0000", 2024-09-18T10:14:52.197 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:52.197 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.197 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.197 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:52.197 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.323452+0000", 2024-09-18T10:14:52.197 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:14:52.197 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.197 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.197 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:52.197 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.323476+0000", 2024-09-18T10:14:52.198 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.3849000000000002e-05 2024-09-18T10:14:52.198 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.198 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.198 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:52.198 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.323504+0000", 2024-09-18T10:14:52.198 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.7602000000000001e-05 2024-09-18T10:14:52.198 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.198 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.198 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:52.198 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.323522+0000", 2024-09-18T10:14:52.198 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7666e-05 2024-09-18T10:14:52.198 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.198 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.199 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:52.199 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.323647+0000", 2024-09-18T10:14:52.199 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000125241 2024-09-18T10:14:52.199 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.199 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.199 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:14:52.199 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.323685+0000", 2024-09-18T10:14:52.199 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8358000000000001e-05 2024-09-18T10:14:52.199 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.199 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.199 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:14:52.199 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.325052+0000", 2024-09-18T10:14:52.199 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0013669120000000001 2024-09-18T10:14:52.199 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.200 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.200 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:52.200 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.325101+0000", 2024-09-18T10:14:52.200 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9064999999999999e-05 2024-09-18T10:14:52.200 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:52.200 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:52.200 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:52.200 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.200 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.200 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.es1 36/32 ECSubWrite(tid=5, reqid=client.4632.0:147, at_version=36'4, trim_to=0'0, roll_forward_to=36'2))", 2024-09-18T10:14:52.200 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:51.329371+0000", 2024-09-18T10:14:52.200 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 0.82646152100000003, 2024-09-18T10:14:52.200 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10310915800000001, 2024-09-18T10:14:52.201 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:52.201 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:14:52.201 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:52.201 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.201 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:52.201 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.329371+0000", 2024-09-18T10:14:52.201 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:52.201 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.201 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.201 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:52.201 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.329371+0000", 2024-09-18T10:14:52.201 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:52.202 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.202 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.202 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:52.202 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.329370+0000", 2024-09-18T10:14:52.202 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:14:52.202 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.202 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.202 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:52.202 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.329445+0000", 2024-09-18T10:14:52.202 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.5643999999999996e-05 2024-09-18T10:14:52.202 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.202 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.203 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:52.203 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.329471+0000", 2024-09-18T10:14:52.203 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5613e-05 2024-09-18T10:14:52.203 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.203 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.203 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:52.203 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.429565+0000", 2024-09-18T10:14:52.203 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100093746 2024-09-18T10:14:52.203 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.203 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.203 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:52.203 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.429690+0000", 2024-09-18T10:14:52.204 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00012519699999999999 2024-09-18T10:14:52.204 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.204 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.204 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:14:52.204 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.429715+0000", 2024-09-18T10:14:52.204 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4769e-05 2024-09-18T10:14:52.204 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.204 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.204 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:14:52.204 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.432431+0000", 2024-09-18T10:14:52.204 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.002715884 2024-09-18T10:14:52.205 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.205 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.205 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:52.205 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.432480+0000", 2024-09-18T10:14:52.205 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9662e-05 2024-09-18T10:14:52.205 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:52.205 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:52.205 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:52.205 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.205 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.205 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.cs6 36/32 ECSubWrite(tid=22, reqid=client.4632.0:151, at_version=0'0, trim_to=0'0, roll_forward_to=36'10))", 2024-09-18T10:14:52.205 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:51.534597+0000", 2024-09-18T10:14:52.205 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 0.62123608100000005, 2024-09-18T10:14:52.206 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.003414767, 2024-09-18T10:14:52.206 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:52.206 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:14:52.206 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:52.206 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.206 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:52.206 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.534597+0000", 2024-09-18T10:14:52.206 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:52.206 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.206 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.206 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:52.206 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.534597+0000", 2024-09-18T10:14:52.206 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:52.207 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.207 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.207 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:52.207 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.534595+0000", 2024-09-18T10:14:52.207 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:14:52.207 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.207 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.207 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:52.207 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.534624+0000", 2024-09-18T10:14:52.207 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9736000000000001e-05 2024-09-18T10:14:52.207 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.207 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.208 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:52.208 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.534659+0000", 2024-09-18T10:14:52.208 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.4498000000000001e-05 2024-09-18T10:14:52.208 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.208 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.208 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:52.208 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.534674+0000", 2024-09-18T10:14:52.208 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.5586e-05 2024-09-18T10:14:52.208 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.208 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.208 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:52.208 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.536909+0000", 2024-09-18T10:14:52.208 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0022346499999999999 2024-09-18T10:14:52.209 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.209 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.209 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:14:52.209 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.536939+0000", 2024-09-18T10:14:52.209 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9714999999999999e-05 2024-09-18T10:14:52.209 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.209 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.209 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:14:52.209 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.537973+0000", 2024-09-18T10:14:52.209 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0010340900000000001 2024-09-18T10:14:52.209 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.209 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.209 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:52.210 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.538011+0000", 2024-09-18T10:14:52.210 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8328999999999998e-05 2024-09-18T10:14:52.210 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:52.210 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:52.210 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:52.210 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.210 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.210 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.cs6 36/32 ECSubWrite(tid=23, reqid=client.4632.0:152, at_version=36'11, trim_to=0'0, roll_forward_to=36'10))", 2024-09-18T10:14:52.210 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:51.536630+0000", 2024-09-18T10:14:52.210 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 0.61920293199999998, 2024-09-18T10:14:52.210 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.19942764099999999, 2024-09-18T10:14:52.210 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:52.210 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:14:52.210 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:52.211 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.211 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:52.211 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.536630+0000", 2024-09-18T10:14:52.211 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:52.211 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.211 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.211 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:52.211 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.536630+0000", 2024-09-18T10:14:52.211 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:52.211 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.211 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.211 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:52.211 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.536628+0000", 2024-09-18T10:14:52.212 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:14:52.212 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.212 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.212 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:52.212 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.536706+0000", 2024-09-18T10:14:52.212 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.8518999999999998e-05 2024-09-18T10:14:52.212 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.212 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.212 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:52.212 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.536788+0000", 2024-09-18T10:14:52.212 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.2148e-05 2024-09-18T10:14:52.212 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.212 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.213 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:52.213 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.536796+0000", 2024-09-18T10:14:52.213 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.3150000000000003e-06 2024-09-18T10:14:52.213 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.213 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.213 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:52.213 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.735249+0000", 2024-09-18T10:14:52.213 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.19845374599999999 2024-09-18T10:14:52.213 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.213 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.213 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:14:52.213 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.735287+0000", 2024-09-18T10:14:52.214 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.7153000000000002e-05 2024-09-18T10:14:52.214 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.214 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.214 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:14:52.214 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.735989+0000", 2024-09-18T10:14:52.214 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00070191999999999997 2024-09-18T10:14:52.214 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.214 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.214 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:52.214 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.736057+0000", 2024-09-18T10:14:52.214 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.8769999999999994e-05 2024-09-18T10:14:52.214 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:52.214 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:52.215 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:52.215 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.215 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.215 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(osd.4.21:1 3.2s0 3:46b2871c:::smithi18925896-34:head [assert-version v6,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:14:52.215 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:51.536666+0000", 2024-09-18T10:14:52.215 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 0.61916661500000003, 2024-09-18T10:14:52.215 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.30016440500000002, 2024-09-18T10:14:52.215 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:52.215 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "started", 2024-09-18T10:14:52.215 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:52.215 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.215 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:52.216 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.536666+0000", 2024-09-18T10:14:52.216 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:52.216 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.216 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.216 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:52.216 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.536666+0000", 2024-09-18T10:14:52.216 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:52.216 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.216 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.216 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:52.216 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.536664+0000", 2024-09-18T10:14:52.216 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:14:52.217 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.217 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.217 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:52.217 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.536673+0000", 2024-09-18T10:14:52.217 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.6319999999999997e-06 2024-09-18T10:14:52.217 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.217 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.217 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:52.217 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.536688+0000", 2024-09-18T10:14:52.217 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.5058999999999999e-05 2024-09-18T10:14:52.217 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.217 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.218 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:52.218 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.536696+0000", 2024-09-18T10:14:52.218 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.8860000000000004e-06 2024-09-18T10:14:52.218 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.218 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.218 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:52.218 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.536868+0000", 2024-09-18T10:14:52.218 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000172475 2024-09-18T10:14:52.218 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.218 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.218 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:14:52.218 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.537125+0000", 2024-09-18T10:14:52.219 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000256906 2024-09-18T10:14:52.219 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.219 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.219 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:52.219 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.836830+0000", 2024-09-18T10:14:52.219 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.299705414 2024-09-18T10:14:52.219 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:52.219 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:52.219 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:52.219 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.219 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.219 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpReadReply(3.2s0 36/32 ECSubReadReply(tid=17, attrs_read=0))", 2024-09-18T10:14:52.219 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:51.538715+0000", 2024-09-18T10:14:52.220 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 0.61711760599999999, 2024-09-18T10:14:52.220 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100758325, 2024-09-18T10:14:52.220 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:52.220 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:14:52.220 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:52.220 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.220 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:52.220 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.538715+0000", 2024-09-18T10:14:52.220 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:52.220 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.220 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.220 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:52.220 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.538715+0000", 2024-09-18T10:14:52.221 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:52.221 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.221 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.221 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:52.221 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.538714+0000", 2024-09-18T10:14:52.221 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999995 2024-09-18T10:14:52.221 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.221 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.221 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:52.222 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.539016+0000", 2024-09-18T10:14:52.222 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00030171100000000003 2024-09-18T10:14:52.222 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.222 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.222 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:52.222 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.539023+0000", 2024-09-18T10:14:52.222 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.4030000000000001e-06 2024-09-18T10:14:52.222 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.222 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.222 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:52.222 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.539028+0000", 2024-09-18T10:14:52.222 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.5050000000000001e-06 2024-09-18T10:14:52.222 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.223 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.223 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:52.223 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.639347+0000", 2024-09-18T10:14:52.223 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100318904 2024-09-18T10:14:52.223 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.223 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.223 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:52.223 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.639473+0000", 2024-09-18T10:14:52.223 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00012650399999999999 2024-09-18T10:14:52.223 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:52.223 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:52.223 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:52.223 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.224 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.224 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpReadReply(3.2s0 36/32 ECSubReadReply(tid=17, attrs_read=0))", 2024-09-18T10:14:52.224 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:51.538722+0000", 2024-09-18T10:14:52.224 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 0.61711051299999997, 2024-09-18T10:14:52.224 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10052570299999999, 2024-09-18T10:14:52.224 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:52.224 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:14:52.224 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:52.224 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.224 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:52.224 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.538722+0000", 2024-09-18T10:14:52.224 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:52.225 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.225 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.225 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:52.225 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.538722+0000", 2024-09-18T10:14:52.225 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:52.225 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.225 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.225 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:52.225 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.538722+0000", 2024-09-18T10:14:52.225 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:52.225 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.226 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.226 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:52.226 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.538722+0000", 2024-09-18T10:14:52.226 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:52.226 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.226 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.226 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:52.226 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.538959+0000", 2024-09-18T10:14:52.226 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00023657300000000001 2024-09-18T10:14:52.226 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.226 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.226 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:52.226 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.539015+0000", 2024-09-18T10:14:52.226 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.6511000000000002e-05 2024-09-18T10:14:52.227 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.227 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.227 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:52.227 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.639175+0000", 2024-09-18T10:14:52.227 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100160339 2024-09-18T10:14:52.227 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.227 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.227 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:52.227 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.639248+0000", 2024-09-18T10:14:52.227 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.2280000000000006e-05 2024-09-18T10:14:52.227 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:52.227 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:52.227 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:52.227 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.227 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.228 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.es1 36/32 ECSubWrite(tid=6, reqid=client.4632.0:147, at_version=0'0, trim_to=0'0, roll_forward_to=36'4))", 2024-09-18T10:14:52.228 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:51.733088+0000", 2024-09-18T10:14:52.228 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 0.42274470200000003, 2024-09-18T10:14:52.228 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0015314020000000001, 2024-09-18T10:14:52.228 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:52.228 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:14:52.228 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:52.228 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.228 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:52.228 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.733088+0000", 2024-09-18T10:14:52.228 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:52.229 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.229 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.229 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:52.229 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.733088+0000", 2024-09-18T10:14:52.229 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:52.229 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.229 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.229 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:52.229 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.733086+0000", 2024-09-18T10:14:52.229 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999976 2024-09-18T10:14:52.229 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.230 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.230 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:52.230 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.733122+0000", 2024-09-18T10:14:52.230 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.6078999999999998e-05 2024-09-18T10:14:52.230 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.230 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.230 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:52.230 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.733170+0000", 2024-09-18T10:14:52.230 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.8201000000000001e-05 2024-09-18T10:14:52.230 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.230 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.230 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:52.230 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.733199+0000", 2024-09-18T10:14:52.230 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9499e-05 2024-09-18T10:14:52.230 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.231 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.231 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:52.231 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.733354+0000", 2024-09-18T10:14:52.231 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00015453600000000001 2024-09-18T10:14:52.231 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.231 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.231 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:14:52.231 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.733386+0000", 2024-09-18T10:14:52.232 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1878999999999998e-05 2024-09-18T10:14:52.232 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.232 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.232 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:14:52.232 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.734565+0000", 2024-09-18T10:14:52.232 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0011787670000000001 2024-09-18T10:14:52.232 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.232 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.232 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:52.233 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.734619+0000", 2024-09-18T10:14:52.233 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.4620999999999998e-05 2024-09-18T10:14:52.233 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:52.233 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:52.233 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:52.233 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.233 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.233 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.7s4 36/32 ECSubWrite(tid=40, reqid=client.4632.0:158, at_version=0'0, trim_to=0'0, roll_forward_to=36'11))", 2024-09-18T10:14:52.233 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:51.733979+0000", 2024-09-18T10:14:52.233 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 0.42185346800000001, 2024-09-18T10:14:52.233 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001511872, 2024-09-18T10:14:52.233 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:52.234 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:14:52.234 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:52.234 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.234 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:52.234 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.733979+0000", 2024-09-18T10:14:52.234 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:52.234 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.234 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.234 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:52.234 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.733979+0000", 2024-09-18T10:14:52.234 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:52.234 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.234 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.234 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:52.235 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.733977+0000", 2024-09-18T10:14:52.235 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:14:52.235 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.235 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.235 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:52.235 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.734010+0000", 2024-09-18T10:14:52.235 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2906e-05 2024-09-18T10:14:52.235 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.235 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.235 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:52.235 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.734043+0000", 2024-09-18T10:14:52.235 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.3300999999999998e-05 2024-09-18T10:14:52.235 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.235 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.236 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:52.236 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.734078+0000", 2024-09-18T10:14:52.236 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.4462000000000003e-05 2024-09-18T10:14:52.236 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.236 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.236 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:52.236 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.734199+0000", 2024-09-18T10:14:52.236 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000120665 2024-09-18T10:14:52.236 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.236 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.236 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:14:52.236 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.734227+0000", 2024-09-18T10:14:52.236 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8408e-05 2024-09-18T10:14:52.236 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.237 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.237 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:14:52.237 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.735463+0000", 2024-09-18T10:14:52.237 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0012361729999999999 2024-09-18T10:14:52.237 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.237 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.237 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:52.237 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.735491+0000", 2024-09-18T10:14:52.237 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.7784e-05 2024-09-18T10:14:52.237 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:52.237 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:52.237 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:52.237 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.238 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.238 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.es1 36/32 ECSubWrite(tid=7, reqid=client.4632.0:148, at_version=36'5, trim_to=0'0, roll_forward_to=36'4))", 2024-09-18T10:14:52.238 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:51.835978+0000", 2024-09-18T10:14:52.238 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 0.31985446499999998, 2024-09-18T10:14:52.238 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001620928, 2024-09-18T10:14:52.238 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:52.238 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:14:52.238 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:52.238 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.238 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:52.238 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.835978+0000", 2024-09-18T10:14:52.238 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:52.238 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.238 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.238 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:52.239 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.835978+0000", 2024-09-18T10:14:52.239 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:52.239 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.239 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.239 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:52.239 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.835977+0000", 2024-09-18T10:14:52.239 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:14:52.239 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.239 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.239 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:52.239 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.836041+0000", 2024-09-18T10:14:52.239 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.4420999999999999e-05 2024-09-18T10:14:52.240 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.240 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.240 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:52.240 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.836081+0000", 2024-09-18T10:14:52.240 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.0252e-05 2024-09-18T10:14:52.240 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.240 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.240 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:52.240 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.836097+0000", 2024-09-18T10:14:52.241 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.5648999999999999e-05 2024-09-18T10:14:52.241 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.241 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.241 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:52.241 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.836231+0000", 2024-09-18T10:14:52.241 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00013359400000000001 2024-09-18T10:14:52.241 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.241 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.241 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:14:52.241 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.836264+0000", 2024-09-18T10:14:52.241 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.3111000000000002e-05 2024-09-18T10:14:52.241 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.241 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.242 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:14:52.242 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.837527+0000", 2024-09-18T10:14:52.242 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0012629410000000001 2024-09-18T10:14:52.242 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.242 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.242 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:52.242 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.837599+0000", 2024-09-18T10:14:52.242 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.2349999999999997e-05 2024-09-18T10:14:52.242 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:52.242 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:52.242 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:52.242 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.242 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.242 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.6s1 36/32 ECSubWrite(tid=19, reqid=client.4632.0:167, at_version=36'11, trim_to=0'0, roll_forward_to=36'10))", 2024-09-18T10:14:52.242 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:51.840327+0000", 2024-09-18T10:14:52.243 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 0.31550524000000002, 2024-09-18T10:14:52.243 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.102147774, 2024-09-18T10:14:52.243 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:52.243 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:14:52.243 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:52.243 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.243 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:52.243 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.840327+0000", 2024-09-18T10:14:52.243 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:52.243 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.243 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.243 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:52.243 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.840327+0000", 2024-09-18T10:14:52.243 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:52.243 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.244 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.244 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:52.244 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.840326+0000", 2024-09-18T10:14:52.244 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:14:52.244 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.244 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.244 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:52.244 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.840534+0000", 2024-09-18T10:14:52.244 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000208168 2024-09-18T10:14:52.244 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.244 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.244 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:52.244 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.840575+0000", 2024-09-18T10:14:52.245 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.0497999999999997e-05 2024-09-18T10:14:52.245 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.245 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.245 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:52.245 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.840593+0000", 2024-09-18T10:14:52.245 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8834999999999998e-05 2024-09-18T10:14:52.245 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.245 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.245 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:52.246 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.940848+0000", 2024-09-18T10:14:52.246 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10025437500000001 2024-09-18T10:14:52.246 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.246 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.246 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:14:52.246 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.940871+0000", 2024-09-18T10:14:52.246 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.3635999999999999e-05 2024-09-18T10:14:52.246 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.246 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.246 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:14:52.246 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.942422+0000", 2024-09-18T10:14:52.246 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0015510809999999999 2024-09-18T10:14:52.246 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.246 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.246 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:52.247 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.942475+0000", 2024-09-18T10:14:52.247 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.2680999999999999e-05 2024-09-18T10:14:52.247 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:52.247 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:52.247 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:52.247 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.247 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.247 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.7s4 36/32 ECSubWrite(tid=43, reqid=client.4632.0:169, at_version=36'12, trim_to=0'0, roll_forward_to=36'11))", 2024-09-18T10:14:52.247 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:51.939335+0000", 2024-09-18T10:14:52.247 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 0.21649732799999999, 2024-09-18T10:14:52.247 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0022859809999999999, 2024-09-18T10:14:52.247 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:52.247 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:14:52.248 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:52.248 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.248 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:52.248 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.939335+0000", 2024-09-18T10:14:52.248 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:52.248 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.248 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.248 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:52.248 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.939335+0000", 2024-09-18T10:14:52.248 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:52.248 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.248 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.248 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:52.249 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.939334+0000", 2024-09-18T10:14:52.249 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:14:52.249 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.249 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.249 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:52.249 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.939603+0000", 2024-09-18T10:14:52.249 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00026838099999999999 2024-09-18T10:14:52.249 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.249 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.249 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:52.249 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.939652+0000", 2024-09-18T10:14:52.249 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9277999999999998e-05 2024-09-18T10:14:52.250 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.250 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.250 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:52.250 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.939671+0000", 2024-09-18T10:14:52.250 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.9352999999999999e-05 2024-09-18T10:14:52.250 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.250 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.250 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:52.250 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.939849+0000", 2024-09-18T10:14:52.250 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000177395 2024-09-18T10:14:52.250 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.250 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.251 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:14:52.251 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.939873+0000", 2024-09-18T10:14:52.251 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4426e-05 2024-09-18T10:14:52.251 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.251 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.251 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:14:52.251 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.941596+0000", 2024-09-18T10:14:52.251 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0017232580000000001 2024-09-18T10:14:52.251 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:52.251 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:52.251 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:52.251 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.941621+0000", 2024-09-18T10:14:52.251 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4898000000000001e-05 2024-09-18T10:14:52.252 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:52.252 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:52.252 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:52.252 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:52.252 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:52.252 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-09-18T10:14:52.253 INFO:tasks.rados.rados.0.smithi189.stdout:55: expect (ObjNum 35 snap 0 seq_num 35) 2024-09-18T10:14:52.254 INFO:tasks.ceph.osd.3.smithi189.stderr:2024-09-18T10:14:52.251+0000 7f2440fc7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22396) UID: 0 2024-09-18T10:14:52.254 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T10:14:52.258 INFO:tasks.rados.rados.0.smithi189.stdout:62: expect (ObjNum 16 snap 0 seq_num 16) 2024-09-18T10:14:52.334 INFO:tasks.rados.rados.0.smithi189.stdout:50: finishing write tid 4 to smithi18925896-50 2024-09-18T10:14:52.334 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 50 v 11 (ObjNum 49 snap 0 seq_num 49) dirty exists 2024-09-18T10:14:52.334 INFO:tasks.rados.rados.0.smithi189.stdout:50: left oid 50 (ObjNum 49 snap 0 seq_num 49) 2024-09-18T10:14:52.355 INFO:tasks.ceph.osd.14.smithi154.stderr:2024-09-18T10:14:52.348+0000 7f75a19e7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 22093) UID: 0 2024-09-18T10:14:52.362 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-18T10:14:52.362 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-18T10:14:52.362 INFO:teuthology.orchestra.run.smithi189.stdout: "num_ops": 0 2024-09-18T10:14:52.363 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-18T10:14:52.372 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T10:14:52.456 INFO:tasks.ceph.osd.14.smithi154.stderr:2024-09-18T10:14:52.452+0000 7f75a19e7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 22093) UID: 0 2024-09-18T10:14:52.516 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-18T10:14:52.516 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-18T10:14:52.516 INFO:teuthology.orchestra.run.smithi189.stdout: "complaint_time": 30, 2024-09-18T10:14:52.516 INFO:teuthology.orchestra.run.smithi189.stdout: "num_blocked_ops": 0 2024-09-18T10:14:52.516 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-18T10:14:52.526 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T10:14:52.538 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15] out_osds: [] dead_osds: [] live_osds: [0, 4, 8, 12, 1, 5, 9, 13, 2, 6, 10, 14, 3, 7, 11, 15] 2024-09-18T10:14:52.538 INFO:tasks.thrashosds.thrasher:choose_action: min_in 8 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T10:14:52.538 INFO:tasks.thrashosds.thrasher:Growing pool unique_pool_0 2024-09-18T10:14:52.539 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T10:14:52.557 INFO:tasks.ceph.osd.3.smithi189.stderr:2024-09-18T10:14:52.555+0000 7f2440fc7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22396) UID: 0 2024-09-18T10:14:52.629 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-18T10:14:52.629 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 20, 2024-09-18T10:14:52.629 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 600, 2024-09-18T10:14:52.629 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [ 2024-09-18T10:14:52.629 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.629 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.2s1 36/32 ECSubWrite(tid=9, reqid=client.4632.0:100, at_version=36'5, trim_to=0'0, roll_forward_to=35'4))", 2024-09-18T10:14:52.629 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:50.301279+0000", 2024-09-18T10:14:52.629 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 2.3244029319999999, 2024-09-18T10:14:52.629 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.104208801, 2024-09-18T10:14:52.629 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:14:52.629 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:14:52.630 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:14:52.630 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.630 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:14:52.630 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.301279+0000", 2024-09-18T10:14:52.630 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:52.630 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.630 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.630 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:14:52.630 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.301279+0000", 2024-09-18T10:14:52.630 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:52.630 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.630 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.630 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:14:52.631 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.301279+0000", 2024-09-18T10:14:52.631 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999995 2024-09-18T10:14:52.631 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.631 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.631 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:14:52.631 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.301498+0000", 2024-09-18T10:14:52.631 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000219259 2024-09-18T10:14:52.631 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.632 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.632 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:14:52.632 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.301512+0000", 2024-09-18T10:14:52.632 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4208e-05 2024-09-18T10:14:52.632 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.632 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.632 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:14:52.632 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.301526+0000", 2024-09-18T10:14:52.632 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4080000000000001e-05 2024-09-18T10:14:52.632 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.632 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.632 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:14:52.633 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.402167+0000", 2024-09-18T10:14:52.633 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100640521 2024-09-18T10:14:52.633 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.633 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.633 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:14:52.633 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.402190+0000", 2024-09-18T10:14:52.633 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3676000000000002e-05 2024-09-18T10:14:52.633 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.633 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.633 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:14:52.633 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.405377+0000", 2024-09-18T10:14:52.633 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0031866899999999998 2024-09-18T10:14:52.633 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.633 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.634 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:14:52.634 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.405488+0000", 2024-09-18T10:14:52.634 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000111013 2024-09-18T10:14:52.634 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:52.634 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:14:52.634 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:52.634 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.634 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.634 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.2s1 36/32 ECSubWrite(tid=10, reqid=client.4632.0:101, at_version=36'6, trim_to=0'0, roll_forward_to=35'4))", 2024-09-18T10:14:52.634 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:50.301535+0000", 2024-09-18T10:14:52.634 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 2.3241469600000002, 2024-09-18T10:14:52.634 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10477768699999999, 2024-09-18T10:14:52.635 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:14:52.635 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:14:52.635 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:14:52.635 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.635 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:14:52.635 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.301535+0000", 2024-09-18T10:14:52.635 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:52.635 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.635 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.635 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:14:52.635 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.301535+0000", 2024-09-18T10:14:52.635 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:52.635 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.636 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.636 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:14:52.636 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.301535+0000", 2024-09-18T10:14:52.636 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999995 2024-09-18T10:14:52.636 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.636 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.636 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:14:52.636 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.301559+0000", 2024-09-18T10:14:52.636 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4090000000000001e-05 2024-09-18T10:14:52.636 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.636 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.636 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:14:52.636 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.301576+0000", 2024-09-18T10:14:52.636 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7399999999999999e-05 2024-09-18T10:14:52.637 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.637 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.637 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:14:52.637 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.301593+0000", 2024-09-18T10:14:52.637 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6605000000000001e-05 2024-09-18T10:14:52.637 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.637 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.637 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:14:52.637 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.404489+0000", 2024-09-18T10:14:52.637 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.102895862 2024-09-18T10:14:52.637 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.637 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.638 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:14:52.638 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.404512+0000", 2024-09-18T10:14:52.638 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.2923000000000001e-05 2024-09-18T10:14:52.638 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.638 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.638 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:14:52.638 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.406197+0000", 2024-09-18T10:14:52.638 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001685303 2024-09-18T10:14:52.638 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.638 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.638 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:14:52.638 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.406313+0000", 2024-09-18T10:14:52.638 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000116047 2024-09-18T10:14:52.638 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:52.639 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:14:52.639 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:52.639 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.639 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.639 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.cs3 36/32 ECSubWrite(tid=12, reqid=client.4632.0:91, at_version=36'5, trim_to=0'0, roll_forward_to=35'4))", 2024-09-18T10:14:52.639 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:50.302210+0000", 2024-09-18T10:14:52.639 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 2.3234720609999999, 2024-09-18T10:14:52.639 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10180185, 2024-09-18T10:14:52.639 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:14:52.639 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:14:52.639 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:14:52.639 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.639 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:14:52.640 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.302210+0000", 2024-09-18T10:14:52.640 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:52.640 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.640 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.640 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:14:52.640 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.302210+0000", 2024-09-18T10:14:52.640 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:52.640 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.640 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.640 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:14:52.640 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.302209+0000", 2024-09-18T10:14:52.640 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:14:52.641 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.641 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.641 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:14:52.641 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.302342+0000", 2024-09-18T10:14:52.641 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000132476 2024-09-18T10:14:52.641 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.641 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.641 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:14:52.641 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.302360+0000", 2024-09-18T10:14:52.641 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.8014000000000001e-05 2024-09-18T10:14:52.642 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.642 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.642 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:14:52.642 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.302372+0000", 2024-09-18T10:14:52.642 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1905e-05 2024-09-18T10:14:52.642 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.642 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.642 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:14:52.642 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.302482+0000", 2024-09-18T10:14:52.642 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000110542 2024-09-18T10:14:52.642 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.642 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.643 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:14:52.643 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.302497+0000", 2024-09-18T10:14:52.643 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5196e-05 2024-09-18T10:14:52.643 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.643 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.643 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:14:52.643 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.403952+0000", 2024-09-18T10:14:52.643 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.101454961 2024-09-18T10:14:52.643 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.643 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.643 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:14:52.643 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.404012+0000", 2024-09-18T10:14:52.643 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9781999999999998e-05 2024-09-18T10:14:52.644 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:52.644 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:14:52.644 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:52.644 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.644 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.644 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.cs3 36/32 ECSubWrite(tid=13, reqid=client.4632.0:92, at_version=36'6, trim_to=0'0, roll_forward_to=35'4))", 2024-09-18T10:14:52.644 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:50.302398+0000", 2024-09-18T10:14:52.644 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 2.3232841190000002, 2024-09-18T10:14:52.644 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.102458168, 2024-09-18T10:14:52.644 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:14:52.644 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:14:52.644 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:14:52.644 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.644 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:14:52.645 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.302398+0000", 2024-09-18T10:14:52.645 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:52.645 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.645 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.645 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:14:52.645 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.302398+0000", 2024-09-18T10:14:52.645 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:52.645 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.645 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.645 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:14:52.645 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.302398+0000", 2024-09-18T10:14:52.645 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999995 2024-09-18T10:14:52.646 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.646 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.646 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:14:52.646 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.302416+0000", 2024-09-18T10:14:52.646 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.8028000000000001e-05 2024-09-18T10:14:52.646 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.646 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.646 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:14:52.646 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.302426+0000", 2024-09-18T10:14:52.646 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0747999999999999e-05 2024-09-18T10:14:52.646 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.646 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.647 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:14:52.647 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.302433+0000", 2024-09-18T10:14:52.647 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.7950000000000002e-06 2024-09-18T10:14:52.647 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.647 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.647 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:14:52.647 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.403441+0000", 2024-09-18T10:14:52.647 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.101007718 2024-09-18T10:14:52.647 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.647 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.647 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:14:52.647 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.403467+0000", 2024-09-18T10:14:52.647 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.6020000000000002e-05 2024-09-18T10:14:52.647 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.648 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.648 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:14:52.648 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.404789+0000", 2024-09-18T10:14:52.648 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001322298 2024-09-18T10:14:52.648 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.648 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.648 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:14:52.648 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.404856+0000", 2024-09-18T10:14:52.648 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.7237999999999996e-05 2024-09-18T10:14:52.648 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:52.648 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:14:52.648 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:52.648 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.649 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.649 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 36/32 ECSubWriteReply(tid=5, last_complete=36'4, committed=1, applied=1))", 2024-09-18T10:14:52.649 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:50.303102+0000", 2024-09-18T10:14:52.649 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 2.3225801829999999, 2024-09-18T10:14:52.649 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20304619600000001, 2024-09-18T10:14:52.649 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:14:52.649 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:14:52.649 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:14:52.649 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.649 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:14:52.649 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.303102+0000", 2024-09-18T10:14:52.649 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:52.650 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.650 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.650 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:14:52.650 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.303102+0000", 2024-09-18T10:14:52.650 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:52.650 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.650 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.650 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:14:52.650 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.303100+0000", 2024-09-18T10:14:52.650 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:14:52.650 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.650 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.651 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:14:52.651 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.303114+0000", 2024-09-18T10:14:52.651 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3866000000000001e-05 2024-09-18T10:14:52.651 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.651 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.651 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:14:52.651 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.303149+0000", 2024-09-18T10:14:52.651 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.5249000000000001e-05 2024-09-18T10:14:52.651 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.651 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.651 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:14:52.651 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.303174+0000", 2024-09-18T10:14:52.651 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4785000000000001e-05 2024-09-18T10:14:52.651 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.652 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.652 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:14:52.652 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.506129+0000", 2024-09-18T10:14:52.652 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20295469199999999 2024-09-18T10:14:52.652 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.652 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.652 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:14:52.652 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.506148+0000", 2024-09-18T10:14:52.652 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9720000000000001e-05 2024-09-18T10:14:52.652 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:52.652 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:14:52.652 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:52.652 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.653 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.653 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 36/32 ECSubWriteReply(tid=5, last_complete=36'4, committed=1, applied=1))", 2024-09-18T10:14:52.653 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:50.305623+0000", 2024-09-18T10:14:52.653 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 2.3200592110000002, 2024-09-18T10:14:52.653 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40085840299999997, 2024-09-18T10:14:52.653 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:14:52.653 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:14:52.653 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:14:52.653 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.653 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:14:52.653 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.305623+0000", 2024-09-18T10:14:52.653 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:52.653 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.653 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.654 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:14:52.654 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.305623+0000", 2024-09-18T10:14:52.654 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:52.654 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.654 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.654 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:14:52.654 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.305621+0000", 2024-09-18T10:14:52.654 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999976 2024-09-18T10:14:52.654 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.654 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.654 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:14:52.654 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.305635+0000", 2024-09-18T10:14:52.654 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4283e-05 2024-09-18T10:14:52.654 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.654 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.655 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:14:52.655 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.305663+0000", 2024-09-18T10:14:52.655 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7475999999999999e-05 2024-09-18T10:14:52.655 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.655 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.655 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:14:52.655 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.305680+0000", 2024-09-18T10:14:52.655 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7609e-05 2024-09-18T10:14:52.655 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.655 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.655 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:14:52.655 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.706451+0000", 2024-09-18T10:14:52.655 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40077027399999998 2024-09-18T10:14:52.655 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.656 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.656 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:14:52.656 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.706481+0000", 2024-09-18T10:14:52.656 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0935000000000001e-05 2024-09-18T10:14:52.656 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:52.656 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:14:52.656 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:52.656 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.656 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.656 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 36/32 ECSubWriteReply(tid=5, last_complete=36'4, committed=1, applied=1))", 2024-09-18T10:14:52.656 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:50.305688+0000", 2024-09-18T10:14:52.656 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 2.3199938470000001, 2024-09-18T10:14:52.656 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40093562900000002, 2024-09-18T10:14:52.656 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:14:52.656 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:14:52.657 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:14:52.657 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.657 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:14:52.657 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.305688+0000", 2024-09-18T10:14:52.657 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:52.657 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.657 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.657 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:14:52.657 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.305688+0000", 2024-09-18T10:14:52.657 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:52.657 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.657 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.657 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:14:52.658 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.305687+0000", 2024-09-18T10:14:52.658 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:14:52.658 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.658 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.658 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:14:52.658 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.305698+0000", 2024-09-18T10:14:52.658 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1763000000000001e-05 2024-09-18T10:14:52.658 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.658 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.658 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:14:52.658 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.305729+0000", 2024-09-18T10:14:52.658 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0502e-05 2024-09-18T10:14:52.658 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.658 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.658 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:14:52.659 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.305741+0000", 2024-09-18T10:14:52.659 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2363e-05 2024-09-18T10:14:52.659 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.659 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.659 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:14:52.659 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.706585+0000", 2024-09-18T10:14:52.659 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.400843265 2024-09-18T10:14:52.659 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.659 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.659 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:14:52.659 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.706624+0000", 2024-09-18T10:14:52.659 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9459000000000003e-05 2024-09-18T10:14:52.660 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:52.660 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:14:52.660 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:52.660 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.660 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.660 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 36/32 ECSubWriteReply(tid=4, last_complete=36'3, committed=1, applied=1))", 2024-09-18T10:14:52.660 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:50.404544+0000", 2024-09-18T10:14:52.660 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 2.2211383319999998, 2024-09-18T10:14:52.660 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.30239659400000002, 2024-09-18T10:14:52.660 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:14:52.660 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:14:52.660 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:14:52.661 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.661 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:14:52.661 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.404544+0000", 2024-09-18T10:14:52.661 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:52.661 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.661 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.661 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:14:52.661 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.404544+0000", 2024-09-18T10:14:52.661 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:52.661 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.661 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.661 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:14:52.662 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.404542+0000", 2024-09-18T10:14:52.662 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:14:52.662 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.662 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.662 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:14:52.662 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.404555+0000", 2024-09-18T10:14:52.662 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3096e-05 2024-09-18T10:14:52.662 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.662 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.662 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:14:52.662 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.404576+0000", 2024-09-18T10:14:52.662 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1165000000000001e-05 2024-09-18T10:14:52.662 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.663 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.663 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:14:52.663 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.404587+0000", 2024-09-18T10:14:52.663 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0549e-05 2024-09-18T10:14:52.663 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.663 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.663 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:14:52.663 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.706657+0000", 2024-09-18T10:14:52.663 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.30206971999999999 2024-09-18T10:14:52.663 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.664 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.664 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:14:52.664 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.706941+0000", 2024-09-18T10:14:52.664 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00028382100000000002 2024-09-18T10:14:52.664 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:52.664 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:14:52.664 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:52.664 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.664 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.664 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 36/32 ECSubWriteReply(tid=5, last_complete=36'4, committed=1, applied=1))", 2024-09-18T10:14:52.664 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:50.406036+0000", 2024-09-18T10:14:52.664 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 2.2196461150000002, 2024-09-18T10:14:52.664 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.30215917399999997, 2024-09-18T10:14:52.665 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:14:52.665 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:14:52.665 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:14:52.665 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.665 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:14:52.665 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.406036+0000", 2024-09-18T10:14:52.665 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:52.665 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.665 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.665 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:14:52.665 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.406036+0000", 2024-09-18T10:14:52.665 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:52.665 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.666 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.666 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:14:52.666 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.406035+0000", 2024-09-18T10:14:52.666 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:14:52.666 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.666 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.666 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:14:52.666 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.406046+0000", 2024-09-18T10:14:52.666 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1704e-05 2024-09-18T10:14:52.667 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.667 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.667 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:14:52.667 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.406065+0000", 2024-09-18T10:14:52.667 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.8448999999999999e-05 2024-09-18T10:14:52.667 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.667 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.667 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:14:52.667 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.406076+0000", 2024-09-18T10:14:52.667 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1286e-05 2024-09-18T10:14:52.667 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.667 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.667 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:14:52.667 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.707005+0000", 2024-09-18T10:14:52.668 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.30092866899999998 2024-09-18T10:14:52.668 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.668 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.668 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:14:52.668 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.708195+0000", 2024-09-18T10:14:52.668 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001190546 2024-09-18T10:14:52.668 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:52.668 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:14:52.668 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:52.668 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.668 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.668 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.cs3 36/32 ECSubWrite(tid=14, reqid=client.4632.0:92, at_version=0'0, trim_to=0'0, roll_forward_to=36'6))", 2024-09-18T10:14:52.668 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:50.504842+0000", 2024-09-18T10:14:52.669 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 2.1208407130000002, 2024-09-18T10:14:52.669 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.202186116, 2024-09-18T10:14:52.669 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:14:52.669 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:14:52.669 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:14:52.669 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.669 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:14:52.669 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.504842+0000", 2024-09-18T10:14:52.669 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:52.669 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.669 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.669 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:14:52.669 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.504842+0000", 2024-09-18T10:14:52.670 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:52.670 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.670 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.670 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:14:52.670 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.504840+0000", 2024-09-18T10:14:52.670 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:14:52.670 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.670 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.670 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:14:52.670 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.504868+0000", 2024-09-18T10:14:52.670 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7809000000000001e-05 2024-09-18T10:14:52.670 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.670 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.671 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:14:52.671 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.504900+0000", 2024-09-18T10:14:52.671 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1921999999999997e-05 2024-09-18T10:14:52.671 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.671 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.671 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:14:52.671 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.605012+0000", 2024-09-18T10:14:52.671 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10011272 2024-09-18T10:14:52.671 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.671 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.671 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:14:52.671 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.605115+0000", 2024-09-18T10:14:52.671 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000103003 2024-09-18T10:14:52.672 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.672 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.672 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:14:52.672 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.605132+0000", 2024-09-18T10:14:52.672 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6359999999999999e-05 2024-09-18T10:14:52.672 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.672 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.672 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:14:52.672 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.706978+0000", 2024-09-18T10:14:52.672 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.101846672 2024-09-18T10:14:52.672 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.672 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.672 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:14:52.673 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.707028+0000", 2024-09-18T10:14:52.673 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9443000000000001e-05 2024-09-18T10:14:52.673 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:52.673 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:14:52.673 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:52.673 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.673 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.673 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.cs3 36/32 ECSubWrite(tid=16, reqid=client.4632.0:106, at_version=36'7, trim_to=0'0, roll_forward_to=36'6))", 2024-09-18T10:14:52.673 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:50.605128+0000", 2024-09-18T10:14:52.673 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 2.0205546019999998, 2024-09-18T10:14:52.673 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.104593478, 2024-09-18T10:14:52.673 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:14:52.673 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:14:52.674 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:14:52.674 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.674 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:14:52.674 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.605128+0000", 2024-09-18T10:14:52.674 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:52.674 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.674 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.674 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:14:52.674 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.605128+0000", 2024-09-18T10:14:52.674 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:52.674 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.674 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.674 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:14:52.674 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.605127+0000", 2024-09-18T10:14:52.675 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999995 2024-09-18T10:14:52.675 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.675 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.675 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:14:52.675 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.605368+0000", 2024-09-18T10:14:52.675 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00024045199999999999 2024-09-18T10:14:52.675 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.675 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.675 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:14:52.675 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.605398+0000", 2024-09-18T10:14:52.675 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0476000000000001e-05 2024-09-18T10:14:52.675 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.676 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.676 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:14:52.676 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.605413+0000", 2024-09-18T10:14:52.676 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4833e-05 2024-09-18T10:14:52.676 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.676 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.676 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:14:52.676 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.705735+0000", 2024-09-18T10:14:52.676 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10032221700000001 2024-09-18T10:14:52.676 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.676 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.676 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:14:52.677 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.705769+0000", 2024-09-18T10:14:52.677 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3516000000000002e-05 2024-09-18T10:14:52.677 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.677 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.677 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:14:52.677 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.709668+0000", 2024-09-18T10:14:52.677 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0038990320000000002 2024-09-18T10:14:52.677 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.677 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.677 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:14:52.677 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.709721+0000", 2024-09-18T10:14:52.677 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3452000000000003e-05 2024-09-18T10:14:52.677 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:52.678 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:14:52.678 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:52.678 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.678 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.678 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.cs3 36/32 ECSubWrite(tid=17, reqid=client.4632.0:107, at_version=36'8, trim_to=0'0, roll_forward_to=36'6))", 2024-09-18T10:14:52.678 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:50.605541+0000", 2024-09-18T10:14:52.678 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 2.020141325, 2024-09-18T10:14:52.678 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.106188687, 2024-09-18T10:14:52.678 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:14:52.678 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:14:52.678 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:14:52.678 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.678 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:14:52.678 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.605541+0000", 2024-09-18T10:14:52.679 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:52.679 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.679 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.679 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:14:52.679 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.605541+0000", 2024-09-18T10:14:52.679 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:52.679 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.679 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.679 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:14:52.679 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.605540+0000", 2024-09-18T10:14:52.679 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:14:52.679 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.679 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.680 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:14:52.680 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.605576+0000", 2024-09-18T10:14:52.680 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.5519000000000003e-05 2024-09-18T10:14:52.680 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.680 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.680 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:14:52.680 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.605600+0000", 2024-09-18T10:14:52.680 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4182e-05 2024-09-18T10:14:52.680 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.680 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.680 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:14:52.680 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.605614+0000", 2024-09-18T10:14:52.680 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3807e-05 2024-09-18T10:14:52.680 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.680 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.681 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:14:52.681 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.706716+0000", 2024-09-18T10:14:52.681 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10110274800000001 2024-09-18T10:14:52.681 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.681 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.681 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:14:52.681 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.706732+0000", 2024-09-18T10:14:52.681 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5254e-05 2024-09-18T10:14:52.681 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.681 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.681 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:14:52.681 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.711692+0000", 2024-09-18T10:14:52.682 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0049602459999999998 2024-09-18T10:14:52.682 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.682 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.682 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:14:52.682 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.711730+0000", 2024-09-18T10:14:52.682 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7803999999999999e-05 2024-09-18T10:14:52.682 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:52.682 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:14:52.682 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:52.682 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.682 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.682 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.es5 36/32 ECSubWrite(tid=0, reqid=client.4632.0:115, at_version=36'1, trim_to=0'0, roll_forward_to=0'0))", 2024-09-18T10:14:52.682 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:50.623805+0000", 2024-09-18T10:14:52.682 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 2.001877323, 2024-09-18T10:14:52.683 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.199174567, 2024-09-18T10:14:52.683 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:14:52.683 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:14:52.683 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:14:52.683 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.683 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:14:52.683 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.623805+0000", 2024-09-18T10:14:52.683 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:52.683 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.683 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.683 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:14:52.683 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.623805+0000", 2024-09-18T10:14:52.683 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:52.684 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.684 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.684 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:14:52.684 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.623804+0000", 2024-09-18T10:14:52.684 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999995 2024-09-18T10:14:52.684 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.684 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.684 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:14:52.684 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.624131+0000", 2024-09-18T10:14:52.684 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00032634999999999999 2024-09-18T10:14:52.684 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.684 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.685 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:14:52.685 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.624160+0000", 2024-09-18T10:14:52.685 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.9269e-05 2024-09-18T10:14:52.685 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.685 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.685 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:14:52.685 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.624173+0000", 2024-09-18T10:14:52.685 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3302000000000001e-05 2024-09-18T10:14:52.685 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.685 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.686 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:14:52.686 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.724319+0000", 2024-09-18T10:14:52.686 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100145894 2024-09-18T10:14:52.686 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.686 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.686 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:14:52.686 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.724326+0000", 2024-09-18T10:14:52.686 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.584e-06 2024-09-18T10:14:52.686 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.686 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.686 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:14:52.686 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.822934+0000", 2024-09-18T10:14:52.686 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.098608285000000004 2024-09-18T10:14:52.687 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.687 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.687 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:14:52.687 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.822980+0000", 2024-09-18T10:14:52.687 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5405999999999999e-05 2024-09-18T10:14:52.687 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:52.687 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:14:52.687 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:52.687 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.687 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.687 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpReadReply(3.0s0 36/32 ECSubReadReply(tid=7, attrs_read=0))", 2024-09-18T10:14:52.687 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:50.709878+0000", 2024-09-18T10:14:52.687 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 1.9158041369999999, 2024-09-18T10:14:52.687 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.19937143800000001, 2024-09-18T10:14:52.688 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:14:52.688 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:14:52.688 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:14:52.688 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.688 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:14:52.688 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.709878+0000", 2024-09-18T10:14:52.688 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:52.688 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.688 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.688 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:14:52.688 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.709878+0000", 2024-09-18T10:14:52.688 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:52.688 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.688 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.689 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:14:52.689 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.709878+0000", 2024-09-18T10:14:52.689 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:52.689 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.689 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.689 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:14:52.689 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.709878+0000", 2024-09-18T10:14:52.689 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:52.689 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.690 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.690 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:14:52.690 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.709950+0000", 2024-09-18T10:14:52.690 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.1475000000000003e-05 2024-09-18T10:14:52.690 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.690 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.690 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:14:52.690 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.709960+0000", 2024-09-18T10:14:52.690 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0309000000000001e-05 2024-09-18T10:14:52.690 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.690 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.690 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:14:52.690 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.909175+0000", 2024-09-18T10:14:52.690 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.19921472300000001 2024-09-18T10:14:52.690 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.691 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.691 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:14:52.691 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.909250+0000", 2024-09-18T10:14:52.691 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.4931000000000005e-05 2024-09-18T10:14:52.691 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:52.691 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:14:52.691 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:52.691 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.691 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.691 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 36/32 ECSubWriteReply(tid=6, last_complete=36'4, committed=1, applied=1))", 2024-09-18T10:14:52.691 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:50.710140+0000", 2024-09-18T10:14:52.691 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 1.915542227, 2024-09-18T10:14:52.691 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20027425800000001, 2024-09-18T10:14:52.691 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:14:52.692 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:14:52.692 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:14:52.692 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.692 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:14:52.692 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.710140+0000", 2024-09-18T10:14:52.692 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:52.692 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.692 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.692 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:14:52.692 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.710140+0000", 2024-09-18T10:14:52.692 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:52.692 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.692 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.693 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:14:52.693 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.710139+0000", 2024-09-18T10:14:52.693 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:14:52.693 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.693 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.693 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:14:52.693 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.710146+0000", 2024-09-18T10:14:52.694 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.4779999999999996e-06 2024-09-18T10:14:52.694 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.694 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.694 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:14:52.694 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.710166+0000", 2024-09-18T10:14:52.694 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0344e-05 2024-09-18T10:14:52.694 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.694 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.694 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:14:52.694 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.810265+0000", 2024-09-18T10:14:52.694 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100099253 2024-09-18T10:14:52.694 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.695 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.695 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:14:52.695 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.910329+0000", 2024-09-18T10:14:52.695 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10006383200000001 2024-09-18T10:14:52.695 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.695 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.695 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:14:52.695 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.910414+0000", 2024-09-18T10:14:52.695 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.5351000000000003e-05 2024-09-18T10:14:52.695 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:52.695 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:14:52.695 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:52.695 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.696 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.696 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpReadReply(3.0s0 36/32 ECSubReadReply(tid=7, attrs_read=0))", 2024-09-18T10:14:52.696 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:50.710609+0000", 2024-09-18T10:14:52.696 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 1.915073077, 2024-09-18T10:14:52.696 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.19872563500000001, 2024-09-18T10:14:52.696 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:14:52.696 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:14:52.696 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:14:52.696 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.696 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:14:52.696 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.710609+0000", 2024-09-18T10:14:52.696 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:52.697 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.697 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.697 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:14:52.697 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.710609+0000", 2024-09-18T10:14:52.697 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:52.697 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.697 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.697 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:14:52.697 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.710608+0000", 2024-09-18T10:14:52.697 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:14:52.697 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.697 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.697 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:14:52.697 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.710624+0000", 2024-09-18T10:14:52.698 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5563999999999999e-05 2024-09-18T10:14:52.698 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.698 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.698 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:14:52.698 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.710642+0000", 2024-09-18T10:14:52.698 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.821e-05 2024-09-18T10:14:52.698 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.698 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.698 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:14:52.698 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.710651+0000", 2024-09-18T10:14:52.698 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.816e-06 2024-09-18T10:14:52.699 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.699 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.699 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:14:52.699 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.909290+0000", 2024-09-18T10:14:52.699 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.19863883399999999 2024-09-18T10:14:52.699 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.699 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.699 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:14:52.699 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.909335+0000", 2024-09-18T10:14:52.699 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5079e-05 2024-09-18T10:14:52.699 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:52.699 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:14:52.699 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:52.699 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.700 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.700 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 36/32 ECSubWriteReply(tid=6, last_complete=36'4, committed=1, applied=1))", 2024-09-18T10:14:52.700 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:50.710837+0000", 2024-09-18T10:14:52.700 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 1.9148452840000001, 2024-09-18T10:14:52.700 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.19854007400000001, 2024-09-18T10:14:52.700 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:14:52.700 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:14:52.700 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:14:52.700 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.700 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:14:52.700 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.710837+0000", 2024-09-18T10:14:52.700 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:52.700 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.701 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.701 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:14:52.701 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.710837+0000", 2024-09-18T10:14:52.701 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:52.701 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.701 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.701 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:14:52.701 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.710835+0000", 2024-09-18T10:14:52.701 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:14:52.701 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.701 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.701 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:14:52.701 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.710850+0000", 2024-09-18T10:14:52.702 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.43e-05 2024-09-18T10:14:52.702 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.702 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.702 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:14:52.702 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.710885+0000", 2024-09-18T10:14:52.702 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4931999999999997e-05 2024-09-18T10:14:52.702 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.702 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.702 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:14:52.702 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.710896+0000", 2024-09-18T10:14:52.702 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0893000000000001e-05 2024-09-18T10:14:52.702 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.702 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.702 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:14:52.703 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.909365+0000", 2024-09-18T10:14:52.703 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.19846915600000001 2024-09-18T10:14:52.703 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.703 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.703 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:14:52.703 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.909377+0000", 2024-09-18T10:14:52.703 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2425999999999999e-05 2024-09-18T10:14:52.703 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:52.703 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:14:52.703 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:52.703 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.703 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.703 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.2s1 36/32 ECSubWrite(tid=11, reqid=client.4632.0:101, at_version=0'0, trim_to=0'0, roll_forward_to=36'6))", 2024-09-18T10:14:52.703 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:50.907380+0000", 2024-09-18T10:14:52.704 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 1.718302481, 2024-09-18T10:14:52.704 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10261464200000001, 2024-09-18T10:14:52.704 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:14:52.704 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:14:52.704 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:14:52.704 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.704 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:14:52.704 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.907380+0000", 2024-09-18T10:14:52.704 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:52.704 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.704 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.704 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:14:52.704 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.907380+0000", 2024-09-18T10:14:52.704 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:52.705 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.705 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.705 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:14:52.705 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.907378+0000", 2024-09-18T10:14:52.705 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:14:52.705 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.705 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.705 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:14:52.705 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.907410+0000", 2024-09-18T10:14:52.705 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1769e-05 2024-09-18T10:14:52.705 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.705 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.705 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:14:52.706 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.907448+0000", 2024-09-18T10:14:52.706 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7904999999999997e-05 2024-09-18T10:14:52.706 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.706 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.706 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:14:52.706 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.907464+0000", 2024-09-18T10:14:52.706 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6022999999999999e-05 2024-09-18T10:14:52.706 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.706 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.706 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:14:52.706 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.007712+0000", 2024-09-18T10:14:52.706 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10024825900000001 2024-09-18T10:14:52.706 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.706 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.707 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:14:52.707 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.007727+0000", 2024-09-18T10:14:52.707 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5206e-05 2024-09-18T10:14:52.707 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.707 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.707 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:14:52.707 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.009952+0000", 2024-09-18T10:14:52.707 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0022251939999999998 2024-09-18T10:14:52.707 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.707 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.707 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:14:52.707 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.009994+0000", 2024-09-18T10:14:52.707 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.2119000000000001e-05 2024-09-18T10:14:52.707 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:52.708 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:14:52.708 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:52.708 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.708 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.708 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.2s1 36/32 ECSubWrite(tid=15, reqid=client.4632.0:125, at_version=0'0, trim_to=0'0, roll_forward_to=36'8))", 2024-09-18T10:14:52.708 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:51.112249+0000", 2024-09-18T10:14:52.708 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 1.5134333259999999, 2024-09-18T10:14:52.708 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.102123384, 2024-09-18T10:14:52.708 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:14:52.708 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:14:52.708 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:14:52.708 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.708 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:14:52.708 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.112249+0000", 2024-09-18T10:14:52.709 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:52.709 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.709 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.709 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:14:52.709 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.112249+0000", 2024-09-18T10:14:52.709 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:52.709 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.709 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.709 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:14:52.709 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.112247+0000", 2024-09-18T10:14:52.709 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:14:52.709 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.710 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.710 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:14:52.710 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.112369+0000", 2024-09-18T10:14:52.710 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000121746 2024-09-18T10:14:52.710 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.710 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.710 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:14:52.710 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.112464+0000", 2024-09-18T10:14:52.710 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.5111000000000005e-05 2024-09-18T10:14:52.710 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.710 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.710 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:14:52.710 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.112484+0000", 2024-09-18T10:14:52.710 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0383000000000001e-05 2024-09-18T10:14:52.711 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.711 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.711 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:14:52.711 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.112754+0000", 2024-09-18T10:14:52.711 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000269611 2024-09-18T10:14:52.711 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.711 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.711 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:14:52.711 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.112791+0000", 2024-09-18T10:14:52.711 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7265000000000003e-05 2024-09-18T10:14:52.711 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.711 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.712 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:14:52.712 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.214299+0000", 2024-09-18T10:14:52.712 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.101507427 2024-09-18T10:14:52.712 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.712 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.712 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:14:52.712 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.214372+0000", 2024-09-18T10:14:52.712 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.3781000000000004e-05 2024-09-18T10:14:52.712 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:52.712 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:14:52.712 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:52.712 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.712 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.712 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.8s3 36/32 ECSubWrite(tid=127, reqid=client.4632.0:154, at_version=36'9, trim_to=0'0, roll_forward_to=36'8))", 2024-09-18T10:14:52.713 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:51.840058+0000", 2024-09-18T10:14:52.713 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.78562434699999995, 2024-09-18T10:14:52.713 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.102237223, 2024-09-18T10:14:52.713 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:14:52.713 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:14:52.713 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:14:52.713 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.713 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:14:52.713 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.840058+0000", 2024-09-18T10:14:52.713 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:52.713 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.713 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.713 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:14:52.713 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.840058+0000", 2024-09-18T10:14:52.714 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:52.714 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.714 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.714 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:14:52.714 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.840057+0000", 2024-09-18T10:14:52.714 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:14:52.714 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.714 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.714 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:14:52.714 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.840226+0000", 2024-09-18T10:14:52.714 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00016943199999999999 2024-09-18T10:14:52.714 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.714 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.714 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:14:52.714 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.840256+0000", 2024-09-18T10:14:52.715 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0309000000000001e-05 2024-09-18T10:14:52.715 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.715 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.715 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:14:52.715 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.940350+0000", 2024-09-18T10:14:52.715 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100093268 2024-09-18T10:14:52.715 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.715 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.715 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:14:52.715 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.940473+0000", 2024-09-18T10:14:52.715 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00012321200000000001 2024-09-18T10:14:52.715 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.715 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.715 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:14:52.715 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.940503+0000", 2024-09-18T10:14:52.716 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0139000000000001e-05 2024-09-18T10:14:52.716 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.716 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.716 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:14:52.716 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.942242+0000", 2024-09-18T10:14:52.716 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001738898 2024-09-18T10:14:52.716 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.716 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.716 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:14:52.716 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.942295+0000", 2024-09-18T10:14:52.716 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3304999999999999e-05 2024-09-18T10:14:52.716 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:52.716 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:14:52.716 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:52.716 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:52.717 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:14:52.717 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-18T10:14:52.719 INFO:teuthology.orchestra.run.smithi029.stderr:nodeep-scrub is unset 2024-09-18T10:14:52.719 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T10:14:52.722 INFO:tasks.rados.rados.0.smithi189.stdout:39: done (10 left) 2024-09-18T10:14:52.722 INFO:tasks.rados.rados.0.smithi189.stdout:49: done (9 left) 2024-09-18T10:14:52.722 INFO:tasks.rados.rados.0.smithi189.stdout:50: done (8 left) 2024-09-18T10:14:52.722 INFO:tasks.rados.rados.0.smithi189.stdout:51: done (7 left) 2024-09-18T10:14:52.722 INFO:tasks.rados.rados.0.smithi189.stdout:52: done (6 left) 2024-09-18T10:14:52.722 INFO:tasks.rados.rados.0.smithi189.stdout:55: done (5 left) 2024-09-18T10:14:52.722 INFO:tasks.rados.rados.0.smithi189.stdout:57: done (4 left) 2024-09-18T10:14:52.722 INFO:tasks.rados.rados.0.smithi189.stdout:58: done (3 left) 2024-09-18T10:14:52.722 INFO:tasks.rados.rados.0.smithi189.stdout:60: done (2 left) 2024-09-18T10:14:52.723 INFO:tasks.rados.rados.0.smithi189.stdout:62: done (1 left) 2024-09-18T10:14:52.723 INFO:tasks.rados.rados.0.smithi189.stdout:63: done (0 left) 2024-09-18T10:14:52.723 INFO:tasks.rados.rados.0.smithi189.stdout:64: copy_from oid 15 from oid 2 current snap is 1 2024-09-18T10:14:52.723 INFO:tasks.rados.rados.0.smithi189.stdout:65: read oid 30 snap -1 2024-09-18T10:14:52.723 INFO:tasks.rados.rados.0.smithi189.stdout:65: expect (ObjNum 29 snap 0 seq_num 29) 2024-09-18T10:14:52.723 INFO:tasks.rados.rados.0.smithi189.stdout:append oid 23 current snap is 1 2024-09-18T10:14:52.723 INFO:tasks.rados.rados.0.smithi189.stdout:66: seq_num 53 ranges {376832=524288,901120=376832} 2024-09-18T10:14:52.723 INFO:tasks.rados.rados.0.smithi189.stdout:66: writing smithi18925896-23 from 376832 to 901120 tid 1 2024-09-18T10:14:52.723 INFO:tasks.rados.rados.0.smithi189.stdout:66: writing smithi18925896-23 from 901120 to 1277952 tid 2 2024-09-18T10:14:52.723 INFO:tasks.rados.rados.0.smithi189.stdout:67: rollback oid 10 current snap is 1 2024-09-18T10:14:52.723 INFO:tasks.rados.rados.0.smithi189.stdout:rollback oid 10 to 0 2024-09-18T10:14:52.723 INFO:tasks.rados.rados.0.smithi189.stdout:68: setattr oid 32 current snap is 1 2024-09-18T10:14:52.723 INFO:tasks.rados.rados.0.smithi189.stdout:69: copy_from oid 7 from oid 17 current snap is 1 2024-09-18T10:14:52.723 INFO:tasks.rados.rados.0.smithi189.stdout:70: read oid 20 snap 0 2024-09-18T10:14:52.724 INFO:tasks.rados.rados.0.smithi189.stdout:70: expect (ObjNum 19 snap 0 seq_num 19) 2024-09-18T10:14:52.724 INFO:tasks.rados.rados.0.smithi189.stdout:71: copy_from oid 24 from oid 29 current snap is 1 2024-09-18T10:14:52.724 INFO:tasks.rados.rados.0.smithi189.stdout:72: rollback oid 6 current snap is 1 2024-09-18T10:14:52.724 INFO:tasks.rados.rados.0.smithi189.stdout:rollback oid 6 to 0 2024-09-18T10:14:52.724 INFO:tasks.rados.rados.0.smithi189.stdout:73: read oid 13 snap 0 2024-09-18T10:14:52.724 INFO:tasks.rados.rados.0.smithi189.stdout:73: expect (ObjNum 12 snap 0 seq_num 12) 2024-09-18T10:14:52.724 INFO:tasks.rados.rados.0.smithi189.stdout:74: rmattr oid 4 current snap is 1 2024-09-18T10:14:52.724 INFO:tasks.rados.rados.0.smithi189.stdout:74: done (10 left) 2024-09-18T10:14:52.724 INFO:tasks.rados.rados.0.smithi189.stdout:75: copy_from oid 48 from oid 1 current snap is 1 2024-09-18T10:14:52.724 INFO:tasks.rados.rados.0.smithi189.stdout:76: read oid 27 snap -1 2024-09-18T10:14:52.724 INFO:tasks.rados.rados.0.smithi189.stdout:76: expect (ObjNum 26 snap 0 seq_num 26) 2024-09-18T10:14:52.724 INFO:tasks.rados.rados.0.smithi189.stdout:77: copy_from oid 28 from oid 34 current snap is 1 2024-09-18T10:14:52.724 INFO:tasks.rados.rados.0.smithi189.stdout:78: read oid 43 snap -1 2024-09-18T10:14:52.725 INFO:tasks.rados.rados.0.smithi189.stdout:78: expect (ObjNum 42 snap 0 seq_num 42) 2024-09-18T10:14:52.725 INFO:tasks.rados.rados.0.smithi189.stdout:79: setattr oid 14 current snap is 1 2024-09-18T10:14:52.725 INFO:tasks.rados.rados.0.smithi189.stdout:80: snap_create 2024-09-18T10:14:52.725 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-18T10:14:52.714+0000 7f578c52f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22134) UID: 0 2024-09-18T10:14:52.799 INFO:tasks.rados.rados.0.smithi189.stdout:67: finishing rollback tid 0 to smithi18925896-10 2024-09-18T10:14:52.819 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-18T10:14:52.814+0000 7f14c6a1a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 22138) UID: 0 2024-09-18T10:14:52.833 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-18T10:14:52.833 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [ 2024-09-18T10:14:52.833 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.833 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:187 3.9s0 3:93224e57:::smithi18925896-10:head [append 0~0] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:14:52.833 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:52.793538+0000", 2024-09-18T10:14:52.834 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.039050211000000001, 2024-09-18T10:14:52.834 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.03908068, 2024-09-18T10:14:52.834 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:14:52.834 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:14:52.834 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:14:52.834 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:14:52.834 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:14:52.834 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 187 2024-09-18T10:14:52.834 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.834 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:14:52.834 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.834 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:14:52.834 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.793538+0000", 2024-09-18T10:14:52.834 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:52.835 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.835 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.835 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:14:52.835 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.793538+0000", 2024-09-18T10:14:52.835 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:52.835 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.835 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.835 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:14:52.835 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.793536+0000", 2024-09-18T10:14:52.835 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:14:52.835 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.835 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.835 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:14:52.836 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.793548+0000", 2024-09-18T10:14:52.836 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2106000000000001e-05 2024-09-18T10:14:52.836 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.836 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.836 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:14:52.836 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.793572+0000", 2024-09-18T10:14:52.836 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4253e-05 2024-09-18T10:14:52.836 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.836 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.836 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:14:52.836 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.793582+0000", 2024-09-18T10:14:52.836 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0135999999999999e-05 2024-09-18T10:14:52.836 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.837 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.837 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:14:52.837 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.803156+0000", 2024-09-18T10:14:52.837 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0095734280000000001 2024-09-18T10:14:52.837 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.837 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.837 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:14:52.837 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.803417+0000", 2024-09-18T10:14:52.837 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00026132199999999998 2024-09-18T10:14:52.837 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.837 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.837 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:14:52.837 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.804492+0000", 2024-09-18T10:14:52.838 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0010750530000000001 2024-09-18T10:14:52.838 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:52.838 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:14:52.838 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:52.838 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.838 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.838 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:202 3.9s0 3.afff8449 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:14:52.838 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:52.793605+0000", 2024-09-18T10:14:52.838 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.038983139999999999, 2024-09-18T10:14:52.838 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.039058586999999999, 2024-09-18T10:14:52.838 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:14:52.838 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "queued for pg", 2024-09-18T10:14:52.838 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:14:52.839 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:14:52.839 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:14:52.839 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 202 2024-09-18T10:14:52.839 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.839 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:14:52.839 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.839 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:14:52.839 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.793605+0000", 2024-09-18T10:14:52.839 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:52.839 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.839 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.839 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:14:52.839 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.793605+0000", 2024-09-18T10:14:52.840 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:52.840 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.840 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.840 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:14:52.840 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.793604+0000", 2024-09-18T10:14:52.840 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:14:52.840 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.840 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.840 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:14:52.840 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.793613+0000", 2024-09-18T10:14:52.840 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.4360000000000008e-06 2024-09-18T10:14:52.840 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.840 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.841 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:14:52.841 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.793628+0000", 2024-09-18T10:14:52.841 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4953000000000001e-05 2024-09-18T10:14:52.841 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.841 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.841 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:14:52.841 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.793634+0000", 2024-09-18T10:14:52.841 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.0170000000000002e-06 2024-09-18T10:14:52.841 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:52.841 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:14:52.841 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:52.841 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.841 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.842 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:203 3.9s0 3.afff8449 (undecoded) ondisk+read+rwordered+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:14:52.842 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:52.793641+0000", 2024-09-18T10:14:52.842 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.038946755, 2024-09-18T10:14:52.842 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.039052077999999997, 2024-09-18T10:14:52.842 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:14:52.842 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "queued for pg", 2024-09-18T10:14:52.842 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:14:52.842 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:14:52.842 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:14:52.843 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 203 2024-09-18T10:14:52.843 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.843 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:14:52.843 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.843 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:14:52.843 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.793641+0000", 2024-09-18T10:14:52.843 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:52.843 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.843 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.843 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:14:52.843 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.793641+0000", 2024-09-18T10:14:52.843 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:52.843 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.844 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.844 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:14:52.844 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.793641+0000", 2024-09-18T10:14:52.844 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:14:52.844 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.844 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.844 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:14:52.844 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.793644+0000", 2024-09-18T10:14:52.844 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7170000000000002e-06 2024-09-18T10:14:52.844 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.844 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.844 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:14:52.844 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.793658+0000", 2024-09-18T10:14:52.845 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3353000000000001e-05 2024-09-18T10:14:52.845 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.845 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.845 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:14:52.845 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.793663+0000", 2024-09-18T10:14:52.845 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.7729999999999996e-06 2024-09-18T10:14:52.845 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:52.845 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:14:52.845 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:52.845 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.845 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.845 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(osd.10.22:1 3.9s0 3.d35743f9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:14:52.845 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:52.796459+0000", 2024-09-18T10:14:52.846 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.036129095999999999, 2024-09-18T10:14:52.846 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.036262943999999998, 2024-09-18T10:14:52.846 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:14:52.846 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "queued for pg", 2024-09-18T10:14:52.846 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:14:52.846 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.846 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:14:52.846 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.796459+0000", 2024-09-18T10:14:52.846 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:52.846 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.846 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.846 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:14:52.846 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.796459+0000", 2024-09-18T10:14:52.846 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:52.847 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.847 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.847 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:14:52.847 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.796457+0000", 2024-09-18T10:14:52.847 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:14:52.847 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.847 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.847 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:14:52.847 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.796468+0000", 2024-09-18T10:14:52.847 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0835999999999999e-05 2024-09-18T10:14:52.847 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.847 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.847 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:14:52.847 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.796488+0000", 2024-09-18T10:14:52.848 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0068000000000002e-05 2024-09-18T10:14:52.848 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.848 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.848 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:14:52.848 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.796501+0000", 2024-09-18T10:14:52.848 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2994e-05 2024-09-18T10:14:52.848 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:52.848 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:14:52.848 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:52.848 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.848 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.848 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.9s0 37/32 ECSubWriteReply(tid=26, last_complete=37'15, committed=1, applied=1))", 2024-09-18T10:14:52.848 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:52.804194+0000", 2024-09-18T10:14:52.849 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.028394072999999999, 2024-09-18T10:14:52.849 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.028553719000000002, 2024-09-18T10:14:52.849 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:14:52.849 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "queued for pg", 2024-09-18T10:14:52.849 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:14:52.849 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.849 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:14:52.849 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.804194+0000", 2024-09-18T10:14:52.849 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:52.849 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.849 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.849 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:14:52.850 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.804194+0000", 2024-09-18T10:14:52.850 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:52.850 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.850 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.850 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:14:52.850 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.804193+0000", 2024-09-18T10:14:52.850 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:14:52.850 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.850 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.850 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:14:52.850 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.804200+0000", 2024-09-18T10:14:52.850 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.5300000000000002e-06 2024-09-18T10:14:52.850 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.850 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.851 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:14:52.851 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.804218+0000", 2024-09-18T10:14:52.851 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.8576999999999998e-05 2024-09-18T10:14:52.851 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.851 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.851 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:14:52.851 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.804228+0000", 2024-09-18T10:14:52.851 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.2229999999999996e-06 2024-09-18T10:14:52.851 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:52.851 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:14:52.851 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:52.851 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.852 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.852 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.9s0 37/32 ECSubWriteReply(tid=27, last_complete=37'16, committed=1, applied=1))", 2024-09-18T10:14:52.852 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:52.804371+0000", 2024-09-18T10:14:52.852 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.028217269, 2024-09-18T10:14:52.852 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.028401640999999998, 2024-09-18T10:14:52.852 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:14:52.852 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "queued for pg", 2024-09-18T10:14:52.852 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:14:52.852 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.852 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:14:52.852 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.804371+0000", 2024-09-18T10:14:52.852 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:52.852 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.852 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.853 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:14:52.853 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.804371+0000", 2024-09-18T10:14:52.853 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:52.853 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.853 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.853 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:14:52.853 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.804370+0000", 2024-09-18T10:14:52.853 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:14:52.853 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.853 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.853 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:14:52.853 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.804379+0000", 2024-09-18T10:14:52.854 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.8199999999999992e-06 2024-09-18T10:14:52.854 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.854 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.854 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:14:52.854 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.804399+0000", 2024-09-18T10:14:52.854 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9236999999999999e-05 2024-09-18T10:14:52.854 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.854 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.854 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:14:52.854 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.804409+0000", 2024-09-18T10:14:52.854 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0438999999999999e-05 2024-09-18T10:14:52.854 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:52.854 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:14:52.854 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:52.855 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.855 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.855 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.9s0 37/32 ECSubWriteReply(tid=27, last_complete=37'16, committed=1, applied=1))", 2024-09-18T10:14:52.855 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:52.806078+0000", 2024-09-18T10:14:52.855 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.026510446, 2024-09-18T10:14:52.855 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.026718872000000001, 2024-09-18T10:14:52.855 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:14:52.855 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "no flag points reached", 2024-09-18T10:14:52.855 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:14:52.855 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.855 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:14:52.855 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.806078+0000", 2024-09-18T10:14:52.855 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:52.855 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.856 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.856 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:14:52.856 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.806078+0000", 2024-09-18T10:14:52.856 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:52.856 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.856 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.856 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:14:52.856 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.806077+0000", 2024-09-18T10:14:52.856 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:14:52.856 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.856 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.856 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:14:52.856 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.806082+0000", 2024-09-18T10:14:52.856 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3199999999999999e-06 2024-09-18T10:14:52.857 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.857 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.857 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:14:52.857 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.806097+0000", 2024-09-18T10:14:52.857 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4786999999999999e-05 2024-09-18T10:14:52.857 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:52.857 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:14:52.857 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:52.857 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.857 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.857 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.9s0 37/32 ECSubWriteReply(tid=27, last_complete=37'16, committed=1, applied=1))", 2024-09-18T10:14:52.857 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:52.806185+0000", 2024-09-18T10:14:52.857 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.026402752000000002, 2024-09-18T10:14:52.858 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.026633126999999999, 2024-09-18T10:14:52.858 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:14:52.858 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "queued for pg", 2024-09-18T10:14:52.858 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:14:52.858 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.858 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:14:52.858 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.806185+0000", 2024-09-18T10:14:52.858 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:52.858 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.858 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.858 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:14:52.858 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.806185+0000", 2024-09-18T10:14:52.858 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:52.858 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.858 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.859 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:14:52.859 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.806184+0000", 2024-09-18T10:14:52.859 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:14:52.859 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.859 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.859 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:14:52.859 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.806192+0000", 2024-09-18T10:14:52.859 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.4050000000000003e-06 2024-09-18T10:14:52.859 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.859 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.859 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:14:52.859 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.806221+0000", 2024-09-18T10:14:52.859 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.9671000000000001e-05 2024-09-18T10:14:52.859 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.860 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.860 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:14:52.860 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.806239+0000", 2024-09-18T10:14:52.860 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7462e-05 2024-09-18T10:14:52.860 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:52.860 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:14:52.860 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:52.860 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.860 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.860 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.9s0 37/32 ECSubWriteReply(tid=27, last_complete=37'16, committed=1, applied=1))", 2024-09-18T10:14:52.860 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:52.806972+0000", 2024-09-18T10:14:52.860 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.025616032, 2024-09-18T10:14:52.860 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.025870249000000001, 2024-09-18T10:14:52.860 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:14:52.861 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "queued for pg", 2024-09-18T10:14:52.861 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:14:52.861 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.861 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:14:52.861 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.806972+0000", 2024-09-18T10:14:52.861 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:52.861 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.861 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.861 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:14:52.861 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.806972+0000", 2024-09-18T10:14:52.861 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:52.861 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.861 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.861 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:14:52.861 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.806971+0000", 2024-09-18T10:14:52.862 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:14:52.862 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.862 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.862 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:14:52.862 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.806979+0000", 2024-09-18T10:14:52.862 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.1200000000000002e-06 2024-09-18T10:14:52.862 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.862 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.862 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:14:52.862 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.806996+0000", 2024-09-18T10:14:52.862 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6772000000000001e-05 2024-09-18T10:14:52.862 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.862 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.862 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:14:52.862 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.807006+0000", 2024-09-18T10:14:52.863 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.7210000000000006e-06 2024-09-18T10:14:52.863 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:52.863 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:14:52.863 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:52.863 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.863 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.863 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.9s0 37/32 ECSubWriteReply(tid=28, last_complete=37'17, committed=1, applied=1))", 2024-09-18T10:14:52.863 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:52.807439+0000", 2024-09-18T10:14:52.863 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.025148839999999999, 2024-09-18T10:14:52.863 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.025428203, 2024-09-18T10:14:52.863 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:14:52.863 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "queued for pg", 2024-09-18T10:14:52.863 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:14:52.863 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.864 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:14:52.864 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.807439+0000", 2024-09-18T10:14:52.864 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:52.864 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.864 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.864 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:14:52.864 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.807439+0000", 2024-09-18T10:14:52.864 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:52.864 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.864 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.864 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:14:52.864 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.807438+0000", 2024-09-18T10:14:52.864 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:14:52.865 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.865 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.865 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:14:52.865 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.807446+0000", 2024-09-18T10:14:52.865 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.6060000000000003e-06 2024-09-18T10:14:52.865 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.865 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.865 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:14:52.865 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.807464+0000", 2024-09-18T10:14:52.865 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7207e-05 2024-09-18T10:14:52.865 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.865 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.865 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:14:52.866 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.807472+0000", 2024-09-18T10:14:52.866 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.3890000000000005e-06 2024-09-18T10:14:52.866 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:52.866 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:14:52.866 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:52.866 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.866 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.866 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.9s0 37/32 ECSubWriteReply(tid=28, last_complete=37'17, committed=1, applied=1))", 2024-09-18T10:14:52.866 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:52.809334+0000", 2024-09-18T10:14:52.866 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.023253995999999999, 2024-09-18T10:14:52.866 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.023556963, 2024-09-18T10:14:52.866 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:14:52.866 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "queued for pg", 2024-09-18T10:14:52.867 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:14:52.867 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.867 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:14:52.867 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.809334+0000", 2024-09-18T10:14:52.867 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:52.867 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.867 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.867 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:14:52.867 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.809334+0000", 2024-09-18T10:14:52.867 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:52.867 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.867 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.868 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:14:52.868 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.809333+0000", 2024-09-18T10:14:52.868 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:14:52.868 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.868 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.868 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:14:52.868 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.809340+0000", 2024-09-18T10:14:52.868 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.8449999999999997e-06 2024-09-18T10:14:52.868 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.868 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.868 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:14:52.868 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.809356+0000", 2024-09-18T10:14:52.868 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5817000000000001e-05 2024-09-18T10:14:52.869 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.869 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.869 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:14:52.869 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.809363+0000", 2024-09-18T10:14:52.869 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.0219999999999999e-06 2024-09-18T10:14:52.869 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:52.869 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:14:52.869 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:52.869 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.869 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.869 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:186 3.9s0 3:93224e57:::smithi18925896-10:head [rollback 2] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:14:52.869 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:52.693370+0000", 2024-09-18T10:14:52.869 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.13921772399999999, 2024-09-18T10:14:52.870 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.13954836700000001, 2024-09-18T10:14:52.870 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:14:52.870 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:14:52.870 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:14:52.870 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:14:52.870 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:14:52.870 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 186 2024-09-18T10:14:52.870 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.870 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:14:52.870 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.870 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:14:52.870 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.693370+0000", 2024-09-18T10:14:52.870 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:52.871 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.871 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.871 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:14:52.871 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.693370+0000", 2024-09-18T10:14:52.871 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:52.871 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.871 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.871 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:14:52.871 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.693370+0000", 2024-09-18T10:14:52.871 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999995 2024-09-18T10:14:52.871 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.871 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.871 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:14:52.872 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.693373+0000", 2024-09-18T10:14:52.872 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.712e-06 2024-09-18T10:14:52.872 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.872 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.872 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:14:52.872 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.693379+0000", 2024-09-18T10:14:52.872 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9320000000000001e-06 2024-09-18T10:14:52.872 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.872 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.872 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:14:52.872 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.793509+0000", 2024-09-18T10:14:52.872 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100130331 2024-09-18T10:14:52.872 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.873 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.873 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:14:52.873 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.800561+0000", 2024-09-18T10:14:52.873 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0070525559999999998 2024-09-18T10:14:52.873 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.873 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.873 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:14:52.873 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.800733+0000", 2024-09-18T10:14:52.873 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00017114100000000001 2024-09-18T10:14:52.873 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:52.873 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:52.873 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:14:52.873 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.802155+0000", 2024-09-18T10:14:52.873 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0014220420000000001 2024-09-18T10:14:52.874 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:52.874 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:14:52.874 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:52.874 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:52.874 INFO:teuthology.orchestra.run.smithi189.stdout: ], 2024-09-18T10:14:52.874 INFO:teuthology.orchestra.run.smithi189.stdout: "num_ops": 12 2024-09-18T10:14:52.874 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-18T10:14:52.877 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T10:14:52.878 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-18T10:14:52.879 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-09-18T10:14:52.879 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":33,"stamp":"2024-09-18T10:14:52.675793+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":7782963,"num_objects":32,"num_object_clones":0,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":133,"num_read_kb":92,"num_write":203,"num_write_kb":9698,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":171,"ondisk_log_size":171,"up":130,"acting":130,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":119,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":16,"kb":1509949440,"kb_used":4775488,"kb_used_data":17736,"kb_used_omap":26,"kb_used_meta":4757477,"kb_avail":1505173952,"statfs":{"total":1546188226560,"available":1541298126848,"internally_reserved":0,"allocated":18161664,"data_stored":14536149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":27264,"internal_metadata":4871656832},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":7192576,"num_objects":28,"num_object_clones":0,"num_object_copies":196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":47,"num_read_kb":19,"num_write":56,"num_write_kb":7024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.479239"},"pg_stats":[{"pgid":"3.f","version":"36'2","reported_seq":18,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-18T10:14:50.699189+0000","last_change":"2024-09-18T10:14:46.904979+0000","last_active":"2024-09-18T10:14:50.699189+0000","last_peered":"2024-09-18T10:14:50.699189+0000","last_clean":"2024-09-18T10:14:50.699189+0000","last_became_active":"2024-09-18T10:14:46.803296+0000","last_became_peered":"2024-09-18T10:14:46.803296+0000","last_unstale":"2024-09-18T10:14:50.699189+0000","last_undegraded":"2024-09-18T10:14:50.699189+0000","last_fullsized":"2024-09-18T10:14:50.699189+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:14:45.576168+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:11.624416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":573440,"num_objects":1,"num_object_clones":0,"num_object_copies":7,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,3,15,14,2,11,0],"acting":[13,3,15,14,2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-18T10:14:47.776324+0000","last_change":"2024-09-18T10:14:46.803615+0000","last_active":"2024-09-18T10:14:47.776324+0000","last_peered":"2024-09-18T10:14:47.776324+0000","last_clean":"2024-09-18T10:14:47.776324+0000","last_became_active":"2024-09-18T10:14:46.801571+0000","last_became_peered":"2024-09-18T10:14:46.801571+0000","last_unstale":"2024-09-18T10:14:47.776324+0000","last_undegraded":"2024-09-18T10:14:47.776324+0000","last_fullsized":"2024-09-18T10:14:47.776324+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:14:45.576168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:15:59.433777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,14,6,13,8,3,12],"acting":[11,14,6,13,8,3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.d","version":"35'2","reported_seq":19,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-18T10:14:49.679077+0000","last_change":"2024-09-18T10:14:46.896050+0000","last_active":"2024-09-18T10:14:49.679077+0000","last_peered":"2024-09-18T10:14:49.679077+0000","last_clean":"2024-09-18T10:14:49.679077+0000","last_became_active":"2024-09-18T10:14:46.793893+0000","last_became_peered":"2024-09-18T10:14:46.793893+0000","last_unstale":"2024-09-18T10:14:49.679077+0000","last_undegraded":"2024-09-18T10:14:49.679077+0000","last_fullsized":"2024-09-18T10:14:49.679077+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:14:45.576168+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:13.756262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":245760,"num_objects":1,"num_object_clones":0,"num_object_copies":7,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":1,"num_write":2,"num_write_kb":240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,1,11,15,9,8,4],"acting":[13,1,11,15,9,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"3.c","version":"35'4","reported_seq":21,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-18T10:14:50.092756+0000","last_change":"2024-09-18T10:14:46.997639+0000","last_active":"2024-09-18T10:14:50.092756+0000","last_peered":"2024-09-18T10:14:50.092756+0000","last_clean":"2024-09-18T10:14:50.092756+0000","last_became_active":"2024-09-18T10:14:46.795995+0000","last_became_peered":"2024-09-18T10:14:46.795995+0000","last_unstale":"2024-09-18T10:14:50.092756+0000","last_undegraded":"2024-09-18T10:14:50.092756+0000","last_fullsized":"2024-09-18T10:14:50.092756+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:14:45.576168+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:14.450533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":475136,"num_objects":2,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":1,"num_write":4,"num_write_kb":464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,13,3,7,12,14],"acting":[5,8,13,3,7,12,14],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.b","version":"35'2","reported_seq":19,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-18T10:14:49.274891+0000","last_change":"2024-09-18T10:14:46.905266+0000","last_active":"2024-09-18T10:14:49.274891+0000","last_peered":"2024-09-18T10:14:49.274891+0000","last_clean":"2024-09-18T10:14:49.274891+0000","last_became_active":"2024-09-18T10:14:46.802834+0000","last_became_peered":"2024-09-18T10:14:46.802834+0000","last_unstale":"2024-09-18T10:14:49.274891+0000","last_undegraded":"2024-09-18T10:14:49.274891+0000","last_fullsized":"2024-09-18T10:14:49.274891+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:14:45.576168+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:04.590511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":49152,"num_objects":1,"num_object_clones":0,"num_object_copies":7,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":1,"num_write":2,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,10,12,11,1,14],"acting":[4,0,10,12,11,1,14],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"36'6","reported_seq":25,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-18T10:14:49.990207+0000","last_change":"2024-09-18T10:14:46.767346+0000","last_active":"2024-09-18T10:14:49.990207+0000","last_peered":"2024-09-18T10:14:49.990207+0000","last_clean":"2024-09-18T10:14:49.990207+0000","last_became_active":"2024-09-18T10:14:46.765703+0000","last_became_peered":"2024-09-18T10:14:46.765703+0000","last_unstale":"2024-09-18T10:14:49.990207+0000","last_undegraded":"2024-09-18T10:14:49.990207+0000","last_fullsized":"2024-09-18T10:14:49.990207+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:14:45.576168+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:08.721290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":819200,"num_objects":3,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":6,"num_read_kb":3,"num_write":6,"num_write_kb":800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,1,6,5,3,15,4],"acting":[13,1,6,5,3,15,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"3.9","version":"35'6","reported_seq":25,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-18T10:14:49.901288+0000","last_change":"2024-09-18T10:14:46.903863+0000","last_active":"2024-09-18T10:14:49.901288+0000","last_peered":"2024-09-18T10:14:49.901288+0000","last_clean":"2024-09-18T10:14:49.901288+0000","last_became_active":"2024-09-18T10:14:46.903238+0000","last_became_peered":"2024-09-18T10:14:46.903238+0000","last_unstale":"2024-09-18T10:14:49.901288+0000","last_undegraded":"2024-09-18T10:14:49.901288+0000","last_fullsized":"2024-09-18T10:14:49.901288+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:14:45.576168+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:15:49.701811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622592,"num_objects":3,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":6,"num_read_kb":3,"num_write":6,"num_write_kb":608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,10,11,5,8,3],"acting":[7,2,10,11,5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"35'4","reported_seq":22,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-18T10:14:49.710871+0000","last_change":"2024-09-18T10:14:46.907794+0000","last_active":"2024-09-18T10:14:49.710871+0000","last_peered":"2024-09-18T10:14:49.710871+0000","last_clean":"2024-09-18T10:14:49.710871+0000","last_became_active":"2024-09-18T10:14:46.905795+0000","last_became_peered":"2024-09-18T10:14:46.905795+0000","last_unstale":"2024-09-18T10:14:49.710871+0000","last_undegraded":"2024-09-18T10:14:49.710871+0000","last_fullsized":"2024-09-18T10:14:49.710871+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:14:45.576168+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:15:48.118890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262144,"num_objects":2,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":4,"num_write_kb":256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,11,3,15,1,7],"acting":[6,0,11,3,15,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"29'112","reported_seq":156,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-18T10:14:48.871006+0000","last_change":"2024-09-18T10:14:32.509942+0000","last_active":"2024-09-18T10:14:48.871006+0000","last_peered":"2024-09-18T10:14:48.871006+0000","last_clean":"2024-09-18T10:14:48.871006+0000","last_became_active":"2024-09-18T10:14:32.508037+0000","last_became_peered":"2024-09-18T10:14:32.508037+0000","last_unstale":"2024-09-18T10:14:48.871006+0000","last_undegraded":"2024-09-18T10:14:48.871006+0000","last_fullsized":"2024-09-18T10:14:48.871006+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:31.267741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:31.267741+0000","last_clean_scrub_stamp":"2024-09-18T10:14:31.267741+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:15:59.262610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,15],"acting":[6,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.3","version":"27'1","reported_seq":23,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-18T10:14:48.667022+0000","last_change":"2024-09-18T10:14:37.801252+0000","last_active":"2024-09-18T10:14:48.667022+0000","last_peered":"2024-09-18T10:14:48.667022+0000","last_clean":"2024-09-18T10:14:48.667022+0000","last_became_active":"2024-09-18T10:14:35.487746+0000","last_became_peered":"2024-09-18T10:14:35.487746+0000","last_unstale":"2024-09-18T10:14:48.667022+0000","last_undegraded":"2024-09-18T10:14:48.667022+0000","last_fullsized":"2024-09-18T10:14:48.667022+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:14:34.288596+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:03.161727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.10072764100000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,5],"acting":[14,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"3.2","version":"35'4","reported_seq":21,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-18T10:14:49.896514+0000","last_change":"2024-09-18T10:14:46.868233+0000","last_active":"2024-09-18T10:14:49.896514+0000","last_peered":"2024-09-18T10:14:49.896514+0000","last_clean":"2024-09-18T10:14:49.896514+0000","last_became_active":"2024-09-18T10:14:46.766019+0000","last_became_peered":"2024-09-18T10:14:46.766019+0000","last_unstale":"2024-09-18T10:14:49.896514+0000","last_undegraded":"2024-09-18T10:14:49.896514+0000","last_fullsized":"2024-09-18T10:14:49.896514+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:14:45.576168+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:15:48.125031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":442368,"num_objects":2,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":1,"num_write":4,"num_write_kb":432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,3,13,1,5,10,7],"acting":[14,3,13,1,5,10,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":21,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-18T10:14:47.689156+0000","last_change":"2024-09-18T10:14:37.600953+0000","last_active":"2024-09-18T10:14:47.689156+0000","last_peered":"2024-09-18T10:14:47.689156+0000","last_clean":"2024-09-18T10:14:47.689156+0000","last_became_active":"2024-09-18T10:14:35.349203+0000","last_became_peered":"2024-09-18T10:14:35.349203+0000","last_unstale":"2024-09-18T10:14:47.689156+0000","last_undegraded":"2024-09-18T10:14:47.689156+0000","last_fullsized":"2024-09-18T10:14:47.689156+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:14:34.288596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:15:37.868112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060130600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3],"acting":[9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1","version":"35'4","reported_seq":22,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-18T10:14:49.983786+0000","last_change":"2024-09-18T10:14:46.734405+0000","last_active":"2024-09-18T10:14:49.983786+0000","last_peered":"2024-09-18T10:14:49.983786+0000","last_clean":"2024-09-18T10:14:49.983786+0000","last_became_active":"2024-09-18T10:14:46.632535+0000","last_became_peered":"2024-09-18T10:14:46.632535+0000","last_unstale":"2024-09-18T10:14:49.983786+0000","last_undegraded":"2024-09-18T10:14:49.983786+0000","last_fullsized":"2024-09-18T10:14:49.983786+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:14:45.576168+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:14.852272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":2,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":4,"num_write_kb":320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,13,14,10,7,3,2],"acting":[4,13,14,10,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-18T10:14:48.667008+0000","last_change":"2024-09-18T10:14:37.601046+0000","last_active":"2024-09-18T10:14:48.667008+0000","last_peered":"2024-09-18T10:14:48.667008+0000","last_clean":"2024-09-18T10:14:48.667008+0000","last_became_active":"2024-09-18T10:14:35.348262+0000","last_became_peered":"2024-09-18T10:14:35.348262+0000","last_unstale":"2024-09-18T10:14:48.667008+0000","last_undegraded":"2024-09-18T10:14:48.667008+0000","last_fullsized":"2024-09-18T10:14:48.667008+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:14:34.288596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:03.520789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063430400000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,0],"acting":[14,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"3.0","version":"35'2","reported_seq":19,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-18T10:14:50.097406+0000","last_change":"2024-09-18T10:14:46.906990+0000","last_active":"2024-09-18T10:14:50.097406+0000","last_peered":"2024-09-18T10:14:50.097406+0000","last_clean":"2024-09-18T10:14:50.097406+0000","last_became_active":"2024-09-18T10:14:46.804481+0000","last_became_peered":"2024-09-18T10:14:46.804481+0000","last_unstale":"2024-09-18T10:14:50.097406+0000","last_undegraded":"2024-09-18T10:14:50.097406+0000","last_fullsized":"2024-09-18T10:14:50.097406+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:14:45.576168+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:15:50.283647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":1,"num_object_clones":0,"num_object_copies":7,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":1,"num_write":2,"num_write_kb":320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,9,2,6,15,4],"acting":[3,7,9,2,6,15,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"29'2","reported_seq":23,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-18T10:14:48.770041+0000","last_change":"2024-09-18T10:14:37.429583+0000","last_active":"2024-09-18T10:14:48.770041+0000","last_peered":"2024-09-18T10:14:48.770041+0000","last_clean":"2024-09-18T10:14:48.770041+0000","last_became_active":"2024-09-18T10:14:35.715374+0000","last_became_peered":"2024-09-18T10:14:35.715374+0000","last_unstale":"2024-09-18T10:14:48.770041+0000","last_undegraded":"2024-09-18T10:14:48.770041+0000","last_fullsized":"2024-09-18T10:14:48.770041+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:14:34.288596+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:03.520789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002421224,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,12],"acting":[13,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"3.3","version":"36'6","reported_seq":23,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-18T10:14:50.195378+0000","last_change":"2024-09-18T10:14:47.104629+0000","last_active":"2024-09-18T10:14:50.195378+0000","last_peered":"2024-09-18T10:14:50.195378+0000","last_clean":"2024-09-18T10:14:50.195378+0000","last_became_active":"2024-09-18T10:14:46.902332+0000","last_became_peered":"2024-09-18T10:14:46.902332+0000","last_unstale":"2024-09-18T10:14:50.195378+0000","last_undegraded":"2024-09-18T10:14:50.195378+0000","last_fullsized":"2024-09-18T10:14:50.195378+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:14:45.576168+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:15:56.108404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":851968,"num_objects":3,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4,"num_read_kb":1,"num_write":6,"num_write_kb":832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4,13,11,2,5],"acting":[10,9,4,13,11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.5","version":"35'2","reported_seq":19,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-18T10:14:49.365193+0000","last_change":"2024-09-18T10:14:46.901426+0000","last_active":"2024-09-18T10:14:49.365193+0000","last_peered":"2024-09-18T10:14:49.365193+0000","last_clean":"2024-09-18T10:14:49.365193+0000","last_became_active":"2024-09-18T10:14:46.800866+0000","last_became_peered":"2024-09-18T10:14:46.800866+0000","last_unstale":"2024-09-18T10:14:49.365193+0000","last_undegraded":"2024-09-18T10:14:49.365193+0000","last_fullsized":"2024-09-18T10:14:49.365193+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:14:45.576168+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:03.748509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65536,"num_objects":1,"num_object_clones":0,"num_object_copies":7,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":1,"num_write":2,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,4,9,12,0,11],"acting":[5,8,4,9,12,0,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":21,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-18T10:14:48.655928+0000","last_change":"2024-09-18T10:14:37.426192+0000","last_active":"2024-09-18T10:14:48.655928+0000","last_peered":"2024-09-18T10:14:48.655928+0000","last_clean":"2024-09-18T10:14:48.655928+0000","last_became_active":"2024-09-18T10:14:35.524102+0000","last_became_peered":"2024-09-18T10:14:35.524102+0000","last_unstale":"2024-09-18T10:14:48.655928+0000","last_undegraded":"2024-09-18T10:14:48.655928+0000","last_fullsized":"2024-09-18T10:14:48.655928+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:14:34.288596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:15:59.190171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.10077985,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"36'4","reported_seq":20,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-18T10:14:49.901615+0000","last_change":"2024-09-18T10:14:46.959375+0000","last_active":"2024-09-18T10:14:49.901615+0000","last_peered":"2024-09-18T10:14:49.901615+0000","last_clean":"2024-09-18T10:14:49.901615+0000","last_became_active":"2024-09-18T10:14:46.757297+0000","last_became_peered":"2024-09-18T10:14:46.757297+0000","last_unstale":"2024-09-18T10:14:49.901615+0000","last_undegraded":"2024-09-18T10:14:49.901615+0000","last_fullsized":"2024-09-18T10:14:49.901615+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:14:45.576168+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:00.604951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":786432,"num_objects":2,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":1,"num_write":4,"num_write_kb":768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,15,4,2,6,10,7],"acting":[8,15,4,2,6,10,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-18T10:14:48.677779+0000","last_change":"2024-09-18T10:14:37.327877+0000","last_active":"2024-09-18T10:14:48.677779+0000","last_peered":"2024-09-18T10:14:48.677779+0000","last_clean":"2024-09-18T10:14:48.677779+0000","last_became_active":"2024-09-18T10:14:35.329198+0000","last_became_peered":"2024-09-18T10:14:35.329198+0000","last_unstale":"2024-09-18T10:14:48.677779+0000","last_undegraded":"2024-09-18T10:14:48.677779+0000","last_fullsized":"2024-09-18T10:14:48.677779+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:14:34.288596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:15:44.196046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062415699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.7","version":"36'6","reported_seq":17,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-18T10:14:49.708160+0000","last_change":"2024-09-18T10:14:47.101051+0000","last_active":"2024-09-18T10:14:49.708160+0000","last_peered":"2024-09-18T10:14:49.708160+0000","last_clean":"2024-09-18T10:14:49.708160+0000","last_became_active":"2024-09-18T10:14:46.798744+0000","last_became_peered":"2024-09-18T10:14:46.798744+0000","last_unstale":"2024-09-18T10:14:49.708160+0000","last_undegraded":"2024-09-18T10:14:49.708160+0000","last_fullsized":"2024-09-18T10:14:49.708160+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:14:45.576168+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:14.843143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1310720,"num_objects":3,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":6,"num_write_kb":1280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6,0,14,10,13],"acting":[4,11,6,0,14,10,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":21,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-18T10:14:48.757155+0000","last_change":"2024-09-18T10:14:38.963300+0000","last_active":"2024-09-18T10:14:48.757155+0000","last_peered":"2024-09-18T10:14:48.757155+0000","last_clean":"2024-09-18T10:14:48.757155+0000","last_became_active":"2024-09-18T10:14:35.429859+0000","last_became_peered":"2024-09-18T10:14:35.429859+0000","last_unstale":"2024-09-18T10:14:48.757155+0000","last_undegraded":"2024-09-18T10:14:48.757155+0000","last_fullsized":"2024-09-18T10:14:48.757155+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:14:34.288596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:15:40.196292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037634499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11],"acting":[8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.6","version":"34'2","reported_seq":18,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-18T10:14:48.964252+0000","last_change":"2024-09-18T10:14:47.029719+0000","last_active":"2024-09-18T10:14:48.964252+0000","last_peered":"2024-09-18T10:14:48.964252+0000","last_clean":"2024-09-18T10:14:48.964252+0000","last_became_active":"2024-09-18T10:14:46.726948+0000","last_became_peered":"2024-09-18T10:14:46.726948+0000","last_unstale":"2024-09-18T10:14:48.964252+0000","last_undegraded":"2024-09-18T10:14:48.964252+0000","last_fullsized":"2024-09-18T10:14:48.964252+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:14:45.576168+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:14.450533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32768,"num_objects":1,"num_object_clones":0,"num_object_copies":7,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":1,"num_write":2,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,14,6,2,5,9,15],"acting":[0,14,6,2,5,9,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":21,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-18T10:14:48.669945+0000","last_change":"2024-09-18T10:14:37.427722+0000","last_active":"2024-09-18T10:14:48.669945+0000","last_peered":"2024-09-18T10:14:48.669945+0000","last_clean":"2024-09-18T10:14:48.669945+0000","last_became_active":"2024-09-18T10:14:35.348943+0000","last_became_peered":"2024-09-18T10:14:35.348943+0000","last_unstale":"2024-09-18T10:14:48.669945+0000","last_undegraded":"2024-09-18T10:14:48.669945+0000","last_fullsized":"2024-09-18T10:14:48.669945+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:14:34.288596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:03.161727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060442999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,10],"acting":[13,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":7192576,"num_objects":28,"num_object_clones":0,"num_object_copies":196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":47,"num_read_kb":19,"num_write":56,"num_write_kb":7024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":15642624,"data_stored":15642624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":56,"ondisk_log_size":56,"up":112,"acting":112,"num_store_stats":16},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":112,"ondisk_log_size":112,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":15,"up_from":22,"seq":94489280518,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299284,"kb_used_data":1920,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94072556,"statfs":{"total":96636764160,"available":96330297344,"internally_reserved":0,"allocated":1966080,"data_stored":1733193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6,10,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68200000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.111}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0840000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0509999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76300000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86199999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3460000000000001}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.294}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4410000000000001}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.663}]}]},{"osd":14,"up_from":22,"seq":94489280518,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299072,"kb_used_data":1712,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94072768,"statfs":{"total":96636764160,"available":96330514432,"internally_reserved":0,"allocated":1753088,"data_stored":1523753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,7,10,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":189.88800000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":79.626000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.417}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":180.19900000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":189.809}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":180.476}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":180.268}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":180.15899999999999}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":189.84999999999999}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":180.59100000000001}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":180.44800000000001}]}]},{"osd":13,"up_from":22,"seq":94489280519,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299552,"kb_used_data":2192,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94072288,"statfs":{"total":96636764160,"available":96330022912,"internally_reserved":0,"allocated":2244608,"data_stored":2011196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6,8,9,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4370000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79100000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":106.372}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":94.935000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85399999999999998}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.1189999999999998}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76700000000000002}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.8739999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0029999999999999}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0569999999999999}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.6980000000000004}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.9749999999999996}]}]},{"osd":0,"up_from":21,"seq":90194313222,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297860,"kb_used_data":504,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073980,"statfs":{"total":96636764160,"available":96331755520,"internally_reserved":0,"allocated":516096,"data_stored":293367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,9,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.069}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3500000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.196}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4119999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6819999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8320000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9710000000000001}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8640000000000001}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.913}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9350000000000001}]}]},{"osd":4,"up_from":21,"seq":90194313224,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298704,"kb_used_data":1344,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073136,"statfs":{"total":96636764160,"available":96330891264,"internally_reserved":0,"allocated":1376256,"data_stored":1146921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2829999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68500000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.018}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.8239999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":197.33199999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.4039999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5289999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.359}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":101.666}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.345}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":101.42700000000001}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":202.191}]}]},{"osd":9,"up_from":21,"seq":90194313222,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297756,"kb_used_data":388,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074084,"statfs":{"total":96636764160,"available":96331862016,"internally_reserved":0,"allocated":397312,"data_stored":180838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,3,4,8,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5700000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1579999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.323}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.385}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5960000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6759999999999999}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6559999999999999}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.802}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7749999999999999}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9299999999999999}]}]},{"osd":11,"up_from":21,"seq":90194313222,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298024,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073816,"statfs":{"total":96636764160,"available":96331587584,"internally_reserved":0,"allocated":684032,"data_stored":461303,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[2,3,6,8,9,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0049999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.878}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.107}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80100000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93300000000000005}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.167}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2150000000000001}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.28}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.526}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.587}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3260000000000001}]}]},{"osd":12,"up_from":21,"seq":90194313223,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297816,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074024,"statfs":{"total":96636764160,"available":96331800576,"internally_reserved":0,"allocated":471040,"data_stored":244234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,9,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60599999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79900000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85799999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68700000000000006}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.099}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.262}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.133}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.042}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1799999999999999}]}]},{"osd":1,"up_from":22,"seq":94489280518,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298004,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073836,"statfs":{"total":96636764160,"available":96331608064,"internally_reserved":0,"allocated":663552,"data_stored":440823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7,8,10,12],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95199999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0449999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.091}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2450000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3120000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.268}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4970000000000001}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.45}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4339999999999999}]}]},{"osd":2,"up_from":22,"seq":94489280518,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298320,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073520,"statfs":{"total":96636764160,"available":96331284480,"internally_reserved":0,"allocated":987136,"data_stored":764407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,3,4,5,6,7,8,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69999999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85999999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.157}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.103}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.073}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2829999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.181}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.464}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4079999999999999}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3420000000000001}]}]},{"osd":3,"up_from":22,"seq":94489280518,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298580,"kb_used_data":1220,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073260,"statfs":{"total":96636764160,"available":96331018240,"internally_reserved":0,"allocated":1249280,"data_stored":1019945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,4,6,7,8,9,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79600000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2070000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1739999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1479999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3839999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4239999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4830000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.964}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0419999999999998}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9219999999999999}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.829}]}]},{"osd":5,"up_from":22,"seq":94489280518,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298096,"kb_used_data":740,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073744,"statfs":{"total":96636764160,"available":96331513856,"internally_reserved":0,"allocated":757760,"data_stored":535031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,6,7,8,9,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":53.121000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":53.093000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":53.167000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":266.911}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":57.936}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68100000000000005}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":260.44200000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80400000000000005}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6479999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0249999999999999}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.089}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1120000000000001}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.056}]}]},{"osd":6,"up_from":22,"seq":94489280518,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298512,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073328,"statfs":{"total":96636764160,"available":96331087872,"internally_reserved":0,"allocated":1175552,"data_stored":949271,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,5,7,9,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93200000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60599999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0209999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0569999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80700000000000005}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96999999999999997}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.329}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1839999999999999}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.389}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2050000000000001}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2969999999999999}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.256}]}]},{"osd":7,"up_from":22,"seq":94489280518,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298964,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94072876,"statfs":{"total":96636764160,"available":96330625024,"internally_reserved":0,"allocated":1642496,"data_stored":1413161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,8,10,11,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.747}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3180000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83599999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96099999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.103}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.524}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.361}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.861}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.488}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4019999999999999}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.885}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6080000000000001}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5900000000000001}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5509999999999999}]}]},{"osd":8,"up_from":22,"seq":94489280518,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298440,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073400,"statfs":{"total":96636764160,"available":96331161600,"internally_reserved":0,"allocated":1105920,"data_stored":876585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[2,4,6,7,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93300000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.083}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88900000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.171}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84999999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0600000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.083}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95899999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.133}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.75}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.107}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":135.95099999999999}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2749999999999999}]}]},{"osd":10,"up_from":22,"seq":94489280519,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298504,"kb_used_data":1144,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073336,"statfs":{"total":96636764160,"available":96331096064,"internally_reserved":0,"allocated":1171456,"data_stored":942121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,4,5,7,9,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79800000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79800000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0920000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":125.23099999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92600000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":34.558}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95399999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1679999999999999}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":34.625999999999998}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":36.210000000000001}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66200000000000003}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":34.713000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":176128,"data_stored":176128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":593920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1257472,"data_stored":1257472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1343488,"data_stored":1343488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1454080,"data_stored":1454080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1355776,"data_stored":1355776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1269760,"data_stored":1269760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1335296,"data_stored":1335296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":675840,"data_stored":675840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":114688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":1232896,"data_stored":1232896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":339968,"data_stored":339968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":53248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":1970176,"data_stored":1970176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":1466368,"data_stored":1466368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":1003520,"data_stored":1003520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T10:14:52.882 INFO:tasks.ceph.ceph_manager.ceph:increase pool size by 10 2024-09-18T10:14:52.882 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set unique_pool_0 pg_num 26 2024-09-18T10:14:52.886 INFO:tasks.rados.rados.0.smithi189.stdout:65: expect (ObjNum 29 snap 0 seq_num 29) 2024-09-18T10:14:52.920 INFO:tasks.ceph.osd.11.smithi189.stderr:2024-09-18T10:14:52.915+0000 7f718b60a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 22400) UID: 0 2024-09-18T10:14:52.983 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-18T10:14:52.983 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-18T10:14:52.983 INFO:teuthology.orchestra.run.smithi189.stdout: "complaint_time": 30, 2024-09-18T10:14:52.983 INFO:teuthology.orchestra.run.smithi189.stdout: "num_blocked_ops": 0 2024-09-18T10:14:52.983 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-18T10:14:52.993 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T10:14:53.005 INFO:tasks.rados.rados.0.smithi189.stdout:64: finishing copy_from racing read to smithi18925896-15 2024-09-18T10:14:53.020 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-18T10:14:53.014+0000 7f302f2c3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22187) UID: 0 2024-09-18T10:14:53.090 INFO:tasks.rados.rados.0.smithi189.stdout:67: finishing rollback tid 1 to smithi18925896-10 2024-09-18T10:14:53.090 INFO:tasks.rados.rados.0.smithi189.stdout:67: finishing rollback tid 2 to smithi18925896-10 2024-09-18T10:14:53.090 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 10 v 17 (ObjNum 9 snap 0 seq_num 9) dirty exists 2024-09-18T10:14:53.098 INFO:tasks.rados.rados.0.smithi189.stdout:69: finishing copy_from racing read to smithi18925896-7 2024-09-18T10:14:53.098 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 32 v 8 (ObjNum 31 snap 0 seq_num 31) dirty exists 2024-09-18T10:14:53.101 INFO:tasks.rados.rados.0.smithi189.stdout:76: expect (ObjNum 26 snap 0 seq_num 26) 2024-09-18T10:14:53.114 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-18T10:14:53.114 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 20, 2024-09-18T10:14:53.114 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 600, 2024-09-18T10:14:53.115 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [ 2024-09-18T10:14:53.115 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.115 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.cs4 36/32 ECSubWrite(tid=17, reqid=client.4632.0:107, at_version=36'8, trim_to=0'0, roll_forward_to=36'6))", 2024-09-18T10:14:53.115 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:50.609171+0000", 2024-09-18T10:14:53.115 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 2.5019900960000001, 2024-09-18T10:14:53.115 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20274229099999999, 2024-09-18T10:14:53.115 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:14:53.115 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:14:53.115 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:14:53.115 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.115 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:14:53.116 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.609171+0000", 2024-09-18T10:14:53.116 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:53.116 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.116 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.116 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:14:53.116 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.609171+0000", 2024-09-18T10:14:53.116 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:53.116 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.116 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.116 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:14:53.116 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.609170+0000", 2024-09-18T10:14:53.117 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:14:53.117 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.117 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.117 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:14:53.117 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.609207+0000", 2024-09-18T10:14:53.117 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6745999999999997e-05 2024-09-18T10:14:53.117 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.117 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.117 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:14:53.117 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.609260+0000", 2024-09-18T10:14:53.117 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3344999999999998e-05 2024-09-18T10:14:53.118 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.118 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.118 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:14:53.118 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.609274+0000", 2024-09-18T10:14:53.118 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3682999999999999e-05 2024-09-18T10:14:53.118 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.118 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.118 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:14:53.118 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.810976+0000", 2024-09-18T10:14:53.118 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.201702717 2024-09-18T10:14:53.118 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.118 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.119 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:14:53.119 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.810993+0000", 2024-09-18T10:14:53.119 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6223e-05 2024-09-18T10:14:53.119 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.119 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.119 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:14:53.119 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.811858+0000", 2024-09-18T10:14:53.119 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00086581600000000005 2024-09-18T10:14:53.119 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.119 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.119 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:14:53.119 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.811913+0000", 2024-09-18T10:14:53.120 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.4945000000000003e-05 2024-09-18T10:14:53.120 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:53.120 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:14:53.120 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:53.120 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.120 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.120 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:112 3.9s0 3:9a1a774c:::smithi18925896-38:head [append 0~622592,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:14:53.120 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:50.615477+0000", 2024-09-18T10:14:53.120 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 2.495683777, 2024-09-18T10:14:53.120 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.30963059500000001, 2024-09-18T10:14:53.120 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:14:53.121 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:14:53.121 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:14:53.121 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:14:53.121 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:14:53.121 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 112 2024-09-18T10:14:53.121 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.121 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:14:53.121 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.121 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:14:53.121 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.615477+0000", 2024-09-18T10:14:53.121 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:53.121 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.122 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.122 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:14:53.122 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.615477+0000", 2024-09-18T10:14:53.122 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:53.122 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.122 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.122 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:14:53.122 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.615474+0000", 2024-09-18T10:14:53.122 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999971 2024-09-18T10:14:53.122 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.122 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.123 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:14:53.123 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.616279+0000", 2024-09-18T10:14:53.123 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00080491199999999997 2024-09-18T10:14:53.123 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.123 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.123 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:14:53.123 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.616316+0000", 2024-09-18T10:14:53.123 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6959000000000003e-05 2024-09-18T10:14:53.123 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.123 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.123 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:14:53.124 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.716408+0000", 2024-09-18T10:14:53.124 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100092056 2024-09-18T10:14:53.124 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.124 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.124 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:14:53.124 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.716492+0000", 2024-09-18T10:14:53.124 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.4223999999999999e-05 2024-09-18T10:14:53.124 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.124 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.124 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:14:53.125 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.716699+0000", 2024-09-18T10:14:53.125 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00020661599999999999 2024-09-18T10:14:53.125 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.125 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.125 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:14:53.125 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.717993+0000", 2024-09-18T10:14:53.125 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0012938520000000001 2024-09-18T10:14:53.125 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.126 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.126 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:14:53.126 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.719349+0000", 2024-09-18T10:14:53.126 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00135629 2024-09-18T10:14:53.126 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.126 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.126 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:14:53.126 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.925043+0000", 2024-09-18T10:14:53.126 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20569420799999999 2024-09-18T10:14:53.126 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.127 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.127 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:14:53.127 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.925108+0000", 2024-09-18T10:14:53.127 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.4428e-05 2024-09-18T10:14:53.127 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:53.127 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:14:53.127 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:53.127 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.127 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.127 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:113 3.9s0 3:9a1a774c:::smithi18925896-38:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:14:53.127 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:50.716437+0000", 2024-09-18T10:14:53.127 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 2.394723854, 2024-09-18T10:14:53.128 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20901476299999999, 2024-09-18T10:14:53.128 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:14:53.128 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:14:53.128 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:14:53.128 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:14:53.128 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:14:53.128 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 113 2024-09-18T10:14:53.128 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.128 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:14:53.128 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.128 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:14:53.129 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.716437+0000", 2024-09-18T10:14:53.129 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:53.129 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.129 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.129 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:14:53.129 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.716437+0000", 2024-09-18T10:14:53.129 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:53.129 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.129 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.129 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:14:53.129 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.716435+0000", 2024-09-18T10:14:53.129 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:14:53.130 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.130 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.130 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:14:53.130 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.716445+0000", 2024-09-18T10:14:53.130 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0152999999999999e-05 2024-09-18T10:14:53.130 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.130 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.130 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:14:53.130 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.716463+0000", 2024-09-18T10:14:53.130 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7756000000000001e-05 2024-09-18T10:14:53.130 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.130 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.130 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:14:53.131 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.816614+0000", 2024-09-18T10:14:53.131 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10015116 2024-09-18T10:14:53.131 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.131 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.131 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:14:53.131 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.816762+0000", 2024-09-18T10:14:53.131 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000148209 2024-09-18T10:14:53.131 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.131 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.131 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:14:53.131 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.816993+0000", 2024-09-18T10:14:53.131 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000230533 2024-09-18T10:14:53.131 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.131 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.132 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:14:53.132 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.817743+0000", 2024-09-18T10:14:53.132 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00074987600000000001 2024-09-18T10:14:53.132 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.132 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.132 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:14:53.132 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.818687+0000", 2024-09-18T10:14:53.132 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00094379999999999996 2024-09-18T10:14:53.132 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.132 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.132 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:14:53.133 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.925381+0000", 2024-09-18T10:14:53.133 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.106694663 2024-09-18T10:14:53.133 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.133 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.133 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:14:53.133 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.925452+0000", 2024-09-18T10:14:53.133 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.0733000000000002e-05 2024-09-18T10:14:53.133 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:53.133 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:14:53.133 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:53.133 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.133 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.134 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:114 3.9s0 3:9a1a774c:::smithi18925896-38:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:14:53.134 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:50.816650+0000", 2024-09-18T10:14:53.134 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 2.2945109160000001, 2024-09-18T10:14:53.134 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.41853457900000002, 2024-09-18T10:14:53.134 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:14:53.134 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:14:53.134 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:14:53.134 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:14:53.134 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:14:53.134 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 114 2024-09-18T10:14:53.134 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.134 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:14:53.134 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.135 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:14:53.135 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.816650+0000", 2024-09-18T10:14:53.135 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:53.135 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.135 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.135 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:14:53.135 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.816650+0000", 2024-09-18T10:14:53.135 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:53.135 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.135 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.135 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:14:53.135 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.816648+0000", 2024-09-18T10:14:53.136 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:14:53.136 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.136 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.136 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:14:53.136 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.816661+0000", 2024-09-18T10:14:53.136 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2619000000000001e-05 2024-09-18T10:14:53.136 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.136 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.137 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:14:53.137 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.816696+0000", 2024-09-18T10:14:53.137 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4885999999999998e-05 2024-09-18T10:14:53.137 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.137 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.137 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:14:53.137 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.816708+0000", 2024-09-18T10:14:53.137 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2843000000000001e-05 2024-09-18T10:14:53.137 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.137 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.137 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:14:53.137 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.818228+0000", 2024-09-18T10:14:53.137 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001519882 2024-09-18T10:14:53.138 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.138 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.138 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for rw locks", 2024-09-18T10:14:53.138 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.818557+0000", 2024-09-18T10:14:53.138 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00032873700000000001 2024-09-18T10:14:53.138 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.138 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.138 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:14:53.138 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.925488+0000", 2024-09-18T10:14:53.138 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.106930502 2024-09-18T10:14:53.138 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.138 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.139 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:14:53.139 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.925563+0000", 2024-09-18T10:14:53.139 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.5386e-05 2024-09-18T10:14:53.139 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.139 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.139 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:14:53.139 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.235185+0000", 2024-09-18T10:14:53.139 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.30962182399999999 2024-09-18T10:14:53.139 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:53.139 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:14:53.139 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:53.139 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.140 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.140 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:136 3.9s0 3:9ab7c4c0:::smithi18925896-46:head [append 0~770048,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:14:53.140 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:50.920266+0000", 2024-09-18T10:14:53.140 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 2.1908953900000001, 2024-09-18T10:14:53.140 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20727352199999999, 2024-09-18T10:14:53.140 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:14:53.140 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:14:53.140 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:14:53.140 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:14:53.140 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:14:53.140 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 136 2024-09-18T10:14:53.140 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.141 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:14:53.141 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.141 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:14:53.141 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.920266+0000", 2024-09-18T10:14:53.141 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:53.141 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.141 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.141 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:14:53.141 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.920266+0000", 2024-09-18T10:14:53.141 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:53.141 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.141 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.141 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:14:53.142 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.920263+0000", 2024-09-18T10:14:53.142 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999976 2024-09-18T10:14:53.142 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.142 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.142 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:14:53.142 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.920921+0000", 2024-09-18T10:14:53.142 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00065782000000000004 2024-09-18T10:14:53.142 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.142 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.142 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:14:53.142 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.920949+0000", 2024-09-18T10:14:53.143 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.8235e-05 2024-09-18T10:14:53.143 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.143 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.143 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:14:53.143 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.920961+0000", 2024-09-18T10:14:53.143 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2112e-05 2024-09-18T10:14:53.143 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.143 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.143 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:14:53.143 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.921044+0000", 2024-09-18T10:14:53.143 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.2106000000000003e-05 2024-09-18T10:14:53.143 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.144 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.144 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:14:53.144 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.921280+0000", 2024-09-18T10:14:53.144 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000236538 2024-09-18T10:14:53.144 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.144 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.144 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:14:53.144 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.922611+0000", 2024-09-18T10:14:53.144 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0013313909999999999 2024-09-18T10:14:53.144 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.144 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.145 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:14:53.145 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.925250+0000", 2024-09-18T10:14:53.145 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0026385050000000002 2024-09-18T10:14:53.145 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.145 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.145 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:14:53.145 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.127403+0000", 2024-09-18T10:14:53.145 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.202152894 2024-09-18T10:14:53.145 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.145 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.145 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:14:53.145 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.127539+0000", 2024-09-18T10:14:53.146 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000136353 2024-09-18T10:14:53.146 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:53.146 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:14:53.146 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:53.146 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.146 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.146 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:137 3.9s0 3:9ab7c4c0:::smithi18925896-46:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:14:53.146 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:50.920989+0000", 2024-09-18T10:14:53.146 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 2.190171984, 2024-09-18T10:14:53.146 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.31321379100000002, 2024-09-18T10:14:53.146 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:14:53.147 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:14:53.147 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:14:53.147 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:14:53.147 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:14:53.147 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 137 2024-09-18T10:14:53.147 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.147 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:14:53.147 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.147 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:14:53.148 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.920989+0000", 2024-09-18T10:14:53.148 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:53.148 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.148 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.148 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:14:53.148 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.920989+0000", 2024-09-18T10:14:53.148 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:53.148 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.148 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.148 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:14:53.148 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.920988+0000", 2024-09-18T10:14:53.149 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:14:53.149 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.149 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.149 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:14:53.149 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.920995+0000", 2024-09-18T10:14:53.149 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.9659999999999999e-06 2024-09-18T10:14:53.149 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.149 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.149 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:14:53.149 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.921005+0000", 2024-09-18T10:14:53.149 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.4409999999999999e-06 2024-09-18T10:14:53.150 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.150 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.150 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:14:53.150 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.921011+0000", 2024-09-18T10:14:53.150 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.7390000000000002e-06 2024-09-18T10:14:53.150 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.150 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.150 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:14:53.150 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.923916+0000", 2024-09-18T10:14:53.150 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.002904968 2024-09-18T10:14:53.150 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.151 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.151 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:14:53.151 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.924060+0000", 2024-09-18T10:14:53.151 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000143337 2024-09-18T10:14:53.151 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.151 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.151 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:14:53.151 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.924629+0000", 2024-09-18T10:14:53.152 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00056904700000000004 2024-09-18T10:14:53.152 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.152 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.152 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:14:53.152 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.925900+0000", 2024-09-18T10:14:53.152 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0012710449999999999 2024-09-18T10:14:53.152 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.152 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.152 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:14:53.152 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.234101+0000", 2024-09-18T10:14:53.152 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.308201593 2024-09-18T10:14:53.152 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.153 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.153 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:14:53.153 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.234203+0000", 2024-09-18T10:14:53.153 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00010160100000000001 2024-09-18T10:14:53.153 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:53.153 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:14:53.153 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:53.153 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.153 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.153 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:138 3.9s0 3:9ab7c4c0:::smithi18925896-46:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:14:53.153 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:50.921054+0000", 2024-09-18T10:14:53.153 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 2.1901066180000002, 2024-09-18T10:14:53.154 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.61288337900000001, 2024-09-18T10:14:53.154 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:14:53.154 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:14:53.154 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:14:53.154 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:14:53.154 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:14:53.154 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 138 2024-09-18T10:14:53.154 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.156 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:14:53.156 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.156 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:14:53.156 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.921054+0000", 2024-09-18T10:14:53.156 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:53.156 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.156 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.156 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:14:53.156 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.921054+0000", 2024-09-18T10:14:53.156 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:53.156 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.157 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.157 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:14:53.157 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.921054+0000", 2024-09-18T10:14:53.157 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999995 2024-09-18T10:14:53.157 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.157 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.157 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:14:53.157 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.921057+0000", 2024-09-18T10:14:53.157 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1829999999999998e-06 2024-09-18T10:14:53.157 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.157 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.157 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:14:53.158 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.921090+0000", 2024-09-18T10:14:53.158 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3281999999999997e-05 2024-09-18T10:14:53.158 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.158 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.158 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:14:53.158 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.921096+0000", 2024-09-18T10:14:53.158 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.0519999999999998e-06 2024-09-18T10:14:53.158 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.158 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.158 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:14:53.158 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.127704+0000", 2024-09-18T10:14:53.159 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20660720499999999 2024-09-18T10:14:53.159 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.159 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.159 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for rw locks", 2024-09-18T10:14:53.159 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.127851+0000", 2024-09-18T10:14:53.159 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00014710999999999999 2024-09-18T10:14:53.159 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.159 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.159 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:14:53.159 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.234300+0000", 2024-09-18T10:14:53.159 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10644893900000001 2024-09-18T10:14:53.159 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.160 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.160 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:14:53.160 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.234462+0000", 2024-09-18T10:14:53.160 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00016218999999999999 2024-09-18T10:14:53.160 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.160 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.160 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:14:53.160 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.533938+0000", 2024-09-18T10:14:53.160 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.29947596999999998 2024-09-18T10:14:53.160 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:53.160 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:14:53.160 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:53.161 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.161 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.161 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.9s0 36/32 ECSubWriteReply(tid=13, last_complete=36'10, committed=1, applied=1))", 2024-09-18T10:14:53.161 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:50.925234+0000", 2024-09-18T10:14:53.161 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 2.1859266310000001, 2024-09-18T10:14:53.161 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20104683500000001, 2024-09-18T10:14:53.161 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:14:53.161 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:14:53.161 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:14:53.161 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.161 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:14:53.161 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.925234+0000", 2024-09-18T10:14:53.162 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:53.162 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.162 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.162 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:14:53.162 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.925234+0000", 2024-09-18T10:14:53.162 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:53.162 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.162 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.162 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:14:53.162 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.925233+0000", 2024-09-18T10:14:53.162 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:14:53.162 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.163 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.163 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:14:53.163 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.925243+0000", 2024-09-18T10:14:53.163 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0125999999999999e-05 2024-09-18T10:14:53.163 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.163 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.163 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:14:53.163 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.925263+0000", 2024-09-18T10:14:53.163 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9619e-05 2024-09-18T10:14:53.163 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.163 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.164 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:14:53.164 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.925274+0000", 2024-09-18T10:14:53.164 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.137e-05 2024-09-18T10:14:53.164 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.164 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.164 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:14:53.164 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.126228+0000", 2024-09-18T10:14:53.164 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20095364499999999 2024-09-18T10:14:53.164 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.164 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.164 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:14:53.164 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.126281+0000", 2024-09-18T10:14:53.165 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3315e-05 2024-09-18T10:14:53.165 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:53.165 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:14:53.165 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:53.165 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.165 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.165 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpRead(3.9s0 36/32 ECSubRead(tid=14, to_read={3:9a1a774c:::smithi18925896-38:head=0,4096,0}, subchunks={3:9a1a774c:::smithi18925896-38:head=[0,1]}, attrs_to_read=))", 2024-09-18T10:14:53.165 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:50.925751+0000", 2024-09-18T10:14:53.165 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 2.1854095720000002, 2024-09-18T10:14:53.165 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20107041, 2024-09-18T10:14:53.165 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:14:53.166 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:14:53.166 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:14:53.166 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.166 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:14:53.166 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.925751+0000", 2024-09-18T10:14:53.166 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:53.166 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.166 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.166 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:14:53.166 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.925751+0000", 2024-09-18T10:14:53.166 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:53.167 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.167 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.167 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:14:53.167 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.925751+0000", 2024-09-18T10:14:53.167 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:53.167 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.167 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.167 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:14:53.167 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.925751+0000", 2024-09-18T10:14:53.167 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:53.167 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.167 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.167 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:14:53.168 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.925796+0000", 2024-09-18T10:14:53.168 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4913000000000002e-05 2024-09-18T10:14:53.168 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.168 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.168 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:14:53.168 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.925860+0000", 2024-09-18T10:14:53.168 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.3905000000000006e-05 2024-09-18T10:14:53.168 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.168 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.168 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:14:53.168 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.126380+0000", 2024-09-18T10:14:53.169 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20051955799999999 2024-09-18T10:14:53.169 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.169 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.169 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:14:53.169 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.126822+0000", 2024-09-18T10:14:53.169 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000442034 2024-09-18T10:14:53.169 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:53.169 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:14:53.169 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:53.169 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.169 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.170 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpReadReply(3.9s0 36/32 ECSubReadReply(tid=14, attrs_read=0))", 2024-09-18T10:14:53.170 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:50.926194+0000", 2024-09-18T10:14:53.170 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 2.1849668370000002, 2024-09-18T10:14:53.170 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.200771486, 2024-09-18T10:14:53.170 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:14:53.170 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:14:53.170 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:14:53.170 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.170 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:14:53.170 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.926194+0000", 2024-09-18T10:14:53.170 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:53.170 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.171 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.171 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:14:53.171 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.926194+0000", 2024-09-18T10:14:53.171 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:53.171 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.171 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.171 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:14:53.171 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.926193+0000", 2024-09-18T10:14:53.171 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:14:53.171 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.171 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.172 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:14:53.172 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.926211+0000", 2024-09-18T10:14:53.172 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.808e-05 2024-09-18T10:14:53.172 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.172 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.172 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:14:53.172 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.926230+0000", 2024-09-18T10:14:53.172 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9106999999999998e-05 2024-09-18T10:14:53.172 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.172 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.172 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:14:53.172 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.026319+0000", 2024-09-18T10:14:53.173 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100088187 2024-09-18T10:14:53.173 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.173 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.173 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:14:53.173 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.126900+0000", 2024-09-18T10:14:53.173 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100581328 2024-09-18T10:14:53.173 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.173 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.173 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:14:53.173 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.126966+0000", 2024-09-18T10:14:53.173 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.5852000000000006e-05 2024-09-18T10:14:53.174 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:53.174 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:14:53.174 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:53.174 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.174 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.174 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:142 3.9s0 3:9221fff5:::smithi18925896-48:head [append 0~425984,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:14:53.174 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:50.929918+0000", 2024-09-18T10:14:53.174 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 2.1812433059999998, 2024-09-18T10:14:53.174 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.39521715200000002, 2024-09-18T10:14:53.174 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:14:53.174 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:14:53.175 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:14:53.175 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:14:53.175 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:14:53.175 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 142 2024-09-18T10:14:53.175 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.175 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:14:53.175 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.175 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:14:53.175 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.929918+0000", 2024-09-18T10:14:53.175 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:53.175 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.176 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.176 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:14:53.176 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.929918+0000", 2024-09-18T10:14:53.176 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:53.176 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.176 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.176 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:14:53.176 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.929916+0000", 2024-09-18T10:14:53.176 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:14:53.176 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.176 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.177 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:14:53.177 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.930096+0000", 2024-09-18T10:14:53.177 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00018038700000000001 2024-09-18T10:14:53.177 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.177 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.177 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:14:53.177 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.930122+0000", 2024-09-18T10:14:53.177 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5927000000000001e-05 2024-09-18T10:14:53.177 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.177 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.177 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:14:53.178 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.930132+0000", 2024-09-18T10:14:53.178 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.6539999999999994e-06 2024-09-18T10:14:53.178 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.178 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.178 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:14:53.178 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.127900+0000", 2024-09-18T10:14:53.178 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.19776816899999999 2024-09-18T10:14:53.178 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.178 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.178 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:14:53.178 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.128112+0000", 2024-09-18T10:14:53.178 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00021222999999999999 2024-09-18T10:14:53.178 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.178 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.179 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:14:53.179 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.129335+0000", 2024-09-18T10:14:53.179 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0012222159999999999 2024-09-18T10:14:53.179 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.179 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.179 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:14:53.179 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.132031+0000", 2024-09-18T10:14:53.179 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0026961300000000001 2024-09-18T10:14:53.179 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.179 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.179 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:14:53.179 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.325013+0000", 2024-09-18T10:14:53.179 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.19298240999999999 2024-09-18T10:14:53.180 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.180 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.180 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:14:53.180 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.325135+0000", 2024-09-18T10:14:53.180 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000121842 2024-09-18T10:14:53.180 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:53.180 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:14:53.180 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:53.180 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.180 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.180 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:143 3.9s0 3:9221fff5:::smithi18925896-48:head [append 0~376832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:14:53.181 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:50.932038+0000", 2024-09-18T10:14:53.181 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 2.1791230170000002, 2024-09-18T10:14:53.181 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.60109878000000005, 2024-09-18T10:14:53.181 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:14:53.181 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:14:53.181 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:14:53.181 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:14:53.181 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:14:53.181 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 143 2024-09-18T10:14:53.181 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.181 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:14:53.182 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.182 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:14:53.182 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932038+0000", 2024-09-18T10:14:53.182 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:53.182 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.182 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.182 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:14:53.182 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932038+0000", 2024-09-18T10:14:53.182 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:53.182 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.182 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.182 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:14:53.183 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932037+0000", 2024-09-18T10:14:53.183 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:14:53.183 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.183 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.183 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:14:53.183 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932309+0000", 2024-09-18T10:14:53.183 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000272086 2024-09-18T10:14:53.183 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.183 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.183 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:14:53.183 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932323+0000", 2024-09-18T10:14:53.183 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4354e-05 2024-09-18T10:14:53.184 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.184 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.184 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:14:53.184 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932329+0000", 2024-09-18T10:14:53.184 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.6840000000000001e-06 2024-09-18T10:14:53.184 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.184 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.184 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:14:53.184 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.130012+0000", 2024-09-18T10:14:53.184 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.19768277100000001 2024-09-18T10:14:53.184 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.185 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.185 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:14:53.185 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.130068+0000", 2024-09-18T10:14:53.185 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.6158e-05 2024-09-18T10:14:53.185 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.185 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.185 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:14:53.185 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.130809+0000", 2024-09-18T10:14:53.185 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00074078700000000002 2024-09-18T10:14:53.185 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.185 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.186 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:14:53.186 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.132840+0000", 2024-09-18T10:14:53.186 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0020308879999999998 2024-09-18T10:14:53.186 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.186 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.186 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:14:53.186 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.533025+0000", 2024-09-18T10:14:53.186 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.400185491 2024-09-18T10:14:53.186 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.186 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.186 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:14:53.186 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.533137+0000", 2024-09-18T10:14:53.187 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000111727 2024-09-18T10:14:53.187 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:53.187 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:14:53.187 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:53.187 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.187 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.187 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:144 3.9s0 3:9221fff5:::smithi18925896-48:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:14:53.187 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:50.932336+0000", 2024-09-18T10:14:53.187 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 2.1788253810000002, 2024-09-18T10:14:53.187 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.60198766699999995, 2024-09-18T10:14:53.187 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:14:53.187 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:14:53.188 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:14:53.188 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:14:53.188 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:14:53.188 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 144 2024-09-18T10:14:53.188 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.188 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:14:53.188 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.188 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:14:53.188 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932336+0000", 2024-09-18T10:14:53.188 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:53.188 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.189 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.189 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:14:53.189 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932336+0000", 2024-09-18T10:14:53.189 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:53.189 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.189 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.189 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:14:53.189 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932335+0000", 2024-09-18T10:14:53.189 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:14:53.189 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.189 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.189 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:14:53.190 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932339+0000", 2024-09-18T10:14:53.190 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8539999999999999e-06 2024-09-18T10:14:53.190 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.190 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.190 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:14:53.190 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932347+0000", 2024-09-18T10:14:53.190 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.2910000000000007e-06 2024-09-18T10:14:53.190 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.190 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.190 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:14:53.190 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932351+0000", 2024-09-18T10:14:53.190 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.5420000000000001e-06 2024-09-18T10:14:53.191 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.191 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.191 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:14:53.191 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.131380+0000", 2024-09-18T10:14:53.191 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.199029236 2024-09-18T10:14:53.191 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.191 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.191 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:14:53.191 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.131436+0000", 2024-09-18T10:14:53.191 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.6098999999999999e-05 2024-09-18T10:14:53.191 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.191 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.192 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:14:53.192 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.131777+0000", 2024-09-18T10:14:53.192 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00034058800000000001 2024-09-18T10:14:53.192 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.192 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.192 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:14:53.192 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.133245+0000", 2024-09-18T10:14:53.192 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0014682300000000001 2024-09-18T10:14:53.192 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.192 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.193 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:14:53.193 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.534187+0000", 2024-09-18T10:14:53.193 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40094253299999999 2024-09-18T10:14:53.193 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.193 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.193 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:14:53.193 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.534323+0000", 2024-09-18T10:14:53.193 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00013606299999999999 2024-09-18T10:14:53.193 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:53.193 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:14:53.193 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:53.193 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.194 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.194 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:145 3.9s0 3:9221fff5:::smithi18925896-48:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:14:53.194 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:50.932355+0000", 2024-09-18T10:14:53.194 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 2.1788058970000002, 2024-09-18T10:14:53.194 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.80502307200000001, 2024-09-18T10:14:53.194 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:14:53.194 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:14:53.194 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:14:53.194 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:14:53.194 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:14:53.194 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 145 2024-09-18T10:14:53.195 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.195 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:14:53.195 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.195 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:14:53.195 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932355+0000", 2024-09-18T10:14:53.195 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:53.195 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.195 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.195 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:14:53.195 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932355+0000", 2024-09-18T10:14:53.195 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:53.195 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.195 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.196 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:14:53.196 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932355+0000", 2024-09-18T10:14:53.196 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999995 2024-09-18T10:14:53.196 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.196 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.196 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:14:53.196 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932358+0000", 2024-09-18T10:14:53.196 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4369999999999999e-06 2024-09-18T10:14:53.196 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.196 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.196 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:14:53.197 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932366+0000", 2024-09-18T10:14:53.197 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.3010000000000007e-06 2024-09-18T10:14:53.197 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.197 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.197 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:14:53.197 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932370+0000", 2024-09-18T10:14:53.197 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.5619999999999999e-06 2024-09-18T10:14:53.197 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.197 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.197 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:14:53.197 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.132210+0000", 2024-09-18T10:14:53.197 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.199840145 2024-09-18T10:14:53.198 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.198 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.198 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for rw locks", 2024-09-18T10:14:53.198 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.132312+0000", 2024-09-18T10:14:53.198 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000102335 2024-09-18T10:14:53.198 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.198 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.198 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:14:53.198 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.535310+0000", 2024-09-18T10:14:53.198 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40299779000000002 2024-09-18T10:14:53.198 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.198 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.199 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:14:53.199 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.535541+0000", 2024-09-18T10:14:53.199 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00023038299999999999 2024-09-18T10:14:53.199 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.199 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.199 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:14:53.199 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.737378+0000", 2024-09-18T10:14:53.199 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.201837604 2024-09-18T10:14:53.199 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:53.199 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:14:53.199 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:53.199 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.199 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.200 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.9s0 36/32 ECSubWriteReply(tid=17, last_complete=36'13, committed=1, applied=1))", 2024-09-18T10:14:53.200 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:51.326043+0000", 2024-09-18T10:14:53.200 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 1.7851179880000001, 2024-09-18T10:14:53.200 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20737249899999999, 2024-09-18T10:14:53.200 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:14:53.200 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:14:53.200 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:14:53.200 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.200 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:14:53.200 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.326043+0000", 2024-09-18T10:14:53.200 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:53.200 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.201 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.201 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:14:53.201 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.326043+0000", 2024-09-18T10:14:53.201 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:53.201 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.201 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.201 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:14:53.201 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.326041+0000", 2024-09-18T10:14:53.201 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:14:53.201 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.202 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.202 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:14:53.202 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.326054+0000", 2024-09-18T10:14:53.202 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2208999999999999e-05 2024-09-18T10:14:53.202 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.202 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.202 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:14:53.202 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.326078+0000", 2024-09-18T10:14:53.202 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3920000000000001e-05 2024-09-18T10:14:53.202 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.202 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.202 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:14:53.203 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.426165+0000", 2024-09-18T10:14:53.203 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100087923 2024-09-18T10:14:53.203 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.203 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.203 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:14:53.203 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.533384+0000", 2024-09-18T10:14:53.203 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.107218428 2024-09-18T10:14:53.203 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.203 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.203 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:14:53.203 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.533416+0000", 2024-09-18T10:14:53.203 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1659000000000003e-05 2024-09-18T10:14:53.204 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:53.204 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:14:53.204 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:53.204 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.204 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.204 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.9s0 36/32 ECSubWriteReply(tid=17, last_complete=36'13, committed=1, applied=1))", 2024-09-18T10:14:53.204 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:51.333140+0000", 2024-09-18T10:14:53.204 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 1.77802101, 2024-09-18T10:14:53.204 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20016851699999999, 2024-09-18T10:14:53.204 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:14:53.204 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:14:53.204 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:14:53.205 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.205 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:14:53.205 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.333140+0000", 2024-09-18T10:14:53.205 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:53.205 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.205 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.205 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:14:53.205 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.333140+0000", 2024-09-18T10:14:53.205 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:53.205 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.205 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.206 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:14:53.206 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.333140+0000", 2024-09-18T10:14:53.206 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999995 2024-09-18T10:14:53.206 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.206 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.206 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:14:53.206 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.333143+0000", 2024-09-18T10:14:53.206 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.743e-06 2024-09-18T10:14:53.206 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.206 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.206 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:14:53.207 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.333157+0000", 2024-09-18T10:14:53.207 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3976e-05 2024-09-18T10:14:53.207 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.207 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.207 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:14:53.207 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.333163+0000", 2024-09-18T10:14:53.207 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.0159999999999997e-06 2024-09-18T10:14:53.207 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.207 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.207 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:14:53.207 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.533282+0000", 2024-09-18T10:14:53.207 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20011900299999999 2024-09-18T10:14:53.208 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.208 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.208 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:14:53.208 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.533309+0000", 2024-09-18T10:14:53.208 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.6248999999999999e-05 2024-09-18T10:14:53.208 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:53.208 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:14:53.208 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:53.208 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.208 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.208 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(osd.6.22:1 3.9s0 3:9bcb6eef:::smithi18925896-11:head [assert-version v4,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:14:53.209 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:51.623104+0000", 2024-09-18T10:14:53.209 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 1.48805659, 2024-09-18T10:14:53.209 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.21424270000000001, 2024-09-18T10:14:53.209 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:14:53.209 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:14:53.209 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:14:53.209 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.209 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:14:53.209 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.623104+0000", 2024-09-18T10:14:53.209 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:53.209 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.210 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.210 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:14:53.210 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.623104+0000", 2024-09-18T10:14:53.210 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:53.210 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.210 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.210 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:14:53.210 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.623102+0000", 2024-09-18T10:14:53.210 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999976 2024-09-18T10:14:53.210 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.210 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.210 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:14:53.211 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.623115+0000", 2024-09-18T10:14:53.211 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3509999999999999e-05 2024-09-18T10:14:53.211 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.211 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.211 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:14:53.211 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.623138+0000", 2024-09-18T10:14:53.211 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.2993000000000002e-05 2024-09-18T10:14:53.211 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.211 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.211 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:14:53.211 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.623157+0000", 2024-09-18T10:14:53.211 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.8243000000000001e-05 2024-09-18T10:14:53.212 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.212 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.212 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:14:53.212 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.623275+0000", 2024-09-18T10:14:53.212 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000118126 2024-09-18T10:14:53.212 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.212 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.212 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:14:53.212 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.623443+0000", 2024-09-18T10:14:53.212 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00016819200000000001 2024-09-18T10:14:53.212 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.213 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.213 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:14:53.213 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.837347+0000", 2024-09-18T10:14:53.213 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.21390421600000001 2024-09-18T10:14:53.213 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:53.213 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:14:53.213 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:53.213 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.213 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.213 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:172 3.9s0 3:9fc2eacb:::smithi18925896-17:head [read 0~278528,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:14:53.213 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:51.836406+0000", 2024-09-18T10:14:53.213 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 1.2747551939999999, 2024-09-18T10:14:53.214 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.205722293, 2024-09-18T10:14:53.214 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:14:53.214 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:14:53.214 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:14:53.214 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:14:53.214 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:14:53.214 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172 2024-09-18T10:14:53.214 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.214 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:14:53.214 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.214 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:14:53.214 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.836406+0000", 2024-09-18T10:14:53.214 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:53.215 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.215 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.215 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:14:53.215 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.836406+0000", 2024-09-18T10:14:53.215 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:53.215 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.215 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.215 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:14:53.215 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.836403+0000", 2024-09-18T10:14:53.215 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999971 2024-09-18T10:14:53.215 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.216 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.216 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:14:53.216 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.836420+0000", 2024-09-18T10:14:53.216 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6724000000000001e-05 2024-09-18T10:14:53.216 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.216 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.216 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:14:53.216 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.836509+0000", 2024-09-18T10:14:53.216 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.9506999999999995e-05 2024-09-18T10:14:53.216 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.216 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.216 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:14:53.217 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.836545+0000", 2024-09-18T10:14:53.217 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.5645000000000002e-05 2024-09-18T10:14:53.217 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.217 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.217 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:14:53.217 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.937603+0000", 2024-09-18T10:14:53.217 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10105837500000001 2024-09-18T10:14:53.217 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.217 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.217 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:14:53.217 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.937752+0000", 2024-09-18T10:14:53.217 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00014940799999999999 2024-09-18T10:14:53.218 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.218 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.218 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:14:53.218 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.042128+0000", 2024-09-18T10:14:53.218 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.104375668 2024-09-18T10:14:53.218 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:53.218 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:14:53.218 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:53.218 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.218 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.218 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:173 3.9s0 3:9fc2eacb:::smithi18925896-17:head [read 0~278528,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:14:53.218 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:51.836582+0000", 2024-09-18T10:14:53.219 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 1.2745795499999999, 2024-09-18T10:14:53.219 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.30525520099999998, 2024-09-18T10:14:53.219 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:14:53.219 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:14:53.219 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:14:53.219 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:14:53.219 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:14:53.219 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 173 2024-09-18T10:14:53.219 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.219 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:14:53.219 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.219 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:14:53.220 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.836582+0000", 2024-09-18T10:14:53.220 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:53.220 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.220 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.220 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:14:53.220 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.836582+0000", 2024-09-18T10:14:53.220 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:53.220 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.220 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.220 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:14:53.220 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.836580+0000", 2024-09-18T10:14:53.221 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:14:53.221 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.221 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.221 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:14:53.221 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.836607+0000", 2024-09-18T10:14:53.221 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.6823000000000001e-05 2024-09-18T10:14:53.221 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.221 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.221 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:14:53.221 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.836627+0000", 2024-09-18T10:14:53.221 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0936000000000001e-05 2024-09-18T10:14:53.221 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.222 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.222 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:14:53.222 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.836640+0000", 2024-09-18T10:14:53.222 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2853e-05 2024-09-18T10:14:53.222 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.222 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.222 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:14:53.222 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.938402+0000", 2024-09-18T10:14:53.222 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.101761796 2024-09-18T10:14:53.222 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.222 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.223 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:14:53.223 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.938666+0000", 2024-09-18T10:14:53.223 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00026422099999999998 2024-09-18T10:14:53.223 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.223 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.223 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:14:53.223 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.141837+0000", 2024-09-18T10:14:53.223 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20317037199999999 2024-09-18T10:14:53.223 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:53.223 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:14:53.224 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:53.224 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.224 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.224 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:174 3.9s0 3:9fc2eacb:::smithi18925896-17:head [read 0~278528,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:14:53.224 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:51.836672+0000", 2024-09-18T10:14:53.224 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 1.2744886420000001, 2024-09-18T10:14:53.224 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.407996215, 2024-09-18T10:14:53.224 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:14:53.224 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:14:53.224 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:14:53.224 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:14:53.225 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:14:53.225 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 174 2024-09-18T10:14:53.225 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.225 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:14:53.225 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.225 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:14:53.225 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.836672+0000", 2024-09-18T10:14:53.225 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:53.226 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.226 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.226 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:14:53.226 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.836672+0000", 2024-09-18T10:14:53.226 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:53.226 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.226 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.226 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:14:53.226 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.836671+0000", 2024-09-18T10:14:53.226 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:14:53.226 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.227 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.227 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:14:53.227 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.836680+0000", 2024-09-18T10:14:53.227 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.5830000000000007e-06 2024-09-18T10:14:53.227 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.227 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.227 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:14:53.227 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.836696+0000", 2024-09-18T10:14:53.227 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6823000000000001e-05 2024-09-18T10:14:53.227 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.227 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.227 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:14:53.228 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.836710+0000", 2024-09-18T10:14:53.228 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.325e-05 2024-09-18T10:14:53.228 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.228 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.228 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:14:53.228 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.142312+0000", 2024-09-18T10:14:53.228 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.30560266400000002 2024-09-18T10:14:53.228 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.228 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.228 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:14:53.228 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.142549+0000", 2024-09-18T10:14:53.228 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00023630299999999999 2024-09-18T10:14:53.229 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.229 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.229 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:14:53.229 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.244669+0000", 2024-09-18T10:14:53.229 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10211993599999999 2024-09-18T10:14:53.229 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:53.229 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:14:53.229 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:53.229 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:53.229 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:14:53.229 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-18T10:14:53.230 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T10:14:53.231 INFO:tasks.rados.rados.0.smithi189.stdout:70: expect (ObjNum 19 snap 0 seq_num 19) 2024-09-18T10:14:53.232 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-18T10:14:53.226+0000 7f00046a8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 22136) UID: 0 2024-09-18T10:14:53.233 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-09-18T10:14:53.234 INFO:tasks.rados.rados.0.smithi189.stdout:64: finishing copy_from to smithi18925896-15 2024-09-18T10:14:53.234 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 15 v 4 (ObjNum 10 snap 0 seq_num 10) dirty exists 2024-09-18T10:14:53.234 INFO:tasks.rados.rados.0.smithi189.stdout:66: finishing write tid 1 to smithi18925896-23 2024-09-18T10:14:53.234 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 14 v 11 (ObjNum 13 snap 0 seq_num 13) dirty exists 2024-09-18T10:14:53.234 INFO:tasks.rados.rados.0.smithi189.stdout:73: expect (ObjNum 12 snap 0 seq_num 12) 2024-09-18T10:14:53.234 INFO:tasks.rados.rados.0.smithi189.stdout:75: finishing copy_from racing read to smithi18925896-48 2024-09-18T10:14:53.234 INFO:tasks.rados.rados.0.smithi189.stdout:78: expect (ObjNum 42 snap 0 seq_num 42) 2024-09-18T10:14:53.235 INFO:tasks.rados.rados.0.smithi189.stdout:77: finishing copy_from racing read to smithi18925896-28 2024-09-18T10:14:53.235 INFO:tasks.rados.rados.0.smithi189.stdout:69: finishing copy_from to smithi18925896-7 2024-09-18T10:14:53.235 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 7 v 6 (ObjNum 16 snap 0 seq_num 16) dirty exists 2024-09-18T10:14:53.235 INFO:tasks.rados.rados.0.smithi189.stdout:66: finishing write tid 2 to smithi18925896-23 2024-09-18T10:14:53.235 INFO:tasks.rados.rados.0.smithi189.stdout:66: finishing write tid 3 to smithi18925896-23 2024-09-18T10:14:53.312 INFO:tasks.rados.rados.0.smithi189.stdout:72: finishing rollback tid 0 to smithi18925896-6 2024-09-18T10:14:53.312 INFO:tasks.rados.rados.0.smithi189.stdout:72: finishing rollback tid 1 to smithi18925896-6 2024-09-18T10:14:53.332 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-18T10:14:53.326+0000 7f1981873700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22132) UID: 0 2024-09-18T10:14:53.333 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-18T10:14:53.333 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-18T10:14:53.333 INFO:teuthology.orchestra.run.smithi189.stdout: "num_ops": 0 2024-09-18T10:14:53.333 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-18T10:14:53.342 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-09-18T10:14:53.378 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-18T10:14:53.404 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-18T10:14:53.407 INFO:tasks.rados.rados.0.smithi189.stdout:75: finishing copy_from to smithi18925896-48 2024-09-18T10:14:53.407 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 48 v 2 (ObjNum 0 snap 0 seq_num 0) dirty exists 2024-09-18T10:14:53.418 INFO:tasks.rados.rados.0.smithi189.stdout:66: finishing write tid 4 to smithi18925896-23 2024-09-18T10:14:53.418 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 23 v 15 (ObjNum 53 snap 1 seq_num 53) dirty exists 2024-09-18T10:14:53.418 INFO:tasks.rados.rados.0.smithi189.stdout:66: left oid 23 (ObjNum 53 snap 1 seq_num 53) 2024-09-18T10:14:53.429 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-18T10:14:53.433 INFO:tasks.ceph.osd.10.smithi154.stderr:2024-09-18T10:14:53.428+0000 7fb73221a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 22091) UID: 0 2024-09-18T10:14:53.440 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-18T10:14:53.440 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-18T10:14:53.440 INFO:teuthology.orchestra.run.smithi189.stdout: "complaint_time": 30, 2024-09-18T10:14:53.440 INFO:teuthology.orchestra.run.smithi189.stdout: "num_blocked_ops": 0 2024-09-18T10:14:53.440 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-18T10:14:53.451 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-09-18T10:14:53.453 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-18T10:14:53.475 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-18T10:14:53.508 INFO:tasks.rados.rados.0.smithi189.stdout:77: finishing copy_from to smithi18925896-28 2024-09-18T10:14:53.508 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 28 v 6 (ObjNum 33 snap 0 seq_num 33) dirty exists 2024-09-18T10:14:53.514 INFO:tasks.rados.rados.0.smithi189.stdout:71: finishing copy_from racing read to smithi18925896-24 2024-09-18T10:14:53.534 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-18T10:14:53.530+0000 7f1981873700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22132) UID: 0 2024-09-18T10:14:53.540 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-18T10:14:53.549 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-18T10:14:53.549 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 20, 2024-09-18T10:14:53.549 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 600, 2024-09-18T10:14:53.549 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [ 2024-09-18T10:14:53.549 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.550 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:146 3.es0 3:78aad6f5:::smithi18925896-49:head [append 0~442368,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:14:53.550 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:51.117496+0000", 2024-09-18T10:14:53.550 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 2.4296043840000001, 2024-09-18T10:14:53.550 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.61437150200000001, 2024-09-18T10:14:53.550 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:14:53.550 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:14:53.550 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:14:53.550 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:14:53.550 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:14:53.550 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 146 2024-09-18T10:14:53.550 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.550 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:14:53.550 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.551 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:14:53.551 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.117496+0000", 2024-09-18T10:14:53.551 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:53.551 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.552 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.552 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:14:53.552 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.117496+0000", 2024-09-18T10:14:53.552 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:53.552 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.552 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.552 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:14:53.552 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.117493+0000", 2024-09-18T10:14:53.552 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999976 2024-09-18T10:14:53.552 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.552 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.552 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:14:53.552 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.117787+0000", 2024-09-18T10:14:53.553 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00029378100000000001 2024-09-18T10:14:53.553 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.553 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.553 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:14:53.553 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.117820+0000", 2024-09-18T10:14:53.553 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3073e-05 2024-09-18T10:14:53.553 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.553 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.553 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:14:53.553 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.117834+0000", 2024-09-18T10:14:53.553 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3369e-05 2024-09-18T10:14:53.553 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.553 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.554 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:14:53.554 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.127090+0000", 2024-09-18T10:14:53.554 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0092563769999999997 2024-09-18T10:14:53.554 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.554 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.554 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:14:53.554 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.127435+0000", 2024-09-18T10:14:53.554 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00034522300000000003 2024-09-18T10:14:53.554 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.554 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.554 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:14:53.554 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.129115+0000", 2024-09-18T10:14:53.554 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0016792980000000001 2024-09-18T10:14:53.554 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.555 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.555 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:14:53.555 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.430505+0000", 2024-09-18T10:14:53.555 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.30139002300000001 2024-09-18T10:14:53.555 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.555 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.555 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:14:53.555 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.731753+0000", 2024-09-18T10:14:53.555 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.30124798800000002 2024-09-18T10:14:53.555 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.555 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.555 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:14:53.556 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.731867+0000", 2024-09-18T10:14:53.556 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000114764 2024-09-18T10:14:53.556 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:53.556 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:14:53.556 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:53.556 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.556 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.556 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:147 3.es0 3:78aad6f5:::smithi18925896-49:head [append 0~376832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:14:53.556 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:51.122092+0000", 2024-09-18T10:14:53.556 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 2.4250079659999999, 2024-09-18T10:14:53.556 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.61047882200000003, 2024-09-18T10:14:53.556 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:14:53.556 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:14:53.556 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:14:53.557 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:14:53.557 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:14:53.557 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 147 2024-09-18T10:14:53.557 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.557 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:14:53.557 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.557 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:14:53.557 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122092+0000", 2024-09-18T10:14:53.557 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:53.557 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.557 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.557 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:14:53.557 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122092+0000", 2024-09-18T10:14:53.558 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:53.558 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.558 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.558 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:14:53.558 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122091+0000", 2024-09-18T10:14:53.558 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:14:53.558 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.558 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.558 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:14:53.559 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122504+0000", 2024-09-18T10:14:53.559 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00041322100000000002 2024-09-18T10:14:53.559 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.559 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.559 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:14:53.559 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122527+0000", 2024-09-18T10:14:53.559 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3504999999999999e-05 2024-09-18T10:14:53.559 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.559 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.559 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:14:53.559 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122537+0000", 2024-09-18T10:14:53.559 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.8460000000000003e-06 2024-09-18T10:14:53.560 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.560 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.560 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:14:53.560 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.327415+0000", 2024-09-18T10:14:53.560 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20487799700000001 2024-09-18T10:14:53.560 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.560 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.560 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:14:53.560 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.327614+0000", 2024-09-18T10:14:53.560 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00019855400000000001 2024-09-18T10:14:53.560 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.560 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.560 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:14:53.560 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.328920+0000", 2024-09-18T10:14:53.561 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001306263 2024-09-18T10:14:53.561 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.561 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.561 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:14:53.561 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.430570+0000", 2024-09-18T10:14:53.561 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.101650508 2024-09-18T10:14:53.561 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.561 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.561 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:14:53.561 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.732477+0000", 2024-09-18T10:14:53.561 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.30190700100000001 2024-09-18T10:14:53.561 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.561 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.562 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:14:53.562 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.732571+0000", 2024-09-18T10:14:53.562 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.3580999999999997e-05 2024-09-18T10:14:53.562 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:53.562 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:14:53.562 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:53.562 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.562 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.562 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:148 3.es0 3:78aad6f5:::smithi18925896-49:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:14:53.562 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:51.122547+0000", 2024-09-18T10:14:53.562 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 2.4245528740000002, 2024-09-18T10:14:53.562 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.91462763400000002, 2024-09-18T10:14:53.562 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:14:53.562 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:14:53.562 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:14:53.563 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:14:53.563 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:14:53.563 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 148 2024-09-18T10:14:53.563 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.563 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:14:53.563 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.563 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:14:53.563 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122547+0000", 2024-09-18T10:14:53.563 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:53.563 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.563 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.563 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:14:53.563 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122547+0000", 2024-09-18T10:14:53.563 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:53.564 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.564 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.564 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:14:53.564 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122546+0000", 2024-09-18T10:14:53.564 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:14:53.564 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.564 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.564 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:14:53.564 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122552+0000", 2024-09-18T10:14:53.564 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3569999999999997e-06 2024-09-18T10:14:53.564 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.564 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.564 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:14:53.564 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122560+0000", 2024-09-18T10:14:53.565 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.191e-06 2024-09-18T10:14:53.565 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.565 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.565 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:14:53.565 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122569+0000", 2024-09-18T10:14:53.565 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.8230000000000001e-06 2024-09-18T10:14:53.565 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.565 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.565 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:14:53.565 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.834492+0000", 2024-09-18T10:14:53.565 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.71192345599999995 2024-09-18T10:14:53.565 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.565 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.566 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:14:53.566 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.834669+0000", 2024-09-18T10:14:53.566 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000177035 2024-09-18T10:14:53.566 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.566 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.566 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:14:53.566 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.835442+0000", 2024-09-18T10:14:53.566 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00077327799999999996 2024-09-18T10:14:53.566 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.566 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.566 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:14:53.566 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.936574+0000", 2024-09-18T10:14:53.566 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.101131163 2024-09-18T10:14:53.566 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.567 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.567 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:14:53.567 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.037028+0000", 2024-09-18T10:14:53.567 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100454026 2024-09-18T10:14:53.567 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.567 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.567 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:14:53.567 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.037175+0000", 2024-09-18T10:14:53.567 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000147305 2024-09-18T10:14:53.567 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:53.567 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:14:53.567 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:53.567 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.567 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.567 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:149 3.es0 3:78aad6f5:::smithi18925896-49:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:14:53.568 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:51.122575+0000", 2024-09-18T10:14:53.568 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 2.4245256660000001, 2024-09-18T10:14:53.568 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.016813872, 2024-09-18T10:14:53.568 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:14:53.568 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:14:53.568 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:14:53.568 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:14:53.568 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:14:53.568 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 149 2024-09-18T10:14:53.568 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.569 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:14:53.569 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.569 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:14:53.569 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122575+0000", 2024-09-18T10:14:53.569 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:53.569 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.569 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.569 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:14:53.569 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122575+0000", 2024-09-18T10:14:53.569 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:53.569 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.569 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.569 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:14:53.569 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122574+0000", 2024-09-18T10:14:53.570 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:14:53.570 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.570 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.570 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:14:53.570 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122579+0000", 2024-09-18T10:14:53.570 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3650000000000003e-06 2024-09-18T10:14:53.570 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.570 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.570 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:14:53.570 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122587+0000", 2024-09-18T10:14:53.570 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.3000000000000002e-06 2024-09-18T10:14:53.570 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.570 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.570 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:14:53.571 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122592+0000", 2024-09-18T10:14:53.571 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9080000000000003e-06 2024-09-18T10:14:53.571 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.571 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.571 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:14:53.571 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.935931+0000", 2024-09-18T10:14:53.571 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.81333878199999998 2024-09-18T10:14:53.571 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.571 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.571 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for rw locks", 2024-09-18T10:14:53.571 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.936216+0000", 2024-09-18T10:14:53.571 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00028463400000000003 2024-09-18T10:14:53.571 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.572 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.572 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:14:53.572 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.038007+0000", 2024-09-18T10:14:53.572 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.101791539 2024-09-18T10:14:53.572 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.572 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.572 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:14:53.572 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.038254+0000", 2024-09-18T10:14:53.572 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000246932 2024-09-18T10:14:53.572 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.572 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.572 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:14:53.572 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.139388+0000", 2024-09-18T10:14:53.572 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10113417199999999 2024-09-18T10:14:53.573 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:53.573 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:14:53.573 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:53.573 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.573 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.573 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.es0 36/32 ECSubWriteReply(tid=2, last_complete=36'2, committed=1, applied=1))", 2024-09-18T10:14:53.573 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:51.126519+0000", 2024-09-18T10:14:53.573 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 2.4205810030000001, 2024-09-18T10:14:53.573 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.30398819900000001, 2024-09-18T10:14:53.573 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:14:53.573 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:14:53.573 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:14:53.573 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.573 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:14:53.573 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.126519+0000", 2024-09-18T10:14:53.574 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:53.574 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.574 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.574 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:14:53.574 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.126519+0000", 2024-09-18T10:14:53.574 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:53.574 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.574 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.574 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:14:53.574 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.126518+0000", 2024-09-18T10:14:53.574 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:14:53.574 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.574 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.574 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:14:53.575 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.126523+0000", 2024-09-18T10:14:53.575 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5680000000000001e-06 2024-09-18T10:14:53.575 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.575 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.575 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:14:53.575 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.126541+0000", 2024-09-18T10:14:53.575 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7890999999999999e-05 2024-09-18T10:14:53.575 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.575 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.575 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:14:53.575 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.226614+0000", 2024-09-18T10:14:53.575 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100072972 2024-09-18T10:14:53.575 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.575 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.575 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:14:53.576 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.430483+0000", 2024-09-18T10:14:53.576 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.203869508 2024-09-18T10:14:53.576 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.576 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.576 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:14:53.576 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.430507+0000", 2024-09-18T10:14:53.576 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3986e-05 2024-09-18T10:14:53.576 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:53.576 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:14:53.576 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:53.576 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.576 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.576 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpRead(3.es0 36/32 ECSubRead(tid=3, to_read={3:7255ddf4:::smithi18925896-39:head=0,4096,0}, subchunks={3:7255ddf4:::smithi18925896-39:head=[0,1]}, attrs_to_read=))", 2024-09-18T10:14:53.576 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:51.126838+0000", 2024-09-18T10:14:53.577 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 2.4202622429999998, 2024-09-18T10:14:53.577 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.707585825, 2024-09-18T10:14:53.577 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:14:53.577 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:14:53.577 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:14:53.577 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.577 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:14:53.577 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.126838+0000", 2024-09-18T10:14:53.577 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:53.577 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.577 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.577 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:14:53.577 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.126838+0000", 2024-09-18T10:14:53.577 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:53.577 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.578 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.578 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:14:53.578 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.126838+0000", 2024-09-18T10:14:53.578 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:53.578 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.578 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.578 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:14:53.578 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.126838+0000", 2024-09-18T10:14:53.578 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:53.578 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.578 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.579 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:14:53.579 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.127004+0000", 2024-09-18T10:14:53.579 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00016626499999999999 2024-09-18T10:14:53.579 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.579 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.579 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:14:53.579 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.227100+0000", 2024-09-18T10:14:53.579 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.1000955 2024-09-18T10:14:53.579 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.579 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.579 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:14:53.579 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.834083+0000", 2024-09-18T10:14:53.579 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.60698300100000002 2024-09-18T10:14:53.579 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.580 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.580 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:14:53.580 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.834424+0000", 2024-09-18T10:14:53.580 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000341059 2024-09-18T10:14:53.580 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:53.580 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:14:53.580 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:53.580 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.580 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.580 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpReadReply(3.es0 36/32 ECSubReadReply(tid=3, attrs_read=0))", 2024-09-18T10:14:53.580 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:51.214895+0000", 2024-09-18T10:14:53.580 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 2.3322055659999998, 2024-09-18T10:14:53.580 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.61857800100000004, 2024-09-18T10:14:53.581 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:14:53.581 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:14:53.581 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:14:53.581 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.581 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:14:53.581 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.214895+0000", 2024-09-18T10:14:53.581 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:53.581 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.581 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.581 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:14:53.581 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.214895+0000", 2024-09-18T10:14:53.581 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:53.581 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.581 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.582 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:14:53.582 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.214893+0000", 2024-09-18T10:14:53.582 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:14:53.582 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.582 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.582 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:14:53.582 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.214927+0000", 2024-09-18T10:14:53.582 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3841999999999999e-05 2024-09-18T10:14:53.582 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.582 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.582 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:14:53.582 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.214953+0000", 2024-09-18T10:14:53.582 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.6519000000000001e-05 2024-09-18T10:14:53.582 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.582 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.583 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:14:53.583 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.214971+0000", 2024-09-18T10:14:53.583 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7535999999999999e-05 2024-09-18T10:14:53.583 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.583 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.583 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:14:53.583 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.833397+0000", 2024-09-18T10:14:53.583 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.61842642599999997 2024-09-18T10:14:53.583 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.583 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.583 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:14:53.583 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.833473+0000", 2024-09-18T10:14:53.583 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.5621e-05 2024-09-18T10:14:53.583 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:53.583 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:14:53.584 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:53.584 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.584 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.584 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.es0 36/32 ECSubWriteReply(tid=2, last_complete=36'2, committed=1, applied=1))", 2024-09-18T10:14:53.584 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:51.215776+0000", 2024-09-18T10:14:53.584 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 2.3313244439999998, 2024-09-18T10:14:53.584 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.214797881, 2024-09-18T10:14:53.584 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:14:53.584 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:14:53.584 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:14:53.584 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.584 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:14:53.584 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.215776+0000", 2024-09-18T10:14:53.584 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:53.584 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.585 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.585 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:14:53.585 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.215776+0000", 2024-09-18T10:14:53.585 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:53.585 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.585 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.585 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:14:53.585 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.215774+0000", 2024-09-18T10:14:53.585 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:14:53.585 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.585 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.585 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:14:53.585 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.215788+0000", 2024-09-18T10:14:53.585 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3896e-05 2024-09-18T10:14:53.586 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.586 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.586 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:14:53.586 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.215828+0000", 2024-09-18T10:14:53.586 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9983e-05 2024-09-18T10:14:53.586 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.586 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.586 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:14:53.586 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.315912+0000", 2024-09-18T10:14:53.586 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10008360299999999 2024-09-18T10:14:53.586 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.586 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.586 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:14:53.586 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.430545+0000", 2024-09-18T10:14:53.586 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.1146334 2024-09-18T10:14:53.587 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.587 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.587 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:14:53.587 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.430574+0000", 2024-09-18T10:14:53.587 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.8711999999999999e-05 2024-09-18T10:14:53.587 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:53.587 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:14:53.587 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:53.587 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.587 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.587 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.es0 36/32 ECSubWriteReply(tid=2, last_complete=36'2, committed=1, applied=1))", 2024-09-18T10:14:53.587 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:51.226907+0000", 2024-09-18T10:14:53.587 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 2.3201927439999999, 2024-09-18T10:14:53.587 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.303948039, 2024-09-18T10:14:53.587 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:14:53.588 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:14:53.588 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:14:53.588 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.588 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:14:53.588 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.226907+0000", 2024-09-18T10:14:53.588 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:53.588 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.588 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.588 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:14:53.588 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.226907+0000", 2024-09-18T10:14:53.588 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:53.588 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.588 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.589 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:14:53.589 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.226907+0000", 2024-09-18T10:14:53.589 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:14:53.589 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.589 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.589 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:14:53.589 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.226912+0000", 2024-09-18T10:14:53.589 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3499999999999996e-06 2024-09-18T10:14:53.589 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.589 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.589 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:14:53.589 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.226925+0000", 2024-09-18T10:14:53.590 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3346000000000001e-05 2024-09-18T10:14:53.590 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.590 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.590 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:14:53.590 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.226931+0000", 2024-09-18T10:14:53.590 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.6729999999999997e-06 2024-09-18T10:14:53.590 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.590 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.590 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:14:53.590 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.530834+0000", 2024-09-18T10:14:53.590 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.30390318799999999 2024-09-18T10:14:53.590 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.590 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.591 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:14:53.591 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.530856+0000", 2024-09-18T10:14:53.591 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1209e-05 2024-09-18T10:14:53.591 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:53.591 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:14:53.591 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:53.591 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.591 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.591 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.9s3 36/32 ECSubWrite(tid=17, reqid=client.4632.0:144, at_version=36'13, trim_to=0'0, roll_forward_to=36'9))", 2024-09-18T10:14:53.591 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:51.227217+0000", 2024-09-18T10:14:53.591 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 2.3198834979999998, 2024-09-18T10:14:53.591 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20878022600000001, 2024-09-18T10:14:53.591 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:14:53.591 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:14:53.592 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:14:53.592 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.592 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:14:53.592 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.227217+0000", 2024-09-18T10:14:53.592 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:53.592 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.592 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.592 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:14:53.592 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.227217+0000", 2024-09-18T10:14:53.592 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:53.592 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.592 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.592 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:14:53.592 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.227216+0000", 2024-09-18T10:14:53.593 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999995 2024-09-18T10:14:53.593 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.593 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.593 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:14:53.593 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.227231+0000", 2024-09-18T10:14:53.593 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5186e-05 2024-09-18T10:14:53.593 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.593 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.593 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:14:53.593 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.227239+0000", 2024-09-18T10:14:53.593 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.7330000000000003e-06 2024-09-18T10:14:53.593 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.593 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.593 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:14:53.594 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.227245+0000", 2024-09-18T10:14:53.594 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.1090000000000003e-06 2024-09-18T10:14:53.594 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.594 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.594 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:14:53.594 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.335110+0000", 2024-09-18T10:14:53.594 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10786511999999999 2024-09-18T10:14:53.594 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.594 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.594 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:14:53.594 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.335126+0000", 2024-09-18T10:14:53.594 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5403000000000001e-05 2024-09-18T10:14:53.594 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.594 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.595 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:14:53.595 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.435957+0000", 2024-09-18T10:14:53.595 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100831078 2024-09-18T10:14:53.595 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.595 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.595 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:14:53.595 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.435997+0000", 2024-09-18T10:14:53.595 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.0028999999999999e-05 2024-09-18T10:14:53.595 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:53.595 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:14:53.595 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:53.595 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.595 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.595 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpReadReply(3.es0 36/32 ECSubReadReply(tid=3, attrs_read=0))", 2024-09-18T10:14:53.596 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:51.227596+0000", 2024-09-18T10:14:53.596 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 2.3195040150000001, 2024-09-18T10:14:53.596 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.60597977400000003, 2024-09-18T10:14:53.596 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:14:53.596 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:14:53.596 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:14:53.596 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.596 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:14:53.596 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.227596+0000", 2024-09-18T10:14:53.596 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:53.596 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.596 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.596 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:14:53.597 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.227596+0000", 2024-09-18T10:14:53.597 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:53.597 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.597 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.597 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:14:53.597 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.227595+0000", 2024-09-18T10:14:53.597 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:14:53.597 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.597 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.597 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:14:53.597 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.227605+0000", 2024-09-18T10:14:53.597 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.7040000000000006e-06 2024-09-18T10:14:53.597 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.597 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.598 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:14:53.598 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.227621+0000", 2024-09-18T10:14:53.598 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6107999999999999e-05 2024-09-18T10:14:53.598 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.598 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.598 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:14:53.598 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.227637+0000", 2024-09-18T10:14:53.598 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5282999999999999e-05 2024-09-18T10:14:53.598 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.598 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.598 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:14:53.598 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.833511+0000", 2024-09-18T10:14:53.598 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.60587403699999998 2024-09-18T10:14:53.598 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.599 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.599 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:14:53.599 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.833576+0000", 2024-09-18T10:14:53.599 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.5392e-05 2024-09-18T10:14:53.599 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:53.599 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:14:53.599 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:53.599 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.599 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.599 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.es0 36/32 ECSubWriteReply(tid=4, last_complete=36'3, committed=1, applied=1))", 2024-09-18T10:14:53.599 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:51.228604+0000", 2024-09-18T10:14:53.599 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 2.318496187, 2024-09-18T10:14:53.599 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.30219310700000002, 2024-09-18T10:14:53.599 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:14:53.600 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:14:53.600 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:14:53.600 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.600 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:14:53.600 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.228604+0000", 2024-09-18T10:14:53.600 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:53.600 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.600 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.600 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:14:53.600 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.228604+0000", 2024-09-18T10:14:53.600 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:53.600 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.600 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.601 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:14:53.601 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.228589+0000", 2024-09-18T10:14:53.601 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999852 2024-09-18T10:14:53.601 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.601 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.601 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:14:53.601 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.228610+0000", 2024-09-18T10:14:53.601 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0676999999999999e-05 2024-09-18T10:14:53.601 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.601 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.601 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:14:53.601 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.228621+0000", 2024-09-18T10:14:53.601 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1154e-05 2024-09-18T10:14:53.601 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.602 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.602 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:14:53.602 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.228628+0000", 2024-09-18T10:14:53.602 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.4250000000000003e-06 2024-09-18T10:14:53.602 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.602 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.602 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:14:53.602 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.530770+0000", 2024-09-18T10:14:53.602 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.30214249799999998 2024-09-18T10:14:53.602 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.602 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.602 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:14:53.602 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.530797+0000", 2024-09-18T10:14:53.602 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7039e-05 2024-09-18T10:14:53.603 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:53.603 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:14:53.603 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:53.603 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.603 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.603 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.es0 36/32 ECSubWriteReply(tid=4, last_complete=36'3, committed=1, applied=1))", 2024-09-18T10:14:53.603 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:51.233227+0000", 2024-09-18T10:14:53.603 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 2.3138730010000002, 2024-09-18T10:14:53.603 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.39820466900000001, 2024-09-18T10:14:53.603 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:14:53.603 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:14:53.603 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:14:53.603 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.603 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:14:53.604 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.233227+0000", 2024-09-18T10:14:53.604 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:53.604 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.604 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.604 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:14:53.604 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.233227+0000", 2024-09-18T10:14:53.604 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:53.604 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.604 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.604 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:14:53.604 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.233226+0000", 2024-09-18T10:14:53.604 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:14:53.604 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.604 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.605 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:14:53.605 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.233232+0000", 2024-09-18T10:14:53.605 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9370000000000001e-06 2024-09-18T10:14:53.605 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.605 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.605 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:14:53.605 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.233248+0000", 2024-09-18T10:14:53.605 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5574999999999999e-05 2024-09-18T10:14:53.605 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.605 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.605 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:14:53.605 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.233254+0000", 2024-09-18T10:14:53.605 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.3659999999999997e-06 2024-09-18T10:14:53.605 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.606 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.606 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:14:53.606 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.631420+0000", 2024-09-18T10:14:53.606 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.39816591699999998 2024-09-18T10:14:53.606 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.606 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.606 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:14:53.606 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.631432+0000", 2024-09-18T10:14:53.606 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1737e-05 2024-09-18T10:14:53.606 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:53.606 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:14:53.606 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:53.606 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.606 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.607 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.es0 36/32 ECSubWriteReply(tid=4, last_complete=36'3, committed=1, applied=1))", 2024-09-18T10:14:53.607 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:51.315941+0000", 2024-09-18T10:14:53.607 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 2.2311587589999999, 2024-09-18T10:14:53.607 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.315333435, 2024-09-18T10:14:53.607 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:14:53.607 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:14:53.607 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:14:53.607 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.607 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:14:53.607 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.315941+0000", 2024-09-18T10:14:53.607 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:53.607 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.607 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.607 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:14:53.607 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.315941+0000", 2024-09-18T10:14:53.607 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:53.608 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.608 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.608 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:14:53.608 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.315940+0000", 2024-09-18T10:14:53.608 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:14:53.608 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.608 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.608 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:14:53.608 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.315950+0000", 2024-09-18T10:14:53.608 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.2329999999999995e-06 2024-09-18T10:14:53.608 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.608 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.608 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:14:53.608 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.315968+0000", 2024-09-18T10:14:53.609 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.8067000000000002e-05 2024-09-18T10:14:53.609 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.609 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.609 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:14:53.609 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.315977+0000", 2024-09-18T10:14:53.609 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.6420000000000002e-06 2024-09-18T10:14:53.609 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.609 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.609 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:14:53.609 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.631262+0000", 2024-09-18T10:14:53.609 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.31528474000000001 2024-09-18T10:14:53.609 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.609 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.609 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:14:53.610 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.631275+0000", 2024-09-18T10:14:53.610 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2799999999999999e-05 2024-09-18T10:14:53.610 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:53.610 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:14:53.610 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:53.610 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.610 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.610 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.es0 36/32 ECSubWriteReply(tid=4, last_complete=36'3, committed=1, applied=1))", 2024-09-18T10:14:53.610 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:51.329637+0000", 2024-09-18T10:14:53.610 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 2.2174628900000002, 2024-09-18T10:14:53.610 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.30156535499999998, 2024-09-18T10:14:53.610 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:14:53.610 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:14:53.610 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:14:53.611 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.611 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:14:53.611 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.329637+0000", 2024-09-18T10:14:53.611 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:53.611 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.611 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.611 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:14:53.611 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.329637+0000", 2024-09-18T10:14:53.611 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:53.611 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.611 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.611 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:14:53.611 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.329636+0000", 2024-09-18T10:14:53.611 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:14:53.611 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.612 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.612 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:14:53.612 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.329648+0000", 2024-09-18T10:14:53.612 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2612000000000001e-05 2024-09-18T10:14:53.612 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.612 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.612 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:14:53.612 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.329678+0000", 2024-09-18T10:14:53.612 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.9369e-05 2024-09-18T10:14:53.612 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.612 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.612 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:14:53.612 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.329706+0000", 2024-09-18T10:14:53.612 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.8073e-05 2024-09-18T10:14:53.613 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.613 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.613 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:14:53.613 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.631181+0000", 2024-09-18T10:14:53.613 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.30147523799999998 2024-09-18T10:14:53.613 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.613 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.613 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:14:53.613 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.631203+0000", 2024-09-18T10:14:53.613 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1795999999999999e-05 2024-09-18T10:14:53.613 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:53.613 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:14:53.613 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:53.613 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.614 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.614 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.es0 36/32 ECSubWriteReply(tid=5, last_complete=36'4, committed=1, applied=1))", 2024-09-18T10:14:53.614 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:51.331048+0000", 2024-09-18T10:14:53.614 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 2.2160525739999999, 2024-09-18T10:14:53.614 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40123123900000002, 2024-09-18T10:14:53.614 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:14:53.614 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:14:53.614 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:14:53.614 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.614 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:14:53.614 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.331048+0000", 2024-09-18T10:14:53.614 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:53.614 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.614 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.615 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:14:53.615 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.331048+0000", 2024-09-18T10:14:53.615 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:53.615 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.615 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.615 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:14:53.615 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.331046+0000", 2024-09-18T10:14:53.615 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:14:53.615 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.615 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.615 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:14:53.615 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.331057+0000", 2024-09-18T10:14:53.615 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1683e-05 2024-09-18T10:14:53.615 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.616 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.616 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:14:53.616 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.331090+0000", 2024-09-18T10:14:53.616 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2576000000000001e-05 2024-09-18T10:14:53.616 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.616 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.616 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:14:53.616 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.331107+0000", 2024-09-18T10:14:53.616 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7382999999999999e-05 2024-09-18T10:14:53.616 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.616 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.616 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:14:53.616 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.732241+0000", 2024-09-18T10:14:53.617 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40113375200000001 2024-09-18T10:14:53.617 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.617 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.617 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:14:53.617 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.732279+0000", 2024-09-18T10:14:53.617 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7719000000000003e-05 2024-09-18T10:14:53.617 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:53.617 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:14:53.617 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:53.617 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.617 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.617 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.es0 36/32 ECSubWriteReply(tid=5, last_complete=36'4, committed=1, applied=1))", 2024-09-18T10:14:53.617 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:51.331214+0000", 2024-09-18T10:14:53.617 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 2.21588618, 2024-09-18T10:14:53.618 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40084109800000001, 2024-09-18T10:14:53.618 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:14:53.618 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:14:53.618 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:14:53.618 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.618 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:14:53.618 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.331214+0000", 2024-09-18T10:14:53.618 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:53.618 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.618 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.618 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:14:53.618 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.331214+0000", 2024-09-18T10:14:53.618 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:53.618 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.619 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.619 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:14:53.619 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.331213+0000", 2024-09-18T10:14:53.619 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:14:53.619 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.619 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.619 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:14:53.619 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.331220+0000", 2024-09-18T10:14:53.619 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.1369999999999996e-06 2024-09-18T10:14:53.619 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.619 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.619 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:14:53.620 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.331235+0000", 2024-09-18T10:14:53.620 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4849e-05 2024-09-18T10:14:53.620 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.620 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.620 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:14:53.620 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.331243+0000", 2024-09-18T10:14:53.620 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.157e-06 2024-09-18T10:14:53.620 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.620 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.620 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:14:53.620 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.732010+0000", 2024-09-18T10:14:53.620 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40076711300000001 2024-09-18T10:14:53.620 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.621 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.621 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:14:53.621 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.732055+0000", 2024-09-18T10:14:53.621 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4821999999999999e-05 2024-09-18T10:14:53.621 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:53.621 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:14:53.621 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:53.621 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.621 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.621 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.es0 36/32 ECSubWriteReply(tid=5, last_complete=36'4, committed=1, applied=1))", 2024-09-18T10:14:53.621 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:51.517145+0000", 2024-09-18T10:14:53.621 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 2.029955443, 2024-09-18T10:14:53.621 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.21501588799999999, 2024-09-18T10:14:53.621 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:14:53.622 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:14:53.622 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:14:53.622 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.622 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:14:53.622 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.517145+0000", 2024-09-18T10:14:53.622 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:53.622 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.622 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.622 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:14:53.622 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.517145+0000", 2024-09-18T10:14:53.622 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:53.622 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.622 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.622 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:14:53.623 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.517144+0000", 2024-09-18T10:14:53.623 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:14:53.623 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.623 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.623 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:14:53.623 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.517153+0000", 2024-09-18T10:14:53.623 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.7990000000000005e-06 2024-09-18T10:14:53.623 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.623 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.623 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:14:53.623 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.517178+0000", 2024-09-18T10:14:53.623 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4783e-05 2024-09-18T10:14:53.623 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.623 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.623 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:14:53.624 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.517193+0000", 2024-09-18T10:14:53.624 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5203e-05 2024-09-18T10:14:53.624 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.624 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.624 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:14:53.624 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.732142+0000", 2024-09-18T10:14:53.624 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.21494856600000001 2024-09-18T10:14:53.624 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.624 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.624 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:14:53.624 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.732161+0000", 2024-09-18T10:14:53.624 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.8776000000000001e-05 2024-09-18T10:14:53.624 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:53.625 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:14:53.625 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:53.625 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.625 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.625 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.es0 36/32 ECSubWriteReply(tid=5, last_complete=36'4, committed=1, applied=1))", 2024-09-18T10:14:53.625 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:51.527659+0000", 2024-09-18T10:14:53.625 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 2.0194414279999999, 2024-09-18T10:14:53.625 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20560199800000001, 2024-09-18T10:14:53.625 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:14:53.625 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:14:53.626 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:14:53.626 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.626 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:14:53.626 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.527659+0000", 2024-09-18T10:14:53.626 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:53.626 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.626 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.626 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:14:53.626 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.527659+0000", 2024-09-18T10:14:53.626 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:53.626 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.626 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.626 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:14:53.626 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.527657+0000", 2024-09-18T10:14:53.627 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:14:53.627 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.627 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.627 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:14:53.627 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.527672+0000", 2024-09-18T10:14:53.627 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4307e-05 2024-09-18T10:14:53.627 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.627 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.627 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:14:53.627 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.527703+0000", 2024-09-18T10:14:53.627 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1461999999999998e-05 2024-09-18T10:14:53.627 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.627 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.628 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:14:53.628 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.527715+0000", 2024-09-18T10:14:53.628 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1875999999999999e-05 2024-09-18T10:14:53.628 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.628 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.628 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:14:53.628 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.732365+0000", 2024-09-18T10:14:53.628 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.204650475 2024-09-18T10:14:53.628 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.628 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.628 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:14:53.628 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.733261+0000", 2024-09-18T10:14:53.628 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00089545200000000003 2024-09-18T10:14:53.628 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:53.628 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:14:53.629 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:53.629 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.629 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.629 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpReadReply(3.es0 36/32 ECSubReadReply(tid=3, attrs_read=0))", 2024-09-18T10:14:53.629 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:51.834406+0000", 2024-09-18T10:14:53.629 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 1.712694704, 2024-09-18T10:14:53.629 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.205083404, 2024-09-18T10:14:53.629 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:14:53.629 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:14:53.629 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:14:53.629 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.629 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:14:53.629 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.834406+0000", 2024-09-18T10:14:53.630 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:53.630 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.630 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.630 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:14:53.630 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.834406+0000", 2024-09-18T10:14:53.630 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:53.630 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.630 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.630 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:14:53.630 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.834406+0000", 2024-09-18T10:14:53.630 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:53.630 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.630 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.630 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:14:53.631 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.834406+0000", 2024-09-18T10:14:53.631 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:14:53.631 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.631 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.631 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:14:53.631 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.834445+0000", 2024-09-18T10:14:53.631 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9564999999999998e-05 2024-09-18T10:14:53.631 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.631 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.631 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:14:53.631 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.834455+0000", 2024-09-18T10:14:53.631 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0316000000000001e-05 2024-09-18T10:14:53.631 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.631 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.631 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:14:53.632 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.038919+0000", 2024-09-18T10:14:53.632 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20446325300000001 2024-09-18T10:14:53.632 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:14:53.632 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:14:53.632 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:14:53.632 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.039489+0000", 2024-09-18T10:14:53.632 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00057027 2024-09-18T10:14:53.632 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:53.632 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:14:53.632 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:53.632 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:14:53.632 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:14:53.632 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-18T10:14:53.634 INFO:teuthology.orchestra.run.smithi029.stdout:osd.6: {} 2024-09-18T10:14:53.634 INFO:teuthology.orchestra.run.smithi029.stdout:osd.7: {} 2024-09-18T10:14:53.634 INFO:teuthology.orchestra.run.smithi029.stdout:osd.8: {} 2024-09-18T10:14:53.634 INFO:teuthology.orchestra.run.smithi029.stdout:osd.9: {} 2024-09-18T10:14:53.635 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_ops_in_flight 2024-09-18T10:14:53.635 INFO:tasks.rados.rados.0.smithi189.stdout:72: finishing rollback tid 2 to smithi18925896-6 2024-09-18T10:14:53.635 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 6 v 19 (ObjNum 5 snap 0 seq_num 5) dirty exists 2024-09-18T10:14:53.635 INFO:tasks.rados.rados.0.smithi189.stdout:71: finishing copy_from to smithi18925896-24 2024-09-18T10:14:53.635 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 24 v 6 (ObjNum 28 snap 0 seq_num 28) dirty exists 2024-09-18T10:14:53.636 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-18T10:14:53.629+0000 7f14c6a1a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 22138) UID: 0 2024-09-18T10:14:53.651 INFO:teuthology.orchestra.run.smithi029.stdout:osd.10: {} 2024-09-18T10:14:53.672 INFO:teuthology.orchestra.run.smithi029.stdout:osd.11: {} 2024-09-18T10:14:53.693 INFO:teuthology.orchestra.run.smithi029.stdout:osd.12: {} 2024-09-18T10:14:53.700 INFO:tasks.rados.rados.0.smithi189.stdout:64: done (15 left) 2024-09-18T10:14:53.700 INFO:tasks.rados.rados.0.smithi189.stdout:65: done (14 left) 2024-09-18T10:14:53.700 INFO:tasks.rados.rados.0.smithi189.stdout:66: done (13 left) 2024-09-18T10:14:53.700 INFO:tasks.rados.rados.0.smithi189.stdout:67: done (12 left) 2024-09-18T10:14:53.700 INFO:tasks.rados.rados.0.smithi189.stdout:68: done (11 left) 2024-09-18T10:14:53.700 INFO:tasks.rados.rados.0.smithi189.stdout:69: done (10 left) 2024-09-18T10:14:53.700 INFO:tasks.rados.rados.0.smithi189.stdout:70: done (9 left) 2024-09-18T10:14:53.700 INFO:tasks.rados.rados.0.smithi189.stdout:71: done (8 left) 2024-09-18T10:14:53.700 INFO:tasks.rados.rados.0.smithi189.stdout:72: done (7 left) 2024-09-18T10:14:53.701 INFO:tasks.rados.rados.0.smithi189.stdout:73: done (6 left) 2024-09-18T10:14:53.701 INFO:tasks.rados.rados.0.smithi189.stdout:75: done (5 left) 2024-09-18T10:14:53.701 INFO:tasks.rados.rados.0.smithi189.stdout:76: done (4 left) 2024-09-18T10:14:53.701 INFO:tasks.rados.rados.0.smithi189.stdout:77: done (3 left) 2024-09-18T10:14:53.701 INFO:tasks.rados.rados.0.smithi189.stdout:78: done (2 left) 2024-09-18T10:14:53.701 INFO:tasks.rados.rados.0.smithi189.stdout:79: done (1 left) 2024-09-18T10:14:53.701 INFO:tasks.rados.rados.0.smithi189.stdout:80: done (0 left) 2024-09-18T10:14:53.701 INFO:tasks.rados.rados.0.smithi189.stdout:81: rollback oid 40 current snap is 2 2024-09-18T10:14:53.701 INFO:tasks.rados.rados.0.smithi189.stdout:rollback oid 40 to 1 2024-09-18T10:14:53.701 INFO:tasks.rados.rados.0.smithi189.stdout:82: snap_create 2024-09-18T10:14:53.705 INFO:teuthology.orchestra.run.smithi029.stderr:set pool 3 pg_num to 26 2024-09-18T10:14:53.716 INFO:teuthology.orchestra.run.smithi029.stdout:osd.13: {} 2024-09-18T10:14:53.738 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-18T10:14:53.733+0000 7f14c6a1a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 22138) UID: 0 2024-09-18T10:14:53.739 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-18T10:14:53.740 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-18T10:14:53.740 INFO:teuthology.orchestra.run.smithi189.stdout: "num_ops": 0 2024-09-18T10:14:53.740 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-18T10:14:53.749 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_blocked_ops 2024-09-18T10:14:53.827 INFO:teuthology.orchestra.run.smithi029.stdout:osd.14: {} 2024-09-18T10:14:53.839 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-18T10:14:53.833+0000 7f00046a8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 22136) UID: 0 2024-09-18T10:14:53.846 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-18T10:14:53.846 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-18T10:14:53.846 INFO:teuthology.orchestra.run.smithi189.stdout: "complaint_time": 30, 2024-09-18T10:14:53.846 INFO:teuthology.orchestra.run.smithi189.stdout: "num_blocked_ops": 0 2024-09-18T10:14:53.846 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-18T10:14:53.850 INFO:teuthology.orchestra.run.smithi029.stdout:osd.15: {} 2024-09-18T10:14:53.852 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T10:14:53.853 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T10:14:53.853 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T10:14:53.853 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T10:14:53.853 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T10:14:53.853 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T10:14:53.853 INFO:teuthology.orchestra.run.smithi029.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T10:14:53.853 INFO:teuthology.orchestra.run.smithi029.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T10:14:53.853 INFO:teuthology.orchestra.run.smithi029.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-18T10:14:53.853 INFO:teuthology.orchestra.run.smithi029.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-18T10:14:53.853 INFO:teuthology.orchestra.run.smithi029.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-18T10:14:53.853 INFO:teuthology.orchestra.run.smithi029.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-18T10:14:53.853 INFO:teuthology.orchestra.run.smithi029.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-18T10:14:53.853 INFO:teuthology.orchestra.run.smithi029.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-18T10:14:53.853 INFO:teuthology.orchestra.run.smithi029.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-18T10:14:53.854 INFO:teuthology.orchestra.run.smithi029.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-18T10:14:53.856 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_historic_ops 2024-09-18T10:14:53.913 INFO:tasks.rados.rados.0.smithi189.stdout:81: finishing rollback tid 0 to smithi18925896-40 2024-09-18T10:14:53.913 INFO:tasks.rados.rados.0.smithi189.stdout:81: finishing rollback tid 1 to smithi18925896-40 2024-09-18T10:14:53.939 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-18T10:14:53.933+0000 7f00046a8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 22136) UID: 0 2024-09-18T10:14:53.952 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-18T10:14:53.953 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:14:53.961 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T10:14:54.012 INFO:tasks.rados.rados.0.smithi189.stdout:81: finishing rollback tid 2 to smithi18925896-40 2024-09-18T10:14:54.012 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 40 v 9 (ObjNum 39 snap 0 seq_num 39) dirty exists 2024-09-18T10:14:54.041 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-18T10:14:54.039+0000 7f14dcc5b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22398) UID: 0 2024-09-18T10:14:54.076 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:14:54.076 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:14:54.086 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T10:14:54.142 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-18T10:14:54.137+0000 7f14c6a1a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 22138) UID: 0 2024-09-18T10:14:54.190 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:14:54.190 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:14:54.199 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T10:14:54.243 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-18T10:14:54.239+0000 7f14dcc5b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22398) UID: 0 2024-09-18T10:14:54.298 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:14:54.298 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:14:54.307 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T10:14:54.344 INFO:tasks.ceph.osd.3.smithi189.stderr:2024-09-18T10:14:54.339+0000 7f2440fc7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22396) UID: 0 2024-09-18T10:14:54.407 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:14:54.407 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:14:54.416 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T10:14:54.445 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-18T10:14:54.441+0000 7f1981873700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22132) UID: 0 2024-09-18T10:14:54.520 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:14:54.520 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:14:54.530 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T10:14:54.546 INFO:tasks.ceph.osd.8.smithi029.stderr:2024-09-18T10:14:54.542+0000 7f2555c27700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 22192) UID: 0 2024-09-18T10:14:54.625 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:14:54.625 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:14:54.633 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-09-18T10:14:54.647 INFO:tasks.ceph.osd.11.smithi189.stderr:2024-09-18T10:14:54.643+0000 7f718b60a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 22400) UID: 0 2024-09-18T10:14:54.708 INFO:tasks.rados.rados.0.smithi189.stdout:81: done (1 left) 2024-09-18T10:14:54.708 INFO:tasks.rados.rados.0.smithi189.stdout:82: done (0 left) 2024-09-18T10:14:54.708 INFO:tasks.rados.rados.0.smithi189.stdout:83: rollback oid 9 current snap is 3 2024-09-18T10:14:54.709 INFO:tasks.rados.rados.0.smithi189.stdout:rollback oid 9 to 0 2024-09-18T10:14:54.709 INFO:tasks.rados.rados.0.smithi189.stdout:84: read oid 20 snap 1 2024-09-18T10:14:54.709 INFO:tasks.rados.rados.0.smithi189.stdout:84: expect (ObjNum 19 snap 0 seq_num 19) 2024-09-18T10:14:54.709 INFO:tasks.rados.rados.0.smithi189.stdout:85: read oid 41 snap -1 2024-09-18T10:14:54.709 INFO:tasks.rados.rados.0.smithi189.stdout:85: expect (ObjNum 40 snap 0 seq_num 40) 2024-09-18T10:14:54.709 INFO:tasks.rados.rados.0.smithi189.stdout:86: read oid 5 snap -1 2024-09-18T10:14:54.710 INFO:tasks.rados.rados.0.smithi189.stdout:86: expect (ObjNum 4 snap 0 seq_num 4) 2024-09-18T10:14:54.710 INFO:tasks.rados.rados.0.smithi189.stdout:87: read oid 4 snap -1 2024-09-18T10:14:54.710 INFO:tasks.rados.rados.0.smithi189.stdout:87: expect (ObjNum 3 snap 0 seq_num 3) 2024-09-18T10:14:54.710 INFO:tasks.rados.rados.0.smithi189.stdout:append oid 3 current snap is 3 2024-09-18T10:14:54.710 INFO:tasks.rados.rados.0.smithi189.stdout:88: seq_num 54 ranges {0=540672,540672=376832} 2024-09-18T10:14:54.715 INFO:tasks.rados.rados.0.smithi189.stdout:88: writing smithi18925896-3 from 0 to 540672 tid 1 2024-09-18T10:14:54.719 INFO:tasks.rados.rados.0.smithi189.stdout:88: writing smithi18925896-3 from 540672 to 917504 tid 2 2024-09-18T10:14:54.719 INFO:tasks.rados.rados.0.smithi189.stdout:89: read oid 27 snap -1 2024-09-18T10:14:54.720 INFO:tasks.rados.rados.0.smithi189.stdout:89: expect (ObjNum 26 snap 0 seq_num 26) 2024-09-18T10:14:54.720 INFO:tasks.rados.rados.0.smithi189.stdout:append oid 17 current snap is 3 2024-09-18T10:14:54.720 INFO:tasks.rados.rados.0.smithi189.stdout:90: seq_num 55 ranges {278528=557056,835584=376832} 2024-09-18T10:14:54.720 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-18T10:14:54.726 INFO:tasks.rados.rados.0.smithi189.stdout:90: writing smithi18925896-17 from 278528 to 835584 tid 1 2024-09-18T10:14:54.728 INFO:tasks.rados.rados.0.smithi189.stdout:90: writing smithi18925896-17 from 835584 to 1212416 tid 2 2024-09-18T10:14:54.728 INFO:tasks.rados.rados.0.smithi189.stdout:91: rollback oid 36 current snap is 3 2024-09-18T10:14:54.729 INFO:tasks.rados.rados.0.smithi189.stdout:rollback oid 36 to 2 2024-09-18T10:14:54.729 INFO:tasks.rados.rados.0.smithi189.stdout:92: rollback oid 7 current snap is 3 2024-09-18T10:14:54.729 INFO:tasks.rados.rados.0.smithi189.stdout:rollback oid 7 to 0 2024-09-18T10:14:54.729 INFO:tasks.rados.rados.0.smithi189.stdout:93: snap_create 2024-09-18T10:14:54.748 INFO:tasks.ceph.osd.14.smithi154.stderr:2024-09-18T10:14:54.744+0000 7f75a19e7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 22093) UID: 0 2024-09-18T10:14:54.786 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:14:54.787 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:14:54.796 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-09-18T10:14:54.849 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-09-18T10:14:54.844+0000 7fdfa8698700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22087) UID: 0 2024-09-18T10:14:54.901 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:14:54.901 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:14:54.915 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-09-18T10:14:54.951 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-18T10:14:54.945+0000 7f578c52f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22134) UID: 0 2024-09-18T10:14:54.967 INFO:tasks.rados.rados.0.smithi189.stdout:83: finishing rollback tid 1 to smithi18925896-9 2024-09-18T10:14:54.967 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 9 v 0 (ObjNum 65 snap 0 seq_num 3838457944) dirty dne 2024-09-18T10:14:55.013 INFO:tasks.rados.rados.0.smithi189.stdout:86: expect (ObjNum 4 snap 0 seq_num 4) 2024-09-18T10:14:55.014 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:14:55.014 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:14:55.023 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_ops_in_flight 2024-09-18T10:14:55.052 INFO:tasks.ceph.osd.3.smithi189.stderr:2024-09-18T10:14:55.047+0000 7f2440fc7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22396) UID: 0 2024-09-18T10:14:55.115 INFO:tasks.rados.rados.0.smithi189.stdout:84: expect (ObjNum 19 snap 0 seq_num 19) 2024-09-18T10:14:55.123 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:14:55.123 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:14:55.133 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_blocked_ops 2024-09-18T10:14:55.134 INFO:tasks.rados.rados.0.smithi189.stdout:91: finishing rollback tid 0 to smithi18925896-36 2024-09-18T10:14:55.134 INFO:tasks.rados.rados.0.smithi189.stdout:91: finishing rollback tid 1 to smithi18925896-36 2024-09-18T10:14:55.134 INFO:tasks.rados.rados.0.smithi189.stdout:91: finishing rollback tid 2 to smithi18925896-36 2024-09-18T10:14:55.134 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 36 v 23 (ObjNum 35 snap 0 seq_num 35) dirty exists 2024-09-18T10:14:55.134 INFO:tasks.rados.rados.0.smithi189.stdout:87: expect (ObjNum 3 snap 0 seq_num 3) 2024-09-18T10:14:55.152 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-18T10:14:55.145+0000 7f14c6a1a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 22138) UID: 0 2024-09-18T10:14:55.220 INFO:tasks.rados.rados.0.smithi189.stdout:88: finishing write tid 1 to smithi18925896-3 2024-09-18T10:14:55.220 INFO:tasks.rados.rados.0.smithi189.stdout:88: finishing write tid 2 to smithi18925896-3 2024-09-18T10:14:55.221 INFO:tasks.rados.rados.0.smithi189.stdout:88: finishing write tid 3 to smithi18925896-3 2024-09-18T10:14:55.222 INFO:tasks.rados.rados.0.smithi189.stdout:92: finishing rollback tid 0 to smithi18925896-7 2024-09-18T10:14:55.223 INFO:tasks.rados.rados.0.smithi189.stdout:92: finishing rollback tid 1 to smithi18925896-7 2024-09-18T10:14:55.231 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:14:55.231 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:14:55.233 INFO:tasks.rados.rados.0.smithi189.stdout:90: finishing write tid 1 to smithi18925896-17 2024-09-18T10:14:55.234 INFO:tasks.rados.rados.0.smithi189.stdout:90: finishing write tid 2 to smithi18925896-17 2024-09-18T10:14:55.234 INFO:tasks.rados.rados.0.smithi189.stdout:90: finishing write tid 3 to smithi18925896-17 2024-09-18T10:14:55.241 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_historic_ops 2024-09-18T10:14:55.253 INFO:tasks.ceph.osd.6.smithi154.stderr:2024-09-18T10:14:55.248+0000 7f0abc9e1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22089) UID: 0 2024-09-18T10:14:55.322 INFO:tasks.rados.rados.0.smithi189.stdout:85: expect (ObjNum 40 snap 0 seq_num 40) 2024-09-18T10:14:55.376 INFO:tasks.rados.rados.0.smithi189.stdout:92: finishing rollback tid 2 to smithi18925896-7 2024-09-18T10:14:55.376 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 7 v 14 (ObjNum 6 snap 0 seq_num 6) dirty exists 2024-09-18T10:14:55.376 INFO:tasks.ceph.osd.11.smithi189.stderr:2024-09-18T10:14:55.371+0000 7f718b60a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 22400) UID: 0 2024-09-18T10:14:55.377 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:14:55.377 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:14:55.377 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T10:14:55.425 INFO:tasks.rados.rados.0.smithi189.stdout:89: expect (ObjNum 26 snap 0 seq_num 26) 2024-09-18T10:14:55.477 INFO:tasks.ceph.osd.3.smithi189.stderr:2024-09-18T10:14:55.475+0000 7f2440fc7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22396) UID: 0 2024-09-18T10:14:55.486 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-18T10:14:55.486 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:14:55.495 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T10:14:55.578 INFO:tasks.ceph.osd.12.smithi029.stderr:2024-09-18T10:14:55.574+0000 7f8e4e627700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 22190) UID: 0 2024-09-18T10:14:55.593 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-18T10:14:55.593 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:14:55.602 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T10:14:55.625 INFO:tasks.rados.rados.0.smithi189.stdout:88: finishing write tid 4 to smithi18925896-3 2024-09-18T10:14:55.625 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 3 v 12 (ObjNum 54 snap 3 seq_num 54) dirty exists 2024-09-18T10:14:55.625 INFO:tasks.rados.rados.0.smithi189.stdout:88: left oid 3 (ObjNum 54 snap 3 seq_num 54) 2024-09-18T10:14:55.637 INFO:tasks.rados.rados.0.smithi189.stdout:90: finishing write tid 4 to smithi18925896-17 2024-09-18T10:14:55.638 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 17 v 23 (ObjNum 55 snap 3 seq_num 55) dirty exists 2024-09-18T10:14:55.638 INFO:tasks.rados.rados.0.smithi189.stdout:90: left oid 17 (ObjNum 55 snap 3 seq_num 55) 2024-09-18T10:14:55.679 INFO:tasks.ceph.osd.8.smithi029.stderr:2024-09-18T10:14:55.674+0000 7f2555c27700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 22192) UID: 0 2024-09-18T10:14:55.698 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-18T10:14:55.698 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:14:55.707 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T10:14:55.721 INFO:tasks.rados.rados.0.smithi189.stdout:83: done (10 left) 2024-09-18T10:14:55.721 INFO:tasks.rados.rados.0.smithi189.stdout:84: done (9 left) 2024-09-18T10:14:55.721 INFO:tasks.rados.rados.0.smithi189.stdout:85: done (8 left) 2024-09-18T10:14:55.721 INFO:tasks.rados.rados.0.smithi189.stdout:86: done (7 left) 2024-09-18T10:14:55.721 INFO:tasks.rados.rados.0.smithi189.stdout:87: done (6 left) 2024-09-18T10:14:55.721 INFO:tasks.rados.rados.0.smithi189.stdout:88: done (5 left) 2024-09-18T10:14:55.722 INFO:tasks.rados.rados.0.smithi189.stdout:89: done (4 left) 2024-09-18T10:14:55.722 INFO:tasks.rados.rados.0.smithi189.stdout:90: done (3 left) 2024-09-18T10:14:55.722 INFO:tasks.rados.rados.0.smithi189.stdout:91: done (2 left) 2024-09-18T10:14:55.722 INFO:tasks.rados.rados.0.smithi189.stdout:92: done (1 left) 2024-09-18T10:14:55.722 INFO:tasks.rados.rados.0.smithi189.stdout:93: done (0 left) 2024-09-18T10:14:55.722 INFO:tasks.rados.rados.0.smithi189.stdout:append oid 27 current snap is 4 2024-09-18T10:14:55.722 INFO:tasks.rados.rados.0.smithi189.stdout:94: seq_num 56 ranges {442368=573440,1015808=376832} 2024-09-18T10:14:55.724 INFO:teuthology.orchestra.run.smithi029.stderr:noscrub is set 2024-09-18T10:14:55.732 INFO:tasks.rados.rados.0.smithi189.stdout:94: writing smithi18925896-27 from 442368 to 1015808 tid 1 2024-09-18T10:14:55.734 INFO:tasks.rados.rados.0.smithi189.stdout:94: writing smithi18925896-27 from 1015808 to 1392640 tid 2 2024-09-18T10:14:55.735 INFO:tasks.rados.rados.0.smithi189.stdout:append oid 39 current snap is 4 2024-09-18T10:14:55.735 INFO:tasks.rados.rados.0.smithi189.stdout:95: seq_num 57 ranges {638976=589824,1228800=376832} 2024-09-18T10:14:55.742 INFO:tasks.rados.rados.0.smithi189.stdout:95: writing smithi18925896-39 from 638976 to 1228800 tid 1 2024-09-18T10:14:55.744 INFO:tasks.rados.rados.0.smithi189.stdout:95: writing smithi18925896-39 from 1228800 to 1605632 tid 2 2024-09-18T10:14:55.744 INFO:tasks.rados.rados.0.smithi189.stdout:96: read oid 45 snap -1 2024-09-18T10:14:55.744 INFO:tasks.rados.rados.0.smithi189.stdout:96: expect (ObjNum 44 snap 0 seq_num 44) 2024-09-18T10:14:55.744 INFO:tasks.rados.rados.0.smithi189.stdout:97: rollback oid 12 current snap is 4 2024-09-18T10:14:55.744 INFO:tasks.rados.rados.0.smithi189.stdout:rollback oid 12 to 3 2024-09-18T10:14:55.744 INFO:tasks.rados.rados.0.smithi189.stdout:98: snap_remove snap 0 2024-09-18T10:14:55.780 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-18T10:14:55.773+0000 7f1981873700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22132) UID: 0 2024-09-18T10:14:55.808 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-18T10:14:55.809 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:14:55.817 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T10:14:55.866 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T10:14:55.881 INFO:tasks.ceph.osd.14.smithi154.stderr:2024-09-18T10:14:55.876+0000 7f75a19e7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 22093) UID: 0 2024-09-18T10:14:55.913 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-18T10:14:55.914 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:14:55.923 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T10:14:55.982 INFO:tasks.ceph.osd.6.smithi154.stderr:2024-09-18T10:14:55.976+0000 7f0abc9e1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22089) UID: 0 2024-09-18T10:14:56.018 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-18T10:14:56.018 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:14:56.020 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-18T10:14:56.028 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-09-18T10:14:56.049 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-18T10:14:56.079 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-18T10:14:56.083 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-09-18T10:14:56.080+0000 7fdfa8698700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22087) UID: 0 2024-09-18T10:14:56.101 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-18T10:14:56.120 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-18T10:14:56.131 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-18T10:14:56.131 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:14:56.140 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-18T10:14:56.141 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-09-18T10:14:56.161 INFO:teuthology.orchestra.run.smithi029.stdout:osd.6: {} 2024-09-18T10:14:56.184 INFO:teuthology.orchestra.run.smithi029.stdout:osd.7: {} 2024-09-18T10:14:56.184 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-18T10:14:56.178+0000 7f302f2c3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22187) UID: 0 2024-09-18T10:14:56.206 INFO:teuthology.orchestra.run.smithi029.stdout:osd.8: {} 2024-09-18T10:14:56.227 INFO:teuthology.orchestra.run.smithi029.stdout:osd.9: {} 2024-09-18T10:14:56.242 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-18T10:14:56.242 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-18T10:14:56.242 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-18T10:14:56.242 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-18T10:14:56.242 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-18T10:14:56.249 INFO:teuthology.orchestra.run.smithi029.stdout:osd.10: {} 2024-09-18T10:14:56.251 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-09-18T10:14:56.274 INFO:teuthology.orchestra.run.smithi029.stdout:osd.11: {} 2024-09-18T10:14:56.285 INFO:tasks.ceph.osd.12.smithi029.stderr:2024-09-18T10:14:56.282+0000 7f8e4e627700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 22190) UID: 0 2024-09-18T10:14:56.293 INFO:teuthology.orchestra.run.smithi029.stdout:osd.12: {} 2024-09-18T10:14:56.315 INFO:teuthology.orchestra.run.smithi029.stdout:osd.13: {} 2024-09-18T10:14:56.352 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-18T10:14:56.352 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-18T10:14:56.352 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-18T10:14:56.352 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-18T10:14:56.352 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.353 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.6s5 36/32 ECSubWrite(tid=17, reqid=client.4632.0:122, at_version=0'0, trim_to=0'0, roll_forward_to=36'10))", 2024-09-18T10:14:56.353 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:51.108726+0000", 2024-09-18T10:14:56.353 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.2412808259999997, 2024-09-18T10:14:56.353 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00128315, 2024-09-18T10:14:56.353 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:14:56.353 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:14:56.353 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:14:56.353 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.353 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:14:56.353 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.108726+0000", 2024-09-18T10:14:56.353 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:14:56.354 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.354 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.354 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:14:56.354 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.108726+0000", 2024-09-18T10:14:56.354 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:14:56.354 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.354 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.354 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:14:56.354 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.108724+0000", 2024-09-18T10:14:56.354 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999981 2024-09-18T10:14:56.354 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.354 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.354 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:14:56.354 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.108751+0000", 2024-09-18T10:14:56.355 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6876000000000002e-05 2024-09-18T10:14:56.355 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.355 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.355 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:14:56.355 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.108785+0000", 2024-09-18T10:14:56.355 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4449000000000002e-05 2024-09-18T10:14:56.355 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.355 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.355 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:14:56.355 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.108798+0000", 2024-09-18T10:14:56.355 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2979000000000001e-05 2024-09-18T10:14:56.356 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.356 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.356 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:14:56.356 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.108919+0000", 2024-09-18T10:14:56.356 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012087300000000001 2024-09-18T10:14:56.356 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.356 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.356 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:14:56.356 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.108940+0000", 2024-09-18T10:14:56.356 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1064999999999999e-05 2024-09-18T10:14:56.356 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.357 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.357 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:14:56.357 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.109951+0000", 2024-09-18T10:14:56.357 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001010178 2024-09-18T10:14:56.357 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.357 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.357 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:14:56.357 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.110009+0000", 2024-09-18T10:14:56.357 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8634000000000001e-05 2024-09-18T10:14:56.357 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:14:56.357 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:14:56.357 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:14:56.357 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.357 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.358 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.6s5 36/32 ECSubWrite(tid=19, reqid=client.4632.0:167, at_version=36'11, trim_to=0'0, roll_forward_to=36'10))", 2024-09-18T10:14:56.358 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:51.840599+0000", 2024-09-18T10:14:56.358 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.5094079130000004, 2024-09-18T10:14:56.358 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.102418184, 2024-09-18T10:14:56.358 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:14:56.358 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:14:56.358 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:14:56.358 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.358 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:14:56.358 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.840599+0000", 2024-09-18T10:14:56.358 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:14:56.358 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.358 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.358 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:14:56.359 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.840599+0000", 2024-09-18T10:14:56.359 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:14:56.359 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.359 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.359 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:14:56.359 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.840592+0000", 2024-09-18T10:14:56.359 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999928 2024-09-18T10:14:56.359 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.359 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.359 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:14:56.359 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.840784+0000", 2024-09-18T10:14:56.359 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019234300000000001 2024-09-18T10:14:56.359 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.359 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.360 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:14:56.360 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.840824+0000", 2024-09-18T10:14:56.360 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9654999999999999e-05 2024-09-18T10:14:56.360 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.360 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.360 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:14:56.360 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.940946+0000", 2024-09-18T10:14:56.360 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10012244300000001 2024-09-18T10:14:56.360 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.360 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.360 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:14:56.360 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.941101+0000", 2024-09-18T10:14:56.360 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015468200000000001 2024-09-18T10:14:56.361 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.361 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.361 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:14:56.361 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.941135+0000", 2024-09-18T10:14:56.361 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4020999999999998e-05 2024-09-18T10:14:56.361 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.361 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.361 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:14:56.361 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.942947+0000", 2024-09-18T10:14:56.361 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001812218 2024-09-18T10:14:56.361 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.361 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.361 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:14:56.361 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.943017+0000", 2024-09-18T10:14:56.362 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9805e-05 2024-09-18T10:14:56.362 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:14:56.362 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:14:56.362 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:14:56.362 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.362 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.362 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.6s5 36/32 ECSubWrite(tid=20, reqid=client.4632.0:167, at_version=0'0, trim_to=0'0, roll_forward_to=36'11))", 2024-09-18T10:14:56.362 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:51.944453+0000", 2024-09-18T10:14:56.362 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.4055539709999998, 2024-09-18T10:14:56.362 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0014126130000000001, 2024-09-18T10:14:56.362 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:14:56.362 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:14:56.362 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:14:56.363 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.363 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:14:56.363 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.944453+0000", 2024-09-18T10:14:56.363 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:14:56.363 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.363 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.363 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:14:56.363 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.944453+0000", 2024-09-18T10:14:56.363 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:14:56.363 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.363 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.363 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:14:56.363 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.944451+0000", 2024-09-18T10:14:56.363 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999986 2024-09-18T10:14:56.364 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.364 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.364 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:14:56.364 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.944476+0000", 2024-09-18T10:14:56.364 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4929000000000001e-05 2024-09-18T10:14:56.364 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.364 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.364 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:14:56.364 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.944507+0000", 2024-09-18T10:14:56.364 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0442000000000001e-05 2024-09-18T10:14:56.364 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.364 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.364 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:14:56.364 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.944520+0000", 2024-09-18T10:14:56.365 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3684e-05 2024-09-18T10:14:56.365 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.365 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.365 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:14:56.365 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.944634+0000", 2024-09-18T10:14:56.365 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000113266 2024-09-18T10:14:56.365 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.365 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.365 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:14:56.365 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.944663+0000", 2024-09-18T10:14:56.365 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9054999999999999e-05 2024-09-18T10:14:56.365 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.365 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.365 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:14:56.366 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.945799+0000", 2024-09-18T10:14:56.366 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0011358189999999999 2024-09-18T10:14:56.366 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.366 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.366 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:14:56.366 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.945865+0000", 2024-09-18T10:14:56.366 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6917999999999999e-05 2024-09-18T10:14:56.366 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:14:56.366 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:14:56.366 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:14:56.366 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.366 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.366 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpRead(3.0s2 37/32 ECSubRead(tid=8, to_read={3:02f544f0:::smithi18925896-30:head=0,122880,0}, subchunks={3:02f544f0:::smithi18925896-30:head=[0,1]}, attrs_to_read=))", 2024-09-18T10:14:56.366 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:52.681732+0000", 2024-09-18T10:14:56.367 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.6682753209999999, 2024-09-18T10:14:56.367 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.101148985, 2024-09-18T10:14:56.367 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:14:56.367 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:14:56.367 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:14:56.367 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.367 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:14:56.367 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.681732+0000", 2024-09-18T10:14:56.367 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:14:56.367 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.367 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.367 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:14:56.367 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.681732+0000", 2024-09-18T10:14:56.368 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:14:56.368 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.368 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.368 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:14:56.368 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.681731+0000", 2024-09-18T10:14:56.368 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999995 2024-09-18T10:14:56.368 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.368 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.368 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:14:56.368 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.681750+0000", 2024-09-18T10:14:56.368 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8885e-05 2024-09-18T10:14:56.368 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.368 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.368 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:14:56.369 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.681794+0000", 2024-09-18T10:14:56.369 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4193000000000002e-05 2024-09-18T10:14:56.369 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.369 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.369 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:14:56.369 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.681809+0000", 2024-09-18T10:14:56.369 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5228999999999999e-05 2024-09-18T10:14:56.369 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.369 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.369 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:14:56.369 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.782079+0000", 2024-09-18T10:14:56.369 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100269752 2024-09-18T10:14:56.369 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.369 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.370 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:14:56.370 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.782880+0000", 2024-09-18T10:14:56.370 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00080150900000000005 2024-09-18T10:14:56.370 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:14:56.370 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:14:56.370 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:14:56.370 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.370 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.370 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpRead(3.0s2 37/32 ECSubRead(tid=9, to_read={3:02f544f0:::smithi18925896-30:head=0,122880,0}, subchunks={3:02f544f0:::smithi18925896-30:head=[0,1]}, attrs_to_read=))", 2024-09-18T10:14:56.370 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:52.682582+0000", 2024-09-18T10:14:56.370 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.6674246410000002, 2024-09-18T10:14:56.370 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100690751, 2024-09-18T10:14:56.370 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:14:56.370 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:14:56.371 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:14:56.371 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.371 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:14:56.371 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.682582+0000", 2024-09-18T10:14:56.371 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:14:56.371 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.371 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.371 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:14:56.371 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.682582+0000", 2024-09-18T10:14:56.371 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:14:56.371 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.371 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.371 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:14:56.371 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.682581+0000", 2024-09-18T10:14:56.371 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999986 2024-09-18T10:14:56.372 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.372 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.372 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:14:56.372 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.682614+0000", 2024-09-18T10:14:56.372 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3305e-05 2024-09-18T10:14:56.372 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.372 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.372 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:14:56.372 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.682641+0000", 2024-09-18T10:14:56.372 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7353000000000001e-05 2024-09-18T10:14:56.372 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.372 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.372 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:14:56.373 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.682662+0000", 2024-09-18T10:14:56.373 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0319e-05 2024-09-18T10:14:56.373 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.373 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.373 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:14:56.373 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.782980+0000", 2024-09-18T10:14:56.373 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100318121 2024-09-18T10:14:56.373 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.373 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.373 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:14:56.373 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.783273+0000", 2024-09-18T10:14:56.373 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00029314599999999999 2024-09-18T10:14:56.373 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:14:56.373 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:14:56.373 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:14:56.374 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.374 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.374 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpRead(3.0s2 37/32 ECSubRead(tid=10, to_read={3:02f544f0:::smithi18925896-30:head=0,122880,0}, subchunks={3:02f544f0:::smithi18925896-30:head=[0,1]}, attrs_to_read=))", 2024-09-18T10:14:56.374 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:52.785078+0000", 2024-09-18T10:14:56.374 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.5649291829999998, 2024-09-18T10:14:56.374 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00040589899999999998, 2024-09-18T10:14:56.374 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:14:56.374 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:14:56.374 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:14:56.374 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.374 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:14:56.374 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.785078+0000", 2024-09-18T10:14:56.374 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:14:56.374 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.375 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.375 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:14:56.375 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.785078+0000", 2024-09-18T10:14:56.375 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:14:56.375 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.375 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.375 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:14:56.375 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.785075+0000", 2024-09-18T10:14:56.375 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999976 2024-09-18T10:14:56.375 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.375 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.375 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:14:56.375 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.785095+0000", 2024-09-18T10:14:56.375 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9902999999999999e-05 2024-09-18T10:14:56.376 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.376 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.376 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:14:56.376 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.785133+0000", 2024-09-18T10:14:56.376 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7852000000000003e-05 2024-09-18T10:14:56.376 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.376 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.376 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:14:56.376 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.785154+0000", 2024-09-18T10:14:56.376 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0953000000000001e-05 2024-09-18T10:14:56.376 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.376 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.376 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:14:56.376 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.785303+0000", 2024-09-18T10:14:56.377 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014873800000000001 2024-09-18T10:14:56.377 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.377 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.377 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:14:56.377 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.785484+0000", 2024-09-18T10:14:56.377 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00018086999999999999 2024-09-18T10:14:56.377 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:14:56.377 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:14:56.377 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:14:56.377 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.377 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.377 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpRead(3.0s2 37/32 ECSubRead(tid=11, to_read={3:0a83d6b3:::smithi18925896-20:head=0,81920,0}, subchunks={3:0a83d6b3:::smithi18925896-20:head=[0,1]}, attrs_to_read=))", 2024-09-18T10:14:56.377 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:52.886968+0000", 2024-09-18T10:14:56.378 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.4630387699999998, 2024-09-18T10:14:56.378 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00092996699999999997, 2024-09-18T10:14:56.378 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:14:56.378 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:14:56.378 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:14:56.378 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.378 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:14:56.378 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.886968+0000", 2024-09-18T10:14:56.378 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:14:56.378 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.378 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.378 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:14:56.378 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.886968+0000", 2024-09-18T10:14:56.378 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:14:56.379 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.379 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.379 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:14:56.379 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.886966+0000", 2024-09-18T10:14:56.379 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999986 2024-09-18T10:14:56.379 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.379 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.379 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:14:56.379 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.886987+0000", 2024-09-18T10:14:56.379 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0276e-05 2024-09-18T10:14:56.379 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.379 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.379 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:14:56.379 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.887022+0000", 2024-09-18T10:14:56.379 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4869000000000001e-05 2024-09-18T10:14:56.380 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.380 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.380 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:14:56.380 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.887035+0000", 2024-09-18T10:14:56.380 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3899e-05 2024-09-18T10:14:56.380 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.380 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.380 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:14:56.380 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.887169+0000", 2024-09-18T10:14:56.380 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013332200000000001 2024-09-18T10:14:56.380 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.380 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.380 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:14:56.380 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.887898+0000", 2024-09-18T10:14:56.381 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00072924100000000005 2024-09-18T10:14:56.381 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:14:56.381 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:14:56.381 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:14:56.381 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.381 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.381 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpRead(3.0s2 37/32 ECSubRead(tid=12, to_read={3:0a83d6b3:::smithi18925896-20:head=0,81920,0}, subchunks={3:0a83d6b3:::smithi18925896-20:head=[0,1]}, attrs_to_read=))", 2024-09-18T10:14:56.381 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:52.888483+0000", 2024-09-18T10:14:56.381 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.4615236490000001, 2024-09-18T10:14:56.381 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00037450999999999999, 2024-09-18T10:14:56.381 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:14:56.381 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:14:56.382 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:14:56.382 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.382 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:14:56.382 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.888483+0000", 2024-09-18T10:14:56.382 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:14:56.382 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.382 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.382 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:14:56.382 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.888483+0000", 2024-09-18T10:14:56.382 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:14:56.382 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.382 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.382 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:14:56.382 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.888482+0000", 2024-09-18T10:14:56.383 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999986 2024-09-18T10:14:56.383 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.383 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.383 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:14:56.383 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.888497+0000", 2024-09-18T10:14:56.383 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5418999999999999e-05 2024-09-18T10:14:56.383 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.383 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.383 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:14:56.383 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.888525+0000", 2024-09-18T10:14:56.383 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7708999999999998e-05 2024-09-18T10:14:56.383 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.383 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.384 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:14:56.384 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.888543+0000", 2024-09-18T10:14:56.384 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8289999999999999e-05 2024-09-18T10:14:56.384 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.384 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.384 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:14:56.384 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.888671+0000", 2024-09-18T10:14:56.384 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012724500000000001 2024-09-18T10:14:56.384 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.384 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.384 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:14:56.384 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.888858+0000", 2024-09-18T10:14:56.384 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000187104 2024-09-18T10:14:56.384 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:14:56.385 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:14:56.385 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:14:56.385 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.385 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.385 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpRead(3.0s2 37/32 ECSubRead(tid=13, to_read={3:0a83d6b3:::smithi18925896-20:head=0,81920,0}, subchunks={3:0a83d6b3:::smithi18925896-20:head=[0,1]}, attrs_to_read=))", 2024-09-18T10:14:56.385 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:52.989582+0000", 2024-09-18T10:14:56.385 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.3604247470000002, 2024-09-18T10:14:56.385 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00043238499999999997, 2024-09-18T10:14:56.385 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:14:56.385 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:14:56.385 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:14:56.385 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.385 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:14:56.386 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.989582+0000", 2024-09-18T10:14:56.386 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:14:56.386 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.386 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.386 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:14:56.386 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.989582+0000", 2024-09-18T10:14:56.386 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:14:56.386 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.386 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.386 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:14:56.386 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.989580+0000", 2024-09-18T10:14:56.386 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999981 2024-09-18T10:14:56.386 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.386 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.387 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:14:56.387 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.989602+0000", 2024-09-18T10:14:56.387 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2289e-05 2024-09-18T10:14:56.387 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.387 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.387 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:14:56.387 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.989638+0000", 2024-09-18T10:14:56.387 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5374999999999999e-05 2024-09-18T10:14:56.387 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.387 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.387 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:14:56.387 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.989652+0000", 2024-09-18T10:14:56.387 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4656e-05 2024-09-18T10:14:56.387 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.388 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.388 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:14:56.388 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.989804+0000", 2024-09-18T10:14:56.388 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015151499999999999 2024-09-18T10:14:56.388 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.388 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.388 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:14:56.388 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.990014+0000", 2024-09-18T10:14:56.388 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00021055600000000001 2024-09-18T10:14:56.388 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:14:56.388 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:14:56.388 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:14:56.388 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.389 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.389 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.ds4 37/32 ECSubWrite(tid=31, reqid=client.4632.0:176, at_version=37'4, trim_to=0'0, roll_forward_to=35'2))", 2024-09-18T10:14:56.389 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:53.002892+0000", 2024-09-18T10:14:56.389 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.3471145170000001, 2024-09-18T10:14:56.389 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10294343, 2024-09-18T10:14:56.389 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:14:56.389 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:14:56.389 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:14:56.389 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.389 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:14:56.389 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.002892+0000", 2024-09-18T10:14:56.389 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:14:56.389 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.389 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.390 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:14:56.390 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.002892+0000", 2024-09-18T10:14:56.390 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:14:56.390 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.390 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.390 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:14:56.390 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.002891+0000", 2024-09-18T10:14:56.390 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:14:56.390 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.390 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.390 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:14:56.390 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.003053+0000", 2024-09-18T10:14:56.390 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00016189399999999999 2024-09-18T10:14:56.390 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.391 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.391 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:14:56.391 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.003089+0000", 2024-09-18T10:14:56.391 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5329e-05 2024-09-18T10:14:56.391 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.391 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.391 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:14:56.391 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.003108+0000", 2024-09-18T10:14:56.391 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8963000000000001e-05 2024-09-18T10:14:56.391 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.391 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.391 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:14:56.391 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.103401+0000", 2024-09-18T10:14:56.391 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100293436 2024-09-18T10:14:56.392 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.392 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.392 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:14:56.392 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.103418+0000", 2024-09-18T10:14:56.392 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6722999999999999e-05 2024-09-18T10:14:56.392 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.392 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.392 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:14:56.392 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.105765+0000", 2024-09-18T10:14:56.392 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0023477279999999999 2024-09-18T10:14:56.392 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.392 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.392 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:14:56.392 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.105836+0000", 2024-09-18T10:14:56.392 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0246999999999993e-05 2024-09-18T10:14:56.393 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:14:56.393 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:14:56.393 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:14:56.393 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.393 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.393 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.3s1 37/32 ECSubWrite(tid=11, reqid=client.4632.0:189, at_version=37'8, trim_to=0'0, roll_forward_to=36'6))", 2024-09-18T10:14:56.393 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:53.096486+0000", 2024-09-18T10:14:56.393 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.2535205770000002, 2024-09-18T10:14:56.393 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10315053, 2024-09-18T10:14:56.393 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:14:56.393 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:14:56.393 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:14:56.393 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.393 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:14:56.394 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.096486+0000", 2024-09-18T10:14:56.394 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:14:56.394 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.394 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.394 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:14:56.394 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.096486+0000", 2024-09-18T10:14:56.394 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:14:56.394 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.394 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.394 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:14:56.394 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.096486+0000", 2024-09-18T10:14:56.394 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999995 2024-09-18T10:14:56.394 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.394 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.395 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:14:56.395 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.096802+0000", 2024-09-18T10:14:56.395 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00031615899999999999 2024-09-18T10:14:56.395 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.395 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.395 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:14:56.395 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.096841+0000", 2024-09-18T10:14:56.395 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9452000000000001e-05 2024-09-18T10:14:56.395 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.395 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.395 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:14:56.395 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.096866+0000", 2024-09-18T10:14:56.395 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5069e-05 2024-09-18T10:14:56.396 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.396 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.396 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:14:56.396 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.197104+0000", 2024-09-18T10:14:56.396 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100237886 2024-09-18T10:14:56.396 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.396 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.396 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:14:56.396 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.197130+0000", 2024-09-18T10:14:56.396 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5666000000000001e-05 2024-09-18T10:14:56.396 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.396 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.396 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:14:56.397 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.199585+0000", 2024-09-18T10:14:56.397 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002454777 2024-09-18T10:14:56.397 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.397 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.397 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:14:56.397 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.199637+0000", 2024-09-18T10:14:56.397 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2221e-05 2024-09-18T10:14:56.397 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:14:56.397 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:14:56.397 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:14:56.397 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.397 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.397 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.ds4 37/32 ECSubWrite(tid=32, reqid=client.4632.0:176, at_version=0'0, trim_to=0'0, roll_forward_to=37'4))", 2024-09-18T10:14:56.397 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:53.107300+0000", 2024-09-18T10:14:56.398 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.2427073210000001, 2024-09-18T10:14:56.398 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.101315192, 2024-09-18T10:14:56.398 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:14:56.398 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:14:56.398 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:14:56.398 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.398 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:14:56.398 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.107300+0000", 2024-09-18T10:14:56.398 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:14:56.398 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.398 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.398 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:14:56.398 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.107300+0000", 2024-09-18T10:14:56.399 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:14:56.399 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.399 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.399 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:14:56.399 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.107298+0000", 2024-09-18T10:14:56.399 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999981 2024-09-18T10:14:56.399 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.399 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.399 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:14:56.399 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.107342+0000", 2024-09-18T10:14:56.399 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4542000000000001e-05 2024-09-18T10:14:56.399 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.399 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.399 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:14:56.400 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.107425+0000", 2024-09-18T10:14:56.400 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2590999999999996e-05 2024-09-18T10:14:56.400 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.400 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.400 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:14:56.400 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.207527+0000", 2024-09-18T10:14:56.400 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100102406 2024-09-18T10:14:56.400 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.400 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.400 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:14:56.400 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.207705+0000", 2024-09-18T10:14:56.400 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017770399999999999 2024-09-18T10:14:56.400 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.400 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.401 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:14:56.401 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.207727+0000", 2024-09-18T10:14:56.401 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1991999999999999e-05 2024-09-18T10:14:56.401 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.401 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.401 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:14:56.401 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.208570+0000", 2024-09-18T10:14:56.401 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00084287200000000004 2024-09-18T10:14:56.401 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.401 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.401 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:14:56.401 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.208615+0000", 2024-09-18T10:14:56.401 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5037999999999998e-05 2024-09-18T10:14:56.401 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:14:56.402 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:14:56.402 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:14:56.402 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.402 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.402 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.3s1 37/32 ECSubWrite(tid=12, reqid=client.4632.0:189, at_version=0'0, trim_to=0'0, roll_forward_to=37'8))", 2024-09-18T10:14:56.402 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:53.210357+0000", 2024-09-18T10:14:56.402 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.139649683, 2024-09-18T10:14:56.402 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00089696800000000005, 2024-09-18T10:14:56.402 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:14:56.402 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:14:56.402 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:14:56.402 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.402 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:14:56.402 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.210357+0000", 2024-09-18T10:14:56.403 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:14:56.403 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.403 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.403 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:14:56.403 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.210357+0000", 2024-09-18T10:14:56.403 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:14:56.403 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.403 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.403 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:14:56.403 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.210356+0000", 2024-09-18T10:14:56.403 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:14:56.403 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.403 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.403 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:14:56.404 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.210370+0000", 2024-09-18T10:14:56.404 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3915e-05 2024-09-18T10:14:56.404 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.404 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.404 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:14:56.404 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.210399+0000", 2024-09-18T10:14:56.404 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9116999999999999e-05 2024-09-18T10:14:56.404 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.404 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.404 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:14:56.404 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.210406+0000", 2024-09-18T10:14:56.404 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1160000000000001e-06 2024-09-18T10:14:56.404 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.404 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.405 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:14:56.405 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.210469+0000", 2024-09-18T10:14:56.405 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2951000000000001e-05 2024-09-18T10:14:56.405 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.405 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.405 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:14:56.405 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.210481+0000", 2024-09-18T10:14:56.405 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1905999999999999e-05 2024-09-18T10:14:56.405 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.405 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.405 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:14:56.405 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.211217+0000", 2024-09-18T10:14:56.405 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00073598900000000004 2024-09-18T10:14:56.405 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.406 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.406 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:14:56.406 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.211254+0000", 2024-09-18T10:14:56.406 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6806e-05 2024-09-18T10:14:56.406 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:14:56.406 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:14:56.406 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:14:56.406 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.406 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.406 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.3s1 37/32 ECSubWrite(tid=13, reqid=client.4632.0:194, at_version=37'10, trim_to=0'0, roll_forward_to=37'8))", 2024-09-18T10:14:56.406 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:53.412495+0000", 2024-09-18T10:14:56.406 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 2.9375118069999999, 2024-09-18T10:14:56.406 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.102940423, 2024-09-18T10:14:56.406 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:14:56.407 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:14:56.407 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:14:56.407 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.407 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:14:56.407 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.412495+0000", 2024-09-18T10:14:56.407 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:14:56.407 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.407 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.407 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:14:56.407 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.412495+0000", 2024-09-18T10:14:56.407 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:14:56.407 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.407 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.407 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:14:56.407 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.412493+0000", 2024-09-18T10:14:56.408 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999976 2024-09-18T10:14:56.408 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.408 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.408 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:14:56.408 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.412821+0000", 2024-09-18T10:14:56.408 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000327986 2024-09-18T10:14:56.408 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.408 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.408 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:14:56.408 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.412875+0000", 2024-09-18T10:14:56.408 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4128000000000001e-05 2024-09-18T10:14:56.408 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.408 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.408 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:14:56.409 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.412895+0000", 2024-09-18T10:14:56.409 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0445000000000002e-05 2024-09-18T10:14:56.409 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.409 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.409 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:14:56.409 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.513126+0000", 2024-09-18T10:14:56.409 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100231025 2024-09-18T10:14:56.409 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.409 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.409 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:14:56.409 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.513146+0000", 2024-09-18T10:14:56.409 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9992000000000001e-05 2024-09-18T10:14:56.409 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.409 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.410 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:14:56.410 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.515351+0000", 2024-09-18T10:14:56.410 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0022045599999999999 2024-09-18T10:14:56.410 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.410 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.410 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:14:56.410 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.515435+0000", 2024-09-18T10:14:56.410 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4536999999999994e-05 2024-09-18T10:14:56.410 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:14:56.410 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:14:56.410 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:14:56.410 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.410 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.411 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.3s1 37/32 ECSubWrite(tid=14, reqid=client.4632.0:194, at_version=0'0, trim_to=0'0, roll_forward_to=37'10))", 2024-09-18T10:14:56.411 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:53.616382+0000", 2024-09-18T10:14:56.411 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 2.7336244519999999, 2024-09-18T10:14:56.411 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0013965220000000001, 2024-09-18T10:14:56.411 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:14:56.411 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:14:56.411 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:14:56.411 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.411 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:14:56.411 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.616382+0000", 2024-09-18T10:14:56.411 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:14:56.411 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.411 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.411 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:14:56.412 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.616382+0000", 2024-09-18T10:14:56.412 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:14:56.412 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.412 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.412 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:14:56.412 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.616380+0000", 2024-09-18T10:14:56.412 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999981 2024-09-18T10:14:56.412 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.412 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.412 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:14:56.412 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.616411+0000", 2024-09-18T10:14:56.412 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0375e-05 2024-09-18T10:14:56.412 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.412 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.413 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:14:56.413 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.616453+0000", 2024-09-18T10:14:56.413 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2692999999999999e-05 2024-09-18T10:14:56.413 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.413 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.413 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:14:56.413 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.616475+0000", 2024-09-18T10:14:56.413 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1169e-05 2024-09-18T10:14:56.413 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.413 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.413 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:14:56.413 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.616651+0000", 2024-09-18T10:14:56.413 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017641999999999999 2024-09-18T10:14:56.413 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.413 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.414 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:14:56.414 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.616675+0000", 2024-09-18T10:14:56.414 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4102999999999999e-05 2024-09-18T10:14:56.414 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.414 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.414 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:14:56.414 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.617695+0000", 2024-09-18T10:14:56.414 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010199670000000001 2024-09-18T10:14:56.414 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.414 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.414 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:14:56.414 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.617779+0000", 2024-09-18T10:14:56.414 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3877999999999999e-05 2024-09-18T10:14:56.415 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:14:56.415 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:14:56.415 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:14:56.415 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:14:56.415 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:14:56.415 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-18T10:14:56.416 INFO:teuthology.orchestra.run.smithi029.stdout:osd.14: {} 2024-09-18T10:14:56.416 INFO:teuthology.orchestra.run.smithi029.stdout:osd.15: {} 2024-09-18T10:14:56.417 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_ops_in_flight 2024-09-18T10:14:56.418 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T10:14:56.418 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T10:14:56.418 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T10:14:56.418 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T10:14:56.418 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T10:14:56.418 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T10:14:56.418 INFO:teuthology.orchestra.run.smithi029.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T10:14:56.418 INFO:teuthology.orchestra.run.smithi029.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T10:14:56.418 INFO:teuthology.orchestra.run.smithi029.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-18T10:14:56.419 INFO:teuthology.orchestra.run.smithi029.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-18T10:14:56.419 INFO:teuthology.orchestra.run.smithi029.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-18T10:14:56.419 INFO:teuthology.orchestra.run.smithi029.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-18T10:14:56.419 INFO:teuthology.orchestra.run.smithi029.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-18T10:14:56.419 INFO:teuthology.orchestra.run.smithi029.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-18T10:14:56.419 INFO:teuthology.orchestra.run.smithi029.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-18T10:14:56.419 INFO:teuthology.orchestra.run.smithi029.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-18T10:14:56.420 INFO:tasks.ceph.osd.6.smithi154.stderr:2024-09-18T10:14:56.412+0000 7f0abc9e1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22089) UID: 0 2024-09-18T10:14:56.517 INFO:tasks.ceph.osd.6.smithi154.stderr:2024-09-18T10:14:56.516+0000 7f0abc9e1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22089) UID: 0 2024-09-18T10:14:56.521 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-18T10:14:56.521 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-18T10:14:56.521 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-09-18T10:14:56.521 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-18T10:14:56.530 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_blocked_ops 2024-09-18T10:14:56.618 INFO:tasks.ceph.osd.6.smithi154.stderr:2024-09-18T10:14:56.616+0000 7f0abc9e1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22089) UID: 0 2024-09-18T10:14:56.626 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-18T10:14:56.626 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-18T10:14:56.626 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-18T10:14:56.626 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-18T10:14:56.626 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-18T10:14:56.635 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_historic_ops 2024-09-18T10:14:56.719 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-18T10:14:56.713+0000 7f578c52f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22134) UID: 0 2024-09-18T10:14:56.725 INFO:tasks.rados.rados.0.smithi189.stdout:98: done (4 left) 2024-09-18T10:14:56.725 INFO:tasks.rados.rados.0.smithi189.stdout:append oid 34 current snap is 4 2024-09-18T10:14:56.725 INFO:tasks.rados.rados.0.smithi189.stdout:95: finishing write tid 1 to smithi18925896-39 2024-09-18T10:14:56.725 INFO:tasks.rados.rados.0.smithi189.stdout:99: seq_num 58 ranges {557056=606208,1163264=376832} 2024-09-18T10:14:56.739 INFO:tasks.rados.rados.0.smithi189.stdout:99: writing smithi18925896-34 from 557056 to 1163264 tid 1 2024-09-18T10:14:56.741 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-18T10:14:56.742 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-18T10:14:56.742 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-18T10:14:56.742 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-18T10:14:56.742 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.742 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4341.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T10:14:56.742 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:36.317595+0000", 2024-09-18T10:14:56.742 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 20.422413409000001, 2024-09-18T10:14:56.742 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.106589221, 2024-09-18T10:14:56.742 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:14:56.742 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:14:56.742 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:14:56.742 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4341", 2024-09-18T10:14:56.743 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.29:0/3001591505", 2024-09-18T10:14:56.743 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 5 2024-09-18T10:14:56.743 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.743 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:14:56.743 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.743 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:14:56.743 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:36.317595+0000", 2024-09-18T10:14:56.743 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:14:56.743 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.743 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.743 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:14:56.743 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:36.317595+0000", 2024-09-18T10:14:56.743 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:14:56.743 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.744 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.744 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:14:56.744 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:36.317592+0000", 2024-09-18T10:14:56.744 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999967 2024-09-18T10:14:56.744 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.744 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.744 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:14:56.744 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:36.317628+0000", 2024-09-18T10:14:56.744 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5945000000000002e-05 2024-09-18T10:14:56.744 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.744 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.744 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:14:56.744 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:36.317665+0000", 2024-09-18T10:14:56.745 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7248e-05 2024-09-18T10:14:56.745 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.745 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.745 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:14:56.745 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:36.317680+0000", 2024-09-18T10:14:56.745 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.505e-05 2024-09-18T10:14:56.745 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.745 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.745 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:14:56.745 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:36.317799+0000", 2024-09-18T10:14:56.745 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000118528 2024-09-18T10:14:56.745 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.745 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.746 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-09-18T10:14:56.746 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:36.317808+0000", 2024-09-18T10:14:56.746 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3260000000000002e-06 2024-09-18T10:14:56.746 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.746 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.746 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:14:56.746 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:36.320780+0000", 2024-09-18T10:14:56.746 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0029716270000000001 2024-09-18T10:14:56.746 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.746 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.746 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:14:56.746 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:36.321160+0000", 2024-09-18T10:14:56.746 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00037985299999999998 2024-09-18T10:14:56.747 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.747 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.747 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 12", 2024-09-18T10:14:56.747 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:36.321559+0000", 2024-09-18T10:14:56.747 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00039961199999999998 2024-09-18T10:14:56.747 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.747 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.747 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-18T10:14:56.747 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:36.323216+0000", 2024-09-18T10:14:56.747 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001656505 2024-09-18T10:14:56.747 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.747 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.747 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:14:56.748 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:36.424027+0000", 2024-09-18T10:14:56.748 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10081152 2024-09-18T10:14:56.748 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.748 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.748 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:14:56.748 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:36.424121+0000", 2024-09-18T10:14:56.748 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3475999999999997e-05 2024-09-18T10:14:56.748 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.748 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.748 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:14:56.748 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:36.424185+0000", 2024-09-18T10:14:56.749 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3751000000000007e-05 2024-09-18T10:14:56.749 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:14:56.749 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:14:56.749 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:14:56.749 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.749 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.749 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpRead(3.2s2 36/32 ECSubRead(tid=16, to_read={3:427b2c0b:::smithi18925896-42:head=0,4096,0}, subchunks={3:427b2c0b:::smithi18925896-42:head=[0,1]}, attrs_to_read=))", 2024-09-18T10:14:56.749 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:51.113712+0000", 2024-09-18T10:14:56.749 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.6262966040000002, 2024-09-18T10:14:56.749 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.200895612, 2024-09-18T10:14:56.749 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:14:56.749 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:14:56.749 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:14:56.750 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.750 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:14:56.750 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.113712+0000", 2024-09-18T10:14:56.750 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:14:56.750 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.750 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.750 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:14:56.750 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.113712+0000", 2024-09-18T10:14:56.750 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:14:56.750 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.750 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.750 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:14:56.751 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.113711+0000", 2024-09-18T10:14:56.751 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999986 2024-09-18T10:14:56.751 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.751 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.751 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:14:56.751 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.113751+0000", 2024-09-18T10:14:56.751 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0049000000000002e-05 2024-09-18T10:14:56.751 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.751 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.751 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:14:56.751 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.113796+0000", 2024-09-18T10:14:56.752 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5595e-05 2024-09-18T10:14:56.752 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.752 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.752 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:14:56.752 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.213891+0000", 2024-09-18T10:14:56.752 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100095142 2024-09-18T10:14:56.752 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.752 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.752 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:14:56.752 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.314098+0000", 2024-09-18T10:14:56.752 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10020617800000001 2024-09-18T10:14:56.752 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.752 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.752 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:14:56.753 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.314608+0000", 2024-09-18T10:14:56.753 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00051021499999999995 2024-09-18T10:14:56.753 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:14:56.753 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:14:56.753 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:14:56.753 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.753 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.753 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.7s6 36/32 ECSubWrite(tid=31, reqid=client.4632.0:128, at_version=0'0, trim_to=0'0, roll_forward_to=36'8))", 2024-09-18T10:14:56.753 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:51.219673+0000", 2024-09-18T10:14:56.753 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.5203358749999998, 2024-09-18T10:14:56.753 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20600959799999999, 2024-09-18T10:14:56.753 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:14:56.753 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:14:56.754 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:14:56.754 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.754 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:14:56.754 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.219673+0000", 2024-09-18T10:14:56.754 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:14:56.754 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.754 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.754 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:14:56.754 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.219673+0000", 2024-09-18T10:14:56.754 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:14:56.754 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.754 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.755 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:14:56.755 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.219671+0000", 2024-09-18T10:14:56.755 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999986 2024-09-18T10:14:56.755 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.755 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.755 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:14:56.755 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.219696+0000", 2024-09-18T10:14:56.755 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4649000000000001e-05 2024-09-18T10:14:56.755 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.755 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.755 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:14:56.755 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.219728+0000", 2024-09-18T10:14:56.755 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2036000000000003e-05 2024-09-18T10:14:56.755 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.756 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.756 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:14:56.756 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.319879+0000", 2024-09-18T10:14:56.756 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100151433 2024-09-18T10:14:56.756 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.756 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.756 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:14:56.756 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.423228+0000", 2024-09-18T10:14:56.756 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.103348542 2024-09-18T10:14:56.756 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.756 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.756 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:14:56.756 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.423241+0000", 2024-09-18T10:14:56.756 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3519999999999999e-05 2024-09-18T10:14:56.757 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.757 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.757 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:14:56.757 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.425642+0000", 2024-09-18T10:14:56.757 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0024009859999999999 2024-09-18T10:14:56.757 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.757 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.757 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:14:56.757 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.425682+0000", 2024-09-18T10:14:56.757 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9999000000000001e-05 2024-09-18T10:14:56.757 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:14:56.757 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:14:56.758 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:14:56.758 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.758 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.758 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.7s6 36/32 ECSubWrite(tid=33, reqid=client.4632.0:156, at_version=36'9, trim_to=0'0, roll_forward_to=36'8))", 2024-09-18T10:14:56.758 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:51.319907+0000", 2024-09-18T10:14:56.758 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.4201012979999996, 2024-09-18T10:14:56.758 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10579651299999999, 2024-09-18T10:14:56.758 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:14:56.758 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:14:56.758 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:14:56.758 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.758 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:14:56.758 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.319907+0000", 2024-09-18T10:14:56.758 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:14:56.759 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.759 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.759 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:14:56.759 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.319907+0000", 2024-09-18T10:14:56.759 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:14:56.759 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.759 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.759 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:14:56.759 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.319907+0000", 2024-09-18T10:14:56.759 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:14:56.759 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.759 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.759 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:14:56.760 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.320101+0000", 2024-09-18T10:14:56.760 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019464600000000001 2024-09-18T10:14:56.760 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.760 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.760 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:14:56.760 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.320126+0000", 2024-09-18T10:14:56.760 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4973e-05 2024-09-18T10:14:56.760 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.760 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.760 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:14:56.760 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.320137+0000", 2024-09-18T10:14:56.760 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0573e-05 2024-09-18T10:14:56.760 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.761 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.761 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:14:56.761 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.423559+0000", 2024-09-18T10:14:56.761 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10342206499999999 2024-09-18T10:14:56.761 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.761 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.761 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:14:56.761 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.423582+0000", 2024-09-18T10:14:56.761 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2969999999999999e-05 2024-09-18T10:14:56.761 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.761 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.761 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:14:56.761 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.425688+0000", 2024-09-18T10:14:56.761 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0021066280000000001 2024-09-18T10:14:56.762 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.762 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.762 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:14:56.762 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.425704+0000", 2024-09-18T10:14:56.762 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5585000000000001e-05 2024-09-18T10:14:56.762 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:14:56.762 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:14:56.762 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:14:56.762 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.762 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.762 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.7s6 36/32 ECSubWrite(tid=34, reqid=client.4632.0:157, at_version=36'10, trim_to=0'0, roll_forward_to=36'8))", 2024-09-18T10:14:56.762 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:51.320147+0000", 2024-09-18T10:14:56.762 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.419862041, 2024-09-18T10:14:56.762 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.106623945, 2024-09-18T10:14:56.763 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:14:56.763 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:14:56.763 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:14:56.763 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.763 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:14:56.763 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.320147+0000", 2024-09-18T10:14:56.763 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:14:56.763 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.763 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.763 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:14:56.763 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.320147+0000", 2024-09-18T10:14:56.763 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:14:56.764 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.764 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.764 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:14:56.764 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.320146+0000", 2024-09-18T10:14:56.764 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999995 2024-09-18T10:14:56.764 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.764 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.764 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:14:56.764 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.320287+0000", 2024-09-18T10:14:56.764 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000140755 2024-09-18T10:14:56.764 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.764 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.764 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:14:56.764 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.320305+0000", 2024-09-18T10:14:56.765 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8009999999999999e-05 2024-09-18T10:14:56.765 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.765 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.765 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:14:56.765 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.320313+0000", 2024-09-18T10:14:56.765 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7789999999999996e-06 2024-09-18T10:14:56.765 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.765 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.765 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:14:56.765 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.424676+0000", 2024-09-18T10:14:56.766 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.104363342 2024-09-18T10:14:56.766 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.766 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.766 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:14:56.766 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.424691+0000", 2024-09-18T10:14:56.766 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.469e-05 2024-09-18T10:14:56.766 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.766 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.766 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:14:56.766 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.426726+0000", 2024-09-18T10:14:56.766 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0020349370000000001 2024-09-18T10:14:56.766 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.766 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.766 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:14:56.767 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.426771+0000", 2024-09-18T10:14:56.767 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5037999999999998e-05 2024-09-18T10:14:56.767 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:14:56.767 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:14:56.767 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:14:56.767 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.767 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.767 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.7s6 36/32 ECSubWrite(tid=36, reqid=client.4632.0:158, at_version=36'11, trim_to=0'0, roll_forward_to=36'8))", 2024-09-18T10:14:56.767 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:51.425170+0000", 2024-09-18T10:14:56.767 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.3148390750000001, 2024-09-18T10:14:56.767 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.124667824, 2024-09-18T10:14:56.767 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:14:56.767 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:14:56.768 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:14:56.768 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.768 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:14:56.768 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.425170+0000", 2024-09-18T10:14:56.768 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:14:56.768 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.768 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.768 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:14:56.768 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.425170+0000", 2024-09-18T10:14:56.768 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:14:56.768 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.768 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.768 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:14:56.768 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.425169+0000", 2024-09-18T10:14:56.769 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:14:56.769 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.769 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.769 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:14:56.769 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.425204+0000", 2024-09-18T10:14:56.769 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5775000000000002e-05 2024-09-18T10:14:56.769 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.769 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.769 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:14:56.769 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.425226+0000", 2024-09-18T10:14:56.769 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1393e-05 2024-09-18T10:14:56.769 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.769 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.769 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:14:56.769 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.525360+0000", 2024-09-18T10:14:56.770 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10013453899999999 2024-09-18T10:14:56.770 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.770 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.770 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:14:56.770 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.525493+0000", 2024-09-18T10:14:56.770 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013232200000000001 2024-09-18T10:14:56.770 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.770 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.770 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:14:56.770 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.525508+0000", 2024-09-18T10:14:56.770 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4826e-05 2024-09-18T10:14:56.770 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.770 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.770 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:14:56.771 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.549772+0000", 2024-09-18T10:14:56.771 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.024264674 2024-09-18T10:14:56.771 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.771 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.771 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:14:56.771 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.549838+0000", 2024-09-18T10:14:56.771 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5265000000000006e-05 2024-09-18T10:14:56.771 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:14:56.771 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:14:56.771 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:14:56.771 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.771 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.771 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:171 3.as0 3:5b104cb9:::smithi18925896-9:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e36)", 2024-09-18T10:14:56.771 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:51.532779+0000", 2024-09-18T10:14:56.771 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.207230161, 2024-09-18T10:14:56.772 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30363525200000002, 2024-09-18T10:14:56.772 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:14:56.772 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:14:56.772 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:14:56.772 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:14:56.772 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:14:56.772 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 171 2024-09-18T10:14:56.772 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.772 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:14:56.772 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.772 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:14:56.772 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.532779+0000", 2024-09-18T10:14:56.772 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:14:56.772 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.773 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.773 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:14:56.773 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.532779+0000", 2024-09-18T10:14:56.773 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:14:56.773 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.773 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.773 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:14:56.773 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.532776+0000", 2024-09-18T10:14:56.773 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999971 2024-09-18T10:14:56.773 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.773 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.773 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:14:56.773 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.532789+0000", 2024-09-18T10:14:56.774 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3339999999999999e-05 2024-09-18T10:14:56.774 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.774 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.774 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:14:56.774 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.532823+0000", 2024-09-18T10:14:56.774 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3627999999999997e-05 2024-09-18T10:14:56.774 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.774 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.774 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:14:56.774 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.532837+0000", 2024-09-18T10:14:56.774 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.385e-05 2024-09-18T10:14:56.774 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.774 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.775 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:14:56.775 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.633025+0000", 2024-09-18T10:14:56.775 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100188176 2024-09-18T10:14:56.775 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.775 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.775 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:14:56.775 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.633191+0000", 2024-09-18T10:14:56.775 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00016606699999999999 2024-09-18T10:14:56.775 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.775 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.775 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:14:56.775 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.633949+0000", 2024-09-18T10:14:56.775 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00075792600000000002 2024-09-18T10:14:56.775 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.776 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.776 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:14:56.776 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.835688+0000", 2024-09-18T10:14:56.776 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20173866700000001 2024-09-18T10:14:56.776 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.776 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.776 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:14:56.776 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.836246+0000", 2024-09-18T10:14:56.776 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00055799699999999996 2024-09-18T10:14:56.776 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.776 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.776 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:14:56.777 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.836414+0000", 2024-09-18T10:14:56.777 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000168274 2024-09-18T10:14:56.777 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:14:56.777 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:14:56.777 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:14:56.777 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.777 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.777 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(osd.0.21:1 3.as0 3:5dc9cbed:::smithi18925896-13:head [assert-version v4,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:14:56.777 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:51.535493+0000", 2024-09-18T10:14:56.777 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.2045152650000004, 2024-09-18T10:14:56.777 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30246291400000003, 2024-09-18T10:14:56.777 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:14:56.777 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-18T10:14:56.777 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:14:56.778 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.778 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:14:56.778 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.535493+0000", 2024-09-18T10:14:56.778 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:14:56.778 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.778 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.778 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:14:56.778 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.535493+0000", 2024-09-18T10:14:56.778 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:14:56.778 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.778 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.778 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:14:56.778 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.535491+0000", 2024-09-18T10:14:56.779 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999971 2024-09-18T10:14:56.779 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.779 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.779 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:14:56.779 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.535508+0000", 2024-09-18T10:14:56.779 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7266e-05 2024-09-18T10:14:56.779 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.779 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.779 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:14:56.779 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.535537+0000", 2024-09-18T10:14:56.779 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8612e-05 2024-09-18T10:14:56.779 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.779 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.779 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:14:56.780 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.535554+0000", 2024-09-18T10:14:56.780 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7575999999999999e-05 2024-09-18T10:14:56.780 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.780 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.780 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:14:56.780 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.634492+0000", 2024-09-18T10:14:56.780 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.098937397999999996 2024-09-18T10:14:56.780 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.780 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.780 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:14:56.780 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.634722+0000", 2024-09-18T10:14:56.780 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00022994500000000001 2024-09-18T10:14:56.780 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.780 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.781 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:14:56.781 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.837956+0000", 2024-09-18T10:14:56.781 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20323484999999999 2024-09-18T10:14:56.781 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:14:56.781 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:14:56.781 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:14:56.781 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.781 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.781 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpRead(3.2s2 36/32 ECSubRead(tid=17, to_read={3:46b2871c:::smithi18925896-34:head=0,139264,0}, subchunks={3:46b2871c:::smithi18925896-34:head=[0,1]}, attrs_to_read=))", 2024-09-18T10:14:56.781 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:51.537987+0000", 2024-09-18T10:14:56.781 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.2020215109999999, 2024-09-18T10:14:56.781 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.297589722, 2024-09-18T10:14:56.782 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:14:56.782 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:14:56.782 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:14:56.782 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.782 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:14:56.782 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.537987+0000", 2024-09-18T10:14:56.782 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:14:56.782 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.782 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.782 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:14:56.782 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.537987+0000", 2024-09-18T10:14:56.782 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:14:56.782 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.783 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.783 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:14:56.783 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.537986+0000", 2024-09-18T10:14:56.783 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999981 2024-09-18T10:14:56.783 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.783 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.783 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:14:56.783 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.538009+0000", 2024-09-18T10:14:56.783 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3506000000000002e-05 2024-09-18T10:14:56.783 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.783 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.783 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:14:56.783 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.538052+0000", 2024-09-18T10:14:56.783 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2982000000000003e-05 2024-09-18T10:14:56.784 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.784 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.784 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:14:56.784 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.538075+0000", 2024-09-18T10:14:56.784 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3178999999999999e-05 2024-09-18T10:14:56.784 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.784 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.784 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:14:56.784 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.834627+0000", 2024-09-18T10:14:56.784 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.29655214299999999 2024-09-18T10:14:56.784 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.784 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.784 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:14:56.784 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.835577+0000", 2024-09-18T10:14:56.785 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00094962200000000001 2024-09-18T10:14:56.785 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:14:56.785 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:14:56.785 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:14:56.785 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.785 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.785 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWriteReply(3.fs0 36/32 ECSubWriteReply(tid=23, last_complete=36'4, committed=1, applied=1))", 2024-09-18T10:14:56.785 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:51.618241+0000", 2024-09-18T10:14:56.785 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.1217677879999997, 2024-09-18T10:14:56.785 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20057209300000001, 2024-09-18T10:14:56.785 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:14:56.785 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:14:56.785 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:14:56.786 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.786 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:14:56.786 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.618241+0000", 2024-09-18T10:14:56.786 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:14:56.786 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.786 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.786 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:14:56.786 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.618241+0000", 2024-09-18T10:14:56.786 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:14:56.786 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.786 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.786 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:14:56.786 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.618230+0000", 2024-09-18T10:14:56.787 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999989 2024-09-18T10:14:56.787 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.787 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.787 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:14:56.787 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.618251+0000", 2024-09-18T10:14:56.787 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0876000000000001e-05 2024-09-18T10:14:56.787 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.787 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.787 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:14:56.787 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.618284+0000", 2024-09-18T10:14:56.787 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3334999999999998e-05 2024-09-18T10:14:56.787 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.787 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.787 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:14:56.788 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.718424+0000", 2024-09-18T10:14:56.788 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100139485 2024-09-18T10:14:56.788 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.788 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.788 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:14:56.788 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.818711+0000", 2024-09-18T10:14:56.788 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100286928 2024-09-18T10:14:56.788 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.788 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.788 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:14:56.788 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.818813+0000", 2024-09-18T10:14:56.788 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000102452 2024-09-18T10:14:56.788 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:14:56.788 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:14:56.789 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:14:56.789 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.789 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.789 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpReadReply(3.as0 36/32 ECSubReadReply(tid=26, attrs_read=0))", 2024-09-18T10:14:56.789 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:51.636169+0000", 2024-09-18T10:14:56.789 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.1038399730000004, 2024-09-18T10:14:56.789 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.19967401500000001, 2024-09-18T10:14:56.789 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:14:56.789 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:14:56.789 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:14:56.789 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.789 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:14:56.789 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.636169+0000", 2024-09-18T10:14:56.789 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:14:56.790 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.790 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.790 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:14:56.790 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.636169+0000", 2024-09-18T10:14:56.790 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:14:56.790 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.790 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.790 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:14:56.790 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.636167+0000", 2024-09-18T10:14:56.790 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999986 2024-09-18T10:14:56.790 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.790 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.790 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:14:56.790 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.636287+0000", 2024-09-18T10:14:56.790 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011957599999999999 2024-09-18T10:14:56.791 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.791 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.791 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:14:56.791 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.636307+0000", 2024-09-18T10:14:56.791 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0472999999999999e-05 2024-09-18T10:14:56.791 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.791 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.791 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:14:56.791 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.736388+0000", 2024-09-18T10:14:56.791 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100080511 2024-09-18T10:14:56.791 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.791 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.791 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:14:56.791 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.835743+0000", 2024-09-18T10:14:56.791 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.099355424999999997 2024-09-18T10:14:56.792 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.792 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.792 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:14:56.792 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.835843+0000", 2024-09-18T10:14:56.792 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9647000000000003e-05 2024-09-18T10:14:56.792 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:14:56.792 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:14:56.792 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:14:56.792 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.792 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.792 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:176 3.ds0 3:b1a14543:::smithi18925896-15:head [copy-from ver 4] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:14:56.792 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:52.680518+0000", 2024-09-18T10:14:56.792 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.0594911680000001, 2024-09-18T10:14:56.792 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.42607371300000002, 2024-09-18T10:14:56.793 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:14:56.793 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:14:56.793 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:14:56.793 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:14:56.793 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:14:56.793 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 176 2024-09-18T10:14:56.793 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.793 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:14:56.793 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.793 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:14:56.793 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.680518+0000", 2024-09-18T10:14:56.793 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:14:56.793 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.794 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.794 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:14:56.794 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.680518+0000", 2024-09-18T10:14:56.794 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:14:56.794 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.794 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.794 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:14:56.794 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.680515+0000", 2024-09-18T10:14:56.794 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999971 2024-09-18T10:14:56.794 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.794 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.794 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:14:56.794 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.680533+0000", 2024-09-18T10:14:56.794 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8756000000000001e-05 2024-09-18T10:14:56.794 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.795 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.795 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:14:56.795 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.680573+0000", 2024-09-18T10:14:56.795 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9431999999999998e-05 2024-09-18T10:14:56.795 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.795 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.795 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:14:56.795 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.680587+0000", 2024-09-18T10:14:56.795 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.447e-05 2024-09-18T10:14:56.795 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.795 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.795 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:14:56.795 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.680755+0000", 2024-09-18T10:14:56.795 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000167584 2024-09-18T10:14:56.795 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.795 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.796 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-09-18T10:14:56.796 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.680784+0000", 2024-09-18T10:14:56.796 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9439000000000001e-05 2024-09-18T10:14:56.796 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.796 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.796 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:14:56.796 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.685398+0000", 2024-09-18T10:14:56.796 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0046140060000000004 2024-09-18T10:14:56.796 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.796 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.796 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:14:56.796 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.685502+0000", 2024-09-18T10:14:56.796 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000103702 2024-09-18T10:14:56.796 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.797 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.797 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:14:56.797 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.003083+0000", 2024-09-18T10:14:56.797 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.31758088600000001 2024-09-18T10:14:56.797 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.797 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.797 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:14:56.797 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.006087+0000", 2024-09-18T10:14:56.797 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0030044820000000002 2024-09-18T10:14:56.797 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.797 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.797 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:14:56.798 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.106397+0000", 2024-09-18T10:14:56.798 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100309469 2024-09-18T10:14:56.798 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.798 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.798 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:14:56.798 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.106591+0000", 2024-09-18T10:14:56.798 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000194513 2024-09-18T10:14:56.798 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:14:56.798 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:14:56.798 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:14:56.798 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.798 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.798 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:177 3.ds0 3:b1a14543:::smithi18925896-15:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:14:56.798 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:52.680610+0000", 2024-09-18T10:14:56.799 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.0593984470000004, 2024-09-18T10:14:56.799 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.32474139800000001, 2024-09-18T10:14:56.799 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:14:56.799 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-18T10:14:56.799 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:14:56.799 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:14:56.799 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:14:56.799 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 177 2024-09-18T10:14:56.799 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.799 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:14:56.799 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.799 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:14:56.799 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.680610+0000", 2024-09-18T10:14:56.800 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:14:56.800 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.800 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.800 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:14:56.800 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.680610+0000", 2024-09-18T10:14:56.800 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:14:56.800 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.800 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.800 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:14:56.800 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.680609+0000", 2024-09-18T10:14:56.800 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:14:56.800 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.800 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.801 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:14:56.801 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.680614+0000", 2024-09-18T10:14:56.801 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6840000000000004e-06 2024-09-18T10:14:56.801 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.801 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.801 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:14:56.801 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.680625+0000", 2024-09-18T10:14:56.801 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0739e-05 2024-09-18T10:14:56.801 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.801 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.801 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:14:56.801 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.680655+0000", 2024-09-18T10:14:56.802 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0022000000000002e-05 2024-09-18T10:14:56.802 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.802 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.802 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:14:56.802 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.680965+0000", 2024-09-18T10:14:56.802 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00031078300000000002 2024-09-18T10:14:56.802 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.802 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.802 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:14:56.802 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.680976+0000", 2024-09-18T10:14:56.802 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0589e-05 2024-09-18T10:14:56.802 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.802 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.803 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:14:56.803 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.785862+0000", 2024-09-18T10:14:56.803 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10488597500000001 2024-09-18T10:14:56.803 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.803 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.803 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2024-09-18T10:14:56.803 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.785963+0000", 2024-09-18T10:14:56.803 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000100523 2024-09-18T10:14:56.803 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.803 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.803 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:14:56.804 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.005029+0000", 2024-09-18T10:14:56.804 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.219066329 2024-09-18T10:14:56.804 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.804 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.804 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:14:56.804 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.005193+0000", 2024-09-18T10:14:56.804 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00016436200000000001 2024-09-18T10:14:56.804 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.804 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.804 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:14:56.804 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.005352+0000", 2024-09-18T10:14:56.804 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000158486 2024-09-18T10:14:56.805 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:14:56.805 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:14:56.805 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:14:56.805 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.805 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.805 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:199 3.as0 3:5dc9cbed:::smithi18925896-13:2 [read 0~212992,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:14:56.805 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:52.694692+0000", 2024-09-18T10:14:56.805 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.0453168399999999, 2024-09-18T10:14:56.805 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.105246667, 2024-09-18T10:14:56.805 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:14:56.805 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-18T10:14:56.805 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:14:56.805 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:14:56.805 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:14:56.806 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 199 2024-09-18T10:14:56.806 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.806 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:14:56.806 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.806 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:14:56.806 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694692+0000", 2024-09-18T10:14:56.806 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:14:56.806 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.806 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.806 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:14:56.806 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694692+0000", 2024-09-18T10:14:56.806 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:14:56.806 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.807 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.807 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:14:56.807 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694690+0000", 2024-09-18T10:14:56.807 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999981 2024-09-18T10:14:56.807 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.807 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.807 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:14:56.807 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694701+0000", 2024-09-18T10:14:56.807 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1562999999999999e-05 2024-09-18T10:14:56.807 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.807 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.807 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:14:56.807 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694725+0000", 2024-09-18T10:14:56.807 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3566000000000001e-05 2024-09-18T10:14:56.808 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.808 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.808 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:14:56.808 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694734+0000", 2024-09-18T10:14:56.808 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4499999999999993e-06 2024-09-18T10:14:56.808 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.808 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.808 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:14:56.808 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694825+0000", 2024-09-18T10:14:56.808 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0816000000000006e-05 2024-09-18T10:14:56.808 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.808 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.809 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:14:56.809 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694956+0000", 2024-09-18T10:14:56.809 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000130536 2024-09-18T10:14:56.809 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.809 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.809 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:14:56.809 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.799939+0000", 2024-09-18T10:14:56.809 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.104982793 2024-09-18T10:14:56.809 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:14:56.809 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:14:56.809 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:14:56.809 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.809 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.810 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:200 3.as0 3:5dc9cbed:::smithi18925896-13:2 [read 0~212992,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:14:56.810 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:52.694751+0000", 2024-09-18T10:14:56.810 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.0452580170000001, 2024-09-18T10:14:56.810 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.40590051599999999, 2024-09-18T10:14:56.810 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:14:56.810 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-18T10:14:56.810 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:14:56.810 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:14:56.810 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:14:56.810 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 200 2024-09-18T10:14:56.810 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.810 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:14:56.810 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.810 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:14:56.811 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694751+0000", 2024-09-18T10:14:56.811 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:14:56.811 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.811 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.811 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:14:56.811 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694751+0000", 2024-09-18T10:14:56.811 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:14:56.811 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.811 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.811 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:14:56.811 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694750+0000", 2024-09-18T10:14:56.811 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999995 2024-09-18T10:14:56.811 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.811 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.811 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:14:56.812 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694756+0000", 2024-09-18T10:14:56.812 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8119999999999996e-06 2024-09-18T10:14:56.812 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.812 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.812 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:14:56.812 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694762+0000", 2024-09-18T10:14:56.812 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8050000000000003e-06 2024-09-18T10:14:56.812 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.812 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.812 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:14:56.812 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694768+0000", 2024-09-18T10:14:56.812 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5549999999999999e-06 2024-09-18T10:14:56.813 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.813 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.813 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:14:56.813 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.695694+0000", 2024-09-18T10:14:56.813 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00092562999999999999 2024-09-18T10:14:56.813 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.813 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.813 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:14:56.813 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.695903+0000", 2024-09-18T10:14:56.813 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00020888699999999999 2024-09-18T10:14:56.813 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.813 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.813 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:14:56.814 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.100651+0000", 2024-09-18T10:14:56.814 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.40474845599999998 2024-09-18T10:14:56.814 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:14:56.814 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:14:56.814 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:14:56.814 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.814 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.814 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:201 3.as0 3:5dc9cbed:::smithi18925896-13:2 [read 0~212992,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:14:56.814 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:52.694813+0000", 2024-09-18T10:14:56.814 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.0451962129999997, 2024-09-18T10:14:56.814 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.50652904899999995, 2024-09-18T10:14:56.814 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:14:56.814 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-18T10:14:56.815 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:14:56.815 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:14:56.815 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:14:56.815 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 201 2024-09-18T10:14:56.815 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.815 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:14:56.815 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.815 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:14:56.815 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694813+0000", 2024-09-18T10:14:56.815 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:14:56.815 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.815 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.816 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:14:56.816 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694813+0000", 2024-09-18T10:14:56.816 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:14:56.816 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.816 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.816 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:14:56.816 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694812+0000", 2024-09-18T10:14:56.816 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999995 2024-09-18T10:14:56.816 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.816 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.816 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:14:56.816 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694816+0000", 2024-09-18T10:14:56.816 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7720000000000001e-06 2024-09-18T10:14:56.817 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.817 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.817 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:14:56.817 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694822+0000", 2024-09-18T10:14:56.817 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5529999999999998e-06 2024-09-18T10:14:56.817 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.817 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.817 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:14:56.817 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694827+0000", 2024-09-18T10:14:56.817 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1500000000000001e-06 2024-09-18T10:14:56.817 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.817 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.818 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:14:56.818 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.797100+0000", 2024-09-18T10:14:56.818 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.102273424 2024-09-18T10:14:56.818 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.818 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.818 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:14:56.818 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.797329+0000", 2024-09-18T10:14:56.818 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00022870200000000001 2024-09-18T10:14:56.818 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.818 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.818 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:14:56.818 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.201342+0000", 2024-09-18T10:14:56.819 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.40401293799999999 2024-09-18T10:14:56.819 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:14:56.819 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:14:56.819 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:14:56.819 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.819 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.819 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:207 3.as0 3:5c566071:::smithi18925896-28:head [copy-from ver 6] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:14:56.819 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:52.694855+0000", 2024-09-18T10:14:56.819 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.0451534110000003, 2024-09-18T10:14:56.819 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.81394701899999999, 2024-09-18T10:14:56.819 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:14:56.819 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:14:56.819 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:14:56.819 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:14:56.820 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:14:56.820 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 207 2024-09-18T10:14:56.820 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.820 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:14:56.820 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.820 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:14:56.820 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694855+0000", 2024-09-18T10:14:56.820 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:14:56.820 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.820 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.820 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:14:56.820 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694855+0000", 2024-09-18T10:14:56.820 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:14:56.821 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.821 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.821 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:14:56.821 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694852+0000", 2024-09-18T10:14:56.821 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999967 2024-09-18T10:14:56.821 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.821 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.821 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:14:56.821 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694859+0000", 2024-09-18T10:14:56.821 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.08e-06 2024-09-18T10:14:56.821 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.821 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.821 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:14:56.822 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694866+0000", 2024-09-18T10:14:56.822 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9249999999999998e-06 2024-09-18T10:14:56.822 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.822 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.822 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:14:56.822 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694870+0000", 2024-09-18T10:14:56.822 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0330000000000002e-06 2024-09-18T10:14:56.822 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.822 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.822 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:14:56.822 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.900743+0000", 2024-09-18T10:14:56.822 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20587325400000001 2024-09-18T10:14:56.822 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.822 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.823 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:14:56.823 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.901026+0000", 2024-09-18T10:14:56.823 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00028313600000000002 2024-09-18T10:14:56.823 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.823 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.823 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:14:56.823 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.204074+0000", 2024-09-18T10:14:56.823 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30304722299999998 2024-09-18T10:14:56.823 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.823 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.823 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:14:56.823 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.207280+0000", 2024-09-18T10:14:56.823 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0032067049999999998 2024-09-18T10:14:56.823 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.824 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.824 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:14:56.824 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.508672+0000", 2024-09-18T10:14:56.824 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30139174800000001 2024-09-18T10:14:56.824 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.824 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.824 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:14:56.824 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.508802+0000", 2024-09-18T10:14:56.824 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013037500000000001 2024-09-18T10:14:56.824 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:14:56.824 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:14:56.824 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:14:56.824 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.824 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.825 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:208 3.as0 3:5c566071:::smithi18925896-28:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:14:56.825 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:52.694875+0000", 2024-09-18T10:14:56.825 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.0451342500000003, 2024-09-18T10:14:56.825 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.51199464299999997, 2024-09-18T10:14:56.825 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:14:56.825 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-18T10:14:56.825 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:14:56.825 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:14:56.825 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:14:56.825 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 208 2024-09-18T10:14:56.825 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.825 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:14:56.825 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.825 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:14:56.826 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694875+0000", 2024-09-18T10:14:56.826 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:14:56.826 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.826 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.826 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:14:56.826 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694875+0000", 2024-09-18T10:14:56.826 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:14:56.826 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.826 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.827 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:14:56.827 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694874+0000", 2024-09-18T10:14:56.827 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999995 2024-09-18T10:14:56.827 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.827 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.827 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:14:56.827 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694876+0000", 2024-09-18T10:14:56.827 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2369999999999999e-06 2024-09-18T10:14:56.827 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.827 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.827 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:14:56.828 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694881+0000", 2024-09-18T10:14:56.828 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0950000000000003e-06 2024-09-18T10:14:56.828 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.828 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.828 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:14:56.828 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694887+0000", 2024-09-18T10:14:56.829 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5690000000000004e-06 2024-09-18T10:14:56.829 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.829 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.829 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:14:56.829 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.901475+0000", 2024-09-18T10:14:56.829 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20658803000000001 2024-09-18T10:14:56.829 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.829 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.829 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2024-09-18T10:14:56.829 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.901550+0000", 2024-09-18T10:14:56.829 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5020999999999999e-05 2024-09-18T10:14:56.829 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.829 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.829 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:14:56.830 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.206415+0000", 2024-09-18T10:14:56.830 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30486486000000002 2024-09-18T10:14:56.830 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.830 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.830 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:14:56.830 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.206634+0000", 2024-09-18T10:14:56.830 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00021925499999999999 2024-09-18T10:14:56.830 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.830 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.830 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:14:56.830 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.206869+0000", 2024-09-18T10:14:56.830 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00023503599999999999 2024-09-18T10:14:56.830 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:14:56.830 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:14:56.831 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:14:56.831 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.831 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.831 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpReadReply(3.as0 36/32 ECSubReadReply(tid=30, attrs_read=0))", 2024-09-18T10:14:56.831 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:52.999782+0000", 2024-09-18T10:14:56.831 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.7402268469999997, 2024-09-18T10:14:56.831 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20160051100000001, 2024-09-18T10:14:56.831 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:14:56.831 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:14:56.831 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:14:56.831 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.831 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:14:56.831 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.999782+0000", 2024-09-18T10:14:56.832 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:14:56.832 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.832 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.832 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:14:56.832 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.999782+0000", 2024-09-18T10:14:56.832 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:14:56.832 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.832 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.832 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:14:56.832 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.999781+0000", 2024-09-18T10:14:56.832 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:14:56.832 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.832 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.832 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:14:56.833 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.999874+0000", 2024-09-18T10:14:56.833 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2983000000000001e-05 2024-09-18T10:14:56.833 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.833 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.833 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:14:56.833 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.999912+0000", 2024-09-18T10:14:56.833 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8077999999999997e-05 2024-09-18T10:14:56.833 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.833 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.833 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:14:56.833 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.999922+0000", 2024-09-18T10:14:56.833 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8099999999999992e-06 2024-09-18T10:14:56.833 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.834 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.834 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:14:56.834 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.200847+0000", 2024-09-18T10:14:56.834 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.200925138 2024-09-18T10:14:56.834 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.834 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.834 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:14:56.834 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.201382+0000", 2024-09-18T10:14:56.834 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00053538499999999998 2024-09-18T10:14:56.834 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:14:56.834 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:14:56.834 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:14:56.834 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.834 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.835 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWriteReply(3.as0 37/32 ECSubWriteReply(tid=33, last_complete=37'9, committed=1, applied=1))", 2024-09-18T10:14:56.836 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:53.312930+0000", 2024-09-18T10:14:56.836 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.4270789669999999, 2024-09-18T10:14:56.836 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.19672636900000001, 2024-09-18T10:14:56.836 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:14:56.836 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:14:56.836 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:14:56.836 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.836 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:14:56.836 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.312930+0000", 2024-09-18T10:14:56.837 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:14:56.837 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.837 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.837 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:14:56.837 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.312930+0000", 2024-09-18T10:14:56.837 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:14:56.837 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.837 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.837 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:14:56.837 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.312928+0000", 2024-09-18T10:14:56.837 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999981 2024-09-18T10:14:56.837 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.837 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.837 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:14:56.838 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.312941+0000", 2024-09-18T10:14:56.838 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2617e-05 2024-09-18T10:14:56.838 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.838 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.838 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:14:56.838 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.312970+0000", 2024-09-18T10:14:56.838 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9549000000000001e-05 2024-09-18T10:14:56.838 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.838 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.838 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:14:56.838 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.312988+0000", 2024-09-18T10:14:56.838 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7346e-05 2024-09-18T10:14:56.838 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.838 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.839 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:14:56.839 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.508535+0000", 2024-09-18T10:14:56.839 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.19554734900000001 2024-09-18T10:14:56.839 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:14:56.839 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:14:56.839 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:14:56.839 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.509656+0000", 2024-09-18T10:14:56.839 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001121248 2024-09-18T10:14:56.839 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:14:56.839 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:14:56.839 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:14:56.839 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:14:56.839 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:14:56.839 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-18T10:14:56.841 INFO:tasks.ceph.osd.8.smithi029.stderr:2024-09-18T10:14:56.834+0000 7f2555c27700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 22192) UID: 0 2024-09-18T10:14:56.842 INFO:tasks.rados.rados.0.smithi189.stdout:99: writing smithi18925896-34 from 1163264 to 1540096 tid 2 2024-09-18T10:14:56.842 INFO:tasks.rados.rados.0.smithi189.stdout:100: read oid 15 snap -1 2024-09-18T10:14:56.842 INFO:tasks.rados.rados.0.smithi189.stdout:100: expect (ObjNum 10 snap 0 seq_num 10) 2024-09-18T10:14:56.842 INFO:tasks.rados.rados.0.smithi189.stdout:101: snap_create 2024-09-18T10:14:56.842 INFO:tasks.rados.rados.0.smithi189.stdout:95: finishing write tid 2 to smithi18925896-39 2024-09-18T10:14:56.842 INFO:tasks.rados.rados.0.smithi189.stdout:95: finishing write tid 3 to smithi18925896-39 2024-09-18T10:14:56.842 INFO:tasks.rados.rados.0.smithi189.stdout:95: finishing write tid 4 to smithi18925896-39 2024-09-18T10:14:56.842 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 39 v 9 (ObjNum 57 snap 4 seq_num 57) dirty exists 2024-09-18T10:14:56.842 INFO:tasks.rados.rados.0.smithi189.stdout:95: left oid 39 (ObjNum 57 snap 4 seq_num 57) 2024-09-18T10:14:56.843 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T10:14:56.941 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-18T10:14:56.937+0000 7f578c52f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22134) UID: 0 2024-09-18T10:14:56.955 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-09-18T10:14:56.955 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-09-18T10:14:56.956 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-09-18T10:14:56.956 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-09-18T10:14:56.965 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T10:14:57.037 INFO:tasks.rados.rados.0.smithi189.stdout:100: expect (ObjNum 10 snap 0 seq_num 10) 2024-09-18T10:14:57.042 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-18T10:14:57.037+0000 7f1981873700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22132) UID: 0 2024-09-18T10:14:57.065 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-09-18T10:14:57.065 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-09-18T10:14:57.065 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-09-18T10:14:57.065 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-09-18T10:14:57.066 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-09-18T10:14:57.075 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T10:14:57.143 INFO:tasks.ceph.osd.14.smithi154.stderr:2024-09-18T10:14:57.140+0000 7f75a19e7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 22093) UID: 0 2024-09-18T10:14:57.176 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-09-18T10:14:57.177 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-09-18T10:14:57.177 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-09-18T10:14:57.177 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-09-18T10:14:57.177 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.177 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.9s1 36/32 ECSubWrite(tid=10, reqid=client.4632.0:112, at_version=36'7, trim_to=0'0, roll_forward_to=35'6))", 2024-09-18T10:14:57.177 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:50.718230+0000", 2024-09-18T10:14:57.177 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 6.4554368029999996, 2024-09-18T10:14:57.177 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.103673577, 2024-09-18T10:14:57.177 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:57.177 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:14:57.177 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:57.178 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.178 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:57.178 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.718230+0000", 2024-09-18T10:14:57.178 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:57.178 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.178 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.178 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:57.178 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.718230+0000", 2024-09-18T10:14:57.178 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:57.178 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.178 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.178 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:57.178 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.718229+0000", 2024-09-18T10:14:57.178 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:14:57.179 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.179 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.179 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:57.179 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.718513+0000", 2024-09-18T10:14:57.179 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000284405 2024-09-18T10:14:57.179 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.179 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.179 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:57.179 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.718544+0000", 2024-09-18T10:14:57.179 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1210000000000001e-05 2024-09-18T10:14:57.179 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.179 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.179 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:57.180 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.818638+0000", 2024-09-18T10:14:57.180 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10009378300000001 2024-09-18T10:14:57.180 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.180 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.180 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:57.180 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.818774+0000", 2024-09-18T10:14:57.180 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000135986 2024-09-18T10:14:57.180 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.180 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.180 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:14:57.180 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.818810+0000", 2024-09-18T10:14:57.180 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.6025000000000002e-05 2024-09-18T10:14:57.180 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.180 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.181 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:14:57.181 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.821869+0000", 2024-09-18T10:14:57.181 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030589049999999998 2024-09-18T10:14:57.181 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.181 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.181 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:57.181 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.821904+0000", 2024-09-18T10:14:57.181 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.4921e-05 2024-09-18T10:14:57.181 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:57.181 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:57.181 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:57.181 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.181 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.182 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.6s3 36/32 ECSubWrite(tid=13, reqid=client.4632.0:80, at_version=36'8, trim_to=0'0, roll_forward_to=36'7))", 2024-09-18T10:14:57.182 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:50.804050+0000", 2024-09-18T10:14:57.182 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 6.3696176409999996, 2024-09-18T10:14:57.182 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.102659627, 2024-09-18T10:14:57.182 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:57.182 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:14:57.182 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:57.182 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.182 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:57.182 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.804050+0000", 2024-09-18T10:14:57.182 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:57.182 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.182 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.182 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:57.183 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.804050+0000", 2024-09-18T10:14:57.183 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:57.183 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.183 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.183 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:57.183 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.804048+0000", 2024-09-18T10:14:57.183 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:14:57.183 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.183 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.183 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:57.183 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.804175+0000", 2024-09-18T10:14:57.183 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00012670199999999999 2024-09-18T10:14:57.183 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.184 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.184 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:57.184 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.804212+0000", 2024-09-18T10:14:57.184 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.7033000000000003e-05 2024-09-18T10:14:57.184 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.184 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.184 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:57.184 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.804225+0000", 2024-09-18T10:14:57.184 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3185e-05 2024-09-18T10:14:57.184 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.184 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.184 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:57.184 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.804775+0000", 2024-09-18T10:14:57.184 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00055026999999999995 2024-09-18T10:14:57.185 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.185 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.185 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:14:57.185 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.804790+0000", 2024-09-18T10:14:57.185 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4538999999999999e-05 2024-09-18T10:14:57.185 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.185 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.185 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:14:57.185 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.906639+0000", 2024-09-18T10:14:57.185 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10184871600000001 2024-09-18T10:14:57.185 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.185 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.185 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:57.186 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.906709+0000", 2024-09-18T10:14:57.186 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0660000000000004e-05 2024-09-18T10:14:57.186 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:57.186 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:57.186 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:57.186 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.186 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.186 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.6s3 36/32 ECSubWrite(tid=14, reqid=client.4632.0:121, at_version=36'9, trim_to=0'0, roll_forward_to=36'7))", 2024-09-18T10:14:57.186 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:50.804306+0000", 2024-09-18T10:14:57.186 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 6.3693612330000002, 2024-09-18T10:14:57.186 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.103984513, 2024-09-18T10:14:57.186 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:57.186 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:14:57.187 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:57.187 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.187 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:57.187 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.804306+0000", 2024-09-18T10:14:57.187 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:57.187 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.187 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.187 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:57.187 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.804306+0000", 2024-09-18T10:14:57.187 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:57.187 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.187 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.187 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:57.187 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.804305+0000", 2024-09-18T10:14:57.188 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:14:57.188 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.188 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.188 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:57.188 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.804579+0000", 2024-09-18T10:14:57.188 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00027352300000000002 2024-09-18T10:14:57.188 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.188 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.188 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:57.188 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.804613+0000", 2024-09-18T10:14:57.188 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.4329999999999998e-05 2024-09-18T10:14:57.188 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.188 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.188 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:57.189 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.804625+0000", 2024-09-18T10:14:57.189 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2335e-05 2024-09-18T10:14:57.189 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.189 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.189 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:57.189 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.904498+0000", 2024-09-18T10:14:57.189 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.099872796999999999 2024-09-18T10:14:57.189 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.189 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.189 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:14:57.189 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.904531+0000", 2024-09-18T10:14:57.189 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.3182000000000002e-05 2024-09-18T10:14:57.189 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.190 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.190 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:14:57.190 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.908265+0000", 2024-09-18T10:14:57.190 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0037334149999999999 2024-09-18T10:14:57.190 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.190 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.190 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:57.190 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.908291+0000", 2024-09-18T10:14:57.190 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5741e-05 2024-09-18T10:14:57.190 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:57.190 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:57.190 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:57.190 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.190 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.191 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.6s3 36/32 ECSubWrite(tid=15, reqid=client.4632.0:122, at_version=36'10, trim_to=0'0, roll_forward_to=36'7))", 2024-09-18T10:14:57.191 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:50.804653+0000", 2024-09-18T10:14:57.191 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 6.3690139080000003, 2024-09-18T10:14:57.191 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.108947892, 2024-09-18T10:14:57.191 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:57.191 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:14:57.191 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:57.191 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.191 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:57.191 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.804653+0000", 2024-09-18T10:14:57.191 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:57.191 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.191 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.191 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:57.192 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.804653+0000", 2024-09-18T10:14:57.192 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:57.192 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.192 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.192 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:57.192 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.804653+0000", 2024-09-18T10:14:57.192 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:14:57.192 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.192 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.192 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:57.192 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.804677+0000", 2024-09-18T10:14:57.192 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4043e-05 2024-09-18T10:14:57.192 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.192 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.193 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:57.193 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.804691+0000", 2024-09-18T10:14:57.193 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.402e-05 2024-09-18T10:14:57.193 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.193 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.193 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:57.193 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.906029+0000", 2024-09-18T10:14:57.193 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.101337911 2024-09-18T10:14:57.193 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.193 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.193 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:57.193 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.906208+0000", 2024-09-18T10:14:57.193 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000179292 2024-09-18T10:14:57.194 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.194 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.194 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:14:57.194 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.906232+0000", 2024-09-18T10:14:57.194 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.3739000000000001e-05 2024-09-18T10:14:57.194 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.194 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.194 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:14:57.194 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.913534+0000", 2024-09-18T10:14:57.194 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0073023029999999996 2024-09-18T10:14:57.194 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.194 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.194 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:57.194 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.913601+0000", 2024-09-18T10:14:57.195 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.7314e-05 2024-09-18T10:14:57.195 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:57.195 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:57.195 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:57.195 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.195 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.195 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.fs4 36/32 ECSubWrite(tid=21, reqid=client.4632.0:109, at_version=36'3, trim_to=0'0, roll_forward_to=36'2))", 2024-09-18T10:14:57.195 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:50.818812+0000", 2024-09-18T10:14:57.195 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 6.3548556630000004, 2024-09-18T10:14:57.195 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10126384400000001, 2024-09-18T10:14:57.195 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:57.195 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:14:57.195 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:57.196 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.196 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:57.196 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.818812+0000", 2024-09-18T10:14:57.196 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:57.196 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.196 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.196 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:57.196 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.818812+0000", 2024-09-18T10:14:57.196 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:57.196 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.196 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.196 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:57.196 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.818811+0000", 2024-09-18T10:14:57.196 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999995 2024-09-18T10:14:57.197 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.197 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.197 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:57.197 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.818966+0000", 2024-09-18T10:14:57.197 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00015498500000000001 2024-09-18T10:14:57.197 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.197 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.197 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:57.197 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.818992+0000", 2024-09-18T10:14:57.197 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6336e-05 2024-09-18T10:14:57.197 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.197 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.197 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:57.197 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.819002+0000", 2024-09-18T10:14:57.198 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.3510000000000008e-06 2024-09-18T10:14:57.198 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.198 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.198 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:57.198 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.919142+0000", 2024-09-18T10:14:57.198 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100140707 2024-09-18T10:14:57.198 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.198 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.198 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:14:57.198 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.919155+0000", 2024-09-18T10:14:57.198 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2762000000000001e-05 2024-09-18T10:14:57.198 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.198 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.198 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:14:57.199 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.920043+0000", 2024-09-18T10:14:57.199 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00088821600000000005 2024-09-18T10:14:57.199 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.199 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.199 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:57.199 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.920075+0000", 2024-09-18T10:14:57.199 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2144000000000003e-05 2024-09-18T10:14:57.199 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:57.199 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:57.199 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:57.199 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.199 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.199 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.fs4 36/32 ECSubWrite(tid=22, reqid=client.4632.0:110, at_version=36'4, trim_to=0'0, roll_forward_to=36'2))", 2024-09-18T10:14:57.199 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:50.819018+0000", 2024-09-18T10:14:57.200 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 6.3546488410000004, 2024-09-18T10:14:57.200 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10139206000000001, 2024-09-18T10:14:57.200 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:57.200 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:14:57.200 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:57.200 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.200 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:57.200 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.819018+0000", 2024-09-18T10:14:57.200 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:57.200 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.200 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.200 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:57.200 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.819018+0000", 2024-09-18T10:14:57.200 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:57.201 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.201 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.201 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:57.201 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.819018+0000", 2024-09-18T10:14:57.201 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999995 2024-09-18T10:14:57.201 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.201 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.201 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:57.201 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.819037+0000", 2024-09-18T10:14:57.201 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8845e-05 2024-09-18T10:14:57.201 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.201 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.201 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:57.202 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.819045+0000", 2024-09-18T10:14:57.202 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.1429999999999998e-06 2024-09-18T10:14:57.202 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.202 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.202 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:57.202 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.819050+0000", 2024-09-18T10:14:57.202 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.0209999999999999e-06 2024-09-18T10:14:57.202 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.202 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.202 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:57.202 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.919643+0000", 2024-09-18T10:14:57.202 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10059288299999999 2024-09-18T10:14:57.202 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.202 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.203 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:14:57.203 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.919657+0000", 2024-09-18T10:14:57.203 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4571e-05 2024-09-18T10:14:57.203 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.203 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.203 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:14:57.203 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.920385+0000", 2024-09-18T10:14:57.203 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00072738700000000002 2024-09-18T10:14:57.203 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.203 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.203 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:57.203 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.920411+0000", 2024-09-18T10:14:57.203 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5656e-05 2024-09-18T10:14:57.203 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:57.204 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:57.204 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:57.204 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.204 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.204 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.6s3 36/32 ECSubWrite(tid=17, reqid=client.4632.0:122, at_version=0'0, trim_to=0'0, roll_forward_to=36'10))", 2024-09-18T10:14:57.204 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:51.108369+0000", 2024-09-18T10:14:57.204 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 6.0652985560000001, 2024-09-18T10:14:57.204 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.101334356, 2024-09-18T10:14:57.204 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:57.204 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:14:57.204 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:57.204 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.204 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:57.205 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.108369+0000", 2024-09-18T10:14:57.205 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:57.205 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.205 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.205 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:57.205 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.108369+0000", 2024-09-18T10:14:57.205 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:57.205 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.205 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.205 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:57.205 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.108367+0000", 2024-09-18T10:14:57.205 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:14:57.205 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.205 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.206 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:57.206 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.108407+0000", 2024-09-18T10:14:57.206 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.0138000000000001e-05 2024-09-18T10:14:57.206 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.206 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.206 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:57.206 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.108452+0000", 2024-09-18T10:14:57.206 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.5491000000000002e-05 2024-09-18T10:14:57.206 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.206 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.206 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:57.206 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.108473+0000", 2024-09-18T10:14:57.206 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.0550000000000001e-05 2024-09-18T10:14:57.207 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.207 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.207 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:57.207 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.108596+0000", 2024-09-18T10:14:57.207 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00012284700000000001 2024-09-18T10:14:57.207 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.207 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.207 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:14:57.207 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.108630+0000", 2024-09-18T10:14:57.207 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.4511000000000002e-05 2024-09-18T10:14:57.207 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.207 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.207 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:14:57.208 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.209655+0000", 2024-09-18T10:14:57.208 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.101024348 2024-09-18T10:14:57.208 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.208 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.208 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:57.208 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.209703+0000", 2024-09-18T10:14:57.208 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.8440999999999999e-05 2024-09-18T10:14:57.208 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:57.208 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:57.208 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:57.208 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.208 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.208 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.9s1 36/32 ECSubWrite(tid=15, reqid=client.4632.0:142, at_version=36'11, trim_to=0'0, roll_forward_to=36'9))", 2024-09-18T10:14:57.209 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:51.129486+0000", 2024-09-18T10:14:57.209 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 6.0441814760000003, 2024-09-18T10:14:57.209 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10248911300000001, 2024-09-18T10:14:57.209 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:57.209 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:14:57.209 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:57.209 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.209 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:57.209 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.129486+0000", 2024-09-18T10:14:57.209 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:57.209 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.209 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.209 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:57.209 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.129486+0000", 2024-09-18T10:14:57.210 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:57.210 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.210 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.210 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:57.210 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.129484+0000", 2024-09-18T10:14:57.210 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:14:57.210 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.210 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.210 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:57.210 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.129675+0000", 2024-09-18T10:14:57.210 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000191585 2024-09-18T10:14:57.210 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.210 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.211 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:57.211 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.129709+0000", 2024-09-18T10:14:57.211 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.3837999999999997e-05 2024-09-18T10:14:57.211 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.211 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.211 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:57.211 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.129724+0000", 2024-09-18T10:14:57.211 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4927e-05 2024-09-18T10:14:57.211 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.211 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.211 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:57.211 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.129866+0000", 2024-09-18T10:14:57.211 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00014166999999999999 2024-09-18T10:14:57.211 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.212 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.212 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:14:57.212 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.129883+0000", 2024-09-18T10:14:57.212 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.6674999999999999e-05 2024-09-18T10:14:57.212 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.212 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.212 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:14:57.212 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.231945+0000", 2024-09-18T10:14:57.212 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.102061982 2024-09-18T10:14:57.212 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.212 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.212 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:57.212 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.231975+0000", 2024-09-18T10:14:57.213 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0412999999999998e-05 2024-09-18T10:14:57.213 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:57.213 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:57.213 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:57.213 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.213 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.213 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.9s1 36/32 ECSubWrite(tid=16, reqid=client.4632.0:143, at_version=36'12, trim_to=0'0, roll_forward_to=36'9))", 2024-09-18T10:14:57.213 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:51.130949+0000", 2024-09-18T10:14:57.213 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 6.0427181440000002, 2024-09-18T10:14:57.213 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.202020333, 2024-09-18T10:14:57.213 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:57.213 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:14:57.213 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:57.214 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.214 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:57.214 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.130949+0000", 2024-09-18T10:14:57.214 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:57.214 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.214 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.214 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:57.214 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.130949+0000", 2024-09-18T10:14:57.214 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:57.214 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.214 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.214 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:57.214 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.130947+0000", 2024-09-18T10:14:57.214 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:14:57.215 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.215 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.215 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:57.215 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.131149+0000", 2024-09-18T10:14:57.215 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00020114800000000001 2024-09-18T10:14:57.215 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.215 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.215 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:57.215 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.131182+0000", 2024-09-18T10:14:57.215 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.3844000000000003e-05 2024-09-18T10:14:57.215 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.215 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.215 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:57.215 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.131204+0000", 2024-09-18T10:14:57.216 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.1753999999999999e-05 2024-09-18T10:14:57.216 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.216 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.216 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:57.216 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.231367+0000", 2024-09-18T10:14:57.216 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100162472 2024-09-18T10:14:57.216 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.216 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.216 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:14:57.216 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.231380+0000", 2024-09-18T10:14:57.216 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3624e-05 2024-09-18T10:14:57.216 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.217 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.217 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:14:57.217 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.332934+0000", 2024-09-18T10:14:57.217 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.101553714 2024-09-18T10:14:57.217 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.217 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.217 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:57.217 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.332969+0000", 2024-09-18T10:14:57.217 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5525999999999998e-05 2024-09-18T10:14:57.217 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:57.217 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:57.217 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:57.217 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.217 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.218 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.9s1 36/32 ECSubWrite(tid=17, reqid=client.4632.0:144, at_version=36'13, trim_to=0'0, roll_forward_to=36'9))", 2024-09-18T10:14:57.218 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:51.131916+0000", 2024-09-18T10:14:57.218 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 6.0417513170000001, 2024-09-18T10:14:57.218 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20110955899999999, 2024-09-18T10:14:57.218 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:57.218 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:14:57.218 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:57.218 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.218 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:57.218 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.131916+0000", 2024-09-18T10:14:57.218 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:57.218 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.218 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.219 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:57.219 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.131916+0000", 2024-09-18T10:14:57.219 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:57.219 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.219 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.219 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:57.219 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.131915+0000", 2024-09-18T10:14:57.219 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:14:57.219 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.219 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.219 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:57.219 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.131949+0000", 2024-09-18T10:14:57.219 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.4576999999999998e-05 2024-09-18T10:14:57.219 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.220 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.220 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:57.220 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.131979+0000", 2024-09-18T10:14:57.220 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9454e-05 2024-09-18T10:14:57.220 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.220 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.220 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:57.220 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.131992+0000", 2024-09-18T10:14:57.220 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3029e-05 2024-09-18T10:14:57.220 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.220 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.220 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:57.220 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.332112+0000", 2024-09-18T10:14:57.220 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.200120295 2024-09-18T10:14:57.221 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.221 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.221 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:14:57.221 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.332126+0000", 2024-09-18T10:14:57.221 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4494999999999999e-05 2024-09-18T10:14:57.221 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.221 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.221 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:14:57.221 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.332996+0000", 2024-09-18T10:14:57.221 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00086989000000000001 2024-09-18T10:14:57.221 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.221 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.221 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:57.221 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.333026+0000", 2024-09-18T10:14:57.222 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9159999999999999e-05 2024-09-18T10:14:57.222 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:57.222 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:57.222 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:57.222 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.222 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.222 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpRead(3.4s3 36/32 ECSubRead(tid=11, to_read={3:2650be24:::smithi18925896-45:head=0,4096,0}, subchunks={3:2650be24:::smithi18925896-45:head=[0,1]}, attrs_to_read=))", 2024-09-18T10:14:57.222 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:51.425236+0000", 2024-09-18T10:14:57.222 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 5.7484312299999996, 2024-09-18T10:14:57.222 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.200812289, 2024-09-18T10:14:57.222 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:57.222 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:14:57.222 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:57.223 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.223 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:57.223 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.425236+0000", 2024-09-18T10:14:57.223 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:57.223 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.223 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.223 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:57.223 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.425236+0000", 2024-09-18T10:14:57.223 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:57.223 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.223 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.223 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:57.223 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.425234+0000", 2024-09-18T10:14:57.223 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:14:57.224 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.224 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.224 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:57.224 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.425253+0000", 2024-09-18T10:14:57.224 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8468999999999999e-05 2024-09-18T10:14:57.224 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.224 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.224 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:57.224 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.425289+0000", 2024-09-18T10:14:57.224 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.6514999999999999e-05 2024-09-18T10:14:57.224 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.224 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.224 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:57.224 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.525424+0000", 2024-09-18T10:14:57.225 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100135157 2024-09-18T10:14:57.225 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.225 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.225 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:57.225 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.625672+0000", 2024-09-18T10:14:57.225 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100247555 2024-09-18T10:14:57.225 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.225 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.225 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:57.225 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.626048+0000", 2024-09-18T10:14:57.225 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00037635000000000001 2024-09-18T10:14:57.225 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:57.225 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:57.226 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:57.226 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.226 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.226 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.9s1 36/32 ECSubWrite(tid=19, reqid=client.4632.0:144, at_version=0'0, trim_to=0'0, roll_forward_to=36'13))", 2024-09-18T10:14:57.226 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:51.534793+0000", 2024-09-18T10:14:57.226 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 5.6388740760000005, 2024-09-18T10:14:57.226 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10177501, 2024-09-18T10:14:57.226 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:57.226 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:14:57.226 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:57.226 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.226 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:57.226 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.534793+0000", 2024-09-18T10:14:57.226 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:57.227 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.227 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.227 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:57.227 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.534793+0000", 2024-09-18T10:14:57.227 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:57.227 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.227 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.227 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:57.227 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.534792+0000", 2024-09-18T10:14:57.227 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:14:57.227 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.227 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.227 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:57.227 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.534818+0000", 2024-09-18T10:14:57.228 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6012000000000001e-05 2024-09-18T10:14:57.228 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.228 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.228 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:57.228 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.534848+0000", 2024-09-18T10:14:57.228 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0716000000000002e-05 2024-09-18T10:14:57.228 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.228 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.228 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:57.228 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.534914+0000", 2024-09-18T10:14:57.228 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.5900000000000003e-05 2024-09-18T10:14:57.228 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.228 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.228 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:57.229 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.635106+0000", 2024-09-18T10:14:57.229 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100191901 2024-09-18T10:14:57.229 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.229 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.229 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:14:57.229 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.635125+0000", 2024-09-18T10:14:57.229 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.9236e-05 2024-09-18T10:14:57.229 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.229 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.229 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:14:57.229 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.636517+0000", 2024-09-18T10:14:57.229 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0013919099999999999 2024-09-18T10:14:57.229 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.230 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.230 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:57.230 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.636568+0000", 2024-09-18T10:14:57.230 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.0868000000000002e-05 2024-09-18T10:14:57.230 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:57.230 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:57.230 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:57.230 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.230 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.230 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.9s1 37/32 ECSubWrite(tid=25, reqid=client.4632.0:185, at_version=37'15, trim_to=0'0, roll_forward_to=36'13))", 2024-09-18T10:14:57.230 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:52.695116+0000", 2024-09-18T10:14:57.230 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 4.4785508739999997, 2024-09-18T10:14:57.230 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.102252275, 2024-09-18T10:14:57.230 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:57.230 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:14:57.231 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:57.231 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.231 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:57.231 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.695116+0000", 2024-09-18T10:14:57.231 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:57.231 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.231 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.231 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:57.231 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.695116+0000", 2024-09-18T10:14:57.231 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:57.231 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.231 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.231 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:57.232 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.695116+0000", 2024-09-18T10:14:57.232 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999995 2024-09-18T10:14:57.232 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.232 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.232 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:57.232 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.695188+0000", 2024-09-18T10:14:57.232 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.2415000000000004e-05 2024-09-18T10:14:57.232 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.232 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.232 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:57.232 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.695211+0000", 2024-09-18T10:14:57.232 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.2963e-05 2024-09-18T10:14:57.232 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.233 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.233 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:57.233 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.695222+0000", 2024-09-18T10:14:57.233 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0319999999999999e-05 2024-09-18T10:14:57.233 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.233 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.233 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:57.233 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.795480+0000", 2024-09-18T10:14:57.233 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100257736 2024-09-18T10:14:57.233 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.233 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.234 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:14:57.234 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.795519+0000", 2024-09-18T10:14:57.234 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.9248000000000001e-05 2024-09-18T10:14:57.234 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.234 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.234 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:14:57.234 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.797296+0000", 2024-09-18T10:14:57.234 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001776775 2024-09-18T10:14:57.234 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.234 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.234 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:57.235 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.797369+0000", 2024-09-18T10:14:57.235 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.3170999999999995e-05 2024-09-18T10:14:57.235 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:57.235 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:57.235 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:57.235 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.235 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.235 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.9s1 37/32 ECSubWrite(tid=26, reqid=client.4632.0:185, at_version=0'0, trim_to=0'0, roll_forward_to=37'15))", 2024-09-18T10:14:57.235 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:52.799766+0000", 2024-09-18T10:14:57.235 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 4.3739015009999997, 2024-09-18T10:14:57.235 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10681106, 2024-09-18T10:14:57.235 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:57.235 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:14:57.236 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:57.236 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.236 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:57.236 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.799766+0000", 2024-09-18T10:14:57.236 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:57.236 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.236 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.236 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:57.236 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.799766+0000", 2024-09-18T10:14:57.236 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:57.236 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.236 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.236 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:57.236 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.799764+0000", 2024-09-18T10:14:57.237 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:14:57.237 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.237 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.237 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:57.237 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.799789+0000", 2024-09-18T10:14:57.237 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4335e-05 2024-09-18T10:14:57.237 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.237 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.237 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:57.237 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.799822+0000", 2024-09-18T10:14:57.237 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.3565999999999997e-05 2024-09-18T10:14:57.237 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.237 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.238 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:57.238 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.799841+0000", 2024-09-18T10:14:57.238 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.9272000000000001e-05 2024-09-18T10:14:57.238 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.238 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.238 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:57.238 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.903502+0000", 2024-09-18T10:14:57.238 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10366048999999999 2024-09-18T10:14:57.238 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.238 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.238 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:14:57.238 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.903526+0000", 2024-09-18T10:14:57.238 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.3958999999999999e-05 2024-09-18T10:14:57.238 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.239 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.239 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:14:57.239 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.906532+0000", 2024-09-18T10:14:57.239 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030064279999999998 2024-09-18T10:14:57.239 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.239 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.239 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:57.239 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.906577+0000", 2024-09-18T10:14:57.239 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4573000000000002e-05 2024-09-18T10:14:57.239 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:57.239 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:57.239 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:57.239 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.240 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.240 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.4s3 37/32 ECSubWrite(tid=12, reqid=client.4632.0:188, at_version=37'8, trim_to=0'0, roll_forward_to=36'6))", 2024-09-18T10:14:57.240 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:52.801107+0000", 2024-09-18T10:14:57.240 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 4.3725600560000002, 2024-09-18T10:14:57.240 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.102732592, 2024-09-18T10:14:57.240 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:57.240 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:14:57.240 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:57.240 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.240 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:57.240 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.801107+0000", 2024-09-18T10:14:57.240 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:57.241 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.241 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.241 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:57.241 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.801107+0000", 2024-09-18T10:14:57.241 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:57.241 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.241 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.241 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:57.241 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.801107+0000", 2024-09-18T10:14:57.241 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999995 2024-09-18T10:14:57.241 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.241 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.241 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:57.242 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.801336+0000", 2024-09-18T10:14:57.242 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00022954 2024-09-18T10:14:57.242 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.242 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.242 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:57.242 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.801390+0000", 2024-09-18T10:14:57.242 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.3634000000000002e-05 2024-09-18T10:14:57.242 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.242 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.242 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:57.242 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.801414+0000", 2024-09-18T10:14:57.242 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4189000000000002e-05 2024-09-18T10:14:57.242 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.242 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.243 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:57.243 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.901637+0000", 2024-09-18T10:14:57.243 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100223012 2024-09-18T10:14:57.243 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.243 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.243 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:14:57.243 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.901666+0000", 2024-09-18T10:14:57.243 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8929999999999999e-05 2024-09-18T10:14:57.243 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.243 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.243 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:14:57.243 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.903775+0000", 2024-09-18T10:14:57.243 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0021094989999999999 2024-09-18T10:14:57.244 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.244 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.244 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:57.244 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.903840+0000", 2024-09-18T10:14:57.244 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.4478000000000002e-05 2024-09-18T10:14:57.244 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:57.244 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:57.244 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:57.244 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.244 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.244 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.9s1 37/32 ECSubWrite(tid=27, reqid=client.4632.0:186, at_version=37'16, trim_to=0'0, roll_forward_to=37'15))", 2024-09-18T10:14:57.244 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:52.803227+0000", 2024-09-18T10:14:57.244 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 4.3704403450000004, 2024-09-18T10:14:57.244 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.103974409, 2024-09-18T10:14:57.245 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:57.245 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:14:57.245 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:57.245 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.245 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:57.245 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.803227+0000", 2024-09-18T10:14:57.245 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:57.245 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.245 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.245 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:57.245 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.803227+0000", 2024-09-18T10:14:57.245 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:57.245 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.246 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.246 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:57.246 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.803226+0000", 2024-09-18T10:14:57.246 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:14:57.246 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.246 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.246 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:57.246 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.803256+0000", 2024-09-18T10:14:57.246 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0083999999999999e-05 2024-09-18T10:14:57.246 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.246 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.246 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:57.246 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.803275+0000", 2024-09-18T10:14:57.246 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8882999999999999e-05 2024-09-18T10:14:57.247 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.247 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.247 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:57.247 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.803291+0000", 2024-09-18T10:14:57.247 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.6500000000000001e-05 2024-09-18T10:14:57.247 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.247 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.247 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:57.247 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.903965+0000", 2024-09-18T10:14:57.247 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100673314 2024-09-18T10:14:57.247 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.247 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.247 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:14:57.247 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.903981+0000", 2024-09-18T10:14:57.248 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.6337e-05 2024-09-18T10:14:57.248 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.248 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.248 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:14:57.248 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.907161+0000", 2024-09-18T10:14:57.248 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0031794549999999999 2024-09-18T10:14:57.248 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.248 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.248 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:57.248 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.907201+0000", 2024-09-18T10:14:57.248 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.0825999999999998e-05 2024-09-18T10:14:57.248 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:57.248 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:57.248 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:57.249 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.249 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.249 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.9s1 37/32 ECSubWrite(tid=28, reqid=client.4632.0:187, at_version=37'17, trim_to=0'0, roll_forward_to=37'15))", 2024-09-18T10:14:57.249 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:52.804637+0000", 2024-09-18T10:14:57.249 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 4.3690304769999999, 2024-09-18T10:14:57.249 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.102583991, 2024-09-18T10:14:57.249 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:57.249 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:14:57.249 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:57.249 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.249 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:57.249 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.804637+0000", 2024-09-18T10:14:57.249 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:57.250 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.250 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.250 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:57.250 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.804637+0000", 2024-09-18T10:14:57.250 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:57.250 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.250 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.250 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:57.250 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.804636+0000", 2024-09-18T10:14:57.250 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:14:57.250 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.250 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.250 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:57.251 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.804661+0000", 2024-09-18T10:14:57.251 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5621000000000001e-05 2024-09-18T10:14:57.251 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.251 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.251 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:57.251 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.804682+0000", 2024-09-18T10:14:57.251 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.0089e-05 2024-09-18T10:14:57.251 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.251 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.251 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:57.251 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.804689+0000", 2024-09-18T10:14:57.251 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.8290000000000008e-06 2024-09-18T10:14:57.251 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.251 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.252 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:57.252 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.904578+0000", 2024-09-18T10:14:57.252 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.099888633000000004 2024-09-18T10:14:57.252 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.252 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.252 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:14:57.252 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.904606+0000", 2024-09-18T10:14:57.252 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.7759e-05 2024-09-18T10:14:57.252 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.252 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.252 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:14:57.252 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.907206+0000", 2024-09-18T10:14:57.252 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0026003440000000001 2024-09-18T10:14:57.252 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.253 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.253 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:57.253 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.907221+0000", 2024-09-18T10:14:57.253 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4728e-05 2024-09-18T10:14:57.253 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:57.253 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:57.253 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:57.253 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.253 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.253 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.9s1 37/32 ECSubWrite(tid=30, reqid=client.4632.0:187, at_version=0'0, trim_to=0'0, roll_forward_to=37'17))", 2024-09-18T10:14:57.253 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:53.090512+0000", 2024-09-18T10:14:57.253 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 4.0831553759999997, 2024-09-18T10:14:57.253 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.21478031, 2024-09-18T10:14:57.254 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:57.254 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:14:57.254 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:57.254 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.254 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:57.254 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.090512+0000", 2024-09-18T10:14:57.254 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:57.254 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.254 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.254 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:57.254 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.090512+0000", 2024-09-18T10:14:57.254 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:57.254 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.254 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.254 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:57.254 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.090510+0000", 2024-09-18T10:14:57.255 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:14:57.255 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.255 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.255 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:57.255 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.090535+0000", 2024-09-18T10:14:57.255 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4848999999999999e-05 2024-09-18T10:14:57.255 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.255 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.255 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:57.255 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.090566+0000", 2024-09-18T10:14:57.255 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1096000000000002e-05 2024-09-18T10:14:57.255 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.255 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.255 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:57.256 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.190713+0000", 2024-09-18T10:14:57.256 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10014709300000001 2024-09-18T10:14:57.256 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.256 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.256 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:57.256 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.290966+0000", 2024-09-18T10:14:57.256 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10025297399999999 2024-09-18T10:14:57.256 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.256 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.256 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:14:57.256 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.290981+0000", 2024-09-18T10:14:57.256 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4960000000000001e-05 2024-09-18T10:14:57.256 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.256 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.257 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:14:57.257 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.305252+0000", 2024-09-18T10:14:57.257 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.014270738999999999 2024-09-18T10:14:57.257 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.257 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.257 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:57.257 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.305292+0000", 2024-09-18T10:14:57.257 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.0092000000000002e-05 2024-09-18T10:14:57.257 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:57.257 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:57.257 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:57.257 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.257 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.258 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.3s5 37/32 ECSubWrite(tid=11, reqid=client.4632.0:189, at_version=37'8, trim_to=0'0, roll_forward_to=36'6))", 2024-09-18T10:14:57.258 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:53.095919+0000", 2024-09-18T10:14:57.258 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 4.0777485540000002, 2024-09-18T10:14:57.258 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.102455383, 2024-09-18T10:14:57.258 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:57.258 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:14:57.258 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:57.258 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.258 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:57.258 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.095919+0000", 2024-09-18T10:14:57.258 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:57.258 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.258 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.259 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:57.259 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.095919+0000", 2024-09-18T10:14:57.259 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:57.259 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.259 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.259 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:57.259 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.095918+0000", 2024-09-18T10:14:57.259 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:14:57.259 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.259 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.259 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:57.259 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.096107+0000", 2024-09-18T10:14:57.259 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000188705 2024-09-18T10:14:57.259 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.260 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.260 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:57.260 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.096151+0000", 2024-09-18T10:14:57.260 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4218000000000003e-05 2024-09-18T10:14:57.260 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.260 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.260 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:57.260 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.096169+0000", 2024-09-18T10:14:57.260 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7883000000000001e-05 2024-09-18T10:14:57.260 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.260 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.260 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:57.260 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.196399+0000", 2024-09-18T10:14:57.260 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100230347 2024-09-18T10:14:57.261 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.261 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.261 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:14:57.261 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.196418+0000", 2024-09-18T10:14:57.261 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8709e-05 2024-09-18T10:14:57.261 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.261 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.261 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:14:57.261 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.198316+0000", 2024-09-18T10:14:57.261 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001898269 2024-09-18T10:14:57.261 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.261 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.261 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:57.261 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.198374+0000", 2024-09-18T10:14:57.262 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.8012e-05 2024-09-18T10:14:57.262 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:57.262 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:57.262 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:57.262 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.262 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.262 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.9s1 37/32 ECSubWrite(tid=31, reqid=client.4632.0:202, at_version=37'19, trim_to=0'0, roll_forward_to=37'17))", 2024-09-18T10:14:57.262 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:53.203769+0000", 2024-09-18T10:14:57.262 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 3.9698983839999999, 2024-09-18T10:14:57.262 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.102550108, 2024-09-18T10:14:57.262 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:57.262 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:14:57.262 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:57.262 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.262 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:57.263 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.203769+0000", 2024-09-18T10:14:57.263 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:57.263 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.263 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.263 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:57.263 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.203769+0000", 2024-09-18T10:14:57.263 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:57.263 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.263 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.263 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:57.263 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.203767+0000", 2024-09-18T10:14:57.263 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:14:57.263 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.263 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.263 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:57.263 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.203917+0000", 2024-09-18T10:14:57.264 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000149491 2024-09-18T10:14:57.264 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.264 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.264 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:57.264 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.203961+0000", 2024-09-18T10:14:57.264 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4317999999999998e-05 2024-09-18T10:14:57.264 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.264 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.264 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:57.264 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.203979+0000", 2024-09-18T10:14:57.264 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8238999999999999e-05 2024-09-18T10:14:57.264 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.264 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.264 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:57.264 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.304282+0000", 2024-09-18T10:14:57.265 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10030295 2024-09-18T10:14:57.265 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.265 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.265 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:14:57.265 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.304297+0000", 2024-09-18T10:14:57.265 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4452e-05 2024-09-18T10:14:57.265 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.265 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.265 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:14:57.265 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.306253+0000", 2024-09-18T10:14:57.265 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019566969999999999 2024-09-18T10:14:57.265 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.265 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.266 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:57.266 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.306319+0000", 2024-09-18T10:14:57.266 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.5613999999999999e-05 2024-09-18T10:14:57.266 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:57.266 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:57.266 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:57.266 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:57.266 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:57.266 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-09-18T10:14:57.268 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T10:14:57.268 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-18T10:14:57.261+0000 7f1981873700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22132) UID: 0 2024-09-18T10:14:57.368 INFO:tasks.ceph.osd.15.smithi189.stderr:2024-09-18T10:14:57.367+0000 7fd0e725e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 22401) UID: 0 2024-09-18T10:14:57.370 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-09-18T10:14:57.371 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-09-18T10:14:57.371 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-09-18T10:14:57.371 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-09-18T10:14:57.380 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T10:14:57.469 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-18T10:14:57.465+0000 7f14c6a1a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 22138) UID: 0 2024-09-18T10:14:57.477 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-09-18T10:14:57.477 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-09-18T10:14:57.477 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-09-18T10:14:57.477 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-09-18T10:14:57.478 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-09-18T10:14:57.486 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T10:14:57.536 INFO:tasks.rados.rados.0.smithi189.stdout:99: finishing write tid 1 to smithi18925896-34 2024-09-18T10:14:57.542 INFO:tasks.rados.rados.0.smithi189.stdout:99: finishing write tid 2 to smithi18925896-34 2024-09-18T10:14:57.542 INFO:tasks.rados.rados.0.smithi189.stdout:99: finishing write tid 3 to smithi18925896-34 2024-09-18T10:14:57.551 INFO:tasks.rados.rados.0.smithi189.stdout:97: finishing rollback tid 0 to smithi18925896-12 2024-09-18T10:14:57.551 INFO:tasks.rados.rados.0.smithi189.stdout:97: finishing rollback tid 1 to smithi18925896-12 2024-09-18T10:14:57.570 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-18T10:14:57.565+0000 7f1981873700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22132) UID: 0 2024-09-18T10:14:57.592 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-09-18T10:14:57.592 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-09-18T10:14:57.592 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-09-18T10:14:57.593 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-09-18T10:14:57.593 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.593 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-18T10:14:57.593 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:35.293815+0000", 2024-09-18T10:14:57.593 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 22.294933168, 2024-09-18T10:14:57.593 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.513288525, 2024-09-18T10:14:57.593 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:57.593 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:14:57.593 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:14:57.593 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:14:57.593 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:14:57.593 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 52 2024-09-18T10:14:57.593 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.594 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:57.594 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.594 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:57.594 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.293815+0000", 2024-09-18T10:14:57.594 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:57.594 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.594 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.594 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:57.594 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.293815+0000", 2024-09-18T10:14:57.594 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:57.594 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.594 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.594 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:57.594 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.293814+0000", 2024-09-18T10:14:57.594 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:14:57.595 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.595 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.595 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:57.595 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.293822+0000", 2024-09-18T10:14:57.595 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.8089999999999993e-06 2024-09-18T10:14:57.595 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.595 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.595 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:57.595 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.293833+0000", 2024-09-18T10:14:57.595 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1389000000000001e-05 2024-09-18T10:14:57.595 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.595 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.595 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:57.595 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.393937+0000", 2024-09-18T10:14:57.596 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10010415 2024-09-18T10:14:57.596 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.596 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.596 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:57.596 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.704491+0000", 2024-09-18T10:14:57.596 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.310554202 2024-09-18T10:14:57.596 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.596 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.596 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:14:57.596 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.704647+0000", 2024-09-18T10:14:57.596 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00015524399999999999 2024-09-18T10:14:57.596 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.596 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.597 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:14:57.597 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.704859+0000", 2024-09-18T10:14:57.597 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000212664 2024-09-18T10:14:57.597 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.597 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.597 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:14:57.597 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.705935+0000", 2024-09-18T10:14:57.597 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001075505 2024-09-18T10:14:57.597 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.597 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.597 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:14:57.597 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.806952+0000", 2024-09-18T10:14:57.597 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10101700299999999 2024-09-18T10:14:57.597 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.598 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.598 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:14:57.598 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.807041+0000", 2024-09-18T10:14:57.598 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.8811000000000001e-05 2024-09-18T10:14:57.598 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.598 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.598 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:57.598 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.807103+0000", 2024-09-18T10:14:57.598 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.2596999999999997e-05 2024-09-18T10:14:57.598 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:57.598 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:57.598 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:57.598 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.599 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.599 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:70 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T10:14:57.599 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:36.786853+0000", 2024-09-18T10:14:57.599 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 20.801895208000001, 2024-09-18T10:14:57.599 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.51178262299999999, 2024-09-18T10:14:57.599 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:57.599 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:14:57.599 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:14:57.599 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:14:57.599 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:14:57.599 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 70 2024-09-18T10:14:57.599 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.599 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:57.599 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.600 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:57.600 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.786853+0000", 2024-09-18T10:14:57.600 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:57.600 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.600 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.600 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:57.600 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.786853+0000", 2024-09-18T10:14:57.600 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:57.600 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.600 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.600 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:57.600 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.786851+0000", 2024-09-18T10:14:57.600 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:14:57.601 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.601 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.601 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:57.601 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.786861+0000", 2024-09-18T10:14:57.601 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.8700000000000004e-06 2024-09-18T10:14:57.601 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.601 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.601 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:57.601 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.786877+0000", 2024-09-18T10:14:57.601 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.5569e-05 2024-09-18T10:14:57.601 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.601 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.601 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:57.602 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.786887+0000", 2024-09-18T10:14:57.602 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0526000000000001e-05 2024-09-18T10:14:57.602 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.602 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.602 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:57.602 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.292790+0000", 2024-09-18T10:14:57.602 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.50590225600000005 2024-09-18T10:14:57.602 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.602 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.602 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:14:57.602 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.292929+0000", 2024-09-18T10:14:57.602 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00013913200000000001 2024-09-18T10:14:57.602 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.603 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.603 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:14:57.603 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.293137+0000", 2024-09-18T10:14:57.603 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00020826199999999999 2024-09-18T10:14:57.603 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.603 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.603 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:14:57.603 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.297461+0000", 2024-09-18T10:14:57.603 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0043237299999999996 2024-09-18T10:14:57.603 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.603 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.603 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:14:57.603 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.298490+0000", 2024-09-18T10:14:57.603 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0010289769999999999 2024-09-18T10:14:57.604 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.604 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.604 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:14:57.604 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.298572+0000", 2024-09-18T10:14:57.604 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.2151999999999995e-05 2024-09-18T10:14:57.604 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.604 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.604 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:57.604 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.298635+0000", 2024-09-18T10:14:57.604 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.3468999999999998e-05 2024-09-18T10:14:57.604 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:57.604 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:57.604 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:57.605 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.605 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.605 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:71 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T10:14:57.605 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:36.786902+0000", 2024-09-18T10:14:57.605 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 20.801846403999999, 2024-09-18T10:14:57.605 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.51197160200000003, 2024-09-18T10:14:57.605 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:57.605 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:14:57.605 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:14:57.605 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:14:57.605 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:14:57.605 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 71 2024-09-18T10:14:57.605 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.605 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:57.606 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.606 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:57.606 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.786902+0000", 2024-09-18T10:14:57.606 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:57.606 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.606 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.606 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:57.606 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.786902+0000", 2024-09-18T10:14:57.606 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:57.606 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.606 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.606 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:57.606 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.786900+0000", 2024-09-18T10:14:57.606 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:14:57.607 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.607 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.607 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:57.607 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.786909+0000", 2024-09-18T10:14:57.607 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.9430000000000006e-06 2024-09-18T10:14:57.607 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.607 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.607 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:57.607 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.786921+0000", 2024-09-18T10:14:57.607 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1789e-05 2024-09-18T10:14:57.607 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.607 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.607 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:57.607 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.786927+0000", 2024-09-18T10:14:57.608 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5770000000000001e-06 2024-09-18T10:14:57.608 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.608 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.608 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:57.608 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.293768+0000", 2024-09-18T10:14:57.608 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.50684175499999995 2024-09-18T10:14:57.608 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.608 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.608 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:14:57.608 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.293894+0000", 2024-09-18T10:14:57.608 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00012588899999999999 2024-09-18T10:14:57.608 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.608 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.608 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:14:57.609 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.294094+0000", 2024-09-18T10:14:57.609 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000199896 2024-09-18T10:14:57.609 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.609 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.609 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:14:57.609 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.298660+0000", 2024-09-18T10:14:57.609 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.004566082 2024-09-18T10:14:57.609 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.609 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.609 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:14:57.609 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.298739+0000", 2024-09-18T10:14:57.609 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.8269000000000006e-05 2024-09-18T10:14:57.609 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.610 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.610 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:14:57.610 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.298814+0000", 2024-09-18T10:14:57.610 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.5582999999999998e-05 2024-09-18T10:14:57.610 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.610 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.610 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:57.610 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.298873+0000", 2024-09-18T10:14:57.610 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.8989000000000001e-05 2024-09-18T10:14:57.610 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:57.610 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:57.610 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:57.610 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.610 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.611 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T10:14:57.611 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:36.786936+0000", 2024-09-18T10:14:57.611 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 20.801812206000001, 2024-09-18T10:14:57.611 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.71130157999999999, 2024-09-18T10:14:57.611 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:57.611 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:14:57.611 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:14:57.611 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:14:57.611 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:14:57.611 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 72 2024-09-18T10:14:57.611 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.611 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:57.611 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.611 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:57.612 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.786936+0000", 2024-09-18T10:14:57.612 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:57.612 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.612 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.612 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:57.612 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.786936+0000", 2024-09-18T10:14:57.612 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:57.612 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.612 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.612 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:57.612 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.786935+0000", 2024-09-18T10:14:57.612 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999995 2024-09-18T10:14:57.612 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.612 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.613 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:57.613 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787018+0000", 2024-09-18T10:14:57.613 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.2511999999999999e-05 2024-09-18T10:14:57.613 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.613 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.613 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:57.613 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787028+0000", 2024-09-18T10:14:57.613 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0709e-05 2024-09-18T10:14:57.613 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.613 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.613 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:57.613 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787034+0000", 2024-09-18T10:14:57.613 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.7769999999999999e-06 2024-09-18T10:14:57.613 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.614 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.614 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:57.614 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.295095+0000", 2024-09-18T10:14:57.614 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.50806109099999996 2024-09-18T10:14:57.614 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.614 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.614 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:14:57.614 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.295221+0000", 2024-09-18T10:14:57.614 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00012563599999999999 2024-09-18T10:14:57.614 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.614 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.614 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:14:57.614 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.295453+0000", 2024-09-18T10:14:57.614 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000232387 2024-09-18T10:14:57.615 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.615 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.615 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:14:57.615 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.301373+0000", 2024-09-18T10:14:57.615 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0059202279999999996 2024-09-18T10:14:57.615 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.615 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.615 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:14:57.615 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.498044+0000", 2024-09-18T10:14:57.615 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.19667104699999999 2024-09-18T10:14:57.615 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.615 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.615 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:14:57.615 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.498133+0000", 2024-09-18T10:14:57.616 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.9044999999999999e-05 2024-09-18T10:14:57.616 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.616 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.616 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:57.616 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.498237+0000", 2024-09-18T10:14:57.616 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00010385100000000001 2024-09-18T10:14:57.616 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:57.616 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:57.616 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:57.616 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.616 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.616 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T10:14:57.616 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:36.787042+0000", 2024-09-18T10:14:57.617 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 20.801705499000001, 2024-09-18T10:14:57.617 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.71152325100000002, 2024-09-18T10:14:57.617 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:57.617 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:14:57.617 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:14:57.617 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:14:57.617 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:14:57.617 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 73 2024-09-18T10:14:57.617 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.617 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:57.617 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.617 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:57.617 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787042+0000", 2024-09-18T10:14:57.618 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:57.618 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.618 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.618 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:57.618 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787042+0000", 2024-09-18T10:14:57.618 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:57.618 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.618 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.618 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:57.618 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787042+0000", 2024-09-18T10:14:57.618 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:14:57.618 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.618 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.618 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:57.618 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787048+0000", 2024-09-18T10:14:57.619 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.8900000000000001e-06 2024-09-18T10:14:57.619 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.619 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.619 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:57.619 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787057+0000", 2024-09-18T10:14:57.619 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.4430000000000003e-06 2024-09-18T10:14:57.619 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.619 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.619 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:57.619 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787063+0000", 2024-09-18T10:14:57.619 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.6459999999999998e-06 2024-09-18T10:14:57.619 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.619 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.619 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:57.620 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.297548+0000", 2024-09-18T10:14:57.620 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.51048589499999997 2024-09-18T10:14:57.620 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.620 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.620 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:14:57.620 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.297668+0000", 2024-09-18T10:14:57.620 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00011917 2024-09-18T10:14:57.620 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.620 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.620 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:14:57.620 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.297891+0000", 2024-09-18T10:14:57.620 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00022301999999999999 2024-09-18T10:14:57.620 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.620 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.621 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:14:57.621 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.301383+0000", 2024-09-18T10:14:57.621 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.003492762 2024-09-18T10:14:57.621 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.621 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.621 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:14:57.621 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.498374+0000", 2024-09-18T10:14:57.621 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.19699053899999999 2024-09-18T10:14:57.621 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.621 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.621 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:14:57.621 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.498506+0000", 2024-09-18T10:14:57.621 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00013246899999999999 2024-09-18T10:14:57.622 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.622 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.622 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:57.622 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.498566+0000", 2024-09-18T10:14:57.622 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.9274000000000002e-05 2024-09-18T10:14:57.622 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:57.622 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:57.622 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:57.622 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.622 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.622 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T10:14:57.622 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:36.787128+0000", 2024-09-18T10:14:57.622 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 20.801620191000001, 2024-09-18T10:14:57.622 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.71169541800000002, 2024-09-18T10:14:57.622 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:57.623 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:14:57.623 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:14:57.623 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:14:57.623 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:14:57.623 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 74 2024-09-18T10:14:57.623 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.623 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:57.623 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.623 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:57.623 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787128+0000", 2024-09-18T10:14:57.623 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:57.623 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.623 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.623 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:57.624 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787128+0000", 2024-09-18T10:14:57.624 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:57.624 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.624 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.624 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:57.624 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787127+0000", 2024-09-18T10:14:57.624 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:14:57.624 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.624 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.624 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:57.624 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787133+0000", 2024-09-18T10:14:57.624 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.6930000000000004e-06 2024-09-18T10:14:57.624 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.624 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.625 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:57.625 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787142+0000", 2024-09-18T10:14:57.625 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.1959999999999997e-06 2024-09-18T10:14:57.625 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.625 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.625 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:57.625 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787150+0000", 2024-09-18T10:14:57.625 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.6399999999999997e-06 2024-09-18T10:14:57.625 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.625 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.625 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:57.625 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.298925+0000", 2024-09-18T10:14:57.625 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.51177523599999997 2024-09-18T10:14:57.625 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.626 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.626 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:14:57.626 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.299064+0000", 2024-09-18T10:14:57.626 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000139622 2024-09-18T10:14:57.626 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.626 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.626 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:14:57.626 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.299287+0000", 2024-09-18T10:14:57.626 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00022309700000000001 2024-09-18T10:14:57.626 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.626 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.626 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:14:57.626 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.301389+0000", 2024-09-18T10:14:57.626 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0021018759999999999 2024-09-18T10:14:57.627 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.627 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.627 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:14:57.627 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.498648+0000", 2024-09-18T10:14:57.627 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.19725873999999999 2024-09-18T10:14:57.627 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.627 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.627 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:14:57.627 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.498731+0000", 2024-09-18T10:14:57.627 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.2429000000000006e-05 2024-09-18T10:14:57.627 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.627 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.627 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:57.627 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.498823+0000", 2024-09-18T10:14:57.628 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.2655e-05 2024-09-18T10:14:57.628 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:57.628 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:57.628 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:57.628 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.628 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.628 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T10:14:57.628 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:36.787157+0000", 2024-09-18T10:14:57.628 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 20.801590965999999, 2024-09-18T10:14:57.628 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.81204889999999996, 2024-09-18T10:14:57.628 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:57.628 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:14:57.628 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:14:57.628 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:14:57.629 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:14:57.629 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 75 2024-09-18T10:14:57.629 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.629 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:57.629 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.629 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:57.629 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787157+0000", 2024-09-18T10:14:57.629 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:57.629 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.629 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.629 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:57.629 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787157+0000", 2024-09-18T10:14:57.629 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:57.629 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.630 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.630 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:57.630 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787156+0000", 2024-09-18T10:14:57.630 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:14:57.630 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.630 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.630 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:57.630 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787236+0000", 2024-09-18T10:14:57.630 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.0175999999999999e-05 2024-09-18T10:14:57.630 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.630 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.630 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:57.630 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787247+0000", 2024-09-18T10:14:57.630 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0448999999999999e-05 2024-09-18T10:14:57.630 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.631 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.631 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:57.631 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787253+0000", 2024-09-18T10:14:57.631 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.2469999999999997e-06 2024-09-18T10:14:57.631 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.631 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.631 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:57.631 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.299974+0000", 2024-09-18T10:14:57.631 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.51272114499999999 2024-09-18T10:14:57.631 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.631 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.631 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:14:57.631 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.300115+0000", 2024-09-18T10:14:57.631 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000141452 2024-09-18T10:14:57.632 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.632 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.632 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:14:57.632 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.300349+0000", 2024-09-18T10:14:57.632 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00023363999999999999 2024-09-18T10:14:57.632 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.632 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.632 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:14:57.632 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.301947+0000", 2024-09-18T10:14:57.632 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0015976510000000001 2024-09-18T10:14:57.632 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.632 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.632 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:14:57.632 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.599027+0000", 2024-09-18T10:14:57.633 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.29708051600000002 2024-09-18T10:14:57.633 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.633 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.633 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:14:57.633 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.599106+0000", 2024-09-18T10:14:57.633 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.9033e-05 2024-09-18T10:14:57.633 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.633 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.633 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:57.633 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.599206+0000", 2024-09-18T10:14:57.633 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.9561000000000004e-05 2024-09-18T10:14:57.633 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:57.633 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:57.633 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:57.633 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.634 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.634 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T10:14:57.634 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:36.787261+0000", 2024-09-18T10:14:57.634 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 20.801487314999999, 2024-09-18T10:14:57.634 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.81223649499999995, 2024-09-18T10:14:57.634 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:57.634 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:14:57.634 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:14:57.634 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:14:57.634 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:14:57.634 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 76 2024-09-18T10:14:57.634 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.635 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:57.635 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.635 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:57.635 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787261+0000", 2024-09-18T10:14:57.635 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:57.635 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.635 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.635 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:57.635 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787261+0000", 2024-09-18T10:14:57.635 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:57.635 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.635 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.635 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:57.635 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787260+0000", 2024-09-18T10:14:57.636 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:14:57.636 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.636 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.636 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:57.636 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787265+0000", 2024-09-18T10:14:57.636 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1590000000000001e-06 2024-09-18T10:14:57.636 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.636 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.636 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:57.636 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787273+0000", 2024-09-18T10:14:57.636 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.4100000000000008e-06 2024-09-18T10:14:57.636 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.636 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.637 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:57.637 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787279+0000", 2024-09-18T10:14:57.637 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.3630000000000002e-06 2024-09-18T10:14:57.637 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.637 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.637 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:57.637 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.496955+0000", 2024-09-18T10:14:57.637 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.70967585099999997 2024-09-18T10:14:57.637 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.637 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.637 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:14:57.637 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.497107+0000", 2024-09-18T10:14:57.637 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000152044 2024-09-18T10:14:57.637 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.638 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.638 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:14:57.638 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.497345+0000", 2024-09-18T10:14:57.638 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000238693 2024-09-18T10:14:57.638 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.638 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.638 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:14:57.638 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.599248+0000", 2024-09-18T10:14:57.638 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.101902836 2024-09-18T10:14:57.638 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.638 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.638 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:14:57.638 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.599329+0000", 2024-09-18T10:14:57.638 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.1064999999999997e-05 2024-09-18T10:14:57.639 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.639 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.639 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:14:57.639 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.599441+0000", 2024-09-18T10:14:57.639 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000111797 2024-09-18T10:14:57.639 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.639 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.639 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:57.639 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.599497+0000", 2024-09-18T10:14:57.639 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.6119999999999998e-05 2024-09-18T10:14:57.639 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:57.639 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:57.639 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:57.639 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.640 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.640 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:93 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T10:14:57.640 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:38.616472+0000", 2024-09-18T10:14:57.640 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 18.972276032, 2024-09-18T10:14:57.640 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.60940747100000003, 2024-09-18T10:14:57.640 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:57.640 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:14:57.640 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:14:57.640 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:14:57.640 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:14:57.640 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 93 2024-09-18T10:14:57.640 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.640 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:57.641 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.641 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:57.641 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.616472+0000", 2024-09-18T10:14:57.641 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:57.641 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.641 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.641 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:57.641 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.616472+0000", 2024-09-18T10:14:57.641 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:57.641 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.641 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.641 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:57.641 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.616470+0000", 2024-09-18T10:14:57.642 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:14:57.642 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.642 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.642 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:57.642 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.616644+0000", 2024-09-18T10:14:57.642 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00017372000000000001 2024-09-18T10:14:57.642 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.642 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.642 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:57.642 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.616695+0000", 2024-09-18T10:14:57.642 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.0991e-05 2024-09-18T10:14:57.642 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.642 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.642 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:57.643 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.616709+0000", 2024-09-18T10:14:57.643 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4008999999999999e-05 2024-09-18T10:14:57.643 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.643 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.643 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:57.643 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.720960+0000", 2024-09-18T10:14:57.643 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.104250624 2024-09-18T10:14:57.643 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.643 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.643 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:14:57.643 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.721068+0000", 2024-09-18T10:14:57.643 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00010814700000000001 2024-09-18T10:14:57.643 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.643 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.644 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:14:57.644 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.721270+0000", 2024-09-18T10:14:57.644 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00020184200000000001 2024-09-18T10:14:57.644 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.644 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.644 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:14:57.644 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.024691+0000", 2024-09-18T10:14:57.644 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.30342105600000002 2024-09-18T10:14:57.644 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.644 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.644 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:14:57.644 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.225725+0000", 2024-09-18T10:14:57.644 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20103395099999999 2024-09-18T10:14:57.644 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.644 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.645 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:14:57.645 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.225816+0000", 2024-09-18T10:14:57.645 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.1928999999999993e-05 2024-09-18T10:14:57.645 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.645 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.645 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:57.645 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.225879+0000", 2024-09-18T10:14:57.645 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.2918999999999998e-05 2024-09-18T10:14:57.645 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:57.645 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:57.645 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:57.645 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.645 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.645 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:94 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T10:14:57.645 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:38.616729+0000", 2024-09-18T10:14:57.646 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 18.972019007, 2024-09-18T10:14:57.646 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.60943170999999996, 2024-09-18T10:14:57.646 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:57.646 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:14:57.646 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:14:57.646 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:14:57.646 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:14:57.646 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 94 2024-09-18T10:14:57.646 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.646 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:57.646 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.646 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:57.646 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.616729+0000", 2024-09-18T10:14:57.646 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:57.647 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.647 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.647 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:57.647 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.616729+0000", 2024-09-18T10:14:57.647 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:57.647 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.647 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.647 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:57.647 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.616727+0000", 2024-09-18T10:14:57.647 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:14:57.647 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.647 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.647 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:57.647 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.616738+0000", 2024-09-18T10:14:57.648 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0776e-05 2024-09-18T10:14:57.648 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.648 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.648 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:57.648 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.616756+0000", 2024-09-18T10:14:57.648 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8079000000000001e-05 2024-09-18T10:14:57.648 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.648 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.648 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:57.648 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.716882+0000", 2024-09-18T10:14:57.648 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10012612999999999 2024-09-18T10:14:57.648 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.648 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.648 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:57.649 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.822315+0000", 2024-09-18T10:14:57.649 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.105433162 2024-09-18T10:14:57.649 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.649 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.649 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:14:57.649 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.822453+0000", 2024-09-18T10:14:57.649 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000137396 2024-09-18T10:14:57.649 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.649 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.649 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:14:57.649 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.822674+0000", 2024-09-18T10:14:57.649 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00022154400000000001 2024-09-18T10:14:57.649 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.649 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.650 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:14:57.650 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.024888+0000", 2024-09-18T10:14:57.650 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20221397199999999 2024-09-18T10:14:57.650 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.650 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.650 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:14:57.650 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.226011+0000", 2024-09-18T10:14:57.651 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20112316299999999 2024-09-18T10:14:57.652 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.652 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.652 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:14:57.652 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.226094+0000", 2024-09-18T10:14:57.652 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.2155000000000002e-05 2024-09-18T10:14:57.652 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.652 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.652 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:57.652 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.226161+0000", 2024-09-18T10:14:57.652 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.7138999999999995e-05 2024-09-18T10:14:57.652 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:57.652 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:57.652 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:57.652 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.653 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.653 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:95 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T10:14:57.653 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:38.716909+0000", 2024-09-18T10:14:57.653 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 18.871839293000001, 2024-09-18T10:14:57.653 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.60960763600000001, 2024-09-18T10:14:57.653 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:57.653 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:14:57.653 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:14:57.653 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:14:57.653 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:14:57.653 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 95 2024-09-18T10:14:57.653 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.653 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:57.653 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.653 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:57.654 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.716909+0000", 2024-09-18T10:14:57.654 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:57.654 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.654 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.654 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:57.654 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.716909+0000", 2024-09-18T10:14:57.654 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:57.654 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.654 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.654 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:57.654 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.716907+0000", 2024-09-18T10:14:57.654 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:14:57.654 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.654 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.655 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:57.655 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.716921+0000", 2024-09-18T10:14:57.655 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4572e-05 2024-09-18T10:14:57.655 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.655 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.655 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:57.655 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.716939+0000", 2024-09-18T10:14:57.655 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7149000000000001e-05 2024-09-18T10:14:57.655 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.655 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.655 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:57.655 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.716948+0000", 2024-09-18T10:14:57.655 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.7000000000000003e-06 2024-09-18T10:14:57.655 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.656 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.656 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:57.656 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.923430+0000", 2024-09-18T10:14:57.656 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20648133299999999 2024-09-18T10:14:57.656 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.656 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.656 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:14:57.656 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.923566+0000", 2024-09-18T10:14:57.656 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00013657599999999999 2024-09-18T10:14:57.656 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.656 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.656 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:14:57.656 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.923782+0000", 2024-09-18T10:14:57.656 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00021599400000000001 2024-09-18T10:14:57.657 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.657 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.657 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:14:57.657 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.024899+0000", 2024-09-18T10:14:57.657 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10111666800000001 2024-09-18T10:14:57.657 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.657 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.657 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:14:57.657 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.326379+0000", 2024-09-18T10:14:57.657 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.30148048799999999 2024-09-18T10:14:57.657 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.657 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.657 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:14:57.657 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.326454+0000", 2024-09-18T10:14:57.658 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.4988999999999996e-05 2024-09-18T10:14:57.658 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.658 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.658 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:57.658 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.326516+0000", 2024-09-18T10:14:57.658 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.1902999999999993e-05 2024-09-18T10:14:57.658 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:57.658 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:57.658 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:57.658 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.658 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.658 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:96 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T10:14:57.658 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:38.716972+0000", 2024-09-18T10:14:57.658 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 18.871776303000001, 2024-09-18T10:14:57.658 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.91539460800000005, 2024-09-18T10:14:57.659 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:57.659 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:14:57.659 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:14:57.659 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:14:57.659 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:14:57.659 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 96 2024-09-18T10:14:57.659 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.659 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:57.659 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.659 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:57.659 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.716972+0000", 2024-09-18T10:14:57.659 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:57.659 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.659 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.660 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:57.660 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.716972+0000", 2024-09-18T10:14:57.660 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:57.660 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.660 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.660 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:57.660 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.716971+0000", 2024-09-18T10:14:57.660 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:14:57.660 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.660 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.660 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:57.660 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717061+0000", 2024-09-18T10:14:57.660 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.0744999999999999e-05 2024-09-18T10:14:57.660 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.661 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.661 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:57.661 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717073+0000", 2024-09-18T10:14:57.661 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1943000000000001e-05 2024-09-18T10:14:57.661 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.661 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.661 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:57.661 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717080+0000", 2024-09-18T10:14:57.661 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0060000000000003e-06 2024-09-18T10:14:57.661 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.661 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.661 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:57.661 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.426708+0000", 2024-09-18T10:14:57.661 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.70962730399999996 2024-09-18T10:14:57.662 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.662 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.662 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:14:57.662 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.426854+0000", 2024-09-18T10:14:57.662 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00014654199999999999 2024-09-18T10:14:57.662 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.662 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.662 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:14:57.662 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.427106+0000", 2024-09-18T10:14:57.662 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00025226600000000001 2024-09-18T10:14:57.662 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.662 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.662 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:14:57.663 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.433087+0000", 2024-09-18T10:14:57.663 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0059804860000000001 2024-09-18T10:14:57.663 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.663 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.663 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:14:57.663 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.632096+0000", 2024-09-18T10:14:57.663 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.19900922099999999 2024-09-18T10:14:57.663 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.663 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.663 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:14:57.663 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.632257+0000", 2024-09-18T10:14:57.663 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00016058399999999999 2024-09-18T10:14:57.663 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.663 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.664 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:57.664 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.632366+0000", 2024-09-18T10:14:57.664 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000109601 2024-09-18T10:14:57.664 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:57.664 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:57.664 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:57.664 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.664 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.664 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:97 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T10:14:57.664 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:38.717093+0000", 2024-09-18T10:14:57.664 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 18.871654493000001, 2024-09-18T10:14:57.664 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.91561460900000002, 2024-09-18T10:14:57.664 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:57.664 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:14:57.665 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:14:57.665 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:14:57.665 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:14:57.665 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 97 2024-09-18T10:14:57.665 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.665 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:57.665 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.665 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:57.665 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717093+0000", 2024-09-18T10:14:57.665 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:57.665 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.665 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.665 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:57.665 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717093+0000", 2024-09-18T10:14:57.666 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:57.666 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.666 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.666 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:57.666 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717092+0000", 2024-09-18T10:14:57.666 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:14:57.666 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.666 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.666 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:57.666 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717099+0000", 2024-09-18T10:14:57.667 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.246e-06 2024-09-18T10:14:57.667 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.667 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.667 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:57.667 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717109+0000", 2024-09-18T10:14:57.667 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0499999999999999e-05 2024-09-18T10:14:57.667 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.667 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.667 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:57.667 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717115+0000", 2024-09-18T10:14:57.667 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.7690000000000002e-06 2024-09-18T10:14:57.667 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.667 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.668 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:57.668 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.428509+0000", 2024-09-18T10:14:57.668 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.71139444299999999 2024-09-18T10:14:57.668 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.668 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.668 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:14:57.668 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.428621+0000", 2024-09-18T10:14:57.668 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00011156 2024-09-18T10:14:57.668 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.668 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.668 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:14:57.668 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.428805+0000", 2024-09-18T10:14:57.668 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000183597 2024-09-18T10:14:57.669 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.669 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.669 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:14:57.669 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.433106+0000", 2024-09-18T10:14:57.669 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0043015019999999996 2024-09-18T10:14:57.669 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.669 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.669 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:14:57.669 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.632474+0000", 2024-09-18T10:14:57.669 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.199368092 2024-09-18T10:14:57.669 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.669 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.669 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:14:57.670 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.632605+0000", 2024-09-18T10:14:57.670 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00013110899999999999 2024-09-18T10:14:57.670 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.670 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.670 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:57.670 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.632708+0000", 2024-09-18T10:14:57.670 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000102748 2024-09-18T10:14:57.670 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:57.670 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:57.670 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:57.670 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.670 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.670 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:98 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T10:14:57.670 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:38.717126+0000", 2024-09-18T10:14:57.670 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 18.871622231, 2024-09-18T10:14:57.670 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.015084141, 2024-09-18T10:14:57.671 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:57.671 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:14:57.671 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:14:57.671 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:14:57.671 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:14:57.671 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 98 2024-09-18T10:14:57.671 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.671 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:57.671 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.671 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:57.671 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717126+0000", 2024-09-18T10:14:57.671 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:57.671 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.671 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.671 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:57.672 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717126+0000", 2024-09-18T10:14:57.672 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:57.672 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.672 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.672 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:57.672 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717125+0000", 2024-09-18T10:14:57.672 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:14:57.672 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.672 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.672 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:57.672 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717130+0000", 2024-09-18T10:14:57.672 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.4160000000000003e-06 2024-09-18T10:14:57.672 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.672 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.673 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:57.673 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717139+0000", 2024-09-18T10:14:57.673 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.6230000000000003e-06 2024-09-18T10:14:57.673 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.673 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.673 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:57.673 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717144+0000", 2024-09-18T10:14:57.673 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5330000000000002e-06 2024-09-18T10:14:57.673 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.673 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.673 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:57.673 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.429491+0000", 2024-09-18T10:14:57.673 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.71234629500000002 2024-09-18T10:14:57.673 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.674 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.674 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:14:57.674 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.429741+0000", 2024-09-18T10:14:57.674 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000250516 2024-09-18T10:14:57.674 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.674 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.674 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:14:57.674 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.429989+0000", 2024-09-18T10:14:57.674 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00024811599999999999 2024-09-18T10:14:57.674 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.674 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.674 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:14:57.674 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.435056+0000", 2024-09-18T10:14:57.675 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0050669670000000003 2024-09-18T10:14:57.675 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.675 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.675 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:14:57.675 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.732068+0000", 2024-09-18T10:14:57.675 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.29701180900000002 2024-09-18T10:14:57.675 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.675 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.675 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:14:57.675 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.732145+0000", 2024-09-18T10:14:57.675 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.7152999999999998e-05 2024-09-18T10:14:57.675 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.675 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.675 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:57.675 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.732210+0000", 2024-09-18T10:14:57.676 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.4503000000000002e-05 2024-09-18T10:14:57.676 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:57.676 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:57.676 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:57.676 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.676 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.676 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:99 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T10:14:57.676 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:38.717230+0000", 2024-09-18T10:14:57.676 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 18.871517670999999, 2024-09-18T10:14:57.676 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0157905819999999, 2024-09-18T10:14:57.676 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:57.676 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:14:57.676 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:14:57.676 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:14:57.677 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:14:57.677 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 99 2024-09-18T10:14:57.677 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.677 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:57.677 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.677 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:57.677 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717230+0000", 2024-09-18T10:14:57.677 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:57.677 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.677 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.677 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:57.677 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717230+0000", 2024-09-18T10:14:57.677 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:57.677 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.678 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.678 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:57.678 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717229+0000", 2024-09-18T10:14:57.678 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:14:57.678 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.678 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.678 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:57.678 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717318+0000", 2024-09-18T10:14:57.678 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.8197999999999998e-05 2024-09-18T10:14:57.678 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.678 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.678 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:57.678 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717342+0000", 2024-09-18T10:14:57.678 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4445000000000001e-05 2024-09-18T10:14:57.678 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.678 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.679 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:57.679 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717360+0000", 2024-09-18T10:14:57.679 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7782999999999999e-05 2024-09-18T10:14:57.679 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.679 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.679 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:57.679 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.430650+0000", 2024-09-18T10:14:57.679 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.71329020200000004 2024-09-18T10:14:57.679 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.679 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.679 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:14:57.679 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.430805+0000", 2024-09-18T10:14:57.679 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00015474900000000001 2024-09-18T10:14:57.679 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.680 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.680 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:14:57.680 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.431021+0000", 2024-09-18T10:14:57.680 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000215911 2024-09-18T10:14:57.680 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.680 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.680 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:14:57.680 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.435099+0000", 2024-09-18T10:14:57.680 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0040787439999999996 2024-09-18T10:14:57.680 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.680 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.680 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:14:57.680 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.732758+0000", 2024-09-18T10:14:57.680 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.297658637 2024-09-18T10:14:57.681 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.681 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.681 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:14:57.681 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.732909+0000", 2024-09-18T10:14:57.681 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00015069899999999999 2024-09-18T10:14:57.681 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.681 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.681 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:57.681 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.733021+0000", 2024-09-18T10:14:57.681 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00011207 2024-09-18T10:14:57.681 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:57.681 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:57.681 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:57.681 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.682 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.682 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:100 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T10:14:57.682 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:38.717370+0000", 2024-09-18T10:14:57.682 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 18.871378108999998, 2024-09-18T10:14:57.682 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1158324850000001, 2024-09-18T10:14:57.682 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:57.682 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:14:57.682 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:14:57.682 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:14:57.682 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:14:57.682 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 100 2024-09-18T10:14:57.682 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.682 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:57.683 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.683 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:57.683 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717370+0000", 2024-09-18T10:14:57.683 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:57.683 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.683 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.683 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:57.683 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717370+0000", 2024-09-18T10:14:57.683 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:57.683 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.683 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.683 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:57.683 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717369+0000", 2024-09-18T10:14:57.683 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:14:57.683 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.684 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.684 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:57.684 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717378+0000", 2024-09-18T10:14:57.684 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.3559999999999999e-06 2024-09-18T10:14:57.684 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.684 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.684 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:57.684 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717398+0000", 2024-09-18T10:14:57.684 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.9962e-05 2024-09-18T10:14:57.684 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.684 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.684 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:57.684 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717407+0000", 2024-09-18T10:14:57.684 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.2699999999999993e-06 2024-09-18T10:14:57.685 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.685 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.685 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:57.685 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.432184+0000", 2024-09-18T10:14:57.685 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.71477630299999995 2024-09-18T10:14:57.685 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.685 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.685 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:14:57.685 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.432316+0000", 2024-09-18T10:14:57.685 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000132082 2024-09-18T10:14:57.685 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.685 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.685 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:14:57.686 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.432526+0000", 2024-09-18T10:14:57.686 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000210659 2024-09-18T10:14:57.686 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.686 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.686 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:14:57.686 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.435651+0000", 2024-09-18T10:14:57.686 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.003124169 2024-09-18T10:14:57.686 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.686 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.686 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:14:57.686 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.832931+0000", 2024-09-18T10:14:57.686 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.39728084899999999 2024-09-18T10:14:57.686 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.686 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.687 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:14:57.687 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.833058+0000", 2024-09-18T10:14:57.687 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000126969 2024-09-18T10:14:57.687 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.687 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.687 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:57.687 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.833202+0000", 2024-09-18T10:14:57.687 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000143922 2024-09-18T10:14:57.687 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:57.687 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:57.687 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:57.687 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.687 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.687 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:132 3.8s0 3:1162852a:::smithi18925896-44:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:14:57.687 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:50.819929+0000", 2024-09-18T10:14:57.687 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 6.7688189169999999, 2024-09-18T10:14:57.688 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.59839508200000002, 2024-09-18T10:14:57.688 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:57.688 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "started", 2024-09-18T10:14:57.688 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:14:57.688 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:14:57.688 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:14:57.688 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 132 2024-09-18T10:14:57.688 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.688 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:57.688 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.688 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:57.688 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.819929+0000", 2024-09-18T10:14:57.688 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:57.688 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.688 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.689 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:57.689 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.819929+0000", 2024-09-18T10:14:57.689 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:57.689 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.689 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.689 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:57.689 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.819927+0000", 2024-09-18T10:14:57.689 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:14:57.689 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.689 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.689 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:57.689 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.819938+0000", 2024-09-18T10:14:57.689 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0502999999999999e-05 2024-09-18T10:14:57.689 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.690 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.690 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:57.690 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.819962+0000", 2024-09-18T10:14:57.690 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4241999999999999e-05 2024-09-18T10:14:57.690 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.690 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.690 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:57.690 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.819973+0000", 2024-09-18T10:14:57.690 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0730000000000001e-05 2024-09-18T10:14:57.690 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.690 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.690 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:57.690 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.908665+0000", 2024-09-18T10:14:57.691 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.088691995999999995 2024-09-18T10:14:57.691 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.691 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.691 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for rw locks", 2024-09-18T10:14:57.691 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.908807+0000", 2024-09-18T10:14:57.691 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00014202400000000001 2024-09-18T10:14:57.691 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.691 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.691 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:57.691 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.214385+0000", 2024-09-18T10:14:57.691 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.30557793100000002 2024-09-18T10:14:57.691 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.691 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.691 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:14:57.691 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.214515+0000", 2024-09-18T10:14:57.692 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00013058699999999999 2024-09-18T10:14:57.692 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.692 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.692 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:57.692 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.418324+0000", 2024-09-18T10:14:57.692 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20380895600000001 2024-09-18T10:14:57.692 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:57.692 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:57.692 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:57.692 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.692 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.692 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:141 3.8s0 3:1c02e63c:::smithi18925896-47:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:14:57.692 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:51.119430+0000", 2024-09-18T10:14:57.692 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 6.4693179049999996, 2024-09-18T10:14:57.693 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.60306919000000003, 2024-09-18T10:14:57.693 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:57.693 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "started", 2024-09-18T10:14:57.693 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:14:57.693 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:14:57.693 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:14:57.693 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 141 2024-09-18T10:14:57.693 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.693 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:57.693 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.693 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:57.693 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.119430+0000", 2024-09-18T10:14:57.693 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:57.693 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.694 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.694 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:57.694 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.119430+0000", 2024-09-18T10:14:57.694 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:57.694 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.694 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.694 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:57.694 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.119430+0000", 2024-09-18T10:14:57.694 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999995 2024-09-18T10:14:57.694 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.694 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.694 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:57.694 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.119432+0000", 2024-09-18T10:14:57.694 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6240000000000002e-06 2024-09-18T10:14:57.695 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.695 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.695 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:57.695 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.119439+0000", 2024-09-18T10:14:57.695 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.8329999999999996e-06 2024-09-18T10:14:57.695 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.695 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.695 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:57.695 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.119444+0000", 2024-09-18T10:14:57.695 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.485e-06 2024-09-18T10:14:57.695 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.695 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.695 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:57.695 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.620626+0000", 2024-09-18T10:14:57.696 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.50118172299999997 2024-09-18T10:14:57.696 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.696 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.696 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:14:57.696 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.620781+0000", 2024-09-18T10:14:57.696 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00015495 2024-09-18T10:14:57.696 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.696 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.696 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:57.696 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.722499+0000", 2024-09-18T10:14:57.696 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.101718085 2024-09-18T10:14:57.696 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:57.696 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:57.697 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:57.697 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.697 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.697 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:154 3.8s0 3:1d386431:::smithi18925896-2:head [copy-from ver 4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:14:57.697 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:51.211083+0000", 2024-09-18T10:14:57.697 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 6.3776646799999996, 2024-09-18T10:14:57.697 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.73193688499999998, 2024-09-18T10:14:57.697 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:57.697 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:14:57.697 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:14:57.697 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:14:57.697 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:14:57.697 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 154 2024-09-18T10:14:57.697 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.698 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:57.698 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.698 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:57.698 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.211083+0000", 2024-09-18T10:14:57.698 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:57.698 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.698 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.698 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:57.698 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.211083+0000", 2024-09-18T10:14:57.698 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:57.698 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.698 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.698 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:57.698 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.211080+0000", 2024-09-18T10:14:57.699 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999971 2024-09-18T10:14:57.699 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.699 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.699 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:57.699 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.211096+0000", 2024-09-18T10:14:57.699 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.5699999999999999e-05 2024-09-18T10:14:57.699 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.699 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.699 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:57.699 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.211134+0000", 2024-09-18T10:14:57.699 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.7694000000000002e-05 2024-09-18T10:14:57.699 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.699 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.699 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:57.699 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.211149+0000", 2024-09-18T10:14:57.700 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4746000000000001e-05 2024-09-18T10:14:57.700 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.700 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.700 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:57.700 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.621341+0000", 2024-09-18T10:14:57.700 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.41019203799999998 2024-09-18T10:14:57.700 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.700 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.700 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:14:57.700 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.621505+0000", 2024-09-18T10:14:57.700 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00016443299999999999 2024-09-18T10:14:57.700 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.700 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.700 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:14:57.701 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.840064+0000", 2024-09-18T10:14:57.701 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.21855904700000001 2024-09-18T10:14:57.701 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.701 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.701 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:14:57.701 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.842430+0000", 2024-09-18T10:14:57.701 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0023664100000000002 2024-09-18T10:14:57.701 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.701 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.701 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:14:57.701 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.942864+0000", 2024-09-18T10:14:57.701 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10043373 2024-09-18T10:14:57.701 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.701 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.702 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:57.702 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.943020+0000", 2024-09-18T10:14:57.702 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00015598000000000001 2024-09-18T10:14:57.702 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:57.702 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:57.702 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:57.702 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.702 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.702 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:155 3.8s0 3:1d386431:::smithi18925896-2:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:14:57.702 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:51.211164+0000", 2024-09-18T10:14:57.702 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 6.3775843099999996, 2024-09-18T10:14:57.702 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.630957349, 2024-09-18T10:14:57.702 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:57.702 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "started", 2024-09-18T10:14:57.702 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:14:57.703 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:14:57.703 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:14:57.703 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 155 2024-09-18T10:14:57.703 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.703 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:57.703 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.703 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:57.703 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.211164+0000", 2024-09-18T10:14:57.703 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:57.703 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.703 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.703 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:57.703 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.211164+0000", 2024-09-18T10:14:57.703 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:57.703 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.704 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.704 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:57.704 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.211163+0000", 2024-09-18T10:14:57.704 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:14:57.704 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.704 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.704 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:57.704 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.211167+0000", 2024-09-18T10:14:57.704 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.6299999999999997e-06 2024-09-18T10:14:57.704 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.704 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.704 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:57.704 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.211178+0000", 2024-09-18T10:14:57.704 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0798999999999999e-05 2024-09-18T10:14:57.705 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.705 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.705 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:57.705 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.211186+0000", 2024-09-18T10:14:57.705 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.5669999999999997e-06 2024-09-18T10:14:57.705 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.705 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.705 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:57.705 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.623024+0000", 2024-09-18T10:14:57.705 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.411838433 2024-09-18T10:14:57.705 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.705 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.705 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for blocked object", 2024-09-18T10:14:57.705 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.623104+0000", 2024-09-18T10:14:57.706 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.0136000000000007e-05 2024-09-18T10:14:57.706 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.706 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.706 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:57.706 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.841600+0000", 2024-09-18T10:14:57.706 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.21849595399999999 2024-09-18T10:14:57.706 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.706 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.706 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:14:57.706 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.841868+0000", 2024-09-18T10:14:57.706 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00026825800000000001 2024-09-18T10:14:57.706 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.706 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.706 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:57.707 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.842121+0000", 2024-09-18T10:14:57.707 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00025266500000000001 2024-09-18T10:14:57.707 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:57.707 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:57.707 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:57.707 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:57.707 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:57.707 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-09-18T10:14:57.708 INFO:tasks.rados.rados.0.smithi189.stdout:96: expect (ObjNum 44 snap 0 seq_num 44) 2024-09-18T10:14:57.708 INFO:tasks.rados.rados.0.smithi189.stdout:97: finishing rollback tid 2 to smithi18925896-12 2024-09-18T10:14:57.709 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 12 v 10 (ObjNum 11 snap 0 seq_num 11) dirty exists 2024-09-18T10:14:57.709 INFO:tasks.ceph.osd.11.smithi189.stderr:2024-09-18T10:14:57.707+0000 7f718b60a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 22400) UID: 0 2024-09-18T10:14:57.709 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-09-18T10:14:57.728 INFO:tasks.rados.rados.0.smithi189.stdout:95: done (6 left) 2024-09-18T10:14:57.728 INFO:tasks.rados.rados.0.smithi189.stdout:96: done (5 left) 2024-09-18T10:14:57.728 INFO:tasks.rados.rados.0.smithi189.stdout:97: done (4 left) 2024-09-18T10:14:57.728 INFO:tasks.rados.rados.0.smithi189.stdout:100: done (3 left) 2024-09-18T10:14:57.728 INFO:tasks.rados.rados.0.smithi189.stdout:101: done (2 left) 2024-09-18T10:14:57.729 INFO:tasks.rados.rados.0.smithi189.stdout:102: snap_remove snap 1 2024-09-18T10:14:57.743 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-18T10:14:57.810 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-18T10:14:57.807+0000 7f14dcc5b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22398) UID: 0 2024-09-18T10:14:57.816 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-09-18T10:14:57.816 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-09-18T10:14:57.816 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.816 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.19s2 41/40 ECSubWrite(tid=41, reqid=osd.7.0:41, at_version=41'25, trim_to=0'0, roll_forward_to=0'0))", 2024-09-18T10:14:57.816 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:57.748903+0000", 2024-09-18T10:14:57.816 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 0.066807769000000003, 2024-09-18T10:14:57.816 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.066846489999999995, 2024-09-18T10:14:57.816 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:57.816 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:14:57.816 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:57.817 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.817 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:57.817 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.748903+0000", 2024-09-18T10:14:57.817 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:57.817 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.817 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.817 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:57.817 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.748903+0000", 2024-09-18T10:14:57.817 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:57.817 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.817 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.817 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:57.817 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.748902+0000", 2024-09-18T10:14:57.817 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:14:57.818 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.818 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.818 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:57.818 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.748946+0000", 2024-09-18T10:14:57.818 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4589999999999998e-05 2024-09-18T10:14:57.818 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.818 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.818 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:57.818 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.748963+0000", 2024-09-18T10:14:57.818 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.6415000000000001e-05 2024-09-18T10:14:57.818 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.818 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.818 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:57.819 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.748974+0000", 2024-09-18T10:14:57.819 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.119e-05 2024-09-18T10:14:57.819 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.819 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.819 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:57.819 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.749096+0000", 2024-09-18T10:14:57.819 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000121907 2024-09-18T10:14:57.819 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.819 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.819 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:14:57.819 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.749120+0000", 2024-09-18T10:14:57.819 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4516999999999999e-05 2024-09-18T10:14:57.819 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:57.819 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:57.820 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:57.820 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.820 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.820 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.1s3 41/40 ECSubWrite(tid=63, reqid=client.4632.0:262, at_version=0'0, trim_to=0'0, roll_forward_to=41'10))", 2024-09-18T10:14:57.820 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:57.750933+0000", 2024-09-18T10:14:57.820 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 0.064777150000000006, 2024-09-18T10:14:57.820 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.064916281000000006, 2024-09-18T10:14:57.820 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:57.820 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:14:57.820 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:57.820 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.820 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:57.820 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.750933+0000", 2024-09-18T10:14:57.820 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:57.821 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.821 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.821 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:57.821 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.750933+0000", 2024-09-18T10:14:57.821 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:57.821 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.821 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.821 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:57.821 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.750932+0000", 2024-09-18T10:14:57.821 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:14:57.821 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.821 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.821 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:57.821 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.750958+0000", 2024-09-18T10:14:57.822 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5473000000000002e-05 2024-09-18T10:14:57.822 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.822 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.822 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:57.822 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.750978+0000", 2024-09-18T10:14:57.822 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.0239999999999999e-05 2024-09-18T10:14:57.822 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.822 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.822 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:57.822 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.750992+0000", 2024-09-18T10:14:57.822 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3393999999999999e-05 2024-09-18T10:14:57.822 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.822 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.822 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:57.823 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.752251+0000", 2024-09-18T10:14:57.823 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0012599449999999999 2024-09-18T10:14:57.823 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.823 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.823 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:14:57.823 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.752271+0000", 2024-09-18T10:14:57.823 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.9289000000000001e-05 2024-09-18T10:14:57.823 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:57.823 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:57.823 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:57.823 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.823 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.823 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWriteReply(3.13s0 41/40 ECSubWriteReply(tid=19, last_complete=41'16, committed=1, applied=1))", 2024-09-18T10:14:57.823 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:57.753336+0000", 2024-09-18T10:14:57.823 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 0.062374376000000002, 2024-09-18T10:14:57.824 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.062593910000000003, 2024-09-18T10:14:57.824 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:57.824 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "no flag points reached", 2024-09-18T10:14:57.824 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:57.824 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.824 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:57.824 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.753336+0000", 2024-09-18T10:14:57.824 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:57.824 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.824 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.824 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:57.824 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.753336+0000", 2024-09-18T10:14:57.824 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:57.825 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.825 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.825 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:57.825 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.753335+0000", 2024-09-18T10:14:57.825 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:14:57.825 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.825 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.825 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:57.825 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.753346+0000", 2024-09-18T10:14:57.825 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0655999999999999e-05 2024-09-18T10:14:57.825 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:57.825 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:57.825 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:57.825 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.753358+0000", 2024-09-18T10:14:57.826 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2493999999999999e-05 2024-09-18T10:14:57.826 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:57.826 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:57.826 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:57.826 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:57.826 INFO:teuthology.orchestra.run.smithi154.stdout: ], 2024-09-18T10:14:57.826 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 3 2024-09-18T10:14:57.826 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-09-18T10:14:57.827 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-09-18T10:14:57.911 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-09-18T10:14:57.904+0000 7fdfa8698700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22087) UID: 0 2024-09-18T10:14:57.924 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-09-18T10:14:57.924 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-09-18T10:14:57.924 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-09-18T10:14:57.924 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-09-18T10:14:57.924 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-09-18T10:14:57.933 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-09-18T10:14:58.011 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-18T10:14:58.005+0000 7f1981873700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22132) UID: 0 2024-09-18T10:14:58.034 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-09-18T10:14:58.034 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-09-18T10:14:58.034 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-09-18T10:14:58.034 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-09-18T10:14:58.034 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.034 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.9s2 36/32 ECSubWrite(tid=12, reqid=client.4632.0:136, at_version=36'9, trim_to=0'0, roll_forward_to=35'6))", 2024-09-18T10:14:58.034 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:51.004061+0000", 2024-09-18T10:14:58.034 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 7.0267346760000002, 2024-09-18T10:14:58.034 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.102012215, 2024-09-18T10:14:58.035 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:58.035 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:14:58.035 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:58.035 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.035 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:58.035 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.004061+0000", 2024-09-18T10:14:58.035 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:58.035 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.035 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.035 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:58.035 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.004061+0000", 2024-09-18T10:14:58.035 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:58.035 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.035 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.036 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:58.036 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.004060+0000", 2024-09-18T10:14:58.036 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:14:58.036 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.036 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.036 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:58.036 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.004355+0000", 2024-09-18T10:14:58.036 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00029493199999999997 2024-09-18T10:14:58.036 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.036 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.036 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:58.036 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.004365+0000", 2024-09-18T10:14:58.037 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.4530000000000008e-06 2024-09-18T10:14:58.037 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.037 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.037 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:58.037 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.004369+0000", 2024-09-18T10:14:58.037 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.6120000000000001e-06 2024-09-18T10:14:58.037 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.037 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.037 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:58.037 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.004454+0000", 2024-09-18T10:14:58.037 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.5050999999999996e-05 2024-09-18T10:14:58.038 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.038 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.038 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:14:58.038 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.004468+0000", 2024-09-18T10:14:58.038 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3521e-05 2024-09-18T10:14:58.038 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.038 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.038 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:14:58.038 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.105999+0000", 2024-09-18T10:14:58.038 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.101530651 2024-09-18T10:14:58.038 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.038 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.038 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:58.038 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.106073+0000", 2024-09-18T10:14:58.039 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.4964999999999998e-05 2024-09-18T10:14:58.039 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:58.039 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:58.039 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:58.039 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.039 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.039 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.9s2 36/32 ECSubWrite(tid=13, reqid=client.4632.0:137, at_version=36'10, trim_to=0'0, roll_forward_to=35'6))", 2024-09-18T10:14:58.039 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:51.004383+0000", 2024-09-18T10:14:58.039 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 7.0264125499999999, 2024-09-18T10:14:58.039 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.101895054, 2024-09-18T10:14:58.039 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:58.039 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:14:58.039 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:58.040 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.040 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:58.040 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.004383+0000", 2024-09-18T10:14:58.040 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:58.040 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.040 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.040 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:58.040 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.004383+0000", 2024-09-18T10:14:58.040 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:58.040 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.040 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.040 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:58.040 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.004383+0000", 2024-09-18T10:14:58.040 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999995 2024-09-18T10:14:58.041 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.041 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.041 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:58.041 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.004400+0000", 2024-09-18T10:14:58.041 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7142999999999998e-05 2024-09-18T10:14:58.041 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.041 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.041 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:58.041 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.004412+0000", 2024-09-18T10:14:58.041 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1970999999999999e-05 2024-09-18T10:14:58.041 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.041 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.041 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:58.041 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.004419+0000", 2024-09-18T10:14:58.042 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.6030000000000001e-06 2024-09-18T10:14:58.042 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.042 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.042 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:58.042 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.105180+0000", 2024-09-18T10:14:58.042 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100760893 2024-09-18T10:14:58.042 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.042 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.042 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:14:58.042 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.105197+0000", 2024-09-18T10:14:58.042 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7872000000000001e-05 2024-09-18T10:14:58.042 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.042 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.042 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:14:58.043 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.106206+0000", 2024-09-18T10:14:58.043 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0010085000000000001 2024-09-18T10:14:58.043 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.043 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.043 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:58.043 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.106278+0000", 2024-09-18T10:14:58.043 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.2478999999999995e-05 2024-09-18T10:14:58.043 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:58.043 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:58.043 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:58.043 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.043 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.043 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.7s5 36/32 ECSubWrite(tid=31, reqid=client.4632.0:128, at_version=0'0, trim_to=0'0, roll_forward_to=36'8))", 2024-09-18T10:14:58.043 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:51.219177+0000", 2024-09-18T10:14:58.044 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 6.8116184650000005, 2024-09-18T10:14:58.044 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.202758309, 2024-09-18T10:14:58.044 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:58.044 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:14:58.044 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:58.044 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.044 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:58.044 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.219177+0000", 2024-09-18T10:14:58.044 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:58.044 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.044 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.044 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:58.044 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.219177+0000", 2024-09-18T10:14:58.045 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:58.045 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.045 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.045 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:58.045 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.219175+0000", 2024-09-18T10:14:58.045 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999976 2024-09-18T10:14:58.045 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.045 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.045 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:58.045 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.219207+0000", 2024-09-18T10:14:58.045 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1285000000000003e-05 2024-09-18T10:14:58.045 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.045 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.046 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:58.046 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.219244+0000", 2024-09-18T10:14:58.046 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.7032000000000001e-05 2024-09-18T10:14:58.046 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.046 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.046 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:58.046 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.219266+0000", 2024-09-18T10:14:58.046 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.2112000000000001e-05 2024-09-18T10:14:58.046 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.046 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.046 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:58.046 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.319431+0000", 2024-09-18T10:14:58.046 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100164869 2024-09-18T10:14:58.046 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.046 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.047 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:14:58.047 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.319446+0000", 2024-09-18T10:14:58.047 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.5945999999999999e-05 2024-09-18T10:14:58.047 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.047 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.047 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:14:58.047 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.421877+0000", 2024-09-18T10:14:58.047 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10243026600000001 2024-09-18T10:14:58.047 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.047 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.047 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:58.047 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.421936+0000", 2024-09-18T10:14:58.047 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.9024999999999998e-05 2024-09-18T10:14:58.047 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:58.048 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:58.048 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:58.048 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.048 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.048 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.7s5 36/32 ECSubWrite(tid=33, reqid=client.4632.0:156, at_version=36'9, trim_to=0'0, roll_forward_to=36'8))", 2024-09-18T10:14:58.048 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:51.227199+0000", 2024-09-18T10:14:58.048 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 6.8035970289999996, 2024-09-18T10:14:58.048 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.19476125799999999, 2024-09-18T10:14:58.048 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:58.048 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:14:58.048 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:58.048 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.048 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:58.049 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.227199+0000", 2024-09-18T10:14:58.049 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:58.049 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.049 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.049 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:58.049 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.227199+0000", 2024-09-18T10:14:58.049 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:58.049 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.049 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.049 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:58.049 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.227198+0000", 2024-09-18T10:14:58.049 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:14:58.049 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.050 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.050 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:58.050 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.227309+0000", 2024-09-18T10:14:58.050 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00011174 2024-09-18T10:14:58.050 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.050 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.050 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:58.050 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.227324+0000", 2024-09-18T10:14:58.050 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4422e-05 2024-09-18T10:14:58.050 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.050 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.050 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:58.050 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.227331+0000", 2024-09-18T10:14:58.050 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.8950000000000001e-06 2024-09-18T10:14:58.051 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.051 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.051 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:58.051 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.419850+0000", 2024-09-18T10:14:58.051 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.19251963599999999 2024-09-18T10:14:58.051 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.051 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.051 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:14:58.051 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.419867+0000", 2024-09-18T10:14:58.051 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.6520000000000001e-05 2024-09-18T10:14:58.051 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.051 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.051 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:14:58.052 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.421943+0000", 2024-09-18T10:14:58.052 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0020762189999999998 2024-09-18T10:14:58.052 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.052 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.052 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:58.052 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.421960+0000", 2024-09-18T10:14:58.052 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7042999999999999e-05 2024-09-18T10:14:58.052 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:58.052 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:58.052 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:58.052 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.052 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.052 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.7s5 36/32 ECSubWrite(tid=34, reqid=client.4632.0:157, at_version=36'10, trim_to=0'0, roll_forward_to=36'8))", 2024-09-18T10:14:58.052 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:51.229493+0000", 2024-09-18T10:14:58.053 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 6.8013032970000005, 2024-09-18T10:14:58.053 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.193027271, 2024-09-18T10:14:58.053 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:58.053 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:14:58.053 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:58.053 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.053 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:58.053 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.229493+0000", 2024-09-18T10:14:58.053 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:58.053 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.053 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.053 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:58.053 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.229493+0000", 2024-09-18T10:14:58.053 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:58.054 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.054 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.054 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:58.054 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.229492+0000", 2024-09-18T10:14:58.054 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999995 2024-09-18T10:14:58.054 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.054 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.054 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:58.054 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.229647+0000", 2024-09-18T10:14:58.054 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00015455800000000001 2024-09-18T10:14:58.054 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.054 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.054 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:58.054 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.229660+0000", 2024-09-18T10:14:58.055 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3567000000000001e-05 2024-09-18T10:14:58.055 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.055 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.055 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:58.055 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.229667+0000", 2024-09-18T10:14:58.055 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.8419999999999999e-06 2024-09-18T10:14:58.055 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.055 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.055 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:58.055 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.420781+0000", 2024-09-18T10:14:58.055 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.19111434199999999 2024-09-18T10:14:58.055 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.055 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.056 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:14:58.056 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.420798+0000", 2024-09-18T10:14:58.056 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7065999999999999e-05 2024-09-18T10:14:58.056 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.056 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.056 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:14:58.056 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.422472+0000", 2024-09-18T10:14:58.056 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001674032 2024-09-18T10:14:58.056 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.056 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.056 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:58.056 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.422520+0000", 2024-09-18T10:14:58.056 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.7543999999999997e-05 2024-09-18T10:14:58.057 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:58.057 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:58.057 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:58.057 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.057 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.057 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.bs2 36/32 ECSubWrite(tid=35, reqid=client.4632.0:160, at_version=36'5, trim_to=0'0, roll_forward_to=36'4))", 2024-09-18T10:14:58.057 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:51.315316+0000", 2024-09-18T10:14:58.057 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 6.7154799690000004, 2024-09-18T10:14:58.057 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.201678933, 2024-09-18T10:14:58.057 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:58.057 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:14:58.057 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:58.057 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.057 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:58.058 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.315316+0000", 2024-09-18T10:14:58.058 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:58.058 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.058 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.058 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:58.058 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.315316+0000", 2024-09-18T10:14:58.058 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:58.058 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.058 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.058 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:58.058 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.315314+0000", 2024-09-18T10:14:58.058 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:14:58.058 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.059 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.059 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:58.059 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.315385+0000", 2024-09-18T10:14:58.059 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.1210000000000004e-05 2024-09-18T10:14:58.059 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.059 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.059 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:58.059 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.315422+0000", 2024-09-18T10:14:58.059 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.7157999999999999e-05 2024-09-18T10:14:58.059 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.059 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.059 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:58.059 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.315441+0000", 2024-09-18T10:14:58.059 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8589000000000001e-05 2024-09-18T10:14:58.059 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.060 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.060 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:58.060 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.515780+0000", 2024-09-18T10:14:58.060 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20033880100000001 2024-09-18T10:14:58.060 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.060 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.060 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:14:58.060 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.515798+0000", 2024-09-18T10:14:58.060 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7762e-05 2024-09-18T10:14:58.060 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.060 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.060 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:14:58.060 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.516927+0000", 2024-09-18T10:14:58.060 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0011289760000000001 2024-09-18T10:14:58.061 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.061 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.061 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:58.061 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.516995+0000", 2024-09-18T10:14:58.061 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.8319999999999996e-05 2024-09-18T10:14:58.061 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:58.061 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:58.061 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:58.061 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.061 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.061 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpRead(3.9s2 36/32 ECSubRead(tid=21, to_read={3:9bcb6eef:::smithi18925896-11:head=0,45056,0}, subchunks={3:9bcb6eef:::smithi18925896-11:head=[0,1]}, attrs_to_read=))", 2024-09-18T10:14:58.061 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:51.624042+0000", 2024-09-18T10:14:58.061 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 6.4067544190000003, 2024-09-18T10:14:58.062 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.11365088800000001, 2024-09-18T10:14:58.062 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:58.062 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:14:58.062 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:58.062 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.062 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:58.062 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.624042+0000", 2024-09-18T10:14:58.062 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:58.062 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.062 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.062 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:58.062 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.624042+0000", 2024-09-18T10:14:58.062 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:58.062 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.063 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.063 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:58.063 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.624039+0000", 2024-09-18T10:14:58.063 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999976 2024-09-18T10:14:58.063 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.063 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.063 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:58.063 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.624061+0000", 2024-09-18T10:14:58.063 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.1605999999999999e-05 2024-09-18T10:14:58.063 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.063 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.063 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:58.063 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.624099+0000", 2024-09-18T10:14:58.063 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.7972000000000002e-05 2024-09-18T10:14:58.064 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.064 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.064 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:58.064 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.624129+0000", 2024-09-18T10:14:58.064 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0555000000000001e-05 2024-09-18T10:14:58.064 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.064 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.064 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:58.064 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.737177+0000", 2024-09-18T10:14:58.064 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.113048077 2024-09-18T10:14:58.064 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.064 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.064 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:58.064 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.737692+0000", 2024-09-18T10:14:58.065 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00051495800000000004 2024-09-18T10:14:58.065 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:58.065 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:58.065 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:58.065 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.065 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.065 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:189 3.3s0 3:ce8164ce:::smithi18925896-7:head [copy-from ver 6] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:14:58.065 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:52.694279+0000", 2024-09-18T10:14:58.065 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 5.3365165599999997, 2024-09-18T10:14:58.065 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.51539283599999997, 2024-09-18T10:14:58.065 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:58.065 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:14:58.065 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:14:58.065 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:14:58.066 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:14:58.066 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 189 2024-09-18T10:14:58.066 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.066 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:58.066 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.066 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:58.066 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694279+0000", 2024-09-18T10:14:58.066 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:58.066 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.066 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.066 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:58.066 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694279+0000", 2024-09-18T10:14:58.066 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:58.067 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.067 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.067 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:58.067 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694277+0000", 2024-09-18T10:14:58.067 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999971 2024-09-18T10:14:58.067 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.067 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.067 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:58.067 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694290+0000", 2024-09-18T10:14:58.067 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3862e-05 2024-09-18T10:14:58.067 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.067 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.067 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:58.067 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694311+0000", 2024-09-18T10:14:58.067 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.0273999999999999e-05 2024-09-18T10:14:58.068 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.068 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.068 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:58.068 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694322+0000", 2024-09-18T10:14:58.068 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1722999999999999e-05 2024-09-18T10:14:58.068 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.068 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.068 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:58.068 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.794494+0000", 2024-09-18T10:14:58.069 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100171653 2024-09-18T10:14:58.069 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.069 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.069 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:14:58.069 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.794705+0000", 2024-09-18T10:14:58.069 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00021126000000000001 2024-09-18T10:14:58.069 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.069 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.069 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:14:58.069 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.096113+0000", 2024-09-18T10:14:58.069 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.30140739100000002 2024-09-18T10:14:58.069 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.069 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.069 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:14:58.070 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.098602+0000", 2024-09-18T10:14:58.070 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.002489593 2024-09-18T10:14:58.070 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.070 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.070 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:14:58.070 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.209538+0000", 2024-09-18T10:14:58.070 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.110935482 2024-09-18T10:14:58.070 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.070 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.070 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:58.070 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.209672+0000", 2024-09-18T10:14:58.070 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00013435399999999999 2024-09-18T10:14:58.070 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:58.070 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:58.071 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:58.071 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.071 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.071 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:190 3.3s0 3:ce8164ce:::smithi18925896-7:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:14:58.071 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:52.694343+0000", 2024-09-18T10:14:58.071 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 5.3364529699999999, 2024-09-18T10:14:58.071 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.40407532499999999, 2024-09-18T10:14:58.071 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:58.071 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "started", 2024-09-18T10:14:58.071 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:14:58.071 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:14:58.071 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:14:58.071 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 190 2024-09-18T10:14:58.072 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.072 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:58.072 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.072 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:58.072 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694343+0000", 2024-09-18T10:14:58.072 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:58.072 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.072 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.072 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:58.072 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694343+0000", 2024-09-18T10:14:58.072 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:58.072 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.072 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.073 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:58.073 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694342+0000", 2024-09-18T10:14:58.073 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:14:58.073 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.073 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.073 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:58.073 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694349+0000", 2024-09-18T10:14:58.073 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.5969999999999997e-06 2024-09-18T10:14:58.073 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.073 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.073 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:58.073 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694367+0000", 2024-09-18T10:14:58.073 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8323000000000001e-05 2024-09-18T10:14:58.074 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.074 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.074 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:58.074 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694376+0000", 2024-09-18T10:14:58.074 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.9740000000000008e-06 2024-09-18T10:14:58.074 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.074 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.074 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:58.074 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.795028+0000", 2024-09-18T10:14:58.074 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10065176000000001 2024-09-18T10:14:58.074 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.074 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.074 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for blocked object", 2024-09-18T10:14:58.074 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.795171+0000", 2024-09-18T10:14:58.074 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00014321399999999999 2024-09-18T10:14:58.075 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.075 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.075 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:58.075 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.098069+0000", 2024-09-18T10:14:58.075 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.30289780700000002 2024-09-18T10:14:58.075 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.075 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.075 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:14:58.075 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.098247+0000", 2024-09-18T10:14:58.075 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00017831300000000001 2024-09-18T10:14:58.075 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.075 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.075 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:58.075 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.098418+0000", 2024-09-18T10:14:58.076 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00017122599999999999 2024-09-18T10:14:58.076 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:58.076 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:58.076 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:58.076 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.076 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.076 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:194 3.3s0 3:c9ce6440:::smithi18925896-24:head [copy-from ver 6] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:14:58.076 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:52.694388+0000", 2024-09-18T10:14:58.076 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 5.3364076970000003, 2024-09-18T10:14:58.076 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.92118689200000003, 2024-09-18T10:14:58.076 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:58.076 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:14:58.076 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:14:58.076 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:14:58.077 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:14:58.077 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 194 2024-09-18T10:14:58.077 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.077 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:58.077 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.077 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:58.077 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694388+0000", 2024-09-18T10:14:58.077 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:58.077 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.077 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.077 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:58.077 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694388+0000", 2024-09-18T10:14:58.077 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:58.077 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.078 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.078 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:58.078 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694388+0000", 2024-09-18T10:14:58.078 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:14:58.078 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.078 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.078 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:58.078 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694393+0000", 2024-09-18T10:14:58.078 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.2299999999999999e-06 2024-09-18T10:14:58.078 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.078 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.078 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:58.078 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694411+0000", 2024-09-18T10:14:58.079 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7802999999999999e-05 2024-09-18T10:14:58.079 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.079 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.079 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:58.079 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694417+0000", 2024-09-18T10:14:58.079 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.4640000000000003e-06 2024-09-18T10:14:58.079 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.079 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.079 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:58.079 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.795289+0000", 2024-09-18T10:14:58.079 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100872288 2024-09-18T10:14:58.079 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.079 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.079 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:14:58.079 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.795548+0000", 2024-09-18T10:14:58.080 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00025869199999999998 2024-09-18T10:14:58.080 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.080 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.080 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:14:58.080 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.412215+0000", 2024-09-18T10:14:58.080 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.61666667100000006 2024-09-18T10:14:58.080 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.080 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.080 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:14:58.080 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.514838+0000", 2024-09-18T10:14:58.080 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10262323399999999 2024-09-18T10:14:58.080 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.080 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.080 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:14:58.081 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.615386+0000", 2024-09-18T10:14:58.081 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100548106 2024-09-18T10:14:58.081 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.081 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.081 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:58.081 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.615575+0000", 2024-09-18T10:14:58.081 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000189128 2024-09-18T10:14:58.081 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:58.081 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:58.081 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:58.081 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.081 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.081 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:195 3.3s0 3:c9ce6440:::smithi18925896-24:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:14:58.082 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:52.694424+0000", 2024-09-18T10:14:58.082 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 5.3363719610000002, 2024-09-18T10:14:58.082 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.82009685499999996, 2024-09-18T10:14:58.082 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:58.082 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "started", 2024-09-18T10:14:58.082 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:14:58.082 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:14:58.082 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:14:58.082 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 195 2024-09-18T10:14:58.082 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.082 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:58.082 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.082 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:58.082 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694424+0000", 2024-09-18T10:14:58.083 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:58.083 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.083 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.083 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:58.083 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694424+0000", 2024-09-18T10:14:58.083 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:58.083 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.083 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.083 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:58.083 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694423+0000", 2024-09-18T10:14:58.083 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999995 2024-09-18T10:14:58.083 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.083 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.084 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:58.084 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694427+0000", 2024-09-18T10:14:58.084 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2650000000000001e-06 2024-09-18T10:14:58.084 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.084 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.084 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:58.084 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694434+0000", 2024-09-18T10:14:58.084 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.5410000000000003e-06 2024-09-18T10:14:58.084 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.084 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.084 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:58.084 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694441+0000", 2024-09-18T10:14:58.084 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.187e-06 2024-09-18T10:14:58.084 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.085 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.085 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:58.085 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.795870+0000", 2024-09-18T10:14:58.085 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.101428617 2024-09-18T10:14:58.085 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.085 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.085 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for blocked object", 2024-09-18T10:14:58.085 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.796040+0000", 2024-09-18T10:14:58.085 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00017008 2024-09-18T10:14:58.085 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.085 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.085 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:58.085 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.514130+0000", 2024-09-18T10:14:58.086 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.71809022199999994 2024-09-18T10:14:58.086 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.086 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.086 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:14:58.086 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.514348+0000", 2024-09-18T10:14:58.086 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000217488 2024-09-18T10:14:58.086 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.086 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.086 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:58.086 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.514521+0000", 2024-09-18T10:14:58.086 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00017300900000000001 2024-09-18T10:14:58.086 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:58.086 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:58.086 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:58.087 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.087 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.087 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWriteReply(3.3s0 37/32 ECSubWriteReply(tid=12, last_complete=37'8, committed=1, applied=1))", 2024-09-18T10:14:58.087 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:53.411334+0000", 2024-09-18T10:14:58.087 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 4.6194622890000003, 2024-09-18T10:14:58.087 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10332827999999999, 2024-09-18T10:14:58.087 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:58.087 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:14:58.087 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:58.087 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.087 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:58.087 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.411334+0000", 2024-09-18T10:14:58.087 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:58.087 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.088 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.088 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:58.088 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.411334+0000", 2024-09-18T10:14:58.088 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:58.088 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.088 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.088 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:58.088 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.411332+0000", 2024-09-18T10:14:58.088 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:14:58.088 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.088 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.088 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:58.088 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.411341+0000", 2024-09-18T10:14:58.088 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.6060000000000003e-06 2024-09-18T10:14:58.089 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.089 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.089 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:58.089 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.411382+0000", 2024-09-18T10:14:58.089 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.1384999999999998e-05 2024-09-18T10:14:58.089 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.089 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.089 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:58.089 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.411394+0000", 2024-09-18T10:14:58.089 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1535999999999999e-05 2024-09-18T10:14:58.089 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.089 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.089 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:58.089 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.514615+0000", 2024-09-18T10:14:58.090 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.103220665 2024-09-18T10:14:58.090 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.090 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.090 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:58.090 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.514662+0000", 2024-09-18T10:14:58.090 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.7318000000000003e-05 2024-09-18T10:14:58.090 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:58.090 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:58.090 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:58.090 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.090 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.090 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.bs2 41/32 ECSubWrite(tid=58, reqid=osd.4.0:58, at_version=41'14, trim_to=0'0, roll_forward_to=39'12))", 2024-09-18T10:14:58.090 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:56.829698+0000", 2024-09-18T10:14:58.090 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 1.2010978670000001, 2024-09-18T10:14:58.091 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.71711602600000002, 2024-09-18T10:14:58.091 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:58.091 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:14:58.091 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:58.091 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.091 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:58.091 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.829698+0000", 2024-09-18T10:14:58.091 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:58.091 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.091 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.091 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:58.091 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.829698+0000", 2024-09-18T10:14:58.091 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:58.091 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.092 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.092 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:58.092 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.829697+0000", 2024-09-18T10:14:58.092 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:14:58.092 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.092 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.092 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:58.092 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.829733+0000", 2024-09-18T10:14:58.092 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.6217999999999998e-05 2024-09-18T10:14:58.092 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.092 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.092 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:58.092 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.829757+0000", 2024-09-18T10:14:58.092 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.3915000000000001e-05 2024-09-18T10:14:58.093 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.093 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.093 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:58.093 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.829769+0000", 2024-09-18T10:14:58.093 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1754e-05 2024-09-18T10:14:58.093 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.093 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.093 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:58.093 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.342965+0000", 2024-09-18T10:14:58.093 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.51319593799999996 2024-09-18T10:14:58.093 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.093 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.093 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:14:58.093 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.343020+0000", 2024-09-18T10:14:58.094 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5367e-05 2024-09-18T10:14:58.094 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.094 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.094 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:14:58.094 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.546772+0000", 2024-09-18T10:14:58.094 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.203751933 2024-09-18T10:14:58.094 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.094 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.094 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:58.094 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.546814+0000", 2024-09-18T10:14:58.094 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.2196000000000001e-05 2024-09-18T10:14:58.094 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:58.094 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:58.094 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:58.095 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.095 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.095 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.2s5 41/40 ECSubWrite(tid=22, reqid=client.4632.0:264, at_version=41'10, trim_to=0'0, roll_forward_to=0'0))", 2024-09-18T10:14:58.095 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:57.230511+0000", 2024-09-18T10:14:58.095 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 0.80028476100000001, 2024-09-18T10:14:58.095 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.19781122500000001, 2024-09-18T10:14:58.095 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:58.095 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:14:58.095 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:58.095 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.095 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:58.095 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.230511+0000", 2024-09-18T10:14:58.095 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:58.095 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.096 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.096 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:58.096 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.230511+0000", 2024-09-18T10:14:58.096 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:58.096 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.096 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.096 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:58.096 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.230509+0000", 2024-09-18T10:14:58.096 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:14:58.096 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.096 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.096 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:58.096 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.230787+0000", 2024-09-18T10:14:58.097 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00027760600000000001 2024-09-18T10:14:58.097 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.097 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.097 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:58.097 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.230832+0000", 2024-09-18T10:14:58.097 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.5117999999999997e-05 2024-09-18T10:14:58.097 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.097 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.097 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:58.097 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.230853+0000", 2024-09-18T10:14:58.097 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.1169e-05 2024-09-18T10:14:58.097 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.097 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.098 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:58.098 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.327944+0000", 2024-09-18T10:14:58.098 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.097091296999999993 2024-09-18T10:14:58.098 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.098 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.098 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:14:58.098 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.327958+0000", 2024-09-18T10:14:58.098 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3844e-05 2024-09-18T10:14:58.098 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.098 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.098 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:14:58.098 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.428275+0000", 2024-09-18T10:14:58.098 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100316583 2024-09-18T10:14:58.098 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.098 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.099 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:58.099 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.428322+0000", 2024-09-18T10:14:58.099 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.7487999999999997e-05 2024-09-18T10:14:58.099 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:58.099 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:58.099 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:58.099 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.099 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.099 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.2s5 41/40 ECSubWrite(tid=23, reqid=client.4632.0:265, at_version=41'11, trim_to=0'0, roll_forward_to=0'0))", 2024-09-18T10:14:58.099 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:57.236340+0000", 2024-09-18T10:14:58.099 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 0.79445637700000005, 2024-09-18T10:14:58.099 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.19202936600000001, 2024-09-18T10:14:58.099 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:58.099 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:14:58.100 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:58.100 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.100 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:58.100 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.236340+0000", 2024-09-18T10:14:58.100 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:58.100 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.100 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.100 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:58.100 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.236340+0000", 2024-09-18T10:14:58.100 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:58.100 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.100 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.100 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:58.101 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.236338+0000", 2024-09-18T10:14:58.101 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:14:58.101 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.101 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.101 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:58.101 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.236453+0000", 2024-09-18T10:14:58.101 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000114586 2024-09-18T10:14:58.101 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.101 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.101 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:58.101 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.236474+0000", 2024-09-18T10:14:58.101 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.1183999999999999e-05 2024-09-18T10:14:58.101 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.102 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.102 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:58.102 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.236489+0000", 2024-09-18T10:14:58.102 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4704e-05 2024-09-18T10:14:58.102 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.102 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.102 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:58.102 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.329218+0000", 2024-09-18T10:14:58.102 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.092729693000000002 2024-09-18T10:14:58.102 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.102 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.102 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:14:58.102 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.329229+0000", 2024-09-18T10:14:58.102 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.06e-05 2024-09-18T10:14:58.103 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.103 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.103 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:14:58.103 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.428335+0000", 2024-09-18T10:14:58.103 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.099105479999999996 2024-09-18T10:14:58.103 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.103 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.103 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:58.103 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.428369+0000", 2024-09-18T10:14:58.103 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.4360999999999998e-05 2024-09-18T10:14:58.103 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:58.103 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:58.103 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:58.103 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.104 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.104 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.2s5 41/40 ECSubWrite(tid=24, reqid=client.4632.0:266, at_version=41'12, trim_to=0'0, roll_forward_to=0'0))", 2024-09-18T10:14:58.104 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:57.238995+0000", 2024-09-18T10:14:58.104 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 0.79180089300000001, 2024-09-18T10:14:58.104 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.18938751000000001, 2024-09-18T10:14:58.104 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:58.104 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:14:58.104 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:58.104 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.104 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:58.104 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.238995+0000", 2024-09-18T10:14:58.104 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:58.104 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.104 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.105 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:58.105 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.238995+0000", 2024-09-18T10:14:58.105 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:58.105 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.105 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.105 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:58.105 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.238994+0000", 2024-09-18T10:14:58.105 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999995 2024-09-18T10:14:58.105 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.105 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.105 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:58.105 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.239012+0000", 2024-09-18T10:14:58.105 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8094e-05 2024-09-18T10:14:58.105 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.106 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.106 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:58.106 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.239028+0000", 2024-09-18T10:14:58.106 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.5556000000000001e-05 2024-09-18T10:14:58.106 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.106 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.106 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:58.106 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.239036+0000", 2024-09-18T10:14:58.106 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.5769999999999996e-06 2024-09-18T10:14:58.106 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.106 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.106 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:58.106 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.330076+0000", 2024-09-18T10:14:58.107 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.091040476999999995 2024-09-18T10:14:58.107 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.107 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.107 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:14:58.107 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.330101+0000", 2024-09-18T10:14:58.107 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5055999999999999e-05 2024-09-18T10:14:58.107 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.107 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.107 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:14:58.107 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.428372+0000", 2024-09-18T10:14:58.107 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.098270645000000004 2024-09-18T10:14:58.107 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.107 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.107 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:58.108 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.428383+0000", 2024-09-18T10:14:58.108 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0750999999999999e-05 2024-09-18T10:14:58.108 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:58.108 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:58.108 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:58.108 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.108 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.108 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.9s2 41/40 ECSubWrite(tid=40, reqid=osd.7.0:39, at_version=0'0, trim_to=0'0, roll_forward_to=41'27))", 2024-09-18T10:14:58.108 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:57.339579+0000", 2024-09-18T10:14:58.108 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 0.69121655900000001, 2024-09-18T10:14:58.108 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.105926179, 2024-09-18T10:14:58.108 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:58.108 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:14:58.108 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:58.109 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.109 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:58.109 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.339579+0000", 2024-09-18T10:14:58.109 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:58.109 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.109 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.109 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:58.109 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.339579+0000", 2024-09-18T10:14:58.109 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:58.109 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.109 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.109 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:58.109 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.339579+0000", 2024-09-18T10:14:58.109 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999995 2024-09-18T10:14:58.110 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.110 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.110 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:58.110 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.339594+0000", 2024-09-18T10:14:58.110 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4895999999999999e-05 2024-09-18T10:14:58.110 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.110 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.110 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:58.110 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.339610+0000", 2024-09-18T10:14:58.110 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.6773e-05 2024-09-18T10:14:58.110 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.110 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.110 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:58.110 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.339634+0000", 2024-09-18T10:14:58.111 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.3662000000000001e-05 2024-09-18T10:14:58.111 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.111 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.111 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:58.111 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.439929+0000", 2024-09-18T10:14:58.111 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100294806 2024-09-18T10:14:58.111 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.111 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.111 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:14:58.111 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.439950+0000", 2024-09-18T10:14:58.111 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.1424999999999999e-05 2024-09-18T10:14:58.111 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.111 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.111 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:14:58.111 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.445467+0000", 2024-09-18T10:14:58.112 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0055167419999999998 2024-09-18T10:14:58.112 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.112 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.112 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:58.112 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.445506+0000", 2024-09-18T10:14:58.112 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8554999999999999e-05 2024-09-18T10:14:58.112 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:58.112 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:58.112 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:58.112 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.112 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.112 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.bs2 41/32 ECSubWrite(tid=62, reqid=osd.4.0:58, at_version=0'0, trim_to=0'0, roll_forward_to=41'14))", 2024-09-18T10:14:58.112 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:57.648412+0000", 2024-09-18T10:14:58.112 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 0.38238358700000002, 2024-09-18T10:14:58.113 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.101839729, 2024-09-18T10:14:58.113 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:58.113 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:14:58.113 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:58.113 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.113 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:58.113 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.648412+0000", 2024-09-18T10:14:58.113 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:58.113 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.113 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.113 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:58.113 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.648412+0000", 2024-09-18T10:14:58.113 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:58.113 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.114 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.114 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:58.114 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.648411+0000", 2024-09-18T10:14:58.114 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:14:58.114 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.114 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.114 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:58.114 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.648439+0000", 2024-09-18T10:14:58.114 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8595999999999999e-05 2024-09-18T10:14:58.114 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.114 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.114 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:58.114 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.648477+0000", 2024-09-18T10:14:58.114 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.7638000000000001e-05 2024-09-18T10:14:58.115 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.115 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.115 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:58.115 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.748568+0000", 2024-09-18T10:14:58.115 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100091455 2024-09-18T10:14:58.115 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.115 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.115 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:58.115 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.748679+0000", 2024-09-18T10:14:58.115 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00011016200000000001 2024-09-18T10:14:58.115 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.115 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.115 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:14:58.115 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.748701+0000", 2024-09-18T10:14:58.116 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.2804999999999999e-05 2024-09-18T10:14:58.116 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.116 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.116 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:14:58.116 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.750212+0000", 2024-09-18T10:14:58.116 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0015107510000000001 2024-09-18T10:14:58.116 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.116 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.116 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:58.116 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.750252+0000", 2024-09-18T10:14:58.116 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.9987000000000002e-05 2024-09-18T10:14:58.116 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:58.116 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:58.116 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:58.117 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.117 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.117 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.4s5 41/40 ECSubWrite(tid=18, reqid=osd.8.0:17, at_version=0'0, trim_to=0'0, roll_forward_to=41'10))", 2024-09-18T10:14:58.117 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:57.651021+0000", 2024-09-18T10:14:58.117 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 0.37977493600000001, 2024-09-18T10:14:58.117 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.102866993, 2024-09-18T10:14:58.117 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:58.117 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:14:58.117 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:58.117 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.117 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:58.117 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.651021+0000", 2024-09-18T10:14:58.117 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:58.117 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.118 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.118 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:58.118 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.651021+0000", 2024-09-18T10:14:58.118 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:58.118 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.118 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.118 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:58.118 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.651020+0000", 2024-09-18T10:14:58.118 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:14:58.118 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.118 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.118 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:58.118 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.651046+0000", 2024-09-18T10:14:58.118 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6599000000000001e-05 2024-09-18T10:14:58.118 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.119 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.119 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:58.119 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.651080+0000", 2024-09-18T10:14:58.119 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.3776000000000003e-05 2024-09-18T10:14:58.119 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.119 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.119 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:58.119 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.651095+0000", 2024-09-18T10:14:58.119 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.5255999999999999e-05 2024-09-18T10:14:58.119 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.119 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.119 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:58.119 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.751346+0000", 2024-09-18T10:14:58.119 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100250595 2024-09-18T10:14:58.120 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.120 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.120 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:14:58.120 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.751371+0000", 2024-09-18T10:14:58.120 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.527e-05 2024-09-18T10:14:58.120 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.120 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.120 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:14:58.120 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.753859+0000", 2024-09-18T10:14:58.120 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0024876270000000001 2024-09-18T10:14:58.120 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.120 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.120 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:58.121 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.753888+0000", 2024-09-18T10:14:58.121 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9269999999999999e-05 2024-09-18T10:14:58.121 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:58.121 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:58.121 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:58.121 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.121 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.121 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.19s2 41/40 ECSubWrite(tid=41, reqid=osd.7.0:41, at_version=41'25, trim_to=0'0, roll_forward_to=0'0))", 2024-09-18T10:14:58.121 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:57.748903+0000", 2024-09-18T10:14:58.121 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 0.28189335999999998, 2024-09-18T10:14:58.121 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.102522723, 2024-09-18T10:14:58.121 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:58.122 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:14:58.122 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:58.122 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.122 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:58.122 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.748903+0000", 2024-09-18T10:14:58.122 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:58.122 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.122 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.122 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:58.122 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.748903+0000", 2024-09-18T10:14:58.122 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:58.122 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.122 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.122 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:58.123 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.748902+0000", 2024-09-18T10:14:58.123 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:14:58.123 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.123 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.123 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:58.123 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.748946+0000", 2024-09-18T10:14:58.123 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4589999999999998e-05 2024-09-18T10:14:58.123 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.123 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.123 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:58.123 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.748963+0000", 2024-09-18T10:14:58.123 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.6415000000000001e-05 2024-09-18T10:14:58.123 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.123 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.124 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:58.124 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.748974+0000", 2024-09-18T10:14:58.124 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.119e-05 2024-09-18T10:14:58.124 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.124 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.124 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:58.124 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.749096+0000", 2024-09-18T10:14:58.124 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000121907 2024-09-18T10:14:58.124 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.124 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.124 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:14:58.124 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.749120+0000", 2024-09-18T10:14:58.124 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4516999999999999e-05 2024-09-18T10:14:58.124 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.124 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.125 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:14:58.125 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.851267+0000", 2024-09-18T10:14:58.125 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.102146448 2024-09-18T10:14:58.125 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.125 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.125 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:58.125 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.851425+0000", 2024-09-18T10:14:58.125 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00015843000000000001 2024-09-18T10:14:58.125 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:58.125 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:58.125 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:58.125 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:58.125 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:58.125 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-09-18T10:14:58.127 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_ops_in_flight 2024-09-18T10:14:58.127 INFO:tasks.ceph.osd.6.smithi154.stderr:2024-09-18T10:14:58.120+0000 7f0abc9e1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22089) UID: 0 2024-09-18T10:14:58.228 INFO:tasks.ceph.osd.3.smithi189.stderr:2024-09-18T10:14:58.227+0000 7f2440fc7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22396) UID: 0 2024-09-18T10:14:58.231 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-09-18T10:14:58.231 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-09-18T10:14:58.231 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-09-18T10:14:58.231 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-09-18T10:14:58.240 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_blocked_ops 2024-09-18T10:14:58.329 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-09-18T10:14:58.324+0000 7fdfa8698700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22087) UID: 0 2024-09-18T10:14:58.336 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-09-18T10:14:58.336 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-09-18T10:14:58.336 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-09-18T10:14:58.336 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-09-18T10:14:58.336 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-09-18T10:14:58.345 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_historic_ops 2024-09-18T10:14:58.420 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T10:14:58.430 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-09-18T10:14:58.424+0000 7fdfa8698700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22087) UID: 0 2024-09-18T10:14:58.491 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-09-18T10:14:58.492 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-09-18T10:14:58.492 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-09-18T10:14:58.492 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-09-18T10:14:58.492 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.492 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4341.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-18T10:14:58.492 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:35.324351+0000", 2024-09-18T10:14:58.492 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 23.164080447, 2024-09-18T10:14:58.492 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.36640713699999999, 2024-09-18T10:14:58.492 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:58.492 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:14:58.492 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:14:58.493 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4341", 2024-09-18T10:14:58.493 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.29:0/3001591505", 2024-09-18T10:14:58.493 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 1 2024-09-18T10:14:58.493 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.493 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:58.493 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.493 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:58.493 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.324351+0000", 2024-09-18T10:14:58.493 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:58.493 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.493 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.493 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:58.493 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.324351+0000", 2024-09-18T10:14:58.493 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:58.494 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.494 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.494 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:58.494 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.324348+0000", 2024-09-18T10:14:58.494 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999976 2024-09-18T10:14:58.494 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.494 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.494 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:58.494 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.324359+0000", 2024-09-18T10:14:58.494 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1048e-05 2024-09-18T10:14:58.494 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.494 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.494 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:58.494 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.324383+0000", 2024-09-18T10:14:58.494 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.3813000000000001e-05 2024-09-18T10:14:58.495 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.495 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.495 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:58.495 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.324421+0000", 2024-09-18T10:14:58.495 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8012000000000001e-05 2024-09-18T10:14:58.495 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.495 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.495 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:58.495 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.324548+0000", 2024-09-18T10:14:58.495 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00012702199999999999 2024-09-18T10:14:58.495 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.495 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.495 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for peered", 2024-09-18T10:14:58.495 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.324565+0000", 2024-09-18T10:14:58.496 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.6325e-05 2024-09-18T10:14:58.496 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.496 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.496 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:58.496 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.588380+0000", 2024-09-18T10:14:58.496 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.263815458 2024-09-18T10:14:58.496 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.496 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.496 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:14:58.496 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.588626+0000", 2024-09-18T10:14:58.496 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00024633499999999998 2024-09-18T10:14:58.496 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.496 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.497 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 5", 2024-09-18T10:14:58.497 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.588803+0000", 2024-09-18T10:14:58.497 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00017640599999999999 2024-09-18T10:14:58.497 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.497 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.497 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:14:58.497 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.589943+0000", 2024-09-18T10:14:58.497 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0011400049999999999 2024-09-18T10:14:58.497 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.497 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.497 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:14:58.497 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.690623+0000", 2024-09-18T10:14:58.497 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100680032 2024-09-18T10:14:58.497 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.498 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.498 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:14:58.498 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.690702+0000", 2024-09-18T10:14:58.498 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.9342e-05 2024-09-18T10:14:58.498 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.498 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.498 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:58.498 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.690758+0000", 2024-09-18T10:14:58.498 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5639e-05 2024-09-18T10:14:58.498 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:58.498 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:58.498 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:58.498 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.498 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.499 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.cs6 36/32 ECSubWrite(tid=20, reqid=client.4632.0:150, at_version=36'9, trim_to=0'0, roll_forward_to=36'8))", 2024-09-18T10:14:58.499 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:51.214349+0000", 2024-09-18T10:14:58.499 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 7.2740816920000002, 2024-09-18T10:14:58.499 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.102450213, 2024-09-18T10:14:58.499 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:58.499 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:14:58.499 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:58.499 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.499 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:58.499 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.214349+0000", 2024-09-18T10:14:58.499 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:58.499 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.499 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.499 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:58.500 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.214349+0000", 2024-09-18T10:14:58.500 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:58.500 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.500 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.500 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:58.500 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.214349+0000", 2024-09-18T10:14:58.500 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:14:58.500 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.500 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.500 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:58.500 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.214497+0000", 2024-09-18T10:14:58.500 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00014846 2024-09-18T10:14:58.500 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.500 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.501 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:58.501 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.214524+0000", 2024-09-18T10:14:58.501 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6778999999999999e-05 2024-09-18T10:14:58.501 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.501 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.501 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:58.501 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.214533+0000", 2024-09-18T10:14:58.501 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.1589999999999999e-06 2024-09-18T10:14:58.501 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.501 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.501 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:58.501 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.214667+0000", 2024-09-18T10:14:58.501 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00013415700000000001 2024-09-18T10:14:58.502 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.502 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.502 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:14:58.502 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.214692+0000", 2024-09-18T10:14:58.502 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4986000000000001e-05 2024-09-18T10:14:58.502 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.502 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.502 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:14:58.502 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.316728+0000", 2024-09-18T10:14:58.502 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.102035442 2024-09-18T10:14:58.502 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.502 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.502 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:58.503 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.316800+0000", 2024-09-18T10:14:58.503 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.1993e-05 2024-09-18T10:14:58.503 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:58.503 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:58.503 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:58.503 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.503 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.503 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.cs6 36/32 ECSubWrite(tid=21, reqid=client.4632.0:151, at_version=36'10, trim_to=0'0, roll_forward_to=36'8))", 2024-09-18T10:14:58.503 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:51.214556+0000", 2024-09-18T10:14:58.503 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 7.2738748419999997, 2024-09-18T10:14:58.503 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10287521099999999, 2024-09-18T10:14:58.503 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:58.503 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:14:58.504 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:58.504 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.504 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:58.504 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.214556+0000", 2024-09-18T10:14:58.504 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:58.504 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.504 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.504 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:58.504 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.214556+0000", 2024-09-18T10:14:58.504 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:58.504 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.504 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.504 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:58.505 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.214556+0000", 2024-09-18T10:14:58.505 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999995 2024-09-18T10:14:58.505 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.505 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.505 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:58.505 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.214657+0000", 2024-09-18T10:14:58.505 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000101139 2024-09-18T10:14:58.505 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.505 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.505 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:58.505 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.214669+0000", 2024-09-18T10:14:58.505 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2243e-05 2024-09-18T10:14:58.505 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.506 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.506 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:58.506 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.214676+0000", 2024-09-18T10:14:58.506 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.413e-06 2024-09-18T10:14:58.506 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.506 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.506 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:58.506 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.315767+0000", 2024-09-18T10:14:58.506 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10109061699999999 2024-09-18T10:14:58.506 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.506 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.506 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:14:58.506 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.315782+0000", 2024-09-18T10:14:58.507 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.537e-05 2024-09-18T10:14:58.507 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.507 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.507 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:14:58.507 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.317393+0000", 2024-09-18T10:14:58.507 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001610708 2024-09-18T10:14:58.507 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.507 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.507 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:58.507 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.317431+0000", 2024-09-18T10:14:58.507 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8393999999999999e-05 2024-09-18T10:14:58.507 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:58.507 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:58.507 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:58.508 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.508 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.508 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.es1 36/32 ECSubWrite(tid=5, reqid=client.4632.0:147, at_version=36'4, trim_to=0'0, roll_forward_to=36'2))", 2024-09-18T10:14:58.508 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:51.329371+0000", 2024-09-18T10:14:58.508 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 7.1590599739999998, 2024-09-18T10:14:58.508 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10310915800000001, 2024-09-18T10:14:58.508 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:58.508 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:14:58.508 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:58.508 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.508 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:58.508 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.329371+0000", 2024-09-18T10:14:58.508 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:58.509 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.509 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.509 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:58.509 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.329371+0000", 2024-09-18T10:14:58.509 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:58.509 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.509 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.509 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:58.509 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.329370+0000", 2024-09-18T10:14:58.509 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:14:58.509 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.509 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.509 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:58.510 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.329445+0000", 2024-09-18T10:14:58.510 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.5643999999999996e-05 2024-09-18T10:14:58.510 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.510 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.510 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:58.510 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.329471+0000", 2024-09-18T10:14:58.510 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5613e-05 2024-09-18T10:14:58.510 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.510 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.510 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:58.510 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.429565+0000", 2024-09-18T10:14:58.510 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100093746 2024-09-18T10:14:58.511 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.511 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.511 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:58.511 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.429690+0000", 2024-09-18T10:14:58.511 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00012519699999999999 2024-09-18T10:14:58.511 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.511 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.511 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:14:58.511 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.429715+0000", 2024-09-18T10:14:58.511 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4769e-05 2024-09-18T10:14:58.511 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.512 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.512 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:14:58.512 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.432431+0000", 2024-09-18T10:14:58.512 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.002715884 2024-09-18T10:14:58.512 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.512 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.512 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:58.512 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.432480+0000", 2024-09-18T10:14:58.512 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9662e-05 2024-09-18T10:14:58.512 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:58.512 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:58.512 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:58.512 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.513 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.513 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.cs6 36/32 ECSubWrite(tid=23, reqid=client.4632.0:152, at_version=36'11, trim_to=0'0, roll_forward_to=36'10))", 2024-09-18T10:14:58.513 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:51.536630+0000", 2024-09-18T10:14:58.513 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 6.9518013849999996, 2024-09-18T10:14:58.513 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.19942764099999999, 2024-09-18T10:14:58.513 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:58.513 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:14:58.513 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:58.513 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.513 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:58.513 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.536630+0000", 2024-09-18T10:14:58.513 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:58.513 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.513 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.514 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:58.514 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.536630+0000", 2024-09-18T10:14:58.514 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:58.514 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.514 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.514 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:58.514 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.536628+0000", 2024-09-18T10:14:58.514 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:14:58.514 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.514 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.514 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:58.514 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.536706+0000", 2024-09-18T10:14:58.514 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.8518999999999998e-05 2024-09-18T10:14:58.515 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.515 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.515 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:58.515 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.536788+0000", 2024-09-18T10:14:58.515 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.2148e-05 2024-09-18T10:14:58.515 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.515 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.515 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:58.515 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.536796+0000", 2024-09-18T10:14:58.515 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.3150000000000003e-06 2024-09-18T10:14:58.515 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.515 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.516 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:58.516 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.735249+0000", 2024-09-18T10:14:58.516 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.19845374599999999 2024-09-18T10:14:58.516 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.516 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.516 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:14:58.516 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.735287+0000", 2024-09-18T10:14:58.516 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.7153000000000002e-05 2024-09-18T10:14:58.516 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.516 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.516 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:14:58.516 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.735989+0000", 2024-09-18T10:14:58.517 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00070191999999999997 2024-09-18T10:14:58.517 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.517 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.517 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:58.517 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.736057+0000", 2024-09-18T10:14:58.517 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.8769999999999994e-05 2024-09-18T10:14:58.517 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:58.517 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:58.517 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:58.517 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.517 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.517 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(osd.4.21:1 3.2s0 3:46b2871c:::smithi18925896-34:head [assert-version v6,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:14:58.517 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:51.536666+0000", 2024-09-18T10:14:58.518 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 6.9517650680000003, 2024-09-18T10:14:58.518 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.30016440500000002, 2024-09-18T10:14:58.518 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:58.518 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "started", 2024-09-18T10:14:58.518 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:58.518 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.518 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:58.518 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.536666+0000", 2024-09-18T10:14:58.518 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:58.518 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.518 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.518 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:58.518 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.536666+0000", 2024-09-18T10:14:58.518 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:58.519 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.519 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.519 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:58.519 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.536664+0000", 2024-09-18T10:14:58.519 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:14:58.519 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.519 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.519 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:58.519 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.536673+0000", 2024-09-18T10:14:58.519 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.6319999999999997e-06 2024-09-18T10:14:58.519 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.519 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.519 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:58.519 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.536688+0000", 2024-09-18T10:14:58.520 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.5058999999999999e-05 2024-09-18T10:14:58.520 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.520 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.520 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:58.520 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.536696+0000", 2024-09-18T10:14:58.520 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.8860000000000004e-06 2024-09-18T10:14:58.520 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.520 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.520 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:58.520 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.536868+0000", 2024-09-18T10:14:58.520 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000172475 2024-09-18T10:14:58.520 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.520 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.520 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:14:58.520 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.537125+0000", 2024-09-18T10:14:58.521 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000256906 2024-09-18T10:14:58.521 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.521 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.521 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:58.521 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.836830+0000", 2024-09-18T10:14:58.521 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.299705414 2024-09-18T10:14:58.521 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:58.521 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:58.521 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:58.521 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.521 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.521 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.6s1 36/32 ECSubWrite(tid=19, reqid=client.4632.0:167, at_version=36'11, trim_to=0'0, roll_forward_to=36'10))", 2024-09-18T10:14:58.521 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:51.840327+0000", 2024-09-18T10:14:58.521 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 6.6481036930000004, 2024-09-18T10:14:58.522 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.102147774, 2024-09-18T10:14:58.522 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:58.522 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:14:58.522 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:58.522 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.522 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:58.522 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.840327+0000", 2024-09-18T10:14:58.522 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:58.522 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.522 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.522 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:58.522 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.840327+0000", 2024-09-18T10:14:58.522 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:58.522 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.522 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.523 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:58.523 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.840326+0000", 2024-09-18T10:14:58.523 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:14:58.523 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.523 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.523 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:58.523 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.840534+0000", 2024-09-18T10:14:58.523 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000208168 2024-09-18T10:14:58.523 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.523 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.523 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:58.523 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.840575+0000", 2024-09-18T10:14:58.523 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.0497999999999997e-05 2024-09-18T10:14:58.523 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.523 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.524 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:58.524 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.840593+0000", 2024-09-18T10:14:58.524 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8834999999999998e-05 2024-09-18T10:14:58.524 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.524 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.524 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:58.524 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.940848+0000", 2024-09-18T10:14:58.524 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10025437500000001 2024-09-18T10:14:58.524 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.524 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.524 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:14:58.524 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.940871+0000", 2024-09-18T10:14:58.524 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.3635999999999999e-05 2024-09-18T10:14:58.524 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.524 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.525 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:14:58.525 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.942422+0000", 2024-09-18T10:14:58.525 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0015510809999999999 2024-09-18T10:14:58.525 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.525 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.525 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:58.525 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.942475+0000", 2024-09-18T10:14:58.525 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.2680999999999999e-05 2024-09-18T10:14:58.525 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:58.525 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:58.525 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:58.525 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.525 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.525 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpRead(3.6s1 37/32 ECSubRead(tid=23, to_read={3:61491e68:::smithi18925896-27:head=0,110592,0}, subchunks={3:61491e68:::smithi18925896-27:head=[0,1]}, attrs_to_read=))", 2024-09-18T10:14:58.525 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:52.798092+0000", 2024-09-18T10:14:58.526 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 5.6903394819999997, 2024-09-18T10:14:58.526 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20065064199999999, 2024-09-18T10:14:58.526 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:58.526 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:14:58.526 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:58.526 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.526 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:58.526 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.798092+0000", 2024-09-18T10:14:58.526 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:58.526 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.526 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.526 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:58.527 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.798092+0000", 2024-09-18T10:14:58.527 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:58.527 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.527 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.527 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:58.527 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.798090+0000", 2024-09-18T10:14:58.527 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:14:58.527 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.527 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.527 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:58.527 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.798120+0000", 2024-09-18T10:14:58.527 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0114999999999999e-05 2024-09-18T10:14:58.527 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.527 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.527 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:58.528 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.798160+0000", 2024-09-18T10:14:58.528 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.0105000000000003e-05 2024-09-18T10:14:58.528 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.528 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.528 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:58.528 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.798188+0000", 2024-09-18T10:14:58.528 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8121999999999999e-05 2024-09-18T10:14:58.528 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.528 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.528 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:58.528 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.998543+0000", 2024-09-18T10:14:58.528 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20035523099999999 2024-09-18T10:14:58.528 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.529 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.529 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:58.529 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.998742+0000", 2024-09-18T10:14:58.529 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00019894199999999999 2024-09-18T10:14:58.529 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:58.529 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:58.529 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:58.529 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.529 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.529 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(osd.13.22:2 3.2s0 3:46b2871c:::smithi18925896-34:head [assert-version v6,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:14:58.529 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:52.902678+0000", 2024-09-18T10:14:58.529 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 5.5857527620000003, 2024-09-18T10:14:58.529 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20245598000000001, 2024-09-18T10:14:58.529 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:58.530 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "started", 2024-09-18T10:14:58.530 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:58.530 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.530 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:58.530 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.902678+0000", 2024-09-18T10:14:58.530 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:58.530 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.530 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.530 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:58.530 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.902678+0000", 2024-09-18T10:14:58.530 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:58.530 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.530 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.531 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:58.531 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.902675+0000", 2024-09-18T10:14:58.531 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999967 2024-09-18T10:14:58.531 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.531 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.531 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:58.531 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.902692+0000", 2024-09-18T10:14:58.531 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.6809000000000001e-05 2024-09-18T10:14:58.531 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.531 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.531 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:58.531 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.902721+0000", 2024-09-18T10:14:58.532 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8915e-05 2024-09-18T10:14:58.532 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.532 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.532 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:58.532 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.902739+0000", 2024-09-18T10:14:58.532 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8278999999999999e-05 2024-09-18T10:14:58.532 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.532 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.532 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:58.532 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.902835+0000", 2024-09-18T10:14:58.532 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.6311999999999995e-05 2024-09-18T10:14:58.532 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.532 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.533 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:14:58.533 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.903032+0000", 2024-09-18T10:14:58.533 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00019696500000000001 2024-09-18T10:14:58.533 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.533 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.533 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:58.533 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.105134+0000", 2024-09-18T10:14:58.533 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20210207299999999 2024-09-18T10:14:58.533 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:58.533 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:58.533 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:58.533 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.533 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.534 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.cs6 41/32 ECSubWrite(tid=46, reqid=osd.5.0:46, at_version=41'27, trim_to=0'0, roll_forward_to=39'23))", 2024-09-18T10:14:58.534 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:56.729725+0000", 2024-09-18T10:14:58.534 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 1.758705881, 2024-09-18T10:14:58.534 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10203801999999999, 2024-09-18T10:14:58.534 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:58.534 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:14:58.534 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:58.534 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.534 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:58.534 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.729725+0000", 2024-09-18T10:14:58.534 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:58.534 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.534 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.534 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:58.534 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.729725+0000", 2024-09-18T10:14:58.535 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:58.535 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.535 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.535 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:58.535 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.729724+0000", 2024-09-18T10:14:58.535 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:14:58.535 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.535 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.535 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:58.535 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.729759+0000", 2024-09-18T10:14:58.535 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.4919999999999998e-05 2024-09-18T10:14:58.535 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.535 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.536 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:58.536 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.729775+0000", 2024-09-18T10:14:58.536 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.5492999999999999e-05 2024-09-18T10:14:58.536 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.536 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.536 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:58.536 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.729784+0000", 2024-09-18T10:14:58.536 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.5070000000000006e-06 2024-09-18T10:14:58.536 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.536 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.536 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:58.536 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.830363+0000", 2024-09-18T10:14:58.537 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100579163 2024-09-18T10:14:58.537 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.537 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.537 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:14:58.537 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.830377+0000", 2024-09-18T10:14:58.537 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3949e-05 2024-09-18T10:14:58.537 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.537 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.537 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:14:58.537 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.831722+0000", 2024-09-18T10:14:58.537 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001345335 2024-09-18T10:14:58.538 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.538 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.538 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:58.538 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.831763+0000", 2024-09-18T10:14:58.538 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.0704000000000002e-05 2024-09-18T10:14:58.538 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:58.538 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:58.539 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:58.539 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.539 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.539 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:264 3.2s0 3:46b2871c:::smithi18925896-34:head [append 0~606208] snapc 5=[5,4,3] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-18T10:14:58.539 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:56.739704+0000", 2024-09-18T10:14:58.539 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 1.748727462, 2024-09-18T10:14:58.539 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.79624036899999995, 2024-09-18T10:14:58.539 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:58.539 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:14:58.539 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:14:58.539 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:14:58.539 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:14:58.539 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 264 2024-09-18T10:14:58.540 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.540 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:58.540 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.540 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:58.540 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.739704+0000", 2024-09-18T10:14:58.540 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:58.540 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.540 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.540 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:58.540 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.739704+0000", 2024-09-18T10:14:58.540 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:58.540 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.540 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.540 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:58.541 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.739702+0000", 2024-09-18T10:14:58.541 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:14:58.541 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.541 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.541 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:58.541 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.740467+0000", 2024-09-18T10:14:58.541 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00076493100000000005 2024-09-18T10:14:58.541 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.541 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.541 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:58.541 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.740485+0000", 2024-09-18T10:14:58.542 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8516e-05 2024-09-18T10:14:58.542 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.542 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.542 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:58.542 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.740495+0000", 2024-09-18T10:14:58.542 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.2669999999999995e-06 2024-09-18T10:14:58.542 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.542 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.542 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:58.542 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.028948+0000", 2024-09-18T10:14:58.542 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.28845313500000003 2024-09-18T10:14:58.542 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.542 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.543 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for peered", 2024-09-18T10:14:58.543 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.028969+0000", 2024-09-18T10:14:58.543 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.1369000000000002e-05 2024-09-18T10:14:58.543 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.543 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.543 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:58.543 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.228324+0000", 2024-09-18T10:14:58.543 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.19935476699999999 2024-09-18T10:14:58.543 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.543 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.543 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:14:58.543 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.228558+0000", 2024-09-18T10:14:58.543 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00023393099999999999 2024-09-18T10:14:58.543 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.544 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.544 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:14:58.544 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.230390+0000", 2024-09-18T10:14:58.544 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018326970000000001 2024-09-18T10:14:58.544 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.544 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.544 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:14:58.544 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.239491+0000", 2024-09-18T10:14:58.544 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0091007950000000001 2024-09-18T10:14:58.544 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.544 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.544 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:14:58.544 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.535834+0000", 2024-09-18T10:14:58.544 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.29634234100000001 2024-09-18T10:14:58.545 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.545 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.545 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:58.545 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.535944+0000", 2024-09-18T10:14:58.545 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000110401 2024-09-18T10:14:58.545 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:58.545 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:58.545 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:58.545 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.546 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.546 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:265 3.2s0 3:46b2871c:::smithi18925896-34:head [append 0~376832] snapc 5=[5,4,3] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-18T10:14:58.546 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:56.741882+0000", 2024-09-18T10:14:58.546 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 1.7465494399999999, 2024-09-18T10:14:58.546 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.80071129500000005, 2024-09-18T10:14:58.546 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:58.546 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:14:58.546 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:14:58.546 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:14:58.546 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:14:58.547 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 265 2024-09-18T10:14:58.547 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.547 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:58.547 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.547 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:58.547 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.741882+0000", 2024-09-18T10:14:58.547 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:58.547 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.547 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.547 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:58.547 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.741882+0000", 2024-09-18T10:14:58.547 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:58.547 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.547 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.548 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:58.548 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.741880+0000", 2024-09-18T10:14:58.548 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:14:58.548 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.548 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.548 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:58.548 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742251+0000", 2024-09-18T10:14:58.548 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00037050499999999998 2024-09-18T10:14:58.548 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.548 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.548 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:58.548 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742264+0000", 2024-09-18T10:14:58.548 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2787e-05 2024-09-18T10:14:58.549 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.549 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.549 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:58.549 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742269+0000", 2024-09-18T10:14:58.549 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.0509999999999996e-06 2024-09-18T10:14:58.549 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.549 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.549 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:58.549 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.234342+0000", 2024-09-18T10:14:58.549 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.492073712 2024-09-18T10:14:58.549 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.549 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.550 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:14:58.550 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.234556+0000", 2024-09-18T10:14:58.550 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00021380699999999999 2024-09-18T10:14:58.550 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.550 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.550 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:14:58.550 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.236316+0000", 2024-09-18T10:14:58.550 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0017597100000000001 2024-09-18T10:14:58.550 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.550 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.550 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:14:58.550 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.241476+0000", 2024-09-18T10:14:58.551 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0051604449999999996 2024-09-18T10:14:58.551 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.551 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.551 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:14:58.551 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.542502+0000", 2024-09-18T10:14:58.551 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.30102573700000002 2024-09-18T10:14:58.551 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.551 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.551 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:58.551 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.542593+0000", 2024-09-18T10:14:58.551 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.0859000000000005e-05 2024-09-18T10:14:58.551 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:58.552 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:58.552 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:58.552 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.552 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.552 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:266 3.2s0 3:46b2871c:::smithi18925896-34:head [setxattr _header (56) in=7b] snapc 5=[5,4,3] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-18T10:14:58.552 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:56.742274+0000", 2024-09-18T10:14:58.552 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 1.7461568089999999, 2024-09-18T10:14:58.552 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.80056005299999999, 2024-09-18T10:14:58.552 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:58.552 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:14:58.552 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:14:58.552 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:14:58.552 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:14:58.552 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 266 2024-09-18T10:14:58.553 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.553 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:58.553 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.553 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:58.553 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742274+0000", 2024-09-18T10:14:58.553 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:58.553 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.553 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.553 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:58.553 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742274+0000", 2024-09-18T10:14:58.553 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:58.553 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.553 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.554 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:58.554 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742274+0000", 2024-09-18T10:14:58.554 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999995 2024-09-18T10:14:58.554 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.554 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.554 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:58.554 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742278+0000", 2024-09-18T10:14:58.554 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.0840000000000002e-06 2024-09-18T10:14:58.554 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.554 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.554 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:58.554 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742284+0000", 2024-09-18T10:14:58.554 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.1090000000000003e-06 2024-09-18T10:14:58.554 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.555 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.555 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:58.555 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742289+0000", 2024-09-18T10:14:58.555 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.4890000000000003e-06 2024-09-18T10:14:58.555 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.555 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.555 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:58.555 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.237660+0000", 2024-09-18T10:14:58.555 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.49537068299999998 2024-09-18T10:14:58.555 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.555 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.555 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:14:58.555 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.237924+0000", 2024-09-18T10:14:58.555 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00026387300000000003 2024-09-18T10:14:58.556 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.556 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.556 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:14:58.556 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.238979+0000", 2024-09-18T10:14:58.556 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0010549380000000001 2024-09-18T10:14:58.556 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.556 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.556 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:14:58.556 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.243571+0000", 2024-09-18T10:14:58.556 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0045919749999999999 2024-09-18T10:14:58.556 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.556 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.557 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:14:58.557 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.542742+0000", 2024-09-18T10:14:58.557 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.29917095999999999 2024-09-18T10:14:58.557 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.557 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.557 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:58.557 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.542834+0000", 2024-09-18T10:14:58.557 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.2536999999999998e-05 2024-09-18T10:14:58.557 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:58.557 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:58.557 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:58.557 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.557 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.557 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:267 3.2s0 3:46b2871c:::smithi18925896-34:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e41)", 2024-09-18T10:14:58.558 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:56.742292+0000", 2024-09-18T10:14:58.558 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 1.7461389710000002, 2024-09-18T10:14:58.558 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.98973012599999999, 2024-09-18T10:14:58.558 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:58.558 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "started", 2024-09-18T10:14:58.558 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:14:58.558 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:14:58.558 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:14:58.558 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 267 2024-09-18T10:14:58.558 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.558 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:58.558 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.558 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:58.558 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742292+0000", 2024-09-18T10:14:58.559 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:58.559 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.559 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.559 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:58.559 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742292+0000", 2024-09-18T10:14:58.559 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:58.559 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.559 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.559 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:58.559 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742292+0000", 2024-09-18T10:14:58.559 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999995 2024-09-18T10:14:58.559 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.559 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.559 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:58.560 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742294+0000", 2024-09-18T10:14:58.560 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.103e-06 2024-09-18T10:14:58.560 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.560 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.560 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:58.560 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742298+0000", 2024-09-18T10:14:58.560 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8070000000000001e-06 2024-09-18T10:14:58.560 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.560 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.560 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:58.560 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742300+0000", 2024-09-18T10:14:58.560 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5890000000000001e-06 2024-09-18T10:14:58.560 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.560 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.561 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:58.561 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.239711+0000", 2024-09-18T10:14:58.561 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.497410625 2024-09-18T10:14:58.561 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.561 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.561 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for rw locks", 2024-09-18T10:14:58.561 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.239786+0000", 2024-09-18T10:14:58.561 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.4827999999999996e-05 2024-09-18T10:14:58.561 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.561 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.561 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:58.561 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.543234+0000", 2024-09-18T10:14:58.561 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.30344799099999997 2024-09-18T10:14:58.562 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.562 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.562 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:14:58.562 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.543335+0000", 2024-09-18T10:14:58.562 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000101625 2024-09-18T10:14:58.562 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.562 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.562 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:58.562 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.732022+0000", 2024-09-18T10:14:58.562 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.18868686700000001 2024-09-18T10:14:58.562 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:58.563 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:58.563 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:58.563 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.563 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.563 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.bs6 41/32 ECSubWrite(tid=58, reqid=osd.4.0:58, at_version=41'14, trim_to=0'0, roll_forward_to=39'12))", 2024-09-18T10:14:58.563 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:56.829914+0000", 2024-09-18T10:14:58.563 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 1.658517365, 2024-09-18T10:14:58.563 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.101819968, 2024-09-18T10:14:58.563 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:58.563 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:14:58.563 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:58.563 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.563 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:58.563 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.829914+0000", 2024-09-18T10:14:58.564 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:58.564 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.564 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.564 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:58.564 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.829914+0000", 2024-09-18T10:14:58.564 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:58.564 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.564 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.564 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:58.564 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.829912+0000", 2024-09-18T10:14:58.564 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:14:58.564 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.564 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.564 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:58.565 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.829967+0000", 2024-09-18T10:14:58.565 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5133999999999998e-05 2024-09-18T10:14:58.565 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.565 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.565 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:58.565 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.829996+0000", 2024-09-18T10:14:58.565 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8872000000000001e-05 2024-09-18T10:14:58.565 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.565 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.565 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:58.565 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.930124+0000", 2024-09-18T10:14:58.565 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10012787099999999 2024-09-18T10:14:58.565 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.566 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.566 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:58.566 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.930282+0000", 2024-09-18T10:14:58.566 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00015779999999999999 2024-09-18T10:14:58.566 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.566 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.566 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:14:58.566 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.930312+0000", 2024-09-18T10:14:58.566 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9862e-05 2024-09-18T10:14:58.566 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.566 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.566 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:14:58.566 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.931672+0000", 2024-09-18T10:14:58.567 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0013604680000000001 2024-09-18T10:14:58.567 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.567 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.567 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:58.567 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.931734+0000", 2024-09-18T10:14:58.567 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.1513999999999994e-05 2024-09-18T10:14:58.567 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:58.567 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:58.567 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:58.567 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.567 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.567 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.es1 41/32 ECSubWrite(tid=16, reqid=osd.11.0:15, at_version=0'0, trim_to=0'0, roll_forward_to=41'11))", 2024-09-18T10:14:58.567 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:56.930285+0000", 2024-09-18T10:14:58.567 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 1.5581460579999999, 2024-09-18T10:14:58.568 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.211817538, 2024-09-18T10:14:58.568 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:58.568 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:14:58.568 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:58.568 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.568 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:58.568 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.930285+0000", 2024-09-18T10:14:58.568 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:58.568 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.568 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.568 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:58.568 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.930285+0000", 2024-09-18T10:14:58.568 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:58.569 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.569 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.569 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:58.569 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.930284+0000", 2024-09-18T10:14:58.569 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:14:58.569 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.569 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.569 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:58.569 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.930313+0000", 2024-09-18T10:14:58.569 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9394999999999999e-05 2024-09-18T10:14:58.569 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.569 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.569 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:58.570 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.930337+0000", 2024-09-18T10:14:58.570 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4239e-05 2024-09-18T10:14:58.570 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.570 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.570 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:58.570 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.930352+0000", 2024-09-18T10:14:58.570 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4577e-05 2024-09-18T10:14:58.570 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.570 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.570 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:58.570 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.030572+0000", 2024-09-18T10:14:58.571 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100219771 2024-09-18T10:14:58.571 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.571 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.571 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:14:58.571 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.030604+0000", 2024-09-18T10:14:58.571 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2654999999999998e-05 2024-09-18T10:14:58.571 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.571 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.571 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:14:58.571 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.142047+0000", 2024-09-18T10:14:58.571 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.111442992 2024-09-18T10:14:58.571 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.571 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.572 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:58.572 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.142103+0000", 2024-09-18T10:14:58.572 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5291999999999998e-05 2024-09-18T10:14:58.572 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:58.572 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:58.572 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:58.572 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.572 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.572 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWriteReply(3.2s0 41/40 ECSubWriteReply(tid=24, last_complete=41'12, committed=1, applied=1))", 2024-09-18T10:14:58.572 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:57.341168+0000", 2024-09-18T10:14:58.572 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 1.1472632279999999, 2024-09-18T10:14:58.572 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.194456411, 2024-09-18T10:14:58.572 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:58.572 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:14:58.573 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:58.573 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.573 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:58.573 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.341168+0000", 2024-09-18T10:14:58.573 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:58.573 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.573 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.573 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:58.573 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.341168+0000", 2024-09-18T10:14:58.573 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:58.573 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.573 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.573 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:58.574 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.341167+0000", 2024-09-18T10:14:58.574 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:14:58.574 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.574 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.574 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:58.574 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.341174+0000", 2024-09-18T10:14:58.574 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.6059999999999999e-06 2024-09-18T10:14:58.574 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.574 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.574 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:58.574 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.341191+0000", 2024-09-18T10:14:58.574 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7122e-05 2024-09-18T10:14:58.574 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.574 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.575 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:58.575 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.341198+0000", 2024-09-18T10:14:58.575 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0330000000000003e-06 2024-09-18T10:14:58.575 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.575 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.575 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:58.575 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.535588+0000", 2024-09-18T10:14:58.575 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.19439076399999999 2024-09-18T10:14:58.575 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.575 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.575 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:58.575 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.535624+0000", 2024-09-18T10:14:58.575 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5791999999999999e-05 2024-09-18T10:14:58.575 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:58.576 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:58.576 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:58.576 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.576 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.576 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWriteReply(3.2s0 41/40 ECSubWriteReply(tid=22, last_complete=41'10, committed=1, applied=1))", 2024-09-18T10:14:58.576 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:57.428484+0000", 2024-09-18T10:14:58.576 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 1.0599471499999999, 2024-09-18T10:14:58.576 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.107484473, 2024-09-18T10:14:58.576 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:58.576 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:14:58.576 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:58.576 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.576 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:58.577 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.428484+0000", 2024-09-18T10:14:58.577 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:58.577 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.577 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.577 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:58.577 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.428484+0000", 2024-09-18T10:14:58.577 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:58.577 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.577 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.577 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:58.577 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.428482+0000", 2024-09-18T10:14:58.577 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999976 2024-09-18T10:14:58.578 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.578 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.578 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:58.578 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.428495+0000", 2024-09-18T10:14:58.578 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3422e-05 2024-09-18T10:14:58.578 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.578 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.578 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:58.578 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.428524+0000", 2024-09-18T10:14:58.578 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8498999999999999e-05 2024-09-18T10:14:58.578 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.578 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.578 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:58.579 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.428541+0000", 2024-09-18T10:14:58.579 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7014999999999999e-05 2024-09-18T10:14:58.579 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.579 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.579 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:58.579 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.535712+0000", 2024-09-18T10:14:58.579 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10717154299999999 2024-09-18T10:14:58.579 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.579 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.579 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:58.579 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.535968+0000", 2024-09-18T10:14:58.579 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000256304 2024-09-18T10:14:58.579 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:58.580 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:58.580 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:58.580 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.580 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.580 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWriteReply(3.2s0 41/40 ECSubWriteReply(tid=23, last_complete=41'11, committed=1, applied=1))", 2024-09-18T10:14:58.580 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:57.428567+0000", 2024-09-18T10:14:58.580 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 1.059863883, 2024-09-18T10:14:58.580 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10848242299999999, 2024-09-18T10:14:58.580 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:58.580 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:14:58.580 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:58.580 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.580 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:58.581 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.428567+0000", 2024-09-18T10:14:58.581 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:58.581 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.581 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.581 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:58.581 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.428567+0000", 2024-09-18T10:14:58.581 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:58.581 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.581 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.581 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:58.581 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.428567+0000", 2024-09-18T10:14:58.581 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999995 2024-09-18T10:14:58.581 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.582 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.582 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:58.582 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.428571+0000", 2024-09-18T10:14:58.582 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.8019999999999999e-06 2024-09-18T10:14:58.582 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.582 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.582 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:58.582 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.428584+0000", 2024-09-18T10:14:58.582 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2247e-05 2024-09-18T10:14:58.582 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.582 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.582 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:58.582 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.528700+0000", 2024-09-18T10:14:58.583 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100116487 2024-09-18T10:14:58.583 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.583 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.583 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:58.583 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.537024+0000", 2024-09-18T10:14:58.583 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0083241979999999997 2024-09-18T10:14:58.583 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.583 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.583 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:58.583 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.537050+0000", 2024-09-18T10:14:58.583 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5179000000000001e-05 2024-09-18T10:14:58.583 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:58.583 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:58.583 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:58.584 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.584 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.584 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.6s1 41/40 ECSubWrite(tid=32, reqid=client.4632.0:250, at_version=41'14, trim_to=0'0, roll_forward_to=0'0))", 2024-09-18T10:14:58.584 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:57.553171+0000", 2024-09-18T10:14:58.584 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 0.93526051899999996, 2024-09-18T10:14:58.584 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.101624537, 2024-09-18T10:14:58.584 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:14:58.584 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:14:58.584 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:14:58.584 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.584 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:14:58.584 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.553171+0000", 2024-09-18T10:14:58.584 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:58.584 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.585 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.585 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:14:58.585 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.553171+0000", 2024-09-18T10:14:58.585 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:14:58.585 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.585 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.585 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:14:58.585 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.553170+0000", 2024-09-18T10:14:58.585 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:14:58.585 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.585 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.585 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:14:58.585 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.553236+0000", 2024-09-18T10:14:58.586 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.6303999999999994e-05 2024-09-18T10:14:58.586 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.586 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.586 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:14:58.586 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.553246+0000", 2024-09-18T10:14:58.586 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0222e-05 2024-09-18T10:14:58.586 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.586 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.586 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:14:58.586 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.553273+0000", 2024-09-18T10:14:58.586 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6344999999999999e-05 2024-09-18T10:14:58.586 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.586 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.587 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:14:58.587 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.553312+0000", 2024-09-18T10:14:58.587 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.9029999999999997e-05 2024-09-18T10:14:58.587 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.587 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.587 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:14:58.587 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.553320+0000", 2024-09-18T10:14:58.587 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.653e-06 2024-09-18T10:14:58.587 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.587 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.587 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:14:58.587 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.654747+0000", 2024-09-18T10:14:58.587 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.101426218 2024-09-18T10:14:58.587 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:14:58.588 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:14:58.588 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:14:58.588 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.654795+0000", 2024-09-18T10:14:58.588 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.8485000000000001e-05 2024-09-18T10:14:58.588 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:58.588 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:58.588 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:58.588 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:14:58.588 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:14:58.588 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-09-18T10:14:58.589 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-18T10:14:58.590 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T10:14:58.591 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-18T10:14:58.585+0000 7f14c6a1a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 22138) UID: 0 2024-09-18T10:14:58.591 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-18T10:14:58.613 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-18T10:14:58.637 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-18T10:14:58.656 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-18T10:14:58.677 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-18T10:14:58.690 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-18T10:14:58.685+0000 7f1981873700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22132) UID: 0 2024-09-18T10:14:58.698 INFO:teuthology.orchestra.run.smithi029.stdout:osd.6: {} 2024-09-18T10:14:58.699 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-18T10:14:58.699 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:14:58.708 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T10:14:58.718 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15] out_osds: [] dead_osds: [] live_osds: [0, 4, 8, 12, 1, 5, 9, 13, 2, 6, 10, 14, 3, 7, 11, 15] 2024-09-18T10:14:58.718 INFO:tasks.thrashosds.thrasher:choose_action: min_in 8 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T10:14:58.718 INFO:tasks.thrashosds.thrasher:inject_pause on osd.9 2024-09-18T10:14:58.719 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-18T10:14:58.719 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-18T10:14:58.719 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok config set filestore_inject_stall 3 2024-09-18T10:14:58.721 INFO:teuthology.orchestra.run.smithi029.stdout:osd.7: {} 2024-09-18T10:14:58.731 INFO:tasks.rados.rados.0.smithi189.stdout:102: done (2 left) 2024-09-18T10:14:58.731 INFO:tasks.rados.rados.0.smithi189.stdout:103: read oid 47 snap 4 2024-09-18T10:14:58.732 INFO:tasks.rados.rados.0.smithi189.stdout:103: expect (ObjNum 46 snap 0 seq_num 46) 2024-09-18T10:14:58.732 INFO:tasks.rados.rados.0.smithi189.stdout:104: read oid 46 snap -1 2024-09-18T10:14:58.732 INFO:tasks.rados.rados.0.smithi189.stdout:104: expect (ObjNum 45 snap 0 seq_num 45) 2024-09-18T10:14:58.732 INFO:tasks.rados.rados.0.smithi189.stdout:105: copy_from oid 1 from oid 4 current snap is 5 2024-09-18T10:14:58.732 INFO:tasks.rados.rados.0.smithi189.stdout:99: finishing write tid 4 to smithi18925896-34 2024-09-18T10:14:58.732 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 34 v 12 (ObjNum 58 snap 4 seq_num 58) dirty exists 2024-09-18T10:14:58.732 INFO:tasks.rados.rados.0.smithi189.stdout:99: left oid 34 (ObjNum 58 snap 4 seq_num 58) 2024-09-18T10:14:58.732 INFO:tasks.rados.rados.0.smithi189.stdout:94: finishing write tid 1 to smithi18925896-27 2024-09-18T10:14:58.732 INFO:tasks.rados.rados.0.smithi189.stdout:94: finishing write tid 2 to smithi18925896-27 2024-09-18T10:14:58.732 INFO:tasks.rados.rados.0.smithi189.stdout:94: finishing write tid 3 to smithi18925896-27 2024-09-18T10:14:58.732 INFO:tasks.rados.rados.0.smithi189.stdout:94: finishing write tid 4 to smithi18925896-27 2024-09-18T10:14:58.733 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 27 v 15 (ObjNum 56 snap 4 seq_num 56) dirty exists 2024-09-18T10:14:58.733 INFO:tasks.rados.rados.0.smithi189.stdout:94: left oid 27 (ObjNum 56 snap 4 seq_num 56) 2024-09-18T10:14:58.733 INFO:tasks.rados.rados.0.smithi189.stdout:94: done (4 left) 2024-09-18T10:14:58.733 INFO:tasks.rados.rados.0.smithi189.stdout:99: done (3 left) 2024-09-18T10:14:58.733 INFO:tasks.rados.rados.0.smithi189.stdout:106: rmattr oid 49 current snap is 5 2024-09-18T10:14:58.733 INFO:tasks.rados.rados.0.smithi189.stdout:106: done (3 left) 2024-09-18T10:14:58.733 INFO:tasks.rados.rados.0.smithi189.stdout:107: snap_remove snap 3 2024-09-18T10:14:58.736 INFO:teuthology.orchestra.run.smithi029.stderr:nodeep-scrub is set 2024-09-18T10:14:58.743 INFO:teuthology.orchestra.run.smithi029.stdout:osd.8: {} 2024-09-18T10:14:58.765 INFO:teuthology.orchestra.run.smithi029.stdout:osd.9: {} 2024-09-18T10:14:58.788 INFO:teuthology.orchestra.run.smithi029.stdout:osd.10: {} 2024-09-18T10:14:58.791 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-18T10:14:58.785+0000 7f578c52f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22134) UID: 0 2024-09-18T10:14:58.809 INFO:teuthology.orchestra.run.smithi029.stdout:osd.11: {} 2024-09-18T10:14:58.810 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-18T10:14:58.810 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:14:58.819 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T10:14:58.821 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-18T10:14:58.822 INFO:teuthology.orchestra.run.smithi136.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-18T10:14:58.822 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-18T10:14:58.830 INFO:teuthology.orchestra.run.smithi029.stdout:osd.12: {} 2024-09-18T10:14:58.854 INFO:teuthology.orchestra.run.smithi029.stdout:osd.13: {} 2024-09-18T10:14:58.892 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-18T10:14:58.885+0000 7f1981873700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22132) UID: 0 2024-09-18T10:14:58.919 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-18T10:14:58.919 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:14:58.928 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T10:14:58.958 INFO:teuthology.orchestra.run.smithi029.stdout:osd.14: {} 2024-09-18T10:14:58.977 INFO:teuthology.orchestra.run.smithi029.stdout:osd.15: {} 2024-09-18T10:14:58.979 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T10:14:58.980 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T10:14:58.980 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T10:14:58.980 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T10:14:58.980 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T10:14:58.980 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T10:14:58.980 INFO:teuthology.orchestra.run.smithi029.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T10:14:58.980 INFO:teuthology.orchestra.run.smithi029.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T10:14:58.980 INFO:teuthology.orchestra.run.smithi029.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-18T10:14:58.980 INFO:teuthology.orchestra.run.smithi029.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-18T10:14:58.980 INFO:teuthology.orchestra.run.smithi029.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-18T10:14:58.980 INFO:teuthology.orchestra.run.smithi029.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-18T10:14:58.980 INFO:teuthology.orchestra.run.smithi029.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-18T10:14:58.981 INFO:teuthology.orchestra.run.smithi029.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-18T10:14:58.981 INFO:teuthology.orchestra.run.smithi029.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-18T10:14:58.981 INFO:teuthology.orchestra.run.smithi029.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-18T10:14:58.992 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-18T10:14:58.985+0000 7f578c52f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22134) UID: 0 2024-09-18T10:14:59.018 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-18T10:14:59.018 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:14:59.028 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T10:14:59.093 INFO:tasks.ceph.osd.11.smithi189.stderr:2024-09-18T10:14:59.091+0000 7f718b60a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 22400) UID: 0 2024-09-18T10:14:59.122 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-18T10:14:59.122 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:14:59.131 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T10:14:59.194 INFO:tasks.ceph.osd.6.smithi154.stderr:2024-09-18T10:14:59.192+0000 7f0abc9e1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22089) UID: 0 2024-09-18T10:14:59.228 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-18T10:14:59.229 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:14:59.238 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-09-18T10:14:59.295 INFO:tasks.ceph.osd.11.smithi189.stderr:2024-09-18T10:14:59.291+0000 7f718b60a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 22400) UID: 0 2024-09-18T10:14:59.338 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-18T10:14:59.339 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:14:59.348 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-09-18T10:14:59.395 INFO:tasks.ceph.osd.14.smithi154.stderr:2024-09-18T10:14:59.392+0000 7f75a19e7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 22093) UID: 0 2024-09-18T10:14:59.491 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-18T10:14:59.491 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:14:59.497 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-18T10:14:59.490+0000 7f302f2c3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22187) UID: 0 2024-09-18T10:14:59.501 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-09-18T10:14:59.597 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-18T10:14:59.598 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:14:59.598 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-18T10:14:59.593+0000 7f00046a8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 22136) UID: 0 2024-09-18T10:14:59.607 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_ops_in_flight 2024-09-18T10:14:59.699 INFO:tasks.ceph.osd.4.smithi029.stderr:2024-09-18T10:14:59.694+0000 7f550cf6e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22189) UID: 0 2024-09-18T10:14:59.703 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-18T10:14:59.704 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:14:59.713 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_blocked_ops 2024-09-18T10:14:59.735 INFO:tasks.rados.rados.0.smithi189.stdout:107: done (3 left) 2024-09-18T10:14:59.735 INFO:tasks.rados.rados.0.smithi189.stdout:108: rmattr oid 3 current snap is 5 2024-09-18T10:14:59.735 INFO:tasks.rados.rados.0.smithi189.stdout:105: finishing copy_from racing read to smithi18925896-1 2024-09-18T10:14:59.735 INFO:tasks.rados.rados.0.smithi189.stdout:103: expect (ObjNum 46 snap 0 seq_num 46) 2024-09-18T10:14:59.759 INFO:tasks.rados.rados.0.smithi189.stdout:104: expect (ObjNum 45 snap 0 seq_num 45) 2024-09-18T10:14:59.779 INFO:tasks.rados.rados.0.smithi189.stdout:105: finishing copy_from to smithi18925896-1 2024-09-18T10:14:59.780 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 1 v 2 (ObjNum 3 snap 0 seq_num 3) dirty exists 2024-09-18T10:14:59.780 INFO:tasks.rados.rados.0.smithi189.stdout:103: done (3 left) 2024-09-18T10:14:59.780 INFO:tasks.rados.rados.0.smithi189.stdout:104: done (2 left) 2024-09-18T10:14:59.780 INFO:tasks.rados.rados.0.smithi189.stdout:105: done (1 left) 2024-09-18T10:14:59.780 INFO:tasks.rados.rados.0.smithi189.stdout:108: done (0 left) 2024-09-18T10:14:59.780 INFO:tasks.rados.rados.0.smithi189.stdout:109: read oid 36 snap 2 2024-09-18T10:14:59.780 INFO:tasks.rados.rados.0.smithi189.stdout:109: expect (ObjNum 35 snap 0 seq_num 35) 2024-09-18T10:14:59.780 INFO:tasks.rados.rados.0.smithi189.stdout:110: snap_remove snap 4 2024-09-18T10:14:59.799 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-18T10:14:59.795+0000 7f14dcc5b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22398) UID: 0 2024-09-18T10:14:59.814 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-18T10:14:59.814 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:14:59.823 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_historic_ops 2024-09-18T10:14:59.902 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-09-18T10:14:59.896+0000 7fdfa8698700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22087) UID: 0 2024-09-18T10:14:59.967 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-18T10:14:59.968 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:14:59.977 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T10:15:00.021 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-18T10:15:00.018+0000 7f302f2c3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22187) UID: 0 2024-09-18T10:15:00.089 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:00.090 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:00.099 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T10:15:00.104 INFO:tasks.ceph.osd.3.smithi189.stderr:2024-09-18T10:15:00.099+0000 7f2440fc7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22396) UID: 0 2024-09-18T10:15:00.203 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-18T10:15:00.201+0000 7f14c6a1a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 22138) UID: 0 2024-09-18T10:15:00.212 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:00.212 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:00.222 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T10:15:00.304 INFO:tasks.ceph.osd.4.smithi029.stderr:2024-09-18T10:15:00.298+0000 7f550cf6e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22189) UID: 0 2024-09-18T10:15:00.322 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:00.322 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:00.331 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T10:15:00.405 INFO:tasks.ceph.osd.15.smithi189.stderr:2024-09-18T10:15:00.403+0000 7fd0e725e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 22401) UID: 0 2024-09-18T10:15:00.475 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:00.475 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:00.485 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T10:15:00.505 INFO:tasks.ceph.osd.8.smithi029.stderr:2024-09-18T10:15:00.502+0000 7f2555c27700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 22192) UID: 0 2024-09-18T10:15:00.581 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:00.582 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:00.591 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T10:15:00.606 INFO:tasks.ceph.osd.14.smithi154.stderr:2024-09-18T10:15:00.600+0000 7f75a19e7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 22093) UID: 0 2024-09-18T10:15:00.707 INFO:tasks.ceph.osd.8.smithi029.stderr:2024-09-18T10:15:00.702+0000 7f2555c27700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 22192) UID: 0 2024-09-18T10:15:00.739 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:00.740 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:00.747 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-18T10:15:00.749 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-09-18T10:15:00.754 INFO:tasks.rados.rados.0.smithi189.stdout:110: done (1 left) 2024-09-18T10:15:00.754 INFO:tasks.rados.rados.0.smithi189.stdout:111: copy_from oid 22 from oid 15 current snap is 5 2024-09-18T10:15:00.754 INFO:tasks.rados.rados.0.smithi189.stdout:109: expect (ObjNum 35 snap 0 seq_num 35) 2024-09-18T10:15:00.777 INFO:tasks.rados.rados.0.smithi189.stdout:109: done (1 left) 2024-09-18T10:15:00.777 INFO:tasks.rados.rados.0.smithi189.stdout:112: read oid 13 snap 2 2024-09-18T10:15:00.777 INFO:tasks.rados.rados.0.smithi189.stdout:112: expect (ObjNum 12 snap 0 seq_num 12) 2024-09-18T10:15:00.777 INFO:tasks.rados.rados.0.smithi189.stdout:append oid 11 current snap is 5 2024-09-18T10:15:00.777 INFO:tasks.rados.rados.0.smithi189.stdout:113: seq_num 59 ranges {180224=622592,802816=376832} 2024-09-18T10:15:00.781 INFO:tasks.rados.rados.0.smithi189.stdout:113: writing smithi18925896-11 from 180224 to 802816 tid 1 2024-09-18T10:15:00.783 INFO:tasks.rados.rados.0.smithi189.stdout:113: writing smithi18925896-11 from 802816 to 1179648 tid 2 2024-09-18T10:15:00.783 INFO:tasks.rados.rados.0.smithi189.stdout:append oid 38 current snap is 5 2024-09-18T10:15:00.783 INFO:tasks.rados.rados.0.smithi189.stdout:114: seq_num 60 ranges {622592=638976,1261568=376832} 2024-09-18T10:15:00.791 INFO:tasks.rados.rados.0.smithi189.stdout:114: writing smithi18925896-38 from 622592 to 1261568 tid 1 2024-09-18T10:15:00.794 INFO:tasks.rados.rados.0.smithi189.stdout:114: writing smithi18925896-38 from 1261568 to 1638400 tid 2 2024-09-18T10:15:00.794 INFO:tasks.rados.rados.0.smithi189.stdout:115: copy_from oid 20 from oid 28 current snap is 5 2024-09-18T10:15:00.794 INFO:tasks.rados.rados.0.smithi189.stdout:append oid 47 current snap is 5 2024-09-18T10:15:00.794 INFO:tasks.rados.rados.0.smithi189.stdout:116: seq_num 61 ranges {786432=655360,1441792=376832} 2024-09-18T10:15:00.802 INFO:tasks.rados.rados.0.smithi189.stdout:116: writing smithi18925896-47 from 786432 to 1441792 tid 1 2024-09-18T10:15:00.804 INFO:tasks.rados.rados.0.smithi189.stdout:116: writing smithi18925896-47 from 1441792 to 1818624 tid 2 2024-09-18T10:15:00.804 INFO:tasks.rados.rados.0.smithi189.stdout:append oid 48 current snap is 5 2024-09-18T10:15:00.805 INFO:tasks.rados.rados.0.smithi189.stdout:117: seq_num 62 ranges {32768=671744,704512=376832} 2024-09-18T10:15:00.808 INFO:tasks.rados.rados.0.smithi189.stdout:117: writing smithi18925896-48 from 32768 to 704512 tid 1 2024-09-18T10:15:00.808 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-18T10:15:00.805+0000 7f578c52f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22134) UID: 0 2024-09-18T10:15:00.810 INFO:tasks.rados.rados.0.smithi189.stdout:117: writing smithi18925896-48 from 704512 to 1081344 tid 2 2024-09-18T10:15:00.810 INFO:tasks.rados.rados.0.smithi189.stdout:118: rollback oid 29 current snap is 5 2024-09-18T10:15:00.810 INFO:tasks.rados.rados.0.smithi189.stdout:rollback oid 29 to 2 2024-09-18T10:15:00.810 INFO:tasks.rados.rados.0.smithi189.stdout:119: delete oid 32 current snap is 5 2024-09-18T10:15:00.892 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:00.893 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:00.902 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-09-18T10:15:00.909 INFO:tasks.ceph.osd.15.smithi189.stderr:2024-09-18T10:15:00.907+0000 7fd0e725e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 22401) UID: 0 2024-09-18T10:15:00.991 INFO:tasks.rados.rados.0.smithi189.stdout:111: finishing copy_from racing read to smithi18925896-22 2024-09-18T10:15:00.991 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T10:15:00.996 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:00.996 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:01.006 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-09-18T10:15:01.009 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-18T10:15:01.007+0000 7f14dcc5b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22398) UID: 0 2024-09-18T10:15:01.110 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-18T10:15:01.105+0000 7f14c6a1a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 22138) UID: 0 2024-09-18T10:15:01.120 INFO:tasks.rados.rados.0.smithi189.stdout:119: done (8 left) 2024-09-18T10:15:01.121 INFO:tasks.rados.rados.0.smithi189.stdout:120: read oid 7 snap -1 2024-09-18T10:15:01.121 INFO:tasks.rados.rados.0.smithi189.stdout:120: expect (ObjNum 6 snap 0 seq_num 6) 2024-09-18T10:15:01.121 INFO:tasks.rados.rados.0.smithi189.stdout:121: rollback oid 37 current snap is 5 2024-09-18T10:15:01.121 INFO:tasks.rados.rados.0.smithi189.stdout:rollback oid 37 to 2 2024-09-18T10:15:01.121 INFO:tasks.rados.rados.0.smithi189.stdout:122: delete oid 6 current snap is 5 2024-09-18T10:15:01.153 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:01.153 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:01.162 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_ops_in_flight 2024-09-18T10:15:01.167 INFO:tasks.rados.rados.0.smithi189.stdout:118: finishing rollback tid 0 to smithi18925896-29 2024-09-18T10:15:01.168 INFO:tasks.rados.rados.0.smithi189.stdout:118: finishing rollback tid 1 to smithi18925896-29 2024-09-18T10:15:01.192 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-18T10:15:01.211 INFO:tasks.ceph.osd.11.smithi189.stderr:2024-09-18T10:15:01.207+0000 7f718b60a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 22400) UID: 0 2024-09-18T10:15:01.220 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-18T10:15:01.244 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-18T10:15:01.261 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:01.261 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:01.265 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-18T10:15:01.268 INFO:tasks.rados.rados.0.smithi189.stdout:118: finishing rollback tid 2 to smithi18925896-29 2024-09-18T10:15:01.269 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 29 v 16 (ObjNum 28 snap 0 seq_num 28) dirty exists 2024-09-18T10:15:01.271 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_blocked_ops 2024-09-18T10:15:01.290 INFO:tasks.rados.rados.0.smithi189.stdout:111: finishing copy_from to smithi18925896-22 2024-09-18T10:15:01.290 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 22 v 4 (ObjNum 10 snap 0 seq_num 10) dirty exists 2024-09-18T10:15:01.292 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-18T10:15:01.312 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-18T10:15:01.309+0000 7f14c6a1a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 22138) UID: 0 2024-09-18T10:15:01.313 INFO:tasks.rados.rados.0.smithi189.stdout:116: finishing write tid 1 to smithi18925896-47 2024-09-18T10:15:01.313 INFO:tasks.rados.rados.0.smithi189.stdout:116: finishing write tid 2 to smithi18925896-47 2024-09-18T10:15:01.313 INFO:tasks.rados.rados.0.smithi189.stdout:116: finishing write tid 3 to smithi18925896-47 2024-09-18T10:15:01.313 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-18T10:15:01.336 INFO:teuthology.orchestra.run.smithi029.stdout:osd.6: {} 2024-09-18T10:15:01.363 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:01.363 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:01.373 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_historic_ops 2024-09-18T10:15:01.414 INFO:tasks.ceph.osd.15.smithi189.stderr:2024-09-18T10:15:01.411+0000 7fd0e725e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 22401) UID: 0 2024-09-18T10:15:01.419 INFO:teuthology.orchestra.run.smithi029.stdout:osd.7: {} 2024-09-18T10:15:01.430 INFO:tasks.rados.rados.0.smithi189.stdout:121: finishing rollback tid 0 to smithi18925896-37 2024-09-18T10:15:01.430 INFO:tasks.rados.rados.0.smithi189.stdout:121: finishing rollback tid 1 to smithi18925896-37 2024-09-18T10:15:01.430 INFO:tasks.rados.rados.0.smithi189.stdout:121: finishing rollback tid 2 to smithi18925896-37 2024-09-18T10:15:01.431 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 37 v 8 (ObjNum 36 snap 0 seq_num 36) dirty exists 2024-09-18T10:15:01.441 INFO:teuthology.orchestra.run.smithi029.stdout:osd.8: {} 2024-09-18T10:15:01.464 INFO:teuthology.orchestra.run.smithi029.stdout:osd.9: {} 2024-09-18T10:15:01.471 INFO:tasks.rados.rados.0.smithi189.stdout:112: expect (ObjNum 12 snap 0 seq_num 12) 2024-09-18T10:15:01.486 INFO:teuthology.orchestra.run.smithi029.stdout:osd.10: {} 2024-09-18T10:15:01.505 INFO:teuthology.orchestra.run.smithi029.stdout:osd.11: {} 2024-09-18T10:15:01.514 INFO:tasks.ceph.osd.10.smithi154.stderr:2024-09-18T10:15:01.508+0000 7fb73221a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 22091) UID: 0 2024-09-18T10:15:01.516 INFO:tasks.rados.rados.0.smithi189.stdout:116: finishing write tid 4 to smithi18925896-47 2024-09-18T10:15:01.516 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 47 v 15 (ObjNum 61 snap 5 seq_num 61) dirty exists 2024-09-18T10:15:01.516 INFO:tasks.rados.rados.0.smithi189.stdout:116: left oid 47 (ObjNum 61 snap 5 seq_num 61) 2024-09-18T10:15:01.520 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:01.520 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:01.530 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T10:15:01.532 INFO:teuthology.orchestra.run.smithi029.stdout:osd.12: {} 2024-09-18T10:15:01.539 INFO:tasks.rados.rados.0.smithi189.stdout:115: finishing copy_from racing read to smithi18925896-20 2024-09-18T10:15:01.555 INFO:teuthology.orchestra.run.smithi029.stdout:osd.13: {} 2024-09-18T10:15:01.579 INFO:teuthology.orchestra.run.smithi029.stdout:osd.14: {} 2024-09-18T10:15:01.601 INFO:teuthology.orchestra.run.smithi029.stdout:osd.15: {} 2024-09-18T10:15:01.604 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T10:15:01.605 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T10:15:01.605 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T10:15:01.605 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T10:15:01.605 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T10:15:01.605 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T10:15:01.605 INFO:teuthology.orchestra.run.smithi029.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T10:15:01.605 INFO:teuthology.orchestra.run.smithi029.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T10:15:01.605 INFO:teuthology.orchestra.run.smithi029.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-18T10:15:01.605 INFO:teuthology.orchestra.run.smithi029.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-18T10:15:01.605 INFO:teuthology.orchestra.run.smithi029.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-18T10:15:01.605 INFO:teuthology.orchestra.run.smithi029.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-18T10:15:01.605 INFO:teuthology.orchestra.run.smithi029.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-18T10:15:01.605 INFO:teuthology.orchestra.run.smithi029.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-18T10:15:01.606 INFO:teuthology.orchestra.run.smithi029.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-18T10:15:01.606 INFO:teuthology.orchestra.run.smithi029.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-18T10:15:01.615 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-18T10:15:01.613+0000 7f1981873700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22132) UID: 0 2024-09-18T10:15:01.626 INFO:tasks.rados.rados.0.smithi189.stdout:120: expect (ObjNum 6 snap 0 seq_num 6) 2024-09-18T10:15:01.640 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-18T10:15:01.640 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-18T10:15:01.640 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-09-18T10:15:01.640 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-18T10:15:01.650 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T10:15:01.711 INFO:tasks.rados.rados.0.smithi189.stdout:113: finishing write tid 1 to smithi18925896-11 2024-09-18T10:15:01.716 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-18T10:15:01.711+0000 7f14dcc5b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22398) UID: 0 2024-09-18T10:15:01.741 INFO:tasks.rados.rados.0.smithi189.stdout:115: finishing copy_from to smithi18925896-20 2024-09-18T10:15:01.741 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 20 v 6 (ObjNum 33 snap 0 seq_num 33) dirty exists 2024-09-18T10:15:01.751 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-18T10:15:01.751 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-18T10:15:01.751 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-18T10:15:01.751 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-18T10:15:01.751 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-18T10:15:01.761 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T10:15:01.778 INFO:teuthology.orchestra.run.smithi029.stderr:noscrub is unset 2024-09-18T10:15:01.795 INFO:tasks.rados.rados.0.smithi189.stdout:111: done (10 left) 2024-09-18T10:15:01.795 INFO:tasks.rados.rados.0.smithi189.stdout:112: done (9 left) 2024-09-18T10:15:01.795 INFO:tasks.rados.rados.0.smithi189.stdout:115: done (8 left) 2024-09-18T10:15:01.795 INFO:tasks.rados.rados.0.smithi189.stdout:116: done (7 left) 2024-09-18T10:15:01.796 INFO:tasks.rados.rados.0.smithi189.stdout:118: done (6 left) 2024-09-18T10:15:01.796 INFO:tasks.rados.rados.0.smithi189.stdout:120: done (5 left) 2024-09-18T10:15:01.796 INFO:tasks.rados.rados.0.smithi189.stdout:121: done (4 left) 2024-09-18T10:15:01.796 INFO:tasks.rados.rados.0.smithi189.stdout:122: done (3 left) 2024-09-18T10:15:01.796 INFO:tasks.rados.rados.0.smithi189.stdout:123: read oid 24 snap 2 2024-09-18T10:15:01.796 INFO:tasks.rados.rados.0.smithi189.stdout:123: expect (ObjNum 28 snap 0 seq_num 28) 2024-09-18T10:15:01.796 INFO:tasks.rados.rados.0.smithi189.stdout:124: rollback oid 27 current snap is 5 2024-09-18T10:15:01.796 INFO:tasks.rados.rados.0.smithi189.stdout:rollback oid 27 to 2 2024-09-18T10:15:01.796 INFO:tasks.rados.rados.0.smithi189.stdout:125: delete oid 35 current snap is 5 2024-09-18T10:15:01.812 INFO:tasks.rados.rados.0.smithi189.stdout:113: finishing write tid 2 to smithi18925896-11 2024-09-18T10:15:01.813 INFO:tasks.rados.rados.0.smithi189.stdout:113: finishing write tid 3 to smithi18925896-11 2024-09-18T10:15:01.817 INFO:tasks.ceph.osd.15.smithi189.stderr:2024-09-18T10:15:01.815+0000 7fd0e725e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 22401) UID: 0 2024-09-18T10:15:01.862 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-18T10:15:01.862 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-18T10:15:01.863 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-18T10:15:01.863 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-18T10:15:01.863 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.863 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.2s3 36/32 ECSubWrite(tid=9, reqid=client.4632.0:100, at_version=36'5, trim_to=0'0, roll_forward_to=35'4))", 2024-09-18T10:15:01.863 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:50.301533+0000", 2024-09-18T10:15:01.863 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 11.557984628, 2024-09-18T10:15:01.863 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.102286764, 2024-09-18T10:15:01.863 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:01.863 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:01.863 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:01.863 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.863 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:01.864 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.301533+0000", 2024-09-18T10:15:01.864 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:01.864 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.864 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.864 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:01.864 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.301533+0000", 2024-09-18T10:15:01.864 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:01.864 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.864 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.864 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:01.864 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.301532+0000", 2024-09-18T10:15:01.864 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:01.864 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.865 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.865 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:01.865 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.301835+0000", 2024-09-18T10:15:01.865 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00030359099999999997 2024-09-18T10:15:01.865 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.865 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.865 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:01.865 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.301862+0000", 2024-09-18T10:15:01.865 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7282999999999999e-05 2024-09-18T10:15:01.865 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.865 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.865 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:01.866 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.301874+0000", 2024-09-18T10:15:01.866 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1590000000000001e-05 2024-09-18T10:15:01.866 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.866 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.866 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:01.866 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.402086+0000", 2024-09-18T10:15:01.866 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10021173899999999 2024-09-18T10:15:01.866 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.866 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.866 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:01.866 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.402106+0000", 2024-09-18T10:15:01.866 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0463e-05 2024-09-18T10:15:01.866 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.867 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.867 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:01.867 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.403765+0000", 2024-09-18T10:15:01.867 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0016588340000000001 2024-09-18T10:15:01.867 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.867 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.867 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:01.867 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.403820+0000", 2024-09-18T10:15:01.867 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5140999999999999e-05 2024-09-18T10:15:01.867 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:01.867 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:01.867 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:01.867 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.868 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.868 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.2s3 36/32 ECSubWrite(tid=10, reqid=client.4632.0:101, at_version=36'6, trim_to=0'0, roll_forward_to=35'4))", 2024-09-18T10:15:01.868 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:50.301900+0000", 2024-09-18T10:15:01.868 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 11.557617609999999, 2024-09-18T10:15:01.868 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10241130599999999, 2024-09-18T10:15:01.868 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:01.868 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:01.868 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:01.868 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.868 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:01.868 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.301900+0000", 2024-09-18T10:15:01.868 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:01.868 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.869 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.869 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:01.869 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.301900+0000", 2024-09-18T10:15:01.869 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:01.869 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.869 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.869 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:01.869 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.301900+0000", 2024-09-18T10:15:01.869 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:01.869 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.869 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.869 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:01.869 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.301917+0000", 2024-09-18T10:15:01.870 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6790999999999999e-05 2024-09-18T10:15:01.870 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.870 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.870 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:01.870 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.301925+0000", 2024-09-18T10:15:01.870 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3599999999999996e-06 2024-09-18T10:15:01.870 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.870 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.870 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:01.870 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.301930+0000", 2024-09-18T10:15:01.870 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4369999999999998e-06 2024-09-18T10:15:01.870 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.870 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.871 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:01.871 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.402886+0000", 2024-09-18T10:15:01.871 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100955429 2024-09-18T10:15:01.871 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.871 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.871 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:01.871 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.402908+0000", 2024-09-18T10:15:01.871 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2276000000000002e-05 2024-09-18T10:15:01.871 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.871 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.871 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:01.871 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.404270+0000", 2024-09-18T10:15:01.871 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001362121 2024-09-18T10:15:01.872 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.872 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.872 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:01.872 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.404312+0000", 2024-09-18T10:15:01.872 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1505999999999999e-05 2024-09-18T10:15:01.872 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:01.872 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:01.872 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:01.872 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.872 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.872 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.bs5 36/32 ECSubWrite(tid=25, reqid=client.4632.0:118, at_version=36'3, trim_to=0'0, roll_forward_to=35'2))", 2024-09-18T10:15:01.872 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:50.709294+0000", 2024-09-18T10:15:01.872 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 11.150223934, 2024-09-18T10:15:01.872 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.102322202, 2024-09-18T10:15:01.873 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:01.873 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:01.873 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:01.873 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.873 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:01.873 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.709294+0000", 2024-09-18T10:15:01.873 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:01.873 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.873 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.873 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:01.873 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.709294+0000", 2024-09-18T10:15:01.873 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:01.874 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.874 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.874 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:01.874 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.709292+0000", 2024-09-18T10:15:01.874 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:01.874 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.874 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.874 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:01.874 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.709955+0000", 2024-09-18T10:15:01.874 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00066278599999999995 2024-09-18T10:15:01.874 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.874 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.874 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:01.874 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.709987+0000", 2024-09-18T10:15:01.874 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2002000000000003e-05 2024-09-18T10:15:01.875 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.875 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.875 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:01.875 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.710001+0000", 2024-09-18T10:15:01.875 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4073000000000001e-05 2024-09-18T10:15:01.875 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.875 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.875 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:01.875 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.810188+0000", 2024-09-18T10:15:01.875 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10018661800000001 2024-09-18T10:15:01.875 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.875 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.875 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:01.876 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.810198+0000", 2024-09-18T10:15:01.876 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0349999999999999e-05 2024-09-18T10:15:01.876 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.876 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.876 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:01.876 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.811559+0000", 2024-09-18T10:15:01.876 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001361115 2024-09-18T10:15:01.876 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.876 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.876 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:01.876 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.811616+0000", 2024-09-18T10:15:01.876 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7238000000000003e-05 2024-09-18T10:15:01.876 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:01.877 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:01.877 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:01.877 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.877 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.877 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.8s5 36/32 ECSubWrite(tid=119, reqid=client.4632.0:130, at_version=36'5, trim_to=0'0, roll_forward_to=35'4))", 2024-09-18T10:15:01.877 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:50.722395+0000", 2024-09-18T10:15:01.877 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 11.13712306, 2024-09-18T10:15:01.877 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10448325999999999, 2024-09-18T10:15:01.877 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:01.877 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:01.877 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:01.877 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.877 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:01.877 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.722395+0000", 2024-09-18T10:15:01.878 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:01.878 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.878 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.878 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:01.878 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.722395+0000", 2024-09-18T10:15:01.878 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:01.878 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.878 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.878 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:01.878 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.722393+0000", 2024-09-18T10:15:01.878 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:01.878 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.878 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.879 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:01.879 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.722993+0000", 2024-09-18T10:15:01.879 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00059969600000000004 2024-09-18T10:15:01.879 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.879 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.879 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:01.879 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.723025+0000", 2024-09-18T10:15:01.879 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2364999999999999e-05 2024-09-18T10:15:01.879 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.879 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.879 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:01.879 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.723039+0000", 2024-09-18T10:15:01.879 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3709999999999999e-05 2024-09-18T10:15:01.880 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.880 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.880 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:01.880 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.823280+0000", 2024-09-18T10:15:01.880 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100241126 2024-09-18T10:15:01.880 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.880 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.880 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:01.880 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.823298+0000", 2024-09-18T10:15:01.880 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7745999999999999e-05 2024-09-18T10:15:01.880 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.880 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.880 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:01.881 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.826820+0000", 2024-09-18T10:15:01.881 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0035221470000000002 2024-09-18T10:15:01.881 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.881 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.881 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:01.881 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.826878+0000", 2024-09-18T10:15:01.881 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8344000000000002e-05 2024-09-18T10:15:01.881 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:01.881 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:01.881 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:01.881 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.881 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.881 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.bs5 36/32 ECSubWrite(tid=26, reqid=client.4632.0:119, at_version=36'4, trim_to=0'0, roll_forward_to=35'2))", 2024-09-18T10:15:01.882 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:50.812131+0000", 2024-09-18T10:15:01.882 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 11.047387254, 2024-09-18T10:15:01.882 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.201739738, 2024-09-18T10:15:01.882 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:01.882 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:01.882 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:01.882 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.882 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:01.882 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.812131+0000", 2024-09-18T10:15:01.882 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:01.882 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.882 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.882 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:01.883 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.812131+0000", 2024-09-18T10:15:01.883 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:01.883 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.883 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.883 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:01.883 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.812129+0000", 2024-09-18T10:15:01.883 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:01.883 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.883 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.883 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:01.883 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.812166+0000", 2024-09-18T10:15:01.883 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6309000000000001e-05 2024-09-18T10:15:01.883 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.883 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.884 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:01.884 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.812199+0000", 2024-09-18T10:15:01.884 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3219000000000002e-05 2024-09-18T10:15:01.884 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.884 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.884 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:01.884 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.812225+0000", 2024-09-18T10:15:01.884 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6302999999999999e-05 2024-09-18T10:15:01.884 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.884 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.884 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:01.884 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.012537+0000", 2024-09-18T10:15:01.885 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20031156899999999 2024-09-18T10:15:01.885 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.885 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.885 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:01.885 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.012564+0000", 2024-09-18T10:15:01.885 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7165000000000002e-05 2024-09-18T10:15:01.885 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.885 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.885 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:01.885 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.013814+0000", 2024-09-18T10:15:01.885 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001249836 2024-09-18T10:15:01.885 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.886 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.886 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:01.886 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.013871+0000", 2024-09-18T10:15:01.886 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6894000000000001e-05 2024-09-18T10:15:01.886 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:01.886 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:01.886 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:01.886 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.886 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.886 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.8s5 36/32 ECSubWrite(tid=121, reqid=client.4632.0:131, at_version=0'0, trim_to=0'0, roll_forward_to=36'6))", 2024-09-18T10:15:01.886 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:51.214254+0000", 2024-09-18T10:15:01.886 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.645263663, 2024-09-18T10:15:01.887 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.101744853, 2024-09-18T10:15:01.887 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:01.887 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:01.887 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:01.887 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.887 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:01.887 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.214254+0000", 2024-09-18T10:15:01.887 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:01.887 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.887 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.887 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:01.888 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.214254+0000", 2024-09-18T10:15:01.888 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:01.888 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.888 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.888 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:01.888 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.214253+0000", 2024-09-18T10:15:01.888 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:01.888 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.888 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.888 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:01.888 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.214281+0000", 2024-09-18T10:15:01.888 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8249e-05 2024-09-18T10:15:01.889 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.889 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.889 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:01.889 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.214311+0000", 2024-09-18T10:15:01.889 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9739000000000001e-05 2024-09-18T10:15:01.889 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.889 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.889 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:01.889 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.314400+0000", 2024-09-18T10:15:01.889 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10008956500000001 2024-09-18T10:15:01.889 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.889 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.889 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:01.889 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.314547+0000", 2024-09-18T10:15:01.890 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014725200000000001 2024-09-18T10:15:01.890 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.890 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.890 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:01.890 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.314581+0000", 2024-09-18T10:15:01.890 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3674999999999999e-05 2024-09-18T10:15:01.890 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.890 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.890 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:01.890 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.315954+0000", 2024-09-18T10:15:01.890 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0013734540000000001 2024-09-18T10:15:01.890 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.890 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.891 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:01.891 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.315999+0000", 2024-09-18T10:15:01.891 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4818999999999999e-05 2024-09-18T10:15:01.891 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:01.891 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:01.891 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:01.891 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.891 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.891 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.bs5 36/32 ECSubWrite(tid=35, reqid=client.4632.0:160, at_version=36'5, trim_to=0'0, roll_forward_to=36'4))", 2024-09-18T10:15:01.891 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:51.315565+0000", 2024-09-18T10:15:01.891 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.543953468, 2024-09-18T10:15:01.891 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10154431999999999, 2024-09-18T10:15:01.891 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:01.891 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:01.892 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:01.892 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.892 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:01.892 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.315565+0000", 2024-09-18T10:15:01.892 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:01.892 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.892 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.892 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:01.892 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.315565+0000", 2024-09-18T10:15:01.892 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:01.892 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.892 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.892 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:01.892 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.315563+0000", 2024-09-18T10:15:01.893 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:01.893 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.893 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.893 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:01.893 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.315603+0000", 2024-09-18T10:15:01.893 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0045e-05 2024-09-18T10:15:01.893 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.893 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.893 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:01.893 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.315623+0000", 2024-09-18T10:15:01.893 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9749999999999999e-05 2024-09-18T10:15:01.893 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.893 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.894 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:01.894 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.315634+0000", 2024-09-18T10:15:01.894 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1399e-05 2024-09-18T10:15:01.894 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.894 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.894 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:01.894 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.415886+0000", 2024-09-18T10:15:01.894 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100251874 2024-09-18T10:15:01.894 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.894 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.894 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:01.894 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.415901+0000", 2024-09-18T10:15:01.894 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4698999999999999e-05 2024-09-18T10:15:01.895 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.895 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.895 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:01.895 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.417028+0000", 2024-09-18T10:15:01.895 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00112751 2024-09-18T10:15:01.895 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.895 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.895 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:01.895 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.417109+0000", 2024-09-18T10:15:01.895 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0829999999999997e-05 2024-09-18T10:15:01.895 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:01.895 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:01.896 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:01.896 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.896 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.896 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.8s5 36/32 ECSubWrite(tid=124, reqid=client.4632.0:140, at_version=36'8, trim_to=0'0, roll_forward_to=36'6))", 2024-09-18T10:15:01.896 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:51.417633+0000", 2024-09-18T10:15:01.896 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.441884733, 2024-09-18T10:15:01.896 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10143042200000001, 2024-09-18T10:15:01.896 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:01.896 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:01.896 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:01.896 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.896 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:01.896 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.417633+0000", 2024-09-18T10:15:01.897 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:01.897 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.897 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.897 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:01.897 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.417633+0000", 2024-09-18T10:15:01.897 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:01.897 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.897 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.897 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:01.897 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.417632+0000", 2024-09-18T10:15:01.897 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:01.898 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.898 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.898 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:01.898 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.417707+0000", 2024-09-18T10:15:01.898 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5537999999999994e-05 2024-09-18T10:15:01.898 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.898 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.898 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:01.898 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.417764+0000", 2024-09-18T10:15:01.898 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7014e-05 2024-09-18T10:15:01.898 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.898 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.898 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:01.899 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.417784+0000", 2024-09-18T10:15:01.899 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0190000000000002e-05 2024-09-18T10:15:01.899 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.899 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.899 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:01.899 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.517966+0000", 2024-09-18T10:15:01.899 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100181626 2024-09-18T10:15:01.899 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.899 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.899 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:01.899 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.517981+0000", 2024-09-18T10:15:01.899 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4486e-05 2024-09-18T10:15:01.899 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.900 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.900 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:01.900 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.518999+0000", 2024-09-18T10:15:01.900 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001018241 2024-09-18T10:15:01.900 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.900 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.900 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:01.900 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.519064+0000", 2024-09-18T10:15:01.900 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4974000000000005e-05 2024-09-18T10:15:01.900 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:01.900 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:01.900 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:01.900 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.901 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.901 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.bs5 36/32 ECSubWrite(tid=37, reqid=client.4632.0:160, at_version=0'0, trim_to=0'0, roll_forward_to=36'5))", 2024-09-18T10:15:01.901 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:51.532347+0000", 2024-09-18T10:15:01.901 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.327170814, 2024-09-18T10:15:01.901 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10127454, 2024-09-18T10:15:01.901 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:01.901 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:01.901 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:01.901 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.901 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:01.901 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.532347+0000", 2024-09-18T10:15:01.901 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:01.901 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.901 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.902 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:01.902 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.532347+0000", 2024-09-18T10:15:01.902 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:01.902 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.902 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.902 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:01.902 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.532346+0000", 2024-09-18T10:15:01.902 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:01.902 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.902 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.902 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:01.902 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.532371+0000", 2024-09-18T10:15:01.902 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5548999999999999e-05 2024-09-18T10:15:01.903 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.903 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.903 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:01.903 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.532403+0000", 2024-09-18T10:15:01.903 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1758999999999999e-05 2024-09-18T10:15:01.903 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.903 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.903 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:01.903 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.532417+0000", 2024-09-18T10:15:01.903 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4413000000000001e-05 2024-09-18T10:15:01.903 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.903 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.903 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:01.903 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.632611+0000", 2024-09-18T10:15:01.904 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100193492 2024-09-18T10:15:01.904 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.904 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.904 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:01.904 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.632625+0000", 2024-09-18T10:15:01.904 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4469e-05 2024-09-18T10:15:01.904 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.904 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.904 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:01.904 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.633557+0000", 2024-09-18T10:15:01.904 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00093178 2024-09-18T10:15:01.904 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.904 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.905 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:01.905 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.633622+0000", 2024-09-18T10:15:01.905 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4861000000000001e-05 2024-09-18T10:15:01.905 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:01.905 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:01.905 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:01.905 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.905 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.905 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.8s5 36/32 ECSubWrite(tid=127, reqid=client.4632.0:154, at_version=36'9, trim_to=0'0, roll_forward_to=36'8))", 2024-09-18T10:15:01.905 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:51.840375+0000", 2024-09-18T10:15:01.905 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.019142816, 2024-09-18T10:15:01.905 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10196929, 2024-09-18T10:15:01.905 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:01.906 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:01.906 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:01.906 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.906 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:01.906 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.840375+0000", 2024-09-18T10:15:01.906 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:01.906 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.906 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.906 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:01.906 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.840375+0000", 2024-09-18T10:15:01.906 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:01.906 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.906 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.906 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:01.907 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.840374+0000", 2024-09-18T10:15:01.907 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:01.907 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.907 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.907 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:01.907 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.840521+0000", 2024-09-18T10:15:01.907 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000147641 2024-09-18T10:15:01.907 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.907 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.907 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:01.907 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.840564+0000", 2024-09-18T10:15:01.907 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2895999999999997e-05 2024-09-18T10:15:01.907 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.907 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.908 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:01.908 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.840585+0000", 2024-09-18T10:15:01.908 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0539000000000001e-05 2024-09-18T10:15:01.908 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.908 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.908 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:01.908 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.940817+0000", 2024-09-18T10:15:01.908 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100232166 2024-09-18T10:15:01.908 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.908 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.908 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:01.908 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.940832+0000", 2024-09-18T10:15:01.908 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5616000000000001e-05 2024-09-18T10:15:01.909 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.909 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.909 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:01.909 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.942287+0000", 2024-09-18T10:15:01.909 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001454398 2024-09-18T10:15:01.909 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.909 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.909 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:01.909 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.942345+0000", 2024-09-18T10:15:01.909 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7813999999999999e-05 2024-09-18T10:15:01.909 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:01.909 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:01.909 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:01.909 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.910 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.910 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.as1 37/32 ECSubWrite(tid=33, reqid=client.4632.0:207, at_version=37'9, trim_to=0'0, roll_forward_to=36'7))", 2024-09-18T10:15:01.910 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:53.209503+0000", 2024-09-18T10:15:01.910 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 8.6500149210000004, 2024-09-18T10:15:01.910 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.103337639, 2024-09-18T10:15:01.910 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:01.910 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:01.910 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:01.910 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.910 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:01.910 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.209503+0000", 2024-09-18T10:15:01.910 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:01.910 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.910 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.911 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:01.911 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.209503+0000", 2024-09-18T10:15:01.911 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:01.911 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.911 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.911 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:01.911 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.209501+0000", 2024-09-18T10:15:01.911 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:01.911 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.911 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.911 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:01.911 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.209748+0000", 2024-09-18T10:15:01.911 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00024739800000000002 2024-09-18T10:15:01.912 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.912 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.912 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:01.912 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.209784+0000", 2024-09-18T10:15:01.912 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5615999999999999e-05 2024-09-18T10:15:01.912 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.912 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.912 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:01.912 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.209799+0000", 2024-09-18T10:15:01.912 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4796e-05 2024-09-18T10:15:01.912 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.912 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.912 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:01.913 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.309992+0000", 2024-09-18T10:15:01.913 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10019278299999999 2024-09-18T10:15:01.913 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.913 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.913 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:01.913 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.310018+0000", 2024-09-18T10:15:01.913 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6392000000000001e-05 2024-09-18T10:15:01.913 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.913 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.913 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:01.913 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.312776+0000", 2024-09-18T10:15:01.913 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0027584200000000001 2024-09-18T10:15:01.914 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.914 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.914 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:01.914 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.312841+0000", 2024-09-18T10:15:01.914 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4348000000000001e-05 2024-09-18T10:15:01.914 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:01.914 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:01.914 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:01.914 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.914 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.914 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.bs5 41/32 ECSubWrite(tid=58, reqid=osd.4.0:58, at_version=41'14, trim_to=0'0, roll_forward_to=39'12))", 2024-09-18T10:15:01.914 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:56.830099+0000", 2024-09-18T10:15:01.914 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.0294186239999998, 2024-09-18T10:15:01.915 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10180473800000001, 2024-09-18T10:15:01.915 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:01.915 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:01.915 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:01.915 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.915 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:01.915 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:56.830099+0000", 2024-09-18T10:15:01.915 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:01.915 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.915 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.915 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:01.915 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:56.830099+0000", 2024-09-18T10:15:01.915 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:01.916 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.916 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.916 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:01.916 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:56.830099+0000", 2024-09-18T10:15:01.916 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:01.916 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.916 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.916 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:01.916 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:56.830180+0000", 2024-09-18T10:15:01.916 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1074000000000003e-05 2024-09-18T10:15:01.916 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.916 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.917 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:01.917 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:56.830220+0000", 2024-09-18T10:15:01.917 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0142000000000003e-05 2024-09-18T10:15:01.917 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.917 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.917 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:01.917 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:56.830238+0000", 2024-09-18T10:15:01.917 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8056000000000001e-05 2024-09-18T10:15:01.917 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.917 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.917 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:01.917 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:56.930521+0000", 2024-09-18T10:15:01.918 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100283026 2024-09-18T10:15:01.918 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.918 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.918 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:01.918 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:56.930599+0000", 2024-09-18T10:15:01.918 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8071000000000005e-05 2024-09-18T10:15:01.918 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.918 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.918 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:01.918 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:56.931860+0000", 2024-09-18T10:15:01.918 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0012610200000000001 2024-09-18T10:15:01.918 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.918 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.919 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:01.919 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:56.931904+0000", 2024-09-18T10:15:01.919 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4058999999999999e-05 2024-09-18T10:15:01.919 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:01.919 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:01.919 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:01.919 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.919 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.919 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.as1 41/32 ECSubWrite(tid=38, reqid=osd.13.0:38, at_version=41'12, trim_to=0'0, roll_forward_to=37'9))", 2024-09-18T10:15:01.919 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:56.934067+0000", 2024-09-18T10:15:01.919 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.9254515730000001, 2024-09-18T10:15:01.919 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.102868934, 2024-09-18T10:15:01.919 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:01.919 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:01.920 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:01.920 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.920 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:01.920 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:56.934067+0000", 2024-09-18T10:15:01.920 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:01.920 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.920 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.920 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:01.920 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:56.934067+0000", 2024-09-18T10:15:01.920 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:01.920 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.920 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.920 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:01.920 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:56.934065+0000", 2024-09-18T10:15:01.921 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:01.921 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.921 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.921 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:01.921 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:56.934108+0000", 2024-09-18T10:15:01.921 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3170999999999997e-05 2024-09-18T10:15:01.921 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.921 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.921 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:01.921 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:56.934137+0000", 2024-09-18T10:15:01.921 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9119e-05 2024-09-18T10:15:01.921 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.921 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.921 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:01.922 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:56.934150+0000", 2024-09-18T10:15:01.922 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2824e-05 2024-09-18T10:15:01.922 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.922 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.922 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:01.922 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.034355+0000", 2024-09-18T10:15:01.922 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10020481000000001 2024-09-18T10:15:01.922 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.922 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.922 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:01.922 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.034398+0000", 2024-09-18T10:15:01.922 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3642e-05 2024-09-18T10:15:01.922 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.922 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.923 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:01.923 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.036893+0000", 2024-09-18T10:15:01.923 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0024942229999999998 2024-09-18T10:15:01.923 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.923 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.923 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:01.923 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.036935+0000", 2024-09-18T10:15:01.923 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2821000000000002e-05 2024-09-18T10:15:01.923 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:01.923 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:01.923 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:01.923 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.923 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.923 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.as1 41/32 ECSubWrite(tid=42, reqid=osd.13.0:38, at_version=0'0, trim_to=0'0, roll_forward_to=41'12))", 2024-09-18T10:15:01.924 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:57.232992+0000", 2024-09-18T10:15:01.924 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.6265259820000004, 2024-09-18T10:15:01.924 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10174161499999999, 2024-09-18T10:15:01.924 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:01.924 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:01.924 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:01.924 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.924 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:01.924 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.232992+0000", 2024-09-18T10:15:01.924 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:01.924 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.924 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.924 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:01.924 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.232992+0000", 2024-09-18T10:15:01.925 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:01.925 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.925 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.925 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:01.925 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.232991+0000", 2024-09-18T10:15:01.925 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:01.925 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.925 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.925 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:01.925 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.233016+0000", 2024-09-18T10:15:01.925 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5456000000000002e-05 2024-09-18T10:15:01.925 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.925 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.926 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:01.926 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.233059+0000", 2024-09-18T10:15:01.926 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3458999999999998e-05 2024-09-18T10:15:01.926 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.926 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.926 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:01.926 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.233079+0000", 2024-09-18T10:15:01.926 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9259e-05 2024-09-18T10:15:01.926 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.926 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.926 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:01.926 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.333278+0000", 2024-09-18T10:15:01.926 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100199184 2024-09-18T10:15:01.927 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.927 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.927 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:01.927 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.333294+0000", 2024-09-18T10:15:01.927 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6252999999999998e-05 2024-09-18T10:15:01.927 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.927 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.927 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:01.927 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.334686+0000", 2024-09-18T10:15:01.927 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0013916110000000001 2024-09-18T10:15:01.927 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.927 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.927 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:01.928 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.334734+0000", 2024-09-18T10:15:01.928 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7936000000000003e-05 2024-09-18T10:15:01.928 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:01.928 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:01.928 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:01.928 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.928 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.928 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.2s3 41/40 ECSubWrite(tid=23, reqid=client.4632.0:265, at_version=41'11, trim_to=0'0, roll_forward_to=0'0))", 2024-09-18T10:15:01.928 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:57.236544+0000", 2024-09-18T10:15:01.928 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.6229740020000003, 2024-09-18T10:15:01.928 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20209979, 2024-09-18T10:15:01.928 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:01.928 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:01.928 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:01.929 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.929 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:01.929 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.236544+0000", 2024-09-18T10:15:01.929 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:01.929 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.929 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.929 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:01.929 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.236544+0000", 2024-09-18T10:15:01.929 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:01.929 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.929 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.929 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:01.929 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.236543+0000", 2024-09-18T10:15:01.930 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:01.930 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.930 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.930 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:01.930 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.236820+0000", 2024-09-18T10:15:01.930 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000277068 2024-09-18T10:15:01.930 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.930 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.930 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:01.930 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.236876+0000", 2024-09-18T10:15:01.930 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6555000000000003e-05 2024-09-18T10:15:01.930 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.930 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.930 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:01.931 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.336982+0000", 2024-09-18T10:15:01.931 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100105679 2024-09-18T10:15:01.931 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.931 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.931 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:01.931 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.337122+0000", 2024-09-18T10:15:01.931 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000139777 2024-09-18T10:15:01.931 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.931 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.931 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:01.931 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.337135+0000", 2024-09-18T10:15:01.931 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3559e-05 2024-09-18T10:15:01.931 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.932 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.932 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:01.932 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.438600+0000", 2024-09-18T10:15:01.932 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.101464572 2024-09-18T10:15:01.932 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.932 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.932 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:01.932 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.438644+0000", 2024-09-18T10:15:01.932 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4085000000000002e-05 2024-09-18T10:15:01.932 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:01.932 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:01.932 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:01.932 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.933 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.933 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.2s3 41/40 ECSubWrite(tid=24, reqid=client.4632.0:266, at_version=41'12, trim_to=0'0, roll_forward_to=0'0))", 2024-09-18T10:15:01.933 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:57.337029+0000", 2024-09-18T10:15:01.933 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.5224891549999997, 2024-09-18T10:15:01.933 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10168448099999999, 2024-09-18T10:15:01.933 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:01.933 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:01.933 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:01.933 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.933 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:01.933 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.337029+0000", 2024-09-18T10:15:01.933 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:01.933 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.934 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.934 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:01.934 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.337029+0000", 2024-09-18T10:15:01.934 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:01.934 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.934 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.934 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:01.934 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.337028+0000", 2024-09-18T10:15:01.934 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:01.934 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.934 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.934 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:01.934 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.337067+0000", 2024-09-18T10:15:01.934 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8991e-05 2024-09-18T10:15:01.935 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.935 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.935 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:01.935 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.337090+0000", 2024-09-18T10:15:01.935 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3332e-05 2024-09-18T10:15:01.935 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.935 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.935 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:01.935 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.337101+0000", 2024-09-18T10:15:01.935 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0872e-05 2024-09-18T10:15:01.935 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.935 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.935 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:01.936 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.337889+0000", 2024-09-18T10:15:01.936 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00078866099999999996 2024-09-18T10:15:01.936 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.936 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.936 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:01.936 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.338008+0000", 2024-09-18T10:15:01.936 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000118313 2024-09-18T10:15:01.936 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.936 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.936 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:01.936 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.438675+0000", 2024-09-18T10:15:01.936 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10066691 2024-09-18T10:15:01.936 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.937 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.937 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:01.937 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.438713+0000", 2024-09-18T10:15:01.937 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8739000000000003e-05 2024-09-18T10:15:01.937 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:01.937 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:01.937 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:01.937 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.937 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.937 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.2s3 41/40 ECSubWrite(tid=25, reqid=client.4632.0:266, at_version=0'0, trim_to=0'0, roll_forward_to=41'12))", 2024-09-18T10:15:01.937 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:57.543345+0000", 2024-09-18T10:15:01.937 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.3161735920000002, 2024-09-18T10:15:01.938 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.18858536100000001, 2024-09-18T10:15:01.938 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:01.938 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:01.938 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:01.938 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.938 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:01.938 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.543345+0000", 2024-09-18T10:15:01.938 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:01.938 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.938 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.938 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:01.938 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.543345+0000", 2024-09-18T10:15:01.938 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:01.938 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.939 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.939 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:01.939 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.543343+0000", 2024-09-18T10:15:01.939 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:01.939 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.939 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.939 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:01.939 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.543385+0000", 2024-09-18T10:15:01.939 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2487999999999997e-05 2024-09-18T10:15:01.939 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.939 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.939 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:01.939 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.543428+0000", 2024-09-18T10:15:01.940 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2315999999999999e-05 2024-09-18T10:15:01.940 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.940 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.940 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:01.940 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.543446+0000", 2024-09-18T10:15:01.940 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8189000000000002e-05 2024-09-18T10:15:01.940 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.940 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.940 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:01.940 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.730560+0000", 2024-09-18T10:15:01.940 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.187114473 2024-09-18T10:15:01.940 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.940 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.941 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:01.941 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.730585+0000", 2024-09-18T10:15:01.941 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4652e-05 2024-09-18T10:15:01.941 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.941 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.941 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:01.941 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.731859+0000", 2024-09-18T10:15:01.941 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001274425 2024-09-18T10:15:01.941 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.941 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.941 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:01.941 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.731930+0000", 2024-09-18T10:15:01.941 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0474000000000003e-05 2024-09-18T10:15:01.941 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:01.942 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:01.942 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:01.942 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.942 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.942 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpRead(3.2s3 41/40 ECSubRead(tid=26, to_read={3:46b2871c:::smithi18925896-34:head=0,4096,0}, subchunks={3:46b2871c:::smithi18925896-34:head=[0,1]}, attrs_to_read=))", 2024-09-18T10:15:01.942 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:57.543972+0000", 2024-09-18T10:15:01.942 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.3155459660000002, 2024-09-18T10:15:01.942 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.18721849300000001, 2024-09-18T10:15:01.942 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:01.942 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:01.942 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:01.942 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.942 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:01.943 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.543972+0000", 2024-09-18T10:15:01.943 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:01.943 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.943 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.943 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:01.943 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.543972+0000", 2024-09-18T10:15:01.943 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:01.943 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.943 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.943 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:01.943 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.543970+0000", 2024-09-18T10:15:01.943 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:01.943 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.944 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.944 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:01.944 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.543996+0000", 2024-09-18T10:15:01.944 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5069e-05 2024-09-18T10:15:01.944 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.944 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.944 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:01.944 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.544056+0000", 2024-09-18T10:15:01.944 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0355000000000001e-05 2024-09-18T10:15:01.944 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.944 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.944 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:01.944 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.544079+0000", 2024-09-18T10:15:01.945 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3396000000000001e-05 2024-09-18T10:15:01.945 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.945 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.945 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:01.945 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.730946+0000", 2024-09-18T10:15:01.945 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.186866644 2024-09-18T10:15:01.945 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.945 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.945 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:01.945 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.731191+0000", 2024-09-18T10:15:01.945 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00024467899999999998 2024-09-18T10:15:01.945 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:01.945 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:01.945 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:01.946 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.946 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.946 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.8s5 41/40 ECSubWrite(tid=132, reqid=osd.6.0:132, at_version=41'13, trim_to=0'0, roll_forward_to=0'0))", 2024-09-18T10:15:01.946 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:57.548222+0000", 2024-09-18T10:15:01.946 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.3112964959999998, 2024-09-18T10:15:01.946 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.102715687, 2024-09-18T10:15:01.946 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:01.946 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:01.946 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:01.946 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.946 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:01.946 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.548222+0000", 2024-09-18T10:15:01.946 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:01.947 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.947 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.947 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:01.947 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.548222+0000", 2024-09-18T10:15:01.947 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:01.947 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.947 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.947 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:01.947 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.548220+0000", 2024-09-18T10:15:01.947 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:01.947 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.947 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.947 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:01.947 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.548274+0000", 2024-09-18T10:15:01.948 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4721e-05 2024-09-18T10:15:01.948 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.948 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.948 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:01.948 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.548307+0000", 2024-09-18T10:15:01.948 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2929000000000003e-05 2024-09-18T10:15:01.948 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.948 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.948 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:01.948 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.548323+0000", 2024-09-18T10:15:01.948 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5534999999999999e-05 2024-09-18T10:15:01.948 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.948 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.949 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:01.949 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.648511+0000", 2024-09-18T10:15:01.949 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10018864700000001 2024-09-18T10:15:01.949 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.949 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.949 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:01.949 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.648541+0000", 2024-09-18T10:15:01.949 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9561999999999999e-05 2024-09-18T10:15:01.949 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.950 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.950 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:01.950 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.650891+0000", 2024-09-18T10:15:01.950 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002349842 2024-09-18T10:15:01.950 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.950 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.950 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:01.950 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.650937+0000", 2024-09-18T10:15:01.950 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6489000000000002e-05 2024-09-18T10:15:01.950 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:01.950 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:01.950 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:01.950 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.950 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.951 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.8s5 42/40 ECSubWrite(tid=133, reqid=osd.6.0:132, at_version=0'0, trim_to=0'0, roll_forward_to=41'13))", 2024-09-18T10:15:01.951 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:58.049905+0000", 2024-09-18T10:15:01.951 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.8096132000000003, 2024-09-18T10:15:01.951 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.101683442, 2024-09-18T10:15:01.951 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:01.951 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:01.951 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:01.951 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.951 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:01.951 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:58.049905+0000", 2024-09-18T10:15:01.951 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:01.951 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.951 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.951 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:01.952 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:58.049905+0000", 2024-09-18T10:15:01.952 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:01.952 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.952 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.952 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:01.952 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:58.049904+0000", 2024-09-18T10:15:01.952 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:01.952 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.952 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.952 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:01.952 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:58.049923+0000", 2024-09-18T10:15:01.952 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8836999999999999e-05 2024-09-18T10:15:01.952 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.953 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.953 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:01.953 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:58.049968+0000", 2024-09-18T10:15:01.953 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4341000000000001e-05 2024-09-18T10:15:01.953 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.953 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.953 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:01.953 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:58.049981+0000", 2024-09-18T10:15:01.953 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.381e-05 2024-09-18T10:15:01.954 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.954 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.954 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:01.954 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:58.150246+0000", 2024-09-18T10:15:01.954 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10026481800000001 2024-09-18T10:15:01.954 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.954 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.954 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:01.954 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:58.150262+0000", 2024-09-18T10:15:01.954 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6197000000000001e-05 2024-09-18T10:15:01.954 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.954 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.954 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:01.954 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:58.151539+0000", 2024-09-18T10:15:01.955 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001276505 2024-09-18T10:15:01.955 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:01.955 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:01.955 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:01.955 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:58.151588+0000", 2024-09-18T10:15:01.955 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9400999999999997e-05 2024-09-18T10:15:01.955 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:01.955 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:01.955 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:01.955 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:01.955 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:01.955 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-18T10:15:01.957 INFO:tasks.rados.rados.0.smithi189.stdout:114: finishing write tid 1 to smithi18925896-38 2024-09-18T10:15:01.957 INFO:tasks.rados.rados.0.smithi189.stdout:114: finishing write tid 2 to smithi18925896-38 2024-09-18T10:15:01.957 INFO:tasks.rados.rados.0.smithi189.stdout:114: finishing write tid 3 to smithi18925896-38 2024-09-18T10:15:01.957 INFO:tasks.rados.rados.0.smithi189.stdout:117: finishing write tid 1 to smithi18925896-48 2024-09-18T10:15:01.957 INFO:tasks.rados.rados.0.smithi189.stdout:117: finishing write tid 2 to smithi18925896-48 2024-09-18T10:15:01.958 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T10:15:01.959 INFO:tasks.ceph.osd.4.smithi029.stderr:2024-09-18T10:15:01.954+0000 7f550cf6e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22189) UID: 0 2024-09-18T10:15:02.018 INFO:tasks.rados.rados.0.smithi189.stdout:117: finishing write tid 3 to smithi18925896-48 2024-09-18T10:15:02.058 INFO:tasks.ceph.osd.11.smithi189.stderr:2024-09-18T10:15:02.055+0000 7f718b60a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 22400) UID: 0 2024-09-18T10:15:02.064 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-18T10:15:02.064 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-18T10:15:02.064 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-09-18T10:15:02.064 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-18T10:15:02.074 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T10:15:02.100 INFO:tasks.rados.rados.0.smithi189.stdout:124: finishing rollback tid 0 to smithi18925896-27 2024-09-18T10:15:02.100 INFO:tasks.rados.rados.0.smithi189.stdout:124: finishing rollback tid 1 to smithi18925896-27 2024-09-18T10:15:02.101 INFO:tasks.rados.rados.0.smithi189.stdout:124: finishing rollback tid 2 to smithi18925896-27 2024-09-18T10:15:02.101 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 27 v 28 (ObjNum 26 snap 0 seq_num 26) dirty exists 2024-09-18T10:15:02.101 INFO:tasks.rados.rados.0.smithi189.stdout:124: done (5 left) 2024-09-18T10:15:02.101 INFO:tasks.rados.rados.0.smithi189.stdout:125: done (4 left) 2024-09-18T10:15:02.101 INFO:tasks.rados.rados.0.smithi189.stdout:126: setattr oid 3 current snap is 5 2024-09-18T10:15:02.104 INFO:tasks.rados.rados.0.smithi189.stdout:123: expect (ObjNum 28 snap 0 seq_num 28) 2024-09-18T10:15:02.122 INFO:tasks.rados.rados.0.smithi189.stdout:123: done (4 left) 2024-09-18T10:15:02.122 INFO:tasks.rados.rados.0.smithi189.stdout:127: snap_create 2024-09-18T10:15:02.159 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-18T10:15:02.157+0000 7f14c6a1a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 22138) UID: 0 2024-09-18T10:15:02.176 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-18T10:15:02.176 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-18T10:15:02.176 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-18T10:15:02.176 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-18T10:15:02.176 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-18T10:15:02.185 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T10:15:02.223 INFO:tasks.rados.rados.0.smithi189.stdout:113: finishing write tid 4 to smithi18925896-11 2024-09-18T10:15:02.223 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 11 v 31 (ObjNum 59 snap 5 seq_num 59) dirty exists 2024-09-18T10:15:02.223 INFO:tasks.rados.rados.0.smithi189.stdout:113: left oid 11 (ObjNum 59 snap 5 seq_num 59) 2024-09-18T10:15:02.228 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 3 v 17 (ObjNum 54 snap 3 seq_num 54) dirty exists 2024-09-18T10:15:02.260 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-18T10:15:02.257+0000 7f578c52f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22134) UID: 0 2024-09-18T10:15:02.285 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-18T10:15:02.285 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-18T10:15:02.286 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-18T10:15:02.286 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-18T10:15:02.286 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.286 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:150 3.cs0 3:3972e5c3:::smithi18925896-50:head [append 0~458752,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:02.287 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:51.125225+0000", 2024-09-18T10:15:02.287 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 11.156812151, 2024-09-18T10:15:02.287 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.40871674099999999, 2024-09-18T10:15:02.287 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:02.287 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:02.287 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:02.287 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:02.287 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:02.287 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 150 2024-09-18T10:15:02.288 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.288 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:02.288 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.288 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:02.288 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.125225+0000", 2024-09-18T10:15:02.288 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:02.288 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.288 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.288 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:02.288 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.125225+0000", 2024-09-18T10:15:02.288 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:02.289 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.289 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.289 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:02.289 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.125222+0000", 2024-09-18T10:15:02.290 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999967 2024-09-18T10:15:02.290 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.290 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.290 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:02.290 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.126038+0000", 2024-09-18T10:15:02.290 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00081693399999999995 2024-09-18T10:15:02.290 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.290 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.290 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:02.290 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.126072+0000", 2024-09-18T10:15:02.290 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4038999999999997e-05 2024-09-18T10:15:02.290 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.290 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.291 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:02.291 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.126084+0000", 2024-09-18T10:15:02.291 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1908e-05 2024-09-18T10:15:02.291 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.291 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.291 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:02.291 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.126184+0000", 2024-09-18T10:15:02.291 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9868e-05 2024-09-18T10:15:02.291 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.291 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.291 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:02.291 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.126439+0000", 2024-09-18T10:15:02.291 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00025457799999999999 2024-09-18T10:15:02.291 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.292 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.292 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:02.292 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.128215+0000", 2024-09-18T10:15:02.292 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0017764499999999999 2024-09-18T10:15:02.292 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.292 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.292 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:02.292 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.232589+0000", 2024-09-18T10:15:02.292 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.104374015 2024-09-18T10:15:02.292 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.292 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.292 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:15:02.292 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.533834+0000", 2024-09-18T10:15:02.292 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30124498 2024-09-18T10:15:02.293 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.293 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.293 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:02.293 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.533941+0000", 2024-09-18T10:15:02.293 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010718299999999999 2024-09-18T10:15:02.293 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:02.293 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:02.293 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:02.293 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.293 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.293 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:151 3.cs0 3:3972e5c3:::smithi18925896-50:head [append 0~376832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:02.293 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:51.127704+0000", 2024-09-18T10:15:02.293 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 11.154333046, 2024-09-18T10:15:02.294 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.40663223799999998, 2024-09-18T10:15:02.294 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:02.294 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:02.294 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:02.294 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:02.294 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:02.294 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 151 2024-09-18T10:15:02.294 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.294 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:02.294 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.294 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:02.294 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.127704+0000", 2024-09-18T10:15:02.294 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:02.294 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.295 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.295 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:02.295 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.127704+0000", 2024-09-18T10:15:02.295 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:02.295 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.295 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.295 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:02.295 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.127702+0000", 2024-09-18T10:15:02.295 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:02.295 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.295 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.295 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:02.295 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.128054+0000", 2024-09-18T10:15:02.296 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00035175199999999999 2024-09-18T10:15:02.296 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.296 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.296 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:02.296 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.128091+0000", 2024-09-18T10:15:02.296 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7035e-05 2024-09-18T10:15:02.296 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.296 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.296 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:02.296 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.128105+0000", 2024-09-18T10:15:02.296 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.428e-05 2024-09-18T10:15:02.296 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.296 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.297 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:02.297 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.129336+0000", 2024-09-18T10:15:02.297 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001231326 2024-09-18T10:15:02.297 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.297 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.297 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:02.297 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.129489+0000", 2024-09-18T10:15:02.297 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015266100000000001 2024-09-18T10:15:02.297 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.297 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.297 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:02.297 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.130757+0000", 2024-09-18T10:15:02.298 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0012684409999999999 2024-09-18T10:15:02.298 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.298 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.298 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:02.298 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.332853+0000", 2024-09-18T10:15:02.298 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20209580599999999 2024-09-18T10:15:02.298 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.298 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.298 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:15:02.298 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.534218+0000", 2024-09-18T10:15:02.298 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.2013653 2024-09-18T10:15:02.298 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.298 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.298 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:02.299 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.534336+0000", 2024-09-18T10:15:02.299 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011760900000000001 2024-09-18T10:15:02.299 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:02.299 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:02.299 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:02.299 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.299 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.299 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:152 3.cs0 3:3972e5c3:::smithi18925896-50:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:02.299 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:51.128123+0000", 2024-09-18T10:15:02.299 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 11.153913479, 2024-09-18T10:15:02.299 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.80287000200000003, 2024-09-18T10:15:02.299 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:02.299 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:02.300 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:02.300 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:02.300 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:02.300 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 152 2024-09-18T10:15:02.300 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.300 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:02.300 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.300 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:02.300 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.128123+0000", 2024-09-18T10:15:02.300 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:02.300 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.300 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.300 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:02.301 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.128123+0000", 2024-09-18T10:15:02.301 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:02.301 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.301 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.301 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:02.301 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.128122+0000", 2024-09-18T10:15:02.301 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:02.301 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.301 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.301 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:02.301 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.128129+0000", 2024-09-18T10:15:02.301 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6730000000000003e-06 2024-09-18T10:15:02.301 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.301 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.302 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:02.302 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.128156+0000", 2024-09-18T10:15:02.302 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7489000000000001e-05 2024-09-18T10:15:02.302 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.302 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.302 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:02.302 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.128173+0000", 2024-09-18T10:15:02.302 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7102999999999999e-05 2024-09-18T10:15:02.302 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.302 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.302 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:02.302 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.535139+0000", 2024-09-18T10:15:02.302 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.40696592599999998 2024-09-18T10:15:02.303 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.303 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.303 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:02.303 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.535380+0000", 2024-09-18T10:15:02.303 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00024023600000000001 2024-09-18T10:15:02.303 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.303 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.303 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:02.303 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.536784+0000", 2024-09-18T10:15:02.303 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0014046950000000001 2024-09-18T10:15:02.303 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.303 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.303 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:02.303 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.537964+0000", 2024-09-18T10:15:02.304 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0011793050000000001 2024-09-18T10:15:02.304 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.304 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.304 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:15:02.304 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.930841+0000", 2024-09-18T10:15:02.304 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.39287700599999997 2024-09-18T10:15:02.304 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.304 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.304 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:02.304 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.930993+0000", 2024-09-18T10:15:02.304 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015280799999999999 2024-09-18T10:15:02.304 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:02.304 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:02.304 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:02.305 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.305 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.305 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:153 3.cs0 3:3972e5c3:::smithi18925896-50:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:02.305 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:51.128181+0000", 2024-09-18T10:15:02.305 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 11.153855584, 2024-09-18T10:15:02.305 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2060332149999999, 2024-09-18T10:15:02.305 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:02.305 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-18T10:15:02.305 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:02.305 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:02.305 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:02.305 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 153 2024-09-18T10:15:02.305 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.306 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:02.306 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.306 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:02.306 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.128181+0000", 2024-09-18T10:15:02.306 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:02.306 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.306 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.306 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:02.306 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.128181+0000", 2024-09-18T10:15:02.306 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:02.306 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.306 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.306 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:02.306 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.128180+0000", 2024-09-18T10:15:02.307 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:15:02.307 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.307 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.307 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:02.307 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.128186+0000", 2024-09-18T10:15:02.307 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.31e-06 2024-09-18T10:15:02.307 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.307 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.307 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:02.307 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.128195+0000", 2024-09-18T10:15:02.307 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5540000000000003e-06 2024-09-18T10:15:02.307 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.307 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.307 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:02.308 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.128201+0000", 2024-09-18T10:15:02.308 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2129999999999996e-06 2024-09-18T10:15:02.308 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.308 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.308 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:02.308 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.537382+0000", 2024-09-18T10:15:02.308 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.40918055399999997 2024-09-18T10:15:02.308 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.308 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.308 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2024-09-18T10:15:02.308 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.537571+0000", 2024-09-18T10:15:02.308 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00018903299999999999 2024-09-18T10:15:02.308 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.308 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.309 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:02.309 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.931789+0000", 2024-09-18T10:15:02.309 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.39421780499999998 2024-09-18T10:15:02.309 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.309 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.309 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:02.309 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.932023+0000", 2024-09-18T10:15:02.309 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000233868 2024-09-18T10:15:02.309 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.309 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.309 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:02.309 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.334215+0000", 2024-09-18T10:15:02.309 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.40219177499999997 2024-09-18T10:15:02.309 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:02.310 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:02.310 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:02.310 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.310 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.310 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWriteReply(3.cs0 36/32 ECSubWriteReply(tid=20, last_complete=36'9, committed=1, applied=1))", 2024-09-18T10:15:02.310 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:51.213977+0000", 2024-09-18T10:15:02.310 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 11.068060306, 2024-09-18T10:15:02.310 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.21934668099999999, 2024-09-18T10:15:02.310 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:02.310 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:02.310 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:02.310 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.310 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:02.311 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.213977+0000", 2024-09-18T10:15:02.311 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:02.311 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.311 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.311 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:02.311 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.213977+0000", 2024-09-18T10:15:02.311 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:02.311 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.311 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.311 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:02.311 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.213975+0000", 2024-09-18T10:15:02.311 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:02.311 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.311 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.312 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:02.312 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.213988+0000", 2024-09-18T10:15:02.312 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3579e-05 2024-09-18T10:15:02.312 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.312 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.312 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:02.312 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.214020+0000", 2024-09-18T10:15:02.312 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2005999999999998e-05 2024-09-18T10:15:02.312 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.312 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.312 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:02.312 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.214032+0000", 2024-09-18T10:15:02.312 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1960000000000001e-05 2024-09-18T10:15:02.312 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.313 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.313 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:02.313 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.433299+0000", 2024-09-18T10:15:02.313 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.21926704299999999 2024-09-18T10:15:02.313 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.313 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.313 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:02.313 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.433323+0000", 2024-09-18T10:15:02.313 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3796000000000001e-05 2024-09-18T10:15:02.313 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:02.313 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:02.313 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:02.313 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.313 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.314 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWriteReply(3.cs0 36/32 ECSubWriteReply(tid=21, last_complete=36'10, committed=1, applied=1))", 2024-09-18T10:15:02.314 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:51.214781+0000", 2024-09-18T10:15:02.314 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 11.067255857999999, 2024-09-18T10:15:02.314 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.218783754, 2024-09-18T10:15:02.314 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:02.314 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:02.314 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:02.314 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.314 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:02.314 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.214781+0000", 2024-09-18T10:15:02.314 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:02.314 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.314 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.314 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:02.315 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.214781+0000", 2024-09-18T10:15:02.315 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:02.315 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.315 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.315 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:02.315 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.214779+0000", 2024-09-18T10:15:02.315 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999976 2024-09-18T10:15:02.315 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.315 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.315 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:02.315 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.214792+0000", 2024-09-18T10:15:02.315 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.322e-05 2024-09-18T10:15:02.315 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.316 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.316 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:02.316 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.214821+0000", 2024-09-18T10:15:02.316 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8812000000000001e-05 2024-09-18T10:15:02.316 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.316 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.316 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:02.316 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.214835+0000", 2024-09-18T10:15:02.316 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4511999999999999e-05 2024-09-18T10:15:02.316 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.316 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.316 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:02.316 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.433549+0000", 2024-09-18T10:15:02.316 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.218713834 2024-09-18T10:15:02.317 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.317 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.317 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:02.317 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.433565+0000", 2024-09-18T10:15:02.317 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5619000000000001e-05 2024-09-18T10:15:02.317 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:02.317 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:02.317 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:02.317 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.317 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.317 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:161 3.cs0 3:34d26c88:::smithi18925896-36:head [read 0~589824,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:02.317 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:51.532533+0000", 2024-09-18T10:15:02.317 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.749503925999999, 2024-09-18T10:15:02.317 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30885369800000001, 2024-09-18T10:15:02.318 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:02.318 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-18T10:15:02.318 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:02.318 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:02.318 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:02.318 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 161 2024-09-18T10:15:02.318 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.318 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:02.318 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.318 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:02.318 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.532533+0000", 2024-09-18T10:15:02.318 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:02.318 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.319 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.319 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:02.319 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.532533+0000", 2024-09-18T10:15:02.319 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:02.319 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.319 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.319 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:02.319 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.532530+0000", 2024-09-18T10:15:02.319 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999971 2024-09-18T10:15:02.319 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.319 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.319 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:02.319 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.532544+0000", 2024-09-18T10:15:02.319 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4219999999999999e-05 2024-09-18T10:15:02.320 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.320 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.320 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:02.320 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.532580+0000", 2024-09-18T10:15:02.320 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5315000000000003e-05 2024-09-18T10:15:02.320 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.320 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.320 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:02.320 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.532594+0000", 2024-09-18T10:15:02.320 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4216000000000001e-05 2024-09-18T10:15:02.320 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.320 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.320 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:02.320 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.538016+0000", 2024-09-18T10:15:02.321 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0054220559999999998 2024-09-18T10:15:02.321 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.321 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.321 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:02.321 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.538264+0000", 2024-09-18T10:15:02.321 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000248471 2024-09-18T10:15:02.321 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.321 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.321 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:02.321 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.841387+0000", 2024-09-18T10:15:02.321 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30312219699999998 2024-09-18T10:15:02.321 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:02.321 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:02.321 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:02.322 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.322 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.322 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:162 3.cs0 3:34d26c88:::smithi18925896-36:head [read 0~589824,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:02.322 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:51.532607+0000", 2024-09-18T10:15:02.322 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.749430242000001, 2024-09-18T10:15:02.322 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.400964512, 2024-09-18T10:15:02.322 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:02.322 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-18T10:15:02.322 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:02.322 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:02.322 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:02.322 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 162 2024-09-18T10:15:02.322 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.322 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:02.323 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.323 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:02.323 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.532607+0000", 2024-09-18T10:15:02.323 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:02.323 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.323 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.323 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:02.323 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.532607+0000", 2024-09-18T10:15:02.323 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:02.323 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.323 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.323 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:02.323 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.532606+0000", 2024-09-18T10:15:02.324 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:15:02.324 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.324 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.324 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:02.324 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.532612+0000", 2024-09-18T10:15:02.324 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0599999999999996e-06 2024-09-18T10:15:02.324 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.324 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.324 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:02.324 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.532621+0000", 2024-09-18T10:15:02.324 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6190000000000006e-06 2024-09-18T10:15:02.324 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.324 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.325 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:02.325 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.532627+0000", 2024-09-18T10:15:02.325 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3469999999999998e-06 2024-09-18T10:15:02.325 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.325 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.325 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:02.325 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.538755+0000", 2024-09-18T10:15:02.325 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0061281729999999998 2024-09-18T10:15:02.325 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.325 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.325 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:02.325 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.538992+0000", 2024-09-18T10:15:02.325 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00023700500000000001 2024-09-18T10:15:02.326 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.326 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.326 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:02.326 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.933571+0000", 2024-09-18T10:15:02.326 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.39457930800000002 2024-09-18T10:15:02.326 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:02.326 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:02.326 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:02.326 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.326 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.326 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:163 3.cs0 3:34d26c88:::smithi18925896-36:head [read 0~589824,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:02.326 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:51.532633+0000", 2024-09-18T10:15:02.326 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.749403997, 2024-09-18T10:15:02.326 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.70024756899999996, 2024-09-18T10:15:02.327 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:02.327 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-18T10:15:02.327 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:02.327 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:02.327 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:02.327 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 163 2024-09-18T10:15:02.327 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.327 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:02.327 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.327 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:02.327 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.532633+0000", 2024-09-18T10:15:02.327 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:02.327 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.327 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.328 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:02.328 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.532633+0000", 2024-09-18T10:15:02.328 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:02.328 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.328 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.328 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:02.328 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.532632+0000", 2024-09-18T10:15:02.328 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:15:02.328 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.328 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.328 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:02.328 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.532636+0000", 2024-09-18T10:15:02.328 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3529999999999997e-06 2024-09-18T10:15:02.328 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.329 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.329 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:02.329 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.532645+0000", 2024-09-18T10:15:02.329 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7099999999999996e-06 2024-09-18T10:15:02.329 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.329 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.329 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:02.329 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.532650+0000", 2024-09-18T10:15:02.329 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3029999999999999e-06 2024-09-18T10:15:02.329 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.329 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.329 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:02.329 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.930003+0000", 2024-09-18T10:15:02.329 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.397352333 2024-09-18T10:15:02.330 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.330 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.330 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:02.330 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.930174+0000", 2024-09-18T10:15:02.330 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017110000000000001 2024-09-18T10:15:02.330 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.330 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.330 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:02.330 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.232880+0000", 2024-09-18T10:15:02.330 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30270652300000001 2024-09-18T10:15:02.330 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:02.330 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:02.330 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:02.331 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.331 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.331 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpReadReply(3.cs0 36/32 ECSubReadReply(tid=25, attrs_read=0))", 2024-09-18T10:15:02.331 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:51.540164+0000", 2024-09-18T10:15:02.331 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.74187251, 2024-09-18T10:15:02.331 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.29990201900000002, 2024-09-18T10:15:02.331 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:02.331 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:02.331 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:02.331 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.331 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:02.332 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.540164+0000", 2024-09-18T10:15:02.332 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:02.332 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.332 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.332 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:02.332 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.540164+0000", 2024-09-18T10:15:02.332 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:02.332 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.332 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.332 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:02.332 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.540164+0000", 2024-09-18T10:15:02.332 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:15:02.332 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.333 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.333 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:02.333 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.540240+0000", 2024-09-18T10:15:02.333 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6821000000000002e-05 2024-09-18T10:15:02.333 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.333 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.333 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:02.333 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.540256+0000", 2024-09-18T10:15:02.333 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5906e-05 2024-09-18T10:15:02.333 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.333 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.333 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:02.333 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.540263+0000", 2024-09-18T10:15:02.333 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2829999999999998e-06 2024-09-18T10:15:02.334 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.334 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.334 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:02.334 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.839993+0000", 2024-09-18T10:15:02.334 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.29973060800000001 2024-09-18T10:15:02.334 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.334 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.334 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:02.334 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.840066+0000", 2024-09-18T10:15:02.334 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3233999999999997e-05 2024-09-18T10:15:02.334 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:02.334 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:02.334 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:02.334 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.335 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.335 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:181 3.cs0 3:3a4fded2:::smithi18925896-23:head [append 0~524288] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:02.335 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:52.691563+0000", 2024-09-18T10:15:02.335 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.5904737660000006, 2024-09-18T10:15:02.335 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.418704519, 2024-09-18T10:15:02.335 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:02.335 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:02.335 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:02.335 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:02.335 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:02.335 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 181 2024-09-18T10:15:02.335 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.335 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:02.335 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.336 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:02.336 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.691563+0000", 2024-09-18T10:15:02.336 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:02.336 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.336 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.336 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:02.336 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.691563+0000", 2024-09-18T10:15:02.336 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:02.336 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.336 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.336 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:02.336 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.691560+0000", 2024-09-18T10:15:02.336 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999971 2024-09-18T10:15:02.337 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.337 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.337 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:02.337 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.692740+0000", 2024-09-18T10:15:02.337 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001180195 2024-09-18T10:15:02.337 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.337 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.337 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:02.337 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.692779+0000", 2024-09-18T10:15:02.337 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8535000000000003e-05 2024-09-18T10:15:02.337 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.337 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.337 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:02.337 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.692794+0000", 2024-09-18T10:15:02.338 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4807000000000001e-05 2024-09-18T10:15:02.338 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.338 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.338 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:02.338 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.793038+0000", 2024-09-18T10:15:02.338 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100244729 2024-09-18T10:15:02.338 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.338 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.338 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-09-18T10:15:02.338 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.793050+0000", 2024-09-18T10:15:02.338 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1776e-05 2024-09-18T10:15:02.338 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.338 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.338 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:02.339 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.893883+0000", 2024-09-18T10:15:02.339 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100833383 2024-09-18T10:15:02.339 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.339 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.339 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:02.339 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.894056+0000", 2024-09-18T10:15:02.339 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017296999999999999 2024-09-18T10:15:02.339 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.339 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.339 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:02.339 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.896702+0000", 2024-09-18T10:15:02.339 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002645728 2024-09-18T10:15:02.339 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.339 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.340 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:02.340 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.001443+0000", 2024-09-18T10:15:02.340 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.104740824 2024-09-18T10:15:02.340 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.340 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.340 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:15:02.340 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.110173+0000", 2024-09-18T10:15:02.340 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.108729645 2024-09-18T10:15:02.340 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.340 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.340 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:02.340 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.110268+0000", 2024-09-18T10:15:02.340 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4987000000000004e-05 2024-09-18T10:15:02.340 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:02.341 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:02.341 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:02.341 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.341 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.341 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:182 3.cs0 3:3a4fded2:::smithi18925896-23:head [append 0~376832] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:02.341 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:52.693795+0000", 2024-09-18T10:15:02.341 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.5882420380000006, 2024-09-18T10:15:02.341 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.51656788200000003, 2024-09-18T10:15:02.341 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:02.341 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:02.341 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:02.341 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:02.341 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:02.342 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 182 2024-09-18T10:15:02.342 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.342 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:02.342 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.342 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:02.342 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.693795+0000", 2024-09-18T10:15:02.342 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:02.342 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.342 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.342 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:02.342 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.693795+0000", 2024-09-18T10:15:02.342 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:02.342 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.342 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.343 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:02.343 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.693787+0000", 2024-09-18T10:15:02.343 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999919 2024-09-18T10:15:02.343 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.343 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.343 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:02.343 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694491+0000", 2024-09-18T10:15:02.343 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00070477199999999999 2024-09-18T10:15:02.343 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.343 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.343 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:02.343 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694526+0000", 2024-09-18T10:15:02.343 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4304999999999998e-05 2024-09-18T10:15:02.343 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.343 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.344 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:02.344 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694539+0000", 2024-09-18T10:15:02.344 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3107e-05 2024-09-18T10:15:02.344 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.344 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.344 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:02.344 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.793145+0000", 2024-09-18T10:15:02.344 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.098605719999999994 2024-09-18T10:15:02.344 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.344 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.344 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:02.344 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.793156+0000", 2024-09-18T10:15:02.344 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1892000000000001e-05 2024-09-18T10:15:02.344 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.345 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.345 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:02.345 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.898472+0000", 2024-09-18T10:15:02.345 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10531558000000001 2024-09-18T10:15:02.345 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.345 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.345 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:02.345 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.898643+0000", 2024-09-18T10:15:02.345 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017116399999999999 2024-09-18T10:15:02.345 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.345 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.345 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:02.345 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.900074+0000", 2024-09-18T10:15:02.346 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0014304299999999999 2024-09-18T10:15:02.346 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.346 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.346 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:02.346 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.001446+0000", 2024-09-18T10:15:02.346 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.101371934 2024-09-18T10:15:02.346 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.346 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.346 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:15:02.346 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.210292+0000", 2024-09-18T10:15:02.346 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.208846693 2024-09-18T10:15:02.346 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.346 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.346 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:02.347 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.210363+0000", 2024-09-18T10:15:02.347 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0470999999999997e-05 2024-09-18T10:15:02.347 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:02.347 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:02.347 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:02.347 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.347 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.347 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:183 3.cs0 3:3a4fded2:::smithi18925896-23:head [setxattr _header (56) in=7b] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:02.347 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:52.694557+0000", 2024-09-18T10:15:02.347 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.5874794780000006, 2024-09-18T10:15:02.347 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.51641106000000003, 2024-09-18T10:15:02.347 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:02.347 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:02.348 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:02.348 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:02.348 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:02.348 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 183 2024-09-18T10:15:02.348 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.348 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:02.348 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.348 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:02.348 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694557+0000", 2024-09-18T10:15:02.348 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:02.348 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.348 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.348 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:02.349 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694557+0000", 2024-09-18T10:15:02.349 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:02.349 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.349 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.349 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:02.349 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694556+0000", 2024-09-18T10:15:02.349 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:02.349 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.349 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.349 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:02.349 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694570+0000", 2024-09-18T10:15:02.349 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4249e-05 2024-09-18T10:15:02.349 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.349 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.350 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:02.350 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694580+0000", 2024-09-18T10:15:02.350 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7459999999999995e-06 2024-09-18T10:15:02.350 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.350 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.350 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:02.350 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694601+0000", 2024-09-18T10:15:02.350 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1583e-05 2024-09-18T10:15:02.350 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.350 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.350 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:02.350 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.793251+0000", 2024-09-18T10:15:02.350 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.098649465000000006 2024-09-18T10:15:02.350 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.351 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.351 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:02.351 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.793262+0000", 2024-09-18T10:15:02.351 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1113e-05 2024-09-18T10:15:02.351 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.351 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.351 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:02.351 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.001995+0000", 2024-09-18T10:15:02.351 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20873336200000001 2024-09-18T10:15:02.351 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.351 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.351 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:02.351 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.002285+0000", 2024-09-18T10:15:02.352 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00028924300000000001 2024-09-18T10:15:02.352 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.352 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.352 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:02.352 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.003993+0000", 2024-09-18T10:15:02.352 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0017085410000000001 2024-09-18T10:15:02.352 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.352 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.352 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:02.352 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.109904+0000", 2024-09-18T10:15:02.352 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.105911038 2024-09-18T10:15:02.352 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.353 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.353 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:15:02.353 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.210762+0000", 2024-09-18T10:15:02.353 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100858146 2024-09-18T10:15:02.353 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.353 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.353 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:02.353 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.210968+0000", 2024-09-18T10:15:02.353 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00020613000000000001 2024-09-18T10:15:02.353 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:02.353 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:02.353 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:02.353 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.354 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.354 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:184 3.cs0 3:3a4fded2:::smithi18925896-23:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:02.354 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:52.694648+0000", 2024-09-18T10:15:02.354 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.5873892539999996, 2024-09-18T10:15:02.354 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.72409976600000003, 2024-09-18T10:15:02.354 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:02.354 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-18T10:15:02.354 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:02.354 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:02.354 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:02.354 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 184 2024-09-18T10:15:02.354 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.354 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:02.355 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.355 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:02.355 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694648+0000", 2024-09-18T10:15:02.355 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:02.355 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.355 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.355 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:02.355 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694648+0000", 2024-09-18T10:15:02.355 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:02.355 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.355 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.355 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:02.355 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694647+0000", 2024-09-18T10:15:02.356 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:15:02.356 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.356 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.356 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:02.356 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694652+0000", 2024-09-18T10:15:02.356 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1170000000000003e-06 2024-09-18T10:15:02.356 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.356 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.356 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:02.356 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694668+0000", 2024-09-18T10:15:02.356 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6019e-05 2024-09-18T10:15:02.356 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.356 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.356 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:02.357 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694674+0000", 2024-09-18T10:15:02.357 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7000000000000002e-06 2024-09-18T10:15:02.357 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.357 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.357 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:02.357 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.893402+0000", 2024-09-18T10:15:02.357 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.19872805700000001 2024-09-18T10:15:02.357 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.357 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.357 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:02.357 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.893413+0000", 2024-09-18T10:15:02.357 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0329999999999999e-05 2024-09-18T10:15:02.357 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.357 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.358 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:02.358 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.004678+0000", 2024-09-18T10:15:02.358 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.111265369 2024-09-18T10:15:02.358 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.358 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.358 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2024-09-18T10:15:02.358 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.004893+0000", 2024-09-18T10:15:02.358 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00021510499999999999 2024-09-18T10:15:02.358 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.358 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.358 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:02.358 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.211096+0000", 2024-09-18T10:15:02.358 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.206202899 2024-09-18T10:15:02.358 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.359 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.359 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:02.359 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.211230+0000", 2024-09-18T10:15:02.359 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000133907 2024-09-18T10:15:02.359 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.359 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.359 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:02.359 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.418747+0000", 2024-09-18T10:15:02.359 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20751733 2024-09-18T10:15:02.359 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:02.359 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:02.359 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:02.359 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.359 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.359 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:196 3.cs0 3:351da1bf:::smithi18925896-6:head [append 0~0] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:02.360 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:52.694697+0000", 2024-09-18T10:15:02.360 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.5873401559999998, 2024-09-18T10:15:02.360 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.61758450099999995, 2024-09-18T10:15:02.360 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:02.360 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:02.360 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:02.360 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:02.360 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:02.360 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 196 2024-09-18T10:15:02.360 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.360 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:02.360 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.360 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:02.361 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694697+0000", 2024-09-18T10:15:02.361 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:02.361 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.361 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.361 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:02.361 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694697+0000", 2024-09-18T10:15:02.361 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:02.361 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.361 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.361 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:02.361 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694696+0000", 2024-09-18T10:15:02.361 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:15:02.361 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.361 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.362 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:02.362 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694705+0000", 2024-09-18T10:15:02.362 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7799999999999995e-06 2024-09-18T10:15:02.362 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.362 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.362 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:02.362 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694721+0000", 2024-09-18T10:15:02.362 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5102e-05 2024-09-18T10:15:02.362 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.362 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.362 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:02.362 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694727+0000", 2024-09-18T10:15:02.362 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2570000000000004e-06 2024-09-18T10:15:02.362 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.363 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.363 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:02.363 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.893457+0000", 2024-09-18T10:15:02.363 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.198729824 2024-09-18T10:15:02.363 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.363 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.363 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:02.363 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.893464+0000", 2024-09-18T10:15:02.363 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6860000000000006e-06 2024-09-18T10:15:02.363 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.363 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.363 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:02.363 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.105086+0000", 2024-09-18T10:15:02.363 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.21162193000000001 2024-09-18T10:15:02.364 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.364 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.364 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:02.364 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.105357+0000", 2024-09-18T10:15:02.364 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00027109699999999998 2024-09-18T10:15:02.364 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.364 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.364 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:02.364 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.107161+0000", 2024-09-18T10:15:02.364 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001803727 2024-09-18T10:15:02.364 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.364 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.364 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:02.365 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.109961+0000", 2024-09-18T10:15:02.365 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0028003080000000001 2024-09-18T10:15:02.365 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.365 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.365 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:15:02.365 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.312174+0000", 2024-09-18T10:15:02.365 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20221219700000001 2024-09-18T10:15:02.365 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.365 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.365 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:02.365 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.312281+0000", 2024-09-18T10:15:02.365 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000107656 2024-09-18T10:15:02.365 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:02.366 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:02.366 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:02.366 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.366 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.366 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:197 3.cs0 3:351da1bf:::smithi18925896-6:head [rollback 2] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:02.366 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:52.694734+0000", 2024-09-18T10:15:02.366 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.5873031140000009, 2024-09-18T10:15:02.366 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.61830483899999999, 2024-09-18T10:15:02.366 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:02.366 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:02.366 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:02.366 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:02.366 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:02.366 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 197 2024-09-18T10:15:02.367 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.367 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:02.367 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.367 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:02.367 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694734+0000", 2024-09-18T10:15:02.367 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:02.367 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.367 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.367 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:02.367 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694734+0000", 2024-09-18T10:15:02.367 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:02.367 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.367 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.367 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:02.368 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694733+0000", 2024-09-18T10:15:02.368 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:15:02.368 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.368 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.368 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:02.368 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694737+0000", 2024-09-18T10:15:02.368 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.636e-06 2024-09-18T10:15:02.368 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.368 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.368 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:02.368 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694752+0000", 2024-09-18T10:15:02.368 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5792999999999999e-05 2024-09-18T10:15:02.368 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.369 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.369 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:02.369 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694758+0000", 2024-09-18T10:15:02.369 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2660000000000001e-06 2024-09-18T10:15:02.369 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.369 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.369 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:02.369 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.108286+0000", 2024-09-18T10:15:02.369 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.41352792700000002 2024-09-18T10:15:02.369 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.369 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.369 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:02.369 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.108540+0000", 2024-09-18T10:15:02.369 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000253928 2024-09-18T10:15:02.370 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.370 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.370 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:02.370 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.109284+0000", 2024-09-18T10:15:02.370 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00074488600000000005 2024-09-18T10:15:02.370 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.370 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.370 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:02.370 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.210174+0000", 2024-09-18T10:15:02.370 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10088973699999999 2024-09-18T10:15:02.370 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.370 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.371 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:15:02.371 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.312857+0000", 2024-09-18T10:15:02.371 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10268295600000001 2024-09-18T10:15:02.371 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.371 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.371 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:02.371 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.313039+0000", 2024-09-18T10:15:02.371 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000181476 2024-09-18T10:15:02.371 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:02.371 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:02.371 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:02.371 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.371 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.371 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWriteReply(3.cs0 37/32 ECSubWriteReply(tid=32, last_complete=37'17, committed=1, applied=1))", 2024-09-18T10:15:02.372 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:53.108966+0000", 2024-09-18T10:15:02.372 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.1730707890000005, 2024-09-18T10:15:02.372 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20265850799999999, 2024-09-18T10:15:02.372 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:02.372 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:02.372 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:02.372 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.372 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:02.372 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.108966+0000", 2024-09-18T10:15:02.372 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:02.372 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.372 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.372 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:02.372 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.108966+0000", 2024-09-18T10:15:02.373 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:02.373 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.373 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.373 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:02.373 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.108965+0000", 2024-09-18T10:15:02.373 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:15:02.373 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.373 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.373 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:02.373 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.108972+0000", 2024-09-18T10:15:02.373 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2960000000000004e-06 2024-09-18T10:15:02.373 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.373 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.374 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:02.374 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.108984+0000", 2024-09-18T10:15:02.374 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.279e-05 2024-09-18T10:15:02.374 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.374 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.374 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:02.374 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.108991+0000", 2024-09-18T10:15:02.374 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1489999999999997e-06 2024-09-18T10:15:02.374 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.374 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.374 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:02.374 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.311600+0000", 2024-09-18T10:15:02.374 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.202608709 2024-09-18T10:15:02.375 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.375 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.375 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:02.375 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.311625+0000", 2024-09-18T10:15:02.375 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4430000000000002e-05 2024-09-18T10:15:02.375 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:02.375 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:02.375 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:02.375 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.375 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.375 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWriteReply(3.cs0 37/32 ECSubWriteReply(tid=32, last_complete=37'17, committed=1, applied=1))", 2024-09-18T10:15:02.375 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:53.110130+0000", 2024-09-18T10:15:02.375 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.1719069389999994, 2024-09-18T10:15:02.375 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20160421000000001, 2024-09-18T10:15:02.376 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:02.376 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:02.376 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:02.376 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.376 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:02.376 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.110130+0000", 2024-09-18T10:15:02.376 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:02.376 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.376 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.376 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:02.376 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.110130+0000", 2024-09-18T10:15:02.376 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:02.376 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.377 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.377 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:02.377 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.110129+0000", 2024-09-18T10:15:02.377 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:15:02.377 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.377 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.377 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:02.377 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.110139+0000", 2024-09-18T10:15:02.377 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4709999999999996e-06 2024-09-18T10:15:02.377 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.377 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.377 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:02.378 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.110155+0000", 2024-09-18T10:15:02.378 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6087999999999999e-05 2024-09-18T10:15:02.378 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.378 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.378 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:02.378 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.110164+0000", 2024-09-18T10:15:02.378 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7089999999999997e-06 2024-09-18T10:15:02.378 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.378 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.378 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:02.378 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.311704+0000", 2024-09-18T10:15:02.378 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20153982500000001 2024-09-18T10:15:02.378 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.378 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.379 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:02.379 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.311734+0000", 2024-09-18T10:15:02.379 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9912000000000001e-05 2024-09-18T10:15:02.379 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:02.379 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:02.379 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:02.379 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.379 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.379 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.as3 41/32 ECSubWrite(tid=42, reqid=osd.13.0:38, at_version=0'0, trim_to=0'0, roll_forward_to=41'12))", 2024-09-18T10:15:02.379 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:57.233115+0000", 2024-09-18T10:15:02.379 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.0489219670000001, 2024-09-18T10:15:02.379 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20438642400000001, 2024-09-18T10:15:02.379 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:02.380 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:02.380 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:02.380 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.380 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:02.380 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.233115+0000", 2024-09-18T10:15:02.380 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:02.380 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.380 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.380 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:02.380 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.233115+0000", 2024-09-18T10:15:02.380 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:02.380 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.380 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.380 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:02.381 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.233113+0000", 2024-09-18T10:15:02.381 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:02.381 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.381 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.381 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:02.381 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.233141+0000", 2024-09-18T10:15:02.381 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7945000000000001e-05 2024-09-18T10:15:02.381 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.381 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.381 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:02.381 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.233176+0000", 2024-09-18T10:15:02.381 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4975999999999999e-05 2024-09-18T10:15:02.381 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.381 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.382 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:02.382 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.333284+0000", 2024-09-18T10:15:02.382 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100107917 2024-09-18T10:15:02.382 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.382 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.382 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:02.382 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.336263+0000", 2024-09-18T10:15:02.382 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0029789059999999999 2024-09-18T10:15:02.382 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.382 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.382 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:02.382 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.336286+0000", 2024-09-18T10:15:02.382 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2926999999999999e-05 2024-09-18T10:15:02.382 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.383 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.383 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:02.383 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.437437+0000", 2024-09-18T10:15:02.383 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10115152600000001 2024-09-18T10:15:02.383 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.383 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.383 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:02.383 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.437501+0000", 2024-09-18T10:15:02.383 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4097999999999995e-05 2024-09-18T10:15:02.383 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:02.383 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:02.383 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:02.383 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.384 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.384 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWriteReply(3.cs0 45/32 ECSubWriteReply(tid=56, last_complete=45'33, committed=1, applied=1))", 2024-09-18T10:15:02.384 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:01.594561+0000", 2024-09-18T10:15:02.384 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.68747543600000005, 2024-09-18T10:15:02.384 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20171572400000001, 2024-09-18T10:15:02.384 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:02.384 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:02.384 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:02.384 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.384 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:02.384 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:01.594561+0000", 2024-09-18T10:15:02.384 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:02.384 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.384 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.385 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:02.385 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:01.594561+0000", 2024-09-18T10:15:02.385 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:02.385 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.385 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.385 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:02.385 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:01.594559+0000", 2024-09-18T10:15:02.385 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:02.385 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.385 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.385 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:02.385 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:01.594574+0000", 2024-09-18T10:15:02.385 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4701999999999999e-05 2024-09-18T10:15:02.386 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.386 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.386 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:02.386 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:01.594610+0000", 2024-09-18T10:15:02.386 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6276000000000003e-05 2024-09-18T10:15:02.386 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.386 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.386 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:02.386 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:01.694744+0000", 2024-09-18T10:15:02.386 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100133343 2024-09-18T10:15:02.386 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.386 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.386 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:02.386 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:01.795016+0000", 2024-09-18T10:15:02.386 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10027169900000001 2024-09-18T10:15:02.387 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.387 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.387 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:02.387 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:01.796277+0000", 2024-09-18T10:15:02.387 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0012616610000000001 2024-09-18T10:15:02.387 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:02.387 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:02.387 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:02.387 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:02.387 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:02.387 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-18T10:15:02.389 INFO:tasks.rados.rados.0.smithi189.stdout:114: finishing write tid 4 to smithi18925896-38 2024-09-18T10:15:02.389 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 38 v 35 (ObjNum 60 snap 5 seq_num 60) dirty exists 2024-09-18T10:15:02.389 INFO:tasks.rados.rados.0.smithi189.stdout:114: left oid 38 (ObjNum 60 snap 5 seq_num 60) 2024-09-18T10:15:02.389 INFO:tasks.rados.rados.0.smithi189.stdout:117: finishing write tid 4 to smithi18925896-48 2024-09-18T10:15:02.389 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 48 v 31 (ObjNum 62 snap 5 seq_num 62) dirty exists 2024-09-18T10:15:02.389 INFO:tasks.rados.rados.0.smithi189.stdout:117: left oid 48 (ObjNum 62 snap 5 seq_num 62) 2024-09-18T10:15:02.389 INFO:tasks.ceph.osd.15.smithi189.stderr:2024-09-18T10:15:02.387+0000 7fd0e725e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 22401) UID: 0 2024-09-18T10:15:02.390 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-09-18T10:15:02.490 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-18T10:15:02.489+0000 7f14c6a1a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 22138) UID: 0 2024-09-18T10:15:02.494 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-18T10:15:02.495 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-18T10:15:02.495 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-09-18T10:15:02.495 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-18T10:15:02.504 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-09-18T10:15:02.590 INFO:tasks.ceph.osd.4.smithi029.stderr:2024-09-18T10:15:02.586+0000 7f550cf6e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22189) UID: 0 2024-09-18T10:15:02.648 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-18T10:15:02.648 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-18T10:15:02.648 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-18T10:15:02.648 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-18T10:15:02.648 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-18T10:15:02.657 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-09-18T10:15:02.692 INFO:tasks.ceph.osd.12.smithi029.stderr:2024-09-18T10:15:02.686+0000 7f8e4e627700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 22190) UID: 0 2024-09-18T10:15:02.762 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-18T10:15:02.763 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-18T10:15:02.763 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-18T10:15:02.763 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-18T10:15:02.763 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.763 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.6s5 36/32 ECSubWrite(tid=19, reqid=client.4632.0:167, at_version=36'11, trim_to=0'0, roll_forward_to=36'10))", 2024-09-18T10:15:02.763 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:51.840599+0000", 2024-09-18T10:15:02.763 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.919287794000001, 2024-09-18T10:15:02.763 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.102418184, 2024-09-18T10:15:02.763 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:02.763 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:02.763 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:02.764 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.764 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:02.764 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.840599+0000", 2024-09-18T10:15:02.764 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:02.764 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.764 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.764 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:02.764 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.840599+0000", 2024-09-18T10:15:02.764 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:02.764 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.764 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.764 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:02.764 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.840592+0000", 2024-09-18T10:15:02.765 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999928 2024-09-18T10:15:02.765 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.765 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.765 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:02.765 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.840784+0000", 2024-09-18T10:15:02.765 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019234300000000001 2024-09-18T10:15:02.765 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.765 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.765 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:02.765 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.840824+0000", 2024-09-18T10:15:02.765 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9654999999999999e-05 2024-09-18T10:15:02.765 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.765 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.765 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:02.766 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.940946+0000", 2024-09-18T10:15:02.766 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10012244300000001 2024-09-18T10:15:02.766 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.766 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.766 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:02.766 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.941101+0000", 2024-09-18T10:15:02.766 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015468200000000001 2024-09-18T10:15:02.767 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.768 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.768 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:02.768 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.941135+0000", 2024-09-18T10:15:02.768 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4020999999999998e-05 2024-09-18T10:15:02.768 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.768 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.768 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:02.768 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.942947+0000", 2024-09-18T10:15:02.768 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001812218 2024-09-18T10:15:02.768 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.768 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.768 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:02.768 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.943017+0000", 2024-09-18T10:15:02.769 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9805e-05 2024-09-18T10:15:02.769 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:02.769 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:02.769 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:02.769 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.769 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.769 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.6s5 36/32 ECSubWrite(tid=20, reqid=client.4632.0:167, at_version=0'0, trim_to=0'0, roll_forward_to=36'11))", 2024-09-18T10:15:02.769 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:51.944453+0000", 2024-09-18T10:15:02.769 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.815433852, 2024-09-18T10:15:02.769 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0014126130000000001, 2024-09-18T10:15:02.769 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:02.769 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:02.769 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:02.769 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.770 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:02.770 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.944453+0000", 2024-09-18T10:15:02.770 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:02.770 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.770 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.770 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:02.770 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.944453+0000", 2024-09-18T10:15:02.770 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:02.770 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.770 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.770 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:02.770 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.944451+0000", 2024-09-18T10:15:02.770 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:02.770 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.771 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.771 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:02.771 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.944476+0000", 2024-09-18T10:15:02.771 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4929000000000001e-05 2024-09-18T10:15:02.771 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.771 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.771 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:02.771 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.944507+0000", 2024-09-18T10:15:02.771 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0442000000000001e-05 2024-09-18T10:15:02.771 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.771 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.771 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:02.771 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.944520+0000", 2024-09-18T10:15:02.771 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3684e-05 2024-09-18T10:15:02.772 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.772 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.772 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:02.772 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.944634+0000", 2024-09-18T10:15:02.772 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000113266 2024-09-18T10:15:02.772 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.772 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.772 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:02.772 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.944663+0000", 2024-09-18T10:15:02.772 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9054999999999999e-05 2024-09-18T10:15:02.772 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.772 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.772 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:02.773 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.945799+0000", 2024-09-18T10:15:02.773 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0011358189999999999 2024-09-18T10:15:02.773 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.773 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.773 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:02.773 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.945865+0000", 2024-09-18T10:15:02.773 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6917999999999999e-05 2024-09-18T10:15:02.773 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:02.773 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:02.773 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:02.773 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.773 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.773 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpRead(3.0s2 37/32 ECSubRead(tid=8, to_read={3:02f544f0:::smithi18925896-30:head=0,122880,0}, subchunks={3:02f544f0:::smithi18925896-30:head=[0,1]}, attrs_to_read=))", 2024-09-18T10:15:02.773 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:52.681732+0000", 2024-09-18T10:15:02.774 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.078155202, 2024-09-18T10:15:02.774 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.101148985, 2024-09-18T10:15:02.774 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:02.774 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:02.774 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:02.774 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.774 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:02.774 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.681732+0000", 2024-09-18T10:15:02.774 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:02.774 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.774 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.774 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:02.774 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.681732+0000", 2024-09-18T10:15:02.774 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:02.775 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.775 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.775 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:02.775 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.681731+0000", 2024-09-18T10:15:02.775 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:02.775 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.775 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.775 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:02.775 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.681750+0000", 2024-09-18T10:15:02.775 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8885e-05 2024-09-18T10:15:02.775 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.775 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.775 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:02.775 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.681794+0000", 2024-09-18T10:15:02.776 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4193000000000002e-05 2024-09-18T10:15:02.776 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.776 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.776 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:02.776 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.681809+0000", 2024-09-18T10:15:02.776 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5228999999999999e-05 2024-09-18T10:15:02.776 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.776 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.776 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:02.776 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.782079+0000", 2024-09-18T10:15:02.776 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100269752 2024-09-18T10:15:02.776 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.776 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.776 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:02.777 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.782880+0000", 2024-09-18T10:15:02.777 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00080150900000000005 2024-09-18T10:15:02.777 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:02.777 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:02.777 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:02.777 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.777 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.777 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpRead(3.0s2 37/32 ECSubRead(tid=9, to_read={3:02f544f0:::smithi18925896-30:head=0,122880,0}, subchunks={3:02f544f0:::smithi18925896-30:head=[0,1]}, attrs_to_read=))", 2024-09-18T10:15:02.777 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:52.682582+0000", 2024-09-18T10:15:02.777 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.077304522, 2024-09-18T10:15:02.777 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100690751, 2024-09-18T10:15:02.777 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:02.777 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:02.777 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:02.778 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.778 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:02.778 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.682582+0000", 2024-09-18T10:15:02.778 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:02.778 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.778 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.778 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:02.778 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.682582+0000", 2024-09-18T10:15:02.778 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:02.778 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.778 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.778 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:02.778 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.682581+0000", 2024-09-18T10:15:02.779 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:02.779 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.779 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.779 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:02.779 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.682614+0000", 2024-09-18T10:15:02.779 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3305e-05 2024-09-18T10:15:02.779 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.779 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.779 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:02.779 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.682641+0000", 2024-09-18T10:15:02.779 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7353000000000001e-05 2024-09-18T10:15:02.779 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.779 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.779 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:02.780 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.682662+0000", 2024-09-18T10:15:02.780 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0319e-05 2024-09-18T10:15:02.780 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.780 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.780 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:02.780 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.782980+0000", 2024-09-18T10:15:02.780 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100318121 2024-09-18T10:15:02.780 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.780 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.780 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:02.780 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.783273+0000", 2024-09-18T10:15:02.780 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00029314599999999999 2024-09-18T10:15:02.780 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:02.781 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:02.781 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:02.781 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.781 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.781 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.ds4 37/32 ECSubWrite(tid=31, reqid=client.4632.0:176, at_version=37'4, trim_to=0'0, roll_forward_to=35'2))", 2024-09-18T10:15:02.781 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:53.002892+0000", 2024-09-18T10:15:02.781 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.7569943979999998, 2024-09-18T10:15:02.781 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10294343, 2024-09-18T10:15:02.781 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:02.781 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:02.781 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:02.781 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.781 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:02.782 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.002892+0000", 2024-09-18T10:15:02.782 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:02.782 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.782 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.782 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:02.782 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.002892+0000", 2024-09-18T10:15:02.782 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:02.782 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.782 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.782 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:02.782 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.002891+0000", 2024-09-18T10:15:02.782 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:15:02.782 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.782 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.783 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:02.783 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.003053+0000", 2024-09-18T10:15:02.783 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00016189399999999999 2024-09-18T10:15:02.783 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.783 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.783 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:02.783 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.003089+0000", 2024-09-18T10:15:02.783 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5329e-05 2024-09-18T10:15:02.783 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.783 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.783 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:02.783 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.003108+0000", 2024-09-18T10:15:02.783 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8963000000000001e-05 2024-09-18T10:15:02.783 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.784 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.784 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:02.784 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.103401+0000", 2024-09-18T10:15:02.784 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100293436 2024-09-18T10:15:02.784 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.784 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.784 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:02.784 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.103418+0000", 2024-09-18T10:15:02.784 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6722999999999999e-05 2024-09-18T10:15:02.784 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.784 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.784 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:02.784 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.105765+0000", 2024-09-18T10:15:02.785 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0023477279999999999 2024-09-18T10:15:02.785 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.785 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.785 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:02.785 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.105836+0000", 2024-09-18T10:15:02.785 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0246999999999993e-05 2024-09-18T10:15:02.785 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:02.785 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:02.785 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:02.785 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.785 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.785 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.3s1 37/32 ECSubWrite(tid=11, reqid=client.4632.0:189, at_version=37'8, trim_to=0'0, roll_forward_to=36'6))", 2024-09-18T10:15:02.785 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:53.096486+0000", 2024-09-18T10:15:02.785 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.6634004579999999, 2024-09-18T10:15:02.786 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10315053, 2024-09-18T10:15:02.786 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:02.786 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:02.786 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:02.786 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.786 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:02.786 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.096486+0000", 2024-09-18T10:15:02.786 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:02.786 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.786 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.786 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:02.786 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.096486+0000", 2024-09-18T10:15:02.786 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:02.787 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.787 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.787 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:02.787 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.096486+0000", 2024-09-18T10:15:02.787 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:02.787 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.787 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.787 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:02.787 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.096802+0000", 2024-09-18T10:15:02.787 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00031615899999999999 2024-09-18T10:15:02.787 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.787 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.787 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:02.787 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.096841+0000", 2024-09-18T10:15:02.788 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9452000000000001e-05 2024-09-18T10:15:02.788 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.788 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.788 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:02.788 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.096866+0000", 2024-09-18T10:15:02.788 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5069e-05 2024-09-18T10:15:02.788 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.788 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.788 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:02.788 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.197104+0000", 2024-09-18T10:15:02.788 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100237886 2024-09-18T10:15:02.788 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.788 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.789 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:02.789 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.197130+0000", 2024-09-18T10:15:02.789 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5666000000000001e-05 2024-09-18T10:15:02.789 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.789 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.789 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:02.789 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.199585+0000", 2024-09-18T10:15:02.789 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002454777 2024-09-18T10:15:02.789 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.789 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.789 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:02.789 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.199637+0000", 2024-09-18T10:15:02.789 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2221e-05 2024-09-18T10:15:02.789 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:02.790 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:02.790 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:02.790 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.790 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.790 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.ds4 37/32 ECSubWrite(tid=32, reqid=client.4632.0:176, at_version=0'0, trim_to=0'0, roll_forward_to=37'4))", 2024-09-18T10:15:02.790 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:53.107300+0000", 2024-09-18T10:15:02.790 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.6525872019999994, 2024-09-18T10:15:02.790 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.101315192, 2024-09-18T10:15:02.790 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:02.790 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:02.790 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:02.790 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.790 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:02.790 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.107300+0000", 2024-09-18T10:15:02.791 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:02.791 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.791 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.791 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:02.791 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.107300+0000", 2024-09-18T10:15:02.791 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:02.791 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.791 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.791 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:02.791 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.107298+0000", 2024-09-18T10:15:02.791 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:02.791 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.791 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.791 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:02.792 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.107342+0000", 2024-09-18T10:15:02.792 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4542000000000001e-05 2024-09-18T10:15:02.792 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.792 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.792 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:02.792 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.107425+0000", 2024-09-18T10:15:02.792 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2590999999999996e-05 2024-09-18T10:15:02.792 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.792 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.792 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:02.792 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.207527+0000", 2024-09-18T10:15:02.792 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100102406 2024-09-18T10:15:02.792 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.792 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.793 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:02.793 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.207705+0000", 2024-09-18T10:15:02.793 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017770399999999999 2024-09-18T10:15:02.793 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.793 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.793 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:02.793 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.207727+0000", 2024-09-18T10:15:02.793 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1991999999999999e-05 2024-09-18T10:15:02.793 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.793 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.793 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:02.793 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.208570+0000", 2024-09-18T10:15:02.793 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00084287200000000004 2024-09-18T10:15:02.794 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.794 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.794 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:02.794 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.208615+0000", 2024-09-18T10:15:02.794 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5037999999999998e-05 2024-09-18T10:15:02.794 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:02.794 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:02.794 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:02.794 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.794 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.794 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.3s1 37/32 ECSubWrite(tid=13, reqid=client.4632.0:194, at_version=37'10, trim_to=0'0, roll_forward_to=37'8))", 2024-09-18T10:15:02.794 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:53.412495+0000", 2024-09-18T10:15:02.794 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.3473916880000001, 2024-09-18T10:15:02.794 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.102940423, 2024-09-18T10:15:02.795 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:02.795 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:02.795 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:02.795 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.795 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:02.795 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.412495+0000", 2024-09-18T10:15:02.795 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:02.795 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.795 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.795 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:02.795 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.412495+0000", 2024-09-18T10:15:02.795 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:02.795 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.795 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.796 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:02.796 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.412493+0000", 2024-09-18T10:15:02.796 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999976 2024-09-18T10:15:02.796 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.796 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.796 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:02.796 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.412821+0000", 2024-09-18T10:15:02.796 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000327986 2024-09-18T10:15:02.796 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.796 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.796 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:02.796 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.412875+0000", 2024-09-18T10:15:02.796 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4128000000000001e-05 2024-09-18T10:15:02.797 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.797 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.797 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:02.797 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.412895+0000", 2024-09-18T10:15:02.797 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0445000000000002e-05 2024-09-18T10:15:02.797 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.797 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.797 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:02.797 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.513126+0000", 2024-09-18T10:15:02.797 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100231025 2024-09-18T10:15:02.797 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.797 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.797 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:02.797 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.513146+0000", 2024-09-18T10:15:02.798 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9992000000000001e-05 2024-09-18T10:15:02.798 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.798 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.798 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:02.798 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.515351+0000", 2024-09-18T10:15:02.798 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0022045599999999999 2024-09-18T10:15:02.798 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.798 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.798 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:02.798 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.515435+0000", 2024-09-18T10:15:02.798 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4536999999999994e-05 2024-09-18T10:15:02.798 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:02.798 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:02.798 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:02.799 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.799 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.799 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.3s1 37/32 ECSubWrite(tid=14, reqid=client.4632.0:194, at_version=0'0, trim_to=0'0, roll_forward_to=37'10))", 2024-09-18T10:15:02.799 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:53.616382+0000", 2024-09-18T10:15:02.799 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.1435043329999992, 2024-09-18T10:15:02.799 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0013965220000000001, 2024-09-18T10:15:02.799 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:02.799 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:02.799 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:02.799 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.799 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:02.799 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.616382+0000", 2024-09-18T10:15:02.799 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:02.799 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.800 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.800 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:02.800 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.616382+0000", 2024-09-18T10:15:02.800 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:02.800 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.800 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.800 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:02.800 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.616380+0000", 2024-09-18T10:15:02.800 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:02.800 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.800 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.800 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:02.800 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.616411+0000", 2024-09-18T10:15:02.801 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0375e-05 2024-09-18T10:15:02.801 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.801 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.801 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:02.801 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.616453+0000", 2024-09-18T10:15:02.801 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2692999999999999e-05 2024-09-18T10:15:02.801 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.801 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.801 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:02.801 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.616475+0000", 2024-09-18T10:15:02.801 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1169e-05 2024-09-18T10:15:02.801 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.801 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.801 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:02.802 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.616651+0000", 2024-09-18T10:15:02.802 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017641999999999999 2024-09-18T10:15:02.802 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.802 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.802 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:02.802 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.616675+0000", 2024-09-18T10:15:02.802 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4102999999999999e-05 2024-09-18T10:15:02.802 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.802 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.802 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:02.802 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.617695+0000", 2024-09-18T10:15:02.802 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010199670000000001 2024-09-18T10:15:02.802 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.802 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.803 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:02.803 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.617779+0000", 2024-09-18T10:15:02.803 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3877999999999999e-05 2024-09-18T10:15:02.803 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:02.803 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:02.803 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:02.803 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.803 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.803 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.ds4 41/32 ECSubWrite(tid=41, reqid=osd.13.0:41, at_version=41'6, trim_to=0'0, roll_forward_to=37'4))", 2024-09-18T10:15:02.803 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:57.138491+0000", 2024-09-18T10:15:02.803 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.6213958850000001, 2024-09-18T10:15:02.803 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.200128009, 2024-09-18T10:15:02.803 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:02.803 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:02.804 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:02.804 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.804 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:02.804 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.138491+0000", 2024-09-18T10:15:02.804 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:02.804 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.804 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.804 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:02.804 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.138491+0000", 2024-09-18T10:15:02.804 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:02.804 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.804 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.804 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:02.805 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.138490+0000", 2024-09-18T10:15:02.805 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:15:02.805 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.805 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.805 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:02.805 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.138555+0000", 2024-09-18T10:15:02.805 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4571000000000002e-05 2024-09-18T10:15:02.805 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.805 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.805 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:02.805 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.138581+0000", 2024-09-18T10:15:02.805 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6069000000000001e-05 2024-09-18T10:15:02.805 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.805 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.805 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:02.806 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.138598+0000", 2024-09-18T10:15:02.806 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7059e-05 2024-09-18T10:15:02.806 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.806 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.806 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:02.806 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.336697+0000", 2024-09-18T10:15:02.806 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.198099517 2024-09-18T10:15:02.806 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.806 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.806 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:02.806 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.336733+0000", 2024-09-18T10:15:02.806 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5369e-05 2024-09-18T10:15:02.806 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.806 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.807 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:02.807 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.338552+0000", 2024-09-18T10:15:02.807 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001819773 2024-09-18T10:15:02.807 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.807 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.807 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:02.807 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.338619+0000", 2024-09-18T10:15:02.807 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6390999999999995e-05 2024-09-18T10:15:02.807 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:02.807 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:02.807 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:02.807 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.807 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.807 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.ds4 41/32 ECSubWrite(tid=43, reqid=osd.13.0:41, at_version=0'0, trim_to=0'0, roll_forward_to=41'6))", 2024-09-18T10:15:02.808 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:57.441404+0000", 2024-09-18T10:15:02.808 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.3184822760000001, 2024-09-18T10:15:02.808 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30108198600000002, 2024-09-18T10:15:02.808 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:02.808 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:02.808 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:02.808 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.808 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:02.808 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.441404+0000", 2024-09-18T10:15:02.808 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:02.808 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.808 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.808 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:02.809 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.441404+0000", 2024-09-18T10:15:02.809 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:02.809 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.809 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.809 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:02.809 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.441403+0000", 2024-09-18T10:15:02.809 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:02.809 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.809 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.809 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:02.809 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.441435+0000", 2024-09-18T10:15:02.809 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2502000000000002e-05 2024-09-18T10:15:02.809 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.809 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.810 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:02.810 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.441473+0000", 2024-09-18T10:15:02.810 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8302e-05 2024-09-18T10:15:02.810 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.810 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.810 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:02.810 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.441524+0000", 2024-09-18T10:15:02.810 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0679e-05 2024-09-18T10:15:02.810 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.810 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.810 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:02.810 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.641739+0000", 2024-09-18T10:15:02.810 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20021524299999999 2024-09-18T10:15:02.810 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.811 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.811 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:02.811 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.641766+0000", 2024-09-18T10:15:02.811 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7081999999999999e-05 2024-09-18T10:15:02.811 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.811 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.811 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:02.811 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.742432+0000", 2024-09-18T10:15:02.811 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10066531300000001 2024-09-18T10:15:02.811 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.811 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.811 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:02.811 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.742486+0000", 2024-09-18T10:15:02.812 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4691999999999997e-05 2024-09-18T10:15:02.812 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:02.812 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:02.812 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:02.812 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.812 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.812 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.6s5 41/40 ECSubWrite(tid=31, reqid=client.4632.0:249, at_version=41'13, trim_to=0'0, roll_forward_to=0'0))", 2024-09-18T10:15:02.812 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:57.550603+0000", 2024-09-18T10:15:02.812 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.2092840860000003, 2024-09-18T10:15:02.812 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.103590501, 2024-09-18T10:15:02.812 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:02.812 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:02.812 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:02.812 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.813 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:02.813 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.550603+0000", 2024-09-18T10:15:02.813 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:02.813 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.813 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.813 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:02.813 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.550603+0000", 2024-09-18T10:15:02.813 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:02.813 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.813 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.813 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:02.813 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.550602+0000", 2024-09-18T10:15:02.813 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:15:02.814 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.814 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.814 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:02.814 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.551120+0000", 2024-09-18T10:15:02.814 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000518867 2024-09-18T10:15:02.814 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.814 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.814 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:02.814 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.551141+0000", 2024-09-18T10:15:02.814 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0873000000000002e-05 2024-09-18T10:15:02.814 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.814 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.814 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:02.815 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.551150+0000", 2024-09-18T10:15:02.815 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.884e-06 2024-09-18T10:15:02.815 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.815 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.815 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:02.815 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.651352+0000", 2024-09-18T10:15:02.815 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100201533 2024-09-18T10:15:02.815 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.815 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.815 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:02.815 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.651384+0000", 2024-09-18T10:15:02.815 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2471999999999997e-05 2024-09-18T10:15:02.815 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.816 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.816 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:02.816 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.654157+0000", 2024-09-18T10:15:02.816 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0027727440000000002 2024-09-18T10:15:02.816 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.816 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.816 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:02.816 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.654193+0000", 2024-09-18T10:15:02.816 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6242000000000003e-05 2024-09-18T10:15:02.816 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:02.816 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:02.816 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:02.816 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.817 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.817 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.6s5 41/40 ECSubWrite(tid=32, reqid=client.4632.0:250, at_version=41'14, trim_to=0'0, roll_forward_to=0'0))", 2024-09-18T10:15:02.817 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:57.552846+0000", 2024-09-18T10:15:02.817 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.2070410909999998, 2024-09-18T10:15:02.817 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10139358900000001, 2024-09-18T10:15:02.817 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:02.817 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:02.817 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:02.817 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.817 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:02.817 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.552846+0000", 2024-09-18T10:15:02.817 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:02.817 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.817 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.818 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:02.818 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.552846+0000", 2024-09-18T10:15:02.818 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:02.818 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.818 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.818 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:02.818 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.552845+0000", 2024-09-18T10:15:02.818 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:15:02.818 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.818 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.818 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:02.818 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.553026+0000", 2024-09-18T10:15:02.818 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00018096599999999999 2024-09-18T10:15:02.819 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.819 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.819 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:02.819 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.553043+0000", 2024-09-18T10:15:02.819 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6818000000000001e-05 2024-09-18T10:15:02.819 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.819 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.819 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:02.819 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.553054+0000", 2024-09-18T10:15:02.819 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1099e-05 2024-09-18T10:15:02.819 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.819 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.819 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:02.820 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.652809+0000", 2024-09-18T10:15:02.820 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.099754887 2024-09-18T10:15:02.820 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.820 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.820 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:02.820 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.652836+0000", 2024-09-18T10:15:02.820 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7582000000000001e-05 2024-09-18T10:15:02.820 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.820 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.820 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:02.820 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.654225+0000", 2024-09-18T10:15:02.820 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0013888170000000001 2024-09-18T10:15:02.820 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.820 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.821 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:02.821 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.654239+0000", 2024-09-18T10:15:02.821 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4260000000000001e-05 2024-09-18T10:15:02.821 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:02.821 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:02.821 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:02.821 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.821 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.821 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.6s5 41/40 ECSubWrite(tid=33, reqid=client.4632.0:251, at_version=41'15, trim_to=0'0, roll_forward_to=0'0))", 2024-09-18T10:15:02.821 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:57.554251+0000", 2024-09-18T10:15:02.821 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.2056358530000004, 2024-09-18T10:15:02.821 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100858745, 2024-09-18T10:15:02.821 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:02.822 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:02.822 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:02.822 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.822 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:02.822 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.554251+0000", 2024-09-18T10:15:02.822 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:02.822 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.822 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.822 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:02.822 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.554251+0000", 2024-09-18T10:15:02.822 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:02.822 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.822 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.822 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:02.823 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.554249+0000", 2024-09-18T10:15:02.823 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:02.823 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.823 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.823 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:02.823 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.554288+0000", 2024-09-18T10:15:02.823 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9147999999999999e-05 2024-09-18T10:15:02.823 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.823 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.823 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:02.823 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.554322+0000", 2024-09-18T10:15:02.823 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3328999999999999e-05 2024-09-18T10:15:02.823 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.824 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.824 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:02.824 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.554337+0000", 2024-09-18T10:15:02.824 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5316e-05 2024-09-18T10:15:02.824 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.824 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.824 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:02.824 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.653990+0000", 2024-09-18T10:15:02.824 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.099652698999999997 2024-09-18T10:15:02.824 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.824 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.824 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:02.824 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.653998+0000", 2024-09-18T10:15:02.825 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1640000000000001e-06 2024-09-18T10:15:02.825 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.825 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.825 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:02.825 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.655081+0000", 2024-09-18T10:15:02.825 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001083297 2024-09-18T10:15:02.825 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.825 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.825 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:02.825 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.655110+0000", 2024-09-18T10:15:02.825 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8365e-05 2024-09-18T10:15:02.825 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:02.825 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:02.825 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:02.826 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.826 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.826 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.13s1 41/40 ECSubWrite(tid=19, reqid=osd.10.0:19, at_version=41'16, trim_to=0'0, roll_forward_to=0'0))", 2024-09-18T10:15:02.826 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:57.750731+0000", 2024-09-18T10:15:02.826 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.0091558650000003, 2024-09-18T10:15:02.826 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0019853919999999999, 2024-09-18T10:15:02.826 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:02.826 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:02.826 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:02.826 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.826 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:02.826 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.750731+0000", 2024-09-18T10:15:02.826 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:02.827 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.827 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.827 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:02.827 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.750731+0000", 2024-09-18T10:15:02.827 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:02.827 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.827 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.827 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:02.827 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.750728+0000", 2024-09-18T10:15:02.827 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999976 2024-09-18T10:15:02.827 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.827 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.827 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:02.827 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.750790+0000", 2024-09-18T10:15:02.828 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1601999999999997e-05 2024-09-18T10:15:02.828 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.828 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.828 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:02.828 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.750838+0000", 2024-09-18T10:15:02.828 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7661999999999999e-05 2024-09-18T10:15:02.828 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.828 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.828 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:02.828 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.750857+0000", 2024-09-18T10:15:02.828 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8912000000000001e-05 2024-09-18T10:15:02.828 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.828 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.828 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:02.829 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.751054+0000", 2024-09-18T10:15:02.829 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000197641 2024-09-18T10:15:02.829 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.829 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.829 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:02.829 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.751088+0000", 2024-09-18T10:15:02.829 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3788000000000002e-05 2024-09-18T10:15:02.829 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.829 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.829 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:02.829 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.752646+0000", 2024-09-18T10:15:02.829 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0015584489999999999 2024-09-18T10:15:02.829 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.829 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.830 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:02.830 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.752716+0000", 2024-09-18T10:15:02.830 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9847999999999999e-05 2024-09-18T10:15:02.830 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:02.830 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:02.830 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:02.830 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.830 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.830 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.13s1 41/40 ECSubWrite(tid=20, reqid=osd.10.0:20, at_version=41'18, trim_to=0'0, roll_forward_to=0'0))", 2024-09-18T10:15:02.830 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:57.752705+0000", 2024-09-18T10:15:02.830 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.0071813660000002, 2024-09-18T10:15:02.830 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0016202930000000001, 2024-09-18T10:15:02.830 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:02.831 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:02.831 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:02.831 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.831 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:02.831 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.752705+0000", 2024-09-18T10:15:02.831 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:02.831 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.831 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.831 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:02.831 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.752705+0000", 2024-09-18T10:15:02.831 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:02.831 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.831 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.831 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:02.832 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.752704+0000", 2024-09-18T10:15:02.832 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:02.832 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.832 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.832 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:02.832 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.752759+0000", 2024-09-18T10:15:02.832 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4845000000000001e-05 2024-09-18T10:15:02.832 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.832 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.832 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:02.832 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.752794+0000", 2024-09-18T10:15:02.832 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5754999999999999e-05 2024-09-18T10:15:02.832 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.833 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.833 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:02.833 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.752807+0000", 2024-09-18T10:15:02.833 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2830000000000001e-05 2024-09-18T10:15:02.833 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.833 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.833 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:02.833 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.752938+0000", 2024-09-18T10:15:02.833 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013076900000000001 2024-09-18T10:15:02.833 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.833 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.833 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:02.833 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.752968+0000", 2024-09-18T10:15:02.833 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0216e-05 2024-09-18T10:15:02.833 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.834 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.834 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:02.834 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.754274+0000", 2024-09-18T10:15:02.834 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001305683 2024-09-18T10:15:02.834 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.834 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.834 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:02.834 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.754326+0000", 2024-09-18T10:15:02.834 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1748e-05 2024-09-18T10:15:02.834 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:02.834 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:02.834 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:02.834 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.834 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.835 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.13s1 42/40 ECSubWrite(tid=21, reqid=osd.10.0:20, at_version=0'0, trim_to=0'0, roll_forward_to=41'18))", 2024-09-18T10:15:02.835 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:58.055945+0000", 2024-09-18T10:15:02.835 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.703942112, 2024-09-18T10:15:02.835 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.101978183, 2024-09-18T10:15:02.835 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:02.835 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:02.835 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:02.835 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.835 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:02.835 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:58.055945+0000", 2024-09-18T10:15:02.835 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:02.835 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.835 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.835 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:02.836 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:58.055945+0000", 2024-09-18T10:15:02.836 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:02.836 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.836 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.836 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:02.836 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:58.055943+0000", 2024-09-18T10:15:02.836 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:02.836 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.836 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.836 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:02.836 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:58.055969+0000", 2024-09-18T10:15:02.836 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6319e-05 2024-09-18T10:15:02.836 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.837 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.837 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:02.837 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:58.056014+0000", 2024-09-18T10:15:02.837 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4866e-05 2024-09-18T10:15:02.837 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.837 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.837 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:02.837 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:58.056031+0000", 2024-09-18T10:15:02.837 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6565000000000001e-05 2024-09-18T10:15:02.837 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.837 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.837 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:02.837 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:58.156261+0000", 2024-09-18T10:15:02.838 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100229871 2024-09-18T10:15:02.838 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.838 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.838 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:02.838 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:58.156276+0000", 2024-09-18T10:15:02.838 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5866e-05 2024-09-18T10:15:02.838 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.838 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.838 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:02.838 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:58.157852+0000", 2024-09-18T10:15:02.838 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0015757880000000001 2024-09-18T10:15:02.838 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.838 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.838 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:02.839 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:58.157923+0000", 2024-09-18T10:15:02.839 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0607999999999999e-05 2024-09-18T10:15:02.839 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:02.839 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:02.839 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:02.839 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.839 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.839 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.10s2 45/40 ECSubWrite(tid=17, reqid=client.4632.0:299, at_version=45'6, trim_to=0'0, roll_forward_to=0'0))", 2024-09-18T10:15:02.839 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:01.537687+0000", 2024-09-18T10:15:02.839 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.222200196, 2024-09-18T10:15:02.839 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0030724150000000002, 2024-09-18T10:15:02.839 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:02.839 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:02.840 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:02.840 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.840 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:02.840 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:01.537687+0000", 2024-09-18T10:15:02.840 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:02.840 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.840 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.840 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:02.840 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:01.537687+0000", 2024-09-18T10:15:02.840 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:02.840 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.840 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.840 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:02.840 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:01.537686+0000", 2024-09-18T10:15:02.841 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:15:02.841 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.841 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.841 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:02.841 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:01.538099+0000", 2024-09-18T10:15:02.841 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00041292400000000001 2024-09-18T10:15:02.841 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.841 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.841 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:02.841 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:01.538131+0000", 2024-09-18T10:15:02.841 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2388999999999998e-05 2024-09-18T10:15:02.841 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.841 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.842 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:02.842 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:01.538145+0000", 2024-09-18T10:15:02.842 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4156e-05 2024-09-18T10:15:02.842 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.842 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.842 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:02.842 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:01.538275+0000", 2024-09-18T10:15:02.842 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012950900000000001 2024-09-18T10:15:02.842 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.842 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.842 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:02.842 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:01.538294+0000", 2024-09-18T10:15:02.842 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.889e-05 2024-09-18T10:15:02.842 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.843 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.843 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:02.843 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:01.540694+0000", 2024-09-18T10:15:02.843 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0024004460000000001 2024-09-18T10:15:02.843 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.843 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.843 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:02.843 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:01.540759+0000", 2024-09-18T10:15:02.843 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4974000000000005e-05 2024-09-18T10:15:02.843 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:02.843 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:02.843 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:02.843 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.843 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.844 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.6s5 46/40 ECSubWrite(tid=46, reqid=client.4632.0:324, at_version=46'27, trim_to=0'0, roll_forward_to=45'25))", 2024-09-18T10:15:02.844 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:01.999968+0000", 2024-09-18T10:15:02.844 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.75991910399999996, 2024-09-18T10:15:02.844 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0020205919999999999, 2024-09-18T10:15:02.844 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:02.844 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:02.844 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:02.844 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.844 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:02.844 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:01.999968+0000", 2024-09-18T10:15:02.844 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:02.844 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.844 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.845 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:02.845 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:01.999968+0000", 2024-09-18T10:15:02.845 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:02.845 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.845 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.845 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:02.845 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:01.999966+0000", 2024-09-18T10:15:02.845 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:02.845 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.845 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.845 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:02.845 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.000000+0000", 2024-09-18T10:15:02.845 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3136000000000003e-05 2024-09-18T10:15:02.846 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.846 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.846 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:02.846 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.000024+0000", 2024-09-18T10:15:02.846 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4978999999999999e-05 2024-09-18T10:15:02.846 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.846 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.846 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:02.846 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.000034+0000", 2024-09-18T10:15:02.846 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8810000000000008e-06 2024-09-18T10:15:02.846 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.846 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.846 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:02.847 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.000178+0000", 2024-09-18T10:15:02.847 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000143776 2024-09-18T10:15:02.847 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.847 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.847 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:02.847 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.000218+0000", 2024-09-18T10:15:02.847 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9969000000000003e-05 2024-09-18T10:15:02.847 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.847 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.847 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:02.847 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.001920+0000", 2024-09-18T10:15:02.847 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001701698 2024-09-18T10:15:02.847 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.847 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.848 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:02.848 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.001988+0000", 2024-09-18T10:15:02.848 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8371e-05 2024-09-18T10:15:02.848 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:02.848 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:02.848 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:02.848 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.848 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.848 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.6s5 46/40 ECSubWrite(tid=47, reqid=client.4632.0:325, at_version=46'28, trim_to=0'0, roll_forward_to=45'25))", 2024-09-18T10:15:02.848 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:02.000059+0000", 2024-09-18T10:15:02.848 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.75982804900000001, 2024-09-18T10:15:02.848 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002312068, 2024-09-18T10:15:02.848 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:02.849 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:02.849 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:02.849 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.849 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:02.849 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.000059+0000", 2024-09-18T10:15:02.849 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:02.849 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.849 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.849 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:02.849 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.000059+0000", 2024-09-18T10:15:02.849 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:02.849 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.849 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.849 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:02.850 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.000058+0000", 2024-09-18T10:15:02.850 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:02.850 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.850 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.850 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:02.850 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.000073+0000", 2024-09-18T10:15:02.850 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5234e-05 2024-09-18T10:15:02.850 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.850 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.850 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:02.850 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.000082+0000", 2024-09-18T10:15:02.850 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8109999999999992e-06 2024-09-18T10:15:02.850 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.851 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.851 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:02.851 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.000088+0000", 2024-09-18T10:15:02.851 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5430000000000001e-06 2024-09-18T10:15:02.851 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.851 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.851 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:02.851 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.001036+0000", 2024-09-18T10:15:02.851 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00094820699999999996 2024-09-18T10:15:02.851 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.851 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.851 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:02.851 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.001061+0000", 2024-09-18T10:15:02.851 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5358999999999999e-05 2024-09-18T10:15:02.851 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.852 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.852 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:02.852 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.002315+0000", 2024-09-18T10:15:02.852 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0012532909999999999 2024-09-18T10:15:02.852 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.852 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.852 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:02.852 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.002371+0000", 2024-09-18T10:15:02.852 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6075000000000001e-05 2024-09-18T10:15:02.852 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:02.852 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:02.852 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:02.852 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:02.853 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:02.853 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-18T10:15:02.855 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_ops_in_flight 2024-09-18T10:15:02.856 INFO:tasks.rados.rados.0.smithi189.stdout:113: done (4 left) 2024-09-18T10:15:02.856 INFO:tasks.rados.rados.0.smithi189.stdout:114: done (3 left) 2024-09-18T10:15:02.856 INFO:tasks.rados.rados.0.smithi189.stdout:117: done (2 left) 2024-09-18T10:15:02.856 INFO:tasks.rados.rados.0.smithi189.stdout:126: done (1 left) 2024-09-18T10:15:02.856 INFO:tasks.rados.rados.0.smithi189.stdout:127: done (0 left) 2024-09-18T10:15:02.856 INFO:tasks.rados.rados.0.smithi189.stdout:128: copy_from oid 25 from oid 6 current snap is 6 2024-09-18T10:15:02.856 INFO:tasks.rados.rados.0.smithi189.stdout:129: setattr oid 23 current snap is 6 2024-09-18T10:15:02.856 INFO:tasks.rados.rados.0.smithi189.stdout:append oid 15 current snap is 6 2024-09-18T10:15:02.857 INFO:tasks.rados.rados.0.smithi189.stdout:130: seq_num 66 ranges {180224=737280,917504=376832} 2024-09-18T10:15:02.857 INFO:tasks.rados.rados.0.smithi189.stdout:130: writing smithi18925896-15 from 180224 to 917504 tid 1 2024-09-18T10:15:02.857 INFO:tasks.rados.rados.0.smithi189.stdout:130: writing smithi18925896-15 from 917504 to 1294336 tid 2 2024-09-18T10:15:02.857 INFO:tasks.rados.rados.0.smithi189.stdout:append oid 20 current snap is 6 2024-09-18T10:15:02.857 INFO:tasks.rados.rados.0.smithi189.stdout:131: seq_num 67 ranges {557056=753664,1310720=376832} 2024-09-18T10:15:02.857 INFO:tasks.rados.rados.0.smithi189.stdout:131: writing smithi18925896-20 from 557056 to 1310720 tid 1 2024-09-18T10:15:02.857 INFO:tasks.rados.rados.0.smithi189.stdout:131: writing smithi18925896-20 from 1310720 to 1687552 tid 2 2024-09-18T10:15:02.857 INFO:tasks.rados.rados.0.smithi189.stdout:132: rollback oid 31 current snap is 6 2024-09-18T10:15:02.857 INFO:tasks.rados.rados.0.smithi189.stdout:rollback oid 31 to 2 2024-09-18T10:15:02.857 INFO:tasks.rados.rados.0.smithi189.stdout:133: read oid 24 snap -1 2024-09-18T10:15:02.857 INFO:tasks.rados.rados.0.smithi189.stdout:133: expect (ObjNum 28 snap 0 seq_num 28) 2024-09-18T10:15:02.857 INFO:tasks.rados.rados.0.smithi189.stdout:append oid 39 current snap is 6 2024-09-18T10:15:02.857 INFO:tasks.rados.rados.0.smithi189.stdout:134: seq_num 68 ranges {1605632=770048,2375680=376832} 2024-09-18T10:15:02.857 INFO:tasks.rados.rados.0.smithi189.stdout:134: writing smithi18925896-39 from 1605632 to 2375680 tid 1 2024-09-18T10:15:02.858 INFO:tasks.rados.rados.0.smithi189.stdout:134: writing smithi18925896-39 from 2375680 to 2752512 tid 2 2024-09-18T10:15:02.858 INFO:tasks.rados.rados.0.smithi189.stdout:135: snap_create 2024-09-18T10:15:02.858 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-18T10:15:02.851+0000 7f14dcc5b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22398) UID: 0 2024-09-18T10:15:02.955 INFO:tasks.ceph.osd.6.smithi154.stderr:2024-09-18T10:15:02.952+0000 7f0abc9e1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22089) UID: 0 2024-09-18T10:15:02.968 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-18T10:15:02.969 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-18T10:15:02.969 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.969 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.cs2 47/32 ECSubWrite(tid=59, reqid=client.4632.0:340, at_version=47'37, trim_to=0'0, roll_forward_to=45'33))", 2024-09-18T10:15:02.969 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:02.910235+0000", 2024-09-18T10:15:02.969 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.056685251999999998, 2024-09-18T10:15:02.969 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.056736302000000002, 2024-09-18T10:15:02.969 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:02.969 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:02.969 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:02.969 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.970 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:02.970 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.910235+0000", 2024-09-18T10:15:02.970 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:02.970 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.970 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.970 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:02.970 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.910235+0000", 2024-09-18T10:15:02.970 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:02.970 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.970 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.970 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:02.970 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.910234+0000", 2024-09-18T10:15:02.970 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:15:02.971 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.971 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.971 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:02.971 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.910263+0000", 2024-09-18T10:15:02.971 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9149000000000002e-05 2024-09-18T10:15:02.971 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.971 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.971 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:02.971 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.910280+0000", 2024-09-18T10:15:02.971 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6441999999999999e-05 2024-09-18T10:15:02.971 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.971 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.971 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:02.971 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.910287+0000", 2024-09-18T10:15:02.972 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8560000000000001e-06 2024-09-18T10:15:02.972 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.972 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.972 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:02.972 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.912209+0000", 2024-09-18T10:15:02.972 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001922223 2024-09-18T10:15:02.972 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.972 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.972 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:02.972 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.912224+0000", 2024-09-18T10:15:02.972 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5393999999999999e-05 2024-09-18T10:15:02.972 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:02.972 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:02.973 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:02.973 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.973 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.973 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.cs2 47/32 ECSubWrite(tid=60, reqid=client.4632.0:341, at_version=47'38, trim_to=0'0, roll_forward_to=45'33))", 2024-09-18T10:15:02.973 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:02.912049+0000", 2024-09-18T10:15:02.973 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.054871418999999998, 2024-09-18T10:15:02.973 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.055014748000000002, 2024-09-18T10:15:02.973 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:02.975 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:02.976 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:02.976 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.976 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:02.976 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.912049+0000", 2024-09-18T10:15:02.976 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:02.976 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.976 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.976 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:02.976 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.912049+0000", 2024-09-18T10:15:02.976 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:02.976 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.976 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.976 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:02.977 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.912048+0000", 2024-09-18T10:15:02.977 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:15:02.977 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.977 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.977 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:02.977 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.912078+0000", 2024-09-18T10:15:02.977 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9815000000000002e-05 2024-09-18T10:15:02.977 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.977 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.977 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:02.977 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.912090+0000", 2024-09-18T10:15:02.977 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2598e-05 2024-09-18T10:15:02.977 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.978 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.978 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:02.978 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.912097+0000", 2024-09-18T10:15:02.978 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5089999999999999e-06 2024-09-18T10:15:02.978 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.978 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.978 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:02.978 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.914129+0000", 2024-09-18T10:15:02.978 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0020318929999999999 2024-09-18T10:15:02.978 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.978 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.978 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:02.978 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.914143+0000", 2024-09-18T10:15:02.978 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4231e-05 2024-09-18T10:15:02.979 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:02.979 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:02.979 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:02.979 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.979 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.979 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.cs2 47/32 ECSubWrite(tid=61, reqid=client.4632.0:342, at_version=47'39, trim_to=0'0, roll_forward_to=45'33))", 2024-09-18T10:15:02.979 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:02.914013+0000", 2024-09-18T10:15:02.979 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.052906893000000003, 2024-09-18T10:15:02.979 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.053133330999999999, 2024-09-18T10:15:02.979 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:02.979 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-18T10:15:02.979 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:02.979 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.979 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:02.980 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.914013+0000", 2024-09-18T10:15:02.980 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:02.980 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.980 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.980 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:02.980 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.914013+0000", 2024-09-18T10:15:02.980 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:02.980 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.980 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.980 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:02.980 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.914012+0000", 2024-09-18T10:15:02.980 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:15:02.980 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.981 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.981 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:02.981 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.914043+0000", 2024-09-18T10:15:02.981 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0539e-05 2024-09-18T10:15:02.981 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.981 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.981 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:02.981 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.914065+0000", 2024-09-18T10:15:02.981 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1838999999999999e-05 2024-09-18T10:15:02.981 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.981 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.981 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:02.981 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.914075+0000", 2024-09-18T10:15:02.981 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0478e-05 2024-09-18T10:15:02.982 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:02.982 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:02.982 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:02.982 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.982 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.982 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:332 3.ds0 3:b1a14543:::smithi18925896-15:head [append 0~737280 in=737280b] snapc b=[b,4] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-18T10:15:02.982 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:02.789344+0000", 2024-09-18T10:15:02.982 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.17757647600000001, 2024-09-18T10:15:02.982 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.177880275, 2024-09-18T10:15:02.982 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:02.982 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-18T10:15:02.982 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:02.982 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:02.982 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:02.983 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 332 2024-09-18T10:15:02.983 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.983 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:02.983 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.983 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:02.983 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.789344+0000", 2024-09-18T10:15:02.983 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:02.983 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.983 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.983 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:02.983 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.789344+0000", 2024-09-18T10:15:02.983 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:02.983 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.984 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.984 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:02.984 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.789342+0000", 2024-09-18T10:15:02.984 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999976 2024-09-18T10:15:02.984 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.984 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.984 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:02.984 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.790370+0000", 2024-09-18T10:15:02.984 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010286150000000001 2024-09-18T10:15:02.984 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.984 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.984 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:02.984 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.790403+0000", 2024-09-18T10:15:02.984 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2549000000000003e-05 2024-09-18T10:15:02.984 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.985 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.985 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:02.985 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.790414+0000", 2024-09-18T10:15:02.985 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1279e-05 2024-09-18T10:15:02.985 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.985 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.985 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:02.985 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.790526+0000", 2024-09-18T10:15:02.985 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000111994 2024-09-18T10:15:02.985 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.985 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.985 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-09-18T10:15:02.985 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.790540+0000", 2024-09-18T10:15:02.986 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4296e-05 2024-09-18T10:15:02.986 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.986 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.986 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:02.986 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.794331+0000", 2024-09-18T10:15:02.986 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0037907029999999999 2024-09-18T10:15:02.986 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.986 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.986 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:02.986 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.794483+0000", 2024-09-18T10:15:02.986 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000151772 2024-09-18T10:15:02.986 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.986 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.987 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:02.987 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.796730+0000", 2024-09-18T10:15:02.987 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0022475009999999998 2024-09-18T10:15:02.987 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.987 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.987 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:02.987 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.804001+0000", 2024-09-18T10:15:02.987 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0072711420000000004 2024-09-18T10:15:02.987 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:02.987 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:02.987 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:02.987 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.987 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.987 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:333 3.ds0 3:b1a14543:::smithi18925896-15:head [append 0~376832 in=376832b] snapc b=[b,4] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-18T10:15:02.988 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:02.792703+0000", 2024-09-18T10:15:02.988 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.174217081, 2024-09-18T10:15:02.988 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.17465049799999999, 2024-09-18T10:15:02.988 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:02.988 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-18T10:15:02.988 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:02.988 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:02.988 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:02.988 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 333 2024-09-18T10:15:02.988 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.988 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:02.988 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.988 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:02.989 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.792703+0000", 2024-09-18T10:15:02.989 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:02.989 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.989 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.989 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:02.989 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.792703+0000", 2024-09-18T10:15:02.989 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:02.989 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.989 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.989 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:02.989 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.792702+0000", 2024-09-18T10:15:02.989 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:02.989 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.989 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.990 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:02.990 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.793113+0000", 2024-09-18T10:15:02.990 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00041176199999999998 2024-09-18T10:15:02.990 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.990 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.990 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:02.990 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.793131+0000", 2024-09-18T10:15:02.990 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7802e-05 2024-09-18T10:15:02.990 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.990 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.990 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:02.990 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.793140+0000", 2024-09-18T10:15:02.990 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0790000000000007e-06 2024-09-18T10:15:02.991 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.991 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.991 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:02.991 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.793223+0000", 2024-09-18T10:15:02.991 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2414000000000001e-05 2024-09-18T10:15:02.991 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.991 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.991 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:02.991 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.793233+0000", 2024-09-18T10:15:02.991 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.076e-05 2024-09-18T10:15:02.991 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.991 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.991 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:02.991 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.799019+0000", 2024-09-18T10:15:02.992 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0057857109999999998 2024-09-18T10:15:02.992 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.992 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.992 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:02.992 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.799289+0000", 2024-09-18T10:15:02.992 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00026981800000000001 2024-09-18T10:15:02.992 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.992 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.992 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:02.992 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.801174+0000", 2024-09-18T10:15:02.992 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00188511 2024-09-18T10:15:02.992 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.992 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.992 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:02.993 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.804009+0000", 2024-09-18T10:15:02.993 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002835181 2024-09-18T10:15:02.993 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:02.993 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:02.993 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:02.993 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.993 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.993 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:334 3.ds0 3:b1a14543:::smithi18925896-15:head [setxattr _header (56) in=63b] snapc b=[b,4] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-18T10:15:02.993 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:02.793164+0000", 2024-09-18T10:15:02.993 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.173755936, 2024-09-18T10:15:02.993 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.17431354399999999, 2024-09-18T10:15:02.993 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:02.993 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-18T10:15:02.994 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:02.994 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:02.994 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:02.994 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 334 2024-09-18T10:15:02.994 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.994 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:02.994 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.994 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:02.994 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.793164+0000", 2024-09-18T10:15:02.994 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:02.994 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.994 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.994 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:02.994 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.793164+0000", 2024-09-18T10:15:02.995 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:02.995 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.995 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.995 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:02.995 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.793163+0000", 2024-09-18T10:15:02.995 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:02.995 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.995 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.995 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:02.995 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.793170+0000", 2024-09-18T10:15:02.995 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3050000000000004e-06 2024-09-18T10:15:02.995 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.995 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.995 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:02.995 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.793180+0000", 2024-09-18T10:15:02.995 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8069999999999994e-06 2024-09-18T10:15:02.996 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.996 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.996 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:02.996 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.793187+0000", 2024-09-18T10:15:02.996 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6879999999999999e-06 2024-09-18T10:15:02.996 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.996 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.996 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:02.996 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.793293+0000", 2024-09-18T10:15:02.996 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000105441 2024-09-18T10:15:02.996 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.996 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.996 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:02.996 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.793302+0000", 2024-09-18T10:15:02.997 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.5769999999999994e-06 2024-09-18T10:15:02.997 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.997 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.997 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:02.997 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.802703+0000", 2024-09-18T10:15:02.997 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0094011400000000005 2024-09-18T10:15:02.997 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.997 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.997 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:02.997 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.802949+0000", 2024-09-18T10:15:02.997 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00024604500000000002 2024-09-18T10:15:02.997 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.997 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.997 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:02.998 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.803628+0000", 2024-09-18T10:15:02.998 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000679391 2024-09-18T10:15:02.998 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.998 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.998 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:02.998 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.804224+0000", 2024-09-18T10:15:02.998 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00059597000000000003 2024-09-18T10:15:02.998 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:02.998 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:02.998 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:02.998 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.998 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.998 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:335 3.ds0 3:b1a14543:::smithi18925896-15:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e47)", 2024-09-18T10:15:02.998 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:02.793201+0000", 2024-09-18T10:15:02.998 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.17371947500000001, 2024-09-18T10:15:02.999 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.17439895599999999, 2024-09-18T10:15:02.999 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:02.999 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "waiting for rw locks", 2024-09-18T10:15:02.999 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:02.999 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:02.999 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:02.999 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 335 2024-09-18T10:15:02.999 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.999 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:02.999 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:02.999 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:02.999 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.793201+0000", 2024-09-18T10:15:02.999 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:02.999 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:02.999 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.000 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:03.000 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.793201+0000", 2024-09-18T10:15:03.000 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:03.000 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.000 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.000 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:03.000 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.793200+0000", 2024-09-18T10:15:03.000 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:03.000 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.000 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.000 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:03.000 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.793206+0000", 2024-09-18T10:15:03.000 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2199999999999997e-06 2024-09-18T10:15:03.000 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.001 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.001 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:03.001 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.793221+0000", 2024-09-18T10:15:03.001 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5092000000000001e-05 2024-09-18T10:15:03.001 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.001 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.001 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:03.001 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.793229+0000", 2024-09-18T10:15:03.001 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1820000000000006e-06 2024-09-18T10:15:03.001 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.001 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.001 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:03.001 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.793367+0000", 2024-09-18T10:15:03.001 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000138057 2024-09-18T10:15:03.002 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.002 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.002 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:03.002 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.793384+0000", 2024-09-18T10:15:03.002 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6731e-05 2024-09-18T10:15:03.002 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.002 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.002 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:03.002 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.804064+0000", 2024-09-18T10:15:03.002 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010680185 2024-09-18T10:15:03.002 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.002 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.002 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2024-09-18T10:15:03.002 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.804199+0000", 2024-09-18T10:15:03.003 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013532999999999999 2024-09-18T10:15:03.003 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:03.003 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:03.003 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:03.003 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.003 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.003 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWriteReply(3.ds0 47/32 ECSubWriteReply(tid=56, last_complete=47'9, committed=1, applied=1))", 2024-09-18T10:15:03.003 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:02.803464+0000", 2024-09-18T10:15:03.003 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.163456343, 2024-09-18T10:15:03.003 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.16424186299999999, 2024-09-18T10:15:03.003 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:03.003 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-18T10:15:03.003 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:03.003 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.004 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:03.004 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.803464+0000", 2024-09-18T10:15:03.004 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:03.004 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.004 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.004 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:03.004 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.803464+0000", 2024-09-18T10:15:03.004 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:03.004 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.004 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.004 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:03.004 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.803463+0000", 2024-09-18T10:15:03.004 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:15:03.005 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.005 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.005 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:03.005 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.803470+0000", 2024-09-18T10:15:03.005 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4780000000000002e-06 2024-09-18T10:15:03.005 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.005 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.005 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:03.005 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.803483+0000", 2024-09-18T10:15:03.005 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2645e-05 2024-09-18T10:15:03.005 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.005 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.005 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:03.005 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.903573+0000", 2024-09-18T10:15:03.006 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10009000899999999 2024-09-18T10:15:03.006 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:03.006 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:03.006 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:03.006 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.006 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.006 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWriteReply(3.ds0 47/32 ECSubWriteReply(tid=55, last_complete=47'8, committed=1, applied=1))", 2024-09-18T10:15:03.006 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:02.900348+0000", 2024-09-18T10:15:03.006 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.066571903000000002, 2024-09-18T10:15:03.006 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.067421817999999994, 2024-09-18T10:15:03.006 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:03.006 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "no flag points reached", 2024-09-18T10:15:03.006 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:03.006 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.007 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:03.007 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.900348+0000", 2024-09-18T10:15:03.007 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:03.007 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.007 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.007 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:03.007 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.900348+0000", 2024-09-18T10:15:03.007 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:03.007 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.007 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.007 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:03.007 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.900347+0000", 2024-09-18T10:15:03.007 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:03.007 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.008 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.008 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:03.008 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.900360+0000", 2024-09-18T10:15:03.008 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2802999999999999e-05 2024-09-18T10:15:03.008 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.008 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.008 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:03.008 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.900391+0000", 2024-09-18T10:15:03.008 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1458999999999998e-05 2024-09-18T10:15:03.008 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:03.008 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:03.008 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:03.008 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.009 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.009 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWriteReply(3.ds0 47/32 ECSubWriteReply(tid=55, last_complete=47'8, committed=1, applied=1))", 2024-09-18T10:15:03.009 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:02.903734+0000", 2024-09-18T10:15:03.009 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.063186081000000005, 2024-09-18T10:15:03.009 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.064092471999999998, 2024-09-18T10:15:03.009 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:03.009 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-18T10:15:03.009 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:03.009 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.009 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:03.009 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.903734+0000", 2024-09-18T10:15:03.009 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:03.009 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.009 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.010 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:03.010 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.903734+0000", 2024-09-18T10:15:03.010 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:03.010 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.010 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.010 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:03.010 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.903734+0000", 2024-09-18T10:15:03.010 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:03.010 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.010 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.010 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:03.010 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.903738+0000", 2024-09-18T10:15:03.010 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6299999999999997e-06 2024-09-18T10:15:03.010 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.010 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.011 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:03.011 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.903753+0000", 2024-09-18T10:15:03.011 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4834000000000001e-05 2024-09-18T10:15:03.011 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.011 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.011 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:03.011 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.903760+0000", 2024-09-18T10:15:03.011 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0029999999999997e-06 2024-09-18T10:15:03.011 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:03.011 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:03.011 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:03.011 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.011 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.011 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWriteReply(3.ds0 47/32 ECSubWriteReply(tid=56, last_complete=47'9, committed=1, applied=1))", 2024-09-18T10:15:03.012 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:02.903766+0000", 2024-09-18T10:15:03.012 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.063154044000000006, 2024-09-18T10:15:03.012 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.064124160999999999, 2024-09-18T10:15:03.012 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:03.012 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-18T10:15:03.012 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:03.012 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.012 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:03.012 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.903766+0000", 2024-09-18T10:15:03.012 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:03.012 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.012 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.012 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:03.013 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.903766+0000", 2024-09-18T10:15:03.013 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:03.013 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.013 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.013 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:03.013 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.903766+0000", 2024-09-18T10:15:03.013 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:03.013 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.013 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.013 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:03.013 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.903769+0000", 2024-09-18T10:15:03.013 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8219999999999998e-06 2024-09-18T10:15:03.013 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.013 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.014 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:03.014 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.903775+0000", 2024-09-18T10:15:03.014 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2859999999999996e-06 2024-09-18T10:15:03.014 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.014 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.014 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:03.014 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.903779+0000", 2024-09-18T10:15:03.014 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7179999999999998e-06 2024-09-18T10:15:03.014 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:03.014 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:03.014 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:03.014 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.014 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.014 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWriteReply(3.ds0 47/32 ECSubWriteReply(tid=57, last_complete=47'10, committed=1, applied=1))", 2024-09-18T10:15:03.015 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:02.905647+0000", 2024-09-18T10:15:03.015 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.061272839000000003, 2024-09-18T10:15:03.015 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.062313742999999998, 2024-09-18T10:15:03.015 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:03.015 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-18T10:15:03.015 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:03.015 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.015 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:03.015 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.905647+0000", 2024-09-18T10:15:03.015 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:03.015 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.015 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.015 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:03.015 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.905647+0000", 2024-09-18T10:15:03.016 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:03.016 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.016 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.016 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:03.016 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.905646+0000", 2024-09-18T10:15:03.016 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:15:03.016 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.016 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.016 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:03.016 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.905653+0000", 2024-09-18T10:15:03.016 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8730000000000001e-06 2024-09-18T10:15:03.016 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.016 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.017 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:03.017 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.905671+0000", 2024-09-18T10:15:03.017 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7838000000000001e-05 2024-09-18T10:15:03.017 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.017 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.017 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:03.017 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.905683+0000", 2024-09-18T10:15:03.017 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1711e-05 2024-09-18T10:15:03.017 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:03.017 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:03.017 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:03.017 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.017 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.017 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWriteReply(3.ds0 47/32 ECSubWriteReply(tid=57, last_complete=47'10, committed=1, applied=1))", 2024-09-18T10:15:03.018 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:02.905852+0000", 2024-09-18T10:15:03.018 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.061068259999999999, 2024-09-18T10:15:03.018 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.062172855999999999, 2024-09-18T10:15:03.018 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:03.018 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-18T10:15:03.018 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:03.018 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.018 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:03.018 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.905852+0000", 2024-09-18T10:15:03.018 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:03.018 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.018 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.018 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:03.018 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.905852+0000", 2024-09-18T10:15:03.019 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:03.019 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.019 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.019 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:03.019 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.905851+0000", 2024-09-18T10:15:03.019 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:15:03.019 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.019 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.019 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:03.019 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.905857+0000", 2024-09-18T10:15:03.019 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3800000000000002e-06 2024-09-18T10:15:03.019 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.019 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.019 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:03.020 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.905872+0000", 2024-09-18T10:15:03.020 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5424999999999999e-05 2024-09-18T10:15:03.020 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.020 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.020 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:03.020 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.905881+0000", 2024-09-18T10:15:03.020 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7560000000000005e-06 2024-09-18T10:15:03.020 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:03.020 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:03.020 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:03.020 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.020 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.020 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.cs2 47/32 ECSubWrite(tid=58, reqid=client.4632.0:331, at_version=47'35, trim_to=0'0, roll_forward_to=45'33))", 2024-09-18T10:15:03.020 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:02.906684+0000", 2024-09-18T10:15:03.021 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.060236591999999999, 2024-09-18T10:15:03.021 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.061410154000000002, 2024-09-18T10:15:03.022 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:03.022 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:03.022 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:03.022 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.022 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:03.022 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.906684+0000", 2024-09-18T10:15:03.022 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:03.022 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.022 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.022 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:03.022 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.906684+0000", 2024-09-18T10:15:03.022 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:03.023 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.023 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.023 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:03.023 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.906683+0000", 2024-09-18T10:15:03.023 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:15:03.023 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.023 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.023 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:03.023 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.906823+0000", 2024-09-18T10:15:03.023 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000139617 2024-09-18T10:15:03.023 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.023 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.023 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:03.023 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.906857+0000", 2024-09-18T10:15:03.024 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4214999999999997e-05 2024-09-18T10:15:03.024 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.024 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.024 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:03.024 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.906908+0000", 2024-09-18T10:15:03.024 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1375000000000002e-05 2024-09-18T10:15:03.024 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.024 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.024 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:03.024 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.910398+0000", 2024-09-18T10:15:03.024 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0034899280000000002 2024-09-18T10:15:03.024 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.024 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.025 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:03.025 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.910424+0000", 2024-09-18T10:15:03.025 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6265999999999999e-05 2024-09-18T10:15:03.025 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:03.025 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:03.025 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:03.025 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:03.025 INFO:teuthology.orchestra.run.smithi136.stdout: ], 2024-09-18T10:15:03.025 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 14 2024-09-18T10:15:03.025 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-18T10:15:03.026 INFO:tasks.rados.rados.0.smithi189.stdout:128: finishing copy_from to smithi18925896-25 2024-09-18T10:15:03.026 INFO:tasks.rados.rados.0.smithi189.stdout:128: got expected ENOENT (src dne) 2024-09-18T10:15:03.027 INFO:tasks.rados.rados.0.smithi189.stdout:128: finishing copy_from racing read to smithi18925896-25 2024-09-18T10:15:03.027 INFO:tasks.rados.rados.0.smithi189.stdout:133: expect (ObjNum 28 snap 0 seq_num 28) 2024-09-18T10:15:03.027 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_blocked_ops 2024-09-18T10:15:03.056 INFO:tasks.ceph.osd.11.smithi189.stderr:2024-09-18T10:15:03.051+0000 7f718b60a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 22400) UID: 0 2024-09-18T10:15:03.100 INFO:tasks.rados.rados.0.smithi189.stdout:130: finishing write tid 1 to smithi18925896-15 2024-09-18T10:15:03.108 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 23 v 35 (ObjNum 53 snap 1 seq_num 53) dirty exists 2024-09-18T10:15:03.109 INFO:tasks.rados.rados.0.smithi189.stdout:132: finishing rollback tid 0 to smithi18925896-31 2024-09-18T10:15:03.109 INFO:tasks.rados.rados.0.smithi189.stdout:132: finishing rollback tid 1 to smithi18925896-31 2024-09-18T10:15:03.110 INFO:tasks.rados.rados.0.smithi189.stdout:132: finishing rollback tid 2 to smithi18925896-31 2024-09-18T10:15:03.110 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 31 v 39 (ObjNum 30 snap 0 seq_num 30) dirty exists 2024-09-18T10:15:03.130 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-18T10:15:03.130 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-18T10:15:03.130 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-18T10:15:03.131 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-18T10:15:03.131 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-18T10:15:03.140 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_historic_ops 2024-09-18T10:15:03.157 INFO:tasks.ceph.osd.3.smithi189.stderr:2024-09-18T10:15:03.155+0000 7f2440fc7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22396) UID: 0 2024-09-18T10:15:03.205 INFO:tasks.rados.rados.0.smithi189.stdout:130: finishing write tid 2 to smithi18925896-15 2024-09-18T10:15:03.206 INFO:tasks.rados.rados.0.smithi189.stdout:130: finishing write tid 3 to smithi18925896-15 2024-09-18T10:15:03.209 INFO:tasks.rados.rados.0.smithi189.stdout:134: finishing write tid 1 to smithi18925896-39 2024-09-18T10:15:03.246 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-18T10:15:03.246 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-18T10:15:03.246 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-18T10:15:03.247 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-18T10:15:03.247 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.247 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.7s6 36/32 ECSubWrite(tid=31, reqid=client.4632.0:128, at_version=0'0, trim_to=0'0, roll_forward_to=36'8))", 2024-09-18T10:15:03.247 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:51.219673+0000", 2024-09-18T10:15:03.247 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 12.023475672, 2024-09-18T10:15:03.247 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20600959799999999, 2024-09-18T10:15:03.247 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:03.247 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:03.247 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:03.247 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.247 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:03.247 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.219673+0000", 2024-09-18T10:15:03.248 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:03.248 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.248 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.248 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:03.249 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.219673+0000", 2024-09-18T10:15:03.249 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:03.249 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.249 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.249 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:03.249 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.219671+0000", 2024-09-18T10:15:03.249 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:03.249 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.249 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.249 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:03.249 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.219696+0000", 2024-09-18T10:15:03.249 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4649000000000001e-05 2024-09-18T10:15:03.249 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.249 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.250 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:03.250 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.219728+0000", 2024-09-18T10:15:03.250 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2036000000000003e-05 2024-09-18T10:15:03.250 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.250 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.250 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:03.250 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.319879+0000", 2024-09-18T10:15:03.250 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100151433 2024-09-18T10:15:03.250 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.250 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.250 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:03.251 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.423228+0000", 2024-09-18T10:15:03.251 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.103348542 2024-09-18T10:15:03.251 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.251 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.251 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:03.251 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.423241+0000", 2024-09-18T10:15:03.251 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3519999999999999e-05 2024-09-18T10:15:03.251 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.251 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.251 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:03.251 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.425642+0000", 2024-09-18T10:15:03.251 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0024009859999999999 2024-09-18T10:15:03.251 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.251 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.252 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:03.252 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.425682+0000", 2024-09-18T10:15:03.252 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9999000000000001e-05 2024-09-18T10:15:03.252 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:03.252 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:03.252 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:03.252 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.252 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.252 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:171 3.as0 3:5b104cb9:::smithi18925896-9:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e36)", 2024-09-18T10:15:03.252 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:51.532779+0000", 2024-09-18T10:15:03.252 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 11.710369957999999, 2024-09-18T10:15:03.252 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30363525200000002, 2024-09-18T10:15:03.252 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:03.253 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:03.253 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:03.253 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:03.253 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:03.253 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 171 2024-09-18T10:15:03.253 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.253 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:03.253 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.253 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:03.253 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.532779+0000", 2024-09-18T10:15:03.253 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:03.254 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.254 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.254 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:03.254 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.532779+0000", 2024-09-18T10:15:03.254 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:03.254 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.254 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.254 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:03.254 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.532776+0000", 2024-09-18T10:15:03.254 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999971 2024-09-18T10:15:03.254 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.254 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.255 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:03.255 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.532789+0000", 2024-09-18T10:15:03.255 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3339999999999999e-05 2024-09-18T10:15:03.255 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.255 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.255 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:03.255 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.532823+0000", 2024-09-18T10:15:03.255 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3627999999999997e-05 2024-09-18T10:15:03.255 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.255 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.255 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:03.255 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.532837+0000", 2024-09-18T10:15:03.255 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.385e-05 2024-09-18T10:15:03.255 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.256 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.256 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:03.256 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.633025+0000", 2024-09-18T10:15:03.256 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100188176 2024-09-18T10:15:03.256 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.256 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.256 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:03.256 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.633191+0000", 2024-09-18T10:15:03.256 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00016606699999999999 2024-09-18T10:15:03.256 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.256 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.256 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:03.256 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.633949+0000", 2024-09-18T10:15:03.257 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00075792600000000002 2024-09-18T10:15:03.257 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.257 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.257 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:03.257 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.835688+0000", 2024-09-18T10:15:03.257 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20173866700000001 2024-09-18T10:15:03.257 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.257 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.257 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:15:03.257 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.836246+0000", 2024-09-18T10:15:03.257 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00055799699999999996 2024-09-18T10:15:03.257 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.257 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.257 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:03.258 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.836414+0000", 2024-09-18T10:15:03.258 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000168274 2024-09-18T10:15:03.258 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:03.258 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:03.258 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:03.258 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.258 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.258 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(osd.0.21:1 3.as0 3:5dc9cbed:::smithi18925896-13:head [assert-version v4,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:03.258 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:51.535493+0000", 2024-09-18T10:15:03.258 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 11.707655062000001, 2024-09-18T10:15:03.258 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30246291400000003, 2024-09-18T10:15:03.258 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:03.258 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-18T10:15:03.258 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:03.259 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.259 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:03.259 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.535493+0000", 2024-09-18T10:15:03.259 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:03.259 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.259 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.259 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:03.259 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.535493+0000", 2024-09-18T10:15:03.259 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:03.260 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.260 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.260 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:03.260 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.535491+0000", 2024-09-18T10:15:03.260 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999971 2024-09-18T10:15:03.260 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.261 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.261 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:03.261 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.535508+0000", 2024-09-18T10:15:03.261 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7266e-05 2024-09-18T10:15:03.261 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.261 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.261 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:03.261 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.535537+0000", 2024-09-18T10:15:03.261 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8612e-05 2024-09-18T10:15:03.261 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.261 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.261 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:03.261 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.535554+0000", 2024-09-18T10:15:03.261 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7575999999999999e-05 2024-09-18T10:15:03.262 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.262 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.262 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:03.262 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.634492+0000", 2024-09-18T10:15:03.262 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.098937397999999996 2024-09-18T10:15:03.262 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.262 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.262 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:03.262 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.634722+0000", 2024-09-18T10:15:03.262 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00022994500000000001 2024-09-18T10:15:03.263 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.263 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.263 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:03.263 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.837956+0000", 2024-09-18T10:15:03.263 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20323484999999999 2024-09-18T10:15:03.263 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:03.263 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:03.263 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:03.263 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.263 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.263 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpRead(3.2s2 36/32 ECSubRead(tid=17, to_read={3:46b2871c:::smithi18925896-34:head=0,139264,0}, subchunks={3:46b2871c:::smithi18925896-34:head=[0,1]}, attrs_to_read=))", 2024-09-18T10:15:03.263 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:51.537987+0000", 2024-09-18T10:15:03.263 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 11.705161307999999, 2024-09-18T10:15:03.264 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.297589722, 2024-09-18T10:15:03.264 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:03.264 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:03.264 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:03.264 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.264 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:03.264 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.537987+0000", 2024-09-18T10:15:03.264 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:03.264 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.264 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.264 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:03.264 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.537987+0000", 2024-09-18T10:15:03.264 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:03.265 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.265 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.265 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:03.265 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.537986+0000", 2024-09-18T10:15:03.265 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:03.265 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.265 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.265 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:03.265 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.538009+0000", 2024-09-18T10:15:03.265 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3506000000000002e-05 2024-09-18T10:15:03.265 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.265 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.265 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:03.265 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.538052+0000", 2024-09-18T10:15:03.266 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2982000000000003e-05 2024-09-18T10:15:03.266 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.266 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.266 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:03.266 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.538075+0000", 2024-09-18T10:15:03.266 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3178999999999999e-05 2024-09-18T10:15:03.266 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.266 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.266 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:03.266 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.834627+0000", 2024-09-18T10:15:03.266 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.29655214299999999 2024-09-18T10:15:03.266 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.266 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.266 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:03.267 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.835577+0000", 2024-09-18T10:15:03.267 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00094962200000000001 2024-09-18T10:15:03.267 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:03.267 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:03.267 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:03.267 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.267 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.267 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:176 3.ds0 3:b1a14543:::smithi18925896-15:head [copy-from ver 4] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:03.267 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:52.680518+0000", 2024-09-18T10:15:03.267 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.562630965, 2024-09-18T10:15:03.267 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.42607371300000002, 2024-09-18T10:15:03.267 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:03.267 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:03.268 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:03.268 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:03.268 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:03.268 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 176 2024-09-18T10:15:03.268 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.268 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:03.268 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.268 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:03.268 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.680518+0000", 2024-09-18T10:15:03.268 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:03.268 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.268 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.268 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:03.268 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.680518+0000", 2024-09-18T10:15:03.269 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:03.269 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.269 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.269 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:03.269 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.680515+0000", 2024-09-18T10:15:03.269 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999971 2024-09-18T10:15:03.269 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.269 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.269 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:03.269 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.680533+0000", 2024-09-18T10:15:03.269 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8756000000000001e-05 2024-09-18T10:15:03.269 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.269 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.269 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:03.270 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.680573+0000", 2024-09-18T10:15:03.270 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9431999999999998e-05 2024-09-18T10:15:03.270 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.270 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.270 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:03.270 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.680587+0000", 2024-09-18T10:15:03.270 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.447e-05 2024-09-18T10:15:03.270 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.270 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.270 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:03.270 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.680755+0000", 2024-09-18T10:15:03.271 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000167584 2024-09-18T10:15:03.271 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.271 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.271 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-09-18T10:15:03.271 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.680784+0000", 2024-09-18T10:15:03.271 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9439000000000001e-05 2024-09-18T10:15:03.271 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.271 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.271 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:03.271 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.685398+0000", 2024-09-18T10:15:03.271 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0046140060000000004 2024-09-18T10:15:03.271 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.272 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.272 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:03.272 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.685502+0000", 2024-09-18T10:15:03.272 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000103702 2024-09-18T10:15:03.272 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.272 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.272 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:03.272 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.003083+0000", 2024-09-18T10:15:03.272 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.31758088600000001 2024-09-18T10:15:03.272 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.273 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.273 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:03.273 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.006087+0000", 2024-09-18T10:15:03.273 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0030044820000000002 2024-09-18T10:15:03.273 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.273 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.273 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:15:03.273 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.106397+0000", 2024-09-18T10:15:03.273 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100309469 2024-09-18T10:15:03.273 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.273 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.273 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:03.273 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.106591+0000", 2024-09-18T10:15:03.273 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000194513 2024-09-18T10:15:03.274 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:03.274 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:03.274 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:03.274 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.274 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.274 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:177 3.ds0 3:b1a14543:::smithi18925896-15:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:03.274 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:52.680610+0000", 2024-09-18T10:15:03.274 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.562538244000001, 2024-09-18T10:15:03.274 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.32474139800000001, 2024-09-18T10:15:03.275 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:03.275 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-18T10:15:03.275 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:03.275 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:03.275 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:03.275 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 177 2024-09-18T10:15:03.275 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.275 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:03.275 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.275 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:03.275 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.680610+0000", 2024-09-18T10:15:03.276 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:03.276 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.276 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.276 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:03.276 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.680610+0000", 2024-09-18T10:15:03.276 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:03.276 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.276 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.276 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:03.276 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.680609+0000", 2024-09-18T10:15:03.276 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:15:03.276 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.276 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.276 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:03.277 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.680614+0000", 2024-09-18T10:15:03.277 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6840000000000004e-06 2024-09-18T10:15:03.277 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.277 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.277 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:03.277 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.680625+0000", 2024-09-18T10:15:03.277 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0739e-05 2024-09-18T10:15:03.278 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.278 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.278 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:03.278 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.680655+0000", 2024-09-18T10:15:03.278 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0022000000000002e-05 2024-09-18T10:15:03.278 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.278 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.278 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:03.278 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.680965+0000", 2024-09-18T10:15:03.278 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00031078300000000002 2024-09-18T10:15:03.278 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.278 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.279 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:03.279 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.680976+0000", 2024-09-18T10:15:03.279 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0589e-05 2024-09-18T10:15:03.279 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.279 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.279 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:03.279 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.785862+0000", 2024-09-18T10:15:03.279 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10488597500000001 2024-09-18T10:15:03.279 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.279 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.279 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2024-09-18T10:15:03.279 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.785963+0000", 2024-09-18T10:15:03.279 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000100523 2024-09-18T10:15:03.280 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.280 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.280 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:03.280 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.005029+0000", 2024-09-18T10:15:03.280 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.219066329 2024-09-18T10:15:03.280 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.280 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.280 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:03.280 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.005193+0000", 2024-09-18T10:15:03.280 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00016436200000000001 2024-09-18T10:15:03.280 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.280 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.280 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:03.281 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.005352+0000", 2024-09-18T10:15:03.281 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000158486 2024-09-18T10:15:03.281 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:03.281 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:03.281 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:03.281 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.281 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.281 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:200 3.as0 3:5dc9cbed:::smithi18925896-13:2 [read 0~212992,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:03.281 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:52.694751+0000", 2024-09-18T10:15:03.281 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.548397813999999, 2024-09-18T10:15:03.282 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.40590051599999999, 2024-09-18T10:15:03.282 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:03.282 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-18T10:15:03.282 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:03.282 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:03.282 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:03.282 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 200 2024-09-18T10:15:03.282 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.282 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:03.282 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.282 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:03.282 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694751+0000", 2024-09-18T10:15:03.282 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:03.283 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.283 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.283 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:03.283 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694751+0000", 2024-09-18T10:15:03.283 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:03.283 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.283 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.283 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:03.283 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694750+0000", 2024-09-18T10:15:03.283 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:03.283 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.283 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.283 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:03.284 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694756+0000", 2024-09-18T10:15:03.284 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8119999999999996e-06 2024-09-18T10:15:03.284 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.284 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.284 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:03.284 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694762+0000", 2024-09-18T10:15:03.284 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8050000000000003e-06 2024-09-18T10:15:03.284 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.284 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.284 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:03.284 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694768+0000", 2024-09-18T10:15:03.284 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5549999999999999e-06 2024-09-18T10:15:03.284 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.284 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.285 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:03.285 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.695694+0000", 2024-09-18T10:15:03.285 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00092562999999999999 2024-09-18T10:15:03.285 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.285 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.285 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:03.286 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.695903+0000", 2024-09-18T10:15:03.286 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00020888699999999999 2024-09-18T10:15:03.286 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.286 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.286 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:03.286 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.100651+0000", 2024-09-18T10:15:03.286 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.40474845599999998 2024-09-18T10:15:03.286 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:03.286 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:03.286 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:03.286 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.286 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.286 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:201 3.as0 3:5dc9cbed:::smithi18925896-13:2 [read 0~212992,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:03.286 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:52.694813+0000", 2024-09-18T10:15:03.287 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.54833601, 2024-09-18T10:15:03.287 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.50652904899999995, 2024-09-18T10:15:03.287 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:03.287 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-18T10:15:03.287 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:03.287 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:03.287 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:03.287 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 201 2024-09-18T10:15:03.287 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.287 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:03.287 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.287 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:03.287 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694813+0000", 2024-09-18T10:15:03.287 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:03.288 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.288 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.288 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:03.288 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694813+0000", 2024-09-18T10:15:03.288 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:03.288 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.289 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.289 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:03.289 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694812+0000", 2024-09-18T10:15:03.289 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:03.289 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.289 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.289 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:03.289 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694816+0000", 2024-09-18T10:15:03.289 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7720000000000001e-06 2024-09-18T10:15:03.289 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.289 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.289 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:03.289 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694822+0000", 2024-09-18T10:15:03.290 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5529999999999998e-06 2024-09-18T10:15:03.290 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.290 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.290 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:03.290 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694827+0000", 2024-09-18T10:15:03.290 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1500000000000001e-06 2024-09-18T10:15:03.290 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.290 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.290 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:03.290 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.797100+0000", 2024-09-18T10:15:03.290 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.102273424 2024-09-18T10:15:03.291 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.291 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.291 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:03.291 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.797329+0000", 2024-09-18T10:15:03.291 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00022870200000000001 2024-09-18T10:15:03.291 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.291 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.291 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:03.291 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.201342+0000", 2024-09-18T10:15:03.291 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.40401293799999999 2024-09-18T10:15:03.291 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:03.291 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:03.291 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:03.291 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.292 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.292 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:207 3.as0 3:5c566071:::smithi18925896-28:head [copy-from ver 6] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:03.292 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:52.694855+0000", 2024-09-18T10:15:03.292 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.548293208, 2024-09-18T10:15:03.292 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.81394701899999999, 2024-09-18T10:15:03.292 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:03.292 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:03.292 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:03.292 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:03.292 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:03.292 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 207 2024-09-18T10:15:03.292 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.292 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:03.293 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.293 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:03.293 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694855+0000", 2024-09-18T10:15:03.293 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:03.293 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.293 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.293 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:03.293 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694855+0000", 2024-09-18T10:15:03.293 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:03.293 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.293 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.293 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:03.293 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694852+0000", 2024-09-18T10:15:03.293 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999967 2024-09-18T10:15:03.293 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.294 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.294 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:03.294 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694859+0000", 2024-09-18T10:15:03.294 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.08e-06 2024-09-18T10:15:03.294 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.294 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.294 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:03.294 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694866+0000", 2024-09-18T10:15:03.294 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9249999999999998e-06 2024-09-18T10:15:03.294 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.294 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.294 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:03.294 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694870+0000", 2024-09-18T10:15:03.294 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0330000000000002e-06 2024-09-18T10:15:03.295 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.295 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.295 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:03.295 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.900743+0000", 2024-09-18T10:15:03.295 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20587325400000001 2024-09-18T10:15:03.295 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.295 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.295 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:03.295 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.901026+0000", 2024-09-18T10:15:03.295 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00028313600000000002 2024-09-18T10:15:03.295 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.295 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.295 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:03.296 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.204074+0000", 2024-09-18T10:15:03.296 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30304722299999998 2024-09-18T10:15:03.296 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.296 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.296 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:03.296 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.207280+0000", 2024-09-18T10:15:03.296 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0032067049999999998 2024-09-18T10:15:03.296 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.296 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.296 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:15:03.296 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.508672+0000", 2024-09-18T10:15:03.296 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30139174800000001 2024-09-18T10:15:03.296 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.297 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.297 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:03.297 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.508802+0000", 2024-09-18T10:15:03.297 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013037500000000001 2024-09-18T10:15:03.297 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:03.297 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:03.297 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:03.297 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.297 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.297 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:208 3.as0 3:5c566071:::smithi18925896-28:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:03.297 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:52.694875+0000", 2024-09-18T10:15:03.297 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.548274047, 2024-09-18T10:15:03.297 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.51199464299999997, 2024-09-18T10:15:03.298 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:03.298 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-18T10:15:03.298 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:03.298 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:03.298 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:03.298 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 208 2024-09-18T10:15:03.298 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.298 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:03.298 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.298 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:03.298 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694875+0000", 2024-09-18T10:15:03.298 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:03.298 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.298 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.299 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:03.299 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694875+0000", 2024-09-18T10:15:03.299 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:03.299 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.299 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.299 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:03.299 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694874+0000", 2024-09-18T10:15:03.299 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:03.299 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.299 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.299 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:03.299 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694876+0000", 2024-09-18T10:15:03.299 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2369999999999999e-06 2024-09-18T10:15:03.299 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.300 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.300 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:03.300 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694881+0000", 2024-09-18T10:15:03.300 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0950000000000003e-06 2024-09-18T10:15:03.300 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.300 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.300 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:03.300 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694887+0000", 2024-09-18T10:15:03.300 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5690000000000004e-06 2024-09-18T10:15:03.300 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.300 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.300 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:03.300 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.901475+0000", 2024-09-18T10:15:03.301 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20658803000000001 2024-09-18T10:15:03.301 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.301 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.301 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2024-09-18T10:15:03.301 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.901550+0000", 2024-09-18T10:15:03.301 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5020999999999999e-05 2024-09-18T10:15:03.301 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.301 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.301 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:03.301 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.206415+0000", 2024-09-18T10:15:03.301 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30486486000000002 2024-09-18T10:15:03.301 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.301 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.302 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:03.302 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.206634+0000", 2024-09-18T10:15:03.302 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00021925499999999999 2024-09-18T10:15:03.302 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.302 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.302 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:03.302 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.206869+0000", 2024-09-18T10:15:03.302 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00023503599999999999 2024-09-18T10:15:03.302 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:03.302 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:03.302 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:03.302 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.302 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.302 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:269 3.ds0 3:b1a14543:::smithi18925896-15:head [read 0~180224,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T10:15:03.303 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:56.742371+0000", 2024-09-18T10:15:03.303 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 6.5007772929999996, 2024-09-18T10:15:03.303 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.29287403000000001, 2024-09-18T10:15:03.303 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:03.303 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-18T10:15:03.303 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:03.303 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:03.303 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:03.303 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 269 2024-09-18T10:15:03.303 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.303 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:03.303 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.303 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:03.304 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:56.742371+0000", 2024-09-18T10:15:03.304 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:03.304 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.304 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.304 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:03.304 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:56.742371+0000", 2024-09-18T10:15:03.304 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:03.304 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.304 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.304 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:03.304 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:56.742371+0000", 2024-09-18T10:15:03.304 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:03.304 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.304 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.305 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:03.305 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:56.742374+0000", 2024-09-18T10:15:03.305 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1389999999999999e-06 2024-09-18T10:15:03.305 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.305 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.305 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:03.305 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:56.742380+0000", 2024-09-18T10:15:03.305 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.711e-06 2024-09-18T10:15:03.305 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.305 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.305 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:03.305 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:56.742386+0000", 2024-09-18T10:15:03.305 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0939999999999996e-06 2024-09-18T10:15:03.306 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.306 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.306 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:03.306 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:56.831667+0000", 2024-09-18T10:15:03.306 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.089281207000000001 2024-09-18T10:15:03.306 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.306 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.306 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:03.306 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:56.831830+0000", 2024-09-18T10:15:03.306 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000163101 2024-09-18T10:15:03.306 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.306 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.307 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:03.307 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.035245+0000", 2024-09-18T10:15:03.307 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.203415278 2024-09-18T10:15:03.307 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:03.307 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:03.307 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:03.307 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.307 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.307 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:270 3.ds0 3:b1a14543:::smithi18925896-15:head [read 0~180224,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T10:15:03.307 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:56.742401+0000", 2024-09-18T10:15:03.307 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 6.5007470710000002, 2024-09-18T10:15:03.307 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.29437073499999999, 2024-09-18T10:15:03.307 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:03.307 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-18T10:15:03.308 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:03.308 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:03.308 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:03.308 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 270 2024-09-18T10:15:03.308 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.308 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:03.308 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.308 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:03.308 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:56.742401+0000", 2024-09-18T10:15:03.308 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:03.308 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.308 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.308 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:03.309 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:56.742401+0000", 2024-09-18T10:15:03.309 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:03.309 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.309 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.309 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:03.309 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:56.742401+0000", 2024-09-18T10:15:03.309 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:03.309 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.309 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.309 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:03.309 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:56.742405+0000", 2024-09-18T10:15:03.309 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7450000000000001e-06 2024-09-18T10:15:03.309 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.310 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.310 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:03.310 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:56.742413+0000", 2024-09-18T10:15:03.310 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0830000000000004e-06 2024-09-18T10:15:03.310 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.310 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.310 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:03.310 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:56.742416+0000", 2024-09-18T10:15:03.310 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6119999999999999e-06 2024-09-18T10:15:03.310 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.310 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.310 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:03.310 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:56.932691+0000", 2024-09-18T10:15:03.311 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.19027452 2024-09-18T10:15:03.311 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.311 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.311 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:03.311 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:56.932882+0000", 2024-09-18T10:15:03.311 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019128099999999999 2024-09-18T10:15:03.311 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.311 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.311 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:03.311 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.036772+0000", 2024-09-18T10:15:03.311 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.103890021 2024-09-18T10:15:03.311 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:03.311 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:03.312 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:03.312 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.312 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.312 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWriteReply(3.as0 41/32 ECSubWriteReply(tid=38, last_complete=41'12, committed=1, applied=1))", 2024-09-18T10:15:03.312 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:56.944491+0000", 2024-09-18T10:15:03.312 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 6.2986577869999998, 2024-09-18T10:15:03.312 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.28757494, 2024-09-18T10:15:03.312 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:03.312 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:03.312 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:03.312 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.312 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:03.312 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:56.944491+0000", 2024-09-18T10:15:03.312 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:03.313 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.313 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.313 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:03.313 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:56.944491+0000", 2024-09-18T10:15:03.313 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:03.313 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.313 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.313 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:03.313 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:56.944490+0000", 2024-09-18T10:15:03.313 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:15:03.313 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.313 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.313 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:03.313 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:56.944496+0000", 2024-09-18T10:15:03.314 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1340000000000001e-06 2024-09-18T10:15:03.314 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.314 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.314 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:03.314 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:56.944514+0000", 2024-09-18T10:15:03.314 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7688000000000001e-05 2024-09-18T10:15:03.314 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.314 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.314 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:03.314 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:56.944521+0000", 2024-09-18T10:15:03.314 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0929999999999997e-06 2024-09-18T10:15:03.314 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.314 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.315 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:03.315 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.232034+0000", 2024-09-18T10:15:03.315 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.28751317799999998 2024-09-18T10:15:03.315 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.315 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.315 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:03.315 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.232066+0000", 2024-09-18T10:15:03.315 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1906000000000003e-05 2024-09-18T10:15:03.315 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:03.315 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:03.315 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:03.315 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.315 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.315 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.2s2 41/40 ECSubWrite(tid=23, reqid=client.4632.0:265, at_version=41'11, trim_to=0'0, roll_forward_to=0'0))", 2024-09-18T10:15:03.316 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:57.236785+0000", 2024-09-18T10:15:03.316 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 6.0063634319999997, 2024-09-18T10:15:03.316 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30570645699999999, 2024-09-18T10:15:03.316 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:03.316 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:03.316 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:03.316 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.316 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:03.316 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.236785+0000", 2024-09-18T10:15:03.316 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:03.316 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.316 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.316 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:03.317 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.236785+0000", 2024-09-18T10:15:03.317 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:03.317 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.317 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.317 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:03.317 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.236783+0000", 2024-09-18T10:15:03.317 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:03.317 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.317 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.317 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:03.317 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.236924+0000", 2024-09-18T10:15:03.317 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000140572 2024-09-18T10:15:03.317 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.317 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.318 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:03.318 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.236956+0000", 2024-09-18T10:15:03.318 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2339000000000003e-05 2024-09-18T10:15:03.318 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.318 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.318 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:03.318 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.236970+0000", 2024-09-18T10:15:03.318 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3145999999999999e-05 2024-09-18T10:15:03.318 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.318 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.318 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:03.318 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.439765+0000", 2024-09-18T10:15:03.318 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20279530100000001 2024-09-18T10:15:03.319 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.319 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.319 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:03.319 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.439790+0000", 2024-09-18T10:15:03.319 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5043000000000001e-05 2024-09-18T10:15:03.319 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.319 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.319 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:03.319 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.542467+0000", 2024-09-18T10:15:03.319 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10267752099999999 2024-09-18T10:15:03.319 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.319 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.319 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:03.319 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.542492+0000", 2024-09-18T10:15:03.320 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4175000000000001e-05 2024-09-18T10:15:03.320 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:03.320 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:03.320 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:03.320 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.320 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.320 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.2s2 41/40 ECSubWrite(tid=24, reqid=client.4632.0:266, at_version=41'12, trim_to=0'0, roll_forward_to=0'0))", 2024-09-18T10:15:03.320 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:57.339428+0000", 2024-09-18T10:15:03.320 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.9037208840000002, 2024-09-18T10:15:03.320 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20308316500000001, 2024-09-18T10:15:03.320 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:03.320 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:03.320 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:03.321 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.321 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:03.321 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.339428+0000", 2024-09-18T10:15:03.321 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:03.321 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.321 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.321 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:03.321 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.339428+0000", 2024-09-18T10:15:03.321 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:03.321 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.321 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.321 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:03.321 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.339426+0000", 2024-09-18T10:15:03.321 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:03.322 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.322 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.322 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:03.322 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.339474+0000", 2024-09-18T10:15:03.322 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8208999999999999e-05 2024-09-18T10:15:03.322 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.322 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.322 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:03.322 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.339523+0000", 2024-09-18T10:15:03.322 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9094999999999997e-05 2024-09-18T10:15:03.323 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.323 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.323 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:03.323 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.339541+0000", 2024-09-18T10:15:03.323 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7376000000000001e-05 2024-09-18T10:15:03.323 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.323 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.323 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:03.323 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.441462+0000", 2024-09-18T10:15:03.323 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.101921652 2024-09-18T10:15:03.323 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.323 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.323 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:03.323 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.441487+0000", 2024-09-18T10:15:03.324 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4182e-05 2024-09-18T10:15:03.324 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.324 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.324 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:03.324 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.542497+0000", 2024-09-18T10:15:03.324 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10101048899999999 2024-09-18T10:15:03.324 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.324 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.324 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:03.324 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.542511+0000", 2024-09-18T10:15:03.324 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3739e-05 2024-09-18T10:15:03.324 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:03.324 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:03.325 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:03.325 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.325 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.325 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWriteReply(3.as0 41/32 ECSubWriteReply(tid=42, last_complete=41'12, committed=1, applied=1))", 2024-09-18T10:15:03.325 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:57.339701+0000", 2024-09-18T10:15:03.325 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.9034474030000004, 2024-09-18T10:15:03.325 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20257603800000001, 2024-09-18T10:15:03.325 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:03.325 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:03.325 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:03.325 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.325 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:03.325 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.339701+0000", 2024-09-18T10:15:03.326 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:03.326 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.326 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.326 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:03.326 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.339701+0000", 2024-09-18T10:15:03.326 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:03.326 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.326 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.326 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:03.326 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.339700+0000", 2024-09-18T10:15:03.326 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:15:03.326 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.326 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.326 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:03.327 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.339708+0000", 2024-09-18T10:15:03.327 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1599999999999998e-06 2024-09-18T10:15:03.327 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.327 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.327 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:03.327 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.339738+0000", 2024-09-18T10:15:03.327 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9281999999999998e-05 2024-09-18T10:15:03.327 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.327 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.327 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:03.327 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.339750+0000", 2024-09-18T10:15:03.327 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2767e-05 2024-09-18T10:15:03.327 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.328 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.328 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:03.328 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.542245+0000", 2024-09-18T10:15:03.328 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20249478700000001 2024-09-18T10:15:03.328 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.328 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.328 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:03.328 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.542277+0000", 2024-09-18T10:15:03.328 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2122000000000002e-05 2024-09-18T10:15:03.328 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:03.328 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:03.328 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:03.328 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.328 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.329 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:326 3.fs0 3:f1c2a816:::smithi18925896-35:head [delete] snapc 7=[4] ondisk+write+known_if_redirected+full_force+supports_pool_eio e45)", 2024-09-18T10:15:03.329 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:01.796087+0000", 2024-09-18T10:15:03.329 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.4470618100000001, 2024-09-18T10:15:03.329 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30577019700000002, 2024-09-18T10:15:03.329 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:03.329 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:03.329 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:03.329 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:03.329 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:03.329 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 326 2024-09-18T10:15:03.329 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.329 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:03.329 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.330 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:03.330 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:01.796087+0000", 2024-09-18T10:15:03.330 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:03.330 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.330 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.330 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:03.330 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:01.796087+0000", 2024-09-18T10:15:03.330 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:03.330 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.330 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.330 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:03.330 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:01.796084+0000", 2024-09-18T10:15:03.330 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999971 2024-09-18T10:15:03.330 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.331 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.331 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:03.331 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:01.796098+0000", 2024-09-18T10:15:03.331 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3811e-05 2024-09-18T10:15:03.331 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.331 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.331 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:03.331 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:01.796124+0000", 2024-09-18T10:15:03.331 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6854000000000001e-05 2024-09-18T10:15:03.331 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.331 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.331 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:03.331 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:01.896289+0000", 2024-09-18T10:15:03.332 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10016480799999999 2024-09-18T10:15:03.332 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.332 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.332 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:03.332 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:01.899288+0000", 2024-09-18T10:15:03.332 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0029986940000000001 2024-09-18T10:15:03.332 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.332 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.332 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:03.332 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:01.899458+0000", 2024-09-18T10:15:03.332 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017041199999999999 2024-09-18T10:15:03.332 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.332 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.332 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:03.333 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:01.900610+0000", 2024-09-18T10:15:03.333 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001151691 2024-09-18T10:15:03.333 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.333 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.333 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:03.333 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:01.902271+0000", 2024-09-18T10:15:03.333 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0016610419999999999 2024-09-18T10:15:03.333 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.333 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.333 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:15:03.333 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.101639+0000", 2024-09-18T10:15:03.333 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.199367616 2024-09-18T10:15:03.334 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.334 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.334 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:03.334 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.101857+0000", 2024-09-18T10:15:03.334 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00021827500000000001 2024-09-18T10:15:03.334 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:03.334 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:03.334 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:03.334 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.334 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.334 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:332 3.ds0 3:b1a14543:::smithi18925896-15:head [append 0~737280 in=737280b] snapc b=[b,4] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-18T10:15:03.334 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:02.789344+0000", 2024-09-18T10:15:03.334 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.45380477600000002, 2024-09-18T10:15:03.334 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.31055802500000002, 2024-09-18T10:15:03.335 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:03.335 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:03.335 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:03.335 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:03.335 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:03.335 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 332 2024-09-18T10:15:03.335 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.335 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:03.335 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.335 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:03.335 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.789344+0000", 2024-09-18T10:15:03.335 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:03.335 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.336 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.336 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:03.336 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.789344+0000", 2024-09-18T10:15:03.336 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:03.336 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.336 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.336 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:03.336 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.789342+0000", 2024-09-18T10:15:03.336 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999976 2024-09-18T10:15:03.336 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.336 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.336 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:03.336 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.790370+0000", 2024-09-18T10:15:03.337 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010286150000000001 2024-09-18T10:15:03.337 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.337 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.337 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:03.337 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.790403+0000", 2024-09-18T10:15:03.337 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2549000000000003e-05 2024-09-18T10:15:03.337 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.337 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.337 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:03.337 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.790414+0000", 2024-09-18T10:15:03.337 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1279e-05 2024-09-18T10:15:03.337 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.337 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.337 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:03.338 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.790526+0000", 2024-09-18T10:15:03.338 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000111994 2024-09-18T10:15:03.338 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.338 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.338 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-09-18T10:15:03.338 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.790540+0000", 2024-09-18T10:15:03.338 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4296e-05 2024-09-18T10:15:03.338 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.338 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.338 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:03.338 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.794331+0000", 2024-09-18T10:15:03.338 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0037907029999999999 2024-09-18T10:15:03.338 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.338 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.339 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:03.339 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.794483+0000", 2024-09-18T10:15:03.339 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000151772 2024-09-18T10:15:03.339 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.339 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.339 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:03.339 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.796730+0000", 2024-09-18T10:15:03.339 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0022475009999999998 2024-09-18T10:15:03.339 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.339 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.339 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:03.339 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.804001+0000", 2024-09-18T10:15:03.339 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0072711420000000004 2024-09-18T10:15:03.340 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.340 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.340 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:15:03.340 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:03.099777+0000", 2024-09-18T10:15:03.340 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.295775552 2024-09-18T10:15:03.340 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.340 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.340 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:03.340 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:03.099902+0000", 2024-09-18T10:15:03.340 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012481999999999999 2024-09-18T10:15:03.340 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:03.340 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:03.340 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:03.341 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.341 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.341 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:333 3.ds0 3:b1a14543:::smithi18925896-15:head [append 0~376832 in=376832b] snapc b=[b,4] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-18T10:15:03.341 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:02.792703+0000", 2024-09-18T10:15:03.341 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.45044538099999998, 2024-09-18T10:15:03.341 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.40779418200000001, 2024-09-18T10:15:03.341 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:03.341 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:03.341 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:03.341 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:03.341 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:03.341 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 333 2024-09-18T10:15:03.341 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.342 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:03.342 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.342 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:03.342 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.792703+0000", 2024-09-18T10:15:03.342 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:03.342 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.342 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.342 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:03.342 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.792703+0000", 2024-09-18T10:15:03.342 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:03.342 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.342 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.342 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:03.342 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.792702+0000", 2024-09-18T10:15:03.343 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:03.343 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.343 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.343 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:03.343 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.793113+0000", 2024-09-18T10:15:03.343 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00041176199999999998 2024-09-18T10:15:03.343 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.343 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.343 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:03.343 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.793131+0000", 2024-09-18T10:15:03.343 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7802e-05 2024-09-18T10:15:03.343 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.343 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.343 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:03.344 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.793140+0000", 2024-09-18T10:15:03.344 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0790000000000007e-06 2024-09-18T10:15:03.344 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.344 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.344 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:03.344 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.793223+0000", 2024-09-18T10:15:03.344 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2414000000000001e-05 2024-09-18T10:15:03.344 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.344 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.344 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:03.344 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.793233+0000", 2024-09-18T10:15:03.344 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.076e-05 2024-09-18T10:15:03.344 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.345 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.345 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:03.345 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.799019+0000", 2024-09-18T10:15:03.345 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0057857109999999998 2024-09-18T10:15:03.345 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.345 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.345 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:03.345 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.799289+0000", 2024-09-18T10:15:03.345 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00026981800000000001 2024-09-18T10:15:03.345 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.345 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.345 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:03.345 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.801174+0000", 2024-09-18T10:15:03.346 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00188511 2024-09-18T10:15:03.346 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.346 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.346 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:03.346 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.804009+0000", 2024-09-18T10:15:03.346 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002835181 2024-09-18T10:15:03.346 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.346 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.346 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:15:03.346 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:03.200390+0000", 2024-09-18T10:15:03.346 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.39638093699999999 2024-09-18T10:15:03.346 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.346 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.346 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:03.347 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:03.200497+0000", 2024-09-18T10:15:03.347 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000107243 2024-09-18T10:15:03.347 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:03.347 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:03.347 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:03.347 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.347 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.347 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:334 3.ds0 3:b1a14543:::smithi18925896-15:head [setxattr _header (56) in=63b] snapc b=[b,4] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-18T10:15:03.347 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:02.793164+0000", 2024-09-18T10:15:03.347 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.44998423599999998, 2024-09-18T10:15:03.347 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.408134104, 2024-09-18T10:15:03.347 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:03.347 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:03.348 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:03.348 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:03.348 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:03.348 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 334 2024-09-18T10:15:03.348 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.348 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:03.348 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.348 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:03.348 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.793164+0000", 2024-09-18T10:15:03.348 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:03.348 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.348 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.349 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:03.349 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.793164+0000", 2024-09-18T10:15:03.349 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:03.349 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.349 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.349 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:03.349 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.793163+0000", 2024-09-18T10:15:03.349 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:03.349 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.349 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.349 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:03.349 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.793170+0000", 2024-09-18T10:15:03.349 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3050000000000004e-06 2024-09-18T10:15:03.350 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.350 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.350 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:03.350 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.793180+0000", 2024-09-18T10:15:03.350 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8069999999999994e-06 2024-09-18T10:15:03.350 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.350 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.350 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:03.350 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.793187+0000", 2024-09-18T10:15:03.350 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6879999999999999e-06 2024-09-18T10:15:03.350 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.350 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.350 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:03.351 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.793293+0000", 2024-09-18T10:15:03.351 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000105441 2024-09-18T10:15:03.351 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.351 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.351 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:03.351 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.793302+0000", 2024-09-18T10:15:03.351 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.5769999999999994e-06 2024-09-18T10:15:03.351 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.351 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.351 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:03.351 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.802703+0000", 2024-09-18T10:15:03.351 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0094011400000000005 2024-09-18T10:15:03.351 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.351 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.352 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:03.352 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.802949+0000", 2024-09-18T10:15:03.352 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00024604500000000002 2024-09-18T10:15:03.352 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.352 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.352 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:03.352 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.803628+0000", 2024-09-18T10:15:03.352 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000679391 2024-09-18T10:15:03.352 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.352 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.352 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:03.352 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.804224+0000", 2024-09-18T10:15:03.352 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00059597000000000003 2024-09-18T10:15:03.352 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.353 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.353 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:15:03.353 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:03.201019+0000", 2024-09-18T10:15:03.353 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.39679527799999997 2024-09-18T10:15:03.353 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:03.353 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:03.353 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:03.353 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:03.201298+0000", 2024-09-18T10:15:03.353 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00027909199999999999 2024-09-18T10:15:03.353 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:03.353 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:03.353 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:03.353 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:03.354 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:03.354 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-18T10:15:03.355 INFO:tasks.rados.rados.0.smithi189.stdout:130: finishing write tid 4 to smithi18925896-15 2024-09-18T10:15:03.355 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 15 v 10 (ObjNum 66 snap 6 seq_num 66) dirty exists 2024-09-18T10:15:03.355 INFO:tasks.rados.rados.0.smithi189.stdout:130: left oid 15 (ObjNum 66 snap 6 seq_num 66) 2024-09-18T10:15:03.356 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T10:15:03.356 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-18T10:15:03.353+0000 7f14c6a1a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 22138) UID: 0 2024-09-18T10:15:03.407 INFO:tasks.rados.rados.0.smithi189.stdout:134: finishing write tid 2 to smithi18925896-39 2024-09-18T10:15:03.408 INFO:tasks.rados.rados.0.smithi189.stdout:134: finishing write tid 3 to smithi18925896-39 2024-09-18T10:15:03.456 INFO:tasks.ceph.osd.14.smithi154.stderr:2024-09-18T10:15:03.451+0000 7f75a19e7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 22093) UID: 0 2024-09-18T10:15:03.465 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-09-18T10:15:03.466 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-09-18T10:15:03.466 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-09-18T10:15:03.466 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-09-18T10:15:03.475 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T10:15:03.544 INFO:tasks.rados.rados.0.smithi189.stdout:128: done (7 left) 2024-09-18T10:15:03.544 INFO:tasks.rados.rados.0.smithi189.stdout:129: done (6 left) 2024-09-18T10:15:03.545 INFO:tasks.rados.rados.0.smithi189.stdout:130: done (5 left) 2024-09-18T10:15:03.545 INFO:tasks.rados.rados.0.smithi189.stdout:132: done (4 left) 2024-09-18T10:15:03.545 INFO:tasks.rados.rados.0.smithi189.stdout:133: done (3 left) 2024-09-18T10:15:03.545 INFO:tasks.rados.rados.0.smithi189.stdout:135: done (2 left) 2024-09-18T10:15:03.545 INFO:tasks.rados.rados.0.smithi189.stdout:136: read oid 38 snap 5 2024-09-18T10:15:03.545 INFO:tasks.rados.rados.0.smithi189.stdout:136: expect (ObjNum 60 snap 5 seq_num 60) 2024-09-18T10:15:03.545 INFO:tasks.rados.rados.0.smithi189.stdout:137: read oid 25 snap 6 2024-09-18T10:15:03.545 INFO:tasks.rados.rados.0.smithi189.stdout:137: expect (ObjNum 50 snap 0 seq_num 50) 2024-09-18T10:15:03.545 INFO:tasks.rados.rados.0.smithi189.stdout:138: delete oid 17 current snap is 7 2024-09-18T10:15:03.557 INFO:tasks.ceph.osd.8.smithi029.stderr:2024-09-18T10:15:03.550+0000 7f2555c27700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 22192) UID: 0 2024-09-18T10:15:03.575 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-09-18T10:15:03.575 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-09-18T10:15:03.575 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-09-18T10:15:03.576 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-09-18T10:15:03.576 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-09-18T10:15:03.585 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T10:15:03.617 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T10:15:03.665 INFO:tasks.ceph.osd.4.smithi029.stderr:2024-09-18T10:15:03.662+0000 7f550cf6e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22189) UID: 0 2024-09-18T10:15:03.685 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-09-18T10:15:03.685 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-09-18T10:15:03.685 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-09-18T10:15:03.685 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-09-18T10:15:03.685 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.685 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.9s1 36/32 ECSubWrite(tid=10, reqid=client.4632.0:112, at_version=36'7, trim_to=0'0, roll_forward_to=35'6))", 2024-09-18T10:15:03.685 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:50.718230+0000", 2024-09-18T10:15:03.685 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 12.964780532000001, 2024-09-18T10:15:03.685 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.103673577, 2024-09-18T10:15:03.686 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:03.686 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:15:03.686 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:03.686 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.686 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:03.686 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.718230+0000", 2024-09-18T10:15:03.686 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:03.686 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.686 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.686 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:03.687 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.718230+0000", 2024-09-18T10:15:03.687 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:03.687 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.687 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.687 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:03.687 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.718229+0000", 2024-09-18T10:15:03.687 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:03.687 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.687 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.687 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:03.687 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.718513+0000", 2024-09-18T10:15:03.687 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000284405 2024-09-18T10:15:03.687 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.688 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.688 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:03.688 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.718544+0000", 2024-09-18T10:15:03.688 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1210000000000001e-05 2024-09-18T10:15:03.688 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.688 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.688 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:03.688 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.818638+0000", 2024-09-18T10:15:03.688 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10009378300000001 2024-09-18T10:15:03.688 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.688 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.688 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:03.688 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.818774+0000", 2024-09-18T10:15:03.689 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000135986 2024-09-18T10:15:03.689 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.689 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.689 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:03.689 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.818810+0000", 2024-09-18T10:15:03.689 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.6025000000000002e-05 2024-09-18T10:15:03.689 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.689 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.689 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:03.689 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.821869+0000", 2024-09-18T10:15:03.689 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030589049999999998 2024-09-18T10:15:03.690 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.690 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.690 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:03.690 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.821904+0000", 2024-09-18T10:15:03.690 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.4921e-05 2024-09-18T10:15:03.690 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:03.690 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:03.690 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:03.690 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.690 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.690 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.6s3 36/32 ECSubWrite(tid=13, reqid=client.4632.0:80, at_version=36'8, trim_to=0'0, roll_forward_to=36'7))", 2024-09-18T10:15:03.690 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:50.804050+0000", 2024-09-18T10:15:03.690 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 12.878961370000001, 2024-09-18T10:15:03.691 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.102659627, 2024-09-18T10:15:03.691 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:03.691 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:15:03.691 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:03.691 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.691 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:03.691 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.804050+0000", 2024-09-18T10:15:03.691 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:03.691 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.691 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.691 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:03.691 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.804050+0000", 2024-09-18T10:15:03.691 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:03.691 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.692 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.692 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:03.692 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.804048+0000", 2024-09-18T10:15:03.692 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:03.692 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.692 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.692 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:03.692 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.804175+0000", 2024-09-18T10:15:03.692 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00012670199999999999 2024-09-18T10:15:03.692 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.692 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.692 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:03.692 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.804212+0000", 2024-09-18T10:15:03.693 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.7033000000000003e-05 2024-09-18T10:15:03.693 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.693 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.693 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:03.693 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.804225+0000", 2024-09-18T10:15:03.693 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3185e-05 2024-09-18T10:15:03.693 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.693 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.693 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:03.693 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.804775+0000", 2024-09-18T10:15:03.693 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00055026999999999995 2024-09-18T10:15:03.693 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.693 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.693 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:03.694 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.804790+0000", 2024-09-18T10:15:03.694 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4538999999999999e-05 2024-09-18T10:15:03.694 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.694 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.694 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:03.694 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.906639+0000", 2024-09-18T10:15:03.694 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10184871600000001 2024-09-18T10:15:03.694 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.694 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.694 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:03.694 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.906709+0000", 2024-09-18T10:15:03.694 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0660000000000004e-05 2024-09-18T10:15:03.694 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:03.694 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:03.695 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:03.695 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.695 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.695 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.6s3 36/32 ECSubWrite(tid=14, reqid=client.4632.0:121, at_version=36'9, trim_to=0'0, roll_forward_to=36'7))", 2024-09-18T10:15:03.695 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:50.804306+0000", 2024-09-18T10:15:03.695 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 12.878704962, 2024-09-18T10:15:03.695 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.103984513, 2024-09-18T10:15:03.695 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:03.695 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:15:03.695 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:03.695 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.695 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:03.695 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.804306+0000", 2024-09-18T10:15:03.695 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:03.696 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.696 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.696 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:03.696 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.804306+0000", 2024-09-18T10:15:03.696 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:03.696 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.696 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.696 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:03.696 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.804305+0000", 2024-09-18T10:15:03.696 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:15:03.696 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.696 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.696 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:03.696 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.804579+0000", 2024-09-18T10:15:03.697 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00027352300000000002 2024-09-18T10:15:03.697 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.697 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.697 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:03.697 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.804613+0000", 2024-09-18T10:15:03.697 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.4329999999999998e-05 2024-09-18T10:15:03.697 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.697 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.697 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:03.697 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.804625+0000", 2024-09-18T10:15:03.697 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2335e-05 2024-09-18T10:15:03.697 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.697 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.697 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:03.698 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.904498+0000", 2024-09-18T10:15:03.698 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.099872796999999999 2024-09-18T10:15:03.698 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.698 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.698 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:03.698 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.904531+0000", 2024-09-18T10:15:03.698 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.3182000000000002e-05 2024-09-18T10:15:03.698 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.698 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.698 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:03.698 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.908265+0000", 2024-09-18T10:15:03.699 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0037334149999999999 2024-09-18T10:15:03.699 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.699 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.699 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:03.699 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.908291+0000", 2024-09-18T10:15:03.699 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5741e-05 2024-09-18T10:15:03.699 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:03.699 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:03.699 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:03.699 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.699 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.699 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.6s3 36/32 ECSubWrite(tid=15, reqid=client.4632.0:122, at_version=36'10, trim_to=0'0, roll_forward_to=36'7))", 2024-09-18T10:15:03.699 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:50.804653+0000", 2024-09-18T10:15:03.699 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 12.878357637000001, 2024-09-18T10:15:03.700 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.108947892, 2024-09-18T10:15:03.700 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:03.700 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:15:03.700 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:03.700 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.700 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:03.700 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.804653+0000", 2024-09-18T10:15:03.700 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:03.700 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.700 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.700 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:03.700 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.804653+0000", 2024-09-18T10:15:03.701 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:03.701 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.701 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.701 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:03.701 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.804653+0000", 2024-09-18T10:15:03.701 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:15:03.701 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.701 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.701 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:03.701 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.804677+0000", 2024-09-18T10:15:03.701 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4043e-05 2024-09-18T10:15:03.701 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.701 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.701 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:03.702 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.804691+0000", 2024-09-18T10:15:03.702 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.402e-05 2024-09-18T10:15:03.702 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.702 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.702 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:03.702 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.906029+0000", 2024-09-18T10:15:03.702 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.101337911 2024-09-18T10:15:03.702 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.702 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.702 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:03.702 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.906208+0000", 2024-09-18T10:15:03.702 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000179292 2024-09-18T10:15:03.703 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.703 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.703 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:03.703 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.906232+0000", 2024-09-18T10:15:03.703 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.3739000000000001e-05 2024-09-18T10:15:03.703 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.703 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.703 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:03.703 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.913534+0000", 2024-09-18T10:15:03.703 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0073023029999999996 2024-09-18T10:15:03.703 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.703 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.703 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:03.703 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.913601+0000", 2024-09-18T10:15:03.704 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.7314e-05 2024-09-18T10:15:03.704 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:03.704 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:03.704 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:03.704 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.704 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.704 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.9s1 36/32 ECSubWrite(tid=16, reqid=client.4632.0:143, at_version=36'12, trim_to=0'0, roll_forward_to=36'9))", 2024-09-18T10:15:03.704 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:51.130949+0000", 2024-09-18T10:15:03.704 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 12.552061873, 2024-09-18T10:15:03.704 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.202020333, 2024-09-18T10:15:03.704 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:03.704 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:15:03.704 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:03.705 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.705 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:03.705 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.130949+0000", 2024-09-18T10:15:03.705 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:03.705 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.705 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.705 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:03.705 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.130949+0000", 2024-09-18T10:15:03.705 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:03.705 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.706 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.706 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:03.706 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.130947+0000", 2024-09-18T10:15:03.706 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:03.706 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.706 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.706 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:03.706 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.131149+0000", 2024-09-18T10:15:03.706 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00020114800000000001 2024-09-18T10:15:03.706 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.706 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.706 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:03.706 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.131182+0000", 2024-09-18T10:15:03.707 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.3844000000000003e-05 2024-09-18T10:15:03.707 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.707 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.707 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:03.707 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.131204+0000", 2024-09-18T10:15:03.707 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.1753999999999999e-05 2024-09-18T10:15:03.707 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.707 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.707 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:03.707 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.231367+0000", 2024-09-18T10:15:03.707 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100162472 2024-09-18T10:15:03.707 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.707 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.708 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:03.708 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.231380+0000", 2024-09-18T10:15:03.708 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3624e-05 2024-09-18T10:15:03.708 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.708 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.708 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:03.708 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.332934+0000", 2024-09-18T10:15:03.708 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.101553714 2024-09-18T10:15:03.708 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.708 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.708 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:03.708 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.332969+0000", 2024-09-18T10:15:03.708 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5525999999999998e-05 2024-09-18T10:15:03.709 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:03.709 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:03.709 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:03.709 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.709 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.709 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.9s1 36/32 ECSubWrite(tid=17, reqid=client.4632.0:144, at_version=36'13, trim_to=0'0, roll_forward_to=36'9))", 2024-09-18T10:15:03.709 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:51.131916+0000", 2024-09-18T10:15:03.709 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 12.551095046, 2024-09-18T10:15:03.709 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20110955899999999, 2024-09-18T10:15:03.709 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:03.709 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:15:03.709 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:03.709 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.710 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:03.710 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.131916+0000", 2024-09-18T10:15:03.710 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:03.710 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.710 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.710 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:03.710 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.131916+0000", 2024-09-18T10:15:03.710 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:03.710 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.710 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.710 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:03.710 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.131915+0000", 2024-09-18T10:15:03.710 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:03.711 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.711 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.711 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:03.711 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.131949+0000", 2024-09-18T10:15:03.711 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.4576999999999998e-05 2024-09-18T10:15:03.711 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.711 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.711 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:03.711 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.131979+0000", 2024-09-18T10:15:03.711 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9454e-05 2024-09-18T10:15:03.711 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.712 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.712 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:03.712 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.131992+0000", 2024-09-18T10:15:03.712 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3029e-05 2024-09-18T10:15:03.712 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.712 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.712 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:03.712 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.332112+0000", 2024-09-18T10:15:03.712 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.200120295 2024-09-18T10:15:03.712 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.712 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.712 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:03.712 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.332126+0000", 2024-09-18T10:15:03.713 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4494999999999999e-05 2024-09-18T10:15:03.713 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.713 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.713 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:03.713 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.332996+0000", 2024-09-18T10:15:03.713 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00086989000000000001 2024-09-18T10:15:03.713 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.713 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.713 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:03.713 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.333026+0000", 2024-09-18T10:15:03.713 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9159999999999999e-05 2024-09-18T10:15:03.713 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:03.713 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:03.713 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:03.714 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.714 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.714 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpRead(3.4s3 36/32 ECSubRead(tid=11, to_read={3:2650be24:::smithi18925896-45:head=0,4096,0}, subchunks={3:2650be24:::smithi18925896-45:head=[0,1]}, attrs_to_read=))", 2024-09-18T10:15:03.714 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:51.425236+0000", 2024-09-18T10:15:03.714 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 12.257774959000001, 2024-09-18T10:15:03.714 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.200812289, 2024-09-18T10:15:03.714 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:03.714 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:15:03.714 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:03.714 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.714 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:03.714 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.425236+0000", 2024-09-18T10:15:03.714 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:03.714 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.715 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.715 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:03.715 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.425236+0000", 2024-09-18T10:15:03.715 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:03.715 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.715 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.715 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:03.715 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.425234+0000", 2024-09-18T10:15:03.715 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:03.715 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.715 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.715 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:03.715 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.425253+0000", 2024-09-18T10:15:03.716 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8468999999999999e-05 2024-09-18T10:15:03.716 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.716 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.716 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:03.716 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.425289+0000", 2024-09-18T10:15:03.716 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.6514999999999999e-05 2024-09-18T10:15:03.716 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.716 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.716 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:03.716 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.525424+0000", 2024-09-18T10:15:03.716 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100135157 2024-09-18T10:15:03.716 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.716 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.717 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:03.717 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.625672+0000", 2024-09-18T10:15:03.717 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100247555 2024-09-18T10:15:03.717 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.717 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.717 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:03.717 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.626048+0000", 2024-09-18T10:15:03.717 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00037635000000000001 2024-09-18T10:15:03.717 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:03.717 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:03.717 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:03.717 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.717 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.717 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.9s1 37/32 ECSubWrite(tid=26, reqid=client.4632.0:185, at_version=0'0, trim_to=0'0, roll_forward_to=37'15))", 2024-09-18T10:15:03.718 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:52.799766+0000", 2024-09-18T10:15:03.718 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 10.88324523, 2024-09-18T10:15:03.718 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10681106, 2024-09-18T10:15:03.718 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:03.718 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:15:03.718 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:03.718 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.718 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:03.718 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.799766+0000", 2024-09-18T10:15:03.718 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:03.718 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.718 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.718 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:03.719 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.799766+0000", 2024-09-18T10:15:03.719 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:03.719 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.719 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.719 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:03.719 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.799764+0000", 2024-09-18T10:15:03.719 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:03.719 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.719 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.719 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:03.719 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.799789+0000", 2024-09-18T10:15:03.719 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4335e-05 2024-09-18T10:15:03.719 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.719 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.720 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:03.720 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.799822+0000", 2024-09-18T10:15:03.720 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.3565999999999997e-05 2024-09-18T10:15:03.720 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.720 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.720 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:03.720 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.799841+0000", 2024-09-18T10:15:03.720 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.9272000000000001e-05 2024-09-18T10:15:03.720 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.720 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.720 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:03.720 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.903502+0000", 2024-09-18T10:15:03.720 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10366048999999999 2024-09-18T10:15:03.721 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.721 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.721 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:03.721 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.903526+0000", 2024-09-18T10:15:03.721 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.3958999999999999e-05 2024-09-18T10:15:03.721 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.721 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.721 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:03.721 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.906532+0000", 2024-09-18T10:15:03.721 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030064279999999998 2024-09-18T10:15:03.721 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.721 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.721 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:03.721 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.906577+0000", 2024-09-18T10:15:03.722 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4573000000000002e-05 2024-09-18T10:15:03.722 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:03.722 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:03.722 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:03.722 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.722 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.722 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.4s3 37/32 ECSubWrite(tid=12, reqid=client.4632.0:188, at_version=37'8, trim_to=0'0, roll_forward_to=36'6))", 2024-09-18T10:15:03.722 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:52.801107+0000", 2024-09-18T10:15:03.722 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 10.881903785, 2024-09-18T10:15:03.722 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.102732592, 2024-09-18T10:15:03.722 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:03.722 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:15:03.722 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:03.723 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.723 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:03.723 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.801107+0000", 2024-09-18T10:15:03.723 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:03.723 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.723 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.723 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:03.723 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.801107+0000", 2024-09-18T10:15:03.723 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:03.723 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.723 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.723 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:03.723 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.801107+0000", 2024-09-18T10:15:03.724 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:03.724 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.724 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.724 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:03.724 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.801336+0000", 2024-09-18T10:15:03.724 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00022954 2024-09-18T10:15:03.724 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.724 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.724 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:03.724 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.801390+0000", 2024-09-18T10:15:03.724 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.3634000000000002e-05 2024-09-18T10:15:03.724 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.724 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.725 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:03.725 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.801414+0000", 2024-09-18T10:15:03.725 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4189000000000002e-05 2024-09-18T10:15:03.725 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.725 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.725 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:03.725 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.901637+0000", 2024-09-18T10:15:03.725 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100223012 2024-09-18T10:15:03.725 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.725 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.725 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:03.725 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.901666+0000", 2024-09-18T10:15:03.725 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8929999999999999e-05 2024-09-18T10:15:03.726 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.726 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.726 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:03.726 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.903775+0000", 2024-09-18T10:15:03.726 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0021094989999999999 2024-09-18T10:15:03.726 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.726 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.726 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:03.726 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.903840+0000", 2024-09-18T10:15:03.726 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.4478000000000002e-05 2024-09-18T10:15:03.726 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:03.726 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:03.726 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:03.726 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.727 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.727 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.9s1 37/32 ECSubWrite(tid=27, reqid=client.4632.0:186, at_version=37'16, trim_to=0'0, roll_forward_to=37'15))", 2024-09-18T10:15:03.727 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:52.803227+0000", 2024-09-18T10:15:03.727 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 10.879784074, 2024-09-18T10:15:03.727 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.103974409, 2024-09-18T10:15:03.727 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:03.727 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:15:03.727 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:03.727 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.727 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:03.727 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.803227+0000", 2024-09-18T10:15:03.727 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:03.727 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.728 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.728 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:03.728 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.803227+0000", 2024-09-18T10:15:03.728 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:03.728 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.728 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.728 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:03.728 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.803226+0000", 2024-09-18T10:15:03.728 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:15:03.728 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.728 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.728 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:03.728 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.803256+0000", 2024-09-18T10:15:03.728 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0083999999999999e-05 2024-09-18T10:15:03.729 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.729 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.729 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:03.729 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.803275+0000", 2024-09-18T10:15:03.729 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8882999999999999e-05 2024-09-18T10:15:03.729 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.729 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.729 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:03.729 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.803291+0000", 2024-09-18T10:15:03.729 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.6500000000000001e-05 2024-09-18T10:15:03.729 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.729 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.729 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:03.730 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.903965+0000", 2024-09-18T10:15:03.730 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100673314 2024-09-18T10:15:03.730 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.730 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.730 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:03.730 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.903981+0000", 2024-09-18T10:15:03.730 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.6337e-05 2024-09-18T10:15:03.730 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.730 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.730 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:03.730 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.907161+0000", 2024-09-18T10:15:03.730 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0031794549999999999 2024-09-18T10:15:03.730 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.731 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.731 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:03.731 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.907201+0000", 2024-09-18T10:15:03.731 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.0825999999999998e-05 2024-09-18T10:15:03.731 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:03.731 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:03.731 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:03.731 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.731 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.731 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.9s1 37/32 ECSubWrite(tid=28, reqid=client.4632.0:187, at_version=37'17, trim_to=0'0, roll_forward_to=37'15))", 2024-09-18T10:15:03.731 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:52.804637+0000", 2024-09-18T10:15:03.731 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 10.878374206, 2024-09-18T10:15:03.731 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.102583991, 2024-09-18T10:15:03.732 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:03.732 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:15:03.732 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:03.732 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.732 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:03.732 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.804637+0000", 2024-09-18T10:15:03.732 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:03.732 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.732 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.732 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:03.732 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.804637+0000", 2024-09-18T10:15:03.732 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:03.732 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.733 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.733 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:03.733 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.804636+0000", 2024-09-18T10:15:03.733 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:15:03.733 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.733 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.733 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:03.733 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.804661+0000", 2024-09-18T10:15:03.733 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5621000000000001e-05 2024-09-18T10:15:03.733 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.733 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.733 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:03.733 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.804682+0000", 2024-09-18T10:15:03.733 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.0089e-05 2024-09-18T10:15:03.734 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.734 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.734 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:03.734 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.804689+0000", 2024-09-18T10:15:03.734 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.8290000000000008e-06 2024-09-18T10:15:03.734 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.734 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.734 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:03.734 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.904578+0000", 2024-09-18T10:15:03.734 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.099888633000000004 2024-09-18T10:15:03.734 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.734 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.734 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:03.735 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.904606+0000", 2024-09-18T10:15:03.735 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.7759e-05 2024-09-18T10:15:03.735 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.735 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.735 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:03.735 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.907206+0000", 2024-09-18T10:15:03.735 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0026003440000000001 2024-09-18T10:15:03.735 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.735 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.735 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:03.735 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.907221+0000", 2024-09-18T10:15:03.735 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4728e-05 2024-09-18T10:15:03.735 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:03.735 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:03.736 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:03.736 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.736 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.736 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.9s1 37/32 ECSubWrite(tid=30, reqid=client.4632.0:187, at_version=0'0, trim_to=0'0, roll_forward_to=37'17))", 2024-09-18T10:15:03.736 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:53.090512+0000", 2024-09-18T10:15:03.736 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 10.592499105, 2024-09-18T10:15:03.736 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.21478031, 2024-09-18T10:15:03.736 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:03.736 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:15:03.736 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:03.736 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.736 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:03.736 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.090512+0000", 2024-09-18T10:15:03.737 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:03.737 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.737 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.737 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:03.737 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.090512+0000", 2024-09-18T10:15:03.737 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:03.737 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.737 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.737 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:03.737 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.090510+0000", 2024-09-18T10:15:03.737 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:03.737 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.737 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.737 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:03.738 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.090535+0000", 2024-09-18T10:15:03.738 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4848999999999999e-05 2024-09-18T10:15:03.738 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.738 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.738 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:03.738 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.090566+0000", 2024-09-18T10:15:03.738 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1096000000000002e-05 2024-09-18T10:15:03.738 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.738 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.738 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:03.738 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.190713+0000", 2024-09-18T10:15:03.738 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10014709300000001 2024-09-18T10:15:03.738 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.738 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.739 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:03.739 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.290966+0000", 2024-09-18T10:15:03.739 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10025297399999999 2024-09-18T10:15:03.739 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.739 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.739 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:03.739 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.290981+0000", 2024-09-18T10:15:03.739 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4960000000000001e-05 2024-09-18T10:15:03.739 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.739 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.739 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:03.739 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.305252+0000", 2024-09-18T10:15:03.739 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.014270738999999999 2024-09-18T10:15:03.740 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.740 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.740 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:03.740 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.305292+0000", 2024-09-18T10:15:03.740 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.0092000000000002e-05 2024-09-18T10:15:03.740 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:03.740 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:03.740 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:03.740 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.740 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.740 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.9s1 37/32 ECSubWrite(tid=31, reqid=client.4632.0:202, at_version=37'19, trim_to=0'0, roll_forward_to=37'17))", 2024-09-18T10:15:03.740 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:53.203769+0000", 2024-09-18T10:15:03.740 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 10.479242113, 2024-09-18T10:15:03.740 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.102550108, 2024-09-18T10:15:03.741 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:03.741 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:15:03.741 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:03.741 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.741 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:03.741 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.203769+0000", 2024-09-18T10:15:03.741 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:03.741 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.741 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.741 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:03.741 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.203769+0000", 2024-09-18T10:15:03.741 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:03.741 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.741 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.742 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:03.742 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.203767+0000", 2024-09-18T10:15:03.742 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:03.742 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.742 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.742 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:03.742 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.203917+0000", 2024-09-18T10:15:03.742 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000149491 2024-09-18T10:15:03.742 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.742 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.742 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:03.742 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.203961+0000", 2024-09-18T10:15:03.742 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4317999999999998e-05 2024-09-18T10:15:03.743 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.743 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.743 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:03.743 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.203979+0000", 2024-09-18T10:15:03.743 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8238999999999999e-05 2024-09-18T10:15:03.743 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.743 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.743 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:03.743 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.304282+0000", 2024-09-18T10:15:03.743 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10030295 2024-09-18T10:15:03.743 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.743 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.743 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:03.744 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.304297+0000", 2024-09-18T10:15:03.744 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4452e-05 2024-09-18T10:15:03.744 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.744 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.744 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:03.744 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.306253+0000", 2024-09-18T10:15:03.744 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019566969999999999 2024-09-18T10:15:03.744 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.744 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.744 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:03.744 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.306319+0000", 2024-09-18T10:15:03.744 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.5613999999999999e-05 2024-09-18T10:15:03.744 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:03.745 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:03.745 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:03.745 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.745 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.745 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.9s1 41/40 ECSubWrite(tid=39, reqid=osd.7.0:39, at_version=41'27, trim_to=0'0, roll_forward_to=0'0))", 2024-09-18T10:15:03.745 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:57.048262+0000", 2024-09-18T10:15:03.745 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 6.6347491410000003, 2024-09-18T10:15:03.745 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.104294998, 2024-09-18T10:15:03.745 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:03.745 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:15:03.745 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:03.745 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.745 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:03.746 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.048262+0000", 2024-09-18T10:15:03.746 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:03.746 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.746 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.746 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:03.746 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.048262+0000", 2024-09-18T10:15:03.746 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:03.746 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.746 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.746 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:03.746 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.048261+0000", 2024-09-18T10:15:03.746 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:15:03.746 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.746 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.747 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:03.747 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.048303+0000", 2024-09-18T10:15:03.747 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.2345999999999997e-05 2024-09-18T10:15:03.747 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.747 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.747 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:03.747 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.048328+0000", 2024-09-18T10:15:03.747 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5148999999999999e-05 2024-09-18T10:15:03.747 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.747 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.747 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:03.747 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.048337+0000", 2024-09-18T10:15:03.748 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.7499999999999992e-06 2024-09-18T10:15:03.748 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.748 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.748 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:03.748 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.148452+0000", 2024-09-18T10:15:03.748 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10011441 2024-09-18T10:15:03.748 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.748 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.748 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:03.748 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.148460+0000", 2024-09-18T10:15:03.748 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.9349999999999994e-06 2024-09-18T10:15:03.748 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.748 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.748 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:03.749 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.152541+0000", 2024-09-18T10:15:03.749 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0040818210000000002 2024-09-18T10:15:03.749 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.749 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.749 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:03.749 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.152557+0000", 2024-09-18T10:15:03.749 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.5553000000000002e-05 2024-09-18T10:15:03.749 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:03.749 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:03.749 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:03.749 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.749 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.749 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.6s3 41/40 ECSubWrite(tid=31, reqid=client.4632.0:249, at_version=41'13, trim_to=0'0, roll_forward_to=0'0))", 2024-09-18T10:15:03.750 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:57.550796+0000", 2024-09-18T10:15:03.750 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 6.1322148490000004, 2024-09-18T10:15:03.750 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10342707900000001, 2024-09-18T10:15:03.750 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:03.750 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:15:03.750 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:03.750 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.750 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:03.750 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.550796+0000", 2024-09-18T10:15:03.750 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:03.750 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.750 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.750 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:03.751 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.550796+0000", 2024-09-18T10:15:03.751 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:03.751 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.751 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.751 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:03.751 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.550796+0000", 2024-09-18T10:15:03.751 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:03.751 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.751 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.751 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:03.751 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.551128+0000", 2024-09-18T10:15:03.751 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00033246399999999998 2024-09-18T10:15:03.751 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.751 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.752 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:03.752 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.551143+0000", 2024-09-18T10:15:03.752 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4997000000000001e-05 2024-09-18T10:15:03.752 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.752 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.752 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:03.752 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.551153+0000", 2024-09-18T10:15:03.752 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.5440000000000004e-06 2024-09-18T10:15:03.752 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.752 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.752 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:03.752 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.651391+0000", 2024-09-18T10:15:03.752 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10023789299999999 2024-09-18T10:15:03.752 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.753 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.753 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:03.753 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.651418+0000", 2024-09-18T10:15:03.753 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.7478999999999999e-05 2024-09-18T10:15:03.753 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.753 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.753 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:03.753 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.654193+0000", 2024-09-18T10:15:03.753 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0027751989999999999 2024-09-18T10:15:03.753 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.753 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.753 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:03.753 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.654223+0000", 2024-09-18T10:15:03.754 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9955e-05 2024-09-18T10:15:03.754 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:03.754 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:03.754 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:03.754 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.754 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.754 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.6s3 41/40 ECSubWrite(tid=32, reqid=client.4632.0:250, at_version=41'14, trim_to=0'0, roll_forward_to=0'0))", 2024-09-18T10:15:03.754 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:57.552410+0000", 2024-09-18T10:15:03.754 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 6.1306010000000004, 2024-09-18T10:15:03.754 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20275494299999999, 2024-09-18T10:15:03.754 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:03.754 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:15:03.754 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:03.755 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.755 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:03.755 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.552410+0000", 2024-09-18T10:15:03.755 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:03.755 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.755 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.755 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:03.755 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.552410+0000", 2024-09-18T10:15:03.755 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:03.755 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.755 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.755 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:03.755 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.552409+0000", 2024-09-18T10:15:03.755 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:03.756 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.756 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.756 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:03.756 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.552478+0000", 2024-09-18T10:15:03.756 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.9084999999999993e-05 2024-09-18T10:15:03.756 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.756 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.756 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:03.756 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.552488+0000", 2024-09-18T10:15:03.756 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.7990000000000005e-06 2024-09-18T10:15:03.756 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.756 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.756 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:03.757 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.552496+0000", 2024-09-18T10:15:03.757 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.3200000000000002e-06 2024-09-18T10:15:03.757 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.757 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.757 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:03.757 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.653130+0000", 2024-09-18T10:15:03.757 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100634549 2024-09-18T10:15:03.757 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.757 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.757 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:03.757 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.653144+0000", 2024-09-18T10:15:03.757 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3619e-05 2024-09-18T10:15:03.757 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.757 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.758 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:03.758 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.755128+0000", 2024-09-18T10:15:03.758 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.101983924 2024-09-18T10:15:03.758 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.758 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.758 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:03.758 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.755165+0000", 2024-09-18T10:15:03.758 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.7361999999999999e-05 2024-09-18T10:15:03.758 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:03.758 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:03.758 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:03.758 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.758 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.759 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.6s3 41/40 ECSubWrite(tid=33, reqid=client.4632.0:251, at_version=41'15, trim_to=0'0, roll_forward_to=0'0))", 2024-09-18T10:15:03.759 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:57.553805+0000", 2024-09-18T10:15:03.759 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 6.12920642, 2024-09-18T10:15:03.759 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20199056600000001, 2024-09-18T10:15:03.759 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:03.759 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:15:03.759 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:03.759 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.759 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:03.759 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.553805+0000", 2024-09-18T10:15:03.759 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:03.759 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.759 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.760 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:03.760 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.553805+0000", 2024-09-18T10:15:03.760 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:03.760 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.760 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.760 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:03.760 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.553804+0000", 2024-09-18T10:15:03.760 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:15:03.760 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.760 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.760 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:03.760 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.553827+0000", 2024-09-18T10:15:03.760 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.3516e-05 2024-09-18T10:15:03.761 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.761 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.761 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:03.761 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.553845+0000", 2024-09-18T10:15:03.761 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7881999999999999e-05 2024-09-18T10:15:03.761 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.761 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.761 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:03.761 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.553853+0000", 2024-09-18T10:15:03.761 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.7109999999999995e-06 2024-09-18T10:15:03.761 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.761 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.761 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:03.762 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.754527+0000", 2024-09-18T10:15:03.762 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.200674198 2024-09-18T10:15:03.762 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.762 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.762 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:03.762 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.754543+0000", 2024-09-18T10:15:03.762 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.5682e-05 2024-09-18T10:15:03.762 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.762 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.762 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:03.762 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.755758+0000", 2024-09-18T10:15:03.762 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0012155860000000001 2024-09-18T10:15:03.762 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.762 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.763 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:03.763 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.755795+0000", 2024-09-18T10:15:03.763 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.6808999999999999e-05 2024-09-18T10:15:03.763 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:03.763 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:03.763 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:03.763 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.763 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.763 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.13s5 41/40 ECSubWrite(tid=19, reqid=osd.10.0:19, at_version=41'16, trim_to=0'0, roll_forward_to=0'0))", 2024-09-18T10:15:03.763 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:57.750351+0000", 2024-09-18T10:15:03.763 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 5.9326597599999999, 2024-09-18T10:15:03.763 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.103362123, 2024-09-18T10:15:03.763 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:03.764 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:15:03.764 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:03.764 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.764 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:03.764 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.750351+0000", 2024-09-18T10:15:03.764 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:03.764 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.764 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.764 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:03.764 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.750351+0000", 2024-09-18T10:15:03.764 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:03.764 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.764 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.765 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:03.765 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.750349+0000", 2024-09-18T10:15:03.765 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:03.765 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.765 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.765 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:03.765 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.750410+0000", 2024-09-18T10:15:03.765 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.0798000000000003e-05 2024-09-18T10:15:03.765 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.765 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.765 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:03.765 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.750442+0000", 2024-09-18T10:15:03.765 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1476000000000002e-05 2024-09-18T10:15:03.765 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.766 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.766 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:03.766 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.750459+0000", 2024-09-18T10:15:03.766 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7147999999999999e-05 2024-09-18T10:15:03.766 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.766 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.766 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:03.766 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.750545+0000", 2024-09-18T10:15:03.766 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.6331999999999999e-05 2024-09-18T10:15:03.766 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.766 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.766 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:03.766 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.750557+0000", 2024-09-18T10:15:03.766 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2312e-05 2024-09-18T10:15:03.767 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.767 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.767 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:03.767 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.853571+0000", 2024-09-18T10:15:03.767 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.103013681 2024-09-18T10:15:03.767 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.767 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.767 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:03.767 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.853713+0000", 2024-09-18T10:15:03.767 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00014226399999999999 2024-09-18T10:15:03.767 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:03.767 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:03.767 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:03.768 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.768 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.768 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.fs4 46/32 ECSubWrite(tid=53, reqid=client.4632.0:326, at_version=46'10, trim_to=0'0, roll_forward_to=45'8))", 2024-09-18T10:15:03.768 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:01.900725+0000", 2024-09-18T10:15:03.768 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 1.782286249, 2024-09-18T10:15:03.768 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10258081199999999, 2024-09-18T10:15:03.768 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:03.768 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:15:03.768 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:03.768 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.768 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:03.768 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:01.900725+0000", 2024-09-18T10:15:03.768 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:03.769 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.769 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.769 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:03.769 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:01.900725+0000", 2024-09-18T10:15:03.769 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:03.769 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.769 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.769 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:03.769 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:01.900723+0000", 2024-09-18T10:15:03.769 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:03.769 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.769 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.769 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:03.769 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:01.900796+0000", 2024-09-18T10:15:03.770 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.2971000000000003e-05 2024-09-18T10:15:03.770 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.770 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.770 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:03.770 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:01.900830+0000", 2024-09-18T10:15:03.770 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.4829000000000001e-05 2024-09-18T10:15:03.770 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.770 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.770 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:03.770 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:01.900848+0000", 2024-09-18T10:15:03.770 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7448999999999998e-05 2024-09-18T10:15:03.770 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.770 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.771 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:03.771 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:02.001066+0000", 2024-09-18T10:15:03.771 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100217915 2024-09-18T10:15:03.771 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.771 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.771 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:03.771 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:02.001084+0000", 2024-09-18T10:15:03.771 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8131999999999999e-05 2024-09-18T10:15:03.771 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.771 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.771 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:03.771 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:02.003237+0000", 2024-09-18T10:15:03.771 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.002152705 2024-09-18T10:15:03.771 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.772 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.772 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:03.772 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:02.003306+0000", 2024-09-18T10:15:03.772 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.8905000000000005e-05 2024-09-18T10:15:03.772 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:03.772 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:03.772 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:03.772 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.772 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.772 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.9s1 45/40 ECSubWrite(tid=60, reqid=client.4632.0:307, at_version=0'0, trim_to=0'0, roll_forward_to=45'31))", 2024-09-18T10:15:03.772 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:02.018294+0000", 2024-09-18T10:15:03.772 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 1.6647173500000001, 2024-09-18T10:15:03.772 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.103142807, 2024-09-18T10:15:03.772 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:03.773 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:15:03.773 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:03.773 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.773 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:03.773 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:02.018294+0000", 2024-09-18T10:15:03.773 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:03.773 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.773 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.773 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:03.773 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:02.018294+0000", 2024-09-18T10:15:03.773 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:03.773 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.773 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.773 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:03.774 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:02.018292+0000", 2024-09-18T10:15:03.774 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:03.774 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.774 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.774 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:03.774 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:02.018321+0000", 2024-09-18T10:15:03.774 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9196e-05 2024-09-18T10:15:03.774 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.774 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.774 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:03.774 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:02.018356+0000", 2024-09-18T10:15:03.774 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.4918e-05 2024-09-18T10:15:03.774 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.774 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.775 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:03.775 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:02.018374+0000", 2024-09-18T10:15:03.775 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8669999999999999e-05 2024-09-18T10:15:03.775 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.775 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.775 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:03.775 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:02.120117+0000", 2024-09-18T10:15:03.775 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.101742289 2024-09-18T10:15:03.775 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.775 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.775 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:03.775 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:02.120132+0000", 2024-09-18T10:15:03.775 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.5373e-05 2024-09-18T10:15:03.775 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.776 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.776 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:03.776 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:02.121372+0000", 2024-09-18T10:15:03.776 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001239483 2024-09-18T10:15:03.776 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:03.776 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:03.776 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:03.776 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:02.121436+0000", 2024-09-18T10:15:03.776 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.4848e-05 2024-09-18T10:15:03.776 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:03.776 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:03.776 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:03.776 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:03.776 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:03.777 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-09-18T10:15:03.778 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-18T10:15:03.778 INFO:tasks.rados.rados.0.smithi189.stdout:134: finishing write tid 4 to smithi18925896-39 2024-09-18T10:15:03.778 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 39 v 19 (ObjNum 68 snap 6 seq_num 68) dirty exists 2024-09-18T10:15:03.778 INFO:tasks.rados.rados.0.smithi189.stdout:134: left oid 39 (ObjNum 68 snap 6 seq_num 68) 2024-09-18T10:15:03.779 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T10:15:03.779 INFO:tasks.ceph.osd.14.smithi154.stderr:2024-09-18T10:15:03.771+0000 7f75a19e7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 22093) UID: 0 2024-09-18T10:15:03.791 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-18T10:15:03.800 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-18T10:15:03.823 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-18T10:15:03.831 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15] out_osds: [] dead_osds: [] live_osds: [0, 4, 8, 12, 1, 5, 9, 13, 2, 6, 10, 14, 3, 7, 11, 15] 2024-09-18T10:15:03.831 INFO:tasks.thrashosds.thrasher:choose_action: min_in 8 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T10:15:03.831 INFO:tasks.thrashosds.thrasher:inject_pause on osd.15 2024-09-18T10:15:03.831 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-18T10:15:03.831 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-18T10:15:03.832 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok config set filestore_inject_stall 3 2024-09-18T10:15:03.849 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-18T10:15:03.876 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-18T10:15:03.879 INFO:tasks.ceph.osd.6.smithi154.stderr:2024-09-18T10:15:03.875+0000 7f0abc9e1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22089) UID: 0 2024-09-18T10:15:03.897 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-18T10:15:03.922 INFO:teuthology.orchestra.run.smithi029.stdout:osd.6: {} 2024-09-18T10:15:03.936 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:03.936 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:03.940 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-18T10:15:03.940 INFO:teuthology.orchestra.run.smithi189.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-18T10:15:03.940 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-18T10:15:03.946 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T10:15:03.955 INFO:tasks.rados.rados.0.smithi189.stdout:137: expect (ObjNum 50 snap 0 seq_num 50) 2024-09-18T10:15:03.959 INFO:teuthology.orchestra.run.smithi029.stdout:osd.7: {} 2024-09-18T10:15:03.980 INFO:teuthology.orchestra.run.smithi029.stdout:osd.8: {} 2024-09-18T10:15:03.991 INFO:tasks.ceph.osd.10.smithi154.stderr:2024-09-18T10:15:03.987+0000 7fb73221a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 22091) UID: 0 2024-09-18T10:15:04.001 INFO:teuthology.orchestra.run.smithi029.stdout:osd.9: {} 2024-09-18T10:15:04.024 INFO:teuthology.orchestra.run.smithi029.stdout:osd.10: {} 2024-09-18T10:15:04.036 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:04.036 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:04.046 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T10:15:04.047 INFO:teuthology.orchestra.run.smithi029.stdout:osd.11: {} 2024-09-18T10:15:04.063 INFO:tasks.rados.rados.0.smithi189.stdout:136: expect (ObjNum 60 snap 5 seq_num 60) 2024-09-18T10:15:04.068 INFO:teuthology.orchestra.run.smithi029.stdout:osd.12: {} 2024-09-18T10:15:04.080 INFO:tasks.ceph.osd.15.smithi189.stderr:2024-09-18T10:15:04.079+0000 7fd0e725e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 22401) UID: 0 2024-09-18T10:15:04.143 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:04.144 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:04.152 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-09-18T10:15:04.162 INFO:tasks.rados.rados.0.smithi189.stdout:134: done (4 left) 2024-09-18T10:15:04.162 INFO:tasks.rados.rados.0.smithi189.stdout:136: done (3 left) 2024-09-18T10:15:04.163 INFO:tasks.rados.rados.0.smithi189.stdout:137: done (2 left) 2024-09-18T10:15:04.163 INFO:tasks.rados.rados.0.smithi189.stdout:138: done (1 left) 2024-09-18T10:15:04.163 INFO:tasks.rados.rados.0.smithi189.stdout:139: rollback oid 47 current snap is 7 2024-09-18T10:15:04.163 INFO:tasks.rados.rados.0.smithi189.stdout:rollback oid 47 to 5 2024-09-18T10:15:04.164 INFO:tasks.rados.rados.0.smithi189.stdout:140: read oid 13 snap -1 2024-09-18T10:15:04.164 INFO:tasks.rados.rados.0.smithi189.stdout:140: expect (ObjNum 12 snap 0 seq_num 12) 2024-09-18T10:15:04.164 INFO:tasks.rados.rados.0.smithi189.stdout:141: setattr oid 48 current snap is 7 2024-09-18T10:15:04.167 INFO:tasks.rados.rados.0.smithi189.stdout:142: delete oid 21 current snap is 7 2024-09-18T10:15:04.181 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-18T10:15:04.181+0000 7f14c6a1a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 22138) UID: 0 2024-09-18T10:15:04.192 INFO:teuthology.orchestra.run.smithi029.stdout:osd.13: {} 2024-09-18T10:15:04.213 INFO:teuthology.orchestra.run.smithi029.stdout:osd.14: {} 2024-09-18T10:15:04.237 INFO:teuthology.orchestra.run.smithi029.stdout:osd.15: {} 2024-09-18T10:15:04.239 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T10:15:04.240 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T10:15:04.240 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T10:15:04.240 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T10:15:04.240 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T10:15:04.240 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T10:15:04.240 INFO:teuthology.orchestra.run.smithi029.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T10:15:04.240 INFO:teuthology.orchestra.run.smithi029.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T10:15:04.240 INFO:teuthology.orchestra.run.smithi029.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-18T10:15:04.240 INFO:teuthology.orchestra.run.smithi029.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-18T10:15:04.240 INFO:teuthology.orchestra.run.smithi029.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-18T10:15:04.240 INFO:teuthology.orchestra.run.smithi029.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-18T10:15:04.240 INFO:teuthology.orchestra.run.smithi029.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-18T10:15:04.240 INFO:teuthology.orchestra.run.smithi029.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-18T10:15:04.241 INFO:teuthology.orchestra.run.smithi029.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-18T10:15:04.241 INFO:teuthology.orchestra.run.smithi029.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-18T10:15:04.251 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:04.251 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:04.260 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-09-18T10:15:04.282 INFO:tasks.ceph.osd.14.smithi154.stderr:2024-09-18T10:15:04.275+0000 7f75a19e7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 22093) UID: 0 2024-09-18T10:15:04.353 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:04.354 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:04.363 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-09-18T10:15:04.383 INFO:tasks.ceph.osd.15.smithi189.stderr:2024-09-18T10:15:04.379+0000 7fd0e725e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 22401) UID: 0 2024-09-18T10:15:04.484 INFO:tasks.ceph.osd.15.smithi189.stderr:2024-09-18T10:15:04.479+0000 7fd0e725e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 22401) UID: 0 2024-09-18T10:15:04.511 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:04.511 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:04.521 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_ops_in_flight 2024-09-18T10:15:04.546 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 48 v 33 (ObjNum 62 snap 5 seq_num 62) dirty exists 2024-09-18T10:15:04.553 INFO:teuthology.orchestra.run.smithi029.stderr:nodeep-scrub is unset 2024-09-18T10:15:04.570 INFO:tasks.rados.rados.0.smithi189.stdout:141: done (4 left) 2024-09-18T10:15:04.571 INFO:tasks.rados.rados.0.smithi189.stdout:142: done (3 left) 2024-09-18T10:15:04.571 INFO:tasks.rados.rados.0.smithi189.stdout:143: snap_remove snap 6 2024-09-18T10:15:04.585 INFO:tasks.ceph.osd.6.smithi154.stderr:2024-09-18T10:15:04.579+0000 7f0abc9e1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22089) UID: 0 2024-09-18T10:15:04.619 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:04.620 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:04.629 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_blocked_ops 2024-09-18T10:15:04.686 INFO:tasks.ceph.osd.10.smithi154.stderr:2024-09-18T10:15:04.679+0000 7fb73221a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 22091) UID: 0 2024-09-18T10:15:04.726 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:04.726 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:04.736 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_historic_ops 2024-09-18T10:15:04.786 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-18T10:15:04.783+0000 7f14dcc5b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22398) UID: 0 2024-09-18T10:15:04.883 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:04.884 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:04.887 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-18T10:15:04.885+0000 7f1981873700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22132) UID: 0 2024-09-18T10:15:04.893 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T10:15:04.988 INFO:tasks.ceph.osd.15.smithi189.stderr:2024-09-18T10:15:04.983+0000 7fd0e725e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 22401) UID: 0 2024-09-18T10:15:05.004 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:05.004 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:05.014 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T10:15:05.089 INFO:tasks.ceph.osd.3.smithi189.stderr:2024-09-18T10:15:05.087+0000 7f2440fc7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22396) UID: 0 2024-09-18T10:15:05.115 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:05.115 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:05.124 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T10:15:05.190 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-18T10:15:05.189+0000 7f14c6a1a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 22138) UID: 0 2024-09-18T10:15:05.223 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:05.223 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:05.233 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T10:15:05.291 INFO:tasks.ceph.osd.12.smithi029.stderr:2024-09-18T10:15:05.286+0000 7f8e4e627700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 22190) UID: 0 2024-09-18T10:15:05.329 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:05.330 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:05.339 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T10:15:05.391 INFO:tasks.ceph.osd.11.smithi189.stderr:2024-09-18T10:15:05.387+0000 7f718b60a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 22400) UID: 0 2024-09-18T10:15:05.441 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:05.442 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:05.451 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T10:15:05.492 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-18T10:15:05.489+0000 7f14c6a1a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 22138) UID: 0 2024-09-18T10:15:05.549 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:05.549 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:05.550 INFO:tasks.rados.rados.0.smithi189.stdout:143: done (3 left) 2024-09-18T10:15:05.550 INFO:tasks.rados.rados.0.smithi189.stdout:144: setattr oid 50 current snap is 7 2024-09-18T10:15:05.550 INFO:tasks.rados.rados.0.smithi189.stdout:131: finishing write tid 1 to smithi18925896-20 2024-09-18T10:15:05.550 INFO:tasks.rados.rados.0.smithi189.stdout:131: finishing write tid 2 to smithi18925896-20 2024-09-18T10:15:05.550 INFO:tasks.rados.rados.0.smithi189.stdout:140: expect (ObjNum 12 snap 0 seq_num 12) 2024-09-18T10:15:05.558 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-09-18T10:15:05.561 INFO:tasks.rados.rados.0.smithi189.stdout:131: finishing write tid 3 to smithi18925896-20 2024-09-18T10:15:05.561 INFO:tasks.rados.rados.0.smithi189.stdout:131: finishing write tid 4 to smithi18925896-20 2024-09-18T10:15:05.561 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 20 v 10 (ObjNum 67 snap 6 seq_num 67) dirty exists 2024-09-18T10:15:05.561 INFO:tasks.rados.rados.0.smithi189.stdout:131: left oid 20 (ObjNum 67 snap 6 seq_num 67) 2024-09-18T10:15:05.561 INFO:tasks.rados.rados.0.smithi189.stdout:139: finishing rollback tid 0 to smithi18925896-47 2024-09-18T10:15:05.561 INFO:tasks.rados.rados.0.smithi189.stdout:131: done (3 left) 2024-09-18T10:15:05.561 INFO:tasks.rados.rados.0.smithi189.stdout:140: done (2 left) 2024-09-18T10:15:05.561 INFO:tasks.rados.rados.0.smithi189.stdout:append oid 26 current snap is 7 2024-09-18T10:15:05.561 INFO:tasks.rados.rados.0.smithi189.stdout:145: seq_num 71 ranges {557056=425984,983040=573440,1556480=196608} 2024-09-18T10:15:05.567 INFO:tasks.rados.rados.0.smithi189.stdout:145: writing smithi18925896-26 from 557056 to 983040 tid 1 2024-09-18T10:15:05.571 INFO:tasks.rados.rados.0.smithi189.stdout:145: writing smithi18925896-26 from 983040 to 1556480 tid 2 2024-09-18T10:15:05.573 INFO:tasks.rados.rados.0.smithi189.stdout:145: writing smithi18925896-26 from 1556480 to 1753088 tid 3 2024-09-18T10:15:05.573 INFO:tasks.rados.rados.0.smithi189.stdout:146: read oid 43 snap 2 2024-09-18T10:15:05.573 INFO:tasks.rados.rados.0.smithi189.stdout:146: expect (ObjNum 42 snap 0 seq_num 42) 2024-09-18T10:15:05.573 INFO:tasks.rados.rados.0.smithi189.stdout:147: rollback oid 34 current snap is 7 2024-09-18T10:15:05.573 INFO:tasks.rados.rados.0.smithi189.stdout:rollback oid 34 to 2 2024-09-18T10:15:05.573 INFO:tasks.rados.rados.0.smithi189.stdout:148: delete oid 7 current snap is 7 2024-09-18T10:15:05.593 INFO:tasks.ceph.osd.8.smithi029.stderr:2024-09-18T10:15:05.590+0000 7f2555c27700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 22192) UID: 0 2024-09-18T10:15:05.654 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:05.654 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:05.663 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-09-18T10:15:05.668 INFO:tasks.rados.rados.0.smithi189.stdout:139: finishing rollback tid 1 to smithi18925896-47 2024-09-18T10:15:05.695 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-18T10:15:05.693+0000 7f578c52f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22134) UID: 0 2024-09-18T10:15:05.755 INFO:tasks.rados.rados.0.smithi189.stdout:148: done (5 left) 2024-09-18T10:15:05.755 INFO:tasks.rados.rados.0.smithi189.stdout:149: delete oid 3 current snap is 7 2024-09-18T10:15:05.764 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:05.764 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:05.774 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-09-18T10:15:05.817 INFO:tasks.ceph.osd.15.smithi189.stderr:2024-09-18T10:15:05.815+0000 7fd0e725e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 22401) UID: 0 2024-09-18T10:15:05.870 INFO:tasks.rados.rados.0.smithi189.stdout:139: finishing rollback tid 2 to smithi18925896-47 2024-09-18T10:15:05.870 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 47 v 19 (ObjNum 61 snap 5 seq_num 61) dirty exists 2024-09-18T10:15:05.887 INFO:tasks.rados.rados.0.smithi189.stdout:145: finishing write tid 1 to smithi18925896-26 2024-09-18T10:15:05.896 INFO:tasks.ceph.osd.12.smithi029.stderr:2024-09-18T10:15:05.890+0000 7f8e4e627700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 22190) UID: 0 2024-09-18T10:15:05.920 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:05.921 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:05.929 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_ops_in_flight 2024-09-18T10:15:05.982 INFO:tasks.rados.rados.0.smithi189.stdout:139: done (5 left) 2024-09-18T10:15:05.982 INFO:tasks.rados.rados.0.smithi189.stdout:149: done (4 left) 2024-09-18T10:15:05.982 INFO:tasks.rados.rados.0.smithi189.stdout:150: copy_from oid 36 from oid 1 current snap is 7 2024-09-18T10:15:05.982 INFO:tasks.rados.rados.0.smithi189.stdout:append oid 24 current snap is 7 2024-09-18T10:15:05.982 INFO:tasks.rados.rados.0.smithi189.stdout:151: seq_num 74 ranges {475136=475136,950272=622592,1572864=147456} 2024-09-18T10:15:05.992 INFO:tasks.rados.rados.0.smithi189.stdout:151: writing smithi18925896-24 from 475136 to 950272 tid 1 2024-09-18T10:15:05.995 INFO:tasks.rados.rados.0.smithi189.stdout:151: writing smithi18925896-24 from 950272 to 1572864 tid 2 2024-09-18T10:15:05.996 INFO:tasks.rados.rados.0.smithi189.stdout:151: writing smithi18925896-24 from 1572864 to 1720320 tid 3 2024-09-18T10:15:05.996 INFO:tasks.rados.rados.0.smithi189.stdout:append oid 6 current snap is 7 2024-09-18T10:15:05.996 INFO:tasks.rados.rados.0.smithi189.stdout:145: finishing write tid 2 to smithi18925896-26 2024-09-18T10:15:05.996 INFO:tasks.rados.rados.0.smithi189.stdout:152: seq_num 75 ranges {0=491520,491520=770048} 2024-09-18T10:15:05.997 INFO:tasks.ceph.osd.11.smithi189.stderr:2024-09-18T10:15:05.995+0000 7f718b60a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 22400) UID: 0 2024-09-18T10:15:05.998 INFO:tasks.rados.rados.0.smithi189.stdout:152: writing smithi18925896-6 from 0 to 491520 tid 1 2024-09-18T10:15:06.003 INFO:tasks.rados.rados.0.smithi189.stdout:152: writing smithi18925896-6 from 491520 to 1261568 tid 2 2024-09-18T10:15:06.003 INFO:tasks.rados.rados.0.smithi189.stdout:153: copy_from oid 32 from oid 37 current snap is 7 2024-09-18T10:15:06.003 INFO:tasks.rados.rados.0.smithi189.stdout:154: copy_from oid 30 from oid 13 current snap is 7 2024-09-18T10:15:06.003 INFO:tasks.rados.rados.0.smithi189.stdout:155: snap_create 2024-09-18T10:15:06.028 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:06.029 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:06.038 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_blocked_ops 2024-09-18T10:15:06.078 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 50 v 41 (ObjNum 49 snap 0 seq_num 49) dirty exists 2024-09-18T10:15:06.082 INFO:tasks.rados.rados.0.smithi189.stdout:147: finishing rollback tid 0 to smithi18925896-34 2024-09-18T10:15:06.082 INFO:tasks.rados.rados.0.smithi189.stdout:147: finishing rollback tid 1 to smithi18925896-34 2024-09-18T10:15:06.093 INFO:tasks.rados.rados.0.smithi189.stdout:153: finishing copy_from racing read to smithi18925896-32 2024-09-18T10:15:06.097 INFO:tasks.ceph.osd.8.smithi029.stderr:2024-09-18T10:15:06.094+0000 7f2555c27700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 22192) UID: 0 2024-09-18T10:15:06.135 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:06.135 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:06.144 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_historic_ops 2024-09-18T10:15:06.188 INFO:tasks.rados.rados.0.smithi189.stdout:145: finishing write tid 3 to smithi18925896-26 2024-09-18T10:15:06.193 INFO:tasks.rados.rados.0.smithi189.stdout:153: finishing copy_from to smithi18925896-32 2024-09-18T10:15:06.193 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 32 v 8 (ObjNum 36 snap 0 seq_num 36) dirty exists 2024-09-18T10:15:06.198 INFO:tasks.ceph.osd.12.smithi029.stderr:2024-09-18T10:15:06.194+0000 7f8e4e627700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 22190) UID: 0 2024-09-18T10:15:06.241 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:06.241 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:06.250 INFO:tasks.rados.rados.0.smithi189.stdout:147: finishing rollback tid 2 to smithi18925896-34 2024-09-18T10:15:06.250 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 34 v 22 (ObjNum 33 snap 0 seq_num 33) dirty exists 2024-09-18T10:15:06.251 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T10:15:06.253 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T10:15:06.279 INFO:tasks.rados.rados.0.smithi189.stdout:154: finishing copy_from racing read to smithi18925896-30 2024-09-18T10:15:06.289 INFO:tasks.rados.rados.0.smithi189.stdout:145: finishing write tid 4 to smithi18925896-26 2024-09-18T10:15:06.299 INFO:tasks.ceph.osd.10.smithi154.stderr:2024-09-18T10:15:06.295+0000 7fb73221a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 22091) UID: 0 2024-09-18T10:15:06.303 INFO:tasks.rados.rados.0.smithi189.stdout:151: finishing write tid 1 to smithi18925896-24 2024-09-18T10:15:06.363 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:06.364 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:06.373 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T10:15:06.384 INFO:tasks.rados.rados.0.smithi189.stdout:154: finishing copy_from to smithi18925896-30 2024-09-18T10:15:06.384 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 30 v 4 (ObjNum 12 snap 0 seq_num 12) dirty exists 2024-09-18T10:15:06.400 INFO:tasks.ceph.osd.10.smithi154.stderr:2024-09-18T10:15:06.395+0000 7fb73221a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 22091) UID: 0 2024-09-18T10:15:06.441 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-18T10:15:06.463 INFO:tasks.rados.rados.0.smithi189.stdout:151: finishing write tid 2 to smithi18925896-24 2024-09-18T10:15:06.464 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-18T10:15:06.465 INFO:tasks.rados.rados.0.smithi189.stdout:151: finishing write tid 3 to smithi18925896-24 2024-09-18T10:15:06.465 INFO:tasks.rados.rados.0.smithi189.stdout:151: finishing write tid 4 to smithi18925896-24 2024-09-18T10:15:06.475 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-18T10:15:06.475 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-18T10:15:06.475 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-18T10:15:06.475 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-18T10:15:06.475 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-18T10:15:06.484 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T10:15:06.486 INFO:tasks.rados.rados.0.smithi189.stdout:152: finishing write tid 1 to smithi18925896-6 2024-09-18T10:15:06.489 INFO:tasks.rados.rados.0.smithi189.stdout:150: finishing copy_from racing read to smithi18925896-36 2024-09-18T10:15:06.491 INFO:tasks.rados.rados.0.smithi189.stdout:152: finishing write tid 2 to smithi18925896-6 2024-09-18T10:15:06.512 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-18T10:15:06.512 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-18T10:15:06.506+0000 7f302f2c3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22187) UID: 0 2024-09-18T10:15:06.549 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-18T10:15:06.552 INFO:tasks.rados.rados.0.smithi189.stdout:144: done (9 left) 2024-09-18T10:15:06.552 INFO:tasks.rados.rados.0.smithi189.stdout:147: done (8 left) 2024-09-18T10:15:06.552 INFO:tasks.rados.rados.0.smithi189.stdout:153: done (7 left) 2024-09-18T10:15:06.552 INFO:tasks.rados.rados.0.smithi189.stdout:154: done (6 left) 2024-09-18T10:15:06.552 INFO:tasks.rados.rados.0.smithi189.stdout:155: done (5 left) 2024-09-18T10:15:06.552 INFO:tasks.rados.rados.0.smithi189.stdout:append oid 46 current snap is 8 2024-09-18T10:15:06.552 INFO:tasks.rados.rados.0.smithi189.stdout:156: seq_num 76 ranges {770048=507904,1277952=507904,1785856=262144} 2024-09-18T10:15:06.566 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-18T10:15:06.570 INFO:tasks.rados.rados.0.smithi189.stdout:156: writing smithi18925896-46 from 770048 to 1277952 tid 1 2024-09-18T10:15:06.573 INFO:tasks.rados.rados.0.smithi189.stdout:156: writing smithi18925896-46 from 1277952 to 1785856 tid 2 2024-09-18T10:15:06.574 INFO:tasks.rados.rados.0.smithi189.stdout:156: writing smithi18925896-46 from 1785856 to 2048000 tid 3 2024-09-18T10:15:06.574 INFO:tasks.rados.rados.0.smithi189.stdout:append oid 47 current snap is 8 2024-09-18T10:15:06.575 INFO:tasks.rados.rados.0.smithi189.stdout:157: seq_num 77 ranges {1818624=524288,2342912=524288,2867200=245760} 2024-09-18T10:15:06.580 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-18T10:15:06.587 INFO:tasks.rados.rados.0.smithi189.stdout:157: writing smithi18925896-47 from 1818624 to 2342912 tid 1 2024-09-18T10:15:06.590 INFO:tasks.rados.rados.0.smithi189.stdout:157: writing smithi18925896-47 from 2342912 to 2867200 tid 2 2024-09-18T10:15:06.592 INFO:tasks.rados.rados.0.smithi189.stdout:157: writing smithi18925896-47 from 2867200 to 3112960 tid 3 2024-09-18T10:15:06.592 INFO:tasks.rados.rados.0.smithi189.stdout:158: rollback oid 17 current snap is 8 2024-09-18T10:15:06.592 INFO:tasks.rados.rados.0.smithi189.stdout:rollback oid 17 to 2 2024-09-18T10:15:06.592 INFO:tasks.rados.rados.0.smithi189.stdout:159: snap_create 2024-09-18T10:15:06.602 INFO:tasks.ceph.osd.14.smithi154.stderr:2024-09-18T10:15:06.595+0000 7f75a19e7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 22093) UID: 0 2024-09-18T10:15:06.614 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-18T10:15:06.614 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-18T10:15:06.614 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-18T10:15:06.614 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-18T10:15:06.614 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.614 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWriteReply(3.6s0 36/32 ECSubWriteReply(tid=11, last_complete=36'7, committed=1, applied=1))", 2024-09-18T10:15:06.614 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:50.290945+0000", 2024-09-18T10:15:06.614 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 16.319136883999999, 2024-09-18T10:15:06.614 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.208748928, 2024-09-18T10:15:06.614 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:06.615 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:06.615 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:06.615 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.615 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:06.615 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.290945+0000", 2024-09-18T10:15:06.615 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:06.615 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.615 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.615 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:06.615 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.290945+0000", 2024-09-18T10:15:06.615 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:06.615 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.616 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.616 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:06.616 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.290945+0000", 2024-09-18T10:15:06.616 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:06.616 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.616 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.616 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:06.616 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.290951+0000", 2024-09-18T10:15:06.616 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.5729999999999996e-06 2024-09-18T10:15:06.616 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.616 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.616 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:06.617 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.290964+0000", 2024-09-18T10:15:06.617 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3012e-05 2024-09-18T10:15:06.617 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.617 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.617 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:06.617 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.290972+0000", 2024-09-18T10:15:06.617 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4699999999999996e-06 2024-09-18T10:15:06.617 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.617 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.617 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:06.617 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.499661+0000", 2024-09-18T10:15:06.617 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.20868911500000001 2024-09-18T10:15:06.617 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.618 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.618 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:06.618 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.499694+0000", 2024-09-18T10:15:06.618 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.3470000000000003e-05 2024-09-18T10:15:06.618 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:06.618 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:06.618 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:06.618 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.618 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.618 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWriteReply(3.6s0 36/32 ECSubWriteReply(tid=11, last_complete=36'7, committed=1, applied=1))", 2024-09-18T10:15:06.618 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:50.291030+0000", 2024-09-18T10:15:06.618 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 16.319052384999999, 2024-09-18T10:15:06.618 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.208840265, 2024-09-18T10:15:06.618 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:06.619 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:06.619 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:06.619 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.619 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:06.619 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.291030+0000", 2024-09-18T10:15:06.619 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:06.619 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.619 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.619 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:06.619 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.291030+0000", 2024-09-18T10:15:06.619 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:06.619 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.619 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.619 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:06.620 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.291030+0000", 2024-09-18T10:15:06.620 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:06.620 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.620 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.620 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:06.620 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.291032+0000", 2024-09-18T10:15:06.620 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.604e-06 2024-09-18T10:15:06.620 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.620 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.620 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:06.620 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.291038+0000", 2024-09-18T10:15:06.620 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8170000000000004e-06 2024-09-18T10:15:06.620 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.621 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.621 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:06.621 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.291042+0000", 2024-09-18T10:15:06.621 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.839e-06 2024-09-18T10:15:06.621 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.621 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.621 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:06.621 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.499812+0000", 2024-09-18T10:15:06.621 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.208769805 2024-09-18T10:15:06.621 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.621 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.621 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:06.621 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.499870+0000", 2024-09-18T10:15:06.622 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8449000000000002e-05 2024-09-18T10:15:06.622 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:06.622 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:06.622 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:06.622 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.622 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.622 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWriteReply(3.6s0 36/32 ECSubWriteReply(tid=11, last_complete=36'7, committed=1, applied=1))", 2024-09-18T10:15:06.622 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:50.302109+0000", 2024-09-18T10:15:06.622 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 16.307973226000001, 2024-09-18T10:15:06.622 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.29912391399999999, 2024-09-18T10:15:06.622 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:06.622 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:06.623 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:06.623 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.623 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:06.623 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.302109+0000", 2024-09-18T10:15:06.623 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:06.623 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.623 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.623 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:06.623 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.302109+0000", 2024-09-18T10:15:06.623 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:06.623 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.623 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.623 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:06.624 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.302107+0000", 2024-09-18T10:15:06.624 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:06.624 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.624 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.624 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:06.624 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.302119+0000", 2024-09-18T10:15:06.624 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1207e-05 2024-09-18T10:15:06.624 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.624 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.624 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:06.624 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.302150+0000", 2024-09-18T10:15:06.624 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.133e-05 2024-09-18T10:15:06.624 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.625 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.625 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:06.625 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.302164+0000", 2024-09-18T10:15:06.625 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3942e-05 2024-09-18T10:15:06.625 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.625 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.625 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:06.625 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.600015+0000", 2024-09-18T10:15:06.625 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.29785072000000001 2024-09-18T10:15:06.625 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.625 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.625 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:06.625 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.601233+0000", 2024-09-18T10:15:06.625 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0012183599999999999 2024-09-18T10:15:06.626 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:06.626 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:06.626 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:06.626 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.626 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.626 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpReadReply(3.6s0 36/32 ECSubReadReply(tid=10, attrs_read=0))", 2024-09-18T10:15:06.626 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:50.302179+0000", 2024-09-18T10:15:06.626 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 16.307903159999999, 2024-09-18T10:15:06.626 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.30043391800000002, 2024-09-18T10:15:06.626 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:06.626 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:06.626 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:06.626 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.627 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:06.627 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.302179+0000", 2024-09-18T10:15:06.627 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:06.627 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.627 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.627 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:06.627 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.302179+0000", 2024-09-18T10:15:06.627 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:06.627 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.627 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.627 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:06.627 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.302178+0000", 2024-09-18T10:15:06.627 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:06.627 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.628 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.628 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:06.628 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.302201+0000", 2024-09-18T10:15:06.628 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2824e-05 2024-09-18T10:15:06.628 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.628 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.628 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:06.628 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.302217+0000", 2024-09-18T10:15:06.628 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5993000000000001e-05 2024-09-18T10:15:06.628 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.628 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.628 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:06.628 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.302223+0000", 2024-09-18T10:15:06.628 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0820000000000004e-06 2024-09-18T10:15:06.629 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.629 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.629 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:06.629 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.602454+0000", 2024-09-18T10:15:06.629 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.30023072499999998 2024-09-18T10:15:06.629 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.629 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.629 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:06.629 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.602613+0000", 2024-09-18T10:15:06.629 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00015896399999999999 2024-09-18T10:15:06.629 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:06.629 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:06.629 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:06.630 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.630 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.630 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpRead(3.7s3 36/32 ECSubRead(tid=22, to_read={3:ee962131:::smithi18925896-29:head=0,4096,0}, subchunks={3:ee962131:::smithi18925896-29:head=[0,1]}, attrs_to_read=))", 2024-09-18T10:15:06.630 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:50.497231+0000", 2024-09-18T10:15:06.630 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 16.112851342999999, 2024-09-18T10:15:06.630 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.20067855700000001, 2024-09-18T10:15:06.630 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:06.630 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:06.630 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:06.630 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.630 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:06.630 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.497231+0000", 2024-09-18T10:15:06.631 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:06.631 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.631 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.631 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:06.631 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.497231+0000", 2024-09-18T10:15:06.631 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:06.631 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.631 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.631 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:06.631 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.497229+0000", 2024-09-18T10:15:06.632 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:06.632 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.632 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.632 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:06.632 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.497252+0000", 2024-09-18T10:15:06.632 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3193e-05 2024-09-18T10:15:06.632 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.632 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.632 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:06.632 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.497287+0000", 2024-09-18T10:15:06.632 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.5040000000000003e-05 2024-09-18T10:15:06.632 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.632 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.633 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:06.633 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.597372+0000", 2024-09-18T10:15:06.633 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10008500200000001 2024-09-18T10:15:06.633 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.633 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.633 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:06.633 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.697601+0000", 2024-09-18T10:15:06.633 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100228709 2024-09-18T10:15:06.633 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.633 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.633 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:06.633 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.697909+0000", 2024-09-18T10:15:06.633 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00030862700000000001 2024-09-18T10:15:06.633 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:06.634 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:06.634 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:06.634 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.634 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.634 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:121 3.6s0 3:6c64c4fa:::smithi18925896-41:head [append 0~688128,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:06.634 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:50.627356+0000", 2024-09-18T10:15:06.634 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 15.982725929000001, 2024-09-18T10:15:06.634 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.38269042599999997, 2024-09-18T10:15:06.634 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:06.634 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:06.634 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:06.634 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:06.634 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:06.635 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 121 2024-09-18T10:15:06.635 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.635 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:06.635 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.635 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:06.635 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.627356+0000", 2024-09-18T10:15:06.635 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:06.635 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.635 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.635 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:06.635 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.627356+0000", 2024-09-18T10:15:06.635 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:06.635 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.635 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.636 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:06.636 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.627353+0000", 2024-09-18T10:15:06.636 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999971 2024-09-18T10:15:06.636 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.636 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.636 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:06.636 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.629112+0000", 2024-09-18T10:15:06.636 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0017585980000000001 2024-09-18T10:15:06.636 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.636 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.636 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:06.636 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.629149+0000", 2024-09-18T10:15:06.636 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.6532999999999997e-05 2024-09-18T10:15:06.637 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.637 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.637 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:06.637 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.629169+0000", 2024-09-18T10:15:06.637 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0699e-05 2024-09-18T10:15:06.637 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.638 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.638 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:06.638 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.706567+0000", 2024-09-18T10:15:06.638 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.077397621999999999 2024-09-18T10:15:06.638 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.638 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.638 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:06.638 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.706944+0000", 2024-09-18T10:15:06.638 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00037710599999999999 2024-09-18T10:15:06.639 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.639 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.639 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:06.639 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.709283+0000", 2024-09-18T10:15:06.639 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0023391689999999999 2024-09-18T10:15:06.639 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.639 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.639 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:06.639 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.711791+0000", 2024-09-18T10:15:06.639 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0025078689999999998 2024-09-18T10:15:06.639 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.639 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.640 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-18T10:15:06.640 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.009925+0000", 2024-09-18T10:15:06.640 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.298134493 2024-09-18T10:15:06.640 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.640 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.640 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:06.640 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.010047+0000", 2024-09-18T10:15:06.640 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000121264 2024-09-18T10:15:06.640 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:06.640 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:06.640 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:06.640 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.640 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.641 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:122 3.6s0 3:6c64c4fa:::smithi18925896-41:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:06.641 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:50.629189+0000", 2024-09-18T10:15:06.641 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 15.980893203000001, 2024-09-18T10:15:06.641 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.47849547399999998, 2024-09-18T10:15:06.641 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:06.641 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:06.641 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:06.641 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:06.641 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:06.641 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 122 2024-09-18T10:15:06.641 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.641 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:06.642 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.642 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:06.642 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.629189+0000", 2024-09-18T10:15:06.642 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:06.642 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.642 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.642 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:06.642 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.629189+0000", 2024-09-18T10:15:06.642 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:06.642 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.642 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.643 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:06.643 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.629187+0000", 2024-09-18T10:15:06.643 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:06.643 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.643 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.643 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:06.643 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.629198+0000", 2024-09-18T10:15:06.643 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0623e-05 2024-09-18T10:15:06.643 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.643 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.643 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:06.643 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.629219+0000", 2024-09-18T10:15:06.643 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0665999999999998e-05 2024-09-18T10:15:06.644 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.644 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.644 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:06.644 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.629231+0000", 2024-09-18T10:15:06.644 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.291e-05 2024-09-18T10:15:06.644 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.644 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.644 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:06.644 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.710475+0000", 2024-09-18T10:15:06.644 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.081244021 2024-09-18T10:15:06.644 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.644 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.645 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:06.645 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.710639+0000", 2024-09-18T10:15:06.645 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000163325 2024-09-18T10:15:06.645 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.645 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.645 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:06.645 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.711435+0000", 2024-09-18T10:15:06.645 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00079575999999999996 2024-09-18T10:15:06.645 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.645 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.645 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:06.646 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.812067+0000", 2024-09-18T10:15:06.646 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10063237799999999 2024-09-18T10:15:06.646 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.646 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.646 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-18T10:15:06.646 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.107562+0000", 2024-09-18T10:15:06.646 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.295494906 2024-09-18T10:15:06.646 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.646 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.646 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:06.646 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.107685+0000", 2024-09-18T10:15:06.646 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00012271100000000001 2024-09-18T10:15:06.646 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:06.646 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:06.647 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:06.647 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.647 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.647 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:123 3.6s0 3:6c64c4fa:::smithi18925896-41:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:06.647 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:50.629246+0000", 2024-09-18T10:15:06.647 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 15.980836376999999, 2024-09-18T10:15:06.647 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.58067324300000001, 2024-09-18T10:15:06.647 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:06.647 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:06.647 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:06.647 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:06.647 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:06.647 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 123 2024-09-18T10:15:06.648 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.648 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:06.648 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.648 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:06.648 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.629246+0000", 2024-09-18T10:15:06.648 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:06.648 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.648 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.648 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:06.648 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.629246+0000", 2024-09-18T10:15:06.648 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:06.648 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.648 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.649 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:06.649 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.629244+0000", 2024-09-18T10:15:06.649 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:06.649 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.649 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.649 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:06.649 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.629254+0000", 2024-09-18T10:15:06.649 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.0759999999999992e-06 2024-09-18T10:15:06.649 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.649 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.649 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:06.649 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.629279+0000", 2024-09-18T10:15:06.649 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.5905999999999999e-05 2024-09-18T10:15:06.649 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.650 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.650 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:06.650 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.629293+0000", 2024-09-18T10:15:06.650 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3073e-05 2024-09-18T10:15:06.650 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.650 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.650 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:06.650 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.806055+0000", 2024-09-18T10:15:06.650 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.17676249999999999 2024-09-18T10:15:06.650 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.650 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.650 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for rw locks", 2024-09-18T10:15:06.650 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.806211+0000", 2024-09-18T10:15:06.651 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000156384 2024-09-18T10:15:06.651 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.651 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.651 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:06.651 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.108025+0000", 2024-09-18T10:15:06.651 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.30181311500000002 2024-09-18T10:15:06.651 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.651 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.651 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:06.651 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.108107+0000", 2024-09-18T10:15:06.651 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.2330000000000006e-05 2024-09-18T10:15:06.651 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.651 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.651 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:06.652 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.209919+0000", 2024-09-18T10:15:06.652 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.101812262 2024-09-18T10:15:06.652 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:06.652 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:06.652 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:06.652 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.652 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.652 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWrite(3.bs1 36/32 ECSubWrite(tid=25, reqid=client.4632.0:118, at_version=36'3, trim_to=0'0, roll_forward_to=35'2))", 2024-09-18T10:15:06.652 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:50.708852+0000", 2024-09-18T10:15:06.652 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 15.901230634000001, 2024-09-18T10:15:06.652 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.202385011, 2024-09-18T10:15:06.652 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:06.652 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:06.653 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:06.653 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.653 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:06.653 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.708852+0000", 2024-09-18T10:15:06.653 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:06.653 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.653 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.653 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:06.653 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.708852+0000", 2024-09-18T10:15:06.653 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:06.653 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.653 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.654 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:06.654 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.708850+0000", 2024-09-18T10:15:06.654 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:06.654 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.654 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.654 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:06.654 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.709003+0000", 2024-09-18T10:15:06.654 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00015311900000000001 2024-09-18T10:15:06.654 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.654 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.654 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:06.654 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.709024+0000", 2024-09-18T10:15:06.654 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0574999999999999e-05 2024-09-18T10:15:06.655 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.655 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.655 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:06.655 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.709036+0000", 2024-09-18T10:15:06.655 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1875999999999999e-05 2024-09-18T10:15:06.655 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.655 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.655 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:06.655 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.909305+0000", 2024-09-18T10:15:06.655 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.20026893000000001 2024-09-18T10:15:06.655 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.655 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.656 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:06.656 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.909325+0000", 2024-09-18T10:15:06.656 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0809999999999999e-05 2024-09-18T10:15:06.656 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.656 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.656 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:06.656 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.911183+0000", 2024-09-18T10:15:06.656 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0018570889999999999 2024-09-18T10:15:06.656 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.656 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.656 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:06.656 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.911237+0000", 2024-09-18T10:15:06.656 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4045999999999997e-05 2024-09-18T10:15:06.657 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:06.657 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:06.657 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:06.657 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.657 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.657 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:167 3.6s0 3:6619023d:::smithi18925896-19:head [copy-from ver 4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:06.657 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:51.531881+0000", 2024-09-18T10:15:06.657 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 15.078201159000001, 2024-09-18T10:15:06.657 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.41157491600000001, 2024-09-18T10:15:06.657 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:06.657 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:06.657 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:06.657 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:06.658 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:06.658 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 167 2024-09-18T10:15:06.658 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.658 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:06.658 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.658 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:06.658 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.531881+0000", 2024-09-18T10:15:06.658 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:06.658 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.658 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.658 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:06.658 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.531881+0000", 2024-09-18T10:15:06.658 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:06.658 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.659 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.659 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:06.659 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.531877+0000", 2024-09-18T10:15:06.659 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999962 2024-09-18T10:15:06.659 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.659 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.659 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:06.659 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.531894+0000", 2024-09-18T10:15:06.659 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7269999999999999e-05 2024-09-18T10:15:06.659 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.659 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.659 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:06.659 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.531939+0000", 2024-09-18T10:15:06.659 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4815999999999999e-05 2024-09-18T10:15:06.660 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.660 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.660 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:06.660 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.531963+0000", 2024-09-18T10:15:06.660 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3312e-05 2024-09-18T10:15:06.660 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.660 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.660 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:06.660 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.532107+0000", 2024-09-18T10:15:06.660 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000144547 2024-09-18T10:15:06.660 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.660 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.660 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:06.661 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.532348+0000", 2024-09-18T10:15:06.661 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000240955 2024-09-18T10:15:06.661 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.661 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.661 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:06.661 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.839634+0000", 2024-09-18T10:15:06.661 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.30728566400000001 2024-09-18T10:15:06.661 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.661 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.661 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:06.661 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.841560+0000", 2024-09-18T10:15:06.661 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001926449 2024-09-18T10:15:06.662 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.662 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.662 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-18T10:15:06.662 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.943372+0000", 2024-09-18T10:15:06.662 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.101812176 2024-09-18T10:15:06.662 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.662 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.662 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:06.662 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.943456+0000", 2024-09-18T10:15:06.662 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.3614000000000003e-05 2024-09-18T10:15:06.662 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:06.662 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:06.662 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:06.663 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.663 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.663 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:168 3.6s0 3:6619023d:::smithi18925896-19:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:06.663 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:51.532000+0000", 2024-09-18T10:15:06.663 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 15.078082172, 2024-09-18T10:15:06.663 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.30898515999999998, 2024-09-18T10:15:06.663 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:06.663 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:06.663 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:06.663 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:06.663 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:06.663 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 168 2024-09-18T10:15:06.663 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.663 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:06.664 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.664 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:06.664 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.532000+0000", 2024-09-18T10:15:06.664 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:06.664 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.664 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.664 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:06.664 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.532000+0000", 2024-09-18T10:15:06.664 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:06.664 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.664 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.664 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:06.665 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.531998+0000", 2024-09-18T10:15:06.665 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:06.665 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.665 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.665 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:06.665 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.532009+0000", 2024-09-18T10:15:06.665 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0988999999999999e-05 2024-09-18T10:15:06.665 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.665 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.665 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:06.665 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.532032+0000", 2024-09-18T10:15:06.665 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2203000000000001e-05 2024-09-18T10:15:06.665 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.666 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.666 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:06.666 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.532043+0000", 2024-09-18T10:15:06.666 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1707e-05 2024-09-18T10:15:06.666 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.666 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.666 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:06.666 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.532663+0000", 2024-09-18T10:15:06.666 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000619993 2024-09-18T10:15:06.666 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.666 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.666 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for blocked object", 2024-09-18T10:15:06.666 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.532735+0000", 2024-09-18T10:15:06.667 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.2025000000000002e-05 2024-09-18T10:15:06.667 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.667 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.667 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:06.667 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.840712+0000", 2024-09-18T10:15:06.667 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.307976582 2024-09-18T10:15:06.667 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.667 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.667 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:06.667 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.840866+0000", 2024-09-18T10:15:06.667 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000154203 2024-09-18T10:15:06.667 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.667 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.667 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:06.668 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.840985+0000", 2024-09-18T10:15:06.668 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000119084 2024-09-18T10:15:06.668 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:06.668 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:06.668 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:06.668 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.668 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.668 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:204 3.6s0 3:61491e68:::smithi18925896-27:head [read 0~442368,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:06.668 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:52.694124+0000", 2024-09-18T10:15:06.668 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 13.915957986, 2024-09-18T10:15:06.668 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.20436779699999999, 2024-09-18T10:15:06.669 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:06.669 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:06.669 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:06.669 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:06.669 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:06.669 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 204 2024-09-18T10:15:06.669 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.669 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:06.669 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.669 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:06.669 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.694124+0000", 2024-09-18T10:15:06.669 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:06.669 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.670 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.670 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:06.670 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.694124+0000", 2024-09-18T10:15:06.670 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:06.670 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.670 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.670 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:06.670 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.694121+0000", 2024-09-18T10:15:06.670 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999971 2024-09-18T10:15:06.670 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.670 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.670 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:06.671 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.694139+0000", 2024-09-18T10:15:06.671 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7277000000000001e-05 2024-09-18T10:15:06.671 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.671 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.671 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:06.671 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.694168+0000", 2024-09-18T10:15:06.671 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9787000000000001e-05 2024-09-18T10:15:06.671 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.671 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.671 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:06.671 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.694184+0000", 2024-09-18T10:15:06.671 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5478999999999999e-05 2024-09-18T10:15:06.671 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.671 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.672 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:06.672 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.694400+0000", 2024-09-18T10:15:06.672 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00021609 2024-09-18T10:15:06.672 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.672 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.672 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op must wait for map", 2024-09-18T10:15:06.672 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.694417+0000", 2024-09-18T10:15:06.672 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7328000000000001e-05 2024-09-18T10:15:06.672 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.672 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.672 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:06.672 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.794979+0000", 2024-09-18T10:15:06.672 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100561635 2024-09-18T10:15:06.673 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.673 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.673 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:06.673 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.795117+0000", 2024-09-18T10:15:06.673 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00013797399999999999 2024-09-18T10:15:06.673 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.673 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.673 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:06.673 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.898492+0000", 2024-09-18T10:15:06.673 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.103375174 2024-09-18T10:15:06.673 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:06.676 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:06.676 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:06.676 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.676 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.676 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:205 3.6s0 3:61491e68:::smithi18925896-27:head [read 0~442368,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:06.676 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:52.694213+0000", 2024-09-18T10:15:06.676 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 13.915869091999999, 2024-09-18T10:15:06.677 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.20580652599999999, 2024-09-18T10:15:06.677 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:06.677 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:06.677 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:06.677 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:06.677 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:06.677 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 205 2024-09-18T10:15:06.677 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.677 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:06.677 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.677 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:06.677 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.694213+0000", 2024-09-18T10:15:06.677 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:06.677 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.678 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.678 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:06.678 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.694213+0000", 2024-09-18T10:15:06.678 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:06.678 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.678 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.678 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:06.678 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.694212+0000", 2024-09-18T10:15:06.678 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:06.678 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.678 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.679 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:06.679 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.694220+0000", 2024-09-18T10:15:06.679 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5250000000000005e-06 2024-09-18T10:15:06.679 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.679 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.679 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:06.679 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.694237+0000", 2024-09-18T10:15:06.679 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.6181e-05 2024-09-18T10:15:06.679 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.679 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.679 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:06.679 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.694246+0000", 2024-09-18T10:15:06.679 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.2029999999999998e-06 2024-09-18T10:15:06.679 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.680 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.680 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:06.680 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.794554+0000", 2024-09-18T10:15:06.680 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10030855 2024-09-18T10:15:06.680 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.680 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.680 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:06.680 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.794563+0000", 2024-09-18T10:15:06.680 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.2230000000000007e-06 2024-09-18T10:15:06.680 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.680 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.680 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:06.681 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.795588+0000", 2024-09-18T10:15:06.681 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001025127 2024-09-18T10:15:06.681 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.681 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.681 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:06.681 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.795756+0000", 2024-09-18T10:15:06.681 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00016793900000000001 2024-09-18T10:15:06.681 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.681 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.681 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:06.681 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.900020+0000", 2024-09-18T10:15:06.681 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.104264018 2024-09-18T10:15:06.681 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:06.681 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:06.682 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:06.682 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.682 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.682 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:206 3.6s0 3:61491e68:::smithi18925896-27:head [read 0~442368,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:06.682 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:52.694386+0000", 2024-09-18T10:15:06.682 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 13.915695835999999, 2024-09-18T10:15:06.682 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.40544065200000001, 2024-09-18T10:15:06.682 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:06.682 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:06.682 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:06.682 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:06.682 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:06.682 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 206 2024-09-18T10:15:06.683 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.683 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:06.683 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.683 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:06.683 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.694386+0000", 2024-09-18T10:15:06.683 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:06.683 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.683 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.683 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:06.683 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.694386+0000", 2024-09-18T10:15:06.683 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:06.683 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.683 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.683 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:06.684 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.694385+0000", 2024-09-18T10:15:06.684 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.999999 2024-09-18T10:15:06.684 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.684 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.684 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:06.684 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.694392+0000", 2024-09-18T10:15:06.684 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7970000000000003e-06 2024-09-18T10:15:06.684 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.684 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.684 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:06.684 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.694405+0000", 2024-09-18T10:15:06.684 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3406e-05 2024-09-18T10:15:06.685 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.685 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.685 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:06.685 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.694413+0000", 2024-09-18T10:15:06.685 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.9070000000000007e-06 2024-09-18T10:15:06.685 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.685 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.685 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:06.685 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.794592+0000", 2024-09-18T10:15:06.685 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100178879 2024-09-18T10:15:06.685 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.685 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.685 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:06.686 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.794598+0000", 2024-09-18T10:15:06.686 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6289999999999998e-06 2024-09-18T10:15:06.686 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.686 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.686 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:06.686 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.796805+0000", 2024-09-18T10:15:06.686 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0022073930000000002 2024-09-18T10:15:06.686 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.686 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.686 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:06.686 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.796886+0000", 2024-09-18T10:15:06.686 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0641000000000003e-05 2024-09-18T10:15:06.686 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.687 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.687 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:06.687 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:53.099827+0000", 2024-09-18T10:15:06.687 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.302941191 2024-09-18T10:15:06.687 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:06.687 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:06.687 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:06.687 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.687 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.687 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:323 3.6s0 3:61491e68:::smithi18925896-27:head [append 0~0] snapc 7=[4] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-18T10:15:06.687 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:01.795354+0000", 2024-09-18T10:15:06.687 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 4.8147283219999997, 2024-09-18T10:15:06.688 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.30418701500000001, 2024-09-18T10:15:06.688 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:06.688 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:06.688 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:06.688 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:06.688 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:06.688 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 323 2024-09-18T10:15:06.688 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.688 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:06.688 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.688 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:06.688 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.795354+0000", 2024-09-18T10:15:06.689 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:06.689 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.689 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.689 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:06.689 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.795354+0000", 2024-09-18T10:15:06.689 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:06.689 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.689 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.689 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:06.689 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.795351+0000", 2024-09-18T10:15:06.689 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999971 2024-09-18T10:15:06.689 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.689 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.690 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:06.690 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.795365+0000", 2024-09-18T10:15:06.690 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3713999999999999e-05 2024-09-18T10:15:06.690 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.690 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.690 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:06.690 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.795406+0000", 2024-09-18T10:15:06.690 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0846999999999997e-05 2024-09-18T10:15:06.690 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.690 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.690 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:06.690 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.795425+0000", 2024-09-18T10:15:06.690 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9936999999999999e-05 2024-09-18T10:15:06.690 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.691 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.691 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:06.691 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.795648+0000", 2024-09-18T10:15:06.691 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000222547 2024-09-18T10:15:06.691 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.691 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.691 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:06.691 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.795872+0000", 2024-09-18T10:15:06.691 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00022384799999999999 2024-09-18T10:15:06.691 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.691 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.691 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:06.691 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.797018+0000", 2024-09-18T10:15:06.692 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001146055 2024-09-18T10:15:06.692 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.692 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.692 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:06.692 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.798609+0000", 2024-09-18T10:15:06.692 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0015905959999999999 2024-09-18T10:15:06.692 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.692 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.692 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-18T10:15:06.692 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.099372+0000", 2024-09-18T10:15:06.692 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.30076301500000002 2024-09-18T10:15:06.692 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.693 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.693 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:06.693 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.099541+0000", 2024-09-18T10:15:06.693 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000169407 2024-09-18T10:15:06.693 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:06.693 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:06.693 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:06.693 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.693 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.693 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:324 3.6s0 3:61491e68:::smithi18925896-27:head [rollback 4] snapc 7=[4] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-18T10:15:06.693 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:01.795482+0000", 2024-09-18T10:15:06.693 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 4.8146007080000004, 2024-09-18T10:15:06.693 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.30451560100000002, 2024-09-18T10:15:06.693 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:06.694 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:06.694 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:06.694 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:06.694 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:06.694 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 324 2024-09-18T10:15:06.694 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.694 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:06.694 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.694 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:06.694 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.795482+0000", 2024-09-18T10:15:06.694 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:06.694 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.694 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.695 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:06.695 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.795482+0000", 2024-09-18T10:15:06.695 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:06.695 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.695 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.695 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:06.695 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.795480+0000", 2024-09-18T10:15:06.695 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:06.695 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.695 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.695 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:06.695 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.795487+0000", 2024-09-18T10:15:06.695 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7170000000000002e-06 2024-09-18T10:15:06.695 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.696 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.696 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:06.696 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.795548+0000", 2024-09-18T10:15:06.696 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1647000000000001e-05 2024-09-18T10:15:06.696 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.696 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.696 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:06.696 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.795566+0000", 2024-09-18T10:15:06.696 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7560000000000001e-05 2024-09-18T10:15:06.696 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.696 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.696 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:06.696 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.797676+0000", 2024-09-18T10:15:06.697 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0021101919999999999 2024-09-18T10:15:06.697 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.697 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.697 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:06.697 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.797742+0000", 2024-09-18T10:15:06.697 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.6351000000000002e-05 2024-09-18T10:15:06.697 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.697 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.697 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:06.697 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.798290+0000", 2024-09-18T10:15:06.697 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000548063 2024-09-18T10:15:06.697 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.697 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.697 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:06.698 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.800131+0000", 2024-09-18T10:15:06.698 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001840759 2024-09-18T10:15:06.698 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.698 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.698 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-18T10:15:06.698 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.099840+0000", 2024-09-18T10:15:06.698 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.29970881300000002 2024-09-18T10:15:06.698 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.698 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.698 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:06.698 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.099997+0000", 2024-09-18T10:15:06.698 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000157124 2024-09-18T10:15:06.698 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:06.699 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:06.699 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:06.699 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.699 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.699 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:325 3.6s0 3:61491e68:::smithi18925896-27:head [append 0~0] snapc 7=[4] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-18T10:15:06.699 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:01.795591+0000", 2024-09-18T10:15:06.699 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 4.8144916530000001, 2024-09-18T10:15:06.699 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.30495726400000001, 2024-09-18T10:15:06.699 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:06.699 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:06.699 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:06.699 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:06.699 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:06.700 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 325 2024-09-18T10:15:06.700 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.700 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:06.700 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.700 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:06.700 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.795591+0000", 2024-09-18T10:15:06.700 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:06.700 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.700 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.700 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:06.701 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.795591+0000", 2024-09-18T10:15:06.701 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:06.701 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.701 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.701 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:06.701 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.795589+0000", 2024-09-18T10:15:06.701 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:06.701 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.701 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.701 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:06.701 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.795598+0000", 2024-09-18T10:15:06.701 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.7940000000000008e-06 2024-09-18T10:15:06.701 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.702 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.702 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:06.702 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.795617+0000", 2024-09-18T10:15:06.702 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8485e-05 2024-09-18T10:15:06.702 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.702 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.702 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:06.702 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.795627+0000", 2024-09-18T10:15:06.702 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0730999999999999e-05 2024-09-18T10:15:06.702 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.702 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.702 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:06.702 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.798672+0000", 2024-09-18T10:15:06.703 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00304484 2024-09-18T10:15:06.703 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.703 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.703 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:06.703 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.798918+0000", 2024-09-18T10:15:06.703 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00024598000000000003 2024-09-18T10:15:06.703 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.703 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.703 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:06.703 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.799703+0000", 2024-09-18T10:15:06.703 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000785262 2024-09-18T10:15:06.703 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.703 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.703 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:06.704 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.900558+0000", 2024-09-18T10:15:06.704 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100854108 2024-09-18T10:15:06.704 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.704 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.704 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-18T10:15:06.704 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.100434+0000", 2024-09-18T10:15:06.704 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.19987638699999999 2024-09-18T10:15:06.704 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.704 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.704 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:06.704 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.100548+0000", 2024-09-18T10:15:06.704 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00011390799999999999 2024-09-18T10:15:06.704 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:06.705 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:06.705 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:06.705 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.705 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.705 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWriteReply(3.6s0 45/40 ECSubWriteReply(tid=45, last_complete=46'26, committed=1, applied=1))", 2024-09-18T10:15:06.705 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:01.898885+0000", 2024-09-18T10:15:06.705 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 4.7111970010000004, 2024-09-18T10:15:06.705 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.200675623, 2024-09-18T10:15:06.705 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:06.705 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:06.705 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:06.705 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.705 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:06.705 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.898885+0000", 2024-09-18T10:15:06.706 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:06.706 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.706 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.706 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:06.706 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.898885+0000", 2024-09-18T10:15:06.706 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:06.706 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.706 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.706 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:06.706 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.898884+0000", 2024-09-18T10:15:06.706 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.999999 2024-09-18T10:15:06.706 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.706 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.706 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:06.707 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.898894+0000", 2024-09-18T10:15:06.707 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.9369999999999999e-06 2024-09-18T10:15:06.707 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.707 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.707 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:06.707 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.898912+0000", 2024-09-18T10:15:06.707 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7472000000000001e-05 2024-09-18T10:15:06.707 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.707 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.707 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:06.707 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.999072+0000", 2024-09-18T10:15:06.707 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.1001601 2024-09-18T10:15:06.707 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.708 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.708 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:06.708 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.099237+0000", 2024-09-18T10:15:06.708 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100165611 2024-09-18T10:15:06.708 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.708 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.708 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:06.708 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.099561+0000", 2024-09-18T10:15:06.708 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00032355300000000002 2024-09-18T10:15:06.708 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:06.709 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:06.709 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:06.709 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.709 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.709 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWriteReply(3.6s0 46/40 ECSubWriteReply(tid=46, last_complete=46'27, committed=1, applied=1))", 2024-09-18T10:15:06.709 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:01.900010+0000", 2024-09-18T10:15:06.709 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 4.7100720259999997, 2024-09-18T10:15:06.709 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.19960616, 2024-09-18T10:15:06.709 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:06.709 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:06.709 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:06.709 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.709 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:06.710 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.900010+0000", 2024-09-18T10:15:06.710 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:06.710 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.710 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.710 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:06.710 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.900010+0000", 2024-09-18T10:15:06.710 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:06.710 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.710 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.710 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:06.710 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.900009+0000", 2024-09-18T10:15:06.710 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:06.710 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.711 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.711 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:06.711 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.900016+0000", 2024-09-18T10:15:06.711 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.3200000000000002e-06 2024-09-18T10:15:06.711 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.711 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.711 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:06.711 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.900029+0000", 2024-09-18T10:15:06.711 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.31e-05 2024-09-18T10:15:06.711 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.711 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.711 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:06.711 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.000137+0000", 2024-09-18T10:15:06.711 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100107454 2024-09-18T10:15:06.712 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.712 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.712 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:06.712 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.099602+0000", 2024-09-18T10:15:06.712 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099465233 2024-09-18T10:15:06.712 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.712 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.712 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:06.712 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.099616+0000", 2024-09-18T10:15:06.712 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4433000000000001e-05 2024-09-18T10:15:06.712 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:06.712 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:06.713 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:06.713 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.713 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.713 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "replica_scrub(pg: 3.8s1,from:0'0,to:41'13,epoch:47/40,start:3:10000000::::0,end:MAX,chunky:1,deep:0,version:9,allow_preemption:1,priority=5)", 2024-09-18T10:15:06.713 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:03.012324+0000", 2024-09-18T10:15:06.713 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 3.5977581669999998, 2024-09-18T10:15:06.713 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.20050876400000001, 2024-09-18T10:15:06.713 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:06.713 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:06.713 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:06.713 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.713 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:06.713 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:03.012324+0000", 2024-09-18T10:15:06.714 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:06.714 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.714 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.714 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:06.714 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:03.012324+0000", 2024-09-18T10:15:06.714 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:06.714 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.714 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.714 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:06.714 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:03.012324+0000", 2024-09-18T10:15:06.714 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:06.715 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.715 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.715 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:06.715 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:03.012347+0000", 2024-09-18T10:15:06.715 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3056000000000001e-05 2024-09-18T10:15:06.715 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.715 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.715 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:06.715 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:03.012374+0000", 2024-09-18T10:15:06.715 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6846e-05 2024-09-18T10:15:06.715 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.715 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.715 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:06.716 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:03.112471+0000", 2024-09-18T10:15:06.716 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100097857 2024-09-18T10:15:06.716 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.716 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.716 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:06.716 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:03.212715+0000", 2024-09-18T10:15:06.716 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10024356 2024-09-18T10:15:06.716 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.716 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.716 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:06.716 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:03.212740+0000", 2024-09-18T10:15:06.716 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.5083e-05 2024-09-18T10:15:06.717 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.717 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.717 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:06.717 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:03.212833+0000", 2024-09-18T10:15:06.717 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.2755000000000002e-05 2024-09-18T10:15:06.717 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:06.717 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:06.717 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:06.717 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:06.717 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:06.717 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-18T10:15:06.719 INFO:tasks.rados.rados.0.smithi189.stdout:151: finishing write tid 5 to smithi18925896-24 2024-09-18T10:15:06.719 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 24 v 27 (ObjNum 74 snap 7 seq_num 74) dirty exists 2024-09-18T10:15:06.719 INFO:tasks.rados.rados.0.smithi189.stdout:151: left oid 24 (ObjNum 74 snap 7 seq_num 74) 2024-09-18T10:15:06.720 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T10:15:06.720 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-18T10:15:06.721 INFO:teuthology.orchestra.run.smithi029.stdout:osd.6: {} 2024-09-18T10:15:06.721 INFO:teuthology.orchestra.run.smithi029.stdout:osd.7: {} 2024-09-18T10:15:06.721 INFO:teuthology.orchestra.run.smithi029.stdout:osd.8: {} 2024-09-18T10:15:06.721 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-18T10:15:06.717+0000 7f00046a8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 22136) UID: 0 2024-09-18T10:15:06.741 INFO:teuthology.orchestra.run.smithi029.stdout:osd.9: {} 2024-09-18T10:15:06.754 INFO:tasks.rados.rados.0.smithi189.stdout:150: finishing copy_from to smithi18925896-36 2024-09-18T10:15:06.754 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 36 v 2 (ObjNum 3 snap 0 seq_num 3) dirty exists 2024-09-18T10:15:06.755 INFO:tasks.rados.rados.0.smithi189.stdout:152: finishing write tid 3 to smithi18925896-6 2024-09-18T10:15:06.793 INFO:tasks.rados.rados.0.smithi189.stdout:145: finishing write tid 5 to smithi18925896-26 2024-09-18T10:15:06.793 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 26 v 17 (ObjNum 71 snap 7 seq_num 71) dirty exists 2024-09-18T10:15:06.793 INFO:tasks.rados.rados.0.smithi189.stdout:145: left oid 26 (ObjNum 71 snap 7 seq_num 71) 2024-09-18T10:15:06.819 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-09-18T10:15:06.815+0000 7fdfa8698700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22087) UID: 0 2024-09-18T10:15:06.840 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-18T10:15:06.840 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-18T10:15:06.840 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.841 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpReadReply(3.7s0 51/40 ECSubReadReply(tid=88, attrs_read=0))", 2024-09-18T10:15:06.841 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:06.793913+0000", 2024-09-18T10:15:06.841 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.045543318999999999, 2024-09-18T10:15:06.841 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.045578072999999997, 2024-09-18T10:15:06.841 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:06.841 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-18T10:15:06.841 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:06.841 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.841 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:06.841 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:06.793913+0000", 2024-09-18T10:15:06.841 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:06.841 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.841 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.842 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:06.842 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:06.793913+0000", 2024-09-18T10:15:06.842 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:06.842 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.842 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.842 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:06.842 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:06.793913+0000", 2024-09-18T10:15:06.842 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:06.842 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.842 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.843 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:06.843 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:06.794050+0000", 2024-09-18T10:15:06.843 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00013724399999999999 2024-09-18T10:15:06.843 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.843 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.843 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:06.843 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:06.794062+0000", 2024-09-18T10:15:06.843 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1719000000000001e-05 2024-09-18T10:15:06.843 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.843 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.843 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:06.843 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:06.794068+0000", 2024-09-18T10:15:06.844 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0310000000000004e-06 2024-09-18T10:15:06.844 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:06.844 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:06.844 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:06.844 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.844 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.844 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpReadReply(3.7s0 51/40 ECSubReadReply(tid=88, attrs_read=0))", 2024-09-18T10:15:06.845 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:06.794767+0000", 2024-09-18T10:15:06.845 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.044689502999999998, 2024-09-18T10:15:06.845 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.044793425999999997, 2024-09-18T10:15:06.845 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:06.845 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-18T10:15:06.845 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:06.845 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.845 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:06.845 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:06.794767+0000", 2024-09-18T10:15:06.845 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:06.846 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.846 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.846 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:06.846 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:06.794767+0000", 2024-09-18T10:15:06.846 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:06.846 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.846 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.846 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:06.846 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:06.794767+0000", 2024-09-18T10:15:06.846 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:06.846 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.846 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.846 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:06.847 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:06.794902+0000", 2024-09-18T10:15:06.847 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000135339 2024-09-18T10:15:06.847 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.847 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.847 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:06.847 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:06.794971+0000", 2024-09-18T10:15:06.847 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.881e-05 2024-09-18T10:15:06.847 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.847 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.847 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:06.847 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:06.794985+0000", 2024-09-18T10:15:06.847 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4014e-05 2024-09-18T10:15:06.847 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:06.848 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:06.848 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:06.848 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.848 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.848 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpRead(3.7s0 51/40 ECSubRead(tid=89, to_read={3:e4da5b5e:::smithi18925896-43:head=0,180224,0}, subchunks={3:e4da5b5e:::smithi18925896-43:head=[0,1]}, attrs_to_read=))", 2024-09-18T10:15:06.848 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:06.795749+0000", 2024-09-18T10:15:06.848 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.043707825999999998, 2024-09-18T10:15:06.848 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.043883534000000002, 2024-09-18T10:15:06.848 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:06.848 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-18T10:15:06.848 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:06.848 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.848 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:06.849 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:06.795749+0000", 2024-09-18T10:15:06.849 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:06.849 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.849 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.849 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:06.849 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:06.795749+0000", 2024-09-18T10:15:06.849 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:06.849 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.849 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.849 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:06.849 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:06.795749+0000", 2024-09-18T10:15:06.849 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:06.850 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.850 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.850 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:06.850 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:06.795749+0000", 2024-09-18T10:15:06.850 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:06.850 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.850 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.850 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:06.850 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:06.795789+0000", 2024-09-18T10:15:06.850 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0367000000000002e-05 2024-09-18T10:15:06.850 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.850 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.850 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:06.850 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:06.795803+0000", 2024-09-18T10:15:06.851 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3576e-05 2024-09-18T10:15:06.851 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:06.851 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:06.851 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:06.851 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.851 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.851 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpReadReply(3.7s0 51/40 ECSubReadReply(tid=89, attrs_read=0))", 2024-09-18T10:15:06.851 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:06.796314+0000", 2024-09-18T10:15:06.851 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.043142119999999999, 2024-09-18T10:15:06.851 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.043373515000000001, 2024-09-18T10:15:06.851 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:06.851 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-18T10:15:06.851 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:06.851 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.852 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:06.852 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:06.796314+0000", 2024-09-18T10:15:06.852 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:06.852 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.852 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.852 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:06.852 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:06.796314+0000", 2024-09-18T10:15:06.852 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:06.852 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.852 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.852 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:06.852 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:06.796313+0000", 2024-09-18T10:15:06.852 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:06.853 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.853 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.853 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:06.853 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:06.796519+0000", 2024-09-18T10:15:06.853 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00020534899999999999 2024-09-18T10:15:06.853 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.853 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.853 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:06.853 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:06.796543+0000", 2024-09-18T10:15:06.853 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.4649000000000001e-05 2024-09-18T10:15:06.853 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:06.853 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:06.853 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:06.854 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:06.796554+0000", 2024-09-18T10:15:06.854 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0308e-05 2024-09-18T10:15:06.854 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:06.854 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:06.854 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:06.854 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:06.854 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-18T10:15:06.854 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 4 2024-09-18T10:15:06.854 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-18T10:15:06.855 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T10:15:06.867 INFO:teuthology.orchestra.run.smithi029.stdout:osd.10: {} 2024-09-18T10:15:06.894 INFO:teuthology.orchestra.run.smithi029.stdout:osd.11: {} 2024-09-18T10:15:06.900 INFO:tasks.rados.rados.0.smithi189.stdout:146: expect (ObjNum 42 snap 0 seq_num 42) 2024-09-18T10:15:06.914 INFO:teuthology.orchestra.run.smithi029.stdout:osd.12: {} 2024-09-18T10:15:06.921 INFO:tasks.ceph.osd.4.smithi029.stderr:2024-09-18T10:15:06.914+0000 7f550cf6e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22189) UID: 0 2024-09-18T10:15:06.935 INFO:teuthology.orchestra.run.smithi029.stdout:osd.13: {} 2024-09-18T10:15:06.957 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-18T10:15:06.957 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-18T10:15:06.957 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-18T10:15:06.957 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-18T10:15:06.957 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-18T10:15:06.958 INFO:teuthology.orchestra.run.smithi029.stdout:osd.14: {} 2024-09-18T10:15:06.966 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T10:15:06.980 INFO:teuthology.orchestra.run.smithi029.stdout:osd.15: {} 2024-09-18T10:15:06.983 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T10:15:06.983 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T10:15:06.983 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T10:15:06.983 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T10:15:06.983 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T10:15:06.983 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T10:15:06.983 INFO:teuthology.orchestra.run.smithi029.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T10:15:06.983 INFO:teuthology.orchestra.run.smithi029.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T10:15:06.984 INFO:teuthology.orchestra.run.smithi029.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-18T10:15:06.984 INFO:teuthology.orchestra.run.smithi029.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-18T10:15:06.984 INFO:teuthology.orchestra.run.smithi029.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-18T10:15:06.984 INFO:teuthology.orchestra.run.smithi029.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-18T10:15:06.984 INFO:teuthology.orchestra.run.smithi029.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-18T10:15:06.984 INFO:teuthology.orchestra.run.smithi029.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-18T10:15:06.984 INFO:teuthology.orchestra.run.smithi029.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-18T10:15:06.984 INFO:teuthology.orchestra.run.smithi029.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-18T10:15:07.004 INFO:tasks.rados.rados.0.smithi189.stdout:157: finishing write tid 1 to smithi18925896-47 2024-09-18T10:15:07.022 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-18T10:15:07.018+0000 7f302f2c3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22187) UID: 0 2024-09-18T10:15:07.059 INFO:tasks.rados.rados.0.smithi189.stdout:152: finishing write tid 4 to smithi18925896-6 2024-09-18T10:15:07.059 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 6 v 46 (ObjNum 75 snap 7 seq_num 75) dirty exists 2024-09-18T10:15:07.059 INFO:tasks.rados.rados.0.smithi189.stdout:152: left oid 6 (ObjNum 75 snap 7 seq_num 75) 2024-09-18T10:15:07.068 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-18T10:15:07.068 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-18T10:15:07.068 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-18T10:15:07.068 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-18T10:15:07.068 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.069 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpReadReply(3.7s0 36/32 ECSubReadReply(tid=20, attrs_read=0))", 2024-09-18T10:15:07.069 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:50.497238+0000", 2024-09-18T10:15:07.069 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 16.566844449000001, 2024-09-18T10:15:07.069 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.40240860299999998, 2024-09-18T10:15:07.069 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:07.069 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:07.069 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:07.069 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.069 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:07.069 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.497238+0000", 2024-09-18T10:15:07.070 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:07.070 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.070 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.070 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:07.070 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.497238+0000", 2024-09-18T10:15:07.070 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:07.070 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.070 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.070 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:07.070 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.497236+0000", 2024-09-18T10:15:07.070 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:07.070 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.070 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.071 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:07.071 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.497270+0000", 2024-09-18T10:15:07.071 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.3392999999999997e-05 2024-09-18T10:15:07.071 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.071 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.071 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:07.071 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.497287+0000", 2024-09-18T10:15:07.071 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7444000000000001e-05 2024-09-18T10:15:07.071 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.071 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.071 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:07.071 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.497299+0000", 2024-09-18T10:15:07.071 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1518000000000001e-05 2024-09-18T10:15:07.071 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.071 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.072 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:07.072 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.899303+0000", 2024-09-18T10:15:07.072 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.40200464699999999 2024-09-18T10:15:07.072 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.072 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.072 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:07.072 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.899647+0000", 2024-09-18T10:15:07.072 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00034345900000000001 2024-09-18T10:15:07.072 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:07.072 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:07.072 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:07.072 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.073 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.073 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:119 3.bs0 3:d62b3dfa:::smithi18925896-40:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:07.073 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:50.705490+0000", 2024-09-18T10:15:07.073 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 16.358592475000002, 2024-09-18T10:15:07.073 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.40546939399999998, 2024-09-18T10:15:07.073 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:07.073 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:07.073 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:07.073 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:07.073 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:07.074 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 119 2024-09-18T10:15:07.074 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.074 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:07.074 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.074 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:07.074 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.705490+0000", 2024-09-18T10:15:07.074 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:07.074 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.074 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.074 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:07.074 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.705490+0000", 2024-09-18T10:15:07.075 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:07.075 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.075 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.075 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:07.075 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.705488+0000", 2024-09-18T10:15:07.075 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:07.075 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.075 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.075 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:07.075 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.705498+0000", 2024-09-18T10:15:07.075 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0393e-05 2024-09-18T10:15:07.075 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.076 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.076 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:07.076 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.705523+0000", 2024-09-18T10:15:07.076 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.499e-05 2024-09-18T10:15:07.076 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.076 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.076 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:07.076 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.705536+0000", 2024-09-18T10:15:07.076 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2833000000000001e-05 2024-09-18T10:15:07.076 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.076 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.077 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:07.077 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.810302+0000", 2024-09-18T10:15:07.077 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.104765231 2024-09-18T10:15:07.077 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.077 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.077 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:07.077 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.810480+0000", 2024-09-18T10:15:07.077 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00017834599999999999 2024-09-18T10:15:07.077 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.077 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.077 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:07.077 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.811463+0000", 2024-09-18T10:15:07.077 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00098265799999999997 2024-09-18T10:15:07.077 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.078 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.078 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:07.078 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.812898+0000", 2024-09-18T10:15:07.078 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0014352360000000001 2024-09-18T10:15:07.078 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.078 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.078 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-18T10:15:07.078 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.110742+0000", 2024-09-18T10:15:07.078 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.29784466900000001 2024-09-18T10:15:07.078 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.078 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.078 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:07.078 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.110959+0000", 2024-09-18T10:15:07.078 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000217035 2024-09-18T10:15:07.079 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:07.079 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:07.079 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:07.079 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.079 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.079 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:120 3.bs0 3:d62b3dfa:::smithi18925896-40:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:07.079 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:50.705551+0000", 2024-09-18T10:15:07.079 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 16.358531632999998, 2024-09-18T10:15:07.079 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.508399406, 2024-09-18T10:15:07.079 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:07.079 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:07.080 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:07.080 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:07.080 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:07.080 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 120 2024-09-18T10:15:07.080 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.080 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:07.080 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.080 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:07.080 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.705551+0000", 2024-09-18T10:15:07.080 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:07.080 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.080 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.080 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:07.080 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.705551+0000", 2024-09-18T10:15:07.081 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:07.081 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.081 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.081 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:07.081 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.705549+0000", 2024-09-18T10:15:07.081 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:07.081 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.081 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.081 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:07.081 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.705557+0000", 2024-09-18T10:15:07.081 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.9500000000000001e-06 2024-09-18T10:15:07.081 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.081 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.082 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:07.082 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.705575+0000", 2024-09-18T10:15:07.082 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7989999999999999e-05 2024-09-18T10:15:07.082 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.082 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.082 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:07.082 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.705587+0000", 2024-09-18T10:15:07.082 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1226000000000001e-05 2024-09-18T10:15:07.082 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.082 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.082 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:07.082 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.812099+0000", 2024-09-18T10:15:07.082 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.106512116 2024-09-18T10:15:07.083 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.083 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.083 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for rw locks", 2024-09-18T10:15:07.083 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.812429+0000", 2024-09-18T10:15:07.083 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00033007300000000001 2024-09-18T10:15:07.083 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.083 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.083 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:07.083 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.111941+0000", 2024-09-18T10:15:07.083 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.29951245700000001 2024-09-18T10:15:07.083 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.083 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.083 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:07.083 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.112099+0000", 2024-09-18T10:15:07.084 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00015813399999999999 2024-09-18T10:15:07.084 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.084 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.084 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:07.084 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.213950+0000", 2024-09-18T10:15:07.084 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10185091 2024-09-18T10:15:07.084 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:07.084 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:07.084 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:07.084 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.084 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.084 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:127 3.7s0 3:e4da5b5e:::smithi18925896-43:head [append 0~720896,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:07.084 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:50.712808+0000", 2024-09-18T10:15:07.085 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 16.351274760999999, 2024-09-18T10:15:07.085 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.40445257800000001, 2024-09-18T10:15:07.085 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:07.085 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:07.085 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:07.085 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:07.085 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:07.085 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 127 2024-09-18T10:15:07.085 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.085 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:07.086 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.086 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:07.086 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.712808+0000", 2024-09-18T10:15:07.086 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:07.086 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.086 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.087 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:07.087 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.712808+0000", 2024-09-18T10:15:07.087 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:07.087 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.087 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.087 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:07.087 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.712805+0000", 2024-09-18T10:15:07.087 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999971 2024-09-18T10:15:07.087 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.087 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.087 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:07.087 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.713866+0000", 2024-09-18T10:15:07.088 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0010617070000000001 2024-09-18T10:15:07.088 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.088 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.088 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:07.088 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.713900+0000", 2024-09-18T10:15:07.088 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.3575999999999998e-05 2024-09-18T10:15:07.088 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.088 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.088 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:07.088 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.713913+0000", 2024-09-18T10:15:07.088 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2823000000000001e-05 2024-09-18T10:15:07.088 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.088 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.088 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:07.089 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.900562+0000", 2024-09-18T10:15:07.089 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.18664886 2024-09-18T10:15:07.089 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.089 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.089 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:07.089 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.900856+0000", 2024-09-18T10:15:07.089 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000294617 2024-09-18T10:15:07.089 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.089 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.089 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:07.089 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.902520+0000", 2024-09-18T10:15:07.089 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00166351 2024-09-18T10:15:07.089 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.090 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.090 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:07.090 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.905574+0000", 2024-09-18T10:15:07.090 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.003054583 2024-09-18T10:15:07.090 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.090 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.090 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-18T10:15:07.090 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.117162+0000", 2024-09-18T10:15:07.090 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.211587309 2024-09-18T10:15:07.090 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.090 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.090 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:07.091 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.117260+0000", 2024-09-18T10:15:07.091 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.8640000000000004e-05 2024-09-18T10:15:07.091 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:07.091 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:07.091 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:07.091 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.091 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.091 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:128 3.7s0 3:e4da5b5e:::smithi18925896-43:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:07.091 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:50.713927+0000", 2024-09-18T10:15:07.091 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 16.350155574999999, 2024-09-18T10:15:07.091 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.50422476699999996, 2024-09-18T10:15:07.091 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:07.091 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:07.091 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:07.092 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:07.092 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:07.092 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 128 2024-09-18T10:15:07.092 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.092 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:07.092 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.092 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:07.092 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.713927+0000", 2024-09-18T10:15:07.092 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:07.092 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.092 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.092 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:07.092 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.713927+0000", 2024-09-18T10:15:07.092 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:07.093 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.093 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.093 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:07.093 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.713925+0000", 2024-09-18T10:15:07.093 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:07.093 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.093 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.093 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:07.093 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.713933+0000", 2024-09-18T10:15:07.093 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.2069999999999998e-06 2024-09-18T10:15:07.093 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.093 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.093 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:07.094 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.713947+0000", 2024-09-18T10:15:07.094 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4249e-05 2024-09-18T10:15:07.094 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.094 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.094 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:07.094 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.713954+0000", 2024-09-18T10:15:07.094 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8070000000000002e-06 2024-09-18T10:15:07.094 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.094 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.094 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:07.094 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.904149+0000", 2024-09-18T10:15:07.094 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.19019514800000001 2024-09-18T10:15:07.094 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.095 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.095 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:07.095 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.904338+0000", 2024-09-18T10:15:07.095 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000189326 2024-09-18T10:15:07.095 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.095 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.095 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:07.095 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.905212+0000", 2024-09-18T10:15:07.095 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00087334100000000003 2024-09-18T10:15:07.095 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.095 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.095 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:07.095 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.906215+0000", 2024-09-18T10:15:07.095 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0010037430000000001 2024-09-18T10:15:07.096 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.096 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.096 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-18T10:15:07.096 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.217903+0000", 2024-09-18T10:15:07.096 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.31168731500000002 2024-09-18T10:15:07.096 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.096 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.096 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:07.096 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.218152+0000", 2024-09-18T10:15:07.096 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00024915799999999997 2024-09-18T10:15:07.096 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:07.096 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:07.096 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:07.097 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.097 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.097 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:129 3.7s0 3:e4da5b5e:::smithi18925896-43:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:07.097 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:50.713961+0000", 2024-09-18T10:15:07.097 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 16.350121406, 2024-09-18T10:15:07.097 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.819334071, 2024-09-18T10:15:07.097 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:07.097 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:07.097 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:07.097 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:07.097 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:07.097 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 129 2024-09-18T10:15:07.097 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.097 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:07.098 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.098 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:07.098 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.713961+0000", 2024-09-18T10:15:07.098 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:07.098 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.098 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.098 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:07.098 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.713961+0000", 2024-09-18T10:15:07.098 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:07.098 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.098 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.098 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:07.098 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.713960+0000", 2024-09-18T10:15:07.099 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.999999 2024-09-18T10:15:07.099 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.099 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.099 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:07.099 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.713964+0000", 2024-09-18T10:15:07.099 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.3059999999999999e-06 2024-09-18T10:15:07.099 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.099 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.099 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:07.099 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.713974+0000", 2024-09-18T10:15:07.099 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.9969999999999993e-06 2024-09-18T10:15:07.099 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.100 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.100 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:07.100 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.713981+0000", 2024-09-18T10:15:07.100 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1229999999999997e-06 2024-09-18T10:15:07.100 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.100 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.100 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:07.100 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.000661+0000", 2024-09-18T10:15:07.100 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.28668068699999999 2024-09-18T10:15:07.101 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.101 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.101 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for rw locks", 2024-09-18T10:15:07.101 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.000844+0000", 2024-09-18T10:15:07.101 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00018289599999999999 2024-09-18T10:15:07.101 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.101 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.101 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:07.101 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.219209+0000", 2024-09-18T10:15:07.102 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.21836466900000001 2024-09-18T10:15:07.102 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.102 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.102 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:07.102 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.219351+0000", 2024-09-18T10:15:07.102 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00014262300000000001 2024-09-18T10:15:07.102 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.102 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.102 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:07.103 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.533295+0000", 2024-09-18T10:15:07.103 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.31394377299999998 2024-09-18T10:15:07.103 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:07.103 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:07.103 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:07.103 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.103 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.103 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:156 3.7s0 3:ead53b0f:::smithi18925896-25:head [append 0~475136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:07.103 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:51.224034+0000", 2024-09-18T10:15:07.103 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 15.840048372, 2024-09-18T10:15:07.103 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.40749072600000003, 2024-09-18T10:15:07.103 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:07.103 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:07.104 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:07.104 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:07.104 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:07.104 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 156 2024-09-18T10:15:07.104 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.104 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:07.104 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.104 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:07.104 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.224034+0000", 2024-09-18T10:15:07.104 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:07.104 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.104 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.104 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:07.105 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.224034+0000", 2024-09-18T10:15:07.105 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:07.105 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.105 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.105 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:07.105 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.224032+0000", 2024-09-18T10:15:07.105 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:07.105 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.105 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.105 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:07.105 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.224742+0000", 2024-09-18T10:15:07.105 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00070980299999999995 2024-09-18T10:15:07.106 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.106 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.106 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:07.106 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.224767+0000", 2024-09-18T10:15:07.106 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.5415999999999999e-05 2024-09-18T10:15:07.106 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.106 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.106 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:07.106 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.224790+0000", 2024-09-18T10:15:07.106 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2932999999999999e-05 2024-09-18T10:15:07.106 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.106 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.106 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:07.107 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.224914+0000", 2024-09-18T10:15:07.107 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000123261 2024-09-18T10:15:07.107 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.107 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.107 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:07.107 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.225098+0000", 2024-09-18T10:15:07.107 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000184606 2024-09-18T10:15:07.107 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.107 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.107 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:07.107 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.226615+0000", 2024-09-18T10:15:07.107 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001517114 2024-09-18T10:15:07.107 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.107 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.108 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:07.108 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.331612+0000", 2024-09-18T10:15:07.108 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.104996909 2024-09-18T10:15:07.108 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.108 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.108 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-18T10:15:07.108 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.631319+0000", 2024-09-18T10:15:07.108 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.29970667299999998 2024-09-18T10:15:07.108 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.108 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.108 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:07.108 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.631525+0000", 2024-09-18T10:15:07.108 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000206011 2024-09-18T10:15:07.109 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:07.109 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:07.109 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:07.109 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.109 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.109 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:157 3.7s0 3:ead53b0f:::smithi18925896-25:head [append 0~376832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:07.109 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:51.226166+0000", 2024-09-18T10:15:07.109 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 15.837916224000001, 2024-09-18T10:15:07.109 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.405859368, 2024-09-18T10:15:07.109 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:07.109 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:07.109 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:07.109 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:07.109 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:07.110 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 157 2024-09-18T10:15:07.110 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.110 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:07.110 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.110 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:07.110 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.226166+0000", 2024-09-18T10:15:07.110 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:07.110 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.110 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.110 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:07.110 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.226166+0000", 2024-09-18T10:15:07.110 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:07.110 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.110 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.111 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:07.111 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.226165+0000", 2024-09-18T10:15:07.111 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:07.111 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.111 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.111 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:07.111 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.226889+0000", 2024-09-18T10:15:07.111 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00072377800000000001 2024-09-18T10:15:07.111 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.111 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.111 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:07.111 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.226913+0000", 2024-09-18T10:15:07.111 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3926000000000001e-05 2024-09-18T10:15:07.111 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.112 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.112 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:07.112 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.226925+0000", 2024-09-18T10:15:07.112 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2575000000000001e-05 2024-09-18T10:15:07.112 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.112 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.112 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:07.112 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.227730+0000", 2024-09-18T10:15:07.112 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00080524299999999997 2024-09-18T10:15:07.112 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.112 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.112 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:07.112 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.227893+0000", 2024-09-18T10:15:07.113 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00016229300000000001 2024-09-18T10:15:07.113 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.113 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.113 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:07.113 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.228969+0000", 2024-09-18T10:15:07.113 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001076341 2024-09-18T10:15:07.113 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.113 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.113 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:07.113 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.331813+0000", 2024-09-18T10:15:07.113 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10284409899999999 2024-09-18T10:15:07.113 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.113 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.114 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-18T10:15:07.114 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.631841+0000", 2024-09-18T10:15:07.114 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.30002807500000001 2024-09-18T10:15:07.114 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.114 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.114 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:07.114 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.632026+0000", 2024-09-18T10:15:07.114 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000184546 2024-09-18T10:15:07.114 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:07.114 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:07.114 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:07.114 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.115 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.115 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:158 3.7s0 3:ead53b0f:::smithi18925896-25:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:07.115 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:51.226948+0000", 2024-09-18T10:15:07.115 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 15.837134279000001, 2024-09-18T10:15:07.115 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.405732802, 2024-09-18T10:15:07.115 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:07.115 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:07.115 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:07.115 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:07.115 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:07.115 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 158 2024-09-18T10:15:07.116 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.116 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:07.116 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.116 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:07.116 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.226948+0000", 2024-09-18T10:15:07.116 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:07.116 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.116 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.116 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:07.116 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.226948+0000", 2024-09-18T10:15:07.116 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:07.116 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.116 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.117 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:07.117 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.226947+0000", 2024-09-18T10:15:07.117 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.999999 2024-09-18T10:15:07.117 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.117 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.117 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:07.117 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.226954+0000", 2024-09-18T10:15:07.117 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.6200000000000001e-06 2024-09-18T10:15:07.117 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.117 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.117 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:07.117 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.226968+0000", 2024-09-18T10:15:07.117 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3866000000000001e-05 2024-09-18T10:15:07.117 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.118 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.118 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:07.118 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.226977+0000", 2024-09-18T10:15:07.118 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.9530000000000005e-06 2024-09-18T10:15:07.118 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.118 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.118 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:07.118 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.329807+0000", 2024-09-18T10:15:07.118 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10283049700000001 2024-09-18T10:15:07.119 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.119 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.119 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:07.119 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.329976+0000", 2024-09-18T10:15:07.119 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000169226 2024-09-18T10:15:07.119 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.119 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.119 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:07.119 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.330989+0000", 2024-09-18T10:15:07.119 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001012825 2024-09-18T10:15:07.119 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.119 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.120 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:07.120 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.432222+0000", 2024-09-18T10:15:07.120 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10123270400000001 2024-09-18T10:15:07.120 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.120 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.120 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-18T10:15:07.120 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.632401+0000", 2024-09-18T10:15:07.120 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.200179352 2024-09-18T10:15:07.120 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.120 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.120 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:07.120 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.632681+0000", 2024-09-18T10:15:07.120 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00027994700000000001 2024-09-18T10:15:07.120 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:07.121 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:07.121 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:07.121 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.121 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.121 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:159 3.7s0 3:ead53b0f:::smithi18925896-25:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:07.121 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:51.226988+0000", 2024-09-18T10:15:07.121 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 15.837094346000001, 2024-09-18T10:15:07.121 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.81465928499999996, 2024-09-18T10:15:07.121 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:07.121 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:07.121 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:07.121 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:07.121 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:07.122 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 159 2024-09-18T10:15:07.122 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.122 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:07.122 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.122 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:07.122 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.226988+0000", 2024-09-18T10:15:07.122 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:07.122 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.122 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.122 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:07.122 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.226988+0000", 2024-09-18T10:15:07.122 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:07.122 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.122 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.123 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:07.123 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.226987+0000", 2024-09-18T10:15:07.123 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.999999 2024-09-18T10:15:07.123 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.123 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.123 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:07.123 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.226992+0000", 2024-09-18T10:15:07.123 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.5900000000000001e-06 2024-09-18T10:15:07.123 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.123 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.123 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:07.123 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.227003+0000", 2024-09-18T10:15:07.123 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1222e-05 2024-09-18T10:15:07.124 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.124 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.124 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:07.124 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.227011+0000", 2024-09-18T10:15:07.124 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.6279999999999996e-06 2024-09-18T10:15:07.124 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.124 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.124 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:07.124 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.533792+0000", 2024-09-18T10:15:07.124 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.30678158100000003 2024-09-18T10:15:07.124 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.124 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.125 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for rw locks", 2024-09-18T10:15:07.125 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.533941+0000", 2024-09-18T10:15:07.125 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00014846 2024-09-18T10:15:07.125 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.125 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.125 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:07.125 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.633576+0000", 2024-09-18T10:15:07.125 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099635153000000004 2024-09-18T10:15:07.125 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.125 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.125 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:07.125 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.633832+0000", 2024-09-18T10:15:07.125 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00025610699999999999 2024-09-18T10:15:07.125 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.126 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.126 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:07.126 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.041647+0000", 2024-09-18T10:15:07.126 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.407815388 2024-09-18T10:15:07.126 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:07.126 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:07.126 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:07.126 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.126 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.126 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:166 3.1s0 3:82ca2386:::smithi18925896-8:head [read 0~131072,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:07.126 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:51.531989+0000", 2024-09-18T10:15:07.126 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 15.532093606, 2024-09-18T10:15:07.126 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.40380508900000001, 2024-09-18T10:15:07.126 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:07.126 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:07.127 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:07.127 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:07.127 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:07.127 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 166 2024-09-18T10:15:07.127 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.127 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:07.127 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.127 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:07.127 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.531989+0000", 2024-09-18T10:15:07.127 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:07.127 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.127 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.127 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:07.128 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.531989+0000", 2024-09-18T10:15:07.128 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:07.128 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.128 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.128 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:07.128 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.531987+0000", 2024-09-18T10:15:07.128 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:07.128 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.128 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.128 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:07.128 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.531997+0000", 2024-09-18T10:15:07.128 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0010000000000001e-05 2024-09-18T10:15:07.128 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.128 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.129 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:07.129 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.532017+0000", 2024-09-18T10:15:07.129 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9358999999999999e-05 2024-09-18T10:15:07.129 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.129 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.129 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:07.129 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.532032+0000", 2024-09-18T10:15:07.129 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4739999999999999e-05 2024-09-18T10:15:07.129 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.129 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.129 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:07.129 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.733011+0000", 2024-09-18T10:15:07.129 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.200979398 2024-09-18T10:15:07.129 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.130 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.130 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:07.130 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.733223+0000", 2024-09-18T10:15:07.130 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00021232799999999999 2024-09-18T10:15:07.130 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.130 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.130 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:07.130 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.935794+0000", 2024-09-18T10:15:07.130 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.20257077400000001 2024-09-18T10:15:07.130 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:07.130 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:07.130 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:07.130 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.131 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.131 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:169 3.7s0 3:e1df0280:::smithi18925896-26:head [copy-from ver 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:07.131 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:51.532055+0000", 2024-09-18T10:15:07.131 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 15.532027158, 2024-09-18T10:15:07.131 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.60955003100000005, 2024-09-18T10:15:07.131 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:07.131 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:07.131 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:07.131 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:07.131 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:07.131 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 169 2024-09-18T10:15:07.131 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.131 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:07.131 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.132 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:07.132 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.532055+0000", 2024-09-18T10:15:07.132 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:07.132 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.132 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.132 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:07.132 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.532055+0000", 2024-09-18T10:15:07.132 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:07.132 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.132 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.132 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:07.132 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.532054+0000", 2024-09-18T10:15:07.132 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:07.133 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.133 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.133 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:07.133 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.532062+0000", 2024-09-18T10:15:07.133 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0430000000000008e-06 2024-09-18T10:15:07.133 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.133 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.133 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:07.133 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.532080+0000", 2024-09-18T10:15:07.133 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8283000000000001e-05 2024-09-18T10:15:07.133 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.133 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.133 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:07.133 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.532092+0000", 2024-09-18T10:15:07.133 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1742999999999999e-05 2024-09-18T10:15:07.134 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.134 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.134 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:07.134 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.534038+0000", 2024-09-18T10:15:07.134 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0019461610000000001 2024-09-18T10:15:07.134 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.134 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.134 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:07.134 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.534285+0000", 2024-09-18T10:15:07.134 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00024667800000000001 2024-09-18T10:15:07.134 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.134 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.134 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:07.135 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.938506+0000", 2024-09-18T10:15:07.135 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.40422132199999999 2024-09-18T10:15:07.135 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.135 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.135 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:07.135 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.041810+0000", 2024-09-18T10:15:07.135 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10330328900000001 2024-09-18T10:15:07.135 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.135 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.135 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-18T10:15:07.135 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.141492+0000", 2024-09-18T10:15:07.135 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099682251999999999 2024-09-18T10:15:07.135 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.135 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.136 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:07.136 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.141605+0000", 2024-09-18T10:15:07.136 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00011349300000000001 2024-09-18T10:15:07.136 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:07.136 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:07.136 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:07.136 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.136 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.136 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:170 3.7s0 3:e1df0280:::smithi18925896-26:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:07.136 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:51.532106+0000", 2024-09-18T10:15:07.136 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 15.531976879, 2024-09-18T10:15:07.136 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.50908735800000005, 2024-09-18T10:15:07.136 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:07.136 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:07.137 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:07.137 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:07.137 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:07.137 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 170 2024-09-18T10:15:07.137 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.137 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:07.137 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.137 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:07.137 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.532106+0000", 2024-09-18T10:15:07.137 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:07.137 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.137 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.137 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:07.138 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.532106+0000", 2024-09-18T10:15:07.138 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:07.138 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.138 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.138 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:07.138 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.532104+0000", 2024-09-18T10:15:07.138 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:07.138 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.138 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.138 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:07.138 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.532113+0000", 2024-09-18T10:15:07.138 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8659999999999995e-06 2024-09-18T10:15:07.138 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.139 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.139 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:07.139 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.532131+0000", 2024-09-18T10:15:07.139 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7723999999999998e-05 2024-09-18T10:15:07.139 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.139 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.139 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:07.139 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.532142+0000", 2024-09-18T10:15:07.139 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0669000000000001e-05 2024-09-18T10:15:07.139 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.139 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.139 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:07.140 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.534705+0000", 2024-09-18T10:15:07.140 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0025635380000000002 2024-09-18T10:15:07.140 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.140 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.140 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for blocked object", 2024-09-18T10:15:07.140 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.534831+0000", 2024-09-18T10:15:07.140 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00012540399999999999 2024-09-18T10:15:07.140 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.140 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.140 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:07.140 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.040545+0000", 2024-09-18T10:15:07.140 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.50571483699999997 2024-09-18T10:15:07.140 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.140 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.141 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:07.141 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.040869+0000", 2024-09-18T10:15:07.141 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00032386999999999997 2024-09-18T10:15:07.141 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.141 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.141 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:07.141 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.041193+0000", 2024-09-18T10:15:07.141 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000323803 2024-09-18T10:15:07.141 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:07.141 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:07.141 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:07.141 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.141 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.142 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:210 3.7s0 3:e4da5b5e:::smithi18925896-43:head [read 0~720896,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:07.142 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:52.694421+0000", 2024-09-18T10:15:07.142 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 14.369661412999999, 2024-09-18T10:15:07.142 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.40910238700000001, 2024-09-18T10:15:07.142 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:07.142 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:07.142 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:07.142 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:07.142 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:07.142 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 210 2024-09-18T10:15:07.142 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.142 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:07.142 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.142 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:07.143 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.694421+0000", 2024-09-18T10:15:07.143 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:07.143 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.143 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.143 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:07.143 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.694421+0000", 2024-09-18T10:15:07.143 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:07.143 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.143 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.143 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:07.143 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.694420+0000", 2024-09-18T10:15:07.143 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:07.143 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.144 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.144 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:07.144 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.694428+0000", 2024-09-18T10:15:07.144 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0520000000000002e-06 2024-09-18T10:15:07.144 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.144 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.144 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:07.144 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.694445+0000", 2024-09-18T10:15:07.144 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7479e-05 2024-09-18T10:15:07.144 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.144 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.144 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:07.145 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.694456+0000", 2024-09-18T10:15:07.145 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0302999999999999e-05 2024-09-18T10:15:07.145 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.145 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.145 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:07.145 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.694858+0000", 2024-09-18T10:15:07.145 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00040244199999999998 2024-09-18T10:15:07.145 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.145 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.145 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:07.145 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.694874+0000", 2024-09-18T10:15:07.145 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.6248000000000001e-05 2024-09-18T10:15:07.145 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.146 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.146 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:07.146 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.896790+0000", 2024-09-18T10:15:07.146 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.201915813 2024-09-18T10:15:07.146 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.146 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.146 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:07.146 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.897060+0000", 2024-09-18T10:15:07.146 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00026979400000000001 2024-09-18T10:15:07.146 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.146 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.146 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:07.147 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:53.103524+0000", 2024-09-18T10:15:07.147 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.20646349999999999 2024-09-18T10:15:07.147 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:07.147 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:07.147 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:07.147 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.147 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.147 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:211 3.7s0 3:e4da5b5e:::smithi18925896-43:head [read 0~720896,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:07.147 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:52.694469+0000", 2024-09-18T10:15:07.147 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 14.369613061000001, 2024-09-18T10:15:07.147 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.51035930399999996, 2024-09-18T10:15:07.147 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:07.147 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:07.147 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:07.148 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:07.148 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:07.148 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 211 2024-09-18T10:15:07.148 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.148 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:07.148 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.148 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:07.148 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.694469+0000", 2024-09-18T10:15:07.148 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:07.148 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.148 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.148 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:07.148 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.694469+0000", 2024-09-18T10:15:07.149 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:07.149 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.149 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.149 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:07.149 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.694469+0000", 2024-09-18T10:15:07.149 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.999999 2024-09-18T10:15:07.149 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.149 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.149 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:07.149 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.694475+0000", 2024-09-18T10:15:07.149 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7349999999999999e-06 2024-09-18T10:15:07.149 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.150 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.150 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:07.150 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.694489+0000", 2024-09-18T10:15:07.150 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4209999999999999e-05 2024-09-18T10:15:07.150 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.150 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.150 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:07.150 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.694498+0000", 2024-09-18T10:15:07.150 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.4339999999999992e-06 2024-09-18T10:15:07.150 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.150 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.150 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:07.150 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.698348+0000", 2024-09-18T10:15:07.150 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0038499670000000001 2024-09-18T10:15:07.151 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.151 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.151 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:07.151 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.698366+0000", 2024-09-18T10:15:07.151 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7999000000000002e-05 2024-09-18T10:15:07.151 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.151 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.151 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:07.151 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:53.000010+0000", 2024-09-18T10:15:07.151 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.30164412699999998 2024-09-18T10:15:07.151 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.151 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.151 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:07.152 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:53.000223+0000", 2024-09-18T10:15:07.152 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000213169 2024-09-18T10:15:07.152 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.152 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.152 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:07.152 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:53.204829+0000", 2024-09-18T10:15:07.152 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.20460561299999999 2024-09-18T10:15:07.152 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:07.152 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:07.152 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:07.152 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.152 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.152 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(osd.10.22:2 3.7s0 3:ee962131:::smithi18925896-29:head [assert-version v6,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:07.152 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:52.798083+0000", 2024-09-18T10:15:07.153 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 14.265999169000001, 2024-09-18T10:15:07.153 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.60989146800000005, 2024-09-18T10:15:07.153 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:07.153 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:07.153 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:07.153 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.153 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:07.153 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.798083+0000", 2024-09-18T10:15:07.153 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:07.153 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.153 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.153 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:07.153 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.798083+0000", 2024-09-18T10:15:07.154 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:07.154 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.154 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.154 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:07.154 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.798080+0000", 2024-09-18T10:15:07.154 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999962 2024-09-18T10:15:07.154 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.154 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.154 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:07.154 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.798097+0000", 2024-09-18T10:15:07.154 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7657000000000001e-05 2024-09-18T10:15:07.154 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.154 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.154 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:07.155 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.798127+0000", 2024-09-18T10:15:07.155 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9532000000000001e-05 2024-09-18T10:15:07.155 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.155 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.155 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:07.155 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.898205+0000", 2024-09-18T10:15:07.155 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100078525 2024-09-18T10:15:07.155 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.155 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.155 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:07.155 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:53.204957+0000", 2024-09-18T10:15:07.155 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.30675138899999999 2024-09-18T10:15:07.155 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.156 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.156 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:07.156 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:53.205092+0000", 2024-09-18T10:15:07.156 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00013556700000000001 2024-09-18T10:15:07.156 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.156 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.156 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:07.156 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:53.407975+0000", 2024-09-18T10:15:07.156 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.20288240799999999 2024-09-18T10:15:07.156 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:07.156 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:07.156 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:07.156 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.156 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.157 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWriteReply(3.1s0 41/40 ECSubWriteReply(tid=60, last_complete=41'9, committed=1, applied=1))", 2024-09-18T10:15:07.157 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:57.147639+0000", 2024-09-18T10:15:07.157 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 9.9164438960000005, 2024-09-18T10:15:07.157 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.403385983, 2024-09-18T10:15:07.157 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:07.157 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:07.157 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:07.157 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.157 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:07.157 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.147639+0000", 2024-09-18T10:15:07.157 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:07.157 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.157 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.157 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:07.158 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.147639+0000", 2024-09-18T10:15:07.158 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:07.158 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.158 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.158 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:07.158 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.147638+0000", 2024-09-18T10:15:07.158 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:07.158 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.158 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.158 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:07.158 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.147642+0000", 2024-09-18T10:15:07.158 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8149999999999999e-06 2024-09-18T10:15:07.158 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.158 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.159 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:07.159 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.147653+0000", 2024-09-18T10:15:07.159 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1226000000000001e-05 2024-09-18T10:15:07.159 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.159 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.159 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:07.159 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.247724+0000", 2024-09-18T10:15:07.159 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10007079300000001 2024-09-18T10:15:07.159 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.159 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.159 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:07.159 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.550998+0000", 2024-09-18T10:15:07.159 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.30327365000000001 2024-09-18T10:15:07.160 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.160 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.160 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:07.160 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.551025+0000", 2024-09-18T10:15:07.160 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6957e-05 2024-09-18T10:15:07.160 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:07.160 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:07.160 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:07.160 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.160 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.160 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWriteReply(3.1s0 41/40 ECSubWriteReply(tid=61, last_complete=41'10, committed=1, applied=1))", 2024-09-18T10:15:07.160 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:57.241844+0000", 2024-09-18T10:15:07.160 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 9.8222386830000001, 2024-09-18T10:15:07.160 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.30877804800000003, 2024-09-18T10:15:07.161 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:07.161 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:07.161 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:07.161 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.161 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:07.161 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.241844+0000", 2024-09-18T10:15:07.161 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:07.161 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.161 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.161 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:07.161 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.241844+0000", 2024-09-18T10:15:07.161 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:07.162 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.162 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.162 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:07.162 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.241842+0000", 2024-09-18T10:15:07.162 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:07.162 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.162 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.162 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:07.162 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.241865+0000", 2024-09-18T10:15:07.162 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3122999999999999e-05 2024-09-18T10:15:07.162 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.162 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.162 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:07.162 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.241890+0000", 2024-09-18T10:15:07.163 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.5029e-05 2024-09-18T10:15:07.163 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.163 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.163 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:07.163 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.241900+0000", 2024-09-18T10:15:07.163 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.004e-05 2024-09-18T10:15:07.163 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.163 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.163 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:07.163 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.550542+0000", 2024-09-18T10:15:07.163 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.30864182699999998 2024-09-18T10:15:07.163 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.164 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.164 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:07.164 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.550622+0000", 2024-09-18T10:15:07.164 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.9589e-05 2024-09-18T10:15:07.164 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:07.164 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:07.164 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:07.164 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.164 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.164 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWriteReply(3.1s0 41/40 ECSubWriteReply(tid=61, last_complete=41'10, committed=1, applied=1))", 2024-09-18T10:15:07.164 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:57.247985+0000", 2024-09-18T10:15:07.164 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 9.8160971559999997, 2024-09-18T10:15:07.164 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.40354383700000002, 2024-09-18T10:15:07.165 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:07.165 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:07.165 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:07.165 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.165 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:07.165 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.247985+0000", 2024-09-18T10:15:07.165 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:07.165 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.165 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.165 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:07.165 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.247985+0000", 2024-09-18T10:15:07.165 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:07.165 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.166 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.166 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:07.166 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.247985+0000", 2024-09-18T10:15:07.166 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:07.166 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.166 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.166 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:07.166 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.247988+0000", 2024-09-18T10:15:07.166 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.3799999999999998e-06 2024-09-18T10:15:07.166 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.166 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.166 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:07.166 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.247996+0000", 2024-09-18T10:15:07.166 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7260000000000002e-06 2024-09-18T10:15:07.167 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.167 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.167 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:07.167 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.248003+0000", 2024-09-18T10:15:07.167 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.0010000000000004e-06 2024-09-18T10:15:07.167 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.167 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.167 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:07.167 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.651512+0000", 2024-09-18T10:15:07.167 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.40350923599999999 2024-09-18T10:15:07.167 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.167 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.167 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:07.167 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.651529+0000", 2024-09-18T10:15:07.168 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.6850999999999998e-05 2024-09-18T10:15:07.168 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:07.168 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:07.168 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:07.168 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.168 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.168 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWriteReply(3.1s0 41/40 ECSubWriteReply(tid=61, last_complete=41'10, committed=1, applied=1))", 2024-09-18T10:15:07.168 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:57.252307+0000", 2024-09-18T10:15:07.168 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 9.811775184, 2024-09-18T10:15:07.168 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.40002617099999999, 2024-09-18T10:15:07.168 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:07.168 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:07.168 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:07.169 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.169 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:07.169 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.252307+0000", 2024-09-18T10:15:07.169 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:07.169 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.169 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.169 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:07.169 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.252307+0000", 2024-09-18T10:15:07.169 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:07.169 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.169 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.169 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:07.169 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.252306+0000", 2024-09-18T10:15:07.169 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.999999 2024-09-18T10:15:07.170 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.170 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.170 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:07.170 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.252313+0000", 2024-09-18T10:15:07.170 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8589999999999999e-06 2024-09-18T10:15:07.170 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.170 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.170 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:07.170 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.252332+0000", 2024-09-18T10:15:07.170 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8989000000000001e-05 2024-09-18T10:15:07.170 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.170 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.170 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:07.170 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.252339+0000", 2024-09-18T10:15:07.171 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.2409999999999998e-06 2024-09-18T10:15:07.171 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.171 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.171 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:07.171 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.651579+0000", 2024-09-18T10:15:07.171 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.399239605 2024-09-18T10:15:07.171 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.171 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.171 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:07.171 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.652334+0000", 2024-09-18T10:15:07.171 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00075449399999999998 2024-09-18T10:15:07.171 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:07.172 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:07.172 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:07.172 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:07.172 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:07.172 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-18T10:15:07.174 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-18T10:15:07.173+0000 7f14c6a1a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 22138) UID: 0 2024-09-18T10:15:07.175 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-09-18T10:15:07.176 INFO:tasks.rados.rados.0.smithi189.stdout:156: finishing write tid 1 to smithi18925896-46 2024-09-18T10:15:07.176 INFO:tasks.rados.rados.0.smithi189.stdout:156: finishing write tid 2 to smithi18925896-46 2024-09-18T10:15:07.176 INFO:tasks.rados.rados.0.smithi189.stdout:156: finishing write tid 3 to smithi18925896-46 2024-09-18T10:15:07.176 INFO:tasks.rados.rados.0.smithi189.stdout:156: finishing write tid 4 to smithi18925896-46 2024-09-18T10:15:07.176 INFO:tasks.rados.rados.0.smithi189.stdout:157: finishing write tid 2 to smithi18925896-47 2024-09-18T10:15:07.176 INFO:tasks.rados.rados.0.smithi189.stdout:157: finishing write tid 3 to smithi18925896-47 2024-09-18T10:15:07.176 INFO:tasks.rados.rados.0.smithi189.stdout:157: finishing write tid 4 to smithi18925896-47 2024-09-18T10:15:07.274 INFO:tasks.ceph.osd.11.smithi189.stderr:2024-09-18T10:15:07.271+0000 7f718b60a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 22400) UID: 0 2024-09-18T10:15:07.281 INFO:tasks.rados.rados.0.smithi189.stdout:156: finishing write tid 5 to smithi18925896-46 2024-09-18T10:15:07.281 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 46 v 44 (ObjNum 76 snap 8 seq_num 76) dirty exists 2024-09-18T10:15:07.281 INFO:tasks.rados.rados.0.smithi189.stdout:156: left oid 46 (ObjNum 76 snap 8 seq_num 76) 2024-09-18T10:15:07.328 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-18T10:15:07.328 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-18T10:15:07.328 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 0 2024-09-18T10:15:07.328 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-18T10:15:07.338 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-09-18T10:15:07.375 INFO:tasks.ceph.osd.3.smithi189.stderr:2024-09-18T10:15:07.371+0000 7f2440fc7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22396) UID: 0 2024-09-18T10:15:07.381 INFO:tasks.rados.rados.0.smithi189.stdout:158: finishing rollback tid 1 to smithi18925896-17 2024-09-18T10:15:07.382 INFO:tasks.rados.rados.0.smithi189.stdout:158: finishing rollback tid 2 to smithi18925896-17 2024-09-18T10:15:07.382 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 17 v 46 (ObjNum 16 snap 0 seq_num 16) dirty exists 2024-09-18T10:15:07.412 INFO:tasks.rados.rados.0.smithi189.stdout:157: finishing write tid 5 to smithi18925896-47 2024-09-18T10:15:07.412 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 47 v 26 (ObjNum 77 snap 8 seq_num 77) dirty exists 2024-09-18T10:15:07.412 INFO:tasks.rados.rados.0.smithi189.stdout:157: left oid 47 (ObjNum 77 snap 8 seq_num 77) 2024-09-18T10:15:07.440 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-18T10:15:07.440 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-18T10:15:07.440 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-18T10:15:07.440 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-18T10:15:07.441 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-18T10:15:07.450 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-09-18T10:15:07.476 INFO:tasks.ceph.osd.6.smithi154.stderr:2024-09-18T10:15:07.471+0000 7f0abc9e1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22089) UID: 0 2024-09-18T10:15:07.550 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-18T10:15:07.551 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-18T10:15:07.551 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-18T10:15:07.551 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-18T10:15:07.551 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.551 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:133 3.4s0 3:2650be24:::smithi18925896-45:head [append 0~753664,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:07.551 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:50.913714+0000", 2024-09-18T10:15:07.551 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 16.632941568, 2024-09-18T10:15:07.551 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.20937973200000001, 2024-09-18T10:15:07.551 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:07.551 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:07.551 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:07.551 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:07.551 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:07.552 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 133 2024-09-18T10:15:07.552 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.552 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:07.552 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.552 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:07.552 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.913714+0000", 2024-09-18T10:15:07.552 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:07.552 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.552 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.552 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:07.552 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.913714+0000", 2024-09-18T10:15:07.552 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:07.552 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.553 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.553 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:07.553 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.913712+0000", 2024-09-18T10:15:07.553 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999976 2024-09-18T10:15:07.553 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.553 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.553 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:07.553 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.915605+0000", 2024-09-18T10:15:07.553 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001893301 2024-09-18T10:15:07.553 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.553 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.553 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:07.553 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.915632+0000", 2024-09-18T10:15:07.553 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6591e-05 2024-09-18T10:15:07.554 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.554 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.554 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:07.554 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.915645+0000", 2024-09-18T10:15:07.554 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3662000000000001e-05 2024-09-18T10:15:07.554 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.554 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.554 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:07.554 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.915722+0000", 2024-09-18T10:15:07.554 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7107000000000006e-05 2024-09-18T10:15:07.554 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.554 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.554 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:07.554 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.915888+0000", 2024-09-18T10:15:07.555 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00016575300000000001 2024-09-18T10:15:07.555 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.555 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.555 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:07.555 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.917139+0000", 2024-09-18T10:15:07.555 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001250903 2024-09-18T10:15:07.555 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.555 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.555 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:07.555 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.020414+0000", 2024-09-18T10:15:07.555 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.103275083 2024-09-18T10:15:07.555 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.555 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.555 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-18T10:15:07.556 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.122988+0000", 2024-09-18T10:15:07.556 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.102573762 2024-09-18T10:15:07.556 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.556 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.556 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:07.556 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.123094+0000", 2024-09-18T10:15:07.556 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000105974 2024-09-18T10:15:07.556 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:07.556 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:07.556 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:07.556 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.556 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.556 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:134 3.4s0 3:2650be24:::smithi18925896-45:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:07.557 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:50.915670+0000", 2024-09-18T10:15:07.557 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 16.630985952, 2024-09-18T10:15:07.557 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.40775830200000002, 2024-09-18T10:15:07.557 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:07.557 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:07.557 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:07.557 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:07.557 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:07.557 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 134 2024-09-18T10:15:07.557 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.557 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:07.557 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.557 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:07.558 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.915670+0000", 2024-09-18T10:15:07.558 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:07.558 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.558 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.558 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:07.558 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.915670+0000", 2024-09-18T10:15:07.558 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:07.558 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.558 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.558 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:07.558 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.915668+0000", 2024-09-18T10:15:07.558 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:07.558 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.558 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.559 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:07.559 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.915676+0000", 2024-09-18T10:15:07.559 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1899999999999995e-06 2024-09-18T10:15:07.559 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.559 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.559 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:07.559 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.915690+0000", 2024-09-18T10:15:07.559 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3827e-05 2024-09-18T10:15:07.559 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.559 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.559 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:07.559 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.915702+0000", 2024-09-18T10:15:07.559 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1477999999999999e-05 2024-09-18T10:15:07.559 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.560 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.560 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:07.560 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.918373+0000", 2024-09-18T10:15:07.560 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0026713000000000002 2024-09-18T10:15:07.560 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.560 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.560 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:07.560 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.918547+0000", 2024-09-18T10:15:07.560 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00017375999999999999 2024-09-18T10:15:07.560 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.560 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.560 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:07.560 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.919485+0000", 2024-09-18T10:15:07.560 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00093826999999999995 2024-09-18T10:15:07.561 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.561 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.561 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:07.561 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.120632+0000", 2024-09-18T10:15:07.561 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.20114729200000001 2024-09-18T10:15:07.561 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.561 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.561 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-18T10:15:07.561 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.323281+0000", 2024-09-18T10:15:07.561 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.20264834500000001 2024-09-18T10:15:07.561 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.561 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.561 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:07.562 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.323428+0000", 2024-09-18T10:15:07.562 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000147334 2024-09-18T10:15:07.562 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:07.562 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:07.562 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:07.562 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.562 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.562 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:135 3.4s0 3:2650be24:::smithi18925896-45:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:07.562 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:50.915712+0000", 2024-09-18T10:15:07.562 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 16.630944093, 2024-09-18T10:15:07.562 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.81161388099999998, 2024-09-18T10:15:07.562 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:07.562 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:07.562 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:07.563 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:07.563 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:07.563 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 135 2024-09-18T10:15:07.563 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.563 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:07.563 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.563 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:07.563 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.915712+0000", 2024-09-18T10:15:07.563 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:07.563 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.563 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.563 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:07.563 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.915712+0000", 2024-09-18T10:15:07.563 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:07.564 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.564 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.564 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:07.564 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.915711+0000", 2024-09-18T10:15:07.564 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.999999 2024-09-18T10:15:07.564 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.564 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.564 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:07.564 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.915715+0000", 2024-09-18T10:15:07.564 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.3159999999999998e-06 2024-09-18T10:15:07.564 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.564 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.564 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:07.564 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.915726+0000", 2024-09-18T10:15:07.565 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1375e-05 2024-09-18T10:15:07.565 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.565 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.565 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:07.565 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.915736+0000", 2024-09-18T10:15:07.565 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.4860000000000003e-06 2024-09-18T10:15:07.565 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.565 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.565 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:07.565 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.919925+0000", 2024-09-18T10:15:07.565 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0041886689999999999 2024-09-18T10:15:07.565 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.565 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.566 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for rw locks", 2024-09-18T10:15:07.566 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.920019+0000", 2024-09-18T10:15:07.566 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.4196000000000004e-05 2024-09-18T10:15:07.566 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.566 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.566 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:07.566 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.424171+0000", 2024-09-18T10:15:07.566 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.50415232399999999 2024-09-18T10:15:07.566 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.566 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.566 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:07.566 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.424320+0000", 2024-09-18T10:15:07.566 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00014877299999999999 2024-09-18T10:15:07.566 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.567 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.567 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:07.567 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.727325+0000", 2024-09-18T10:15:07.567 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.30300555400000001 2024-09-18T10:15:07.567 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:07.567 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:07.567 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:07.567 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.567 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.567 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 36/32 ECSubWriteReply(tid=9, last_complete=36'6, committed=1, applied=1))", 2024-09-18T10:15:07.567 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:51.122866+0000", 2024-09-18T10:15:07.567 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 16.423789546999998, 2024-09-18T10:15:07.567 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.20110556800000001, 2024-09-18T10:15:07.567 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:07.568 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:07.568 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:07.568 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.568 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:07.568 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.122866+0000", 2024-09-18T10:15:07.568 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:07.568 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.568 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.568 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:07.568 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.122866+0000", 2024-09-18T10:15:07.568 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:07.568 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.568 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.569 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:07.569 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.122865+0000", 2024-09-18T10:15:07.569 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:07.569 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.569 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.569 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:07.569 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.122870+0000", 2024-09-18T10:15:07.569 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0239999999999997e-06 2024-09-18T10:15:07.569 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.569 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.569 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:07.570 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.122885+0000", 2024-09-18T10:15:07.570 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5027e-05 2024-09-18T10:15:07.570 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.570 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.570 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:07.570 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.222973+0000", 2024-09-18T10:15:07.570 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100087623 2024-09-18T10:15:07.570 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.570 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.570 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:07.570 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.323132+0000", 2024-09-18T10:15:07.571 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10015884799999999 2024-09-18T10:15:07.571 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.571 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.571 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:07.571 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.323972+0000", 2024-09-18T10:15:07.571 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00083970500000000005 2024-09-18T10:15:07.571 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:07.571 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:07.571 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:07.571 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.571 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.571 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWrite(3.cs1 36/32 ECSubWrite(tid=20, reqid=client.4632.0:150, at_version=36'9, trim_to=0'0, roll_forward_to=36'8))", 2024-09-18T10:15:07.571 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:51.127568+0000", 2024-09-18T10:15:07.571 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 16.419087936, 2024-09-18T10:15:07.572 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.29747172100000002, 2024-09-18T10:15:07.572 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:07.572 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:07.572 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:07.572 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.572 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:07.572 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.127568+0000", 2024-09-18T10:15:07.572 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:07.572 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.572 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.572 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:07.572 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.127568+0000", 2024-09-18T10:15:07.572 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:07.572 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.573 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.573 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:07.573 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.127566+0000", 2024-09-18T10:15:07.573 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:07.573 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.573 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.573 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:07.573 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.127654+0000", 2024-09-18T10:15:07.573 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8181000000000002e-05 2024-09-18T10:15:07.573 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.573 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.573 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:07.573 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.127681+0000", 2024-09-18T10:15:07.574 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6282e-05 2024-09-18T10:15:07.574 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.574 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.574 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:07.574 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.127694+0000", 2024-09-18T10:15:07.574 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3159999999999999e-05 2024-09-18T10:15:07.574 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.574 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.574 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:07.574 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.221115+0000", 2024-09-18T10:15:07.575 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.093420807999999994 2024-09-18T10:15:07.575 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.575 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.575 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:07.575 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.221132+0000", 2024-09-18T10:15:07.575 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7725999999999999e-05 2024-09-18T10:15:07.575 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.575 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.575 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:07.575 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.424921+0000", 2024-09-18T10:15:07.575 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.20378872000000001 2024-09-18T10:15:07.575 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.575 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.576 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:07.576 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.425039+0000", 2024-09-18T10:15:07.576 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000118267 2024-09-18T10:15:07.576 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:07.576 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:07.576 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:07.576 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.576 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.576 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWrite(3.cs1 36/32 ECSubWrite(tid=21, reqid=client.4632.0:151, at_version=36'10, trim_to=0'0, roll_forward_to=36'8))", 2024-09-18T10:15:07.576 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:51.130232+0000", 2024-09-18T10:15:07.576 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 16.416423665, 2024-09-18T10:15:07.576 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.294885956, 2024-09-18T10:15:07.576 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:07.577 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:07.577 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:07.577 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.577 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:07.577 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.130232+0000", 2024-09-18T10:15:07.577 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:07.577 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.577 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.577 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:07.577 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.130232+0000", 2024-09-18T10:15:07.577 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:07.577 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.577 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.578 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:07.578 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.130226+0000", 2024-09-18T10:15:07.578 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999943 2024-09-18T10:15:07.578 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.578 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.578 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:07.578 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.130394+0000", 2024-09-18T10:15:07.578 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000167205 2024-09-18T10:15:07.578 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.578 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.578 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:07.578 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.130426+0000", 2024-09-18T10:15:07.578 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2625e-05 2024-09-18T10:15:07.578 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.579 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.579 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:07.579 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.130443+0000", 2024-09-18T10:15:07.579 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.6558999999999998e-05 2024-09-18T10:15:07.579 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.579 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.579 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:07.579 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.330703+0000", 2024-09-18T10:15:07.579 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.200260614 2024-09-18T10:15:07.579 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.579 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.579 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:07.579 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.330718+0000", 2024-09-18T10:15:07.579 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4647999999999999e-05 2024-09-18T10:15:07.580 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.580 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.580 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:07.580 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.425072+0000", 2024-09-18T10:15:07.580 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.094353426000000004 2024-09-18T10:15:07.580 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.580 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.580 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:07.580 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.425118+0000", 2024-09-18T10:15:07.580 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6378999999999998e-05 2024-09-18T10:15:07.580 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:07.580 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:07.580 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:07.581 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.581 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.581 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWrite(3.es4 36/32 ECSubWrite(tid=2, reqid=client.4632.0:116, at_version=0'0, trim_to=0'0, roll_forward_to=36'2))", 2024-09-18T10:15:07.581 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:51.223504+0000", 2024-09-18T10:15:07.581 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 16.323151607, 2024-09-18T10:15:07.581 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.20181344600000001, 2024-09-18T10:15:07.581 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:07.581 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:07.581 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:07.581 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.581 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:07.581 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.223504+0000", 2024-09-18T10:15:07.581 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:07.581 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.582 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.582 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:07.582 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.223504+0000", 2024-09-18T10:15:07.582 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:07.582 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.582 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.582 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:07.582 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.223504+0000", 2024-09-18T10:15:07.582 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:07.582 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.582 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.582 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:07.582 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.223515+0000", 2024-09-18T10:15:07.582 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1464000000000001e-05 2024-09-18T10:15:07.583 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.583 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.583 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:07.583 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.223528+0000", 2024-09-18T10:15:07.583 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3423e-05 2024-09-18T10:15:07.583 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.583 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.583 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:07.583 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.323617+0000", 2024-09-18T10:15:07.583 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100088249 2024-09-18T10:15:07.583 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.583 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.583 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:07.583 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.423850+0000", 2024-09-18T10:15:07.584 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100233212 2024-09-18T10:15:07.584 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.584 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.584 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:07.584 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.423866+0000", 2024-09-18T10:15:07.584 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.59e-05 2024-09-18T10:15:07.584 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.584 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.584 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:07.584 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.425270+0000", 2024-09-18T10:15:07.584 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001403783 2024-09-18T10:15:07.584 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.584 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.585 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:07.585 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.425317+0000", 2024-09-18T10:15:07.585 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7892000000000002e-05 2024-09-18T10:15:07.585 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:07.585 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:07.585 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:07.585 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.585 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.585 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWrite(3.es4 36/32 ECSubWrite(tid=4, reqid=client.4632.0:146, at_version=36'3, trim_to=0'0, roll_forward_to=36'2))", 2024-09-18T10:15:07.585 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:51.323664+0000", 2024-09-18T10:15:07.585 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 16.222991446000002, 2024-09-18T10:15:07.585 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.20269168300000001, 2024-09-18T10:15:07.585 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:07.585 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:07.586 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:07.586 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.586 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:07.586 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.323664+0000", 2024-09-18T10:15:07.586 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:07.586 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.586 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.586 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:07.586 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.323664+0000", 2024-09-18T10:15:07.586 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:07.586 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.586 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.586 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:07.587 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.323663+0000", 2024-09-18T10:15:07.587 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:07.587 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.587 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.587 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:07.587 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.323918+0000", 2024-09-18T10:15:07.587 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00025554100000000003 2024-09-18T10:15:07.587 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.587 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.587 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:07.587 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.323944+0000", 2024-09-18T10:15:07.587 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.5749e-05 2024-09-18T10:15:07.587 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.587 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.587 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:07.588 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.323962+0000", 2024-09-18T10:15:07.588 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7632999999999999e-05 2024-09-18T10:15:07.588 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.588 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.588 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:07.588 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.424243+0000", 2024-09-18T10:15:07.588 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10028129500000001 2024-09-18T10:15:07.588 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.588 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.588 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:07.588 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.424300+0000", 2024-09-18T10:15:07.588 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7262000000000002e-05 2024-09-18T10:15:07.588 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.589 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.589 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:07.589 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.526281+0000", 2024-09-18T10:15:07.589 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10198104099999999 2024-09-18T10:15:07.589 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.589 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.589 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:07.589 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.526356+0000", 2024-09-18T10:15:07.589 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4658000000000002e-05 2024-09-18T10:15:07.589 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:07.589 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:07.589 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:07.589 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.589 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.590 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpRead(3.4s0 36/32 ECSubRead(tid=11, to_read={3:2650be24:::smithi18925896-45:head=0,4096,0}, subchunks={3:2650be24:::smithi18925896-45:head=[0,1]}, attrs_to_read=))", 2024-09-18T10:15:07.590 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:51.424635+0000", 2024-09-18T10:15:07.590 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 16.122021006000001, 2024-09-18T10:15:07.590 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.20168914099999999, 2024-09-18T10:15:07.590 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:07.590 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:07.590 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:07.590 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.590 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:07.590 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.424635+0000", 2024-09-18T10:15:07.590 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:07.590 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.590 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.590 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:07.591 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.424635+0000", 2024-09-18T10:15:07.591 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:07.591 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.591 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.591 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:07.591 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.424635+0000", 2024-09-18T10:15:07.591 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:07.591 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.591 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.591 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:07.591 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.424635+0000", 2024-09-18T10:15:07.591 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:07.591 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.591 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.592 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:07.592 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.424703+0000", 2024-09-18T10:15:07.592 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8015999999999994e-05 2024-09-18T10:15:07.592 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.592 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.592 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:07.592 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.524847+0000", 2024-09-18T10:15:07.592 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100144646 2024-09-18T10:15:07.592 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.592 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.592 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:07.592 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.626011+0000", 2024-09-18T10:15:07.592 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10116393 2024-09-18T10:15:07.593 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.593 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.593 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:07.593 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.626324+0000", 2024-09-18T10:15:07.593 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00031254900000000002 2024-09-18T10:15:07.593 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:07.593 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:07.593 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:07.593 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.593 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.593 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWrite(3.cs1 36/32 ECSubWrite(tid=22, reqid=client.4632.0:151, at_version=0'0, trim_to=0'0, roll_forward_to=36'10))", 2024-09-18T10:15:07.593 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:51.533998+0000", 2024-09-18T10:15:07.593 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 16.012657612000002, 2024-09-18T10:15:07.593 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.29424693400000002, 2024-09-18T10:15:07.594 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:07.594 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:07.594 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:07.594 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.594 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:07.594 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.533998+0000", 2024-09-18T10:15:07.594 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:07.594 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.594 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.594 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:07.594 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.533998+0000", 2024-09-18T10:15:07.594 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:07.594 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.595 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.595 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:07.595 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.533996+0000", 2024-09-18T10:15:07.595 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:07.595 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.595 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.595 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:07.595 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.534023+0000", 2024-09-18T10:15:07.595 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.675e-05 2024-09-18T10:15:07.595 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.595 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.595 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:07.595 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.534056+0000", 2024-09-18T10:15:07.596 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.3288999999999999e-05 2024-09-18T10:15:07.596 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.596 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.596 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:07.596 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.634158+0000", 2024-09-18T10:15:07.596 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100101632 2024-09-18T10:15:07.596 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.596 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.596 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:07.596 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.827026+0000", 2024-09-18T10:15:07.596 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.19286773800000001 2024-09-18T10:15:07.596 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.596 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.596 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:07.597 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.827036+0000", 2024-09-18T10:15:07.597 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0261999999999999e-05 2024-09-18T10:15:07.597 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.597 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.597 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:07.597 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.828227+0000", 2024-09-18T10:15:07.597 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001191398 2024-09-18T10:15:07.597 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.597 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.597 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:07.597 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.828245+0000", 2024-09-18T10:15:07.597 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7604999999999998e-05 2024-09-18T10:15:07.597 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:07.598 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:07.598 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:07.598 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.598 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.598 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:188 3.4s0 3:213f864e:::smithi18925896-32:head [create,setxattr XNeZS[lOPtVca\\Pa[\\rdfbinb[bcjhtYSpmcafZm (1928) in=40b,setxattr TfflYYqPnps]lo^qSlp___TitNdd`]kaPTVOTcsh (1648) in=40b,setxattr TNb_onpY[j_dkhfmnWtm]Yktg]]RNtbQe[OUTNlX (770) in=40b,setxattr ZcgnkhY\\orTYaZh]PPldj`SNq`p\\cX`k]h\\bZY_Y (700) in=40b,setxattr gPQVnqhpsVhW_kq`QnafXrlRZtZ\\YnRftgQTishR (595) in=40b,setxattr U`XqYi[WhoqnpSPRc\\RhnppZp^qY\\qUfjpqs[r^a (1892) in=40b,setxattr cTmhd`pWgrRf_VfVgOtgeOP^ShP]lsOtpoTj[\\gl (418) in=40b,setxattr aa_taYTfslg`lRRrOkkraN\\XiW^V\\ljl]NqXin`a (314) in=40b,setxattr ibl]`igtgVThY^Qse\\okYYUtTmolX\\W]p]Q[P\\il (1403) in=40b,setxattr \\ZabSO[UUqeZa`hddNh[ZjaRRgUZoQtkn\\QdsX]W (1541) in=40b,setxattr jUkQUPTNbo`rZNmYlUmhb]ojc]aeheoVRfp]bPWf (5052) in=40b,setxattr Ofj`oQ^cR]da^SPsTaXmmPse\\OZ`VSqQ]U`efsZl (419) in=40b,setxattr a[cdkWos\\OrUVPciUaa^_^`lWVTScffiWkd\\[kgf (1261) in=40b,setxattr jXb]\\WhbatdZ[qTTiZO]kPd^`l[kajOYiQ[iqSYc (244) in=40b,setxattr \\im]jQhbanUn\\_^h`jaU]W`ReaZN_je[]kgg (1578) in=36b,setxattr ZbosoRORS^_de]rN^_daOiha\\Q\\eRae^[sQUjXsV (9120) in=40b,setxattr gcn]XpZUkOZr\\pkrVVZpYiR\\alh_PaZa_S[PYcn] (384) in=40b,setxattr \\^qUOaNXSTjtmafhfbo\\gseTTkV`mgnacYq^bkcc (34) in=40b,setxattr iUa^mOQ]TqjVSUU_nXfY]f`NjljiiaYbrm[o]aes (1611) in=40b,setxattr tXaQPQbc^aiV[^reVOPpib^caoWf^XgqilO^_g[k (1016) in=40b] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:07.598 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:52.693780+0000", 2024-09-18T10:15:07.598 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 14.852875515999999, 2024-09-18T10:15:07.598 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.40440725999999999, 2024-09-18T10:15:07.598 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:07.598 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:07.598 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:07.598 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:07.598 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:07.598 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 188 2024-09-18T10:15:07.599 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.599 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:07.599 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.599 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:07.599 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.693780+0000", 2024-09-18T10:15:07.599 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:07.599 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.599 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.599 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:07.599 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.693780+0000", 2024-09-18T10:15:07.599 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:07.599 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.599 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.599 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:07.600 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.693777+0000", 2024-09-18T10:15:07.600 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999967 2024-09-18T10:15:07.600 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.600 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.600 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:07.600 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.693873+0000", 2024-09-18T10:15:07.600 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.6496000000000006e-05 2024-09-18T10:15:07.600 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.600 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.600 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:07.600 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.693926+0000", 2024-09-18T10:15:07.600 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2148999999999998e-05 2024-09-18T10:15:07.600 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.601 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.601 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:07.601 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.693945+0000", 2024-09-18T10:15:07.601 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9296999999999998e-05 2024-09-18T10:15:07.601 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.601 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.601 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:07.601 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.694035+0000", 2024-09-18T10:15:07.601 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.0439999999999995e-05 2024-09-18T10:15:07.601 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.601 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.601 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op must wait for map", 2024-09-18T10:15:07.601 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.694044+0000", 2024-09-18T10:15:07.601 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.1479999999999995e-06 2024-09-18T10:15:07.602 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.602 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.602 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:07.602 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.797388+0000", 2024-09-18T10:15:07.602 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10334315199999999 2024-09-18T10:15:07.602 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.602 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.602 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:07.602 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.797847+0000", 2024-09-18T10:15:07.602 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00045971299999999999 2024-09-18T10:15:07.602 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.602 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.602 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:07.603 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.800981+0000", 2024-09-18T10:15:07.603 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0031333189999999999 2024-09-18T10:15:07.603 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.603 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.603 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:07.603 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.803005+0000", 2024-09-18T10:15:07.603 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0020248359999999999 2024-09-18T10:15:07.603 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.603 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.603 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-18T10:15:07.604 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:53.097948+0000", 2024-09-18T10:15:07.604 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.29494210599999998 2024-09-18T10:15:07.604 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.604 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.604 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:07.604 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:53.098187+0000", 2024-09-18T10:15:07.604 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000239769 2024-09-18T10:15:07.604 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:07.604 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:07.604 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:07.604 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.604 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.604 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWrite(3.9s5 37/32 ECSubWrite(tid=27, reqid=client.4632.0:186, at_version=37'16, trim_to=0'0, roll_forward_to=37'15))", 2024-09-18T10:15:07.605 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:52.801767+0000", 2024-09-18T10:15:07.605 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 14.744888574000001, 2024-09-18T10:15:07.605 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.200908524, 2024-09-18T10:15:07.605 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:07.605 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:07.605 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:07.605 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.605 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:07.605 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.801767+0000", 2024-09-18T10:15:07.605 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:07.605 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.605 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.605 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:07.605 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.801767+0000", 2024-09-18T10:15:07.606 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:07.606 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.606 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.606 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:07.606 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.801766+0000", 2024-09-18T10:15:07.606 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:07.606 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.606 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.606 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:07.606 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.801802+0000", 2024-09-18T10:15:07.606 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.6378999999999998e-05 2024-09-18T10:15:07.606 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.606 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.606 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:07.607 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.801823+0000", 2024-09-18T10:15:07.607 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.037e-05 2024-09-18T10:15:07.607 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.607 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.607 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:07.607 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.901919+0000", 2024-09-18T10:15:07.607 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10009599800000001 2024-09-18T10:15:07.607 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.607 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.607 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:07.607 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.901973+0000", 2024-09-18T10:15:07.607 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4033000000000003e-05 2024-09-18T10:15:07.607 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.607 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.608 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:07.608 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.901979+0000", 2024-09-18T10:15:07.608 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8820000000000003e-06 2024-09-18T10:15:07.608 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.608 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.608 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:07.608 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:53.002634+0000", 2024-09-18T10:15:07.608 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100654695 2024-09-18T10:15:07.608 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.608 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.608 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:07.608 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:53.002676+0000", 2024-09-18T10:15:07.608 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1393999999999997e-05 2024-09-18T10:15:07.609 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:07.609 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:07.609 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:07.609 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.609 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.609 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWrite(3.cs1 37/32 ECSubWrite(tid=29, reqid=client.4632.0:181, at_version=37'13, trim_to=0'0, roll_forward_to=36'11))", 2024-09-18T10:15:07.609 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:52.895376+0000", 2024-09-18T10:15:07.609 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 14.651279848, 2024-09-18T10:15:07.609 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.20359808099999999, 2024-09-18T10:15:07.609 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:07.609 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:07.609 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:07.609 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.609 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:07.610 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.895376+0000", 2024-09-18T10:15:07.610 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:07.610 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.610 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.610 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:07.610 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.895376+0000", 2024-09-18T10:15:07.610 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:07.610 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.610 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.610 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:07.610 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.895375+0000", 2024-09-18T10:15:07.610 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.999999 2024-09-18T10:15:07.610 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.611 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.611 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:07.611 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.895708+0000", 2024-09-18T10:15:07.611 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00033264600000000003 2024-09-18T10:15:07.611 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.611 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.611 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:07.611 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.895744+0000", 2024-09-18T10:15:07.611 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.6718999999999999e-05 2024-09-18T10:15:07.611 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.611 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.611 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:07.611 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.895761+0000", 2024-09-18T10:15:07.611 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7125999999999998e-05 2024-09-18T10:15:07.612 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.612 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.612 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:07.612 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.995989+0000", 2024-09-18T10:15:07.612 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100227143 2024-09-18T10:15:07.612 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.612 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.612 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:07.612 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.996005+0000", 2024-09-18T10:15:07.612 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.6566e-05 2024-09-18T10:15:07.612 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.612 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.612 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:07.613 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:53.098923+0000", 2024-09-18T10:15:07.613 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.102917966 2024-09-18T10:15:07.613 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.613 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.613 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:07.613 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:53.098974+0000", 2024-09-18T10:15:07.613 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0742000000000003e-05 2024-09-18T10:15:07.613 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:07.613 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:07.613 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:07.613 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.613 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.613 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWrite(3.cs1 41/32 ECSubWrite(tid=47, reqid=osd.5.0:46, at_version=0'0, trim_to=0'0, roll_forward_to=41'27))", 2024-09-18T10:15:07.614 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:56.935584+0000", 2024-09-18T10:15:07.614 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 10.611071819999999, 2024-09-18T10:15:07.614 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.60626800999999997, 2024-09-18T10:15:07.614 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:07.614 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:07.614 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:07.614 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.614 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:07.614 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.935584+0000", 2024-09-18T10:15:07.614 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:07.614 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.614 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.614 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:07.614 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.935584+0000", 2024-09-18T10:15:07.615 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:07.615 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.615 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.615 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:07.615 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.935582+0000", 2024-09-18T10:15:07.615 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:07.615 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.615 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.615 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:07.615 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.935616+0000", 2024-09-18T10:15:07.615 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.4588999999999997e-05 2024-09-18T10:15:07.615 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.615 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.615 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:07.616 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.935661+0000", 2024-09-18T10:15:07.616 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4705e-05 2024-09-18T10:15:07.616 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.616 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.616 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:07.616 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.935681+0000", 2024-09-18T10:15:07.616 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0290000000000001e-05 2024-09-18T10:15:07.616 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.616 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.616 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:07.616 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.339911+0000", 2024-09-18T10:15:07.616 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.40422952000000001 2024-09-18T10:15:07.616 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.617 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.617 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:07.617 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.339925+0000", 2024-09-18T10:15:07.617 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4523e-05 2024-09-18T10:15:07.617 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.617 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.617 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:07.617 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.541788+0000", 2024-09-18T10:15:07.617 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.20186222000000001 2024-09-18T10:15:07.617 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.617 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.617 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:07.617 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.541852+0000", 2024-09-18T10:15:07.617 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.4171999999999995e-05 2024-09-18T10:15:07.618 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:07.618 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:07.618 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:07.618 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.618 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.618 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWrite(3.cs1 41/32 ECSubWrite(tid=48, reqid=osd.5.0:48, at_version=41'29, trim_to=0'0, roll_forward_to=41'27))", 2024-09-18T10:15:07.618 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:56.937458+0000", 2024-09-18T10:15:07.618 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 10.609198031, 2024-09-18T10:15:07.618 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.60464621900000004, 2024-09-18T10:15:07.618 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:07.618 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:07.618 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:07.618 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.619 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:07.619 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.937458+0000", 2024-09-18T10:15:07.619 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:07.619 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.619 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.619 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:07.619 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.937458+0000", 2024-09-18T10:15:07.619 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:07.619 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.619 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.619 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:07.619 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.937456+0000", 2024-09-18T10:15:07.619 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.999999 2024-09-18T10:15:07.619 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.620 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.620 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:07.620 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.937505+0000", 2024-09-18T10:15:07.620 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8809e-05 2024-09-18T10:15:07.620 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.620 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.620 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:07.620 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.937540+0000", 2024-09-18T10:15:07.620 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.4662000000000001e-05 2024-09-18T10:15:07.620 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.620 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.620 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:07.620 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.937554+0000", 2024-09-18T10:15:07.620 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4137e-05 2024-09-18T10:15:07.621 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.621 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.621 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:07.621 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.340361+0000", 2024-09-18T10:15:07.621 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.40280652500000003 2024-09-18T10:15:07.621 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.621 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.621 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:07.621 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.340382+0000", 2024-09-18T10:15:07.621 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.16e-05 2024-09-18T10:15:07.621 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.621 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.621 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:07.622 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.542055+0000", 2024-09-18T10:15:07.622 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.20167278999999999 2024-09-18T10:15:07.622 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.622 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.622 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:07.622 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.542104+0000", 2024-09-18T10:15:07.622 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8866000000000003e-05 2024-09-18T10:15:07.622 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:07.622 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:07.622 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:07.622 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.622 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.622 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpRead(3.4s0 41/40 ECSubRead(tid=14, to_read={3:2650be24:::smithi18925896-45:head=0,188416,0}, subchunks={3:2650be24:::smithi18925896-45:head=[0,1]}, attrs_to_read=))", 2024-09-18T10:15:07.623 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:57.341244+0000", 2024-09-18T10:15:07.623 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 10.205411667, 2024-09-18T10:15:07.623 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.201520594, 2024-09-18T10:15:07.623 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:07.623 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:07.623 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:07.623 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.623 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:07.623 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.341244+0000", 2024-09-18T10:15:07.623 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:07.623 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.623 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.623 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:07.623 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.341244+0000", 2024-09-18T10:15:07.624 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:07.624 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.624 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.624 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:07.624 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.341244+0000", 2024-09-18T10:15:07.624 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:07.624 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.624 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.624 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:07.624 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.341244+0000", 2024-09-18T10:15:07.624 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:07.624 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.624 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.625 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:07.625 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.341281+0000", 2024-09-18T10:15:07.625 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7413999999999998e-05 2024-09-18T10:15:07.626 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.626 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.626 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:07.626 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.441387+0000", 2024-09-18T10:15:07.626 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100105729 2024-09-18T10:15:07.626 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.626 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.626 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:07.626 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.542088+0000", 2024-09-18T10:15:07.626 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10070077600000001 2024-09-18T10:15:07.626 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.626 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.626 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:07.627 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.542765+0000", 2024-09-18T10:15:07.627 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00067667499999999995 2024-09-18T10:15:07.627 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:07.627 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:07.627 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:07.627 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.627 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.627 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpRead(3.4s0 41/40 ECSubRead(tid=15, to_read={3:2650be24:::smithi18925896-45:head=0,188416,0}, subchunks={3:2650be24:::smithi18925896-45:head=[0,1]}, attrs_to_read=))", 2024-09-18T10:15:07.627 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:57.341910+0000", 2024-09-18T10:15:07.627 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 10.204745681, 2024-09-18T10:15:07.627 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.20191420500000001, 2024-09-18T10:15:07.627 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:07.627 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:07.628 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:07.628 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.628 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:07.628 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.341910+0000", 2024-09-18T10:15:07.628 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:07.628 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.628 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.628 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:07.628 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.341910+0000", 2024-09-18T10:15:07.628 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:07.628 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.628 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.628 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:07.628 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.341910+0000", 2024-09-18T10:15:07.629 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:07.629 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.629 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.629 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:07.629 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.341910+0000", 2024-09-18T10:15:07.629 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:07.629 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.629 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.629 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:07.629 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.441437+0000", 2024-09-18T10:15:07.629 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099527529000000003 2024-09-18T10:15:07.629 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.629 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.629 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:07.630 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.441461+0000", 2024-09-18T10:15:07.630 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3767000000000001e-05 2024-09-18T10:15:07.630 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.630 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.630 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:07.630 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.543542+0000", 2024-09-18T10:15:07.630 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10208060200000001 2024-09-18T10:15:07.630 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.630 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.630 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:07.630 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.543824+0000", 2024-09-18T10:15:07.630 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00028230699999999998 2024-09-18T10:15:07.630 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:07.630 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:07.631 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:07.631 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.631 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.631 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpReadReply(3.4s0 41/40 ECSubReadReply(tid=14, attrs_read=0))", 2024-09-18T10:15:07.631 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:57.342538+0000", 2024-09-18T10:15:07.631 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 10.204117492, 2024-09-18T10:15:07.631 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.20040434200000001, 2024-09-18T10:15:07.631 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:07.631 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:07.631 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:07.631 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.631 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:07.631 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.342538+0000", 2024-09-18T10:15:07.631 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:07.632 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.632 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.632 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:07.632 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.342538+0000", 2024-09-18T10:15:07.632 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:07.632 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.632 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.632 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:07.632 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.342537+0000", 2024-09-18T10:15:07.632 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.999999 2024-09-18T10:15:07.632 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.632 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.632 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:07.633 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.446172+0000", 2024-09-18T10:15:07.633 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.103635227 2024-09-18T10:15:07.633 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.633 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.633 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:07.633 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.446200+0000", 2024-09-18T10:15:07.633 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.775e-05 2024-09-18T10:15:07.633 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.633 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.633 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:07.633 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.446213+0000", 2024-09-18T10:15:07.633 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2729e-05 2024-09-18T10:15:07.633 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.633 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.634 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:07.634 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.542854+0000", 2024-09-18T10:15:07.634 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.096640734000000006 2024-09-18T10:15:07.634 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.634 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.634 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:07.634 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.542942+0000", 2024-09-18T10:15:07.634 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8894999999999995e-05 2024-09-18T10:15:07.634 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:07.634 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:07.634 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:07.634 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.634 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.634 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpRead(3.4s0 41/40 ECSubRead(tid=16, to_read={3:2650be24:::smithi18925896-45:head=0,188416,0}, subchunks={3:2650be24:::smithi18925896-45:head=[0,1]}, attrs_to_read=))", 2024-09-18T10:15:07.634 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:57.342646+0000", 2024-09-18T10:15:07.635 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 10.204009605, 2024-09-18T10:15:07.635 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.20341867, 2024-09-18T10:15:07.635 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:07.635 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:07.635 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:07.635 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.635 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:07.635 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.342646+0000", 2024-09-18T10:15:07.635 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:07.635 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.635 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.635 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:07.635 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.342646+0000", 2024-09-18T10:15:07.636 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:07.636 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.636 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.636 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:07.636 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.342646+0000", 2024-09-18T10:15:07.636 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:07.636 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.636 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.636 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:07.636 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.342646+0000", 2024-09-18T10:15:07.636 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:07.636 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.636 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.636 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:07.637 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.441499+0000", 2024-09-18T10:15:07.637 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.098853294999999994 2024-09-18T10:15:07.637 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.637 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.637 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:07.637 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.441514+0000", 2024-09-18T10:15:07.637 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5057e-05 2024-09-18T10:15:07.637 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.637 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.637 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:07.637 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.545709+0000", 2024-09-18T10:15:07.637 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10419476900000001 2024-09-18T10:15:07.637 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.637 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.638 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:07.638 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.546065+0000", 2024-09-18T10:15:07.638 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00035554899999999998 2024-09-18T10:15:07.638 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:07.638 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:07.638 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:07.638 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.638 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.638 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpReadReply(3.4s0 41/40 ECSubReadReply(tid=16, attrs_read=0))", 2024-09-18T10:15:07.638 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:57.446638+0000", 2024-09-18T10:15:07.638 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 10.100017632, 2024-09-18T10:15:07.638 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.202725567, 2024-09-18T10:15:07.638 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:07.638 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:07.639 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:07.639 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.639 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:07.639 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.446638+0000", 2024-09-18T10:15:07.639 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:07.639 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.639 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.639 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:07.639 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.446638+0000", 2024-09-18T10:15:07.639 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:07.639 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.639 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.639 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:07.639 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.446637+0000", 2024-09-18T10:15:07.640 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.999999 2024-09-18T10:15:07.640 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.640 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.640 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:07.640 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.446985+0000", 2024-09-18T10:15:07.640 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00034822999999999999 2024-09-18T10:15:07.640 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.640 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.640 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:07.640 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.447007+0000", 2024-09-18T10:15:07.640 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1279000000000001e-05 2024-09-18T10:15:07.640 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.640 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.640 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:07.641 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.547113+0000", 2024-09-18T10:15:07.641 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100106532 2024-09-18T10:15:07.641 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.641 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.641 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:07.641 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.647607+0000", 2024-09-18T10:15:07.641 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100493575 2024-09-18T10:15:07.641 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.641 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.641 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:07.641 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.649364+0000", 2024-09-18T10:15:07.641 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001756861 2024-09-18T10:15:07.641 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:07.641 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:07.642 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:07.642 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:07.642 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:07.642 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-18T10:15:07.643 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-09-18T10:15:07.639+0000 7fdfa8698700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22087) UID: 0 2024-09-18T10:15:07.644 INFO:tasks.rados.rados.0.smithi189.stdout:145: done (8 left) 2024-09-18T10:15:07.644 INFO:tasks.rados.rados.0.smithi189.stdout:146: done (7 left) 2024-09-18T10:15:07.644 INFO:tasks.rados.rados.0.smithi189.stdout:150: done (6 left) 2024-09-18T10:15:07.644 INFO:tasks.rados.rados.0.smithi189.stdout:151: done (5 left) 2024-09-18T10:15:07.644 INFO:tasks.rados.rados.0.smithi189.stdout:152: done (4 left) 2024-09-18T10:15:07.644 INFO:tasks.rados.rados.0.smithi189.stdout:156: done (3 left) 2024-09-18T10:15:07.645 INFO:tasks.rados.rados.0.smithi189.stdout:157: done (2 left) 2024-09-18T10:15:07.645 INFO:tasks.rados.rados.0.smithi189.stdout:158: done (1 left) 2024-09-18T10:15:07.645 INFO:tasks.rados.rados.0.smithi189.stdout:159: done (0 left) 2024-09-18T10:15:07.645 INFO:tasks.rados.rados.0.smithi189.stdout:160: read oid 33 snap -1 2024-09-18T10:15:07.645 INFO:tasks.rados.rados.0.smithi189.stdout:160: expect (ObjNum 32 snap 0 seq_num 32) 2024-09-18T10:15:07.645 INFO:tasks.rados.rados.0.smithi189.stdout:161: read oid 45 snap -1 2024-09-18T10:15:07.645 INFO:tasks.rados.rados.0.smithi189.stdout:161: expect (ObjNum 44 snap 0 seq_num 44) 2024-09-18T10:15:07.645 INFO:tasks.rados.rados.0.smithi189.stdout:162: snap_create 2024-09-18T10:15:07.645 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_ops_in_flight 2024-09-18T10:15:07.646 INFO:teuthology.orchestra.run.smithi029.stderr:noscrub is set 2024-09-18T10:15:07.745 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-18T10:15:07.741+0000 7f578c52f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22134) UID: 0 2024-09-18T10:15:07.755 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-18T10:15:07.756 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-18T10:15:07.756 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 0 2024-09-18T10:15:07.756 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-18T10:15:07.767 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_blocked_ops 2024-09-18T10:15:07.845 INFO:tasks.ceph.osd.11.smithi189.stderr:2024-09-18T10:15:07.843+0000 7f718b60a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 22400) UID: 0 2024-09-18T10:15:07.864 INFO:tasks.rados.rados.0.smithi189.stdout:161: expect (ObjNum 44 snap 0 seq_num 44) 2024-09-18T10:15:07.868 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-18T10:15:07.868 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-18T10:15:07.869 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-18T10:15:07.869 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-18T10:15:07.869 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-18T10:15:07.878 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_historic_ops 2024-09-18T10:15:07.946 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-18T10:15:07.942+0000 7f302f2c3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22187) UID: 0 2024-09-18T10:15:07.967 INFO:tasks.rados.rados.0.smithi189.stdout:160: expect (ObjNum 32 snap 0 seq_num 32) 2024-09-18T10:15:07.975 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-18T10:15:07.975 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-18T10:15:07.975 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-18T10:15:07.975 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-18T10:15:07.976 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.976 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWrite(3.bs3 36/32 ECSubWrite(tid=29, reqid=client.4632.0:119, at_version=0'0, trim_to=0'0, roll_forward_to=36'4))", 2024-09-18T10:15:07.976 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:51.111475+0000", 2024-09-18T10:15:07.976 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 16.860206521999999, 2024-09-18T10:15:07.976 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.101555193, 2024-09-18T10:15:07.976 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:07.976 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:07.976 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:07.977 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.977 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:07.977 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.111475+0000", 2024-09-18T10:15:07.977 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:07.977 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.977 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.977 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:07.977 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.111475+0000", 2024-09-18T10:15:07.977 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:07.977 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.977 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.977 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:07.977 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.111474+0000", 2024-09-18T10:15:07.978 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:07.978 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.978 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.978 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:07.978 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.111492+0000", 2024-09-18T10:15:07.978 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7985000000000001e-05 2024-09-18T10:15:07.978 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.978 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.978 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:07.978 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.111511+0000", 2024-09-18T10:15:07.978 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8584e-05 2024-09-18T10:15:07.978 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.978 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.978 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:07.979 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.111521+0000", 2024-09-18T10:15:07.979 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0513000000000001e-05 2024-09-18T10:15:07.979 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.979 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.979 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:07.979 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.211779+0000", 2024-09-18T10:15:07.979 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10025748800000001 2024-09-18T10:15:07.979 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.979 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.979 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:07.979 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.211796+0000", 2024-09-18T10:15:07.980 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7655999999999998e-05 2024-09-18T10:15:07.980 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.980 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.980 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:07.980 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.212965+0000", 2024-09-18T10:15:07.980 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0011683679999999999 2024-09-18T10:15:07.980 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.980 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.980 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:07.980 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.213031+0000", 2024-09-18T10:15:07.980 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.5962000000000003e-05 2024-09-18T10:15:07.980 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:07.980 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:07.981 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:07.981 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.981 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.981 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpRead(3.bs3 36/32 ECSubRead(tid=30, to_read={3:d62b3dfa:::smithi18925896-40:head=0,4096,0}, subchunks={3:d62b3dfa:::smithi18925896-40:head=[0,1]}, attrs_to_read=))", 2024-09-18T10:15:07.981 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:51.112426+0000", 2024-09-18T10:15:07.981 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 16.859255887, 2024-09-18T10:15:07.981 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10039024000000001, 2024-09-18T10:15:07.981 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:07.981 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:07.981 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:07.981 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.981 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:07.981 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.112426+0000", 2024-09-18T10:15:07.982 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:07.982 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.982 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.982 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:07.982 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.112426+0000", 2024-09-18T10:15:07.982 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:07.982 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.982 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.982 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:07.982 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.112424+0000", 2024-09-18T10:15:07.982 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:07.982 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.982 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.982 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:07.983 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.112457+0000", 2024-09-18T10:15:07.983 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.3121999999999999e-05 2024-09-18T10:15:07.983 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.983 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.983 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:07.983 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.112512+0000", 2024-09-18T10:15:07.983 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4929000000000002e-05 2024-09-18T10:15:07.983 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.983 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.983 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:07.983 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.112535+0000", 2024-09-18T10:15:07.983 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3376000000000001e-05 2024-09-18T10:15:07.983 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.984 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.984 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:07.984 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.212254+0000", 2024-09-18T10:15:07.984 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099718673999999993 2024-09-18T10:15:07.984 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.984 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.984 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:07.984 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.212816+0000", 2024-09-18T10:15:07.984 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00056218499999999999 2024-09-18T10:15:07.984 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:07.984 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:07.984 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:07.984 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.984 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.985 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWrite(3.cs5 36/32 ECSubWrite(tid=20, reqid=client.4632.0:150, at_version=36'9, trim_to=0'0, roll_forward_to=36'8))", 2024-09-18T10:15:07.985 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:51.127656+0000", 2024-09-18T10:15:07.985 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 16.844025917, 2024-09-18T10:15:07.985 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10369326700000001, 2024-09-18T10:15:07.985 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:07.985 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:07.985 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:07.985 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.985 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:07.985 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.127656+0000", 2024-09-18T10:15:07.985 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:07.985 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.986 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.986 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:07.986 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.127656+0000", 2024-09-18T10:15:07.986 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:07.986 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.986 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.986 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:07.986 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.127655+0000", 2024-09-18T10:15:07.986 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.999999 2024-09-18T10:15:07.986 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.986 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.986 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:07.986 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.127882+0000", 2024-09-18T10:15:07.987 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00022688799999999999 2024-09-18T10:15:07.987 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.987 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.987 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:07.987 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.127906+0000", 2024-09-18T10:15:07.987 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.4749e-05 2024-09-18T10:15:07.987 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.987 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.987 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:07.988 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.127917+0000", 2024-09-18T10:15:07.988 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0344e-05 2024-09-18T10:15:07.988 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.988 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.988 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:07.988 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.228119+0000", 2024-09-18T10:15:07.988 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100201944 2024-09-18T10:15:07.988 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.988 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.988 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:07.988 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.228135+0000", 2024-09-18T10:15:07.988 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.6354e-05 2024-09-18T10:15:07.988 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.989 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.989 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:07.989 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.231252+0000", 2024-09-18T10:15:07.989 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0031171200000000001 2024-09-18T10:15:07.989 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.989 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.989 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:07.989 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.231349+0000", 2024-09-18T10:15:07.989 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.7021999999999994e-05 2024-09-18T10:15:07.989 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:07.989 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:07.989 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:07.990 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.990 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.990 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWrite(3.cs5 36/32 ECSubWrite(tid=21, reqid=client.4632.0:151, at_version=36'10, trim_to=0'0, roll_forward_to=36'8))", 2024-09-18T10:15:07.990 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:51.130378+0000", 2024-09-18T10:15:07.990 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 16.841303858, 2024-09-18T10:15:07.990 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10150170999999999, 2024-09-18T10:15:07.990 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:07.990 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:07.990 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:07.990 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.990 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:07.990 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.130378+0000", 2024-09-18T10:15:07.990 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:07.990 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.991 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.991 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:07.991 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.130378+0000", 2024-09-18T10:15:07.991 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:07.991 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.991 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.991 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:07.991 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.130375+0000", 2024-09-18T10:15:07.991 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999967 2024-09-18T10:15:07.991 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.991 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.991 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:07.992 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.130518+0000", 2024-09-18T10:15:07.992 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00014364299999999999 2024-09-18T10:15:07.992 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.992 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.992 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:07.992 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.130540+0000", 2024-09-18T10:15:07.992 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2072000000000001e-05 2024-09-18T10:15:07.992 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.992 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.992 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:07.992 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.130553+0000", 2024-09-18T10:15:07.993 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2432999999999999e-05 2024-09-18T10:15:07.993 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.993 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.993 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:07.993 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.230723+0000", 2024-09-18T10:15:07.993 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10016994899999999 2024-09-18T10:15:07.993 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.993 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.993 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:07.993 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.230734+0000", 2024-09-18T10:15:07.993 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0874e-05 2024-09-18T10:15:07.993 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.993 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.993 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:07.994 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.231854+0000", 2024-09-18T10:15:07.994 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0011205399999999999 2024-09-18T10:15:07.994 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.994 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.994 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:07.994 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.231880+0000", 2024-09-18T10:15:07.994 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.5571e-05 2024-09-18T10:15:07.994 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:07.994 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:07.994 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:07.994 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.994 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.995 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWrite(3.es6 36/32 ECSubWrite(tid=4, reqid=client.4632.0:146, at_version=36'3, trim_to=0'0, roll_forward_to=36'2))", 2024-09-18T10:15:07.995 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:51.226533+0000", 2024-09-18T10:15:07.995 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 16.745148620999998, 2024-09-18T10:15:07.995 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.102071464, 2024-09-18T10:15:07.995 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:07.995 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:07.995 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:07.995 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.995 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:07.995 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.226533+0000", 2024-09-18T10:15:07.995 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:07.995 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.995 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.996 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:07.996 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.226533+0000", 2024-09-18T10:15:07.996 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:07.996 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.996 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.996 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:07.996 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.226531+0000", 2024-09-18T10:15:07.996 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:07.996 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.996 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.996 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:07.996 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.226807+0000", 2024-09-18T10:15:07.996 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00027599800000000001 2024-09-18T10:15:07.997 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.997 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.997 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:07.997 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.226849+0000", 2024-09-18T10:15:07.997 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1439000000000001e-05 2024-09-18T10:15:07.997 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.997 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.997 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:07.997 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.326947+0000", 2024-09-18T10:15:07.997 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100098859 2024-09-18T10:15:07.997 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.997 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.997 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:07.997 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.327061+0000", 2024-09-18T10:15:07.998 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00011343199999999999 2024-09-18T10:15:07.998 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.998 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.998 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:07.998 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.327083+0000", 2024-09-18T10:15:07.998 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2509000000000001e-05 2024-09-18T10:15:07.998 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.998 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.998 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:07.998 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.328552+0000", 2024-09-18T10:15:07.998 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0014686339999999999 2024-09-18T10:15:07.998 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.998 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.999 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:07.999 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.328605+0000", 2024-09-18T10:15:07.999 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2639000000000002e-05 2024-09-18T10:15:07.999 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:07.999 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:07.999 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:07.999 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:07.999 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:07.999 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWrite(3.bs3 36/32 ECSubWrite(tid=37, reqid=client.4632.0:160, at_version=0'0, trim_to=0'0, roll_forward_to=36'5))", 2024-09-18T10:15:07.999 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:51.531559+0000", 2024-09-18T10:15:07.999 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 16.440122431999999, 2024-09-18T10:15:07.999 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10164537799999999, 2024-09-18T10:15:07.999 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:08.000 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:08.000 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:08.000 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.000 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:08.000 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.531559+0000", 2024-09-18T10:15:08.000 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:08.000 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.000 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.000 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:08.000 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.531559+0000", 2024-09-18T10:15:08.000 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:08.000 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.001 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.001 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:08.001 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.531558+0000", 2024-09-18T10:15:08.001 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:08.001 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.001 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.001 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:08.001 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.531584+0000", 2024-09-18T10:15:08.001 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6066000000000001e-05 2024-09-18T10:15:08.001 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.001 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.001 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:08.001 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.531620+0000", 2024-09-18T10:15:08.002 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.6625999999999998e-05 2024-09-18T10:15:08.002 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.002 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.002 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:08.002 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.531635+0000", 2024-09-18T10:15:08.002 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4175999999999999e-05 2024-09-18T10:15:08.002 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.002 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.002 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:08.002 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.631831+0000", 2024-09-18T10:15:08.002 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10019641 2024-09-18T10:15:08.002 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.002 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.002 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:08.003 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.631859+0000", 2024-09-18T10:15:08.003 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.8289e-05 2024-09-18T10:15:08.003 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.003 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.003 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:08.003 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.633127+0000", 2024-09-18T10:15:08.003 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001267287 2024-09-18T10:15:08.003 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.003 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.003 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:08.003 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.633205+0000", 2024-09-18T10:15:08.003 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.8244000000000005e-05 2024-09-18T10:15:08.003 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:08.004 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:08.004 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:08.004 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.004 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.004 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWrite(3.es6 36/32 ECSubWrite(tid=7, reqid=client.4632.0:148, at_version=36'5, trim_to=0'0, roll_forward_to=36'4))", 2024-09-18T10:15:08.004 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:51.835142+0000", 2024-09-18T10:15:08.004 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 16.136539891999998, 2024-09-18T10:15:08.004 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.101412908, 2024-09-18T10:15:08.004 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:08.004 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:08.004 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:08.004 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.004 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:08.005 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.835142+0000", 2024-09-18T10:15:08.005 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:08.005 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.005 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.005 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:08.005 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.835142+0000", 2024-09-18T10:15:08.005 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:08.005 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.005 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.006 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:08.006 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.835140+0000", 2024-09-18T10:15:08.006 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:08.006 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.006 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.006 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:08.006 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.835164+0000", 2024-09-18T10:15:08.006 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3464000000000001e-05 2024-09-18T10:15:08.006 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.006 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.006 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:08.006 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.835183+0000", 2024-09-18T10:15:08.006 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.91e-05 2024-09-18T10:15:08.006 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.007 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.007 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:08.007 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.835192+0000", 2024-09-18T10:15:08.007 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.064e-06 2024-09-18T10:15:08.007 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.007 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.007 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:08.007 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.935399+0000", 2024-09-18T10:15:08.007 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100206792 2024-09-18T10:15:08.007 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.007 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.007 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:08.008 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.935414+0000", 2024-09-18T10:15:08.008 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5119e-05 2024-09-18T10:15:08.008 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.008 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.008 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:08.008 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.936521+0000", 2024-09-18T10:15:08.008 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0011073890000000001 2024-09-18T10:15:08.008 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.008 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.008 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:08.008 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.936555+0000", 2024-09-18T10:15:08.008 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.349e-05 2024-09-18T10:15:08.009 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:08.009 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:08.009 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:08.009 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.009 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.009 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWrite(3.cs5 37/32 ECSubWrite(tid=29, reqid=client.4632.0:181, at_version=37'13, trim_to=0'0, roll_forward_to=36'11))", 2024-09-18T10:15:08.009 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:52.895578+0000", 2024-09-18T10:15:08.009 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 15.076103363, 2024-09-18T10:15:08.009 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.102246569, 2024-09-18T10:15:08.009 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:08.009 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:08.009 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:08.009 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.010 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:08.010 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.895578+0000", 2024-09-18T10:15:08.010 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:08.010 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.010 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.010 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:08.010 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.895578+0000", 2024-09-18T10:15:08.010 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:08.010 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.010 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.010 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:08.010 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.895578+0000", 2024-09-18T10:15:08.010 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:08.010 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.011 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.011 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:08.011 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.895958+0000", 2024-09-18T10:15:08.011 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00038039299999999997 2024-09-18T10:15:08.011 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.011 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.011 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:08.011 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.895978+0000", 2024-09-18T10:15:08.011 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9896999999999999e-05 2024-09-18T10:15:08.011 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.011 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.011 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:08.012 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.895989+0000", 2024-09-18T10:15:08.012 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0438000000000001e-05 2024-09-18T10:15:08.012 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.012 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.012 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:08.012 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.996171+0000", 2024-09-18T10:15:08.012 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10018293 2024-09-18T10:15:08.012 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.012 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.012 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:08.012 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.996182+0000", 2024-09-18T10:15:08.012 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.006e-05 2024-09-18T10:15:08.012 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.013 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.013 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:08.013 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.997758+0000", 2024-09-18T10:15:08.013 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0015764520000000001 2024-09-18T10:15:08.013 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.013 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.013 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:08.013 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.997825+0000", 2024-09-18T10:15:08.013 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7061e-05 2024-09-18T10:15:08.013 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:08.013 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:08.013 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:08.013 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.014 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.014 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWrite(3.cs5 37/32 ECSubWrite(tid=30, reqid=client.4632.0:182, at_version=37'14, trim_to=0'0, roll_forward_to=36'11))", 2024-09-18T10:15:08.014 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:52.899572+0000", 2024-09-18T10:15:08.014 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 15.072109379, 2024-09-18T10:15:08.014 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099223695000000001, 2024-09-18T10:15:08.014 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:08.014 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:08.014 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:08.014 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.014 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:08.014 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.899572+0000", 2024-09-18T10:15:08.014 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:08.014 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.015 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.015 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:08.015 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.899572+0000", 2024-09-18T10:15:08.015 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:08.015 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.015 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.015 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:08.015 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.899571+0000", 2024-09-18T10:15:08.015 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:08.015 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.015 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.015 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:08.015 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.899811+0000", 2024-09-18T10:15:08.015 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00024048900000000001 2024-09-18T10:15:08.016 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.016 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.016 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:08.016 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.899846+0000", 2024-09-18T10:15:08.016 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.4702000000000001e-05 2024-09-18T10:15:08.016 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.016 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.016 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:08.016 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.899862+0000", 2024-09-18T10:15:08.016 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.6155e-05 2024-09-18T10:15:08.016 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.016 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.016 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:08.017 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.997160+0000", 2024-09-18T10:15:08.017 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.097298481000000006 2024-09-18T10:15:08.017 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.017 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.017 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:08.017 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.997174+0000", 2024-09-18T10:15:08.017 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3162e-05 2024-09-18T10:15:08.017 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.017 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.017 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:08.017 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.998760+0000", 2024-09-18T10:15:08.017 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0015868200000000001 2024-09-18T10:15:08.017 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.017 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.018 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:08.018 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.998796+0000", 2024-09-18T10:15:08.018 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.5771e-05 2024-09-18T10:15:08.018 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:08.018 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:08.018 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:08.018 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.018 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.018 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWrite(3.cs5 37/32 ECSubWrite(tid=33, reqid=client.4632.0:197, at_version=37'18, trim_to=0'0, roll_forward_to=36'11))", 2024-09-18T10:15:08.018 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:53.108605+0000", 2024-09-18T10:15:08.018 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 14.863076595999999, 2024-09-18T10:15:08.018 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.20153053300000001, 2024-09-18T10:15:08.018 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:08.019 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:08.019 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:08.019 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.019 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:08.019 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:53.108605+0000", 2024-09-18T10:15:08.019 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:08.019 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.019 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.019 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:08.019 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:53.108605+0000", 2024-09-18T10:15:08.019 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:08.019 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.019 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.019 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:08.020 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:53.108604+0000", 2024-09-18T10:15:08.020 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.999999 2024-09-18T10:15:08.020 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.020 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.020 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:08.020 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:53.108627+0000", 2024-09-18T10:15:08.020 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2954000000000001e-05 2024-09-18T10:15:08.020 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.020 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.020 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:08.020 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:53.108644+0000", 2024-09-18T10:15:08.020 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.6867e-05 2024-09-18T10:15:08.020 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.021 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.021 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:08.021 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:53.208724+0000", 2024-09-18T10:15:08.021 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10007999099999999 2024-09-18T10:15:08.021 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.021 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.021 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:08.021 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:53.308920+0000", 2024-09-18T10:15:08.021 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100196046 2024-09-18T10:15:08.021 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.021 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.021 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:08.021 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:53.308936+0000", 2024-09-18T10:15:08.022 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5537e-05 2024-09-18T10:15:08.022 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.022 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.022 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:08.022 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:53.310086+0000", 2024-09-18T10:15:08.022 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0011503850000000001 2024-09-18T10:15:08.022 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.022 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.022 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:08.022 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:53.310136+0000", 2024-09-18T10:15:08.022 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9514000000000001e-05 2024-09-18T10:15:08.022 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:08.022 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:08.022 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:08.023 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.023 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.023 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWrite(3.cs5 41/32 ECSubWrite(tid=45, reqid=osd.5.0:45, at_version=41'25, trim_to=0'0, roll_forward_to=39'23))", 2024-09-18T10:15:08.023 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:56.728134+0000", 2024-09-18T10:15:08.023 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 11.243548111000001, 2024-09-18T10:15:08.023 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0050947600000000003, 2024-09-18T10:15:08.023 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:08.023 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:08.023 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:08.023 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.023 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:08.023 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.728134+0000", 2024-09-18T10:15:08.024 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:08.024 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.024 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.024 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:08.024 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.728134+0000", 2024-09-18T10:15:08.024 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:08.024 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.024 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.024 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:08.024 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.728132+0000", 2024-09-18T10:15:08.024 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:08.024 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.025 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.025 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:08.025 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.728256+0000", 2024-09-18T10:15:08.025 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00012371999999999999 2024-09-18T10:15:08.025 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.025 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.025 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:08.025 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.728306+0000", 2024-09-18T10:15:08.025 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9725999999999998e-05 2024-09-18T10:15:08.025 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.025 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.026 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:08.026 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.728329+0000", 2024-09-18T10:15:08.026 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3785999999999999e-05 2024-09-18T10:15:08.026 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.026 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.026 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:08.026 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.729430+0000", 2024-09-18T10:15:08.026 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001100497 2024-09-18T10:15:08.026 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.026 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.026 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:08.026 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.729458+0000", 2024-09-18T10:15:08.026 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.8362000000000001e-05 2024-09-18T10:15:08.027 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.027 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.027 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:08.027 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.733159+0000", 2024-09-18T10:15:08.027 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0037007699999999999 2024-09-18T10:15:08.027 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.027 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.027 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:08.027 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.733229+0000", 2024-09-18T10:15:08.027 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9496e-05 2024-09-18T10:15:08.027 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:08.027 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:08.027 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:08.028 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.028 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.028 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWrite(3.cs5 41/32 ECSubWrite(tid=46, reqid=osd.5.0:46, at_version=41'27, trim_to=0'0, roll_forward_to=39'23))", 2024-09-18T10:15:08.028 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:56.729037+0000", 2024-09-18T10:15:08.028 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 11.242645251000001, 2024-09-18T10:15:08.028 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0053448910000000001, 2024-09-18T10:15:08.028 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:08.028 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:08.028 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:08.028 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.028 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:08.028 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.729037+0000", 2024-09-18T10:15:08.029 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:08.029 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.029 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.029 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:08.029 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.729037+0000", 2024-09-18T10:15:08.029 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:08.029 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.029 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.029 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:08.029 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.729035+0000", 2024-09-18T10:15:08.029 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:08.029 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.030 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.030 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:08.030 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.729181+0000", 2024-09-18T10:15:08.030 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00014593399999999999 2024-09-18T10:15:08.030 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.030 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.030 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:08.030 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.729243+0000", 2024-09-18T10:15:08.030 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.2192000000000004e-05 2024-09-18T10:15:08.030 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.030 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.030 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:08.031 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.729263+0000", 2024-09-18T10:15:08.031 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9602999999999998e-05 2024-09-18T10:15:08.031 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.031 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.031 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:08.031 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.731783+0000", 2024-09-18T10:15:08.031 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0025209490000000002 2024-09-18T10:15:08.031 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.031 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.031 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:08.031 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.731808+0000", 2024-09-18T10:15:08.031 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.4363e-05 2024-09-18T10:15:08.031 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.031 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.032 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:08.032 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.734350+0000", 2024-09-18T10:15:08.032 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.002541728 2024-09-18T10:15:08.032 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.032 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.032 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:08.032 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.734381+0000", 2024-09-18T10:15:08.032 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1909000000000002e-05 2024-09-18T10:15:08.032 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:08.032 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:08.032 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:08.032 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.032 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.033 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWrite(3.bs3 41/32 ECSubWrite(tid=58, reqid=osd.4.0:58, at_version=41'14, trim_to=0'0, roll_forward_to=39'12))", 2024-09-18T10:15:08.033 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:56.830265+0000", 2024-09-18T10:15:08.033 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 11.141416751, 2024-09-18T10:15:08.033 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10214519699999999, 2024-09-18T10:15:08.033 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:08.033 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:08.033 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:08.033 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.033 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:08.033 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.830265+0000", 2024-09-18T10:15:08.033 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:08.033 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.033 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.033 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:08.034 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.830265+0000", 2024-09-18T10:15:08.034 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:08.034 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.034 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.034 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:08.034 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.830263+0000", 2024-09-18T10:15:08.034 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:08.034 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.034 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.034 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:08.034 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.830323+0000", 2024-09-18T10:15:08.034 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9846000000000003e-05 2024-09-18T10:15:08.034 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.035 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.035 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:08.035 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.830370+0000", 2024-09-18T10:15:08.035 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6798999999999997e-05 2024-09-18T10:15:08.035 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.035 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.035 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:08.035 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.830389+0000", 2024-09-18T10:15:08.035 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9530000000000001e-05 2024-09-18T10:15:08.035 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.035 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.035 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:08.035 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.930612+0000", 2024-09-18T10:15:08.035 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100222795 2024-09-18T10:15:08.036 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.036 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.036 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:08.036 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.930648+0000", 2024-09-18T10:15:08.036 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.6115999999999998e-05 2024-09-18T10:15:08.036 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.036 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.036 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:08.036 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.932338+0000", 2024-09-18T10:15:08.036 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0016898130000000001 2024-09-18T10:15:08.036 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.036 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.036 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:08.036 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.932410+0000", 2024-09-18T10:15:08.037 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.2214999999999999e-05 2024-09-18T10:15:08.037 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:08.037 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:08.037 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:08.037 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.037 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.037 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWrite(3.es6 41/32 ECSubWrite(tid=16, reqid=osd.11.0:15, at_version=0'0, trim_to=0'0, roll_forward_to=41'11))", 2024-09-18T10:15:08.037 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:56.839285+0000", 2024-09-18T10:15:08.037 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 11.132396749, 2024-09-18T10:15:08.037 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100907127, 2024-09-18T10:15:08.037 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:08.037 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:08.037 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:08.038 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.038 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:08.038 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.839285+0000", 2024-09-18T10:15:08.038 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:08.038 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.038 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.038 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:08.038 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.839285+0000", 2024-09-18T10:15:08.038 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:08.038 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.038 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.038 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:08.038 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.839284+0000", 2024-09-18T10:15:08.039 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.999999 2024-09-18T10:15:08.039 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.039 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.039 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:08.039 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.839300+0000", 2024-09-18T10:15:08.039 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5852e-05 2024-09-18T10:15:08.039 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.039 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.039 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:08.040 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.839319+0000", 2024-09-18T10:15:08.040 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9242999999999999e-05 2024-09-18T10:15:08.040 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.040 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.040 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:08.040 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.839329+0000", 2024-09-18T10:15:08.040 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.4849999999999998e-06 2024-09-18T10:15:08.040 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.040 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.040 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:08.040 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.939487+0000", 2024-09-18T10:15:08.040 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100158518 2024-09-18T10:15:08.040 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.041 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.041 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:08.041 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.939503+0000", 2024-09-18T10:15:08.041 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5305999999999999e-05 2024-09-18T10:15:08.041 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.041 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.041 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:08.041 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.940151+0000", 2024-09-18T10:15:08.041 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00064866700000000002 2024-09-18T10:15:08.041 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.041 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.041 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:08.041 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.940192+0000", 2024-09-18T10:15:08.042 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0976000000000002e-05 2024-09-18T10:15:08.042 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:08.042 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:08.042 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:08.042 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.042 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.042 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWrite(3.bs3 41/32 ECSubWrite(tid=62, reqid=osd.4.0:58, at_version=0'0, trim_to=0'0, roll_forward_to=41'14))", 2024-09-18T10:15:08.042 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:57.647903+0000", 2024-09-18T10:15:08.042 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 10.32377878, 2024-09-18T10:15:08.042 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.20108230399999999, 2024-09-18T10:15:08.042 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:08.042 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:08.042 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:08.042 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.043 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:08.043 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.647903+0000", 2024-09-18T10:15:08.043 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:08.043 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.043 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.043 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:08.043 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.647903+0000", 2024-09-18T10:15:08.043 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:08.043 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.043 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.043 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:08.043 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.647901+0000", 2024-09-18T10:15:08.043 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999976 2024-09-18T10:15:08.043 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.044 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.044 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:08.044 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.647936+0000", 2024-09-18T10:15:08.044 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.5329e-05 2024-09-18T10:15:08.044 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.044 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.044 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:08.044 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.647978+0000", 2024-09-18T10:15:08.044 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2145999999999999e-05 2024-09-18T10:15:08.044 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.044 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.044 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:08.044 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.647996+0000", 2024-09-18T10:15:08.045 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7331999999999999e-05 2024-09-18T10:15:08.045 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.045 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.045 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:08.045 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.748248+0000", 2024-09-18T10:15:08.045 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100252307 2024-09-18T10:15:08.045 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.045 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.045 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:08.045 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.748280+0000", 2024-09-18T10:15:08.045 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2316e-05 2024-09-18T10:15:08.045 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.046 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.046 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:08.046 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.848945+0000", 2024-09-18T10:15:08.046 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100664434 2024-09-18T10:15:08.046 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.046 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.046 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:08.046 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.848985+0000", 2024-09-18T10:15:08.046 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0605999999999997e-05 2024-09-18T10:15:08.046 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:08.046 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:08.046 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:08.046 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.046 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.047 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWrite(3.bs3 46/32 ECSubWrite(tid=75, reqid=client.4632.0:327, at_version=0'0, trim_to=0'0, roll_forward_to=46'17))", 2024-09-18T10:15:08.047 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:02.228270+0000", 2024-09-18T10:15:08.047 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 5.743412094, 2024-09-18T10:15:08.047 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.101562179, 2024-09-18T10:15:08.047 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:08.047 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:08.047 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:08.047 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.047 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:08.047 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.228270+0000", 2024-09-18T10:15:08.047 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:08.047 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.047 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.048 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:08.048 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.228270+0000", 2024-09-18T10:15:08.048 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:08.048 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.048 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.048 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:08.048 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.228268+0000", 2024-09-18T10:15:08.048 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:08.048 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.048 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.048 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:08.048 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.228295+0000", 2024-09-18T10:15:08.048 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6809e-05 2024-09-18T10:15:08.048 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.049 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.049 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:08.049 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.228369+0000", 2024-09-18T10:15:08.049 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.3645000000000004e-05 2024-09-18T10:15:08.049 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.049 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.049 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:08.049 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.228388+0000", 2024-09-18T10:15:08.049 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9403e-05 2024-09-18T10:15:08.049 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.049 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.049 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:08.049 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.328674+0000", 2024-09-18T10:15:08.049 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100285687 2024-09-18T10:15:08.050 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.050 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.050 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:08.050 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.328690+0000", 2024-09-18T10:15:08.050 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5956000000000001e-05 2024-09-18T10:15:08.050 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.050 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.050 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:08.050 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.329701+0000", 2024-09-18T10:15:08.050 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0010117609999999999 2024-09-18T10:15:08.050 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.050 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.050 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:08.051 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.329832+0000", 2024-09-18T10:15:08.051 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000130541 2024-09-18T10:15:08.051 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:08.051 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:08.051 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:08.051 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.051 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.051 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWrite(3.es6 47/32 ECSubWrite(tid=21, reqid=client.4632.0:346, at_version=47'17, trim_to=0'0, roll_forward_to=44'15))", 2024-09-18T10:15:08.051 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:02.905682+0000", 2024-09-18T10:15:08.051 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 5.0659998709999998, 2024-09-18T10:15:08.051 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.104106903, 2024-09-18T10:15:08.051 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:08.051 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:08.051 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:08.052 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.052 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:08.052 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.905682+0000", 2024-09-18T10:15:08.052 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:08.052 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.052 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.052 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:08.052 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.905682+0000", 2024-09-18T10:15:08.052 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:08.052 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.052 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.052 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:08.052 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.905682+0000", 2024-09-18T10:15:08.053 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:08.053 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.053 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.053 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:08.053 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.906384+0000", 2024-09-18T10:15:08.053 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000702362 2024-09-18T10:15:08.053 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.053 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.053 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:08.054 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.906428+0000", 2024-09-18T10:15:08.054 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4199000000000002e-05 2024-09-18T10:15:08.054 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.054 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.054 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:08.054 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:03.006578+0000", 2024-09-18T10:15:08.054 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100149618 2024-09-18T10:15:08.054 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.054 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.054 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:08.054 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:03.006758+0000", 2024-09-18T10:15:08.054 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00018075599999999999 2024-09-18T10:15:08.054 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.055 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.055 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:08.055 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:03.006775+0000", 2024-09-18T10:15:08.055 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.6673000000000001e-05 2024-09-18T10:15:08.055 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.055 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.055 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:08.055 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:03.009712+0000", 2024-09-18T10:15:08.055 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.002936569 2024-09-18T10:15:08.055 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.055 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.055 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:08.055 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:03.009789+0000", 2024-09-18T10:15:08.055 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7169000000000006e-05 2024-09-18T10:15:08.056 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:08.056 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:08.056 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:08.056 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.056 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.056 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWrite(3.cs5 47/32 ECSubWrite(tid=58, reqid=client.4632.0:331, at_version=47'35, trim_to=0'0, roll_forward_to=45'33))", 2024-09-18T10:15:08.056 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:02.906433+0000", 2024-09-18T10:15:08.056 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 5.0652488199999999, 2024-09-18T10:15:08.056 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0040846720000000001, 2024-09-18T10:15:08.056 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:08.056 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:08.056 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:08.056 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.057 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:08.057 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.906433+0000", 2024-09-18T10:15:08.057 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:08.057 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.057 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.057 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:08.057 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.906433+0000", 2024-09-18T10:15:08.057 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:08.057 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.057 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.057 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:08.057 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.906432+0000", 2024-09-18T10:15:08.057 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:08.058 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.058 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.058 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:08.058 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.906738+0000", 2024-09-18T10:15:08.058 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00030501999999999998 2024-09-18T10:15:08.058 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.058 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.058 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:08.058 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.906770+0000", 2024-09-18T10:15:08.058 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2079000000000002e-05 2024-09-18T10:15:08.058 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.058 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.058 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:08.058 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.906788+0000", 2024-09-18T10:15:08.059 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.819e-05 2024-09-18T10:15:08.059 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.059 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.059 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:08.059 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.906978+0000", 2024-09-18T10:15:08.059 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00018976600000000001 2024-09-18T10:15:08.059 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.059 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.059 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:08.059 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.907013+0000", 2024-09-18T10:15:08.059 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.5366e-05 2024-09-18T10:15:08.059 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.059 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.060 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:08.060 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.910480+0000", 2024-09-18T10:15:08.060 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0034668759999999998 2024-09-18T10:15:08.060 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.060 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.060 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:08.060 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.910518+0000", 2024-09-18T10:15:08.060 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7929000000000002e-05 2024-09-18T10:15:08.060 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:08.060 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:08.060 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:08.060 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.060 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.061 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWrite(3.cs5 47/32 ECSubWrite(tid=60, reqid=client.4632.0:341, at_version=47'38, trim_to=0'0, roll_forward_to=45'33))", 2024-09-18T10:15:08.061 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:02.911299+0000", 2024-09-18T10:15:08.061 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 5.0603832449999997, 2024-09-18T10:15:08.061 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.101776582, 2024-09-18T10:15:08.061 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:08.061 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:08.061 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:08.061 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.061 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:08.061 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.911299+0000", 2024-09-18T10:15:08.061 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:08.061 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.061 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.062 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:08.062 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.911299+0000", 2024-09-18T10:15:08.062 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:08.062 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.062 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.062 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:08.062 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.911298+0000", 2024-09-18T10:15:08.062 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.999999 2024-09-18T10:15:08.062 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.062 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.062 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:08.062 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.911361+0000", 2024-09-18T10:15:08.063 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3900999999999997e-05 2024-09-18T10:15:08.063 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.063 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.063 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:08.063 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.911392+0000", 2024-09-18T10:15:08.063 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0275000000000001e-05 2024-09-18T10:15:08.063 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.063 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.063 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:08.063 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.911404+0000", 2024-09-18T10:15:08.063 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2682e-05 2024-09-18T10:15:08.063 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.063 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.064 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:08.064 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:03.011587+0000", 2024-09-18T10:15:08.064 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10018239 2024-09-18T10:15:08.064 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.064 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.064 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:08.064 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:03.011607+0000", 2024-09-18T10:15:08.064 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0329000000000001e-05 2024-09-18T10:15:08.064 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.064 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.064 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:08.065 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:03.013018+0000", 2024-09-18T10:15:08.065 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001410863 2024-09-18T10:15:08.065 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.065 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.065 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:08.065 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:03.013075+0000", 2024-09-18T10:15:08.065 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7182000000000002e-05 2024-09-18T10:15:08.065 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:08.065 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:08.065 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:08.065 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.065 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.066 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWrite(3.cs5 47/32 ECSubWrite(tid=61, reqid=client.4632.0:342, at_version=47'39, trim_to=0'0, roll_forward_to=45'33))", 2024-09-18T10:15:08.066 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:02.913383+0000", 2024-09-18T10:15:08.066 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 5.0582991320000001, 2024-09-18T10:15:08.066 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10027179, 2024-09-18T10:15:08.066 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:08.066 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:08.066 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:08.066 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.066 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:08.066 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.913383+0000", 2024-09-18T10:15:08.066 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:08.066 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.066 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.066 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:08.067 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.913383+0000", 2024-09-18T10:15:08.067 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:08.067 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.067 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.067 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:08.067 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.913377+0000", 2024-09-18T10:15:08.067 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999938 2024-09-18T10:15:08.067 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.067 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.067 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:08.067 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.913410+0000", 2024-09-18T10:15:08.067 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.3611000000000001e-05 2024-09-18T10:15:08.067 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.068 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.068 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:08.068 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.913441+0000", 2024-09-18T10:15:08.068 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1100999999999999e-05 2024-09-18T10:15:08.068 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.068 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.068 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:08.068 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.913457+0000", 2024-09-18T10:15:08.068 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5954e-05 2024-09-18T10:15:08.068 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.068 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.068 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:08.068 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:03.012156+0000", 2024-09-18T10:15:08.068 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.098699091000000003 2024-09-18T10:15:08.069 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.069 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.069 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:08.069 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:03.012171+0000", 2024-09-18T10:15:08.069 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4807000000000001e-05 2024-09-18T10:15:08.069 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.069 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.069 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:08.069 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:03.013597+0000", 2024-09-18T10:15:08.070 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001425976 2024-09-18T10:15:08.070 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:08.070 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:08.070 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:08.070 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:03.013655+0000", 2024-09-18T10:15:08.070 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7281999999999998e-05 2024-09-18T10:15:08.070 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:08.070 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:08.070 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:08.070 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:08.070 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:08.070 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-18T10:15:08.072 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-18T10:15:08.067+0000 7f14dcc5b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22398) UID: 0 2024-09-18T10:15:08.072 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T10:15:08.172 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-18T10:15:08.169+0000 7f14c6a1a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 22138) UID: 0 2024-09-18T10:15:08.180 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-18T10:15:08.180 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-18T10:15:08.180 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-09-18T10:15:08.180 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-18T10:15:08.190 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T10:15:08.274 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-18T10:15:08.270+0000 7f302f2c3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22187) UID: 0 2024-09-18T10:15:08.286 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-18T10:15:08.286 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-18T10:15:08.286 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-18T10:15:08.286 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-18T10:15:08.286 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-18T10:15:08.295 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T10:15:08.375 INFO:tasks.ceph.osd.8.smithi029.stderr:2024-09-18T10:15:08.370+0000 7f2555c27700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 22192) UID: 0 2024-09-18T10:15:08.393 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-18T10:15:08.393 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-18T10:15:08.393 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-18T10:15:08.393 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-18T10:15:08.393 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.393 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.2s3 36/32 ECSubWrite(tid=9, reqid=client.4632.0:100, at_version=36'5, trim_to=0'0, roll_forward_to=35'4))", 2024-09-18T10:15:08.393 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:50.301533+0000", 2024-09-18T10:15:08.393 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 18.089955195999998, 2024-09-18T10:15:08.393 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.102286764, 2024-09-18T10:15:08.394 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:08.394 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:08.394 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:08.394 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.394 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:08.394 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.301533+0000", 2024-09-18T10:15:08.394 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:08.394 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.394 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.394 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:08.394 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.301533+0000", 2024-09-18T10:15:08.394 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:08.394 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.394 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.395 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:08.395 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.301532+0000", 2024-09-18T10:15:08.395 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:08.395 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.395 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.395 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:08.395 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.301835+0000", 2024-09-18T10:15:08.395 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00030359099999999997 2024-09-18T10:15:08.395 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.395 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.395 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:08.395 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.301862+0000", 2024-09-18T10:15:08.395 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7282999999999999e-05 2024-09-18T10:15:08.396 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.396 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.396 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:08.396 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.301874+0000", 2024-09-18T10:15:08.396 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1590000000000001e-05 2024-09-18T10:15:08.396 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.396 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.397 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:08.397 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.402086+0000", 2024-09-18T10:15:08.397 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10021173899999999 2024-09-18T10:15:08.397 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.397 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.397 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:08.397 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.402106+0000", 2024-09-18T10:15:08.397 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0463e-05 2024-09-18T10:15:08.397 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.397 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.397 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:08.398 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.403765+0000", 2024-09-18T10:15:08.398 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0016588340000000001 2024-09-18T10:15:08.398 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.398 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.398 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:08.398 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.403820+0000", 2024-09-18T10:15:08.398 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5140999999999999e-05 2024-09-18T10:15:08.398 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:08.398 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:08.398 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:08.398 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.398 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.399 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.2s3 36/32 ECSubWrite(tid=10, reqid=client.4632.0:101, at_version=36'6, trim_to=0'0, roll_forward_to=35'4))", 2024-09-18T10:15:08.399 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:50.301900+0000", 2024-09-18T10:15:08.399 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 18.089588178, 2024-09-18T10:15:08.399 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10241130599999999, 2024-09-18T10:15:08.399 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:08.399 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:08.399 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:08.399 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.400 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:08.400 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.301900+0000", 2024-09-18T10:15:08.400 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:08.400 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.400 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.400 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:08.400 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.301900+0000", 2024-09-18T10:15:08.400 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:08.400 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.400 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.400 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:08.400 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.301900+0000", 2024-09-18T10:15:08.400 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:08.401 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.401 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.401 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:08.401 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.301917+0000", 2024-09-18T10:15:08.401 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6790999999999999e-05 2024-09-18T10:15:08.401 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.401 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.401 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:08.401 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.301925+0000", 2024-09-18T10:15:08.401 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3599999999999996e-06 2024-09-18T10:15:08.401 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.401 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.401 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:08.402 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.301930+0000", 2024-09-18T10:15:08.402 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4369999999999998e-06 2024-09-18T10:15:08.402 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.402 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.402 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:08.402 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.402886+0000", 2024-09-18T10:15:08.402 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100955429 2024-09-18T10:15:08.402 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.402 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.402 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:08.402 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.402908+0000", 2024-09-18T10:15:08.402 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2276000000000002e-05 2024-09-18T10:15:08.402 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.402 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.403 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:08.403 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.404270+0000", 2024-09-18T10:15:08.403 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001362121 2024-09-18T10:15:08.403 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.403 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.403 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:08.403 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.404312+0000", 2024-09-18T10:15:08.403 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1505999999999999e-05 2024-09-18T10:15:08.403 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:08.403 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:08.403 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:08.403 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.403 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.403 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.bs5 36/32 ECSubWrite(tid=25, reqid=client.4632.0:118, at_version=36'3, trim_to=0'0, roll_forward_to=35'2))", 2024-09-18T10:15:08.404 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:50.709294+0000", 2024-09-18T10:15:08.404 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 17.682194502000002, 2024-09-18T10:15:08.404 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.102322202, 2024-09-18T10:15:08.404 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:08.404 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:08.404 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:08.404 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.404 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:08.404 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.709294+0000", 2024-09-18T10:15:08.404 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:08.404 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.404 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.404 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:08.404 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.709294+0000", 2024-09-18T10:15:08.405 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:08.405 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.405 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.405 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:08.405 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.709292+0000", 2024-09-18T10:15:08.405 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:08.405 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.405 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.405 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:08.405 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.709955+0000", 2024-09-18T10:15:08.405 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00066278599999999995 2024-09-18T10:15:08.405 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.405 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.406 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:08.406 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.709987+0000", 2024-09-18T10:15:08.406 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2002000000000003e-05 2024-09-18T10:15:08.406 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.406 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.406 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:08.406 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.710001+0000", 2024-09-18T10:15:08.406 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4073000000000001e-05 2024-09-18T10:15:08.406 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.406 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.406 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:08.406 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.810188+0000", 2024-09-18T10:15:08.406 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10018661800000001 2024-09-18T10:15:08.406 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.406 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.407 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:08.407 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.810198+0000", 2024-09-18T10:15:08.407 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0349999999999999e-05 2024-09-18T10:15:08.407 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.407 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.407 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:08.407 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.811559+0000", 2024-09-18T10:15:08.407 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001361115 2024-09-18T10:15:08.407 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.407 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.407 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:08.407 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.811616+0000", 2024-09-18T10:15:08.407 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7238000000000003e-05 2024-09-18T10:15:08.408 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:08.408 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:08.408 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:08.408 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.408 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.408 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.8s5 36/32 ECSubWrite(tid=119, reqid=client.4632.0:130, at_version=36'5, trim_to=0'0, roll_forward_to=35'4))", 2024-09-18T10:15:08.408 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:50.722395+0000", 2024-09-18T10:15:08.408 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 17.669093627999999, 2024-09-18T10:15:08.408 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10448325999999999, 2024-09-18T10:15:08.408 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:08.408 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:08.408 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:08.408 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.409 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:08.409 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.722395+0000", 2024-09-18T10:15:08.409 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:08.409 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.409 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.409 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:08.409 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.722395+0000", 2024-09-18T10:15:08.409 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:08.409 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.409 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.409 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:08.409 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.722393+0000", 2024-09-18T10:15:08.409 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:08.410 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.410 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.410 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:08.410 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.722993+0000", 2024-09-18T10:15:08.410 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00059969600000000004 2024-09-18T10:15:08.410 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.410 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.410 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:08.410 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.723025+0000", 2024-09-18T10:15:08.410 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2364999999999999e-05 2024-09-18T10:15:08.410 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.410 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.410 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:08.411 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.723039+0000", 2024-09-18T10:15:08.411 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3709999999999999e-05 2024-09-18T10:15:08.411 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.411 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.411 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:08.411 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.823280+0000", 2024-09-18T10:15:08.411 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100241126 2024-09-18T10:15:08.411 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.411 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.411 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:08.411 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.823298+0000", 2024-09-18T10:15:08.411 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7745999999999999e-05 2024-09-18T10:15:08.411 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.412 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.412 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:08.412 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.826820+0000", 2024-09-18T10:15:08.412 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0035221470000000002 2024-09-18T10:15:08.412 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.412 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.412 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:08.412 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.826878+0000", 2024-09-18T10:15:08.412 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8344000000000002e-05 2024-09-18T10:15:08.412 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:08.412 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:08.412 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:08.412 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.412 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.413 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.bs5 36/32 ECSubWrite(tid=26, reqid=client.4632.0:119, at_version=36'4, trim_to=0'0, roll_forward_to=35'2))", 2024-09-18T10:15:08.413 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:50.812131+0000", 2024-09-18T10:15:08.413 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 17.579357821999999, 2024-09-18T10:15:08.413 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.201739738, 2024-09-18T10:15:08.413 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:08.413 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:08.413 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:08.413 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.413 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:08.413 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.812131+0000", 2024-09-18T10:15:08.413 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:08.413 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.414 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.414 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:08.414 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.812131+0000", 2024-09-18T10:15:08.414 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:08.414 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.414 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.414 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:08.414 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.812129+0000", 2024-09-18T10:15:08.414 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:08.414 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.414 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.414 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:08.414 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.812166+0000", 2024-09-18T10:15:08.415 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6309000000000001e-05 2024-09-18T10:15:08.415 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.415 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.415 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:08.415 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.812199+0000", 2024-09-18T10:15:08.415 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3219000000000002e-05 2024-09-18T10:15:08.415 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.415 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.415 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:08.415 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.812225+0000", 2024-09-18T10:15:08.415 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6302999999999999e-05 2024-09-18T10:15:08.415 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.415 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.416 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:08.416 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.012537+0000", 2024-09-18T10:15:08.416 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20031156899999999 2024-09-18T10:15:08.416 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.416 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.416 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:08.416 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.012564+0000", 2024-09-18T10:15:08.416 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7165000000000002e-05 2024-09-18T10:15:08.416 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.416 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.416 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:08.416 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.013814+0000", 2024-09-18T10:15:08.416 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001249836 2024-09-18T10:15:08.417 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.417 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.417 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:08.417 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.013871+0000", 2024-09-18T10:15:08.417 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6894000000000001e-05 2024-09-18T10:15:08.417 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:08.417 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:08.417 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:08.417 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.417 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.417 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.8s5 36/32 ECSubWrite(tid=121, reqid=client.4632.0:131, at_version=0'0, trim_to=0'0, roll_forward_to=36'6))", 2024-09-18T10:15:08.417 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:51.214254+0000", 2024-09-18T10:15:08.417 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 17.177234231, 2024-09-18T10:15:08.418 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.101744853, 2024-09-18T10:15:08.418 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:08.418 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:08.418 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:08.418 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.418 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:08.418 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.214254+0000", 2024-09-18T10:15:08.418 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:08.418 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.418 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.418 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:08.418 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.214254+0000", 2024-09-18T10:15:08.418 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:08.419 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.419 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.419 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:08.419 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.214253+0000", 2024-09-18T10:15:08.419 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:08.419 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.419 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.419 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:08.419 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.214281+0000", 2024-09-18T10:15:08.419 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8249e-05 2024-09-18T10:15:08.419 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.419 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.419 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:08.419 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.214311+0000", 2024-09-18T10:15:08.420 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9739000000000001e-05 2024-09-18T10:15:08.420 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.420 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.420 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:08.420 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.314400+0000", 2024-09-18T10:15:08.420 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10008956500000001 2024-09-18T10:15:08.420 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.420 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.420 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:08.420 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.314547+0000", 2024-09-18T10:15:08.420 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014725200000000001 2024-09-18T10:15:08.420 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.420 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.421 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:08.421 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.314581+0000", 2024-09-18T10:15:08.421 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3674999999999999e-05 2024-09-18T10:15:08.421 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.421 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.421 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:08.421 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.315954+0000", 2024-09-18T10:15:08.421 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0013734540000000001 2024-09-18T10:15:08.421 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.421 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.421 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:08.422 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.315999+0000", 2024-09-18T10:15:08.422 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4818999999999999e-05 2024-09-18T10:15:08.422 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:08.422 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:08.422 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:08.422 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.422 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.422 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.bs5 36/32 ECSubWrite(tid=35, reqid=client.4632.0:160, at_version=36'5, trim_to=0'0, roll_forward_to=36'4))", 2024-09-18T10:15:08.422 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:51.315565+0000", 2024-09-18T10:15:08.422 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 17.075924036, 2024-09-18T10:15:08.422 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10154431999999999, 2024-09-18T10:15:08.422 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:08.422 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:08.422 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:08.423 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.423 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:08.423 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.315565+0000", 2024-09-18T10:15:08.423 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:08.423 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.423 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.423 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:08.423 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.315565+0000", 2024-09-18T10:15:08.423 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:08.423 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.423 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.423 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:08.423 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.315563+0000", 2024-09-18T10:15:08.424 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:08.424 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.424 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.424 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:08.424 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.315603+0000", 2024-09-18T10:15:08.424 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0045e-05 2024-09-18T10:15:08.424 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.424 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.424 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:08.424 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.315623+0000", 2024-09-18T10:15:08.424 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9749999999999999e-05 2024-09-18T10:15:08.424 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.424 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.425 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:08.425 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.315634+0000", 2024-09-18T10:15:08.425 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1399e-05 2024-09-18T10:15:08.425 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.425 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.425 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:08.425 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.415886+0000", 2024-09-18T10:15:08.425 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100251874 2024-09-18T10:15:08.425 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.425 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.425 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:08.425 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.415901+0000", 2024-09-18T10:15:08.425 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4698999999999999e-05 2024-09-18T10:15:08.426 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.426 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.426 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:08.426 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.417028+0000", 2024-09-18T10:15:08.426 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00112751 2024-09-18T10:15:08.426 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.426 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.426 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:08.426 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.417109+0000", 2024-09-18T10:15:08.426 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0829999999999997e-05 2024-09-18T10:15:08.426 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:08.426 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:08.426 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:08.426 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.427 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.427 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.8s5 36/32 ECSubWrite(tid=127, reqid=client.4632.0:154, at_version=36'9, trim_to=0'0, roll_forward_to=36'8))", 2024-09-18T10:15:08.427 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:51.840375+0000", 2024-09-18T10:15:08.427 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 16.551113384000001, 2024-09-18T10:15:08.427 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10196929, 2024-09-18T10:15:08.427 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:08.427 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:08.427 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:08.427 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.427 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:08.427 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.840375+0000", 2024-09-18T10:15:08.427 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:08.427 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.428 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.428 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:08.428 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.840375+0000", 2024-09-18T10:15:08.428 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:08.428 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.428 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.428 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:08.428 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.840374+0000", 2024-09-18T10:15:08.428 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:08.428 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.428 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.428 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:08.429 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.840521+0000", 2024-09-18T10:15:08.429 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000147641 2024-09-18T10:15:08.429 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.429 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.429 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:08.429 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.840564+0000", 2024-09-18T10:15:08.429 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2895999999999997e-05 2024-09-18T10:15:08.429 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.429 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.429 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:08.429 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.840585+0000", 2024-09-18T10:15:08.429 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0539000000000001e-05 2024-09-18T10:15:08.429 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.429 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.430 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:08.430 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.940817+0000", 2024-09-18T10:15:08.430 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100232166 2024-09-18T10:15:08.430 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.430 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.430 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:08.430 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.940832+0000", 2024-09-18T10:15:08.430 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5616000000000001e-05 2024-09-18T10:15:08.430 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.430 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.430 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:08.430 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.942287+0000", 2024-09-18T10:15:08.430 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001454398 2024-09-18T10:15:08.431 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.431 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.431 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:08.431 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.942345+0000", 2024-09-18T10:15:08.431 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7813999999999999e-05 2024-09-18T10:15:08.431 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:08.431 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:08.431 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:08.431 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.431 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.431 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.as1 37/32 ECSubWrite(tid=33, reqid=client.4632.0:207, at_version=37'9, trim_to=0'0, roll_forward_to=36'7))", 2024-09-18T10:15:08.431 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:53.209503+0000", 2024-09-18T10:15:08.431 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 15.181985489000001, 2024-09-18T10:15:08.431 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.103337639, 2024-09-18T10:15:08.432 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:08.432 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:08.432 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:08.432 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.432 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:08.432 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.209503+0000", 2024-09-18T10:15:08.432 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:08.432 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.432 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.432 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:08.432 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.209503+0000", 2024-09-18T10:15:08.432 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:08.432 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.433 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.433 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:08.433 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.209501+0000", 2024-09-18T10:15:08.433 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:08.433 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.433 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.433 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:08.433 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.209748+0000", 2024-09-18T10:15:08.433 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00024739800000000002 2024-09-18T10:15:08.433 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.433 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.433 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:08.433 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.209784+0000", 2024-09-18T10:15:08.434 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5615999999999999e-05 2024-09-18T10:15:08.434 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.434 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.434 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:08.434 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.209799+0000", 2024-09-18T10:15:08.434 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4796e-05 2024-09-18T10:15:08.434 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.434 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.434 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:08.434 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.309992+0000", 2024-09-18T10:15:08.434 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10019278299999999 2024-09-18T10:15:08.434 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.434 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.435 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:08.435 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.310018+0000", 2024-09-18T10:15:08.435 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6392000000000001e-05 2024-09-18T10:15:08.435 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.435 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.435 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:08.435 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.312776+0000", 2024-09-18T10:15:08.435 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0027584200000000001 2024-09-18T10:15:08.435 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.435 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.435 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:08.435 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.312841+0000", 2024-09-18T10:15:08.435 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4348000000000001e-05 2024-09-18T10:15:08.435 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:08.436 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:08.436 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:08.436 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.436 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.436 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.bs5 41/32 ECSubWrite(tid=58, reqid=osd.4.0:58, at_version=41'14, trim_to=0'0, roll_forward_to=39'12))", 2024-09-18T10:15:08.436 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:56.830099+0000", 2024-09-18T10:15:08.436 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 11.561389192, 2024-09-18T10:15:08.436 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10180473800000001, 2024-09-18T10:15:08.436 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:08.436 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:08.436 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:08.436 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.436 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:08.437 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:56.830099+0000", 2024-09-18T10:15:08.437 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:08.437 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.437 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.437 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:08.437 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:56.830099+0000", 2024-09-18T10:15:08.437 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:08.437 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.437 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.437 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:08.437 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:56.830099+0000", 2024-09-18T10:15:08.437 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:08.437 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.438 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.438 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:08.438 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:56.830180+0000", 2024-09-18T10:15:08.438 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1074000000000003e-05 2024-09-18T10:15:08.438 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.438 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.438 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:08.438 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:56.830220+0000", 2024-09-18T10:15:08.438 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0142000000000003e-05 2024-09-18T10:15:08.438 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.438 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.438 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:08.438 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:56.830238+0000", 2024-09-18T10:15:08.438 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8056000000000001e-05 2024-09-18T10:15:08.439 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.439 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.439 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:08.439 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:56.930521+0000", 2024-09-18T10:15:08.439 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100283026 2024-09-18T10:15:08.439 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.439 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.439 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:08.439 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:56.930599+0000", 2024-09-18T10:15:08.439 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8071000000000005e-05 2024-09-18T10:15:08.440 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.440 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.440 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:08.440 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:56.931860+0000", 2024-09-18T10:15:08.440 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0012610200000000001 2024-09-18T10:15:08.440 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.440 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.440 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:08.440 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:56.931904+0000", 2024-09-18T10:15:08.440 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4058999999999999e-05 2024-09-18T10:15:08.440 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:08.441 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:08.441 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:08.441 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.441 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.441 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.as1 41/32 ECSubWrite(tid=38, reqid=osd.13.0:38, at_version=41'12, trim_to=0'0, roll_forward_to=37'9))", 2024-09-18T10:15:08.441 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:56.934067+0000", 2024-09-18T10:15:08.441 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 11.457422141, 2024-09-18T10:15:08.441 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.102868934, 2024-09-18T10:15:08.441 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:08.441 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:08.441 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:08.441 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.442 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:08.442 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:56.934067+0000", 2024-09-18T10:15:08.442 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:08.442 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.442 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.442 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:08.442 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:56.934067+0000", 2024-09-18T10:15:08.442 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:08.442 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.442 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.442 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:08.442 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:56.934065+0000", 2024-09-18T10:15:08.442 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:08.443 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.443 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.443 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:08.443 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:56.934108+0000", 2024-09-18T10:15:08.443 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3170999999999997e-05 2024-09-18T10:15:08.443 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.443 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.443 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:08.443 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:56.934137+0000", 2024-09-18T10:15:08.443 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9119e-05 2024-09-18T10:15:08.443 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.443 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.443 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:08.443 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:56.934150+0000", 2024-09-18T10:15:08.444 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2824e-05 2024-09-18T10:15:08.444 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.444 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.444 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:08.444 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.034355+0000", 2024-09-18T10:15:08.444 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10020481000000001 2024-09-18T10:15:08.444 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.444 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.444 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:08.444 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.034398+0000", 2024-09-18T10:15:08.444 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3642e-05 2024-09-18T10:15:08.444 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.444 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.444 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:08.445 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.036893+0000", 2024-09-18T10:15:08.445 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0024942229999999998 2024-09-18T10:15:08.445 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.445 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.445 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:08.445 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.036935+0000", 2024-09-18T10:15:08.445 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2821000000000002e-05 2024-09-18T10:15:08.445 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:08.445 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:08.445 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:08.445 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.445 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.445 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.as1 41/32 ECSubWrite(tid=42, reqid=osd.13.0:38, at_version=0'0, trim_to=0'0, roll_forward_to=41'12))", 2024-09-18T10:15:08.446 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:57.232992+0000", 2024-09-18T10:15:08.446 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 11.158496550000001, 2024-09-18T10:15:08.446 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10174161499999999, 2024-09-18T10:15:08.446 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:08.446 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:08.446 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:08.446 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.446 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:08.446 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.232992+0000", 2024-09-18T10:15:08.446 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:08.446 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.446 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.446 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:08.447 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.232992+0000", 2024-09-18T10:15:08.447 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:08.447 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.447 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.447 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:08.447 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.232991+0000", 2024-09-18T10:15:08.447 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:08.447 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.447 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.447 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:08.447 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.233016+0000", 2024-09-18T10:15:08.447 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5456000000000002e-05 2024-09-18T10:15:08.447 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.447 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.448 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:08.448 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.233059+0000", 2024-09-18T10:15:08.448 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3458999999999998e-05 2024-09-18T10:15:08.448 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.448 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.448 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:08.448 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.233079+0000", 2024-09-18T10:15:08.448 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9259e-05 2024-09-18T10:15:08.448 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.448 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.448 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:08.448 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.333278+0000", 2024-09-18T10:15:08.448 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100199184 2024-09-18T10:15:08.448 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.449 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.449 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:08.449 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.333294+0000", 2024-09-18T10:15:08.449 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6252999999999998e-05 2024-09-18T10:15:08.449 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.449 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.449 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:08.449 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.334686+0000", 2024-09-18T10:15:08.449 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0013916110000000001 2024-09-18T10:15:08.449 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.449 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.449 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:08.449 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.334734+0000", 2024-09-18T10:15:08.449 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7936000000000003e-05 2024-09-18T10:15:08.450 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:08.450 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:08.450 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:08.450 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.450 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.450 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.2s3 41/40 ECSubWrite(tid=23, reqid=client.4632.0:265, at_version=41'11, trim_to=0'0, roll_forward_to=0'0))", 2024-09-18T10:15:08.450 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:57.236544+0000", 2024-09-18T10:15:08.450 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 11.15494457, 2024-09-18T10:15:08.450 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20209979, 2024-09-18T10:15:08.450 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:08.450 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:08.450 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:08.450 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.451 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:08.451 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.236544+0000", 2024-09-18T10:15:08.451 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:08.451 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.451 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.451 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:08.451 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.236544+0000", 2024-09-18T10:15:08.451 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:08.451 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.451 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.451 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:08.451 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.236543+0000", 2024-09-18T10:15:08.451 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:08.451 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.452 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.452 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:08.452 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.236820+0000", 2024-09-18T10:15:08.452 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000277068 2024-09-18T10:15:08.452 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.452 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.452 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:08.452 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.236876+0000", 2024-09-18T10:15:08.452 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6555000000000003e-05 2024-09-18T10:15:08.452 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.452 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.452 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:08.452 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.336982+0000", 2024-09-18T10:15:08.452 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100105679 2024-09-18T10:15:08.453 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.453 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.453 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:08.453 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.337122+0000", 2024-09-18T10:15:08.453 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000139777 2024-09-18T10:15:08.453 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.453 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.453 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:08.453 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.337135+0000", 2024-09-18T10:15:08.453 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3559e-05 2024-09-18T10:15:08.453 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.453 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.453 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:08.454 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.438600+0000", 2024-09-18T10:15:08.454 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.101464572 2024-09-18T10:15:08.454 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.454 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.454 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:08.454 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.438644+0000", 2024-09-18T10:15:08.454 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4085000000000002e-05 2024-09-18T10:15:08.454 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:08.454 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:08.454 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:08.454 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.454 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.454 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.2s3 41/40 ECSubWrite(tid=24, reqid=client.4632.0:266, at_version=41'12, trim_to=0'0, roll_forward_to=0'0))", 2024-09-18T10:15:08.455 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:57.337029+0000", 2024-09-18T10:15:08.455 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 11.054459723000001, 2024-09-18T10:15:08.455 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10168448099999999, 2024-09-18T10:15:08.455 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:08.455 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:08.455 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:08.455 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.455 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:08.455 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.337029+0000", 2024-09-18T10:15:08.455 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:08.455 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.455 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.455 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:08.455 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.337029+0000", 2024-09-18T10:15:08.456 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:08.456 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.456 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.456 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:08.456 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.337028+0000", 2024-09-18T10:15:08.456 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:08.456 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.456 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.456 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:08.456 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.337067+0000", 2024-09-18T10:15:08.456 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8991e-05 2024-09-18T10:15:08.456 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.456 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.457 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:08.457 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.337090+0000", 2024-09-18T10:15:08.457 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3332e-05 2024-09-18T10:15:08.457 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.457 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.457 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:08.457 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.337101+0000", 2024-09-18T10:15:08.457 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0872e-05 2024-09-18T10:15:08.457 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.457 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.457 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:08.457 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.337889+0000", 2024-09-18T10:15:08.457 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00078866099999999996 2024-09-18T10:15:08.458 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.458 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.458 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:08.458 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.338008+0000", 2024-09-18T10:15:08.458 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000118313 2024-09-18T10:15:08.458 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.458 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.458 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:08.458 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.438675+0000", 2024-09-18T10:15:08.458 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10066691 2024-09-18T10:15:08.458 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.458 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.458 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:08.458 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.438713+0000", 2024-09-18T10:15:08.459 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8739000000000003e-05 2024-09-18T10:15:08.459 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:08.459 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:08.459 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:08.459 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.459 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.459 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.2s3 41/40 ECSubWrite(tid=25, reqid=client.4632.0:266, at_version=0'0, trim_to=0'0, roll_forward_to=41'12))", 2024-09-18T10:15:08.459 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:57.543345+0000", 2024-09-18T10:15:08.459 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.84814416, 2024-09-18T10:15:08.459 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.18858536100000001, 2024-09-18T10:15:08.459 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:08.459 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:08.459 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:08.459 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.460 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:08.460 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.543345+0000", 2024-09-18T10:15:08.460 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:08.460 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.460 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.460 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:08.460 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.543345+0000", 2024-09-18T10:15:08.460 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:08.460 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.460 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.460 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:08.460 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.543343+0000", 2024-09-18T10:15:08.460 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:08.461 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.461 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.461 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:08.461 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.543385+0000", 2024-09-18T10:15:08.461 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2487999999999997e-05 2024-09-18T10:15:08.461 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.461 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.461 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:08.461 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.543428+0000", 2024-09-18T10:15:08.461 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2315999999999999e-05 2024-09-18T10:15:08.461 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.461 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.461 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:08.462 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.543446+0000", 2024-09-18T10:15:08.462 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8189000000000002e-05 2024-09-18T10:15:08.462 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.462 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.462 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:08.462 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.730560+0000", 2024-09-18T10:15:08.462 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.187114473 2024-09-18T10:15:08.462 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.462 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.462 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:08.462 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.730585+0000", 2024-09-18T10:15:08.462 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4652e-05 2024-09-18T10:15:08.462 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.463 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.463 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:08.463 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.731859+0000", 2024-09-18T10:15:08.463 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001274425 2024-09-18T10:15:08.463 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.463 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.463 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:08.463 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.731930+0000", 2024-09-18T10:15:08.463 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0474000000000003e-05 2024-09-18T10:15:08.463 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:08.463 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:08.463 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:08.463 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.463 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.464 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpRead(3.2s3 41/40 ECSubRead(tid=26, to_read={3:46b2871c:::smithi18925896-34:head=0,4096,0}, subchunks={3:46b2871c:::smithi18925896-34:head=[0,1]}, attrs_to_read=))", 2024-09-18T10:15:08.464 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:57.543972+0000", 2024-09-18T10:15:08.464 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.847516534, 2024-09-18T10:15:08.464 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.18721849300000001, 2024-09-18T10:15:08.464 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:08.464 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:08.464 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:08.464 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.464 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:08.464 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.543972+0000", 2024-09-18T10:15:08.464 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:08.464 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.464 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.465 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:08.465 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.543972+0000", 2024-09-18T10:15:08.465 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:08.465 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.465 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.465 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:08.465 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.543970+0000", 2024-09-18T10:15:08.465 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:08.465 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.465 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.465 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:08.465 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.543996+0000", 2024-09-18T10:15:08.465 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5069e-05 2024-09-18T10:15:08.466 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.466 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.466 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:08.466 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.544056+0000", 2024-09-18T10:15:08.466 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0355000000000001e-05 2024-09-18T10:15:08.466 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.466 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.466 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:08.466 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.544079+0000", 2024-09-18T10:15:08.466 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3396000000000001e-05 2024-09-18T10:15:08.466 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.466 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.466 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:08.466 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.730946+0000", 2024-09-18T10:15:08.467 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.186866644 2024-09-18T10:15:08.467 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.467 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.467 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:08.467 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.731191+0000", 2024-09-18T10:15:08.467 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00024467899999999998 2024-09-18T10:15:08.467 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:08.467 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:08.467 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:08.467 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.467 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.467 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.8s5 41/40 ECSubWrite(tid=132, reqid=osd.6.0:132, at_version=41'13, trim_to=0'0, roll_forward_to=0'0))", 2024-09-18T10:15:08.467 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:57.548222+0000", 2024-09-18T10:15:08.468 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.843267063999999, 2024-09-18T10:15:08.468 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.102715687, 2024-09-18T10:15:08.468 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:08.468 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:08.468 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:08.468 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.468 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:08.468 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.548222+0000", 2024-09-18T10:15:08.468 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:08.468 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.468 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.468 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:08.469 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.548222+0000", 2024-09-18T10:15:08.469 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:08.469 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.469 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.469 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:08.469 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.548220+0000", 2024-09-18T10:15:08.469 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:08.469 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.469 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.469 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:08.469 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.548274+0000", 2024-09-18T10:15:08.469 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4721e-05 2024-09-18T10:15:08.469 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.470 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.470 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:08.470 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.548307+0000", 2024-09-18T10:15:08.470 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2929000000000003e-05 2024-09-18T10:15:08.470 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.470 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.470 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:08.470 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.548323+0000", 2024-09-18T10:15:08.470 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5534999999999999e-05 2024-09-18T10:15:08.470 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.470 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.470 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:08.470 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.648511+0000", 2024-09-18T10:15:08.470 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10018864700000001 2024-09-18T10:15:08.471 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.471 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.471 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:08.471 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.648541+0000", 2024-09-18T10:15:08.471 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9561999999999999e-05 2024-09-18T10:15:08.471 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.471 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.471 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:08.471 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.650891+0000", 2024-09-18T10:15:08.471 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002349842 2024-09-18T10:15:08.471 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.471 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.471 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:08.472 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.650937+0000", 2024-09-18T10:15:08.472 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6489000000000002e-05 2024-09-18T10:15:08.472 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:08.472 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:08.472 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:08.472 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.472 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.472 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.8s5 42/40 ECSubWrite(tid=133, reqid=osd.6.0:132, at_version=0'0, trim_to=0'0, roll_forward_to=41'13))", 2024-09-18T10:15:08.472 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:58.049905+0000", 2024-09-18T10:15:08.472 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.341583768, 2024-09-18T10:15:08.472 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.101683442, 2024-09-18T10:15:08.472 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:08.473 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:08.473 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:08.473 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.473 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:08.473 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:58.049905+0000", 2024-09-18T10:15:08.473 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:08.473 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.473 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.473 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:08.473 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:58.049905+0000", 2024-09-18T10:15:08.473 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:08.473 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.473 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.473 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:08.474 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:58.049904+0000", 2024-09-18T10:15:08.474 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:08.474 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.474 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.474 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:08.474 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:58.049923+0000", 2024-09-18T10:15:08.474 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8836999999999999e-05 2024-09-18T10:15:08.474 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.474 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.474 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:08.474 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:58.049968+0000", 2024-09-18T10:15:08.474 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4341000000000001e-05 2024-09-18T10:15:08.474 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.475 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.475 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:08.475 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:58.049981+0000", 2024-09-18T10:15:08.475 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.381e-05 2024-09-18T10:15:08.475 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.475 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.475 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:08.475 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:58.150246+0000", 2024-09-18T10:15:08.475 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10026481800000001 2024-09-18T10:15:08.475 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.475 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.475 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:08.475 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:58.150262+0000", 2024-09-18T10:15:08.476 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6197000000000001e-05 2024-09-18T10:15:08.476 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.476 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.476 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:08.476 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:58.151539+0000", 2024-09-18T10:15:08.476 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001276505 2024-09-18T10:15:08.476 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.476 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.476 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:08.476 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:58.151588+0000", 2024-09-18T10:15:08.476 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9400999999999997e-05 2024-09-18T10:15:08.476 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:08.476 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:08.477 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:08.477 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.477 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.477 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.ds1 47/32 ECSubWrite(tid=55, reqid=client.4632.0:332, at_version=47'8, trim_to=0'0, roll_forward_to=41'6))", 2024-09-18T10:15:08.477 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:02.796404+0000", 2024-09-18T10:15:08.477 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.5950849380000003, 2024-09-18T10:15:08.477 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.103838836, 2024-09-18T10:15:08.477 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:08.477 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:08.477 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:08.478 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.478 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:08.478 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.796404+0000", 2024-09-18T10:15:08.478 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:08.478 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.478 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.478 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:08.478 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.796404+0000", 2024-09-18T10:15:08.478 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:08.479 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.479 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.479 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:08.479 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.796403+0000", 2024-09-18T10:15:08.479 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:15:08.479 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.479 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.479 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:08.479 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.796757+0000", 2024-09-18T10:15:08.479 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00035467799999999998 2024-09-18T10:15:08.479 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.479 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.479 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:08.479 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.796832+0000", 2024-09-18T10:15:08.480 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4661999999999998e-05 2024-09-18T10:15:08.480 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.480 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.480 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:08.480 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.896954+0000", 2024-09-18T10:15:08.480 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10012193799999999 2024-09-18T10:15:08.480 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.480 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.480 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:08.480 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.897127+0000", 2024-09-18T10:15:08.480 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017293500000000001 2024-09-18T10:15:08.480 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.480 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.480 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:08.481 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.897149+0000", 2024-09-18T10:15:08.481 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2262000000000001e-05 2024-09-18T10:15:08.481 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.481 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.481 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:08.481 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.900201+0000", 2024-09-18T10:15:08.481 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0030515379999999999 2024-09-18T10:15:08.481 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.481 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.481 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:08.481 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.900243+0000", 2024-09-18T10:15:08.481 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1758999999999998e-05 2024-09-18T10:15:08.482 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:08.482 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:08.482 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:08.482 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.482 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.482 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.18s5 51/40 ECSubWrite(tid=156, reqid=client.4632.0:403, at_version=51'24, trim_to=0'0, roll_forward_to=50'21))", 2024-09-18T10:15:08.482 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:06.699348+0000", 2024-09-18T10:15:08.482 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.692140306, 2024-09-18T10:15:08.482 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.102561949, 2024-09-18T10:15:08.482 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:08.482 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:08.482 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:08.482 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.482 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:08.483 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:06.699348+0000", 2024-09-18T10:15:08.483 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:08.483 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.483 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.483 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:08.483 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:06.699348+0000", 2024-09-18T10:15:08.483 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:08.483 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.483 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.483 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:08.483 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:06.699347+0000", 2024-09-18T10:15:08.483 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:08.483 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.484 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.484 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:08.484 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:06.699672+0000", 2024-09-18T10:15:08.484 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000325297 2024-09-18T10:15:08.484 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.484 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.484 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:08.484 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:06.699699+0000", 2024-09-18T10:15:08.484 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7259000000000001e-05 2024-09-18T10:15:08.484 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.484 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.484 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:08.484 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:06.699720+0000", 2024-09-18T10:15:08.485 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0403000000000001e-05 2024-09-18T10:15:08.485 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.485 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.485 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:08.485 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:06.799900+0000", 2024-09-18T10:15:08.485 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10018077 2024-09-18T10:15:08.486 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.486 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.486 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:08.486 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:06.799924+0000", 2024-09-18T10:15:08.486 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3615999999999999e-05 2024-09-18T10:15:08.486 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.486 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.486 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:08.486 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:06.801851+0000", 2024-09-18T10:15:08.486 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0019266159999999999 2024-09-18T10:15:08.486 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.486 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.487 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:08.487 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:06.801910+0000", 2024-09-18T10:15:08.487 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9781000000000002e-05 2024-09-18T10:15:08.487 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:08.487 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:08.487 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:08.487 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:08.487 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:08.487 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-18T10:15:08.489 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T10:15:08.489 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-18T10:15:08.485+0000 7f1981873700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22132) UID: 0 2024-09-18T10:15:08.547 INFO:tasks.rados.rados.0.smithi189.stdout:160: done (2 left) 2024-09-18T10:15:08.547 INFO:tasks.rados.rados.0.smithi189.stdout:161: done (1 left) 2024-09-18T10:15:08.547 INFO:tasks.rados.rados.0.smithi189.stdout:162: done (0 left) 2024-09-18T10:15:08.547 INFO:tasks.rados.rados.0.smithi189.stdout:163: copy_from oid 48 from oid 42 current snap is 10 2024-09-18T10:15:08.547 INFO:tasks.rados.rados.0.smithi189.stdout:164: delete oid 29 current snap is 10 2024-09-18T10:15:08.590 INFO:tasks.ceph.osd.6.smithi154.stderr:2024-09-18T10:15:08.583+0000 7f0abc9e1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22089) UID: 0 2024-09-18T10:15:08.593 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-18T10:15:08.593 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-18T10:15:08.593 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-09-18T10:15:08.593 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-18T10:15:08.603 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T10:15:08.690 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-09-18T10:15:08.683+0000 7fdfa8698700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22087) UID: 0 2024-09-18T10:15:08.698 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-18T10:15:08.698 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-18T10:15:08.699 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-18T10:15:08.699 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-18T10:15:08.699 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-18T10:15:08.708 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T10:15:08.791 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-18T10:15:08.789+0000 7f00046a8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 22136) UID: 0 2024-09-18T10:15:08.809 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-18T10:15:08.809 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-18T10:15:08.809 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-18T10:15:08.809 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-18T10:15:08.809 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.809 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:150 3.cs0 3:3972e5c3:::smithi18925896-50:head [append 0~458752,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:08.809 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:51.125225+0000", 2024-09-18T10:15:08.809 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 17.679902013, 2024-09-18T10:15:08.809 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.40871674099999999, 2024-09-18T10:15:08.809 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:08.809 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:08.810 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:08.810 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:08.810 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:08.810 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 150 2024-09-18T10:15:08.810 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.810 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:08.810 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.810 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:08.810 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.125225+0000", 2024-09-18T10:15:08.810 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:08.810 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.810 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.811 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:08.811 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.125225+0000", 2024-09-18T10:15:08.811 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:08.811 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.811 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.811 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:08.811 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.125222+0000", 2024-09-18T10:15:08.811 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999967 2024-09-18T10:15:08.811 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.811 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.811 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:08.811 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.126038+0000", 2024-09-18T10:15:08.811 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00081693399999999995 2024-09-18T10:15:08.812 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.812 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.812 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:08.812 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.126072+0000", 2024-09-18T10:15:08.812 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4038999999999997e-05 2024-09-18T10:15:08.812 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.812 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.812 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:08.812 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.126084+0000", 2024-09-18T10:15:08.812 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1908e-05 2024-09-18T10:15:08.812 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.812 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.812 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:08.813 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.126184+0000", 2024-09-18T10:15:08.813 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9868e-05 2024-09-18T10:15:08.813 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.813 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.813 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:08.813 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.126439+0000", 2024-09-18T10:15:08.813 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00025457799999999999 2024-09-18T10:15:08.813 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.813 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.813 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:08.813 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.128215+0000", 2024-09-18T10:15:08.813 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0017764499999999999 2024-09-18T10:15:08.813 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.814 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.814 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:08.814 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.232589+0000", 2024-09-18T10:15:08.814 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.104374015 2024-09-18T10:15:08.814 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.814 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.814 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:15:08.814 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.533834+0000", 2024-09-18T10:15:08.814 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30124498 2024-09-18T10:15:08.814 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.814 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.814 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:08.814 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.533941+0000", 2024-09-18T10:15:08.814 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010718299999999999 2024-09-18T10:15:08.815 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:08.815 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:08.815 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:08.815 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.815 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.815 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:151 3.cs0 3:3972e5c3:::smithi18925896-50:head [append 0~376832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:08.815 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:51.127704+0000", 2024-09-18T10:15:08.815 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 17.677422908, 2024-09-18T10:15:08.815 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.40663223799999998, 2024-09-18T10:15:08.815 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:08.815 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:08.815 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:08.815 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:08.816 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:08.816 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 151 2024-09-18T10:15:08.816 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.816 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:08.816 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.816 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:08.816 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.127704+0000", 2024-09-18T10:15:08.816 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:08.816 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.816 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.816 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:08.816 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.127704+0000", 2024-09-18T10:15:08.816 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:08.817 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.817 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.817 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:08.817 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.127702+0000", 2024-09-18T10:15:08.817 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:08.817 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.817 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.817 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:08.817 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.128054+0000", 2024-09-18T10:15:08.817 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00035175199999999999 2024-09-18T10:15:08.817 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.817 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.817 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:08.817 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.128091+0000", 2024-09-18T10:15:08.818 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7035e-05 2024-09-18T10:15:08.818 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.818 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.818 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:08.818 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.128105+0000", 2024-09-18T10:15:08.818 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.428e-05 2024-09-18T10:15:08.818 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.818 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.818 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:08.818 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.129336+0000", 2024-09-18T10:15:08.818 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001231326 2024-09-18T10:15:08.818 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.818 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.819 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:08.819 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.129489+0000", 2024-09-18T10:15:08.819 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015266100000000001 2024-09-18T10:15:08.819 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.819 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.819 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:08.819 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.130757+0000", 2024-09-18T10:15:08.819 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0012684409999999999 2024-09-18T10:15:08.819 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.819 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.819 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:08.819 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.332853+0000", 2024-09-18T10:15:08.819 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20209580599999999 2024-09-18T10:15:08.819 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.819 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.820 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:15:08.820 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.534218+0000", 2024-09-18T10:15:08.820 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.2013653 2024-09-18T10:15:08.820 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.820 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.820 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:08.820 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.534336+0000", 2024-09-18T10:15:08.820 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011760900000000001 2024-09-18T10:15:08.820 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:08.820 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:08.820 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:08.820 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.820 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.821 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:152 3.cs0 3:3972e5c3:::smithi18925896-50:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:08.821 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:51.128123+0000", 2024-09-18T10:15:08.821 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 17.677003340999999, 2024-09-18T10:15:08.821 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.80287000200000003, 2024-09-18T10:15:08.821 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:08.821 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:08.821 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:08.821 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:08.821 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:08.821 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 152 2024-09-18T10:15:08.821 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.821 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:08.821 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.822 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:08.822 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.128123+0000", 2024-09-18T10:15:08.822 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:08.822 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.822 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.822 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:08.822 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.128123+0000", 2024-09-18T10:15:08.822 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:08.822 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.822 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.822 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:08.822 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.128122+0000", 2024-09-18T10:15:08.822 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:08.822 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.823 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.823 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:08.823 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.128129+0000", 2024-09-18T10:15:08.823 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6730000000000003e-06 2024-09-18T10:15:08.823 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.823 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.823 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:08.823 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.128156+0000", 2024-09-18T10:15:08.823 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7489000000000001e-05 2024-09-18T10:15:08.823 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.823 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.823 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:08.823 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.128173+0000", 2024-09-18T10:15:08.824 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7102999999999999e-05 2024-09-18T10:15:08.824 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.824 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.824 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:08.824 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.535139+0000", 2024-09-18T10:15:08.824 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.40696592599999998 2024-09-18T10:15:08.824 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.824 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.824 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:08.824 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.535380+0000", 2024-09-18T10:15:08.824 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00024023600000000001 2024-09-18T10:15:08.824 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.825 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.825 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:08.825 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.536784+0000", 2024-09-18T10:15:08.825 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0014046950000000001 2024-09-18T10:15:08.825 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.825 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.825 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:08.825 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.537964+0000", 2024-09-18T10:15:08.825 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0011793050000000001 2024-09-18T10:15:08.825 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.825 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.825 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:15:08.825 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.930841+0000", 2024-09-18T10:15:08.826 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.39287700599999997 2024-09-18T10:15:08.826 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.826 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.826 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:08.826 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.930993+0000", 2024-09-18T10:15:08.826 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015280799999999999 2024-09-18T10:15:08.826 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:08.826 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:08.826 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:08.826 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.826 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.826 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:153 3.cs0 3:3972e5c3:::smithi18925896-50:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:08.826 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:51.128181+0000", 2024-09-18T10:15:08.827 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 17.676945446000001, 2024-09-18T10:15:08.827 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2060332149999999, 2024-09-18T10:15:08.827 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:08.827 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-18T10:15:08.827 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:08.827 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:08.827 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:08.827 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 153 2024-09-18T10:15:08.827 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.827 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:08.827 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.827 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:08.827 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.128181+0000", 2024-09-18T10:15:08.827 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:08.828 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.828 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.828 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:08.828 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.128181+0000", 2024-09-18T10:15:08.828 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:08.828 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.828 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.828 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:08.828 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.128180+0000", 2024-09-18T10:15:08.828 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:15:08.828 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.828 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.828 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:08.829 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.128186+0000", 2024-09-18T10:15:08.829 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.31e-06 2024-09-18T10:15:08.829 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.829 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.829 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:08.829 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.128195+0000", 2024-09-18T10:15:08.829 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5540000000000003e-06 2024-09-18T10:15:08.829 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.829 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.829 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:08.829 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.128201+0000", 2024-09-18T10:15:08.829 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2129999999999996e-06 2024-09-18T10:15:08.830 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.830 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.830 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:08.830 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.537382+0000", 2024-09-18T10:15:08.830 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.40918055399999997 2024-09-18T10:15:08.830 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.830 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.830 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2024-09-18T10:15:08.830 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.537571+0000", 2024-09-18T10:15:08.830 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00018903299999999999 2024-09-18T10:15:08.830 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.830 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.830 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:08.830 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.931789+0000", 2024-09-18T10:15:08.831 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.39421780499999998 2024-09-18T10:15:08.831 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.831 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.831 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:08.831 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.932023+0000", 2024-09-18T10:15:08.831 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000233868 2024-09-18T10:15:08.831 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.831 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.832 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:08.832 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.334215+0000", 2024-09-18T10:15:08.832 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.40219177499999997 2024-09-18T10:15:08.832 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:08.832 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:08.832 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:08.832 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.832 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.832 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWriteReply(3.cs0 36/32 ECSubWriteReply(tid=20, last_complete=36'9, committed=1, applied=1))", 2024-09-18T10:15:08.832 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:51.213977+0000", 2024-09-18T10:15:08.832 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 17.591150167999999, 2024-09-18T10:15:08.832 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.21934668099999999, 2024-09-18T10:15:08.833 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:08.833 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:08.833 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:08.833 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.833 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:08.833 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.213977+0000", 2024-09-18T10:15:08.833 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:08.833 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.833 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.833 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:08.833 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.213977+0000", 2024-09-18T10:15:08.833 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:08.834 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.834 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.834 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:08.834 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.213975+0000", 2024-09-18T10:15:08.834 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:08.834 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.834 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.834 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:08.834 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.213988+0000", 2024-09-18T10:15:08.834 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3579e-05 2024-09-18T10:15:08.834 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.834 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.834 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:08.835 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.214020+0000", 2024-09-18T10:15:08.835 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2005999999999998e-05 2024-09-18T10:15:08.835 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.835 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.835 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:08.835 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.214032+0000", 2024-09-18T10:15:08.835 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1960000000000001e-05 2024-09-18T10:15:08.835 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.835 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.835 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:08.835 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.433299+0000", 2024-09-18T10:15:08.835 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.21926704299999999 2024-09-18T10:15:08.835 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.835 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.836 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:08.836 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.433323+0000", 2024-09-18T10:15:08.836 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3796000000000001e-05 2024-09-18T10:15:08.836 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:08.836 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:08.836 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:08.836 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.836 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.836 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWriteReply(3.cs0 36/32 ECSubWriteReply(tid=21, last_complete=36'10, committed=1, applied=1))", 2024-09-18T10:15:08.836 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:51.214781+0000", 2024-09-18T10:15:08.836 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 17.590345719999998, 2024-09-18T10:15:08.836 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.218783754, 2024-09-18T10:15:08.836 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:08.837 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:08.837 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:08.837 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.837 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:08.837 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.214781+0000", 2024-09-18T10:15:08.837 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:08.837 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.837 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.837 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:08.837 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.214781+0000", 2024-09-18T10:15:08.837 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:08.837 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.837 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.838 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:08.838 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.214779+0000", 2024-09-18T10:15:08.838 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999976 2024-09-18T10:15:08.838 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.838 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.838 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:08.838 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.214792+0000", 2024-09-18T10:15:08.838 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.322e-05 2024-09-18T10:15:08.838 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.838 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.838 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:08.838 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.214821+0000", 2024-09-18T10:15:08.838 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8812000000000001e-05 2024-09-18T10:15:08.838 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.839 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.839 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:08.839 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.214835+0000", 2024-09-18T10:15:08.839 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4511999999999999e-05 2024-09-18T10:15:08.839 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.839 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.839 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:08.839 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.433549+0000", 2024-09-18T10:15:08.839 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.218713834 2024-09-18T10:15:08.839 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.839 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.839 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:08.839 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.433565+0000", 2024-09-18T10:15:08.840 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5619000000000001e-05 2024-09-18T10:15:08.840 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:08.840 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:08.840 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:08.840 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.840 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.840 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:161 3.cs0 3:34d26c88:::smithi18925896-36:head [read 0~589824,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:08.840 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:51.532533+0000", 2024-09-18T10:15:08.840 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 17.272593788000002, 2024-09-18T10:15:08.840 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30885369800000001, 2024-09-18T10:15:08.840 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:08.840 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-18T10:15:08.840 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:08.840 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:08.841 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:08.841 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 161 2024-09-18T10:15:08.841 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.841 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:08.841 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.841 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:08.841 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.532533+0000", 2024-09-18T10:15:08.841 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:08.841 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.841 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.841 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:08.841 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.532533+0000", 2024-09-18T10:15:08.841 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:08.842 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.842 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.842 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:08.842 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.532530+0000", 2024-09-18T10:15:08.842 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999971 2024-09-18T10:15:08.842 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.842 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.842 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:08.842 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.532544+0000", 2024-09-18T10:15:08.842 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4219999999999999e-05 2024-09-18T10:15:08.842 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.842 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.843 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:08.843 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.532580+0000", 2024-09-18T10:15:08.843 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5315000000000003e-05 2024-09-18T10:15:08.843 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.843 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.843 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:08.843 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.532594+0000", 2024-09-18T10:15:08.843 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4216000000000001e-05 2024-09-18T10:15:08.843 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.843 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.843 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:08.843 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.538016+0000", 2024-09-18T10:15:08.843 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0054220559999999998 2024-09-18T10:15:08.844 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.844 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.844 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:08.844 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.538264+0000", 2024-09-18T10:15:08.844 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000248471 2024-09-18T10:15:08.844 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.844 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.844 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:08.844 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.841387+0000", 2024-09-18T10:15:08.844 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30312219699999998 2024-09-18T10:15:08.844 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:08.844 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:08.844 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:08.845 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.845 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.845 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:162 3.cs0 3:34d26c88:::smithi18925896-36:head [read 0~589824,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:08.845 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:51.532607+0000", 2024-09-18T10:15:08.845 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 17.272520104000002, 2024-09-18T10:15:08.845 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.400964512, 2024-09-18T10:15:08.845 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:08.845 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-18T10:15:08.845 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:08.845 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:08.845 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:08.845 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 162 2024-09-18T10:15:08.846 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.846 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:08.846 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.846 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:08.846 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.532607+0000", 2024-09-18T10:15:08.846 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:08.846 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.846 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.846 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:08.846 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.532607+0000", 2024-09-18T10:15:08.846 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:08.846 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.846 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.846 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:08.847 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.532606+0000", 2024-09-18T10:15:08.847 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:15:08.847 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.847 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.847 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:08.847 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.532612+0000", 2024-09-18T10:15:08.847 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0599999999999996e-06 2024-09-18T10:15:08.847 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.847 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.847 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:08.847 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.532621+0000", 2024-09-18T10:15:08.847 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6190000000000006e-06 2024-09-18T10:15:08.847 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.847 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.848 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:08.848 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.532627+0000", 2024-09-18T10:15:08.848 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3469999999999998e-06 2024-09-18T10:15:08.848 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.848 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.848 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:08.848 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.538755+0000", 2024-09-18T10:15:08.848 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0061281729999999998 2024-09-18T10:15:08.848 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.848 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.848 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:08.848 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.538992+0000", 2024-09-18T10:15:08.849 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00023700500000000001 2024-09-18T10:15:08.849 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.849 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.849 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:08.849 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.933571+0000", 2024-09-18T10:15:08.849 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.39457930800000002 2024-09-18T10:15:08.849 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:08.849 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:08.849 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:08.849 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.849 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.850 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:163 3.cs0 3:34d26c88:::smithi18925896-36:head [read 0~589824,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:08.850 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:51.532633+0000", 2024-09-18T10:15:08.850 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 17.272493859000001, 2024-09-18T10:15:08.850 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.70024756899999996, 2024-09-18T10:15:08.850 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:08.850 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-18T10:15:08.850 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:08.850 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:08.850 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:08.850 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 163 2024-09-18T10:15:08.850 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.850 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:08.850 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.850 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:08.851 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.532633+0000", 2024-09-18T10:15:08.851 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:08.851 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.851 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.851 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:08.851 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.532633+0000", 2024-09-18T10:15:08.851 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:08.851 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.851 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.851 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:08.851 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.532632+0000", 2024-09-18T10:15:08.851 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:15:08.851 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.852 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.852 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:08.852 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.532636+0000", 2024-09-18T10:15:08.852 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3529999999999997e-06 2024-09-18T10:15:08.852 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.852 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.852 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:08.852 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.532645+0000", 2024-09-18T10:15:08.852 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7099999999999996e-06 2024-09-18T10:15:08.852 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.852 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.852 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:08.852 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.532650+0000", 2024-09-18T10:15:08.852 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3029999999999999e-06 2024-09-18T10:15:08.853 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.853 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.853 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:08.853 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.930003+0000", 2024-09-18T10:15:08.853 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.397352333 2024-09-18T10:15:08.853 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.853 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.853 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:08.853 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.930174+0000", 2024-09-18T10:15:08.853 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017110000000000001 2024-09-18T10:15:08.853 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.853 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.853 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:08.853 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.232880+0000", 2024-09-18T10:15:08.854 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30270652300000001 2024-09-18T10:15:08.854 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:08.854 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:08.854 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:08.854 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.854 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.854 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpReadReply(3.cs0 36/32 ECSubReadReply(tid=25, attrs_read=0))", 2024-09-18T10:15:08.854 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:51.540164+0000", 2024-09-18T10:15:08.854 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 17.264962371999999, 2024-09-18T10:15:08.854 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.29990201900000002, 2024-09-18T10:15:08.854 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:08.854 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:08.854 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:08.854 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.854 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:08.855 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.540164+0000", 2024-09-18T10:15:08.855 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:08.855 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.855 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.855 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:08.855 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.540164+0000", 2024-09-18T10:15:08.855 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:08.855 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.855 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.855 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:08.855 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.540164+0000", 2024-09-18T10:15:08.855 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:15:08.855 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.855 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.856 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:08.856 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.540240+0000", 2024-09-18T10:15:08.856 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6821000000000002e-05 2024-09-18T10:15:08.856 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.856 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.856 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:08.856 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.540256+0000", 2024-09-18T10:15:08.856 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5906e-05 2024-09-18T10:15:08.856 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.856 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.857 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:08.857 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.540263+0000", 2024-09-18T10:15:08.857 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2829999999999998e-06 2024-09-18T10:15:08.857 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.857 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.857 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:08.857 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.839993+0000", 2024-09-18T10:15:08.857 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.29973060800000001 2024-09-18T10:15:08.857 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.857 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.857 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:08.857 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.840066+0000", 2024-09-18T10:15:08.858 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3233999999999997e-05 2024-09-18T10:15:08.858 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:08.858 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:08.858 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:08.858 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.858 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.858 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:181 3.cs0 3:3a4fded2:::smithi18925896-23:head [append 0~524288] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:08.858 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:52.691563+0000", 2024-09-18T10:15:08.858 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 16.113563628000001, 2024-09-18T10:15:08.858 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.418704519, 2024-09-18T10:15:08.858 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:08.858 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:08.859 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:08.859 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:08.859 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:08.859 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 181 2024-09-18T10:15:08.859 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.859 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:08.859 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.859 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:08.859 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.691563+0000", 2024-09-18T10:15:08.859 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:08.859 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.859 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.859 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:08.860 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.691563+0000", 2024-09-18T10:15:08.860 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:08.860 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.860 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.860 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:08.860 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.691560+0000", 2024-09-18T10:15:08.860 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999971 2024-09-18T10:15:08.860 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.860 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.860 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:08.860 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.692740+0000", 2024-09-18T10:15:08.860 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001180195 2024-09-18T10:15:08.860 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.861 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.861 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:08.861 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.692779+0000", 2024-09-18T10:15:08.861 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8535000000000003e-05 2024-09-18T10:15:08.861 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.861 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.861 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:08.862 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.692794+0000", 2024-09-18T10:15:08.862 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4807000000000001e-05 2024-09-18T10:15:08.862 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.862 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.862 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:08.862 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.793038+0000", 2024-09-18T10:15:08.862 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100244729 2024-09-18T10:15:08.862 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.862 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.862 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-09-18T10:15:08.862 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.793050+0000", 2024-09-18T10:15:08.862 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1776e-05 2024-09-18T10:15:08.862 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.863 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.863 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:08.863 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.893883+0000", 2024-09-18T10:15:08.863 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100833383 2024-09-18T10:15:08.863 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.863 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.863 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:08.863 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.894056+0000", 2024-09-18T10:15:08.863 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017296999999999999 2024-09-18T10:15:08.863 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.863 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.863 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:08.863 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.896702+0000", 2024-09-18T10:15:08.864 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002645728 2024-09-18T10:15:08.864 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.864 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.864 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:08.864 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.001443+0000", 2024-09-18T10:15:08.864 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.104740824 2024-09-18T10:15:08.864 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.864 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.864 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:15:08.864 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.110173+0000", 2024-09-18T10:15:08.864 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.108729645 2024-09-18T10:15:08.864 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.865 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.865 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:08.865 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.110268+0000", 2024-09-18T10:15:08.865 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4987000000000004e-05 2024-09-18T10:15:08.865 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:08.865 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:08.865 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:08.865 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.865 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.865 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:182 3.cs0 3:3a4fded2:::smithi18925896-23:head [append 0~376832] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:08.865 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:52.693795+0000", 2024-09-18T10:15:08.865 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 16.1113319, 2024-09-18T10:15:08.865 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.51656788200000003, 2024-09-18T10:15:08.866 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:08.866 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:08.866 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:08.866 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:08.866 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:08.866 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 182 2024-09-18T10:15:08.866 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.866 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:08.866 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.866 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:08.866 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.693795+0000", 2024-09-18T10:15:08.866 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:08.866 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.866 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.867 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:08.867 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.693795+0000", 2024-09-18T10:15:08.867 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:08.867 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.867 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.867 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:08.867 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.693787+0000", 2024-09-18T10:15:08.867 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999919 2024-09-18T10:15:08.867 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.867 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.867 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:08.867 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694491+0000", 2024-09-18T10:15:08.867 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00070477199999999999 2024-09-18T10:15:08.868 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.868 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.868 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:08.868 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694526+0000", 2024-09-18T10:15:08.868 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4304999999999998e-05 2024-09-18T10:15:08.868 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.868 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.868 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:08.868 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694539+0000", 2024-09-18T10:15:08.868 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3107e-05 2024-09-18T10:15:08.868 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.868 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.869 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:08.869 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.793145+0000", 2024-09-18T10:15:08.869 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.098605719999999994 2024-09-18T10:15:08.869 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.869 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.869 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:08.869 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.793156+0000", 2024-09-18T10:15:08.869 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1892000000000001e-05 2024-09-18T10:15:08.869 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.869 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.869 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:08.869 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.898472+0000", 2024-09-18T10:15:08.869 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10531558000000001 2024-09-18T10:15:08.870 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.870 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.870 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:08.870 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.898643+0000", 2024-09-18T10:15:08.870 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017116399999999999 2024-09-18T10:15:08.870 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.870 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.870 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:08.870 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.900074+0000", 2024-09-18T10:15:08.870 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0014304299999999999 2024-09-18T10:15:08.870 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.870 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.870 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:08.871 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.001446+0000", 2024-09-18T10:15:08.871 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.101371934 2024-09-18T10:15:08.871 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.871 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.871 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:15:08.871 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.210292+0000", 2024-09-18T10:15:08.871 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.208846693 2024-09-18T10:15:08.871 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.871 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.871 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:08.871 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.210363+0000", 2024-09-18T10:15:08.871 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0470999999999997e-05 2024-09-18T10:15:08.871 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:08.872 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:08.872 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:08.872 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.872 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.872 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:183 3.cs0 3:3a4fded2:::smithi18925896-23:head [setxattr _header (56) in=7b] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:08.872 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:52.694557+0000", 2024-09-18T10:15:08.872 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 16.110569340000001, 2024-09-18T10:15:08.872 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.51641106000000003, 2024-09-18T10:15:08.872 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:08.872 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:08.872 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:08.872 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:08.872 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:08.873 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 183 2024-09-18T10:15:08.873 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.873 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:08.873 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.873 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:08.873 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694557+0000", 2024-09-18T10:15:08.873 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:08.873 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.873 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.873 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:08.873 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694557+0000", 2024-09-18T10:15:08.873 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:08.873 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.874 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.874 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:08.874 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694556+0000", 2024-09-18T10:15:08.874 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:08.874 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.874 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.874 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:08.874 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694570+0000", 2024-09-18T10:15:08.874 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4249e-05 2024-09-18T10:15:08.874 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.874 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.874 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:08.874 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694580+0000", 2024-09-18T10:15:08.874 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7459999999999995e-06 2024-09-18T10:15:08.874 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.875 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.875 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:08.875 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694601+0000", 2024-09-18T10:15:08.875 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1583e-05 2024-09-18T10:15:08.875 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.875 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.875 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:08.875 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.793251+0000", 2024-09-18T10:15:08.875 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.098649465000000006 2024-09-18T10:15:08.875 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.875 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.875 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:08.875 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.793262+0000", 2024-09-18T10:15:08.876 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1113e-05 2024-09-18T10:15:08.876 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.876 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.876 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:08.876 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.001995+0000", 2024-09-18T10:15:08.876 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20873336200000001 2024-09-18T10:15:08.876 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.876 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.876 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:08.876 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.002285+0000", 2024-09-18T10:15:08.876 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00028924300000000001 2024-09-18T10:15:08.876 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.876 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.877 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:08.877 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.003993+0000", 2024-09-18T10:15:08.877 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0017085410000000001 2024-09-18T10:15:08.877 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.877 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.877 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:08.877 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.109904+0000", 2024-09-18T10:15:08.877 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.105911038 2024-09-18T10:15:08.877 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.877 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.877 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:15:08.877 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.210762+0000", 2024-09-18T10:15:08.877 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100858146 2024-09-18T10:15:08.877 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.878 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.878 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:08.878 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.210968+0000", 2024-09-18T10:15:08.878 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00020613000000000001 2024-09-18T10:15:08.878 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:08.878 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:08.878 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:08.878 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.878 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.878 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:184 3.cs0 3:3a4fded2:::smithi18925896-23:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:08.878 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:52.694648+0000", 2024-09-18T10:15:08.878 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 16.110479116, 2024-09-18T10:15:08.879 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.72409976600000003, 2024-09-18T10:15:08.879 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:08.879 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-18T10:15:08.879 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:08.879 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:08.879 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:08.879 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 184 2024-09-18T10:15:08.879 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.879 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:08.879 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.879 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:08.879 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694648+0000", 2024-09-18T10:15:08.879 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:08.880 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.880 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.880 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:08.880 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694648+0000", 2024-09-18T10:15:08.880 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:08.880 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.880 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.880 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:08.880 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694647+0000", 2024-09-18T10:15:08.880 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:15:08.880 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.880 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.881 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:08.881 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694652+0000", 2024-09-18T10:15:08.881 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1170000000000003e-06 2024-09-18T10:15:08.881 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.881 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.881 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:08.881 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694668+0000", 2024-09-18T10:15:08.881 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6019e-05 2024-09-18T10:15:08.881 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.881 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.881 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:08.881 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694674+0000", 2024-09-18T10:15:08.881 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7000000000000002e-06 2024-09-18T10:15:08.882 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.882 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.882 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:08.882 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.893402+0000", 2024-09-18T10:15:08.882 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.19872805700000001 2024-09-18T10:15:08.882 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.882 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.882 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:08.882 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.893413+0000", 2024-09-18T10:15:08.882 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0329999999999999e-05 2024-09-18T10:15:08.882 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.882 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.882 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:08.882 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.004678+0000", 2024-09-18T10:15:08.883 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.111265369 2024-09-18T10:15:08.883 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.883 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.883 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2024-09-18T10:15:08.883 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.004893+0000", 2024-09-18T10:15:08.883 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00021510499999999999 2024-09-18T10:15:08.883 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.883 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.883 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:08.883 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.211096+0000", 2024-09-18T10:15:08.883 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.206202899 2024-09-18T10:15:08.884 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.884 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.884 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:08.884 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.211230+0000", 2024-09-18T10:15:08.884 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000133907 2024-09-18T10:15:08.884 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.885 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.885 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:08.885 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.418747+0000", 2024-09-18T10:15:08.885 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20751733 2024-09-18T10:15:08.885 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:08.885 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:08.885 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:08.885 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.885 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.885 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:196 3.cs0 3:351da1bf:::smithi18925896-6:head [append 0~0] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:08.885 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:52.694697+0000", 2024-09-18T10:15:08.885 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 16.110430017999999, 2024-09-18T10:15:08.885 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.61758450099999995, 2024-09-18T10:15:08.886 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:08.886 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:08.886 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:08.886 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:08.886 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:08.886 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 196 2024-09-18T10:15:08.886 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.886 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:08.886 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.886 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:08.886 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694697+0000", 2024-09-18T10:15:08.886 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:08.886 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.886 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.887 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:08.887 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694697+0000", 2024-09-18T10:15:08.887 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:08.887 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.887 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.887 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:08.887 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694696+0000", 2024-09-18T10:15:08.887 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:15:08.887 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.887 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.887 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:08.887 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694705+0000", 2024-09-18T10:15:08.887 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7799999999999995e-06 2024-09-18T10:15:08.887 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.888 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.888 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:08.888 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694721+0000", 2024-09-18T10:15:08.888 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5102e-05 2024-09-18T10:15:08.888 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.888 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.888 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:08.888 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694727+0000", 2024-09-18T10:15:08.888 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2570000000000004e-06 2024-09-18T10:15:08.888 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.888 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.888 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:08.889 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.893457+0000", 2024-09-18T10:15:08.889 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.198729824 2024-09-18T10:15:08.889 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.889 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.889 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:08.889 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.893464+0000", 2024-09-18T10:15:08.889 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6860000000000006e-06 2024-09-18T10:15:08.889 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.889 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.889 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:08.889 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.105086+0000", 2024-09-18T10:15:08.889 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.21162193000000001 2024-09-18T10:15:08.889 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.889 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.890 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:08.890 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.105357+0000", 2024-09-18T10:15:08.890 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00027109699999999998 2024-09-18T10:15:08.890 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.890 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.890 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:08.890 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.107161+0000", 2024-09-18T10:15:08.890 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001803727 2024-09-18T10:15:08.890 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.890 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.890 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:08.890 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.109961+0000", 2024-09-18T10:15:08.891 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0028003080000000001 2024-09-18T10:15:08.891 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.891 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.891 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:15:08.891 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.312174+0000", 2024-09-18T10:15:08.891 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20221219700000001 2024-09-18T10:15:08.891 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.891 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.891 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:08.891 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.312281+0000", 2024-09-18T10:15:08.891 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000107656 2024-09-18T10:15:08.891 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:08.891 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:08.891 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:08.892 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.892 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.892 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:197 3.cs0 3:351da1bf:::smithi18925896-6:head [rollback 2] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:08.892 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:52.694734+0000", 2024-09-18T10:15:08.892 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 16.110392976, 2024-09-18T10:15:08.892 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.61830483899999999, 2024-09-18T10:15:08.892 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:08.892 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:08.892 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:08.892 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:08.892 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:08.892 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 197 2024-09-18T10:15:08.892 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.893 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:08.893 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.893 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:08.893 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694734+0000", 2024-09-18T10:15:08.893 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:08.893 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.893 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.893 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:08.893 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694734+0000", 2024-09-18T10:15:08.893 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:08.893 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.893 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.893 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:08.893 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694733+0000", 2024-09-18T10:15:08.894 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:15:08.894 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.894 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.894 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:08.894 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694737+0000", 2024-09-18T10:15:08.894 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.636e-06 2024-09-18T10:15:08.894 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.894 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.894 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:08.894 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694752+0000", 2024-09-18T10:15:08.894 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5792999999999999e-05 2024-09-18T10:15:08.894 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.895 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.895 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:08.895 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694758+0000", 2024-09-18T10:15:08.895 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2660000000000001e-06 2024-09-18T10:15:08.895 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.895 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.895 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:08.895 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.108286+0000", 2024-09-18T10:15:08.895 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.41352792700000002 2024-09-18T10:15:08.895 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.895 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.895 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:08.895 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.108540+0000", 2024-09-18T10:15:08.896 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000253928 2024-09-18T10:15:08.896 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.896 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.896 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:08.896 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.109284+0000", 2024-09-18T10:15:08.896 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00074488600000000005 2024-09-18T10:15:08.896 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.896 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.896 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:08.896 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.210174+0000", 2024-09-18T10:15:08.896 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10088973699999999 2024-09-18T10:15:08.896 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.896 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.897 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:15:08.897 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.312857+0000", 2024-09-18T10:15:08.897 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10268295600000001 2024-09-18T10:15:08.897 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.897 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.897 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:08.897 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.313039+0000", 2024-09-18T10:15:08.897 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000181476 2024-09-18T10:15:08.897 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:08.897 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:08.897 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:08.897 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.897 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.898 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:331 3.cs0 3:3a4fded2:::smithi18925896-23:head [create,setxattr qPd\\P_iff[nbYodrbXe[dQ_okmVWqe[RceaqQRNm (630) in=40b,setxattr NWYlZi[dPXnppNo]`fatPqfSaTlsQVetOmrQm^ap (138) in=40b,setxattr iVfcaeRT]TQppeYbq_`lRobpUsaVcWXphplasbgY (1507) in=40b,setxattr YeaeOf[jt`g\\k^]VfXTjOONb`jT[WYUgdiglVjYl (8841) in=40b,setxattr [iqZhSWl_sclXkajVYTXlOX]eeeWkPi\\dOYYt_Wb (1928) in=40b,setxattr \\[gaSPWYO`NQXpTccfbk`SZPaU`lbpTk]a] (1262) in=35b,setxattr ]\\]U^niYqQmjcgO[dsO`YpRZeRV]`X]bU[X^_Qis (1086) in=40b,setxattr VafomYY]aa`a^pbWebSRaithSn_dQjQYq]aTarm\\ (1402) in=40b,setxattr acj^VZhfaaikXrXq]_haYqkl[^_atWUTaloiVj`a (105) in=40b,setxattr VNUlaeZOglfodan^Ycb`dqVeZhSTmO_rabViYVch (70) in=40b,setxattr Qka\\PoUoghRc\\Qcnl`kVbj[rs\\hSdVXmlNqha`ta (594) in=40b,setxattr qsSQhb_apjdTlikdpj\\maaVdS[mVfsh_Vqi[ogWo (771) in=40b,setxattr Tm]oi_Rf_V`pSY[VhYSXSQN^bqjtgmm^fUo^f\\ZV (1087) in=40b,setxattr \\dTq (1262) in=4b,setxattr nettWfThbXj]cUpsYrTjZ^qk^\\[ZoZ[qb]Sf[ZbN (1684) in=40b,setxattr Sa\\hrdN^YombnT\\torZ`N]aRqicS_lP^XgP`e`Oc (1963) in=40b,setxattr dWhoTgYPSTh]jfWhPTs^iVdYbtfhi_WjOdes]NXh (1051) in=40b,setxattr Qbp_QrhXQf`O]blcqeqgRchO]RrhbshoV\\cW[_es (10033) in=40b,setxattr WYbQqgdRjmdbeoaP]U]n]PTWOYkP^lWOanNTnaOr (1998) in=40b,setxattr Rfbdg`[^ZYrN]cl][Ypr^tpXmb_mbh[gf\\idocW] (10804) in=40b] snapc b=[b,4] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-18T10:15:08.898 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:02.781073+0000", 2024-09-18T10:15:08.898 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 6.0240536679999996, 2024-09-18T10:15:08.898 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.32781067699999999, 2024-09-18T10:15:08.898 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:08.898 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:08.898 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:08.898 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:08.898 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:08.898 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 331 2024-09-18T10:15:08.898 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.898 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:08.898 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.899 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:08.899 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.781073+0000", 2024-09-18T10:15:08.899 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:08.899 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.899 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.899 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:08.899 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.781073+0000", 2024-09-18T10:15:08.899 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:08.899 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.899 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.899 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:08.899 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.781070+0000", 2024-09-18T10:15:08.899 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999971 2024-09-18T10:15:08.899 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.900 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.900 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:08.900 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.781166+0000", 2024-09-18T10:15:08.900 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5959999999999996e-05 2024-09-18T10:15:08.900 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.900 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.900 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:08.900 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.781206+0000", 2024-09-18T10:15:08.900 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9307999999999997e-05 2024-09-18T10:15:08.900 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.900 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.900 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:08.900 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.781221+0000", 2024-09-18T10:15:08.901 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5549999999999999e-05 2024-09-18T10:15:08.901 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.901 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.901 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:08.901 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.903358+0000", 2024-09-18T10:15:08.901 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.12213652799999999 2024-09-18T10:15:08.901 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.901 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.901 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-09-18T10:15:08.901 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.903375+0000", 2024-09-18T10:15:08.901 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7856999999999999e-05 2024-09-18T10:15:08.901 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.902 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.902 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:08.902 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.904269+0000", 2024-09-18T10:15:08.902 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00089356799999999997 2024-09-18T10:15:08.902 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.902 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.902 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:08.902 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.904590+0000", 2024-09-18T10:15:08.902 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00032100100000000001 2024-09-18T10:15:08.902 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.902 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.902 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:08.903 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.906644+0000", 2024-09-18T10:15:08.903 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0020539830000000001 2024-09-18T10:15:08.903 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.903 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.903 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:08.903 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.912414+0000", 2024-09-18T10:15:08.903 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0057699070000000003 2024-09-18T10:15:08.903 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.903 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.903 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:15:08.903 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:03.108757+0000", 2024-09-18T10:15:08.903 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.196343085 2024-09-18T10:15:08.903 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.903 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.903 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:08.904 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:03.108884+0000", 2024-09-18T10:15:08.904 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012672000000000001 2024-09-18T10:15:08.904 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:08.904 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:08.904 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:08.904 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.904 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.904 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:340 3.cs0 3:3d6b9cb0:::smithi18925896-31:head [append 0~0] snapc b=[b,4] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-18T10:15:08.904 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:02.803124+0000", 2024-09-18T10:15:08.904 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 6.0020028539999997, 2024-09-18T10:15:08.904 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30615964099999998, 2024-09-18T10:15:08.905 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:08.905 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:08.905 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:08.905 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:08.905 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:08.905 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 340 2024-09-18T10:15:08.905 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.905 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:08.905 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.905 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:08.905 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.803124+0000", 2024-09-18T10:15:08.905 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:08.905 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.906 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.906 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:08.906 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.803124+0000", 2024-09-18T10:15:08.906 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:08.906 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.906 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.906 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:08.906 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.803122+0000", 2024-09-18T10:15:08.906 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999976 2024-09-18T10:15:08.906 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.906 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.906 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:08.906 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.803131+0000", 2024-09-18T10:15:08.906 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1929999999999999e-06 2024-09-18T10:15:08.907 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.907 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.907 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:08.907 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.803151+0000", 2024-09-18T10:15:08.907 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9998000000000001e-05 2024-09-18T10:15:08.907 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.907 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.907 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:08.907 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.803160+0000", 2024-09-18T10:15:08.907 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3729999999999999e-06 2024-09-18T10:15:08.907 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.907 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.908 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:08.908 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.903508+0000", 2024-09-18T10:15:08.908 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100347275 2024-09-18T10:15:08.908 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.908 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.908 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:08.908 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.903516+0000", 2024-09-18T10:15:08.908 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2260000000000005e-06 2024-09-18T10:15:08.908 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.908 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.908 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:08.908 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.908957+0000", 2024-09-18T10:15:08.908 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00544138 2024-09-18T10:15:08.908 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.909 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.909 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:08.909 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.909232+0000", 2024-09-18T10:15:08.909 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00027507699999999998 2024-09-18T10:15:08.909 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.909 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.909 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:08.909 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.910206+0000", 2024-09-18T10:15:08.909 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00097342100000000003 2024-09-18T10:15:08.909 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.909 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.909 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:08.909 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.912421+0000", 2024-09-18T10:15:08.910 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0022157470000000001 2024-09-18T10:15:08.910 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.910 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.910 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:15:08.910 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:03.109124+0000", 2024-09-18T10:15:08.910 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.19670262499999999 2024-09-18T10:15:08.910 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.910 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.910 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:08.910 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:03.109284+0000", 2024-09-18T10:15:08.910 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000159512 2024-09-18T10:15:08.910 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:08.910 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:08.910 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:08.911 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.911 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.911 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:341 3.cs0 3:3d6b9cb0:::smithi18925896-31:head [rollback 4] snapc b=[b,4] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-18T10:15:08.911 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:02.803176+0000", 2024-09-18T10:15:08.911 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 6.001951, 2024-09-18T10:15:08.911 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30684217000000003, 2024-09-18T10:15:08.911 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:08.911 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:08.911 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:08.911 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:08.911 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:08.911 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 341 2024-09-18T10:15:08.911 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.911 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:08.912 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.912 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:08.912 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.803176+0000", 2024-09-18T10:15:08.912 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:08.912 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.912 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.912 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:08.912 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.803176+0000", 2024-09-18T10:15:08.912 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:08.912 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.912 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.912 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:08.912 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.803175+0000", 2024-09-18T10:15:08.913 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:08.913 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.913 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.913 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:08.913 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.803181+0000", 2024-09-18T10:15:08.913 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3579999999999999e-06 2024-09-18T10:15:08.913 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.913 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.913 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:08.913 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.803198+0000", 2024-09-18T10:15:08.913 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6855e-05 2024-09-18T10:15:08.913 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.913 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.914 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:08.914 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.803213+0000", 2024-09-18T10:15:08.914 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4626e-05 2024-09-18T10:15:08.914 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.914 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.914 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:08.914 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.911165+0000", 2024-09-18T10:15:08.914 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.107952006 2024-09-18T10:15:08.914 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.914 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.914 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:08.914 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.911317+0000", 2024-09-18T10:15:08.914 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015228999999999999 2024-09-18T10:15:08.915 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.915 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.915 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:08.915 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.911914+0000", 2024-09-18T10:15:08.915 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000597056 2024-09-18T10:15:08.915 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.915 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.915 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:08.915 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:03.012693+0000", 2024-09-18T10:15:08.915 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100778485 2024-09-18T10:15:08.915 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.915 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.915 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:15:08.916 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:03.109913+0000", 2024-09-18T10:15:08.916 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.097220438000000006 2024-09-18T10:15:08.916 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.916 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.916 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:08.916 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:03.110018+0000", 2024-09-18T10:15:08.916 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000104746 2024-09-18T10:15:08.916 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:08.916 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:08.916 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:08.916 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.916 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.916 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:342 3.cs0 3:3d6b9cb0:::smithi18925896-31:head [append 0~0] snapc b=[b,4] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-18T10:15:08.917 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:02.803223+0000", 2024-09-18T10:15:08.917 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 6.0019041120000001, 2024-09-18T10:15:08.917 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30756660499999999, 2024-09-18T10:15:08.917 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:08.917 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:08.917 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:08.917 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:08.917 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:08.917 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 342 2024-09-18T10:15:08.917 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.917 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:08.917 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.917 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:08.918 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.803223+0000", 2024-09-18T10:15:08.918 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:08.918 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.918 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.918 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:08.918 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.803223+0000", 2024-09-18T10:15:08.918 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:08.918 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.918 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.918 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:08.918 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.803222+0000", 2024-09-18T10:15:08.918 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:15:08.918 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.918 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.919 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:08.919 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.803226+0000", 2024-09-18T10:15:08.919 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1289999999999998e-06 2024-09-18T10:15:08.919 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.919 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.919 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:08.919 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.803235+0000", 2024-09-18T10:15:08.919 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1120000000000002e-06 2024-09-18T10:15:08.919 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.919 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.919 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:08.919 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.803239+0000", 2024-09-18T10:15:08.919 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8639999999999998e-06 2024-09-18T10:15:08.919 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.919 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.920 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:08.920 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.913431+0000", 2024-09-18T10:15:08.920 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.110192522 2024-09-18T10:15:08.920 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.920 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.920 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:08.920 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.913523+0000", 2024-09-18T10:15:08.920 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1885000000000005e-05 2024-09-18T10:15:08.920 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.920 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.920 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:08.920 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.913971+0000", 2024-09-18T10:15:08.920 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00044766599999999999 2024-09-18T10:15:08.920 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.921 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.921 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:08.921 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:03.012696+0000", 2024-09-18T10:15:08.921 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.098725270000000004 2024-09-18T10:15:08.921 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.921 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.921 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:15:08.921 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:03.110683+0000", 2024-09-18T10:15:08.921 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.097986670999999997 2024-09-18T10:15:08.921 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:08.921 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:08.921 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:08.921 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:03.110789+0000", 2024-09-18T10:15:08.921 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000106543 2024-09-18T10:15:08.922 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:08.922 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:08.922 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:08.922 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:08.922 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:08.922 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-18T10:15:08.923 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-09-18T10:15:08.924 INFO:tasks.ceph.osd.14.smithi154.stderr:2024-09-18T10:15:08.919+0000 7f75a19e7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 22093) UID: 0 2024-09-18T10:15:08.950 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15] out_osds: [] dead_osds: [] live_osds: [0, 4, 8, 12, 1, 5, 9, 13, 2, 6, 10, 14, 3, 7, 11, 15] 2024-09-18T10:15:08.950 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-18T10:15:08.954 INFO:tasks.rados.rados.0.smithi189.stdout:164: done (1 left) 2024-09-18T10:15:08.954 INFO:tasks.rados.rados.0.smithi189.stdout:165: snap_create 2024-09-18T10:15:08.993 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T10:15:09.024 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-18T10:15:09.021+0000 7f1981873700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22132) UID: 0 2024-09-18T10:15:09.029 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-18T10:15:09.029 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-18T10:15:09.029 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-09-18T10:15:09.029 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-18T10:15:09.039 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-09-18T10:15:09.045 INFO:tasks.rados.rados.0.smithi189.stdout:163: finishing copy_from racing read to smithi18925896-48 2024-09-18T10:15:09.125 INFO:tasks.ceph.osd.3.smithi189.stderr:2024-09-18T10:15:09.123+0000 7f2440fc7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22396) UID: 0 2024-09-18T10:15:09.134 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-18T10:15:09.134 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-18T10:15:09.134 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-18T10:15:09.134 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-18T10:15:09.134 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-18T10:15:09.143 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-09-18T10:15:09.145 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-18T10:15:09.174 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-18T10:15:09.195 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-18T10:15:09.218 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-18T10:15:09.225 INFO:tasks.ceph.osd.11.smithi189.stderr:2024-09-18T10:15:09.223+0000 7f718b60a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 22400) UID: 0 2024-09-18T10:15:09.237 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-18T10:15:09.243 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-18T10:15:09.243 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-18T10:15:09.244 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-18T10:15:09.244 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-18T10:15:09.244 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.244 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.6s5 36/32 ECSubWrite(tid=19, reqid=client.4632.0:167, at_version=36'11, trim_to=0'0, roll_forward_to=36'10))", 2024-09-18T10:15:09.244 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:51.840599+0000", 2024-09-18T10:15:09.244 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 17.401531605999999, 2024-09-18T10:15:09.244 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.102418184, 2024-09-18T10:15:09.244 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:09.244 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:09.244 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:09.244 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.244 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:09.245 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.840599+0000", 2024-09-18T10:15:09.245 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:09.245 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.245 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.245 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:09.245 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.840599+0000", 2024-09-18T10:15:09.245 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:09.245 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.245 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.246 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:09.246 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.840592+0000", 2024-09-18T10:15:09.246 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999928 2024-09-18T10:15:09.246 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.246 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.246 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:09.246 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.840784+0000", 2024-09-18T10:15:09.246 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019234300000000001 2024-09-18T10:15:09.246 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.246 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.246 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:09.246 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.840824+0000", 2024-09-18T10:15:09.247 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9654999999999999e-05 2024-09-18T10:15:09.247 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.247 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.247 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:09.247 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.940946+0000", 2024-09-18T10:15:09.247 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10012244300000001 2024-09-18T10:15:09.247 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.247 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.247 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:09.247 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.941101+0000", 2024-09-18T10:15:09.247 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015468200000000001 2024-09-18T10:15:09.247 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.247 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.247 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:09.248 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.941135+0000", 2024-09-18T10:15:09.248 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4020999999999998e-05 2024-09-18T10:15:09.248 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.248 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.248 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:09.248 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.942947+0000", 2024-09-18T10:15:09.248 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001812218 2024-09-18T10:15:09.248 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.248 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.248 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:09.248 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.943017+0000", 2024-09-18T10:15:09.248 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9805e-05 2024-09-18T10:15:09.248 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:09.249 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:09.249 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:09.249 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.249 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.249 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpRead(3.0s2 37/32 ECSubRead(tid=8, to_read={3:02f544f0:::smithi18925896-30:head=0,122880,0}, subchunks={3:02f544f0:::smithi18925896-30:head=[0,1]}, attrs_to_read=))", 2024-09-18T10:15:09.249 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:52.681732+0000", 2024-09-18T10:15:09.249 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 16.560399014000001, 2024-09-18T10:15:09.249 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.101148985, 2024-09-18T10:15:09.249 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:09.249 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:09.249 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:09.249 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.249 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:09.249 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.681732+0000", 2024-09-18T10:15:09.250 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:09.250 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.250 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.250 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:09.250 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.681732+0000", 2024-09-18T10:15:09.250 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:09.250 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.250 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.250 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:09.250 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.681731+0000", 2024-09-18T10:15:09.251 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:09.251 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.251 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.251 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:09.251 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.681750+0000", 2024-09-18T10:15:09.251 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8885e-05 2024-09-18T10:15:09.251 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.251 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.251 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:09.251 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.681794+0000", 2024-09-18T10:15:09.251 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4193000000000002e-05 2024-09-18T10:15:09.251 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.251 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.251 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:09.252 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.681809+0000", 2024-09-18T10:15:09.252 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5228999999999999e-05 2024-09-18T10:15:09.252 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.252 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.252 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:09.252 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.782079+0000", 2024-09-18T10:15:09.252 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100269752 2024-09-18T10:15:09.252 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.252 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.252 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:09.252 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.782880+0000", 2024-09-18T10:15:09.252 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00080150900000000005 2024-09-18T10:15:09.252 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:09.253 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:09.253 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:09.253 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.253 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.253 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpRead(3.0s2 37/32 ECSubRead(tid=9, to_read={3:02f544f0:::smithi18925896-30:head=0,122880,0}, subchunks={3:02f544f0:::smithi18925896-30:head=[0,1]}, attrs_to_read=))", 2024-09-18T10:15:09.253 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:52.682582+0000", 2024-09-18T10:15:09.253 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 16.559548333999999, 2024-09-18T10:15:09.253 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100690751, 2024-09-18T10:15:09.253 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:09.253 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:09.253 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:09.253 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.254 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:09.254 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.682582+0000", 2024-09-18T10:15:09.254 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:09.254 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.254 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.254 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:09.254 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.682582+0000", 2024-09-18T10:15:09.254 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:09.254 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.254 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.254 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:09.254 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.682581+0000", 2024-09-18T10:15:09.254 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:09.255 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.255 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.255 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:09.255 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.682614+0000", 2024-09-18T10:15:09.255 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3305e-05 2024-09-18T10:15:09.255 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.255 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.255 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:09.255 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.682641+0000", 2024-09-18T10:15:09.255 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7353000000000001e-05 2024-09-18T10:15:09.255 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.255 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.255 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:09.255 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.682662+0000", 2024-09-18T10:15:09.256 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0319e-05 2024-09-18T10:15:09.256 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.256 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.256 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:09.256 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.782980+0000", 2024-09-18T10:15:09.256 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100318121 2024-09-18T10:15:09.256 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.256 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.256 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:09.256 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.783273+0000", 2024-09-18T10:15:09.256 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00029314599999999999 2024-09-18T10:15:09.256 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:09.256 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:09.257 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:09.257 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.257 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.257 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.ds4 37/32 ECSubWrite(tid=31, reqid=client.4632.0:176, at_version=37'4, trim_to=0'0, roll_forward_to=35'2))", 2024-09-18T10:15:09.257 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:53.002892+0000", 2024-09-18T10:15:09.257 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 16.23923821, 2024-09-18T10:15:09.257 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10294343, 2024-09-18T10:15:09.257 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:09.257 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:09.257 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:09.257 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.257 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:09.257 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.002892+0000", 2024-09-18T10:15:09.258 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:09.258 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.258 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.258 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:09.258 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.002892+0000", 2024-09-18T10:15:09.258 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:09.258 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.258 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.258 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:09.258 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.002891+0000", 2024-09-18T10:15:09.258 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:15:09.258 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.258 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.258 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:09.259 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.003053+0000", 2024-09-18T10:15:09.259 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00016189399999999999 2024-09-18T10:15:09.259 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.259 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.259 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:09.259 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.003089+0000", 2024-09-18T10:15:09.259 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5329e-05 2024-09-18T10:15:09.259 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.259 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.259 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:09.259 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.003108+0000", 2024-09-18T10:15:09.259 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8963000000000001e-05 2024-09-18T10:15:09.259 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.260 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.260 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:09.260 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.103401+0000", 2024-09-18T10:15:09.260 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100293436 2024-09-18T10:15:09.260 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.260 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.260 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:09.260 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.103418+0000", 2024-09-18T10:15:09.260 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6722999999999999e-05 2024-09-18T10:15:09.260 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.260 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.260 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:09.260 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.105765+0000", 2024-09-18T10:15:09.260 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0023477279999999999 2024-09-18T10:15:09.261 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.261 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.261 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:09.261 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.105836+0000", 2024-09-18T10:15:09.261 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0246999999999993e-05 2024-09-18T10:15:09.261 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:09.261 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:09.261 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:09.261 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.261 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.261 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.3s1 37/32 ECSubWrite(tid=11, reqid=client.4632.0:189, at_version=37'8, trim_to=0'0, roll_forward_to=36'6))", 2024-09-18T10:15:09.261 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:53.096486+0000", 2024-09-18T10:15:09.261 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 16.145644269999998, 2024-09-18T10:15:09.262 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10315053, 2024-09-18T10:15:09.262 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:09.262 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:09.262 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:09.262 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.262 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:09.262 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.096486+0000", 2024-09-18T10:15:09.262 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:09.262 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.262 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.262 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:09.262 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.096486+0000", 2024-09-18T10:15:09.262 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:09.262 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.263 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.263 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:09.263 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.096486+0000", 2024-09-18T10:15:09.263 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:09.263 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.263 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.263 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:09.263 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.096802+0000", 2024-09-18T10:15:09.263 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00031615899999999999 2024-09-18T10:15:09.263 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.263 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.263 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:09.263 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.096841+0000", 2024-09-18T10:15:09.264 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9452000000000001e-05 2024-09-18T10:15:09.264 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.264 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.264 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:09.264 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.096866+0000", 2024-09-18T10:15:09.264 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5069e-05 2024-09-18T10:15:09.264 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.264 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.264 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:09.264 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.197104+0000", 2024-09-18T10:15:09.264 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100237886 2024-09-18T10:15:09.264 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.264 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.264 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:09.265 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.197130+0000", 2024-09-18T10:15:09.265 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5666000000000001e-05 2024-09-18T10:15:09.265 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.265 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.265 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:09.265 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.199585+0000", 2024-09-18T10:15:09.265 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002454777 2024-09-18T10:15:09.265 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.265 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.265 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:09.265 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.199637+0000", 2024-09-18T10:15:09.265 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2221e-05 2024-09-18T10:15:09.265 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:09.265 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:09.266 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:09.266 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.266 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.266 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.ds4 37/32 ECSubWrite(tid=32, reqid=client.4632.0:176, at_version=0'0, trim_to=0'0, roll_forward_to=37'4))", 2024-09-18T10:15:09.266 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:53.107300+0000", 2024-09-18T10:15:09.266 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 16.134831014, 2024-09-18T10:15:09.266 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.101315192, 2024-09-18T10:15:09.266 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:09.266 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:09.266 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:09.266 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.266 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:09.266 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.107300+0000", 2024-09-18T10:15:09.267 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:09.267 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.267 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.267 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:09.267 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.107300+0000", 2024-09-18T10:15:09.267 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:09.267 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.267 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.267 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:09.268 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.107298+0000", 2024-09-18T10:15:09.268 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:09.268 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.268 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.268 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:09.268 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.107342+0000", 2024-09-18T10:15:09.268 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4542000000000001e-05 2024-09-18T10:15:09.268 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.268 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.268 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:09.268 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.107425+0000", 2024-09-18T10:15:09.268 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2590999999999996e-05 2024-09-18T10:15:09.268 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.269 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.269 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:09.269 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.207527+0000", 2024-09-18T10:15:09.269 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100102406 2024-09-18T10:15:09.269 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.269 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.269 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:09.269 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.207705+0000", 2024-09-18T10:15:09.269 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017770399999999999 2024-09-18T10:15:09.269 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.269 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.269 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:09.269 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.207727+0000", 2024-09-18T10:15:09.270 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1991999999999999e-05 2024-09-18T10:15:09.270 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.270 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.270 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:09.270 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.208570+0000", 2024-09-18T10:15:09.270 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00084287200000000004 2024-09-18T10:15:09.270 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.270 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.270 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:09.270 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.208615+0000", 2024-09-18T10:15:09.270 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5037999999999998e-05 2024-09-18T10:15:09.270 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:09.270 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:09.271 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:09.271 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.271 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.271 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.3s1 37/32 ECSubWrite(tid=13, reqid=client.4632.0:194, at_version=37'10, trim_to=0'0, roll_forward_to=37'8))", 2024-09-18T10:15:09.271 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:53.412495+0000", 2024-09-18T10:15:09.271 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 15.8296355, 2024-09-18T10:15:09.271 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.102940423, 2024-09-18T10:15:09.271 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:09.271 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:09.271 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:09.271 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.271 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:09.271 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.412495+0000", 2024-09-18T10:15:09.271 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:09.272 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.272 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.272 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:09.272 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.412495+0000", 2024-09-18T10:15:09.272 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:09.272 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.272 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.272 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:09.272 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.412493+0000", 2024-09-18T10:15:09.272 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999976 2024-09-18T10:15:09.272 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.272 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.272 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:09.273 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.412821+0000", 2024-09-18T10:15:09.273 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000327986 2024-09-18T10:15:09.273 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.273 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.273 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:09.273 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.412875+0000", 2024-09-18T10:15:09.273 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4128000000000001e-05 2024-09-18T10:15:09.273 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.273 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.273 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:09.273 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.412895+0000", 2024-09-18T10:15:09.273 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0445000000000002e-05 2024-09-18T10:15:09.273 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.274 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.274 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:09.274 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.513126+0000", 2024-09-18T10:15:09.274 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100231025 2024-09-18T10:15:09.274 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.274 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.274 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:09.274 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.513146+0000", 2024-09-18T10:15:09.274 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9992000000000001e-05 2024-09-18T10:15:09.274 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.274 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.274 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:09.274 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.515351+0000", 2024-09-18T10:15:09.274 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0022045599999999999 2024-09-18T10:15:09.274 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.275 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.275 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:09.275 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.515435+0000", 2024-09-18T10:15:09.275 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4536999999999994e-05 2024-09-18T10:15:09.275 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:09.275 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:09.275 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:09.275 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.275 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.275 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.ds4 41/32 ECSubWrite(tid=41, reqid=osd.13.0:41, at_version=41'6, trim_to=0'0, roll_forward_to=37'4))", 2024-09-18T10:15:09.275 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:57.138491+0000", 2024-09-18T10:15:09.275 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 12.103639697, 2024-09-18T10:15:09.275 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.200128009, 2024-09-18T10:15:09.275 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:09.276 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:09.276 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:09.276 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.276 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:09.276 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.138491+0000", 2024-09-18T10:15:09.276 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:09.276 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.276 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.276 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:09.276 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.138491+0000", 2024-09-18T10:15:09.276 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:09.276 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.276 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.277 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:09.277 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.138490+0000", 2024-09-18T10:15:09.277 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:15:09.277 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.277 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.277 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:09.277 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.138555+0000", 2024-09-18T10:15:09.277 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4571000000000002e-05 2024-09-18T10:15:09.277 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.277 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.277 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:09.277 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.138581+0000", 2024-09-18T10:15:09.277 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6069000000000001e-05 2024-09-18T10:15:09.277 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.278 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.278 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:09.278 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.138598+0000", 2024-09-18T10:15:09.278 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7059e-05 2024-09-18T10:15:09.278 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.278 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.278 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:09.278 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.336697+0000", 2024-09-18T10:15:09.278 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.198099517 2024-09-18T10:15:09.278 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.278 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.278 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:09.278 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.336733+0000", 2024-09-18T10:15:09.279 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5369e-05 2024-09-18T10:15:09.279 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.279 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.279 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:09.279 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.338552+0000", 2024-09-18T10:15:09.279 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001819773 2024-09-18T10:15:09.279 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.279 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.279 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:09.279 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.338619+0000", 2024-09-18T10:15:09.279 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6390999999999995e-05 2024-09-18T10:15:09.279 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:09.279 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:09.280 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:09.280 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.280 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.280 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.ds4 41/32 ECSubWrite(tid=43, reqid=osd.13.0:41, at_version=0'0, trim_to=0'0, roll_forward_to=41'6))", 2024-09-18T10:15:09.280 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:57.441404+0000", 2024-09-18T10:15:09.280 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 11.800726087999999, 2024-09-18T10:15:09.280 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30108198600000002, 2024-09-18T10:15:09.280 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:09.280 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:09.280 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:09.280 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.280 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:09.280 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.441404+0000", 2024-09-18T10:15:09.280 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:09.281 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.281 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.281 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:09.281 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.441404+0000", 2024-09-18T10:15:09.281 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:09.281 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.281 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.281 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:09.281 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.441403+0000", 2024-09-18T10:15:09.281 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:09.281 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.281 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.281 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:09.281 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.441435+0000", 2024-09-18T10:15:09.282 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2502000000000002e-05 2024-09-18T10:15:09.282 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.282 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.282 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:09.282 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.441473+0000", 2024-09-18T10:15:09.282 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8302e-05 2024-09-18T10:15:09.282 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.282 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.282 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:09.282 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.441524+0000", 2024-09-18T10:15:09.282 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0679e-05 2024-09-18T10:15:09.282 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.282 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.283 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:09.283 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.641739+0000", 2024-09-18T10:15:09.283 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20021524299999999 2024-09-18T10:15:09.283 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.283 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.283 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:09.283 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.641766+0000", 2024-09-18T10:15:09.283 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7081999999999999e-05 2024-09-18T10:15:09.283 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.283 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.283 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:09.283 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.742432+0000", 2024-09-18T10:15:09.283 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10066531300000001 2024-09-18T10:15:09.283 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.284 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.284 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:09.284 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.742486+0000", 2024-09-18T10:15:09.284 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4691999999999997e-05 2024-09-18T10:15:09.284 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:09.284 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:09.284 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:09.284 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.284 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.284 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.6s5 41/40 ECSubWrite(tid=31, reqid=client.4632.0:249, at_version=41'13, trim_to=0'0, roll_forward_to=0'0))", 2024-09-18T10:15:09.284 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:57.550603+0000", 2024-09-18T10:15:09.284 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 11.691527898, 2024-09-18T10:15:09.284 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.103590501, 2024-09-18T10:15:09.285 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:09.285 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:09.285 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:09.285 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.285 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:09.285 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.550603+0000", 2024-09-18T10:15:09.285 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:09.285 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.285 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.285 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:09.285 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.550603+0000", 2024-09-18T10:15:09.285 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:09.285 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.285 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.286 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:09.286 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.550602+0000", 2024-09-18T10:15:09.286 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:15:09.286 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.286 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.286 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:09.286 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.551120+0000", 2024-09-18T10:15:09.286 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000518867 2024-09-18T10:15:09.286 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.286 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.286 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:09.286 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.551141+0000", 2024-09-18T10:15:09.286 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0873000000000002e-05 2024-09-18T10:15:09.286 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.287 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.287 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:09.287 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.551150+0000", 2024-09-18T10:15:09.287 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.884e-06 2024-09-18T10:15:09.287 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.287 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.287 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:09.287 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.651352+0000", 2024-09-18T10:15:09.287 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100201533 2024-09-18T10:15:09.287 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.287 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.287 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:09.287 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.651384+0000", 2024-09-18T10:15:09.288 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2471999999999997e-05 2024-09-18T10:15:09.288 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.288 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.288 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:09.288 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.654157+0000", 2024-09-18T10:15:09.288 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0027727440000000002 2024-09-18T10:15:09.288 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.288 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.288 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:09.288 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.654193+0000", 2024-09-18T10:15:09.288 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6242000000000003e-05 2024-09-18T10:15:09.288 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:09.289 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:09.289 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:09.289 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.289 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.289 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.6s5 41/40 ECSubWrite(tid=32, reqid=client.4632.0:250, at_version=41'14, trim_to=0'0, roll_forward_to=0'0))", 2024-09-18T10:15:09.289 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:57.552846+0000", 2024-09-18T10:15:09.289 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 11.689284903000001, 2024-09-18T10:15:09.289 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10139358900000001, 2024-09-18T10:15:09.289 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:09.290 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:09.290 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:09.290 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.290 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:09.290 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.552846+0000", 2024-09-18T10:15:09.290 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:09.290 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.290 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.290 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:09.290 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.552846+0000", 2024-09-18T10:15:09.290 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:09.290 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.291 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.291 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:09.291 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.552845+0000", 2024-09-18T10:15:09.291 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:15:09.291 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.291 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.291 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:09.291 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.553026+0000", 2024-09-18T10:15:09.291 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00018096599999999999 2024-09-18T10:15:09.291 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.291 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.291 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:09.291 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.553043+0000", 2024-09-18T10:15:09.291 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6818000000000001e-05 2024-09-18T10:15:09.292 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.292 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.292 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:09.292 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.553054+0000", 2024-09-18T10:15:09.292 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1099e-05 2024-09-18T10:15:09.292 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.292 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.292 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:09.292 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.652809+0000", 2024-09-18T10:15:09.292 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.099754887 2024-09-18T10:15:09.292 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.292 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.292 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:09.293 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.652836+0000", 2024-09-18T10:15:09.293 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7582000000000001e-05 2024-09-18T10:15:09.293 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.293 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.293 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:09.293 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.654225+0000", 2024-09-18T10:15:09.293 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0013888170000000001 2024-09-18T10:15:09.293 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.293 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.293 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:09.293 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.654239+0000", 2024-09-18T10:15:09.293 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4260000000000001e-05 2024-09-18T10:15:09.293 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:09.294 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:09.294 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:09.294 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.294 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.294 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.6s5 41/40 ECSubWrite(tid=33, reqid=client.4632.0:251, at_version=41'15, trim_to=0'0, roll_forward_to=0'0))", 2024-09-18T10:15:09.294 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:57.554251+0000", 2024-09-18T10:15:09.294 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 11.687879665000001, 2024-09-18T10:15:09.294 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100858745, 2024-09-18T10:15:09.294 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:09.294 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:09.294 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:09.294 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.294 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:09.295 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.554251+0000", 2024-09-18T10:15:09.295 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:09.295 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.295 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.295 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:09.295 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.554251+0000", 2024-09-18T10:15:09.295 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:09.295 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.295 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.295 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:09.295 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.554249+0000", 2024-09-18T10:15:09.295 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:09.295 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.296 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.296 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:09.296 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.554288+0000", 2024-09-18T10:15:09.296 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9147999999999999e-05 2024-09-18T10:15:09.296 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.296 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.296 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:09.296 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.554322+0000", 2024-09-18T10:15:09.296 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3328999999999999e-05 2024-09-18T10:15:09.296 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.296 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.296 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:09.296 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.554337+0000", 2024-09-18T10:15:09.296 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5316e-05 2024-09-18T10:15:09.297 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.297 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.297 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:09.297 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.653990+0000", 2024-09-18T10:15:09.297 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.099652698999999997 2024-09-18T10:15:09.297 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.297 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.297 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:09.297 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.653998+0000", 2024-09-18T10:15:09.297 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1640000000000001e-06 2024-09-18T10:15:09.297 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.297 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.297 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:09.298 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.655081+0000", 2024-09-18T10:15:09.298 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001083297 2024-09-18T10:15:09.298 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.298 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.298 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:09.298 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.655110+0000", 2024-09-18T10:15:09.298 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8365e-05 2024-09-18T10:15:09.298 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:09.298 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:09.298 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:09.298 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.298 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.298 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.13s1 42/40 ECSubWrite(tid=21, reqid=osd.10.0:20, at_version=0'0, trim_to=0'0, roll_forward_to=41'18))", 2024-09-18T10:15:09.299 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:58.055945+0000", 2024-09-18T10:15:09.299 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 11.186185924, 2024-09-18T10:15:09.299 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.101978183, 2024-09-18T10:15:09.299 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:09.299 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:09.299 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:09.299 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.299 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:09.299 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:58.055945+0000", 2024-09-18T10:15:09.299 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:09.299 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.299 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.299 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:09.300 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:58.055945+0000", 2024-09-18T10:15:09.300 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:09.300 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.300 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.300 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:09.300 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:58.055943+0000", 2024-09-18T10:15:09.300 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:09.300 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.300 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.300 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:09.300 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:58.055969+0000", 2024-09-18T10:15:09.300 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6319e-05 2024-09-18T10:15:09.300 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.300 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.301 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:09.301 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:58.056014+0000", 2024-09-18T10:15:09.301 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4866e-05 2024-09-18T10:15:09.301 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.301 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.301 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:09.301 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:58.056031+0000", 2024-09-18T10:15:09.301 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6565000000000001e-05 2024-09-18T10:15:09.301 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.301 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.301 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:09.301 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:58.156261+0000", 2024-09-18T10:15:09.301 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100229871 2024-09-18T10:15:09.301 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.302 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.302 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:09.302 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:58.156276+0000", 2024-09-18T10:15:09.302 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5866e-05 2024-09-18T10:15:09.302 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.302 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.302 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:09.302 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:58.157852+0000", 2024-09-18T10:15:09.302 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0015757880000000001 2024-09-18T10:15:09.302 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.302 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.302 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:09.302 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:58.157923+0000", 2024-09-18T10:15:09.303 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0607999999999999e-05 2024-09-18T10:15:09.303 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:09.303 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:09.303 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:09.303 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.303 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.303 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.ds4 47/32 ECSubWrite(tid=55, reqid=client.4632.0:332, at_version=47'8, trim_to=0'0, roll_forward_to=41'6))", 2024-09-18T10:15:09.303 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:02.796736+0000", 2024-09-18T10:15:09.303 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 6.445394319, 2024-09-18T10:15:09.303 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.106482128, 2024-09-18T10:15:09.303 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:09.303 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:09.304 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:09.304 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.304 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:09.304 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.796736+0000", 2024-09-18T10:15:09.304 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:09.304 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.304 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.304 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:09.304 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.796736+0000", 2024-09-18T10:15:09.304 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:09.305 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.305 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.305 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:09.305 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.796735+0000", 2024-09-18T10:15:09.305 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:15:09.305 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.305 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.305 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:09.305 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.796958+0000", 2024-09-18T10:15:09.305 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00022249800000000001 2024-09-18T10:15:09.305 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.305 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.305 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:09.306 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.796983+0000", 2024-09-18T10:15:09.306 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5454000000000001e-05 2024-09-18T10:15:09.306 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.306 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.306 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:09.306 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.897088+0000", 2024-09-18T10:15:09.306 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10010422099999999 2024-09-18T10:15:09.306 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.306 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.306 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:09.306 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.900995+0000", 2024-09-18T10:15:09.306 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0039075589999999997 2024-09-18T10:15:09.306 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.307 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.307 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:09.307 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.901018+0000", 2024-09-18T10:15:09.307 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2592999999999999e-05 2024-09-18T10:15:09.307 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.307 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.307 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:09.307 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.903153+0000", 2024-09-18T10:15:09.307 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0021355300000000001 2024-09-18T10:15:09.307 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.307 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.307 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:09.307 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.903218+0000", 2024-09-18T10:15:09.308 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5135000000000006e-05 2024-09-18T10:15:09.308 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:09.308 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:09.308 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:09.308 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.308 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.308 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.ds4 47/32 ECSubWrite(tid=56, reqid=client.4632.0:333, at_version=47'9, trim_to=0'0, roll_forward_to=41'6))", 2024-09-18T10:15:09.308 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:02.897119+0000", 2024-09-18T10:15:09.308 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 6.3450112189999999, 2024-09-18T10:15:09.308 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.102697966, 2024-09-18T10:15:09.308 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:09.308 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:09.309 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:09.309 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.309 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:09.309 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.897119+0000", 2024-09-18T10:15:09.309 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:09.309 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.309 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.309 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:09.309 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.897119+0000", 2024-09-18T10:15:09.309 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:09.309 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.309 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.310 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:09.310 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.897118+0000", 2024-09-18T10:15:09.310 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:15:09.310 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.310 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.310 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:09.310 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.897266+0000", 2024-09-18T10:15:09.310 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014762799999999999 2024-09-18T10:15:09.310 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.310 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.310 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:09.310 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.897290+0000", 2024-09-18T10:15:09.310 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3835000000000001e-05 2024-09-18T10:15:09.310 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.311 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.311 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:09.311 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.997426+0000", 2024-09-18T10:15:09.311 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100136738 2024-09-18T10:15:09.311 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.311 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.311 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:09.311 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.997559+0000", 2024-09-18T10:15:09.311 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000133036 2024-09-18T10:15:09.311 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.311 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.311 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:09.311 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.997600+0000", 2024-09-18T10:15:09.311 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0324999999999998e-05 2024-09-18T10:15:09.311 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.312 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.312 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:09.312 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.999772+0000", 2024-09-18T10:15:09.312 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0021719740000000001 2024-09-18T10:15:09.312 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.312 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.312 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:09.312 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.999817+0000", 2024-09-18T10:15:09.312 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5575000000000003e-05 2024-09-18T10:15:09.312 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:09.312 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:09.312 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:09.313 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.313 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.313 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDScrubReserve(3.0s2 REQUEST e47)", 2024-09-18T10:15:09.313 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:03.036584+0000", 2024-09-18T10:15:09.313 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 6.2055465940000003, 2024-09-18T10:15:09.313 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100321234, 2024-09-18T10:15:09.313 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:09.313 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-18T10:15:09.313 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:09.313 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.313 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:09.313 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:03.036584+0000", 2024-09-18T10:15:09.313 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:09.313 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.313 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.314 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:09.314 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:03.036584+0000", 2024-09-18T10:15:09.314 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:09.314 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.314 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.314 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:09.314 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:03.036584+0000", 2024-09-18T10:15:09.314 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:09.314 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.314 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.314 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:09.314 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:03.036588+0000", 2024-09-18T10:15:09.314 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0570000000000003e-06 2024-09-18T10:15:09.314 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.315 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.315 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:09.315 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:03.036611+0000", 2024-09-18T10:15:09.315 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3688000000000001e-05 2024-09-18T10:15:09.315 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.315 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.315 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:09.315 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:03.136754+0000", 2024-09-18T10:15:09.315 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100142588 2024-09-18T10:15:09.315 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.315 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.315 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:09.315 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:03.136831+0000", 2024-09-18T10:15:09.316 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6852000000000002e-05 2024-09-18T10:15:09.316 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.316 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.316 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:09.316 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:03.136865+0000", 2024-09-18T10:15:09.316 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4551000000000002e-05 2024-09-18T10:15:09.316 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.316 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.316 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:09.316 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:03.136905+0000", 2024-09-18T10:15:09.316 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9869000000000001e-05 2024-09-18T10:15:09.316 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:09.316 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:09.316 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:09.317 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.317 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.317 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.ds4 47/32 ECSubWrite(tid=58, reqid=client.4632.0:334, at_version=0'0, trim_to=0'0, roll_forward_to=47'10))", 2024-09-18T10:15:09.317 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:03.205963+0000", 2024-09-18T10:15:09.317 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 6.0361673429999998, 2024-09-18T10:15:09.317 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10154212, 2024-09-18T10:15:09.317 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:09.317 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:09.317 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:09.317 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.317 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:09.317 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:03.205963+0000", 2024-09-18T10:15:09.317 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:09.317 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.318 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.318 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:09.318 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:03.205963+0000", 2024-09-18T10:15:09.318 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:09.318 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.318 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.318 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:09.318 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:03.205961+0000", 2024-09-18T10:15:09.318 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:09.318 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.318 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.318 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:09.318 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:03.205988+0000", 2024-09-18T10:15:09.318 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6305999999999999e-05 2024-09-18T10:15:09.318 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.319 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.319 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:09.319 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:03.206017+0000", 2024-09-18T10:15:09.319 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8994999999999999e-05 2024-09-18T10:15:09.319 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.319 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.319 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:09.319 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:03.206029+0000", 2024-09-18T10:15:09.319 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2106999999999999e-05 2024-09-18T10:15:09.319 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.319 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.319 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:09.319 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:03.306212+0000", 2024-09-18T10:15:09.319 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100182779 2024-09-18T10:15:09.320 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.320 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.320 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:09.320 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:03.306231+0000", 2024-09-18T10:15:09.320 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9910000000000001e-05 2024-09-18T10:15:09.320 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.320 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.320 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:09.320 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:03.307445+0000", 2024-09-18T10:15:09.320 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0012138120000000001 2024-09-18T10:15:09.320 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.320 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.320 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:09.320 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:03.307505+0000", 2024-09-18T10:15:09.321 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0047999999999998e-05 2024-09-18T10:15:09.321 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:09.321 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:09.321 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:09.321 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.321 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.321 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "replica_scrub(pg: 3.5s3,from:0'0,to:0'0,epoch:48/40,start:3:a0000000::::0,end:MAX,chunky:1,deep:0,version:9,allow_preemption:1,priority=5)", 2024-09-18T10:15:09.321 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:03.633524+0000", 2024-09-18T10:15:09.321 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.6086060670000002, 2024-09-18T10:15:09.321 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100433762, 2024-09-18T10:15:09.321 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:09.321 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-18T10:15:09.321 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:09.321 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.321 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:09.322 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:03.633524+0000", 2024-09-18T10:15:09.322 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:09.322 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.322 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.322 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:09.322 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:03.633524+0000", 2024-09-18T10:15:09.322 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:09.322 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.322 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.322 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:09.322 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:03.633524+0000", 2024-09-18T10:15:09.322 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:15:09.322 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.323 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.323 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:09.323 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:03.633550+0000", 2024-09-18T10:15:09.323 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.669e-05 2024-09-18T10:15:09.323 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.323 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.323 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:09.323 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:03.633583+0000", 2024-09-18T10:15:09.323 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3121999999999999e-05 2024-09-18T10:15:09.323 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.323 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.323 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:09.323 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:03.733732+0000", 2024-09-18T10:15:09.324 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100148444 2024-09-18T10:15:09.324 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.324 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.324 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:09.324 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:03.733855+0000", 2024-09-18T10:15:09.324 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012269000000000001 2024-09-18T10:15:09.324 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.324 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.324 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:09.324 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:03.733881+0000", 2024-09-18T10:15:09.324 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6072e-05 2024-09-18T10:15:09.324 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.324 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.324 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:09.325 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:03.733958+0000", 2024-09-18T10:15:09.325 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7571000000000007e-05 2024-09-18T10:15:09.325 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:09.325 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:09.325 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:09.325 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.325 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.325 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDPGPush(3.10s6 48/46 [PushOp(3:0a83d6b3:::smithi18925896-20:head, version: 45'6, data_included: [0~139264], data_size: 139264, omap_header_size: 0, omap_entries_size: 0, attrset_size: 4, recovery_info: ObjectRecoveryInfo(3:0a83d6b3:::smithi18925896-20:head@45'6, size: 557056, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: 1), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:557056, data_complete:true, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:true, error:false))])", 2024-09-18T10:15:09.325 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:03.727506+0000", 2024-09-18T10:15:09.325 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.51462436, 2024-09-18T10:15:09.325 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.101802727, 2024-09-18T10:15:09.325 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:09.325 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:09.326 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:09.326 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.326 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:09.326 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:03.727506+0000", 2024-09-18T10:15:09.326 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:09.326 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.326 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.326 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:09.326 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:03.727506+0000", 2024-09-18T10:15:09.326 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:09.326 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.326 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.326 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:09.327 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:03.727504+0000", 2024-09-18T10:15:09.327 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:09.327 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.327 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.327 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:09.327 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:03.727977+0000", 2024-09-18T10:15:09.327 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00047210600000000002 2024-09-18T10:15:09.327 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.327 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.327 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:09.327 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:03.728033+0000", 2024-09-18T10:15:09.327 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6161e-05 2024-09-18T10:15:09.327 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.328 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.328 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:09.328 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:03.728062+0000", 2024-09-18T10:15:09.328 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9275999999999999e-05 2024-09-18T10:15:09.328 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.328 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.328 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:09.328 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:03.828393+0000", 2024-09-18T10:15:09.328 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10033117499999999 2024-09-18T10:15:09.328 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.328 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.328 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:09.328 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:03.829309+0000", 2024-09-18T10:15:09.328 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00091576899999999998 2024-09-18T10:15:09.329 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:09.329 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:09.329 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:09.329 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.329 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.329 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDScrubReserve(3.15s3 RELEASE e52)", 2024-09-18T10:15:09.329 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:07.817321+0000", 2024-09-18T10:15:09.329 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.4248097829999999, 2024-09-18T10:15:09.329 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10048929700000001, 2024-09-18T10:15:09.329 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:09.329 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-18T10:15:09.329 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:09.330 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.330 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:09.330 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:07.817321+0000", 2024-09-18T10:15:09.330 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:09.330 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.330 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.330 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:09.330 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:07.817321+0000", 2024-09-18T10:15:09.330 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:09.330 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.330 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.330 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:09.330 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:07.817296+0000", 2024-09-18T10:15:09.330 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999752 2024-09-18T10:15:09.331 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.331 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.331 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:09.331 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:07.817359+0000", 2024-09-18T10:15:09.331 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2774999999999994e-05 2024-09-18T10:15:09.331 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.331 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.331 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:09.331 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:07.817394+0000", 2024-09-18T10:15:09.331 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5074999999999999e-05 2024-09-18T10:15:09.331 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.331 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.331 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:09.331 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:07.817411+0000", 2024-09-18T10:15:09.331 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6952999999999998e-05 2024-09-18T10:15:09.332 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.332 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.332 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:09.332 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:07.917685+0000", 2024-09-18T10:15:09.332 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10027429 2024-09-18T10:15:09.332 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.332 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.332 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:09.332 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:07.917734+0000", 2024-09-18T10:15:09.332 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8918000000000002e-05 2024-09-18T10:15:09.332 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:09.332 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:09.332 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:09.332 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:07.917810+0000", 2024-09-18T10:15:09.333 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5951999999999994e-05 2024-09-18T10:15:09.333 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:09.333 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:09.333 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:09.333 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:09.333 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:09.333 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-18T10:15:09.334 INFO:tasks.rados.rados.0.smithi189.stdout:163: finishing copy_from to smithi18925896-48 2024-09-18T10:15:09.334 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 48 v 8 (ObjNum 41 snap 0 seq_num 41) dirty exists 2024-09-18T10:15:09.335 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-18T10:15:09.335 INFO:teuthology.orchestra.run.smithi029.stdout:osd.6: {} 2024-09-18T10:15:09.335 INFO:teuthology.orchestra.run.smithi029.stdout:osd.7: {} 2024-09-18T10:15:09.335 INFO:teuthology.orchestra.run.smithi029.stdout:osd.8: {} 2024-09-18T10:15:09.336 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-18T10:15:09.330+0000 7f302f2c3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22187) UID: 0 2024-09-18T10:15:09.336 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_ops_in_flight 2024-09-18T10:15:09.351 INFO:teuthology.orchestra.run.smithi029.stdout:osd.9: {} 2024-09-18T10:15:09.374 INFO:teuthology.orchestra.run.smithi029.stdout:osd.10: {} 2024-09-18T10:15:09.397 INFO:teuthology.orchestra.run.smithi029.stdout:osd.11: {} 2024-09-18T10:15:09.418 INFO:teuthology.orchestra.run.smithi029.stdout:osd.12: {} 2024-09-18T10:15:09.435 INFO:tasks.ceph.osd.6.smithi154.stderr:2024-09-18T10:15:09.427+0000 7f0abc9e1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22089) UID: 0 2024-09-18T10:15:09.439 INFO:teuthology.orchestra.run.smithi029.stdout:osd.13: {} 2024-09-18T10:15:09.442 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:09.442 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:09.451 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_blocked_ops 2024-09-18T10:15:09.461 INFO:teuthology.orchestra.run.smithi029.stdout:osd.14: {} 2024-09-18T10:15:09.485 INFO:teuthology.orchestra.run.smithi029.stdout:osd.15: {} 2024-09-18T10:15:09.488 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T10:15:09.488 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T10:15:09.488 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T10:15:09.488 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T10:15:09.488 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T10:15:09.489 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T10:15:09.489 INFO:teuthology.orchestra.run.smithi029.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T10:15:09.489 INFO:teuthology.orchestra.run.smithi029.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T10:15:09.489 INFO:teuthology.orchestra.run.smithi029.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-18T10:15:09.489 INFO:teuthology.orchestra.run.smithi029.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-18T10:15:09.489 INFO:teuthology.orchestra.run.smithi029.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-18T10:15:09.489 INFO:teuthology.orchestra.run.smithi029.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-18T10:15:09.489 INFO:teuthology.orchestra.run.smithi029.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-18T10:15:09.489 INFO:teuthology.orchestra.run.smithi029.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-18T10:15:09.489 INFO:teuthology.orchestra.run.smithi029.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-18T10:15:09.489 INFO:teuthology.orchestra.run.smithi029.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-18T10:15:09.537 INFO:tasks.ceph.osd.4.smithi029.stderr:2024-09-18T10:15:09.534+0000 7f550cf6e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22189) UID: 0 2024-09-18T10:15:09.554 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:09.554 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:09.563 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_historic_ops 2024-09-18T10:15:09.638 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-18T10:15:09.637+0000 7f14c6a1a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 22138) UID: 0 2024-09-18T10:15:09.648 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-18T10:15:09.662 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:09.662 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:09.672 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T10:15:09.739 INFO:tasks.ceph.osd.15.smithi189.stderr:2024-09-18T10:15:09.735+0000 7fd0e725e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 22401) UID: 0 2024-09-18T10:15:09.781 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:09.781 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:09.791 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T10:15:09.815 INFO:tasks.rados.rados.0.smithi189.stdout:163: done (1 left) 2024-09-18T10:15:09.815 INFO:tasks.rados.rados.0.smithi189.stdout:165: done (0 left) 2024-09-18T10:15:09.815 INFO:tasks.rados.rados.0.smithi189.stdout:append oid 43 current snap is 11 2024-09-18T10:15:09.815 INFO:tasks.rados.rados.0.smithi189.stdout:166: seq_num 79 ranges {720896=573440,1294336=425984,1720320=344064} 2024-09-18T10:15:09.820 INFO:teuthology.orchestra.run.smithi029.stderr:reweighted osd.0 to 1 (10000) 2024-09-18T10:15:09.828 INFO:tasks.rados.rados.0.smithi189.stdout:166: writing smithi18925896-43 from 720896 to 1294336 tid 1 2024-09-18T10:15:09.831 INFO:tasks.rados.rados.0.smithi189.stdout:166: writing smithi18925896-43 from 1294336 to 1720320 tid 2 2024-09-18T10:15:09.833 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-18T10:15:09.833 INFO:tasks.rados.rados.0.smithi189.stdout:166: writing smithi18925896-43 from 1720320 to 2064384 tid 3 2024-09-18T10:15:09.833 INFO:tasks.rados.rados.0.smithi189.stdout:append oid 10 current snap is 11 2024-09-18T10:15:09.833 INFO:tasks.rados.rados.0.smithi189.stdout:167: seq_num 80 ranges {163840=589824,753664=573440,1327104=196608} 2024-09-18T10:15:09.837 INFO:tasks.rados.rados.0.smithi189.stdout:167: writing smithi18925896-10 from 163840 to 753664 tid 1 2024-09-18T10:15:09.839 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-18T10:15:09.837+0000 7f578c52f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22134) UID: 0 2024-09-18T10:15:09.840 INFO:tasks.rados.rados.0.smithi189.stdout:167: writing smithi18925896-10 from 753664 to 1327104 tid 2 2024-09-18T10:15:09.842 INFO:tasks.rados.rados.0.smithi189.stdout:167: writing smithi18925896-10 from 1327104 to 1523712 tid 3 2024-09-18T10:15:09.842 INFO:tasks.rados.rados.0.smithi189.stdout:168: snap_create 2024-09-18T10:15:09.890 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:09.890 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:09.899 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T10:15:09.940 INFO:tasks.ceph.osd.15.smithi189.stderr:2024-09-18T10:15:09.935+0000 7fd0e725e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 22401) UID: 0 2024-09-18T10:15:10.003 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:10.003 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:10.012 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T10:15:10.042 INFO:tasks.ceph.osd.8.smithi029.stderr:2024-09-18T10:15:10.038+0000 7f2555c27700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 22192) UID: 0 2024-09-18T10:15:10.113 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:10.113 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:10.122 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T10:15:10.138 INFO:tasks.rados.rados.0.smithi189.stdout:166: finishing write tid 1 to smithi18925896-43 2024-09-18T10:15:10.142 INFO:tasks.ceph.osd.12.smithi029.stderr:2024-09-18T10:15:10.138+0000 7f8e4e627700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 22190) UID: 0 2024-09-18T10:15:10.216 INFO:tasks.rados.rados.0.smithi189.stdout:166: finishing write tid 2 to smithi18925896-43 2024-09-18T10:15:10.224 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:10.224 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:10.233 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T10:15:10.238 INFO:tasks.rados.rados.0.smithi189.stdout:166: finishing write tid 3 to smithi18925896-43 2024-09-18T10:15:10.243 INFO:tasks.ceph.osd.12.smithi029.stderr:2024-09-18T10:15:10.238+0000 7f8e4e627700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 22190) UID: 0 2024-09-18T10:15:10.249 INFO:tasks.rados.rados.0.smithi189.stdout:167: finishing write tid 1 to smithi18925896-10 2024-09-18T10:15:10.252 INFO:tasks.rados.rados.0.smithi189.stdout:167: finishing write tid 2 to smithi18925896-10 2024-09-18T10:15:10.252 INFO:tasks.rados.rados.0.smithi189.stdout:167: finishing write tid 3 to smithi18925896-10 2024-09-18T10:15:10.329 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:10.329 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:10.338 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-09-18T10:15:10.339 INFO:tasks.rados.rados.0.smithi189.stdout:166: finishing write tid 4 to smithi18925896-43 2024-09-18T10:15:10.344 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-18T10:15:10.341+0000 7f578c52f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22134) UID: 0 2024-09-18T10:15:10.440 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:10.440 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:10.445 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-18T10:15:10.443+0000 7f14dcc5b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22398) UID: 0 2024-09-18T10:15:10.449 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-09-18T10:15:10.452 INFO:tasks.rados.rados.0.smithi189.stdout:167: finishing write tid 4 to smithi18925896-10 2024-09-18T10:15:10.546 INFO:tasks.ceph.osd.11.smithi189.stderr:2024-09-18T10:15:10.543+0000 7f718b60a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 22400) UID: 0 2024-09-18T10:15:10.552 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:10.553 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:10.562 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-09-18T10:15:10.642 INFO:tasks.rados.rados.0.smithi189.stdout:166: finishing write tid 5 to smithi18925896-43 2024-09-18T10:15:10.642 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 43 v 22 (ObjNum 79 snap 11 seq_num 79) dirty exists 2024-09-18T10:15:10.642 INFO:tasks.rados.rados.0.smithi189.stdout:166: left oid 43 (ObjNum 79 snap 11 seq_num 79) 2024-09-18T10:15:10.646 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-18T10:15:10.645+0000 7f00046a8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 22136) UID: 0 2024-09-18T10:15:10.657 INFO:tasks.rados.rados.0.smithi189.stdout:167: finishing write tid 5 to smithi18925896-10 2024-09-18T10:15:10.657 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 10 v 42 (ObjNum 80 snap 11 seq_num 80) dirty exists 2024-09-18T10:15:10.657 INFO:tasks.rados.rados.0.smithi189.stdout:167: left oid 10 (ObjNum 80 snap 11 seq_num 80) 2024-09-18T10:15:10.663 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:10.663 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:10.672 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_ops_in_flight 2024-09-18T10:15:10.748 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-18T10:15:10.745+0000 7f1981873700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22132) UID: 0 2024-09-18T10:15:10.777 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:10.777 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:10.784 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_blocked_ops 2024-09-18T10:15:10.824 INFO:tasks.rados.rados.0.smithi189.stdout:166: done (2 left) 2024-09-18T10:15:10.824 INFO:tasks.rados.rados.0.smithi189.stdout:167: done (1 left) 2024-09-18T10:15:10.824 INFO:tasks.rados.rados.0.smithi189.stdout:168: done (0 left) 2024-09-18T10:15:10.824 INFO:tasks.rados.rados.0.smithi189.stdout:169: snap_create 2024-09-18T10:15:10.829 INFO:teuthology.orchestra.run.smithi029.stderr:reweighted osd.1 to 1 (10000) 2024-09-18T10:15:10.830 INFO:teuthology.orchestra.run.smithi029.stderr:nodeep-scrub is set 2024-09-18T10:15:10.843 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-18T10:15:10.848 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-09-18T10:15:10.843+0000 7fdfa8698700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22087) UID: 0 2024-09-18T10:15:10.885 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:10.885 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:10.895 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_historic_ops 2024-09-18T10:15:10.949 INFO:tasks.ceph.osd.11.smithi189.stderr:2024-09-18T10:15:10.947+0000 7f718b60a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 22400) UID: 0 2024-09-18T10:15:11.041 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:11.042 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:11.050 INFO:tasks.ceph.osd.12.smithi029.stderr:2024-09-18T10:15:11.046+0000 7f8e4e627700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 22190) UID: 0 2024-09-18T10:15:11.051 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T10:15:11.152 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-18T10:15:11.149+0000 7f578c52f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22134) UID: 0 2024-09-18T10:15:11.160 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:11.160 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:11.170 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T10:15:11.253 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-18T10:15:11.249+0000 7f00046a8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 22136) UID: 0 2024-09-18T10:15:11.268 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:11.268 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:11.278 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T10:15:11.355 INFO:tasks.ceph.osd.14.smithi154.stderr:2024-09-18T10:15:11.351+0000 7f75a19e7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 22093) UID: 0 2024-09-18T10:15:11.377 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:11.377 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:11.386 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T10:15:11.456 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-18T10:15:11.451+0000 7f14dcc5b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22398) UID: 0 2024-09-18T10:15:11.486 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:11.486 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:11.495 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T10:15:11.501 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T10:15:11.558 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-18T10:15:11.557+0000 7f14c6a1a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 22138) UID: 0 2024-09-18T10:15:11.596 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:11.597 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:11.606 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T10:15:11.654 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-18T10:15:11.658 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-18T10:15:11.657+0000 7f578c52f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22134) UID: 0 2024-09-18T10:15:11.678 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-18T10:15:11.706 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-18T10:15:11.707 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:11.707 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:11.716 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-09-18T10:15:11.736 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-18T10:15:11.757 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-18T10:15:11.759 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-18T10:15:11.755+0000 7f14dcc5b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22398) UID: 0 2024-09-18T10:15:11.780 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-18T10:15:11.804 INFO:teuthology.orchestra.run.smithi029.stdout:osd.6: {} 2024-09-18T10:15:11.815 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:11.815 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:11.824 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-09-18T10:15:11.827 INFO:tasks.rados.rados.0.smithi189.stdout:169: done (0 left) 2024-09-18T10:15:11.827 INFO:tasks.rados.rados.0.smithi189.stdout:170: read oid 42 snap -1 2024-09-18T10:15:11.827 INFO:tasks.rados.rados.0.smithi189.stdout:170: expect (ObjNum 41 snap 0 seq_num 41) 2024-09-18T10:15:11.827 INFO:tasks.rados.rados.0.smithi189.stdout:171: rmattr oid 17 current snap is 13 2024-09-18T10:15:11.827 INFO:tasks.rados.rados.0.smithi189.stdout:171: done (1 left) 2024-09-18T10:15:11.827 INFO:tasks.rados.rados.0.smithi189.stdout:172: setattr oid 28 current snap is 13 2024-09-18T10:15:11.828 INFO:teuthology.orchestra.run.smithi029.stdout:osd.7: {} 2024-09-18T10:15:11.830 INFO:teuthology.orchestra.run.smithi029.stderr:reweighted osd.2 to 1 (10000) 2024-09-18T10:15:11.831 INFO:tasks.rados.rados.0.smithi189.stdout:173: copy_from oid 33 from oid 16 current snap is 13 2024-09-18T10:15:11.831 INFO:tasks.rados.rados.0.smithi189.stdout:174: setattr oid 32 current snap is 13 2024-09-18T10:15:11.834 INFO:tasks.rados.rados.0.smithi189.stdout:175: snap_remove snap 2 2024-09-18T10:15:11.844 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-18T10:15:11.852 INFO:teuthology.orchestra.run.smithi029.stdout:osd.8: {} 2024-09-18T10:15:11.861 INFO:tasks.ceph.osd.14.smithi154.stderr:2024-09-18T10:15:11.855+0000 7f75a19e7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 22093) UID: 0 2024-09-18T10:15:11.872 INFO:teuthology.orchestra.run.smithi029.stdout:osd.9: {} 2024-09-18T10:15:11.924 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:11.924 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:11.934 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-09-18T10:15:11.954 INFO:teuthology.orchestra.run.smithi029.stdout:osd.10: {} 2024-09-18T10:15:11.961 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-18T10:15:11.957+0000 7f578c52f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22134) UID: 0 2024-09-18T10:15:11.985 INFO:teuthology.orchestra.run.smithi029.stdout:osd.11: {} 2024-09-18T10:15:12.009 INFO:teuthology.orchestra.run.smithi029.stdout:osd.12: {} 2024-09-18T10:15:12.029 INFO:teuthology.orchestra.run.smithi029.stdout:osd.13: {} 2024-09-18T10:15:12.062 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-18T10:15:12.061+0000 7f14c6a1a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 22138) UID: 0 2024-09-18T10:15:12.083 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-18T10:15:12.083 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 20, 2024-09-18T10:15:12.083 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 600, 2024-09-18T10:15:12.083 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [ 2024-09-18T10:15:12.084 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.084 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:146 3.es0 3:78aad6f5:::smithi18925896-49:head [append 0~442368,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:12.084 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:51.117496+0000", 2024-09-18T10:15:12.084 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 20.962648323, 2024-09-18T10:15:12.084 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.61437150200000001, 2024-09-18T10:15:12.084 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:12.084 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:12.084 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:12.084 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:12.084 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:12.084 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 146 2024-09-18T10:15:12.084 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.084 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:12.085 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.085 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:12.085 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.117496+0000", 2024-09-18T10:15:12.085 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:12.085 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.085 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.085 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:12.085 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.117496+0000", 2024-09-18T10:15:12.085 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:12.085 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.086 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.086 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:12.086 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.117493+0000", 2024-09-18T10:15:12.086 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999976 2024-09-18T10:15:12.086 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.086 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.086 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:12.086 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.117787+0000", 2024-09-18T10:15:12.086 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00029378100000000001 2024-09-18T10:15:12.086 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.086 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.086 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:12.086 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.117820+0000", 2024-09-18T10:15:12.087 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3073e-05 2024-09-18T10:15:12.087 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.087 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.087 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:12.087 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.117834+0000", 2024-09-18T10:15:12.087 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3369e-05 2024-09-18T10:15:12.087 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.087 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.087 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:12.087 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.127090+0000", 2024-09-18T10:15:12.087 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0092563769999999997 2024-09-18T10:15:12.087 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.087 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.088 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:12.088 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.127435+0000", 2024-09-18T10:15:12.088 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00034522300000000003 2024-09-18T10:15:12.088 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.088 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.088 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:12.088 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.129115+0000", 2024-09-18T10:15:12.088 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0016792980000000001 2024-09-18T10:15:12.088 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.088 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.089 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:12.089 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.430505+0000", 2024-09-18T10:15:12.089 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.30139002300000001 2024-09-18T10:15:12.089 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.089 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.089 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:12.089 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.731753+0000", 2024-09-18T10:15:12.090 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.30124798800000002 2024-09-18T10:15:12.090 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.090 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.090 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:12.090 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.731867+0000", 2024-09-18T10:15:12.090 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000114764 2024-09-18T10:15:12.090 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:12.090 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:12.090 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:12.090 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.090 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.090 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:147 3.es0 3:78aad6f5:::smithi18925896-49:head [append 0~376832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:12.091 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:51.122092+0000", 2024-09-18T10:15:12.091 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 20.958051905000001, 2024-09-18T10:15:12.091 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.61047882200000003, 2024-09-18T10:15:12.091 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:12.091 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:12.091 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:12.091 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:12.091 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:12.091 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 147 2024-09-18T10:15:12.091 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.092 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:12.092 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.092 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:12.092 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122092+0000", 2024-09-18T10:15:12.092 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:12.092 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.092 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.092 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:12.093 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122092+0000", 2024-09-18T10:15:12.093 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:12.093 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.093 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.093 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:12.093 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122091+0000", 2024-09-18T10:15:12.093 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:12.093 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.093 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.093 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:12.093 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122504+0000", 2024-09-18T10:15:12.093 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00041322100000000002 2024-09-18T10:15:12.093 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.094 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.094 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:12.094 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122527+0000", 2024-09-18T10:15:12.094 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3504999999999999e-05 2024-09-18T10:15:12.094 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.094 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.094 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:12.094 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122537+0000", 2024-09-18T10:15:12.094 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.8460000000000003e-06 2024-09-18T10:15:12.094 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.094 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.094 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:12.094 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.327415+0000", 2024-09-18T10:15:12.095 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20487799700000001 2024-09-18T10:15:12.095 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.095 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.095 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:12.095 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.327614+0000", 2024-09-18T10:15:12.095 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00019855400000000001 2024-09-18T10:15:12.095 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.095 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.095 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:12.095 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.328920+0000", 2024-09-18T10:15:12.095 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001306263 2024-09-18T10:15:12.096 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.096 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.096 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:12.096 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.430570+0000", 2024-09-18T10:15:12.096 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.101650508 2024-09-18T10:15:12.096 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.096 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.096 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:12.096 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.732477+0000", 2024-09-18T10:15:12.096 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.30190700100000001 2024-09-18T10:15:12.096 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.096 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.097 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:12.097 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.732571+0000", 2024-09-18T10:15:12.097 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.3580999999999997e-05 2024-09-18T10:15:12.097 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:12.097 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:12.097 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:12.097 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.097 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.097 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:148 3.es0 3:78aad6f5:::smithi18925896-49:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:12.097 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:51.122547+0000", 2024-09-18T10:15:12.097 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 20.957596812999999, 2024-09-18T10:15:12.097 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.91462763400000002, 2024-09-18T10:15:12.097 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:12.097 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:12.098 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:12.098 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:12.098 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:12.098 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 148 2024-09-18T10:15:12.098 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.098 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:12.098 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.098 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:12.098 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122547+0000", 2024-09-18T10:15:12.098 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:12.098 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.098 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.098 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:12.098 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122547+0000", 2024-09-18T10:15:12.099 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:12.099 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.099 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.099 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:12.099 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122546+0000", 2024-09-18T10:15:12.099 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:15:12.100 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.100 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.100 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:12.100 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122552+0000", 2024-09-18T10:15:12.100 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3569999999999997e-06 2024-09-18T10:15:12.100 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.100 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.100 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:12.100 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122560+0000", 2024-09-18T10:15:12.100 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.191e-06 2024-09-18T10:15:12.100 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.100 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.100 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:12.101 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122569+0000", 2024-09-18T10:15:12.101 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.8230000000000001e-06 2024-09-18T10:15:12.101 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.101 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.101 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:12.101 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.834492+0000", 2024-09-18T10:15:12.101 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.71192345599999995 2024-09-18T10:15:12.101 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.101 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.101 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:12.101 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.834669+0000", 2024-09-18T10:15:12.101 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000177035 2024-09-18T10:15:12.101 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.102 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.102 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:12.102 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.835442+0000", 2024-09-18T10:15:12.102 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00077327799999999996 2024-09-18T10:15:12.102 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.102 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.102 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:12.102 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.936574+0000", 2024-09-18T10:15:12.102 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.101131163 2024-09-18T10:15:12.102 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.102 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.102 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:12.102 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.037028+0000", 2024-09-18T10:15:12.102 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100454026 2024-09-18T10:15:12.103 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.103 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.103 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:12.103 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.037175+0000", 2024-09-18T10:15:12.103 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000147305 2024-09-18T10:15:12.103 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:12.103 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:12.103 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:12.103 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.103 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.103 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:149 3.es0 3:78aad6f5:::smithi18925896-49:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:12.104 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:51.122575+0000", 2024-09-18T10:15:12.104 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 20.957569605, 2024-09-18T10:15:12.104 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.016813872, 2024-09-18T10:15:12.104 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:12.104 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:15:12.104 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:12.104 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:12.104 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:12.104 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 149 2024-09-18T10:15:12.104 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.104 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:12.104 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.105 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:12.105 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122575+0000", 2024-09-18T10:15:12.105 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:12.105 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.105 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.105 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:12.105 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122575+0000", 2024-09-18T10:15:12.105 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:12.105 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.105 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.105 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:12.105 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122574+0000", 2024-09-18T10:15:12.105 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:15:12.105 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.106 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.106 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:12.106 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122579+0000", 2024-09-18T10:15:12.106 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3650000000000003e-06 2024-09-18T10:15:12.106 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.106 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.106 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:12.106 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122587+0000", 2024-09-18T10:15:12.106 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.3000000000000002e-06 2024-09-18T10:15:12.106 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.106 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.106 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:12.106 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122592+0000", 2024-09-18T10:15:12.107 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9080000000000003e-06 2024-09-18T10:15:12.107 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.107 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.107 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:12.107 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.935931+0000", 2024-09-18T10:15:12.107 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.81333878199999998 2024-09-18T10:15:12.107 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.107 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.107 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for rw locks", 2024-09-18T10:15:12.107 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.936216+0000", 2024-09-18T10:15:12.107 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00028463400000000003 2024-09-18T10:15:12.107 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.107 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.107 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:12.108 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.038007+0000", 2024-09-18T10:15:12.108 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.101791539 2024-09-18T10:15:12.108 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.108 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.108 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:12.108 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.038254+0000", 2024-09-18T10:15:12.108 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000246932 2024-09-18T10:15:12.108 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.108 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.108 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:12.108 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.139388+0000", 2024-09-18T10:15:12.108 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10113417199999999 2024-09-18T10:15:12.109 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:12.110 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:12.110 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:12.110 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.110 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.110 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.es0 36/32 ECSubWriteReply(tid=2, last_complete=36'2, committed=1, applied=1))", 2024-09-18T10:15:12.110 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:51.126519+0000", 2024-09-18T10:15:12.110 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 20.953624942000001, 2024-09-18T10:15:12.110 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.30398819900000001, 2024-09-18T10:15:12.110 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:12.110 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:12.110 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:12.110 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.110 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:12.110 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.126519+0000", 2024-09-18T10:15:12.111 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:12.111 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.111 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.111 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:12.111 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.126519+0000", 2024-09-18T10:15:12.111 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:12.111 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.111 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.111 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:12.111 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.126518+0000", 2024-09-18T10:15:12.111 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:15:12.111 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.111 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.111 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:12.112 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.126523+0000", 2024-09-18T10:15:12.112 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5680000000000001e-06 2024-09-18T10:15:12.112 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.112 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.112 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:12.112 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.126541+0000", 2024-09-18T10:15:12.112 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7890999999999999e-05 2024-09-18T10:15:12.112 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.112 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.112 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:12.112 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.226614+0000", 2024-09-18T10:15:12.112 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100072972 2024-09-18T10:15:12.112 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.112 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.113 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:12.113 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.430483+0000", 2024-09-18T10:15:12.113 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.203869508 2024-09-18T10:15:12.113 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.113 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.113 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:12.113 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.430507+0000", 2024-09-18T10:15:12.113 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3986e-05 2024-09-18T10:15:12.113 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:12.113 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:12.113 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:12.113 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.113 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.113 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpRead(3.es0 36/32 ECSubRead(tid=3, to_read={3:7255ddf4:::smithi18925896-39:head=0,4096,0}, subchunks={3:7255ddf4:::smithi18925896-39:head=[0,1]}, attrs_to_read=))", 2024-09-18T10:15:12.114 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:51.126838+0000", 2024-09-18T10:15:12.114 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 20.953306181999999, 2024-09-18T10:15:12.114 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.707585825, 2024-09-18T10:15:12.114 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:12.114 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:12.114 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:12.114 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.114 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:12.114 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.126838+0000", 2024-09-18T10:15:12.115 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:12.115 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.115 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.115 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:12.115 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.126838+0000", 2024-09-18T10:15:12.115 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:12.115 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.115 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.115 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:12.115 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.126838+0000", 2024-09-18T10:15:12.115 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:12.115 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.115 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.115 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:12.116 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.126838+0000", 2024-09-18T10:15:12.116 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:12.116 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.116 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.116 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:12.116 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.127004+0000", 2024-09-18T10:15:12.116 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00016626499999999999 2024-09-18T10:15:12.116 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.116 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.116 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:12.116 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.227100+0000", 2024-09-18T10:15:12.116 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.1000955 2024-09-18T10:15:12.116 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.117 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.117 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:12.117 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.834083+0000", 2024-09-18T10:15:12.117 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.60698300100000002 2024-09-18T10:15:12.117 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.117 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.117 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:12.117 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.834424+0000", 2024-09-18T10:15:12.117 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000341059 2024-09-18T10:15:12.117 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:12.117 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:12.117 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:12.117 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.117 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.118 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpReadReply(3.es0 36/32 ECSubReadReply(tid=3, attrs_read=0))", 2024-09-18T10:15:12.118 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:51.214895+0000", 2024-09-18T10:15:12.118 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 20.865249505000001, 2024-09-18T10:15:12.118 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.61857800100000004, 2024-09-18T10:15:12.118 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:12.118 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:12.118 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:12.118 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.118 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:12.118 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.214895+0000", 2024-09-18T10:15:12.118 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:12.118 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.118 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.118 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:12.119 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.214895+0000", 2024-09-18T10:15:12.119 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:12.119 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.119 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.119 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:12.119 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.214893+0000", 2024-09-18T10:15:12.119 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:12.120 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.120 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.120 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:12.120 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.214927+0000", 2024-09-18T10:15:12.120 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3841999999999999e-05 2024-09-18T10:15:12.120 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.120 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.120 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:12.121 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.214953+0000", 2024-09-18T10:15:12.121 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.6519000000000001e-05 2024-09-18T10:15:12.121 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.121 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.121 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:12.121 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.214971+0000", 2024-09-18T10:15:12.121 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7535999999999999e-05 2024-09-18T10:15:12.121 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.121 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.122 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:12.122 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.833397+0000", 2024-09-18T10:15:12.122 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.61842642599999997 2024-09-18T10:15:12.122 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.122 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.122 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:12.122 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.833473+0000", 2024-09-18T10:15:12.122 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.5621e-05 2024-09-18T10:15:12.122 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:12.122 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:12.122 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:12.122 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.122 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.123 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.es0 36/32 ECSubWriteReply(tid=2, last_complete=36'2, committed=1, applied=1))", 2024-09-18T10:15:12.123 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:51.226907+0000", 2024-09-18T10:15:12.123 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 20.853236682999999, 2024-09-18T10:15:12.123 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.303948039, 2024-09-18T10:15:12.123 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:12.123 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:12.123 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:12.123 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.123 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:12.123 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.226907+0000", 2024-09-18T10:15:12.123 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:12.123 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.123 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.123 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:12.124 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.226907+0000", 2024-09-18T10:15:12.124 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:12.124 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.124 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.124 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:12.124 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.226907+0000", 2024-09-18T10:15:12.124 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:15:12.124 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.124 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.124 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:12.124 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.226912+0000", 2024-09-18T10:15:12.124 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3499999999999996e-06 2024-09-18T10:15:12.124 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.124 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.125 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:12.125 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.226925+0000", 2024-09-18T10:15:12.125 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3346000000000001e-05 2024-09-18T10:15:12.125 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.125 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.125 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:12.125 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.226931+0000", 2024-09-18T10:15:12.125 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.6729999999999997e-06 2024-09-18T10:15:12.125 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.125 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.125 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:12.125 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.530834+0000", 2024-09-18T10:15:12.125 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.30390318799999999 2024-09-18T10:15:12.126 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.126 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.126 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:12.126 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.530856+0000", 2024-09-18T10:15:12.126 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1209e-05 2024-09-18T10:15:12.126 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:12.126 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:12.126 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:12.126 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.126 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.126 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpReadReply(3.es0 36/32 ECSubReadReply(tid=3, attrs_read=0))", 2024-09-18T10:15:12.126 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:51.227596+0000", 2024-09-18T10:15:12.126 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 20.852547953999998, 2024-09-18T10:15:12.126 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.60597977400000003, 2024-09-18T10:15:12.127 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:12.127 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:12.127 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:12.127 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.127 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:12.127 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.227596+0000", 2024-09-18T10:15:12.127 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:12.127 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.127 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.127 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:12.127 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.227596+0000", 2024-09-18T10:15:12.127 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:12.127 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.127 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.128 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:12.128 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.227595+0000", 2024-09-18T10:15:12.128 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:15:12.128 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.128 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.128 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:12.128 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.227605+0000", 2024-09-18T10:15:12.128 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.7040000000000006e-06 2024-09-18T10:15:12.128 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.128 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.128 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:12.128 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.227621+0000", 2024-09-18T10:15:12.128 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6107999999999999e-05 2024-09-18T10:15:12.128 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.129 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.129 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:12.129 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.227637+0000", 2024-09-18T10:15:12.129 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5282999999999999e-05 2024-09-18T10:15:12.129 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.129 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.129 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:12.129 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.833511+0000", 2024-09-18T10:15:12.129 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.60587403699999998 2024-09-18T10:15:12.129 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.129 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.129 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:12.129 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.833576+0000", 2024-09-18T10:15:12.129 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.5392e-05 2024-09-18T10:15:12.130 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:12.130 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:12.130 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:12.130 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.130 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.130 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.es0 36/32 ECSubWriteReply(tid=4, last_complete=36'3, committed=1, applied=1))", 2024-09-18T10:15:12.130 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:51.228604+0000", 2024-09-18T10:15:12.130 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 20.851540126, 2024-09-18T10:15:12.130 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.30219310700000002, 2024-09-18T10:15:12.131 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:12.131 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:12.131 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:12.131 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.131 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:12.131 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.228604+0000", 2024-09-18T10:15:12.131 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:12.131 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.131 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.131 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:12.131 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.228604+0000", 2024-09-18T10:15:12.131 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:12.131 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.131 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.132 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:12.132 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.228589+0000", 2024-09-18T10:15:12.132 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999852 2024-09-18T10:15:12.132 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.132 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.132 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:12.132 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.228610+0000", 2024-09-18T10:15:12.132 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0676999999999999e-05 2024-09-18T10:15:12.132 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.132 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.132 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:12.132 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.228621+0000", 2024-09-18T10:15:12.132 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1154e-05 2024-09-18T10:15:12.133 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.133 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.133 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:12.133 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.228628+0000", 2024-09-18T10:15:12.134 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.4250000000000003e-06 2024-09-18T10:15:12.134 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.134 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.134 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:12.134 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.530770+0000", 2024-09-18T10:15:12.134 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.30214249799999998 2024-09-18T10:15:12.134 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.134 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.134 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:12.134 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.530797+0000", 2024-09-18T10:15:12.134 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7039e-05 2024-09-18T10:15:12.134 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:12.134 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:12.135 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:12.135 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.135 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.135 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.es0 36/32 ECSubWriteReply(tid=4, last_complete=36'3, committed=1, applied=1))", 2024-09-18T10:15:12.135 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:51.233227+0000", 2024-09-18T10:15:12.135 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 20.84691694, 2024-09-18T10:15:12.135 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.39820466900000001, 2024-09-18T10:15:12.135 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:12.135 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:12.135 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:12.135 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.135 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:12.135 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.233227+0000", 2024-09-18T10:15:12.135 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:12.136 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.136 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.136 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:12.136 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.233227+0000", 2024-09-18T10:15:12.136 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:12.136 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.136 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.136 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:12.136 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.233226+0000", 2024-09-18T10:15:12.136 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:15:12.136 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.136 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.136 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:12.136 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.233232+0000", 2024-09-18T10:15:12.137 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9370000000000001e-06 2024-09-18T10:15:12.137 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.137 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.137 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:12.137 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.233248+0000", 2024-09-18T10:15:12.137 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5574999999999999e-05 2024-09-18T10:15:12.137 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.137 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.137 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:12.137 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.233254+0000", 2024-09-18T10:15:12.137 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.3659999999999997e-06 2024-09-18T10:15:12.138 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.138 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.138 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:12.138 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.631420+0000", 2024-09-18T10:15:12.138 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.39816591699999998 2024-09-18T10:15:12.138 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.138 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.138 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:12.138 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.631432+0000", 2024-09-18T10:15:12.138 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1737e-05 2024-09-18T10:15:12.138 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:12.138 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:12.139 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:12.139 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.139 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.139 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.es0 36/32 ECSubWriteReply(tid=4, last_complete=36'3, committed=1, applied=1))", 2024-09-18T10:15:12.139 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:51.315941+0000", 2024-09-18T10:15:12.139 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 20.764202697999998, 2024-09-18T10:15:12.139 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.315333435, 2024-09-18T10:15:12.139 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:12.140 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:12.141 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:12.141 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.141 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:12.141 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.315941+0000", 2024-09-18T10:15:12.141 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:12.141 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.141 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.141 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:12.141 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.315941+0000", 2024-09-18T10:15:12.141 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:12.141 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.141 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.142 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:12.142 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.315940+0000", 2024-09-18T10:15:12.142 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:15:12.142 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.142 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.142 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:12.142 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.315950+0000", 2024-09-18T10:15:12.142 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.2329999999999995e-06 2024-09-18T10:15:12.142 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.142 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.142 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:12.142 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.315968+0000", 2024-09-18T10:15:12.142 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.8067000000000002e-05 2024-09-18T10:15:12.143 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.143 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.143 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:12.143 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.315977+0000", 2024-09-18T10:15:12.143 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.6420000000000002e-06 2024-09-18T10:15:12.143 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.143 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.143 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:12.143 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.631262+0000", 2024-09-18T10:15:12.143 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.31528474000000001 2024-09-18T10:15:12.143 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.144 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.144 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:12.144 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.631275+0000", 2024-09-18T10:15:12.144 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2799999999999999e-05 2024-09-18T10:15:12.144 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:12.144 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:12.145 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:12.145 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.145 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.145 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.es0 36/32 ECSubWriteReply(tid=4, last_complete=36'3, committed=1, applied=1))", 2024-09-18T10:15:12.145 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:51.329637+0000", 2024-09-18T10:15:12.145 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 20.750506828999999, 2024-09-18T10:15:12.145 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.30156535499999998, 2024-09-18T10:15:12.145 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:12.145 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:12.145 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:12.145 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.145 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:12.145 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.329637+0000", 2024-09-18T10:15:12.146 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:12.146 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.146 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.146 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:12.146 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.329637+0000", 2024-09-18T10:15:12.146 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:12.146 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.146 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.146 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:12.146 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.329636+0000", 2024-09-18T10:15:12.146 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:12.146 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.146 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.146 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:12.147 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.329648+0000", 2024-09-18T10:15:12.147 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2612000000000001e-05 2024-09-18T10:15:12.147 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.147 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.147 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:12.147 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.329678+0000", 2024-09-18T10:15:12.147 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.9369e-05 2024-09-18T10:15:12.147 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.147 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.147 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:12.147 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.329706+0000", 2024-09-18T10:15:12.147 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.8073e-05 2024-09-18T10:15:12.147 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.147 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.148 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:12.148 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.631181+0000", 2024-09-18T10:15:12.148 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.30147523799999998 2024-09-18T10:15:12.148 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.148 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.148 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:12.148 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.631203+0000", 2024-09-18T10:15:12.148 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1795999999999999e-05 2024-09-18T10:15:12.148 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:12.149 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:12.149 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:12.149 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.149 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.149 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.es0 36/32 ECSubWriteReply(tid=5, last_complete=36'4, committed=1, applied=1))", 2024-09-18T10:15:12.149 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:51.331048+0000", 2024-09-18T10:15:12.149 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 20.749096513000001, 2024-09-18T10:15:12.150 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40123123900000002, 2024-09-18T10:15:12.150 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:12.150 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:12.150 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:12.150 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.150 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:12.150 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.331048+0000", 2024-09-18T10:15:12.150 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:12.150 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.150 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.150 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:12.150 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.331048+0000", 2024-09-18T10:15:12.150 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:12.150 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.151 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.151 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:12.151 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.331046+0000", 2024-09-18T10:15:12.151 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:12.151 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.151 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.151 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:12.151 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.331057+0000", 2024-09-18T10:15:12.151 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1683e-05 2024-09-18T10:15:12.151 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.151 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.151 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:12.151 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.331090+0000", 2024-09-18T10:15:12.151 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2576000000000001e-05 2024-09-18T10:15:12.152 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.152 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.152 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:12.152 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.331107+0000", 2024-09-18T10:15:12.152 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7382999999999999e-05 2024-09-18T10:15:12.152 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.152 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.152 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:12.152 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.732241+0000", 2024-09-18T10:15:12.152 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40113375200000001 2024-09-18T10:15:12.152 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.152 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.153 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:12.153 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.732279+0000", 2024-09-18T10:15:12.153 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7719000000000003e-05 2024-09-18T10:15:12.153 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:12.153 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:12.153 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:12.153 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.153 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.153 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.es0 36/32 ECSubWriteReply(tid=5, last_complete=36'4, committed=1, applied=1))", 2024-09-18T10:15:12.153 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:51.331214+0000", 2024-09-18T10:15:12.153 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 20.748930119000001, 2024-09-18T10:15:12.153 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40084109800000001, 2024-09-18T10:15:12.153 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:12.153 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:12.154 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:12.154 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.154 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:12.154 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.331214+0000", 2024-09-18T10:15:12.154 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:12.154 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.154 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.154 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:12.154 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.331214+0000", 2024-09-18T10:15:12.154 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:12.154 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.154 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.154 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:12.154 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.331213+0000", 2024-09-18T10:15:12.155 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:15:12.155 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.155 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.155 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:12.155 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.331220+0000", 2024-09-18T10:15:12.155 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.1369999999999996e-06 2024-09-18T10:15:12.155 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.155 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.155 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:12.155 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.331235+0000", 2024-09-18T10:15:12.155 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4849e-05 2024-09-18T10:15:12.155 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.155 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.155 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:12.156 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.331243+0000", 2024-09-18T10:15:12.156 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.157e-06 2024-09-18T10:15:12.156 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.156 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.156 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:12.156 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.732010+0000", 2024-09-18T10:15:12.156 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40076711300000001 2024-09-18T10:15:12.156 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.156 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.156 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:12.156 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.732055+0000", 2024-09-18T10:15:12.156 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4821999999999999e-05 2024-09-18T10:15:12.156 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:12.157 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:12.157 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:12.157 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.157 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.157 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:346 3.es0 3:7255ddf4:::smithi18925896-39:head [append 0~770048] snapc b=[b,4] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-18T10:15:12.157 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:02.898102+0000", 2024-09-18T10:15:12.157 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 9.1820426570000002, 2024-09-18T10:15:12.157 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.31123798899999999, 2024-09-18T10:15:12.157 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:12.157 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:12.157 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:12.157 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:12.157 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:12.157 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 346 2024-09-18T10:15:12.158 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.158 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:12.158 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.158 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:12.158 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.898102+0000", 2024-09-18T10:15:12.158 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:12.158 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.158 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.158 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:12.158 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.898102+0000", 2024-09-18T10:15:12.159 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:12.159 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.159 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.159 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:12.159 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.898099+0000", 2024-09-18T10:15:12.159 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999971 2024-09-18T10:15:12.159 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.159 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.159 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:12.160 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.898914+0000", 2024-09-18T10:15:12.160 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00081561999999999997 2024-09-18T10:15:12.160 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.160 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.160 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:12.160 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.898942+0000", 2024-09-18T10:15:12.160 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.8019999999999999e-05 2024-09-18T10:15:12.160 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.160 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.160 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:12.160 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.898962+0000", 2024-09-18T10:15:12.160 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0026000000000001e-05 2024-09-18T10:15:12.160 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.161 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.161 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:12.161 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.902525+0000", 2024-09-18T10:15:12.161 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0035628090000000001 2024-09-18T10:15:12.161 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.161 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.161 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:12.161 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.902844+0000", 2024-09-18T10:15:12.161 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00031879400000000001 2024-09-18T10:15:12.161 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.161 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.161 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:12.161 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.905818+0000", 2024-09-18T10:15:12.161 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0029740600000000002 2024-09-18T10:15:12.162 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.162 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.162 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:12.162 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.008488+0000", 2024-09-18T10:15:12.162 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10266977100000001 2024-09-18T10:15:12.162 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.162 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.162 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:12.162 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.209150+0000", 2024-09-18T10:15:12.162 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20066203799999999 2024-09-18T10:15:12.162 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.162 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.162 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:12.162 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.209339+0000", 2024-09-18T10:15:12.162 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00018959099999999999 2024-09-18T10:15:12.163 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:12.163 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:12.163 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:12.163 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.163 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.163 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:347 3.es0 3:7255ddf4:::smithi18925896-39:head [append 0~376832] snapc b=[b,4] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-18T10:15:12.163 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:02.898980+0000", 2024-09-18T10:15:12.163 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 9.1811640949999997, 2024-09-18T10:15:12.163 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.50845237799999998, 2024-09-18T10:15:12.163 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:12.163 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:12.163 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:12.163 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:12.163 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:12.164 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 347 2024-09-18T10:15:12.164 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.164 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:12.164 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.164 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:12.164 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.898980+0000", 2024-09-18T10:15:12.164 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:12.164 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.164 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.164 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:12.164 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.898980+0000", 2024-09-18T10:15:12.164 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:12.164 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.164 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.165 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:12.165 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.898978+0000", 2024-09-18T10:15:12.165 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:12.165 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.165 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.165 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:12.165 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.899378+0000", 2024-09-18T10:15:12.165 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000399465 2024-09-18T10:15:12.165 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.165 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.165 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:12.165 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.899397+0000", 2024-09-18T10:15:12.165 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9219999999999999e-05 2024-09-18T10:15:12.165 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.166 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.166 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:12.166 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.899410+0000", 2024-09-18T10:15:12.166 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2486e-05 2024-09-18T10:15:12.166 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.166 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.166 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:12.166 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.103248+0000", 2024-09-18T10:15:12.166 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20383807200000001 2024-09-18T10:15:12.166 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.166 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.166 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:12.166 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.103447+0000", 2024-09-18T10:15:12.166 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00019970400000000001 2024-09-18T10:15:12.167 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.167 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.167 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:12.167 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.104894+0000", 2024-09-18T10:15:12.167 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001447083 2024-09-18T10:15:12.167 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.167 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.167 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:12.167 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.209378+0000", 2024-09-18T10:15:12.167 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.104483942 2024-09-18T10:15:12.167 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.167 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.167 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:12.167 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.407310+0000", 2024-09-18T10:15:12.168 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.197931406 2024-09-18T10:15:12.168 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.168 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.168 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:12.168 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.407432+0000", 2024-09-18T10:15:12.168 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00012266999999999999 2024-09-18T10:15:12.168 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:12.168 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:12.168 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:12.168 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.169 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.169 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:348 3.es0 3:7255ddf4:::smithi18925896-39:head [setxattr _header (56) in=7b] snapc b=[b,4] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-18T10:15:12.169 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:02.899423+0000", 2024-09-18T10:15:12.169 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 9.1807206669999992, 2024-09-18T10:15:12.169 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.50859120400000002, 2024-09-18T10:15:12.169 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:12.169 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:12.169 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:12.169 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:12.169 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:12.169 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 348 2024-09-18T10:15:12.169 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.169 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:12.170 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.170 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:12.170 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.899423+0000", 2024-09-18T10:15:12.170 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:12.170 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.170 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.170 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:12.170 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.899423+0000", 2024-09-18T10:15:12.170 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:12.170 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.170 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.170 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:12.170 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.899422+0000", 2024-09-18T10:15:12.170 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:12.171 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.171 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.171 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:12.171 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.899431+0000", 2024-09-18T10:15:12.171 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.5960000000000004e-06 2024-09-18T10:15:12.171 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.171 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.171 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:12.171 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.899451+0000", 2024-09-18T10:15:12.171 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0089999999999999e-05 2024-09-18T10:15:12.171 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.171 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.171 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:12.171 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.899461+0000", 2024-09-18T10:15:12.172 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0373e-05 2024-09-18T10:15:12.172 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.172 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.172 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:12.172 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.106191+0000", 2024-09-18T10:15:12.172 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20672991700000001 2024-09-18T10:15:12.172 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.172 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.172 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:12.172 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.106370+0000", 2024-09-18T10:15:12.172 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00017880500000000001 2024-09-18T10:15:12.172 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.172 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.173 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:12.173 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.107175+0000", 2024-09-18T10:15:12.173 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00080489699999999995 2024-09-18T10:15:12.173 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.173 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.173 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:12.173 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.209383+0000", 2024-09-18T10:15:12.173 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10220804999999999 2024-09-18T10:15:12.173 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.173 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.173 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:12.173 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.407801+0000", 2024-09-18T10:15:12.173 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.19841824199999999 2024-09-18T10:15:12.173 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.174 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.174 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:12.174 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.408015+0000", 2024-09-18T10:15:12.174 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00021364100000000001 2024-09-18T10:15:12.174 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:12.174 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:12.174 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:12.174 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.174 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.174 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:349 3.es0 3:7255ddf4:::smithi18925896-39:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e47)", 2024-09-18T10:15:12.174 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:02.899474+0000", 2024-09-18T10:15:12.174 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 9.1806704559999996, 2024-09-18T10:15:12.174 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.81294787599999996, 2024-09-18T10:15:12.174 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:12.175 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:15:12.175 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:12.175 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:12.175 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:12.175 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 349 2024-09-18T10:15:12.175 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.175 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:12.175 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.175 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:12.175 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.899474+0000", 2024-09-18T10:15:12.175 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:12.175 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.175 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.175 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:12.175 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.899474+0000", 2024-09-18T10:15:12.176 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:12.176 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.176 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.176 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:12.176 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.899472+0000", 2024-09-18T10:15:12.176 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:12.176 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.176 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.176 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:12.176 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.899480+0000", 2024-09-18T10:15:12.176 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.0129999999999993e-06 2024-09-18T10:15:12.176 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.176 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.176 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:12.177 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.899501+0000", 2024-09-18T10:15:12.177 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0959999999999999e-05 2024-09-18T10:15:12.177 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.177 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.177 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:12.177 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.899513+0000", 2024-09-18T10:15:12.177 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1736000000000001e-05 2024-09-18T10:15:12.177 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.177 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.177 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:12.177 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.208436+0000", 2024-09-18T10:15:12.177 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.30892317800000002 2024-09-18T10:15:12.177 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.177 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.178 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for rw locks", 2024-09-18T10:15:12.178 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.208762+0000", 2024-09-18T10:15:12.178 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00032566099999999998 2024-09-18T10:15:12.178 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.178 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.178 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:12.178 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.408786+0000", 2024-09-18T10:15:12.178 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.200024389 2024-09-18T10:15:12.178 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.178 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.178 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:12.178 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.409088+0000", 2024-09-18T10:15:12.178 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000301272 2024-09-18T10:15:12.178 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.178 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.179 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:12.179 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.712422+0000", 2024-09-18T10:15:12.179 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.303333937 2024-09-18T10:15:12.179 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:12.179 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:12.179 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:12.179 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.179 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.179 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.es0 47/32 ECSubWriteReply(tid=24, last_complete=47'19, committed=1, applied=1))", 2024-09-18T10:15:12.180 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:03.410052+0000", 2024-09-18T10:15:12.180 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 8.6700917680000007, 2024-09-18T10:15:12.180 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.30055966699999997, 2024-09-18T10:15:12.180 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:12.180 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:12.180 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:12.180 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.180 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:12.180 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.410052+0000", 2024-09-18T10:15:12.180 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:12.180 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.180 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.180 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:12.180 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.410052+0000", 2024-09-18T10:15:12.181 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:12.181 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.181 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.181 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:12.181 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.410051+0000", 2024-09-18T10:15:12.181 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:12.181 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.181 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.181 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:12.181 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.410063+0000", 2024-09-18T10:15:12.181 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2804e-05 2024-09-18T10:15:12.181 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.181 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.181 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:12.182 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.410098+0000", 2024-09-18T10:15:12.182 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4375000000000002e-05 2024-09-18T10:15:12.182 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.182 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.182 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:12.182 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.510200+0000", 2024-09-18T10:15:12.182 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100101738 2024-09-18T10:15:12.182 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.182 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.182 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:12.182 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.710585+0000", 2024-09-18T10:15:12.182 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20038515400000001 2024-09-18T10:15:12.182 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.182 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.183 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:12.183 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.710612+0000", 2024-09-18T10:15:12.183 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7333000000000001e-05 2024-09-18T10:15:12.183 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:12.183 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:12.183 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:12.183 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:12.183 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:12.183 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-18T10:15:12.184 INFO:teuthology.orchestra.run.smithi029.stdout:osd.14: {} 2024-09-18T10:15:12.185 INFO:teuthology.orchestra.run.smithi029.stdout:osd.15: {} 2024-09-18T10:15:12.185 INFO:tasks.ceph.osd.8.smithi029.stderr:2024-09-18T10:15:12.178+0000 7f2555c27700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 22192) UID: 0 2024-09-18T10:15:12.185 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_ops_in_flight 2024-09-18T10:15:12.186 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T10:15:12.186 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T10:15:12.186 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T10:15:12.186 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T10:15:12.186 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T10:15:12.186 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T10:15:12.187 INFO:teuthology.orchestra.run.smithi029.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T10:15:12.187 INFO:teuthology.orchestra.run.smithi029.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T10:15:12.187 INFO:teuthology.orchestra.run.smithi029.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-18T10:15:12.187 INFO:teuthology.orchestra.run.smithi029.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-18T10:15:12.187 INFO:teuthology.orchestra.run.smithi029.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-18T10:15:12.187 INFO:teuthology.orchestra.run.smithi029.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-18T10:15:12.187 INFO:teuthology.orchestra.run.smithi029.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-18T10:15:12.187 INFO:teuthology.orchestra.run.smithi029.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-18T10:15:12.187 INFO:teuthology.orchestra.run.smithi029.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-18T10:15:12.187 INFO:teuthology.orchestra.run.smithi029.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-18T10:15:12.285 INFO:tasks.ceph.osd.8.smithi029.stderr:2024-09-18T10:15:12.282+0000 7f2555c27700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 22192) UID: 0 2024-09-18T10:15:12.294 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-18T10:15:12.294 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-18T10:15:12.294 INFO:teuthology.orchestra.run.smithi189.stdout: "num_ops": 0 2024-09-18T10:15:12.294 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-18T10:15:12.304 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_blocked_ops 2024-09-18T10:15:12.386 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-18T10:15:12.385+0000 7f578c52f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22134) UID: 0 2024-09-18T10:15:12.402 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-18T10:15:12.402 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-18T10:15:12.402 INFO:teuthology.orchestra.run.smithi189.stdout: "complaint_time": 30, 2024-09-18T10:15:12.402 INFO:teuthology.orchestra.run.smithi189.stdout: "num_blocked_ops": 0 2024-09-18T10:15:12.402 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-18T10:15:12.411 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_historic_ops 2024-09-18T10:15:12.487 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-18T10:15:12.485+0000 7f578c52f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22134) UID: 0 2024-09-18T10:15:12.512 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-18T10:15:12.512 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 20, 2024-09-18T10:15:12.512 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 600, 2024-09-18T10:15:12.512 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [ 2024-09-18T10:15:12.512 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.512 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4257.0:19 1.0 e24/23 1:30306672:devicehealth::main.db.0000000000000000:head v 24'9, mlcod=24'7)", 2024-09-18T10:15:12.512 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:33.346192+0000", 2024-09-18T10:15:12.512 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 39.162383507000001, 2024-09-18T10:15:12.512 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20178458699999999, 2024-09-18T10:15:12.512 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:12.512 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:12.513 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:12.513 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.513 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:12.513 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:33.346192+0000", 2024-09-18T10:15:12.513 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:12.513 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.513 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.513 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:12.513 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:33.346192+0000", 2024-09-18T10:15:12.513 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:12.513 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.513 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.513 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:12.514 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:33.346190+0000", 2024-09-18T10:15:12.514 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:12.514 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.514 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.514 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:12.514 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:33.346206+0000", 2024-09-18T10:15:12.514 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6257e-05 2024-09-18T10:15:12.514 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.514 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.514 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:12.514 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:33.346237+0000", 2024-09-18T10:15:12.514 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0471999999999999e-05 2024-09-18T10:15:12.514 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.514 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.515 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:12.515 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:33.446377+0000", 2024-09-18T10:15:12.515 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10014044499999999 2024-09-18T10:15:12.515 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.515 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.515 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:12.515 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:33.546722+0000", 2024-09-18T10:15:12.515 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100344593 2024-09-18T10:15:12.515 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.515 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.515 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:12.515 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:33.546787+0000", 2024-09-18T10:15:12.515 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.5347999999999998e-05 2024-09-18T10:15:12.515 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.516 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.516 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:12.516 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:33.547891+0000", 2024-09-18T10:15:12.516 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001104379 2024-09-18T10:15:12.516 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.516 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.516 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:12.516 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:33.547976+0000", 2024-09-18T10:15:12.516 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.5023000000000002e-05 2024-09-18T10:15:12.516 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:12.516 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:12.516 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:12.516 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.516 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.517 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4257.0:48 1.0 e26/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 26'30, mlcod=26'22)", 2024-09-18T10:15:12.517 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:35.294144+0000", 2024-09-18T10:15:12.517 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 37.214431363999999, 2024-09-18T10:15:12.517 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20346468400000001, 2024-09-18T10:15:12.517 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:12.517 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:12.517 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:12.517 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.517 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:12.517 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:35.294144+0000", 2024-09-18T10:15:12.517 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:12.517 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.517 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.518 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:12.518 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:35.294144+0000", 2024-09-18T10:15:12.518 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:12.518 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.518 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.518 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:12.518 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:35.294142+0000", 2024-09-18T10:15:12.518 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:12.518 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.518 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.518 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:12.518 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:35.294156+0000", 2024-09-18T10:15:12.518 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4059e-05 2024-09-18T10:15:12.518 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.519 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.519 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:12.519 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:35.294209+0000", 2024-09-18T10:15:12.519 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.2164999999999999e-05 2024-09-18T10:15:12.519 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.519 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.519 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:12.519 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:35.394305+0000", 2024-09-18T10:15:12.519 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10009605000000001 2024-09-18T10:15:12.519 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.519 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.519 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:12.519 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:35.394422+0000", 2024-09-18T10:15:12.519 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000117093 2024-09-18T10:15:12.520 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.520 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.520 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:12.520 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:35.394460+0000", 2024-09-18T10:15:12.520 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8037999999999997e-05 2024-09-18T10:15:12.520 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.520 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.520 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:12.520 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:35.497437+0000", 2024-09-18T10:15:12.520 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.102977206 2024-09-18T10:15:12.520 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.520 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.520 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:12.521 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:35.497609+0000", 2024-09-18T10:15:12.521 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00017165299999999999 2024-09-18T10:15:12.521 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:12.521 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:12.521 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:12.521 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.521 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.521 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4257.0:96 1.0 e29/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 29'70, mlcod=29'68)", 2024-09-18T10:15:12.521 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:39.427232+0000", 2024-09-18T10:15:12.521 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 33.081343240000002, 2024-09-18T10:15:12.521 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20408627300000001, 2024-09-18T10:15:12.521 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:12.521 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:12.522 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:12.522 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.522 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:12.522 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.427232+0000", 2024-09-18T10:15:12.522 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:12.522 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.522 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.522 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:12.522 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.427232+0000", 2024-09-18T10:15:12.522 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:12.522 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.522 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.522 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:12.523 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.427230+0000", 2024-09-18T10:15:12.523 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:12.523 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.523 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.523 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:12.523 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.427399+0000", 2024-09-18T10:15:12.523 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000168787 2024-09-18T10:15:12.523 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.523 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.523 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:12.523 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.427431+0000", 2024-09-18T10:15:12.523 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2372000000000001e-05 2024-09-18T10:15:12.523 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.523 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.524 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:12.524 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.427444+0000", 2024-09-18T10:15:12.524 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2303000000000001e-05 2024-09-18T10:15:12.524 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.524 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.524 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:12.524 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.427580+0000", 2024-09-18T10:15:12.524 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000136019 2024-09-18T10:15:12.524 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.525 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.525 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:12.525 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.427622+0000", 2024-09-18T10:15:12.525 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.2551e-05 2024-09-18T10:15:12.525 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.525 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.525 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:12.525 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.631238+0000", 2024-09-18T10:15:12.525 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20361580100000001 2024-09-18T10:15:12.525 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.525 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.525 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:12.525 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.631318+0000", 2024-09-18T10:15:12.527 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.0370000000000005e-05 2024-09-18T10:15:12.527 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:12.527 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:12.527 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:12.527 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.527 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.527 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4257.0:97 1.0 e29/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 29'71, mlcod=29'68)", 2024-09-18T10:15:12.527 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:39.428800+0000", 2024-09-18T10:15:12.527 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 33.079775298000001, 2024-09-18T10:15:12.527 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.202566529, 2024-09-18T10:15:12.527 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:12.527 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:12.527 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:12.528 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.528 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:12.528 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.428800+0000", 2024-09-18T10:15:12.528 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:12.528 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.528 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.528 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:12.528 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.428800+0000", 2024-09-18T10:15:12.528 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:12.528 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.528 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.528 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:12.528 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.428798+0000", 2024-09-18T10:15:12.529 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:12.529 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.529 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.529 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:12.529 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.428814+0000", 2024-09-18T10:15:12.529 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5262999999999999e-05 2024-09-18T10:15:12.529 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.529 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.529 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:12.529 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.428836+0000", 2024-09-18T10:15:12.529 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.2096e-05 2024-09-18T10:15:12.529 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.529 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.530 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:12.530 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.428847+0000", 2024-09-18T10:15:12.530 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1272e-05 2024-09-18T10:15:12.530 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.530 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.530 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:12.530 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.529081+0000", 2024-09-18T10:15:12.530 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100233853 2024-09-18T10:15:12.530 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.530 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.530 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:12.530 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.529115+0000", 2024-09-18T10:15:12.530 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3862000000000002e-05 2024-09-18T10:15:12.530 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.531 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.531 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:12.531 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.631329+0000", 2024-09-18T10:15:12.531 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.102214465 2024-09-18T10:15:12.531 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.531 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.531 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:12.531 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.631367+0000", 2024-09-18T10:15:12.531 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7450999999999998e-05 2024-09-18T10:15:12.531 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:12.531 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:12.531 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:12.531 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.531 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.532 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4257.0:98 1.0 e29/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 29'72, mlcod=29'68)", 2024-09-18T10:15:12.532 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:39.429995+0000", 2024-09-18T10:15:12.532 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 33.078580074999998, 2024-09-18T10:15:12.532 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20141884500000001, 2024-09-18T10:15:12.532 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:12.532 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:12.532 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:12.532 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.532 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:12.532 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.429995+0000", 2024-09-18T10:15:12.532 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:12.532 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.532 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.533 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:12.533 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.429995+0000", 2024-09-18T10:15:12.533 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:12.533 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.533 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.533 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:12.533 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.429994+0000", 2024-09-18T10:15:12.533 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:12.533 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.533 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.533 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:12.533 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.430008+0000", 2024-09-18T10:15:12.533 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4185999999999999e-05 2024-09-18T10:15:12.534 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.534 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.534 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:12.534 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.430036+0000", 2024-09-18T10:15:12.534 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7878999999999999e-05 2024-09-18T10:15:12.534 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.534 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.534 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:12.534 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.430048+0000", 2024-09-18T10:15:12.534 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1725999999999999e-05 2024-09-18T10:15:12.534 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.534 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.534 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:12.534 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.629731+0000", 2024-09-18T10:15:12.535 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.19968341100000001 2024-09-18T10:15:12.535 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.535 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.535 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:12.535 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.629764+0000", 2024-09-18T10:15:12.535 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3572999999999999e-05 2024-09-18T10:15:12.535 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.535 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.535 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:12.535 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.631372+0000", 2024-09-18T10:15:12.535 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0016074279999999999 2024-09-18T10:15:12.535 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.535 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.535 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:12.536 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.631414+0000", 2024-09-18T10:15:12.536 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.2191999999999999e-05 2024-09-18T10:15:12.536 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:12.536 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:12.536 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:12.536 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.536 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.536 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4257.0:99 1.0 e29/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 29'73, mlcod=29'68)", 2024-09-18T10:15:12.536 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:39.431146+0000", 2024-09-18T10:15:12.536 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 33.077429350999999, 2024-09-18T10:15:12.536 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.200928938, 2024-09-18T10:15:12.536 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:12.536 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:12.537 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:12.537 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.537 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:12.537 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.431146+0000", 2024-09-18T10:15:12.537 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:12.537 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.537 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.537 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:12.537 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.431146+0000", 2024-09-18T10:15:12.537 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:12.537 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.537 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.537 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:12.537 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.431144+0000", 2024-09-18T10:15:12.538 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:12.538 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.538 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.538 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:12.538 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.431218+0000", 2024-09-18T10:15:12.538 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.3577000000000004e-05 2024-09-18T10:15:12.538 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.538 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.538 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:12.538 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.431250+0000", 2024-09-18T10:15:12.538 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2268999999999999e-05 2024-09-18T10:15:12.538 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.538 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.539 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:12.539 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.431262+0000", 2024-09-18T10:15:12.539 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2053e-05 2024-09-18T10:15:12.539 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.539 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.539 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:12.539 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.630328+0000", 2024-09-18T10:15:12.539 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.19906532900000001 2024-09-18T10:15:12.539 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.539 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.539 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:12.539 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.630359+0000", 2024-09-18T10:15:12.539 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1606000000000002e-05 2024-09-18T10:15:12.539 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.540 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.540 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:12.540 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.632015+0000", 2024-09-18T10:15:12.540 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0016557900000000001 2024-09-18T10:15:12.540 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.540 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.540 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:12.540 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.632075+0000", 2024-09-18T10:15:12.540 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9904000000000001e-05 2024-09-18T10:15:12.540 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:12.540 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:12.540 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:12.540 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.540 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.541 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4257.0:100 1.0 e29/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 29'74, mlcod=29'68)", 2024-09-18T10:15:12.541 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:39.432631+0000", 2024-09-18T10:15:12.541 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 33.075944262, 2024-09-18T10:15:12.541 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20014804899999999, 2024-09-18T10:15:12.541 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:12.541 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:12.541 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:12.541 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.541 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:12.541 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.432631+0000", 2024-09-18T10:15:12.541 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:12.541 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.541 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.542 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:12.542 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.432631+0000", 2024-09-18T10:15:12.542 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:12.542 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.542 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.542 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:12.542 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.432630+0000", 2024-09-18T10:15:12.542 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:12.542 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.542 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.542 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:12.542 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.432643+0000", 2024-09-18T10:15:12.542 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3525e-05 2024-09-18T10:15:12.543 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.543 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.543 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:12.543 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.432674+0000", 2024-09-18T10:15:12.543 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1183000000000003e-05 2024-09-18T10:15:12.543 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.543 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.543 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:12.543 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.432690+0000", 2024-09-18T10:15:12.543 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5665999999999999e-05 2024-09-18T10:15:12.543 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.543 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.543 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:12.544 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.631457+0000", 2024-09-18T10:15:12.544 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.19876732899999999 2024-09-18T10:15:12.544 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.544 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.544 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:12.544 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.631488+0000", 2024-09-18T10:15:12.544 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1059000000000002e-05 2024-09-18T10:15:12.544 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.544 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.544 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:12.544 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.632717+0000", 2024-09-18T10:15:12.544 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0012285219999999999 2024-09-18T10:15:12.544 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.544 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.545 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:12.545 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.632779+0000", 2024-09-18T10:15:12.545 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.2328000000000004e-05 2024-09-18T10:15:12.545 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:12.545 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:12.545 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:12.545 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.545 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.545 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4257.0:103 1.0 e29/23 1:30306672:devicehealth::main.db.0000000000000000:head v 29'77, mlcod=29'73)", 2024-09-18T10:15:12.545 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:40.036738+0000", 2024-09-18T10:15:12.545 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 32.471837292000004, 2024-09-18T10:15:12.545 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20132596899999999, 2024-09-18T10:15:12.545 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:12.546 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:12.546 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:12.546 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.546 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:12.546 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.036738+0000", 2024-09-18T10:15:12.546 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:12.546 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.546 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.546 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:12.546 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.036738+0000", 2024-09-18T10:15:12.546 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:12.546 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.546 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.546 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:12.547 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.036737+0000", 2024-09-18T10:15:12.547 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:12.547 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.547 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.547 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:12.547 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.036804+0000", 2024-09-18T10:15:12.547 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.7178e-05 2024-09-18T10:15:12.547 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.547 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.547 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:12.547 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.036821+0000", 2024-09-18T10:15:12.547 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7153e-05 2024-09-18T10:15:12.547 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.547 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.548 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:12.548 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.136954+0000", 2024-09-18T10:15:12.548 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100132572 2024-09-18T10:15:12.548 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.548 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.548 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:12.548 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.137090+0000", 2024-09-18T10:15:12.548 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000136332 2024-09-18T10:15:12.548 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.548 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.548 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:12.548 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.137124+0000", 2024-09-18T10:15:12.548 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3458999999999999e-05 2024-09-18T10:15:12.549 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.549 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.549 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:12.549 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.237999+0000", 2024-09-18T10:15:12.549 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100875385 2024-09-18T10:15:12.549 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.549 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.549 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:12.549 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.238064+0000", 2024-09-18T10:15:12.549 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.5086999999999995e-05 2024-09-18T10:15:12.549 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:12.549 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:12.549 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:12.549 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.550 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.550 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4257.0:113 1.0 e29/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 29'83, mlcod=29'80)", 2024-09-18T10:15:12.550 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:40.801786+0000", 2024-09-18T10:15:12.550 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 31.706789788000002, 2024-09-18T10:15:12.550 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.203408961, 2024-09-18T10:15:12.550 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:12.550 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:12.550 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:12.550 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.550 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:12.550 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.801786+0000", 2024-09-18T10:15:12.550 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:12.550 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.550 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.551 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:12.551 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.801786+0000", 2024-09-18T10:15:12.551 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:12.551 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.551 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.551 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:12.551 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.801784+0000", 2024-09-18T10:15:12.551 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:12.551 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.551 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.551 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:12.551 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.801801+0000", 2024-09-18T10:15:12.551 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7223000000000001e-05 2024-09-18T10:15:12.551 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.552 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.552 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:12.552 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.801823+0000", 2024-09-18T10:15:12.552 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1469000000000001e-05 2024-09-18T10:15:12.552 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.552 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.552 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:12.552 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.801835+0000", 2024-09-18T10:15:12.552 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2033e-05 2024-09-18T10:15:12.552 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.552 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.552 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:12.552 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.902049+0000", 2024-09-18T10:15:12.552 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100213659 2024-09-18T10:15:12.553 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.553 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.553 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:12.553 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.902082+0000", 2024-09-18T10:15:12.553 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3668999999999999e-05 2024-09-18T10:15:12.553 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.553 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.553 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:12.553 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.005131+0000", 2024-09-18T10:15:12.553 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.103048379 2024-09-18T10:15:12.553 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.553 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.553 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:12.553 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.005195+0000", 2024-09-18T10:15:12.553 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.3805000000000003e-05 2024-09-18T10:15:12.554 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:12.554 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:12.554 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:12.554 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.554 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.554 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4257.0:114 1.0 e29/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 29'84, mlcod=29'80)", 2024-09-18T10:15:12.554 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:40.802657+0000", 2024-09-18T10:15:12.554 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 31.705918654000001, 2024-09-18T10:15:12.554 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20256802600000001, 2024-09-18T10:15:12.554 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:12.554 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:12.554 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:12.554 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.554 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:12.555 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.802657+0000", 2024-09-18T10:15:12.555 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:12.555 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.555 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.555 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:12.555 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.802657+0000", 2024-09-18T10:15:12.555 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:12.555 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.555 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.555 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:12.555 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.802656+0000", 2024-09-18T10:15:12.555 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:15:12.555 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.555 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.555 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:12.556 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.802824+0000", 2024-09-18T10:15:12.556 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000168784 2024-09-18T10:15:12.556 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.556 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.556 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:12.556 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.802844+0000", 2024-09-18T10:15:12.556 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0058999999999999e-05 2024-09-18T10:15:12.556 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.556 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.556 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:12.556 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.802857+0000", 2024-09-18T10:15:12.556 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2163e-05 2024-09-18T10:15:12.556 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.557 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.557 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:12.557 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.902531+0000", 2024-09-18T10:15:12.557 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.099674591000000007 2024-09-18T10:15:12.557 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.557 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.557 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:12.557 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.902559+0000", 2024-09-18T10:15:12.557 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7483000000000001e-05 2024-09-18T10:15:12.557 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.557 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.557 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:12.557 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.005198+0000", 2024-09-18T10:15:12.557 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.102638853 2024-09-18T10:15:12.558 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.558 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.558 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:12.558 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.005225+0000", 2024-09-18T10:15:12.558 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7206e-05 2024-09-18T10:15:12.558 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:12.558 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:12.558 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:12.558 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.558 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.558 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4257.0:115 1.0 e29/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 29'85, mlcod=29'80)", 2024-09-18T10:15:12.558 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:40.803806+0000", 2024-09-18T10:15:12.558 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 31.704769267, 2024-09-18T10:15:12.558 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.201443711, 2024-09-18T10:15:12.559 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:12.559 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:12.559 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:12.559 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.559 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:12.559 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.803806+0000", 2024-09-18T10:15:12.559 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:12.559 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.559 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.559 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:12.559 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.803806+0000", 2024-09-18T10:15:12.559 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:12.559 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.559 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.559 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:12.560 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.803804+0000", 2024-09-18T10:15:12.560 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:12.560 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.560 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.560 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:12.560 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.803818+0000", 2024-09-18T10:15:12.560 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3477e-05 2024-09-18T10:15:12.560 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.560 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.560 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:12.560 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.803851+0000", 2024-09-18T10:15:12.560 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3086000000000001e-05 2024-09-18T10:15:12.560 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.560 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.561 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:12.561 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.803867+0000", 2024-09-18T10:15:12.561 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5885000000000001e-05 2024-09-18T10:15:12.561 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.561 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.561 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:12.561 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.903206+0000", 2024-09-18T10:15:12.561 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.099338990000000002 2024-09-18T10:15:12.561 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.561 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.561 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:12.561 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.903253+0000", 2024-09-18T10:15:12.561 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.7419000000000001e-05 2024-09-18T10:15:12.562 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.562 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.562 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:12.562 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.005227+0000", 2024-09-18T10:15:12.562 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.101973199 2024-09-18T10:15:12.562 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.562 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.562 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:12.562 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.005250+0000", 2024-09-18T10:15:12.562 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3261999999999999e-05 2024-09-18T10:15:12.562 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:12.562 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:12.562 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:12.562 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.562 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.563 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4257.0:116 1.0 e29/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 29'86, mlcod=29'80)", 2024-09-18T10:15:12.563 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:40.804730+0000", 2024-09-18T10:15:12.563 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 31.703845288, 2024-09-18T10:15:12.563 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20068456000000001, 2024-09-18T10:15:12.563 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:12.563 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:12.563 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:12.563 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.563 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:12.563 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.804730+0000", 2024-09-18T10:15:12.563 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:12.563 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.563 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.563 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:12.564 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.804730+0000", 2024-09-18T10:15:12.564 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:12.564 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.564 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.564 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:12.564 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.804729+0000", 2024-09-18T10:15:12.564 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:12.564 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.564 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.564 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:12.564 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.804739+0000", 2024-09-18T10:15:12.564 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0669000000000001e-05 2024-09-18T10:15:12.564 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.565 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.565 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:12.565 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.804762+0000", 2024-09-18T10:15:12.565 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3062999999999999e-05 2024-09-18T10:15:12.565 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.565 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.565 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:12.565 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.804776+0000", 2024-09-18T10:15:12.565 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3273e-05 2024-09-18T10:15:12.565 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.565 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.565 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:12.565 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.003889+0000", 2024-09-18T10:15:12.565 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.19911304099999999 2024-09-18T10:15:12.565 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.566 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.566 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:12.566 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.003922+0000", 2024-09-18T10:15:12.566 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3621999999999998e-05 2024-09-18T10:15:12.566 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.566 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.566 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:12.566 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.005347+0000", 2024-09-18T10:15:12.566 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0014251050000000001 2024-09-18T10:15:12.566 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.566 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.566 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:12.566 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.005415+0000", 2024-09-18T10:15:12.566 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.7154000000000001e-05 2024-09-18T10:15:12.567 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:12.567 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:12.567 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:12.567 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.567 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.567 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4257.0:117 1.0 e29/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 29'87, mlcod=29'80)", 2024-09-18T10:15:12.567 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:40.805727+0000", 2024-09-18T10:15:12.567 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 31.702848028999998, 2024-09-18T10:15:12.567 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.200946191, 2024-09-18T10:15:12.567 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:12.567 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:12.567 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:12.567 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.567 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:12.568 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.805727+0000", 2024-09-18T10:15:12.568 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:12.568 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.568 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.568 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:12.568 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.805727+0000", 2024-09-18T10:15:12.568 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:12.568 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.568 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.568 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:12.568 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.805726+0000", 2024-09-18T10:15:12.568 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:12.568 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.568 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.569 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:12.569 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.805783+0000", 2024-09-18T10:15:12.569 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.7068000000000003e-05 2024-09-18T10:15:12.569 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.569 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.569 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:12.569 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.805817+0000", 2024-09-18T10:15:12.569 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4471999999999998e-05 2024-09-18T10:15:12.569 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.569 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.569 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:12.569 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.805833+0000", 2024-09-18T10:15:12.569 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6313000000000001e-05 2024-09-18T10:15:12.569 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.570 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.570 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:12.570 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.004481+0000", 2024-09-18T10:15:12.570 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.198647823 2024-09-18T10:15:12.570 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.570 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.570 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:12.570 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.004520+0000", 2024-09-18T10:15:12.570 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8992000000000002e-05 2024-09-18T10:15:12.570 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.570 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.570 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:12.570 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.006585+0000", 2024-09-18T10:15:12.570 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0020647389999999999 2024-09-18T10:15:12.571 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.571 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.571 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:12.571 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.006673+0000", 2024-09-18T10:15:12.571 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.8443999999999995e-05 2024-09-18T10:15:12.571 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:12.571 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:12.571 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:12.571 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.571 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.571 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4257.0:128 1.0 e29/23 1:30306672:devicehealth::main.db.0000000000000000:head v 29'98, mlcod=29'95)", 2024-09-18T10:15:12.571 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:41.222806+0000", 2024-09-18T10:15:12.571 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 31.285769387999999, 2024-09-18T10:15:12.572 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20200015099999999, 2024-09-18T10:15:12.572 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:12.572 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:12.572 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:12.572 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.572 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:12.572 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.222806+0000", 2024-09-18T10:15:12.572 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:12.572 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.572 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.572 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:12.572 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.222806+0000", 2024-09-18T10:15:12.572 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:12.572 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.573 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.573 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:12.573 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.222804+0000", 2024-09-18T10:15:12.573 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:12.573 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.573 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.573 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:12.573 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.222885+0000", 2024-09-18T10:15:12.573 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.0140000000000002e-05 2024-09-18T10:15:12.573 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.573 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.573 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:12.573 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.222915+0000", 2024-09-18T10:15:12.573 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0168999999999999e-05 2024-09-18T10:15:12.574 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.574 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.574 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:12.574 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.323060+0000", 2024-09-18T10:15:12.574 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100145475 2024-09-18T10:15:12.574 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.574 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.574 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:12.574 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.423304+0000", 2024-09-18T10:15:12.574 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100243924 2024-09-18T10:15:12.574 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.574 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.574 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:12.574 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.423338+0000", 2024-09-18T10:15:12.574 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3819000000000003e-05 2024-09-18T10:15:12.575 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.575 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.575 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:12.575 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.424724+0000", 2024-09-18T10:15:12.575 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0013862690000000001 2024-09-18T10:15:12.575 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.575 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.575 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:12.575 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.424806+0000", 2024-09-18T10:15:12.575 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.1901000000000001e-05 2024-09-18T10:15:12.575 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:12.575 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:12.575 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:12.576 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.576 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.576 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.4s1 41/40 ECSubWrite(tid=17, reqid=osd.8.0:17, at_version=41'10, trim_to=0'0, roll_forward_to=0'0))", 2024-09-18T10:15:12.576 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:57.445149+0000", 2024-09-18T10:15:12.576 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 15.063426826000001, 2024-09-18T10:15:12.576 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20394425099999999, 2024-09-18T10:15:12.576 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:12.576 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:12.576 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:12.576 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.576 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:12.576 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.445149+0000", 2024-09-18T10:15:12.576 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:12.576 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.577 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.577 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:12.577 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.445149+0000", 2024-09-18T10:15:12.577 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:12.577 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.577 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.577 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:12.577 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.445147+0000", 2024-09-18T10:15:12.577 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:12.577 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.577 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.577 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:12.577 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.445221+0000", 2024-09-18T10:15:12.577 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.4140999999999994e-05 2024-09-18T10:15:12.578 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.578 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.578 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:12.578 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.445260+0000", 2024-09-18T10:15:12.578 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8998999999999997e-05 2024-09-18T10:15:12.578 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.578 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.578 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:12.578 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.545386+0000", 2024-09-18T10:15:12.578 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10012621300000001 2024-09-18T10:15:12.578 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.578 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.579 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:12.579 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.645963+0000", 2024-09-18T10:15:12.579 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10057662000000001 2024-09-18T10:15:12.579 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.579 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.579 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:12.579 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.645988+0000", 2024-09-18T10:15:12.579 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5449e-05 2024-09-18T10:15:12.579 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.579 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.579 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:12.579 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.649028+0000", 2024-09-18T10:15:12.579 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0030390840000000001 2024-09-18T10:15:12.579 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.580 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.580 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:12.580 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.649093+0000", 2024-09-18T10:15:12.580 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.5258000000000004e-05 2024-09-18T10:15:12.580 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:12.580 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:12.580 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:12.580 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.580 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.580 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.6s6 41/40 ECSubWrite(tid=31, reqid=client.4632.0:249, at_version=41'13, trim_to=0'0, roll_forward_to=0'0))", 2024-09-18T10:15:12.580 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:57.550336+0000", 2024-09-18T10:15:12.580 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 14.958239787, 2024-09-18T10:15:12.580 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.203590574, 2024-09-18T10:15:12.580 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:12.581 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:12.581 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:12.581 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.581 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:12.581 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.550336+0000", 2024-09-18T10:15:12.581 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:12.581 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.581 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.581 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:12.581 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.550336+0000", 2024-09-18T10:15:12.581 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:12.581 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.581 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.581 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:12.582 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.550329+0000", 2024-09-18T10:15:12.582 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999933 2024-09-18T10:15:12.582 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.582 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.582 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:12.582 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.550932+0000", 2024-09-18T10:15:12.582 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00060276999999999998 2024-09-18T10:15:12.582 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.582 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.582 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:12.582 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.550955+0000", 2024-09-18T10:15:12.582 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.2884999999999999e-05 2024-09-18T10:15:12.582 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.582 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.582 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:12.583 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.651052+0000", 2024-09-18T10:15:12.583 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100097174 2024-09-18T10:15:12.583 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.583 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.583 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:12.583 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.651160+0000", 2024-09-18T10:15:12.583 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000108483 2024-09-18T10:15:12.583 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.583 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.583 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:12.584 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.651172+0000", 2024-09-18T10:15:12.584 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1790999999999999e-05 2024-09-18T10:15:12.584 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.584 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.584 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:12.584 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.753875+0000", 2024-09-18T10:15:12.584 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10270337 2024-09-18T10:15:12.584 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.584 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.584 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:12.584 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.753926+0000", 2024-09-18T10:15:12.584 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0767999999999999e-05 2024-09-18T10:15:12.584 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:12.584 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:12.585 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:12.585 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.585 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.585 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.6s6 41/40 ECSubWrite(tid=33, reqid=client.4632.0:251, at_version=41'15, trim_to=0'0, roll_forward_to=0'0))", 2024-09-18T10:15:12.585 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:57.651390+0000", 2024-09-18T10:15:12.585 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 14.857185175, 2024-09-18T10:15:12.585 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.105666251, 2024-09-18T10:15:12.585 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:12.585 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:12.585 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:12.585 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.585 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:12.585 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.651390+0000", 2024-09-18T10:15:12.586 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:12.586 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.586 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.586 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:12.586 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.651390+0000", 2024-09-18T10:15:12.586 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:12.586 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.586 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.586 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:12.586 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.651390+0000", 2024-09-18T10:15:12.586 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:12.586 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.586 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.586 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:12.587 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.651405+0000", 2024-09-18T10:15:12.587 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5512e-05 2024-09-18T10:15:12.587 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.587 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.587 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:12.587 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.651413+0000", 2024-09-18T10:15:12.587 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.9249999999999995e-06 2024-09-18T10:15:12.587 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.587 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.587 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:12.587 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.651418+0000", 2024-09-18T10:15:12.587 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0799999999999996e-06 2024-09-18T10:15:12.587 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.587 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.588 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:12.588 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.753439+0000", 2024-09-18T10:15:12.588 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.102020576 2024-09-18T10:15:12.588 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.588 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.588 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:12.588 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.753470+0000", 2024-09-18T10:15:12.588 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1296e-05 2024-09-18T10:15:12.588 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.588 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.588 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:12.588 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.757028+0000", 2024-09-18T10:15:12.588 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0035574510000000001 2024-09-18T10:15:12.588 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.589 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.589 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:12.589 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.757056+0000", 2024-09-18T10:15:12.589 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.8894000000000002e-05 2024-09-18T10:15:12.589 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:12.589 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:12.589 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:12.589 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.589 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.589 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.8s4 42/40 ECSubWrite(tid=133, reqid=osd.6.0:132, at_version=0'0, trim_to=0'0, roll_forward_to=41'13))", 2024-09-18T10:15:12.589 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:58.049763+0000", 2024-09-18T10:15:12.589 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 14.458812051000001, 2024-09-18T10:15:12.589 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.21446268399999999, 2024-09-18T10:15:12.589 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:12.590 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:12.590 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:12.590 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.590 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:12.590 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:58.049763+0000", 2024-09-18T10:15:12.590 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:12.590 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.590 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.590 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:12.590 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:58.049763+0000", 2024-09-18T10:15:12.590 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:12.590 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.590 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.590 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:12.591 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:58.049763+0000", 2024-09-18T10:15:12.591 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:12.591 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.591 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.591 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:12.591 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:58.049787+0000", 2024-09-18T10:15:12.591 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3819e-05 2024-09-18T10:15:12.591 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.591 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.591 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:12.591 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:58.049817+0000", 2024-09-18T10:15:12.591 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0749e-05 2024-09-18T10:15:12.591 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.591 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.592 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:12.592 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:58.049836+0000", 2024-09-18T10:15:12.592 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9182999999999999e-05 2024-09-18T10:15:12.592 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.592 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.592 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:12.592 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:58.163409+0000", 2024-09-18T10:15:12.592 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.11357204899999999 2024-09-18T10:15:12.592 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.592 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.592 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:12.592 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:58.163424+0000", 2024-09-18T10:15:12.592 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5403000000000001e-05 2024-09-18T10:15:12.592 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.593 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.593 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:12.593 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:58.264161+0000", 2024-09-18T10:15:12.593 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100736942 2024-09-18T10:15:12.593 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.593 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.593 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:12.593 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:58.264226+0000", 2024-09-18T10:15:12.593 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.5085000000000005e-05 2024-09-18T10:15:12.593 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:12.593 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:12.593 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:12.593 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.593 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.594 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.ds3 47/32 ECSubWrite(tid=56, reqid=client.4632.0:333, at_version=47'9, trim_to=0'0, roll_forward_to=41'6))", 2024-09-18T10:15:12.594 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:02.801124+0000", 2024-09-18T10:15:12.594 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 9.7074508930000007, 2024-09-18T10:15:12.594 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20185083000000001, 2024-09-18T10:15:12.594 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:12.594 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:12.594 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:12.594 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.594 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:12.594 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.801124+0000", 2024-09-18T10:15:12.594 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:12.594 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.594 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.594 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:12.595 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.801124+0000", 2024-09-18T10:15:12.595 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:12.595 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.595 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.595 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:12.595 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.801103+0000", 2024-09-18T10:15:12.595 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999785 2024-09-18T10:15:12.595 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.595 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.595 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:12.595 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.801366+0000", 2024-09-18T10:15:12.595 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00026269299999999999 2024-09-18T10:15:12.595 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.595 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.596 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:12.596 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.801395+0000", 2024-09-18T10:15:12.596 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.9572000000000001e-05 2024-09-18T10:15:12.596 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.596 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.596 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:12.596 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.801412+0000", 2024-09-18T10:15:12.596 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6872999999999999e-05 2024-09-18T10:15:12.596 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.596 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.596 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:12.596 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.000543+0000", 2024-09-18T10:15:12.596 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.19913046700000001 2024-09-18T10:15:12.596 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.597 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.597 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:12.597 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.000579+0000", 2024-09-18T10:15:12.597 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6649000000000001e-05 2024-09-18T10:15:12.597 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.597 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.597 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:12.597 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.002922+0000", 2024-09-18T10:15:12.597 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0023426139999999998 2024-09-18T10:15:12.597 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.597 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.597 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:12.597 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.002975+0000", 2024-09-18T10:15:12.597 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3232000000000001e-05 2024-09-18T10:15:12.598 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:12.598 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:12.598 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:12.598 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.598 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.598 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.ds3 47/32 ECSubWrite(tid=57, reqid=client.4632.0:334, at_version=47'10, trim_to=0'0, roll_forward_to=41'6))", 2024-09-18T10:15:12.598 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:02.803374+0000", 2024-09-18T10:15:12.598 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 9.7052009810000008, 2024-09-18T10:15:12.598 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20018619500000001, 2024-09-18T10:15:12.598 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:12.598 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:12.598 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:12.598 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.598 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:12.599 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.803374+0000", 2024-09-18T10:15:12.599 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:12.599 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.599 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.599 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:12.599 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.803374+0000", 2024-09-18T10:15:12.599 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:12.599 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.599 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.599 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:12.599 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.803373+0000", 2024-09-18T10:15:12.599 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:15:12.599 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.599 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.600 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:12.600 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.803399+0000", 2024-09-18T10:15:12.600 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5738e-05 2024-09-18T10:15:12.600 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.600 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.600 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:12.600 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.803419+0000", 2024-09-18T10:15:12.600 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9890000000000001e-05 2024-09-18T10:15:12.600 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.600 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.600 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:12.600 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.803430+0000", 2024-09-18T10:15:12.600 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.117e-05 2024-09-18T10:15:12.600 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.601 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.601 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:12.601 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.001843+0000", 2024-09-18T10:15:12.601 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.198412951 2024-09-18T10:15:12.601 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.601 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.601 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:12.601 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.001860+0000", 2024-09-18T10:15:12.601 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.658e-05 2024-09-18T10:15:12.601 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.601 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.601 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:12.601 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.003525+0000", 2024-09-18T10:15:12.601 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0016650160000000001 2024-09-18T10:15:12.602 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:12.602 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:12.602 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:12.602 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.003561+0000", 2024-09-18T10:15:12.602 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.5886000000000002e-05 2024-09-18T10:15:12.602 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:12.602 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:12.602 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:12.602 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:12.602 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:12.602 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-18T10:15:12.604 INFO:tasks.ceph.osd.15.smithi189.stderr:2024-09-18T10:15:12.603+0000 7fd0e725e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 22401) UID: 0 2024-09-18T10:15:12.604 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T10:15:12.705 INFO:tasks.ceph.osd.3.smithi189.stderr:2024-09-18T10:15:12.703+0000 7f2440fc7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22396) UID: 0 2024-09-18T10:15:12.717 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-18T10:15:12.717 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-18T10:15:12.717 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 0 2024-09-18T10:15:12.717 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-18T10:15:12.727 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T10:15:12.806 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-18T10:15:12.803+0000 7f14dcc5b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22398) UID: 0 2024-09-18T10:15:12.833 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-18T10:15:12.833 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-18T10:15:12.833 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-18T10:15:12.833 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-18T10:15:12.833 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-18T10:15:12.841 INFO:tasks.rados.rados.0.smithi189.stdout:175: done (4 left) 2024-09-18T10:15:12.841 INFO:tasks.rados.rados.0.smithi189.stdout:append oid 25 current snap is 13 2024-09-18T10:15:12.841 INFO:tasks.rados.rados.0.smithi189.stdout:176: seq_num 81 ranges {1261568=606208,1867776=720896,2588672=49152} 2024-09-18T10:15:12.844 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T10:15:12.844 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-18T10:15:12.846 INFO:teuthology.orchestra.run.smithi029.stderr:reweighted osd.3 to 1 (10000) 2024-09-18T10:15:12.857 INFO:tasks.rados.rados.0.smithi189.stdout:176: writing smithi18925896-25 from 1261568 to 1867776 tid 1 2024-09-18T10:15:12.858 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-18T10:15:12.861 INFO:tasks.rados.rados.0.smithi189.stdout:176: writing smithi18925896-25 from 1867776 to 2588672 tid 2 2024-09-18T10:15:12.862 INFO:tasks.rados.rados.0.smithi189.stdout:176: writing smithi18925896-25 from 2588672 to 2637824 tid 3 2024-09-18T10:15:12.862 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 28 v 14 (ObjNum 33 snap 0 seq_num 33) dirty exists 2024-09-18T10:15:12.862 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 32 v 15 (ObjNum 36 snap 0 seq_num 36) dirty exists 2024-09-18T10:15:12.862 INFO:tasks.rados.rados.0.smithi189.stdout:170: expect (ObjNum 41 snap 0 seq_num 41) 2024-09-18T10:15:12.882 INFO:tasks.rados.rados.0.smithi189.stdout:173: finishing copy_from racing read to smithi18925896-33 2024-09-18T10:15:12.883 INFO:tasks.rados.rados.0.smithi189.stdout:173: finishing copy_from to smithi18925896-33 2024-09-18T10:15:12.883 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 33 v 2 (ObjNum 15 snap 0 seq_num 15) dirty exists 2024-09-18T10:15:12.883 INFO:tasks.rados.rados.0.smithi189.stdout:170: done (4 left) 2024-09-18T10:15:12.883 INFO:tasks.rados.rados.0.smithi189.stdout:172: done (3 left) 2024-09-18T10:15:12.883 INFO:tasks.rados.rados.0.smithi189.stdout:173: done (2 left) 2024-09-18T10:15:12.883 INFO:tasks.rados.rados.0.smithi189.stdout:174: done (1 left) 2024-09-18T10:15:12.884 INFO:tasks.rados.rados.0.smithi189.stdout:append oid 45 current snap is 13 2024-09-18T10:15:12.884 INFO:tasks.rados.rados.0.smithi189.stdout:177: seq_num 82 ranges {753664=622592,1376256=475136,1851392=294912} 2024-09-18T10:15:12.892 INFO:tasks.rados.rados.0.smithi189.stdout:177: writing smithi18925896-45 from 753664 to 1376256 tid 1 2024-09-18T10:15:12.896 INFO:tasks.rados.rados.0.smithi189.stdout:177: writing smithi18925896-45 from 1376256 to 1851392 tid 2 2024-09-18T10:15:12.899 INFO:tasks.rados.rados.0.smithi189.stdout:177: writing smithi18925896-45 from 1851392 to 2146304 tid 3 2024-09-18T10:15:12.899 INFO:tasks.rados.rados.0.smithi189.stdout:178: snap_remove snap 12 2024-09-18T10:15:12.906 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-18T10:15:12.905+0000 7f578c52f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22134) UID: 0 2024-09-18T10:15:12.943 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-18T10:15:12.943 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-18T10:15:12.943 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-18T10:15:12.943 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-18T10:15:12.943 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.943 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWriteReply(3.6s0 36/32 ECSubWriteReply(tid=11, last_complete=36'7, committed=1, applied=1))", 2024-09-18T10:15:12.943 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:50.290945+0000", 2024-09-18T10:15:12.943 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 22.64865653, 2024-09-18T10:15:12.943 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.208748928, 2024-09-18T10:15:12.943 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:12.944 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:12.944 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:12.944 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.944 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:12.944 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.290945+0000", 2024-09-18T10:15:12.944 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:12.944 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:12.944 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.944 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:12.944 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.290945+0000", 2024-09-18T10:15:12.944 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:12.944 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:12.944 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.944 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:12.945 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.290945+0000", 2024-09-18T10:15:12.945 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:12.945 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:12.945 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.945 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:12.945 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.290951+0000", 2024-09-18T10:15:12.945 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.5729999999999996e-06 2024-09-18T10:15:12.945 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:12.945 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.945 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:12.945 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.290964+0000", 2024-09-18T10:15:12.945 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3012e-05 2024-09-18T10:15:12.945 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:12.946 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.946 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:12.946 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.290972+0000", 2024-09-18T10:15:12.946 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4699999999999996e-06 2024-09-18T10:15:12.946 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:12.946 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.946 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:12.946 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.499661+0000", 2024-09-18T10:15:12.946 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.20868911500000001 2024-09-18T10:15:12.946 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:12.946 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.946 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:12.946 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.499694+0000", 2024-09-18T10:15:12.947 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.3470000000000003e-05 2024-09-18T10:15:12.947 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:12.947 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:12.947 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:12.947 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:12.947 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.947 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWriteReply(3.6s0 36/32 ECSubWriteReply(tid=11, last_complete=36'7, committed=1, applied=1))", 2024-09-18T10:15:12.947 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:50.291030+0000", 2024-09-18T10:15:12.947 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 22.648572031, 2024-09-18T10:15:12.947 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.208840265, 2024-09-18T10:15:12.947 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:12.947 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:12.947 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:12.947 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.948 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:12.948 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.291030+0000", 2024-09-18T10:15:12.948 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:12.948 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:12.948 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.948 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:12.948 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.291030+0000", 2024-09-18T10:15:12.948 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:12.948 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:12.948 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.948 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:12.948 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.291030+0000", 2024-09-18T10:15:12.948 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:12.948 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:12.949 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.949 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:12.949 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.291032+0000", 2024-09-18T10:15:12.949 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.604e-06 2024-09-18T10:15:12.949 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:12.949 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.949 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:12.949 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.291038+0000", 2024-09-18T10:15:12.949 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8170000000000004e-06 2024-09-18T10:15:12.949 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:12.949 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.949 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:12.949 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.291042+0000", 2024-09-18T10:15:12.950 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.839e-06 2024-09-18T10:15:12.950 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:12.950 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.950 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:12.950 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.499812+0000", 2024-09-18T10:15:12.950 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.208769805 2024-09-18T10:15:12.950 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:12.950 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.950 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:12.950 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.499870+0000", 2024-09-18T10:15:12.950 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8449000000000002e-05 2024-09-18T10:15:12.950 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:12.950 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:12.950 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:12.951 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:12.951 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.951 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWriteReply(3.6s0 36/32 ECSubWriteReply(tid=11, last_complete=36'7, committed=1, applied=1))", 2024-09-18T10:15:12.951 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:50.302109+0000", 2024-09-18T10:15:12.951 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 22.637492871999999, 2024-09-18T10:15:12.951 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.29912391399999999, 2024-09-18T10:15:12.951 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:12.951 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:12.951 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:12.951 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.951 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:12.951 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.302109+0000", 2024-09-18T10:15:12.951 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:12.951 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:12.952 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.952 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:12.952 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.302109+0000", 2024-09-18T10:15:12.952 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:12.952 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:12.952 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.952 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:12.952 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.302107+0000", 2024-09-18T10:15:12.952 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:12.952 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:12.952 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.952 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:12.952 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.302119+0000", 2024-09-18T10:15:12.952 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1207e-05 2024-09-18T10:15:12.953 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:12.953 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.953 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:12.953 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.302150+0000", 2024-09-18T10:15:12.953 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.133e-05 2024-09-18T10:15:12.953 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:12.953 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.953 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:12.953 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.302164+0000", 2024-09-18T10:15:12.953 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3942e-05 2024-09-18T10:15:12.953 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:12.953 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.953 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:12.954 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.600015+0000", 2024-09-18T10:15:12.954 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.29785072000000001 2024-09-18T10:15:12.954 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:12.954 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.954 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:12.954 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.601233+0000", 2024-09-18T10:15:12.954 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0012183599999999999 2024-09-18T10:15:12.954 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:12.954 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:12.954 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:12.954 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:12.954 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.954 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpReadReply(3.6s0 36/32 ECSubReadReply(tid=10, attrs_read=0))", 2024-09-18T10:15:12.954 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:50.302179+0000", 2024-09-18T10:15:12.955 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 22.637422806, 2024-09-18T10:15:12.955 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.30043391800000002, 2024-09-18T10:15:12.955 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:12.955 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:12.955 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:12.955 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.955 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:12.955 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.302179+0000", 2024-09-18T10:15:12.955 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:12.955 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:12.955 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.955 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:12.955 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.302179+0000", 2024-09-18T10:15:12.955 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:12.956 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:12.956 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.956 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:12.956 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.302178+0000", 2024-09-18T10:15:12.956 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:12.956 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:12.956 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.956 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:12.956 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.302201+0000", 2024-09-18T10:15:12.956 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2824e-05 2024-09-18T10:15:12.956 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:12.956 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.956 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:12.956 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.302217+0000", 2024-09-18T10:15:12.957 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5993000000000001e-05 2024-09-18T10:15:12.957 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:12.957 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.957 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:12.957 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.302223+0000", 2024-09-18T10:15:12.957 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0820000000000004e-06 2024-09-18T10:15:12.957 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:12.957 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.957 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:12.957 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.602454+0000", 2024-09-18T10:15:12.957 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.30023072499999998 2024-09-18T10:15:12.958 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:12.958 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.958 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:12.958 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.602613+0000", 2024-09-18T10:15:12.958 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00015896399999999999 2024-09-18T10:15:12.958 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:12.958 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:12.958 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:12.958 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:12.958 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.959 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpRead(3.7s3 36/32 ECSubRead(tid=22, to_read={3:ee962131:::smithi18925896-29:head=0,4096,0}, subchunks={3:ee962131:::smithi18925896-29:head=[0,1]}, attrs_to_read=))", 2024-09-18T10:15:12.959 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:50.497231+0000", 2024-09-18T10:15:12.959 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 22.442370989, 2024-09-18T10:15:12.959 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.20067855700000001, 2024-09-18T10:15:12.959 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:12.959 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:12.959 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:12.959 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.959 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:12.959 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.497231+0000", 2024-09-18T10:15:12.959 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:12.959 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:12.959 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.959 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:12.960 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.497231+0000", 2024-09-18T10:15:12.960 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:12.960 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:12.960 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.960 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:12.960 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.497229+0000", 2024-09-18T10:15:12.960 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:12.960 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:12.960 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.960 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:12.960 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.497252+0000", 2024-09-18T10:15:12.960 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3193e-05 2024-09-18T10:15:12.960 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:12.961 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.961 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:12.961 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.497287+0000", 2024-09-18T10:15:12.961 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.5040000000000003e-05 2024-09-18T10:15:12.961 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:12.961 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.961 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:12.961 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.597372+0000", 2024-09-18T10:15:12.961 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10008500200000001 2024-09-18T10:15:12.961 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:12.961 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.961 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:12.961 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.697601+0000", 2024-09-18T10:15:12.961 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100228709 2024-09-18T10:15:12.962 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:12.962 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.962 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:12.962 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.697909+0000", 2024-09-18T10:15:12.962 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00030862700000000001 2024-09-18T10:15:12.962 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:12.962 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:12.962 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:12.962 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:12.962 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.962 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:121 3.6s0 3:6c64c4fa:::smithi18925896-41:head [append 0~688128,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:12.962 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:50.627356+0000", 2024-09-18T10:15:12.962 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 22.312245574999999, 2024-09-18T10:15:12.963 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.38269042599999997, 2024-09-18T10:15:12.963 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:12.963 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:12.963 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:12.963 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:12.963 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:12.963 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 121 2024-09-18T10:15:12.963 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:12.963 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:12.963 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.963 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:12.963 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.627356+0000", 2024-09-18T10:15:12.963 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:12.963 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:12.964 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.964 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:12.964 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.627356+0000", 2024-09-18T10:15:12.964 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:12.964 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:12.964 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.964 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:12.964 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.627353+0000", 2024-09-18T10:15:12.964 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999971 2024-09-18T10:15:12.964 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:12.964 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.964 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:12.964 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.629112+0000", 2024-09-18T10:15:12.965 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0017585980000000001 2024-09-18T10:15:12.965 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:12.965 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.965 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:12.965 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.629149+0000", 2024-09-18T10:15:12.965 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.6532999999999997e-05 2024-09-18T10:15:12.965 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:12.965 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.965 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:12.965 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.629169+0000", 2024-09-18T10:15:12.965 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0699e-05 2024-09-18T10:15:12.965 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:12.965 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.965 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:12.966 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.706567+0000", 2024-09-18T10:15:12.966 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.077397621999999999 2024-09-18T10:15:12.966 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:12.966 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.966 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:12.966 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.706944+0000", 2024-09-18T10:15:12.966 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00037710599999999999 2024-09-18T10:15:12.966 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:12.966 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.966 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:12.966 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.709283+0000", 2024-09-18T10:15:12.966 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0023391689999999999 2024-09-18T10:15:12.966 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:12.966 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.967 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:12.967 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.711791+0000", 2024-09-18T10:15:12.967 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0025078689999999998 2024-09-18T10:15:12.967 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:12.967 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.967 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-18T10:15:12.967 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.009925+0000", 2024-09-18T10:15:12.967 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.298134493 2024-09-18T10:15:12.967 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:12.967 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.967 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:12.967 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.010047+0000", 2024-09-18T10:15:12.967 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000121264 2024-09-18T10:15:12.967 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:12.968 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:12.968 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:12.968 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:12.968 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.968 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:122 3.6s0 3:6c64c4fa:::smithi18925896-41:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:12.968 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:50.629189+0000", 2024-09-18T10:15:12.968 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 22.310412848999999, 2024-09-18T10:15:12.968 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.47849547399999998, 2024-09-18T10:15:12.968 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:12.968 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:12.968 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:12.968 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:12.968 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:12.968 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 122 2024-09-18T10:15:12.969 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:12.969 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:12.969 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.969 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:12.969 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.629189+0000", 2024-09-18T10:15:12.969 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:12.969 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:12.969 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.969 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:12.969 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.629189+0000", 2024-09-18T10:15:12.969 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:12.969 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:12.969 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.970 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:12.970 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.629187+0000", 2024-09-18T10:15:12.970 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:12.970 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:12.970 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.970 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:12.970 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.629198+0000", 2024-09-18T10:15:12.970 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0623e-05 2024-09-18T10:15:12.970 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:12.970 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.970 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:12.970 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.629219+0000", 2024-09-18T10:15:12.970 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0665999999999998e-05 2024-09-18T10:15:12.970 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:12.970 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.971 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:12.971 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.629231+0000", 2024-09-18T10:15:12.971 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.291e-05 2024-09-18T10:15:12.971 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:12.971 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.971 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:12.971 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.710475+0000", 2024-09-18T10:15:12.971 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.081244021 2024-09-18T10:15:12.971 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:12.971 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.971 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:12.971 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.710639+0000", 2024-09-18T10:15:12.971 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000163325 2024-09-18T10:15:12.971 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:12.972 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.972 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:12.972 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.711435+0000", 2024-09-18T10:15:12.972 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00079575999999999996 2024-09-18T10:15:12.972 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:12.972 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.972 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:12.972 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.812067+0000", 2024-09-18T10:15:12.972 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10063237799999999 2024-09-18T10:15:12.972 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:12.972 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.972 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-18T10:15:12.972 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.107562+0000", 2024-09-18T10:15:12.973 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.295494906 2024-09-18T10:15:12.973 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:12.973 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.973 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:12.973 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.107685+0000", 2024-09-18T10:15:12.973 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00012271100000000001 2024-09-18T10:15:12.973 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:12.973 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:12.973 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:12.973 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:12.973 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.973 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:123 3.6s0 3:6c64c4fa:::smithi18925896-41:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:12.973 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:50.629246+0000", 2024-09-18T10:15:12.973 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 22.310356023000001, 2024-09-18T10:15:12.974 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.58067324300000001, 2024-09-18T10:15:12.974 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:12.974 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:12.974 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:12.974 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:12.974 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:12.974 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 123 2024-09-18T10:15:12.974 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:12.974 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:12.974 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.974 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:12.974 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.629246+0000", 2024-09-18T10:15:12.974 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:12.974 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:12.975 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.976 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:12.976 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.629246+0000", 2024-09-18T10:15:12.977 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:12.978 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:12.978 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.978 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:12.978 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.629244+0000", 2024-09-18T10:15:12.978 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:12.978 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:12.978 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.978 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:12.978 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.629254+0000", 2024-09-18T10:15:12.978 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.0759999999999992e-06 2024-09-18T10:15:12.979 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:12.979 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.979 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:12.979 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.629279+0000", 2024-09-18T10:15:12.979 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.5905999999999999e-05 2024-09-18T10:15:12.979 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:12.979 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.979 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:12.979 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.629293+0000", 2024-09-18T10:15:12.979 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3073e-05 2024-09-18T10:15:12.979 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:12.979 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.979 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:12.980 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.806055+0000", 2024-09-18T10:15:12.980 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.17676249999999999 2024-09-18T10:15:12.980 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:12.980 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.980 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for rw locks", 2024-09-18T10:15:12.980 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.806211+0000", 2024-09-18T10:15:12.980 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000156384 2024-09-18T10:15:12.980 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:12.980 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.980 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:12.980 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.108025+0000", 2024-09-18T10:15:12.980 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.30181311500000002 2024-09-18T10:15:12.980 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:12.980 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.981 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:12.981 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.108107+0000", 2024-09-18T10:15:12.981 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.2330000000000006e-05 2024-09-18T10:15:12.981 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:12.981 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.981 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:12.981 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.209919+0000", 2024-09-18T10:15:12.981 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.101812262 2024-09-18T10:15:12.981 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:12.981 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:12.981 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:12.981 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:12.982 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.982 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWrite(3.bs1 36/32 ECSubWrite(tid=25, reqid=client.4632.0:118, at_version=36'3, trim_to=0'0, roll_forward_to=35'2))", 2024-09-18T10:15:12.982 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:50.708852+0000", 2024-09-18T10:15:12.982 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 22.230750279999999, 2024-09-18T10:15:12.982 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.202385011, 2024-09-18T10:15:12.982 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:12.982 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:12.982 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:12.982 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.982 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:12.982 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.708852+0000", 2024-09-18T10:15:12.982 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:12.982 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:12.983 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.983 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:12.983 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.708852+0000", 2024-09-18T10:15:12.983 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:12.983 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:12.983 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.983 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:12.983 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.708850+0000", 2024-09-18T10:15:12.983 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:12.983 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:12.983 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.983 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:12.983 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.709003+0000", 2024-09-18T10:15:12.984 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00015311900000000001 2024-09-18T10:15:12.984 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:12.984 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.984 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:12.984 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.709024+0000", 2024-09-18T10:15:12.984 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0574999999999999e-05 2024-09-18T10:15:12.984 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:12.984 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.984 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:12.984 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.709036+0000", 2024-09-18T10:15:12.984 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1875999999999999e-05 2024-09-18T10:15:12.984 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:12.984 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.984 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:12.985 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.909305+0000", 2024-09-18T10:15:12.985 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.20026893000000001 2024-09-18T10:15:12.985 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:12.985 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.985 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:12.985 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.909325+0000", 2024-09-18T10:15:12.985 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0809999999999999e-05 2024-09-18T10:15:12.985 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:12.985 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.985 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:12.985 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.911183+0000", 2024-09-18T10:15:12.985 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0018570889999999999 2024-09-18T10:15:12.985 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:12.985 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.986 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:12.986 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.911237+0000", 2024-09-18T10:15:12.986 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4045999999999997e-05 2024-09-18T10:15:12.986 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:12.986 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:12.986 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:12.986 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:12.986 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.986 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:167 3.6s0 3:6619023d:::smithi18925896-19:head [copy-from ver 4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:12.986 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:51.531881+0000", 2024-09-18T10:15:12.986 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 21.407720805, 2024-09-18T10:15:12.986 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.41157491600000001, 2024-09-18T10:15:12.986 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:12.987 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:12.987 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:12.987 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:12.987 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:12.987 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 167 2024-09-18T10:15:12.987 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:12.987 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:12.987 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.987 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:12.988 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.531881+0000", 2024-09-18T10:15:12.988 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:12.988 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:12.988 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.988 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:12.988 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.531881+0000", 2024-09-18T10:15:12.988 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:12.988 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:12.989 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.989 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:12.989 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.531877+0000", 2024-09-18T10:15:12.989 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999962 2024-09-18T10:15:12.989 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:12.989 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.989 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:12.989 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.531894+0000", 2024-09-18T10:15:12.989 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7269999999999999e-05 2024-09-18T10:15:12.989 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:12.989 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.989 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:12.989 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.531939+0000", 2024-09-18T10:15:12.990 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4815999999999999e-05 2024-09-18T10:15:12.990 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:12.990 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.990 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:12.990 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.531963+0000", 2024-09-18T10:15:12.990 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3312e-05 2024-09-18T10:15:12.990 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:12.990 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.990 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:12.990 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.532107+0000", 2024-09-18T10:15:12.990 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000144547 2024-09-18T10:15:12.990 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:12.990 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.990 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:12.991 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.532348+0000", 2024-09-18T10:15:12.991 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000240955 2024-09-18T10:15:12.991 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:12.991 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.991 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:12.991 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.839634+0000", 2024-09-18T10:15:12.991 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.30728566400000001 2024-09-18T10:15:12.991 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:12.991 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.991 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:12.991 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.841560+0000", 2024-09-18T10:15:12.991 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001926449 2024-09-18T10:15:12.992 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:12.992 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.992 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-18T10:15:12.992 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.943372+0000", 2024-09-18T10:15:12.992 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.101812176 2024-09-18T10:15:12.992 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:12.992 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.992 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:12.992 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.943456+0000", 2024-09-18T10:15:12.992 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.3614000000000003e-05 2024-09-18T10:15:12.992 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:12.992 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:12.992 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:12.993 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:12.993 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.993 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:168 3.6s0 3:6619023d:::smithi18925896-19:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:12.993 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:51.532000+0000", 2024-09-18T10:15:12.993 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 21.407601818, 2024-09-18T10:15:12.993 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.30898515999999998, 2024-09-18T10:15:12.993 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:12.993 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:12.993 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:12.993 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:12.993 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:12.993 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 168 2024-09-18T10:15:12.993 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:12.994 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:12.994 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.994 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:12.994 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.532000+0000", 2024-09-18T10:15:12.994 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:12.994 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:12.994 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.995 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:12.995 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.532000+0000", 2024-09-18T10:15:12.995 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:12.995 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:12.995 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.995 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:12.995 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.531998+0000", 2024-09-18T10:15:12.995 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:12.995 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:12.995 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.995 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:12.995 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.532009+0000", 2024-09-18T10:15:12.995 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0988999999999999e-05 2024-09-18T10:15:12.995 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:12.996 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.996 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:12.996 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.532032+0000", 2024-09-18T10:15:12.996 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2203000000000001e-05 2024-09-18T10:15:12.996 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:12.996 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.996 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:12.996 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.532043+0000", 2024-09-18T10:15:12.996 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1707e-05 2024-09-18T10:15:12.996 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:12.996 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.996 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:12.996 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.532663+0000", 2024-09-18T10:15:12.997 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000619993 2024-09-18T10:15:12.997 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:12.997 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.997 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for blocked object", 2024-09-18T10:15:12.997 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.532735+0000", 2024-09-18T10:15:12.997 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.2025000000000002e-05 2024-09-18T10:15:12.997 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:12.997 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.997 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:12.997 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.840712+0000", 2024-09-18T10:15:12.997 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.307976582 2024-09-18T10:15:12.997 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:12.997 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.998 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:12.998 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.840866+0000", 2024-09-18T10:15:12.998 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000154203 2024-09-18T10:15:12.998 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:12.998 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.998 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:12.998 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.840985+0000", 2024-09-18T10:15:12.998 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000119084 2024-09-18T10:15:12.998 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:12.998 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:12.998 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:12.998 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:12.998 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:12.998 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:204 3.6s0 3:61491e68:::smithi18925896-27:head [read 0~442368,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:12.999 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:52.694124+0000", 2024-09-18T10:15:12.999 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 20.245477632, 2024-09-18T10:15:12.999 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.20436779699999999, 2024-09-18T10:15:12.999 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:12.999 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:12.999 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:12.999 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:12.999 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:12.999 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 204 2024-09-18T10:15:12.999 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:12.999 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:12.999 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.000 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:13.000 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.694124+0000", 2024-09-18T10:15:13.000 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.000 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.000 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.000 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:13.000 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.694124+0000", 2024-09-18T10:15:13.000 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.000 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.000 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.000 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:13.000 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.694121+0000", 2024-09-18T10:15:13.001 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999971 2024-09-18T10:15:13.001 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.001 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.001 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:13.001 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.694139+0000", 2024-09-18T10:15:13.001 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7277000000000001e-05 2024-09-18T10:15:13.001 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.001 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.001 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:13.001 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.694168+0000", 2024-09-18T10:15:13.001 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9787000000000001e-05 2024-09-18T10:15:13.001 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.001 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.002 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:13.002 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.694184+0000", 2024-09-18T10:15:13.002 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5478999999999999e-05 2024-09-18T10:15:13.002 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.002 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.002 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:13.002 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.694400+0000", 2024-09-18T10:15:13.002 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00021609 2024-09-18T10:15:13.002 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.003 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.003 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op must wait for map", 2024-09-18T10:15:13.003 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.694417+0000", 2024-09-18T10:15:13.003 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7328000000000001e-05 2024-09-18T10:15:13.003 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.003 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.003 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:13.003 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.794979+0000", 2024-09-18T10:15:13.003 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100561635 2024-09-18T10:15:13.003 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.003 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.003 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:13.003 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.795117+0000", 2024-09-18T10:15:13.003 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00013797399999999999 2024-09-18T10:15:13.004 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.004 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.004 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:13.004 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.898492+0000", 2024-09-18T10:15:13.004 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.103375174 2024-09-18T10:15:13.004 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.004 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:13.004 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.004 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.004 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.004 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:205 3.6s0 3:61491e68:::smithi18925896-27:head [read 0~442368,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:13.004 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:52.694213+0000", 2024-09-18T10:15:13.004 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 20.245388737999999, 2024-09-18T10:15:13.004 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.20580652599999999, 2024-09-18T10:15:13.004 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:13.005 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:13.005 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:13.005 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:13.005 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:13.005 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 205 2024-09-18T10:15:13.005 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.005 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:13.005 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.005 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:13.005 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.694213+0000", 2024-09-18T10:15:13.005 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.005 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.005 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.006 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:13.006 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.694213+0000", 2024-09-18T10:15:13.006 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.006 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.006 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.006 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:13.006 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.694212+0000", 2024-09-18T10:15:13.006 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:13.006 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.006 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.006 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:13.006 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.694220+0000", 2024-09-18T10:15:13.006 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5250000000000005e-06 2024-09-18T10:15:13.006 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.007 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.007 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:13.007 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.694237+0000", 2024-09-18T10:15:13.007 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.6181e-05 2024-09-18T10:15:13.007 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.007 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.007 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:13.007 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.694246+0000", 2024-09-18T10:15:13.007 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.2029999999999998e-06 2024-09-18T10:15:13.007 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.007 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.007 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:13.007 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.794554+0000", 2024-09-18T10:15:13.007 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10030855 2024-09-18T10:15:13.008 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.008 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.008 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:13.008 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.794563+0000", 2024-09-18T10:15:13.008 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.2230000000000007e-06 2024-09-18T10:15:13.008 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.008 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.008 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:13.008 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.795588+0000", 2024-09-18T10:15:13.008 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001025127 2024-09-18T10:15:13.008 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.008 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.008 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:13.009 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.795756+0000", 2024-09-18T10:15:13.009 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00016793900000000001 2024-09-18T10:15:13.009 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.009 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.009 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:13.009 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.900020+0000", 2024-09-18T10:15:13.009 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.104264018 2024-09-18T10:15:13.009 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.009 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:13.009 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.009 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.009 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.009 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:206 3.6s0 3:61491e68:::smithi18925896-27:head [read 0~442368,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:13.009 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:52.694386+0000", 2024-09-18T10:15:13.009 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 20.245215481999999, 2024-09-18T10:15:13.010 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.40544065200000001, 2024-09-18T10:15:13.010 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:13.010 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:13.010 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:13.010 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:13.010 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:13.010 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 206 2024-09-18T10:15:13.010 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.010 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:13.010 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.010 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:13.010 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.694386+0000", 2024-09-18T10:15:13.011 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.011 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.011 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.011 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:13.011 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.694386+0000", 2024-09-18T10:15:13.011 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.011 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.011 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.011 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:13.011 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.694385+0000", 2024-09-18T10:15:13.011 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.999999 2024-09-18T10:15:13.011 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.012 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.012 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:13.012 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.694392+0000", 2024-09-18T10:15:13.012 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7970000000000003e-06 2024-09-18T10:15:13.012 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.012 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.012 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:13.012 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.694405+0000", 2024-09-18T10:15:13.012 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3406e-05 2024-09-18T10:15:13.012 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.012 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.012 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:13.012 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.694413+0000", 2024-09-18T10:15:13.012 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.9070000000000007e-06 2024-09-18T10:15:13.013 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.013 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.013 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:13.013 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.794592+0000", 2024-09-18T10:15:13.013 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100178879 2024-09-18T10:15:13.013 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.013 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.013 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:13.013 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.794598+0000", 2024-09-18T10:15:13.013 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6289999999999998e-06 2024-09-18T10:15:13.013 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.013 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.013 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:13.013 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.796805+0000", 2024-09-18T10:15:13.014 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0022073930000000002 2024-09-18T10:15:13.014 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.014 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.014 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:13.014 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.796886+0000", 2024-09-18T10:15:13.014 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0641000000000003e-05 2024-09-18T10:15:13.014 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.014 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.014 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:13.014 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:53.099827+0000", 2024-09-18T10:15:13.014 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.302941191 2024-09-18T10:15:13.014 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.014 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:13.014 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.015 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.015 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.015 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:323 3.6s0 3:61491e68:::smithi18925896-27:head [append 0~0] snapc 7=[4] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-18T10:15:13.015 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:01.795354+0000", 2024-09-18T10:15:13.015 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 11.144247968, 2024-09-18T10:15:13.015 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.30418701500000001, 2024-09-18T10:15:13.015 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:13.015 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:13.015 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:13.015 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:13.015 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:13.015 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 323 2024-09-18T10:15:13.015 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.015 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:13.015 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.016 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:13.016 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.795354+0000", 2024-09-18T10:15:13.016 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.016 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.016 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.016 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:13.016 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.795354+0000", 2024-09-18T10:15:13.016 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.016 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.016 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.016 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:13.016 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.795351+0000", 2024-09-18T10:15:13.016 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999971 2024-09-18T10:15:13.016 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.017 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.017 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:13.017 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.795365+0000", 2024-09-18T10:15:13.017 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3713999999999999e-05 2024-09-18T10:15:13.017 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.017 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.017 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:13.017 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.795406+0000", 2024-09-18T10:15:13.017 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0846999999999997e-05 2024-09-18T10:15:13.017 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.017 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.017 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:13.017 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.795425+0000", 2024-09-18T10:15:13.017 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9936999999999999e-05 2024-09-18T10:15:13.018 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.018 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.018 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:13.018 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.795648+0000", 2024-09-18T10:15:13.018 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000222547 2024-09-18T10:15:13.018 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.018 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.018 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:13.018 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.795872+0000", 2024-09-18T10:15:13.018 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00022384799999999999 2024-09-18T10:15:13.018 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.018 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.018 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:13.018 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.797018+0000", 2024-09-18T10:15:13.018 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001146055 2024-09-18T10:15:13.019 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.019 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.019 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:13.019 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.798609+0000", 2024-09-18T10:15:13.019 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0015905959999999999 2024-09-18T10:15:13.019 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.019 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.019 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-18T10:15:13.019 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.099372+0000", 2024-09-18T10:15:13.019 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.30076301500000002 2024-09-18T10:15:13.019 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.019 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.019 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:13.019 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.099541+0000", 2024-09-18T10:15:13.020 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000169407 2024-09-18T10:15:13.020 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.020 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:13.020 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.020 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.020 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.020 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:324 3.6s0 3:61491e68:::smithi18925896-27:head [rollback 4] snapc 7=[4] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-18T10:15:13.020 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:01.795482+0000", 2024-09-18T10:15:13.020 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 11.144120354, 2024-09-18T10:15:13.020 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.30451560100000002, 2024-09-18T10:15:13.020 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:13.020 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:13.020 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:13.021 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:13.021 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:13.021 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 324 2024-09-18T10:15:13.021 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.021 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:13.021 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.021 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:13.021 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.795482+0000", 2024-09-18T10:15:13.021 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.021 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.021 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.021 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:13.021 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.795482+0000", 2024-09-18T10:15:13.021 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.022 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.022 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.022 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:13.022 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.795480+0000", 2024-09-18T10:15:13.022 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:13.022 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.022 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.022 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:13.022 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.795487+0000", 2024-09-18T10:15:13.022 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7170000000000002e-06 2024-09-18T10:15:13.022 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.022 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.022 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:13.022 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.795548+0000", 2024-09-18T10:15:13.023 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1647000000000001e-05 2024-09-18T10:15:13.023 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.023 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.023 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:13.023 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.795566+0000", 2024-09-18T10:15:13.023 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7560000000000001e-05 2024-09-18T10:15:13.023 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.023 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.023 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:13.023 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.797676+0000", 2024-09-18T10:15:13.023 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0021101919999999999 2024-09-18T10:15:13.023 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.023 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.023 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:13.024 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.797742+0000", 2024-09-18T10:15:13.024 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.6351000000000002e-05 2024-09-18T10:15:13.024 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.024 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.024 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:13.024 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.798290+0000", 2024-09-18T10:15:13.024 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000548063 2024-09-18T10:15:13.024 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.024 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.024 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:13.024 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.800131+0000", 2024-09-18T10:15:13.024 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001840759 2024-09-18T10:15:13.024 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.025 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.025 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-18T10:15:13.025 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.099840+0000", 2024-09-18T10:15:13.025 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.29970881300000002 2024-09-18T10:15:13.025 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.025 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.025 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:13.025 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.099997+0000", 2024-09-18T10:15:13.025 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000157124 2024-09-18T10:15:13.025 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.025 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:13.025 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.025 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.025 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.026 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:325 3.6s0 3:61491e68:::smithi18925896-27:head [append 0~0] snapc 7=[4] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-18T10:15:13.026 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:01.795591+0000", 2024-09-18T10:15:13.026 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 11.144011299000001, 2024-09-18T10:15:13.026 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.30495726400000001, 2024-09-18T10:15:13.026 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:13.026 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:13.026 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:13.026 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:13.026 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:13.026 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 325 2024-09-18T10:15:13.026 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.026 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:13.026 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.026 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:13.027 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.795591+0000", 2024-09-18T10:15:13.027 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.027 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.027 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.027 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:13.027 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.795591+0000", 2024-09-18T10:15:13.027 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.027 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.027 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.027 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:13.027 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.795589+0000", 2024-09-18T10:15:13.027 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:13.027 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.027 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.027 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:13.028 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.795598+0000", 2024-09-18T10:15:13.028 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.7940000000000008e-06 2024-09-18T10:15:13.028 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.028 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.028 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:13.028 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.795617+0000", 2024-09-18T10:15:13.028 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8485e-05 2024-09-18T10:15:13.028 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.028 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.028 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:13.028 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.795627+0000", 2024-09-18T10:15:13.028 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0730999999999999e-05 2024-09-18T10:15:13.028 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.029 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.029 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:13.029 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.798672+0000", 2024-09-18T10:15:13.029 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00304484 2024-09-18T10:15:13.029 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.029 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.029 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:13.029 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.798918+0000", 2024-09-18T10:15:13.029 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00024598000000000003 2024-09-18T10:15:13.029 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.029 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.029 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:13.029 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.799703+0000", 2024-09-18T10:15:13.029 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000785262 2024-09-18T10:15:13.030 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.030 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.030 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:13.030 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.900558+0000", 2024-09-18T10:15:13.030 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100854108 2024-09-18T10:15:13.030 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.030 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.030 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-18T10:15:13.030 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.100434+0000", 2024-09-18T10:15:13.030 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.19987638699999999 2024-09-18T10:15:13.030 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.030 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.030 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:13.030 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.100548+0000", 2024-09-18T10:15:13.031 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00011390799999999999 2024-09-18T10:15:13.031 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.031 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:13.031 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.031 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.031 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.031 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWriteReply(3.6s0 45/40 ECSubWriteReply(tid=45, last_complete=46'26, committed=1, applied=1))", 2024-09-18T10:15:13.031 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:01.898885+0000", 2024-09-18T10:15:13.031 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 11.040716647, 2024-09-18T10:15:13.031 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.200675623, 2024-09-18T10:15:13.031 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:13.031 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:13.031 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:13.031 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.032 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:13.032 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.898885+0000", 2024-09-18T10:15:13.032 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.032 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.032 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.032 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:13.032 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.898885+0000", 2024-09-18T10:15:13.032 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.032 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.032 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.032 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:13.032 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.898884+0000", 2024-09-18T10:15:13.032 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.999999 2024-09-18T10:15:13.032 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.033 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.033 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:13.033 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.898894+0000", 2024-09-18T10:15:13.033 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.9369999999999999e-06 2024-09-18T10:15:13.033 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.033 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.033 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:13.033 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.898912+0000", 2024-09-18T10:15:13.033 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7472000000000001e-05 2024-09-18T10:15:13.033 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.033 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.033 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:13.033 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.999072+0000", 2024-09-18T10:15:13.033 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.1001601 2024-09-18T10:15:13.033 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.034 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.034 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:13.034 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.099237+0000", 2024-09-18T10:15:13.034 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100165611 2024-09-18T10:15:13.034 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.034 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.036 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:13.036 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.099561+0000", 2024-09-18T10:15:13.036 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00032355300000000002 2024-09-18T10:15:13.036 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.036 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:13.036 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.036 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.036 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.037 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "replica_scrub(pg: 3.8s1,from:0'0,to:41'13,epoch:47/40,start:3:10000000::::0,end:MAX,chunky:1,deep:0,version:9,allow_preemption:1,priority=5)", 2024-09-18T10:15:13.037 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:03.012324+0000", 2024-09-18T10:15:13.037 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 9.9272778129999999, 2024-09-18T10:15:13.037 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.20050876400000001, 2024-09-18T10:15:13.037 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:13.037 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:13.037 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:13.037 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.037 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:13.037 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:03.012324+0000", 2024-09-18T10:15:13.037 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.037 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.037 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.037 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:13.038 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:03.012324+0000", 2024-09-18T10:15:13.038 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.038 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.038 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.038 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:13.038 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:03.012324+0000", 2024-09-18T10:15:13.038 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:13.038 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.038 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.038 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:13.038 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:03.012347+0000", 2024-09-18T10:15:13.038 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3056000000000001e-05 2024-09-18T10:15:13.038 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.038 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.039 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:13.039 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:03.012374+0000", 2024-09-18T10:15:13.039 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6846e-05 2024-09-18T10:15:13.039 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.039 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.039 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:13.039 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:03.112471+0000", 2024-09-18T10:15:13.039 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100097857 2024-09-18T10:15:13.039 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.039 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.039 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:13.039 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:03.212715+0000", 2024-09-18T10:15:13.039 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10024356 2024-09-18T10:15:13.039 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.040 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.040 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:13.040 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:03.212740+0000", 2024-09-18T10:15:13.040 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.5083e-05 2024-09-18T10:15:13.040 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.040 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.040 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:13.040 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:03.212833+0000", 2024-09-18T10:15:13.040 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.2755000000000002e-05 2024-09-18T10:15:13.040 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.040 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:13.040 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.040 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.041 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.041 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDScrubReserve(3.6s0 GRANT e50)", 2024-09-18T10:15:13.041 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:06.485987+0000", 2024-09-18T10:15:13.041 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 6.4536152849999997, 2024-09-18T10:15:13.041 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.20085092399999999, 2024-09-18T10:15:13.041 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:13.041 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:13.041 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:13.041 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.041 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:13.041 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:06.485987+0000", 2024-09-18T10:15:13.041 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.041 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.041 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.042 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:13.042 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:06.485987+0000", 2024-09-18T10:15:13.042 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.042 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.042 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.042 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:13.042 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:06.485985+0000", 2024-09-18T10:15:13.042 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:13.042 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.042 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.042 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:13.042 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:06.485995+0000", 2024-09-18T10:15:13.042 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.6860000000000001e-06 2024-09-18T10:15:13.042 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.043 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.043 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:13.043 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:06.486025+0000", 2024-09-18T10:15:13.043 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0673000000000003e-05 2024-09-18T10:15:13.043 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.043 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.043 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:13.043 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:06.486051+0000", 2024-09-18T10:15:13.043 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6188000000000001e-05 2024-09-18T10:15:13.043 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.043 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.043 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:13.043 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:06.686765+0000", 2024-09-18T10:15:13.043 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.200713951 2024-09-18T10:15:13.044 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.044 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.044 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:13.044 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:06.686809+0000", 2024-09-18T10:15:13.044 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.3140999999999999e-05 2024-09-18T10:15:13.044 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.044 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.044 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:13.044 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:06.686838+0000", 2024-09-18T10:15:13.044 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.8985000000000001e-05 2024-09-18T10:15:13.044 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.044 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:13.044 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.045 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.045 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:13.045 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-18T10:15:13.046 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T10:15:13.047 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-18T10:15:13.045+0000 7f1981873700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22132) UID: 0 2024-09-18T10:15:13.141 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-18T10:15:13.142 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-18T10:15:13.142 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.142 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:441 3.17s0 3:ead53b0f:::smithi18925896-25:head [append 0~606208 in=606208b] snapc 13=[13,12,11,10,f,e,b] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-18T10:15:13.142 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:12.856902+0000", 2024-09-18T10:15:13.142 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.28300346999999998, 2024-09-18T10:15:13.142 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.283041979, 2024-09-18T10:15:13.142 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:13.142 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:13.142 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:13.142 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:13.143 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:13.143 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 441 2024-09-18T10:15:13.143 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.143 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:13.143 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.143 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:13.143 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.856902+0000", 2024-09-18T10:15:13.143 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.143 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.143 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.143 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:13.143 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.856902+0000", 2024-09-18T10:15:13.144 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.144 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.144 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.144 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:13.144 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.856900+0000", 2024-09-18T10:15:13.144 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999976 2024-09-18T10:15:13.144 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.144 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.144 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:13.144 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.857946+0000", 2024-09-18T10:15:13.144 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0010459620000000001 2024-09-18T10:15:13.144 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.144 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.145 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:13.145 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.857973+0000", 2024-09-18T10:15:13.145 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6758999999999999e-05 2024-09-18T10:15:13.145 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.145 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.145 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:13.145 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.857983+0000", 2024-09-18T10:15:13.145 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0105000000000001e-05 2024-09-18T10:15:13.145 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.145 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.145 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:13.145 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.946351+0000", 2024-09-18T10:15:13.146 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.088368302999999995 2024-09-18T10:15:13.146 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.146 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.146 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:13.146 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.946465+0000", 2024-09-18T10:15:13.146 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00011425999999999999 2024-09-18T10:15:13.146 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.146 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.146 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:13.146 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.947656+0000", 2024-09-18T10:15:13.146 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0011901710000000001 2024-09-18T10:15:13.146 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.146 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.147 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:13.147 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.040655+0000", 2024-09-18T10:15:13.147 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.092999126000000001 2024-09-18T10:15:13.147 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.147 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:13.147 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.147 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.147 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.147 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:442 3.17s0 3:ead53b0f:::smithi18925896-25:head [append 0~720896 in=720896b] snapc 13=[13,12,11,10,f,e,b] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-18T10:15:13.147 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:12.861303+0000", 2024-09-18T10:15:13.147 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.27860307600000001, 2024-09-18T10:15:13.148 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.27874046000000002, 2024-09-18T10:15:13.148 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:13.148 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:13.148 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:13.148 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:13.148 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:13.148 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 442 2024-09-18T10:15:13.148 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.148 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:13.148 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.149 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:13.149 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.861303+0000", 2024-09-18T10:15:13.149 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.149 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.149 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.149 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:13.149 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.861303+0000", 2024-09-18T10:15:13.149 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.149 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.149 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.149 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:13.149 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.861301+0000", 2024-09-18T10:15:13.149 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:13.150 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.150 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.150 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:13.150 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.862343+0000", 2024-09-18T10:15:13.150 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0010422820000000001 2024-09-18T10:15:13.150 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.150 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.150 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:13.150 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.862381+0000", 2024-09-18T10:15:13.150 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7407000000000003e-05 2024-09-18T10:15:13.150 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.150 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.151 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:13.151 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.862391+0000", 2024-09-18T10:15:13.151 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0572e-05 2024-09-18T10:15:13.151 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.151 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.151 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:13.151 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.948744+0000", 2024-09-18T10:15:13.151 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.086352683999999999 2024-09-18T10:15:13.151 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.151 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.151 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:13.151 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.948864+0000", 2024-09-18T10:15:13.152 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000119855 2024-09-18T10:15:13.152 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.152 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.152 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:13.152 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.949834+0000", 2024-09-18T10:15:13.152 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00097020199999999998 2024-09-18T10:15:13.152 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.152 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.152 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:13.152 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.040659+0000", 2024-09-18T10:15:13.152 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.090824659000000002 2024-09-18T10:15:13.152 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.152 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:13.153 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.153 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.153 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.153 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:443 3.17s0 3:ead53b0f:::smithi18925896-25:head [append 0~49152 in=49152b] snapc 13=[13,12,11,10,f,e,b] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-18T10:15:13.153 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:12.862402+0000", 2024-09-18T10:15:13.153 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.27750329800000001, 2024-09-18T10:15:13.153 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.27773002400000002, 2024-09-18T10:15:13.153 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:13.153 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:13.153 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:13.153 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:13.153 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:13.153 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 443 2024-09-18T10:15:13.153 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.154 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:13.154 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.154 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:13.154 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.862402+0000", 2024-09-18T10:15:13.154 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.154 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.154 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.154 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:13.154 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.862402+0000", 2024-09-18T10:15:13.154 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.154 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.154 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.154 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:13.154 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.862402+0000", 2024-09-18T10:15:13.155 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.999999 2024-09-18T10:15:13.155 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.155 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.155 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:13.155 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.862441+0000", 2024-09-18T10:15:13.155 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9116000000000003e-05 2024-09-18T10:15:13.155 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.155 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.155 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:13.155 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.862455+0000", 2024-09-18T10:15:13.155 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4501999999999999e-05 2024-09-18T10:15:13.155 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.155 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.156 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:13.156 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.862462+0000", 2024-09-18T10:15:13.156 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.4659999999999996e-06 2024-09-18T10:15:13.156 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.156 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.156 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:13.156 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.950570+0000", 2024-09-18T10:15:13.156 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.088107850000000001 2024-09-18T10:15:13.156 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.156 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.156 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:13.156 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.950675+0000", 2024-09-18T10:15:13.156 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000105445 2024-09-18T10:15:13.157 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.157 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.157 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:13.157 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.951279+0000", 2024-09-18T10:15:13.157 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00060383599999999996 2024-09-18T10:15:13.157 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.157 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.157 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:13.157 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.040661+0000", 2024-09-18T10:15:13.157 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.089382448000000003 2024-09-18T10:15:13.157 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.157 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:13.157 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.158 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.158 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.158 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:444 3.17s0 3:ead53b0f:::smithi18925896-25:head [setxattr _header (57) in=64b] snapc 13=[13,12,11,10,f,e,b] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-18T10:15:13.158 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:12.862469+0000", 2024-09-18T10:15:13.158 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.277437088, 2024-09-18T10:15:13.158 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.27776176600000002, 2024-09-18T10:15:13.158 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:13.158 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:13.158 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:13.158 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:13.158 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:13.158 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 444 2024-09-18T10:15:13.158 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.159 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:13.159 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.159 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:13.159 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.862469+0000", 2024-09-18T10:15:13.159 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.159 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.159 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.159 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:13.159 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.862469+0000", 2024-09-18T10:15:13.159 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.159 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.159 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.159 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:13.160 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.862468+0000", 2024-09-18T10:15:13.160 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:13.160 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.160 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.160 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:13.160 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.862473+0000", 2024-09-18T10:15:13.160 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.5839999999999996e-06 2024-09-18T10:15:13.160 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.160 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.160 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:13.160 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.862483+0000", 2024-09-18T10:15:13.160 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0188e-05 2024-09-18T10:15:13.161 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.161 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.161 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:13.161 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.862487+0000", 2024-09-18T10:15:13.161 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6580000000000001e-06 2024-09-18T10:15:13.161 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.161 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.161 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:13.161 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.952057+0000", 2024-09-18T10:15:13.161 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.089569930000000006 2024-09-18T10:15:13.161 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.161 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.161 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:13.162 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.952227+0000", 2024-09-18T10:15:13.162 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000170106 2024-09-18T10:15:13.162 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.162 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.162 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:13.162 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.953127+0000", 2024-09-18T10:15:13.162 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00089958300000000002 2024-09-18T10:15:13.162 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.162 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.162 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:13.162 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.040664+0000", 2024-09-18T10:15:13.162 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.087536931999999998 2024-09-18T10:15:13.163 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.163 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:13.163 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.163 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.163 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.163 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:445 3.17s0 3:ead53b0f:::smithi18925896-25:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e57)", 2024-09-18T10:15:13.163 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:12.862493+0000", 2024-09-18T10:15:13.163 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.27741322000000002, 2024-09-18T10:15:13.163 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.277823816, 2024-09-18T10:15:13.163 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:13.163 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "waiting for rw locks", 2024-09-18T10:15:13.163 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:13.164 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:13.164 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:13.164 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 445 2024-09-18T10:15:13.164 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.164 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:13.164 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.164 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:13.164 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.862493+0000", 2024-09-18T10:15:13.164 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.164 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.164 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.164 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:13.165 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.862493+0000", 2024-09-18T10:15:13.165 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.165 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.165 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.165 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:13.165 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.862492+0000", 2024-09-18T10:15:13.165 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:13.165 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.165 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.165 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:13.165 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.862495+0000", 2024-09-18T10:15:13.165 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1829999999999998e-06 2024-09-18T10:15:13.165 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.166 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.166 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:13.166 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.862506+0000", 2024-09-18T10:15:13.166 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0386e-05 2024-09-18T10:15:13.166 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.166 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.166 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:13.166 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.962582+0000", 2024-09-18T10:15:13.166 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100076741 2024-09-18T10:15:13.166 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.166 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.166 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:13.167 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.962632+0000", 2024-09-18T10:15:13.167 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9364999999999999e-05 2024-09-18T10:15:13.167 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.167 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.167 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for rw locks", 2024-09-18T10:15:13.167 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.962715+0000", 2024-09-18T10:15:13.167 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.3617999999999998e-05 2024-09-18T10:15:13.167 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.167 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:13.167 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.167 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.167 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.167 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWriteReply(3.17s0 57/40 ECSubWriteReply(tid=105, last_complete=57'20, committed=1, applied=1))", 2024-09-18T10:15:13.168 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:13.054814+0000", 2024-09-18T10:15:13.168 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.085091525000000001, 2024-09-18T10:15:13.168 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.08557853, 2024-09-18T10:15:13.168 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:13.168 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-18T10:15:13.168 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:13.168 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.168 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:13.168 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.054814+0000", 2024-09-18T10:15:13.168 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.168 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.168 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.168 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:13.168 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.054814+0000", 2024-09-18T10:15:13.169 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.169 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.169 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.169 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:13.169 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.054813+0000", 2024-09-18T10:15:13.169 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:13.169 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.169 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.169 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:13.169 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.054825+0000", 2024-09-18T10:15:13.169 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2135e-05 2024-09-18T10:15:13.169 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.169 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.170 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:13.170 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.054844+0000", 2024-09-18T10:15:13.170 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8536999999999999e-05 2024-09-18T10:15:13.170 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.170 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.170 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:13.170 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.054852+0000", 2024-09-18T10:15:13.170 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.585e-06 2024-09-18T10:15:13.170 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.170 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:13.170 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.170 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.170 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.171 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWriteReply(3.17s0 57/40 ECSubWriteReply(tid=106, last_complete=57'22, committed=1, applied=1))", 2024-09-18T10:15:13.171 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:13.054873+0000", 2024-09-18T10:15:13.171 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.085032907000000005, 2024-09-18T10:15:13.171 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.085569972999999994, 2024-09-18T10:15:13.171 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:13.171 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-18T10:15:13.171 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:13.171 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.171 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:13.171 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.054873+0000", 2024-09-18T10:15:13.171 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.171 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.171 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.172 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:13.172 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.054873+0000", 2024-09-18T10:15:13.172 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.172 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.172 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.172 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:13.172 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.054872+0000", 2024-09-18T10:15:13.172 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:13.172 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.172 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.172 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:13.172 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.054877+0000", 2024-09-18T10:15:13.172 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.5179999999999998e-06 2024-09-18T10:15:13.173 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.173 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.173 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:13.173 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.054888+0000", 2024-09-18T10:15:13.173 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1396999999999999e-05 2024-09-18T10:15:13.173 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.173 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.173 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:13.173 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.054896+0000", 2024-09-18T10:15:13.173 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4399999999999999e-06 2024-09-18T10:15:13.173 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.173 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:13.173 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.174 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.174 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.174 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWriteReply(3.17s0 57/40 ECSubWriteReply(tid=106, last_complete=57'22, committed=1, applied=1))", 2024-09-18T10:15:13.174 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:13.054951+0000", 2024-09-18T10:15:13.174 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.084954981999999998, 2024-09-18T10:15:13.174 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.085543040000000001, 2024-09-18T10:15:13.174 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:13.174 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-18T10:15:13.174 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:13.174 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.174 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:13.174 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.054951+0000", 2024-09-18T10:15:13.174 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.175 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.175 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.175 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:13.175 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.054951+0000", 2024-09-18T10:15:13.175 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.175 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.175 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.175 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:13.175 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.054950+0000", 2024-09-18T10:15:13.175 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:13.175 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.175 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.175 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:13.175 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.054955+0000", 2024-09-18T10:15:13.175 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6979999999999997e-06 2024-09-18T10:15:13.176 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.176 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.176 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:13.176 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.054965+0000", 2024-09-18T10:15:13.176 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.8349999999999999e-06 2024-09-18T10:15:13.176 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.176 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.176 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:13.176 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.054970+0000", 2024-09-18T10:15:13.176 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2549999999999997e-06 2024-09-18T10:15:13.176 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.176 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:13.176 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.177 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.177 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.177 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWriteReply(3.17s0 57/40 ECSubWriteReply(tid=105, last_complete=57'20, committed=1, applied=1))", 2024-09-18T10:15:13.177 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:13.055991+0000", 2024-09-18T10:15:13.177 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.083914318000000002, 2024-09-18T10:15:13.177 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.084551870000000001, 2024-09-18T10:15:13.177 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:13.177 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-18T10:15:13.177 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:13.177 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.177 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:13.177 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.055991+0000", 2024-09-18T10:15:13.177 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.178 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.178 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.178 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:13.178 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.055991+0000", 2024-09-18T10:15:13.178 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.178 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.178 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.178 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:13.178 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.055990+0000", 2024-09-18T10:15:13.178 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.999999 2024-09-18T10:15:13.178 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.178 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.179 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:13.179 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.055998+0000", 2024-09-18T10:15:13.179 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7489999999999998e-06 2024-09-18T10:15:13.179 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.179 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.179 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:13.179 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.056019+0000", 2024-09-18T10:15:13.179 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0564000000000002e-05 2024-09-18T10:15:13.179 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.179 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.179 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:13.179 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.056115+0000", 2024-09-18T10:15:13.179 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.6160000000000001e-05 2024-09-18T10:15:13.180 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.180 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:13.180 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.180 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.180 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.180 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWriteReply(3.17s0 57/40 ECSubWriteReply(tid=106, last_complete=57'22, committed=1, applied=1))", 2024-09-18T10:15:13.180 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:13.056127+0000", 2024-09-18T10:15:13.180 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.083778351000000001, 2024-09-18T10:15:13.180 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.084465272999999993, 2024-09-18T10:15:13.180 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:13.180 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-18T10:15:13.180 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:13.180 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.181 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:13.181 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.056127+0000", 2024-09-18T10:15:13.181 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.181 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.181 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.181 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:13.181 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.056127+0000", 2024-09-18T10:15:13.181 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.181 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.181 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.181 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:13.181 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.056127+0000", 2024-09-18T10:15:13.181 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:13.182 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.182 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.182 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:13.182 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.056133+0000", 2024-09-18T10:15:13.182 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.913e-06 2024-09-18T10:15:13.182 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.182 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.182 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:13.182 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.056141+0000", 2024-09-18T10:15:13.182 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.602e-06 2024-09-18T10:15:13.182 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.182 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.182 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:13.182 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.056152+0000", 2024-09-18T10:15:13.183 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0237999999999999e-05 2024-09-18T10:15:13.183 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.183 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:13.183 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.183 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.183 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-18T10:15:13.183 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 10 2024-09-18T10:15:13.183 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-18T10:15:13.185 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T10:15:13.185 INFO:tasks.ceph.osd.14.smithi154.stderr:2024-09-18T10:15:13.179+0000 7f75a19e7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 22093) UID: 0 2024-09-18T10:15:13.285 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-09-18T10:15:13.279+0000 7fdfa8698700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22087) UID: 0 2024-09-18T10:15:13.289 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-18T10:15:13.289 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-18T10:15:13.289 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-18T10:15:13.289 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-18T10:15:13.290 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-18T10:15:13.299 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T10:15:13.386 INFO:tasks.ceph.osd.14.smithi154.stderr:2024-09-18T10:15:13.379+0000 7f75a19e7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 22093) UID: 0 2024-09-18T10:15:13.409 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-18T10:15:13.410 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-18T10:15:13.410 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-18T10:15:13.410 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-18T10:15:13.410 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.410 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:119 3.bs0 3:d62b3dfa:::smithi18925896-40:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:13.410 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:50.705490+0000", 2024-09-18T10:15:13.410 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 22.699796343999999, 2024-09-18T10:15:13.410 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.40546939399999998, 2024-09-18T10:15:13.410 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:13.410 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:13.410 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:13.411 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:13.411 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:13.411 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 119 2024-09-18T10:15:13.411 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.411 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:13.411 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.411 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:13.411 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.705490+0000", 2024-09-18T10:15:13.411 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.411 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.411 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.411 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:13.411 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.705490+0000", 2024-09-18T10:15:13.411 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.412 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.412 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.412 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:13.412 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.705488+0000", 2024-09-18T10:15:13.412 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:13.412 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.412 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.412 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:13.412 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.705498+0000", 2024-09-18T10:15:13.412 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0393e-05 2024-09-18T10:15:13.412 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.412 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.413 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:13.413 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.705523+0000", 2024-09-18T10:15:13.413 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.499e-05 2024-09-18T10:15:13.413 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.413 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.413 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:13.413 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.705536+0000", 2024-09-18T10:15:13.413 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2833000000000001e-05 2024-09-18T10:15:13.413 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.413 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.413 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:13.413 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.810302+0000", 2024-09-18T10:15:13.413 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.104765231 2024-09-18T10:15:13.414 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.414 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.414 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:13.414 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.810480+0000", 2024-09-18T10:15:13.414 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00017834599999999999 2024-09-18T10:15:13.414 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.414 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.414 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:13.414 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.811463+0000", 2024-09-18T10:15:13.414 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00098265799999999997 2024-09-18T10:15:13.414 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.414 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.414 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:13.415 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.812898+0000", 2024-09-18T10:15:13.415 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0014352360000000001 2024-09-18T10:15:13.415 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.415 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.415 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-18T10:15:13.415 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.110742+0000", 2024-09-18T10:15:13.415 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.29784466900000001 2024-09-18T10:15:13.415 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.415 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.415 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:13.415 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.110959+0000", 2024-09-18T10:15:13.415 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000217035 2024-09-18T10:15:13.416 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.416 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:13.416 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.416 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.416 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.416 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:120 3.bs0 3:d62b3dfa:::smithi18925896-40:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:13.416 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:50.705551+0000", 2024-09-18T10:15:13.416 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 22.699735501999999, 2024-09-18T10:15:13.416 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.508399406, 2024-09-18T10:15:13.416 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:13.416 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:13.416 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:13.417 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:13.417 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:13.417 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 120 2024-09-18T10:15:13.417 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.417 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:13.417 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.417 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:13.417 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.705551+0000", 2024-09-18T10:15:13.417 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.417 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.417 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.417 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:13.417 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.705551+0000", 2024-09-18T10:15:13.418 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.418 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.418 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.418 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:13.418 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.705549+0000", 2024-09-18T10:15:13.418 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:13.418 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.418 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.418 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:13.418 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.705557+0000", 2024-09-18T10:15:13.418 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.9500000000000001e-06 2024-09-18T10:15:13.418 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.419 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.419 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:13.419 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.705575+0000", 2024-09-18T10:15:13.419 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7989999999999999e-05 2024-09-18T10:15:13.419 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.419 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.419 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:13.419 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.705587+0000", 2024-09-18T10:15:13.419 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1226000000000001e-05 2024-09-18T10:15:13.419 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.419 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.419 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:13.419 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.812099+0000", 2024-09-18T10:15:13.419 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.106512116 2024-09-18T10:15:13.420 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.420 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.420 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for rw locks", 2024-09-18T10:15:13.420 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.812429+0000", 2024-09-18T10:15:13.420 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00033007300000000001 2024-09-18T10:15:13.420 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.420 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.420 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:13.420 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.111941+0000", 2024-09-18T10:15:13.420 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.29951245700000001 2024-09-18T10:15:13.420 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.420 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.420 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:13.420 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.112099+0000", 2024-09-18T10:15:13.421 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00015813399999999999 2024-09-18T10:15:13.421 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.421 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.421 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:13.421 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.213950+0000", 2024-09-18T10:15:13.421 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10185091 2024-09-18T10:15:13.421 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.421 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:13.421 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.421 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.421 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.421 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:127 3.7s0 3:e4da5b5e:::smithi18925896-43:head [append 0~720896,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:13.421 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:50.712808+0000", 2024-09-18T10:15:13.422 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 22.69247863, 2024-09-18T10:15:13.422 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.40445257800000001, 2024-09-18T10:15:13.422 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:13.422 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:13.422 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:13.422 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:13.422 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:13.422 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 127 2024-09-18T10:15:13.422 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.422 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:13.422 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.422 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:13.422 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.712808+0000", 2024-09-18T10:15:13.423 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.423 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.423 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.423 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:13.423 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.712808+0000", 2024-09-18T10:15:13.423 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.423 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.423 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.423 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:13.423 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.712805+0000", 2024-09-18T10:15:13.423 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999971 2024-09-18T10:15:13.423 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.423 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.424 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:13.424 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.713866+0000", 2024-09-18T10:15:13.424 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0010617070000000001 2024-09-18T10:15:13.424 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.424 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.424 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:13.424 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.713900+0000", 2024-09-18T10:15:13.424 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.3575999999999998e-05 2024-09-18T10:15:13.424 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.424 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.424 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:13.424 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.713913+0000", 2024-09-18T10:15:13.424 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2823000000000001e-05 2024-09-18T10:15:13.424 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.425 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.425 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:13.425 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.900562+0000", 2024-09-18T10:15:13.425 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.18664886 2024-09-18T10:15:13.425 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.425 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.425 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:13.425 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.900856+0000", 2024-09-18T10:15:13.425 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000294617 2024-09-18T10:15:13.425 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.425 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.425 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:13.426 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.902520+0000", 2024-09-18T10:15:13.426 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00166351 2024-09-18T10:15:13.426 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.426 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.426 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:13.426 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.905574+0000", 2024-09-18T10:15:13.426 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.003054583 2024-09-18T10:15:13.426 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.426 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.426 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-18T10:15:13.426 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.117162+0000", 2024-09-18T10:15:13.426 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.211587309 2024-09-18T10:15:13.426 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.427 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.427 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:13.427 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.117260+0000", 2024-09-18T10:15:13.427 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.8640000000000004e-05 2024-09-18T10:15:13.427 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.428 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:13.428 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.428 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.428 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.428 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:128 3.7s0 3:e4da5b5e:::smithi18925896-43:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:13.428 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:50.713927+0000", 2024-09-18T10:15:13.428 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 22.691359444, 2024-09-18T10:15:13.428 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.50422476699999996, 2024-09-18T10:15:13.429 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:13.429 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:13.429 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:13.429 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:13.429 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:13.429 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 128 2024-09-18T10:15:13.429 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.429 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:13.429 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.429 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:13.429 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.713927+0000", 2024-09-18T10:15:13.429 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.429 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.430 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.430 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:13.430 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.713927+0000", 2024-09-18T10:15:13.430 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.430 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.430 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.430 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:13.430 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.713925+0000", 2024-09-18T10:15:13.430 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:13.430 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.430 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.430 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:13.430 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.713933+0000", 2024-09-18T10:15:13.430 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.2069999999999998e-06 2024-09-18T10:15:13.430 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.431 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.431 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:13.431 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.713947+0000", 2024-09-18T10:15:13.431 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4249e-05 2024-09-18T10:15:13.431 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.431 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.431 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:13.431 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.713954+0000", 2024-09-18T10:15:13.431 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8070000000000002e-06 2024-09-18T10:15:13.431 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.431 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.431 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:13.431 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.904149+0000", 2024-09-18T10:15:13.432 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.19019514800000001 2024-09-18T10:15:13.432 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.432 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.432 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:13.432 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.904338+0000", 2024-09-18T10:15:13.432 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000189326 2024-09-18T10:15:13.432 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.432 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.432 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:13.432 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.905212+0000", 2024-09-18T10:15:13.432 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00087334100000000003 2024-09-18T10:15:13.432 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.433 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.433 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:13.433 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.906215+0000", 2024-09-18T10:15:13.433 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0010037430000000001 2024-09-18T10:15:13.433 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.433 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.433 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-18T10:15:13.433 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.217903+0000", 2024-09-18T10:15:13.433 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.31168731500000002 2024-09-18T10:15:13.433 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.433 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.433 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:13.433 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.218152+0000", 2024-09-18T10:15:13.434 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00024915799999999997 2024-09-18T10:15:13.434 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.434 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:13.434 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.434 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.434 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.434 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:129 3.7s0 3:e4da5b5e:::smithi18925896-43:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:13.434 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:50.713961+0000", 2024-09-18T10:15:13.434 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 22.691325275000001, 2024-09-18T10:15:13.434 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.819334071, 2024-09-18T10:15:13.434 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:13.434 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:13.435 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:13.435 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:13.435 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:13.435 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 129 2024-09-18T10:15:13.435 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.435 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:13.435 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.435 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:13.435 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.713961+0000", 2024-09-18T10:15:13.435 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.435 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.436 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.436 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:13.436 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.713961+0000", 2024-09-18T10:15:13.436 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.436 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.436 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.436 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:13.436 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.713960+0000", 2024-09-18T10:15:13.436 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.999999 2024-09-18T10:15:13.436 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.436 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.436 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:13.436 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.713964+0000", 2024-09-18T10:15:13.437 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.3059999999999999e-06 2024-09-18T10:15:13.437 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.437 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.437 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:13.437 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.713974+0000", 2024-09-18T10:15:13.437 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.9969999999999993e-06 2024-09-18T10:15:13.437 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.437 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.437 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:13.437 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.713981+0000", 2024-09-18T10:15:13.437 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1229999999999997e-06 2024-09-18T10:15:13.437 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.438 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.438 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:13.438 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.000661+0000", 2024-09-18T10:15:13.438 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.28668068699999999 2024-09-18T10:15:13.438 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.438 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.438 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for rw locks", 2024-09-18T10:15:13.438 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.000844+0000", 2024-09-18T10:15:13.438 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00018289599999999999 2024-09-18T10:15:13.438 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.438 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.438 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:13.438 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.219209+0000", 2024-09-18T10:15:13.439 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.21836466900000001 2024-09-18T10:15:13.439 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.439 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.439 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:13.439 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.219351+0000", 2024-09-18T10:15:13.439 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00014262300000000001 2024-09-18T10:15:13.439 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.439 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.439 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:13.439 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.533295+0000", 2024-09-18T10:15:13.439 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.31394377299999998 2024-09-18T10:15:13.439 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.439 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:13.439 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.440 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.440 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.440 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:156 3.7s0 3:ead53b0f:::smithi18925896-25:head [append 0~475136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:13.440 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:51.224034+0000", 2024-09-18T10:15:13.440 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 22.181252240999999, 2024-09-18T10:15:13.440 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.40749072600000003, 2024-09-18T10:15:13.440 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:13.440 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:13.440 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:13.440 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:13.440 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:13.440 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 156 2024-09-18T10:15:13.440 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.440 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:13.441 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.441 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:13.441 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.224034+0000", 2024-09-18T10:15:13.441 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.441 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.441 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.441 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:13.441 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.224034+0000", 2024-09-18T10:15:13.441 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.441 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.441 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.444 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:13.444 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.224032+0000", 2024-09-18T10:15:13.444 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:13.444 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.444 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.444 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:13.444 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.224742+0000", 2024-09-18T10:15:13.444 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00070980299999999995 2024-09-18T10:15:13.444 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.444 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.444 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:13.444 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.224767+0000", 2024-09-18T10:15:13.445 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.5415999999999999e-05 2024-09-18T10:15:13.445 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.445 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.445 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:13.445 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.224790+0000", 2024-09-18T10:15:13.445 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2932999999999999e-05 2024-09-18T10:15:13.445 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.445 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.445 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:13.445 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.224914+0000", 2024-09-18T10:15:13.445 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000123261 2024-09-18T10:15:13.445 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.445 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.446 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:13.446 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.225098+0000", 2024-09-18T10:15:13.446 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000184606 2024-09-18T10:15:13.446 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.446 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.446 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:13.446 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.226615+0000", 2024-09-18T10:15:13.446 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001517114 2024-09-18T10:15:13.446 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.446 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.446 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:13.446 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.331612+0000", 2024-09-18T10:15:13.446 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.104996909 2024-09-18T10:15:13.447 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.447 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.447 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-18T10:15:13.447 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.631319+0000", 2024-09-18T10:15:13.447 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.29970667299999998 2024-09-18T10:15:13.447 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.447 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.447 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:13.447 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.631525+0000", 2024-09-18T10:15:13.447 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000206011 2024-09-18T10:15:13.447 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.447 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:13.447 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.447 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.448 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.448 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:157 3.7s0 3:ead53b0f:::smithi18925896-25:head [append 0~376832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:13.448 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:51.226166+0000", 2024-09-18T10:15:13.448 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 22.179120093000002, 2024-09-18T10:15:13.448 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.405859368, 2024-09-18T10:15:13.448 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:13.448 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:13.448 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:13.448 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:13.448 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:13.448 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 157 2024-09-18T10:15:13.448 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.449 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:13.449 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.449 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:13.449 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.226166+0000", 2024-09-18T10:15:13.449 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.449 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.449 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.449 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:13.449 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.226166+0000", 2024-09-18T10:15:13.449 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.449 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.449 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.450 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:13.450 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.226165+0000", 2024-09-18T10:15:13.450 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:13.450 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.450 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.450 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:13.450 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.226889+0000", 2024-09-18T10:15:13.450 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00072377800000000001 2024-09-18T10:15:13.450 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.450 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.450 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:13.450 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.226913+0000", 2024-09-18T10:15:13.450 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3926000000000001e-05 2024-09-18T10:15:13.451 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.451 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.451 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:13.451 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.226925+0000", 2024-09-18T10:15:13.451 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2575000000000001e-05 2024-09-18T10:15:13.451 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.451 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.451 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:13.451 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.227730+0000", 2024-09-18T10:15:13.451 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00080524299999999997 2024-09-18T10:15:13.451 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.451 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.451 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:13.452 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.227893+0000", 2024-09-18T10:15:13.452 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00016229300000000001 2024-09-18T10:15:13.452 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.452 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.452 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:13.452 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.228969+0000", 2024-09-18T10:15:13.452 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001076341 2024-09-18T10:15:13.452 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.452 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.452 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:13.452 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.331813+0000", 2024-09-18T10:15:13.452 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10284409899999999 2024-09-18T10:15:13.453 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.453 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.453 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-18T10:15:13.453 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.631841+0000", 2024-09-18T10:15:13.453 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.30002807500000001 2024-09-18T10:15:13.453 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.453 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.453 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:13.453 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.632026+0000", 2024-09-18T10:15:13.453 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000184546 2024-09-18T10:15:13.453 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.453 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:13.454 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.454 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.454 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.454 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:158 3.7s0 3:ead53b0f:::smithi18925896-25:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:13.454 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:51.226948+0000", 2024-09-18T10:15:13.454 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 22.178338148000002, 2024-09-18T10:15:13.454 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.405732802, 2024-09-18T10:15:13.454 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:13.454 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:13.454 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:13.454 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:13.454 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:13.454 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 158 2024-09-18T10:15:13.455 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.455 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:13.455 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.455 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:13.455 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.226948+0000", 2024-09-18T10:15:13.455 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.455 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.455 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.455 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:13.455 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.226948+0000", 2024-09-18T10:15:13.455 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.455 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.455 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.455 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:13.456 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.226947+0000", 2024-09-18T10:15:13.456 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.999999 2024-09-18T10:15:13.456 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.456 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.456 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:13.456 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.226954+0000", 2024-09-18T10:15:13.456 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.6200000000000001e-06 2024-09-18T10:15:13.456 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.456 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.456 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:13.456 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.226968+0000", 2024-09-18T10:15:13.456 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3866000000000001e-05 2024-09-18T10:15:13.456 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.457 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.457 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:13.457 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.226977+0000", 2024-09-18T10:15:13.457 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.9530000000000005e-06 2024-09-18T10:15:13.457 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.457 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.457 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:13.457 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.329807+0000", 2024-09-18T10:15:13.457 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10283049700000001 2024-09-18T10:15:13.457 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.457 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.457 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:13.457 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.329976+0000", 2024-09-18T10:15:13.458 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000169226 2024-09-18T10:15:13.458 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.458 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.458 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:13.458 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.330989+0000", 2024-09-18T10:15:13.458 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001012825 2024-09-18T10:15:13.458 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.458 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.458 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:13.458 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.432222+0000", 2024-09-18T10:15:13.458 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10123270400000001 2024-09-18T10:15:13.458 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.458 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.459 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-18T10:15:13.459 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.632401+0000", 2024-09-18T10:15:13.459 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.200179352 2024-09-18T10:15:13.459 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.459 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.459 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:13.459 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.632681+0000", 2024-09-18T10:15:13.459 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00027994700000000001 2024-09-18T10:15:13.459 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.459 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:13.459 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.460 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.460 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.460 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:159 3.7s0 3:ead53b0f:::smithi18925896-25:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:13.460 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:51.226988+0000", 2024-09-18T10:15:13.460 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 22.178298215000002, 2024-09-18T10:15:13.460 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.81465928499999996, 2024-09-18T10:15:13.460 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:13.460 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:13.460 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:13.460 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:13.460 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:13.460 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 159 2024-09-18T10:15:13.461 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.461 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:13.461 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.461 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:13.461 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.226988+0000", 2024-09-18T10:15:13.461 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.461 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.461 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.461 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:13.461 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.226988+0000", 2024-09-18T10:15:13.461 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.461 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.461 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.462 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:13.462 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.226987+0000", 2024-09-18T10:15:13.462 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.999999 2024-09-18T10:15:13.462 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.462 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.462 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:13.462 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.226992+0000", 2024-09-18T10:15:13.462 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.5900000000000001e-06 2024-09-18T10:15:13.462 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.462 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.462 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:13.462 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.227003+0000", 2024-09-18T10:15:13.462 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1222e-05 2024-09-18T10:15:13.463 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.463 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.463 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:13.463 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.227011+0000", 2024-09-18T10:15:13.463 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.6279999999999996e-06 2024-09-18T10:15:13.463 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.463 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.463 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:13.463 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.533792+0000", 2024-09-18T10:15:13.463 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.30678158100000003 2024-09-18T10:15:13.463 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.463 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.463 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for rw locks", 2024-09-18T10:15:13.464 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.533941+0000", 2024-09-18T10:15:13.464 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00014846 2024-09-18T10:15:13.464 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.464 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.464 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:13.464 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.633576+0000", 2024-09-18T10:15:13.464 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099635153000000004 2024-09-18T10:15:13.464 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.464 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.464 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:13.464 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.633832+0000", 2024-09-18T10:15:13.464 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00025610699999999999 2024-09-18T10:15:13.464 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.464 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.465 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:13.465 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.041647+0000", 2024-09-18T10:15:13.465 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.407815388 2024-09-18T10:15:13.465 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.465 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:13.465 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.465 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.465 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.465 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:166 3.1s0 3:82ca2386:::smithi18925896-8:head [read 0~131072,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:13.465 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:51.531989+0000", 2024-09-18T10:15:13.465 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 21.873297475000001, 2024-09-18T10:15:13.465 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.40380508900000001, 2024-09-18T10:15:13.465 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:13.465 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:13.466 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:13.466 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:13.466 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:13.466 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 166 2024-09-18T10:15:13.466 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.466 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:13.466 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.466 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:13.466 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.531989+0000", 2024-09-18T10:15:13.466 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.466 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.466 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.466 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:13.467 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.531989+0000", 2024-09-18T10:15:13.467 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.467 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.467 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.467 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:13.467 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.531987+0000", 2024-09-18T10:15:13.467 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:13.467 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.467 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.467 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:13.467 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.531997+0000", 2024-09-18T10:15:13.467 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0010000000000001e-05 2024-09-18T10:15:13.467 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.468 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.468 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:13.468 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.532017+0000", 2024-09-18T10:15:13.468 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9358999999999999e-05 2024-09-18T10:15:13.468 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.468 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.468 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:13.468 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.532032+0000", 2024-09-18T10:15:13.468 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4739999999999999e-05 2024-09-18T10:15:13.468 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.468 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.468 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:13.469 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.733011+0000", 2024-09-18T10:15:13.469 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.200979398 2024-09-18T10:15:13.469 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.469 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.469 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:13.469 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.733223+0000", 2024-09-18T10:15:13.469 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00021232799999999999 2024-09-18T10:15:13.469 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.469 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.469 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:13.469 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.935794+0000", 2024-09-18T10:15:13.469 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.20257077400000001 2024-09-18T10:15:13.469 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.469 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:13.470 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.470 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.470 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.470 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:169 3.7s0 3:e1df0280:::smithi18925896-26:head [copy-from ver 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:13.470 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:51.532055+0000", 2024-09-18T10:15:13.470 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 21.873231026999999, 2024-09-18T10:15:13.470 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.60955003100000005, 2024-09-18T10:15:13.470 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:13.470 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:13.470 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:13.470 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:13.470 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:13.470 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 169 2024-09-18T10:15:13.471 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.471 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:13.471 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.471 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:13.471 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.532055+0000", 2024-09-18T10:15:13.471 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.471 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.471 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.471 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:13.471 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.532055+0000", 2024-09-18T10:15:13.471 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.471 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.471 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.471 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:13.471 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.532054+0000", 2024-09-18T10:15:13.472 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:13.472 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.472 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.472 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:13.472 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.532062+0000", 2024-09-18T10:15:13.472 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0430000000000008e-06 2024-09-18T10:15:13.472 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.472 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.472 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:13.472 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.532080+0000", 2024-09-18T10:15:13.472 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8283000000000001e-05 2024-09-18T10:15:13.472 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.472 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.473 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:13.473 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.532092+0000", 2024-09-18T10:15:13.473 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1742999999999999e-05 2024-09-18T10:15:13.473 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.473 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.473 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:13.473 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.534038+0000", 2024-09-18T10:15:13.473 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0019461610000000001 2024-09-18T10:15:13.473 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.473 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.473 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:13.473 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.534285+0000", 2024-09-18T10:15:13.473 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00024667800000000001 2024-09-18T10:15:13.474 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.474 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.474 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:13.474 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.938506+0000", 2024-09-18T10:15:13.474 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.40422132199999999 2024-09-18T10:15:13.474 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.474 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.474 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:13.474 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.041810+0000", 2024-09-18T10:15:13.474 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10330328900000001 2024-09-18T10:15:13.474 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.474 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.474 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-18T10:15:13.475 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.141492+0000", 2024-09-18T10:15:13.475 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099682251999999999 2024-09-18T10:15:13.475 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.475 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.475 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:13.475 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.141605+0000", 2024-09-18T10:15:13.475 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00011349300000000001 2024-09-18T10:15:13.475 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.475 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:13.475 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.475 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.475 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.475 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:170 3.7s0 3:e1df0280:::smithi18925896-26:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:13.476 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:51.532106+0000", 2024-09-18T10:15:13.476 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 21.873180747999999, 2024-09-18T10:15:13.476 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.50908735800000005, 2024-09-18T10:15:13.476 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:13.476 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:13.476 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:13.476 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:13.476 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:13.476 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 170 2024-09-18T10:15:13.476 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.476 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:13.476 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.476 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:13.477 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.532106+0000", 2024-09-18T10:15:13.477 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.477 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.477 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.477 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:13.477 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.532106+0000", 2024-09-18T10:15:13.477 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.477 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.477 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.477 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:13.477 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.532104+0000", 2024-09-18T10:15:13.477 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:13.477 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.477 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.478 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:13.478 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.532113+0000", 2024-09-18T10:15:13.478 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8659999999999995e-06 2024-09-18T10:15:13.478 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.478 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.478 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:13.478 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.532131+0000", 2024-09-18T10:15:13.478 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7723999999999998e-05 2024-09-18T10:15:13.478 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.478 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.478 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:13.478 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.532142+0000", 2024-09-18T10:15:13.478 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0669000000000001e-05 2024-09-18T10:15:13.478 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.479 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.479 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:13.479 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.534705+0000", 2024-09-18T10:15:13.479 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0025635380000000002 2024-09-18T10:15:13.479 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.479 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.479 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for blocked object", 2024-09-18T10:15:13.479 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.534831+0000", 2024-09-18T10:15:13.479 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00012540399999999999 2024-09-18T10:15:13.479 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.479 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.479 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:13.480 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.040545+0000", 2024-09-18T10:15:13.480 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.50571483699999997 2024-09-18T10:15:13.480 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.480 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.480 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:13.480 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.040869+0000", 2024-09-18T10:15:13.480 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00032386999999999997 2024-09-18T10:15:13.480 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.480 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.480 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:13.480 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.041193+0000", 2024-09-18T10:15:13.480 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000323803 2024-09-18T10:15:13.481 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.481 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:13.481 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.481 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.481 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.481 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:210 3.7s0 3:e4da5b5e:::smithi18925896-43:head [read 0~720896,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:13.481 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:52.694421+0000", 2024-09-18T10:15:13.481 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 20.710865282, 2024-09-18T10:15:13.481 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.40910238700000001, 2024-09-18T10:15:13.481 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:13.481 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:13.481 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:13.482 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:13.482 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:13.482 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 210 2024-09-18T10:15:13.482 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.482 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:13.482 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.482 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:13.482 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.694421+0000", 2024-09-18T10:15:13.482 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.482 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.482 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.482 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:13.483 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.694421+0000", 2024-09-18T10:15:13.483 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.483 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.483 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.483 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:13.483 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.694420+0000", 2024-09-18T10:15:13.483 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:13.483 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.483 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.483 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:13.483 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.694428+0000", 2024-09-18T10:15:13.483 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0520000000000002e-06 2024-09-18T10:15:13.483 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.483 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.484 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:13.484 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.694445+0000", 2024-09-18T10:15:13.484 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7479e-05 2024-09-18T10:15:13.484 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.484 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.484 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:13.484 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.694456+0000", 2024-09-18T10:15:13.484 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0302999999999999e-05 2024-09-18T10:15:13.484 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.484 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.484 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:13.484 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.694858+0000", 2024-09-18T10:15:13.484 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00040244199999999998 2024-09-18T10:15:13.485 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.485 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.485 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:13.485 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.694874+0000", 2024-09-18T10:15:13.485 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.6248000000000001e-05 2024-09-18T10:15:13.485 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.485 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.485 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:13.485 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.896790+0000", 2024-09-18T10:15:13.485 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.201915813 2024-09-18T10:15:13.485 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.485 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.485 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:13.486 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.897060+0000", 2024-09-18T10:15:13.486 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00026979400000000001 2024-09-18T10:15:13.486 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.486 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.486 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:13.486 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:53.103524+0000", 2024-09-18T10:15:13.486 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.20646349999999999 2024-09-18T10:15:13.486 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.486 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:13.486 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.486 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.486 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.487 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:211 3.7s0 3:e4da5b5e:::smithi18925896-43:head [read 0~720896,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:13.487 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:52.694469+0000", 2024-09-18T10:15:13.487 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 20.71081693, 2024-09-18T10:15:13.487 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.51035930399999996, 2024-09-18T10:15:13.487 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:13.487 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:13.487 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:13.487 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:13.487 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:13.487 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 211 2024-09-18T10:15:13.487 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.487 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:13.487 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.487 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:13.488 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.694469+0000", 2024-09-18T10:15:13.488 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.488 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.488 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.488 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:13.488 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.694469+0000", 2024-09-18T10:15:13.488 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.488 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.488 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.488 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:13.488 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.694469+0000", 2024-09-18T10:15:13.488 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.999999 2024-09-18T10:15:13.489 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.489 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.489 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:13.489 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.694475+0000", 2024-09-18T10:15:13.489 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7349999999999999e-06 2024-09-18T10:15:13.489 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.489 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.489 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:13.489 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.694489+0000", 2024-09-18T10:15:13.489 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4209999999999999e-05 2024-09-18T10:15:13.489 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.489 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.490 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:13.490 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.694498+0000", 2024-09-18T10:15:13.490 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.4339999999999992e-06 2024-09-18T10:15:13.490 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.490 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.490 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:13.490 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.698348+0000", 2024-09-18T10:15:13.490 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0038499670000000001 2024-09-18T10:15:13.490 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.490 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.490 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:13.490 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.698366+0000", 2024-09-18T10:15:13.490 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7999000000000002e-05 2024-09-18T10:15:13.491 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.491 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.491 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:13.491 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:53.000010+0000", 2024-09-18T10:15:13.491 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.30164412699999998 2024-09-18T10:15:13.491 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.491 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.491 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:13.491 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:53.000223+0000", 2024-09-18T10:15:13.491 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000213169 2024-09-18T10:15:13.491 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.491 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.491 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:13.491 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:53.204829+0000", 2024-09-18T10:15:13.492 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.20460561299999999 2024-09-18T10:15:13.492 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.492 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:13.492 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.492 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.492 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.492 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(osd.10.22:2 3.7s0 3:ee962131:::smithi18925896-29:head [assert-version v6,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:13.492 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:52.798083+0000", 2024-09-18T10:15:13.492 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 20.607203038000002, 2024-09-18T10:15:13.492 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.60989146800000005, 2024-09-18T10:15:13.492 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:13.492 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:13.492 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:13.493 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.493 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:13.493 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.798083+0000", 2024-09-18T10:15:13.493 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.493 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.493 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.493 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:13.493 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.798083+0000", 2024-09-18T10:15:13.493 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.493 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.493 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.493 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:13.493 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.798080+0000", 2024-09-18T10:15:13.494 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999962 2024-09-18T10:15:13.494 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.494 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.494 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:13.494 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.798097+0000", 2024-09-18T10:15:13.494 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7657000000000001e-05 2024-09-18T10:15:13.494 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.494 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.494 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:13.494 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.798127+0000", 2024-09-18T10:15:13.494 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9532000000000001e-05 2024-09-18T10:15:13.494 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.494 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.495 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:13.495 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.898205+0000", 2024-09-18T10:15:13.495 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100078525 2024-09-18T10:15:13.495 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.495 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.495 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:13.495 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:53.204957+0000", 2024-09-18T10:15:13.495 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.30675138899999999 2024-09-18T10:15:13.495 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.495 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.495 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:13.495 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:53.205092+0000", 2024-09-18T10:15:13.495 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00013556700000000001 2024-09-18T10:15:13.496 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.496 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.496 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:13.496 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:53.407975+0000", 2024-09-18T10:15:13.496 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.20288240799999999 2024-09-18T10:15:13.496 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.496 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:13.497 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.497 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.497 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.497 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:411 3.1s0 3:82c9401b:::smithi18925896-33:head [read 0~540672,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-18T10:15:13.497 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:07.554062+0000", 2024-09-18T10:15:13.498 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 5.8512247989999997, 2024-09-18T10:15:13.498 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.41060208599999998, 2024-09-18T10:15:13.498 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:13.498 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:13.498 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:13.498 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:13.498 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:13.498 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 411 2024-09-18T10:15:13.498 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.498 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:13.498 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.498 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:13.498 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.554062+0000", 2024-09-18T10:15:13.499 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.499 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.499 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.499 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:13.499 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.554062+0000", 2024-09-18T10:15:13.499 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.499 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.499 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.499 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:13.499 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.554060+0000", 2024-09-18T10:15:13.499 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:13.499 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.499 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.499 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:13.500 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.554071+0000", 2024-09-18T10:15:13.500 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1207e-05 2024-09-18T10:15:13.500 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.500 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.500 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:13.501 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.554093+0000", 2024-09-18T10:15:13.501 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1956000000000001e-05 2024-09-18T10:15:13.501 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.501 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.501 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:13.501 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.554106+0000", 2024-09-18T10:15:13.501 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2866e-05 2024-09-18T10:15:13.501 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.501 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.501 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:13.501 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.554190+0000", 2024-09-18T10:15:13.501 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.4133000000000002e-05 2024-09-18T10:15:13.501 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.502 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.502 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:13.502 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.554196+0000", 2024-09-18T10:15:13.502 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0850000000000002e-06 2024-09-18T10:15:13.502 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.502 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.502 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:13.502 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.658151+0000", 2024-09-18T10:15:13.502 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10395459899999999 2024-09-18T10:15:13.502 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.502 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.502 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:13.502 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.658470+0000", 2024-09-18T10:15:13.502 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00031953999999999999 2024-09-18T10:15:13.502 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.503 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.503 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:13.503 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.964664+0000", 2024-09-18T10:15:13.503 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.30619342300000002 2024-09-18T10:15:13.503 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.503 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:13.503 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.503 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.503 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.503 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:412 3.1s0 3:82c9401b:::smithi18925896-33:head [read 0~540672,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-18T10:15:13.503 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:07.554121+0000", 2024-09-18T10:15:13.503 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 5.8511651960000002, 2024-09-18T10:15:13.503 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.41171843299999999, 2024-09-18T10:15:13.503 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:13.504 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:13.504 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:13.504 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:13.504 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:13.504 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 412 2024-09-18T10:15:13.504 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.504 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:13.504 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.504 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:13.504 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.554121+0000", 2024-09-18T10:15:13.504 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.504 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.504 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.505 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:13.505 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.554121+0000", 2024-09-18T10:15:13.505 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.505 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.505 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.505 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:13.505 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.554120+0000", 2024-09-18T10:15:13.505 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:13.505 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.505 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.505 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:13.505 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.554128+0000", 2024-09-18T10:15:13.505 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.943e-06 2024-09-18T10:15:13.505 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.505 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.506 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:13.506 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.554146+0000", 2024-09-18T10:15:13.506 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7880000000000002e-05 2024-09-18T10:15:13.506 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.506 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.506 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:13.506 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.554162+0000", 2024-09-18T10:15:13.506 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5931999999999999e-05 2024-09-18T10:15:13.506 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.506 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.506 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:13.506 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.554405+0000", 2024-09-18T10:15:13.506 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000243515 2024-09-18T10:15:13.506 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.507 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.507 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:13.507 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.554428+0000", 2024-09-18T10:15:13.507 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2476e-05 2024-09-18T10:15:13.507 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.507 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.507 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:13.507 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.659266+0000", 2024-09-18T10:15:13.507 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10483785299999999 2024-09-18T10:15:13.507 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.507 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.507 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:13.507 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.659461+0000", 2024-09-18T10:15:13.507 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00019562200000000001 2024-09-18T10:15:13.508 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.508 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.508 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:13.508 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.965840+0000", 2024-09-18T10:15:13.508 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.30637847200000001 2024-09-18T10:15:13.508 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.508 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:13.508 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.508 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.508 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.508 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:419 3.17s0 3:ee962131:::smithi18925896-29:head [delete] snapc 10=[10,f,e,b,4] ondisk+write+known_if_redirected+full_force+supports_pool_eio e53)", 2024-09-18T10:15:13.508 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:08.547155+0000", 2024-09-18T10:15:13.508 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 4.8581314259999999, 2024-09-18T10:15:13.509 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.40648379899999998, 2024-09-18T10:15:13.509 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:13.509 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:13.509 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:13.509 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:13.509 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:13.509 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 419 2024-09-18T10:15:13.509 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.509 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:13.509 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.509 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:13.509 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:08.547155+0000", 2024-09-18T10:15:13.509 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.510 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.510 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.510 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:13.510 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:08.547155+0000", 2024-09-18T10:15:13.510 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.510 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.510 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.510 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:13.510 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:08.547152+0000", 2024-09-18T10:15:13.510 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999976 2024-09-18T10:15:13.510 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.510 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.510 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:13.511 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:08.547166+0000", 2024-09-18T10:15:13.511 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4008999999999999e-05 2024-09-18T10:15:13.511 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.511 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.511 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:13.511 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:08.547197+0000", 2024-09-18T10:15:13.511 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0703000000000001e-05 2024-09-18T10:15:13.511 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.511 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.511 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:13.511 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:08.547213+0000", 2024-09-18T10:15:13.511 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5713e-05 2024-09-18T10:15:13.511 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.511 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.512 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:13.512 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:08.547344+0000", 2024-09-18T10:15:13.512 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00013137000000000001 2024-09-18T10:15:13.512 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.512 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.512 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op must wait for map", 2024-09-18T10:15:13.512 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:08.547364+0000", 2024-09-18T10:15:13.512 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9879999999999999e-05 2024-09-18T10:15:13.512 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.512 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.512 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:13.512 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:08.745910+0000", 2024-09-18T10:15:13.512 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.19854606399999999 2024-09-18T10:15:13.513 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.513 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.513 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:13.513 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:08.746238+0000", 2024-09-18T10:15:13.513 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00032790899999999998 2024-09-18T10:15:13.513 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.513 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.513 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:13.513 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:08.747857+0000", 2024-09-18T10:15:13.513 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0016184420000000001 2024-09-18T10:15:13.513 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.513 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.513 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:13.513 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:08.749417+0000", 2024-09-18T10:15:13.514 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001560929 2024-09-18T10:15:13.514 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.514 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.514 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-18T10:15:13.514 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:08.953500+0000", 2024-09-18T10:15:13.514 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.204082122 2024-09-18T10:15:13.514 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.514 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.514 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:13.514 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:08.953639+0000", 2024-09-18T10:15:13.514 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000139231 2024-09-18T10:15:13.514 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.514 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:13.514 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.515 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.515 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.515 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:437 3.1s0 3:82c9401b:::smithi18925896-33:head [copy-from ver 2] snapc 13=[13,12,11,10,f,e,b,4] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-18T10:15:13.515 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:11.831109+0000", 2024-09-18T10:15:13.515 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 1.5741772599999999, 2024-09-18T10:15:13.515 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.709244349, 2024-09-18T10:15:13.515 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:13.515 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:13.515 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:13.515 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:13.515 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:13.515 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 437 2024-09-18T10:15:13.515 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.516 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:13.516 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.516 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:13.516 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:11.831109+0000", 2024-09-18T10:15:13.516 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.516 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.516 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.516 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:13.516 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:11.831109+0000", 2024-09-18T10:15:13.516 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.516 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.516 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.516 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:13.516 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:11.831106+0000", 2024-09-18T10:15:13.517 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999971 2024-09-18T10:15:13.517 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.517 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.517 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:13.517 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:11.831122+0000", 2024-09-18T10:15:13.517 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5971e-05 2024-09-18T10:15:13.517 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.517 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.517 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:13.517 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:11.831155+0000", 2024-09-18T10:15:13.517 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.3259999999999997e-05 2024-09-18T10:15:13.517 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.517 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.517 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:13.518 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:11.831172+0000", 2024-09-18T10:15:13.518 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.6433999999999999e-05 2024-09-18T10:15:13.518 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.518 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.518 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:13.518 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:11.931398+0000", 2024-09-18T10:15:13.518 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10022644 2024-09-18T10:15:13.518 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.518 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.518 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op must wait for map", 2024-09-18T10:15:13.518 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:11.931412+0000", 2024-09-18T10:15:13.518 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3586e-05 2024-09-18T10:15:13.518 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.518 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.519 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:13.519 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:11.934231+0000", 2024-09-18T10:15:13.519 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0028191819999999999 2024-09-18T10:15:13.519 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.519 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.519 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:13.519 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:11.934501+0000", 2024-09-18T10:15:13.519 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00026996099999999999 2024-09-18T10:15:13.519 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.519 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.519 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:13.519 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.352429+0000", 2024-09-18T10:15:13.519 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.41792792699999998 2024-09-18T10:15:13.519 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.520 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.520 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:13.520 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.356112+0000", 2024-09-18T10:15:13.520 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0036833199999999999 2024-09-18T10:15:13.520 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.520 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.520 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-18T10:15:13.520 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.540229+0000", 2024-09-18T10:15:13.520 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.18411702599999999 2024-09-18T10:15:13.520 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.520 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.520 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:13.520 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.540354+0000", 2024-09-18T10:15:13.521 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00012415399999999999 2024-09-18T10:15:13.521 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.521 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:13.521 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.521 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.521 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.521 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:438 3.1s0 3:82c9401b:::smithi18925896-33:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e56)", 2024-09-18T10:15:13.521 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:11.831206+0000", 2024-09-18T10:15:13.521 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 1.5740806780000001, 2024-09-18T10:15:13.521 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.52445609999999998, 2024-09-18T10:15:13.521 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:13.521 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:13.521 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:13.521 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:13.522 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:13.522 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 438 2024-09-18T10:15:13.522 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.522 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:13.522 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.522 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:13.522 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:11.831206+0000", 2024-09-18T10:15:13.522 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.522 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.522 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.522 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:13.522 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:11.831206+0000", 2024-09-18T10:15:13.522 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.522 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.523 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.523 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:13.523 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:11.831205+0000", 2024-09-18T10:15:13.523 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.999999 2024-09-18T10:15:13.523 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.523 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.523 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:13.523 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:11.831210+0000", 2024-09-18T10:15:13.523 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8539999999999996e-06 2024-09-18T10:15:13.523 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.523 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.523 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:13.523 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:11.831230+0000", 2024-09-18T10:15:13.523 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9899999999999999e-05 2024-09-18T10:15:13.524 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.524 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.524 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:13.524 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:11.831238+0000", 2024-09-18T10:15:13.524 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.7639999999999994e-06 2024-09-18T10:15:13.524 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.524 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.524 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:13.524 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:11.931577+0000", 2024-09-18T10:15:13.524 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10033874299999999 2024-09-18T10:15:13.524 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.524 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.524 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:13.524 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:11.931604+0000", 2024-09-18T10:15:13.525 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.7239000000000001e-05 2024-09-18T10:15:13.525 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.525 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.525 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:13.525 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:11.935139+0000", 2024-09-18T10:15:13.525 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00353484 2024-09-18T10:15:13.525 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.525 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.525 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for blocked object", 2024-09-18T10:15:13.525 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:11.935380+0000", 2024-09-18T10:15:13.525 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00024033400000000001 2024-09-18T10:15:13.525 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.525 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.525 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:13.526 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.355102+0000", 2024-09-18T10:15:13.526 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.41972236200000002 2024-09-18T10:15:13.526 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.526 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.526 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:13.526 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.355403+0000", 2024-09-18T10:15:13.526 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000301537 2024-09-18T10:15:13.526 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.526 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.526 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:13.526 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.355662+0000", 2024-09-18T10:15:13.526 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000258421 2024-09-18T10:15:13.526 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.526 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:13.527 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.527 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.527 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:13.527 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-18T10:15:13.528 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-09-18T10:15:13.529 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-18T10:15:13.525+0000 7f1981873700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22132) UID: 0 2024-09-18T10:15:13.551 INFO:tasks.rados.rados.0.smithi189.stdout:178: done (2 left) 2024-09-18T10:15:13.552 INFO:tasks.rados.rados.0.smithi189.stdout:179: delete oid 22 current snap is 13 2024-09-18T10:15:13.556 INFO:teuthology.orchestra.run.smithi029.stderr:reweighted osd.4 to 1 (10000) 2024-09-18T10:15:13.556 INFO:teuthology.orchestra.run.smithi029.stderr:noscrub is unset 2024-09-18T10:15:13.574 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-18T10:15:13.629 INFO:tasks.ceph.osd.4.smithi029.stderr:2024-09-18T10:15:13.622+0000 7f550cf6e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22189) UID: 0 2024-09-18T10:15:13.639 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-18T10:15:13.640 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-18T10:15:13.640 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.640 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 57/40 ECSubWriteReply(tid=32, last_complete=57'20, committed=1, applied=1))", 2024-09-18T10:15:13.640 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:13.469653+0000", 2024-09-18T10:15:13.640 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.16752433999999999, 2024-09-18T10:15:13.640 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.16757960199999999, 2024-09-18T10:15:13.640 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:13.640 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-18T10:15:13.640 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:13.640 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.640 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:13.641 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.469653+0000", 2024-09-18T10:15:13.641 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.641 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.641 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.641 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:13.641 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.469653+0000", 2024-09-18T10:15:13.641 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.641 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.641 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.641 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:13.642 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.469651+0000", 2024-09-18T10:15:13.642 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:13.642 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.642 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.642 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:13.642 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.469662+0000", 2024-09-18T10:15:13.642 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0519999999999999e-05 2024-09-18T10:15:13.642 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.642 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.642 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:13.642 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.469693+0000", 2024-09-18T10:15:13.642 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1566000000000002e-05 2024-09-18T10:15:13.642 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.643 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.643 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:13.643 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.569787+0000", 2024-09-18T10:15:13.643 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10009314800000001 2024-09-18T10:15:13.643 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.643 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:13.643 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.643 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.643 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.643 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 57/40 ECSubWriteReply(tid=31, last_complete=57'19, committed=1, applied=1))", 2024-09-18T10:15:13.643 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:13.547860+0000", 2024-09-18T10:15:13.643 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.089317231999999996, 2024-09-18T10:15:13.643 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.089539634000000007, 2024-09-18T10:15:13.643 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:13.644 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-18T10:15:13.644 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:13.644 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.644 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:13.644 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.547860+0000", 2024-09-18T10:15:13.644 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.644 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.644 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.644 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:13.644 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.547860+0000", 2024-09-18T10:15:13.644 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.644 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.644 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.645 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:13.645 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.547859+0000", 2024-09-18T10:15:13.645 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.999999 2024-09-18T10:15:13.645 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.645 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.645 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:13.645 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.547871+0000", 2024-09-18T10:15:13.645 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.242e-05 2024-09-18T10:15:13.645 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.646 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.646 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:13.646 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.547924+0000", 2024-09-18T10:15:13.646 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2522000000000003e-05 2024-09-18T10:15:13.646 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.646 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.646 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:13.646 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.547938+0000", 2024-09-18T10:15:13.646 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4117000000000001e-05 2024-09-18T10:15:13.646 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.646 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:13.646 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.646 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.647 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.647 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 57/40 ECSubWriteReply(tid=32, last_complete=57'20, committed=1, applied=1))", 2024-09-18T10:15:13.647 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:13.547948+0000", 2024-09-18T10:15:13.647 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.089228590999999996, 2024-09-18T10:15:13.647 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.089529125000000001, 2024-09-18T10:15:13.647 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:13.647 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-18T10:15:13.647 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:13.647 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.647 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:13.647 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.547948+0000", 2024-09-18T10:15:13.647 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.647 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.648 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.648 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:13.648 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.547948+0000", 2024-09-18T10:15:13.648 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.648 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.648 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.648 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:13.648 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.547948+0000", 2024-09-18T10:15:13.648 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:13.648 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.648 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.648 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:13.648 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.547953+0000", 2024-09-18T10:15:13.649 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2660000000000001e-06 2024-09-18T10:15:13.649 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.649 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.649 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:13.649 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.547968+0000", 2024-09-18T10:15:13.649 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5183e-05 2024-09-18T10:15:13.649 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.649 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.649 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:13.649 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.547973+0000", 2024-09-18T10:15:13.649 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1900000000000003e-06 2024-09-18T10:15:13.649 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.649 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:13.649 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.650 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.650 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.650 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 57/40 ECSubWriteReply(tid=33, last_complete=57'21, committed=1, applied=1))", 2024-09-18T10:15:13.650 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:13.547984+0000", 2024-09-18T10:15:13.650 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.089193301000000003, 2024-09-18T10:15:13.650 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.089567655999999995, 2024-09-18T10:15:13.650 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:13.650 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-18T10:15:13.650 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:13.650 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.650 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:13.650 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.547984+0000", 2024-09-18T10:15:13.650 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.650 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.651 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.651 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:13.651 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.547984+0000", 2024-09-18T10:15:13.651 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.651 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.651 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.651 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:13.651 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.547983+0000", 2024-09-18T10:15:13.651 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:13.651 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.651 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.651 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:13.651 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.547987+0000", 2024-09-18T10:15:13.651 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8469999999999998e-06 2024-09-18T10:15:13.652 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.652 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.652 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:13.652 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.548000+0000", 2024-09-18T10:15:13.652 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2593e-05 2024-09-18T10:15:13.652 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.652 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.652 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:13.652 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.548006+0000", 2024-09-18T10:15:13.652 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.4559999999999997e-06 2024-09-18T10:15:13.652 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.652 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:13.652 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.652 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.653 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.653 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 57/40 ECSubWriteReply(tid=34, last_complete=57'22, committed=1, applied=1))", 2024-09-18T10:15:13.653 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:13.548016+0000", 2024-09-18T10:15:13.653 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.089160991999999994, 2024-09-18T10:15:13.653 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.089607409999999998, 2024-09-18T10:15:13.653 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:13.653 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-18T10:15:13.653 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:13.653 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.653 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:13.653 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.548016+0000", 2024-09-18T10:15:13.653 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.653 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.653 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.654 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:13.654 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.548016+0000", 2024-09-18T10:15:13.654 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.654 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.654 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.654 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:13.654 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.548016+0000", 2024-09-18T10:15:13.654 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:13.654 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.654 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.654 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:13.654 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.548019+0000", 2024-09-18T10:15:13.654 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.4400000000000001e-06 2024-09-18T10:15:13.654 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.655 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.655 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:13.655 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.548032+0000", 2024-09-18T10:15:13.655 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2386e-05 2024-09-18T10:15:13.655 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.655 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.655 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:13.655 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.548038+0000", 2024-09-18T10:15:13.655 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3430000000000001e-06 2024-09-18T10:15:13.655 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.655 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:13.655 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.656 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.656 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.656 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 57/40 ECSubWriteReply(tid=33, last_complete=57'21, committed=1, applied=1))", 2024-09-18T10:15:13.656 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:13.569808+0000", 2024-09-18T10:15:13.656 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.067368603999999999, 2024-09-18T10:15:13.656 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.067888722999999998, 2024-09-18T10:15:13.656 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:13.656 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-18T10:15:13.656 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:13.656 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.656 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:13.656 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.569808+0000", 2024-09-18T10:15:13.656 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.656 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.657 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.657 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:13.657 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.569808+0000", 2024-09-18T10:15:13.657 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.657 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.657 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.657 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:13.657 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.569808+0000", 2024-09-18T10:15:13.657 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.999999 2024-09-18T10:15:13.657 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.657 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.657 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:13.657 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.569815+0000", 2024-09-18T10:15:13.658 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.3540000000000002e-06 2024-09-18T10:15:13.658 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.658 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.658 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:13.658 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.569829+0000", 2024-09-18T10:15:13.658 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3879999999999999e-05 2024-09-18T10:15:13.658 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.658 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.658 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:13.658 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.569838+0000", 2024-09-18T10:15:13.658 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6119999999999999e-06 2024-09-18T10:15:13.658 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.658 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:13.659 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.659 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.659 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.659 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 57/40 ECSubWriteReply(tid=34, last_complete=57'22, committed=1, applied=1))", 2024-09-18T10:15:13.659 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:13.569854+0000", 2024-09-18T10:15:13.659 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.067323530000000006, 2024-09-18T10:15:13.659 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.067915424000000002, 2024-09-18T10:15:13.659 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:13.659 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-18T10:15:13.659 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:13.659 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.659 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:13.659 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.569854+0000", 2024-09-18T10:15:13.660 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.660 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.660 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.660 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:13.660 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.569854+0000", 2024-09-18T10:15:13.660 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.660 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.660 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.660 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:13.660 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.569853+0000", 2024-09-18T10:15:13.660 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967296 2024-09-18T10:15:13.660 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.660 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.661 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:13.661 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.569855+0000", 2024-09-18T10:15:13.661 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.911e-06 2024-09-18T10:15:13.661 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.661 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.661 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:13.661 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.569862+0000", 2024-09-18T10:15:13.661 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8750000000000002e-06 2024-09-18T10:15:13.661 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.661 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.661 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:13.661 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.569865+0000", 2024-09-18T10:15:13.661 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0180000000000002e-06 2024-09-18T10:15:13.661 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.662 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:13.662 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.662 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.662 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.662 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:446 3.4s0 3:2650be24:::smithi18925896-45:head [append 0~622592 in=622592b] snapc 13=[13,12,11,10,f,e,b] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-18T10:15:13.662 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:12.892106+0000", 2024-09-18T10:15:13.662 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.74507077600000005, 2024-09-18T10:15:13.662 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.74575570499999999, 2024-09-18T10:15:13.662 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:13.662 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:13.662 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:13.662 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:13.662 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:13.662 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 446 2024-09-18T10:15:13.663 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.663 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:13.663 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.663 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:13.663 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.892106+0000", 2024-09-18T10:15:13.663 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.663 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.663 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.663 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:13.663 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.892106+0000", 2024-09-18T10:15:13.663 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.663 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.663 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.664 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:13.664 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.892103+0000", 2024-09-18T10:15:13.664 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999971 2024-09-18T10:15:13.664 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.664 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.664 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:13.664 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.893050+0000", 2024-09-18T10:15:13.664 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00094607000000000003 2024-09-18T10:15:13.664 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.664 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.664 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:13.664 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.893084+0000", 2024-09-18T10:15:13.664 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.4759999999999999e-05 2024-09-18T10:15:13.665 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.665 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.665 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:13.665 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.893098+0000", 2024-09-18T10:15:13.665 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3859000000000001e-05 2024-09-18T10:15:13.665 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.665 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.665 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:13.665 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.145987+0000", 2024-09-18T10:15:13.665 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.252889115 2024-09-18T10:15:13.665 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.665 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.665 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:13.665 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.146112+0000", 2024-09-18T10:15:13.666 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00012469299999999999 2024-09-18T10:15:13.666 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.666 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.666 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:13.666 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.146923+0000", 2024-09-18T10:15:13.666 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00081091500000000005 2024-09-18T10:15:13.666 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.666 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.666 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:13.666 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.152437+0000", 2024-09-18T10:15:13.666 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0055141169999999998 2024-09-18T10:15:13.666 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.666 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:13.666 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.667 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.667 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.667 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:447 3.4s0 3:2650be24:::smithi18925896-45:head [append 0~475136 in=475136b] snapc 13=[13,12,11,10,f,e,b] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-18T10:15:13.667 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:12.896134+0000", 2024-09-18T10:15:13.667 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.74104341299999998, 2024-09-18T10:15:13.667 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.74186489600000005, 2024-09-18T10:15:13.667 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:13.667 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:13.667 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:13.667 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:13.667 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:13.667 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 447 2024-09-18T10:15:13.667 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.667 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:13.668 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.668 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:13.668 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.896134+0000", 2024-09-18T10:15:13.668 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.668 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.668 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.668 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:13.668 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.896134+0000", 2024-09-18T10:15:13.668 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.668 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.668 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.668 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:13.668 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.896131+0000", 2024-09-18T10:15:13.668 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999976 2024-09-18T10:15:13.669 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.669 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.669 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:13.669 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.896661+0000", 2024-09-18T10:15:13.669 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00052970299999999997 2024-09-18T10:15:13.669 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.669 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.669 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:13.669 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.896693+0000", 2024-09-18T10:15:13.669 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1903000000000003e-05 2024-09-18T10:15:13.669 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.669 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.669 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:13.670 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.896705+0000", 2024-09-18T10:15:13.670 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2288e-05 2024-09-18T10:15:13.670 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.670 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.670 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:13.670 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.147534+0000", 2024-09-18T10:15:13.670 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.25082866300000001 2024-09-18T10:15:13.670 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.670 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.670 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:13.670 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.147612+0000", 2024-09-18T10:15:13.670 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7789999999999999e-05 2024-09-18T10:15:13.670 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.670 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.671 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:13.671 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.148220+0000", 2024-09-18T10:15:13.671 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00060859599999999999 2024-09-18T10:15:13.671 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.671 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.671 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:13.671 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.152441+0000", 2024-09-18T10:15:13.671 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0042208250000000001 2024-09-18T10:15:13.671 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.671 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:13.671 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.671 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.671 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.671 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:448 3.4s0 3:2650be24:::smithi18925896-45:head [append 0~294912 in=294912b] snapc 13=[13,12,11,10,f,e,b] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-18T10:15:13.672 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:12.897981+0000", 2024-09-18T10:15:13.672 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.73919623499999998, 2024-09-18T10:15:13.672 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.74014470799999998, 2024-09-18T10:15:13.672 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:13.672 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:13.672 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:13.672 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:13.672 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:13.672 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 448 2024-09-18T10:15:13.672 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.672 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:13.672 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.672 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:13.673 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.897981+0000", 2024-09-18T10:15:13.673 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.673 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.673 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.673 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:13.673 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.897981+0000", 2024-09-18T10:15:13.673 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.673 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.673 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.673 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:13.673 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.897979+0000", 2024-09-18T10:15:13.673 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:13.674 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.674 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.674 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:13.674 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.898390+0000", 2024-09-18T10:15:13.674 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00041124999999999999 2024-09-18T10:15:13.674 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.674 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.674 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:13.674 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.898416+0000", 2024-09-18T10:15:13.674 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.525e-05 2024-09-18T10:15:13.674 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.675 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.675 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:13.675 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.898434+0000", 2024-09-18T10:15:13.675 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8615999999999999e-05 2024-09-18T10:15:13.675 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.675 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.675 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:13.675 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.148720+0000", 2024-09-18T10:15:13.675 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.25028612900000002 2024-09-18T10:15:13.675 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.675 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.675 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:13.675 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.148799+0000", 2024-09-18T10:15:13.675 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.8481999999999998e-05 2024-09-18T10:15:13.676 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.676 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.676 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:13.676 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.149383+0000", 2024-09-18T10:15:13.676 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000584255 2024-09-18T10:15:13.676 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.676 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.676 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:13.676 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.152442+0000", 2024-09-18T10:15:13.676 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0030591049999999999 2024-09-18T10:15:13.676 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.676 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:13.676 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.677 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.677 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.677 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:449 3.4s0 3:2650be24:::smithi18925896-45:head [setxattr _header (57) in=64b] snapc 13=[13,12,11,10,f,e,b] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-18T10:15:13.677 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:12.898446+0000", 2024-09-18T10:15:13.677 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.73873071700000004, 2024-09-18T10:15:13.677 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.73980457700000002, 2024-09-18T10:15:13.677 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:13.677 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:13.677 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:13.677 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:13.677 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:13.677 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 449 2024-09-18T10:15:13.677 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.678 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:13.678 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.678 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:13.678 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.898446+0000", 2024-09-18T10:15:13.678 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.678 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.678 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.678 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:13.678 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.898446+0000", 2024-09-18T10:15:13.678 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.678 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.678 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.678 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:13.678 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.898445+0000", 2024-09-18T10:15:13.679 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.999999 2024-09-18T10:15:13.679 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.679 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.679 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:13.679 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.898451+0000", 2024-09-18T10:15:13.679 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.054e-06 2024-09-18T10:15:13.679 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.679 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.679 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:13.679 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.898462+0000", 2024-09-18T10:15:13.679 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.081e-05 2024-09-18T10:15:13.679 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.679 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.679 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:13.680 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.898467+0000", 2024-09-18T10:15:13.680 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4220000000000002e-06 2024-09-18T10:15:13.680 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.680 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.680 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:13.680 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.149769+0000", 2024-09-18T10:15:13.680 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.25130268 2024-09-18T10:15:13.680 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.680 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.680 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:13.680 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.149964+0000", 2024-09-18T10:15:13.680 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00019506699999999999 2024-09-18T10:15:13.680 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.680 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.681 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:13.681 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.151037+0000", 2024-09-18T10:15:13.681 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0010723200000000001 2024-09-18T10:15:13.681 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.681 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.681 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:13.681 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.152623+0000", 2024-09-18T10:15:13.681 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0015865829999999999 2024-09-18T10:15:13.681 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.681 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:13.681 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.681 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.681 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.682 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWrite(3.19s5 57/40 ECSubWrite(tid=96, reqid=osd.7.0:96, at_version=57'54, trim_to=0'0, roll_forward_to=57'50))", 2024-09-18T10:15:13.682 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:13.464289+0000", 2024-09-18T10:15:13.682 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.17288801000000001, 2024-09-18T10:15:13.682 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.17408264800000001, 2024-09-18T10:15:13.682 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:13.682 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:13.682 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:13.682 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.682 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:13.682 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.464289+0000", 2024-09-18T10:15:13.682 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.682 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.682 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.682 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:13.683 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.464289+0000", 2024-09-18T10:15:13.683 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.683 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.683 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.683 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:13.683 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.464288+0000", 2024-09-18T10:15:13.683 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:13.683 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.683 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.683 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:13.683 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.464317+0000", 2024-09-18T10:15:13.683 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.8588000000000001e-05 2024-09-18T10:15:13.683 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.683 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.683 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:13.684 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.464327+0000", 2024-09-18T10:15:13.684 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.8330000000000006e-06 2024-09-18T10:15:13.684 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.684 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.684 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:13.684 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.464333+0000", 2024-09-18T10:15:13.684 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0769999999999996e-06 2024-09-18T10:15:13.684 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.684 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.684 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:13.684 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.564959+0000", 2024-09-18T10:15:13.684 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10062602900000001 2024-09-18T10:15:13.684 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.685 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.685 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:13.685 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.564974+0000", 2024-09-18T10:15:13.685 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4877e-05 2024-09-18T10:15:13.685 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.685 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:13.685 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.685 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.685 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.685 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:450 3.4s0 3:2650be24:::smithi18925896-45:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e57)", 2024-09-18T10:15:13.685 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:12.898472+0000", 2024-09-18T10:15:13.685 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.73870513199999999, 2024-09-18T10:15:13.685 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.74000580699999996, 2024-09-18T10:15:13.686 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:13.686 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "waiting for rw locks", 2024-09-18T10:15:13.686 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:13.686 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:13.686 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:13.686 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 450 2024-09-18T10:15:13.686 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.686 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:13.686 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.686 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:13.686 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.898472+0000", 2024-09-18T10:15:13.686 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.686 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.687 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.687 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:13.687 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.898472+0000", 2024-09-18T10:15:13.687 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.687 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.687 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.687 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:13.687 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.898471+0000", 2024-09-18T10:15:13.687 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:13.687 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.687 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.687 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:13.687 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.898475+0000", 2024-09-18T10:15:13.687 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1480000000000002e-06 2024-09-18T10:15:13.688 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.688 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.688 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:13.688 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.898485+0000", 2024-09-18T10:15:13.688 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0342e-05 2024-09-18T10:15:13.688 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.688 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.688 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:13.688 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.898490+0000", 2024-09-18T10:15:13.688 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7060000000000003e-06 2024-09-18T10:15:13.688 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.688 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.688 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:13.688 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.151519+0000", 2024-09-18T10:15:13.689 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.25302985300000003 2024-09-18T10:15:13.689 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.689 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.689 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for rw locks", 2024-09-18T10:15:13.689 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.151633+0000", 2024-09-18T10:15:13.689 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000113121 2024-09-18T10:15:13.689 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.689 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:13.689 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.689 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.689 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-18T10:15:13.689 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 13 2024-09-18T10:15:13.689 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-18T10:15:13.690 INFO:tasks.rados.rados.0.smithi189.stdout:176: finishing write tid 1 to smithi18925896-25 2024-09-18T10:15:13.691 INFO:tasks.rados.rados.0.smithi189.stdout:177: finishing write tid 1 to smithi18925896-45 2024-09-18T10:15:13.691 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-09-18T10:15:13.741 INFO:tasks.ceph.osd.4.smithi029.stderr:2024-09-18T10:15:13.738+0000 7f550cf6e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22189) UID: 0 2024-09-18T10:15:13.759 INFO:tasks.rados.rados.0.smithi189.stdout:176: finishing write tid 2 to smithi18925896-25 2024-09-18T10:15:13.768 INFO:tasks.rados.rados.0.smithi189.stdout:177: finishing write tid 2 to smithi18925896-45 2024-09-18T10:15:13.769 INFO:tasks.rados.rados.0.smithi189.stdout:177: finishing write tid 3 to smithi18925896-45 2024-09-18T10:15:13.769 INFO:tasks.rados.rados.0.smithi189.stdout:177: finishing write tid 4 to smithi18925896-45 2024-09-18T10:15:13.789 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-18T10:15:13.789 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-18T10:15:13.790 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-18T10:15:13.790 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-18T10:15:13.790 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-18T10:15:13.799 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-09-18T10:15:13.831 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-18T10:15:13.829+0000 7f14c6a1a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 22138) UID: 0 2024-09-18T10:15:13.863 INFO:tasks.rados.rados.0.smithi189.stdout:179: done (2 left) 2024-09-18T10:15:13.863 INFO:tasks.rados.rados.0.smithi189.stdout:180: snap_create 2024-09-18T10:15:13.901 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-18T10:15:13.901 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-18T10:15:13.901 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-18T10:15:13.901 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-18T10:15:13.901 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.902 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:134 3.4s0 3:2650be24:::smithi18925896-45:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:13.902 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:50.915670+0000", 2024-09-18T10:15:13.902 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 22.981729221999998, 2024-09-18T10:15:13.902 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.40775830200000002, 2024-09-18T10:15:13.902 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:13.902 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:13.902 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:13.902 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:13.902 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:13.902 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 134 2024-09-18T10:15:13.902 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.902 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:13.902 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.902 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:13.903 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.915670+0000", 2024-09-18T10:15:13.903 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.903 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.903 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.903 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:13.903 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.915670+0000", 2024-09-18T10:15:13.903 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.903 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.903 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.903 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:13.903 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.915668+0000", 2024-09-18T10:15:13.903 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:13.903 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.903 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.904 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:13.904 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.915676+0000", 2024-09-18T10:15:13.904 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1899999999999995e-06 2024-09-18T10:15:13.904 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.904 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.904 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:13.904 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.915690+0000", 2024-09-18T10:15:13.904 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3827e-05 2024-09-18T10:15:13.904 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.904 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.904 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:13.904 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.915702+0000", 2024-09-18T10:15:13.904 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1477999999999999e-05 2024-09-18T10:15:13.904 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.905 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.905 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:13.905 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.918373+0000", 2024-09-18T10:15:13.905 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0026713000000000002 2024-09-18T10:15:13.905 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.905 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.905 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:13.905 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.918547+0000", 2024-09-18T10:15:13.905 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00017375999999999999 2024-09-18T10:15:13.905 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.905 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.905 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:13.905 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.919485+0000", 2024-09-18T10:15:13.905 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00093826999999999995 2024-09-18T10:15:13.906 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.906 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.906 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:13.906 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.120632+0000", 2024-09-18T10:15:13.906 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.20114729200000001 2024-09-18T10:15:13.906 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.906 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.906 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-18T10:15:13.906 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.323281+0000", 2024-09-18T10:15:13.906 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.20264834500000001 2024-09-18T10:15:13.906 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.906 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.906 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:13.906 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.323428+0000", 2024-09-18T10:15:13.907 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000147334 2024-09-18T10:15:13.907 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.907 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:13.907 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.907 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.907 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.907 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:135 3.4s0 3:2650be24:::smithi18925896-45:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:13.907 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:50.915712+0000", 2024-09-18T10:15:13.907 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 22.981687362999999, 2024-09-18T10:15:13.907 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.81161388099999998, 2024-09-18T10:15:13.907 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:13.907 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:13.907 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:13.908 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:13.908 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:13.908 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 135 2024-09-18T10:15:13.908 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.908 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:13.908 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.908 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:13.908 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.915712+0000", 2024-09-18T10:15:13.908 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.908 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.908 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.908 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:13.908 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.915712+0000", 2024-09-18T10:15:13.908 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.909 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.909 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.909 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:13.909 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.915711+0000", 2024-09-18T10:15:13.909 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.999999 2024-09-18T10:15:13.909 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.909 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.909 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:13.909 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.915715+0000", 2024-09-18T10:15:13.909 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.3159999999999998e-06 2024-09-18T10:15:13.909 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.909 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.909 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:13.909 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.915726+0000", 2024-09-18T10:15:13.910 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1375e-05 2024-09-18T10:15:13.910 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.910 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.910 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:13.910 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.915736+0000", 2024-09-18T10:15:13.910 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.4860000000000003e-06 2024-09-18T10:15:13.910 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.910 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.910 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:13.910 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.919925+0000", 2024-09-18T10:15:13.910 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0041886689999999999 2024-09-18T10:15:13.910 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.910 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.910 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for rw locks", 2024-09-18T10:15:13.911 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.920019+0000", 2024-09-18T10:15:13.911 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.4196000000000004e-05 2024-09-18T10:15:13.911 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.911 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.911 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:13.911 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.424171+0000", 2024-09-18T10:15:13.911 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.50415232399999999 2024-09-18T10:15:13.911 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.911 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.911 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:13.911 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.424320+0000", 2024-09-18T10:15:13.911 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00014877299999999999 2024-09-18T10:15:13.911 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.911 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.911 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:13.912 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.727325+0000", 2024-09-18T10:15:13.912 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.30300555400000001 2024-09-18T10:15:13.912 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.912 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:13.912 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.912 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.912 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.912 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWrite(3.cs1 36/32 ECSubWrite(tid=20, reqid=client.4632.0:150, at_version=36'9, trim_to=0'0, roll_forward_to=36'8))", 2024-09-18T10:15:13.912 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:51.127568+0000", 2024-09-18T10:15:13.912 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 22.769831205999999, 2024-09-18T10:15:13.912 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.29747172100000002, 2024-09-18T10:15:13.912 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:13.912 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:13.912 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:13.913 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.913 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:13.913 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.127568+0000", 2024-09-18T10:15:13.913 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.913 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.913 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.913 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:13.913 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.127568+0000", 2024-09-18T10:15:13.913 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.913 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.913 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.913 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:13.913 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.127566+0000", 2024-09-18T10:15:13.913 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:13.913 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.914 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.914 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:13.914 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.127654+0000", 2024-09-18T10:15:13.914 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8181000000000002e-05 2024-09-18T10:15:13.914 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.914 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.914 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:13.914 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.127681+0000", 2024-09-18T10:15:13.914 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6282e-05 2024-09-18T10:15:13.914 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.914 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.914 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:13.914 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.127694+0000", 2024-09-18T10:15:13.914 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3159999999999999e-05 2024-09-18T10:15:13.914 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.915 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.915 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:13.915 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.221115+0000", 2024-09-18T10:15:13.915 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.093420807999999994 2024-09-18T10:15:13.915 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.915 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.915 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:13.915 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.221132+0000", 2024-09-18T10:15:13.915 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7725999999999999e-05 2024-09-18T10:15:13.915 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.915 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.915 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:13.915 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.424921+0000", 2024-09-18T10:15:13.916 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.20378872000000001 2024-09-18T10:15:13.916 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.916 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.916 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:13.916 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.425039+0000", 2024-09-18T10:15:13.916 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000118267 2024-09-18T10:15:13.916 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.916 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:13.916 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.916 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.916 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.916 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWrite(3.cs1 36/32 ECSubWrite(tid=21, reqid=client.4632.0:151, at_version=36'10, trim_to=0'0, roll_forward_to=36'8))", 2024-09-18T10:15:13.916 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:51.130232+0000", 2024-09-18T10:15:13.916 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 22.767166934999999, 2024-09-18T10:15:13.917 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.294885956, 2024-09-18T10:15:13.917 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:13.917 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:13.917 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:13.917 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.917 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:13.917 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.130232+0000", 2024-09-18T10:15:13.917 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.917 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.917 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.917 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:13.917 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.130232+0000", 2024-09-18T10:15:13.917 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.918 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.918 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.918 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:13.919 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.130226+0000", 2024-09-18T10:15:13.919 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999943 2024-09-18T10:15:13.919 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.919 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.919 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:13.919 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.130394+0000", 2024-09-18T10:15:13.919 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000167205 2024-09-18T10:15:13.919 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.919 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.919 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:13.919 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.130426+0000", 2024-09-18T10:15:13.919 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2625e-05 2024-09-18T10:15:13.919 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.919 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.920 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:13.920 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.130443+0000", 2024-09-18T10:15:13.920 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.6558999999999998e-05 2024-09-18T10:15:13.920 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.920 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.920 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:13.920 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.330703+0000", 2024-09-18T10:15:13.920 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.200260614 2024-09-18T10:15:13.920 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.920 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.920 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:13.920 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.330718+0000", 2024-09-18T10:15:13.920 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4647999999999999e-05 2024-09-18T10:15:13.921 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.921 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.921 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:13.921 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.425072+0000", 2024-09-18T10:15:13.921 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.094353426000000004 2024-09-18T10:15:13.921 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.921 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.921 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:13.921 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.425118+0000", 2024-09-18T10:15:13.921 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6378999999999998e-05 2024-09-18T10:15:13.921 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.921 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:13.921 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.921 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.922 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.922 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWrite(3.cs1 36/32 ECSubWrite(tid=22, reqid=client.4632.0:151, at_version=0'0, trim_to=0'0, roll_forward_to=36'10))", 2024-09-18T10:15:13.922 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:51.533998+0000", 2024-09-18T10:15:13.922 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 22.363400882000001, 2024-09-18T10:15:13.922 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.29424693400000002, 2024-09-18T10:15:13.922 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:13.922 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:13.922 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:13.922 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.922 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:13.922 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.533998+0000", 2024-09-18T10:15:13.922 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.922 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.922 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.923 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:13.923 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.533998+0000", 2024-09-18T10:15:13.923 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.923 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.923 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.923 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:13.923 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.533996+0000", 2024-09-18T10:15:13.923 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:13.923 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.923 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.923 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:13.923 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.534023+0000", 2024-09-18T10:15:13.923 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.675e-05 2024-09-18T10:15:13.924 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.924 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.924 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:13.924 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.534056+0000", 2024-09-18T10:15:13.924 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.3288999999999999e-05 2024-09-18T10:15:13.924 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.924 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.924 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:13.924 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.634158+0000", 2024-09-18T10:15:13.924 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100101632 2024-09-18T10:15:13.924 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.924 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.924 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:13.925 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.827026+0000", 2024-09-18T10:15:13.925 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.19286773800000001 2024-09-18T10:15:13.925 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.925 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.925 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:13.925 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.827036+0000", 2024-09-18T10:15:13.925 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0261999999999999e-05 2024-09-18T10:15:13.925 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.925 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.925 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:13.925 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.828227+0000", 2024-09-18T10:15:13.925 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001191398 2024-09-18T10:15:13.925 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.925 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.926 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:13.926 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.828245+0000", 2024-09-18T10:15:13.926 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7604999999999998e-05 2024-09-18T10:15:13.926 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.926 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:13.926 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.926 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.926 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.926 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:188 3.4s0 3:213f864e:::smithi18925896-32:head [create,setxattr XNeZS[lOPtVca\\Pa[\\rdfbinb[bcjhtYSpmcafZm (1928) in=40b,setxattr TfflYYqPnps]lo^qSlp___TitNdd`]kaPTVOTcsh (1648) in=40b,setxattr TNb_onpY[j_dkhfmnWtm]Yktg]]RNtbQe[OUTNlX (770) in=40b,setxattr ZcgnkhY\\orTYaZh]PPldj`SNq`p\\cX`k]h\\bZY_Y (700) in=40b,setxattr gPQVnqhpsVhW_kq`QnafXrlRZtZ\\YnRftgQTishR (595) in=40b,setxattr U`XqYi[WhoqnpSPRc\\RhnppZp^qY\\qUfjpqs[r^a (1892) in=40b,setxattr cTmhd`pWgrRf_VfVgOtgeOP^ShP]lsOtpoTj[\\gl (418) in=40b,setxattr aa_taYTfslg`lRRrOkkraN\\XiW^V\\ljl]NqXin`a (314) in=40b,setxattr ibl]`igtgVThY^Qse\\okYYUtTmolX\\W]p]Q[P\\il (1403) in=40b,setxattr \\ZabSO[UUqeZa`hddNh[ZjaRRgUZoQtkn\\QdsX]W (1541) in=40b,setxattr jUkQUPTNbo`rZNmYlUmhb]ojc]aeheoVRfp]bPWf (5052) in=40b,setxattr Ofj`oQ^cR]da^SPsTaXmmPse\\OZ`VSqQ]U`efsZl (419) in=40b,setxattr a[cdkWos\\OrUVPciUaa^_^`lWVTScffiWkd\\[kgf (1261) in=40b,setxattr jXb]\\WhbatdZ[qTTiZO]kPd^`l[kajOYiQ[iqSYc (244) in=40b,setxattr \\im]jQhbanUn\\_^h`jaU]W`ReaZN_je[]kgg (1578) in=36b,setxattr ZbosoRORS^_de]rN^_daOiha\\Q\\eRae^[sQUjXsV (9120) in=40b,setxattr gcn]XpZUkOZr\\pkrVVZpYiR\\alh_PaZa_S[PYcn] (384) in=40b,setxattr \\^qUOaNXSTjtmafhfbo\\gseTTkV`mgnacYq^bkcc (34) in=40b,setxattr iUa^mOQ]TqjVSUU_nXfY]f`NjljiiaYbrm[o]aes (1611) in=40b,setxattr tXaQPQbc^aiV[^reVOPpib^caoWf^XgqilO^_g[k (1016) in=40b] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:13.926 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:52.693780+0000", 2024-09-18T10:15:13.926 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 21.203618786, 2024-09-18T10:15:13.926 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.40440725999999999, 2024-09-18T10:15:13.926 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:13.927 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:13.927 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:13.927 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:13.927 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:13.927 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 188 2024-09-18T10:15:13.927 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.927 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:13.927 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.927 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:13.927 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.693780+0000", 2024-09-18T10:15:13.927 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.927 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.927 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.927 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:13.928 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.693780+0000", 2024-09-18T10:15:13.928 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.928 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.928 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.928 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:13.928 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.693777+0000", 2024-09-18T10:15:13.928 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999967 2024-09-18T10:15:13.928 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.928 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.928 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:13.928 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.693873+0000", 2024-09-18T10:15:13.928 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.6496000000000006e-05 2024-09-18T10:15:13.928 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.929 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.929 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:13.929 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.693926+0000", 2024-09-18T10:15:13.929 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2148999999999998e-05 2024-09-18T10:15:13.929 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.929 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.929 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:13.929 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.693945+0000", 2024-09-18T10:15:13.929 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9296999999999998e-05 2024-09-18T10:15:13.929 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.929 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.929 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:13.929 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.694035+0000", 2024-09-18T10:15:13.930 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.0439999999999995e-05 2024-09-18T10:15:13.930 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.930 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.930 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op must wait for map", 2024-09-18T10:15:13.930 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.694044+0000", 2024-09-18T10:15:13.930 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.1479999999999995e-06 2024-09-18T10:15:13.930 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.930 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.930 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:13.930 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.797388+0000", 2024-09-18T10:15:13.930 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10334315199999999 2024-09-18T10:15:13.930 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.930 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.930 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:13.931 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.797847+0000", 2024-09-18T10:15:13.931 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00045971299999999999 2024-09-18T10:15:13.931 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.931 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.931 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:13.931 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.800981+0000", 2024-09-18T10:15:13.931 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0031333189999999999 2024-09-18T10:15:13.931 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.931 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.931 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:13.931 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.803005+0000", 2024-09-18T10:15:13.931 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0020248359999999999 2024-09-18T10:15:13.931 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.931 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.932 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-18T10:15:13.932 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:53.097948+0000", 2024-09-18T10:15:13.932 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.29494210599999998 2024-09-18T10:15:13.932 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.932 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.932 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:13.932 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:53.098187+0000", 2024-09-18T10:15:13.932 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000239769 2024-09-18T10:15:13.932 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.932 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:13.932 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.932 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.932 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.933 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWrite(3.cs1 41/32 ECSubWrite(tid=47, reqid=osd.5.0:46, at_version=0'0, trim_to=0'0, roll_forward_to=41'27))", 2024-09-18T10:15:13.933 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:56.935584+0000", 2024-09-18T10:15:13.933 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 16.961815090000002, 2024-09-18T10:15:13.933 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.60626800999999997, 2024-09-18T10:15:13.933 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:13.933 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:13.933 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:13.933 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.933 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:13.933 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.935584+0000", 2024-09-18T10:15:13.933 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.933 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.933 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.933 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:13.934 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.935584+0000", 2024-09-18T10:15:13.934 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.934 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.934 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.934 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:13.934 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.935582+0000", 2024-09-18T10:15:13.934 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:13.934 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.934 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.934 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:13.934 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.935616+0000", 2024-09-18T10:15:13.934 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.4588999999999997e-05 2024-09-18T10:15:13.934 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.934 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.935 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:13.935 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.935661+0000", 2024-09-18T10:15:13.935 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4705e-05 2024-09-18T10:15:13.935 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.935 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.935 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:13.935 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.935681+0000", 2024-09-18T10:15:13.935 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0290000000000001e-05 2024-09-18T10:15:13.935 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.935 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.935 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:13.935 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.339911+0000", 2024-09-18T10:15:13.935 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.40422952000000001 2024-09-18T10:15:13.935 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.936 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.936 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:13.936 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.339925+0000", 2024-09-18T10:15:13.936 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4523e-05 2024-09-18T10:15:13.936 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.936 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.936 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:13.936 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.541788+0000", 2024-09-18T10:15:13.936 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.20186222000000001 2024-09-18T10:15:13.936 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.936 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.936 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:13.936 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.541852+0000", 2024-09-18T10:15:13.936 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.4171999999999995e-05 2024-09-18T10:15:13.937 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.937 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:13.937 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.937 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.937 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.937 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWrite(3.cs1 41/32 ECSubWrite(tid=48, reqid=osd.5.0:48, at_version=41'29, trim_to=0'0, roll_forward_to=41'27))", 2024-09-18T10:15:13.937 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:56.937458+0000", 2024-09-18T10:15:13.937 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 16.959941301000001, 2024-09-18T10:15:13.937 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.60464621900000004, 2024-09-18T10:15:13.937 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:13.937 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:13.937 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:13.937 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.937 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:13.938 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.937458+0000", 2024-09-18T10:15:13.938 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.938 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.938 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.938 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:13.938 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.937458+0000", 2024-09-18T10:15:13.938 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.938 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.938 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.938 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:13.938 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.937456+0000", 2024-09-18T10:15:13.938 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.999999 2024-09-18T10:15:13.938 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.938 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.939 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:13.939 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.937505+0000", 2024-09-18T10:15:13.939 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8809e-05 2024-09-18T10:15:13.939 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.939 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.939 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:13.939 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.937540+0000", 2024-09-18T10:15:13.939 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.4662000000000001e-05 2024-09-18T10:15:13.939 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.939 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.939 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:13.939 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.937554+0000", 2024-09-18T10:15:13.939 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4137e-05 2024-09-18T10:15:13.940 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.940 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.940 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:13.940 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.340361+0000", 2024-09-18T10:15:13.940 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.40280652500000003 2024-09-18T10:15:13.940 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.940 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.940 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:13.940 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.340382+0000", 2024-09-18T10:15:13.940 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.16e-05 2024-09-18T10:15:13.940 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.940 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.940 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:13.940 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.542055+0000", 2024-09-18T10:15:13.941 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.20167278999999999 2024-09-18T10:15:13.941 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.941 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.941 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:13.941 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.542104+0000", 2024-09-18T10:15:13.941 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8866000000000003e-05 2024-09-18T10:15:13.941 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.941 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:13.941 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.941 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.941 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.941 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:413 3.4s0 3:2650be24:::smithi18925896-45:head [read 0~753664,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-18T10:15:13.941 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:07.553960+0000", 2024-09-18T10:15:13.941 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 6.3434391190000001, 2024-09-18T10:15:13.941 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.30658152999999999, 2024-09-18T10:15:13.942 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:13.942 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:13.942 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:13.942 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:13.942 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:13.942 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 413 2024-09-18T10:15:13.942 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.942 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:13.942 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.942 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:13.942 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.553960+0000", 2024-09-18T10:15:13.942 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.942 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.942 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.943 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:13.943 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.553960+0000", 2024-09-18T10:15:13.943 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.943 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.943 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.943 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:13.943 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.553956+0000", 2024-09-18T10:15:13.943 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999967 2024-09-18T10:15:13.943 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.943 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.943 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:13.943 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.553973+0000", 2024-09-18T10:15:13.943 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.6602000000000001e-05 2024-09-18T10:15:13.943 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.944 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.944 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:13.944 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.554010+0000", 2024-09-18T10:15:13.944 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7129999999999999e-05 2024-09-18T10:15:13.944 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.944 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.944 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:13.944 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.554032+0000", 2024-09-18T10:15:13.944 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2316000000000001e-05 2024-09-18T10:15:13.944 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.944 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.944 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:13.944 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.554187+0000", 2024-09-18T10:15:13.945 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00015407499999999999 2024-09-18T10:15:13.945 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.945 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.945 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op must wait for map", 2024-09-18T10:15:13.945 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.554232+0000", 2024-09-18T10:15:13.945 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.5173999999999998e-05 2024-09-18T10:15:13.945 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.945 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.945 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:13.945 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.655961+0000", 2024-09-18T10:15:13.945 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10172934 2024-09-18T10:15:13.945 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.945 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.946 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:13.946 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.656152+0000", 2024-09-18T10:15:13.946 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00019075599999999999 2024-09-18T10:15:13.946 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.946 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.946 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:13.946 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.860541+0000", 2024-09-18T10:15:13.946 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.20438944000000001 2024-09-18T10:15:13.946 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.946 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:13.946 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.946 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.946 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.946 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:414 3.4s0 3:2650be24:::smithi18925896-45:head [read 0~753664,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-18T10:15:13.947 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:07.554078+0000", 2024-09-18T10:15:13.947 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 6.3433208150000002, 2024-09-18T10:15:13.947 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.30769232899999999, 2024-09-18T10:15:13.947 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:13.947 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:13.947 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:13.947 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:13.947 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:13.947 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 414 2024-09-18T10:15:13.947 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.947 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:13.947 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.947 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:13.947 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.554078+0000", 2024-09-18T10:15:13.948 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.948 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.948 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.948 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:13.948 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.554078+0000", 2024-09-18T10:15:13.948 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.948 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.948 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.948 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:13.948 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.554077+0000", 2024-09-18T10:15:13.948 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:13.948 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.948 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.949 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:13.949 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.554086+0000", 2024-09-18T10:15:13.949 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.6569999999999992e-06 2024-09-18T10:15:13.949 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.949 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.949 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:13.949 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.554106+0000", 2024-09-18T10:15:13.949 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9969000000000001e-05 2024-09-18T10:15:13.949 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.949 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.949 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:13.949 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.554118+0000", 2024-09-18T10:15:13.949 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.217e-05 2024-09-18T10:15:13.949 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.950 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.950 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:13.950 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.554389+0000", 2024-09-18T10:15:13.950 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00027051000000000001 2024-09-18T10:15:13.950 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.950 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.950 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:13.950 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.554407+0000", 2024-09-18T10:15:13.950 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7677e-05 2024-09-18T10:15:13.950 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.950 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.950 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:13.950 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.656732+0000", 2024-09-18T10:15:13.950 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.102325597 2024-09-18T10:15:13.951 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.951 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.951 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:13.951 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.656894+0000", 2024-09-18T10:15:13.951 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000161383 2024-09-18T10:15:13.951 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.951 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.951 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:13.951 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.861770+0000", 2024-09-18T10:15:13.951 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.20487676599999999 2024-09-18T10:15:13.951 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.951 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:13.951 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.951 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.951 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.952 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:415 3.4s0 3:2650be24:::smithi18925896-45:head [read 0~753664,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-18T10:15:13.952 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:07.554169+0000", 2024-09-18T10:15:13.952 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 6.3432303540000001, 2024-09-18T10:15:13.952 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.308598176, 2024-09-18T10:15:13.952 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:13.952 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:13.952 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:13.952 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:13.952 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:13.952 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 415 2024-09-18T10:15:13.952 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.952 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:13.952 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.952 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:13.953 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.554169+0000", 2024-09-18T10:15:13.953 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.953 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.953 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.953 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:13.953 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.554169+0000", 2024-09-18T10:15:13.953 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.953 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.953 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.953 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:13.953 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.554167+0000", 2024-09-18T10:15:13.953 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:13.953 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.954 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.954 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:13.954 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.554176+0000", 2024-09-18T10:15:13.954 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6430000000000001e-06 2024-09-18T10:15:13.954 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.954 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.954 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:13.954 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.554197+0000", 2024-09-18T10:15:13.954 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1209e-05 2024-09-18T10:15:13.954 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.954 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.954 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:13.954 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.554206+0000", 2024-09-18T10:15:13.954 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.3570000000000004e-06 2024-09-18T10:15:13.955 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.955 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.955 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:13.955 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.554515+0000", 2024-09-18T10:15:13.955 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00030821000000000001 2024-09-18T10:15:13.955 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.955 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.955 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:13.955 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.554535+0000", 2024-09-18T10:15:13.955 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0092999999999999e-05 2024-09-18T10:15:13.955 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.955 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.955 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:13.955 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.756075+0000", 2024-09-18T10:15:13.956 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.201540831 2024-09-18T10:15:13.956 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.956 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.956 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:13.956 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.756260+0000", 2024-09-18T10:15:13.956 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00018466000000000001 2024-09-18T10:15:13.956 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.956 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.956 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:13.956 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.862767+0000", 2024-09-18T10:15:13.956 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.106506579 2024-09-18T10:15:13.956 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.956 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:13.957 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.957 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.957 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.957 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(osd.4.21:3 3.4s0 3:2619bf65:::smithi18925896-16:head [assert-version v2,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-18T10:15:13.957 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:11.936400+0000", 2024-09-18T10:15:13.957 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 1.960999199, 2024-09-18T10:15:13.957 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.312109203, 2024-09-18T10:15:13.957 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:13.957 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:13.957 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:13.957 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.957 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:13.957 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:11.936400+0000", 2024-09-18T10:15:13.957 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.958 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.958 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.958 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:13.958 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:11.936400+0000", 2024-09-18T10:15:13.958 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.958 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.958 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.958 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:13.958 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:11.936396+0000", 2024-09-18T10:15:13.958 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999962 2024-09-18T10:15:13.958 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.958 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.958 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:13.958 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:11.936434+0000", 2024-09-18T10:15:13.959 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7580000000000003e-05 2024-09-18T10:15:13.959 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.959 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.959 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:13.959 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:11.936473+0000", 2024-09-18T10:15:13.959 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9876000000000002e-05 2024-09-18T10:15:13.959 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.959 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.959 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:13.959 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:11.936499+0000", 2024-09-18T10:15:13.959 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6012000000000001e-05 2024-09-18T10:15:13.959 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.959 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.959 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:13.960 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.039131+0000", 2024-09-18T10:15:13.960 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10263111799999999 2024-09-18T10:15:13.960 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.960 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.960 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:13.960 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.039402+0000", 2024-09-18T10:15:13.960 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00027099399999999999 2024-09-18T10:15:13.960 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.960 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.960 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:13.960 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.248509+0000", 2024-09-18T10:15:13.960 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.20910732700000001 2024-09-18T10:15:13.960 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.961 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:13.961 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.961 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.961 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.961 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWrite(3.cs1 57/32 ECSubWrite(tid=71, reqid=osd.5.0:71, at_version=57'48, trim_to=0'0, roll_forward_to=50'46))", 2024-09-18T10:15:13.961 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:12.854680+0000", 2024-09-18T10:15:13.961 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 1.0427185699999999, 2024-09-18T10:15:13.961 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.29685314200000001, 2024-09-18T10:15:13.961 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:13.961 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:13.962 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:13.962 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.962 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:13.962 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.854680+0000", 2024-09-18T10:15:13.962 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.962 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.962 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.962 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:13.962 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.854680+0000", 2024-09-18T10:15:13.962 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.962 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.962 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.962 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:13.962 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.854680+0000", 2024-09-18T10:15:13.963 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:13.963 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.963 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.963 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:13.963 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.854707+0000", 2024-09-18T10:15:13.963 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.7025999999999998e-05 2024-09-18T10:15:13.963 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.963 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.963 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:13.963 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.854715+0000", 2024-09-18T10:15:13.963 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5569999999999996e-06 2024-09-18T10:15:13.963 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.963 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.963 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:13.964 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.854724+0000", 2024-09-18T10:15:13.964 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.9809999999999992e-06 2024-09-18T10:15:13.964 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.964 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.964 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:13.964 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.047729+0000", 2024-09-18T10:15:13.964 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.193004817 2024-09-18T10:15:13.964 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.964 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.964 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:13.964 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.047756+0000", 2024-09-18T10:15:13.964 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6991e-05 2024-09-18T10:15:13.964 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.965 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.965 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:13.965 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.151492+0000", 2024-09-18T10:15:13.965 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.103735538 2024-09-18T10:15:13.965 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.965 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.965 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:13.965 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.151533+0000", 2024-09-18T10:15:13.965 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1699000000000002e-05 2024-09-18T10:15:13.965 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.965 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:13.965 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.965 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.965 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.966 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWrite(3.cs1 57/32 ECSubWrite(tid=72, reqid=osd.5.0:72, at_version=57'50, trim_to=0'0, roll_forward_to=50'46))", 2024-09-18T10:15:13.966 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:12.855564+0000", 2024-09-18T10:15:13.966 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 1.0418348200000001, 2024-09-18T10:15:13.966 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.296299965, 2024-09-18T10:15:13.966 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:13.966 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:13.966 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:13.966 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.966 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:13.966 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.855564+0000", 2024-09-18T10:15:13.966 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.966 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.966 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.966 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:13.967 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.855564+0000", 2024-09-18T10:15:13.967 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.967 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.967 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.967 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:13.967 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.855563+0000", 2024-09-18T10:15:13.967 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:13.967 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.967 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.967 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:13.967 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.855583+0000", 2024-09-18T10:15:13.967 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9187000000000001e-05 2024-09-18T10:15:13.967 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.967 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.968 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:13.968 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.855599+0000", 2024-09-18T10:15:13.968 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.6331999999999999e-05 2024-09-18T10:15:13.968 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.968 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.968 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:13.968 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.855608+0000", 2024-09-18T10:15:13.968 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.7239999999999998e-06 2024-09-18T10:15:13.968 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.968 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.968 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:13.968 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.048592+0000", 2024-09-18T10:15:13.968 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.192984664 2024-09-18T10:15:13.968 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.969 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.969 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:13.969 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.048615+0000", 2024-09-18T10:15:13.969 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2628000000000001e-05 2024-09-18T10:15:13.969 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.969 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.969 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:13.969 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.151787+0000", 2024-09-18T10:15:13.969 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.103171704 2024-09-18T10:15:13.969 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.969 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.969 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:13.969 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.151864+0000", 2024-09-18T10:15:13.969 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7302e-05 2024-09-18T10:15:13.970 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.970 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:13.970 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.970 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.970 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.970 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:446 3.4s0 3:2650be24:::smithi18925896-45:head [append 0~622592 in=622592b] snapc 13=[13,12,11,10,f,e,b] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-18T10:15:13.970 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:12.892106+0000", 2024-09-18T10:15:13.970 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 1.0052925699999999, 2024-09-18T10:15:13.970 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.77541641100000003, 2024-09-18T10:15:13.970 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:13.970 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:13.970 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:13.970 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:13.971 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:13.971 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 446 2024-09-18T10:15:13.971 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.971 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:13.972 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.972 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:13.972 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.892106+0000", 2024-09-18T10:15:13.972 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.972 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.972 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.972 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:13.972 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.892106+0000", 2024-09-18T10:15:13.972 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.972 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.972 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.972 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:13.972 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.892103+0000", 2024-09-18T10:15:13.973 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999971 2024-09-18T10:15:13.973 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.973 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.973 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:13.973 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.893050+0000", 2024-09-18T10:15:13.973 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00094607000000000003 2024-09-18T10:15:13.973 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.974 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.974 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:13.974 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.893084+0000", 2024-09-18T10:15:13.974 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.4759999999999999e-05 2024-09-18T10:15:13.974 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.974 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.974 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:13.974 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.893098+0000", 2024-09-18T10:15:13.974 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3859000000000001e-05 2024-09-18T10:15:13.974 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.975 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.975 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:13.975 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.145987+0000", 2024-09-18T10:15:13.975 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.252889115 2024-09-18T10:15:13.975 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.975 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.975 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:13.975 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.146112+0000", 2024-09-18T10:15:13.975 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00012469299999999999 2024-09-18T10:15:13.975 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.975 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.975 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:13.975 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.146923+0000", 2024-09-18T10:15:13.976 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00081091500000000005 2024-09-18T10:15:13.976 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.976 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.976 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:13.976 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.152437+0000", 2024-09-18T10:15:13.976 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0055141169999999998 2024-09-18T10:15:13.976 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.976 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.976 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-18T10:15:13.976 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.667347+0000", 2024-09-18T10:15:13.976 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.51491041199999998 2024-09-18T10:15:13.976 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.976 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.976 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:13.977 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.667523+0000", 2024-09-18T10:15:13.977 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00017533199999999999 2024-09-18T10:15:13.977 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.977 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:13.977 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.977 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.977 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.977 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:447 3.4s0 3:2650be24:::smithi18925896-45:head [append 0~475136 in=475136b] snapc 13=[13,12,11,10,f,e,b] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-18T10:15:13.977 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:12.896134+0000", 2024-09-18T10:15:13.977 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 1.0012652070000001, 2024-09-18T10:15:13.977 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.87223328099999997, 2024-09-18T10:15:13.977 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:13.977 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:13.978 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:13.978 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:13.978 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:13.978 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 447 2024-09-18T10:15:13.978 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.978 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:13.978 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.978 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:13.978 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.896134+0000", 2024-09-18T10:15:13.978 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.978 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.978 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.978 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:13.978 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.896134+0000", 2024-09-18T10:15:13.979 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.979 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.979 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.979 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:13.979 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.896131+0000", 2024-09-18T10:15:13.979 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999976 2024-09-18T10:15:13.979 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.979 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.979 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:13.979 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.896661+0000", 2024-09-18T10:15:13.979 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00052970299999999997 2024-09-18T10:15:13.979 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.979 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.979 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:13.980 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.896693+0000", 2024-09-18T10:15:13.980 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1903000000000003e-05 2024-09-18T10:15:13.980 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.980 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.980 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:13.980 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.896705+0000", 2024-09-18T10:15:13.980 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2288e-05 2024-09-18T10:15:13.980 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.980 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.980 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:13.980 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.147534+0000", 2024-09-18T10:15:13.980 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.25082866300000001 2024-09-18T10:15:13.980 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.980 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.981 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:13.981 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.147612+0000", 2024-09-18T10:15:13.981 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7789999999999999e-05 2024-09-18T10:15:13.981 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.981 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.981 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:13.981 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.148220+0000", 2024-09-18T10:15:13.981 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00060859599999999999 2024-09-18T10:15:13.981 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.981 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.981 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:13.981 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.152441+0000", 2024-09-18T10:15:13.981 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0042208250000000001 2024-09-18T10:15:13.981 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.982 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.982 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-18T10:15:13.982 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.768118+0000", 2024-09-18T10:15:13.982 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.615676471 2024-09-18T10:15:13.982 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.982 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.982 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:13.982 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.768367+0000", 2024-09-18T10:15:13.982 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00024930800000000001 2024-09-18T10:15:13.982 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.982 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:13.983 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.983 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.983 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.983 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:448 3.4s0 3:2650be24:::smithi18925896-45:head [append 0~294912 in=294912b] snapc 13=[13,12,11,10,f,e,b] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-18T10:15:13.983 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:12.897981+0000", 2024-09-18T10:15:13.983 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.99941802899999999, 2024-09-18T10:15:13.983 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.87089765500000005, 2024-09-18T10:15:13.983 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:13.983 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:13.983 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:13.983 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:13.983 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:13.983 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 448 2024-09-18T10:15:13.984 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.984 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:13.984 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.984 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:13.984 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.897981+0000", 2024-09-18T10:15:13.984 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.984 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.984 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.984 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:13.984 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.897981+0000", 2024-09-18T10:15:13.984 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.984 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.984 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.984 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:13.985 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.897979+0000", 2024-09-18T10:15:13.985 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:13.985 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.985 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.985 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:13.985 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.898390+0000", 2024-09-18T10:15:13.985 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00041124999999999999 2024-09-18T10:15:13.985 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.985 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.985 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:13.985 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.898416+0000", 2024-09-18T10:15:13.985 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.525e-05 2024-09-18T10:15:13.985 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.985 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.985 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:13.986 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.898434+0000", 2024-09-18T10:15:13.986 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8615999999999999e-05 2024-09-18T10:15:13.986 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.986 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.986 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:13.986 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.148720+0000", 2024-09-18T10:15:13.986 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.25028612900000002 2024-09-18T10:15:13.986 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.986 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.986 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:13.986 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.148799+0000", 2024-09-18T10:15:13.986 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.8481999999999998e-05 2024-09-18T10:15:13.986 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.987 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.987 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:13.987 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.149383+0000", 2024-09-18T10:15:13.987 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000584255 2024-09-18T10:15:13.987 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.987 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.987 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:13.987 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.152442+0000", 2024-09-18T10:15:13.987 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0030591049999999999 2024-09-18T10:15:13.987 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.987 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.987 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-18T10:15:13.987 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.768702+0000", 2024-09-18T10:15:13.987 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.61626031599999997 2024-09-18T10:15:13.987 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.988 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.988 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:13.988 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.768878+0000", 2024-09-18T10:15:13.988 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00017600600000000001 2024-09-18T10:15:13.988 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.988 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:13.988 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.988 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.988 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.988 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:449 3.4s0 3:2650be24:::smithi18925896-45:head [setxattr _header (57) in=64b] snapc 13=[13,12,11,10,f,e,b] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-18T10:15:13.988 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:12.898446+0000", 2024-09-18T10:15:13.988 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.99895251100000004, 2024-09-18T10:15:13.988 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.87093100999999995, 2024-09-18T10:15:13.989 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:13.989 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:13.989 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:13.989 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:13.989 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:13.989 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 449 2024-09-18T10:15:13.989 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.989 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:13.989 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.989 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:13.989 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.898446+0000", 2024-09-18T10:15:13.989 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.989 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.989 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.990 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:13.990 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.898446+0000", 2024-09-18T10:15:13.990 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.990 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.990 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.990 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:13.990 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.898445+0000", 2024-09-18T10:15:13.990 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.999999 2024-09-18T10:15:13.990 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.990 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.990 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:13.990 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.898451+0000", 2024-09-18T10:15:13.990 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.054e-06 2024-09-18T10:15:13.991 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.991 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.991 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:13.991 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.898462+0000", 2024-09-18T10:15:13.991 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.081e-05 2024-09-18T10:15:13.991 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.991 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.991 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:13.991 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.898467+0000", 2024-09-18T10:15:13.991 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4220000000000002e-06 2024-09-18T10:15:13.991 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.991 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.991 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:13.991 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.149769+0000", 2024-09-18T10:15:13.991 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.25130268 2024-09-18T10:15:13.992 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.992 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.992 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:13.992 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.149964+0000", 2024-09-18T10:15:13.992 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00019506699999999999 2024-09-18T10:15:13.992 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.992 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.992 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:13.992 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.151037+0000", 2024-09-18T10:15:13.992 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0010723200000000001 2024-09-18T10:15:13.992 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.993 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.993 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:13.993 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.152623+0000", 2024-09-18T10:15:13.993 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0015865829999999999 2024-09-18T10:15:13.993 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.993 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.993 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-18T10:15:13.993 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.769215+0000", 2024-09-18T10:15:13.993 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.61659198800000004 2024-09-18T10:15:13.993 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.993 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.993 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:13.993 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.769377+0000", 2024-09-18T10:15:13.993 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000162134 2024-09-18T10:15:13.994 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.994 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:13.994 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.994 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.994 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.994 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 57/40 ECSubWriteReply(tid=32, last_complete=57'20, committed=1, applied=1))", 2024-09-18T10:15:13.994 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:13.469653+0000", 2024-09-18T10:15:13.994 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.427746134, 2024-09-18T10:15:13.994 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.29816387999999999, 2024-09-18T10:15:13.994 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:13.994 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:13.994 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:13.994 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.995 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:13.995 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.469653+0000", 2024-09-18T10:15:13.995 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.995 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.995 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.995 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:13.995 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.469653+0000", 2024-09-18T10:15:13.995 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.995 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.995 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.995 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:13.995 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.469651+0000", 2024-09-18T10:15:13.995 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:13.995 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.996 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.996 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:13.996 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.469662+0000", 2024-09-18T10:15:13.996 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0519999999999999e-05 2024-09-18T10:15:13.996 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.996 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.996 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:13.996 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.469693+0000", 2024-09-18T10:15:13.996 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1566000000000002e-05 2024-09-18T10:15:13.996 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.996 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.996 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:13.996 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.569787+0000", 2024-09-18T10:15:13.996 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10009314800000001 2024-09-18T10:15:13.997 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.997 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.997 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:13.997 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.767752+0000", 2024-09-18T10:15:13.997 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.197965274 2024-09-18T10:15:13.997 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.997 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.997 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:13.997 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.767817+0000", 2024-09-18T10:15:13.997 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.4764999999999994e-05 2024-09-18T10:15:13.997 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.997 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:13.997 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:13.997 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.998 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.998 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 57/40 ECSubWriteReply(tid=34, last_complete=57'22, committed=1, applied=1))", 2024-09-18T10:15:13.998 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:13.548016+0000", 2024-09-18T10:15:13.998 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.349382786, 2024-09-18T10:15:13.998 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.22234759000000001, 2024-09-18T10:15:13.998 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:13.998 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:13.998 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:13.998 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.998 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:13.998 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.548016+0000", 2024-09-18T10:15:13.998 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.998 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.998 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.999 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:13.999 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.548016+0000", 2024-09-18T10:15:13.999 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:13.999 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.999 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.999 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:13.999 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.548016+0000", 2024-09-18T10:15:13.999 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:13.999 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.999 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:13.999 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:13.999 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.548019+0000", 2024-09-18T10:15:13.999 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.4400000000000001e-06 2024-09-18T10:15:13.999 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:13.999 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.000 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:14.000 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.548032+0000", 2024-09-18T10:15:14.000 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2386e-05 2024-09-18T10:15:14.000 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.000 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.000 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:14.000 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.548038+0000", 2024-09-18T10:15:14.000 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3430000000000001e-06 2024-09-18T10:15:14.000 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.000 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.000 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:14.000 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.769091+0000", 2024-09-18T10:15:14.000 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.22105259199999999 2024-09-18T10:15:14.000 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.001 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.001 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:14.001 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.770364+0000", 2024-09-18T10:15:14.001 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001273139 2024-09-18T10:15:14.001 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:14.001 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:14.001 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:14.001 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:14.001 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:14.001 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-18T10:15:14.002 INFO:tasks.rados.rados.0.smithi189.stdout:176: finishing write tid 3 to smithi18925896-25 2024-09-18T10:15:14.002 INFO:tasks.rados.rados.0.smithi189.stdout:176: finishing write tid 4 to smithi18925896-25 2024-09-18T10:15:14.003 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_ops_in_flight 2024-09-18T10:15:14.004 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-18T10:15:13.998+0000 7f302f2c3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22187) UID: 0 2024-09-18T10:15:14.057 INFO:tasks.rados.rados.0.smithi189.stdout:176: finishing write tid 5 to smithi18925896-25 2024-09-18T10:15:14.057 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 25 v 25 (ObjNum 81 snap 13 seq_num 81) dirty exists 2024-09-18T10:15:14.057 INFO:tasks.rados.rados.0.smithi189.stdout:176: left oid 25 (ObjNum 81 snap 13 seq_num 81) 2024-09-18T10:15:14.072 INFO:tasks.rados.rados.0.smithi189.stdout:177: finishing write tid 5 to smithi18925896-45 2024-09-18T10:15:14.072 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 45 v 22 (ObjNum 82 snap 13 seq_num 82) dirty exists 2024-09-18T10:15:14.072 INFO:tasks.rados.rados.0.smithi189.stdout:177: left oid 45 (ObjNum 82 snap 13 seq_num 82) 2024-09-18T10:15:14.103 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-18T10:15:14.099+0000 7f14dcc5b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22398) UID: 0 2024-09-18T10:15:14.114 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-18T10:15:14.114 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-18T10:15:14.114 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 0 2024-09-18T10:15:14.114 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-18T10:15:14.124 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_blocked_ops 2024-09-18T10:15:14.197 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T10:15:14.205 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-18T10:15:14.198+0000 7f302f2c3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22187) UID: 0 2024-09-18T10:15:14.235 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-18T10:15:14.235 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-18T10:15:14.235 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-18T10:15:14.235 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-18T10:15:14.235 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-18T10:15:14.248 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_historic_ops 2024-09-18T10:15:14.306 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-18T10:15:14.303+0000 7f14dcc5b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22398) UID: 0 2024-09-18T10:15:14.351 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-18T10:15:14.371 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-18T10:15:14.371 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-18T10:15:14.371 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-18T10:15:14.371 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-18T10:15:14.372 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.372 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWrite(3.bs3 36/32 ECSubWrite(tid=29, reqid=client.4632.0:119, at_version=0'0, trim_to=0'0, roll_forward_to=36'4))", 2024-09-18T10:15:14.372 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:51.111475+0000", 2024-09-18T10:15:14.372 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 23.255273200000001, 2024-09-18T10:15:14.372 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.101555193, 2024-09-18T10:15:14.373 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:14.373 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:14.373 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:14.373 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.373 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:14.373 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.111475+0000", 2024-09-18T10:15:14.374 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:14.374 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.374 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.374 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:14.374 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.111475+0000", 2024-09-18T10:15:14.375 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:14.375 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.375 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.375 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:14.375 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.111474+0000", 2024-09-18T10:15:14.375 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:14.376 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.376 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.376 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:14.376 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.111492+0000", 2024-09-18T10:15:14.376 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7985000000000001e-05 2024-09-18T10:15:14.376 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.377 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.377 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:14.377 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.111511+0000", 2024-09-18T10:15:14.377 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8584e-05 2024-09-18T10:15:14.377 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.378 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.378 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:14.378 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.111521+0000", 2024-09-18T10:15:14.378 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0513000000000001e-05 2024-09-18T10:15:14.378 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.379 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.379 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:14.379 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.211779+0000", 2024-09-18T10:15:14.379 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10025748800000001 2024-09-18T10:15:14.379 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.379 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.380 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:14.380 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.211796+0000", 2024-09-18T10:15:14.380 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7655999999999998e-05 2024-09-18T10:15:14.380 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.380 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.381 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:14.381 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.212965+0000", 2024-09-18T10:15:14.381 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0011683679999999999 2024-09-18T10:15:14.381 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.381 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.382 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:14.382 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.213031+0000", 2024-09-18T10:15:14.382 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.5962000000000003e-05 2024-09-18T10:15:14.382 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:14.383 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:14.383 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:14.383 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.383 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.383 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpRead(3.bs3 36/32 ECSubRead(tid=30, to_read={3:d62b3dfa:::smithi18925896-40:head=0,4096,0}, subchunks={3:d62b3dfa:::smithi18925896-40:head=[0,1]}, attrs_to_read=))", 2024-09-18T10:15:14.383 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:51.112426+0000", 2024-09-18T10:15:14.384 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 23.254322564999999, 2024-09-18T10:15:14.384 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10039024000000001, 2024-09-18T10:15:14.384 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:14.384 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:14.385 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:14.385 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.385 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:14.385 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.112426+0000", 2024-09-18T10:15:14.385 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:14.385 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.386 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.386 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:14.386 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.112426+0000", 2024-09-18T10:15:14.386 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:14.386 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.387 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.387 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:14.387 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.112424+0000", 2024-09-18T10:15:14.387 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:14.387 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.388 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.388 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:14.388 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.112457+0000", 2024-09-18T10:15:14.388 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.3121999999999999e-05 2024-09-18T10:15:14.388 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.388 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.389 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:14.389 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.112512+0000", 2024-09-18T10:15:14.389 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.4929000000000002e-05 2024-09-18T10:15:14.389 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.389 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.390 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:14.390 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.112535+0000", 2024-09-18T10:15:14.390 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3376000000000001e-05 2024-09-18T10:15:14.390 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.390 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.390 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:14.391 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.212254+0000", 2024-09-18T10:15:14.391 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099718673999999993 2024-09-18T10:15:14.391 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.391 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.391 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:14.392 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.212816+0000", 2024-09-18T10:15:14.392 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00056218499999999999 2024-09-18T10:15:14.392 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:14.392 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:14.392 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:14.393 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.393 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.393 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWrite(3.cs5 36/32 ECSubWrite(tid=20, reqid=client.4632.0:150, at_version=36'9, trim_to=0'0, roll_forward_to=36'8))", 2024-09-18T10:15:14.393 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:51.127656+0000", 2024-09-18T10:15:14.393 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 23.239092594999999, 2024-09-18T10:15:14.394 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10369326700000001, 2024-09-18T10:15:14.394 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:14.394 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:14.394 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:14.394 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.394 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:14.395 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.127656+0000", 2024-09-18T10:15:14.395 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:14.395 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.395 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.395 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:14.396 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.127656+0000", 2024-09-18T10:15:14.396 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:14.396 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.396 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.396 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:14.396 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.127655+0000", 2024-09-18T10:15:14.397 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.999999 2024-09-18T10:15:14.397 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.397 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.397 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:14.397 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.127882+0000", 2024-09-18T10:15:14.397 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00022688799999999999 2024-09-18T10:15:14.398 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.398 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.398 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:14.398 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.127906+0000", 2024-09-18T10:15:14.398 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.4749e-05 2024-09-18T10:15:14.398 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.399 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.399 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:14.399 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.127917+0000", 2024-09-18T10:15:14.399 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0344e-05 2024-09-18T10:15:14.399 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.400 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.400 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:14.400 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.228119+0000", 2024-09-18T10:15:14.400 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100201944 2024-09-18T10:15:14.400 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.401 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.401 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:14.401 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.228135+0000", 2024-09-18T10:15:14.401 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.6354e-05 2024-09-18T10:15:14.401 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.401 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.402 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:14.402 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.231252+0000", 2024-09-18T10:15:14.402 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0031171200000000001 2024-09-18T10:15:14.402 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.402 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.402 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:14.403 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.231349+0000", 2024-09-18T10:15:14.403 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.7021999999999994e-05 2024-09-18T10:15:14.403 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:14.403 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:14.403 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:14.403 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.403 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.403 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWrite(3.cs5 36/32 ECSubWrite(tid=21, reqid=client.4632.0:151, at_version=36'10, trim_to=0'0, roll_forward_to=36'8))", 2024-09-18T10:15:14.403 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:51.130378+0000", 2024-09-18T10:15:14.403 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 23.236370535999999, 2024-09-18T10:15:14.404 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10150170999999999, 2024-09-18T10:15:14.404 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:14.404 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:14.404 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:14.404 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.404 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:14.404 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.130378+0000", 2024-09-18T10:15:14.404 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:14.404 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.404 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.405 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:14.405 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.130378+0000", 2024-09-18T10:15:14.405 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:14.405 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.405 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.405 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:14.405 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.130375+0000", 2024-09-18T10:15:14.405 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999967 2024-09-18T10:15:14.405 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.406 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.406 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:14.406 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.130518+0000", 2024-09-18T10:15:14.406 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00014364299999999999 2024-09-18T10:15:14.406 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.406 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.406 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:14.406 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.130540+0000", 2024-09-18T10:15:14.406 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2072000000000001e-05 2024-09-18T10:15:14.407 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.407 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.407 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:14.407 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.130553+0000", 2024-09-18T10:15:14.407 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2432999999999999e-05 2024-09-18T10:15:14.407 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.407 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.407 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:14.407 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.230723+0000", 2024-09-18T10:15:14.407 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10016994899999999 2024-09-18T10:15:14.408 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.408 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.408 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:14.408 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.230734+0000", 2024-09-18T10:15:14.408 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0874e-05 2024-09-18T10:15:14.408 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.408 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.408 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:14.409 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.231854+0000", 2024-09-18T10:15:14.409 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0011205399999999999 2024-09-18T10:15:14.409 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.409 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.409 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:14.409 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.231880+0000", 2024-09-18T10:15:14.409 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.5571e-05 2024-09-18T10:15:14.409 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:14.409 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:14.409 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:14.410 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.410 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.410 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWrite(3.es6 36/32 ECSubWrite(tid=4, reqid=client.4632.0:146, at_version=36'3, trim_to=0'0, roll_forward_to=36'2))", 2024-09-18T10:15:14.410 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:51.226533+0000", 2024-09-18T10:15:14.410 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 23.140215299000001, 2024-09-18T10:15:14.410 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.102071464, 2024-09-18T10:15:14.410 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:14.410 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:14.410 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:14.411 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.411 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:14.411 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.226533+0000", 2024-09-18T10:15:14.411 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:14.411 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.411 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.411 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:14.411 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.226533+0000", 2024-09-18T10:15:14.411 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:14.411 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.412 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.412 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:14.412 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.226531+0000", 2024-09-18T10:15:14.412 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:14.412 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.412 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.412 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:14.412 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.226807+0000", 2024-09-18T10:15:14.412 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00027599800000000001 2024-09-18T10:15:14.412 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.412 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.412 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:14.413 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.226849+0000", 2024-09-18T10:15:14.413 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1439000000000001e-05 2024-09-18T10:15:14.413 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.413 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.413 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:14.413 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.326947+0000", 2024-09-18T10:15:14.413 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100098859 2024-09-18T10:15:14.413 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.413 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.413 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:14.414 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.327061+0000", 2024-09-18T10:15:14.414 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00011343199999999999 2024-09-18T10:15:14.414 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.414 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.414 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:14.414 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.327083+0000", 2024-09-18T10:15:14.414 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2509000000000001e-05 2024-09-18T10:15:14.414 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.414 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.415 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:14.415 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.328552+0000", 2024-09-18T10:15:14.415 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0014686339999999999 2024-09-18T10:15:14.415 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.415 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.415 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:14.415 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.328605+0000", 2024-09-18T10:15:14.415 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2639000000000002e-05 2024-09-18T10:15:14.415 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:14.415 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:14.415 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:14.415 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.415 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.415 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWrite(3.bs3 36/32 ECSubWrite(tid=37, reqid=client.4632.0:160, at_version=0'0, trim_to=0'0, roll_forward_to=36'5))", 2024-09-18T10:15:14.416 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:51.531559+0000", 2024-09-18T10:15:14.416 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 22.835189110000002, 2024-09-18T10:15:14.416 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10164537799999999, 2024-09-18T10:15:14.416 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:14.416 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:14.416 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:14.416 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.416 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:14.416 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.531559+0000", 2024-09-18T10:15:14.416 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:14.416 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.416 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.416 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:14.416 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.531559+0000", 2024-09-18T10:15:14.417 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:14.417 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.417 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.417 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:14.417 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.531558+0000", 2024-09-18T10:15:14.417 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:14.417 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.417 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.417 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:14.417 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.531584+0000", 2024-09-18T10:15:14.417 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6066000000000001e-05 2024-09-18T10:15:14.418 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.418 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.418 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:14.418 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.531620+0000", 2024-09-18T10:15:14.418 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.6625999999999998e-05 2024-09-18T10:15:14.418 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.418 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.418 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:14.418 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.531635+0000", 2024-09-18T10:15:14.418 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4175999999999999e-05 2024-09-18T10:15:14.418 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.418 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.418 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:14.418 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.631831+0000", 2024-09-18T10:15:14.419 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10019641 2024-09-18T10:15:14.419 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.419 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.419 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:14.419 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.631859+0000", 2024-09-18T10:15:14.419 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.8289e-05 2024-09-18T10:15:14.419 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.419 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.419 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:14.419 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.633127+0000", 2024-09-18T10:15:14.419 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001267287 2024-09-18T10:15:14.419 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.419 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.419 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:14.420 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.633205+0000", 2024-09-18T10:15:14.420 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.8244000000000005e-05 2024-09-18T10:15:14.420 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:14.420 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:14.420 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:14.420 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.420 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.420 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWrite(3.es6 36/32 ECSubWrite(tid=7, reqid=client.4632.0:148, at_version=36'5, trim_to=0'0, roll_forward_to=36'4))", 2024-09-18T10:15:14.420 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:51.835142+0000", 2024-09-18T10:15:14.420 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 22.531606570000001, 2024-09-18T10:15:14.420 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.101412908, 2024-09-18T10:15:14.420 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:14.420 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:14.421 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:14.421 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.421 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:14.421 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.835142+0000", 2024-09-18T10:15:14.421 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:14.421 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.421 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.421 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:14.421 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.835142+0000", 2024-09-18T10:15:14.421 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:14.421 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.421 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.421 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:14.421 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.835140+0000", 2024-09-18T10:15:14.422 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:14.422 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.422 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.422 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:14.422 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.835164+0000", 2024-09-18T10:15:14.422 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3464000000000001e-05 2024-09-18T10:15:14.422 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.422 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.422 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:14.422 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.835183+0000", 2024-09-18T10:15:14.422 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.91e-05 2024-09-18T10:15:14.422 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.422 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.422 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:14.423 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.835192+0000", 2024-09-18T10:15:14.423 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.064e-06 2024-09-18T10:15:14.423 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.423 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.423 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:14.423 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.935399+0000", 2024-09-18T10:15:14.423 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100206792 2024-09-18T10:15:14.423 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.423 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.423 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:14.423 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.935414+0000", 2024-09-18T10:15:14.423 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5119e-05 2024-09-18T10:15:14.423 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.423 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.424 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:14.424 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.936521+0000", 2024-09-18T10:15:14.424 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0011073890000000001 2024-09-18T10:15:14.424 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.424 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.424 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:14.424 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.936555+0000", 2024-09-18T10:15:14.424 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.349e-05 2024-09-18T10:15:14.424 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:14.424 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:14.424 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:14.424 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.424 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.425 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWrite(3.cs5 37/32 ECSubWrite(tid=29, reqid=client.4632.0:181, at_version=37'13, trim_to=0'0, roll_forward_to=36'11))", 2024-09-18T10:15:14.425 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:52.895578+0000", 2024-09-18T10:15:14.425 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 21.471170041000001, 2024-09-18T10:15:14.425 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.102246569, 2024-09-18T10:15:14.425 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:14.425 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:14.425 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:14.425 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.425 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:14.426 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.895578+0000", 2024-09-18T10:15:14.426 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:14.426 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.426 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.426 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:14.426 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.895578+0000", 2024-09-18T10:15:14.426 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:14.426 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.427 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.427 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:14.427 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.895578+0000", 2024-09-18T10:15:14.427 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:14.427 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.427 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.427 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:14.427 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.895958+0000", 2024-09-18T10:15:14.427 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00038039299999999997 2024-09-18T10:15:14.428 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.428 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.428 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:14.428 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.895978+0000", 2024-09-18T10:15:14.428 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9896999999999999e-05 2024-09-18T10:15:14.428 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.428 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.428 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:14.428 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.895989+0000", 2024-09-18T10:15:14.428 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0438000000000001e-05 2024-09-18T10:15:14.428 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.428 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.428 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:14.428 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.996171+0000", 2024-09-18T10:15:14.429 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10018293 2024-09-18T10:15:14.429 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.429 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.429 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:14.429 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.996182+0000", 2024-09-18T10:15:14.430 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.006e-05 2024-09-18T10:15:14.430 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.430 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.430 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:14.430 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.997758+0000", 2024-09-18T10:15:14.430 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0015764520000000001 2024-09-18T10:15:14.431 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.431 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.431 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:14.431 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.997825+0000", 2024-09-18T10:15:14.431 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7061e-05 2024-09-18T10:15:14.432 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:14.432 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:14.432 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:14.432 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.432 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.432 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWrite(3.cs5 37/32 ECSubWrite(tid=33, reqid=client.4632.0:197, at_version=37'18, trim_to=0'0, roll_forward_to=36'11))", 2024-09-18T10:15:14.432 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:53.108605+0000", 2024-09-18T10:15:14.433 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 21.258143273999998, 2024-09-18T10:15:14.433 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.20153053300000001, 2024-09-18T10:15:14.433 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:14.433 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:14.433 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:14.433 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.433 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:14.433 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:53.108605+0000", 2024-09-18T10:15:14.433 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:14.433 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.433 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.433 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:14.434 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:53.108605+0000", 2024-09-18T10:15:14.434 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:14.434 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.434 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.434 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:14.434 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:53.108604+0000", 2024-09-18T10:15:14.435 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.999999 2024-09-18T10:15:14.435 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.435 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.435 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:14.435 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:53.108627+0000", 2024-09-18T10:15:14.435 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2954000000000001e-05 2024-09-18T10:15:14.435 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.436 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.436 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:14.436 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:53.108644+0000", 2024-09-18T10:15:14.436 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.6867e-05 2024-09-18T10:15:14.436 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.436 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.436 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:14.436 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:53.208724+0000", 2024-09-18T10:15:14.436 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10007999099999999 2024-09-18T10:15:14.436 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.436 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.436 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:14.436 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:53.308920+0000", 2024-09-18T10:15:14.437 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100196046 2024-09-18T10:15:14.437 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.437 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.437 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:14.437 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:53.308936+0000", 2024-09-18T10:15:14.437 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5537e-05 2024-09-18T10:15:14.437 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.437 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.437 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:14.437 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:53.310086+0000", 2024-09-18T10:15:14.437 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0011503850000000001 2024-09-18T10:15:14.437 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.437 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.437 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:14.438 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:53.310136+0000", 2024-09-18T10:15:14.438 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9514000000000001e-05 2024-09-18T10:15:14.438 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:14.438 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:14.438 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:14.438 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.438 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.438 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWrite(3.bs3 41/32 ECSubWrite(tid=58, reqid=osd.4.0:58, at_version=41'14, trim_to=0'0, roll_forward_to=39'12))", 2024-09-18T10:15:14.438 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:56.830265+0000", 2024-09-18T10:15:14.438 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 17.536483429, 2024-09-18T10:15:14.438 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10214519699999999, 2024-09-18T10:15:14.438 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:14.438 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:14.439 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:14.439 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.439 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:14.439 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.830265+0000", 2024-09-18T10:15:14.439 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:14.439 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.439 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.439 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:14.439 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.830265+0000", 2024-09-18T10:15:14.439 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:14.439 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.439 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.439 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:14.439 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.830263+0000", 2024-09-18T10:15:14.439 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:14.440 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.440 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.440 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:14.440 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.830323+0000", 2024-09-18T10:15:14.440 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9846000000000003e-05 2024-09-18T10:15:14.440 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.440 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.440 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:14.440 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.830370+0000", 2024-09-18T10:15:14.440 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6798999999999997e-05 2024-09-18T10:15:14.440 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.440 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.440 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:14.441 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.830389+0000", 2024-09-18T10:15:14.441 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9530000000000001e-05 2024-09-18T10:15:14.441 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.441 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.441 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:14.441 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.930612+0000", 2024-09-18T10:15:14.441 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100222795 2024-09-18T10:15:14.441 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.441 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.441 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:14.441 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.930648+0000", 2024-09-18T10:15:14.441 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.6115999999999998e-05 2024-09-18T10:15:14.441 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.441 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.442 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:14.442 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.932338+0000", 2024-09-18T10:15:14.442 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0016898130000000001 2024-09-18T10:15:14.442 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.442 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.442 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:14.442 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.932410+0000", 2024-09-18T10:15:14.442 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.2214999999999999e-05 2024-09-18T10:15:14.442 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:14.442 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:14.442 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:14.442 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.442 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.442 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWrite(3.es6 41/32 ECSubWrite(tid=16, reqid=osd.11.0:15, at_version=0'0, trim_to=0'0, roll_forward_to=41'11))", 2024-09-18T10:15:14.443 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:56.839285+0000", 2024-09-18T10:15:14.443 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 17.527463427000001, 2024-09-18T10:15:14.443 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100907127, 2024-09-18T10:15:14.443 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:14.443 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:14.443 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:14.443 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.443 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:14.443 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.839285+0000", 2024-09-18T10:15:14.443 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:14.443 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.443 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.443 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:14.444 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.839285+0000", 2024-09-18T10:15:14.444 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:14.444 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.444 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.444 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:14.444 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.839284+0000", 2024-09-18T10:15:14.444 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.999999 2024-09-18T10:15:14.444 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.444 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.444 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:14.444 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.839300+0000", 2024-09-18T10:15:14.444 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5852e-05 2024-09-18T10:15:14.445 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.445 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.445 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:14.445 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.839319+0000", 2024-09-18T10:15:14.445 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9242999999999999e-05 2024-09-18T10:15:14.445 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.445 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.445 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:14.445 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.839329+0000", 2024-09-18T10:15:14.445 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.4849999999999998e-06 2024-09-18T10:15:14.445 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.445 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.445 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:14.446 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.939487+0000", 2024-09-18T10:15:14.446 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100158518 2024-09-18T10:15:14.446 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.446 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.446 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:14.446 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.939503+0000", 2024-09-18T10:15:14.446 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5305999999999999e-05 2024-09-18T10:15:14.446 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.446 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.446 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:14.446 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.940151+0000", 2024-09-18T10:15:14.447 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00064866700000000002 2024-09-18T10:15:14.447 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.447 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.447 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:14.447 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.940192+0000", 2024-09-18T10:15:14.447 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0976000000000002e-05 2024-09-18T10:15:14.447 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:14.447 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:14.447 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:14.447 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.447 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.447 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWrite(3.bs3 41/32 ECSubWrite(tid=62, reqid=osd.4.0:58, at_version=0'0, trim_to=0'0, roll_forward_to=41'14))", 2024-09-18T10:15:14.447 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:57.647903+0000", 2024-09-18T10:15:14.447 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 16.718845458000001, 2024-09-18T10:15:14.448 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.20108230399999999, 2024-09-18T10:15:14.448 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:14.448 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:14.448 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:14.448 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.448 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:14.448 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.647903+0000", 2024-09-18T10:15:14.448 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:14.448 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.448 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.448 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:14.448 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.647903+0000", 2024-09-18T10:15:14.448 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:14.448 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.449 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.449 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:14.449 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.647901+0000", 2024-09-18T10:15:14.449 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999976 2024-09-18T10:15:14.449 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.449 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.449 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:14.449 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.647936+0000", 2024-09-18T10:15:14.449 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.5329e-05 2024-09-18T10:15:14.449 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.449 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.449 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:14.449 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.647978+0000", 2024-09-18T10:15:14.449 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2145999999999999e-05 2024-09-18T10:15:14.450 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.450 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.450 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:14.450 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.647996+0000", 2024-09-18T10:15:14.450 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7331999999999999e-05 2024-09-18T10:15:14.450 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.450 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.450 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:14.450 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.748248+0000", 2024-09-18T10:15:14.450 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100252307 2024-09-18T10:15:14.450 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.450 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.450 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:14.450 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.748280+0000", 2024-09-18T10:15:14.451 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2316e-05 2024-09-18T10:15:14.451 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.451 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.451 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:14.451 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.848945+0000", 2024-09-18T10:15:14.451 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100664434 2024-09-18T10:15:14.451 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.451 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.451 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:14.451 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.848985+0000", 2024-09-18T10:15:14.451 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0605999999999997e-05 2024-09-18T10:15:14.451 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:14.451 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:14.451 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:14.452 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.452 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.452 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWrite(3.bs3 46/32 ECSubWrite(tid=75, reqid=client.4632.0:327, at_version=0'0, trim_to=0'0, roll_forward_to=46'17))", 2024-09-18T10:15:14.452 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:02.228270+0000", 2024-09-18T10:15:14.452 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 12.138478771999999, 2024-09-18T10:15:14.452 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.101562179, 2024-09-18T10:15:14.452 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:14.452 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:14.452 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:14.452 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.452 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:14.452 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.228270+0000", 2024-09-18T10:15:14.452 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:14.453 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.453 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.453 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:14.453 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.228270+0000", 2024-09-18T10:15:14.453 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:14.453 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.453 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.453 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:14.453 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.228268+0000", 2024-09-18T10:15:14.453 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:14.453 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.453 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.454 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:14.454 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.228295+0000", 2024-09-18T10:15:14.454 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6809e-05 2024-09-18T10:15:14.454 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.454 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.454 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:14.454 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.228369+0000", 2024-09-18T10:15:14.454 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.3645000000000004e-05 2024-09-18T10:15:14.454 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.454 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.454 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:14.454 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.228388+0000", 2024-09-18T10:15:14.454 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9403e-05 2024-09-18T10:15:14.455 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.455 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.455 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:14.455 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.328674+0000", 2024-09-18T10:15:14.455 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100285687 2024-09-18T10:15:14.455 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.455 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.455 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:14.455 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.328690+0000", 2024-09-18T10:15:14.455 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5956000000000001e-05 2024-09-18T10:15:14.455 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.456 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.456 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:14.456 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.329701+0000", 2024-09-18T10:15:14.456 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0010117609999999999 2024-09-18T10:15:14.456 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.456 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.456 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:14.456 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.329832+0000", 2024-09-18T10:15:14.456 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000130541 2024-09-18T10:15:14.456 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:14.457 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:14.457 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:14.457 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.457 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.457 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWrite(3.es6 47/32 ECSubWrite(tid=21, reqid=client.4632.0:346, at_version=47'17, trim_to=0'0, roll_forward_to=44'15))", 2024-09-18T10:15:14.457 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:02.905682+0000", 2024-09-18T10:15:14.457 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 11.461066549, 2024-09-18T10:15:14.457 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.104106903, 2024-09-18T10:15:14.457 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:14.457 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:14.457 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:14.458 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.458 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:14.458 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.905682+0000", 2024-09-18T10:15:14.458 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:14.458 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.458 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.458 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:14.458 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.905682+0000", 2024-09-18T10:15:14.458 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:14.458 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.458 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.458 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:14.458 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.905682+0000", 2024-09-18T10:15:14.458 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:14.459 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.459 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.459 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:14.459 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.906384+0000", 2024-09-18T10:15:14.459 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000702362 2024-09-18T10:15:14.459 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.459 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.459 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:14.459 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.906428+0000", 2024-09-18T10:15:14.459 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4199000000000002e-05 2024-09-18T10:15:14.459 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.459 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.459 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:14.459 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:03.006578+0000", 2024-09-18T10:15:14.460 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100149618 2024-09-18T10:15:14.460 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.460 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.460 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:14.460 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:03.006758+0000", 2024-09-18T10:15:14.460 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00018075599999999999 2024-09-18T10:15:14.460 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.460 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.460 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:14.460 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:03.006775+0000", 2024-09-18T10:15:14.461 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.6673000000000001e-05 2024-09-18T10:15:14.461 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.461 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.461 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:14.461 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:03.009712+0000", 2024-09-18T10:15:14.461 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.002936569 2024-09-18T10:15:14.461 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.461 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.461 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:14.461 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:03.009789+0000", 2024-09-18T10:15:14.462 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7169000000000006e-05 2024-09-18T10:15:14.462 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:14.462 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:14.462 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:14.462 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.462 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.462 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWrite(3.cs5 47/32 ECSubWrite(tid=60, reqid=client.4632.0:341, at_version=47'38, trim_to=0'0, roll_forward_to=45'33))", 2024-09-18T10:15:14.462 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:02.911299+0000", 2024-09-18T10:15:14.462 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 11.455449923, 2024-09-18T10:15:14.462 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.101776582, 2024-09-18T10:15:14.463 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:14.463 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:14.463 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:14.463 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.463 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:14.463 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.911299+0000", 2024-09-18T10:15:14.463 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:14.463 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.463 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.463 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:14.463 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.911299+0000", 2024-09-18T10:15:14.463 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:14.463 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.464 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.464 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:14.464 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.911298+0000", 2024-09-18T10:15:14.464 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.999999 2024-09-18T10:15:14.464 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.464 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.464 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:14.464 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.911361+0000", 2024-09-18T10:15:14.464 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3900999999999997e-05 2024-09-18T10:15:14.464 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.465 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.465 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:14.465 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.911392+0000", 2024-09-18T10:15:14.465 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0275000000000001e-05 2024-09-18T10:15:14.465 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.465 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.465 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:14.465 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.911404+0000", 2024-09-18T10:15:14.465 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2682e-05 2024-09-18T10:15:14.465 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.465 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.465 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:14.466 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:03.011587+0000", 2024-09-18T10:15:14.466 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10018239 2024-09-18T10:15:14.466 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.466 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.466 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:14.466 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:03.011607+0000", 2024-09-18T10:15:14.466 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0329000000000001e-05 2024-09-18T10:15:14.466 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.466 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.466 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:14.466 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:03.013018+0000", 2024-09-18T10:15:14.466 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001410863 2024-09-18T10:15:14.467 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.467 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.467 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:14.467 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:03.013075+0000", 2024-09-18T10:15:14.467 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7182000000000002e-05 2024-09-18T10:15:14.467 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:14.467 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:14.467 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:14.467 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.467 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.467 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWrite(3.cs5 47/32 ECSubWrite(tid=61, reqid=client.4632.0:342, at_version=47'39, trim_to=0'0, roll_forward_to=45'33))", 2024-09-18T10:15:14.468 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:02.913383+0000", 2024-09-18T10:15:14.468 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 11.453365809999999, 2024-09-18T10:15:14.468 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10027179, 2024-09-18T10:15:14.468 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:14.468 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:14.468 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:14.468 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.468 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:14.468 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.913383+0000", 2024-09-18T10:15:14.468 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:14.468 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.468 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.469 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:14.469 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.913383+0000", 2024-09-18T10:15:14.469 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:14.469 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.469 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.469 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:14.469 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.913377+0000", 2024-09-18T10:15:14.470 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999938 2024-09-18T10:15:14.470 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.470 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.470 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:14.470 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.913410+0000", 2024-09-18T10:15:14.470 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.3611000000000001e-05 2024-09-18T10:15:14.470 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.470 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.470 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:14.470 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.913441+0000", 2024-09-18T10:15:14.470 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1100999999999999e-05 2024-09-18T10:15:14.471 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.471 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.471 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:14.471 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.913457+0000", 2024-09-18T10:15:14.471 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5954e-05 2024-09-18T10:15:14.471 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.471 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.471 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:14.471 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:03.012156+0000", 2024-09-18T10:15:14.471 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.098699091000000003 2024-09-18T10:15:14.471 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.471 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.471 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:14.471 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:03.012171+0000", 2024-09-18T10:15:14.472 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4807000000000001e-05 2024-09-18T10:15:14.472 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.472 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.472 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:14.472 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:03.013597+0000", 2024-09-18T10:15:14.473 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001425976 2024-09-18T10:15:14.473 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.473 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.473 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:14.473 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:03.013655+0000", 2024-09-18T10:15:14.473 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7281999999999998e-05 2024-09-18T10:15:14.474 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:14.474 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:14.474 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:14.474 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.474 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.474 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDScrubReserve(3.15s4 RELEASE e52)", 2024-09-18T10:15:14.474 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:07.816881+0000", 2024-09-18T10:15:14.475 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 6.5498670910000003, 2024-09-18T10:15:14.475 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.103296636, 2024-09-18T10:15:14.475 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:14.475 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:14.475 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:14.475 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.475 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:14.475 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.816881+0000", 2024-09-18T10:15:14.475 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:14.475 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.475 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.475 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:14.476 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.816881+0000", 2024-09-18T10:15:14.476 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:14.476 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.476 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.476 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:14.476 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.816881+0000", 2024-09-18T10:15:14.477 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:14.477 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.477 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.477 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:14.477 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.816886+0000", 2024-09-18T10:15:14.477 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0769999999999998e-06 2024-09-18T10:15:14.477 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.478 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.478 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:14.478 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.816911+0000", 2024-09-18T10:15:14.478 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.495e-05 2024-09-18T10:15:14.478 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.478 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.478 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:14.478 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.816926+0000", 2024-09-18T10:15:14.478 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5148e-05 2024-09-18T10:15:14.478 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.478 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.478 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:14.478 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.817027+0000", 2024-09-18T10:15:14.479 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00010067499999999999 2024-09-18T10:15:14.479 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.479 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.479 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op must wait for map", 2024-09-18T10:15:14.479 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.817046+0000", 2024-09-18T10:15:14.479 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8836999999999999e-05 2024-09-18T10:15:14.479 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.479 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.479 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:14.479 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.920112+0000", 2024-09-18T10:15:14.479 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.103066402 2024-09-18T10:15:14.479 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.479 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.479 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:14.480 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.920137+0000", 2024-09-18T10:15:14.480 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.4454e-05 2024-09-18T10:15:14.480 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.480 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.480 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:14.480 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.920178+0000", 2024-09-18T10:15:14.480 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1570999999999999e-05 2024-09-18T10:15:14.480 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:14.480 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:14.481 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:14.481 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.481 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.481 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWrite(3.es6 57/32 ECSubWrite(tid=26, reqid=osd.11.0:26, at_version=57'21, trim_to=0'0, roll_forward_to=47'19))", 2024-09-18T10:15:14.481 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:13.052342+0000", 2024-09-18T10:15:14.481 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 1.3144069090000001, 2024-09-18T10:15:14.481 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10095557199999999, 2024-09-18T10:15:14.481 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:14.481 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:14.481 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:14.481 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.481 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:14.481 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.052342+0000", 2024-09-18T10:15:14.481 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:14.482 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.482 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.482 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:14.482 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.052342+0000", 2024-09-18T10:15:14.482 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:14.482 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.482 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.482 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:14.482 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.052341+0000", 2024-09-18T10:15:14.482 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:14.482 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.482 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.482 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:14.482 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.052385+0000", 2024-09-18T10:15:14.483 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4277000000000003e-05 2024-09-18T10:15:14.483 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.483 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.483 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:14.483 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.052406+0000", 2024-09-18T10:15:14.483 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0903999999999998e-05 2024-09-18T10:15:14.483 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.483 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.483 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:14.483 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.052422+0000", 2024-09-18T10:15:14.483 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5193e-05 2024-09-18T10:15:14.484 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.484 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.484 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:14.484 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.152588+0000", 2024-09-18T10:15:14.484 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100166009 2024-09-18T10:15:14.484 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.484 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.484 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:14.484 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.152596+0000", 2024-09-18T10:15:14.484 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.4910000000000005e-06 2024-09-18T10:15:14.484 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.484 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.484 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:14.485 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.153271+0000", 2024-09-18T10:15:14.485 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00067446900000000002 2024-09-18T10:15:14.485 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.485 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.485 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:14.485 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.153297+0000", 2024-09-18T10:15:14.485 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6696999999999999e-05 2024-09-18T10:15:14.485 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:14.485 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:14.485 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:14.485 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.485 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.485 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWrite(3.cs5 57/32 ECSubWrite(tid=75, reqid=osd.5.0:75, at_version=57'54, trim_to=0'0, roll_forward_to=57'50))", 2024-09-18T10:15:14.486 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:13.258407+0000", 2024-09-18T10:15:14.486 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 1.108341421, 2024-09-18T10:15:14.486 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10139938799999999, 2024-09-18T10:15:14.486 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:14.486 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:14.486 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:14.486 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.486 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:14.486 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.258407+0000", 2024-09-18T10:15:14.486 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:14.486 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.486 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.486 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:14.487 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.258407+0000", 2024-09-18T10:15:14.487 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:14.487 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.487 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.487 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:14.487 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.258406+0000", 2024-09-18T10:15:14.487 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:14.487 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.487 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.487 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:14.487 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.258450+0000", 2024-09-18T10:15:14.487 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4685999999999999e-05 2024-09-18T10:15:14.487 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.487 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.488 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:14.488 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.258477+0000", 2024-09-18T10:15:14.488 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6906e-05 2024-09-18T10:15:14.488 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.488 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.488 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:14.488 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.258490+0000", 2024-09-18T10:15:14.488 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2723e-05 2024-09-18T10:15:14.488 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.488 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.489 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:14.489 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.358659+0000", 2024-09-18T10:15:14.489 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100169231 2024-09-18T10:15:14.489 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.489 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.489 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:14.489 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.358679+0000", 2024-09-18T10:15:14.489 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9324999999999999e-05 2024-09-18T10:15:14.489 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.489 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.489 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:14.489 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.359767+0000", 2024-09-18T10:15:14.489 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0010879640000000001 2024-09-18T10:15:14.490 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.490 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.490 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:14.490 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.359807+0000", 2024-09-18T10:15:14.490 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9860000000000001e-05 2024-09-18T10:15:14.490 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:14.490 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:14.490 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:14.490 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.490 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.490 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWrite(3.cs5 58/32 ECSubWrite(tid=78, reqid=osd.5.0:77, at_version=0'0, trim_to=0'0, roll_forward_to=57'56))", 2024-09-18T10:15:14.490 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:13.858719+0000", 2024-09-18T10:15:14.491 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.50802942699999998, 2024-09-18T10:15:14.491 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.105635402, 2024-09-18T10:15:14.491 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:14.491 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:14.491 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:14.491 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.491 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:14.491 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.858719+0000", 2024-09-18T10:15:14.491 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:14.491 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.491 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.491 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:14.491 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.858719+0000", 2024-09-18T10:15:14.492 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:14.492 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.492 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.492 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:14.492 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.858719+0000", 2024-09-18T10:15:14.492 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:14.492 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.492 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.492 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:14.492 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.858731+0000", 2024-09-18T10:15:14.492 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2007999999999999e-05 2024-09-18T10:15:14.492 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.493 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.493 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:14.493 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.858755+0000", 2024-09-18T10:15:14.493 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.4681e-05 2024-09-18T10:15:14.493 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.493 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.493 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:14.493 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.958911+0000", 2024-09-18T10:15:14.493 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10015579300000001 2024-09-18T10:15:14.493 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.493 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.494 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:14.494 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.962752+0000", 2024-09-18T10:15:14.494 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0038403259999999998 2024-09-18T10:15:14.494 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.494 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.494 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:14.494 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.962767+0000", 2024-09-18T10:15:14.494 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5494000000000001e-05 2024-09-18T10:15:14.494 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.494 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.495 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:14.495 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.964314+0000", 2024-09-18T10:15:14.495 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0015469189999999999 2024-09-18T10:15:14.495 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:14.495 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:14.495 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:14.495 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.964355+0000", 2024-09-18T10:15:14.495 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0491000000000002e-05 2024-09-18T10:15:14.495 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:14.495 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:14.495 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:14.495 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:14.495 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:14.495 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-18T10:15:14.497 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T10:15:14.498 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-18T10:15:14.499 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-18T10:15:14.499 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-18T10:15:14.499 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-18T10:15:14.499 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-18T10:15:14.499 INFO:tasks.ceph.osd.15.smithi189.stderr:2024-09-18T10:15:14.495+0000 7fd0e725e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 22401) UID: 0 2024-09-18T10:15:14.502 INFO:teuthology.orchestra.run.smithi029.stdout:osd.6: {} 2024-09-18T10:15:14.525 INFO:teuthology.orchestra.run.smithi029.stdout:osd.7: {} 2024-09-18T10:15:14.553 INFO:tasks.rados.rados.0.smithi189.stdout:176: done (2 left) 2024-09-18T10:15:14.553 INFO:tasks.rados.rados.0.smithi189.stdout:177: done (1 left) 2024-09-18T10:15:14.553 INFO:tasks.rados.rados.0.smithi189.stdout:180: done (0 left) 2024-09-18T10:15:14.553 INFO:tasks.rados.rados.0.smithi189.stdout:append oid 31 current snap is 14 2024-09-18T10:15:14.553 INFO:tasks.rados.rados.0.smithi189.stdout:181: seq_num 84 ranges {507904=655360,1163264=638976,1802240=131072} 2024-09-18T10:15:14.557 INFO:teuthology.orchestra.run.smithi029.stderr:reweighted osd.5 to 1 (10000) 2024-09-18T10:15:14.567 INFO:tasks.rados.rados.0.smithi189.stdout:181: writing smithi18925896-31 from 507904 to 1163264 tid 1 2024-09-18T10:15:14.571 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-18T10:15:14.571 INFO:tasks.rados.rados.0.smithi189.stdout:181: writing smithi18925896-31 from 1163264 to 1802240 tid 2 2024-09-18T10:15:14.572 INFO:tasks.rados.rados.0.smithi189.stdout:181: writing smithi18925896-31 from 1802240 to 1933312 tid 3 2024-09-18T10:15:14.572 INFO:tasks.rados.rados.0.smithi189.stdout:append oid 16 current snap is 14 2024-09-18T10:15:14.572 INFO:tasks.rados.rados.0.smithi189.stdout:182: seq_num 85 ranges {262144=671744,933888=770048} 2024-09-18T10:15:14.577 INFO:tasks.rados.rados.0.smithi189.stdout:182: writing smithi18925896-16 from 262144 to 933888 tid 1 2024-09-18T10:15:14.581 INFO:tasks.rados.rados.0.smithi189.stdout:182: writing smithi18925896-16 from 933888 to 1703936 tid 2 2024-09-18T10:15:14.582 INFO:tasks.rados.rados.0.smithi189.stdout:183: copy_from oid 10 from oid 39 current snap is 14 2024-09-18T10:15:14.582 INFO:tasks.rados.rados.0.smithi189.stdout:184: read oid 48 snap 9 2024-09-18T10:15:14.582 INFO:tasks.rados.rados.0.smithi189.stdout:184: expect (ObjNum 62 snap 5 seq_num 62) 2024-09-18T10:15:14.582 INFO:tasks.rados.rados.0.smithi189.stdout:append oid 50 current snap is 14 2024-09-18T10:15:14.582 INFO:tasks.rados.rados.0.smithi189.stdout:185: seq_num 86 ranges {835584=688128,1523712=540672,2064384=229376} 2024-09-18T10:15:14.590 INFO:tasks.rados.rados.0.smithi189.stdout:185: writing smithi18925896-50 from 835584 to 1523712 tid 1 2024-09-18T10:15:14.593 INFO:tasks.rados.rados.0.smithi189.stdout:185: writing smithi18925896-50 from 1523712 to 2064384 tid 2 2024-09-18T10:15:14.594 INFO:tasks.rados.rados.0.smithi189.stdout:185: writing smithi18925896-50 from 2064384 to 2293760 tid 3 2024-09-18T10:15:14.594 INFO:tasks.rados.rados.0.smithi189.stdout:186: delete oid 13 current snap is 14 2024-09-18T10:15:14.598 INFO:tasks.ceph.osd.8.smithi029.stderr:2024-09-18T10:15:14.594+0000 7f2555c27700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 22192) UID: 0 2024-09-18T10:15:14.605 INFO:teuthology.orchestra.run.smithi029.stdout:osd.8: {} 2024-09-18T10:15:14.615 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:14.615 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:14.626 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T10:15:14.626 INFO:teuthology.orchestra.run.smithi029.stdout:osd.9: {} 2024-09-18T10:15:14.655 INFO:teuthology.orchestra.run.smithi029.stdout:osd.10: {} 2024-09-18T10:15:14.686 INFO:teuthology.orchestra.run.smithi029.stdout:osd.11: {} 2024-09-18T10:15:14.698 INFO:tasks.ceph.osd.8.smithi029.stderr:2024-09-18T10:15:14.694+0000 7f2555c27700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 22192) UID: 0 2024-09-18T10:15:14.707 INFO:teuthology.orchestra.run.smithi029.stdout:osd.12: {} 2024-09-18T10:15:14.729 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:14.729 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:14.739 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T10:15:14.741 INFO:teuthology.orchestra.run.smithi029.stdout:osd.13: {} 2024-09-18T10:15:14.775 INFO:teuthology.orchestra.run.smithi029.stdout:osd.14: {} 2024-09-18T10:15:14.800 INFO:tasks.ceph.osd.11.smithi189.stderr:2024-09-18T10:15:14.795+0000 7f718b60a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 22400) UID: 0 2024-09-18T10:15:14.809 INFO:teuthology.orchestra.run.smithi029.stdout:osd.15: {} 2024-09-18T10:15:14.811 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T10:15:14.812 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T10:15:14.812 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T10:15:14.812 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T10:15:14.812 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T10:15:14.812 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T10:15:14.812 INFO:teuthology.orchestra.run.smithi029.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T10:15:14.812 INFO:teuthology.orchestra.run.smithi029.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T10:15:14.812 INFO:teuthology.orchestra.run.smithi029.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-18T10:15:14.812 INFO:teuthology.orchestra.run.smithi029.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-18T10:15:14.812 INFO:teuthology.orchestra.run.smithi029.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-18T10:15:14.812 INFO:teuthology.orchestra.run.smithi029.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-18T10:15:14.813 INFO:teuthology.orchestra.run.smithi029.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-18T10:15:14.813 INFO:teuthology.orchestra.run.smithi029.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-18T10:15:14.813 INFO:teuthology.orchestra.run.smithi029.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-18T10:15:14.813 INFO:teuthology.orchestra.run.smithi029.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-18T10:15:14.840 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:14.840 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:14.849 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T10:15:14.887 INFO:tasks.rados.rados.0.smithi189.stdout:181: finishing write tid 1 to smithi18925896-31 2024-09-18T10:15:14.887 INFO:tasks.rados.rados.0.smithi189.stdout:181: finishing write tid 2 to smithi18925896-31 2024-09-18T10:15:14.891 INFO:tasks.rados.rados.0.smithi189.stdout:184: expect (ObjNum 62 snap 5 seq_num 62) 2024-09-18T10:15:14.900 INFO:tasks.ceph.osd.8.smithi029.stderr:2024-09-18T10:15:14.894+0000 7f2555c27700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 22192) UID: 0 2024-09-18T10:15:14.930 INFO:tasks.rados.rados.0.smithi189.stdout:183: finishing copy_from racing read to smithi18925896-10 2024-09-18T10:15:14.930 INFO:tasks.rados.rados.0.smithi189.stdout:184: done (5 left) 2024-09-18T10:15:14.930 INFO:tasks.rados.rados.0.smithi189.stdout:186: done (4 left) 2024-09-18T10:15:14.930 INFO:tasks.rados.rados.0.smithi189.stdout:187: delete oid 41 current snap is 14 2024-09-18T10:15:14.949 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:14.949 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:14.958 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T10:15:14.983 INFO:tasks.rados.rados.0.smithi189.stdout:182: finishing write tid 1 to smithi18925896-16 2024-09-18T10:15:15.001 INFO:tasks.ceph.osd.10.smithi154.stderr:2024-09-18T10:15:14.995+0000 7fb73221a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 22091) UID: 0 2024-09-18T10:15:15.055 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:15.055 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:15.064 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T10:15:15.102 INFO:tasks.ceph.osd.14.smithi154.stderr:2024-09-18T10:15:15.095+0000 7f75a19e7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 22093) UID: 0 2024-09-18T10:15:15.166 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:15.166 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:15.175 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-09-18T10:15:15.204 INFO:tasks.ceph.osd.3.smithi189.stderr:2024-09-18T10:15:15.199+0000 7f2440fc7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22396) UID: 0 2024-09-18T10:15:15.273 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:15.273 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:15.282 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-09-18T10:15:15.294 INFO:tasks.rados.rados.0.smithi189.stdout:183: finishing copy_from to smithi18925896-10 2024-09-18T10:15:15.294 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 10 v 19 (ObjNum 68 snap 6 seq_num 68) dirty exists 2024-09-18T10:15:15.304 INFO:tasks.ceph.osd.11.smithi189.stderr:2024-09-18T10:15:15.299+0000 7f718b60a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 22400) UID: 0 2024-09-18T10:15:15.336 INFO:tasks.rados.rados.0.smithi189.stdout:183: done (4 left) 2024-09-18T10:15:15.336 INFO:tasks.rados.rados.0.smithi189.stdout:187: done (3 left) 2024-09-18T10:15:15.336 INFO:tasks.rados.rados.0.smithi189.stdout:188: rmattr oid 41 current snap is 14 2024-09-18T10:15:15.336 INFO:tasks.rados.rados.0.smithi189.stdout:188: done (3 left) 2024-09-18T10:15:15.336 INFO:tasks.rados.rados.0.smithi189.stdout:189: snap_remove snap 5 2024-09-18T10:15:15.381 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:15.381 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:15.390 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-09-18T10:15:15.405 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-18T10:15:15.401+0000 7f14c6a1a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 22138) UID: 0 2024-09-18T10:15:15.488 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:15.488 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:15.497 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_ops_in_flight 2024-09-18T10:15:15.506 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-18T10:15:15.502+0000 7f302f2c3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22187) UID: 0 2024-09-18T10:15:15.574 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-18T10:15:15.592 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:15.592 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:15.602 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_blocked_ops 2024-09-18T10:15:15.606 INFO:tasks.ceph.osd.10.smithi154.stderr:2024-09-18T10:15:15.599+0000 7fb73221a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 22091) UID: 0 2024-09-18T10:15:15.702 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:15.702 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:15.707 INFO:tasks.ceph.osd.15.smithi189.stderr:2024-09-18T10:15:15.703+0000 7fd0e725e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 22401) UID: 0 2024-09-18T10:15:15.708 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_historic_ops 2024-09-18T10:15:15.808 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-18T10:15:15.805+0000 7f578c52f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22134) UID: 0 2024-09-18T10:15:15.845 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:15.845 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:15.846 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T10:15:15.869 INFO:tasks.rados.rados.0.smithi189.stdout:189: done (3 left) 2024-09-18T10:15:15.869 INFO:tasks.rados.rados.0.smithi189.stdout:190: read oid 37 snap -1 2024-09-18T10:15:15.869 INFO:tasks.rados.rados.0.smithi189.stdout:190: expect (ObjNum 36 snap 0 seq_num 36) 2024-09-18T10:15:15.869 INFO:tasks.rados.rados.0.smithi189.stdout:191: rmattr oid 11 current snap is 14 2024-09-18T10:15:15.869 INFO:tasks.rados.rados.0.smithi189.stdout:191: done (4 left) 2024-09-18T10:15:15.869 INFO:tasks.rados.rados.0.smithi189.stdout:192: snap_create 2024-09-18T10:15:15.869 INFO:tasks.rados.rados.0.smithi189.stdout:181: finishing write tid 3 to smithi18925896-31 2024-09-18T10:15:15.869 INFO:tasks.rados.rados.0.smithi189.stdout:181: finishing write tid 4 to smithi18925896-31 2024-09-18T10:15:15.870 INFO:tasks.rados.rados.0.smithi189.stdout:182: finishing write tid 2 to smithi18925896-16 2024-09-18T10:15:15.870 INFO:tasks.rados.rados.0.smithi189.stdout:185: finishing write tid 1 to smithi18925896-50 2024-09-18T10:15:15.870 INFO:tasks.rados.rados.0.smithi189.stdout:185: finishing write tid 2 to smithi18925896-50 2024-09-18T10:15:15.870 INFO:tasks.rados.rados.0.smithi189.stdout:185: finishing write tid 3 to smithi18925896-50 2024-09-18T10:15:15.870 INFO:tasks.rados.rados.0.smithi189.stdout:182: finishing write tid 3 to smithi18925896-16 2024-09-18T10:15:15.870 INFO:tasks.rados.rados.0.smithi189.stdout:185: finishing write tid 4 to smithi18925896-50 2024-09-18T10:15:15.872 INFO:teuthology.orchestra.run.smithi029.stderr:reweighted osd.6 to 1 (10000) 2024-09-18T10:15:15.887 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-18T10:15:15.901 INFO:tasks.rados.rados.0.smithi189.stdout:181: finishing write tid 5 to smithi18925896-31 2024-09-18T10:15:15.902 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 31 v 61 (ObjNum 84 snap 14 seq_num 84) dirty exists 2024-09-18T10:15:15.902 INFO:tasks.rados.rados.0.smithi189.stdout:181: left oid 31 (ObjNum 84 snap 14 seq_num 84) 2024-09-18T10:15:15.903 INFO:tasks.rados.rados.0.smithi189.stdout:185: finishing write tid 5 to smithi18925896-50 2024-09-18T10:15:15.903 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 50 v 66 (ObjNum 86 snap 14 seq_num 86) dirty exists 2024-09-18T10:15:15.903 INFO:tasks.rados.rados.0.smithi189.stdout:185: left oid 50 (ObjNum 86 snap 14 seq_num 86) 2024-09-18T10:15:15.909 INFO:tasks.ceph.osd.12.smithi029.stderr:2024-09-18T10:15:15.902+0000 7f8e4e627700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 22190) UID: 0 2024-09-18T10:15:15.955 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:15.955 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:15.965 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T10:15:16.001 INFO:tasks.rados.rados.0.smithi189.stdout:182: finishing write tid 4 to smithi18925896-16 2024-09-18T10:15:16.001 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 16 v 30 (ObjNum 85 snap 14 seq_num 85) dirty exists 2024-09-18T10:15:16.001 INFO:tasks.rados.rados.0.smithi189.stdout:182: left oid 16 (ObjNum 85 snap 14 seq_num 85) 2024-09-18T10:15:16.010 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-18T10:15:16.009+0000 7f14c6a1a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 22138) UID: 0 2024-09-18T10:15:16.071 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:16.071 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:16.080 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T10:15:16.113 INFO:tasks.ceph.osd.15.smithi189.stderr:2024-09-18T10:15:16.110+0000 7fd0e725e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 22401) UID: 0 2024-09-18T10:15:16.177 INFO:tasks.rados.rados.0.smithi189.stdout:190: expect (ObjNum 36 snap 0 seq_num 36) 2024-09-18T10:15:16.181 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:16.181 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:16.191 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T10:15:16.214 INFO:tasks.ceph.osd.11.smithi189.stderr:2024-09-18T10:15:16.210+0000 7f718b60a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 22400) UID: 0 2024-09-18T10:15:16.295 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:16.295 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:16.304 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T10:15:16.317 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-18T10:15:16.313+0000 7f00046a8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 22136) UID: 0 2024-09-18T10:15:16.399 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:16.399 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:16.408 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T10:15:16.418 INFO:tasks.ceph.osd.4.smithi029.stderr:2024-09-18T10:15:16.414+0000 7f550cf6e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22189) UID: 0 2024-09-18T10:15:16.507 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:16.507 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:16.515 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-09-18T10:15:16.522 INFO:tasks.ceph.osd.11.smithi189.stderr:2024-09-18T10:15:16.518+0000 7f718b60a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 22400) UID: 0 2024-09-18T10:15:16.609 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:16.609 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:16.618 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-09-18T10:15:16.622 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-09-18T10:15:16.615+0000 7fdfa8698700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22087) UID: 0 2024-09-18T10:15:16.714 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:16.714 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:16.726 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-09-18T10:15:16.727 INFO:tasks.ceph.osd.12.smithi029.stderr:2024-09-18T10:15:16.722+0000 7f8e4e627700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 22190) UID: 0 2024-09-18T10:15:16.825 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T10:15:16.827 INFO:tasks.ceph.osd.4.smithi029.stderr:2024-09-18T10:15:16.822+0000 7f550cf6e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22189) UID: 0 2024-09-18T10:15:16.828 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:16.829 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:16.839 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_ops_in_flight 2024-09-18T10:15:16.873 INFO:tasks.rados.rados.0.smithi189.stdout:181: done (4 left) 2024-09-18T10:15:16.873 INFO:tasks.rados.rados.0.smithi189.stdout:182: done (3 left) 2024-09-18T10:15:16.873 INFO:tasks.rados.rados.0.smithi189.stdout:185: done (2 left) 2024-09-18T10:15:16.874 INFO:tasks.rados.rados.0.smithi189.stdout:190: done (1 left) 2024-09-18T10:15:16.874 INFO:tasks.rados.rados.0.smithi189.stdout:192: done (0 left) 2024-09-18T10:15:16.874 INFO:tasks.rados.rados.0.smithi189.stdout:193: copy_from oid 17 from oid 8 current snap is 15 2024-09-18T10:15:16.874 INFO:tasks.rados.rados.0.smithi189.stdout:194: read oid 32 snap -1 2024-09-18T10:15:16.874 INFO:tasks.rados.rados.0.smithi189.stdout:194: expect (ObjNum 36 snap 0 seq_num 36) 2024-09-18T10:15:16.874 INFO:tasks.rados.rados.0.smithi189.stdout:append oid 43 current snap is 15 2024-09-18T10:15:16.874 INFO:tasks.rados.rados.0.smithi189.stdout:195: seq_num 89 ranges {2064384=737280,2801664=573440,3375104=196608} 2024-09-18T10:15:16.879 INFO:teuthology.orchestra.run.smithi029.stderr:reweighted osd.7 to 1 (10000) 2024-09-18T10:15:16.881 INFO:teuthology.orchestra.run.smithi029.stderr:nodeep-scrub is unset 2024-09-18T10:15:16.892 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 8 1 2024-09-18T10:15:16.895 INFO:tasks.rados.rados.0.smithi189.stdout:195: writing smithi18925896-43 from 2064384 to 2801664 tid 1 2024-09-18T10:15:16.898 INFO:tasks.rados.rados.0.smithi189.stdout:195: writing smithi18925896-43 from 2801664 to 3375104 tid 2 2024-09-18T10:15:16.900 INFO:tasks.rados.rados.0.smithi189.stdout:195: writing smithi18925896-43 from 3375104 to 3571712 tid 3 2024-09-18T10:15:16.900 INFO:tasks.rados.rados.0.smithi189.stdout:196: setattr oid 39 current snap is 15 2024-09-18T10:15:16.901 INFO:tasks.rados.rados.0.smithi189.stdout:197: read oid 14 snap 13 2024-09-18T10:15:16.901 INFO:tasks.rados.rados.0.smithi189.stdout:197: expect (ObjNum 13 snap 0 seq_num 13) 2024-09-18T10:15:16.901 INFO:tasks.rados.rados.0.smithi189.stdout:198: snap_create 2024-09-18T10:15:16.928 INFO:tasks.ceph.osd.4.smithi029.stderr:2024-09-18T10:15:16.922+0000 7f550cf6e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22189) UID: 0 2024-09-18T10:15:16.940 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:16.941 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:16.950 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_blocked_ops 2024-09-18T10:15:16.978 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-18T10:15:17.011 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 39 v 25 (ObjNum 68 snap 6 seq_num 68) dirty exists 2024-09-18T10:15:17.029 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-18T10:15:17.025+0000 7f14c6a1a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 22138) UID: 0 2024-09-18T10:15:17.055 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:17.055 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:17.064 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_historic_ops 2024-09-18T10:15:17.065 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-18T10:15:17.088 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-18T10:15:17.110 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-18T10:15:17.117 INFO:tasks.rados.rados.0.smithi189.stdout:194: expect (ObjNum 36 snap 0 seq_num 36) 2024-09-18T10:15:17.130 INFO:tasks.ceph.osd.15.smithi189.stderr:2024-09-18T10:15:17.126+0000 7fd0e725e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 22401) UID: 0 2024-09-18T10:15:17.165 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:17.165 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:17.168 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-18T10:15:17.174 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T10:15:17.190 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-18T10:15:17.212 INFO:teuthology.orchestra.run.smithi029.stdout:osd.6: {} 2024-09-18T10:15:17.231 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-18T10:15:17.229+0000 7f14c6a1a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 22138) UID: 0 2024-09-18T10:15:17.233 INFO:teuthology.orchestra.run.smithi029.stdout:osd.7: {} 2024-09-18T10:15:17.252 INFO:tasks.rados.rados.0.smithi189.stdout:193: finishing copy_from racing read to smithi18925896-17 2024-09-18T10:15:17.255 INFO:teuthology.orchestra.run.smithi029.stdout:osd.8: {} 2024-09-18T10:15:17.277 INFO:teuthology.orchestra.run.smithi029.stdout:osd.9: {} 2024-09-18T10:15:17.285 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-18T10:15:17.285 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [ 2024-09-18T10:15:17.285 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.285 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.19s6 61/40 ECSubWrite(tid=109, reqid=client.4632.0:480, at_version=61'60, trim_to=0'0, roll_forward_to=60'58))", 2024-09-18T10:15:17.285 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:17.249146+0000", 2024-09-18T10:15:17.285 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.035769281999999999, 2024-09-18T10:15:17.286 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.035790067000000002, 2024-09-18T10:15:17.286 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:17.286 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "no flag points reached", 2024-09-18T10:15:17.286 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:17.286 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.286 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:17.286 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.249146+0000", 2024-09-18T10:15:17.286 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:17.286 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.286 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.286 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:17.287 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.249146+0000", 2024-09-18T10:15:17.287 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:17.287 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.287 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.287 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:17.287 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.249145+0000", 2024-09-18T10:15:17.287 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:15:17.287 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.287 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.287 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:17.287 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.249195+0000", 2024-09-18T10:15:17.287 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9747999999999999e-05 2024-09-18T10:15:17.288 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.288 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.288 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:17.288 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.249213+0000", 2024-09-18T10:15:17.288 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.8088999999999999e-05 2024-09-18T10:15:17.288 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:17.288 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:17.288 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:17.288 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:17.288 INFO:teuthology.orchestra.run.smithi189.stdout: ], 2024-09-18T10:15:17.288 INFO:teuthology.orchestra.run.smithi189.stdout: "num_ops": 1 2024-09-18T10:15:17.288 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-18T10:15:17.295 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T10:15:17.333 INFO:tasks.ceph.osd.6.smithi154.stderr:2024-09-18T10:15:17.327+0000 7f0abc9e1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22089) UID: 0 2024-09-18T10:15:17.377 INFO:teuthology.orchestra.run.smithi029.stdout:osd.10: {} 2024-09-18T10:15:17.399 INFO:teuthology.orchestra.run.smithi029.stdout:osd.11: {} 2024-09-18T10:15:17.400 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-18T10:15:17.400 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-18T10:15:17.400 INFO:teuthology.orchestra.run.smithi189.stdout: "complaint_time": 30, 2024-09-18T10:15:17.400 INFO:teuthology.orchestra.run.smithi189.stdout: "num_blocked_ops": 0 2024-09-18T10:15:17.400 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-18T10:15:17.409 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T10:15:17.413 INFO:tasks.rados.rados.0.smithi189.stdout:197: expect (ObjNum 13 snap 0 seq_num 13) 2024-09-18T10:15:17.422 INFO:teuthology.orchestra.run.smithi029.stdout:osd.12: {} 2024-09-18T10:15:17.433 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-18T10:15:17.429+0000 7f14c6a1a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 22138) UID: 0 2024-09-18T10:15:17.452 INFO:tasks.rados.rados.0.smithi189.stdout:195: finishing write tid 1 to smithi18925896-43 2024-09-18T10:15:17.452 INFO:tasks.rados.rados.0.smithi189.stdout:193: finishing copy_from to smithi18925896-17 2024-09-18T10:15:17.452 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 17 v 2 (ObjNum 7 snap 0 seq_num 7) dirty exists 2024-09-18T10:15:17.453 INFO:teuthology.orchestra.run.smithi029.stdout:osd.13: {} 2024-09-18T10:15:17.453 INFO:tasks.rados.rados.0.smithi189.stdout:195: finishing write tid 2 to smithi18925896-43 2024-09-18T10:15:17.478 INFO:teuthology.orchestra.run.smithi029.stdout:osd.14: {} 2024-09-18T10:15:17.501 INFO:teuthology.orchestra.run.smithi029.stdout:osd.15: {} 2024-09-18T10:15:17.504 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T10:15:17.504 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T10:15:17.505 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T10:15:17.505 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T10:15:17.505 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T10:15:17.505 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T10:15:17.505 INFO:teuthology.orchestra.run.smithi029.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T10:15:17.505 INFO:teuthology.orchestra.run.smithi029.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T10:15:17.505 INFO:teuthology.orchestra.run.smithi029.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-18T10:15:17.505 INFO:teuthology.orchestra.run.smithi029.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-18T10:15:17.505 INFO:teuthology.orchestra.run.smithi029.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-18T10:15:17.505 INFO:teuthology.orchestra.run.smithi029.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-18T10:15:17.505 INFO:teuthology.orchestra.run.smithi029.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-18T10:15:17.506 INFO:teuthology.orchestra.run.smithi029.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-18T10:15:17.506 INFO:teuthology.orchestra.run.smithi029.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-18T10:15:17.506 INFO:teuthology.orchestra.run.smithi029.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-18T10:15:17.512 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-18T10:15:17.512 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 20, 2024-09-18T10:15:17.512 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 600, 2024-09-18T10:15:17.512 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [ 2024-09-18T10:15:17.513 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.513 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 36/32 ECSubWriteReply(tid=5, last_complete=36'4, committed=1, applied=1))", 2024-09-18T10:15:17.513 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:50.303102+0000", 2024-09-18T10:15:17.513 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 27.206149404000001, 2024-09-18T10:15:17.513 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20304619600000001, 2024-09-18T10:15:17.513 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:17.513 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:17.513 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:17.513 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.513 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:17.513 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.303102+0000", 2024-09-18T10:15:17.513 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:17.514 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.514 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.514 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:17.514 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.303102+0000", 2024-09-18T10:15:17.514 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:17.514 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.514 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.514 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:17.514 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.303100+0000", 2024-09-18T10:15:17.514 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:17.514 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.514 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.514 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:17.515 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.303114+0000", 2024-09-18T10:15:17.515 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3866000000000001e-05 2024-09-18T10:15:17.515 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.515 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.515 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:17.515 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.303149+0000", 2024-09-18T10:15:17.515 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.5249000000000001e-05 2024-09-18T10:15:17.515 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.515 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.515 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:17.515 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.303174+0000", 2024-09-18T10:15:17.515 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4785000000000001e-05 2024-09-18T10:15:17.515 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.516 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.516 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:17.516 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.506129+0000", 2024-09-18T10:15:17.516 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20295469199999999 2024-09-18T10:15:17.516 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.516 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.516 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:17.516 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.506148+0000", 2024-09-18T10:15:17.516 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9720000000000001e-05 2024-09-18T10:15:17.516 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:17.516 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:17.516 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:17.516 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.517 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.517 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 36/32 ECSubWriteReply(tid=5, last_complete=36'4, committed=1, applied=1))", 2024-09-18T10:15:17.517 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:50.305623+0000", 2024-09-18T10:15:17.517 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 27.203628431999999, 2024-09-18T10:15:17.517 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40085840299999997, 2024-09-18T10:15:17.517 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:17.517 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:17.517 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:17.517 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.517 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:17.517 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.305623+0000", 2024-09-18T10:15:17.517 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:17.517 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.517 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.518 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:17.518 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.305623+0000", 2024-09-18T10:15:17.518 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:17.518 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.518 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.518 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:17.518 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.305621+0000", 2024-09-18T10:15:17.518 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999976 2024-09-18T10:15:17.518 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.518 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.518 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:17.518 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.305635+0000", 2024-09-18T10:15:17.518 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4283e-05 2024-09-18T10:15:17.518 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.518 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.519 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:17.519 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.305663+0000", 2024-09-18T10:15:17.519 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7475999999999999e-05 2024-09-18T10:15:17.519 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.519 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.519 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:17.519 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.305680+0000", 2024-09-18T10:15:17.519 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7609e-05 2024-09-18T10:15:17.519 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.519 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.519 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:17.519 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.706451+0000", 2024-09-18T10:15:17.519 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40077027399999998 2024-09-18T10:15:17.519 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.519 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.520 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:17.520 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.706481+0000", 2024-09-18T10:15:17.520 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0935000000000001e-05 2024-09-18T10:15:17.520 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:17.520 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:17.520 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:17.520 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.520 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.520 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 36/32 ECSubWriteReply(tid=5, last_complete=36'4, committed=1, applied=1))", 2024-09-18T10:15:17.520 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:50.305688+0000", 2024-09-18T10:15:17.520 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 27.203563068000001, 2024-09-18T10:15:17.520 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40093562900000002, 2024-09-18T10:15:17.520 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:17.520 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:17.521 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:17.521 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.521 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:17.521 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.305688+0000", 2024-09-18T10:15:17.521 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:17.521 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.521 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.521 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:17.521 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.305688+0000", 2024-09-18T10:15:17.521 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:17.521 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.521 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.521 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:17.522 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.305687+0000", 2024-09-18T10:15:17.522 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:17.522 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.522 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.522 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:17.522 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.305698+0000", 2024-09-18T10:15:17.522 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1763000000000001e-05 2024-09-18T10:15:17.522 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.522 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.522 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:17.522 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.305729+0000", 2024-09-18T10:15:17.522 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0502e-05 2024-09-18T10:15:17.522 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.522 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.522 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:17.523 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.305741+0000", 2024-09-18T10:15:17.523 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2363e-05 2024-09-18T10:15:17.523 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.523 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.523 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:17.523 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.706585+0000", 2024-09-18T10:15:17.523 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.400843265 2024-09-18T10:15:17.523 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.523 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.523 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:17.523 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.706624+0000", 2024-09-18T10:15:17.523 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9459000000000003e-05 2024-09-18T10:15:17.523 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:17.523 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:17.524 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:17.524 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.524 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.524 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 36/32 ECSubWriteReply(tid=4, last_complete=36'3, committed=1, applied=1))", 2024-09-18T10:15:17.524 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:50.404544+0000", 2024-09-18T10:15:17.524 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 27.104707553000001, 2024-09-18T10:15:17.524 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.30239659400000002, 2024-09-18T10:15:17.524 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:17.524 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:17.524 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:17.524 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.524 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:17.524 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.404544+0000", 2024-09-18T10:15:17.524 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:17.524 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.525 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.525 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:17.525 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.404544+0000", 2024-09-18T10:15:17.525 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:17.525 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.525 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.525 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:17.525 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.404542+0000", 2024-09-18T10:15:17.525 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:17.525 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.525 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.525 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:17.525 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.404555+0000", 2024-09-18T10:15:17.525 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3096e-05 2024-09-18T10:15:17.526 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.526 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.526 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:17.526 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.404576+0000", 2024-09-18T10:15:17.526 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1165000000000001e-05 2024-09-18T10:15:17.526 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.526 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.526 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:17.526 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.404587+0000", 2024-09-18T10:15:17.526 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0549e-05 2024-09-18T10:15:17.526 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.526 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.526 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:17.527 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.706657+0000", 2024-09-18T10:15:17.527 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.30206971999999999 2024-09-18T10:15:17.527 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.527 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.527 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:17.527 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.706941+0000", 2024-09-18T10:15:17.527 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00028382100000000002 2024-09-18T10:15:17.527 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:17.527 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:17.527 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:17.527 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.527 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.527 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 36/32 ECSubWriteReply(tid=5, last_complete=36'4, committed=1, applied=1))", 2024-09-18T10:15:17.528 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:50.406036+0000", 2024-09-18T10:15:17.528 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 27.103215336000002, 2024-09-18T10:15:17.528 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.30215917399999997, 2024-09-18T10:15:17.528 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:17.528 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:17.528 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:17.528 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.528 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:17.528 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.406036+0000", 2024-09-18T10:15:17.528 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:17.528 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.528 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.528 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:17.528 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.406036+0000", 2024-09-18T10:15:17.529 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:17.529 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.529 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.529 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:17.529 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.406035+0000", 2024-09-18T10:15:17.529 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:17.529 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.529 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.529 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:17.529 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.406046+0000", 2024-09-18T10:15:17.529 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1704e-05 2024-09-18T10:15:17.529 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.529 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.529 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:17.530 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.406065+0000", 2024-09-18T10:15:17.530 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.8448999999999999e-05 2024-09-18T10:15:17.530 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.530 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.530 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:17.530 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.406076+0000", 2024-09-18T10:15:17.530 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1286e-05 2024-09-18T10:15:17.530 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.530 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.530 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:17.530 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.707005+0000", 2024-09-18T10:15:17.530 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.30092866899999998 2024-09-18T10:15:17.530 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.530 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.530 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:17.531 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.708195+0000", 2024-09-18T10:15:17.531 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001190546 2024-09-18T10:15:17.531 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:17.531 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:17.531 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:17.531 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.531 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.531 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.cs3 36/32 ECSubWrite(tid=14, reqid=client.4632.0:92, at_version=0'0, trim_to=0'0, roll_forward_to=36'6))", 2024-09-18T10:15:17.531 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:50.504842+0000", 2024-09-18T10:15:17.531 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 27.004409934000002, 2024-09-18T10:15:17.531 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.202186116, 2024-09-18T10:15:17.531 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:17.531 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:17.531 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:17.531 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.532 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:17.532 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.504842+0000", 2024-09-18T10:15:17.532 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:17.532 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.532 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.532 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:17.532 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.504842+0000", 2024-09-18T10:15:17.532 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:17.532 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.532 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.532 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:17.532 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.504840+0000", 2024-09-18T10:15:17.532 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:17.533 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.533 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.533 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:17.533 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.504868+0000", 2024-09-18T10:15:17.533 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7809000000000001e-05 2024-09-18T10:15:17.533 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.533 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.533 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:17.533 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.504900+0000", 2024-09-18T10:15:17.533 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1921999999999997e-05 2024-09-18T10:15:17.533 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.533 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.533 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:17.533 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.605012+0000", 2024-09-18T10:15:17.534 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10011272 2024-09-18T10:15:17.534 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.534 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.534 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:17.534 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.605115+0000", 2024-09-18T10:15:17.534 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000103003 2024-09-18T10:15:17.534 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.534 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.534 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:17.534 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.605132+0000", 2024-09-18T10:15:17.534 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6359999999999999e-05 2024-09-18T10:15:17.534 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.534 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.535 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:17.535 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.706978+0000", 2024-09-18T10:15:17.535 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.101846672 2024-09-18T10:15:17.535 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.535 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.535 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:17.535 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.707028+0000", 2024-09-18T10:15:17.535 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9443000000000001e-05 2024-09-18T10:15:17.535 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:17.535 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:17.535 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:17.536 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.536 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.536 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:179 3.0s0 3:02f544f0:::smithi18925896-30:head [read 0~491520,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:17.536 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:52.680315+0000", 2024-09-18T10:15:17.536 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 24.828936805000001, 2024-09-18T10:15:17.536 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20435319499999999, 2024-09-18T10:15:17.536 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:17.536 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:15:17.536 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:17.536 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:17.536 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:17.536 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 179 2024-09-18T10:15:17.536 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.537 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:17.537 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.537 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:17.537 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.680315+0000", 2024-09-18T10:15:17.537 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:17.537 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.537 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.537 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:17.537 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.680315+0000", 2024-09-18T10:15:17.537 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:17.537 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.537 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.538 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:17.538 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.680313+0000", 2024-09-18T10:15:17.538 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:17.538 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.538 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.538 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:17.538 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.680322+0000", 2024-09-18T10:15:17.538 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.9299999999999992e-06 2024-09-18T10:15:17.538 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.538 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.538 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:17.538 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.680343+0000", 2024-09-18T10:15:17.538 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0556000000000001e-05 2024-09-18T10:15:17.539 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.539 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.539 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:17.539 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.680373+0000", 2024-09-18T10:15:17.539 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0592000000000001e-05 2024-09-18T10:15:17.539 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.539 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.539 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:17.539 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.681360+0000", 2024-09-18T10:15:17.539 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00098610400000000002 2024-09-18T10:15:17.539 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.539 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.539 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:17.539 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.681624+0000", 2024-09-18T10:15:17.539 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000263992 2024-09-18T10:15:17.539 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.540 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.540 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:17.540 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.884668+0000", 2024-09-18T10:15:17.540 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.203044321 2024-09-18T10:15:17.540 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:17.540 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:17.540 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:17.540 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.540 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.540 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:180 3.0s0 3:02f544f0:::smithi18925896-30:head [read 0~491520,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:17.540 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:52.680394+0000", 2024-09-18T10:15:17.540 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 24.828857726999999, 2024-09-18T10:15:17.540 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20540486599999999, 2024-09-18T10:15:17.540 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:17.540 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:15:17.540 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:17.541 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:17.541 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:17.541 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 180 2024-09-18T10:15:17.541 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.541 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:17.541 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.541 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:17.545 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.680394+0000", 2024-09-18T10:15:17.545 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:17.545 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.546 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.546 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:17.546 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.680394+0000", 2024-09-18T10:15:17.546 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:17.546 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.546 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.546 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:17.546 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.680393+0000", 2024-09-18T10:15:17.546 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:17.546 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.546 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.546 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:17.546 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.680401+0000", 2024-09-18T10:15:17.546 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.4169999999999992e-06 2024-09-18T10:15:17.547 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.547 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.547 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:17.547 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.680422+0000", 2024-09-18T10:15:17.547 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1186e-05 2024-09-18T10:15:17.547 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.547 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.547 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:17.547 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.680433+0000", 2024-09-18T10:15:17.547 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1103e-05 2024-09-18T10:15:17.547 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.547 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.547 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:17.547 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.783888+0000", 2024-09-18T10:15:17.547 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.103454746 2024-09-18T10:15:17.548 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.548 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.548 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:17.548 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.784067+0000", 2024-09-18T10:15:17.548 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00017865399999999999 2024-09-18T10:15:17.548 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.548 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.548 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:17.548 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.885799+0000", 2024-09-18T10:15:17.548 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10173201699999999 2024-09-18T10:15:17.548 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:17.548 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:17.548 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:17.548 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.548 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.549 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpReadReply(3.0s0 36/32 ECSubReadReply(tid=9, attrs_read=0))", 2024-09-18T10:15:17.549 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:52.683258+0000", 2024-09-18T10:15:17.549 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 24.825993934, 2024-09-18T10:15:17.549 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.201447083, 2024-09-18T10:15:17.549 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:17.549 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:17.549 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:17.549 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.549 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:17.549 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.683258+0000", 2024-09-18T10:15:17.549 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:17.549 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.549 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.549 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:17.549 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.683258+0000", 2024-09-18T10:15:17.550 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:17.550 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.550 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.550 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:17.550 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.683256+0000", 2024-09-18T10:15:17.550 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:17.550 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.550 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.550 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:17.550 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.683482+0000", 2024-09-18T10:15:17.550 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00022592599999999999 2024-09-18T10:15:17.550 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.550 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.550 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:17.551 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.683508+0000", 2024-09-18T10:15:17.551 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5732999999999999e-05 2024-09-18T10:15:17.551 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.551 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.551 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:17.551 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.783587+0000", 2024-09-18T10:15:17.551 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100079109 2024-09-18T10:15:17.551 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.551 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.551 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:17.551 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.883944+0000", 2024-09-18T10:15:17.551 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100357268 2024-09-18T10:15:17.551 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.551 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.552 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:17.552 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.884705+0000", 2024-09-18T10:15:17.552 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00076026999999999996 2024-09-18T10:15:17.552 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:17.552 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:17.552 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:17.552 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.552 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.552 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:191 3.0s0 3:0a83d6b3:::smithi18925896-20:2 [read 0~327680,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:17.552 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:52.694194+0000", 2024-09-18T10:15:17.552 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 24.815057286999998, 2024-09-18T10:15:17.552 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.29753027399999998, 2024-09-18T10:15:17.552 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:17.552 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:15:17.553 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:17.553 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:17.553 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:17.553 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 191 2024-09-18T10:15:17.553 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.553 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:17.553 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.553 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:17.553 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.694194+0000", 2024-09-18T10:15:17.553 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:17.553 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.553 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.553 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:17.553 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.694194+0000", 2024-09-18T10:15:17.554 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:17.554 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.554 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.554 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:17.554 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.694192+0000", 2024-09-18T10:15:17.554 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:17.554 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.554 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.554 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:17.554 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.694204+0000", 2024-09-18T10:15:17.554 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2233e-05 2024-09-18T10:15:17.554 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.554 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.554 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:17.554 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.694280+0000", 2024-09-18T10:15:17.554 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.5584e-05 2024-09-18T10:15:17.555 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.555 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.555 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:17.555 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.694292+0000", 2024-09-18T10:15:17.555 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2478e-05 2024-09-18T10:15:17.555 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.555 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.555 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:17.555 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.885869+0000", 2024-09-18T10:15:17.555 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.191576304 2024-09-18T10:15:17.555 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.555 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.555 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:17.556 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.886008+0000", 2024-09-18T10:15:17.556 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00013880300000000001 2024-09-18T10:15:17.556 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.556 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.556 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:17.556 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.991725+0000", 2024-09-18T10:15:17.556 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10571691 2024-09-18T10:15:17.556 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:17.556 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:17.556 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:17.556 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.556 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.556 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:192 3.0s0 3:0a83d6b3:::smithi18925896-20:2 [read 0~327680,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:17.557 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:52.694306+0000", 2024-09-18T10:15:17.557 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 24.814945337000001, 2024-09-18T10:15:17.557 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.41051474799999998, 2024-09-18T10:15:17.557 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:17.557 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:15:17.557 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:17.557 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:17.557 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:17.557 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 192 2024-09-18T10:15:17.557 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.557 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:17.557 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.557 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:17.557 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.694306+0000", 2024-09-18T10:15:17.558 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:17.558 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.558 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.558 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:17.558 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.694306+0000", 2024-09-18T10:15:17.558 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:17.558 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.558 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.558 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:17.558 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.694305+0000", 2024-09-18T10:15:17.558 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:17.558 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.558 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.558 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:17.559 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.694312+0000", 2024-09-18T10:15:17.559 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.5129999999999999e-06 2024-09-18T10:15:17.559 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.559 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.559 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:17.559 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.694328+0000", 2024-09-18T10:15:17.559 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5766000000000001e-05 2024-09-18T10:15:17.559 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.559 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.559 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:17.559 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.694349+0000", 2024-09-18T10:15:17.559 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0990000000000001e-05 2024-09-18T10:15:17.559 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.559 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.560 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:17.560 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.887405+0000", 2024-09-18T10:15:17.560 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.193055791 2024-09-18T10:15:17.560 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.560 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.560 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:17.560 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.887535+0000", 2024-09-18T10:15:17.560 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00013043000000000001 2024-09-18T10:15:17.560 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.560 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.560 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:17.560 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:53.104821+0000", 2024-09-18T10:15:17.560 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.217285486 2024-09-18T10:15:17.561 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:17.561 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:17.561 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:17.561 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.561 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.561 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:193 3.0s0 3:0a83d6b3:::smithi18925896-20:2 [read 0~327680,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:17.561 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:52.694359+0000", 2024-09-18T10:15:17.561 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 24.814892155999999, 2024-09-18T10:15:17.561 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.411282918, 2024-09-18T10:15:17.561 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:17.561 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:15:17.561 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:17.561 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:17.562 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:17.562 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 193 2024-09-18T10:15:17.562 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.562 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:17.562 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.562 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:17.562 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.694359+0000", 2024-09-18T10:15:17.562 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:17.562 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.562 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.562 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:17.562 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.694359+0000", 2024-09-18T10:15:17.562 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:17.563 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.563 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.563 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:17.563 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.694358+0000", 2024-09-18T10:15:17.563 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:15:17.563 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.563 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.563 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:17.563 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.694364+0000", 2024-09-18T10:15:17.563 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8089999999999998e-06 2024-09-18T10:15:17.563 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.563 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.563 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:17.564 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.694381+0000", 2024-09-18T10:15:17.564 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6949e-05 2024-09-18T10:15:17.564 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.564 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.564 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:17.564 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.694388+0000", 2024-09-18T10:15:17.564 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.8079999999999999e-06 2024-09-18T10:15:17.564 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.564 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.564 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:17.564 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.988184+0000", 2024-09-18T10:15:17.564 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.29379642900000003 2024-09-18T10:15:17.564 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.564 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.565 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:17.565 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.988399+0000", 2024-09-18T10:15:17.565 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000214537 2024-09-18T10:15:17.565 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.565 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.565 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:17.565 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:53.105642+0000", 2024-09-18T10:15:17.565 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.11724332699999999 2024-09-18T10:15:17.565 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:17.565 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:17.565 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:17.565 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.566 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.566 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.10s0 45/40 ECSubWriteReply(tid=18, last_complete=45'6, committed=1, applied=1))", 2024-09-18T10:15:17.566 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:01.742819+0000", 2024-09-18T10:15:17.566 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 15.766432553, 2024-09-18T10:15:17.566 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.30317059800000001, 2024-09-18T10:15:17.566 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:17.566 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:17.566 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:17.566 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.566 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:17.566 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.742819+0000", 2024-09-18T10:15:17.566 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:17.566 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.567 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.567 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:17.567 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.742819+0000", 2024-09-18T10:15:17.567 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:17.567 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.567 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.567 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:17.567 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.742817+0000", 2024-09-18T10:15:17.567 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:17.567 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.567 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.567 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:17.567 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.742832+0000", 2024-09-18T10:15:17.568 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.466e-05 2024-09-18T10:15:17.568 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.568 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.568 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:17.568 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.742866+0000", 2024-09-18T10:15:17.568 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4242000000000002e-05 2024-09-18T10:15:17.568 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.568 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.568 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:17.568 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.843033+0000", 2024-09-18T10:15:17.568 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100166479 2024-09-18T10:15:17.568 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.569 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.569 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:17.569 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.045959+0000", 2024-09-18T10:15:17.569 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.202926 2024-09-18T10:15:17.569 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.569 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.569 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:17.569 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.045990+0000", 2024-09-18T10:15:17.569 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0889999999999997e-05 2024-09-18T10:15:17.569 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:17.569 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:17.569 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:17.569 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.569 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.570 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.10s0 45/40 ECSubWriteReply(tid=18, last_complete=45'6, committed=1, applied=1))", 2024-09-18T10:15:17.570 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:01.843113+0000", 2024-09-18T10:15:17.570 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 15.666138707, 2024-09-18T10:15:17.570 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.203080129, 2024-09-18T10:15:17.570 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:17.570 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:17.570 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:17.570 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.570 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:17.570 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.843113+0000", 2024-09-18T10:15:17.570 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:17.570 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.570 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.571 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:17.571 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.843113+0000", 2024-09-18T10:15:17.571 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:17.571 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.571 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.571 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:17.571 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.843111+0000", 2024-09-18T10:15:17.571 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:17.571 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.571 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.571 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:17.571 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.843122+0000", 2024-09-18T10:15:17.571 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0199e-05 2024-09-18T10:15:17.571 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.572 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.572 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:17.572 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.843147+0000", 2024-09-18T10:15:17.572 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5113000000000002e-05 2024-09-18T10:15:17.572 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.572 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.572 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:17.572 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.943289+0000", 2024-09-18T10:15:17.572 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100141813 2024-09-18T10:15:17.572 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.572 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.572 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:17.572 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.046146+0000", 2024-09-18T10:15:17.572 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.102857768 2024-09-18T10:15:17.572 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.573 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.573 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:17.573 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.046193+0000", 2024-09-18T10:15:17.573 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.6566000000000001e-05 2024-09-18T10:15:17.573 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:17.573 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:17.573 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:17.573 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.573 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.573 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDScrubReserve(3.8s3 REQUEST e46)", 2024-09-18T10:15:17.573 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:02.499249+0000", 2024-09-18T10:15:17.573 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 15.010002232, 2024-09-18T10:15:17.573 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20056184199999999, 2024-09-18T10:15:17.573 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:17.573 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:15:17.574 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:17.574 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.574 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:17.574 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.499249+0000", 2024-09-18T10:15:17.574 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:17.574 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.574 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.574 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:17.574 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.499249+0000", 2024-09-18T10:15:17.574 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:17.574 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.574 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.574 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:17.574 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.499247+0000", 2024-09-18T10:15:17.575 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:17.575 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.575 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.575 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:17.575 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.499327+0000", 2024-09-18T10:15:17.575 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.9883999999999996e-05 2024-09-18T10:15:17.575 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.575 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.575 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:17.575 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.499369+0000", 2024-09-18T10:15:17.575 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1572000000000001e-05 2024-09-18T10:15:17.575 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.575 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.575 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:17.576 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.499386+0000", 2024-09-18T10:15:17.576 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7370000000000001e-05 2024-09-18T10:15:17.576 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.576 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.576 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:17.576 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.699717+0000", 2024-09-18T10:15:17.576 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20033121600000001 2024-09-18T10:15:17.576 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.576 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.576 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:17.576 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.699751+0000", 2024-09-18T10:15:17.576 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3828999999999997e-05 2024-09-18T10:15:17.577 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.577 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.577 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:17.577 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.699811+0000", 2024-09-18T10:15:17.577 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.0090999999999998e-05 2024-09-18T10:15:17.577 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:17.577 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:17.577 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:17.577 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.577 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.577 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.19s6 51/40 ECSubWrite(tid=78, reqid=client.4632.0:408, at_version=51'46, trim_to=0'0, roll_forward_to=50'39))", 2024-09-18T10:15:17.577 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:06.779231+0000", 2024-09-18T10:15:17.577 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 10.730020817, 2024-09-18T10:15:17.578 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.201476816, 2024-09-18T10:15:17.578 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:17.578 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:17.578 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:17.578 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.578 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:17.578 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:06.779231+0000", 2024-09-18T10:15:17.578 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:17.578 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.578 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.578 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:17.578 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:06.779231+0000", 2024-09-18T10:15:17.578 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:17.579 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.579 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.579 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:17.579 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:06.779229+0000", 2024-09-18T10:15:17.579 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:17.579 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.579 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.579 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:17.579 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:06.779264+0000", 2024-09-18T10:15:17.579 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4699000000000001e-05 2024-09-18T10:15:17.579 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.579 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.579 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:17.579 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:06.779288+0000", 2024-09-18T10:15:17.580 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3821000000000001e-05 2024-09-18T10:15:17.580 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.580 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.580 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:17.580 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:06.879428+0000", 2024-09-18T10:15:17.580 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10014011 2024-09-18T10:15:17.580 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.580 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.580 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:17.580 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:06.979654+0000", 2024-09-18T10:15:17.580 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100225544 2024-09-18T10:15:17.580 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.580 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.580 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:17.581 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:06.979671+0000", 2024-09-18T10:15:17.581 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7244999999999998e-05 2024-09-18T10:15:17.581 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.581 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.581 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:17.581 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:06.980659+0000", 2024-09-18T10:15:17.581 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00098774499999999994 2024-09-18T10:15:17.581 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.581 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.581 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:17.581 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:06.980708+0000", 2024-09-18T10:15:17.581 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.8897999999999998e-05 2024-09-18T10:15:17.581 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:17.581 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:17.582 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:17.582 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.582 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.582 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.19s6 51/40 ECSubWrite(tid=80, reqid=client.4632.0:408, at_version=0'0, trim_to=0'0, roll_forward_to=51'46))", 2024-09-18T10:15:17.582 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:07.549703+0000", 2024-09-18T10:15:17.582 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 9.9595481439999993, 2024-09-18T10:15:17.582 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20120351, 2024-09-18T10:15:17.582 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:17.582 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:17.582 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:17.582 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.582 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:17.582 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:07.549703+0000", 2024-09-18T10:15:17.582 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:17.583 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.583 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.583 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:17.583 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:07.549703+0000", 2024-09-18T10:15:17.583 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:17.583 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.583 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.583 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:17.583 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:07.549702+0000", 2024-09-18T10:15:17.583 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:17.583 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.583 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.583 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:17.583 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:07.549735+0000", 2024-09-18T10:15:17.583 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3238999999999998e-05 2024-09-18T10:15:17.584 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.584 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.584 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:17.584 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:07.549765+0000", 2024-09-18T10:15:17.584 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.9955999999999999e-05 2024-09-18T10:15:17.584 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.584 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.584 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:17.584 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:07.549782+0000", 2024-09-18T10:15:17.584 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6503000000000001e-05 2024-09-18T10:15:17.584 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.584 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.584 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:17.584 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:07.549945+0000", 2024-09-18T10:15:17.585 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000163392 2024-09-18T10:15:17.585 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.585 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.585 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:17.585 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:07.549962+0000", 2024-09-18T10:15:17.585 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.645e-05 2024-09-18T10:15:17.585 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.585 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.585 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:17.585 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:07.750853+0000", 2024-09-18T10:15:17.585 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20089121500000001 2024-09-18T10:15:17.585 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.585 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.586 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:17.586 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:07.750907+0000", 2024-09-18T10:15:17.586 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.4112e-05 2024-09-18T10:15:17.586 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:17.586 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:17.586 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:17.586 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.586 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.586 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.1s5 56/40 ECSubWrite(tid=102, reqid=client.4632.0:437, at_version=0'0, trim_to=0'0, roll_forward_to=56'16))", 2024-09-18T10:15:17.586 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:12.541097+0000", 2024-09-18T10:15:17.586 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 4.9681543450000003, 2024-09-18T10:15:17.586 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20186336399999999, 2024-09-18T10:15:17.586 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:17.586 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:17.587 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:17.587 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.587 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:17.587 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:12.541097+0000", 2024-09-18T10:15:17.587 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:17.587 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.587 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.587 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:17.587 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:12.541097+0000", 2024-09-18T10:15:17.587 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:17.587 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.587 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.587 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:17.588 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:12.541095+0000", 2024-09-18T10:15:17.588 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:17.588 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.588 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.588 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:17.588 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:12.541147+0000", 2024-09-18T10:15:17.588 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.2203000000000001e-05 2024-09-18T10:15:17.588 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.588 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.588 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:17.588 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:12.541184+0000", 2024-09-18T10:15:17.588 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6495000000000002e-05 2024-09-18T10:15:17.588 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.588 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.589 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:17.589 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:12.641359+0000", 2024-09-18T10:15:17.589 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100175135 2024-09-18T10:15:17.589 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.589 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.589 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:17.589 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:12.741675+0000", 2024-09-18T10:15:17.589 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100316457 2024-09-18T10:15:17.589 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.589 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.589 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:17.589 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:12.741693+0000", 2024-09-18T10:15:17.589 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7269e-05 2024-09-18T10:15:17.590 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.590 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.590 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:17.590 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:12.742894+0000", 2024-09-18T10:15:17.590 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0012013639999999999 2024-09-18T10:15:17.590 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.590 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.590 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:17.590 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:12.742961+0000", 2024-09-18T10:15:17.590 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.6575000000000005e-05 2024-09-18T10:15:17.590 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:17.590 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:17.590 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:17.590 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.591 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.591 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.1s5 57/40 ECSubWrite(tid=111, reqid=osd.4.0:111, at_version=57'18, trim_to=0'0, roll_forward_to=56'16))", 2024-09-18T10:15:17.591 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:13.054749+0000", 2024-09-18T10:15:17.591 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 4.4545022840000001, 2024-09-18T10:15:17.591 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.201940535, 2024-09-18T10:15:17.591 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:17.591 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:17.591 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:17.591 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.591 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:17.591 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.054749+0000", 2024-09-18T10:15:17.591 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:17.591 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.591 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.591 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:17.592 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.054749+0000", 2024-09-18T10:15:17.592 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:17.592 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.592 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.592 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:17.592 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.054749+0000", 2024-09-18T10:15:17.592 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:17.592 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.592 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.592 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:17.592 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.054785+0000", 2024-09-18T10:15:17.592 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6480999999999999e-05 2024-09-18T10:15:17.592 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.593 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.593 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:17.593 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.054808+0000", 2024-09-18T10:15:17.593 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.2351e-05 2024-09-18T10:15:17.593 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.593 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.593 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:17.593 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.154914+0000", 2024-09-18T10:15:17.593 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100106862 2024-09-18T10:15:17.593 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.593 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.593 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:17.593 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.155065+0000", 2024-09-18T10:15:17.593 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000150762 2024-09-18T10:15:17.594 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.594 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.594 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:17.594 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.155083+0000", 2024-09-18T10:15:17.594 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.8207e-05 2024-09-18T10:15:17.594 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.594 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.594 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:17.594 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.256244+0000", 2024-09-18T10:15:17.594 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10116046200000001 2024-09-18T10:15:17.594 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.594 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.594 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:17.594 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.256690+0000", 2024-09-18T10:15:17.595 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00044597000000000001 2024-09-18T10:15:17.595 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:17.595 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:17.595 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:17.595 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.595 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.595 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 58/40 ECSubWriteReply(tid=21, last_complete=57'8, committed=1, applied=1))", 2024-09-18T10:15:17.595 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:13.649729+0000", 2024-09-18T10:15:17.595 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 3.8595226770000002, 2024-09-18T10:15:17.595 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20172436599999999, 2024-09-18T10:15:17.595 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:17.595 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:17.595 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:17.595 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.596 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:17.596 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.649729+0000", 2024-09-18T10:15:17.596 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:17.596 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.596 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.596 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:17.596 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.649729+0000", 2024-09-18T10:15:17.596 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:17.596 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.596 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.596 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:17.596 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.649728+0000", 2024-09-18T10:15:17.596 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:17.596 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.596 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.597 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:17.597 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.649736+0000", 2024-09-18T10:15:17.597 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.0530000000000007e-06 2024-09-18T10:15:17.597 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.597 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.597 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:17.597 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.649748+0000", 2024-09-18T10:15:17.597 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1173e-05 2024-09-18T10:15:17.597 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.597 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.597 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:17.597 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.749876+0000", 2024-09-18T10:15:17.597 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10012805 2024-09-18T10:15:17.597 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.598 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.598 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:17.598 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.850098+0000", 2024-09-18T10:15:17.598 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100222371 2024-09-18T10:15:17.598 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.598 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.598 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:17.598 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.851453+0000", 2024-09-18T10:15:17.598 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0013552219999999999 2024-09-18T10:15:17.598 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:17.598 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:17.598 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:17.598 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:17.599 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:17.599 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-18T10:15:17.601 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T10:15:17.601 INFO:tasks.rados.rados.0.smithi189.stdout:195: finishing write tid 3 to smithi18925896-43 2024-09-18T10:15:17.602 INFO:tasks.rados.rados.0.smithi189.stdout:195: finishing write tid 4 to smithi18925896-43 2024-09-18T10:15:17.602 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-18T10:15:17.594+0000 7f14dcc5b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22398) UID: 0 2024-09-18T10:15:17.659 INFO:tasks.rados.rados.0.smithi189.stdout:195: finishing write tid 5 to smithi18925896-43 2024-09-18T10:15:17.659 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 43 v 35 (ObjNum 89 snap 15 seq_num 89) dirty exists 2024-09-18T10:15:17.659 INFO:tasks.rados.rados.0.smithi189.stdout:195: left oid 43 (ObjNum 89 snap 15 seq_num 89) 2024-09-18T10:15:17.701 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-18T10:15:17.701+0000 7f14c6a1a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 22138) UID: 0 2024-09-18T10:15:17.708 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-18T10:15:17.708 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [ 2024-09-18T10:15:17.708 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.708 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.19s0 61/40 ECSubWriteReply(tid=110, last_complete=61'60, committed=1, applied=1))", 2024-09-18T10:15:17.708 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:17.554461+0000", 2024-09-18T10:15:17.709 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.15334928, 2024-09-18T10:15:17.709 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.15338700699999999, 2024-09-18T10:15:17.709 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:17.709 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "queued for pg", 2024-09-18T10:15:17.709 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:17.709 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.709 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:17.709 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.554461+0000", 2024-09-18T10:15:17.709 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:17.709 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.709 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.709 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:17.709 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.554461+0000", 2024-09-18T10:15:17.710 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:17.710 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.710 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.710 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:17.710 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.554458+0000", 2024-09-18T10:15:17.710 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999976 2024-09-18T10:15:17.710 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.710 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.710 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:17.710 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.554472+0000", 2024-09-18T10:15:17.710 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4032999999999999e-05 2024-09-18T10:15:17.710 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.710 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.710 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:17.711 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.554507+0000", 2024-09-18T10:15:17.711 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4869000000000001e-05 2024-09-18T10:15:17.711 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.711 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.711 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:17.711 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.554521+0000", 2024-09-18T10:15:17.711 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3613e-05 2024-09-18T10:15:17.711 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:17.711 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:17.711 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:17.711 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.711 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.711 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDScrubReserve(3.1s4 REQUEST e61)", 2024-09-18T10:15:17.711 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:17.657331+0000", 2024-09-18T10:15:17.712 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.050478561999999998, 2024-09-18T10:15:17.712 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.050590830000000003, 2024-09-18T10:15:17.712 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:17.712 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "no flag points reached", 2024-09-18T10:15:17.712 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:17.712 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.712 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:17.712 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.657331+0000", 2024-09-18T10:15:17.712 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:17.712 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.712 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.712 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:17.712 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.657331+0000", 2024-09-18T10:15:17.713 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:17.713 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.713 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.713 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:17.713 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.657331+0000", 2024-09-18T10:15:17.713 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:17.713 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.713 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.713 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:17.713 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.657336+0000", 2024-09-18T10:15:17.713 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.2000000000000002e-06 2024-09-18T10:15:17.713 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.713 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.713 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:17.714 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.657367+0000", 2024-09-18T10:15:17.714 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0354000000000001e-05 2024-09-18T10:15:17.714 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:17.714 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:17.714 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:17.714 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:17.714 INFO:teuthology.orchestra.run.smithi189.stdout: ], 2024-09-18T10:15:17.714 INFO:teuthology.orchestra.run.smithi189.stdout: "num_ops": 2 2024-09-18T10:15:17.714 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-18T10:15:17.717 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T10:15:17.802 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-18T10:15:17.798+0000 7f302f2c3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22187) UID: 0 2024-09-18T10:15:17.814 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-18T10:15:17.814 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-18T10:15:17.815 INFO:teuthology.orchestra.run.smithi189.stdout: "complaint_time": 30, 2024-09-18T10:15:17.815 INFO:teuthology.orchestra.run.smithi189.stdout: "num_blocked_ops": 0 2024-09-18T10:15:17.815 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-18T10:15:17.824 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T10:15:17.890 INFO:tasks.rados.rados.0.smithi189.stdout:193: done (5 left) 2024-09-18T10:15:17.890 INFO:tasks.rados.rados.0.smithi189.stdout:194: done (4 left) 2024-09-18T10:15:17.890 INFO:tasks.rados.rados.0.smithi189.stdout:195: done (3 left) 2024-09-18T10:15:17.890 INFO:tasks.rados.rados.0.smithi189.stdout:196: done (2 left) 2024-09-18T10:15:17.890 INFO:tasks.rados.rados.0.smithi189.stdout:197: done (1 left) 2024-09-18T10:15:17.890 INFO:tasks.rados.rados.0.smithi189.stdout:198: done (0 left) 2024-09-18T10:15:17.890 INFO:tasks.rados.rados.0.smithi189.stdout:199: rmattr oid 47 current snap is 16 2024-09-18T10:15:17.890 INFO:tasks.rados.rados.0.smithi189.stdout:199: done (0 left) 2024-09-18T10:15:17.890 INFO:tasks.rados.rados.0.smithi189.stdout:200: rmattr oid 39 current snap is 16 2024-09-18T10:15:17.891 INFO:tasks.rados.rados.0.smithi189.stdout:201: read oid 5 snap -1 2024-09-18T10:15:17.891 INFO:tasks.rados.rados.0.smithi189.stdout:201: expect (ObjNum 4 snap 0 seq_num 4) 2024-09-18T10:15:17.891 INFO:tasks.rados.rados.0.smithi189.stdout:202: read oid 48 snap -1 2024-09-18T10:15:17.891 INFO:tasks.rados.rados.0.smithi189.stdout:202: expect (ObjNum 41 snap 0 seq_num 41) 2024-09-18T10:15:17.891 INFO:tasks.rados.rados.0.smithi189.stdout:203: setattr oid 38 current snap is 16 2024-09-18T10:15:17.893 INFO:teuthology.orchestra.run.smithi029.stderr:reweighted osd.8 to 1 (10000) 2024-09-18T10:15:17.895 INFO:tasks.rados.rados.0.smithi189.stdout:append oid 37 current snap is 16 2024-09-18T10:15:17.895 INFO:tasks.rados.rados.0.smithi189.stdout:204: seq_num 90 ranges {606208=753664,1359872=720896,2080768=49152} 2024-09-18T10:15:17.903 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-18T10:15:17.901+0000 7f14c6a1a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 22138) UID: 0 2024-09-18T10:15:17.908 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 9 1 2024-09-18T10:15:17.909 INFO:tasks.rados.rados.0.smithi189.stdout:204: writing smithi18925896-37 from 606208 to 1359872 tid 1 2024-09-18T10:15:17.912 INFO:tasks.rados.rados.0.smithi189.stdout:204: writing smithi18925896-37 from 1359872 to 2080768 tid 2 2024-09-18T10:15:17.913 INFO:tasks.rados.rados.0.smithi189.stdout:204: writing smithi18925896-37 from 2080768 to 2129920 tid 3 2024-09-18T10:15:17.913 INFO:tasks.rados.rados.0.smithi189.stdout:205: copy_from oid 41 from oid 31 current snap is 16 2024-09-18T10:15:17.913 INFO:tasks.rados.rados.0.smithi189.stdout:206: snap_create 2024-09-18T10:15:17.927 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-18T10:15:17.927 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 20, 2024-09-18T10:15:17.927 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 600, 2024-09-18T10:15:17.927 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [ 2024-09-18T10:15:17.927 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.928 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:114 3.9s0 3:9a1a774c:::smithi18925896-38:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:17.928 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:50.816650+0000", 2024-09-18T10:15:17.928 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 27.108439041, 2024-09-18T10:15:17.929 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.41853457900000002, 2024-09-18T10:15:17.929 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:17.929 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:15:17.929 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:17.929 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:17.930 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:17.930 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 114 2024-09-18T10:15:17.930 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.930 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:17.930 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.930 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:17.930 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.816650+0000", 2024-09-18T10:15:17.930 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:17.930 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.930 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.930 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:17.930 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.816650+0000", 2024-09-18T10:15:17.931 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:17.931 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.931 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.931 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:17.931 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.816648+0000", 2024-09-18T10:15:17.931 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:17.931 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.931 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.931 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:17.931 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.816661+0000", 2024-09-18T10:15:17.931 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2619000000000001e-05 2024-09-18T10:15:17.931 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.931 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.932 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:17.932 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.816696+0000", 2024-09-18T10:15:17.932 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4885999999999998e-05 2024-09-18T10:15:17.932 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.932 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.932 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:17.932 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.816708+0000", 2024-09-18T10:15:17.932 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2843000000000001e-05 2024-09-18T10:15:17.932 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.932 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.932 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:17.932 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.818228+0000", 2024-09-18T10:15:17.932 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001519882 2024-09-18T10:15:17.933 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.933 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.933 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for rw locks", 2024-09-18T10:15:17.933 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.818557+0000", 2024-09-18T10:15:17.933 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00032873700000000001 2024-09-18T10:15:17.933 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.933 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.933 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:17.933 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.925488+0000", 2024-09-18T10:15:17.933 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.106930502 2024-09-18T10:15:17.933 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.933 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.933 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:17.933 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.925563+0000", 2024-09-18T10:15:17.934 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.5386e-05 2024-09-18T10:15:17.934 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.934 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.934 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:17.934 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.235185+0000", 2024-09-18T10:15:17.934 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.30962182399999999 2024-09-18T10:15:17.934 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:17.934 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:17.934 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:17.934 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.934 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.934 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:138 3.9s0 3:9ab7c4c0:::smithi18925896-46:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:17.934 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:50.921054+0000", 2024-09-18T10:15:17.934 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 27.004034742999998, 2024-09-18T10:15:17.935 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.61288337900000001, 2024-09-18T10:15:17.935 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:17.935 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:15:17.935 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:17.935 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:17.935 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:17.935 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 138 2024-09-18T10:15:17.935 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.935 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:17.935 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.935 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:17.935 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.921054+0000", 2024-09-18T10:15:17.935 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:17.935 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.936 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.936 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:17.936 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.921054+0000", 2024-09-18T10:15:17.936 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:17.936 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.936 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.936 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:17.936 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.921054+0000", 2024-09-18T10:15:17.936 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:17.936 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.936 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.936 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:17.937 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.921057+0000", 2024-09-18T10:15:17.937 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1829999999999998e-06 2024-09-18T10:15:17.937 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.937 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.937 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:17.937 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.921090+0000", 2024-09-18T10:15:17.937 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3281999999999997e-05 2024-09-18T10:15:17.937 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.937 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.937 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:17.937 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.921096+0000", 2024-09-18T10:15:17.937 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.0519999999999998e-06 2024-09-18T10:15:17.937 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.938 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.938 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:17.938 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.127704+0000", 2024-09-18T10:15:17.938 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20660720499999999 2024-09-18T10:15:17.938 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.938 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.938 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for rw locks", 2024-09-18T10:15:17.938 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.127851+0000", 2024-09-18T10:15:17.938 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00014710999999999999 2024-09-18T10:15:17.938 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.938 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.938 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:17.938 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.234300+0000", 2024-09-18T10:15:17.938 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10644893900000001 2024-09-18T10:15:17.939 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.939 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.939 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:17.939 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.234462+0000", 2024-09-18T10:15:17.939 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00016218999999999999 2024-09-18T10:15:17.939 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.939 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.939 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:17.939 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.533938+0000", 2024-09-18T10:15:17.939 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.29947596999999998 2024-09-18T10:15:17.939 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:17.939 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:17.939 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:17.939 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.940 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.940 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:142 3.9s0 3:9221fff5:::smithi18925896-48:head [append 0~425984,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:17.940 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:50.929918+0000", 2024-09-18T10:15:17.940 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 26.995171430999999, 2024-09-18T10:15:17.940 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.39521715200000002, 2024-09-18T10:15:17.940 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:17.940 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:17.940 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:17.940 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:17.940 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:17.940 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 142 2024-09-18T10:15:17.940 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.940 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:17.940 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.941 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:17.941 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.929918+0000", 2024-09-18T10:15:17.941 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:17.941 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.941 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.941 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:17.941 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.929918+0000", 2024-09-18T10:15:17.941 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:17.941 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.941 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.941 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:17.941 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.929916+0000", 2024-09-18T10:15:17.941 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:17.941 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.942 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.942 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:17.942 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.930096+0000", 2024-09-18T10:15:17.942 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00018038700000000001 2024-09-18T10:15:17.942 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.942 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.942 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:17.942 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.930122+0000", 2024-09-18T10:15:17.942 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5927000000000001e-05 2024-09-18T10:15:17.942 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.942 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.942 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:17.942 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.930132+0000", 2024-09-18T10:15:17.942 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.6539999999999994e-06 2024-09-18T10:15:17.943 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.943 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.943 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:17.943 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.127900+0000", 2024-09-18T10:15:17.943 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.19776816899999999 2024-09-18T10:15:17.943 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.944 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.944 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:17.944 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.128112+0000", 2024-09-18T10:15:17.944 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00021222999999999999 2024-09-18T10:15:17.944 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.944 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.944 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:17.944 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.129335+0000", 2024-09-18T10:15:17.944 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0012222159999999999 2024-09-18T10:15:17.944 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.944 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.944 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:17.944 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.132031+0000", 2024-09-18T10:15:17.945 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0026961300000000001 2024-09-18T10:15:17.945 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.945 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.945 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:17.945 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.325013+0000", 2024-09-18T10:15:17.945 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.19298240999999999 2024-09-18T10:15:17.945 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.945 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.945 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:17.945 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.325135+0000", 2024-09-18T10:15:17.945 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000121842 2024-09-18T10:15:17.945 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:17.946 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:17.946 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:17.946 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.946 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.946 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:143 3.9s0 3:9221fff5:::smithi18925896-48:head [append 0~376832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:17.946 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:50.932038+0000", 2024-09-18T10:15:17.946 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 26.993051141999999, 2024-09-18T10:15:17.946 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.60109878000000005, 2024-09-18T10:15:17.946 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:17.946 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:17.946 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:17.946 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:17.946 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:17.947 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 143 2024-09-18T10:15:17.947 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.947 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:17.947 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.947 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:17.947 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932038+0000", 2024-09-18T10:15:17.947 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:17.947 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.947 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.947 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:17.947 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932038+0000", 2024-09-18T10:15:17.947 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:17.947 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.947 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.948 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:17.948 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932037+0000", 2024-09-18T10:15:17.948 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:15:17.948 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.948 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.948 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:17.948 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932309+0000", 2024-09-18T10:15:17.948 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000272086 2024-09-18T10:15:17.948 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.948 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.948 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:17.948 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932323+0000", 2024-09-18T10:15:17.948 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4354e-05 2024-09-18T10:15:17.949 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.949 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.949 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:17.949 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932329+0000", 2024-09-18T10:15:17.949 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.6840000000000001e-06 2024-09-18T10:15:17.949 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.949 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.949 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:17.949 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.130012+0000", 2024-09-18T10:15:17.949 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.19768277100000001 2024-09-18T10:15:17.949 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.949 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.949 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:17.950 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.130068+0000", 2024-09-18T10:15:17.950 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.6158e-05 2024-09-18T10:15:17.950 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.950 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.950 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:17.950 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.130809+0000", 2024-09-18T10:15:17.950 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00074078700000000002 2024-09-18T10:15:17.950 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.950 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.950 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:17.950 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.132840+0000", 2024-09-18T10:15:17.952 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0020308879999999998 2024-09-18T10:15:17.952 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.952 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.952 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:17.953 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.533025+0000", 2024-09-18T10:15:17.953 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.400185491 2024-09-18T10:15:17.953 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.953 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.953 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:17.953 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.533137+0000", 2024-09-18T10:15:17.953 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000111727 2024-09-18T10:15:17.953 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:17.953 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:17.953 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:17.953 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.954 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.954 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:144 3.9s0 3:9221fff5:::smithi18925896-48:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:17.954 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:50.932336+0000", 2024-09-18T10:15:17.954 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 26.992753506, 2024-09-18T10:15:17.954 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.60198766699999995, 2024-09-18T10:15:17.954 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:17.954 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:17.954 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:17.954 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:17.954 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:17.954 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 144 2024-09-18T10:15:17.955 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.955 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:17.955 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.955 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:17.955 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932336+0000", 2024-09-18T10:15:17.955 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:17.955 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.955 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.955 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:17.955 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932336+0000", 2024-09-18T10:15:17.956 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:17.956 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.956 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.956 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:17.956 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932335+0000", 2024-09-18T10:15:17.956 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:15:17.956 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.956 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.956 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:17.956 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932339+0000", 2024-09-18T10:15:17.956 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8539999999999999e-06 2024-09-18T10:15:17.956 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.957 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.957 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:17.957 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932347+0000", 2024-09-18T10:15:17.957 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.2910000000000007e-06 2024-09-18T10:15:17.957 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.957 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.957 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:17.957 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932351+0000", 2024-09-18T10:15:17.957 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.5420000000000001e-06 2024-09-18T10:15:17.957 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.957 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.957 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:17.958 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.131380+0000", 2024-09-18T10:15:17.958 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.199029236 2024-09-18T10:15:17.958 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.958 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.958 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:17.958 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.131436+0000", 2024-09-18T10:15:17.958 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.6098999999999999e-05 2024-09-18T10:15:17.958 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.958 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.958 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:17.958 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.131777+0000", 2024-09-18T10:15:17.958 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00034058800000000001 2024-09-18T10:15:17.958 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.959 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.959 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:17.959 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.133245+0000", 2024-09-18T10:15:17.959 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0014682300000000001 2024-09-18T10:15:17.959 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.959 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.959 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:17.959 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.534187+0000", 2024-09-18T10:15:17.959 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40094253299999999 2024-09-18T10:15:17.959 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.959 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.959 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:17.959 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.534323+0000", 2024-09-18T10:15:17.960 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00013606299999999999 2024-09-18T10:15:17.960 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:17.960 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:17.960 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:17.960 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.960 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.960 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:145 3.9s0 3:9221fff5:::smithi18925896-48:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:17.960 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:50.932355+0000", 2024-09-18T10:15:17.960 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 26.992734022, 2024-09-18T10:15:17.960 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.80502307200000001, 2024-09-18T10:15:17.960 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:17.961 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:15:17.961 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:17.961 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:17.961 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:17.961 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 145 2024-09-18T10:15:17.961 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.961 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:17.961 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.961 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:17.961 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932355+0000", 2024-09-18T10:15:17.961 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:17.961 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.962 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.962 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:17.962 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932355+0000", 2024-09-18T10:15:17.962 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:17.962 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.962 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.962 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:17.962 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932355+0000", 2024-09-18T10:15:17.962 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:17.962 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.962 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.962 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:17.963 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932358+0000", 2024-09-18T10:15:17.963 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4369999999999999e-06 2024-09-18T10:15:17.963 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.963 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.963 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:17.963 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932366+0000", 2024-09-18T10:15:17.963 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.3010000000000007e-06 2024-09-18T10:15:17.963 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.963 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.963 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:17.963 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932370+0000", 2024-09-18T10:15:17.963 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.5619999999999999e-06 2024-09-18T10:15:17.964 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.964 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.964 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:17.964 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.132210+0000", 2024-09-18T10:15:17.964 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.199840145 2024-09-18T10:15:17.964 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.964 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.964 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for rw locks", 2024-09-18T10:15:17.964 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.132312+0000", 2024-09-18T10:15:17.965 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000102335 2024-09-18T10:15:17.965 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.965 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.965 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:17.965 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.535310+0000", 2024-09-18T10:15:17.965 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40299779000000002 2024-09-18T10:15:17.965 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.965 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.965 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:17.965 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.535541+0000", 2024-09-18T10:15:17.965 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00023038299999999999 2024-09-18T10:15:17.966 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.966 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.966 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:17.966 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.737378+0000", 2024-09-18T10:15:17.966 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.201837604 2024-09-18T10:15:17.966 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:17.966 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:17.966 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:17.966 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.966 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.966 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:174 3.9s0 3:9fc2eacb:::smithi18925896-17:head [read 0~278528,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:17.967 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:51.836672+0000", 2024-09-18T10:15:17.967 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 26.088416766999998, 2024-09-18T10:15:17.967 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.407996215, 2024-09-18T10:15:17.967 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:17.967 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:15:17.967 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:17.967 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:17.967 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:17.967 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 174 2024-09-18T10:15:17.967 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.968 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:17.968 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.968 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:17.968 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.836672+0000", 2024-09-18T10:15:17.968 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:17.968 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.968 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.968 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:17.968 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.836672+0000", 2024-09-18T10:15:17.968 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:17.968 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.969 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.969 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:17.969 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.836671+0000", 2024-09-18T10:15:17.969 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:17.969 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.969 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.969 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:17.969 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.836680+0000", 2024-09-18T10:15:17.969 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.5830000000000007e-06 2024-09-18T10:15:17.970 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.970 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.970 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:17.970 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.836696+0000", 2024-09-18T10:15:17.970 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6823000000000001e-05 2024-09-18T10:15:17.970 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.970 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.970 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:17.970 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.836710+0000", 2024-09-18T10:15:17.970 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.325e-05 2024-09-18T10:15:17.970 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.971 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.971 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:17.971 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.142312+0000", 2024-09-18T10:15:17.971 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.30560266400000002 2024-09-18T10:15:17.971 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.971 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.971 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:17.971 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.142549+0000", 2024-09-18T10:15:17.971 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00023630299999999999 2024-09-18T10:15:17.971 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.972 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.972 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:17.972 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.244669+0000", 2024-09-18T10:15:17.972 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10211993599999999 2024-09-18T10:15:17.972 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:17.972 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:17.972 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:17.972 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.972 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.972 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:186 3.9s0 3:93224e57:::smithi18925896-10:head [rollback 2] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:17.972 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:52.693370+0000", 2024-09-18T10:15:17.973 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 25.231718755999999, 2024-09-18T10:15:17.973 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.39644133999999998, 2024-09-18T10:15:17.973 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:17.973 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:17.973 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:17.973 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:17.973 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:17.973 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 186 2024-09-18T10:15:17.973 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.973 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:17.973 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.974 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:17.974 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.693370+0000", 2024-09-18T10:15:17.974 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:17.974 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.974 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.974 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:17.974 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.693370+0000", 2024-09-18T10:15:17.974 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:17.974 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.974 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.974 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:17.974 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.693370+0000", 2024-09-18T10:15:17.974 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:17.975 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.975 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.975 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:17.975 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.693373+0000", 2024-09-18T10:15:17.975 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.712e-06 2024-09-18T10:15:17.975 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.975 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.975 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:17.975 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.693379+0000", 2024-09-18T10:15:17.975 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9320000000000001e-06 2024-09-18T10:15:17.975 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.975 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.976 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:17.976 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.793509+0000", 2024-09-18T10:15:17.976 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100130331 2024-09-18T10:15:17.976 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.976 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.976 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:17.976 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.800561+0000", 2024-09-18T10:15:17.976 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0070525559999999998 2024-09-18T10:15:17.976 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.976 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.976 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:17.976 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.800733+0000", 2024-09-18T10:15:17.977 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00017114100000000001 2024-09-18T10:15:17.977 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.977 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.977 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:17.977 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.802155+0000", 2024-09-18T10:15:17.977 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0014220420000000001 2024-09-18T10:15:17.977 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.977 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.977 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:17.977 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.905163+0000", 2024-09-18T10:15:17.977 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.103008407 2024-09-18T10:15:17.977 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.978 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.978 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:17.978 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:53.089742+0000", 2024-09-18T10:15:17.978 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.184578568 2024-09-18T10:15:17.978 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.978 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.978 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:17.978 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:53.089812+0000", 2024-09-18T10:15:17.978 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.0203000000000005e-05 2024-09-18T10:15:17.978 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:17.978 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:17.979 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:17.979 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.979 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.979 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:202 3.9s0 3:9221fff5:::smithi18925896-48:head [copy-from ver 2] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:17.979 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:52.793605+0000", 2024-09-18T10:15:17.979 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 25.131484172, 2024-09-18T10:15:17.979 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.61381690899999997, 2024-09-18T10:15:17.979 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:17.979 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:17.979 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:17.979 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:17.980 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:17.980 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 202 2024-09-18T10:15:17.980 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.980 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:17.980 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.980 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:17.980 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.793605+0000", 2024-09-18T10:15:17.980 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:17.980 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.980 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.980 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:17.981 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.793605+0000", 2024-09-18T10:15:17.981 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:17.981 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.981 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.981 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:17.981 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.793604+0000", 2024-09-18T10:15:17.981 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:15:17.981 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.981 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.981 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:17.981 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.793613+0000", 2024-09-18T10:15:17.982 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.4360000000000008e-06 2024-09-18T10:15:17.982 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.982 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.982 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:17.982 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.793628+0000", 2024-09-18T10:15:17.982 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4953000000000001e-05 2024-09-18T10:15:17.982 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.982 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.982 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:17.982 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.793634+0000", 2024-09-18T10:15:17.983 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.0170000000000002e-06 2024-09-18T10:15:17.983 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.983 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.983 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:17.983 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.905359+0000", 2024-09-18T10:15:17.983 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.111724328 2024-09-18T10:15:17.983 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.983 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.983 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:17.983 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.905540+0000", 2024-09-18T10:15:17.983 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000181552 2024-09-18T10:15:17.983 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.984 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.984 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:17.984 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:53.203724+0000", 2024-09-18T10:15:17.984 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.29818339500000002 2024-09-18T10:15:17.984 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.984 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.984 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:17.984 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:53.206308+0000", 2024-09-18T10:15:17.984 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0025841620000000001 2024-09-18T10:15:17.984 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.984 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.984 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:17.985 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:53.407237+0000", 2024-09-18T10:15:17.985 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.200928739 2024-09-18T10:15:17.985 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.985 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.985 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:17.985 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:53.407422+0000", 2024-09-18T10:15:17.985 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000185384 2024-09-18T10:15:17.985 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:17.985 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:17.985 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:17.985 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.986 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.986 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:203 3.9s0 3:9221fff5:::smithi18925896-48:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:17.986 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:52.793641+0000", 2024-09-18T10:15:17.986 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 25.131447786999999, 2024-09-18T10:15:17.986 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.41220821099999999, 2024-09-18T10:15:17.986 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:17.986 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:15:17.986 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:17.986 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:17.986 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:17.986 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 203 2024-09-18T10:15:17.986 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.987 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:17.987 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.987 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:17.987 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.793641+0000", 2024-09-18T10:15:17.987 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:17.987 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.987 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.987 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:17.987 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.793641+0000", 2024-09-18T10:15:17.987 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:17.987 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.987 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.988 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:17.988 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.793641+0000", 2024-09-18T10:15:17.988 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:15:17.988 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.988 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.988 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:17.988 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.793644+0000", 2024-09-18T10:15:17.988 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7170000000000002e-06 2024-09-18T10:15:17.988 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.988 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.988 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:17.988 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.793658+0000", 2024-09-18T10:15:17.989 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3353000000000001e-05 2024-09-18T10:15:17.989 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.989 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.989 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:17.989 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.793663+0000", 2024-09-18T10:15:17.989 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.7729999999999996e-06 2024-09-18T10:15:17.989 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.989 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.989 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:17.989 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.905813+0000", 2024-09-18T10:15:17.990 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.112150033 2024-09-18T10:15:17.990 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.990 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.990 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for blocked object", 2024-09-18T10:15:17.990 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.905901+0000", 2024-09-18T10:15:17.990 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.7319999999999997e-05 2024-09-18T10:15:17.990 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.990 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.990 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:17.990 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:53.205317+0000", 2024-09-18T10:15:17.990 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.299415826 2024-09-18T10:15:17.991 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.991 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.991 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:17.991 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:53.205528+0000", 2024-09-18T10:15:17.991 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000211485 2024-09-18T10:15:17.991 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.991 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.991 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:17.991 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:53.205850+0000", 2024-09-18T10:15:17.992 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000321544 2024-09-18T10:15:17.992 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:17.992 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:17.992 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:17.992 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.992 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.992 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.19s0 45/40 ECSubWriteReply(tid=52, last_complete=45'34, committed=1, applied=1))", 2024-09-18T10:15:17.992 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:01.593698+0000", 2024-09-18T10:15:17.992 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 16.33139078, 2024-09-18T10:15:17.992 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.32117952799999999, 2024-09-18T10:15:17.992 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:17.993 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:17.993 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:17.993 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.993 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:17.993 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.593698+0000", 2024-09-18T10:15:17.993 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:17.993 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.993 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.993 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:17.993 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.593698+0000", 2024-09-18T10:15:17.994 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:17.994 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.994 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.994 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:17.994 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.593697+0000", 2024-09-18T10:15:17.994 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:15:17.994 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.994 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.994 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:17.994 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.593705+0000", 2024-09-18T10:15:17.994 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.9489999999999996e-06 2024-09-18T10:15:17.994 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.995 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.995 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:17.995 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.593728+0000", 2024-09-18T10:15:17.995 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.2197000000000001e-05 2024-09-18T10:15:17.995 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.995 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.995 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:17.995 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.593738+0000", 2024-09-18T10:15:17.995 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0098999999999999e-05 2024-09-18T10:15:17.995 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.995 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.995 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:17.995 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.914447+0000", 2024-09-18T10:15:17.996 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.32070974800000002 2024-09-18T10:15:17.996 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.996 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.996 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:17.996 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.914878+0000", 2024-09-18T10:15:17.996 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00043047099999999998 2024-09-18T10:15:17.996 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:17.996 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:17.996 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:17.996 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.997 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.997 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.19s0 45/40 ECSubWriteReply(tid=53, last_complete=45'35, committed=1, applied=1))", 2024-09-18T10:15:17.997 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:01.593750+0000", 2024-09-18T10:15:17.997 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 16.331339594999999, 2024-09-18T10:15:17.997 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.32251513599999998, 2024-09-18T10:15:17.997 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:17.997 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:17.997 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:17.997 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.997 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:17.997 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.593750+0000", 2024-09-18T10:15:17.997 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:17.998 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.998 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.998 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:17.998 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.593750+0000", 2024-09-18T10:15:17.998 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:17.998 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.998 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.998 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:17.998 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.593749+0000", 2024-09-18T10:15:17.998 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:17.998 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.998 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.999 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:17.999 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.593755+0000", 2024-09-18T10:15:17.999 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.0599999999999996e-06 2024-09-18T10:15:17.999 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.999 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.999 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:17.999 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.593776+0000", 2024-09-18T10:15:17.999 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0914999999999999e-05 2024-09-18T10:15:17.999 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:17.999 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:17.999 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:18.000 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.593785+0000", 2024-09-18T10:15:18.000 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.3370000000000006e-06 2024-09-18T10:15:18.000 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.000 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.000 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:18.000 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.914974+0000", 2024-09-18T10:15:18.000 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.32118843899999999 2024-09-18T10:15:18.000 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.000 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.000 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:18.000 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.916265+0000", 2024-09-18T10:15:18.001 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001291001 2024-09-18T10:15:18.001 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:18.001 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:18.001 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:18.001 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.001 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.001 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.9s0 45/40 ECSubWriteReply(tid=55, last_complete=45'30, committed=1, applied=1))", 2024-09-18T10:15:18.001 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:01.594361+0000", 2024-09-18T10:15:18.001 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 16.330728151999999, 2024-09-18T10:15:18.001 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.32295218399999998, 2024-09-18T10:15:18.001 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:18.002 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:18.002 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:18.002 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.002 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:18.002 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.594361+0000", 2024-09-18T10:15:18.002 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:18.002 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.002 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.002 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:18.002 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.594361+0000", 2024-09-18T10:15:18.002 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:18.003 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.003 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.003 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:18.003 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.594360+0000", 2024-09-18T10:15:18.003 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:18.003 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.003 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.003 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:18.003 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.594366+0000", 2024-09-18T10:15:18.003 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.716e-06 2024-09-18T10:15:18.003 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.004 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.004 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:18.004 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.594381+0000", 2024-09-18T10:15:18.004 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4813e-05 2024-09-18T10:15:18.004 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.004 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.004 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:18.004 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.594390+0000", 2024-09-18T10:15:18.004 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.2199999999999998e-06 2024-09-18T10:15:18.004 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.004 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.005 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:18.005 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.916963+0000", 2024-09-18T10:15:18.005 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.32257234600000001 2024-09-18T10:15:18.005 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.005 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.005 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:18.005 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.917313+0000", 2024-09-18T10:15:18.005 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00035066200000000001 2024-09-18T10:15:18.005 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:18.005 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:18.005 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:18.005 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.006 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.006 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.9s0 45/40 ECSubWriteReply(tid=56, last_complete=45'31, committed=1, applied=1))", 2024-09-18T10:15:18.006 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:01.595816+0000", 2024-09-18T10:15:18.006 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 16.329273408999999, 2024-09-18T10:15:18.006 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.321651562, 2024-09-18T10:15:18.006 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:18.006 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:18.006 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:18.006 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.006 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:18.006 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.595816+0000", 2024-09-18T10:15:18.007 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:18.007 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.007 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.007 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:18.007 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.595816+0000", 2024-09-18T10:15:18.007 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:18.007 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.007 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.007 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:18.007 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.595814+0000", 2024-09-18T10:15:18.007 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:18.007 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.008 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.008 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:18.008 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.595824+0000", 2024-09-18T10:15:18.008 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.8560000000000002e-06 2024-09-18T10:15:18.008 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.008 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.008 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:18.008 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.595849+0000", 2024-09-18T10:15:18.008 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5006000000000001e-05 2024-09-18T10:15:18.008 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.008 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.008 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:18.009 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.595865+0000", 2024-09-18T10:15:18.009 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5546e-05 2024-09-18T10:15:18.009 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.009 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.009 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:18.009 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.917411+0000", 2024-09-18T10:15:18.009 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.32154633700000002 2024-09-18T10:15:18.009 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.009 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.009 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:18.009 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.917467+0000", 2024-09-18T10:15:18.009 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.6246999999999999e-05 2024-09-18T10:15:18.010 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:18.010 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:18.010 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:18.010 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.010 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.010 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.9s0 45/40 ECSubWriteReply(tid=56, last_complete=45'31, committed=1, applied=1))", 2024-09-18T10:15:18.010 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:01.595995+0000", 2024-09-18T10:15:18.010 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 16.329094048000002, 2024-09-18T10:15:18.010 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.32158524599999999, 2024-09-18T10:15:18.010 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:18.010 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:18.010 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:18.011 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.011 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:18.011 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.595995+0000", 2024-09-18T10:15:18.011 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:18.011 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.011 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.011 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:18.011 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.595995+0000", 2024-09-18T10:15:18.011 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:18.011 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.011 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.011 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:18.011 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.595994+0000", 2024-09-18T10:15:18.012 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:15:18.012 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.012 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.012 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:18.012 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.596005+0000", 2024-09-18T10:15:18.012 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1005999999999999e-05 2024-09-18T10:15:18.012 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.012 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.012 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:18.012 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.596024+0000", 2024-09-18T10:15:18.013 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.8862999999999999e-05 2024-09-18T10:15:18.013 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.013 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.013 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:18.013 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.596059+0000", 2024-09-18T10:15:18.013 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4941999999999999e-05 2024-09-18T10:15:18.013 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.013 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.013 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:18.013 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.917550+0000", 2024-09-18T10:15:18.013 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.32149124600000001 2024-09-18T10:15:18.014 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.014 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.014 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:18.014 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.917580+0000", 2024-09-18T10:15:18.014 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0241999999999999e-05 2024-09-18T10:15:18.014 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:18.014 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:18.014 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:18.014 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.014 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.014 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.9s0 45/40 ECSubWriteReply(tid=56, last_complete=45'31, committed=1, applied=1))", 2024-09-18T10:15:18.014 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:01.696021+0000", 2024-09-18T10:15:18.015 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 16.229068221999999, 2024-09-18T10:15:18.015 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.32268668299999997, 2024-09-18T10:15:18.015 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:18.015 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:18.015 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:18.015 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.015 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:18.015 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.696021+0000", 2024-09-18T10:15:18.015 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:18.015 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.015 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.015 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:18.016 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.696021+0000", 2024-09-18T10:15:18.016 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:18.016 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.016 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.016 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:18.016 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.696019+0000", 2024-09-18T10:15:18.016 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:18.016 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.016 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.016 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:18.017 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.696031+0000", 2024-09-18T10:15:18.017 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1026000000000001e-05 2024-09-18T10:15:18.017 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.017 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.017 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:18.017 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.696064+0000", 2024-09-18T10:15:18.017 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3902000000000002e-05 2024-09-18T10:15:18.017 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.017 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.017 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:18.017 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.696081+0000", 2024-09-18T10:15:18.017 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.666e-05 2024-09-18T10:15:18.018 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.018 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.018 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:18.018 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.017550+0000", 2024-09-18T10:15:18.018 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.32146865699999999 2024-09-18T10:15:18.018 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.018 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.018 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:18.018 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.018708+0000", 2024-09-18T10:15:18.018 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001157911 2024-09-18T10:15:18.019 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:18.019 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:18.019 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:18.019 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.019 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.019 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:351 3.19s0 3:9a1a774c:::smithi18925896-38:b [read 0~1638400,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-18T10:15:18.019 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:03.544302+0000", 2024-09-18T10:15:18.019 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 14.380786783, 2024-09-18T10:15:18.019 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.41512137599999999, 2024-09-18T10:15:18.020 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:18.020 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:15:18.020 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:18.020 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:18.020 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:18.020 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 351 2024-09-18T10:15:18.020 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.020 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:18.020 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.020 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:18.020 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.544302+0000", 2024-09-18T10:15:18.021 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:18.021 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.021 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.021 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:18.021 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.544302+0000", 2024-09-18T10:15:18.021 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:18.021 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.021 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.021 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:18.021 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.544298+0000", 2024-09-18T10:15:18.021 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999957 2024-09-18T10:15:18.022 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.022 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.022 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:18.022 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.544320+0000", 2024-09-18T10:15:18.022 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1549e-05 2024-09-18T10:15:18.022 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.022 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.022 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:18.022 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.544366+0000", 2024-09-18T10:15:18.022 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.6752999999999998e-05 2024-09-18T10:15:18.022 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.023 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.023 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:18.023 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.544387+0000", 2024-09-18T10:15:18.023 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1052000000000001e-05 2024-09-18T10:15:18.023 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.023 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.023 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:18.023 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.544758+0000", 2024-09-18T10:15:18.023 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00037069700000000002 2024-09-18T10:15:18.023 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.023 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.023 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "op must wait for map", 2024-09-18T10:15:18.024 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.544786+0000", 2024-09-18T10:15:18.024 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7472000000000001e-05 2024-09-18T10:15:18.024 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.024 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.024 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:18.024 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.649236+0000", 2024-09-18T10:15:18.024 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.104450378 2024-09-18T10:15:18.024 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.024 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.024 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:18.024 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.649492+0000", 2024-09-18T10:15:18.025 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00025605300000000001 2024-09-18T10:15:18.025 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.025 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.025 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:18.025 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.959424+0000", 2024-09-18T10:15:18.025 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.30993174899999998 2024-09-18T10:15:18.025 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:18.025 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:18.025 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:18.025 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.025 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.026 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:417 3.9s0 3:9221fff5:::smithi18925896-48:head [copy-from ver 8] snapc 10=[10,f,e,b,4] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-18T10:15:18.026 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:08.547348+0000", 2024-09-18T10:15:18.026 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 9.3777415570000002, 2024-09-18T10:15:18.026 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.69832097199999998, 2024-09-18T10:15:18.026 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:18.026 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:18.026 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:18.026 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:18.026 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:18.026 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 417 2024-09-18T10:15:18.026 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.027 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:18.027 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.027 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:18.027 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:08.547348+0000", 2024-09-18T10:15:18.027 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:18.027 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.027 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.027 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:18.027 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:08.547348+0000", 2024-09-18T10:15:18.027 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:18.027 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.027 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.028 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:18.028 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:08.547344+0000", 2024-09-18T10:15:18.028 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999967 2024-09-18T10:15:18.028 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.028 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.028 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:18.028 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:08.547367+0000", 2024-09-18T10:15:18.028 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.2316000000000001e-05 2024-09-18T10:15:18.028 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.028 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.028 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:18.029 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:08.547415+0000", 2024-09-18T10:15:18.029 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.8541999999999997e-05 2024-09-18T10:15:18.029 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.029 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.029 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:18.029 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:08.647531+0000", 2024-09-18T10:15:18.029 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10011588 2024-09-18T10:15:18.029 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.029 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.029 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:18.029 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:08.647713+0000", 2024-09-18T10:15:18.030 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000181965 2024-09-18T10:15:18.030 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.030 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.030 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "op must wait for map", 2024-09-18T10:15:18.030 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:08.647726+0000", 2024-09-18T10:15:18.030 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3576e-05 2024-09-18T10:15:18.030 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.030 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.030 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:18.030 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:08.754339+0000", 2024-09-18T10:15:18.030 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10661229699999999 2024-09-18T10:15:18.030 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.031 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.031 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:18.031 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:08.754855+0000", 2024-09-18T10:15:18.031 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00051627900000000002 2024-09-18T10:15:18.031 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.031 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.031 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:18.031 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:09.040700+0000", 2024-09-18T10:15:18.031 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.285844922 2024-09-18T10:15:18.031 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.032 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.032 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:18.032 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:09.045961+0000", 2024-09-18T10:15:18.032 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0052605769999999998 2024-09-18T10:15:18.032 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.032 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.032 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:18.032 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:09.245481+0000", 2024-09-18T10:15:18.032 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.19952081599999999 2024-09-18T10:15:18.032 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.032 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.033 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:18.033 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:09.245669+0000", 2024-09-18T10:15:18.033 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000187245 2024-09-18T10:15:18.033 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:18.033 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:18.033 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:18.033 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.033 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.033 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:418 3.9s0 3:9221fff5:::smithi18925896-48:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e53)", 2024-09-18T10:15:18.033 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:08.647572+0000", 2024-09-18T10:15:18.033 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 9.2775174059999994, 2024-09-18T10:15:18.034 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.39760114600000002, 2024-09-18T10:15:18.034 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:18.034 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:15:18.034 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:18.034 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:18.034 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:18.034 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 418 2024-09-18T10:15:18.034 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.034 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:18.034 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.035 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:18.035 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:08.647572+0000", 2024-09-18T10:15:18.035 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:18.035 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.035 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.035 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:18.035 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:08.647572+0000", 2024-09-18T10:15:18.035 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:18.035 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.035 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.036 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:18.036 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:08.647570+0000", 2024-09-18T10:15:18.036 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:18.036 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.036 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.036 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:18.036 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:08.647581+0000", 2024-09-18T10:15:18.036 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0829000000000001e-05 2024-09-18T10:15:18.036 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.036 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.036 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:18.037 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:08.647599+0000", 2024-09-18T10:15:18.037 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7846000000000002e-05 2024-09-18T10:15:18.037 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.037 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.037 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:18.037 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:08.647609+0000", 2024-09-18T10:15:18.037 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0339999999999999e-05 2024-09-18T10:15:18.037 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.037 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.037 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:18.037 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:08.747808+0000", 2024-09-18T10:15:18.038 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100199012 2024-09-18T10:15:18.038 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.038 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.038 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:18.038 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:08.747819+0000", 2024-09-18T10:15:18.038 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1276e-05 2024-09-18T10:15:18.038 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.038 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.038 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:18.038 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:08.854528+0000", 2024-09-18T10:15:18.038 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.106708267 2024-09-18T10:15:18.039 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.039 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.039 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for blocked object", 2024-09-18T10:15:18.039 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:08.854638+0000", 2024-09-18T10:15:18.039 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000110214 2024-09-18T10:15:18.039 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.039 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.039 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:18.039 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:09.044871+0000", 2024-09-18T10:15:18.039 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.190232762 2024-09-18T10:15:18.039 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.039 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.040 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:18.040 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:09.045037+0000", 2024-09-18T10:15:18.040 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00016675500000000001 2024-09-18T10:15:18.040 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.040 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.040 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:18.040 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:09.045173+0000", 2024-09-18T10:15:18.040 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000135575 2024-09-18T10:15:18.040 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:18.040 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:18.040 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:18.040 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.041 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.041 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.1s4 57/40 ECSubWrite(tid=112, reqid=osd.4.0:112, at_version=57'20, trim_to=0'0, roll_forward_to=56'16))", 2024-09-18T10:15:18.041 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:13.062699+0000", 2024-09-18T10:15:18.041 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 4.8623898680000002, 2024-09-18T10:15:18.041 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40470361999999999, 2024-09-18T10:15:18.041 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:18.041 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:18.041 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:18.041 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.041 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:18.041 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.062699+0000", 2024-09-18T10:15:18.041 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:18.042 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.042 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.042 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:18.042 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.062699+0000", 2024-09-18T10:15:18.042 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:18.042 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.042 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.042 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:18.042 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.062699+0000", 2024-09-18T10:15:18.042 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:18.042 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.042 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.043 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:18.043 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.062734+0000", 2024-09-18T10:15:18.043 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.5108999999999999e-05 2024-09-18T10:15:18.043 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.043 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.043 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:18.043 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.062746+0000", 2024-09-18T10:15:18.043 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2062000000000001e-05 2024-09-18T10:15:18.043 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.043 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.044 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:18.044 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.062754+0000", 2024-09-18T10:15:18.044 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.5800000000000003e-06 2024-09-18T10:15:18.044 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.044 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.044 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:18.044 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.254656+0000", 2024-09-18T10:15:18.044 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.19190244400000001 2024-09-18T10:15:18.044 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.044 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.044 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:18.044 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.254673+0000", 2024-09-18T10:15:18.044 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7146000000000002e-05 2024-09-18T10:15:18.045 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.045 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.045 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:18.045 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.467347+0000", 2024-09-18T10:15:18.045 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.21267335500000001 2024-09-18T10:15:18.045 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.045 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.045 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:18.045 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.467403+0000", 2024-09-18T10:15:18.045 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.6443999999999997e-05 2024-09-18T10:15:18.045 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:18.045 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:18.046 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:18.046 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:18.046 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:18.046 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-18T10:15:18.048 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-09-18T10:15:18.048 INFO:tasks.ceph.osd.15.smithi189.stderr:2024-09-18T10:15:18.042+0000 7fd0e725e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 22401) UID: 0 2024-09-18T10:15:18.148 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-18T10:15:18.145+0000 7f14c6a1a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 22138) UID: 0 2024-09-18T10:15:18.156 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-18T10:15:18.156 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [ 2024-09-18T10:15:18.156 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.157 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:495 3.es0 3:7255ddf4:::smithi18925896-39:head [rmxattr R^`al]TqifatYOiOU]p`ZegfXmNsXVaZcdd]b]YX in=40b,rmxattr TNQYR^jgR`pPaYitmrcliV[dttobncTntcfTlQkt in=40b,rmxattr W_aq[RVsaUpQtX_RpV^Vpqrg`QgQViPeVaRqZSmf in=40b,rmxattr aYfgr^q[\\aPg_aZsakfUfsdhYbra^QXjlkrpdY\\\\ in=40b,rmxattr dgXPi in=5b,rmxattr p]OS_Znl[dNa^qTn`WW[j\\dtgbNac_bijYmg in=36b,rmxattr tZ]grhlaq]kUO[k^d^dr``XNio^^Oaebpj\\[Ytdq in=40b] snapc 19=[19,18,16,12,11,10,f,e] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T10:15:18.157 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:17.889335+0000", 2024-09-18T10:15:18.157 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.26618517400000002, 2024-09-18T10:15:18.157 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.26625626299999999, 2024-09-18T10:15:18.157 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:18.157 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:15:18.157 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:18.157 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:18.157 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:18.158 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 495 2024-09-18T10:15:18.158 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.158 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:18.158 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.158 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:18.158 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889335+0000", 2024-09-18T10:15:18.158 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:18.158 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.158 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.158 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:18.158 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889335+0000", 2024-09-18T10:15:18.158 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:18.158 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.158 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.159 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:18.159 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889333+0000", 2024-09-18T10:15:18.159 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999976 2024-09-18T10:15:18.159 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.159 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.159 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:18.159 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889351+0000", 2024-09-18T10:15:18.159 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7795000000000002e-05 2024-09-18T10:15:18.159 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.159 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.159 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:18.159 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889383+0000", 2024-09-18T10:15:18.159 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2740999999999997e-05 2024-09-18T10:15:18.159 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.160 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.160 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:18.160 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889396+0000", 2024-09-18T10:15:18.160 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2926e-05 2024-09-18T10:15:18.160 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.160 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.160 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:18.160 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.989725+0000", 2024-09-18T10:15:18.160 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10032845999999999 2024-09-18T10:15:18.160 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.160 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.160 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "op must wait for map", 2024-09-18T10:15:18.160 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.989751+0000", 2024-09-18T10:15:18.160 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5819000000000001e-05 2024-09-18T10:15:18.161 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.161 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.161 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:18.161 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.999619+0000", 2024-09-18T10:15:18.161 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0098683920000000001 2024-09-18T10:15:18.161 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.161 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.161 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:18.161 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.999776+0000", 2024-09-18T10:15:18.161 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000157336 2024-09-18T10:15:18.161 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.161 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.161 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:18.161 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.001374+0000", 2024-09-18T10:15:18.161 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0015982419999999999 2024-09-18T10:15:18.162 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.162 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.162 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:18.162 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.003908+0000", 2024-09-18T10:15:18.162 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0025331870000000001 2024-09-18T10:15:18.162 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:18.162 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:18.162 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:18.162 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.162 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.162 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.es0 62/32 ECSubWriteReply(tid=33, last_complete=62'27, committed=1, applied=1))", 2024-09-18T10:15:18.162 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:18.014530+0000", 2024-09-18T10:15:18.163 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.140990327, 2024-09-18T10:15:18.163 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.14120718199999999, 2024-09-18T10:15:18.163 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:18.163 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "queued for pg", 2024-09-18T10:15:18.163 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:18.163 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.163 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:18.163 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.014530+0000", 2024-09-18T10:15:18.163 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:18.163 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.163 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.163 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:18.163 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.014530+0000", 2024-09-18T10:15:18.163 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:18.164 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.164 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.164 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:18.164 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.014529+0000", 2024-09-18T10:15:18.164 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:18.164 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.164 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.164 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:18.164 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.014539+0000", 2024-09-18T10:15:18.164 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0093e-05 2024-09-18T10:15:18.164 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.164 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.165 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:18.165 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.014570+0000", 2024-09-18T10:15:18.165 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1245999999999999e-05 2024-09-18T10:15:18.165 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.165 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.165 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:18.165 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.014582+0000", 2024-09-18T10:15:18.165 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1610000000000001e-05 2024-09-18T10:15:18.165 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:18.165 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:18.165 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:18.165 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.165 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.166 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.es0 62/32 ECSubWriteReply(tid=33, last_complete=62'27, committed=1, applied=1))", 2024-09-18T10:15:18.166 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:18.105221+0000", 2024-09-18T10:15:18.166 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.050299984999999998, 2024-09-18T10:15:18.166 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.050587642000000002, 2024-09-18T10:15:18.166 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:18.166 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "queued for pg", 2024-09-18T10:15:18.166 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:18.166 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.166 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:18.166 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.105221+0000", 2024-09-18T10:15:18.166 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:18.166 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.166 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.167 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:18.167 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.105221+0000", 2024-09-18T10:15:18.167 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:18.167 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.167 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.167 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:18.167 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.105218+0000", 2024-09-18T10:15:18.167 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999976 2024-09-18T10:15:18.167 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.167 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.167 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:18.167 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.105232+0000", 2024-09-18T10:15:18.167 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4133e-05 2024-09-18T10:15:18.168 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.168 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.168 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:18.168 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.105266+0000", 2024-09-18T10:15:18.168 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4001999999999997e-05 2024-09-18T10:15:18.168 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.168 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.168 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:18.168 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.105279+0000", 2024-09-18T10:15:18.168 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2486999999999999e-05 2024-09-18T10:15:18.168 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:18.168 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:18.168 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:18.168 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:18.169 INFO:teuthology.orchestra.run.smithi189.stdout: ], 2024-09-18T10:15:18.169 INFO:teuthology.orchestra.run.smithi189.stdout: "num_ops": 3 2024-09-18T10:15:18.169 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-18T10:15:18.170 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-09-18T10:15:18.218 INFO:tasks.rados.rados.0.smithi189.stdout:201: expect (ObjNum 4 snap 0 seq_num 4) 2024-09-18T10:15:18.249 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-09-18T10:15:18.243+0000 7fdfa8698700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22087) UID: 0 2024-09-18T10:15:18.267 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-18T10:15:18.268 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-18T10:15:18.268 INFO:teuthology.orchestra.run.smithi189.stdout: "complaint_time": 30, 2024-09-18T10:15:18.268 INFO:teuthology.orchestra.run.smithi189.stdout: "num_blocked_ops": 0 2024-09-18T10:15:18.268 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-18T10:15:18.277 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-09-18T10:15:18.318 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 39 v 27 (ObjNum 68 snap 6 seq_num 68) dirty exists 2024-09-18T10:15:18.350 INFO:tasks.ceph.osd.3.smithi189.stderr:2024-09-18T10:15:18.346+0000 7f2440fc7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22396) UID: 0 2024-09-18T10:15:18.376 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-18T10:15:18.376 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 20, 2024-09-18T10:15:18.376 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 600, 2024-09-18T10:15:18.377 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [ 2024-09-18T10:15:18.377 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.377 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:146 3.es0 3:78aad6f5:::smithi18925896-49:head [append 0~442368,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:18.377 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:51.117496+0000", 2024-09-18T10:15:18.377 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 27.255424003000002, 2024-09-18T10:15:18.377 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.61437150200000001, 2024-09-18T10:15:18.377 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:18.377 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:18.377 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:18.377 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:18.377 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:18.377 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 146 2024-09-18T10:15:18.378 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.378 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:18.378 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.378 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:18.378 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.117496+0000", 2024-09-18T10:15:18.378 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:18.378 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.378 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.378 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:18.378 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.117496+0000", 2024-09-18T10:15:18.378 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:18.378 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.378 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.379 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:18.379 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.117493+0000", 2024-09-18T10:15:18.379 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999976 2024-09-18T10:15:18.379 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.379 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.379 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:18.379 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.117787+0000", 2024-09-18T10:15:18.379 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00029378100000000001 2024-09-18T10:15:18.379 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.379 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.379 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:18.379 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.117820+0000", 2024-09-18T10:15:18.379 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3073e-05 2024-09-18T10:15:18.379 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.380 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.380 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:18.380 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.117834+0000", 2024-09-18T10:15:18.380 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3369e-05 2024-09-18T10:15:18.380 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.380 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.380 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:18.380 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.127090+0000", 2024-09-18T10:15:18.380 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0092563769999999997 2024-09-18T10:15:18.380 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.380 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.380 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:18.381 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.127435+0000", 2024-09-18T10:15:18.381 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00034522300000000003 2024-09-18T10:15:18.381 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.381 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.381 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:18.381 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.129115+0000", 2024-09-18T10:15:18.381 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0016792980000000001 2024-09-18T10:15:18.381 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.381 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.381 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:18.381 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.430505+0000", 2024-09-18T10:15:18.381 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.30139002300000001 2024-09-18T10:15:18.381 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.382 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.382 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:18.382 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.731753+0000", 2024-09-18T10:15:18.382 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.30124798800000002 2024-09-18T10:15:18.382 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.382 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.382 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:18.382 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.731867+0000", 2024-09-18T10:15:18.382 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000114764 2024-09-18T10:15:18.382 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:18.382 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:18.382 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:18.383 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.383 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.383 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:147 3.es0 3:78aad6f5:::smithi18925896-49:head [append 0~376832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:18.383 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:51.122092+0000", 2024-09-18T10:15:18.383 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 27.250827585, 2024-09-18T10:15:18.383 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.61047882200000003, 2024-09-18T10:15:18.383 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:18.383 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:18.383 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:18.383 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:18.383 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:18.383 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 147 2024-09-18T10:15:18.383 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.384 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:18.384 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.384 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:18.384 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122092+0000", 2024-09-18T10:15:18.384 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:18.384 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.384 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.384 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:18.384 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122092+0000", 2024-09-18T10:15:18.384 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:18.384 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.384 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.384 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:18.385 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122091+0000", 2024-09-18T10:15:18.385 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:18.385 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.385 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.385 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:18.385 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122504+0000", 2024-09-18T10:15:18.385 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00041322100000000002 2024-09-18T10:15:18.385 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.385 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.385 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:18.385 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122527+0000", 2024-09-18T10:15:18.385 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3504999999999999e-05 2024-09-18T10:15:18.386 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.386 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.386 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:18.386 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122537+0000", 2024-09-18T10:15:18.386 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.8460000000000003e-06 2024-09-18T10:15:18.386 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.386 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.386 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:18.386 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.327415+0000", 2024-09-18T10:15:18.386 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20487799700000001 2024-09-18T10:15:18.386 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.386 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.386 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:18.387 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.327614+0000", 2024-09-18T10:15:18.387 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00019855400000000001 2024-09-18T10:15:18.387 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.387 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.387 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:18.387 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.328920+0000", 2024-09-18T10:15:18.387 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001306263 2024-09-18T10:15:18.387 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.387 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.387 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:18.387 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.430570+0000", 2024-09-18T10:15:18.387 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.101650508 2024-09-18T10:15:18.387 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.387 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.388 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:18.388 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.732477+0000", 2024-09-18T10:15:18.388 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.30190700100000001 2024-09-18T10:15:18.388 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.388 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.388 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:18.388 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.732571+0000", 2024-09-18T10:15:18.388 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.3580999999999997e-05 2024-09-18T10:15:18.388 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:18.388 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:18.388 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:18.388 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.388 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.389 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:148 3.es0 3:78aad6f5:::smithi18925896-49:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:18.389 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:51.122547+0000", 2024-09-18T10:15:18.389 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 27.250372493, 2024-09-18T10:15:18.389 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.91462763400000002, 2024-09-18T10:15:18.389 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:18.389 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:18.389 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:18.389 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:18.389 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:18.389 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 148 2024-09-18T10:15:18.389 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.389 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:18.389 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.389 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:18.390 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122547+0000", 2024-09-18T10:15:18.390 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:18.390 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.390 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.390 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:18.390 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122547+0000", 2024-09-18T10:15:18.390 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:18.390 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.390 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.390 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:18.390 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122546+0000", 2024-09-18T10:15:18.390 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:15:18.390 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.390 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.390 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:18.391 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122552+0000", 2024-09-18T10:15:18.391 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3569999999999997e-06 2024-09-18T10:15:18.391 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.391 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.391 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:18.391 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122560+0000", 2024-09-18T10:15:18.391 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.191e-06 2024-09-18T10:15:18.391 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.391 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.391 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:18.391 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122569+0000", 2024-09-18T10:15:18.391 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.8230000000000001e-06 2024-09-18T10:15:18.391 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.392 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.392 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:18.392 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.834492+0000", 2024-09-18T10:15:18.392 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.71192345599999995 2024-09-18T10:15:18.392 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.392 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.392 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:18.392 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.834669+0000", 2024-09-18T10:15:18.392 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000177035 2024-09-18T10:15:18.392 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.392 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.392 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:18.392 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.835442+0000", 2024-09-18T10:15:18.392 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00077327799999999996 2024-09-18T10:15:18.393 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.393 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.393 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:18.393 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.936574+0000", 2024-09-18T10:15:18.393 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.101131163 2024-09-18T10:15:18.393 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.393 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.393 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:18.393 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.037028+0000", 2024-09-18T10:15:18.393 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100454026 2024-09-18T10:15:18.393 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.393 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.393 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:18.394 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.037175+0000", 2024-09-18T10:15:18.394 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000147305 2024-09-18T10:15:18.394 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:18.394 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:18.394 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:18.394 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.394 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.394 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:149 3.es0 3:78aad6f5:::smithi18925896-49:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:18.394 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:51.122575+0000", 2024-09-18T10:15:18.394 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 27.250345285000002, 2024-09-18T10:15:18.394 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.016813872, 2024-09-18T10:15:18.394 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:18.395 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:15:18.395 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:18.395 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:18.395 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:18.395 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 149 2024-09-18T10:15:18.395 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.395 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:18.395 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.395 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:18.396 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122575+0000", 2024-09-18T10:15:18.396 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:18.396 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.396 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.396 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:18.396 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122575+0000", 2024-09-18T10:15:18.396 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:18.396 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.396 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.396 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:18.396 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122574+0000", 2024-09-18T10:15:18.396 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:15:18.396 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.397 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.397 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:18.397 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122579+0000", 2024-09-18T10:15:18.397 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3650000000000003e-06 2024-09-18T10:15:18.397 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.397 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.397 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:18.397 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122587+0000", 2024-09-18T10:15:18.397 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.3000000000000002e-06 2024-09-18T10:15:18.397 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.397 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.397 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:18.397 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122592+0000", 2024-09-18T10:15:18.398 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9080000000000003e-06 2024-09-18T10:15:18.398 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.398 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.398 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:18.398 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.935931+0000", 2024-09-18T10:15:18.398 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.81333878199999998 2024-09-18T10:15:18.398 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.398 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.398 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for rw locks", 2024-09-18T10:15:18.398 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.936216+0000", 2024-09-18T10:15:18.398 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00028463400000000003 2024-09-18T10:15:18.398 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.398 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.398 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:18.398 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.038007+0000", 2024-09-18T10:15:18.399 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.101791539 2024-09-18T10:15:18.399 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.399 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.399 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:18.399 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.038254+0000", 2024-09-18T10:15:18.399 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000246932 2024-09-18T10:15:18.399 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.399 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.399 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:18.399 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.139388+0000", 2024-09-18T10:15:18.399 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10113417199999999 2024-09-18T10:15:18.399 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:18.399 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:18.399 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:18.400 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.400 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.400 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpRead(3.es0 36/32 ECSubRead(tid=3, to_read={3:7255ddf4:::smithi18925896-39:head=0,4096,0}, subchunks={3:7255ddf4:::smithi18925896-39:head=[0,1]}, attrs_to_read=))", 2024-09-18T10:15:18.400 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:51.126838+0000", 2024-09-18T10:15:18.400 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 27.246081862, 2024-09-18T10:15:18.400 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.707585825, 2024-09-18T10:15:18.400 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:18.400 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:18.400 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:18.400 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.400 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:18.400 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.126838+0000", 2024-09-18T10:15:18.400 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:18.401 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.401 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.401 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:18.401 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.126838+0000", 2024-09-18T10:15:18.401 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:18.401 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.401 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.401 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:18.401 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.126838+0000", 2024-09-18T10:15:18.401 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:18.401 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.401 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.401 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:18.402 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.126838+0000", 2024-09-18T10:15:18.402 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:18.402 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.402 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.402 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:18.402 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.127004+0000", 2024-09-18T10:15:18.402 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00016626499999999999 2024-09-18T10:15:18.402 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.402 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.402 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:18.402 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.227100+0000", 2024-09-18T10:15:18.402 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.1000955 2024-09-18T10:15:18.402 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.402 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.403 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:18.403 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.834083+0000", 2024-09-18T10:15:18.403 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.60698300100000002 2024-09-18T10:15:18.403 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.403 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.403 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:18.403 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.834424+0000", 2024-09-18T10:15:18.403 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000341059 2024-09-18T10:15:18.403 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:18.403 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:18.403 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:18.403 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.403 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.404 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpReadReply(3.es0 36/32 ECSubReadReply(tid=3, attrs_read=0))", 2024-09-18T10:15:18.404 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:51.214895+0000", 2024-09-18T10:15:18.404 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 27.158025185, 2024-09-18T10:15:18.404 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.61857800100000004, 2024-09-18T10:15:18.404 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:18.404 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:18.404 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:18.404 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.404 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:18.404 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.214895+0000", 2024-09-18T10:15:18.404 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:18.404 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.404 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.405 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:18.405 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.214895+0000", 2024-09-18T10:15:18.405 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:18.405 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.405 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.405 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:18.405 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.214893+0000", 2024-09-18T10:15:18.405 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:18.405 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.405 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.405 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:18.405 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.214927+0000", 2024-09-18T10:15:18.406 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3841999999999999e-05 2024-09-18T10:15:18.406 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.406 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.406 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:18.406 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.214953+0000", 2024-09-18T10:15:18.406 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.6519000000000001e-05 2024-09-18T10:15:18.406 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.406 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.406 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:18.406 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.214971+0000", 2024-09-18T10:15:18.406 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7535999999999999e-05 2024-09-18T10:15:18.406 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.406 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.407 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:18.407 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.833397+0000", 2024-09-18T10:15:18.407 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.61842642599999997 2024-09-18T10:15:18.407 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.407 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.407 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:18.407 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.833473+0000", 2024-09-18T10:15:18.407 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.5621e-05 2024-09-18T10:15:18.407 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:18.407 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:18.407 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:18.407 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.407 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.407 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpReadReply(3.es0 36/32 ECSubReadReply(tid=3, attrs_read=0))", 2024-09-18T10:15:18.408 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:51.227596+0000", 2024-09-18T10:15:18.408 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 27.145323634, 2024-09-18T10:15:18.408 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.60597977400000003, 2024-09-18T10:15:18.408 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:18.408 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:18.408 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:18.408 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.408 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:18.408 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.227596+0000", 2024-09-18T10:15:18.408 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:18.408 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.408 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.408 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:18.409 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.227596+0000", 2024-09-18T10:15:18.409 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:18.409 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.409 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.409 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:18.409 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.227595+0000", 2024-09-18T10:15:18.409 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:15:18.409 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.409 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.409 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:18.409 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.227605+0000", 2024-09-18T10:15:18.409 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.7040000000000006e-06 2024-09-18T10:15:18.409 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.409 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.410 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:18.410 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.227621+0000", 2024-09-18T10:15:18.410 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6107999999999999e-05 2024-09-18T10:15:18.410 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.410 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.410 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:18.410 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.227637+0000", 2024-09-18T10:15:18.410 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5282999999999999e-05 2024-09-18T10:15:18.410 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.410 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.410 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:18.410 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.833511+0000", 2024-09-18T10:15:18.410 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.60587403699999998 2024-09-18T10:15:18.411 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.411 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.411 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:18.411 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.833576+0000", 2024-09-18T10:15:18.411 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.5392e-05 2024-09-18T10:15:18.411 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:18.411 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:18.411 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:18.411 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.411 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.411 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.es0 36/32 ECSubWriteReply(tid=4, last_complete=36'3, committed=1, applied=1))", 2024-09-18T10:15:18.411 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:51.233227+0000", 2024-09-18T10:15:18.411 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 27.139692620000002, 2024-09-18T10:15:18.411 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.39820466900000001, 2024-09-18T10:15:18.412 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:18.412 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:18.412 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:18.412 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.412 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:18.412 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.233227+0000", 2024-09-18T10:15:18.412 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:18.412 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.412 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.412 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:18.412 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.233227+0000", 2024-09-18T10:15:18.412 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:18.412 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.413 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.413 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:18.413 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.233226+0000", 2024-09-18T10:15:18.413 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:15:18.413 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.413 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.413 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:18.413 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.233232+0000", 2024-09-18T10:15:18.413 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9370000000000001e-06 2024-09-18T10:15:18.413 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.413 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.413 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:18.413 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.233248+0000", 2024-09-18T10:15:18.414 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5574999999999999e-05 2024-09-18T10:15:18.414 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.414 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.414 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:18.414 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.233254+0000", 2024-09-18T10:15:18.414 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.3659999999999997e-06 2024-09-18T10:15:18.414 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.414 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.414 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:18.414 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.631420+0000", 2024-09-18T10:15:18.414 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.39816591699999998 2024-09-18T10:15:18.414 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.414 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.415 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:18.415 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.631432+0000", 2024-09-18T10:15:18.415 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1737e-05 2024-09-18T10:15:18.415 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:18.415 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:18.415 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:18.415 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.415 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.415 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.es0 36/32 ECSubWriteReply(tid=4, last_complete=36'3, committed=1, applied=1))", 2024-09-18T10:15:18.415 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:51.315941+0000", 2024-09-18T10:15:18.415 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 27.056978378, 2024-09-18T10:15:18.415 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.315333435, 2024-09-18T10:15:18.416 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:18.416 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:18.416 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:18.416 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.416 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:18.416 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.315941+0000", 2024-09-18T10:15:18.416 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:18.416 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.416 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.416 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:18.416 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.315941+0000", 2024-09-18T10:15:18.416 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:18.416 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.416 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.417 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:18.417 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.315940+0000", 2024-09-18T10:15:18.417 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:15:18.417 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.417 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.417 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:18.417 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.315950+0000", 2024-09-18T10:15:18.417 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.2329999999999995e-06 2024-09-18T10:15:18.417 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.417 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.417 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:18.417 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.315968+0000", 2024-09-18T10:15:18.417 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.8067000000000002e-05 2024-09-18T10:15:18.417 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.418 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.418 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:18.418 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.315977+0000", 2024-09-18T10:15:18.418 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.6420000000000002e-06 2024-09-18T10:15:18.418 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.418 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.418 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:18.418 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.631262+0000", 2024-09-18T10:15:18.418 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.31528474000000001 2024-09-18T10:15:18.418 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.418 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.418 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:18.418 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.631275+0000", 2024-09-18T10:15:18.418 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2799999999999999e-05 2024-09-18T10:15:18.418 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:18.419 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:18.419 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:18.419 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.419 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.419 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.es0 36/32 ECSubWriteReply(tid=5, last_complete=36'4, committed=1, applied=1))", 2024-09-18T10:15:18.419 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:51.331048+0000", 2024-09-18T10:15:18.419 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 27.041872193, 2024-09-18T10:15:18.419 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40123123900000002, 2024-09-18T10:15:18.419 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:18.419 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:18.419 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:18.419 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.419 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:18.420 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.331048+0000", 2024-09-18T10:15:18.420 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:18.420 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.420 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.420 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:18.420 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.331048+0000", 2024-09-18T10:15:18.420 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:18.420 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.420 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.420 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:18.420 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.331046+0000", 2024-09-18T10:15:18.420 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:18.420 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.421 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.421 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:18.421 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.331057+0000", 2024-09-18T10:15:18.421 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1683e-05 2024-09-18T10:15:18.421 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.421 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.421 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:18.421 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.331090+0000", 2024-09-18T10:15:18.421 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2576000000000001e-05 2024-09-18T10:15:18.421 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.421 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.421 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:18.422 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.331107+0000", 2024-09-18T10:15:18.422 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7382999999999999e-05 2024-09-18T10:15:18.422 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.422 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.422 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:18.422 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.732241+0000", 2024-09-18T10:15:18.422 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40113375200000001 2024-09-18T10:15:18.422 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.422 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.422 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:18.422 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.732279+0000", 2024-09-18T10:15:18.422 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7719000000000003e-05 2024-09-18T10:15:18.422 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:18.423 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:18.423 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:18.423 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.423 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.423 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.es0 36/32 ECSubWriteReply(tid=5, last_complete=36'4, committed=1, applied=1))", 2024-09-18T10:15:18.423 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:51.331214+0000", 2024-09-18T10:15:18.423 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 27.041705798999999, 2024-09-18T10:15:18.423 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40084109800000001, 2024-09-18T10:15:18.423 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:18.423 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:18.423 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:18.423 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.423 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:18.424 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.331214+0000", 2024-09-18T10:15:18.424 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:18.424 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.424 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.424 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:18.424 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.331214+0000", 2024-09-18T10:15:18.424 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:18.424 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.424 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.424 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:18.424 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.331213+0000", 2024-09-18T10:15:18.424 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:15:18.425 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.425 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.425 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:18.425 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.331220+0000", 2024-09-18T10:15:18.425 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.1369999999999996e-06 2024-09-18T10:15:18.425 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.425 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.425 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:18.425 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.331235+0000", 2024-09-18T10:15:18.425 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4849e-05 2024-09-18T10:15:18.425 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.425 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.426 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:18.426 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.331243+0000", 2024-09-18T10:15:18.426 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.157e-06 2024-09-18T10:15:18.426 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.426 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.426 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:18.426 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.732010+0000", 2024-09-18T10:15:18.426 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40076711300000001 2024-09-18T10:15:18.426 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.426 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.426 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:18.426 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.732055+0000", 2024-09-18T10:15:18.426 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4821999999999999e-05 2024-09-18T10:15:18.427 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:18.427 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:18.427 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:18.427 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.427 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.427 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:346 3.es0 3:7255ddf4:::smithi18925896-39:head [append 0~770048] snapc b=[b,4] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-18T10:15:18.427 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:02.898102+0000", 2024-09-18T10:15:18.427 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 15.474818337, 2024-09-18T10:15:18.427 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.31123798899999999, 2024-09-18T10:15:18.427 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:18.427 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:18.428 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:18.428 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:18.428 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:18.428 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 346 2024-09-18T10:15:18.428 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.428 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:18.428 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.428 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:18.428 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.898102+0000", 2024-09-18T10:15:18.428 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:18.428 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.428 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.429 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:18.429 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.898102+0000", 2024-09-18T10:15:18.429 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:18.429 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.429 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.429 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:18.429 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.898099+0000", 2024-09-18T10:15:18.429 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999971 2024-09-18T10:15:18.429 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.429 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.429 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:18.430 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.898914+0000", 2024-09-18T10:15:18.430 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00081561999999999997 2024-09-18T10:15:18.430 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.430 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.430 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:18.430 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.898942+0000", 2024-09-18T10:15:18.430 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.8019999999999999e-05 2024-09-18T10:15:18.430 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.430 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.430 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:18.430 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.898962+0000", 2024-09-18T10:15:18.430 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0026000000000001e-05 2024-09-18T10:15:18.430 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.430 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.431 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:18.431 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.902525+0000", 2024-09-18T10:15:18.431 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0035628090000000001 2024-09-18T10:15:18.431 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.431 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.431 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:18.431 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.902844+0000", 2024-09-18T10:15:18.431 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00031879400000000001 2024-09-18T10:15:18.431 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.431 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.432 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:18.432 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.905818+0000", 2024-09-18T10:15:18.432 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0029740600000000002 2024-09-18T10:15:18.432 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.432 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.432 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:18.432 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.008488+0000", 2024-09-18T10:15:18.432 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10266977100000001 2024-09-18T10:15:18.432 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.432 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.432 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:18.432 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.209150+0000", 2024-09-18T10:15:18.432 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20066203799999999 2024-09-18T10:15:18.432 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.433 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.433 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:18.433 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.209339+0000", 2024-09-18T10:15:18.433 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00018959099999999999 2024-09-18T10:15:18.433 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:18.433 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:18.433 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:18.433 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.433 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.433 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:347 3.es0 3:7255ddf4:::smithi18925896-39:head [append 0~376832] snapc b=[b,4] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-18T10:15:18.433 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:02.898980+0000", 2024-09-18T10:15:18.433 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 15.473939775, 2024-09-18T10:15:18.433 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.50845237799999998, 2024-09-18T10:15:18.433 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:18.434 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:18.434 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:18.434 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:18.434 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:18.434 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 347 2024-09-18T10:15:18.434 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.434 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:18.434 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.434 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:18.434 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.898980+0000", 2024-09-18T10:15:18.434 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:18.434 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.434 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.434 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:18.435 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.898980+0000", 2024-09-18T10:15:18.435 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:18.435 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.435 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.435 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:18.435 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.898978+0000", 2024-09-18T10:15:18.435 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:18.435 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.435 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.435 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:18.435 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.899378+0000", 2024-09-18T10:15:18.435 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000399465 2024-09-18T10:15:18.435 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.435 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.435 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:18.436 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.899397+0000", 2024-09-18T10:15:18.436 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9219999999999999e-05 2024-09-18T10:15:18.436 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.436 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.436 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:18.436 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.899410+0000", 2024-09-18T10:15:18.436 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2486e-05 2024-09-18T10:15:18.436 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.436 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.436 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:18.436 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.103248+0000", 2024-09-18T10:15:18.436 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20383807200000001 2024-09-18T10:15:18.436 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.436 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.437 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:18.437 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.103447+0000", 2024-09-18T10:15:18.437 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00019970400000000001 2024-09-18T10:15:18.437 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.437 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.437 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:18.437 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.104894+0000", 2024-09-18T10:15:18.437 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001447083 2024-09-18T10:15:18.437 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.437 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.437 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:18.437 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.209378+0000", 2024-09-18T10:15:18.437 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.104483942 2024-09-18T10:15:18.438 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.438 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.438 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:18.438 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.407310+0000", 2024-09-18T10:15:18.438 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.197931406 2024-09-18T10:15:18.438 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.438 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.438 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:18.438 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.407432+0000", 2024-09-18T10:15:18.438 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00012266999999999999 2024-09-18T10:15:18.438 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:18.438 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:18.438 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:18.438 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.439 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.439 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:348 3.es0 3:7255ddf4:::smithi18925896-39:head [setxattr _header (56) in=7b] snapc b=[b,4] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-18T10:15:18.439 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:02.899423+0000", 2024-09-18T10:15:18.439 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 15.473496346999999, 2024-09-18T10:15:18.439 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.50859120400000002, 2024-09-18T10:15:18.439 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:18.439 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:18.439 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:18.439 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:18.439 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:18.439 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 348 2024-09-18T10:15:18.439 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.439 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:18.440 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.440 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:18.440 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.899423+0000", 2024-09-18T10:15:18.440 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:18.440 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.440 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.440 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:18.440 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.899423+0000", 2024-09-18T10:15:18.440 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:18.440 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.440 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.440 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:18.440 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.899422+0000", 2024-09-18T10:15:18.440 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:18.441 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.441 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.441 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:18.441 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.899431+0000", 2024-09-18T10:15:18.441 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.5960000000000004e-06 2024-09-18T10:15:18.441 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.441 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.441 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:18.441 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.899451+0000", 2024-09-18T10:15:18.441 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0089999999999999e-05 2024-09-18T10:15:18.441 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.441 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.441 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:18.441 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.899461+0000", 2024-09-18T10:15:18.442 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0373e-05 2024-09-18T10:15:18.442 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.442 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.442 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:18.442 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.106191+0000", 2024-09-18T10:15:18.442 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20672991700000001 2024-09-18T10:15:18.442 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.442 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.442 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:18.442 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.106370+0000", 2024-09-18T10:15:18.442 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00017880500000000001 2024-09-18T10:15:18.442 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.442 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.443 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:18.443 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.107175+0000", 2024-09-18T10:15:18.443 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00080489699999999995 2024-09-18T10:15:18.443 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.443 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.443 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:18.443 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.209383+0000", 2024-09-18T10:15:18.443 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10220804999999999 2024-09-18T10:15:18.443 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.443 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.443 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:18.443 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.407801+0000", 2024-09-18T10:15:18.443 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.19841824199999999 2024-09-18T10:15:18.443 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.443 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.444 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:18.444 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.408015+0000", 2024-09-18T10:15:18.444 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00021364100000000001 2024-09-18T10:15:18.444 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:18.444 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:18.444 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:18.444 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.444 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.444 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:349 3.es0 3:7255ddf4:::smithi18925896-39:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e47)", 2024-09-18T10:15:18.444 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:02.899474+0000", 2024-09-18T10:15:18.444 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 15.473446136, 2024-09-18T10:15:18.444 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.81294787599999996, 2024-09-18T10:15:18.444 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:18.445 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:15:18.445 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:18.445 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:18.445 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:18.445 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 349 2024-09-18T10:15:18.445 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.445 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:18.445 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.445 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:18.445 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.899474+0000", 2024-09-18T10:15:18.445 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:18.445 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.445 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.445 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:18.446 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.899474+0000", 2024-09-18T10:15:18.446 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:18.446 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.446 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.446 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:18.446 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.899472+0000", 2024-09-18T10:15:18.446 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:18.446 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.446 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.446 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:18.446 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.899480+0000", 2024-09-18T10:15:18.446 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.0129999999999993e-06 2024-09-18T10:15:18.446 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.447 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.447 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:18.447 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.899501+0000", 2024-09-18T10:15:18.447 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0959999999999999e-05 2024-09-18T10:15:18.447 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.447 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.447 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:18.447 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.899513+0000", 2024-09-18T10:15:18.447 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1736000000000001e-05 2024-09-18T10:15:18.447 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.447 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.447 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:18.447 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.208436+0000", 2024-09-18T10:15:18.447 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.30892317800000002 2024-09-18T10:15:18.447 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.448 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.448 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for rw locks", 2024-09-18T10:15:18.448 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.208762+0000", 2024-09-18T10:15:18.448 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00032566099999999998 2024-09-18T10:15:18.448 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.448 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.448 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:18.448 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.408786+0000", 2024-09-18T10:15:18.448 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.200024389 2024-09-18T10:15:18.448 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.448 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.448 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:18.448 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.409088+0000", 2024-09-18T10:15:18.448 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000301272 2024-09-18T10:15:18.449 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.449 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.449 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:18.449 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.712422+0000", 2024-09-18T10:15:18.449 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.303333937 2024-09-18T10:15:18.449 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:18.449 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:18.449 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:18.449 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.449 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.449 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.17s1 57/40 ECSubWrite(tid=106, reqid=client.4632.0:441, at_version=57'22, trim_to=0'0, roll_forward_to=53'18))", 2024-09-18T10:15:18.449 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:12.963189+0000", 2024-09-18T10:15:18.449 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 5.4097304199999998, 2024-09-18T10:15:18.450 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.404840753, 2024-09-18T10:15:18.450 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:18.450 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:18.450 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:18.450 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.450 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:18.450 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:12.963189+0000", 2024-09-18T10:15:18.450 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:18.450 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.450 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.450 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:18.450 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:12.963189+0000", 2024-09-18T10:15:18.450 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:18.450 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.451 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.451 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:18.451 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:12.963189+0000", 2024-09-18T10:15:18.451 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:15:18.451 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.451 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.451 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:18.451 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:12.963511+0000", 2024-09-18T10:15:18.451 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00032195800000000001 2024-09-18T10:15:18.451 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.451 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.451 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:18.452 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:12.963536+0000", 2024-09-18T10:15:18.452 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5795e-05 2024-09-18T10:15:18.452 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.452 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.452 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:18.452 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:12.963548+0000", 2024-09-18T10:15:18.452 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1235e-05 2024-09-18T10:15:18.452 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.452 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.452 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:18.452 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.164194+0000", 2024-09-18T10:15:18.452 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.200646044 2024-09-18T10:15:18.452 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.453 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.453 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:18.453 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.164208+0000", 2024-09-18T10:15:18.453 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4107000000000001e-05 2024-09-18T10:15:18.453 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.453 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.453 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:18.453 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.367962+0000", 2024-09-18T10:15:18.453 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20375375800000001 2024-09-18T10:15:18.453 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.453 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.453 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:18.453 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.368030+0000", 2024-09-18T10:15:18.454 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.8664999999999994e-05 2024-09-18T10:15:18.454 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:18.454 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:18.454 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:18.454 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.454 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.454 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.17s1 57/40 ECSubWrite(tid=107, reqid=client.4632.0:442, at_version=57'23, trim_to=0'0, roll_forward_to=53'18))", 2024-09-18T10:15:18.454 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:12.963562+0000", 2024-09-18T10:15:18.454 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 5.4093577010000002, 2024-09-18T10:15:18.454 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40594254800000001, 2024-09-18T10:15:18.454 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:18.454 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:18.454 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:18.454 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.455 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:18.455 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:12.963562+0000", 2024-09-18T10:15:18.455 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:18.455 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.455 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.455 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:18.455 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:12.963562+0000", 2024-09-18T10:15:18.455 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:18.455 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.455 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.455 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:18.455 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:12.963561+0000", 2024-09-18T10:15:18.455 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:18.455 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.456 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.456 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:18.456 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:12.963869+0000", 2024-09-18T10:15:18.456 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00030785799999999999 2024-09-18T10:15:18.456 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.456 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.456 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:18.456 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:12.963886+0000", 2024-09-18T10:15:18.456 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6582000000000001e-05 2024-09-18T10:15:18.456 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.456 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.456 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:18.456 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.063959+0000", 2024-09-18T10:15:18.457 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10007302699999999 2024-09-18T10:15:18.457 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.457 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.457 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:18.457 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.166049+0000", 2024-09-18T10:15:18.457 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.102089845 2024-09-18T10:15:18.457 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.457 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.457 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:18.457 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.166064+0000", 2024-09-18T10:15:18.457 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5287000000000001e-05 2024-09-18T10:15:18.457 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.457 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.457 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:18.458 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.369452+0000", 2024-09-18T10:15:18.458 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20338777 2024-09-18T10:15:18.458 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.458 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.458 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:18.458 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.369505+0000", 2024-09-18T10:15:18.458 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.2868000000000003e-05 2024-09-18T10:15:18.458 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:18.458 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:18.458 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:18.458 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.458 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.458 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.17s1 57/40 ECSubWrite(tid=108, reqid=client.4632.0:443, at_version=57'24, trim_to=0'0, roll_forward_to=53'18))", 2024-09-18T10:15:18.459 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:13.063971+0000", 2024-09-18T10:15:18.459 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 5.3089483839999998, 2024-09-18T10:15:18.459 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.30555857800000003, 2024-09-18T10:15:18.459 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:18.459 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:18.459 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:18.459 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.459 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:18.459 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.063971+0000", 2024-09-18T10:15:18.459 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:18.459 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.459 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.459 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:18.459 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.063971+0000", 2024-09-18T10:15:18.460 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:18.460 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.460 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.460 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:18.460 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.063971+0000", 2024-09-18T10:15:18.460 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:18.460 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.460 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.460 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:18.460 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.064015+0000", 2024-09-18T10:15:18.460 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3918000000000002e-05 2024-09-18T10:15:18.460 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.460 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.461 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:18.461 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.064025+0000", 2024-09-18T10:15:18.461 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.9029999999999999e-06 2024-09-18T10:15:18.461 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.461 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.461 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:18.461 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.064030+0000", 2024-09-18T10:15:18.461 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.6910000000000002e-06 2024-09-18T10:15:18.461 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.461 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.461 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:18.461 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.166744+0000", 2024-09-18T10:15:18.462 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10271409300000001 2024-09-18T10:15:18.462 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.462 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.462 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:18.462 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.166755+0000", 2024-09-18T10:15:18.462 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0125e-05 2024-09-18T10:15:18.462 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.462 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.462 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:18.462 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.369511+0000", 2024-09-18T10:15:18.462 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20275652299999999 2024-09-18T10:15:18.462 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.462 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.463 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:18.463 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.369530+0000", 2024-09-18T10:15:18.463 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.893e-05 2024-09-18T10:15:18.463 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:18.463 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:18.463 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:18.463 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.463 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.463 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.17s1 57/40 ECSubWrite(tid=109, reqid=client.4632.0:444, at_version=57'25, trim_to=0'0, roll_forward_to=53'18))", 2024-09-18T10:15:18.463 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:13.064037+0000", 2024-09-18T10:15:18.463 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 5.308882874, 2024-09-18T10:15:18.463 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.30551024300000001, 2024-09-18T10:15:18.464 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:18.464 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:18.464 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:18.464 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.464 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:18.464 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.064037+0000", 2024-09-18T10:15:18.464 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:18.464 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.464 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.464 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:18.464 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.064037+0000", 2024-09-18T10:15:18.464 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:18.464 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.465 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.465 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:18.465 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.064037+0000", 2024-09-18T10:15:18.465 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:18.465 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.465 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.465 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:18.465 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.064053+0000", 2024-09-18T10:15:18.465 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5964000000000002e-05 2024-09-18T10:15:18.465 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.465 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.465 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:18.465 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.064062+0000", 2024-09-18T10:15:18.466 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.1479999999999995e-06 2024-09-18T10:15:18.466 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.466 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.466 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:18.466 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.064071+0000", 2024-09-18T10:15:18.466 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.5459999999999997e-06 2024-09-18T10:15:18.466 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.466 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.466 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:18.466 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.266847+0000", 2024-09-18T10:15:18.466 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20277574100000001 2024-09-18T10:15:18.466 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.466 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.467 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:18.467 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.266860+0000", 2024-09-18T10:15:18.467 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3264999999999999e-05 2024-09-18T10:15:18.467 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.467 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.467 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:18.467 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.369533+0000", 2024-09-18T10:15:18.467 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.102672676 2024-09-18T10:15:18.467 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.467 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.467 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:18.467 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.369547+0000", 2024-09-18T10:15:18.467 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4319e-05 2024-09-18T10:15:18.467 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:18.467 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:18.468 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:18.468 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.468 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.468 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:495 3.es0 3:7255ddf4:::smithi18925896-39:head [rmxattr R^`al]TqifatYOiOU]p`ZegfXmNsXVaZcdd]b]YX in=40b,rmxattr TNQYR^jgR`pPaYitmrcliV[dttobncTntcfTlQkt in=40b,rmxattr W_aq[RVsaUpQtX_RpV^Vpqrg`QgQViPeVaRqZSmf in=40b,rmxattr aYfgr^q[\\aPg_aZsakfUfsdhYbra^QXjlkrpdY\\\\ in=40b,rmxattr dgXPi in=5b,rmxattr p]OS_Znl[dNa^qTn`WW[j\\dtgbNac_bijYmg in=36b,rmxattr tZ]grhlaq]kUO[k^d^dr``XNio^^Oaebpj\\[Ytdq in=40b] snapc 19=[19,18,16,12,11,10,f,e] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T10:15:18.468 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:17.889335+0000", 2024-09-18T10:15:18.468 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.48358446799999999, 2024-09-18T10:15:18.468 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.42846536899999998, 2024-09-18T10:15:18.468 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:18.468 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:18.468 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:18.468 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:18.468 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:18.468 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 495 2024-09-18T10:15:18.469 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.469 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:18.469 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.469 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:18.469 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889335+0000", 2024-09-18T10:15:18.469 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:18.469 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.469 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.469 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:18.469 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889335+0000", 2024-09-18T10:15:18.469 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:18.469 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.469 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.469 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:18.469 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889333+0000", 2024-09-18T10:15:18.470 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999976 2024-09-18T10:15:18.470 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.470 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.470 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:18.470 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889351+0000", 2024-09-18T10:15:18.470 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7795000000000002e-05 2024-09-18T10:15:18.470 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.470 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.470 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:18.471 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889383+0000", 2024-09-18T10:15:18.471 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2740999999999997e-05 2024-09-18T10:15:18.471 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.471 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.471 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:18.471 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889396+0000", 2024-09-18T10:15:18.471 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2926e-05 2024-09-18T10:15:18.471 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.471 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.471 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:18.471 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.989725+0000", 2024-09-18T10:15:18.471 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10032845999999999 2024-09-18T10:15:18.471 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.471 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.472 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "op must wait for map", 2024-09-18T10:15:18.472 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.989751+0000", 2024-09-18T10:15:18.472 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5819000000000001e-05 2024-09-18T10:15:18.472 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.472 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.472 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:18.472 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.999619+0000", 2024-09-18T10:15:18.472 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0098683920000000001 2024-09-18T10:15:18.472 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.472 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.472 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:18.472 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.999776+0000", 2024-09-18T10:15:18.472 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000157336 2024-09-18T10:15:18.473 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.473 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.473 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:18.473 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.001374+0000", 2024-09-18T10:15:18.473 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0015982419999999999 2024-09-18T10:15:18.473 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.473 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.473 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:18.473 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.003908+0000", 2024-09-18T10:15:18.473 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0025331870000000001 2024-09-18T10:15:18.473 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.473 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.473 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:18.473 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.317708+0000", 2024-09-18T10:15:18.474 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.31380064800000002 2024-09-18T10:15:18.474 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.474 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.474 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:18.474 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.317801+0000", 2024-09-18T10:15:18.474 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.2427000000000001e-05 2024-09-18T10:15:18.474 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:18.474 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:18.474 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:18.474 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:18.474 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:18.474 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-18T10:15:18.476 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_ops_in_flight 2024-09-18T10:15:18.477 INFO:tasks.rados.rados.0.smithi189.stdout:202: expect (ObjNum 41 snap 0 seq_num 41) 2024-09-18T10:15:18.477 INFO:tasks.rados.rados.0.smithi189.stdout:204: finishing write tid 1 to smithi18925896-37 2024-09-18T10:15:18.477 INFO:tasks.ceph.osd.15.smithi189.stderr:2024-09-18T10:15:18.470+0000 7fd0e725e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 22401) UID: 0 2024-09-18T10:15:18.508 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 38 v 62 (ObjNum 60 snap 5 seq_num 60) dirty exists 2024-09-18T10:15:18.524 INFO:tasks.rados.rados.0.smithi189.stdout:204: finishing write tid 2 to smithi18925896-37 2024-09-18T10:15:18.577 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-09-18T10:15:18.571+0000 7fdfa8698700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22087) UID: 0 2024-09-18T10:15:18.615 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-18T10:15:18.615 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-18T10:15:18.616 INFO:teuthology.orchestra.run.smithi189.stdout: "num_ops": 0 2024-09-18T10:15:18.616 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-18T10:15:18.625 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_blocked_ops 2024-09-18T10:15:18.626 INFO:tasks.rados.rados.0.smithi189.stdout:204: finishing write tid 3 to smithi18925896-37 2024-09-18T10:15:18.626 INFO:tasks.rados.rados.0.smithi189.stdout:204: finishing write tid 4 to smithi18925896-37 2024-09-18T10:15:18.678 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-18T10:15:18.674+0000 7f14dcc5b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22398) UID: 0 2024-09-18T10:15:18.720 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-18T10:15:18.721 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-18T10:15:18.721 INFO:teuthology.orchestra.run.smithi189.stdout: "complaint_time": 30, 2024-09-18T10:15:18.721 INFO:teuthology.orchestra.run.smithi189.stdout: "num_blocked_ops": 0 2024-09-18T10:15:18.721 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-18T10:15:18.730 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_historic_ops 2024-09-18T10:15:18.779 INFO:tasks.ceph.osd.12.smithi029.stderr:2024-09-18T10:15:18.774+0000 7f8e4e627700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 22190) UID: 0 2024-09-18T10:15:18.836 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-18T10:15:18.836 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 20, 2024-09-18T10:15:18.837 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 600, 2024-09-18T10:15:18.837 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [ 2024-09-18T10:15:18.837 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.837 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4257.0:19 1.0 e24/23 1:30306672:devicehealth::main.db.0000000000000000:head v 24'9, mlcod=24'7)", 2024-09-18T10:15:18.837 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:33.346192+0000", 2024-09-18T10:15:18.837 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 45.486193798000002, 2024-09-18T10:15:18.837 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20178458699999999, 2024-09-18T10:15:18.837 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:18.837 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:18.837 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:18.837 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.838 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:18.838 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:33.346192+0000", 2024-09-18T10:15:18.838 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:18.838 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.838 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.838 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:18.839 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:33.346192+0000", 2024-09-18T10:15:18.839 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:18.839 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.839 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.839 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:18.839 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:33.346190+0000", 2024-09-18T10:15:18.839 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:18.839 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.839 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.839 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:18.839 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:33.346206+0000", 2024-09-18T10:15:18.839 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6257e-05 2024-09-18T10:15:18.840 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.840 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.840 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:18.840 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:33.346237+0000", 2024-09-18T10:15:18.840 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0471999999999999e-05 2024-09-18T10:15:18.840 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.840 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.840 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:18.840 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:33.446377+0000", 2024-09-18T10:15:18.840 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10014044499999999 2024-09-18T10:15:18.841 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.841 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.841 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:18.841 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:33.546722+0000", 2024-09-18T10:15:18.841 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100344593 2024-09-18T10:15:18.841 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.841 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.841 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:18.842 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:33.546787+0000", 2024-09-18T10:15:18.842 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.5347999999999998e-05 2024-09-18T10:15:18.842 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.842 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.842 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:18.842 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:33.547891+0000", 2024-09-18T10:15:18.842 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001104379 2024-09-18T10:15:18.842 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.842 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.842 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:18.842 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:33.547976+0000", 2024-09-18T10:15:18.842 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.5023000000000002e-05 2024-09-18T10:15:18.842 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:18.842 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:18.843 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:18.843 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.843 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.843 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4257.0:48 1.0 e26/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 26'30, mlcod=26'22)", 2024-09-18T10:15:18.843 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:35.294144+0000", 2024-09-18T10:15:18.843 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 43.538241655, 2024-09-18T10:15:18.843 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20346468400000001, 2024-09-18T10:15:18.843 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:18.843 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:18.843 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:18.843 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.843 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:18.843 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:35.294144+0000", 2024-09-18T10:15:18.844 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:18.844 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.844 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.844 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:18.844 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:35.294144+0000", 2024-09-18T10:15:18.844 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:18.844 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.844 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.844 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:18.844 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:35.294142+0000", 2024-09-18T10:15:18.844 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:18.844 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.845 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.845 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:18.845 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:35.294156+0000", 2024-09-18T10:15:18.845 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4059e-05 2024-09-18T10:15:18.845 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.845 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.846 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:18.846 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:35.294209+0000", 2024-09-18T10:15:18.846 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.2164999999999999e-05 2024-09-18T10:15:18.846 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.846 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.846 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:18.846 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:35.394305+0000", 2024-09-18T10:15:18.846 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10009605000000001 2024-09-18T10:15:18.847 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.847 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.847 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:18.847 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:35.394422+0000", 2024-09-18T10:15:18.847 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000117093 2024-09-18T10:15:18.847 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.847 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.847 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:18.847 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:35.394460+0000", 2024-09-18T10:15:18.847 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8037999999999997e-05 2024-09-18T10:15:18.847 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.847 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.847 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:18.847 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:35.497437+0000", 2024-09-18T10:15:18.848 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.102977206 2024-09-18T10:15:18.848 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.848 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.848 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:18.848 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:35.497609+0000", 2024-09-18T10:15:18.848 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00017165299999999999 2024-09-18T10:15:18.848 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:18.848 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:18.848 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:18.848 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.848 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.848 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4257.0:96 1.0 e29/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 29'70, mlcod=29'68)", 2024-09-18T10:15:18.848 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:39.427232+0000", 2024-09-18T10:15:18.849 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 39.405153531000003, 2024-09-18T10:15:18.849 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20408627300000001, 2024-09-18T10:15:18.849 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:18.849 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:18.849 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:18.849 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.849 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:18.849 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.427232+0000", 2024-09-18T10:15:18.850 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:18.850 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.850 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.850 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:18.850 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.427232+0000", 2024-09-18T10:15:18.850 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:18.850 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.851 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.851 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:18.851 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.427230+0000", 2024-09-18T10:15:18.851 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:18.851 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.851 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.852 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:18.852 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.427399+0000", 2024-09-18T10:15:18.852 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000168787 2024-09-18T10:15:18.852 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.852 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.852 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:18.853 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.427431+0000", 2024-09-18T10:15:18.853 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2372000000000001e-05 2024-09-18T10:15:18.853 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.853 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.853 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:18.853 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.427444+0000", 2024-09-18T10:15:18.853 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2303000000000001e-05 2024-09-18T10:15:18.853 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.853 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.853 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:18.854 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.427580+0000", 2024-09-18T10:15:18.854 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000136019 2024-09-18T10:15:18.854 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.854 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.854 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:18.854 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.427622+0000", 2024-09-18T10:15:18.854 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.2551e-05 2024-09-18T10:15:18.854 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.854 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.854 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:18.854 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.631238+0000", 2024-09-18T10:15:18.854 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20361580100000001 2024-09-18T10:15:18.854 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.855 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.855 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:18.855 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.631318+0000", 2024-09-18T10:15:18.855 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.0370000000000005e-05 2024-09-18T10:15:18.855 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:18.855 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:18.855 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:18.855 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.855 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.855 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4257.0:97 1.0 e29/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 29'71, mlcod=29'68)", 2024-09-18T10:15:18.855 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:39.428800+0000", 2024-09-18T10:15:18.855 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 39.403585589000002, 2024-09-18T10:15:18.855 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.202566529, 2024-09-18T10:15:18.855 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:18.856 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:18.856 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:18.856 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.856 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:18.856 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.428800+0000", 2024-09-18T10:15:18.856 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:18.856 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.856 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.856 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:18.856 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.428800+0000", 2024-09-18T10:15:18.856 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:18.856 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.856 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.857 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:18.857 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.428798+0000", 2024-09-18T10:15:18.857 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:18.857 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.857 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.857 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:18.857 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.428814+0000", 2024-09-18T10:15:18.857 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5262999999999999e-05 2024-09-18T10:15:18.857 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.857 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.857 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:18.857 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.428836+0000", 2024-09-18T10:15:18.857 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.2096e-05 2024-09-18T10:15:18.857 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.858 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.858 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:18.858 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.428847+0000", 2024-09-18T10:15:18.858 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1272e-05 2024-09-18T10:15:18.858 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.858 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.858 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:18.858 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.529081+0000", 2024-09-18T10:15:18.858 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100233853 2024-09-18T10:15:18.858 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.858 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.858 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:18.858 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.529115+0000", 2024-09-18T10:15:18.858 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3862000000000002e-05 2024-09-18T10:15:18.859 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.859 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.859 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:18.859 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.631329+0000", 2024-09-18T10:15:18.859 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.102214465 2024-09-18T10:15:18.859 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.859 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.859 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:18.859 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.631367+0000", 2024-09-18T10:15:18.859 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7450999999999998e-05 2024-09-18T10:15:18.859 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:18.859 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:18.859 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:18.859 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.860 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.860 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4257.0:98 1.0 e29/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 29'72, mlcod=29'68)", 2024-09-18T10:15:18.860 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:39.429995+0000", 2024-09-18T10:15:18.860 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 39.402390365999999, 2024-09-18T10:15:18.860 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20141884500000001, 2024-09-18T10:15:18.860 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:18.860 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:18.860 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:18.860 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.860 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:18.860 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.429995+0000", 2024-09-18T10:15:18.860 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:18.860 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.861 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.861 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:18.861 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.429995+0000", 2024-09-18T10:15:18.861 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:18.861 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.861 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.861 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:18.861 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.429994+0000", 2024-09-18T10:15:18.861 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:18.861 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.861 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.861 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:18.861 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.430008+0000", 2024-09-18T10:15:18.862 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4185999999999999e-05 2024-09-18T10:15:18.862 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.862 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.862 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:18.862 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.430036+0000", 2024-09-18T10:15:18.862 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7878999999999999e-05 2024-09-18T10:15:18.862 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.863 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.863 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:18.863 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.430048+0000", 2024-09-18T10:15:18.863 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1725999999999999e-05 2024-09-18T10:15:18.863 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.863 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.863 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:18.863 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.629731+0000", 2024-09-18T10:15:18.863 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.19968341100000001 2024-09-18T10:15:18.864 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.864 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.864 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:18.864 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.629764+0000", 2024-09-18T10:15:18.864 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3572999999999999e-05 2024-09-18T10:15:18.864 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.864 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.864 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:18.864 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.631372+0000", 2024-09-18T10:15:18.864 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0016074279999999999 2024-09-18T10:15:18.864 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.864 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.864 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:18.865 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.631414+0000", 2024-09-18T10:15:18.865 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.2191999999999999e-05 2024-09-18T10:15:18.865 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:18.865 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:18.865 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:18.865 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.865 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.865 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4257.0:99 1.0 e29/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 29'73, mlcod=29'68)", 2024-09-18T10:15:18.865 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:39.431146+0000", 2024-09-18T10:15:18.865 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 39.401239642, 2024-09-18T10:15:18.865 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.200928938, 2024-09-18T10:15:18.865 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:18.866 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:18.866 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:18.866 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.866 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:18.866 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.431146+0000", 2024-09-18T10:15:18.866 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:18.866 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.866 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.866 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:18.866 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.431146+0000", 2024-09-18T10:15:18.866 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:18.866 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.866 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.866 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:18.867 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.431144+0000", 2024-09-18T10:15:18.867 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:18.867 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.867 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.867 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:18.867 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.431218+0000", 2024-09-18T10:15:18.867 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.3577000000000004e-05 2024-09-18T10:15:18.867 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.867 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.867 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:18.867 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.431250+0000", 2024-09-18T10:15:18.867 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2268999999999999e-05 2024-09-18T10:15:18.867 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.867 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.868 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:18.868 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.431262+0000", 2024-09-18T10:15:18.868 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2053e-05 2024-09-18T10:15:18.868 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.868 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.868 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:18.868 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.630328+0000", 2024-09-18T10:15:18.868 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.19906532900000001 2024-09-18T10:15:18.868 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.868 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.868 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:18.868 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.630359+0000", 2024-09-18T10:15:18.868 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1606000000000002e-05 2024-09-18T10:15:18.868 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.869 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.869 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:18.869 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.632015+0000", 2024-09-18T10:15:18.869 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0016557900000000001 2024-09-18T10:15:18.869 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.869 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.869 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:18.869 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.632075+0000", 2024-09-18T10:15:18.869 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9904000000000001e-05 2024-09-18T10:15:18.869 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:18.869 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:18.869 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:18.869 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.870 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.870 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4257.0:100 1.0 e29/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 29'74, mlcod=29'68)", 2024-09-18T10:15:18.870 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:39.432631+0000", 2024-09-18T10:15:18.870 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 39.399754553000001, 2024-09-18T10:15:18.870 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20014804899999999, 2024-09-18T10:15:18.870 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:18.870 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:18.870 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:18.870 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.870 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:18.870 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.432631+0000", 2024-09-18T10:15:18.870 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:18.870 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.871 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.871 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:18.871 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.432631+0000", 2024-09-18T10:15:18.871 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:18.871 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.871 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.871 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:18.871 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.432630+0000", 2024-09-18T10:15:18.871 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:18.871 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.871 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.872 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:18.872 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.432643+0000", 2024-09-18T10:15:18.872 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3525e-05 2024-09-18T10:15:18.872 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.872 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.872 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:18.872 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.432674+0000", 2024-09-18T10:15:18.872 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1183000000000003e-05 2024-09-18T10:15:18.872 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.872 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.872 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:18.872 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.432690+0000", 2024-09-18T10:15:18.872 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5665999999999999e-05 2024-09-18T10:15:18.872 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.873 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.873 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:18.873 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.631457+0000", 2024-09-18T10:15:18.873 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.19876732899999999 2024-09-18T10:15:18.873 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.873 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.873 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:18.873 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.631488+0000", 2024-09-18T10:15:18.873 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1059000000000002e-05 2024-09-18T10:15:18.873 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.873 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.873 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:18.873 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.632717+0000", 2024-09-18T10:15:18.873 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0012285219999999999 2024-09-18T10:15:18.874 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.874 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.874 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:18.874 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.632779+0000", 2024-09-18T10:15:18.874 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.2328000000000004e-05 2024-09-18T10:15:18.874 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:18.874 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:18.874 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:18.874 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.874 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.874 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4257.0:103 1.0 e29/23 1:30306672:devicehealth::main.db.0000000000000000:head v 29'77, mlcod=29'73)", 2024-09-18T10:15:18.874 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:40.036738+0000", 2024-09-18T10:15:18.874 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 38.795647582999997, 2024-09-18T10:15:18.875 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20132596899999999, 2024-09-18T10:15:18.875 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:18.875 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:18.875 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:18.875 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.875 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:18.875 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.036738+0000", 2024-09-18T10:15:18.875 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:18.875 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.875 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.875 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:18.875 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.036738+0000", 2024-09-18T10:15:18.875 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:18.875 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.876 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.876 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:18.876 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.036737+0000", 2024-09-18T10:15:18.876 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:18.876 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.876 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.876 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:18.876 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.036804+0000", 2024-09-18T10:15:18.876 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.7178e-05 2024-09-18T10:15:18.876 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.876 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.876 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:18.876 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.036821+0000", 2024-09-18T10:15:18.877 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7153e-05 2024-09-18T10:15:18.877 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.877 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.877 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:18.877 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.136954+0000", 2024-09-18T10:15:18.877 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100132572 2024-09-18T10:15:18.877 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.877 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.877 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:18.877 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.137090+0000", 2024-09-18T10:15:18.877 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000136332 2024-09-18T10:15:18.877 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.877 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.878 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:18.878 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.137124+0000", 2024-09-18T10:15:18.878 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3458999999999999e-05 2024-09-18T10:15:18.878 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.878 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.878 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:18.878 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.237999+0000", 2024-09-18T10:15:18.878 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100875385 2024-09-18T10:15:18.878 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.878 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.878 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:18.878 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.238064+0000", 2024-09-18T10:15:18.878 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.5086999999999995e-05 2024-09-18T10:15:18.878 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:18.879 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:18.879 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:18.879 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.879 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.879 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4257.0:113 1.0 e29/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 29'83, mlcod=29'80)", 2024-09-18T10:15:18.879 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:40.801786+0000", 2024-09-18T10:15:18.879 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 38.030600079000003, 2024-09-18T10:15:18.879 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.203408961, 2024-09-18T10:15:18.879 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:18.879 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:18.879 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:18.879 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.879 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:18.879 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.801786+0000", 2024-09-18T10:15:18.880 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:18.880 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.880 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.880 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:18.880 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.801786+0000", 2024-09-18T10:15:18.880 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:18.880 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.880 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.880 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:18.880 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.801784+0000", 2024-09-18T10:15:18.880 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:18.880 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.880 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.880 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:18.881 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.801801+0000", 2024-09-18T10:15:18.881 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7223000000000001e-05 2024-09-18T10:15:18.881 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.881 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.881 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:18.881 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.801823+0000", 2024-09-18T10:15:18.881 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1469000000000001e-05 2024-09-18T10:15:18.881 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.881 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.881 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:18.881 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.801835+0000", 2024-09-18T10:15:18.881 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2033e-05 2024-09-18T10:15:18.882 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.882 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.882 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:18.882 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.902049+0000", 2024-09-18T10:15:18.882 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100213659 2024-09-18T10:15:18.882 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.882 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.882 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:18.882 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.902082+0000", 2024-09-18T10:15:18.882 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3668999999999999e-05 2024-09-18T10:15:18.882 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.883 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.883 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:18.883 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.005131+0000", 2024-09-18T10:15:18.883 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.103048379 2024-09-18T10:15:18.883 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.883 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.883 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:18.883 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.005195+0000", 2024-09-18T10:15:18.883 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.3805000000000003e-05 2024-09-18T10:15:18.883 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:18.883 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:18.883 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:18.883 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.883 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.884 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4257.0:114 1.0 e29/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 29'84, mlcod=29'80)", 2024-09-18T10:15:18.884 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:40.802657+0000", 2024-09-18T10:15:18.884 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 38.029728945000002, 2024-09-18T10:15:18.884 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20256802600000001, 2024-09-18T10:15:18.884 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:18.884 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:18.884 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:18.884 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.884 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:18.884 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.802657+0000", 2024-09-18T10:15:18.884 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:18.884 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.884 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.884 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:18.885 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.802657+0000", 2024-09-18T10:15:18.885 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:18.885 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.885 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.885 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:18.885 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.802656+0000", 2024-09-18T10:15:18.885 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:15:18.885 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.885 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.885 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:18.885 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.802824+0000", 2024-09-18T10:15:18.885 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000168784 2024-09-18T10:15:18.885 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.885 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.885 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:18.886 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.802844+0000", 2024-09-18T10:15:18.886 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0058999999999999e-05 2024-09-18T10:15:18.886 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.886 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.886 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:18.886 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.802857+0000", 2024-09-18T10:15:18.886 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2163e-05 2024-09-18T10:15:18.886 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.886 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.886 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:18.886 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.902531+0000", 2024-09-18T10:15:18.886 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.099674591000000007 2024-09-18T10:15:18.886 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.887 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.887 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:18.887 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.902559+0000", 2024-09-18T10:15:18.887 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7483000000000001e-05 2024-09-18T10:15:18.887 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.887 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.887 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:18.887 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.005198+0000", 2024-09-18T10:15:18.887 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.102638853 2024-09-18T10:15:18.887 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.887 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.887 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:18.887 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.005225+0000", 2024-09-18T10:15:18.888 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7206e-05 2024-09-18T10:15:18.888 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:18.888 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:18.888 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:18.888 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.888 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.888 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4257.0:115 1.0 e29/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 29'85, mlcod=29'80)", 2024-09-18T10:15:18.888 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:40.803806+0000", 2024-09-18T10:15:18.888 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 38.028579557999997, 2024-09-18T10:15:18.888 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.201443711, 2024-09-18T10:15:18.888 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:18.888 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:18.889 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:18.889 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.889 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:18.889 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.803806+0000", 2024-09-18T10:15:18.889 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:18.889 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.889 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.889 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:18.889 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.803806+0000", 2024-09-18T10:15:18.889 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:18.889 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.889 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.890 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:18.890 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.803804+0000", 2024-09-18T10:15:18.890 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:18.890 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.890 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.890 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:18.891 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.803818+0000", 2024-09-18T10:15:18.891 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3477e-05 2024-09-18T10:15:18.891 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.891 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.891 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:18.891 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.803851+0000", 2024-09-18T10:15:18.891 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3086000000000001e-05 2024-09-18T10:15:18.891 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.892 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.892 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:18.892 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.803867+0000", 2024-09-18T10:15:18.892 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5885000000000001e-05 2024-09-18T10:15:18.892 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.892 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.892 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:18.892 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.903206+0000", 2024-09-18T10:15:18.892 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.099338990000000002 2024-09-18T10:15:18.892 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.892 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.893 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:18.893 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.903253+0000", 2024-09-18T10:15:18.893 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.7419000000000001e-05 2024-09-18T10:15:18.893 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.893 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.893 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:18.893 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.005227+0000", 2024-09-18T10:15:18.894 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.101973199 2024-09-18T10:15:18.894 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.894 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.894 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:18.894 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.005250+0000", 2024-09-18T10:15:18.894 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3261999999999999e-05 2024-09-18T10:15:18.894 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:18.894 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:18.895 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:18.895 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.895 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.895 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4257.0:116 1.0 e29/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 29'86, mlcod=29'80)", 2024-09-18T10:15:18.895 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:40.804730+0000", 2024-09-18T10:15:18.895 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 38.027655578999997, 2024-09-18T10:15:18.895 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20068456000000001, 2024-09-18T10:15:18.895 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:18.895 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:18.895 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:18.896 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.896 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:18.896 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.804730+0000", 2024-09-18T10:15:18.896 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:18.896 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.896 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.896 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:18.896 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.804730+0000", 2024-09-18T10:15:18.897 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:18.897 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.897 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.897 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:18.897 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.804729+0000", 2024-09-18T10:15:18.897 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:18.897 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.897 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.897 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:18.897 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.804739+0000", 2024-09-18T10:15:18.898 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0669000000000001e-05 2024-09-18T10:15:18.898 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.898 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.898 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:18.898 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.804762+0000", 2024-09-18T10:15:18.898 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3062999999999999e-05 2024-09-18T10:15:18.898 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.898 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.898 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:18.898 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.804776+0000", 2024-09-18T10:15:18.898 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3273e-05 2024-09-18T10:15:18.898 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.898 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.899 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:18.899 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.003889+0000", 2024-09-18T10:15:18.899 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.19911304099999999 2024-09-18T10:15:18.899 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.899 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.899 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:18.899 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.003922+0000", 2024-09-18T10:15:18.899 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3621999999999998e-05 2024-09-18T10:15:18.899 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.899 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.899 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:18.900 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.005347+0000", 2024-09-18T10:15:18.900 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0014251050000000001 2024-09-18T10:15:18.900 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.900 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.900 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:18.900 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.005415+0000", 2024-09-18T10:15:18.900 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.7154000000000001e-05 2024-09-18T10:15:18.900 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:18.900 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:18.900 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:18.900 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.900 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.901 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4257.0:117 1.0 e29/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 29'87, mlcod=29'80)", 2024-09-18T10:15:18.901 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:40.805727+0000", 2024-09-18T10:15:18.901 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 38.026658320000003, 2024-09-18T10:15:18.901 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.200946191, 2024-09-18T10:15:18.901 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:18.901 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:18.901 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:18.901 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.901 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:18.901 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.805727+0000", 2024-09-18T10:15:18.901 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:18.901 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.901 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.902 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:18.902 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.805727+0000", 2024-09-18T10:15:18.902 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:18.902 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.902 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.902 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:18.902 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.805726+0000", 2024-09-18T10:15:18.902 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:18.902 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.902 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.902 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:18.902 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.805783+0000", 2024-09-18T10:15:18.903 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.7068000000000003e-05 2024-09-18T10:15:18.903 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.903 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.903 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:18.903 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.805817+0000", 2024-09-18T10:15:18.904 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4471999999999998e-05 2024-09-18T10:15:18.904 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.904 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.904 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:18.904 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.805833+0000", 2024-09-18T10:15:18.904 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6313000000000001e-05 2024-09-18T10:15:18.904 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.904 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.904 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:18.904 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.004481+0000", 2024-09-18T10:15:18.904 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.198647823 2024-09-18T10:15:18.904 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.904 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.904 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:18.905 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.004520+0000", 2024-09-18T10:15:18.905 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8992000000000002e-05 2024-09-18T10:15:18.905 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.905 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.905 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:18.905 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.006585+0000", 2024-09-18T10:15:18.905 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0020647389999999999 2024-09-18T10:15:18.905 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.905 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.905 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:18.905 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.006673+0000", 2024-09-18T10:15:18.905 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.8443999999999995e-05 2024-09-18T10:15:18.906 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:18.906 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:18.906 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:18.906 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.906 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.906 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4257.0:128 1.0 e29/23 1:30306672:devicehealth::main.db.0000000000000000:head v 29'98, mlcod=29'95)", 2024-09-18T10:15:18.906 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:41.222806+0000", 2024-09-18T10:15:18.906 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 37.609579678999999, 2024-09-18T10:15:18.906 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20200015099999999, 2024-09-18T10:15:18.906 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:18.906 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:18.906 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:18.906 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.906 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:18.906 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.222806+0000", 2024-09-18T10:15:18.907 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:18.907 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.907 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.907 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:18.907 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.222806+0000", 2024-09-18T10:15:18.907 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:18.907 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.907 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.907 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:18.907 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.222804+0000", 2024-09-18T10:15:18.907 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:18.907 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.907 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.907 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:18.908 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.222885+0000", 2024-09-18T10:15:18.908 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.0140000000000002e-05 2024-09-18T10:15:18.908 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.908 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.908 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:18.908 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.222915+0000", 2024-09-18T10:15:18.908 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0168999999999999e-05 2024-09-18T10:15:18.908 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.908 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.908 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:18.908 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.323060+0000", 2024-09-18T10:15:18.908 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100145475 2024-09-18T10:15:18.908 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.909 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.909 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:18.909 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.423304+0000", 2024-09-18T10:15:18.909 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100243924 2024-09-18T10:15:18.909 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.909 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.909 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:18.909 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.423338+0000", 2024-09-18T10:15:18.909 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3819000000000003e-05 2024-09-18T10:15:18.909 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.909 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.909 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:18.909 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.424724+0000", 2024-09-18T10:15:18.909 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0013862690000000001 2024-09-18T10:15:18.909 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.910 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.910 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:18.910 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.424806+0000", 2024-09-18T10:15:18.910 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.1901000000000001e-05 2024-09-18T10:15:18.910 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:18.910 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:18.910 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:18.910 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.910 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.910 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.4s1 41/40 ECSubWrite(tid=17, reqid=osd.8.0:17, at_version=41'10, trim_to=0'0, roll_forward_to=0'0))", 2024-09-18T10:15:18.910 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:57.445149+0000", 2024-09-18T10:15:18.910 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 21.387237117000002, 2024-09-18T10:15:18.910 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20394425099999999, 2024-09-18T10:15:18.911 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:18.911 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:18.911 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:18.911 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.911 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:18.911 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.445149+0000", 2024-09-18T10:15:18.911 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:18.911 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.911 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.911 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:18.911 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.445149+0000", 2024-09-18T10:15:18.911 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:18.911 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.911 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.912 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:18.912 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.445147+0000", 2024-09-18T10:15:18.912 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:18.912 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.912 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.912 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:18.912 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.445221+0000", 2024-09-18T10:15:18.912 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.4140999999999994e-05 2024-09-18T10:15:18.912 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.912 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.912 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:18.912 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.445260+0000", 2024-09-18T10:15:18.912 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8998999999999997e-05 2024-09-18T10:15:18.912 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.913 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.913 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:18.913 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.545386+0000", 2024-09-18T10:15:18.913 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10012621300000001 2024-09-18T10:15:18.913 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.913 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.913 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:18.913 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.645963+0000", 2024-09-18T10:15:18.913 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10057662000000001 2024-09-18T10:15:18.913 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.913 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.913 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:18.913 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.645988+0000", 2024-09-18T10:15:18.913 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5449e-05 2024-09-18T10:15:18.914 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.914 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.914 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:18.914 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.649028+0000", 2024-09-18T10:15:18.914 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0030390840000000001 2024-09-18T10:15:18.914 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.914 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.914 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:18.914 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.649093+0000", 2024-09-18T10:15:18.914 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.5258000000000004e-05 2024-09-18T10:15:18.914 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:18.914 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:18.914 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:18.914 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.914 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.915 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.6s6 41/40 ECSubWrite(tid=31, reqid=client.4632.0:249, at_version=41'13, trim_to=0'0, roll_forward_to=0'0))", 2024-09-18T10:15:18.915 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:57.550336+0000", 2024-09-18T10:15:18.915 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 21.282050078000001, 2024-09-18T10:15:18.915 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.203590574, 2024-09-18T10:15:18.915 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:18.915 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:18.915 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:18.915 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.915 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:18.915 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.550336+0000", 2024-09-18T10:15:18.915 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:18.915 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.915 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.915 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:18.916 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.550336+0000", 2024-09-18T10:15:18.916 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:18.916 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.916 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.916 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:18.916 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.550329+0000", 2024-09-18T10:15:18.916 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999933 2024-09-18T10:15:18.916 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.916 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.916 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:18.916 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.550932+0000", 2024-09-18T10:15:18.916 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00060276999999999998 2024-09-18T10:15:18.916 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.917 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.917 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:18.917 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.550955+0000", 2024-09-18T10:15:18.917 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.2884999999999999e-05 2024-09-18T10:15:18.917 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.917 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.917 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:18.917 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.651052+0000", 2024-09-18T10:15:18.917 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100097174 2024-09-18T10:15:18.917 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.917 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.917 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:18.917 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.651160+0000", 2024-09-18T10:15:18.917 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000108483 2024-09-18T10:15:18.918 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.918 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.918 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:18.918 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.651172+0000", 2024-09-18T10:15:18.918 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1790999999999999e-05 2024-09-18T10:15:18.918 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.918 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.918 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:18.918 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.753875+0000", 2024-09-18T10:15:18.918 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10270337 2024-09-18T10:15:18.918 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.918 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.918 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:18.919 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.753926+0000", 2024-09-18T10:15:18.919 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0767999999999999e-05 2024-09-18T10:15:18.919 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:18.919 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:18.919 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:18.919 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.919 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.919 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.6s6 41/40 ECSubWrite(tid=33, reqid=client.4632.0:251, at_version=41'15, trim_to=0'0, roll_forward_to=0'0))", 2024-09-18T10:15:18.919 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:57.651390+0000", 2024-09-18T10:15:18.919 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 21.180995465999999, 2024-09-18T10:15:18.919 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.105666251, 2024-09-18T10:15:18.919 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:18.919 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:18.920 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:18.920 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.920 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:18.920 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.651390+0000", 2024-09-18T10:15:18.920 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:18.920 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.920 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.920 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:18.920 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.651390+0000", 2024-09-18T10:15:18.920 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:18.920 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.920 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.920 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:18.920 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.651390+0000", 2024-09-18T10:15:18.921 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:18.921 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.921 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.921 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:18.921 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.651405+0000", 2024-09-18T10:15:18.921 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5512e-05 2024-09-18T10:15:18.921 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.921 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.921 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:18.921 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.651413+0000", 2024-09-18T10:15:18.921 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.9249999999999995e-06 2024-09-18T10:15:18.921 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.921 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.922 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:18.922 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.651418+0000", 2024-09-18T10:15:18.922 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0799999999999996e-06 2024-09-18T10:15:18.922 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.922 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.922 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:18.922 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.753439+0000", 2024-09-18T10:15:18.922 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.102020576 2024-09-18T10:15:18.922 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.922 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.922 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:18.922 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.753470+0000", 2024-09-18T10:15:18.922 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1296e-05 2024-09-18T10:15:18.922 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.923 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.923 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:18.923 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.757028+0000", 2024-09-18T10:15:18.923 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0035574510000000001 2024-09-18T10:15:18.923 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.923 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.923 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:18.923 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.757056+0000", 2024-09-18T10:15:18.923 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.8894000000000002e-05 2024-09-18T10:15:18.923 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:18.923 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:18.923 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:18.923 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.923 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.924 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.8s4 42/40 ECSubWrite(tid=133, reqid=osd.6.0:132, at_version=0'0, trim_to=0'0, roll_forward_to=41'13))", 2024-09-18T10:15:18.924 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:58.049763+0000", 2024-09-18T10:15:18.924 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 20.782622342, 2024-09-18T10:15:18.924 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.21446268399999999, 2024-09-18T10:15:18.924 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:18.924 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:18.924 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:18.924 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.924 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:18.924 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:58.049763+0000", 2024-09-18T10:15:18.924 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:18.924 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.924 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.924 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:18.925 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:58.049763+0000", 2024-09-18T10:15:18.925 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:18.925 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.925 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.925 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:18.925 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:58.049763+0000", 2024-09-18T10:15:18.925 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:18.925 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.925 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.925 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:18.925 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:58.049787+0000", 2024-09-18T10:15:18.925 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3819e-05 2024-09-18T10:15:18.925 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.925 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.926 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:18.926 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:58.049817+0000", 2024-09-18T10:15:18.926 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0749e-05 2024-09-18T10:15:18.926 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.926 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.926 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:18.926 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:58.049836+0000", 2024-09-18T10:15:18.926 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9182999999999999e-05 2024-09-18T10:15:18.926 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.926 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.926 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:18.926 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:58.163409+0000", 2024-09-18T10:15:18.926 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.11357204899999999 2024-09-18T10:15:18.926 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.927 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.927 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:18.927 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:58.163424+0000", 2024-09-18T10:15:18.927 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5403000000000001e-05 2024-09-18T10:15:18.927 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.927 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.927 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:18.927 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:58.264161+0000", 2024-09-18T10:15:18.927 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100736942 2024-09-18T10:15:18.927 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.927 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.927 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:18.927 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:58.264226+0000", 2024-09-18T10:15:18.928 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.5085000000000005e-05 2024-09-18T10:15:18.928 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:18.928 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:18.928 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:18.928 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.928 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.928 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.ds3 47/32 ECSubWrite(tid=56, reqid=client.4632.0:333, at_version=47'9, trim_to=0'0, roll_forward_to=41'6))", 2024-09-18T10:15:18.928 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:02.801124+0000", 2024-09-18T10:15:18.928 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 16.031261184000002, 2024-09-18T10:15:18.928 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20185083000000001, 2024-09-18T10:15:18.928 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:18.928 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:18.928 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:18.928 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.929 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:18.929 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.801124+0000", 2024-09-18T10:15:18.929 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:18.929 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.929 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.929 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:18.929 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.801124+0000", 2024-09-18T10:15:18.929 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:18.929 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.929 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.929 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:18.930 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.801103+0000", 2024-09-18T10:15:18.930 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999785 2024-09-18T10:15:18.930 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.930 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.930 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:18.930 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.801366+0000", 2024-09-18T10:15:18.930 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00026269299999999999 2024-09-18T10:15:18.931 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.931 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.931 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:18.931 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.801395+0000", 2024-09-18T10:15:18.931 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.9572000000000001e-05 2024-09-18T10:15:18.931 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.931 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.931 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:18.932 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.801412+0000", 2024-09-18T10:15:18.932 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6872999999999999e-05 2024-09-18T10:15:18.932 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.932 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.932 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:18.932 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.000543+0000", 2024-09-18T10:15:18.932 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.19913046700000001 2024-09-18T10:15:18.932 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.932 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.933 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:18.933 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.000579+0000", 2024-09-18T10:15:18.933 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6649000000000001e-05 2024-09-18T10:15:18.933 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.933 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.933 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:18.933 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.002922+0000", 2024-09-18T10:15:18.933 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0023426139999999998 2024-09-18T10:15:18.933 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.933 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.933 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:18.933 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.002975+0000", 2024-09-18T10:15:18.933 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3232000000000001e-05 2024-09-18T10:15:18.933 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:18.934 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:18.934 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:18.934 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.934 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.934 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.ds3 47/32 ECSubWrite(tid=57, reqid=client.4632.0:334, at_version=47'10, trim_to=0'0, roll_forward_to=41'6))", 2024-09-18T10:15:18.934 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:02.803374+0000", 2024-09-18T10:15:18.934 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 16.029011272000002, 2024-09-18T10:15:18.934 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20018619500000001, 2024-09-18T10:15:18.934 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:18.934 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:18.934 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:18.934 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.934 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:18.934 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.803374+0000", 2024-09-18T10:15:18.935 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:18.935 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.935 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.935 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:18.935 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.803374+0000", 2024-09-18T10:15:18.935 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:18.935 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.935 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.935 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:18.935 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.803373+0000", 2024-09-18T10:15:18.935 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:15:18.935 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.935 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.935 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:18.936 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.803399+0000", 2024-09-18T10:15:18.936 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5738e-05 2024-09-18T10:15:18.936 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.936 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.936 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:18.936 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.803419+0000", 2024-09-18T10:15:18.936 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9890000000000001e-05 2024-09-18T10:15:18.936 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.936 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.936 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:18.936 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.803430+0000", 2024-09-18T10:15:18.936 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.117e-05 2024-09-18T10:15:18.936 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.936 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.937 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:18.937 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.001843+0000", 2024-09-18T10:15:18.937 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.198412951 2024-09-18T10:15:18.937 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.937 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.937 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:18.937 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.001860+0000", 2024-09-18T10:15:18.937 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.658e-05 2024-09-18T10:15:18.937 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.937 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.937 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:18.937 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.003525+0000", 2024-09-18T10:15:18.937 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0016650160000000001 2024-09-18T10:15:18.937 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:18.938 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:18.938 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:18.938 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.003561+0000", 2024-09-18T10:15:18.938 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.5886000000000002e-05 2024-09-18T10:15:18.938 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:18.938 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:18.938 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:18.938 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:18.938 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:18.938 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-18T10:15:18.939 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-18T10:15:18.941 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-18T10:15:18.937+0000 7f578c52f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22134) UID: 0 2024-09-18T10:15:18.942 INFO:teuthology.orchestra.run.smithi029.stderr:reweighted osd.9 to 1 (10000) 2024-09-18T10:15:18.943 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 10 1 2024-09-18T10:15:18.943 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T10:15:18.944 INFO:tasks.rados.rados.0.smithi189.stdout:200: done (6 left) 2024-09-18T10:15:18.944 INFO:tasks.rados.rados.0.smithi189.stdout:201: done (5 left) 2024-09-18T10:15:18.944 INFO:tasks.rados.rados.0.smithi189.stdout:202: done (4 left) 2024-09-18T10:15:18.944 INFO:tasks.rados.rados.0.smithi189.stdout:203: done (3 left) 2024-09-18T10:15:18.945 INFO:tasks.rados.rados.0.smithi189.stdout:206: done (2 left) 2024-09-18T10:15:18.945 INFO:tasks.rados.rados.0.smithi189.stdout:207: delete oid 30 current snap is 17 2024-09-18T10:15:18.945 INFO:tasks.rados.rados.0.smithi189.stdout:204: finishing write tid 5 to smithi18925896-37 2024-09-18T10:15:18.945 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 37 v 19 (ObjNum 90 snap 16 seq_num 90) dirty exists 2024-09-18T10:15:18.945 INFO:tasks.rados.rados.0.smithi189.stdout:204: left oid 37 (ObjNum 90 snap 16 seq_num 90) 2024-09-18T10:15:19.041 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-18T10:15:19.037+0000 7f00046a8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 22136) UID: 0 2024-09-18T10:15:19.047 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-18T10:15:19.047 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-18T10:15:19.047 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 0 2024-09-18T10:15:19.048 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-18T10:15:19.058 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T10:15:19.110 INFO:tasks.rados.rados.0.smithi189.stdout:204: done (2 left) 2024-09-18T10:15:19.110 INFO:tasks.rados.rados.0.smithi189.stdout:207: done (1 left) 2024-09-18T10:15:19.110 INFO:tasks.rados.rados.0.smithi189.stdout:208: delete oid 4 current snap is 17 2024-09-18T10:15:19.141 INFO:tasks.ceph.osd.8.smithi029.stderr:2024-09-18T10:15:19.138+0000 7f2555c27700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 22192) UID: 0 2024-09-18T10:15:19.165 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-18T10:15:19.165 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-18T10:15:19.165 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-18T10:15:19.165 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-18T10:15:19.165 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-18T10:15:19.178 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T10:15:19.215 INFO:tasks.rados.rados.0.smithi189.stdout:208: done (1 left) 2024-09-18T10:15:19.215 INFO:tasks.rados.rados.0.smithi189.stdout:209: copy_from oid 17 from oid 19 current snap is 17 2024-09-18T10:15:19.216 INFO:tasks.rados.rados.0.smithi189.stdout:210: delete oid 24 current snap is 17 2024-09-18T10:15:19.242 INFO:tasks.ceph.osd.15.smithi189.stderr:2024-09-18T10:15:19.238+0000 7fd0e725e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 22401) UID: 0 2024-09-18T10:15:19.274 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-18T10:15:19.274 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-18T10:15:19.275 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-18T10:15:19.275 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-18T10:15:19.275 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.275 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWriteReply(3.6s0 36/32 ECSubWriteReply(tid=11, last_complete=36'7, committed=1, applied=1))", 2024-09-18T10:15:19.275 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:50.290945+0000", 2024-09-18T10:15:19.275 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 28.979888498000001, 2024-09-18T10:15:19.275 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.208748928, 2024-09-18T10:15:19.275 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:19.275 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:19.275 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:19.275 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.275 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:19.275 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.290945+0000", 2024-09-18T10:15:19.276 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:19.276 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.276 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.276 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:19.276 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.290945+0000", 2024-09-18T10:15:19.276 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:19.276 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.276 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.276 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:19.276 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.290945+0000", 2024-09-18T10:15:19.276 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:19.276 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.276 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.277 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:19.277 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.290951+0000", 2024-09-18T10:15:19.277 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.5729999999999996e-06 2024-09-18T10:15:19.277 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.277 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.277 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:19.277 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.290964+0000", 2024-09-18T10:15:19.277 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3012e-05 2024-09-18T10:15:19.277 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.277 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.277 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:19.277 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.290972+0000", 2024-09-18T10:15:19.277 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4699999999999996e-06 2024-09-18T10:15:19.278 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.278 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.278 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:19.278 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.499661+0000", 2024-09-18T10:15:19.278 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.20868911500000001 2024-09-18T10:15:19.278 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.278 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.278 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:19.278 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.499694+0000", 2024-09-18T10:15:19.278 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.3470000000000003e-05 2024-09-18T10:15:19.278 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:19.279 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:19.279 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:19.279 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.279 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.279 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWriteReply(3.6s0 36/32 ECSubWriteReply(tid=11, last_complete=36'7, committed=1, applied=1))", 2024-09-18T10:15:19.279 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:50.291030+0000", 2024-09-18T10:15:19.279 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 28.979803999000001, 2024-09-18T10:15:19.279 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.208840265, 2024-09-18T10:15:19.279 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:19.279 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:19.279 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:19.279 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.279 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:19.280 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.291030+0000", 2024-09-18T10:15:19.280 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:19.280 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.280 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.280 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:19.280 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.291030+0000", 2024-09-18T10:15:19.280 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:19.280 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.280 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.280 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:19.281 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.291030+0000", 2024-09-18T10:15:19.281 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:19.281 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.281 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.281 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:19.281 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.291032+0000", 2024-09-18T10:15:19.281 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.604e-06 2024-09-18T10:15:19.281 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.281 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.281 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:19.281 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.291038+0000", 2024-09-18T10:15:19.281 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8170000000000004e-06 2024-09-18T10:15:19.281 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.282 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.282 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:19.282 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.291042+0000", 2024-09-18T10:15:19.282 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.839e-06 2024-09-18T10:15:19.282 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.282 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.282 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:19.282 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.499812+0000", 2024-09-18T10:15:19.282 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.208769805 2024-09-18T10:15:19.282 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.282 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.283 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:19.283 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.499870+0000", 2024-09-18T10:15:19.283 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8449000000000002e-05 2024-09-18T10:15:19.283 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:19.283 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:19.283 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:19.283 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.284 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.284 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWriteReply(3.6s0 36/32 ECSubWriteReply(tid=11, last_complete=36'7, committed=1, applied=1))", 2024-09-18T10:15:19.284 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:50.302109+0000", 2024-09-18T10:15:19.284 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 28.96872484, 2024-09-18T10:15:19.284 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.29912391399999999, 2024-09-18T10:15:19.284 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:19.284 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:19.284 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:19.284 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.284 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:19.284 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.302109+0000", 2024-09-18T10:15:19.284 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:19.284 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.285 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.285 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:19.285 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.302109+0000", 2024-09-18T10:15:19.285 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:19.285 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.285 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.285 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:19.285 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.302107+0000", 2024-09-18T10:15:19.285 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:19.285 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.285 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.285 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:19.286 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.302119+0000", 2024-09-18T10:15:19.286 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1207e-05 2024-09-18T10:15:19.286 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.286 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.286 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:19.286 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.302150+0000", 2024-09-18T10:15:19.286 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.133e-05 2024-09-18T10:15:19.286 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.286 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.286 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:19.286 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.302164+0000", 2024-09-18T10:15:19.286 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3942e-05 2024-09-18T10:15:19.286 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.286 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.286 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:19.287 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.600015+0000", 2024-09-18T10:15:19.287 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.29785072000000001 2024-09-18T10:15:19.287 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.287 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.287 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:19.287 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.601233+0000", 2024-09-18T10:15:19.287 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0012183599999999999 2024-09-18T10:15:19.287 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:19.287 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:19.287 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:19.287 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.288 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.288 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpReadReply(3.6s0 36/32 ECSubReadReply(tid=10, attrs_read=0))", 2024-09-18T10:15:19.288 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:50.302179+0000", 2024-09-18T10:15:19.288 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 28.968654774000001, 2024-09-18T10:15:19.288 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.30043391800000002, 2024-09-18T10:15:19.288 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:19.289 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:19.289 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:19.289 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.289 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:19.289 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.302179+0000", 2024-09-18T10:15:19.289 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:19.289 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.289 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.289 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:19.289 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.302179+0000", 2024-09-18T10:15:19.289 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:19.290 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.290 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.290 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:19.290 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.302178+0000", 2024-09-18T10:15:19.290 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:19.290 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.290 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.290 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:19.290 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.302201+0000", 2024-09-18T10:15:19.290 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2824e-05 2024-09-18T10:15:19.290 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.290 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.290 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:19.291 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.302217+0000", 2024-09-18T10:15:19.291 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5993000000000001e-05 2024-09-18T10:15:19.291 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.291 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.291 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:19.291 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.302223+0000", 2024-09-18T10:15:19.291 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0820000000000004e-06 2024-09-18T10:15:19.291 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.291 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.291 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:19.291 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.602454+0000", 2024-09-18T10:15:19.292 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.30023072499999998 2024-09-18T10:15:19.292 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.292 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.292 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:19.292 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.602613+0000", 2024-09-18T10:15:19.292 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00015896399999999999 2024-09-18T10:15:19.293 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:19.293 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:19.293 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:19.293 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.293 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.293 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:121 3.6s0 3:6c64c4fa:::smithi18925896-41:head [append 0~688128,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:19.293 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:50.627356+0000", 2024-09-18T10:15:19.293 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 28.643477542999999, 2024-09-18T10:15:19.293 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.38269042599999997, 2024-09-18T10:15:19.293 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:19.293 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:19.293 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:19.294 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:19.294 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:19.294 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 121 2024-09-18T10:15:19.294 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.294 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:19.294 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.294 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:19.294 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.627356+0000", 2024-09-18T10:15:19.294 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:19.294 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.294 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.294 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:19.294 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.627356+0000", 2024-09-18T10:15:19.295 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:19.295 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.295 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.295 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:19.295 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.627353+0000", 2024-09-18T10:15:19.295 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999971 2024-09-18T10:15:19.295 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.295 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.296 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:19.296 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.629112+0000", 2024-09-18T10:15:19.296 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0017585980000000001 2024-09-18T10:15:19.296 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.296 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.297 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:19.297 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.629149+0000", 2024-09-18T10:15:19.297 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.6532999999999997e-05 2024-09-18T10:15:19.297 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.297 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.297 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:19.298 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.629169+0000", 2024-09-18T10:15:19.298 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0699e-05 2024-09-18T10:15:19.298 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.298 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.298 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:19.298 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.706567+0000", 2024-09-18T10:15:19.299 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.077397621999999999 2024-09-18T10:15:19.299 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.299 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.299 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:19.299 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.706944+0000", 2024-09-18T10:15:19.299 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00037710599999999999 2024-09-18T10:15:19.299 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.299 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.299 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:19.299 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.709283+0000", 2024-09-18T10:15:19.299 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0023391689999999999 2024-09-18T10:15:19.299 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.300 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.300 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:19.300 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.711791+0000", 2024-09-18T10:15:19.300 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0025078689999999998 2024-09-18T10:15:19.300 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.301 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.301 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-18T10:15:19.301 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.009925+0000", 2024-09-18T10:15:19.301 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.298134493 2024-09-18T10:15:19.301 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.301 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.301 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:19.301 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.010047+0000", 2024-09-18T10:15:19.301 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000121264 2024-09-18T10:15:19.301 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:19.301 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:19.301 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:19.302 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.302 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.302 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:122 3.6s0 3:6c64c4fa:::smithi18925896-41:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:19.302 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:50.629189+0000", 2024-09-18T10:15:19.302 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 28.641644817, 2024-09-18T10:15:19.302 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.47849547399999998, 2024-09-18T10:15:19.302 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:19.302 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:19.302 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:19.302 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:19.302 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:19.302 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 122 2024-09-18T10:15:19.302 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.302 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:19.303 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.303 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:19.303 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.629189+0000", 2024-09-18T10:15:19.303 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:19.303 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.303 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.303 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:19.303 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.629189+0000", 2024-09-18T10:15:19.303 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:19.303 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.303 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.303 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:19.303 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.629187+0000", 2024-09-18T10:15:19.304 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:19.304 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.304 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.304 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:19.304 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.629198+0000", 2024-09-18T10:15:19.304 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0623e-05 2024-09-18T10:15:19.305 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.305 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.305 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:19.305 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.629219+0000", 2024-09-18T10:15:19.305 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0665999999999998e-05 2024-09-18T10:15:19.306 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.306 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.306 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:19.306 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.629231+0000", 2024-09-18T10:15:19.306 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.291e-05 2024-09-18T10:15:19.306 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.306 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.306 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:19.306 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.710475+0000", 2024-09-18T10:15:19.306 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.081244021 2024-09-18T10:15:19.306 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.307 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.307 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:19.307 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.710639+0000", 2024-09-18T10:15:19.307 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000163325 2024-09-18T10:15:19.307 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.307 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.307 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:19.307 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.711435+0000", 2024-09-18T10:15:19.307 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00079575999999999996 2024-09-18T10:15:19.307 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.307 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.307 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:19.307 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.812067+0000", 2024-09-18T10:15:19.307 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10063237799999999 2024-09-18T10:15:19.308 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.308 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.308 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-18T10:15:19.308 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.107562+0000", 2024-09-18T10:15:19.308 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.295494906 2024-09-18T10:15:19.308 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.308 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.308 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:19.308 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.107685+0000", 2024-09-18T10:15:19.308 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00012271100000000001 2024-09-18T10:15:19.308 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:19.308 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:19.308 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:19.308 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.308 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.309 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:123 3.6s0 3:6c64c4fa:::smithi18925896-41:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:19.309 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:50.629246+0000", 2024-09-18T10:15:19.309 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 28.641587991000002, 2024-09-18T10:15:19.309 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.58067324300000001, 2024-09-18T10:15:19.309 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:19.309 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:19.309 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:19.309 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:19.309 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:19.309 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 123 2024-09-18T10:15:19.309 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.309 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:19.310 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.310 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:19.310 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.629246+0000", 2024-09-18T10:15:19.310 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:19.310 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.310 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.310 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:19.310 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.629246+0000", 2024-09-18T10:15:19.310 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:19.310 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.310 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.311 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:19.311 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.629244+0000", 2024-09-18T10:15:19.311 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:19.311 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.311 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.311 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:19.311 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.629254+0000", 2024-09-18T10:15:19.311 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.0759999999999992e-06 2024-09-18T10:15:19.311 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.311 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.311 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:19.311 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.629279+0000", 2024-09-18T10:15:19.312 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.5905999999999999e-05 2024-09-18T10:15:19.312 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.312 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.312 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:19.312 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.629293+0000", 2024-09-18T10:15:19.312 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3073e-05 2024-09-18T10:15:19.312 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.312 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.312 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:19.312 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.806055+0000", 2024-09-18T10:15:19.312 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.17676249999999999 2024-09-18T10:15:19.312 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.312 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.313 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for rw locks", 2024-09-18T10:15:19.313 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.806211+0000", 2024-09-18T10:15:19.313 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000156384 2024-09-18T10:15:19.313 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.313 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.313 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:19.313 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.108025+0000", 2024-09-18T10:15:19.313 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.30181311500000002 2024-09-18T10:15:19.313 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.313 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.314 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:19.314 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.108107+0000", 2024-09-18T10:15:19.314 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.2330000000000006e-05 2024-09-18T10:15:19.314 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.314 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.314 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:19.314 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.209919+0000", 2024-09-18T10:15:19.314 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.101812262 2024-09-18T10:15:19.314 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:19.314 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:19.314 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:19.314 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.314 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.315 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:167 3.6s0 3:6619023d:::smithi18925896-19:head [copy-from ver 4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:19.315 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:51.531881+0000", 2024-09-18T10:15:19.315 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 27.738952773000001, 2024-09-18T10:15:19.315 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.41157491600000001, 2024-09-18T10:15:19.315 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:19.315 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:19.316 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:19.316 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:19.316 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:19.316 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 167 2024-09-18T10:15:19.316 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.316 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:19.317 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.317 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:19.317 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.531881+0000", 2024-09-18T10:15:19.317 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:19.317 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.318 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.318 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:19.318 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.531881+0000", 2024-09-18T10:15:19.318 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:19.318 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.318 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.318 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:19.318 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.531877+0000", 2024-09-18T10:15:19.318 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999962 2024-09-18T10:15:19.318 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.318 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.319 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:19.319 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.531894+0000", 2024-09-18T10:15:19.319 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7269999999999999e-05 2024-09-18T10:15:19.319 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.319 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.319 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:19.319 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.531939+0000", 2024-09-18T10:15:19.319 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4815999999999999e-05 2024-09-18T10:15:19.319 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.319 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.319 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:19.319 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.531963+0000", 2024-09-18T10:15:19.319 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3312e-05 2024-09-18T10:15:19.319 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.319 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.320 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:19.320 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.532107+0000", 2024-09-18T10:15:19.320 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000144547 2024-09-18T10:15:19.320 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.320 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.320 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:19.320 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.532348+0000", 2024-09-18T10:15:19.320 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000240955 2024-09-18T10:15:19.320 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.320 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.320 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:19.320 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.839634+0000", 2024-09-18T10:15:19.320 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.30728566400000001 2024-09-18T10:15:19.321 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.321 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.321 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:19.321 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.841560+0000", 2024-09-18T10:15:19.321 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001926449 2024-09-18T10:15:19.321 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.321 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.321 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-18T10:15:19.321 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.943372+0000", 2024-09-18T10:15:19.321 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.101812176 2024-09-18T10:15:19.321 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.321 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.321 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:19.321 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.943456+0000", 2024-09-18T10:15:19.322 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.3614000000000003e-05 2024-09-18T10:15:19.322 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:19.322 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:19.322 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:19.322 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.322 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.322 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:168 3.6s0 3:6619023d:::smithi18925896-19:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:19.322 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:51.532000+0000", 2024-09-18T10:15:19.322 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 27.738833786000001, 2024-09-18T10:15:19.322 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.30898515999999998, 2024-09-18T10:15:19.322 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:19.322 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:19.322 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:19.323 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:19.323 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:19.323 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 168 2024-09-18T10:15:19.323 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.323 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:19.323 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.323 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:19.323 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.532000+0000", 2024-09-18T10:15:19.323 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:19.323 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.323 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.323 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:19.323 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.532000+0000", 2024-09-18T10:15:19.323 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:19.324 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.324 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.324 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:19.324 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.531998+0000", 2024-09-18T10:15:19.324 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:19.324 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.324 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.324 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:19.324 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.532009+0000", 2024-09-18T10:15:19.324 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0988999999999999e-05 2024-09-18T10:15:19.324 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.324 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.324 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:19.324 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.532032+0000", 2024-09-18T10:15:19.325 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2203000000000001e-05 2024-09-18T10:15:19.325 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.325 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.325 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:19.325 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.532043+0000", 2024-09-18T10:15:19.325 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1707e-05 2024-09-18T10:15:19.325 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.325 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.325 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:19.325 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.532663+0000", 2024-09-18T10:15:19.325 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000619993 2024-09-18T10:15:19.325 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.325 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.325 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for blocked object", 2024-09-18T10:15:19.326 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.532735+0000", 2024-09-18T10:15:19.326 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.2025000000000002e-05 2024-09-18T10:15:19.326 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.326 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.326 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:19.326 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.840712+0000", 2024-09-18T10:15:19.326 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.307976582 2024-09-18T10:15:19.326 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.326 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.326 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:19.326 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.840866+0000", 2024-09-18T10:15:19.326 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000154203 2024-09-18T10:15:19.326 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.327 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.327 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:19.327 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.840985+0000", 2024-09-18T10:15:19.327 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000119084 2024-09-18T10:15:19.327 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:19.327 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:19.327 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:19.327 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.327 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.327 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:205 3.6s0 3:61491e68:::smithi18925896-27:head [read 0~442368,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:19.327 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:52.694213+0000", 2024-09-18T10:15:19.328 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 26.576620706, 2024-09-18T10:15:19.328 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.20580652599999999, 2024-09-18T10:15:19.328 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:19.328 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:19.328 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:19.328 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:19.328 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:19.328 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 205 2024-09-18T10:15:19.329 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.329 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:19.329 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.329 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:19.329 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.694213+0000", 2024-09-18T10:15:19.330 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:19.330 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.330 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.330 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:19.330 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.694213+0000", 2024-09-18T10:15:19.330 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:19.331 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.331 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.331 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:19.331 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.694212+0000", 2024-09-18T10:15:19.331 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:19.331 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.332 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.332 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:19.332 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.694220+0000", 2024-09-18T10:15:19.332 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5250000000000005e-06 2024-09-18T10:15:19.332 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.332 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.333 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:19.333 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.694237+0000", 2024-09-18T10:15:19.333 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.6181e-05 2024-09-18T10:15:19.333 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.333 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.333 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:19.333 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.694246+0000", 2024-09-18T10:15:19.334 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.2029999999999998e-06 2024-09-18T10:15:19.334 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.334 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.334 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:19.334 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.794554+0000", 2024-09-18T10:15:19.334 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10030855 2024-09-18T10:15:19.334 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.335 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.335 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:19.335 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.794563+0000", 2024-09-18T10:15:19.335 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.2230000000000007e-06 2024-09-18T10:15:19.335 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.335 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.335 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:19.336 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.795588+0000", 2024-09-18T10:15:19.336 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001025127 2024-09-18T10:15:19.336 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.336 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.336 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:19.336 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.795756+0000", 2024-09-18T10:15:19.337 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00016793900000000001 2024-09-18T10:15:19.337 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.337 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.337 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:19.337 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.900020+0000", 2024-09-18T10:15:19.337 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.104264018 2024-09-18T10:15:19.337 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:19.338 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:19.338 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:19.338 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.338 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.338 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:206 3.6s0 3:61491e68:::smithi18925896-27:head [read 0~442368,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:19.338 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:52.694386+0000", 2024-09-18T10:15:19.338 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 26.57644745, 2024-09-18T10:15:19.338 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.40544065200000001, 2024-09-18T10:15:19.339 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:19.339 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:19.339 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:19.339 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:19.339 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:19.339 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 206 2024-09-18T10:15:19.339 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.340 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:19.340 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.340 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:19.340 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.694386+0000", 2024-09-18T10:15:19.340 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:19.340 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.340 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.341 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:19.341 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.694386+0000", 2024-09-18T10:15:19.341 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:19.341 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.341 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.341 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:19.342 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.694385+0000", 2024-09-18T10:15:19.342 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.999999 2024-09-18T10:15:19.342 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.342 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.342 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:19.342 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.694392+0000", 2024-09-18T10:15:19.343 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7970000000000003e-06 2024-09-18T10:15:19.343 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.343 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.343 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:19.343 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.694405+0000", 2024-09-18T10:15:19.343 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3406e-05 2024-09-18T10:15:19.344 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.344 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.344 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:19.344 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.694413+0000", 2024-09-18T10:15:19.344 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.9070000000000007e-06 2024-09-18T10:15:19.344 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.344 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.344 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:19.344 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.794592+0000", 2024-09-18T10:15:19.345 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100178879 2024-09-18T10:15:19.345 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.345 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.345 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:19.345 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.794598+0000", 2024-09-18T10:15:19.345 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6289999999999998e-06 2024-09-18T10:15:19.345 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.345 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.345 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:19.345 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.796805+0000", 2024-09-18T10:15:19.345 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0022073930000000002 2024-09-18T10:15:19.345 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.345 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.346 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:19.346 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.796886+0000", 2024-09-18T10:15:19.346 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0641000000000003e-05 2024-09-18T10:15:19.346 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.346 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.346 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:19.346 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:53.099827+0000", 2024-09-18T10:15:19.346 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.302941191 2024-09-18T10:15:19.346 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:19.346 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:19.346 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:19.346 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.346 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.346 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:323 3.6s0 3:61491e68:::smithi18925896-27:head [append 0~0] snapc 7=[4] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-18T10:15:19.347 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:01.795354+0000", 2024-09-18T10:15:19.347 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 17.475479935999999, 2024-09-18T10:15:19.347 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.30418701500000001, 2024-09-18T10:15:19.347 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:19.347 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:19.347 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:19.347 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:19.347 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:19.347 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 323 2024-09-18T10:15:19.347 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.347 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:19.347 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.348 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:19.348 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.795354+0000", 2024-09-18T10:15:19.348 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:19.348 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.348 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.348 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:19.348 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.795354+0000", 2024-09-18T10:15:19.349 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:19.349 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.349 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.349 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:19.349 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.795351+0000", 2024-09-18T10:15:19.349 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999971 2024-09-18T10:15:19.350 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.350 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.350 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:19.350 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.795365+0000", 2024-09-18T10:15:19.350 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3713999999999999e-05 2024-09-18T10:15:19.350 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.350 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.351 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:19.351 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.795406+0000", 2024-09-18T10:15:19.351 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0846999999999997e-05 2024-09-18T10:15:19.351 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.351 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.351 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:19.351 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.795425+0000", 2024-09-18T10:15:19.352 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9936999999999999e-05 2024-09-18T10:15:19.352 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.352 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.352 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:19.352 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.795648+0000", 2024-09-18T10:15:19.352 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000222547 2024-09-18T10:15:19.353 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.353 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.353 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:19.353 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.795872+0000", 2024-09-18T10:15:19.353 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00022384799999999999 2024-09-18T10:15:19.353 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.354 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.354 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:19.354 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.797018+0000", 2024-09-18T10:15:19.354 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001146055 2024-09-18T10:15:19.354 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.354 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.355 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:19.355 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.798609+0000", 2024-09-18T10:15:19.355 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0015905959999999999 2024-09-18T10:15:19.355 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.355 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.355 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-18T10:15:19.356 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.099372+0000", 2024-09-18T10:15:19.356 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.30076301500000002 2024-09-18T10:15:19.356 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.356 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.356 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:19.357 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.099541+0000", 2024-09-18T10:15:19.357 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000169407 2024-09-18T10:15:19.357 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:19.357 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:19.357 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:19.358 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.358 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.358 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:324 3.6s0 3:61491e68:::smithi18925896-27:head [rollback 4] snapc 7=[4] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-18T10:15:19.358 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:01.795482+0000", 2024-09-18T10:15:19.358 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 17.475352321999999, 2024-09-18T10:15:19.358 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.30451560100000002, 2024-09-18T10:15:19.359 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:19.359 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:19.359 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:19.359 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:19.359 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:19.359 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 324 2024-09-18T10:15:19.360 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.360 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:19.360 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.360 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:19.360 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.795482+0000", 2024-09-18T10:15:19.360 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:19.361 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.361 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.361 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:19.361 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.795482+0000", 2024-09-18T10:15:19.361 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:19.361 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.362 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.362 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:19.362 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.795480+0000", 2024-09-18T10:15:19.362 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:19.362 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.362 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.362 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:19.363 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.795487+0000", 2024-09-18T10:15:19.363 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7170000000000002e-06 2024-09-18T10:15:19.363 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.363 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.363 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:19.363 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.795548+0000", 2024-09-18T10:15:19.364 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1647000000000001e-05 2024-09-18T10:15:19.364 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.364 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.364 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:19.364 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.795566+0000", 2024-09-18T10:15:19.364 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7560000000000001e-05 2024-09-18T10:15:19.365 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.365 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.365 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:19.365 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.797676+0000", 2024-09-18T10:15:19.365 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0021101919999999999 2024-09-18T10:15:19.365 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.366 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.366 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:19.366 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.797742+0000", 2024-09-18T10:15:19.366 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.6351000000000002e-05 2024-09-18T10:15:19.366 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.366 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.367 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:19.367 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.798290+0000", 2024-09-18T10:15:19.367 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000548063 2024-09-18T10:15:19.367 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.367 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.368 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:19.368 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.800131+0000", 2024-09-18T10:15:19.368 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001840759 2024-09-18T10:15:19.368 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.368 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.368 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-18T10:15:19.369 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.099840+0000", 2024-09-18T10:15:19.369 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.29970881300000002 2024-09-18T10:15:19.369 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.369 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.369 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:19.369 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.099997+0000", 2024-09-18T10:15:19.370 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000157124 2024-09-18T10:15:19.370 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:19.370 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:19.370 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:19.370 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.370 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.371 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:325 3.6s0 3:61491e68:::smithi18925896-27:head [append 0~0] snapc 7=[4] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-18T10:15:19.371 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:01.795591+0000", 2024-09-18T10:15:19.371 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 17.475243267, 2024-09-18T10:15:19.371 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.30495726400000001, 2024-09-18T10:15:19.371 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:19.371 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:19.372 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:19.372 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:19.372 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:19.372 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 325 2024-09-18T10:15:19.372 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.372 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:19.373 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.373 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:19.373 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.795591+0000", 2024-09-18T10:15:19.373 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:19.373 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.373 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.374 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:19.374 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.795591+0000", 2024-09-18T10:15:19.374 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:19.374 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.374 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.374 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:19.375 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.795589+0000", 2024-09-18T10:15:19.375 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:19.375 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.375 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.375 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:19.376 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.795598+0000", 2024-09-18T10:15:19.376 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.7940000000000008e-06 2024-09-18T10:15:19.376 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.376 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.376 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:19.376 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.795617+0000", 2024-09-18T10:15:19.377 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8485e-05 2024-09-18T10:15:19.377 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.377 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.377 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:19.377 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.795627+0000", 2024-09-18T10:15:19.378 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0730999999999999e-05 2024-09-18T10:15:19.378 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.378 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.378 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:19.378 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.798672+0000", 2024-09-18T10:15:19.378 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00304484 2024-09-18T10:15:19.378 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.379 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.379 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:19.379 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.798918+0000", 2024-09-18T10:15:19.379 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00024598000000000003 2024-09-18T10:15:19.379 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.379 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.380 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:19.380 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.799703+0000", 2024-09-18T10:15:19.380 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000785262 2024-09-18T10:15:19.380 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.380 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.381 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:19.381 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.900558+0000", 2024-09-18T10:15:19.381 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100854108 2024-09-18T10:15:19.381 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.381 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.381 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-18T10:15:19.381 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.100434+0000", 2024-09-18T10:15:19.382 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.19987638699999999 2024-09-18T10:15:19.382 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.382 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.382 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:19.382 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.100548+0000", 2024-09-18T10:15:19.382 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00011390799999999999 2024-09-18T10:15:19.383 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:19.383 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:19.383 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:19.383 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.383 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.383 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWriteReply(3.6s0 57/40 ECSubWriteReply(tid=50, last_complete=57'30, committed=1, applied=1))", 2024-09-18T10:15:19.384 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:13.143948+0000", 2024-09-18T10:15:19.384 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 6.1268862249999998, 2024-09-18T10:15:19.384 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.21095676099999999, 2024-09-18T10:15:19.384 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:19.384 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:19.384 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:19.385 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.385 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:19.385 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.143948+0000", 2024-09-18T10:15:19.385 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:19.385 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.385 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.386 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:19.386 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.143948+0000", 2024-09-18T10:15:19.386 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:19.386 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.386 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.386 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:19.387 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.143947+0000", 2024-09-18T10:15:19.387 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:19.387 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.387 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.387 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:19.387 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.143950+0000", 2024-09-18T10:15:19.387 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.8959999999999999e-06 2024-09-18T10:15:19.387 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.388 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.388 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:19.388 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.143958+0000", 2024-09-18T10:15:19.388 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7230000000000004e-06 2024-09-18T10:15:19.388 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.388 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.388 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:19.388 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.143961+0000", 2024-09-18T10:15:19.388 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0010000000000002e-06 2024-09-18T10:15:19.389 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.389 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.389 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:19.389 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.354877+0000", 2024-09-18T10:15:19.389 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.21091578599999999 2024-09-18T10:15:19.389 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.389 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.389 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:19.389 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.354904+0000", 2024-09-18T10:15:19.390 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.779e-05 2024-09-18T10:15:19.390 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:19.390 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:19.390 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:19.390 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.390 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.390 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWriteReply(3.6s0 57/40 ECSubWriteReply(tid=51, last_complete=57'32, committed=1, applied=1))", 2024-09-18T10:15:19.390 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:13.144581+0000", 2024-09-18T10:15:19.390 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 6.1262525300000004, 2024-09-18T10:15:19.391 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.210571697, 2024-09-18T10:15:19.391 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:19.391 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:19.391 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:19.391 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.391 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:19.391 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.144581+0000", 2024-09-18T10:15:19.391 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:19.391 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.391 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.392 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:19.392 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.144581+0000", 2024-09-18T10:15:19.392 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:19.392 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.392 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.392 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:19.392 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.144581+0000", 2024-09-18T10:15:19.392 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:19.392 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.393 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.393 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:19.393 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.144584+0000", 2024-09-18T10:15:19.393 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.4760000000000001e-06 2024-09-18T10:15:19.393 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.393 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.393 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:19.393 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.144591+0000", 2024-09-18T10:15:19.393 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1969999999999999e-06 2024-09-18T10:15:19.393 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.394 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.394 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:19.394 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.144595+0000", 2024-09-18T10:15:19.394 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9899999999999999e-06 2024-09-18T10:15:19.394 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.394 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.394 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:19.394 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.355136+0000", 2024-09-18T10:15:19.394 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.21054089200000001 2024-09-18T10:15:19.395 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.395 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.395 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:19.395 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.355153+0000", 2024-09-18T10:15:19.395 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7442999999999999e-05 2024-09-18T10:15:19.395 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:19.395 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:19.395 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:19.395 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.395 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.396 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWriteReply(3.6s0 57/40 ECSubWriteReply(tid=51, last_complete=57'32, committed=1, applied=1))", 2024-09-18T10:15:19.396 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:13.144719+0000", 2024-09-18T10:15:19.396 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 6.126114587, 2024-09-18T10:15:19.396 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.210498871, 2024-09-18T10:15:19.396 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:19.396 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:19.396 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:19.396 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.396 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:19.397 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.144719+0000", 2024-09-18T10:15:19.397 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:19.397 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.397 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.397 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:19.397 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.144719+0000", 2024-09-18T10:15:19.397 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:19.397 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.397 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.397 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:19.398 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.144719+0000", 2024-09-18T10:15:19.398 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:19.398 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.398 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.398 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:19.398 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.144721+0000", 2024-09-18T10:15:19.398 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0169999999999999e-06 2024-09-18T10:15:19.398 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.399 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.399 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:19.399 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.144728+0000", 2024-09-18T10:15:19.399 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9600000000000003e-06 2024-09-18T10:15:19.399 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.399 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.399 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:19.400 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.144731+0000", 2024-09-18T10:15:19.400 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.7760000000000002e-06 2024-09-18T10:15:19.400 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.400 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.400 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:19.401 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.355198+0000", 2024-09-18T10:15:19.401 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.210467401 2024-09-18T10:15:19.401 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.401 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.401 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:19.401 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.355218+0000", 2024-09-18T10:15:19.401 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9979999999999998e-05 2024-09-18T10:15:19.401 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:19.401 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:19.401 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:19.401 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.401 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.402 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWriteReply(3.6s0 57/40 ECSubWriteReply(tid=50, last_complete=57'30, committed=1, applied=1))", 2024-09-18T10:15:19.402 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:13.149416+0000", 2024-09-18T10:15:19.402 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 6.1214178029999999, 2024-09-18T10:15:19.402 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.205566108, 2024-09-18T10:15:19.402 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:19.402 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:19.402 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:19.402 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.402 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:19.402 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.149416+0000", 2024-09-18T10:15:19.402 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:19.402 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.402 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.402 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:19.403 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.149416+0000", 2024-09-18T10:15:19.403 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:19.403 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.403 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.403 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:19.403 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.149414+0000", 2024-09-18T10:15:19.403 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999976 2024-09-18T10:15:19.403 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.403 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.403 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:19.403 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.149431+0000", 2024-09-18T10:15:19.403 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.6631000000000001e-05 2024-09-18T10:15:19.403 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.404 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.404 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:19.404 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.149472+0000", 2024-09-18T10:15:19.404 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1668000000000002e-05 2024-09-18T10:15:19.404 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.404 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.404 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:19.404 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.149490+0000", 2024-09-18T10:15:19.404 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8108e-05 2024-09-18T10:15:19.404 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.404 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.404 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:19.404 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.354955+0000", 2024-09-18T10:15:19.404 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.205464444 2024-09-18T10:15:19.405 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.405 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.405 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:19.405 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.354982+0000", 2024-09-18T10:15:19.405 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.7443000000000001e-05 2024-09-18T10:15:19.405 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:19.405 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:19.405 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:19.405 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.405 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.405 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWriteReply(3.6s0 57/40 ECSubWriteReply(tid=51, last_complete=57'32, committed=1, applied=1))", 2024-09-18T10:15:19.405 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:13.150190+0000", 2024-09-18T10:15:19.405 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 6.1206439369999996, 2024-09-18T10:15:19.406 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.205097152, 2024-09-18T10:15:19.406 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:19.406 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:19.406 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:19.406 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.406 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:19.406 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.150190+0000", 2024-09-18T10:15:19.406 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:19.406 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.406 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.406 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:19.406 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.150190+0000", 2024-09-18T10:15:19.406 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:19.406 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.406 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.407 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:19.407 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.150189+0000", 2024-09-18T10:15:19.407 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:19.407 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.407 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.407 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:19.407 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.150200+0000", 2024-09-18T10:15:19.407 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1286e-05 2024-09-18T10:15:19.407 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.407 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.407 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:19.407 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.150230+0000", 2024-09-18T10:15:19.407 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.021e-05 2024-09-18T10:15:19.408 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.408 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.408 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:19.408 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.150250+0000", 2024-09-18T10:15:19.408 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9933999999999999e-05 2024-09-18T10:15:19.408 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.408 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.408 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:19.408 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.355268+0000", 2024-09-18T10:15:19.408 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.20501817999999999 2024-09-18T10:15:19.408 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.408 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.408 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:19.409 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.355287+0000", 2024-09-18T10:15:19.409 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8913999999999999e-05 2024-09-18T10:15:19.409 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:19.409 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:19.409 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:19.409 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.409 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.409 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDScrubReserve(3.6s0 REJECT e62)", 2024-09-18T10:15:19.409 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:18.118418+0000", 2024-09-18T10:15:19.409 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 1.15241623, 2024-09-18T10:15:19.409 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.379472953, 2024-09-18T10:15:19.409 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:19.409 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:19.410 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:19.410 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.410 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:19.410 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:18.118418+0000", 2024-09-18T10:15:19.410 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:19.410 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.410 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.410 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:19.410 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:18.118418+0000", 2024-09-18T10:15:19.410 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:19.410 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.410 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.410 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:19.410 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:18.118416+0000", 2024-09-18T10:15:19.411 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:19.411 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.411 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.411 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:19.411 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:18.118424+0000", 2024-09-18T10:15:19.411 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.576e-06 2024-09-18T10:15:19.411 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.411 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.411 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:19.411 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:18.118450+0000", 2024-09-18T10:15:19.411 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6673e-05 2024-09-18T10:15:19.411 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.411 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.411 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:19.412 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:18.218541+0000", 2024-09-18T10:15:19.412 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100090107 2024-09-18T10:15:19.412 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.412 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.412 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:19.412 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:18.497802+0000", 2024-09-18T10:15:19.412 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.279261392 2024-09-18T10:15:19.412 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.412 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.412 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:19.412 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:18.497866+0000", 2024-09-18T10:15:19.412 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3989e-05 2024-09-18T10:15:19.412 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.413 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.413 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:19.413 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:18.497873+0000", 2024-09-18T10:15:19.413 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1500000000000002e-06 2024-09-18T10:15:19.413 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.413 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.413 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:19.413 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:18.497891+0000", 2024-09-18T10:15:19.413 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7462999999999999e-05 2024-09-18T10:15:19.413 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:19.413 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:19.413 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:19.413 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:19.413 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:19.414 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-18T10:15:19.415 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T10:15:19.416 INFO:tasks.ceph.osd.12.smithi029.stderr:2024-09-18T10:15:19.410+0000 7f8e4e627700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 22190) UID: 0 2024-09-18T10:15:19.422 INFO:tasks.rados.rados.0.smithi189.stdout:210: done (2 left) 2024-09-18T10:15:19.422 INFO:tasks.rados.rados.0.smithi189.stdout:append oid 5 current snap is 17 2024-09-18T10:15:19.422 INFO:tasks.rados.rados.0.smithi189.stdout:211: seq_num 94 ranges {81920=425984,507904=786432,1294336=376832} 2024-09-18T10:15:19.430 INFO:tasks.rados.rados.0.smithi189.stdout:211: writing smithi18925896-5 from 81920 to 507904 tid 1 2024-09-18T10:15:19.439 INFO:tasks.rados.rados.0.smithi189.stdout:211: writing smithi18925896-5 from 507904 to 1294336 tid 2 2024-09-18T10:15:19.442 INFO:tasks.rados.rados.0.smithi189.stdout:211: writing smithi18925896-5 from 1294336 to 1671168 tid 3 2024-09-18T10:15:19.442 INFO:tasks.rados.rados.0.smithi189.stdout:212: read oid 28 snap -1 2024-09-18T10:15:19.442 INFO:tasks.rados.rados.0.smithi189.stdout:212: expect (ObjNum 33 snap 0 seq_num 33) 2024-09-18T10:15:19.442 INFO:tasks.rados.rados.0.smithi189.stdout:append oid 15 current snap is 17 2024-09-18T10:15:19.442 INFO:tasks.rados.rados.0.smithi189.stdout:213: seq_num 95 ranges {1294336=442368,1736704=540672,2277376=622592} 2024-09-18T10:15:19.451 INFO:tasks.rados.rados.0.smithi189.stdout:213: writing smithi18925896-15 from 1294336 to 1736704 tid 1 2024-09-18T10:15:19.454 INFO:tasks.rados.rados.0.smithi189.stdout:213: writing smithi18925896-15 from 1736704 to 2277376 tid 2 2024-09-18T10:15:19.458 INFO:tasks.rados.rados.0.smithi189.stdout:213: writing smithi18925896-15 from 2277376 to 2899968 tid 3 2024-09-18T10:15:19.458 INFO:tasks.rados.rados.0.smithi189.stdout:214: rollback oid 10 current snap is 17 2024-09-18T10:15:19.458 INFO:tasks.rados.rados.0.smithi189.stdout:rollback oid 10 to 16 2024-09-18T10:15:19.458 INFO:tasks.rados.rados.0.smithi189.stdout:215: snap_remove snap 15 2024-09-18T10:15:19.515 INFO:tasks.ceph.osd.4.smithi029.stderr:2024-09-18T10:15:19.510+0000 7f550cf6e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22189) UID: 0 2024-09-18T10:15:19.529 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-18T10:15:19.529 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-18T10:15:19.529 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 0 2024-09-18T10:15:19.529 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-18T10:15:19.538 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T10:15:19.601 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T10:15:19.617 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-18T10:15:19.613+0000 7f1981873700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22132) UID: 0 2024-09-18T10:15:19.643 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-18T10:15:19.644 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-18T10:15:19.644 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-18T10:15:19.644 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-18T10:15:19.644 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-18T10:15:19.655 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T10:15:19.717 INFO:tasks.ceph.osd.12.smithi029.stderr:2024-09-18T10:15:19.714+0000 7f8e4e627700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 22190) UID: 0 2024-09-18T10:15:19.733 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-18T10:15:19.755 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-18T10:15:19.778 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-18T10:15:19.800 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-18T10:15:19.800 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-18T10:15:19.800 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-18T10:15:19.800 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-18T10:15:19.800 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.800 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:120 3.bs0 3:d62b3dfa:::smithi18925896-40:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:19.800 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:50.705551+0000", 2024-09-18T10:15:19.800 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 29.090339211, 2024-09-18T10:15:19.801 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.508399406, 2024-09-18T10:15:19.801 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:19.801 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:19.801 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:19.801 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:19.801 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:19.801 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 120 2024-09-18T10:15:19.801 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.801 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:19.801 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.801 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:19.801 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.705551+0000", 2024-09-18T10:15:19.802 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:19.802 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.802 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.802 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:19.802 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.705551+0000", 2024-09-18T10:15:19.802 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:19.802 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.802 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.802 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:19.802 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.705549+0000", 2024-09-18T10:15:19.802 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:19.803 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.803 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.803 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:19.803 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.705557+0000", 2024-09-18T10:15:19.803 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.9500000000000001e-06 2024-09-18T10:15:19.803 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.803 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.803 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:19.803 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.705575+0000", 2024-09-18T10:15:19.803 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7989999999999999e-05 2024-09-18T10:15:19.803 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.803 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.804 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:19.804 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.705587+0000", 2024-09-18T10:15:19.804 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1226000000000001e-05 2024-09-18T10:15:19.804 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.804 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.804 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:19.804 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.812099+0000", 2024-09-18T10:15:19.804 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.106512116 2024-09-18T10:15:19.804 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.804 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.804 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for rw locks", 2024-09-18T10:15:19.804 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.812429+0000", 2024-09-18T10:15:19.804 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00033007300000000001 2024-09-18T10:15:19.805 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.805 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.805 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:19.805 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.111941+0000", 2024-09-18T10:15:19.805 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.29951245700000001 2024-09-18T10:15:19.805 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.805 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.805 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:19.805 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.112099+0000", 2024-09-18T10:15:19.805 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00015813399999999999 2024-09-18T10:15:19.805 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.805 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.805 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:19.805 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.213950+0000", 2024-09-18T10:15:19.806 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10185091 2024-09-18T10:15:19.806 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:19.806 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:19.806 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:19.806 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.806 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.806 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:128 3.7s0 3:e4da5b5e:::smithi18925896-43:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:19.806 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:50.713927+0000", 2024-09-18T10:15:19.806 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 29.081963153, 2024-09-18T10:15:19.806 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.50422476699999996, 2024-09-18T10:15:19.806 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:19.806 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:19.806 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:19.806 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:19.807 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:19.807 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 128 2024-09-18T10:15:19.807 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.807 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:19.807 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.807 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:19.807 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.713927+0000", 2024-09-18T10:15:19.807 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:19.807 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.807 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.807 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:19.807 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.713927+0000", 2024-09-18T10:15:19.807 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:19.808 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.808 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.808 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:19.808 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.713925+0000", 2024-09-18T10:15:19.808 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:19.808 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.808 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.808 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:19.808 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.713933+0000", 2024-09-18T10:15:19.808 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.2069999999999998e-06 2024-09-18T10:15:19.808 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.808 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.808 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:19.809 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.713947+0000", 2024-09-18T10:15:19.809 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4249e-05 2024-09-18T10:15:19.809 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.809 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.809 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:19.809 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.713954+0000", 2024-09-18T10:15:19.809 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8070000000000002e-06 2024-09-18T10:15:19.809 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.809 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.809 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:19.809 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.904149+0000", 2024-09-18T10:15:19.809 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.19019514800000001 2024-09-18T10:15:19.809 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.809 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.810 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:19.810 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.904338+0000", 2024-09-18T10:15:19.810 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000189326 2024-09-18T10:15:19.810 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.810 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.810 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:19.810 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.905212+0000", 2024-09-18T10:15:19.810 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00087334100000000003 2024-09-18T10:15:19.810 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.810 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.810 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:19.810 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.906215+0000", 2024-09-18T10:15:19.810 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0010037430000000001 2024-09-18T10:15:19.810 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.811 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.811 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-18T10:15:19.811 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.217903+0000", 2024-09-18T10:15:19.811 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.31168731500000002 2024-09-18T10:15:19.811 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.811 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.811 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:19.811 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.218152+0000", 2024-09-18T10:15:19.811 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00024915799999999997 2024-09-18T10:15:19.811 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:19.811 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:19.811 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:19.811 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.812 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.812 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:129 3.7s0 3:e4da5b5e:::smithi18925896-43:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:19.812 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:50.713961+0000", 2024-09-18T10:15:19.812 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 29.081928984000001, 2024-09-18T10:15:19.812 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.819334071, 2024-09-18T10:15:19.812 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:19.812 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:19.812 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:19.812 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:19.812 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:19.812 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 129 2024-09-18T10:15:19.812 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.812 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:19.813 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.813 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:19.813 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.713961+0000", 2024-09-18T10:15:19.813 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:19.813 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.813 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.813 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:19.813 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.713961+0000", 2024-09-18T10:15:19.813 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:19.813 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.813 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.813 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:19.813 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.713960+0000", 2024-09-18T10:15:19.814 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.999999 2024-09-18T10:15:19.814 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.814 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.814 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:19.814 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.713964+0000", 2024-09-18T10:15:19.814 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.3059999999999999e-06 2024-09-18T10:15:19.814 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.814 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.814 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:19.814 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.713974+0000", 2024-09-18T10:15:19.814 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.9969999999999993e-06 2024-09-18T10:15:19.814 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.814 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.814 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:19.814 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.713981+0000", 2024-09-18T10:15:19.815 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1229999999999997e-06 2024-09-18T10:15:19.815 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.815 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.815 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:19.815 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.000661+0000", 2024-09-18T10:15:19.815 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.28668068699999999 2024-09-18T10:15:19.815 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.815 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.815 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for rw locks", 2024-09-18T10:15:19.815 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.000844+0000", 2024-09-18T10:15:19.815 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00018289599999999999 2024-09-18T10:15:19.815 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.815 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.816 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:19.816 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.219209+0000", 2024-09-18T10:15:19.816 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.21836466900000001 2024-09-18T10:15:19.816 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.816 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.816 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:19.816 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.219351+0000", 2024-09-18T10:15:19.816 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00014262300000000001 2024-09-18T10:15:19.816 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.816 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.816 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:19.816 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.533295+0000", 2024-09-18T10:15:19.816 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.31394377299999998 2024-09-18T10:15:19.816 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:19.817 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:19.817 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:19.817 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.817 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.817 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:159 3.7s0 3:ead53b0f:::smithi18925896-25:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:19.817 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:51.226988+0000", 2024-09-18T10:15:19.817 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 28.568901923999999, 2024-09-18T10:15:19.817 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.81465928499999996, 2024-09-18T10:15:19.817 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:19.817 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:19.817 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:19.817 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:19.817 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:19.818 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 159 2024-09-18T10:15:19.818 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.818 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:19.818 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.818 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:19.818 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.226988+0000", 2024-09-18T10:15:19.818 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:19.818 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.818 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.818 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:19.818 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.226988+0000", 2024-09-18T10:15:19.818 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:19.818 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.819 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.819 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:19.819 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.226987+0000", 2024-09-18T10:15:19.819 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.999999 2024-09-18T10:15:19.819 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.819 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.819 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:19.819 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.226992+0000", 2024-09-18T10:15:19.819 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.5900000000000001e-06 2024-09-18T10:15:19.819 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.819 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.819 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:19.819 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.227003+0000", 2024-09-18T10:15:19.819 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1222e-05 2024-09-18T10:15:19.820 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.820 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.820 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:19.820 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.227011+0000", 2024-09-18T10:15:19.820 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.6279999999999996e-06 2024-09-18T10:15:19.820 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.820 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.820 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:19.820 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.533792+0000", 2024-09-18T10:15:19.820 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.30678158100000003 2024-09-18T10:15:19.820 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.820 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.820 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for rw locks", 2024-09-18T10:15:19.821 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.533941+0000", 2024-09-18T10:15:19.821 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00014846 2024-09-18T10:15:19.821 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.821 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.822 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:19.822 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.633576+0000", 2024-09-18T10:15:19.822 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099635153000000004 2024-09-18T10:15:19.822 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.822 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.822 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:19.822 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.633832+0000", 2024-09-18T10:15:19.822 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00025610699999999999 2024-09-18T10:15:19.822 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.822 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.822 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:19.822 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.041647+0000", 2024-09-18T10:15:19.823 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.407815388 2024-09-18T10:15:19.823 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:19.823 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:19.823 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:19.823 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.823 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.823 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:169 3.7s0 3:e1df0280:::smithi18925896-26:head [copy-from ver 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:19.823 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:51.532055+0000", 2024-09-18T10:15:19.823 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 28.263834736, 2024-09-18T10:15:19.823 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.60955003100000005, 2024-09-18T10:15:19.823 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:19.823 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:19.823 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:19.824 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:19.824 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:19.824 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 169 2024-09-18T10:15:19.824 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.824 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:19.824 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.824 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:19.824 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.532055+0000", 2024-09-18T10:15:19.824 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:19.824 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.824 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.824 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:19.825 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.532055+0000", 2024-09-18T10:15:19.825 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:19.825 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.825 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.825 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:19.825 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.532054+0000", 2024-09-18T10:15:19.825 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:19.825 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.825 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.825 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:19.825 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.532062+0000", 2024-09-18T10:15:19.825 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0430000000000008e-06 2024-09-18T10:15:19.825 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.826 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.826 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:19.826 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.532080+0000", 2024-09-18T10:15:19.826 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8283000000000001e-05 2024-09-18T10:15:19.826 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.826 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.826 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:19.826 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.532092+0000", 2024-09-18T10:15:19.826 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1742999999999999e-05 2024-09-18T10:15:19.826 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.826 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.826 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:19.826 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.534038+0000", 2024-09-18T10:15:19.826 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0019461610000000001 2024-09-18T10:15:19.827 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.827 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.827 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:19.827 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.534285+0000", 2024-09-18T10:15:19.827 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00024667800000000001 2024-09-18T10:15:19.827 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.827 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.827 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:19.827 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.938506+0000", 2024-09-18T10:15:19.827 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.40422132199999999 2024-09-18T10:15:19.827 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.827 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.827 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:19.828 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.041810+0000", 2024-09-18T10:15:19.828 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10330328900000001 2024-09-18T10:15:19.828 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.828 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.828 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-18T10:15:19.828 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.141492+0000", 2024-09-18T10:15:19.828 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099682251999999999 2024-09-18T10:15:19.828 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.828 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.828 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:19.828 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.141605+0000", 2024-09-18T10:15:19.828 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00011349300000000001 2024-09-18T10:15:19.828 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:19.829 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:19.829 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:19.829 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.829 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.829 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:170 3.7s0 3:e1df0280:::smithi18925896-26:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:19.829 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:51.532106+0000", 2024-09-18T10:15:19.829 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 28.263784457, 2024-09-18T10:15:19.829 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.50908735800000005, 2024-09-18T10:15:19.829 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:19.829 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:19.829 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:19.829 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:19.829 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:19.830 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 170 2024-09-18T10:15:19.830 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.830 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:19.830 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.830 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:19.830 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.532106+0000", 2024-09-18T10:15:19.830 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:19.830 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.830 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.830 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:19.830 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.532106+0000", 2024-09-18T10:15:19.830 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:19.830 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.831 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.831 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:19.831 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.532104+0000", 2024-09-18T10:15:19.831 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:19.831 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.831 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.831 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:19.831 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.532113+0000", 2024-09-18T10:15:19.831 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8659999999999995e-06 2024-09-18T10:15:19.831 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.831 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.831 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:19.831 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.532131+0000", 2024-09-18T10:15:19.832 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7723999999999998e-05 2024-09-18T10:15:19.832 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.832 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.832 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:19.832 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.532142+0000", 2024-09-18T10:15:19.832 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0669000000000001e-05 2024-09-18T10:15:19.832 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.832 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.832 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:19.832 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.534705+0000", 2024-09-18T10:15:19.832 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0025635380000000002 2024-09-18T10:15:19.832 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.833 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.833 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for blocked object", 2024-09-18T10:15:19.833 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.534831+0000", 2024-09-18T10:15:19.833 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00012540399999999999 2024-09-18T10:15:19.833 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.833 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.833 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:19.833 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.040545+0000", 2024-09-18T10:15:19.833 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.50571483699999997 2024-09-18T10:15:19.833 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.833 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.833 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:19.833 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.040869+0000", 2024-09-18T10:15:19.834 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00032386999999999997 2024-09-18T10:15:19.834 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.834 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.834 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:19.834 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.041193+0000", 2024-09-18T10:15:19.834 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000323803 2024-09-18T10:15:19.834 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:19.834 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:19.834 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:19.834 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.834 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.834 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:211 3.7s0 3:e4da5b5e:::smithi18925896-43:head [read 0~720896,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:19.834 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:52.694469+0000", 2024-09-18T10:15:19.834 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 27.101420639000001, 2024-09-18T10:15:19.835 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.51035930399999996, 2024-09-18T10:15:19.835 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:19.835 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:19.835 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:19.835 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:19.835 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:19.835 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 211 2024-09-18T10:15:19.835 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.835 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:19.835 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.835 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:19.835 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.694469+0000", 2024-09-18T10:15:19.835 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:19.836 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.836 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.836 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:19.836 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.694469+0000", 2024-09-18T10:15:19.836 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:19.836 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.836 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.836 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:19.836 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.694469+0000", 2024-09-18T10:15:19.836 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.999999 2024-09-18T10:15:19.836 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.836 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.836 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:19.837 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.694475+0000", 2024-09-18T10:15:19.837 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7349999999999999e-06 2024-09-18T10:15:19.837 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.837 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.837 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:19.837 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.694489+0000", 2024-09-18T10:15:19.837 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4209999999999999e-05 2024-09-18T10:15:19.837 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.837 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.837 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:19.837 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.694498+0000", 2024-09-18T10:15:19.837 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.4339999999999992e-06 2024-09-18T10:15:19.837 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.838 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.838 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:19.838 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.698348+0000", 2024-09-18T10:15:19.838 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0038499670000000001 2024-09-18T10:15:19.838 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.838 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.838 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:19.838 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.698366+0000", 2024-09-18T10:15:19.838 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7999000000000002e-05 2024-09-18T10:15:19.838 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.838 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.838 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:19.838 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:53.000010+0000", 2024-09-18T10:15:19.839 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.30164412699999998 2024-09-18T10:15:19.839 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.839 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.839 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:19.839 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:53.000223+0000", 2024-09-18T10:15:19.839 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000213169 2024-09-18T10:15:19.839 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.839 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.839 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:19.839 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:53.204829+0000", 2024-09-18T10:15:19.839 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.20460561299999999 2024-09-18T10:15:19.839 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:19.839 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:19.839 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:19.840 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.840 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.840 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(osd.10.22:2 3.7s0 3:ee962131:::smithi18925896-29:head [assert-version v6,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:19.840 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:52.798083+0000", 2024-09-18T10:15:19.840 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 26.997806746999998, 2024-09-18T10:15:19.840 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.60989146800000005, 2024-09-18T10:15:19.840 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:19.840 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:19.840 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:19.840 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.840 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:19.840 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.798083+0000", 2024-09-18T10:15:19.840 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:19.841 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.841 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.841 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:19.841 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.798083+0000", 2024-09-18T10:15:19.841 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:19.841 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.841 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.841 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:19.841 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.798080+0000", 2024-09-18T10:15:19.841 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999962 2024-09-18T10:15:19.841 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.841 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.841 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:19.841 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.798097+0000", 2024-09-18T10:15:19.842 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7657000000000001e-05 2024-09-18T10:15:19.842 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.842 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.842 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:19.842 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.798127+0000", 2024-09-18T10:15:19.842 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9532000000000001e-05 2024-09-18T10:15:19.842 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.842 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.842 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:19.842 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.898205+0000", 2024-09-18T10:15:19.842 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100078525 2024-09-18T10:15:19.842 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.842 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.843 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:19.843 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:53.204957+0000", 2024-09-18T10:15:19.843 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.30675138899999999 2024-09-18T10:15:19.843 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.843 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.843 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:19.843 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:53.205092+0000", 2024-09-18T10:15:19.843 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00013556700000000001 2024-09-18T10:15:19.843 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.843 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.843 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:19.843 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:53.407975+0000", 2024-09-18T10:15:19.843 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.20288240799999999 2024-09-18T10:15:19.844 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:19.844 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:19.844 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:19.844 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.844 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.844 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:412 3.1s0 3:82c9401b:::smithi18925896-33:head [read 0~540672,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-18T10:15:19.844 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:07.554121+0000", 2024-09-18T10:15:19.844 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 12.241768905000001, 2024-09-18T10:15:19.844 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.41171843299999999, 2024-09-18T10:15:19.844 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:19.845 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:19.845 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:19.845 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:19.845 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:19.845 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 412 2024-09-18T10:15:19.845 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.845 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:19.845 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.845 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:19.845 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.554121+0000", 2024-09-18T10:15:19.845 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:19.845 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.846 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.846 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:19.846 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.554121+0000", 2024-09-18T10:15:19.846 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:19.846 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.846 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.846 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:19.846 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.554120+0000", 2024-09-18T10:15:19.846 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:19.846 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.846 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.846 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:19.846 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.554128+0000", 2024-09-18T10:15:19.847 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.943e-06 2024-09-18T10:15:19.847 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.847 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.847 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:19.847 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.554146+0000", 2024-09-18T10:15:19.847 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7880000000000002e-05 2024-09-18T10:15:19.847 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.847 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.847 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:19.847 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.554162+0000", 2024-09-18T10:15:19.847 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5931999999999999e-05 2024-09-18T10:15:19.847 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.847 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.848 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:19.848 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.554405+0000", 2024-09-18T10:15:19.848 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000243515 2024-09-18T10:15:19.848 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.848 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.848 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:19.848 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.554428+0000", 2024-09-18T10:15:19.848 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2476e-05 2024-09-18T10:15:19.848 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.848 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.848 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:19.848 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.659266+0000", 2024-09-18T10:15:19.848 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10483785299999999 2024-09-18T10:15:19.849 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.849 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.849 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:19.849 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.659461+0000", 2024-09-18T10:15:19.849 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00019562200000000001 2024-09-18T10:15:19.849 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.849 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.849 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:19.849 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.965840+0000", 2024-09-18T10:15:19.849 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.30637847200000001 2024-09-18T10:15:19.849 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:19.849 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:19.850 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:19.850 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.850 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.850 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:437 3.1s0 3:82c9401b:::smithi18925896-33:head [copy-from ver 2] snapc 13=[13,12,11,10,f,e,b,4] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-18T10:15:19.850 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:11.831109+0000", 2024-09-18T10:15:19.850 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 7.9647809689999995, 2024-09-18T10:15:19.850 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.709244349, 2024-09-18T10:15:19.850 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:19.850 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:19.850 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:19.850 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:19.850 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:19.850 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 437 2024-09-18T10:15:19.850 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.851 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:19.851 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.851 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:19.851 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:11.831109+0000", 2024-09-18T10:15:19.851 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:19.851 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.851 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.851 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:19.851 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:11.831109+0000", 2024-09-18T10:15:19.851 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:19.851 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.851 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.851 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:19.852 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:11.831106+0000", 2024-09-18T10:15:19.852 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999971 2024-09-18T10:15:19.852 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.852 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.852 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:19.852 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:11.831122+0000", 2024-09-18T10:15:19.852 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5971e-05 2024-09-18T10:15:19.852 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.852 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.852 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:19.852 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:11.831155+0000", 2024-09-18T10:15:19.852 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.3259999999999997e-05 2024-09-18T10:15:19.853 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.853 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.853 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:19.853 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:11.831172+0000", 2024-09-18T10:15:19.853 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.6433999999999999e-05 2024-09-18T10:15:19.853 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.853 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.853 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:19.853 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:11.931398+0000", 2024-09-18T10:15:19.853 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10022644 2024-09-18T10:15:19.853 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.854 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.854 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op must wait for map", 2024-09-18T10:15:19.854 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:11.931412+0000", 2024-09-18T10:15:19.854 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3586e-05 2024-09-18T10:15:19.854 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.854 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.854 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:19.854 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:11.934231+0000", 2024-09-18T10:15:19.854 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0028191819999999999 2024-09-18T10:15:19.854 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.854 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.854 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:19.854 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:11.934501+0000", 2024-09-18T10:15:19.855 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00026996099999999999 2024-09-18T10:15:19.855 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.855 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.855 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:19.855 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.352429+0000", 2024-09-18T10:15:19.855 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.41792792699999998 2024-09-18T10:15:19.855 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.855 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.855 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:19.855 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.356112+0000", 2024-09-18T10:15:19.855 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0036833199999999999 2024-09-18T10:15:19.855 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.855 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.856 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-18T10:15:19.856 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.540229+0000", 2024-09-18T10:15:19.856 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.18411702599999999 2024-09-18T10:15:19.856 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.856 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.856 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:19.856 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.540354+0000", 2024-09-18T10:15:19.856 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00012415399999999999 2024-09-18T10:15:19.856 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:19.856 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:19.856 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:19.856 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.856 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.857 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:438 3.1s0 3:82c9401b:::smithi18925896-33:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e56)", 2024-09-18T10:15:19.857 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:11.831206+0000", 2024-09-18T10:15:19.857 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 7.9646843870000001, 2024-09-18T10:15:19.857 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.52445609999999998, 2024-09-18T10:15:19.857 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:19.857 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:19.857 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:19.857 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:19.857 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:19.857 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 438 2024-09-18T10:15:19.857 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.857 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:19.857 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.858 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:19.858 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:11.831206+0000", 2024-09-18T10:15:19.858 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:19.858 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.858 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.858 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:19.858 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:11.831206+0000", 2024-09-18T10:15:19.858 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:19.858 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.858 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.858 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:19.858 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:11.831205+0000", 2024-09-18T10:15:19.858 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.999999 2024-09-18T10:15:19.858 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.859 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.859 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:19.859 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:11.831210+0000", 2024-09-18T10:15:19.859 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8539999999999996e-06 2024-09-18T10:15:19.859 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.859 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.859 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:19.859 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:11.831230+0000", 2024-09-18T10:15:19.859 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9899999999999999e-05 2024-09-18T10:15:19.859 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.859 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.859 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:19.859 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:11.831238+0000", 2024-09-18T10:15:19.859 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.7639999999999994e-06 2024-09-18T10:15:19.860 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.860 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.860 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:19.860 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:11.931577+0000", 2024-09-18T10:15:19.860 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10033874299999999 2024-09-18T10:15:19.860 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.860 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.860 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:19.860 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:11.931604+0000", 2024-09-18T10:15:19.860 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.7239000000000001e-05 2024-09-18T10:15:19.860 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.860 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.861 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:19.861 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:11.935139+0000", 2024-09-18T10:15:19.861 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00353484 2024-09-18T10:15:19.861 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.861 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.861 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for blocked object", 2024-09-18T10:15:19.861 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:11.935380+0000", 2024-09-18T10:15:19.861 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00024033400000000001 2024-09-18T10:15:19.861 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.861 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.861 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:19.861 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.355102+0000", 2024-09-18T10:15:19.861 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.41972236200000002 2024-09-18T10:15:19.861 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.862 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.862 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:19.862 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.355403+0000", 2024-09-18T10:15:19.862 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000301537 2024-09-18T10:15:19.862 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.862 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.862 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:19.862 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.355662+0000", 2024-09-18T10:15:19.862 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000258421 2024-09-18T10:15:19.862 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:19.862 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:19.862 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:19.862 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.863 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.863 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:441 3.17s0 3:ead53b0f:::smithi18925896-25:head [append 0~606208 in=606208b] snapc 13=[13,12,11,10,f,e,b] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-18T10:15:19.863 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:12.856902+0000", 2024-09-18T10:15:19.863 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 6.9389878669999998, 2024-09-18T10:15:19.863 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.80182116999999997, 2024-09-18T10:15:19.863 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:19.863 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:19.863 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:19.863 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:19.863 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:19.863 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 441 2024-09-18T10:15:19.863 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.863 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:19.864 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.864 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:19.864 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.856902+0000", 2024-09-18T10:15:19.864 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:19.864 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.864 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.864 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:19.864 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.856902+0000", 2024-09-18T10:15:19.864 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:19.864 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.864 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.864 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:19.864 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.856900+0000", 2024-09-18T10:15:19.865 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999976 2024-09-18T10:15:19.865 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.865 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.865 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:19.865 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.857946+0000", 2024-09-18T10:15:19.865 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0010459620000000001 2024-09-18T10:15:19.865 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.865 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.865 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:19.865 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.857973+0000", 2024-09-18T10:15:19.865 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6758999999999999e-05 2024-09-18T10:15:19.865 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.865 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.866 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:19.866 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.857983+0000", 2024-09-18T10:15:19.866 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0105000000000001e-05 2024-09-18T10:15:19.866 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.866 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.866 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:19.866 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.946351+0000", 2024-09-18T10:15:19.866 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.088368302999999995 2024-09-18T10:15:19.866 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.866 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.866 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:19.866 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.946465+0000", 2024-09-18T10:15:19.866 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00011425999999999999 2024-09-18T10:15:19.866 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.867 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.867 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:19.867 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.947656+0000", 2024-09-18T10:15:19.867 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0011901710000000001 2024-09-18T10:15:19.867 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.867 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.867 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:19.867 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.040655+0000", 2024-09-18T10:15:19.867 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.092999126000000001 2024-09-18T10:15:19.867 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.867 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.867 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-18T10:15:19.867 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.658646+0000", 2024-09-18T10:15:19.868 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.61799096399999998 2024-09-18T10:15:19.868 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.868 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.868 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:19.868 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.658723+0000", 2024-09-18T10:15:19.868 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7805000000000005e-05 2024-09-18T10:15:19.868 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:19.868 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:19.868 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:19.868 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.868 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.868 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:442 3.17s0 3:ead53b0f:::smithi18925896-25:head [append 0~720896 in=720896b] snapc 13=[13,12,11,10,f,e,b] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-18T10:15:19.868 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:12.861303+0000", 2024-09-18T10:15:19.869 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 6.9345874729999997, 2024-09-18T10:15:19.869 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.89783094100000005, 2024-09-18T10:15:19.869 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:19.869 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:19.869 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:19.869 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:19.869 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:19.869 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 442 2024-09-18T10:15:19.869 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.869 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:19.869 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.869 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:19.869 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.861303+0000", 2024-09-18T10:15:19.869 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:19.870 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.870 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.870 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:19.870 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.861303+0000", 2024-09-18T10:15:19.870 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:19.870 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.870 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.870 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:19.870 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.861301+0000", 2024-09-18T10:15:19.870 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:19.870 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.870 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.870 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:19.870 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.862343+0000", 2024-09-18T10:15:19.871 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0010422820000000001 2024-09-18T10:15:19.871 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.871 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.871 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:19.871 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.862381+0000", 2024-09-18T10:15:19.871 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7407000000000003e-05 2024-09-18T10:15:19.871 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.871 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.871 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:19.871 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.862391+0000", 2024-09-18T10:15:19.871 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0572e-05 2024-09-18T10:15:19.871 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.871 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.871 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:19.872 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.948744+0000", 2024-09-18T10:15:19.872 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.086352683999999999 2024-09-18T10:15:19.872 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.872 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.872 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:19.872 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.948864+0000", 2024-09-18T10:15:19.872 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000119855 2024-09-18T10:15:19.872 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.872 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.872 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:19.872 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.949834+0000", 2024-09-18T10:15:19.872 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00097020199999999998 2024-09-18T10:15:19.872 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.873 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.873 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:19.873 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.040659+0000", 2024-09-18T10:15:19.873 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.090824659000000002 2024-09-18T10:15:19.873 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.873 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.873 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-18T10:15:19.873 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.759042+0000", 2024-09-18T10:15:19.873 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.71838323100000001 2024-09-18T10:15:19.873 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.873 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.873 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:19.873 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.759134+0000", 2024-09-18T10:15:19.873 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.1760000000000002e-05 2024-09-18T10:15:19.874 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:19.874 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:19.874 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:19.874 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.874 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.874 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:443 3.17s0 3:ead53b0f:::smithi18925896-25:head [append 0~49152 in=49152b] snapc 13=[13,12,11,10,f,e,b] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-18T10:15:19.874 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:12.862402+0000", 2024-09-18T10:15:19.874 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 6.9334876950000002, 2024-09-18T10:15:19.874 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.99734852100000004, 2024-09-18T10:15:19.874 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:19.874 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:19.874 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:19.874 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:19.875 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:19.875 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 443 2024-09-18T10:15:19.875 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.875 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:19.875 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.875 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:19.875 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.862402+0000", 2024-09-18T10:15:19.875 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:19.875 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.875 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.875 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:19.875 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.862402+0000", 2024-09-18T10:15:19.875 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:19.875 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.876 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.876 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:19.876 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.862402+0000", 2024-09-18T10:15:19.876 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.999999 2024-09-18T10:15:19.876 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.876 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.876 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:19.876 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.862441+0000", 2024-09-18T10:15:19.876 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9116000000000003e-05 2024-09-18T10:15:19.876 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.876 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.876 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:19.876 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.862455+0000", 2024-09-18T10:15:19.877 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4501999999999999e-05 2024-09-18T10:15:19.877 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.877 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.877 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:19.877 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.862462+0000", 2024-09-18T10:15:19.877 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.4659999999999996e-06 2024-09-18T10:15:19.877 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.877 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.877 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:19.877 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.950570+0000", 2024-09-18T10:15:19.877 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.088107850000000001 2024-09-18T10:15:19.877 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.877 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.877 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:19.878 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.950675+0000", 2024-09-18T10:15:19.878 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000105445 2024-09-18T10:15:19.878 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.878 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.878 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:19.878 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.951279+0000", 2024-09-18T10:15:19.878 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00060383599999999996 2024-09-18T10:15:19.878 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.878 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.878 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:19.878 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.040661+0000", 2024-09-18T10:15:19.878 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.089382448000000003 2024-09-18T10:15:19.878 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.878 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.879 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-18T10:15:19.879 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.859668+0000", 2024-09-18T10:15:19.879 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.81900669800000003 2024-09-18T10:15:19.879 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.879 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.879 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:19.879 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.859751+0000", 2024-09-18T10:15:19.879 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.2943999999999997e-05 2024-09-18T10:15:19.879 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:19.879 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:19.879 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:19.879 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.879 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.880 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:444 3.17s0 3:ead53b0f:::smithi18925896-25:head [setxattr _header (57) in=64b] snapc 13=[13,12,11,10,f,e,b] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-18T10:15:19.880 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:12.862469+0000", 2024-09-18T10:15:19.880 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 6.9334214850000002, 2024-09-18T10:15:19.880 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.99750343200000002, 2024-09-18T10:15:19.880 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:19.880 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:19.880 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:19.880 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:19.880 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:19.880 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 444 2024-09-18T10:15:19.880 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.880 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:19.880 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.880 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:19.881 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.862469+0000", 2024-09-18T10:15:19.881 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:19.881 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.881 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.881 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:19.881 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.862469+0000", 2024-09-18T10:15:19.881 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:19.881 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.881 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.881 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:19.881 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.862468+0000", 2024-09-18T10:15:19.881 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:19.881 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.882 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.882 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:19.882 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.862473+0000", 2024-09-18T10:15:19.882 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.5839999999999996e-06 2024-09-18T10:15:19.882 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.882 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.882 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:19.882 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.862483+0000", 2024-09-18T10:15:19.882 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0188e-05 2024-09-18T10:15:19.882 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.882 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.882 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:19.882 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.862487+0000", 2024-09-18T10:15:19.882 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6580000000000001e-06 2024-09-18T10:15:19.883 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.883 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.883 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:19.883 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.952057+0000", 2024-09-18T10:15:19.883 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.089569930000000006 2024-09-18T10:15:19.883 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.883 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.883 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:19.883 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.952227+0000", 2024-09-18T10:15:19.883 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000170106 2024-09-18T10:15:19.883 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.883 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.883 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:19.884 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.953127+0000", 2024-09-18T10:15:19.884 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00089958300000000002 2024-09-18T10:15:19.884 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.884 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.884 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:19.884 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.040664+0000", 2024-09-18T10:15:19.884 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.087536931999999998 2024-09-18T10:15:19.884 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.884 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.884 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-18T10:15:19.884 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.859892+0000", 2024-09-18T10:15:19.884 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.81922844699999997 2024-09-18T10:15:19.884 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.884 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.885 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:19.885 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.859972+0000", 2024-09-18T10:15:19.885 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.9662999999999999e-05 2024-09-18T10:15:19.885 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:19.885 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:19.885 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:19.885 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.885 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.885 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:445 3.17s0 3:ead53b0f:::smithi18925896-25:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e57)", 2024-09-18T10:15:19.885 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:12.862493+0000", 2024-09-18T10:15:19.885 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 6.9333976169999998, 2024-09-18T10:15:19.885 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1938188730000001, 2024-09-18T10:15:19.885 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:19.886 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:19.886 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:19.886 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:19.886 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:19.886 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 445 2024-09-18T10:15:19.886 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.886 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:19.886 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.886 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:19.886 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.862493+0000", 2024-09-18T10:15:19.886 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:19.886 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.886 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.886 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:19.887 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.862493+0000", 2024-09-18T10:15:19.887 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:19.887 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.887 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.887 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:19.887 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.862492+0000", 2024-09-18T10:15:19.887 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:19.887 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.887 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.887 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:19.887 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.862495+0000", 2024-09-18T10:15:19.887 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1829999999999998e-06 2024-09-18T10:15:19.887 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.887 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.888 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:19.888 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.862506+0000", 2024-09-18T10:15:19.888 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0386e-05 2024-09-18T10:15:19.888 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.888 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.888 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:19.888 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.962582+0000", 2024-09-18T10:15:19.888 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100076741 2024-09-18T10:15:19.888 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.888 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.888 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:19.888 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.962632+0000", 2024-09-18T10:15:19.888 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9364999999999999e-05 2024-09-18T10:15:19.889 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.889 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.889 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for rw locks", 2024-09-18T10:15:19.889 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.962715+0000", 2024-09-18T10:15:19.889 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.3617999999999998e-05 2024-09-18T10:15:19.889 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.889 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.889 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:19.889 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.860479+0000", 2024-09-18T10:15:19.889 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.89776372699999996 2024-09-18T10:15:19.889 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.889 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.889 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:19.890 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.860908+0000", 2024-09-18T10:15:19.890 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00042866099999999999 2024-09-18T10:15:19.890 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.890 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.890 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:19.890 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:14.056311+0000", 2024-09-18T10:15:19.890 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.19540364199999999 2024-09-18T10:15:19.890 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:19.890 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:19.890 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:19.890 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.890 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.890 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWriteReply(3.7s0 57/40 ECSubWriteReply(tid=110, last_complete=57'26, committed=1, applied=1))", 2024-09-18T10:15:19.890 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:13.156807+0000", 2024-09-18T10:15:19.891 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 6.6390835399999997, 2024-09-18T10:15:19.891 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.50166774700000005, 2024-09-18T10:15:19.891 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:19.891 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:19.891 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:19.891 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.891 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:19.891 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.156807+0000", 2024-09-18T10:15:19.891 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:19.891 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.891 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.891 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:19.891 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.156807+0000", 2024-09-18T10:15:19.891 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:19.892 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.892 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.892 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:19.892 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.156806+0000", 2024-09-18T10:15:19.892 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:19.892 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.892 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.892 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:19.892 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.156810+0000", 2024-09-18T10:15:19.892 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.535e-06 2024-09-18T10:15:19.892 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.892 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.892 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:19.893 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.156819+0000", 2024-09-18T10:15:19.893 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.6940000000000007e-06 2024-09-18T10:15:19.893 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.893 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.893 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:19.893 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.156824+0000", 2024-09-18T10:15:19.893 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2899999999999996e-06 2024-09-18T10:15:19.893 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.893 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.893 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:19.893 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.658462+0000", 2024-09-18T10:15:19.893 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.50163808600000004 2024-09-18T10:15:19.893 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.894 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.894 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:19.894 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.658474+0000", 2024-09-18T10:15:19.894 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2619999999999999e-05 2024-09-18T10:15:19.894 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:19.894 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:19.894 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:19.894 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.894 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.894 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWriteReply(3.7s0 57/40 ECSubWriteReply(tid=110, last_complete=57'26, committed=1, applied=1))", 2024-09-18T10:15:19.894 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:13.351324+0000", 2024-09-18T10:15:19.894 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 6.4445664679999997, 2024-09-18T10:15:19.894 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.50822421699999998, 2024-09-18T10:15:19.894 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:19.895 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:19.895 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:19.895 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.895 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:19.895 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.351324+0000", 2024-09-18T10:15:19.895 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:19.895 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.895 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.895 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:19.895 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.351324+0000", 2024-09-18T10:15:19.895 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:19.895 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.895 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.896 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:19.896 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.351323+0000", 2024-09-18T10:15:19.896 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:19.896 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.896 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.896 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:19.896 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.351326+0000", 2024-09-18T10:15:19.896 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.943e-06 2024-09-18T10:15:19.896 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.896 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.896 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:19.896 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.351337+0000", 2024-09-18T10:15:19.896 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0922e-05 2024-09-18T10:15:19.897 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.897 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.897 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:19.897 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.451449+0000", 2024-09-18T10:15:19.897 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10011148 2024-09-18T10:15:19.897 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.897 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.897 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:19.897 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.859533+0000", 2024-09-18T10:15:19.897 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.40808401300000002 2024-09-18T10:15:19.897 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.897 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.897 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:19.898 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.859548+0000", 2024-09-18T10:15:19.898 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5153e-05 2024-09-18T10:15:19.898 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:19.898 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:19.898 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:19.898 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.898 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.898 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWriteReply(3.17s0 57/40 ECSubWriteReply(tid=108, last_complete=57'24, committed=1, applied=1))", 2024-09-18T10:15:19.898 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:13.447023+0000", 2024-09-18T10:15:19.898 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 6.3488671380000001, 2024-09-18T10:15:19.898 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.41274579700000003, 2024-09-18T10:15:19.898 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:19.898 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:19.899 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:19.899 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.899 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:19.899 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.447023+0000", 2024-09-18T10:15:19.899 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:19.899 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.899 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.899 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:19.899 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.447023+0000", 2024-09-18T10:15:19.899 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:19.899 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.899 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.899 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:19.899 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.447023+0000", 2024-09-18T10:15:19.900 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:19.900 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.900 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.900 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:19.900 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.447027+0000", 2024-09-18T10:15:19.900 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0539999999999996e-06 2024-09-18T10:15:19.900 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.900 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.900 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:19.900 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.447040+0000", 2024-09-18T10:15:19.900 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2873e-05 2024-09-18T10:15:19.900 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.900 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.901 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:19.901 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.447045+0000", 2024-09-18T10:15:19.901 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7729999999999996e-06 2024-09-18T10:15:19.901 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.901 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.901 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:19.901 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.859560+0000", 2024-09-18T10:15:19.901 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.41251467400000003 2024-09-18T10:15:19.901 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.901 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.901 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:19.901 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.859769+0000", 2024-09-18T10:15:19.901 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00020882299999999999 2024-09-18T10:15:19.901 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:19.902 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:19.902 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:19.902 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.902 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.902 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWriteReply(3.17s0 57/40 ECSubWriteReply(tid=109, last_complete=57'25, committed=1, applied=1))", 2024-09-18T10:15:19.902 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:13.447052+0000", 2024-09-18T10:15:19.902 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 6.3488384780000002, 2024-09-18T10:15:19.902 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.41337315400000002, 2024-09-18T10:15:19.902 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:19.902 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:19.902 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:19.902 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.902 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:19.903 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.447052+0000", 2024-09-18T10:15:19.903 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:19.903 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.903 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.903 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:19.903 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.447052+0000", 2024-09-18T10:15:19.903 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:19.903 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.903 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.903 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:19.903 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.447051+0000", 2024-09-18T10:15:19.903 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:19.903 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.904 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.904 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:19.904 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.447055+0000", 2024-09-18T10:15:19.904 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.743e-06 2024-09-18T10:15:19.904 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.904 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.904 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:19.904 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.447063+0000", 2024-09-18T10:15:19.904 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1200000000000002e-06 2024-09-18T10:15:19.904 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.904 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.904 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:19.904 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.547143+0000", 2024-09-18T10:15:19.905 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100079747 2024-09-18T10:15:19.905 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.905 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.905 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:19.905 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.859819+0000", 2024-09-18T10:15:19.905 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.31267598499999999 2024-09-18T10:15:19.905 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:19.905 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:19.905 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:19.905 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.860425+0000", 2024-09-18T10:15:19.905 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00060597600000000004 2024-09-18T10:15:19.905 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:19.905 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:19.906 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:19.906 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:19.906 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:19.906 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-18T10:15:19.907 INFO:tasks.rados.rados.0.smithi189.stdout:215: done (6 left) 2024-09-18T10:15:19.907 INFO:tasks.rados.rados.0.smithi189.stdout:216: snap_remove snap 14 2024-09-18T10:15:19.907 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-18T10:15:19.908 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-09-18T10:15:19.909 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-18T10:15:19.909 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-18T10:15:19.909 INFO:teuthology.orchestra.run.smithi029.stderr:noscrub is set 2024-09-18T10:15:19.909 INFO:teuthology.orchestra.run.smithi029.stderr:reweighted osd.10 to 1 (10000) 2024-09-18T10:15:19.910 INFO:tasks.ceph.osd.10.smithi154.stderr:2024-09-18T10:15:19.899+0000 7fb73221a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 22091) UID: 0 2024-09-18T10:15:19.927 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 11 1 2024-09-18T10:15:19.963 INFO:teuthology.orchestra.run.smithi029.stdout:osd.6: {} 2024-09-18T10:15:19.988 INFO:teuthology.orchestra.run.smithi029.stdout:osd.7: {} 2024-09-18T10:15:20.008 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-18T10:15:20.005+0000 7f1981873700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22132) UID: 0 2024-09-18T10:15:20.036 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-18T10:15:20.036 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-18T10:15:20.036 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:20.036 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWrite(3.16s3 64/62 ECSubWrite(tid=2, reqid=client.4632.0:508, at_version=64'16, trim_to=0'0, roll_forward_to=0'0))", 2024-09-18T10:15:20.037 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:19.968947+0000", 2024-09-18T10:15:20.037 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.066370855000000006, 2024-09-18T10:15:20.037 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.066410640000000007, 2024-09-18T10:15:20.037 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:20.037 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-18T10:15:20.037 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:20.037 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:20.037 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:20.037 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:19.968947+0000", 2024-09-18T10:15:20.037 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:20.037 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:20.037 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:20.037 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:20.038 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:19.968947+0000", 2024-09-18T10:15:20.038 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:20.038 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:20.038 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:20.038 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:20.038 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:19.968947+0000", 2024-09-18T10:15:20.038 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:20.038 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:20.038 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:20.038 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:20.038 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:19.970994+0000", 2024-09-18T10:15:20.038 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0020469379999999999 2024-09-18T10:15:20.038 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:20.039 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:20.039 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:20.039 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:19.971029+0000", 2024-09-18T10:15:20.039 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.5392999999999998e-05 2024-09-18T10:15:20.039 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:20.039 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:20.039 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:20.039 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:19.971050+0000", 2024-09-18T10:15:20.039 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.018e-05 2024-09-18T10:15:20.039 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:20.039 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:20.039 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:20.039 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:20.039 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-18T10:15:20.040 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 1 2024-09-18T10:15:20.040 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-18T10:15:20.040 INFO:teuthology.orchestra.run.smithi029.stdout:osd.8: {} 2024-09-18T10:15:20.047 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-09-18T10:15:20.060 INFO:teuthology.orchestra.run.smithi029.stdout:osd.9: {} 2024-09-18T10:15:20.089 INFO:teuthology.orchestra.run.smithi029.stdout:osd.10: {} 2024-09-18T10:15:20.109 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-09-18T10:15:20.103+0000 7fdfa8698700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22087) UID: 0 2024-09-18T10:15:20.124 INFO:teuthology.orchestra.run.smithi029.stdout:osd.11: {} 2024-09-18T10:15:20.158 INFO:teuthology.orchestra.run.smithi029.stdout:osd.12: {} 2024-09-18T10:15:20.176 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:20.176 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:20.184 INFO:teuthology.orchestra.run.smithi029.stdout:osd.13: {} 2024-09-18T10:15:20.187 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-09-18T10:15:20.205 INFO:teuthology.orchestra.run.smithi029.stdout:osd.14: {} 2024-09-18T10:15:20.210 INFO:tasks.ceph.osd.14.smithi154.stderr:2024-09-18T10:15:20.207+0000 7f75a19e7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 22093) UID: 0 2024-09-18T10:15:20.297 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:20.297 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:20.306 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_ops_in_flight 2024-09-18T10:15:20.310 INFO:tasks.ceph.osd.11.smithi189.stderr:2024-09-18T10:15:20.306+0000 7f718b60a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 22400) UID: 0 2024-09-18T10:15:20.317 INFO:teuthology.orchestra.run.smithi029.stdout:osd.15: {} 2024-09-18T10:15:20.320 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T10:15:20.320 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T10:15:20.320 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T10:15:20.320 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T10:15:20.320 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T10:15:20.320 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T10:15:20.320 INFO:teuthology.orchestra.run.smithi029.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T10:15:20.320 INFO:teuthology.orchestra.run.smithi029.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T10:15:20.321 INFO:teuthology.orchestra.run.smithi029.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-18T10:15:20.321 INFO:teuthology.orchestra.run.smithi029.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-18T10:15:20.321 INFO:teuthology.orchestra.run.smithi029.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-18T10:15:20.321 INFO:teuthology.orchestra.run.smithi029.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-18T10:15:20.321 INFO:teuthology.orchestra.run.smithi029.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-18T10:15:20.321 INFO:teuthology.orchestra.run.smithi029.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-18T10:15:20.321 INFO:teuthology.orchestra.run.smithi029.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-18T10:15:20.321 INFO:teuthology.orchestra.run.smithi029.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-18T10:15:20.407 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:20.408 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:20.411 INFO:tasks.ceph.osd.4.smithi029.stderr:2024-09-18T10:15:20.406+0000 7f550cf6e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22189) UID: 0 2024-09-18T10:15:20.417 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_blocked_ops 2024-09-18T10:15:20.513 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-18T10:15:20.509+0000 7f00046a8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 22136) UID: 0 2024-09-18T10:15:20.516 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:20.516 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:20.525 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_historic_ops 2024-09-18T10:15:20.614 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-18T10:15:20.610+0000 7f302f2c3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22187) UID: 0 2024-09-18T10:15:20.623 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:20.624 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:20.633 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T10:15:20.715 INFO:tasks.ceph.osd.4.smithi029.stderr:2024-09-18T10:15:20.710+0000 7f550cf6e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22189) UID: 0 2024-09-18T10:15:20.744 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:20.744 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:20.754 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T10:15:20.816 INFO:tasks.ceph.osd.4.smithi029.stderr:2024-09-18T10:15:20.810+0000 7f550cf6e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22189) UID: 0 2024-09-18T10:15:20.857 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:20.857 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:20.866 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T10:15:20.905 INFO:tasks.rados.rados.0.smithi189.stdout:216: done (6 left) 2024-09-18T10:15:20.905 INFO:tasks.rados.rados.0.smithi189.stdout:append oid 46 current snap is 17 2024-09-18T10:15:20.905 INFO:tasks.rados.rados.0.smithi189.stdout:209: finishing copy_from racing read to smithi18925896-17 2024-09-18T10:15:20.905 INFO:tasks.rados.rados.0.smithi189.stdout:217: seq_num 96 ranges {2048000=458752,2506752=688128,3194880=475136} 2024-09-18T10:15:20.917 INFO:teuthology.orchestra.run.smithi029.stderr:reweighted osd.11 to 1 (10000) 2024-09-18T10:15:20.917 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-18T10:15:20.914+0000 7f302f2c3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22187) UID: 0 2024-09-18T10:15:20.924 INFO:tasks.rados.rados.0.smithi189.stdout:217: writing smithi18925896-46 from 2048000 to 2506752 tid 1 2024-09-18T10:15:20.928 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 12 1 2024-09-18T10:15:20.928 INFO:tasks.rados.rados.0.smithi189.stdout:217: writing smithi18925896-46 from 2506752 to 3194880 tid 2 2024-09-18T10:15:20.931 INFO:tasks.rados.rados.0.smithi189.stdout:217: writing smithi18925896-46 from 3194880 to 3670016 tid 3 2024-09-18T10:15:20.931 INFO:tasks.rados.rados.0.smithi189.stdout:218: copy_from oid 47 from oid 42 current snap is 17 2024-09-18T10:15:20.931 INFO:tasks.rados.rados.0.smithi189.stdout:212: expect (ObjNum 33 snap 0 seq_num 33) 2024-09-18T10:15:20.950 INFO:tasks.rados.rados.0.smithi189.stdout:205: finishing copy_from racing read to smithi18925896-41 2024-09-18T10:15:20.950 INFO:tasks.rados.rados.0.smithi189.stdout:213: finishing write tid 1 to smithi18925896-15 2024-09-18T10:15:20.950 INFO:tasks.rados.rados.0.smithi189.stdout:211: finishing write tid 1 to smithi18925896-5 2024-09-18T10:15:20.950 INFO:tasks.rados.rados.0.smithi189.stdout:212: done (7 left) 2024-09-18T10:15:20.951 INFO:tasks.rados.rados.0.smithi189.stdout:219: delete oid 21 current snap is 17 2024-09-18T10:15:20.951 INFO:tasks.rados.rados.0.smithi189.stdout:211: finishing write tid 2 to smithi18925896-5 2024-09-18T10:15:20.951 INFO:tasks.rados.rados.0.smithi189.stdout:211: finishing write tid 3 to smithi18925896-5 2024-09-18T10:15:20.951 INFO:tasks.rados.rados.0.smithi189.stdout:211: finishing write tid 4 to smithi18925896-5 2024-09-18T10:15:20.951 INFO:tasks.rados.rados.0.smithi189.stdout:213: finishing write tid 2 to smithi18925896-15 2024-09-18T10:15:20.951 INFO:tasks.rados.rados.0.smithi189.stdout:213: finishing write tid 3 to smithi18925896-15 2024-09-18T10:15:20.951 INFO:tasks.rados.rados.0.smithi189.stdout:213: finishing write tid 4 to smithi18925896-15 2024-09-18T10:15:20.951 INFO:tasks.rados.rados.0.smithi189.stdout:214: finishing rollback tid 0 to smithi18925896-10 2024-09-18T10:15:20.951 INFO:tasks.rados.rados.0.smithi189.stdout:214: finishing rollback tid 1 to smithi18925896-10 2024-09-18T10:15:20.951 INFO:tasks.rados.rados.0.smithi189.stdout:209: finishing copy_from to smithi18925896-17 2024-09-18T10:15:20.951 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 17 v 4 (ObjNum 12 snap 0 seq_num 12) dirty exists 2024-09-18T10:15:20.951 INFO:tasks.rados.rados.0.smithi189.stdout:214: finishing rollback tid 2 to smithi18925896-10 2024-09-18T10:15:20.951 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 10 v 56 (ObjNum 68 snap 6 seq_num 68) dirty exists 2024-09-18T10:15:20.951 INFO:tasks.rados.rados.0.smithi189.stdout:205: finishing copy_from to smithi18925896-41 2024-09-18T10:15:20.952 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 41 v 61 (ObjNum 84 snap 14 seq_num 84) dirty exists 2024-09-18T10:15:20.952 INFO:tasks.rados.rados.0.smithi189.stdout:211: finishing write tid 5 to smithi18925896-5 2024-09-18T10:15:20.952 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 5 v 37 (ObjNum 94 snap 17 seq_num 94) dirty exists 2024-09-18T10:15:20.952 INFO:tasks.rados.rados.0.smithi189.stdout:211: left oid 5 (ObjNum 94 snap 17 seq_num 94) 2024-09-18T10:15:20.952 INFO:tasks.rados.rados.0.smithi189.stdout:213: finishing write tid 5 to smithi18925896-15 2024-09-18T10:15:20.952 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 15 v 19 (ObjNum 95 snap 17 seq_num 95) dirty exists 2024-09-18T10:15:20.952 INFO:tasks.rados.rados.0.smithi189.stdout:213: left oid 15 (ObjNum 95 snap 17 seq_num 95) 2024-09-18T10:15:20.973 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:20.974 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:20.983 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T10:15:21.021 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-18T10:15:21.014+0000 7f302f2c3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22187) UID: 0 2024-09-18T10:15:21.081 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:21.081 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:21.090 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T10:15:21.118 INFO:tasks.ceph.osd.14.smithi154.stderr:2024-09-18T10:15:21.111+0000 7f75a19e7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 22093) UID: 0 2024-09-18T10:15:21.186 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:21.186 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:21.195 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T10:15:21.221 INFO:tasks.ceph.osd.15.smithi189.stderr:2024-09-18T10:15:21.218+0000 7fd0e725e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 22401) UID: 0 2024-09-18T10:15:21.292 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:21.292 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:21.301 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-09-18T10:15:21.319 INFO:tasks.rados.rados.0.smithi189.stdout:218: finishing copy_from racing read to smithi18925896-47 2024-09-18T10:15:21.322 INFO:tasks.ceph.osd.8.smithi029.stderr:2024-09-18T10:15:21.318+0000 7f2555c27700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 22192) UID: 0 2024-09-18T10:15:21.396 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:21.397 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:21.406 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-09-18T10:15:21.424 INFO:tasks.rados.rados.0.smithi189.stdout:205: done (7 left) 2024-09-18T10:15:21.424 INFO:tasks.rados.rados.0.smithi189.stdout:209: done (6 left) 2024-09-18T10:15:21.424 INFO:tasks.rados.rados.0.smithi189.stdout:211: done (5 left) 2024-09-18T10:15:21.424 INFO:tasks.rados.rados.0.smithi189.stdout:213: done (4 left) 2024-09-18T10:15:21.424 INFO:tasks.rados.rados.0.smithi189.stdout:214: done (3 left) 2024-09-18T10:15:21.425 INFO:tasks.rados.rados.0.smithi189.stdout:219: done (2 left) 2024-09-18T10:15:21.425 INFO:tasks.rados.rados.0.smithi189.stdout:220: delete oid 39 current snap is 17 2024-09-18T10:15:21.425 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-18T10:15:21.421+0000 7f14c6a1a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 22138) UID: 0 2024-09-18T10:15:21.504 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:21.504 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:21.513 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-09-18T10:15:21.525 INFO:tasks.ceph.osd.12.smithi029.stderr:2024-09-18T10:15:21.518+0000 7f8e4e627700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 22190) UID: 0 2024-09-18T10:15:21.615 INFO:tasks.rados.rados.0.smithi189.stdout:218: finishing copy_from to smithi18925896-47 2024-09-18T10:15:21.615 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 47 v 8 (ObjNum 41 snap 0 seq_num 41) dirty exists 2024-09-18T10:15:21.618 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:21.618 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:21.619 INFO:tasks.rados.rados.0.smithi189.stdout:217: finishing write tid 1 to smithi18925896-46 2024-09-18T10:15:21.619 INFO:tasks.rados.rados.0.smithi189.stdout:217: finishing write tid 2 to smithi18925896-46 2024-09-18T10:15:21.620 INFO:tasks.rados.rados.0.smithi189.stdout:217: finishing write tid 3 to smithi18925896-46 2024-09-18T10:15:21.621 INFO:tasks.rados.rados.0.smithi189.stdout:217: finishing write tid 4 to smithi18925896-46 2024-09-18T10:15:21.626 INFO:tasks.ceph.osd.4.smithi029.stderr:2024-09-18T10:15:21.622+0000 7f550cf6e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22189) UID: 0 2024-09-18T10:15:21.628 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_ops_in_flight 2024-09-18T10:15:21.723 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:21.723 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:21.726 INFO:tasks.ceph.osd.14.smithi154.stderr:2024-09-18T10:15:21.723+0000 7f75a19e7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 22093) UID: 0 2024-09-18T10:15:21.732 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_blocked_ops 2024-09-18T10:15:21.758 INFO:tasks.rados.rados.0.smithi189.stdout:218: done (2 left) 2024-09-18T10:15:21.759 INFO:tasks.rados.rados.0.smithi189.stdout:220: done (1 left) 2024-09-18T10:15:21.759 INFO:tasks.rados.rados.0.smithi189.stdout:221: setattr oid 48 current snap is 17 2024-09-18T10:15:21.762 INFO:tasks.rados.rados.0.smithi189.stdout:222: snap_remove snap 7 2024-09-18T10:15:21.828 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-18T10:15:21.825+0000 7f578c52f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22134) UID: 0 2024-09-18T10:15:21.832 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:21.832 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:21.842 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_historic_ops 2024-09-18T10:15:21.926 INFO:tasks.rados.rados.0.smithi189.stdout:222: done (2 left) 2024-09-18T10:15:21.926 INFO:tasks.rados.rados.0.smithi189.stdout:append oid 11 current snap is 17 2024-09-18T10:15:21.926 INFO:tasks.rados.rados.0.smithi189.stdout:223: seq_num 99 ranges {1179648=507904,1687552=606208,2293760=557056} 2024-09-18T10:15:21.927 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-18T10:15:21.936 INFO:tasks.ceph.osd.15.smithi189.stderr:2024-09-18T10:15:21.922+0000 7fd0e725e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 22401) UID: 0 2024-09-18T10:15:21.937 INFO:teuthology.orchestra.run.smithi029.stderr:reweighted osd.12 to 1 (10000) 2024-09-18T10:15:21.940 INFO:tasks.rados.rados.0.smithi189.stdout:223: writing smithi18925896-11 from 1179648 to 1687552 tid 1 2024-09-18T10:15:21.940 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:21.940 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:21.943 INFO:tasks.rados.rados.0.smithi189.stdout:223: writing smithi18925896-11 from 1687552 to 2293760 tid 2 2024-09-18T10:15:21.946 INFO:tasks.rados.rados.0.smithi189.stdout:223: writing smithi18925896-11 from 2293760 to 2850816 tid 3 2024-09-18T10:15:21.946 INFO:tasks.rados.rados.0.smithi189.stdout:append oid 50 current snap is 17 2024-09-18T10:15:21.947 INFO:tasks.rados.rados.0.smithi189.stdout:224: seq_num 100 ranges {2293760=524288,2818048=753664,3571712=409600} 2024-09-18T10:15:21.949 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T10:15:21.951 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 13 1 2024-09-18T10:15:21.962 INFO:tasks.rados.rados.0.smithi189.stdout:224: writing smithi18925896-50 from 2293760 to 2818048 tid 1 2024-09-18T10:15:21.966 INFO:tasks.rados.rados.0.smithi189.stdout:224: writing smithi18925896-50 from 2818048 to 3571712 tid 2 2024-09-18T10:15:21.969 INFO:tasks.rados.rados.0.smithi189.stdout:224: writing smithi18925896-50 from 3571712 to 3981312 tid 3 2024-09-18T10:15:21.969 INFO:tasks.rados.rados.0.smithi189.stdout:225: copy_from oid 49 from oid 39 current snap is 17 2024-09-18T10:15:21.969 INFO:tasks.rados.rados.0.smithi189.stdout:226: snap_create 2024-09-18T10:15:22.028 INFO:tasks.rados.rados.0.smithi189.stdout:217: finishing write tid 5 to smithi18925896-46 2024-09-18T10:15:22.028 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 46 v 73 (ObjNum 96 snap 17 seq_num 96) dirty exists 2024-09-18T10:15:22.028 INFO:tasks.rados.rados.0.smithi189.stdout:217: left oid 46 (ObjNum 96 snap 17 seq_num 96) 2024-09-18T10:15:22.029 INFO:tasks.ceph.osd.4.smithi029.stderr:2024-09-18T10:15:22.026+0000 7f550cf6e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22189) UID: 0 2024-09-18T10:15:22.060 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:22.060 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:22.069 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T10:15:22.130 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-18T10:15:22.126+0000 7f302f2c3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22187) UID: 0 2024-09-18T10:15:22.133 INFO:tasks.rados.rados.0.smithi189.stdout:225: finishing copy_from to smithi18925896-49 2024-09-18T10:15:22.133 INFO:tasks.rados.rados.0.smithi189.stdout:225: got expected ENOENT (src dne) 2024-09-18T10:15:22.134 INFO:tasks.rados.rados.0.smithi189.stdout:225: finishing copy_from racing read to smithi18925896-49 2024-09-18T10:15:22.166 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:22.166 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:22.175 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T10:15:22.231 INFO:tasks.ceph.osd.8.smithi029.stderr:2024-09-18T10:15:22.226+0000 7f2555c27700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 22192) UID: 0 2024-09-18T10:15:22.272 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:22.272 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:22.281 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T10:15:22.332 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-18T10:15:22.329+0000 7f14c6a1a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 22138) UID: 0 2024-09-18T10:15:22.337 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T10:15:22.388 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:22.388 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:22.397 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T10:15:22.433 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-18T10:15:22.426+0000 7f302f2c3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22187) UID: 0 2024-09-18T10:15:22.484 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-18T10:15:22.499 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:22.499 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:22.507 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-18T10:15:22.508 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T10:15:22.529 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-18T10:15:22.533 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-18T10:15:22.533+0000 7f14c6a1a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 22138) UID: 0 2024-09-18T10:15:22.572 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-18T10:15:22.585 INFO:tasks.rados.rados.0.smithi189.stdout:224: finishing write tid 1 to smithi18925896-50 2024-09-18T10:15:22.586 INFO:tasks.rados.rados.0.smithi189.stdout:224: finishing write tid 2 to smithi18925896-50 2024-09-18T10:15:22.586 INFO:tasks.rados.rados.0.smithi189.stdout:224: finishing write tid 3 to smithi18925896-50 2024-09-18T10:15:22.586 INFO:tasks.rados.rados.0.smithi189.stdout:224: finishing write tid 4 to smithi18925896-50 2024-09-18T10:15:22.594 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-18T10:15:22.609 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:22.609 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:22.616 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-18T10:15:22.619 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-09-18T10:15:22.634 INFO:tasks.ceph.osd.3.smithi189.stderr:2024-09-18T10:15:22.630+0000 7f2440fc7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22396) UID: 0 2024-09-18T10:15:22.640 INFO:teuthology.orchestra.run.smithi029.stdout:osd.6: {} 2024-09-18T10:15:22.664 INFO:teuthology.orchestra.run.smithi029.stdout:osd.7: {} 2024-09-18T10:15:22.687 INFO:teuthology.orchestra.run.smithi029.stdout:osd.8: {} 2024-09-18T10:15:22.711 INFO:teuthology.orchestra.run.smithi029.stdout:osd.9: {} 2024-09-18T10:15:22.716 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:22.716 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:22.725 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-09-18T10:15:22.734 INFO:teuthology.orchestra.run.smithi029.stdout:osd.10: {} 2024-09-18T10:15:22.735 INFO:tasks.ceph.osd.4.smithi029.stderr:2024-09-18T10:15:22.730+0000 7f550cf6e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22189) UID: 0 2024-09-18T10:15:22.738 INFO:tasks.rados.rados.0.smithi189.stdout:224: finishing write tid 5 to smithi18925896-50 2024-09-18T10:15:22.738 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 50 v 79 (ObjNum 100 snap 17 seq_num 100) dirty exists 2024-09-18T10:15:22.738 INFO:tasks.rados.rados.0.smithi189.stdout:224: left oid 50 (ObjNum 100 snap 17 seq_num 100) 2024-09-18T10:15:22.743 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 48 v 62 (ObjNum 41 snap 0 seq_num 41) dirty exists 2024-09-18T10:15:22.744 INFO:tasks.rados.rados.0.smithi189.stdout:223: finishing write tid 1 to smithi18925896-11 2024-09-18T10:15:22.744 INFO:tasks.rados.rados.0.smithi189.stdout:223: finishing write tid 2 to smithi18925896-11 2024-09-18T10:15:22.758 INFO:teuthology.orchestra.run.smithi029.stdout:osd.11: {} 2024-09-18T10:15:22.777 INFO:teuthology.orchestra.run.smithi029.stdout:osd.12: {} 2024-09-18T10:15:22.798 INFO:teuthology.orchestra.run.smithi029.stdout:osd.13: {} 2024-09-18T10:15:22.821 INFO:teuthology.orchestra.run.smithi029.stdout:osd.14: {} 2024-09-18T10:15:22.825 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-09-18T10:15:22.825 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-09-18T10:15:22.825 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-09-18T10:15:22.825 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-09-18T10:15:22.825 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-09-18T10:15:22.834 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-09-18T10:15:22.835 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-18T10:15:22.830+0000 7f14dcc5b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22398) UID: 0 2024-09-18T10:15:22.845 INFO:tasks.rados.rados.0.smithi189.stdout:223: finishing write tid 3 to smithi18925896-11 2024-09-18T10:15:22.845 INFO:teuthology.orchestra.run.smithi029.stdout:osd.15: {} 2024-09-18T10:15:22.845 INFO:tasks.rados.rados.0.smithi189.stdout:223: finishing write tid 4 to smithi18925896-11 2024-09-18T10:15:22.849 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T10:15:22.849 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T10:15:22.849 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T10:15:22.849 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T10:15:22.849 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T10:15:22.849 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T10:15:22.849 INFO:teuthology.orchestra.run.smithi029.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T10:15:22.849 INFO:teuthology.orchestra.run.smithi029.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T10:15:22.850 INFO:teuthology.orchestra.run.smithi029.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-18T10:15:22.850 INFO:teuthology.orchestra.run.smithi029.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-18T10:15:22.850 INFO:teuthology.orchestra.run.smithi029.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-18T10:15:22.850 INFO:teuthology.orchestra.run.smithi029.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-18T10:15:22.850 INFO:teuthology.orchestra.run.smithi029.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-18T10:15:22.850 INFO:teuthology.orchestra.run.smithi029.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-18T10:15:22.850 INFO:teuthology.orchestra.run.smithi029.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-18T10:15:22.850 INFO:teuthology.orchestra.run.smithi029.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-18T10:15:22.928 INFO:tasks.rados.rados.0.smithi189.stdout:217: done (5 left) 2024-09-18T10:15:22.929 INFO:tasks.rados.rados.0.smithi189.stdout:221: done (4 left) 2024-09-18T10:15:22.929 INFO:tasks.rados.rados.0.smithi189.stdout:224: done (3 left) 2024-09-18T10:15:22.929 INFO:tasks.rados.rados.0.smithi189.stdout:225: done (2 left) 2024-09-18T10:15:22.929 INFO:tasks.rados.rados.0.smithi189.stdout:226: done (1 left) 2024-09-18T10:15:22.929 INFO:tasks.rados.rados.0.smithi189.stdout:227: snap_create 2024-09-18T10:15:22.933 INFO:teuthology.orchestra.run.smithi029.stderr:reweighted osd.13 to 1 (10000) 2024-09-18T10:15:22.933 INFO:teuthology.orchestra.run.smithi029.stderr:nodeep-scrub is set 2024-09-18T10:15:22.938 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-18T10:15:22.933+0000 7f14c6a1a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 22138) UID: 0 2024-09-18T10:15:22.939 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-09-18T10:15:22.939 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-09-18T10:15:22.939 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-09-18T10:15:22.939 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-09-18T10:15:22.939 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.939 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.7s5 36/32 ECSubWrite(tid=31, reqid=client.4632.0:128, at_version=0'0, trim_to=0'0, roll_forward_to=36'8))", 2024-09-18T10:15:22.939 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:51.219177+0000", 2024-09-18T10:15:22.939 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 31.7174725, 2024-09-18T10:15:22.939 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.202758309, 2024-09-18T10:15:22.939 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:22.939 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:15:22.940 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:22.940 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.940 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:22.940 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.219177+0000", 2024-09-18T10:15:22.940 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:22.940 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.940 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.940 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:22.940 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.219177+0000", 2024-09-18T10:15:22.940 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:22.940 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.940 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.940 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:22.941 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.219175+0000", 2024-09-18T10:15:22.941 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999976 2024-09-18T10:15:22.941 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.941 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.941 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:22.941 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.219207+0000", 2024-09-18T10:15:22.941 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1285000000000003e-05 2024-09-18T10:15:22.941 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.942 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.942 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:22.942 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.219244+0000", 2024-09-18T10:15:22.942 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.7032000000000001e-05 2024-09-18T10:15:22.942 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.942 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.942 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:22.942 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.219266+0000", 2024-09-18T10:15:22.942 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.2112000000000001e-05 2024-09-18T10:15:22.942 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.942 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.942 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:22.942 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.319431+0000", 2024-09-18T10:15:22.942 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100164869 2024-09-18T10:15:22.943 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.943 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.943 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:22.943 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.319446+0000", 2024-09-18T10:15:22.943 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.5945999999999999e-05 2024-09-18T10:15:22.943 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.943 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.943 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:22.943 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.421877+0000", 2024-09-18T10:15:22.943 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10243026600000001 2024-09-18T10:15:22.943 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.943 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.943 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:22.944 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.421936+0000", 2024-09-18T10:15:22.944 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.9024999999999998e-05 2024-09-18T10:15:22.944 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:22.944 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:22.944 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:22.944 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.944 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.944 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.bs2 36/32 ECSubWrite(tid=35, reqid=client.4632.0:160, at_version=36'5, trim_to=0'0, roll_forward_to=36'4))", 2024-09-18T10:15:22.944 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:51.315316+0000", 2024-09-18T10:15:22.944 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 31.621334004000001, 2024-09-18T10:15:22.944 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.201678933, 2024-09-18T10:15:22.944 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:22.944 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:15:22.945 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:22.945 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.945 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:22.945 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.315316+0000", 2024-09-18T10:15:22.945 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:22.945 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.945 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.945 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:22.945 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.315316+0000", 2024-09-18T10:15:22.945 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:22.945 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.945 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.945 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:22.946 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.315314+0000", 2024-09-18T10:15:22.946 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:22.946 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.946 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.946 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:22.946 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.315385+0000", 2024-09-18T10:15:22.946 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.1210000000000004e-05 2024-09-18T10:15:22.946 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.946 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.946 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:22.946 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.315422+0000", 2024-09-18T10:15:22.946 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.7157999999999999e-05 2024-09-18T10:15:22.946 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.946 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.947 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:22.947 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.315441+0000", 2024-09-18T10:15:22.947 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8589000000000001e-05 2024-09-18T10:15:22.947 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.947 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.947 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:22.947 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.515780+0000", 2024-09-18T10:15:22.947 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20033880100000001 2024-09-18T10:15:22.947 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.947 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.948 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:22.948 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.515798+0000", 2024-09-18T10:15:22.948 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7762e-05 2024-09-18T10:15:22.948 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.948 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.948 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:22.948 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.516927+0000", 2024-09-18T10:15:22.948 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0011289760000000001 2024-09-18T10:15:22.948 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.948 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.948 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:22.948 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.516995+0000", 2024-09-18T10:15:22.949 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.8319999999999996e-05 2024-09-18T10:15:22.949 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:22.949 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:22.949 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:22.949 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.949 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.949 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:189 3.3s0 3:ce8164ce:::smithi18925896-7:head [copy-from ver 6] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:22.949 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:52.694279+0000", 2024-09-18T10:15:22.949 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 30.242370595000001, 2024-09-18T10:15:22.949 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.51539283599999997, 2024-09-18T10:15:22.949 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:22.949 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:22.949 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:22.949 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:15:22.950 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:22.950 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 189 2024-09-18T10:15:22.950 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.950 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:22.950 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.950 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:22.950 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694279+0000", 2024-09-18T10:15:22.950 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:22.950 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.950 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.950 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:22.950 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694279+0000", 2024-09-18T10:15:22.950 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:22.950 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.951 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.951 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:22.951 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694277+0000", 2024-09-18T10:15:22.951 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999971 2024-09-18T10:15:22.951 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.951 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.951 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:22.951 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694290+0000", 2024-09-18T10:15:22.951 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3862e-05 2024-09-18T10:15:22.951 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.951 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.951 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:22.951 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694311+0000", 2024-09-18T10:15:22.952 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.0273999999999999e-05 2024-09-18T10:15:22.952 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.952 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.952 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:22.952 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694322+0000", 2024-09-18T10:15:22.952 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1722999999999999e-05 2024-09-18T10:15:22.952 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.952 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.952 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:22.952 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.794494+0000", 2024-09-18T10:15:22.952 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100171653 2024-09-18T10:15:22.952 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.952 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.952 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:22.953 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.794705+0000", 2024-09-18T10:15:22.953 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00021126000000000001 2024-09-18T10:15:22.953 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.953 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.953 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:22.953 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.096113+0000", 2024-09-18T10:15:22.953 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.30140739100000002 2024-09-18T10:15:22.953 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.953 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.953 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:22.953 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.098602+0000", 2024-09-18T10:15:22.953 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.002489593 2024-09-18T10:15:22.953 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.953 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.954 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:15:22.954 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.209538+0000", 2024-09-18T10:15:22.954 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.110935482 2024-09-18T10:15:22.954 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.954 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.954 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:22.954 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.209672+0000", 2024-09-18T10:15:22.954 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00013435399999999999 2024-09-18T10:15:22.954 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:22.954 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:22.954 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:22.954 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.954 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.954 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:190 3.3s0 3:ce8164ce:::smithi18925896-7:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:22.954 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:52.694343+0000", 2024-09-18T10:15:22.955 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 30.242307005000001, 2024-09-18T10:15:22.955 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.40407532499999999, 2024-09-18T10:15:22.955 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:22.955 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "started", 2024-09-18T10:15:22.955 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:22.955 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:15:22.955 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:22.955 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 190 2024-09-18T10:15:22.955 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.955 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:22.955 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.955 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:22.955 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694343+0000", 2024-09-18T10:15:22.955 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:22.956 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.956 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.956 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:22.956 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694343+0000", 2024-09-18T10:15:22.956 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:22.956 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.956 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.956 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:22.956 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694342+0000", 2024-09-18T10:15:22.956 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:15:22.956 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.956 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.956 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:22.957 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694349+0000", 2024-09-18T10:15:22.957 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.5969999999999997e-06 2024-09-18T10:15:22.957 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.957 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.957 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:22.957 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694367+0000", 2024-09-18T10:15:22.957 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8323000000000001e-05 2024-09-18T10:15:22.957 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.957 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.958 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:22.958 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694376+0000", 2024-09-18T10:15:22.958 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.9740000000000008e-06 2024-09-18T10:15:22.958 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.958 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.958 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:22.958 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.795028+0000", 2024-09-18T10:15:22.958 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10065176000000001 2024-09-18T10:15:22.958 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.958 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.958 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for blocked object", 2024-09-18T10:15:22.958 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.795171+0000", 2024-09-18T10:15:22.958 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00014321399999999999 2024-09-18T10:15:22.959 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.959 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.959 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:22.959 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.098069+0000", 2024-09-18T10:15:22.959 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.30289780700000002 2024-09-18T10:15:22.959 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.959 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.959 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:22.959 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.098247+0000", 2024-09-18T10:15:22.959 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00017831300000000001 2024-09-18T10:15:22.959 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.959 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.960 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:22.960 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.098418+0000", 2024-09-18T10:15:22.960 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00017122599999999999 2024-09-18T10:15:22.960 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:22.960 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:22.960 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:22.960 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.960 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.960 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:194 3.3s0 3:c9ce6440:::smithi18925896-24:head [copy-from ver 6] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:22.960 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:52.694388+0000", 2024-09-18T10:15:22.960 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 30.242261731999999, 2024-09-18T10:15:22.960 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.92118689200000003, 2024-09-18T10:15:22.960 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:22.961 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:22.961 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:22.961 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:15:22.961 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:22.961 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 194 2024-09-18T10:15:22.961 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.961 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:22.961 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.961 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:22.961 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694388+0000", 2024-09-18T10:15:22.961 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:22.961 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.961 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.962 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:22.962 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694388+0000", 2024-09-18T10:15:22.962 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:22.962 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.962 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.962 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:22.962 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694388+0000", 2024-09-18T10:15:22.962 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:15:22.962 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.962 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.962 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:22.962 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694393+0000", 2024-09-18T10:15:22.962 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.2299999999999999e-06 2024-09-18T10:15:22.962 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.963 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.963 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:22.963 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694411+0000", 2024-09-18T10:15:22.963 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7802999999999999e-05 2024-09-18T10:15:22.963 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.963 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.963 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:22.963 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694417+0000", 2024-09-18T10:15:22.963 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.4640000000000003e-06 2024-09-18T10:15:22.963 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.963 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.963 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:22.963 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.795289+0000", 2024-09-18T10:15:22.964 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100872288 2024-09-18T10:15:22.964 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.964 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.964 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:22.964 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.795548+0000", 2024-09-18T10:15:22.964 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00025869199999999998 2024-09-18T10:15:22.964 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.964 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.964 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:22.964 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.412215+0000", 2024-09-18T10:15:22.964 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.61666667100000006 2024-09-18T10:15:22.964 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.964 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.965 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:22.965 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.514838+0000", 2024-09-18T10:15:22.965 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10262323399999999 2024-09-18T10:15:22.965 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.965 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.965 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:15:22.965 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.615386+0000", 2024-09-18T10:15:22.965 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100548106 2024-09-18T10:15:22.965 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.965 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.965 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:22.965 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.615575+0000", 2024-09-18T10:15:22.965 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000189128 2024-09-18T10:15:22.966 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:22.966 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:22.966 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:22.966 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.966 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.966 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:195 3.3s0 3:c9ce6440:::smithi18925896-24:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:22.966 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:52.694424+0000", 2024-09-18T10:15:22.966 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 30.242225995999998, 2024-09-18T10:15:22.966 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.82009685499999996, 2024-09-18T10:15:22.966 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:22.966 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "started", 2024-09-18T10:15:22.966 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:22.966 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:15:22.966 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:22.967 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 195 2024-09-18T10:15:22.967 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.967 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:22.967 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.967 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:22.967 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694424+0000", 2024-09-18T10:15:22.967 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:22.967 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.967 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.967 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:22.967 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694424+0000", 2024-09-18T10:15:22.967 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:22.967 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.967 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.968 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:22.968 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694423+0000", 2024-09-18T10:15:22.968 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:22.968 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.968 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.968 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:22.968 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694427+0000", 2024-09-18T10:15:22.968 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2650000000000001e-06 2024-09-18T10:15:22.968 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.968 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.968 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:22.968 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694434+0000", 2024-09-18T10:15:22.968 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.5410000000000003e-06 2024-09-18T10:15:22.968 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.969 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.969 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:22.969 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694441+0000", 2024-09-18T10:15:22.969 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.187e-06 2024-09-18T10:15:22.969 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.969 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.969 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:22.969 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.795870+0000", 2024-09-18T10:15:22.969 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.101428617 2024-09-18T10:15:22.969 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.969 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.969 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for blocked object", 2024-09-18T10:15:22.969 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.796040+0000", 2024-09-18T10:15:22.970 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00017008 2024-09-18T10:15:22.970 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.970 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.970 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:22.970 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.514130+0000", 2024-09-18T10:15:22.970 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.71809022199999994 2024-09-18T10:15:22.970 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.970 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.970 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:22.970 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.514348+0000", 2024-09-18T10:15:22.970 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000217488 2024-09-18T10:15:22.970 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.970 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.970 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:22.971 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.514521+0000", 2024-09-18T10:15:22.971 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00017300900000000001 2024-09-18T10:15:22.971 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:22.971 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:22.971 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:22.971 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.971 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.971 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.bs2 41/32 ECSubWrite(tid=58, reqid=osd.4.0:58, at_version=41'14, trim_to=0'0, roll_forward_to=39'12))", 2024-09-18T10:15:22.971 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:56.829698+0000", 2024-09-18T10:15:22.971 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 26.106951901999999, 2024-09-18T10:15:22.971 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.71711602600000002, 2024-09-18T10:15:22.971 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:22.971 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:15:22.972 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:22.972 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.972 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:22.972 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.829698+0000", 2024-09-18T10:15:22.972 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:22.972 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.972 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.972 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:22.972 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.829698+0000", 2024-09-18T10:15:22.972 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:22.972 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.972 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.972 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:22.973 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.829697+0000", 2024-09-18T10:15:22.973 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:22.973 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.973 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.973 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:22.973 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.829733+0000", 2024-09-18T10:15:22.973 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.6217999999999998e-05 2024-09-18T10:15:22.973 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.973 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.973 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:22.973 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.829757+0000", 2024-09-18T10:15:22.973 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.3915000000000001e-05 2024-09-18T10:15:22.973 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.973 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.974 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:22.974 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.829769+0000", 2024-09-18T10:15:22.974 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1754e-05 2024-09-18T10:15:22.974 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.974 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.974 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:22.974 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.342965+0000", 2024-09-18T10:15:22.974 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.51319593799999996 2024-09-18T10:15:22.974 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.974 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.974 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:22.974 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.343020+0000", 2024-09-18T10:15:22.974 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5367e-05 2024-09-18T10:15:22.974 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.975 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.975 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:22.975 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.546772+0000", 2024-09-18T10:15:22.975 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.203751933 2024-09-18T10:15:22.975 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.975 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.975 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:22.975 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.546814+0000", 2024-09-18T10:15:22.975 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.2196000000000001e-05 2024-09-18T10:15:22.975 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:22.975 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:22.975 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:22.975 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.976 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.976 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWriteReply(3.13s0 41/40 ECSubWriteReply(tid=20, last_complete=41'18, committed=1, applied=1))", 2024-09-18T10:15:22.976 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:57.854385+0000", 2024-09-18T10:15:22.976 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 25.082264484, 2024-09-18T10:15:22.976 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20186811199999999, 2024-09-18T10:15:22.976 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:22.976 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:15:22.976 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:22.976 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.976 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:22.976 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.854385+0000", 2024-09-18T10:15:22.976 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:22.976 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.976 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.977 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:22.977 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.854385+0000", 2024-09-18T10:15:22.977 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:22.977 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.977 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.977 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:22.977 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.854384+0000", 2024-09-18T10:15:22.977 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:22.977 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.977 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.977 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:22.977 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.854396+0000", 2024-09-18T10:15:22.977 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2295e-05 2024-09-18T10:15:22.978 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.978 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.978 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:22.978 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.854417+0000", 2024-09-18T10:15:22.978 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.0621000000000001e-05 2024-09-18T10:15:22.978 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.978 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.978 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:22.978 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.854431+0000", 2024-09-18T10:15:22.978 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.435e-05 2024-09-18T10:15:22.978 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.978 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.978 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:22.979 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:58.054900+0000", 2024-09-18T10:15:22.979 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20046845299999999 2024-09-18T10:15:22.979 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.979 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.979 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:22.979 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:58.056254+0000", 2024-09-18T10:15:22.979 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0013538630000000001 2024-09-18T10:15:22.979 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:22.979 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:22.979 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:22.979 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.979 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.979 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:320 3.13s0 3:c9ce6440:::smithi18925896-24:4 [read 0~475136,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T10:15:22.979 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:01.795519+0000", 2024-09-18T10:15:22.980 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 21.141130908000001, 2024-09-18T10:15:22.980 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20609297800000001, 2024-09-18T10:15:22.980 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:22.980 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "started", 2024-09-18T10:15:22.980 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:22.980 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:15:22.980 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:22.980 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 320 2024-09-18T10:15:22.980 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.980 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:22.980 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.980 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:22.980 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:01.795519+0000", 2024-09-18T10:15:22.980 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:22.981 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.981 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.981 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:22.981 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:01.795519+0000", 2024-09-18T10:15:22.981 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:22.981 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.981 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.981 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:22.981 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:01.795516+0000", 2024-09-18T10:15:22.981 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999967 2024-09-18T10:15:22.981 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.981 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.981 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:22.982 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:01.795532+0000", 2024-09-18T10:15:22.982 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.6498e-05 2024-09-18T10:15:22.982 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.982 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.982 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:22.982 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:01.795569+0000", 2024-09-18T10:15:22.982 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.6665999999999998e-05 2024-09-18T10:15:22.982 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.982 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.982 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:22.982 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:01.795583+0000", 2024-09-18T10:15:22.982 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4025999999999999e-05 2024-09-18T10:15:22.982 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.982 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.983 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:22.983 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:01.795692+0000", 2024-09-18T10:15:22.983 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00010935 2024-09-18T10:15:22.983 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.983 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.983 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:22.983 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:01.795877+0000", 2024-09-18T10:15:22.983 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00018476699999999999 2024-09-18T10:15:22.983 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.983 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.983 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:22.983 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:02.001612+0000", 2024-09-18T10:15:22.983 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20573480399999999 2024-09-18T10:15:22.983 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:22.984 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:22.984 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:22.984 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.984 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.984 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:321 3.13s0 3:c9ce6440:::smithi18925896-24:4 [read 0~475136,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T10:15:22.984 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:01.795610+0000", 2024-09-18T10:15:22.984 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 21.141040231000002, 2024-09-18T10:15:22.984 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20673176600000001, 2024-09-18T10:15:22.984 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:22.984 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "started", 2024-09-18T10:15:22.984 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:22.984 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:15:22.984 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:22.985 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 321 2024-09-18T10:15:22.985 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.985 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:22.985 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.985 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:22.985 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:01.795610+0000", 2024-09-18T10:15:22.985 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:22.985 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.985 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.985 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:22.985 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:01.795610+0000", 2024-09-18T10:15:22.985 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:22.986 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.986 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.986 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:22.986 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:01.795609+0000", 2024-09-18T10:15:22.986 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:15:22.986 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.986 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.986 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:22.986 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:01.795618+0000", 2024-09-18T10:15:22.986 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.4499999999999993e-06 2024-09-18T10:15:22.986 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.986 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.986 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:22.986 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:01.795632+0000", 2024-09-18T10:15:22.987 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3876999999999999e-05 2024-09-18T10:15:22.987 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.987 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.987 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:22.987 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:01.795641+0000", 2024-09-18T10:15:22.987 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.2420000000000006e-06 2024-09-18T10:15:22.987 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.987 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.987 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:22.987 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:01.896465+0000", 2024-09-18T10:15:22.987 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100823489 2024-09-18T10:15:22.987 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.987 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.988 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:22.988 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:01.896591+0000", 2024-09-18T10:15:22.988 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00012644899999999999 2024-09-18T10:15:22.988 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.988 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.988 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:22.988 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:02.002342+0000", 2024-09-18T10:15:22.988 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.105750235 2024-09-18T10:15:22.988 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:22.988 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:22.988 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:22.988 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.988 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.989 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:322 3.13s0 3:c9ce6440:::smithi18925896-24:4 [read 0~475136,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T10:15:22.989 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:01.795716+0000", 2024-09-18T10:15:22.989 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 21.140933816, 2024-09-18T10:15:22.989 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.30787060799999999, 2024-09-18T10:15:22.989 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:22.989 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "started", 2024-09-18T10:15:22.989 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:22.989 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:15:22.989 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:22.989 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 322 2024-09-18T10:15:22.989 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.989 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:22.989 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.989 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:22.990 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:01.795716+0000", 2024-09-18T10:15:22.990 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:22.990 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.990 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.990 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:22.990 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:01.795716+0000", 2024-09-18T10:15:22.990 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:22.990 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.990 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.990 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:22.990 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:01.795715+0000", 2024-09-18T10:15:22.990 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:15:22.990 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.990 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.991 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:22.991 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:01.795723+0000", 2024-09-18T10:15:22.991 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.5730000000000002e-06 2024-09-18T10:15:22.991 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.991 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.991 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:22.991 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:01.795737+0000", 2024-09-18T10:15:22.991 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4158999999999999e-05 2024-09-18T10:15:22.991 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.991 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.991 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:22.991 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:01.895874+0000", 2024-09-18T10:15:22.991 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100137485 2024-09-18T10:15:22.991 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.992 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.992 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:22.992 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:01.999648+0000", 2024-09-18T10:15:22.992 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10377353 2024-09-18T10:15:22.992 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.992 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.992 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:22.992 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:01.999827+0000", 2024-09-18T10:15:22.992 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0001794 2024-09-18T10:15:22.992 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.992 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.992 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:22.992 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:02.103587+0000", 2024-09-18T10:15:22.993 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.103759648 2024-09-18T10:15:22.993 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:22.993 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:22.993 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:22.993 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.993 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.993 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.19s2 45/40 ECSubWrite(tid=58, reqid=client.4632.0:297, at_version=0'0, trim_to=0'0, roll_forward_to=45'35))", 2024-09-18T10:15:22.993 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:01.915741+0000", 2024-09-18T10:15:22.993 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 21.020908922, 2024-09-18T10:15:22.993 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.28605987500000002, 2024-09-18T10:15:22.993 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:22.993 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:15:22.993 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:22.993 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.994 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:22.994 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:01.915741+0000", 2024-09-18T10:15:22.994 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:22.994 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.994 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.994 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:22.994 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:01.915741+0000", 2024-09-18T10:15:22.994 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:22.994 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.994 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.994 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:22.994 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:01.915739+0000", 2024-09-18T10:15:22.994 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:22.994 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.995 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.995 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:22.995 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:01.915763+0000", 2024-09-18T10:15:22.995 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.3938999999999999e-05 2024-09-18T10:15:22.995 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.995 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.995 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:22.995 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:01.915792+0000", 2024-09-18T10:15:22.995 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8232999999999999e-05 2024-09-18T10:15:22.995 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.995 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.995 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:22.995 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:01.915807+0000", 2024-09-18T10:15:22.995 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.5682e-05 2024-09-18T10:15:22.996 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.996 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.996 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:22.996 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:02.100734+0000", 2024-09-18T10:15:22.996 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.18492666999999999 2024-09-18T10:15:22.996 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.996 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.996 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:22.996 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:02.100766+0000", 2024-09-18T10:15:22.996 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2132000000000003e-05 2024-09-18T10:15:22.996 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.996 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.996 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:22.996 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:02.201732+0000", 2024-09-18T10:15:22.997 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10096622700000001 2024-09-18T10:15:22.997 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.997 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.997 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:22.997 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:02.201801+0000", 2024-09-18T10:15:22.997 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.8694000000000003e-05 2024-09-18T10:15:22.997 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:22.997 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:22.997 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:22.997 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.997 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.997 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:344 3.13s0 3:c9ce6440:::smithi18925896-24:head [read 0~475136,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-18T10:15:22.997 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:02.803036+0000", 2024-09-18T10:15:22.998 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 20.133613614000001, 2024-09-18T10:15:22.998 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.21069362, 2024-09-18T10:15:22.998 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:22.998 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "started", 2024-09-18T10:15:22.998 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:22.998 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:15:22.998 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:22.998 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 344 2024-09-18T10:15:22.998 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.998 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:22.998 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.998 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:22.998 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:02.803036+0000", 2024-09-18T10:15:22.999 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:22.999 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.999 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.999 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:22.999 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:02.803036+0000", 2024-09-18T10:15:22.999 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:22.999 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.999 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.999 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:22.999 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:02.803035+0000", 2024-09-18T10:15:22.999 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:22.999 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:22.999 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:22.999 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:23.000 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:02.803046+0000", 2024-09-18T10:15:23.000 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1347e-05 2024-09-18T10:15:23.000 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.000 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.000 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:23.000 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:02.803070+0000", 2024-09-18T10:15:23.000 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.3349e-05 2024-09-18T10:15:23.000 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.000 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.000 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:23.000 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:02.803081+0000", 2024-09-18T10:15:23.000 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1629e-05 2024-09-18T10:15:23.000 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.000 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.001 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:23.001 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:02.803319+0000", 2024-09-18T10:15:23.001 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000238061 2024-09-18T10:15:23.001 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.001 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.001 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:23.001 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:02.803346+0000", 2024-09-18T10:15:23.001 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.7059e-05 2024-09-18T10:15:23.001 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.001 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.001 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:23.001 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:02.808250+0000", 2024-09-18T10:15:23.001 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0049038650000000003 2024-09-18T10:15:23.002 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.002 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.002 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:23.002 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:02.808332+0000", 2024-09-18T10:15:23.002 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.1538000000000004e-05 2024-09-18T10:15:23.002 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.002 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.002 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:23.002 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:03.013730+0000", 2024-09-18T10:15:23.002 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20539824200000001 2024-09-18T10:15:23.002 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:23.002 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:23.002 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:23.003 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.003 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.003 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:345 3.13s0 3:c9ce6440:::smithi18925896-24:head [read 0~475136,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-18T10:15:23.003 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:02.803095+0000", 2024-09-18T10:15:23.003 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 20.133554812, 2024-09-18T10:15:23.003 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.21197828799999999, 2024-09-18T10:15:23.003 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:23.003 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "started", 2024-09-18T10:15:23.003 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:23.003 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:15:23.003 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:23.003 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 345 2024-09-18T10:15:23.003 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.004 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:23.004 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.004 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:23.004 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:02.803095+0000", 2024-09-18T10:15:23.004 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:23.004 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.004 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.004 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:23.004 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:02.803095+0000", 2024-09-18T10:15:23.004 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:23.004 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.004 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.004 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:23.005 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:02.803094+0000", 2024-09-18T10:15:23.005 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:15:23.005 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.005 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.005 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:23.005 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:02.803104+0000", 2024-09-18T10:15:23.005 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.6430000000000007e-06 2024-09-18T10:15:23.005 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.005 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.005 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:23.005 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:02.803142+0000", 2024-09-18T10:15:23.005 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8591999999999999e-05 2024-09-18T10:15:23.005 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.006 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.006 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:23.006 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:02.803155+0000", 2024-09-18T10:15:23.006 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2466000000000001e-05 2024-09-18T10:15:23.006 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.006 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.006 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:23.006 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:02.803466+0000", 2024-09-18T10:15:23.006 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00031142499999999999 2024-09-18T10:15:23.006 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.006 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.006 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:23.006 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:02.803487+0000", 2024-09-18T10:15:23.006 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.0305999999999998e-05 2024-09-18T10:15:23.007 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.007 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.007 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:23.007 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:02.808581+0000", 2024-09-18T10:15:23.007 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0050947400000000004 2024-09-18T10:15:23.007 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.007 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.007 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:23.007 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:02.808678+0000", 2024-09-18T10:15:23.007 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.6797000000000002e-05 2024-09-18T10:15:23.007 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.007 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.007 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:23.007 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:03.015073+0000", 2024-09-18T10:15:23.008 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20639537599999999 2024-09-18T10:15:23.008 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:23.008 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:23.008 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:23.008 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.008 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.008 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpReadReply(3.13s0 47/40 ECSubReadReply(tid=32, attrs_read=0))", 2024-09-18T10:15:23.008 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:02.809839+0000", 2024-09-18T10:15:23.008 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 20.126810763000002, 2024-09-18T10:15:23.008 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20412009, 2024-09-18T10:15:23.008 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:23.008 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:15:23.009 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:23.009 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.009 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:23.009 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:02.809839+0000", 2024-09-18T10:15:23.009 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:23.009 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.009 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.009 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:23.009 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:02.809839+0000", 2024-09-18T10:15:23.009 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:23.009 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.009 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.009 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:23.010 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:02.809839+0000", 2024-09-18T10:15:23.010 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:23.010 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.010 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.010 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:23.010 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:02.809895+0000", 2024-09-18T10:15:23.010 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.6668e-05 2024-09-18T10:15:23.010 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.010 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.010 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:23.010 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:02.809913+0000", 2024-09-18T10:15:23.010 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7757999999999998e-05 2024-09-18T10:15:23.010 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.011 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.011 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:23.011 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:02.809917+0000", 2024-09-18T10:15:23.011 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.664e-06 2024-09-18T10:15:23.011 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.011 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.011 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:23.011 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:03.013886+0000", 2024-09-18T10:15:23.011 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20396906400000001 2024-09-18T10:15:23.011 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.011 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.011 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:23.011 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:03.013959+0000", 2024-09-18T10:15:23.011 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.3323000000000003e-05 2024-09-18T10:15:23.012 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:23.012 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:23.012 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:23.012 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.012 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.012 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.19s2 51/40 ECSubWrite(tid=74, reqid=client.4632.0:398, at_version=51'42, trim_to=0'0, roll_forward_to=50'39))", 2024-09-18T10:15:23.012 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:06.867249+0000", 2024-09-18T10:15:23.012 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 16.069401268, 2024-09-18T10:15:23.012 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20415097099999999, 2024-09-18T10:15:23.012 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:23.012 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:15:23.012 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:23.012 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.013 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:23.013 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:06.867249+0000", 2024-09-18T10:15:23.013 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:23.013 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.013 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.013 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:23.013 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:06.867249+0000", 2024-09-18T10:15:23.013 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:23.013 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.013 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.013 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:23.013 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:06.867248+0000", 2024-09-18T10:15:23.013 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:23.014 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.014 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.014 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:23.014 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:06.867490+0000", 2024-09-18T10:15:23.014 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000241995 2024-09-18T10:15:23.014 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.014 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.014 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:23.014 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:06.867512+0000", 2024-09-18T10:15:23.014 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.2215999999999999e-05 2024-09-18T10:15:23.014 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.014 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.014 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:23.015 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:06.967650+0000", 2024-09-18T10:15:23.015 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10013731100000001 2024-09-18T10:15:23.015 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.015 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.015 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:23.015 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:06.967806+0000", 2024-09-18T10:15:23.015 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00015614099999999999 2024-09-18T10:15:23.015 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.015 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.015 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:23.015 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:06.967838+0000", 2024-09-18T10:15:23.015 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2719000000000003e-05 2024-09-18T10:15:23.015 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.015 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.016 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:23.016 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:07.071328+0000", 2024-09-18T10:15:23.016 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.103489888 2024-09-18T10:15:23.016 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.016 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.016 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:23.016 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:07.071400+0000", 2024-09-18T10:15:23.016 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.1390999999999995e-05 2024-09-18T10:15:23.016 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:23.016 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:23.016 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:23.016 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.016 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.017 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.17s5 57/40 ECSubWrite(tid=107, reqid=client.4632.0:442, at_version=57'23, trim_to=0'0, roll_forward_to=53'18))", 2024-09-18T10:15:23.017 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:12.950378+0000", 2024-09-18T10:15:23.017 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 9.9862723599999992, 2024-09-18T10:15:23.017 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20447374900000001, 2024-09-18T10:15:23.017 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:23.017 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:15:23.017 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:23.017 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.017 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:23.017 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:12.950378+0000", 2024-09-18T10:15:23.017 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:23.017 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.017 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.017 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:23.018 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:12.950378+0000", 2024-09-18T10:15:23.018 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:23.018 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.018 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.018 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:23.018 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:12.950376+0000", 2024-09-18T10:15:23.018 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:23.018 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.018 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.018 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:23.018 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:12.950580+0000", 2024-09-18T10:15:23.018 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00020451900000000001 2024-09-18T10:15:23.018 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.019 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.019 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:23.019 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:12.950601+0000", 2024-09-18T10:15:23.019 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.0803000000000001e-05 2024-09-18T10:15:23.019 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.019 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.019 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:23.019 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:12.950612+0000", 2024-09-18T10:15:23.019 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.078e-05 2024-09-18T10:15:23.019 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.019 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.019 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:23.019 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.053115+0000", 2024-09-18T10:15:23.019 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.102502629 2024-09-18T10:15:23.020 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.020 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.020 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:23.020 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.053141+0000", 2024-09-18T10:15:23.020 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6536000000000001e-05 2024-09-18T10:15:23.020 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.020 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.020 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:23.020 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.154811+0000", 2024-09-18T10:15:23.020 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.101669755 2024-09-18T10:15:23.020 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.020 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.020 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:23.021 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.154851+0000", 2024-09-18T10:15:23.021 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.0392999999999998e-05 2024-09-18T10:15:23.021 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:23.021 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:23.021 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:23.021 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.021 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.021 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.17s5 57/40 ECSubWrite(tid=108, reqid=client.4632.0:443, at_version=57'24, trim_to=0'0, roll_forward_to=53'18))", 2024-09-18T10:15:23.021 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:12.952216+0000", 2024-09-18T10:15:23.021 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 9.9844342009999991, 2024-09-18T10:15:23.021 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20265246100000001, 2024-09-18T10:15:23.021 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:23.021 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:15:23.021 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:23.022 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.022 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:23.022 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:12.952216+0000", 2024-09-18T10:15:23.022 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:23.022 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.022 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.022 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:23.022 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:12.952216+0000", 2024-09-18T10:15:23.022 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:23.022 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.022 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.022 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:23.022 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:12.952214+0000", 2024-09-18T10:15:23.022 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:23.022 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.023 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.023 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:23.023 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:12.952263+0000", 2024-09-18T10:15:23.023 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.8267999999999999e-05 2024-09-18T10:15:23.023 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.023 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.023 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:23.023 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:12.952280+0000", 2024-09-18T10:15:23.023 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7368999999999999e-05 2024-09-18T10:15:23.023 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.023 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.023 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:23.023 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:12.952292+0000", 2024-09-18T10:15:23.023 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1657e-05 2024-09-18T10:15:23.024 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.024 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.024 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:23.024 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.054056+0000", 2024-09-18T10:15:23.024 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.101764519 2024-09-18T10:15:23.024 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.024 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.024 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:23.024 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.054063+0000", 2024-09-18T10:15:23.024 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.3320000000000003e-06 2024-09-18T10:15:23.024 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.024 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.024 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:23.024 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.154858+0000", 2024-09-18T10:15:23.025 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100794891 2024-09-18T10:15:23.025 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.025 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.025 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:23.025 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.154868+0000", 2024-09-18T10:15:23.025 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.9140000000000003e-06 2024-09-18T10:15:23.025 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:23.025 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:23.025 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:23.025 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.025 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.025 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.17s5 57/40 ECSubWrite(tid=109, reqid=client.4632.0:444, at_version=57'25, trim_to=0'0, roll_forward_to=53'18))", 2024-09-18T10:15:23.025 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:12.953422+0000", 2024-09-18T10:15:23.026 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 9.9832283640000004, 2024-09-18T10:15:23.026 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.201458103, 2024-09-18T10:15:23.026 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:23.026 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:15:23.026 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:23.026 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.026 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:23.026 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:12.953422+0000", 2024-09-18T10:15:23.026 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:23.026 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.026 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.026 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:23.026 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:12.953422+0000", 2024-09-18T10:15:23.026 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:23.027 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.027 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.027 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:23.027 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:12.953420+0000", 2024-09-18T10:15:23.027 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:23.027 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.027 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.027 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:23.027 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:12.953471+0000", 2024-09-18T10:15:23.027 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1016000000000001e-05 2024-09-18T10:15:23.027 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.027 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.027 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:23.027 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:12.953492+0000", 2024-09-18T10:15:23.028 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.1126000000000001e-05 2024-09-18T10:15:23.028 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.028 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.028 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:23.028 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:12.953506+0000", 2024-09-18T10:15:23.028 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4008999999999999e-05 2024-09-18T10:15:23.028 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.028 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.028 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:23.028 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.054683+0000", 2024-09-18T10:15:23.028 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.1011769 2024-09-18T10:15:23.028 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.028 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.029 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:23.029 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.054704+0000", 2024-09-18T10:15:23.029 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.0492999999999998e-05 2024-09-18T10:15:23.029 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.029 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.029 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:23.029 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.154870+0000", 2024-09-18T10:15:23.029 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100166777 2024-09-18T10:15:23.029 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.029 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.030 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:23.030 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.154880+0000", 2024-09-18T10:15:23.030 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.3759999999999997e-06 2024-09-18T10:15:23.030 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:23.030 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:23.030 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:23.030 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.030 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.030 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.4s5 57/40 ECSubWrite(tid=31, reqid=client.4632.0:446, at_version=57'19, trim_to=0'0, roll_forward_to=57'17))", 2024-09-18T10:15:23.031 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:13.265591+0000", 2024-09-18T10:15:23.031 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 9.6710586969999994, 2024-09-18T10:15:23.031 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20344775900000001, 2024-09-18T10:15:23.031 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:23.031 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:15:23.031 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:23.031 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.031 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:23.031 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.265591+0000", 2024-09-18T10:15:23.031 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:23.031 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.031 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.031 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:23.032 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.265591+0000", 2024-09-18T10:15:23.032 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:23.032 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.032 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.032 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:23.032 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.265591+0000", 2024-09-18T10:15:23.032 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:15:23.032 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.032 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.032 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:23.032 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.265740+0000", 2024-09-18T10:15:23.032 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00014936100000000001 2024-09-18T10:15:23.033 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.033 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.033 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:23.033 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.265761+0000", 2024-09-18T10:15:23.033 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.1348e-05 2024-09-18T10:15:23.033 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.033 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.033 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:23.033 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.365849+0000", 2024-09-18T10:15:23.033 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10008776 2024-09-18T10:15:23.033 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.033 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.033 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:23.034 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.466053+0000", 2024-09-18T10:15:23.034 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10020398699999999 2024-09-18T10:15:23.034 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.034 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.034 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:23.034 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.466069+0000", 2024-09-18T10:15:23.034 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.6163000000000001e-05 2024-09-18T10:15:23.034 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.034 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.034 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:23.034 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.469022+0000", 2024-09-18T10:15:23.034 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0029528499999999999 2024-09-18T10:15:23.034 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.034 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.034 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:23.035 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.469039+0000", 2024-09-18T10:15:23.035 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7034e-05 2024-09-18T10:15:23.035 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:23.035 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:23.035 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:23.035 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:23.035 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:23.035 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-09-18T10:15:23.037 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 14 1 2024-09-18T10:15:23.038 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_ops_in_flight 2024-09-18T10:15:23.040 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-18T10:15:23.037+0000 7f14c6a1a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 22138) UID: 0 2024-09-18T10:15:23.051 INFO:tasks.rados.rados.0.smithi189.stdout:223: finishing write tid 5 to smithi18925896-11 2024-09-18T10:15:23.051 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 11 v 82 (ObjNum 99 snap 17 seq_num 99) dirty exists 2024-09-18T10:15:23.051 INFO:tasks.rados.rados.0.smithi189.stdout:223: left oid 11 (ObjNum 99 snap 17 seq_num 99) 2024-09-18T10:15:23.141 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-18T10:15:23.137+0000 7f578c52f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22134) UID: 0 2024-09-18T10:15:23.148 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-09-18T10:15:23.148 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-09-18T10:15:23.148 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-09-18T10:15:23.148 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-09-18T10:15:23.159 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_blocked_ops 2024-09-18T10:15:23.242 INFO:tasks.ceph.osd.11.smithi189.stderr:2024-09-18T10:15:23.238+0000 7f718b60a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 22400) UID: 0 2024-09-18T10:15:23.254 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-09-18T10:15:23.254 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-09-18T10:15:23.254 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-09-18T10:15:23.254 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-09-18T10:15:23.254 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-09-18T10:15:23.264 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_historic_ops 2024-09-18T10:15:23.342 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-09-18T10:15:23.339+0000 7fdfa8698700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22087) UID: 0 2024-09-18T10:15:23.361 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-09-18T10:15:23.362 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-09-18T10:15:23.362 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-09-18T10:15:23.362 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-09-18T10:15:23.362 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.362 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4341.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-18T10:15:23.362 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:35.324351+0000", 2024-09-18T10:15:23.362 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 48.034225022000001, 2024-09-18T10:15:23.362 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.36640713699999999, 2024-09-18T10:15:23.362 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:23.362 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:23.362 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:23.363 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4341", 2024-09-18T10:15:23.363 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.29:0/3001591505", 2024-09-18T10:15:23.363 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 1 2024-09-18T10:15:23.363 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.363 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:23.363 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.363 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:23.363 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.324351+0000", 2024-09-18T10:15:23.363 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:23.363 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.363 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.363 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:23.363 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.324351+0000", 2024-09-18T10:15:23.363 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:23.364 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.364 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.364 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:23.364 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.324348+0000", 2024-09-18T10:15:23.364 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999976 2024-09-18T10:15:23.364 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.364 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.364 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:23.364 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.324359+0000", 2024-09-18T10:15:23.364 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1048e-05 2024-09-18T10:15:23.364 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.364 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.364 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:23.365 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.324383+0000", 2024-09-18T10:15:23.365 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.3813000000000001e-05 2024-09-18T10:15:23.365 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.365 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.365 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:23.365 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.324421+0000", 2024-09-18T10:15:23.365 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8012000000000001e-05 2024-09-18T10:15:23.365 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.365 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.365 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:23.365 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.324548+0000", 2024-09-18T10:15:23.365 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00012702199999999999 2024-09-18T10:15:23.365 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.366 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.366 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for peered", 2024-09-18T10:15:23.366 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.324565+0000", 2024-09-18T10:15:23.366 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.6325e-05 2024-09-18T10:15:23.366 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.366 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.366 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:23.366 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.588380+0000", 2024-09-18T10:15:23.366 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.263815458 2024-09-18T10:15:23.367 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.367 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.367 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:23.367 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.588626+0000", 2024-09-18T10:15:23.367 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00024633499999999998 2024-09-18T10:15:23.367 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.367 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.367 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 5", 2024-09-18T10:15:23.367 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.588803+0000", 2024-09-18T10:15:23.367 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00017640599999999999 2024-09-18T10:15:23.367 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.367 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.367 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:15:23.367 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.589943+0000", 2024-09-18T10:15:23.367 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0011400049999999999 2024-09-18T10:15:23.368 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.368 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.368 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:15:23.368 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.690623+0000", 2024-09-18T10:15:23.368 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100680032 2024-09-18T10:15:23.368 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.368 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.368 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:15:23.368 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.690702+0000", 2024-09-18T10:15:23.368 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.9342e-05 2024-09-18T10:15:23.368 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.368 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.368 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:23.369 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.690758+0000", 2024-09-18T10:15:23.369 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5639e-05 2024-09-18T10:15:23.369 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:23.369 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:23.369 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:23.369 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.369 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.369 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(osd.4.21:1 3.2s0 3:46b2871c:::smithi18925896-34:head [assert-version v6,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:23.370 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:51.536666+0000", 2024-09-18T10:15:23.370 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 31.821909643000001, 2024-09-18T10:15:23.370 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.30016440500000002, 2024-09-18T10:15:23.370 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:23.370 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "started", 2024-09-18T10:15:23.370 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:23.370 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.370 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:23.370 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.536666+0000", 2024-09-18T10:15:23.370 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:23.370 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.370 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.371 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:23.371 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.536666+0000", 2024-09-18T10:15:23.371 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:23.371 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.371 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.371 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:23.371 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.536664+0000", 2024-09-18T10:15:23.371 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:23.371 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.372 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.372 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:23.372 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.536673+0000", 2024-09-18T10:15:23.372 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.6319999999999997e-06 2024-09-18T10:15:23.372 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.372 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.372 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:23.372 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.536688+0000", 2024-09-18T10:15:23.372 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.5058999999999999e-05 2024-09-18T10:15:23.372 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.372 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.372 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:23.373 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.536696+0000", 2024-09-18T10:15:23.373 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.8860000000000004e-06 2024-09-18T10:15:23.373 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.373 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.373 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:23.373 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.536868+0000", 2024-09-18T10:15:23.373 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000172475 2024-09-18T10:15:23.373 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.373 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.373 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:23.373 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.537125+0000", 2024-09-18T10:15:23.373 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000256906 2024-09-18T10:15:23.373 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.374 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.374 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:23.374 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.836830+0000", 2024-09-18T10:15:23.374 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.299705414 2024-09-18T10:15:23.374 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:23.374 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:23.374 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:23.374 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.374 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.374 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:264 3.2s0 3:46b2871c:::smithi18925896-34:head [append 0~606208] snapc 5=[5,4,3] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-18T10:15:23.374 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:56.739704+0000", 2024-09-18T10:15:23.374 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 26.618872036999999, 2024-09-18T10:15:23.374 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.79624036899999995, 2024-09-18T10:15:23.374 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:23.375 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:23.375 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:23.375 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:15:23.375 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:23.375 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 264 2024-09-18T10:15:23.375 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.375 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:23.375 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.375 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:23.375 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.739704+0000", 2024-09-18T10:15:23.375 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:23.375 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.375 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.376 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:23.376 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.739704+0000", 2024-09-18T10:15:23.376 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:23.376 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.376 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.376 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:23.376 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.739702+0000", 2024-09-18T10:15:23.376 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:23.376 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.376 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.376 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:23.376 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.740467+0000", 2024-09-18T10:15:23.377 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00076493100000000005 2024-09-18T10:15:23.377 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.377 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.377 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:23.377 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.740485+0000", 2024-09-18T10:15:23.377 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8516e-05 2024-09-18T10:15:23.377 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.377 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.377 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:23.377 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.740495+0000", 2024-09-18T10:15:23.377 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.2669999999999995e-06 2024-09-18T10:15:23.377 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.377 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.378 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:23.378 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.028948+0000", 2024-09-18T10:15:23.378 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.28845313500000003 2024-09-18T10:15:23.378 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.378 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.378 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for peered", 2024-09-18T10:15:23.378 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.028969+0000", 2024-09-18T10:15:23.378 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.1369000000000002e-05 2024-09-18T10:15:23.378 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.378 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.378 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:23.378 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.228324+0000", 2024-09-18T10:15:23.378 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.19935476699999999 2024-09-18T10:15:23.378 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.379 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.379 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:23.379 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.228558+0000", 2024-09-18T10:15:23.379 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00023393099999999999 2024-09-18T10:15:23.379 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.379 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.379 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:23.379 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.230390+0000", 2024-09-18T10:15:23.379 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018326970000000001 2024-09-18T10:15:23.379 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.379 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.379 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:23.379 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.239491+0000", 2024-09-18T10:15:23.379 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0091007950000000001 2024-09-18T10:15:23.380 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.380 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.380 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:15:23.380 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.535834+0000", 2024-09-18T10:15:23.380 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.29634234100000001 2024-09-18T10:15:23.380 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.380 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.380 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:23.380 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.535944+0000", 2024-09-18T10:15:23.380 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000110401 2024-09-18T10:15:23.380 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:23.380 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:23.380 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:23.381 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.381 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.381 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:265 3.2s0 3:46b2871c:::smithi18925896-34:head [append 0~376832] snapc 5=[5,4,3] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-18T10:15:23.381 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:56.741882+0000", 2024-09-18T10:15:23.381 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 26.616694015, 2024-09-18T10:15:23.381 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.80071129500000005, 2024-09-18T10:15:23.381 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:23.381 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:23.381 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:23.381 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:15:23.381 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:23.381 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 265 2024-09-18T10:15:23.381 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.381 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:23.382 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.382 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:23.382 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.741882+0000", 2024-09-18T10:15:23.382 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:23.382 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.382 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.382 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:23.382 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.741882+0000", 2024-09-18T10:15:23.382 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:23.382 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.382 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.382 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:23.382 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.741880+0000", 2024-09-18T10:15:23.382 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:23.383 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.383 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.383 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:23.383 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742251+0000", 2024-09-18T10:15:23.383 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00037050499999999998 2024-09-18T10:15:23.383 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.383 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.383 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:23.383 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742264+0000", 2024-09-18T10:15:23.383 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2787e-05 2024-09-18T10:15:23.383 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.383 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.383 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:23.383 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742269+0000", 2024-09-18T10:15:23.384 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.0509999999999996e-06 2024-09-18T10:15:23.384 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.384 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.384 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:23.384 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.234342+0000", 2024-09-18T10:15:23.384 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.492073712 2024-09-18T10:15:23.384 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.384 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.384 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:23.384 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.234556+0000", 2024-09-18T10:15:23.384 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00021380699999999999 2024-09-18T10:15:23.384 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.384 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.384 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:23.385 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.236316+0000", 2024-09-18T10:15:23.385 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0017597100000000001 2024-09-18T10:15:23.385 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.385 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.385 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:23.385 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.241476+0000", 2024-09-18T10:15:23.385 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0051604449999999996 2024-09-18T10:15:23.385 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.385 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.385 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:15:23.385 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.542502+0000", 2024-09-18T10:15:23.385 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.30102573700000002 2024-09-18T10:15:23.385 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.386 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.386 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:23.386 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.542593+0000", 2024-09-18T10:15:23.386 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.0859000000000005e-05 2024-09-18T10:15:23.386 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:23.386 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:23.386 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:23.386 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.386 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.386 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:266 3.2s0 3:46b2871c:::smithi18925896-34:head [setxattr _header (56) in=7b] snapc 5=[5,4,3] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-18T10:15:23.386 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:56.742274+0000", 2024-09-18T10:15:23.386 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 26.616301384, 2024-09-18T10:15:23.387 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.80056005299999999, 2024-09-18T10:15:23.387 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:23.387 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:23.387 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:23.387 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:15:23.387 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:23.387 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 266 2024-09-18T10:15:23.387 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.387 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:23.387 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.387 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:23.387 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742274+0000", 2024-09-18T10:15:23.387 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:23.388 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.388 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.388 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:23.388 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742274+0000", 2024-09-18T10:15:23.388 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:23.388 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.388 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.388 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:23.388 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742274+0000", 2024-09-18T10:15:23.388 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:23.388 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.388 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.388 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:23.389 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742278+0000", 2024-09-18T10:15:23.389 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.0840000000000002e-06 2024-09-18T10:15:23.389 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.389 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.389 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:23.389 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742284+0000", 2024-09-18T10:15:23.389 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.1090000000000003e-06 2024-09-18T10:15:23.389 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.389 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.389 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:23.389 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742289+0000", 2024-09-18T10:15:23.389 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.4890000000000003e-06 2024-09-18T10:15:23.389 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.390 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.390 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:23.390 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.237660+0000", 2024-09-18T10:15:23.390 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.49537068299999998 2024-09-18T10:15:23.390 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.390 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.390 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:23.390 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.237924+0000", 2024-09-18T10:15:23.390 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00026387300000000003 2024-09-18T10:15:23.390 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.390 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.390 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:23.390 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.238979+0000", 2024-09-18T10:15:23.391 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0010549380000000001 2024-09-18T10:15:23.391 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.391 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.391 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:23.391 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.243571+0000", 2024-09-18T10:15:23.391 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0045919749999999999 2024-09-18T10:15:23.391 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.391 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.391 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:15:23.391 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.542742+0000", 2024-09-18T10:15:23.391 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.29917095999999999 2024-09-18T10:15:23.391 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.391 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.391 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:23.392 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.542834+0000", 2024-09-18T10:15:23.392 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.2536999999999998e-05 2024-09-18T10:15:23.392 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:23.392 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:23.392 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:23.392 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.392 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.392 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:267 3.2s0 3:46b2871c:::smithi18925896-34:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e41)", 2024-09-18T10:15:23.392 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:56.742292+0000", 2024-09-18T10:15:23.392 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 26.616283545999998, 2024-09-18T10:15:23.392 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.98973012599999999, 2024-09-18T10:15:23.392 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:23.392 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "started", 2024-09-18T10:15:23.393 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:23.393 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:15:23.393 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:23.393 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 267 2024-09-18T10:15:23.393 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.393 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:23.393 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.393 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:23.393 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742292+0000", 2024-09-18T10:15:23.393 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:23.393 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.393 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.393 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:23.394 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742292+0000", 2024-09-18T10:15:23.394 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:23.394 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.394 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.394 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:23.394 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742292+0000", 2024-09-18T10:15:23.394 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:23.394 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.394 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.394 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:23.394 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742294+0000", 2024-09-18T10:15:23.394 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.103e-06 2024-09-18T10:15:23.394 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.395 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.395 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:23.395 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742298+0000", 2024-09-18T10:15:23.395 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8070000000000001e-06 2024-09-18T10:15:23.395 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.395 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.395 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:23.395 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742300+0000", 2024-09-18T10:15:23.395 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5890000000000001e-06 2024-09-18T10:15:23.395 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.395 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.395 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:23.395 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.239711+0000", 2024-09-18T10:15:23.396 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.497410625 2024-09-18T10:15:23.396 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.396 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.396 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for rw locks", 2024-09-18T10:15:23.396 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.239786+0000", 2024-09-18T10:15:23.396 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.4827999999999996e-05 2024-09-18T10:15:23.396 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.396 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.396 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:23.396 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.543234+0000", 2024-09-18T10:15:23.396 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.30344799099999997 2024-09-18T10:15:23.396 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.397 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.397 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:23.397 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.543335+0000", 2024-09-18T10:15:23.397 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000101625 2024-09-18T10:15:23.397 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.397 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.397 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:23.397 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.732022+0000", 2024-09-18T10:15:23.397 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.18868686700000001 2024-09-18T10:15:23.397 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:23.397 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:23.397 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:23.397 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.398 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.398 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.es1 41/32 ECSubWrite(tid=16, reqid=osd.11.0:15, at_version=0'0, trim_to=0'0, roll_forward_to=41'11))", 2024-09-18T10:15:23.398 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:56.930285+0000", 2024-09-18T10:15:23.398 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 26.428290633, 2024-09-18T10:15:23.398 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.211817538, 2024-09-18T10:15:23.398 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:23.398 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:15:23.398 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:23.398 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.398 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:23.398 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.930285+0000", 2024-09-18T10:15:23.398 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:23.398 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.399 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.399 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:23.399 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.930285+0000", 2024-09-18T10:15:23.399 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:23.399 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.399 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.399 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:23.399 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.930284+0000", 2024-09-18T10:15:23.399 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:23.399 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.399 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.399 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:23.400 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.930313+0000", 2024-09-18T10:15:23.400 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9394999999999999e-05 2024-09-18T10:15:23.400 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.400 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.400 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:23.400 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.930337+0000", 2024-09-18T10:15:23.400 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4239e-05 2024-09-18T10:15:23.400 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.400 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.400 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:23.400 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.930352+0000", 2024-09-18T10:15:23.400 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4577e-05 2024-09-18T10:15:23.400 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.400 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.401 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:23.401 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.030572+0000", 2024-09-18T10:15:23.401 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100219771 2024-09-18T10:15:23.401 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.401 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.401 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:23.401 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.030604+0000", 2024-09-18T10:15:23.401 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2654999999999998e-05 2024-09-18T10:15:23.401 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.401 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.401 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:23.401 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.142047+0000", 2024-09-18T10:15:23.401 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.111442992 2024-09-18T10:15:23.402 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.402 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.402 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:23.402 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.142103+0000", 2024-09-18T10:15:23.402 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5291999999999998e-05 2024-09-18T10:15:23.402 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:23.402 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:23.402 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:23.402 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.402 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.402 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.es1 47/32 ECSubWrite(tid=21, reqid=client.4632.0:346, at_version=47'17, trim_to=0'0, roll_forward_to=44'15))", 2024-09-18T10:15:23.402 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:02.906267+0000", 2024-09-18T10:15:23.402 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 20.452308334000001, 2024-09-18T10:15:23.402 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20475042399999999, 2024-09-18T10:15:23.403 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:23.403 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:15:23.403 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:23.403 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.403 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:23.403 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:02.906267+0000", 2024-09-18T10:15:23.403 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:23.403 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.403 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.403 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:23.403 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:02.906267+0000", 2024-09-18T10:15:23.403 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:23.403 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.404 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.404 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:23.404 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:02.906267+0000", 2024-09-18T10:15:23.404 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:23.404 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.404 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.404 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:23.404 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:02.907107+0000", 2024-09-18T10:15:23.404 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00084074799999999999 2024-09-18T10:15:23.404 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.404 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.404 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:23.405 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:02.907144+0000", 2024-09-18T10:15:23.405 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.6902e-05 2024-09-18T10:15:23.405 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.405 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.405 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:23.405 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:03.007227+0000", 2024-09-18T10:15:23.405 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100083167 2024-09-18T10:15:23.405 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.405 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.405 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:23.405 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:03.105676+0000", 2024-09-18T10:15:23.405 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.098448716000000006 2024-09-18T10:15:23.405 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.406 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.406 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:23.406 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:03.105708+0000", 2024-09-18T10:15:23.406 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2119000000000002e-05 2024-09-18T10:15:23.406 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.406 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.406 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:23.406 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:03.110947+0000", 2024-09-18T10:15:23.406 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0052382210000000004 2024-09-18T10:15:23.406 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.406 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.406 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:23.406 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:03.111018+0000", 2024-09-18T10:15:23.406 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.1204000000000005e-05 2024-09-18T10:15:23.407 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:23.407 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:23.407 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:23.407 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.407 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.407 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(osd.7.22:2 3.2s0 3:427b2c0b:::smithi18925896-42:head [assert-version v8,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-18T10:15:23.407 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:08.756861+0000", 2024-09-18T10:15:23.407 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 14.60171482, 2024-09-18T10:15:23.407 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.27903143600000002, 2024-09-18T10:15:23.407 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:23.407 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "started", 2024-09-18T10:15:23.407 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:23.408 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.408 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:23.408 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:08.756861+0000", 2024-09-18T10:15:23.408 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:23.408 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.408 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.408 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:23.408 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:08.756861+0000", 2024-09-18T10:15:23.408 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:23.408 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.408 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.408 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:23.408 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:08.756859+0000", 2024-09-18T10:15:23.409 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999976 2024-09-18T10:15:23.409 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.409 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.409 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:23.409 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:08.756869+0000", 2024-09-18T10:15:23.409 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0389e-05 2024-09-18T10:15:23.409 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.409 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.409 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:23.409 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:08.756887+0000", 2024-09-18T10:15:23.409 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8094e-05 2024-09-18T10:15:23.409 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.409 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.410 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:23.410 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:08.756896+0000", 2024-09-18T10:15:23.410 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.9290000000000004e-06 2024-09-18T10:15:23.410 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.410 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.410 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:23.410 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:08.757008+0000", 2024-09-18T10:15:23.410 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00011184 2024-09-18T10:15:23.410 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.410 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.410 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op must wait for map", 2024-09-18T10:15:23.410 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:08.757025+0000", 2024-09-18T10:15:23.410 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.6912999999999999e-05 2024-09-18T10:15:23.411 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.411 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.411 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:23.411 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:08.931998+0000", 2024-09-18T10:15:23.411 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.174973289 2024-09-18T10:15:23.411 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.411 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.411 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:23.412 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:08.932274+0000", 2024-09-18T10:15:23.412 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00027573799999999998 2024-09-18T10:15:23.412 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.412 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.412 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:23.412 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:09.035892+0000", 2024-09-18T10:15:23.412 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10361846700000001 2024-09-18T10:15:23.412 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:23.412 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:23.412 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:23.412 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.412 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.413 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.17s4 57/40 ECSubWrite(tid=107, reqid=client.4632.0:442, at_version=57'23, trim_to=0'0, roll_forward_to=53'18))", 2024-09-18T10:15:23.413 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:12.950559+0000", 2024-09-18T10:15:23.413 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 10.408016465999999, 2024-09-18T10:15:23.413 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20580406000000001, 2024-09-18T10:15:23.413 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:23.413 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:15:23.413 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:23.413 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.413 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:23.413 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:12.950559+0000", 2024-09-18T10:15:23.413 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:23.413 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.414 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.414 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:23.414 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:12.950559+0000", 2024-09-18T10:15:23.414 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:23.414 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.414 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.414 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:23.414 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:12.950558+0000", 2024-09-18T10:15:23.414 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:23.414 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.414 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.414 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:23.414 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:12.951020+0000", 2024-09-18T10:15:23.415 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00046179100000000002 2024-09-18T10:15:23.415 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.415 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.415 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:23.415 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:12.951064+0000", 2024-09-18T10:15:23.415 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4359e-05 2024-09-18T10:15:23.415 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.415 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.415 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:23.415 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:12.951079+0000", 2024-09-18T10:15:23.415 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.5469e-05 2024-09-18T10:15:23.415 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.415 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.416 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:23.416 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.053581+0000", 2024-09-18T10:15:23.416 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10250205499999999 2024-09-18T10:15:23.416 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.416 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.416 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:23.416 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.053594+0000", 2024-09-18T10:15:23.416 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2452e-05 2024-09-18T10:15:23.416 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.416 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.416 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:23.416 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.156337+0000", 2024-09-18T10:15:23.416 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.102743495 2024-09-18T10:15:23.417 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.417 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.417 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:23.417 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.156363+0000", 2024-09-18T10:15:23.417 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5862000000000001e-05 2024-09-18T10:15:23.417 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:23.417 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:23.417 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:23.417 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.417 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.417 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.17s4 57/40 ECSubWrite(tid=108, reqid=client.4632.0:443, at_version=57'24, trim_to=0'0, roll_forward_to=53'18))", 2024-09-18T10:15:23.417 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:12.952248+0000", 2024-09-18T10:15:23.417 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 10.406327835999999, 2024-09-18T10:15:23.418 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20414522900000001, 2024-09-18T10:15:23.418 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:23.418 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:15:23.418 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:23.418 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.418 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:23.418 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:12.952248+0000", 2024-09-18T10:15:23.418 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:23.418 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.418 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.418 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:23.418 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:12.952248+0000", 2024-09-18T10:15:23.418 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:23.419 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.419 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.419 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:23.419 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:12.952247+0000", 2024-09-18T10:15:23.419 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:15:23.419 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.419 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.419 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:23.419 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:12.952310+0000", 2024-09-18T10:15:23.419 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.3102000000000007e-05 2024-09-18T10:15:23.419 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.419 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.420 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:23.420 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:12.952336+0000", 2024-09-18T10:15:23.420 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5930000000000001e-05 2024-09-18T10:15:23.420 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.420 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.420 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:23.420 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:12.952346+0000", 2024-09-18T10:15:23.420 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0739e-05 2024-09-18T10:15:23.420 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.420 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.420 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:23.420 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.054995+0000", 2024-09-18T10:15:23.420 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.102648218 2024-09-18T10:15:23.421 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.421 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.421 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:23.421 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.055003+0000", 2024-09-18T10:15:23.421 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.6540000000000005e-06 2024-09-18T10:15:23.421 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.421 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.421 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:23.421 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.156370+0000", 2024-09-18T10:15:23.421 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.101367055 2024-09-18T10:15:23.421 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.421 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.422 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:23.422 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.156393+0000", 2024-09-18T10:15:23.422 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.2637e-05 2024-09-18T10:15:23.422 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:23.422 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:23.422 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:23.422 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.422 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.422 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:452 3.2s0 3:47202842:::smithi18925896-22:head [delete] snapc 13=[12,11,10,f,e,b] ondisk+write+known_if_redirected+full_force+supports_pool_eio e58)", 2024-09-18T10:15:23.422 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:13.551955+0000", 2024-09-18T10:15:23.422 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 9.8066205330000003, 2024-09-18T10:15:23.422 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.31127974800000002, 2024-09-18T10:15:23.422 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:23.422 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:23.423 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:23.423 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:15:23.423 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:23.423 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 452 2024-09-18T10:15:23.423 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.423 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:23.423 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.423 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:23.423 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.551955+0000", 2024-09-18T10:15:23.423 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:23.423 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.423 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.423 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:23.424 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.551955+0000", 2024-09-18T10:15:23.424 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:23.424 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.424 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.424 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:23.424 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.551952+0000", 2024-09-18T10:15:23.424 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999967 2024-09-18T10:15:23.424 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.424 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.424 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:23.424 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.551970+0000", 2024-09-18T10:15:23.424 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8366999999999999e-05 2024-09-18T10:15:23.424 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.425 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.425 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:23.425 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.552016+0000", 2024-09-18T10:15:23.425 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.5868000000000002e-05 2024-09-18T10:15:23.425 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.425 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.425 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:23.425 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.552037+0000", 2024-09-18T10:15:23.425 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.1311999999999999e-05 2024-09-18T10:15:23.425 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.425 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.425 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:23.425 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.560400+0000", 2024-09-18T10:15:23.425 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0083625920000000003 2024-09-18T10:15:23.426 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.426 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.426 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op must wait for map", 2024-09-18T10:15:23.426 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.560411+0000", 2024-09-18T10:15:23.426 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1043000000000001e-05 2024-09-18T10:15:23.426 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.426 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.426 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:23.426 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.562108+0000", 2024-09-18T10:15:23.426 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0016968479999999999 2024-09-18T10:15:23.426 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.426 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.426 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:23.427 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.562286+0000", 2024-09-18T10:15:23.427 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00017818599999999999 2024-09-18T10:15:23.427 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.427 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.427 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:23.427 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.563336+0000", 2024-09-18T10:15:23.427 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001050051 2024-09-18T10:15:23.427 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.427 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.427 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:23.427 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.565490+0000", 2024-09-18T10:15:23.427 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0021545119999999999 2024-09-18T10:15:23.427 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.427 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.428 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:15:23.428 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.863047+0000", 2024-09-18T10:15:23.428 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.29755694199999999 2024-09-18T10:15:23.428 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.428 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.428 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:23.428 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.863235+0000", 2024-09-18T10:15:23.428 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000187524 2024-09-18T10:15:23.428 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:23.428 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:23.428 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:23.428 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.428 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.429 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDPGPushReply(3.15s0 61/60 [PushReplyOp(3:acecabac:::smithi18925896-4:head)])", 2024-09-18T10:15:23.429 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:17.756036+0000", 2024-09-18T10:15:23.429 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 5.6025398309999996, 2024-09-18T10:15:23.429 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.30209286600000002, 2024-09-18T10:15:23.429 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:23.429 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:15:23.429 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:23.429 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.429 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:23.429 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.756036+0000", 2024-09-18T10:15:23.429 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:23.429 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.429 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.429 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:23.430 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.756036+0000", 2024-09-18T10:15:23.430 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:23.430 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.430 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.430 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:23.430 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.756035+0000", 2024-09-18T10:15:23.430 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:15:23.430 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.430 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.430 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:23.430 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.756042+0000", 2024-09-18T10:15:23.430 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.6630000000000004e-06 2024-09-18T10:15:23.430 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.431 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.431 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:23.431 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.756057+0000", 2024-09-18T10:15:23.431 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.5719e-05 2024-09-18T10:15:23.431 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.431 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.431 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:23.431 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.756068+0000", 2024-09-18T10:15:23.431 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0207000000000001e-05 2024-09-18T10:15:23.431 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.432 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.432 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:23.432 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:18.058067+0000", 2024-09-18T10:15:23.432 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.30199969599999998 2024-09-18T10:15:23.432 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.432 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.432 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:23.432 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:18.058129+0000", 2024-09-18T10:15:23.432 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.1303999999999994e-05 2024-09-18T10:15:23.432 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:23.432 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:23.432 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:23.432 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.433 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.433 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:496 3.2s0 3:44cb9e27:::smithi18925896-5:head [read 0~81920,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T10:15:23.433 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:17.889532+0000", 2024-09-18T10:15:23.433 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 5.4690439270000004, 2024-09-18T10:15:23.433 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.32688004999999998, 2024-09-18T10:15:23.433 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:23.433 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "started", 2024-09-18T10:15:23.433 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:23.433 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:15:23.433 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:23.433 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 496 2024-09-18T10:15:23.433 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.434 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:23.434 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.434 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:23.434 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889532+0000", 2024-09-18T10:15:23.434 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:23.434 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.434 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.434 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:23.434 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889532+0000", 2024-09-18T10:15:23.434 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:23.434 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.434 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.435 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:23.435 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889529+0000", 2024-09-18T10:15:23.435 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999971 2024-09-18T10:15:23.435 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.435 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.435 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:23.435 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889546+0000", 2024-09-18T10:15:23.435 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.6690000000000001e-05 2024-09-18T10:15:23.435 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.435 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.435 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:23.435 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889580+0000", 2024-09-18T10:15:23.435 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.4638999999999998e-05 2024-09-18T10:15:23.435 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.436 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.436 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:23.436 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889596+0000", 2024-09-18T10:15:23.436 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.5571999999999999e-05 2024-09-18T10:15:23.436 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.436 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.436 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:23.436 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.989817+0000", 2024-09-18T10:15:23.436 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100221713 2024-09-18T10:15:23.436 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.436 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.436 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op must wait for map", 2024-09-18T10:15:23.436 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.989836+0000", 2024-09-18T10:15:23.437 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8516e-05 2024-09-18T10:15:23.437 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.437 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.437 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:23.437 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.990420+0000", 2024-09-18T10:15:23.437 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000583918 2024-09-18T10:15:23.437 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.437 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.437 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:23.437 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.990693+0000", 2024-09-18T10:15:23.437 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00027301499999999999 2024-09-18T10:15:23.437 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.437 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.438 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:23.438 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:18.216412+0000", 2024-09-18T10:15:23.438 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.22571880699999999 2024-09-18T10:15:23.438 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:23.438 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:23.438 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:23.438 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.438 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.438 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:497 3.2s0 3:44cb9e27:::smithi18925896-5:head [read 0~81920,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T10:15:23.438 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:17.889627+0000", 2024-09-18T10:15:23.438 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 5.4689490100000002, 2024-09-18T10:15:23.438 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.32738385800000003, 2024-09-18T10:15:23.439 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:23.439 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "started", 2024-09-18T10:15:23.439 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:23.439 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:15:23.439 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:23.439 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 497 2024-09-18T10:15:23.439 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.439 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:23.439 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.439 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:23.439 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889627+0000", 2024-09-18T10:15:23.439 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:23.439 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.440 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.440 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:23.440 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889627+0000", 2024-09-18T10:15:23.440 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:23.440 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.440 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.440 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:23.440 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889625+0000", 2024-09-18T10:15:23.440 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:15:23.440 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.440 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.440 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:23.441 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889633+0000", 2024-09-18T10:15:23.441 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.4170000000000003e-06 2024-09-18T10:15:23.441 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.441 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.441 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:23.441 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889664+0000", 2024-09-18T10:15:23.441 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0589000000000001e-05 2024-09-18T10:15:23.441 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.441 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.441 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:23.441 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889672+0000", 2024-09-18T10:15:23.441 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.3729999999999993e-06 2024-09-18T10:15:23.441 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.442 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.442 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:23.442 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.990017+0000", 2024-09-18T10:15:23.442 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100345608 2024-09-18T10:15:23.442 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.442 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.442 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:23.442 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.990046+0000", 2024-09-18T10:15:23.442 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8912999999999999e-05 2024-09-18T10:15:23.442 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.442 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.442 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:23.442 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.991459+0000", 2024-09-18T10:15:23.442 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0014122340000000001 2024-09-18T10:15:23.442 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.443 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.443 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:23.443 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.991631+0000", 2024-09-18T10:15:23.443 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000172029 2024-09-18T10:15:23.443 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.443 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.443 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:23.443 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:18.217010+0000", 2024-09-18T10:15:23.443 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.22537980499999999 2024-09-18T10:15:23.443 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:23.444 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:23.444 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:23.444 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.444 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.444 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:498 3.2s0 3:44cb9e27:::smithi18925896-5:head [read 0~81920,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T10:15:23.444 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:17.889709+0000", 2024-09-18T10:15:23.444 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 5.468866652, 2024-09-18T10:15:23.444 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.32775850600000001, 2024-09-18T10:15:23.444 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:23.444 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "started", 2024-09-18T10:15:23.444 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:23.444 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:15:23.444 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:23.445 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 498 2024-09-18T10:15:23.445 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.445 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:23.445 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.445 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:23.445 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889709+0000", 2024-09-18T10:15:23.445 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:23.445 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.445 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.445 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:23.445 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889709+0000", 2024-09-18T10:15:23.445 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:23.445 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.445 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.446 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:23.446 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889708+0000", 2024-09-18T10:15:23.446 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:15:23.446 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.446 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.446 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:23.446 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889716+0000", 2024-09-18T10:15:23.446 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.0989999999999999e-06 2024-09-18T10:15:23.446 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.446 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.446 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:23.446 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889732+0000", 2024-09-18T10:15:23.446 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.6093e-05 2024-09-18T10:15:23.447 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.447 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.447 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:23.447 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889742+0000", 2024-09-18T10:15:23.447 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.4599999999999992e-06 2024-09-18T10:15:23.447 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.447 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.447 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:23.447 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.992175+0000", 2024-09-18T10:15:23.447 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10243377200000001 2024-09-18T10:15:23.447 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.447 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.447 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:23.447 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.992333+0000", 2024-09-18T10:15:23.447 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00015753499999999999 2024-09-18T10:15:23.448 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.448 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.448 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:23.448 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:18.217467+0000", 2024-09-18T10:15:23.448 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.22513459999999999 2024-09-18T10:15:23.448 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:23.448 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:23.448 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:23.448 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.448 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.448 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:516 3.2s0 3:44cb9e27:::smithi18925896-5:head [append 0~425984] snapc 1a=[1a,19,18,16,12,11,10,f,e] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-18T10:15:23.448 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:19.430973+0000", 2024-09-18T10:15:23.448 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 3.9276022629999998, 2024-09-18T10:15:23.448 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.59812562800000002, 2024-09-18T10:15:23.448 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:23.449 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:23.449 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:23.449 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:15:23.449 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:23.449 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 516 2024-09-18T10:15:23.449 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.449 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:23.449 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.449 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:23.449 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.430973+0000", 2024-09-18T10:15:23.449 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:23.449 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.449 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.450 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:23.450 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.430973+0000", 2024-09-18T10:15:23.450 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:23.450 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.450 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.450 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:23.450 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.430970+0000", 2024-09-18T10:15:23.450 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999967 2024-09-18T10:15:23.450 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.450 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.450 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:23.450 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.431995+0000", 2024-09-18T10:15:23.450 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001024896 2024-09-18T10:15:23.450 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.450 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.451 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:23.451 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.432030+0000", 2024-09-18T10:15:23.451 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5241999999999999e-05 2024-09-18T10:15:23.451 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.451 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.451 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:23.451 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.432044+0000", 2024-09-18T10:15:23.451 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.411e-05 2024-09-18T10:15:23.451 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.451 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.451 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:23.451 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.432226+0000", 2024-09-18T10:15:23.451 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000181841 2024-09-18T10:15:23.451 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.451 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.451 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:23.452 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.432421+0000", 2024-09-18T10:15:23.452 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000194527 2024-09-18T10:15:23.452 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.452 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.452 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:23.452 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.434150+0000", 2024-09-18T10:15:23.452 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001729742 2024-09-18T10:15:23.452 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.452 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.452 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:23.452 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.436513+0000", 2024-09-18T10:15:23.452 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0023622299999999999 2024-09-18T10:15:23.452 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.452 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.453 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:15:23.453 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:20.029005+0000", 2024-09-18T10:15:23.453 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.59249230100000005 2024-09-18T10:15:23.453 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.453 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.453 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:23.453 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:20.029099+0000", 2024-09-18T10:15:23.453 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.4042000000000005e-05 2024-09-18T10:15:23.453 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:23.453 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:23.453 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:23.453 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.453 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.453 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:517 3.2s0 3:44cb9e27:::smithi18925896-5:head [append 0~786432] snapc 1a=[1a,19,18,16,12,11,10,f,e] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-18T10:15:23.454 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:19.439831+0000", 2024-09-18T10:15:23.454 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 3.9187447239999997, 2024-09-18T10:15:23.454 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.58958403800000003, 2024-09-18T10:15:23.454 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:23.454 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:23.454 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:23.454 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:15:23.454 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:23.454 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 517 2024-09-18T10:15:23.454 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.454 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:23.454 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.454 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:23.454 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.439831+0000", 2024-09-18T10:15:23.454 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:23.455 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.455 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.455 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:23.455 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.439831+0000", 2024-09-18T10:15:23.455 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:23.455 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.455 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.455 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:23.455 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.439828+0000", 2024-09-18T10:15:23.455 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999976 2024-09-18T10:15:23.455 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.455 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.455 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:23.456 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.441109+0000", 2024-09-18T10:15:23.456 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001280988 2024-09-18T10:15:23.456 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.456 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.456 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:23.456 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.441168+0000", 2024-09-18T10:15:23.456 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.8869000000000002e-05 2024-09-18T10:15:23.456 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.456 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.456 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:23.456 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.541302+0000", 2024-09-18T10:15:23.456 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100134055 2024-09-18T10:15:23.456 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.457 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.457 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:23.457 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.637962+0000", 2024-09-18T10:15:23.457 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.096659682999999996 2024-09-18T10:15:23.457 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.457 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.457 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:23.457 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.638164+0000", 2024-09-18T10:15:23.457 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00020226700000000001 2024-09-18T10:15:23.457 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.457 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.457 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:23.457 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.639933+0000", 2024-09-18T10:15:23.458 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001768501 2024-09-18T10:15:23.458 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.458 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.458 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:23.458 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.646572+0000", 2024-09-18T10:15:23.458 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.006639069 2024-09-18T10:15:23.458 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.458 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.458 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:15:23.458 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:20.029341+0000", 2024-09-18T10:15:23.458 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.38276939599999998 2024-09-18T10:15:23.458 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.458 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.459 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:23.459 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:20.029415+0000", 2024-09-18T10:15:23.459 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.3726000000000005e-05 2024-09-18T10:15:23.459 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:23.459 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:23.459 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:23.459 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.459 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.459 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:518 3.2s0 3:44cb9e27:::smithi18925896-5:head [append 0~376832] snapc 1a=[1a,19,18,16,12,11,10,f,e] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-18T10:15:23.459 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:19.541352+0000", 2024-09-18T10:15:23.459 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 3.8172236860000002, 2024-09-18T10:15:23.459 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.48828847600000003, 2024-09-18T10:15:23.459 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:23.460 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:23.460 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:23.460 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:15:23.460 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:23.460 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 518 2024-09-18T10:15:23.460 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.460 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:23.460 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.460 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:23.460 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.541352+0000", 2024-09-18T10:15:23.460 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:23.460 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.460 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.461 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:23.461 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.541352+0000", 2024-09-18T10:15:23.461 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:23.461 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.461 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.461 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:23.461 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.541350+0000", 2024-09-18T10:15:23.461 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:23.461 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.461 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.461 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:23.461 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.541834+0000", 2024-09-18T10:15:23.461 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00048402500000000002 2024-09-18T10:15:23.462 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.462 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.462 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:23.462 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.541861+0000", 2024-09-18T10:15:23.462 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.7333000000000001e-05 2024-09-18T10:15:23.462 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.462 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.462 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:23.462 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.541875+0000", 2024-09-18T10:15:23.462 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4025999999999999e-05 2024-09-18T10:15:23.462 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.462 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.462 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:23.463 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.641684+0000", 2024-09-18T10:15:23.463 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.099808872000000007 2024-09-18T10:15:23.463 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.463 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.463 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:23.463 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.641873+0000", 2024-09-18T10:15:23.463 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00018933200000000001 2024-09-18T10:15:23.463 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.463 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.463 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:23.463 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.643401+0000", 2024-09-18T10:15:23.463 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001527104 2024-09-18T10:15:23.463 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.463 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.464 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:23.464 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.646854+0000", 2024-09-18T10:15:23.464 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0034532069999999998 2024-09-18T10:15:23.464 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.464 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.464 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:15:23.464 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:20.029569+0000", 2024-09-18T10:15:23.464 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.38271509599999998 2024-09-18T10:15:23.464 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.464 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.464 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:23.464 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:20.029640+0000", 2024-09-18T10:15:23.464 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.1594000000000006e-05 2024-09-18T10:15:23.465 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:23.465 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:23.465 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:23.465 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.465 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.465 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:519 3.2s0 3:44cb9e27:::smithi18925896-5:head [setxattr _header (56) in=7b] snapc 1a=[1a,19,18,16,12,11,10,f,e] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-18T10:15:23.465 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:19.541886+0000", 2024-09-18T10:15:23.465 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 3.8166899990000003, 2024-09-18T10:15:23.465 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.488165768, 2024-09-18T10:15:23.465 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:23.465 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:23.465 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:23.465 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:15:23.466 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:23.466 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 519 2024-09-18T10:15:23.466 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.466 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:23.466 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.466 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:23.466 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.541886+0000", 2024-09-18T10:15:23.466 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:23.466 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.466 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.466 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:23.466 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.541886+0000", 2024-09-18T10:15:23.466 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:23.466 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.467 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.467 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:23.467 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.541884+0000", 2024-09-18T10:15:23.467 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:23.467 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.467 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.467 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:23.467 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.541890+0000", 2024-09-18T10:15:23.467 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.0589999999999999e-06 2024-09-18T10:15:23.467 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.467 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.467 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:23.467 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.541899+0000", 2024-09-18T10:15:23.467 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.02e-06 2024-09-18T10:15:23.467 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.468 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.468 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:23.468 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.541905+0000", 2024-09-18T10:15:23.468 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.9399999999999999e-06 2024-09-18T10:15:23.468 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.468 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.468 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:23.468 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.644561+0000", 2024-09-18T10:15:23.468 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.102655408 2024-09-18T10:15:23.468 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.468 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.468 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:23.468 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.644869+0000", 2024-09-18T10:15:23.469 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00030766700000000002 2024-09-18T10:15:23.469 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.469 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.469 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:23.469 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.645745+0000", 2024-09-18T10:15:23.469 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000876314 2024-09-18T10:15:23.469 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.469 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.469 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:23.469 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.647092+0000", 2024-09-18T10:15:23.470 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0013469269999999999 2024-09-18T10:15:23.470 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.470 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.470 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:15:23.470 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:20.029901+0000", 2024-09-18T10:15:23.470 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.38280895199999998 2024-09-18T10:15:23.470 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:23.470 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:23.470 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:23.470 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:20.030051+0000", 2024-09-18T10:15:23.470 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00015067699999999999 2024-09-18T10:15:23.470 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:23.470 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:23.471 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:23.471 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:23.471 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:23.471 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-09-18T10:15:23.473 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T10:15:23.473 INFO:tasks.ceph.osd.14.smithi154.stderr:2024-09-18T10:15:23.471+0000 7f75a19e7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 22093) UID: 0 2024-09-18T10:15:23.574 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-18T10:15:23.570+0000 7f14dcc5b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22398) UID: 0 2024-09-18T10:15:23.581 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-18T10:15:23.582 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-18T10:15:23.582 INFO:teuthology.orchestra.run.smithi189.stdout: "num_ops": 0 2024-09-18T10:15:23.582 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-18T10:15:23.591 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T10:15:23.675 INFO:tasks.ceph.osd.8.smithi029.stderr:2024-09-18T10:15:23.670+0000 7f2555c27700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 22192) UID: 0 2024-09-18T10:15:23.689 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-18T10:15:23.689 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-18T10:15:23.689 INFO:teuthology.orchestra.run.smithi189.stdout: "complaint_time": 30, 2024-09-18T10:15:23.689 INFO:teuthology.orchestra.run.smithi189.stdout: "num_blocked_ops": 0 2024-09-18T10:15:23.689 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-18T10:15:23.698 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T10:15:23.776 INFO:tasks.ceph.osd.14.smithi154.stderr:2024-09-18T10:15:23.775+0000 7f75a19e7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 22093) UID: 0 2024-09-18T10:15:23.797 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-18T10:15:23.798 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 20, 2024-09-18T10:15:23.798 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 600, 2024-09-18T10:15:23.798 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [ 2024-09-18T10:15:23.798 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.798 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 36/32 ECSubWriteReply(tid=5, last_complete=36'4, committed=1, applied=1))", 2024-09-18T10:15:23.798 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:50.303102+0000", 2024-09-18T10:15:23.798 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 33.491319404000002, 2024-09-18T10:15:23.798 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20304619600000001, 2024-09-18T10:15:23.798 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:23.798 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:23.798 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:23.798 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.799 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:23.799 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.303102+0000", 2024-09-18T10:15:23.799 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:23.799 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.799 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.799 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:23.799 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.303102+0000", 2024-09-18T10:15:23.799 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:23.799 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.799 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.799 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:23.799 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.303100+0000", 2024-09-18T10:15:23.799 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:23.800 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.800 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.800 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:23.800 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.303114+0000", 2024-09-18T10:15:23.800 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3866000000000001e-05 2024-09-18T10:15:23.800 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.800 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.800 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:23.800 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.303149+0000", 2024-09-18T10:15:23.800 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.5249000000000001e-05 2024-09-18T10:15:23.800 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.800 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.800 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:23.801 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.303174+0000", 2024-09-18T10:15:23.801 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4785000000000001e-05 2024-09-18T10:15:23.801 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.801 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.801 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:23.801 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.506129+0000", 2024-09-18T10:15:23.801 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20295469199999999 2024-09-18T10:15:23.802 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.802 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.802 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:23.802 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.506148+0000", 2024-09-18T10:15:23.802 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9720000000000001e-05 2024-09-18T10:15:23.802 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:23.802 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:23.802 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:23.802 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.802 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.802 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 36/32 ECSubWriteReply(tid=5, last_complete=36'4, committed=1, applied=1))", 2024-09-18T10:15:23.802 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:50.305623+0000", 2024-09-18T10:15:23.802 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 33.488798432000003, 2024-09-18T10:15:23.803 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40085840299999997, 2024-09-18T10:15:23.803 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:23.803 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:23.803 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:23.803 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.803 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:23.803 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.305623+0000", 2024-09-18T10:15:23.803 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:23.803 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.803 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.804 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:23.804 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.305623+0000", 2024-09-18T10:15:23.804 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:23.804 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.804 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.804 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:23.804 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.305621+0000", 2024-09-18T10:15:23.804 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999976 2024-09-18T10:15:23.804 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.804 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.804 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:23.804 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.305635+0000", 2024-09-18T10:15:23.804 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4283e-05 2024-09-18T10:15:23.805 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.805 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.805 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:23.805 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.305663+0000", 2024-09-18T10:15:23.805 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7475999999999999e-05 2024-09-18T10:15:23.805 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.805 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.805 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:23.805 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.305680+0000", 2024-09-18T10:15:23.805 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7609e-05 2024-09-18T10:15:23.805 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.805 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.805 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:23.806 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.706451+0000", 2024-09-18T10:15:23.806 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40077027399999998 2024-09-18T10:15:23.806 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.806 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.806 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:23.806 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.706481+0000", 2024-09-18T10:15:23.806 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0935000000000001e-05 2024-09-18T10:15:23.806 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:23.806 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:23.806 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:23.806 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.806 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.806 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 36/32 ECSubWriteReply(tid=5, last_complete=36'4, committed=1, applied=1))", 2024-09-18T10:15:23.807 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:50.305688+0000", 2024-09-18T10:15:23.807 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 33.488733068000002, 2024-09-18T10:15:23.807 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40093562900000002, 2024-09-18T10:15:23.807 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:23.807 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:23.807 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:23.807 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.807 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:23.808 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.305688+0000", 2024-09-18T10:15:23.808 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:23.808 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.808 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.808 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:23.808 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.305688+0000", 2024-09-18T10:15:23.808 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:23.808 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.808 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.808 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:23.808 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.305687+0000", 2024-09-18T10:15:23.808 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:23.809 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.809 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.809 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:23.809 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.305698+0000", 2024-09-18T10:15:23.809 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1763000000000001e-05 2024-09-18T10:15:23.810 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.810 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.810 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:23.810 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.305729+0000", 2024-09-18T10:15:23.810 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0502e-05 2024-09-18T10:15:23.810 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.810 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.810 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:23.811 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.305741+0000", 2024-09-18T10:15:23.811 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2363e-05 2024-09-18T10:15:23.811 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.811 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.811 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:23.811 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.706585+0000", 2024-09-18T10:15:23.811 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.400843265 2024-09-18T10:15:23.811 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.811 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.811 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:23.811 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.706624+0000", 2024-09-18T10:15:23.811 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9459000000000003e-05 2024-09-18T10:15:23.811 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:23.811 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:23.812 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:23.812 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.812 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.812 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 36/32 ECSubWriteReply(tid=4, last_complete=36'3, committed=1, applied=1))", 2024-09-18T10:15:23.812 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:50.404544+0000", 2024-09-18T10:15:23.812 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 33.389877552999998, 2024-09-18T10:15:23.812 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.30239659400000002, 2024-09-18T10:15:23.812 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:23.812 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:23.812 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:23.812 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.812 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:23.813 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.404544+0000", 2024-09-18T10:15:23.813 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:23.813 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.813 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.813 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:23.813 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.404544+0000", 2024-09-18T10:15:23.813 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:23.813 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.813 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.813 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:23.813 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.404542+0000", 2024-09-18T10:15:23.813 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:23.813 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.813 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.814 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:23.814 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.404555+0000", 2024-09-18T10:15:23.814 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3096e-05 2024-09-18T10:15:23.814 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.814 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.814 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:23.814 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.404576+0000", 2024-09-18T10:15:23.814 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1165000000000001e-05 2024-09-18T10:15:23.814 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.814 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.814 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:23.814 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.404587+0000", 2024-09-18T10:15:23.814 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0549e-05 2024-09-18T10:15:23.815 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.815 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.815 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:23.815 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.706657+0000", 2024-09-18T10:15:23.815 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.30206971999999999 2024-09-18T10:15:23.815 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.815 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.815 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:23.815 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.706941+0000", 2024-09-18T10:15:23.815 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00028382100000000002 2024-09-18T10:15:23.815 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:23.815 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:23.815 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:23.815 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.816 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.816 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 36/32 ECSubWriteReply(tid=5, last_complete=36'4, committed=1, applied=1))", 2024-09-18T10:15:23.816 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:50.406036+0000", 2024-09-18T10:15:23.816 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 33.388385335999999, 2024-09-18T10:15:23.816 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.30215917399999997, 2024-09-18T10:15:23.816 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:23.816 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:23.816 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:23.816 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.816 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:23.816 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.406036+0000", 2024-09-18T10:15:23.816 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:23.817 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.817 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.817 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:23.817 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.406036+0000", 2024-09-18T10:15:23.817 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:23.817 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.817 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.817 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:23.817 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.406035+0000", 2024-09-18T10:15:23.817 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:23.817 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.817 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.817 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:23.818 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.406046+0000", 2024-09-18T10:15:23.818 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1704e-05 2024-09-18T10:15:23.818 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.818 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.818 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:23.818 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.406065+0000", 2024-09-18T10:15:23.818 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.8448999999999999e-05 2024-09-18T10:15:23.818 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.818 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.818 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:23.818 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.406076+0000", 2024-09-18T10:15:23.818 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1286e-05 2024-09-18T10:15:23.818 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.818 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.819 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:23.819 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.707005+0000", 2024-09-18T10:15:23.819 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.30092866899999998 2024-09-18T10:15:23.819 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.819 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.819 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:23.819 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.708195+0000", 2024-09-18T10:15:23.819 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001190546 2024-09-18T10:15:23.819 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:23.819 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:23.819 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:23.819 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.819 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.820 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.cs3 36/32 ECSubWrite(tid=14, reqid=client.4632.0:92, at_version=0'0, trim_to=0'0, roll_forward_to=36'6))", 2024-09-18T10:15:23.820 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:50.504842+0000", 2024-09-18T10:15:23.820 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 33.289579934000002, 2024-09-18T10:15:23.820 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.202186116, 2024-09-18T10:15:23.820 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:23.820 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:23.820 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:23.820 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.820 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:23.820 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.504842+0000", 2024-09-18T10:15:23.820 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:23.820 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.820 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.821 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:23.821 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.504842+0000", 2024-09-18T10:15:23.821 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:23.821 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.821 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.821 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:23.821 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.504840+0000", 2024-09-18T10:15:23.821 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:23.821 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.821 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.821 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:23.821 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.504868+0000", 2024-09-18T10:15:23.821 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7809000000000001e-05 2024-09-18T10:15:23.822 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.822 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.822 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:23.822 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.504900+0000", 2024-09-18T10:15:23.822 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1921999999999997e-05 2024-09-18T10:15:23.822 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.822 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.822 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:23.822 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.605012+0000", 2024-09-18T10:15:23.822 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10011272 2024-09-18T10:15:23.822 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.822 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.822 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:23.822 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.605115+0000", 2024-09-18T10:15:23.823 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000103003 2024-09-18T10:15:23.823 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.823 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.823 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:23.823 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.605132+0000", 2024-09-18T10:15:23.823 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6359999999999999e-05 2024-09-18T10:15:23.823 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.823 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.823 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:23.823 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.706978+0000", 2024-09-18T10:15:23.823 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.101846672 2024-09-18T10:15:23.823 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.823 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.824 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:23.824 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.707028+0000", 2024-09-18T10:15:23.824 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9443000000000001e-05 2024-09-18T10:15:23.824 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:23.824 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:23.824 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:23.824 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.824 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.824 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:179 3.0s0 3:02f544f0:::smithi18925896-30:head [read 0~491520,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:23.824 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:52.680315+0000", 2024-09-18T10:15:23.824 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 31.114106804999999, 2024-09-18T10:15:23.824 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20435319499999999, 2024-09-18T10:15:23.825 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:23.825 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:15:23.825 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:23.825 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:23.825 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:23.825 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 179 2024-09-18T10:15:23.825 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.825 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:23.825 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.825 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:23.825 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.680315+0000", 2024-09-18T10:15:23.825 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:23.825 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.826 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.826 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:23.826 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.680315+0000", 2024-09-18T10:15:23.826 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:23.826 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.826 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.826 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:23.826 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.680313+0000", 2024-09-18T10:15:23.826 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:23.826 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.826 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.826 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:23.826 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.680322+0000", 2024-09-18T10:15:23.826 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.9299999999999992e-06 2024-09-18T10:15:23.827 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.827 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.827 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:23.827 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.680343+0000", 2024-09-18T10:15:23.827 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0556000000000001e-05 2024-09-18T10:15:23.827 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.827 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.827 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:23.827 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.680373+0000", 2024-09-18T10:15:23.827 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0592000000000001e-05 2024-09-18T10:15:23.827 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.827 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.827 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:23.828 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.681360+0000", 2024-09-18T10:15:23.828 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00098610400000000002 2024-09-18T10:15:23.828 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.828 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.828 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:23.828 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.681624+0000", 2024-09-18T10:15:23.828 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000263992 2024-09-18T10:15:23.828 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.828 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.828 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:23.828 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.884668+0000", 2024-09-18T10:15:23.828 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.203044321 2024-09-18T10:15:23.828 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:23.829 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:23.829 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:23.829 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.829 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.829 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:180 3.0s0 3:02f544f0:::smithi18925896-30:head [read 0~491520,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:23.829 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:52.680394+0000", 2024-09-18T10:15:23.829 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 31.114027727, 2024-09-18T10:15:23.829 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20540486599999999, 2024-09-18T10:15:23.829 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:23.829 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:15:23.829 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:23.829 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:23.829 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:23.829 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 180 2024-09-18T10:15:23.830 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.830 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:23.830 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.830 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:23.830 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.680394+0000", 2024-09-18T10:15:23.830 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:23.830 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.830 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.830 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:23.830 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.680394+0000", 2024-09-18T10:15:23.830 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:23.830 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.830 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.830 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:23.831 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.680393+0000", 2024-09-18T10:15:23.831 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:23.831 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.831 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.831 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:23.831 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.680401+0000", 2024-09-18T10:15:23.831 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.4169999999999992e-06 2024-09-18T10:15:23.831 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.831 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.831 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:23.831 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.680422+0000", 2024-09-18T10:15:23.831 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1186e-05 2024-09-18T10:15:23.831 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.831 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.832 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:23.832 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.680433+0000", 2024-09-18T10:15:23.832 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1103e-05 2024-09-18T10:15:23.832 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.832 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.832 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:23.832 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.783888+0000", 2024-09-18T10:15:23.832 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.103454746 2024-09-18T10:15:23.832 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.832 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.832 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:23.832 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.784067+0000", 2024-09-18T10:15:23.832 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00017865399999999999 2024-09-18T10:15:23.833 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.833 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.833 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:23.833 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.885799+0000", 2024-09-18T10:15:23.833 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10173201699999999 2024-09-18T10:15:23.833 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:23.833 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:23.833 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:23.833 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.833 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.833 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpReadReply(3.0s0 36/32 ECSubReadReply(tid=9, attrs_read=0))", 2024-09-18T10:15:23.833 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:52.683258+0000", 2024-09-18T10:15:23.833 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 31.111163934, 2024-09-18T10:15:23.834 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.201447083, 2024-09-18T10:15:23.834 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:23.834 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:23.834 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:23.834 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.834 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:23.834 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.683258+0000", 2024-09-18T10:15:23.834 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:23.834 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.834 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.834 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:23.834 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.683258+0000", 2024-09-18T10:15:23.834 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:23.834 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.835 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.835 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:23.835 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.683256+0000", 2024-09-18T10:15:23.835 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:23.835 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.835 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.835 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:23.835 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.683482+0000", 2024-09-18T10:15:23.835 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00022592599999999999 2024-09-18T10:15:23.835 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.835 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.835 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:23.835 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.683508+0000", 2024-09-18T10:15:23.836 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5732999999999999e-05 2024-09-18T10:15:23.836 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.836 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.836 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:23.836 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.783587+0000", 2024-09-18T10:15:23.836 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100079109 2024-09-18T10:15:23.836 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.836 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.836 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:23.836 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.883944+0000", 2024-09-18T10:15:23.836 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100357268 2024-09-18T10:15:23.836 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.837 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.837 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:23.837 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.884705+0000", 2024-09-18T10:15:23.837 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00076026999999999996 2024-09-18T10:15:23.837 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:23.837 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:23.837 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:23.837 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.837 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.837 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:191 3.0s0 3:0a83d6b3:::smithi18925896-20:2 [read 0~327680,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:23.837 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:52.694194+0000", 2024-09-18T10:15:23.837 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 31.100227286999999, 2024-09-18T10:15:23.837 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.29753027399999998, 2024-09-18T10:15:23.837 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:23.838 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:15:23.838 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:23.838 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:23.838 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:23.838 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 191 2024-09-18T10:15:23.838 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.838 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:23.838 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.838 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:23.838 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.694194+0000", 2024-09-18T10:15:23.838 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:23.838 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.838 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.839 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:23.839 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.694194+0000", 2024-09-18T10:15:23.839 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:23.839 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.839 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.839 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:23.839 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.694192+0000", 2024-09-18T10:15:23.839 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:23.839 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.839 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.839 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:23.839 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.694204+0000", 2024-09-18T10:15:23.839 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2233e-05 2024-09-18T10:15:23.839 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.840 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.840 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:23.840 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.694280+0000", 2024-09-18T10:15:23.840 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.5584e-05 2024-09-18T10:15:23.840 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.840 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.840 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:23.840 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.694292+0000", 2024-09-18T10:15:23.840 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2478e-05 2024-09-18T10:15:23.840 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.840 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.840 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:23.840 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.885869+0000", 2024-09-18T10:15:23.841 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.191576304 2024-09-18T10:15:23.841 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.841 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.841 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:23.841 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.886008+0000", 2024-09-18T10:15:23.841 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00013880300000000001 2024-09-18T10:15:23.841 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.841 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.841 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:23.841 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.991725+0000", 2024-09-18T10:15:23.841 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10571691 2024-09-18T10:15:23.841 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:23.841 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:23.842 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:23.842 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.842 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.842 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:192 3.0s0 3:0a83d6b3:::smithi18925896-20:2 [read 0~327680,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:23.842 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:52.694306+0000", 2024-09-18T10:15:23.842 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 31.100115336999998, 2024-09-18T10:15:23.842 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.41051474799999998, 2024-09-18T10:15:23.842 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:23.842 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:15:23.842 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:23.842 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:23.842 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:23.842 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 192 2024-09-18T10:15:23.842 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.843 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:23.843 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.843 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:23.843 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.694306+0000", 2024-09-18T10:15:23.843 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:23.843 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.843 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.843 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:23.843 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.694306+0000", 2024-09-18T10:15:23.843 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:23.843 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.843 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.843 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:23.843 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.694305+0000", 2024-09-18T10:15:23.844 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:23.844 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.844 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.844 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:23.844 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.694312+0000", 2024-09-18T10:15:23.844 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.5129999999999999e-06 2024-09-18T10:15:23.844 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.844 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.844 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:23.844 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.694328+0000", 2024-09-18T10:15:23.844 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5766000000000001e-05 2024-09-18T10:15:23.844 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.844 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.844 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:23.845 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.694349+0000", 2024-09-18T10:15:23.845 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0990000000000001e-05 2024-09-18T10:15:23.845 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.845 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.845 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:23.845 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.887405+0000", 2024-09-18T10:15:23.845 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.193055791 2024-09-18T10:15:23.845 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.845 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.845 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:23.845 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.887535+0000", 2024-09-18T10:15:23.845 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00013043000000000001 2024-09-18T10:15:23.845 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.846 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.846 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:23.846 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:53.104821+0000", 2024-09-18T10:15:23.846 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.217285486 2024-09-18T10:15:23.846 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:23.846 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:23.846 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:23.846 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.846 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.846 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:193 3.0s0 3:0a83d6b3:::smithi18925896-20:2 [read 0~327680,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:23.846 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:52.694359+0000", 2024-09-18T10:15:23.846 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 31.100062156, 2024-09-18T10:15:23.846 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.411282918, 2024-09-18T10:15:23.846 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:23.847 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:15:23.847 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:23.847 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:23.847 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:23.847 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 193 2024-09-18T10:15:23.847 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.847 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:23.847 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.847 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:23.847 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.694359+0000", 2024-09-18T10:15:23.847 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:23.847 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.847 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.848 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:23.848 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.694359+0000", 2024-09-18T10:15:23.848 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:23.848 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.848 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.848 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:23.848 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.694358+0000", 2024-09-18T10:15:23.848 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:15:23.848 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.848 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.848 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:23.848 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.694364+0000", 2024-09-18T10:15:23.848 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8089999999999998e-06 2024-09-18T10:15:23.849 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.849 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.849 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:23.849 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.694381+0000", 2024-09-18T10:15:23.849 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6949e-05 2024-09-18T10:15:23.849 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.849 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.849 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:23.849 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.694388+0000", 2024-09-18T10:15:23.849 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.8079999999999999e-06 2024-09-18T10:15:23.850 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.850 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.850 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:23.850 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.988184+0000", 2024-09-18T10:15:23.850 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.29379642900000003 2024-09-18T10:15:23.850 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.850 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.850 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:23.850 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.988399+0000", 2024-09-18T10:15:23.850 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000214537 2024-09-18T10:15:23.850 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.850 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.850 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:23.851 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:53.105642+0000", 2024-09-18T10:15:23.851 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.11724332699999999 2024-09-18T10:15:23.851 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:23.851 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:23.851 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:23.851 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.851 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.851 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.10s0 45/40 ECSubWriteReply(tid=18, last_complete=45'6, committed=1, applied=1))", 2024-09-18T10:15:23.851 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:01.742819+0000", 2024-09-18T10:15:23.851 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 22.051602552999999, 2024-09-18T10:15:23.851 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.30317059800000001, 2024-09-18T10:15:23.851 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:23.851 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:23.851 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:23.852 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.852 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:23.852 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.742819+0000", 2024-09-18T10:15:23.852 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:23.852 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.852 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.852 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:23.852 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.742819+0000", 2024-09-18T10:15:23.852 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:23.852 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.852 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.852 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:23.852 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.742817+0000", 2024-09-18T10:15:23.852 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:23.853 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.853 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.853 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:23.853 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.742832+0000", 2024-09-18T10:15:23.853 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.466e-05 2024-09-18T10:15:23.853 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.853 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.853 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:23.853 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.742866+0000", 2024-09-18T10:15:23.853 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4242000000000002e-05 2024-09-18T10:15:23.853 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.853 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.853 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:23.854 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.843033+0000", 2024-09-18T10:15:23.854 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100166479 2024-09-18T10:15:23.854 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.854 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.854 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:23.854 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.045959+0000", 2024-09-18T10:15:23.854 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.202926 2024-09-18T10:15:23.854 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.854 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.854 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:23.854 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.045990+0000", 2024-09-18T10:15:23.854 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0889999999999997e-05 2024-09-18T10:15:23.854 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:23.854 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:23.855 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:23.855 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.855 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.855 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.10s0 45/40 ECSubWriteReply(tid=18, last_complete=45'6, committed=1, applied=1))", 2024-09-18T10:15:23.855 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:01.843113+0000", 2024-09-18T10:15:23.855 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 21.951308706999999, 2024-09-18T10:15:23.855 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.203080129, 2024-09-18T10:15:23.855 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:23.855 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:23.855 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:23.855 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.855 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:23.855 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.843113+0000", 2024-09-18T10:15:23.855 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:23.856 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.856 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.856 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:23.856 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.843113+0000", 2024-09-18T10:15:23.856 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:23.856 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.856 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.856 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:23.856 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.843111+0000", 2024-09-18T10:15:23.856 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:23.856 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.856 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.856 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:23.857 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.843122+0000", 2024-09-18T10:15:23.857 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0199e-05 2024-09-18T10:15:23.857 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.857 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.857 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:23.857 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.843147+0000", 2024-09-18T10:15:23.857 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5113000000000002e-05 2024-09-18T10:15:23.857 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.857 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.857 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:23.857 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.943289+0000", 2024-09-18T10:15:23.857 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100141813 2024-09-18T10:15:23.857 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.857 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.858 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:23.858 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.046146+0000", 2024-09-18T10:15:23.858 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.102857768 2024-09-18T10:15:23.858 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.858 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.858 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:23.858 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.046193+0000", 2024-09-18T10:15:23.858 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.6566000000000001e-05 2024-09-18T10:15:23.858 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:23.858 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:23.858 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:23.858 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.858 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.858 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.19s6 51/40 ECSubWrite(tid=78, reqid=client.4632.0:408, at_version=51'46, trim_to=0'0, roll_forward_to=50'39))", 2024-09-18T10:15:23.859 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:06.779231+0000", 2024-09-18T10:15:23.859 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 17.015190817000001, 2024-09-18T10:15:23.859 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.201476816, 2024-09-18T10:15:23.859 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:23.859 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:23.859 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:23.859 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.859 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:23.859 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:06.779231+0000", 2024-09-18T10:15:23.859 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:23.859 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.859 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.859 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:23.859 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:06.779231+0000", 2024-09-18T10:15:23.860 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:23.860 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.860 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.860 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:23.860 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:06.779229+0000", 2024-09-18T10:15:23.860 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:23.860 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.860 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.860 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:23.860 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:06.779264+0000", 2024-09-18T10:15:23.860 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4699000000000001e-05 2024-09-18T10:15:23.860 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.861 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.861 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:23.861 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:06.779288+0000", 2024-09-18T10:15:23.861 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3821000000000001e-05 2024-09-18T10:15:23.861 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.861 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.861 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:23.861 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:06.879428+0000", 2024-09-18T10:15:23.861 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10014011 2024-09-18T10:15:23.861 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.861 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.861 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:23.861 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:06.979654+0000", 2024-09-18T10:15:23.861 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100225544 2024-09-18T10:15:23.862 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.862 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.862 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:23.862 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:06.979671+0000", 2024-09-18T10:15:23.862 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7244999999999998e-05 2024-09-18T10:15:23.862 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.862 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.862 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:23.862 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:06.980659+0000", 2024-09-18T10:15:23.862 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00098774499999999994 2024-09-18T10:15:23.862 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.862 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.862 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:23.862 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:06.980708+0000", 2024-09-18T10:15:23.863 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.8897999999999998e-05 2024-09-18T10:15:23.863 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:23.863 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:23.863 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:23.863 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.863 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.863 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.1s5 56/40 ECSubWrite(tid=102, reqid=client.4632.0:437, at_version=0'0, trim_to=0'0, roll_forward_to=56'16))", 2024-09-18T10:15:23.863 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:12.541097+0000", 2024-09-18T10:15:23.863 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 11.253324344999999, 2024-09-18T10:15:23.863 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20186336399999999, 2024-09-18T10:15:23.863 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:23.863 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:23.863 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:23.863 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.864 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:23.864 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:12.541097+0000", 2024-09-18T10:15:23.864 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:23.864 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.864 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.864 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:23.864 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:12.541097+0000", 2024-09-18T10:15:23.864 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:23.864 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.864 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.864 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:23.864 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:12.541095+0000", 2024-09-18T10:15:23.864 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:23.865 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.865 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.865 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:23.865 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:12.541147+0000", 2024-09-18T10:15:23.865 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.2203000000000001e-05 2024-09-18T10:15:23.865 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.865 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.865 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:23.865 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:12.541184+0000", 2024-09-18T10:15:23.866 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6495000000000002e-05 2024-09-18T10:15:23.866 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.866 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.866 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:23.866 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:12.641359+0000", 2024-09-18T10:15:23.866 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100175135 2024-09-18T10:15:23.866 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.866 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.866 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:23.866 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:12.741675+0000", 2024-09-18T10:15:23.866 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100316457 2024-09-18T10:15:23.867 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.867 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.867 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:23.867 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:12.741693+0000", 2024-09-18T10:15:23.867 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7269e-05 2024-09-18T10:15:23.867 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.867 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.867 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:23.868 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:12.742894+0000", 2024-09-18T10:15:23.868 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0012013639999999999 2024-09-18T10:15:23.868 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.868 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.868 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:23.868 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:12.742961+0000", 2024-09-18T10:15:23.868 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.6575000000000005e-05 2024-09-18T10:15:23.868 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:23.868 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:23.868 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:23.868 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.868 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.868 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.1s5 57/40 ECSubWrite(tid=111, reqid=osd.4.0:111, at_version=57'18, trim_to=0'0, roll_forward_to=56'16))", 2024-09-18T10:15:23.869 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:13.054749+0000", 2024-09-18T10:15:23.869 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 10.739672283999999, 2024-09-18T10:15:23.869 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.201940535, 2024-09-18T10:15:23.869 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:23.869 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:23.869 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:23.869 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.869 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:23.869 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.054749+0000", 2024-09-18T10:15:23.869 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:23.869 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.869 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.870 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:23.870 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.054749+0000", 2024-09-18T10:15:23.870 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:23.870 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.870 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.870 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:23.870 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.054749+0000", 2024-09-18T10:15:23.870 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:23.870 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.871 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.871 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:23.871 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.054785+0000", 2024-09-18T10:15:23.871 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6480999999999999e-05 2024-09-18T10:15:23.871 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.871 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.871 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:23.871 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.054808+0000", 2024-09-18T10:15:23.871 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.2351e-05 2024-09-18T10:15:23.871 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.871 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.871 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:23.872 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.154914+0000", 2024-09-18T10:15:23.872 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100106862 2024-09-18T10:15:23.872 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.872 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.872 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:23.872 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.155065+0000", 2024-09-18T10:15:23.872 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000150762 2024-09-18T10:15:23.872 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.872 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.872 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:23.872 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.155083+0000", 2024-09-18T10:15:23.872 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.8207e-05 2024-09-18T10:15:23.873 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.873 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.873 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:23.873 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.256244+0000", 2024-09-18T10:15:23.873 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10116046200000001 2024-09-18T10:15:23.873 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.873 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.873 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:23.873 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.256690+0000", 2024-09-18T10:15:23.873 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00044597000000000001 2024-09-18T10:15:23.873 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:23.873 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:23.873 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:23.873 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.874 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.874 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 58/40 ECSubWriteReply(tid=21, last_complete=57'8, committed=1, applied=1))", 2024-09-18T10:15:23.874 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:13.649729+0000", 2024-09-18T10:15:23.874 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 10.144692677, 2024-09-18T10:15:23.874 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20172436599999999, 2024-09-18T10:15:23.874 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:23.874 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:23.874 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:23.874 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.874 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:23.874 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.649729+0000", 2024-09-18T10:15:23.874 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:23.874 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.875 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.875 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:23.875 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.649729+0000", 2024-09-18T10:15:23.875 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:23.875 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.875 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.875 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:23.875 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.649728+0000", 2024-09-18T10:15:23.875 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:23.875 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.875 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.875 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:23.875 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.649736+0000", 2024-09-18T10:15:23.876 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.0530000000000007e-06 2024-09-18T10:15:23.876 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.876 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.876 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:23.876 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.649748+0000", 2024-09-18T10:15:23.876 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1173e-05 2024-09-18T10:15:23.876 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.876 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.876 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:23.876 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.749876+0000", 2024-09-18T10:15:23.876 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10012805 2024-09-18T10:15:23.876 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.876 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.877 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:23.877 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.850098+0000", 2024-09-18T10:15:23.877 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100222371 2024-09-18T10:15:23.877 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.877 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.877 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:23.877 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.851453+0000", 2024-09-18T10:15:23.877 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0013552219999999999 2024-09-18T10:15:23.877 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:23.877 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:23.877 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:23.877 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.877 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.877 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:511 3.0s0 3:02f544f0:::smithi18925896-30:head [stat,delete] snapc 1a=[1a,19,18,16,12,11,10,f,e] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-18T10:15:23.878 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:18.899317+0000", 2024-09-18T10:15:23.878 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 4.8951049070000003, 2024-09-18T10:15:23.878 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.21051439599999999, 2024-09-18T10:15:23.878 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:23.878 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:23.878 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:23.878 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:23.878 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:23.878 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 511 2024-09-18T10:15:23.878 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.878 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:23.878 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.878 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:23.879 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.899317+0000", 2024-09-18T10:15:23.879 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:23.879 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.879 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.879 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:23.879 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.899317+0000", 2024-09-18T10:15:23.879 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:23.879 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.879 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.879 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:23.879 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.899314+0000", 2024-09-18T10:15:23.879 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999971 2024-09-18T10:15:23.879 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.880 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.880 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:23.880 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.899329+0000", 2024-09-18T10:15:23.880 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5160000000000001e-05 2024-09-18T10:15:23.880 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.880 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.880 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:23.880 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.899364+0000", 2024-09-18T10:15:23.880 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.5456000000000001e-05 2024-09-18T10:15:23.880 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.880 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.880 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:23.880 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.899385+0000", 2024-09-18T10:15:23.881 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0279e-05 2024-09-18T10:15:23.881 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.881 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.881 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:23.881 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.899497+0000", 2024-09-18T10:15:23.881 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000111773 2024-09-18T10:15:23.881 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.881 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.881 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "op must wait for map", 2024-09-18T10:15:23.881 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.899508+0000", 2024-09-18T10:15:23.881 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1177e-05 2024-09-18T10:15:23.884 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.884 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.884 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:23.884 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.903698+0000", 2024-09-18T10:15:23.885 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0041902850000000002 2024-09-18T10:15:23.885 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.885 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.885 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:23.885 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.903878+0000", 2024-09-18T10:15:23.885 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00018009499999999999 2024-09-18T10:15:23.885 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.885 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.885 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:23.885 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.905286+0000", 2024-09-18T10:15:23.885 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0014077720000000001 2024-09-18T10:15:23.885 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.885 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.886 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:23.886 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.003449+0000", 2024-09-18T10:15:23.886 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.098163081999999999 2024-09-18T10:15:23.886 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.886 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.886 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:23.886 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.109709+0000", 2024-09-18T10:15:23.886 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10625963400000001 2024-09-18T10:15:23.886 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.886 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.886 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:23.886 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.109831+0000", 2024-09-18T10:15:23.886 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00012246699999999999 2024-09-18T10:15:23.886 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:23.887 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:23.887 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:23.887 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.887 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.887 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.cs3 66/32 ECSubWrite(tid=106, reqid=osd.5.0:105, at_version=0'0, trim_to=0'0, roll_forward_to=66'83))", 2024-09-18T10:15:23.887 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:22.845036+0000", 2024-09-18T10:15:23.887 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.94938511000000003, 2024-09-18T10:15:23.887 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20128633700000001, 2024-09-18T10:15:23.887 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:23.887 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:23.887 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:23.887 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.887 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:23.887 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:22.845036+0000", 2024-09-18T10:15:23.887 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:23.888 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.888 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.888 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:23.888 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:22.845036+0000", 2024-09-18T10:15:23.888 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:23.888 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.888 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.888 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:23.888 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:22.845035+0000", 2024-09-18T10:15:23.888 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:23.888 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.888 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.888 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:23.889 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:22.845061+0000", 2024-09-18T10:15:23.889 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.6122000000000002e-05 2024-09-18T10:15:23.889 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.889 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.889 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:23.889 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:22.845101+0000", 2024-09-18T10:15:23.889 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9888999999999997e-05 2024-09-18T10:15:23.889 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.889 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.889 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:23.889 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:22.845115+0000", 2024-09-18T10:15:23.889 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4654000000000001e-05 2024-09-18T10:15:23.890 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.890 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.890 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:23.890 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:22.945327+0000", 2024-09-18T10:15:23.890 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100211944 2024-09-18T10:15:23.890 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.890 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.890 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:23.890 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:22.945344+0000", 2024-09-18T10:15:23.890 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7025999999999999e-05 2024-09-18T10:15:23.890 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.890 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.890 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:23.890 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.046181+0000", 2024-09-18T10:15:23.891 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100837125 2024-09-18T10:15:23.891 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:23.891 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:23.891 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:23.891 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.046323+0000", 2024-09-18T10:15:23.891 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000141364 2024-09-18T10:15:23.891 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:23.891 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:23.891 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:23.891 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:23.891 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:23.891 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-18T10:15:23.893 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T10:15:23.893 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-18T10:15:23.886+0000 7f302f2c3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22187) UID: 0 2024-09-18T10:15:23.937 INFO:tasks.rados.rados.0.smithi189.stdout:223: done (1 left) 2024-09-18T10:15:23.937 INFO:tasks.rados.rados.0.smithi189.stdout:227: done (0 left) 2024-09-18T10:15:23.937 INFO:tasks.rados.rados.0.smithi189.stdout:append oid 17 current snap is 19 2024-09-18T10:15:23.938 INFO:tasks.rados.rados.0.smithi189.stdout:228: seq_num 101 ranges {212992=540672,753664=507904,1261568=622592,1884160=32768} 2024-09-18T10:15:23.942 INFO:teuthology.orchestra.run.smithi029.stderr:reweighted osd.14 to 1 (10000) 2024-09-18T10:15:23.949 INFO:tasks.rados.rados.0.smithi189.stdout:228: writing smithi18925896-17 from 212992 to 753664 tid 1 2024-09-18T10:15:23.952 INFO:tasks.rados.rados.0.smithi189.stdout:228: writing smithi18925896-17 from 753664 to 1261568 tid 2 2024-09-18T10:15:23.955 INFO:tasks.rados.rados.0.smithi189.stdout:228: writing smithi18925896-17 from 1261568 to 1884160 tid 3 2024-09-18T10:15:23.956 INFO:tasks.rados.rados.0.smithi189.stdout:228: writing smithi18925896-17 from 1884160 to 1916928 tid 4 2024-09-18T10:15:23.956 INFO:tasks.rados.rados.0.smithi189.stdout:229: rollback oid 8 current snap is 19 2024-09-18T10:15:23.956 INFO:tasks.rados.rados.0.smithi189.stdout:rollback oid 8 to 9 2024-09-18T10:15:23.956 INFO:tasks.rados.rados.0.smithi189.stdout:230: rmattr oid 26 current snap is 19 2024-09-18T10:15:23.956 INFO:tasks.rados.rados.0.smithi189.stdout:230: done (2 left) 2024-09-18T10:15:23.956 INFO:tasks.rados.rados.0.smithi189.stdout:231: delete oid 2 current snap is 19 2024-09-18T10:15:23.957 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 15 1 2024-09-18T10:15:24.002 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-18T10:15:23.998+0000 7f302f2c3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22187) UID: 0 2024-09-18T10:15:24.003 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-18T10:15:24.004 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [ 2024-09-18T10:15:24.004 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.004 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:559 3.19s0 3.d35743f9 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-18T10:15:24.004 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:23.949274+0000", 2024-09-18T10:15:24.004 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.052867822000000002, 2024-09-18T10:15:24.004 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.052911097999999997, 2024-09-18T10:15:24.004 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:24.004 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "op must wait for map", 2024-09-18T10:15:24.004 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:24.004 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:24.004 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:24.005 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 559 2024-09-18T10:15:24.005 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.005 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:24.005 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.005 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:24.005 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.949274+0000", 2024-09-18T10:15:24.005 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:24.005 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.005 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.005 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:24.005 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.949274+0000", 2024-09-18T10:15:24.005 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:24.005 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.006 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.006 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:24.006 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.949271+0000", 2024-09-18T10:15:24.006 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999967 2024-09-18T10:15:24.006 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.006 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.006 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:24.006 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.949853+0000", 2024-09-18T10:15:24.006 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00058219299999999995 2024-09-18T10:15:24.006 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.006 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.006 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:24.006 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.949900+0000", 2024-09-18T10:15:24.007 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.6158000000000001e-05 2024-09-18T10:15:24.007 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.007 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.007 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:24.007 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.949916+0000", 2024-09-18T10:15:24.007 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6603e-05 2024-09-18T10:15:24.007 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.007 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.007 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:24.007 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.950009+0000", 2024-09-18T10:15:24.007 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.2582000000000002e-05 2024-09-18T10:15:24.007 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.007 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.008 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "op must wait for map", 2024-09-18T10:15:24.008 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.950020+0000", 2024-09-18T10:15:24.008 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1576000000000001e-05 2024-09-18T10:15:24.008 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:24.008 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:24.008 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:24.008 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.008 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.008 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:560 3.19s0 3.d35743f9 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-18T10:15:24.008 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:23.952150+0000", 2024-09-18T10:15:24.008 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.049992228999999999, 2024-09-18T10:15:24.008 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.050152645000000003, 2024-09-18T10:15:24.008 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:24.008 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting_for_map not empty", 2024-09-18T10:15:24.009 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:24.009 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:24.009 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:24.009 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 560 2024-09-18T10:15:24.009 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.009 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:24.009 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.009 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:24.009 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.952150+0000", 2024-09-18T10:15:24.009 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:24.009 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.010 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.010 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:24.010 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.952150+0000", 2024-09-18T10:15:24.010 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:24.010 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.010 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.010 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:24.010 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.952148+0000", 2024-09-18T10:15:24.010 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:24.010 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.010 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.010 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:24.010 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.952647+0000", 2024-09-18T10:15:24.011 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00049869699999999999 2024-09-18T10:15:24.011 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.011 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.011 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:24.011 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.952672+0000", 2024-09-18T10:15:24.011 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5301999999999999e-05 2024-09-18T10:15:24.011 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.011 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.011 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:24.011 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.952685+0000", 2024-09-18T10:15:24.011 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2719e-05 2024-09-18T10:15:24.011 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.011 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.011 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:24.012 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.952773+0000", 2024-09-18T10:15:24.012 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.8608000000000003e-05 2024-09-18T10:15:24.012 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.012 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.012 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:24.012 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.952783+0000", 2024-09-18T10:15:24.012 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.7729999999999994e-06 2024-09-18T10:15:24.012 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:24.012 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:24.012 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:24.012 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.012 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.013 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:561 3.19s0 3.d35743f9 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-18T10:15:24.013 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:23.955639+0000", 2024-09-18T10:15:24.013 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.046503605000000003, 2024-09-18T10:15:24.013 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.046765068, 2024-09-18T10:15:24.013 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:24.013 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting_for_map not empty", 2024-09-18T10:15:24.013 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:24.013 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:24.013 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:24.013 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 561 2024-09-18T10:15:24.013 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.013 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:24.013 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.013 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:24.014 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.955639+0000", 2024-09-18T10:15:24.014 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:24.014 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.014 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.014 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:24.014 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.955639+0000", 2024-09-18T10:15:24.014 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:24.014 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.014 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.014 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:24.014 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.955637+0000", 2024-09-18T10:15:24.014 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:24.014 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.014 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.015 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:24.015 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956211+0000", 2024-09-18T10:15:24.015 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00057413099999999997 2024-09-18T10:15:24.015 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.015 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.015 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:24.015 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956241+0000", 2024-09-18T10:15:24.015 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0244999999999999e-05 2024-09-18T10:15:24.015 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.015 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.015 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:24.015 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956253+0000", 2024-09-18T10:15:24.015 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1824000000000001e-05 2024-09-18T10:15:24.016 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.016 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.016 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:24.016 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956338+0000", 2024-09-18T10:15:24.016 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.5544999999999995e-05 2024-09-18T10:15:24.016 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.016 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.016 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:24.016 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956349+0000", 2024-09-18T10:15:24.016 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0203e-05 2024-09-18T10:15:24.016 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:24.016 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:24.016 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:24.016 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.017 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.017 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:562 3.19s0 3.d35743f9 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-18T10:15:24.017 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:23.956275+0000", 2024-09-18T10:15:24.017 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.045867617999999999, 2024-09-18T10:15:24.017 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.046226889, 2024-09-18T10:15:24.017 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:24.017 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting_for_map not empty", 2024-09-18T10:15:24.017 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:24.017 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:24.017 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:24.017 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 562 2024-09-18T10:15:24.017 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.017 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:24.017 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.018 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:24.018 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956275+0000", 2024-09-18T10:15:24.018 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:24.018 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.018 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.018 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:24.018 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956275+0000", 2024-09-18T10:15:24.018 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:24.018 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.018 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.018 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:24.018 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956273+0000", 2024-09-18T10:15:24.018 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:24.018 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.019 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.019 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:24.019 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956355+0000", 2024-09-18T10:15:24.019 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.1661000000000003e-05 2024-09-18T10:15:24.019 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.019 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.019 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:24.019 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956366+0000", 2024-09-18T10:15:24.019 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1192000000000001e-05 2024-09-18T10:15:24.019 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.019 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.019 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:24.019 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956373+0000", 2024-09-18T10:15:24.020 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.5599999999999999e-06 2024-09-18T10:15:24.020 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.020 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.020 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:24.020 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956434+0000", 2024-09-18T10:15:24.020 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.0761999999999999e-05 2024-09-18T10:15:24.020 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.020 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.020 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:24.020 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956441+0000", 2024-09-18T10:15:24.020 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.1430000000000001e-06 2024-09-18T10:15:24.021 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:24.021 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:24.021 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:24.021 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.021 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.021 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:563 3.19s0 3.d35743f9 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-18T10:15:24.021 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:23.956389+0000", 2024-09-18T10:15:24.021 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.045753271999999998, 2024-09-18T10:15:24.021 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.046210793, 2024-09-18T10:15:24.021 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:24.021 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting_for_map not empty", 2024-09-18T10:15:24.021 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:24.021 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:24.022 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:24.022 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 563 2024-09-18T10:15:24.022 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.022 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:24.022 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.022 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:24.022 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956389+0000", 2024-09-18T10:15:24.022 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:24.022 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.022 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.022 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:24.022 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956389+0000", 2024-09-18T10:15:24.022 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:24.022 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.023 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.023 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:24.023 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956387+0000", 2024-09-18T10:15:24.023 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:24.023 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.023 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.023 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:24.023 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956395+0000", 2024-09-18T10:15:24.023 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.537e-06 2024-09-18T10:15:24.023 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.023 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.023 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:24.023 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956405+0000", 2024-09-18T10:15:24.023 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0057999999999999e-05 2024-09-18T10:15:24.024 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.024 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.024 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:24.024 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956416+0000", 2024-09-18T10:15:24.024 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1311999999999999e-05 2024-09-18T10:15:24.024 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.024 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.024 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:24.024 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956502+0000", 2024-09-18T10:15:24.024 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.5413000000000004e-05 2024-09-18T10:15:24.024 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.024 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.024 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:24.024 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956512+0000", 2024-09-18T10:15:24.025 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0299000000000001e-05 2024-09-18T10:15:24.025 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:24.025 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:24.025 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:24.025 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.025 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.025 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:564 3.19s0 3.d35743f9 (undecoded) ondisk+read+rwordered+known_if_redirected+supports_pool_eio e68)", 2024-09-18T10:15:24.025 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:23.956428+0000", 2024-09-18T10:15:24.025 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.045714171999999997, 2024-09-18T10:15:24.025 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.046269644999999998, 2024-09-18T10:15:24.025 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:24.025 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting_for_map not empty", 2024-09-18T10:15:24.025 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:24.026 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:24.026 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:24.026 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 564 2024-09-18T10:15:24.026 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.026 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:24.026 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.026 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:24.026 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956428+0000", 2024-09-18T10:15:24.026 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:24.026 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.026 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.026 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:24.026 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956428+0000", 2024-09-18T10:15:24.026 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:24.027 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.027 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.027 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:24.027 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956427+0000", 2024-09-18T10:15:24.027 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:15:24.027 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.027 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.027 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:24.027 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956432+0000", 2024-09-18T10:15:24.027 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9980000000000003e-06 2024-09-18T10:15:24.027 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.027 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.027 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:24.027 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956446+0000", 2024-09-18T10:15:24.028 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3602e-05 2024-09-18T10:15:24.028 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.028 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.028 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:24.028 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956454+0000", 2024-09-18T10:15:24.028 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.2369999999999992e-06 2024-09-18T10:15:24.028 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.028 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.028 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:24.028 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956551+0000", 2024-09-18T10:15:24.028 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.7424999999999996e-05 2024-09-18T10:15:24.028 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.028 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.028 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:24.029 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956558+0000", 2024-09-18T10:15:24.029 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.8050000000000001e-06 2024-09-18T10:15:24.029 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:24.029 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:24.029 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:24.029 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:24.029 INFO:teuthology.orchestra.run.smithi189.stdout: ], 2024-09-18T10:15:24.029 INFO:teuthology.orchestra.run.smithi189.stdout: "num_ops": 6 2024-09-18T10:15:24.029 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-18T10:15:24.030 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T10:15:24.065 INFO:tasks.rados.rados.0.smithi189.stdout:229: finishing rollback tid 0 to smithi18925896-8 2024-09-18T10:15:24.094 INFO:tasks.ceph.osd.12.smithi029.stderr:2024-09-18T10:15:24.090+0000 7f8e4e627700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 22190) UID: 0 2024-09-18T10:15:24.131 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-18T10:15:24.131 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-18T10:15:24.131 INFO:teuthology.orchestra.run.smithi189.stdout: "complaint_time": 30, 2024-09-18T10:15:24.131 INFO:teuthology.orchestra.run.smithi189.stdout: "num_blocked_ops": 0 2024-09-18T10:15:24.131 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-18T10:15:24.140 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T10:15:24.163 INFO:tasks.rados.rados.0.smithi189.stdout:231: done (2 left) 2024-09-18T10:15:24.164 INFO:tasks.rados.rados.0.smithi189.stdout:232: rmattr oid 24 current snap is 19 2024-09-18T10:15:24.164 INFO:tasks.rados.rados.0.smithi189.stdout:232: done (2 left) 2024-09-18T10:15:24.164 INFO:tasks.rados.rados.0.smithi189.stdout:233: snap_create 2024-09-18T10:15:24.166 INFO:tasks.rados.rados.0.smithi189.stdout:229: finishing rollback tid 1 to smithi18925896-8 2024-09-18T10:15:24.166 INFO:tasks.rados.rados.0.smithi189.stdout:229: finishing rollback tid 2 to smithi18925896-8 2024-09-18T10:15:24.166 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 8 v 30 (ObjNum 7 snap 0 seq_num 7) dirty exists 2024-09-18T10:15:24.196 INFO:tasks.ceph.osd.10.smithi154.stderr:2024-09-18T10:15:24.195+0000 7fb73221a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 22091) UID: 0 2024-09-18T10:15:24.255 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-18T10:15:24.255 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 20, 2024-09-18T10:15:24.255 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 600, 2024-09-18T10:15:24.255 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [ 2024-09-18T10:15:24.255 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.255 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:114 3.9s0 3:9a1a774c:::smithi18925896-38:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:24.256 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:50.816650+0000", 2024-09-18T10:15:24.256 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 33.434261775000003, 2024-09-18T10:15:24.256 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.41853457900000002, 2024-09-18T10:15:24.256 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:24.256 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:15:24.256 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:24.256 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:24.257 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:24.257 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 114 2024-09-18T10:15:24.257 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.257 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:24.257 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.257 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:24.257 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.816650+0000", 2024-09-18T10:15:24.257 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:24.257 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.257 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.257 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:24.258 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.816650+0000", 2024-09-18T10:15:24.258 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:24.258 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.258 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.258 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:24.258 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.816648+0000", 2024-09-18T10:15:24.258 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:24.258 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.258 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.259 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:24.259 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.816661+0000", 2024-09-18T10:15:24.259 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2619000000000001e-05 2024-09-18T10:15:24.259 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.259 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.259 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:24.259 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.816696+0000", 2024-09-18T10:15:24.259 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4885999999999998e-05 2024-09-18T10:15:24.259 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.259 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.259 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:24.259 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.816708+0000", 2024-09-18T10:15:24.260 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2843000000000001e-05 2024-09-18T10:15:24.260 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.260 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.260 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:24.260 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.818228+0000", 2024-09-18T10:15:24.260 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001519882 2024-09-18T10:15:24.260 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.260 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.260 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for rw locks", 2024-09-18T10:15:24.260 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.818557+0000", 2024-09-18T10:15:24.260 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00032873700000000001 2024-09-18T10:15:24.261 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.261 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.261 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:24.261 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.925488+0000", 2024-09-18T10:15:24.261 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.106930502 2024-09-18T10:15:24.261 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.261 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.261 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:24.261 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.925563+0000", 2024-09-18T10:15:24.261 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.5386e-05 2024-09-18T10:15:24.261 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.261 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.262 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:24.262 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.235185+0000", 2024-09-18T10:15:24.262 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.30962182399999999 2024-09-18T10:15:24.262 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:24.262 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:24.262 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:24.262 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.262 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.262 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:138 3.9s0 3:9ab7c4c0:::smithi18925896-46:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:24.262 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:50.921054+0000", 2024-09-18T10:15:24.262 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 33.329857476999997, 2024-09-18T10:15:24.263 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.61288337900000001, 2024-09-18T10:15:24.263 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:24.263 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:15:24.263 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:24.263 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:24.263 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:24.263 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 138 2024-09-18T10:15:24.263 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.263 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:24.263 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.263 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:24.264 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.921054+0000", 2024-09-18T10:15:24.264 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:24.264 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.264 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.264 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:24.264 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.921054+0000", 2024-09-18T10:15:24.264 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:24.264 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.264 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.264 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:24.264 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.921054+0000", 2024-09-18T10:15:24.265 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:24.265 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.265 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.265 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:24.265 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.921057+0000", 2024-09-18T10:15:24.265 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1829999999999998e-06 2024-09-18T10:15:24.265 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.265 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.265 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:24.265 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.921090+0000", 2024-09-18T10:15:24.266 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3281999999999997e-05 2024-09-18T10:15:24.266 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.266 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.266 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:24.266 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.921096+0000", 2024-09-18T10:15:24.266 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.0519999999999998e-06 2024-09-18T10:15:24.266 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.266 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.266 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:24.266 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.127704+0000", 2024-09-18T10:15:24.266 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20660720499999999 2024-09-18T10:15:24.267 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.267 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.267 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for rw locks", 2024-09-18T10:15:24.267 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.127851+0000", 2024-09-18T10:15:24.267 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00014710999999999999 2024-09-18T10:15:24.267 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.267 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.267 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:24.267 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.234300+0000", 2024-09-18T10:15:24.267 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10644893900000001 2024-09-18T10:15:24.267 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.267 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.268 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:24.268 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.234462+0000", 2024-09-18T10:15:24.268 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00016218999999999999 2024-09-18T10:15:24.268 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.268 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.268 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:24.268 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.533938+0000", 2024-09-18T10:15:24.268 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.29947596999999998 2024-09-18T10:15:24.268 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:24.268 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:24.268 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:24.269 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.269 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.269 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:142 3.9s0 3:9221fff5:::smithi18925896-48:head [append 0~425984,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:24.269 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:50.929918+0000", 2024-09-18T10:15:24.269 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 33.320994165000002, 2024-09-18T10:15:24.269 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.39521715200000002, 2024-09-18T10:15:24.269 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:24.269 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:24.269 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:24.269 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:24.269 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:24.269 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 142 2024-09-18T10:15:24.270 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.270 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:24.270 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.270 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:24.270 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.929918+0000", 2024-09-18T10:15:24.270 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:24.270 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.270 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.270 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:24.270 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.929918+0000", 2024-09-18T10:15:24.270 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:24.270 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.271 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.271 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:24.271 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.929916+0000", 2024-09-18T10:15:24.271 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:24.271 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.271 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.271 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:24.271 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.930096+0000", 2024-09-18T10:15:24.271 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00018038700000000001 2024-09-18T10:15:24.271 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.271 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.272 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:24.272 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.930122+0000", 2024-09-18T10:15:24.272 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5927000000000001e-05 2024-09-18T10:15:24.272 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.272 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.272 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:24.272 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.930132+0000", 2024-09-18T10:15:24.272 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.6539999999999994e-06 2024-09-18T10:15:24.272 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.272 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.272 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:24.272 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.127900+0000", 2024-09-18T10:15:24.272 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.19776816899999999 2024-09-18T10:15:24.273 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.273 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.273 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:24.273 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.128112+0000", 2024-09-18T10:15:24.273 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00021222999999999999 2024-09-18T10:15:24.273 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.273 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.273 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:24.273 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.129335+0000", 2024-09-18T10:15:24.273 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0012222159999999999 2024-09-18T10:15:24.273 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.274 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.274 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:24.274 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.132031+0000", 2024-09-18T10:15:24.274 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0026961300000000001 2024-09-18T10:15:24.274 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.274 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.274 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:24.274 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.325013+0000", 2024-09-18T10:15:24.274 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.19298240999999999 2024-09-18T10:15:24.274 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.274 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.274 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:24.275 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.325135+0000", 2024-09-18T10:15:24.275 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000121842 2024-09-18T10:15:24.275 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:24.275 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:24.275 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:24.275 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.275 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.275 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:143 3.9s0 3:9221fff5:::smithi18925896-48:head [append 0~376832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:24.275 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:50.932038+0000", 2024-09-18T10:15:24.275 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 33.318873875999998, 2024-09-18T10:15:24.275 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.60109878000000005, 2024-09-18T10:15:24.276 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:24.276 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:24.276 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:24.276 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:24.276 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:24.276 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 143 2024-09-18T10:15:24.276 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.276 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:24.276 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.276 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:24.276 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932038+0000", 2024-09-18T10:15:24.277 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:24.277 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.277 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.277 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:24.277 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932038+0000", 2024-09-18T10:15:24.277 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:24.277 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.277 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.277 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:24.277 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932037+0000", 2024-09-18T10:15:24.277 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:15:24.277 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.277 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.278 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:24.278 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932309+0000", 2024-09-18T10:15:24.278 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000272086 2024-09-18T10:15:24.278 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.278 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.278 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:24.278 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932323+0000", 2024-09-18T10:15:24.278 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4354e-05 2024-09-18T10:15:24.278 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.278 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.278 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:24.278 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932329+0000", 2024-09-18T10:15:24.279 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.6840000000000001e-06 2024-09-18T10:15:24.279 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.279 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.279 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:24.279 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.130012+0000", 2024-09-18T10:15:24.279 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.19768277100000001 2024-09-18T10:15:24.279 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.279 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.279 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:24.279 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.130068+0000", 2024-09-18T10:15:24.279 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.6158e-05 2024-09-18T10:15:24.279 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.280 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.280 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:24.280 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.130809+0000", 2024-09-18T10:15:24.280 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00074078700000000002 2024-09-18T10:15:24.280 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.280 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.280 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:24.280 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.132840+0000", 2024-09-18T10:15:24.280 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0020308879999999998 2024-09-18T10:15:24.280 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.280 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.281 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:24.281 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.533025+0000", 2024-09-18T10:15:24.281 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.400185491 2024-09-18T10:15:24.281 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.281 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.281 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:24.281 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.533137+0000", 2024-09-18T10:15:24.281 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000111727 2024-09-18T10:15:24.281 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:24.281 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:24.282 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:24.282 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.282 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.282 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:144 3.9s0 3:9221fff5:::smithi18925896-48:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:24.282 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:50.932336+0000", 2024-09-18T10:15:24.282 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 33.318576239999999, 2024-09-18T10:15:24.282 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.60198766699999995, 2024-09-18T10:15:24.282 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:24.282 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:24.282 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:24.282 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:24.282 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:24.282 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 144 2024-09-18T10:15:24.283 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.283 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:24.283 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.283 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:24.283 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932336+0000", 2024-09-18T10:15:24.283 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:24.283 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.283 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.283 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:24.283 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932336+0000", 2024-09-18T10:15:24.283 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:24.283 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.284 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.284 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:24.284 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932335+0000", 2024-09-18T10:15:24.284 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:15:24.284 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.284 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.284 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:24.284 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932339+0000", 2024-09-18T10:15:24.284 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8539999999999999e-06 2024-09-18T10:15:24.284 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.284 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.284 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:24.285 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932347+0000", 2024-09-18T10:15:24.285 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.2910000000000007e-06 2024-09-18T10:15:24.285 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.285 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.285 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:24.285 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932351+0000", 2024-09-18T10:15:24.285 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.5420000000000001e-06 2024-09-18T10:15:24.285 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.285 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.285 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:24.285 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.131380+0000", 2024-09-18T10:15:24.285 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.199029236 2024-09-18T10:15:24.286 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.286 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.286 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:24.286 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.131436+0000", 2024-09-18T10:15:24.286 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.6098999999999999e-05 2024-09-18T10:15:24.286 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.286 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.286 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:24.286 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.131777+0000", 2024-09-18T10:15:24.286 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00034058800000000001 2024-09-18T10:15:24.286 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.286 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.287 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:24.287 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.133245+0000", 2024-09-18T10:15:24.287 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0014682300000000001 2024-09-18T10:15:24.287 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.287 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.287 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:24.287 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.534187+0000", 2024-09-18T10:15:24.287 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40094253299999999 2024-09-18T10:15:24.287 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.287 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.287 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:24.287 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.534323+0000", 2024-09-18T10:15:24.287 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00013606299999999999 2024-09-18T10:15:24.287 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:24.288 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:24.288 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:24.288 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.288 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.288 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:145 3.9s0 3:9221fff5:::smithi18925896-48:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:24.288 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:50.932355+0000", 2024-09-18T10:15:24.288 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 33.318556756, 2024-09-18T10:15:24.288 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.80502307200000001, 2024-09-18T10:15:24.288 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:24.288 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:15:24.288 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:24.288 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:24.289 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:24.289 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 145 2024-09-18T10:15:24.289 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.289 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:24.289 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.289 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:24.289 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932355+0000", 2024-09-18T10:15:24.289 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:24.289 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.289 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.290 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:24.290 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932355+0000", 2024-09-18T10:15:24.290 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:24.290 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.290 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.290 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:24.290 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932355+0000", 2024-09-18T10:15:24.290 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:24.290 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.290 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.290 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:24.291 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932358+0000", 2024-09-18T10:15:24.291 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4369999999999999e-06 2024-09-18T10:15:24.291 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.291 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.291 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:24.291 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932366+0000", 2024-09-18T10:15:24.291 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.3010000000000007e-06 2024-09-18T10:15:24.291 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.291 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.291 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:24.291 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932370+0000", 2024-09-18T10:15:24.291 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.5619999999999999e-06 2024-09-18T10:15:24.292 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.292 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.292 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:24.292 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.132210+0000", 2024-09-18T10:15:24.292 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.199840145 2024-09-18T10:15:24.292 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.292 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.292 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for rw locks", 2024-09-18T10:15:24.292 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.132312+0000", 2024-09-18T10:15:24.292 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000102335 2024-09-18T10:15:24.292 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.293 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.293 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:24.293 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.535310+0000", 2024-09-18T10:15:24.293 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40299779000000002 2024-09-18T10:15:24.293 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.293 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.293 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:24.293 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.535541+0000", 2024-09-18T10:15:24.293 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00023038299999999999 2024-09-18T10:15:24.293 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.293 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.296 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:24.296 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.737378+0000", 2024-09-18T10:15:24.296 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.201837604 2024-09-18T10:15:24.297 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:24.297 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:24.297 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:24.297 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.297 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.297 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:174 3.9s0 3:9fc2eacb:::smithi18925896-17:head [read 0~278528,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:24.297 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:51.836672+0000", 2024-09-18T10:15:24.297 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 32.414239500999997, 2024-09-18T10:15:24.297 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.407996215, 2024-09-18T10:15:24.297 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:24.297 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:15:24.298 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:24.298 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:24.298 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:24.298 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 174 2024-09-18T10:15:24.298 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.298 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:24.298 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.298 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:24.298 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.836672+0000", 2024-09-18T10:15:24.298 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:24.298 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.299 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.299 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:24.299 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.836672+0000", 2024-09-18T10:15:24.299 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:24.299 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.299 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.299 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:24.299 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.836671+0000", 2024-09-18T10:15:24.299 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:24.299 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.299 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.299 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:24.300 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.836680+0000", 2024-09-18T10:15:24.300 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.5830000000000007e-06 2024-09-18T10:15:24.300 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.300 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.300 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:24.300 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.836696+0000", 2024-09-18T10:15:24.300 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6823000000000001e-05 2024-09-18T10:15:24.300 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.300 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.300 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:24.300 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.836710+0000", 2024-09-18T10:15:24.301 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.325e-05 2024-09-18T10:15:24.301 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.301 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.301 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:24.301 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.142312+0000", 2024-09-18T10:15:24.301 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.30560266400000002 2024-09-18T10:15:24.301 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.301 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.301 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:24.301 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.142549+0000", 2024-09-18T10:15:24.302 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00023630299999999999 2024-09-18T10:15:24.302 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.302 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.302 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:24.302 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.244669+0000", 2024-09-18T10:15:24.302 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10211993599999999 2024-09-18T10:15:24.302 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:24.302 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:24.303 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:24.303 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.303 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.304 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:186 3.9s0 3:93224e57:::smithi18925896-10:head [rollback 2] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:24.304 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:52.693370+0000", 2024-09-18T10:15:24.304 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 31.557541489999998, 2024-09-18T10:15:24.304 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.39644133999999998, 2024-09-18T10:15:24.304 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:24.304 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:24.304 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:24.304 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:24.304 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:24.304 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 186 2024-09-18T10:15:24.304 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.305 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:24.305 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.305 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:24.305 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.693370+0000", 2024-09-18T10:15:24.305 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:24.305 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.305 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.305 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:24.305 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.693370+0000", 2024-09-18T10:15:24.305 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:24.305 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.305 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.306 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:24.306 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.693370+0000", 2024-09-18T10:15:24.306 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:24.306 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.306 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.306 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:24.306 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.693373+0000", 2024-09-18T10:15:24.306 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.712e-06 2024-09-18T10:15:24.306 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.306 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.306 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:24.306 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.693379+0000", 2024-09-18T10:15:24.307 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9320000000000001e-06 2024-09-18T10:15:24.307 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.307 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.307 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:24.307 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.793509+0000", 2024-09-18T10:15:24.307 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100130331 2024-09-18T10:15:24.307 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.307 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.307 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:24.307 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.800561+0000", 2024-09-18T10:15:24.307 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0070525559999999998 2024-09-18T10:15:24.307 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.308 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.308 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:24.308 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.800733+0000", 2024-09-18T10:15:24.308 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00017114100000000001 2024-09-18T10:15:24.308 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.308 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.308 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:24.308 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.802155+0000", 2024-09-18T10:15:24.308 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0014220420000000001 2024-09-18T10:15:24.308 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.308 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.308 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:24.309 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.905163+0000", 2024-09-18T10:15:24.309 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.103008407 2024-09-18T10:15:24.309 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.309 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.309 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:24.309 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:53.089742+0000", 2024-09-18T10:15:24.309 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.184578568 2024-09-18T10:15:24.309 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.309 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.309 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:24.309 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:53.089812+0000", 2024-09-18T10:15:24.309 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.0203000000000005e-05 2024-09-18T10:15:24.310 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:24.310 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:24.310 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:24.310 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.310 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.310 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:202 3.9s0 3:9221fff5:::smithi18925896-48:head [copy-from ver 2] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:24.310 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:52.793605+0000", 2024-09-18T10:15:24.310 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 31.457306905999999, 2024-09-18T10:15:24.310 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.61381690899999997, 2024-09-18T10:15:24.310 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:24.310 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:24.310 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:24.311 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:24.311 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:24.311 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 202 2024-09-18T10:15:24.311 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.311 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:24.311 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.311 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:24.311 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.793605+0000", 2024-09-18T10:15:24.311 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:24.311 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.311 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.311 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:24.312 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.793605+0000", 2024-09-18T10:15:24.312 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:24.312 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.312 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.312 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:24.312 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.793604+0000", 2024-09-18T10:15:24.312 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:15:24.312 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.312 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.312 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:24.312 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.793613+0000", 2024-09-18T10:15:24.312 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.4360000000000008e-06 2024-09-18T10:15:24.313 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.313 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.313 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:24.313 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.793628+0000", 2024-09-18T10:15:24.313 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4953000000000001e-05 2024-09-18T10:15:24.313 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.313 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.313 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:24.313 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.793634+0000", 2024-09-18T10:15:24.313 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.0170000000000002e-06 2024-09-18T10:15:24.313 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.314 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.314 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:24.314 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.905359+0000", 2024-09-18T10:15:24.314 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.111724328 2024-09-18T10:15:24.314 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.314 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.314 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:24.314 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.905540+0000", 2024-09-18T10:15:24.314 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000181552 2024-09-18T10:15:24.314 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.314 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.314 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:24.315 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:53.203724+0000", 2024-09-18T10:15:24.315 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.29818339500000002 2024-09-18T10:15:24.315 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.315 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.315 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:24.315 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:53.206308+0000", 2024-09-18T10:15:24.315 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0025841620000000001 2024-09-18T10:15:24.315 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.315 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.315 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:24.315 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:53.407237+0000", 2024-09-18T10:15:24.316 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.200928739 2024-09-18T10:15:24.316 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.316 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.316 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:24.316 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:53.407422+0000", 2024-09-18T10:15:24.316 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000185384 2024-09-18T10:15:24.316 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:24.316 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:24.316 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:24.316 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.316 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.317 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:203 3.9s0 3:9221fff5:::smithi18925896-48:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:24.317 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:52.793641+0000", 2024-09-18T10:15:24.317 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 31.457270521000002, 2024-09-18T10:15:24.317 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.41220821099999999, 2024-09-18T10:15:24.317 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:24.317 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:15:24.317 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:24.317 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:24.317 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:24.317 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 203 2024-09-18T10:15:24.317 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.318 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:24.318 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.318 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:24.318 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.793641+0000", 2024-09-18T10:15:24.318 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:24.318 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.318 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.318 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:24.318 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.793641+0000", 2024-09-18T10:15:24.318 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:24.318 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.318 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.319 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:24.319 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.793641+0000", 2024-09-18T10:15:24.319 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:15:24.319 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.319 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.319 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:24.319 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.793644+0000", 2024-09-18T10:15:24.319 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7170000000000002e-06 2024-09-18T10:15:24.319 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.319 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.319 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:24.320 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.793658+0000", 2024-09-18T10:15:24.320 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3353000000000001e-05 2024-09-18T10:15:24.320 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.320 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.320 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:24.320 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.793663+0000", 2024-09-18T10:15:24.320 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.7729999999999996e-06 2024-09-18T10:15:24.320 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.320 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.320 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:24.321 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.905813+0000", 2024-09-18T10:15:24.321 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.112150033 2024-09-18T10:15:24.321 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.321 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.321 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for blocked object", 2024-09-18T10:15:24.321 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.905901+0000", 2024-09-18T10:15:24.322 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.7319999999999997e-05 2024-09-18T10:15:24.322 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.322 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.322 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:24.322 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:53.205317+0000", 2024-09-18T10:15:24.322 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.299415826 2024-09-18T10:15:24.322 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.322 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.322 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:24.322 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:53.205528+0000", 2024-09-18T10:15:24.323 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000211485 2024-09-18T10:15:24.323 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.323 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.323 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:24.323 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:53.205850+0000", 2024-09-18T10:15:24.323 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000321544 2024-09-18T10:15:24.323 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:24.323 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:24.323 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:24.323 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.323 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.324 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.9s0 45/40 ECSubWriteReply(tid=55, last_complete=45'30, committed=1, applied=1))", 2024-09-18T10:15:24.324 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:01.594361+0000", 2024-09-18T10:15:24.324 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 22.656550886000002, 2024-09-18T10:15:24.324 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.32295218399999998, 2024-09-18T10:15:24.324 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:24.324 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:24.324 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:24.324 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.324 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:24.324 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.594361+0000", 2024-09-18T10:15:24.324 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:24.325 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.325 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.325 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:24.325 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.594361+0000", 2024-09-18T10:15:24.325 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:24.325 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.325 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.325 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:24.325 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.594360+0000", 2024-09-18T10:15:24.325 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:24.325 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.326 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.326 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:24.326 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.594366+0000", 2024-09-18T10:15:24.326 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.716e-06 2024-09-18T10:15:24.326 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.326 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.326 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:24.326 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.594381+0000", 2024-09-18T10:15:24.326 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4813e-05 2024-09-18T10:15:24.326 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.326 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.327 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:24.327 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.594390+0000", 2024-09-18T10:15:24.327 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.2199999999999998e-06 2024-09-18T10:15:24.327 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.327 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.327 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:24.327 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.916963+0000", 2024-09-18T10:15:24.327 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.32257234600000001 2024-09-18T10:15:24.327 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.327 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.327 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:24.327 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.917313+0000", 2024-09-18T10:15:24.328 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00035066200000000001 2024-09-18T10:15:24.328 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:24.328 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:24.328 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:24.328 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.328 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.328 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:351 3.19s0 3:9a1a774c:::smithi18925896-38:b [read 0~1638400,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-18T10:15:24.328 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:03.544302+0000", 2024-09-18T10:15:24.328 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 20.706609517, 2024-09-18T10:15:24.328 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.41512137599999999, 2024-09-18T10:15:24.329 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:24.329 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:15:24.329 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:24.329 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:24.329 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:24.329 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 351 2024-09-18T10:15:24.329 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.329 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:24.329 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.329 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:24.330 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.544302+0000", 2024-09-18T10:15:24.330 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:24.330 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.330 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.330 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:24.330 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.544302+0000", 2024-09-18T10:15:24.330 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:24.330 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.330 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.330 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:24.331 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.544298+0000", 2024-09-18T10:15:24.331 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999957 2024-09-18T10:15:24.331 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.331 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.331 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:24.331 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.544320+0000", 2024-09-18T10:15:24.331 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1549e-05 2024-09-18T10:15:24.331 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.331 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.331 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:24.331 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.544366+0000", 2024-09-18T10:15:24.332 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.6752999999999998e-05 2024-09-18T10:15:24.332 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.332 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.332 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:24.332 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.544387+0000", 2024-09-18T10:15:24.332 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1052000000000001e-05 2024-09-18T10:15:24.332 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.332 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.332 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:24.333 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.544758+0000", 2024-09-18T10:15:24.333 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00037069700000000002 2024-09-18T10:15:24.333 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.333 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.333 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "op must wait for map", 2024-09-18T10:15:24.333 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.544786+0000", 2024-09-18T10:15:24.333 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7472000000000001e-05 2024-09-18T10:15:24.333 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.333 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.333 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:24.333 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.649236+0000", 2024-09-18T10:15:24.334 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.104450378 2024-09-18T10:15:24.334 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.334 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.334 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:24.334 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.649492+0000", 2024-09-18T10:15:24.334 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00025605300000000001 2024-09-18T10:15:24.334 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.334 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.334 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:24.334 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.959424+0000", 2024-09-18T10:15:24.334 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.30993174899999998 2024-09-18T10:15:24.335 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:24.335 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:24.335 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:24.335 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.335 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.335 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:417 3.9s0 3:9221fff5:::smithi18925896-48:head [copy-from ver 8] snapc 10=[10,f,e,b,4] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-18T10:15:24.335 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:08.547348+0000", 2024-09-18T10:15:24.335 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 15.703564290999999, 2024-09-18T10:15:24.335 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.69832097199999998, 2024-09-18T10:15:24.336 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:24.336 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:24.336 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:24.336 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:24.336 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:24.336 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 417 2024-09-18T10:15:24.336 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.336 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:24.336 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.337 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:24.337 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:08.547348+0000", 2024-09-18T10:15:24.337 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:24.337 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.337 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.337 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:24.337 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:08.547348+0000", 2024-09-18T10:15:24.337 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:24.337 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.337 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.338 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:24.338 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:08.547344+0000", 2024-09-18T10:15:24.338 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999967 2024-09-18T10:15:24.338 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.338 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.338 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:24.338 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:08.547367+0000", 2024-09-18T10:15:24.338 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.2316000000000001e-05 2024-09-18T10:15:24.338 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.339 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.339 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:24.339 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:08.547415+0000", 2024-09-18T10:15:24.339 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.8541999999999997e-05 2024-09-18T10:15:24.339 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.339 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.339 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:24.339 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:08.647531+0000", 2024-09-18T10:15:24.339 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10011588 2024-09-18T10:15:24.339 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.340 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.340 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:24.340 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:08.647713+0000", 2024-09-18T10:15:24.340 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000181965 2024-09-18T10:15:24.340 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.340 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.340 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "op must wait for map", 2024-09-18T10:15:24.340 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:08.647726+0000", 2024-09-18T10:15:24.340 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3576e-05 2024-09-18T10:15:24.340 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.341 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.341 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:24.341 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:08.754339+0000", 2024-09-18T10:15:24.341 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10661229699999999 2024-09-18T10:15:24.341 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.341 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.341 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:24.341 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:08.754855+0000", 2024-09-18T10:15:24.341 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00051627900000000002 2024-09-18T10:15:24.341 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.342 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.342 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:24.342 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:09.040700+0000", 2024-09-18T10:15:24.342 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.285844922 2024-09-18T10:15:24.342 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.342 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.342 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:24.342 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:09.045961+0000", 2024-09-18T10:15:24.342 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0052605769999999998 2024-09-18T10:15:24.342 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.342 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.343 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:24.343 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:09.245481+0000", 2024-09-18T10:15:24.343 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.19952081599999999 2024-09-18T10:15:24.343 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.343 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.343 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:24.343 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:09.245669+0000", 2024-09-18T10:15:24.343 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000187245 2024-09-18T10:15:24.343 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:24.343 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:24.343 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:24.344 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.344 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.344 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:418 3.9s0 3:9221fff5:::smithi18925896-48:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e53)", 2024-09-18T10:15:24.344 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:08.647572+0000", 2024-09-18T10:15:24.344 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 15.60334014, 2024-09-18T10:15:24.344 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.39760114600000002, 2024-09-18T10:15:24.344 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:24.344 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:15:24.344 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:24.344 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:24.344 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:24.345 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 418 2024-09-18T10:15:24.345 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.345 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:24.345 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.345 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:24.345 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:08.647572+0000", 2024-09-18T10:15:24.345 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:24.345 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.345 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.345 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:24.345 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:08.647572+0000", 2024-09-18T10:15:24.346 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:24.346 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.346 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.346 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:24.346 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:08.647570+0000", 2024-09-18T10:15:24.346 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:24.346 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.346 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.346 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:24.346 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:08.647581+0000", 2024-09-18T10:15:24.346 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0829000000000001e-05 2024-09-18T10:15:24.347 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.347 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.347 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:24.347 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:08.647599+0000", 2024-09-18T10:15:24.347 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7846000000000002e-05 2024-09-18T10:15:24.347 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.347 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.347 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:24.347 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:08.647609+0000", 2024-09-18T10:15:24.347 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0339999999999999e-05 2024-09-18T10:15:24.347 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.348 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.348 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:24.348 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:08.747808+0000", 2024-09-18T10:15:24.348 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100199012 2024-09-18T10:15:24.348 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.348 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.348 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:24.348 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:08.747819+0000", 2024-09-18T10:15:24.348 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1276e-05 2024-09-18T10:15:24.348 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.348 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.349 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:24.349 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:08.854528+0000", 2024-09-18T10:15:24.349 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.106708267 2024-09-18T10:15:24.349 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.349 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.349 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for blocked object", 2024-09-18T10:15:24.349 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:08.854638+0000", 2024-09-18T10:15:24.349 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000110214 2024-09-18T10:15:24.349 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.349 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.350 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:24.350 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:09.044871+0000", 2024-09-18T10:15:24.350 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.190232762 2024-09-18T10:15:24.350 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.350 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.350 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:24.350 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:09.045037+0000", 2024-09-18T10:15:24.350 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00016675500000000001 2024-09-18T10:15:24.350 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.350 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.350 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:24.351 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:09.045173+0000", 2024-09-18T10:15:24.351 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000135575 2024-09-18T10:15:24.351 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:24.351 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:24.351 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:24.351 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.351 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.351 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.1s4 57/40 ECSubWrite(tid=112, reqid=osd.4.0:112, at_version=57'20, trim_to=0'0, roll_forward_to=56'16))", 2024-09-18T10:15:24.351 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:13.062699+0000", 2024-09-18T10:15:24.351 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 11.188212602, 2024-09-18T10:15:24.351 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40470361999999999, 2024-09-18T10:15:24.352 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:24.352 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:24.352 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:24.352 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.352 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:24.352 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.062699+0000", 2024-09-18T10:15:24.352 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:24.352 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.352 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.352 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:24.352 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.062699+0000", 2024-09-18T10:15:24.352 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:24.353 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.353 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.353 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:24.353 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.062699+0000", 2024-09-18T10:15:24.353 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:24.353 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.353 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.353 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:24.353 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.062734+0000", 2024-09-18T10:15:24.353 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.5108999999999999e-05 2024-09-18T10:15:24.353 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.354 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.354 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:24.354 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.062746+0000", 2024-09-18T10:15:24.354 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2062000000000001e-05 2024-09-18T10:15:24.354 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.354 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.354 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:24.354 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.062754+0000", 2024-09-18T10:15:24.354 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.5800000000000003e-06 2024-09-18T10:15:24.354 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.354 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.354 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:24.355 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.254656+0000", 2024-09-18T10:15:24.355 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.19190244400000001 2024-09-18T10:15:24.355 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.355 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.355 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:24.355 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.254673+0000", 2024-09-18T10:15:24.355 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7146000000000002e-05 2024-09-18T10:15:24.355 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.355 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.355 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:24.356 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.467347+0000", 2024-09-18T10:15:24.356 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.21267335500000001 2024-09-18T10:15:24.356 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.356 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.356 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:24.356 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.467403+0000", 2024-09-18T10:15:24.356 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.6443999999999997e-05 2024-09-18T10:15:24.356 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:24.356 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:24.356 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:24.356 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.356 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.357 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:499 3.9s0 3:9221fff5:::smithi18925896-48:head [read 0~704512,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T10:15:24.357 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:17.889501+0000", 2024-09-18T10:15:24.357 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 6.3614105649999999, 2024-09-18T10:15:24.357 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.41188610399999998, 2024-09-18T10:15:24.357 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:24.357 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:15:24.357 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:24.357 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:24.357 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:24.357 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 499 2024-09-18T10:15:24.357 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.358 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:24.358 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.358 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:24.358 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889501+0000", 2024-09-18T10:15:24.358 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:24.358 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.358 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.358 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:24.358 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889501+0000", 2024-09-18T10:15:24.358 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:24.358 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.359 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.359 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:24.359 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889499+0000", 2024-09-18T10:15:24.359 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999976 2024-09-18T10:15:24.359 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.359 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.359 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:24.359 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889514+0000", 2024-09-18T10:15:24.359 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4793999999999999e-05 2024-09-18T10:15:24.359 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.359 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.360 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:24.360 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889538+0000", 2024-09-18T10:15:24.360 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4218000000000001e-05 2024-09-18T10:15:24.360 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.360 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.360 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:24.360 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889552+0000", 2024-09-18T10:15:24.360 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4021e-05 2024-09-18T10:15:24.360 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.360 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.360 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:24.361 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889705+0000", 2024-09-18T10:15:24.361 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00015229499999999999 2024-09-18T10:15:24.361 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.361 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.361 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "op must wait for map", 2024-09-18T10:15:24.361 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889725+0000", 2024-09-18T10:15:24.361 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0251999999999999e-05 2024-09-18T10:15:24.361 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.361 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.361 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:24.361 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.894166+0000", 2024-09-18T10:15:24.362 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0044407969999999998 2024-09-18T10:15:24.362 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.362 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.362 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:24.362 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.894321+0000", 2024-09-18T10:15:24.362 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000155403 2024-09-18T10:15:24.362 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.362 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.362 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:24.362 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.301387+0000", 2024-09-18T10:15:24.362 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40706650900000002 2024-09-18T10:15:24.362 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:24.363 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:24.363 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:24.363 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.363 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.363 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:500 3.9s0 3:9221fff5:::smithi18925896-48:head [read 0~704512,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T10:15:24.363 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:17.889581+0000", 2024-09-18T10:15:24.363 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 6.3613310240000001, 2024-09-18T10:15:24.363 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.41275783799999999, 2024-09-18T10:15:24.363 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:24.363 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:15:24.363 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:24.364 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:24.364 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:24.364 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 500 2024-09-18T10:15:24.364 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.364 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:24.364 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.364 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:24.364 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889581+0000", 2024-09-18T10:15:24.364 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:24.364 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.364 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.365 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:24.365 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889581+0000", 2024-09-18T10:15:24.365 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:24.365 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.365 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.365 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:24.365 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889580+0000", 2024-09-18T10:15:24.365 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:24.365 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.365 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.365 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:24.366 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889586+0000", 2024-09-18T10:15:24.366 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8340000000000004e-06 2024-09-18T10:15:24.366 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.366 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.366 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:24.366 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889594+0000", 2024-09-18T10:15:24.366 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.8320000000000006e-06 2024-09-18T10:15:24.366 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.366 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.366 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:24.367 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889599+0000", 2024-09-18T10:15:24.367 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.7890000000000002e-06 2024-09-18T10:15:24.367 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.367 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.367 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:24.367 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889906+0000", 2024-09-18T10:15:24.367 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00030686400000000001 2024-09-18T10:15:24.367 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.367 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.367 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:24.367 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889930+0000", 2024-09-18T10:15:24.368 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4318999999999999e-05 2024-09-18T10:15:24.368 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.368 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.368 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:24.368 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.895541+0000", 2024-09-18T10:15:24.368 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0056113029999999998 2024-09-18T10:15:24.368 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.368 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.368 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:24.368 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.895817+0000", 2024-09-18T10:15:24.369 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000275612 2024-09-18T10:15:24.369 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.369 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.369 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:24.369 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.302339+0000", 2024-09-18T10:15:24.369 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40652197699999998 2024-09-18T10:15:24.369 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:24.369 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:24.369 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:24.369 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.369 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.370 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:501 3.9s0 3:9221fff5:::smithi18925896-48:head [read 0~704512,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T10:15:24.370 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:17.889668+0000", 2024-09-18T10:15:24.370 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 6.3612439140000001, 2024-09-18T10:15:24.370 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.51380426099999998, 2024-09-18T10:15:24.370 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:24.370 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:15:24.370 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:24.370 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:24.370 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:24.370 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 501 2024-09-18T10:15:24.371 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.371 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:24.371 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.371 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:24.371 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889668+0000", 2024-09-18T10:15:24.371 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:24.371 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.371 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.371 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:24.371 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889668+0000", 2024-09-18T10:15:24.372 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:24.372 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.372 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.372 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:24.372 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889667+0000", 2024-09-18T10:15:24.372 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:24.372 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.372 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.372 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:24.372 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889671+0000", 2024-09-18T10:15:24.372 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7570000000000002e-06 2024-09-18T10:15:24.373 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.373 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.373 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:24.373 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889682+0000", 2024-09-18T10:15:24.373 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0709e-05 2024-09-18T10:15:24.373 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.373 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.373 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:24.373 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889686+0000", 2024-09-18T10:15:24.373 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.437e-06 2024-09-18T10:15:24.373 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.374 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.374 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:24.374 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889996+0000", 2024-09-18T10:15:24.374 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00030937899999999998 2024-09-18T10:15:24.374 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.374 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.374 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:24.374 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.890018+0000", 2024-09-18T10:15:24.374 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.2592999999999999e-05 2024-09-18T10:15:24.374 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.374 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.374 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:24.375 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.996777+0000", 2024-09-18T10:15:24.375 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.106758192 2024-09-18T10:15:24.375 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.375 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.375 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:24.375 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.997045+0000", 2024-09-18T10:15:24.375 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00026851300000000001 2024-09-18T10:15:24.375 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.375 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.375 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:24.375 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.403472+0000", 2024-09-18T10:15:24.376 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40642724899999999 2024-09-18T10:15:24.376 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:24.376 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:24.376 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:24.376 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.376 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.376 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:502 3.19s0 3:9a1a774c:::smithi18925896-38:head [create,setxattr Rj\\mZOfQXo^b_ZWWfqN]m[iPt^aRVgsYNetVd`VO (19508) in=40b,setxattr [adnUNmYmWdN__]tR]msPnVSRiY^psh`dY\\Z (630) in=36b,setxattr cPsWXtVeXTPagdn_WVZbVcV\\UjSZUqsVlPZbY` (630) in=38b,setxattr pqbrReOSsgZlmftc]`Z`]USaXjsacf`fao_VP`eU (1190) in=40b,setxattr aYfgr^q[\\aPg_aZsakfUfsdhYbra^QXjlkrpdY\\\\ (138) in=40b,setxattr e\\XXWiVknihNQQ[OsThQUbRmtprrfYPeRX[[mRcQ (1719) in=40b,setxattr TNQYR^jgR`pPaYitmrcliV[dttobncTntcfTlQkt (8420) in=40b,setxattr W_aq[RVsaUpQtX_RpV^Vpqrg`QgQViPeVaRqZSmf (525) in=40b,setxattr RWdkeaaUP\\PZteTNscS^UVRtjo`oqqdNqe]cdZUa (245) in=40b,setxattr R^`al]TqifatYOiOU]p`ZegfXmNsXVaZcdd]b]YX (1435) in=40b,setxattr p]OS_Znl[dNa^qTn`WW[j\\dtgbNac_bijYmg (630) in=36b,setxattr n_jr[tV[VhYPqU]UenciobhtYfY^a^c\\hd^YrcQ[ (13472) in=40b,setxattr gReQSUg`cdl_ilU_lo[ggg\\fZsVQY[PgY]jPsZ_U (14664) in=40b,setxattr cNWaQrsdt^dic_qN_^mTXmc`_X]YOTXNk[WldRe` (523) in=40b,setxattr tZ]grhlaq]kUO[k^d^dr``XNio^^Oaebpj\\[Ytdq (33) in=40b,setxattr jqS_j^cefaXab`pQoPXfRkSmdako`fZ_heb^_fma (1191) in=40b,setxattr dgXPi (630) in=5b,setxattr Or_kkpbnNkTrQ\\apUaUsZR\\XW^nabhaiQ\\]gOjR\\ (525) in=40b,setxattr Rf`aR[fdtjQ\\NaqZke]l_rUssrVpOPXYWpXkl_kd (2876) in=40b,setxattr _sUSYWYUmOrlUVhZajZeYbbnsdra[]`Q`^Ogea[m (631) in=40b] snapc 19=[19,18,16,12,11,10,f,e] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T10:15:24.376 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:17.894821+0000", 2024-09-18T10:15:24.376 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 6.3560908859999996, 2024-09-18T10:15:24.376 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.61297755899999995, 2024-09-18T10:15:24.376 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:24.377 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:24.377 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:24.377 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:24.377 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:24.377 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 502 2024-09-18T10:15:24.377 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.377 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:24.377 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.377 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:24.377 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.894821+0000", 2024-09-18T10:15:24.378 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:24.378 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.378 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.378 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:24.378 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.894821+0000", 2024-09-18T10:15:24.378 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:24.378 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.378 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.378 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:24.378 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.894819+0000", 2024-09-18T10:15:24.378 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:24.378 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.379 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.379 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:24.379 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.894942+0000", 2024-09-18T10:15:24.379 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00012299699999999999 2024-09-18T10:15:24.379 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.379 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.379 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:24.379 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.894987+0000", 2024-09-18T10:15:24.379 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.515e-05 2024-09-18T10:15:24.379 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.379 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.379 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:24.380 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.895021+0000", 2024-09-18T10:15:24.380 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3905000000000001e-05 2024-09-18T10:15:24.380 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.380 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.380 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:24.380 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.302420+0000", 2024-09-18T10:15:24.380 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40739912099999998 2024-09-18T10:15:24.380 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.380 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.380 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:24.380 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.302573+0000", 2024-09-18T10:15:24.380 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00015244400000000001 2024-09-18T10:15:24.381 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.381 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.381 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:24.381 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.304119+0000", 2024-09-18T10:15:24.381 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001546504 2024-09-18T10:15:24.381 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.381 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.381 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:24.381 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.306309+0000", 2024-09-18T10:15:24.381 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.002189992 2024-09-18T10:15:24.381 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.381 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.382 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:24.382 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.507653+0000", 2024-09-18T10:15:24.382 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20134429600000001 2024-09-18T10:15:24.382 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.382 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.382 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:24.382 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.507799+0000", 2024-09-18T10:15:24.382 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000145153 2024-09-18T10:15:24.382 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:24.382 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:24.382 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:24.383 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.383 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.383 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:514 3.19s0 3:9fc2eacb:::smithi18925896-17:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e63)", 2024-09-18T10:15:24.383 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:19.215577+0000", 2024-09-18T10:15:24.383 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 5.0353346449999998, 2024-09-18T10:15:24.383 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.55261972699999995, 2024-09-18T10:15:24.383 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:24.383 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:15:24.383 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:24.383 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:24.383 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:24.383 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 514 2024-09-18T10:15:24.384 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.384 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:24.384 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.384 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:24.384 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.215577+0000", 2024-09-18T10:15:24.384 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:24.384 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.384 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.384 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:24.384 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.215577+0000", 2024-09-18T10:15:24.384 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:24.385 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.385 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.385 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:24.385 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.215575+0000", 2024-09-18T10:15:24.385 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:24.385 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.385 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.385 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:24.385 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.215590+0000", 2024-09-18T10:15:24.385 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4153e-05 2024-09-18T10:15:24.386 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.386 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.386 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:24.386 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.215610+0000", 2024-09-18T10:15:24.386 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9962e-05 2024-09-18T10:15:24.386 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.386 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.386 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:24.386 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.215622+0000", 2024-09-18T10:15:24.386 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2714000000000001e-05 2024-09-18T10:15:24.386 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.386 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.387 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:24.387 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.216288+0000", 2024-09-18T10:15:24.387 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00066566900000000003 2024-09-18T10:15:24.387 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.387 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.387 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for blocked object", 2024-09-18T10:15:24.387 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.216400+0000", 2024-09-18T10:15:24.387 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000112101 2024-09-18T10:15:24.387 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.387 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.387 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:24.387 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.767868+0000", 2024-09-18T10:15:24.388 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.55146753900000001 2024-09-18T10:15:24.388 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.388 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.388 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:24.388 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.768017+0000", 2024-09-18T10:15:24.388 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000149567 2024-09-18T10:15:24.388 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.388 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.388 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:24.388 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.768197+0000", 2024-09-18T10:15:24.388 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00017981200000000001 2024-09-18T10:15:24.388 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:24.389 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:24.389 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:24.389 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:24.389 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:24.389 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-18T10:15:24.391 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-09-18T10:15:24.391 INFO:tasks.rados.rados.0.smithi189.stdout:228: finishing write tid 1 to smithi18925896-17 2024-09-18T10:15:24.391 INFO:tasks.rados.rados.0.smithi189.stdout:228: finishing write tid 2 to smithi18925896-17 2024-09-18T10:15:24.392 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-18T10:15:24.389+0000 7f14c6a1a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 22138) UID: 0 2024-09-18T10:15:24.468 INFO:tasks.rados.rados.0.smithi189.stdout:228: finishing write tid 3 to smithi18925896-17 2024-09-18T10:15:24.491 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-18T10:15:24.489+0000 7f578c52f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22134) UID: 0 2024-09-18T10:15:24.539 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-18T10:15:24.539 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-18T10:15:24.540 INFO:teuthology.orchestra.run.smithi189.stdout: "num_ops": 0 2024-09-18T10:15:24.540 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-18T10:15:24.549 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-09-18T10:15:24.569 INFO:tasks.rados.rados.0.smithi189.stdout:228: finishing write tid 4 to smithi18925896-17 2024-09-18T10:15:24.569 INFO:tasks.rados.rados.0.smithi189.stdout:228: finishing write tid 5 to smithi18925896-17 2024-09-18T10:15:24.591 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-18T10:15:24.589+0000 7f1981873700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22132) UID: 0 2024-09-18T10:15:24.651 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-18T10:15:24.652 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-18T10:15:24.652 INFO:teuthology.orchestra.run.smithi189.stdout: "complaint_time": 30, 2024-09-18T10:15:24.652 INFO:teuthology.orchestra.run.smithi189.stdout: "num_blocked_ops": 0 2024-09-18T10:15:24.652 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-18T10:15:24.660 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-09-18T10:15:24.705 INFO:tasks.ceph.osd.15.smithi189.stderr:2024-09-18T10:15:24.702+0000 7fd0e725e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 22401) UID: 0 2024-09-18T10:15:24.757 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-18T10:15:24.758 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 20, 2024-09-18T10:15:24.758 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 600, 2024-09-18T10:15:24.758 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [ 2024-09-18T10:15:24.758 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.758 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:146 3.es0 3:78aad6f5:::smithi18925896-49:head [append 0~442368,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:24.758 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:51.117496+0000", 2024-09-18T10:15:24.758 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 33.636673983999998, 2024-09-18T10:15:24.758 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.61437150200000001, 2024-09-18T10:15:24.758 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:24.759 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:24.759 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:24.759 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:24.759 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:24.759 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 146 2024-09-18T10:15:24.759 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.759 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:24.759 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.759 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:24.759 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.117496+0000", 2024-09-18T10:15:24.759 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:24.759 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.760 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.760 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:24.760 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.117496+0000", 2024-09-18T10:15:24.760 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:24.760 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.760 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.760 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:24.760 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.117493+0000", 2024-09-18T10:15:24.760 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999976 2024-09-18T10:15:24.760 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.760 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.760 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:24.760 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.117787+0000", 2024-09-18T10:15:24.760 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00029378100000000001 2024-09-18T10:15:24.761 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.761 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.761 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:24.761 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.117820+0000", 2024-09-18T10:15:24.761 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3073e-05 2024-09-18T10:15:24.761 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.761 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.761 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:24.761 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.117834+0000", 2024-09-18T10:15:24.761 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3369e-05 2024-09-18T10:15:24.761 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.761 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.761 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:24.761 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.127090+0000", 2024-09-18T10:15:24.761 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0092563769999999997 2024-09-18T10:15:24.762 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.762 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.762 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:24.762 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.127435+0000", 2024-09-18T10:15:24.762 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00034522300000000003 2024-09-18T10:15:24.762 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.762 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.762 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:24.762 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.129115+0000", 2024-09-18T10:15:24.762 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0016792980000000001 2024-09-18T10:15:24.762 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.762 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.762 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:24.762 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.430505+0000", 2024-09-18T10:15:24.762 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.30139002300000001 2024-09-18T10:15:24.762 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.763 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.763 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:24.763 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.731753+0000", 2024-09-18T10:15:24.763 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.30124798800000002 2024-09-18T10:15:24.763 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.763 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.763 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:24.763 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.731867+0000", 2024-09-18T10:15:24.763 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000114764 2024-09-18T10:15:24.763 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:24.763 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:24.763 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:24.763 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.763 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.763 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:147 3.es0 3:78aad6f5:::smithi18925896-49:head [append 0~376832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:24.764 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:51.122092+0000", 2024-09-18T10:15:24.764 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 33.632077566, 2024-09-18T10:15:24.764 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.61047882200000003, 2024-09-18T10:15:24.764 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:24.764 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:24.764 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:24.764 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:24.764 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:24.764 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 147 2024-09-18T10:15:24.764 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.764 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:24.764 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.764 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:24.764 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122092+0000", 2024-09-18T10:15:24.764 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:24.765 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.765 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.765 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:24.765 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122092+0000", 2024-09-18T10:15:24.765 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:24.765 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.765 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.765 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:24.765 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122091+0000", 2024-09-18T10:15:24.765 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:24.765 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.765 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.765 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:24.766 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122504+0000", 2024-09-18T10:15:24.766 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00041322100000000002 2024-09-18T10:15:24.766 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.766 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.766 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:24.766 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122527+0000", 2024-09-18T10:15:24.766 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3504999999999999e-05 2024-09-18T10:15:24.766 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.766 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.766 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:24.766 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122537+0000", 2024-09-18T10:15:24.766 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.8460000000000003e-06 2024-09-18T10:15:24.766 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.766 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.767 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:24.767 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.327415+0000", 2024-09-18T10:15:24.767 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20487799700000001 2024-09-18T10:15:24.767 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.767 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.767 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:24.767 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.327614+0000", 2024-09-18T10:15:24.767 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00019855400000000001 2024-09-18T10:15:24.767 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.767 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.767 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:24.767 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.328920+0000", 2024-09-18T10:15:24.767 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001306263 2024-09-18T10:15:24.767 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.768 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.768 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:24.768 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.430570+0000", 2024-09-18T10:15:24.768 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.101650508 2024-09-18T10:15:24.768 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.768 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.768 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:24.768 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.732477+0000", 2024-09-18T10:15:24.768 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.30190700100000001 2024-09-18T10:15:24.768 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.768 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.768 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:24.768 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.732571+0000", 2024-09-18T10:15:24.768 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.3580999999999997e-05 2024-09-18T10:15:24.769 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:24.769 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:24.769 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:24.769 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.769 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.769 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:148 3.es0 3:78aad6f5:::smithi18925896-49:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:24.769 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:51.122547+0000", 2024-09-18T10:15:24.769 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 33.631622473999997, 2024-09-18T10:15:24.769 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.91462763400000002, 2024-09-18T10:15:24.769 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:24.769 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:24.769 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:24.769 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:24.769 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:24.769 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 148 2024-09-18T10:15:24.770 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.770 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:24.770 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.770 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:24.770 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122547+0000", 2024-09-18T10:15:24.770 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:24.770 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.770 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.770 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:24.770 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122547+0000", 2024-09-18T10:15:24.770 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:24.770 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.771 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.771 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:24.771 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122546+0000", 2024-09-18T10:15:24.771 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:15:24.771 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.771 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.771 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:24.771 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122552+0000", 2024-09-18T10:15:24.771 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3569999999999997e-06 2024-09-18T10:15:24.771 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.772 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.772 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:24.772 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122560+0000", 2024-09-18T10:15:24.772 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.191e-06 2024-09-18T10:15:24.772 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.772 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.772 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:24.772 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122569+0000", 2024-09-18T10:15:24.772 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.8230000000000001e-06 2024-09-18T10:15:24.772 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.772 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.772 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:24.772 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.834492+0000", 2024-09-18T10:15:24.773 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.71192345599999995 2024-09-18T10:15:24.773 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.773 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.773 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:24.773 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.834669+0000", 2024-09-18T10:15:24.773 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000177035 2024-09-18T10:15:24.773 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.773 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.773 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:24.773 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.835442+0000", 2024-09-18T10:15:24.773 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00077327799999999996 2024-09-18T10:15:24.773 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.774 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.774 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:24.774 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.936574+0000", 2024-09-18T10:15:24.774 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.101131163 2024-09-18T10:15:24.774 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.774 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.774 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:24.774 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.037028+0000", 2024-09-18T10:15:24.774 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100454026 2024-09-18T10:15:24.774 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.774 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.774 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:24.774 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.037175+0000", 2024-09-18T10:15:24.774 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000147305 2024-09-18T10:15:24.775 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:24.775 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:24.775 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:24.775 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.775 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.775 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:149 3.es0 3:78aad6f5:::smithi18925896-49:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:24.775 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:51.122575+0000", 2024-09-18T10:15:24.775 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 33.631595265999998, 2024-09-18T10:15:24.775 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.016813872, 2024-09-18T10:15:24.775 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:24.775 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:15:24.775 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:24.775 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:24.776 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:24.776 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 149 2024-09-18T10:15:24.776 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.776 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:24.776 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.776 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:24.776 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122575+0000", 2024-09-18T10:15:24.776 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:24.776 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.776 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.776 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:24.776 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122575+0000", 2024-09-18T10:15:24.776 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:24.776 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.777 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.777 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:24.777 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122574+0000", 2024-09-18T10:15:24.777 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:15:24.777 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.777 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.777 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:24.777 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122579+0000", 2024-09-18T10:15:24.777 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3650000000000003e-06 2024-09-18T10:15:24.777 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.777 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.777 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:24.777 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122587+0000", 2024-09-18T10:15:24.777 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.3000000000000002e-06 2024-09-18T10:15:24.777 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.778 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.778 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:24.778 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122592+0000", 2024-09-18T10:15:24.778 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9080000000000003e-06 2024-09-18T10:15:24.778 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.778 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.778 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:24.778 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.935931+0000", 2024-09-18T10:15:24.778 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.81333878199999998 2024-09-18T10:15:24.778 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.778 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.778 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for rw locks", 2024-09-18T10:15:24.778 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.936216+0000", 2024-09-18T10:15:24.778 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00028463400000000003 2024-09-18T10:15:24.778 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.778 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.779 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:24.779 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.038007+0000", 2024-09-18T10:15:24.779 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.101791539 2024-09-18T10:15:24.779 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.779 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.779 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:24.779 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.038254+0000", 2024-09-18T10:15:24.779 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000246932 2024-09-18T10:15:24.779 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.779 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.779 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:24.779 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.139388+0000", 2024-09-18T10:15:24.779 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10113417199999999 2024-09-18T10:15:24.779 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:24.779 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:24.779 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:24.780 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.780 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.780 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpRead(3.es0 36/32 ECSubRead(tid=3, to_read={3:7255ddf4:::smithi18925896-39:head=0,4096,0}, subchunks={3:7255ddf4:::smithi18925896-39:head=[0,1]}, attrs_to_read=))", 2024-09-18T10:15:24.780 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:51.126838+0000", 2024-09-18T10:15:24.780 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 33.627331843, 2024-09-18T10:15:24.780 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.707585825, 2024-09-18T10:15:24.780 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:24.780 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:24.780 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:24.780 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.780 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:24.780 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.126838+0000", 2024-09-18T10:15:24.780 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:24.780 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.781 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.781 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:24.781 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.126838+0000", 2024-09-18T10:15:24.781 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:24.781 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.781 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.781 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:24.781 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.126838+0000", 2024-09-18T10:15:24.781 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:24.781 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.781 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.781 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:24.781 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.126838+0000", 2024-09-18T10:15:24.782 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:24.782 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.782 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.782 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:24.782 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.127004+0000", 2024-09-18T10:15:24.782 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00016626499999999999 2024-09-18T10:15:24.782 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.782 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.782 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:24.782 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.227100+0000", 2024-09-18T10:15:24.782 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.1000955 2024-09-18T10:15:24.782 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.782 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.782 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:24.783 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.834083+0000", 2024-09-18T10:15:24.783 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.60698300100000002 2024-09-18T10:15:24.783 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.783 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.783 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:24.783 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.834424+0000", 2024-09-18T10:15:24.783 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000341059 2024-09-18T10:15:24.783 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:24.783 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:24.783 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:24.783 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.783 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.783 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpReadReply(3.es0 36/32 ECSubReadReply(tid=3, attrs_read=0))", 2024-09-18T10:15:24.784 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:51.214895+0000", 2024-09-18T10:15:24.784 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 33.539275166000003, 2024-09-18T10:15:24.784 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.61857800100000004, 2024-09-18T10:15:24.784 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:24.784 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:24.784 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:24.784 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.784 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:24.784 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.214895+0000", 2024-09-18T10:15:24.784 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:24.784 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.784 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.784 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:24.785 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.214895+0000", 2024-09-18T10:15:24.785 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:24.785 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.785 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.785 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:24.785 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.214893+0000", 2024-09-18T10:15:24.785 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:24.785 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.785 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.785 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:24.785 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.214927+0000", 2024-09-18T10:15:24.785 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3841999999999999e-05 2024-09-18T10:15:24.785 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.785 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.786 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:24.786 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.214953+0000", 2024-09-18T10:15:24.786 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.6519000000000001e-05 2024-09-18T10:15:24.786 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.786 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.786 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:24.786 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.214971+0000", 2024-09-18T10:15:24.786 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7535999999999999e-05 2024-09-18T10:15:24.786 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.786 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.786 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:24.786 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.833397+0000", 2024-09-18T10:15:24.786 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.61842642599999997 2024-09-18T10:15:24.786 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.787 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.787 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:24.787 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.833473+0000", 2024-09-18T10:15:24.787 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.5621e-05 2024-09-18T10:15:24.787 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:24.787 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:24.787 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:24.787 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.787 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.787 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpReadReply(3.es0 36/32 ECSubReadReply(tid=3, attrs_read=0))", 2024-09-18T10:15:24.787 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:51.227596+0000", 2024-09-18T10:15:24.787 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 33.526573614999997, 2024-09-18T10:15:24.787 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.60597977400000003, 2024-09-18T10:15:24.787 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:24.787 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:24.787 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:24.788 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.788 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:24.788 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.227596+0000", 2024-09-18T10:15:24.788 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:24.788 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.788 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.788 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:24.788 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.227596+0000", 2024-09-18T10:15:24.788 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:24.788 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.788 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.788 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:24.788 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.227595+0000", 2024-09-18T10:15:24.788 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:15:24.789 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.789 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.789 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:24.789 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.227605+0000", 2024-09-18T10:15:24.789 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.7040000000000006e-06 2024-09-18T10:15:24.789 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.789 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.789 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:24.789 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.227621+0000", 2024-09-18T10:15:24.789 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6107999999999999e-05 2024-09-18T10:15:24.789 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.789 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.789 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:24.790 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.227637+0000", 2024-09-18T10:15:24.790 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5282999999999999e-05 2024-09-18T10:15:24.790 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.790 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.790 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:24.790 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.833511+0000", 2024-09-18T10:15:24.790 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.60587403699999998 2024-09-18T10:15:24.790 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.790 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.790 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:24.790 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.833576+0000", 2024-09-18T10:15:24.790 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.5392e-05 2024-09-18T10:15:24.790 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:24.790 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:24.791 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:24.791 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.791 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.791 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.es0 36/32 ECSubWriteReply(tid=4, last_complete=36'3, committed=1, applied=1))", 2024-09-18T10:15:24.791 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:51.233227+0000", 2024-09-18T10:15:24.791 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 33.520942601000002, 2024-09-18T10:15:24.791 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.39820466900000001, 2024-09-18T10:15:24.791 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:24.792 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:24.792 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:24.792 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.792 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:24.792 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.233227+0000", 2024-09-18T10:15:24.792 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:24.792 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.792 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.792 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:24.792 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.233227+0000", 2024-09-18T10:15:24.792 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:24.792 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.792 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.792 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:24.792 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.233226+0000", 2024-09-18T10:15:24.793 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:15:24.793 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.793 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.793 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:24.793 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.233232+0000", 2024-09-18T10:15:24.793 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9370000000000001e-06 2024-09-18T10:15:24.793 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.793 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.793 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:24.793 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.233248+0000", 2024-09-18T10:15:24.793 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5574999999999999e-05 2024-09-18T10:15:24.793 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.793 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.793 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:24.793 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.233254+0000", 2024-09-18T10:15:24.794 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.3659999999999997e-06 2024-09-18T10:15:24.794 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.794 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.794 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:24.794 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.631420+0000", 2024-09-18T10:15:24.794 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.39816591699999998 2024-09-18T10:15:24.794 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.794 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.794 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:24.794 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.631432+0000", 2024-09-18T10:15:24.794 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1737e-05 2024-09-18T10:15:24.794 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:24.794 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:24.794 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:24.795 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.795 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.795 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.es0 36/32 ECSubWriteReply(tid=4, last_complete=36'3, committed=1, applied=1))", 2024-09-18T10:15:24.795 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:51.315941+0000", 2024-09-18T10:15:24.795 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 33.438228359, 2024-09-18T10:15:24.795 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.315333435, 2024-09-18T10:15:24.795 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:24.795 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:24.795 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:24.795 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.796 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:24.796 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.315941+0000", 2024-09-18T10:15:24.796 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:24.796 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.796 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.796 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:24.796 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.315941+0000", 2024-09-18T10:15:24.796 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:24.796 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.796 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.797 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:24.797 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.315940+0000", 2024-09-18T10:15:24.797 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:15:24.797 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.797 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.797 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:24.797 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.315950+0000", 2024-09-18T10:15:24.797 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.2329999999999995e-06 2024-09-18T10:15:24.797 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.797 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.797 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:24.797 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.315968+0000", 2024-09-18T10:15:24.797 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.8067000000000002e-05 2024-09-18T10:15:24.797 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.798 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.798 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:24.798 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.315977+0000", 2024-09-18T10:15:24.798 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.6420000000000002e-06 2024-09-18T10:15:24.798 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.798 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.798 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:24.798 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.631262+0000", 2024-09-18T10:15:24.798 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.31528474000000001 2024-09-18T10:15:24.798 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.798 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.798 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:24.798 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.631275+0000", 2024-09-18T10:15:24.798 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2799999999999999e-05 2024-09-18T10:15:24.799 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:24.799 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:24.799 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:24.799 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.799 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.799 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.es0 36/32 ECSubWriteReply(tid=5, last_complete=36'4, committed=1, applied=1))", 2024-09-18T10:15:24.799 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:51.331048+0000", 2024-09-18T10:15:24.799 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 33.423122174, 2024-09-18T10:15:24.799 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40123123900000002, 2024-09-18T10:15:24.799 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:24.799 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:24.799 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:24.799 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.800 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:24.800 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.331048+0000", 2024-09-18T10:15:24.800 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:24.800 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.800 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.800 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:24.800 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.331048+0000", 2024-09-18T10:15:24.800 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:24.800 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.800 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.800 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:24.800 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.331046+0000", 2024-09-18T10:15:24.800 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:24.800 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.801 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.801 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:24.801 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.331057+0000", 2024-09-18T10:15:24.801 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1683e-05 2024-09-18T10:15:24.801 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.801 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.801 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:24.801 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.331090+0000", 2024-09-18T10:15:24.801 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2576000000000001e-05 2024-09-18T10:15:24.801 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.801 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.801 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:24.801 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.331107+0000", 2024-09-18T10:15:24.801 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7382999999999999e-05 2024-09-18T10:15:24.802 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.802 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.802 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:24.802 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.732241+0000", 2024-09-18T10:15:24.802 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40113375200000001 2024-09-18T10:15:24.802 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.802 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.802 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:24.802 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.732279+0000", 2024-09-18T10:15:24.802 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7719000000000003e-05 2024-09-18T10:15:24.802 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:24.802 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:24.802 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:24.803 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.803 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.803 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.es0 36/32 ECSubWriteReply(tid=5, last_complete=36'4, committed=1, applied=1))", 2024-09-18T10:15:24.803 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:51.331214+0000", 2024-09-18T10:15:24.803 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 33.422955780000002, 2024-09-18T10:15:24.803 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40084109800000001, 2024-09-18T10:15:24.803 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:24.803 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:24.803 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:24.803 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.803 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:24.803 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.331214+0000", 2024-09-18T10:15:24.803 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:24.804 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.804 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.804 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:24.804 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.331214+0000", 2024-09-18T10:15:24.804 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:24.804 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.804 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.804 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:24.804 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.331213+0000", 2024-09-18T10:15:24.804 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:15:24.804 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.804 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.805 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:24.805 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.331220+0000", 2024-09-18T10:15:24.805 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.1369999999999996e-06 2024-09-18T10:15:24.805 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.805 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.805 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:24.805 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.331235+0000", 2024-09-18T10:15:24.805 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4849e-05 2024-09-18T10:15:24.805 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.805 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.805 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:24.805 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.331243+0000", 2024-09-18T10:15:24.805 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.157e-06 2024-09-18T10:15:24.806 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.806 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.806 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:24.806 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.732010+0000", 2024-09-18T10:15:24.806 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40076711300000001 2024-09-18T10:15:24.806 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.806 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.806 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:24.806 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.732055+0000", 2024-09-18T10:15:24.806 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4821999999999999e-05 2024-09-18T10:15:24.806 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:24.806 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:24.806 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:24.806 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.807 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.807 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:346 3.es0 3:7255ddf4:::smithi18925896-39:head [append 0~770048] snapc b=[b,4] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-18T10:15:24.807 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:02.898102+0000", 2024-09-18T10:15:24.807 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 21.856068317999998, 2024-09-18T10:15:24.807 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.31123798899999999, 2024-09-18T10:15:24.807 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:24.807 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:24.807 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:24.807 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:24.807 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:24.807 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 346 2024-09-18T10:15:24.807 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.807 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:24.807 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.807 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:24.808 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.898102+0000", 2024-09-18T10:15:24.808 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:24.808 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.808 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.808 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:24.808 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.898102+0000", 2024-09-18T10:15:24.808 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:24.808 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.808 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.808 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:24.808 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.898099+0000", 2024-09-18T10:15:24.808 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999971 2024-09-18T10:15:24.808 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.808 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.808 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:24.808 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.898914+0000", 2024-09-18T10:15:24.809 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00081561999999999997 2024-09-18T10:15:24.809 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.809 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.809 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:24.809 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.898942+0000", 2024-09-18T10:15:24.809 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.8019999999999999e-05 2024-09-18T10:15:24.809 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.809 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.809 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:24.809 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.898962+0000", 2024-09-18T10:15:24.809 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0026000000000001e-05 2024-09-18T10:15:24.809 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.809 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.809 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:24.809 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.902525+0000", 2024-09-18T10:15:24.810 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0035628090000000001 2024-09-18T10:15:24.810 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.810 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.810 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:24.810 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.902844+0000", 2024-09-18T10:15:24.810 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00031879400000000001 2024-09-18T10:15:24.810 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.810 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.810 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:24.810 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.905818+0000", 2024-09-18T10:15:24.810 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0029740600000000002 2024-09-18T10:15:24.810 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.810 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.810 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:24.810 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.008488+0000", 2024-09-18T10:15:24.810 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10266977100000001 2024-09-18T10:15:24.811 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.811 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.811 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:24.811 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.209150+0000", 2024-09-18T10:15:24.811 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20066203799999999 2024-09-18T10:15:24.811 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.811 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.811 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:24.811 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.209339+0000", 2024-09-18T10:15:24.811 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00018959099999999999 2024-09-18T10:15:24.811 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:24.811 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:24.811 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:24.811 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.811 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.811 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:347 3.es0 3:7255ddf4:::smithi18925896-39:head [append 0~376832] snapc b=[b,4] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-18T10:15:24.812 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:02.898980+0000", 2024-09-18T10:15:24.812 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 21.855189756000001, 2024-09-18T10:15:24.812 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.50845237799999998, 2024-09-18T10:15:24.812 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:24.812 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:24.812 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:24.812 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:24.812 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:24.812 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 347 2024-09-18T10:15:24.812 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.812 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:24.812 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.812 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:24.812 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.898980+0000", 2024-09-18T10:15:24.813 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:24.813 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.813 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.813 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:24.813 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.898980+0000", 2024-09-18T10:15:24.813 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:24.813 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.813 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.813 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:24.813 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.898978+0000", 2024-09-18T10:15:24.813 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:24.813 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.813 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.813 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:24.813 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.899378+0000", 2024-09-18T10:15:24.814 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000399465 2024-09-18T10:15:24.814 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.814 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.814 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:24.814 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.899397+0000", 2024-09-18T10:15:24.814 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9219999999999999e-05 2024-09-18T10:15:24.814 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.814 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.814 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:24.814 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.899410+0000", 2024-09-18T10:15:24.814 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2486e-05 2024-09-18T10:15:24.814 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.814 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.814 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:24.815 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.103248+0000", 2024-09-18T10:15:24.815 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20383807200000001 2024-09-18T10:15:24.815 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.815 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.815 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:24.815 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.103447+0000", 2024-09-18T10:15:24.815 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00019970400000000001 2024-09-18T10:15:24.815 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.815 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.815 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:24.815 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.104894+0000", 2024-09-18T10:15:24.815 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001447083 2024-09-18T10:15:24.815 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.816 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.816 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:24.816 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.209378+0000", 2024-09-18T10:15:24.816 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.104483942 2024-09-18T10:15:24.816 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.816 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.816 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:24.816 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.407310+0000", 2024-09-18T10:15:24.816 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.197931406 2024-09-18T10:15:24.816 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.816 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.816 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:24.816 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.407432+0000", 2024-09-18T10:15:24.817 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00012266999999999999 2024-09-18T10:15:24.817 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:24.817 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:24.817 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:24.817 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.817 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.817 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:348 3.es0 3:7255ddf4:::smithi18925896-39:head [setxattr _header (56) in=7b] snapc b=[b,4] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-18T10:15:24.817 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:02.899423+0000", 2024-09-18T10:15:24.817 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 21.854746328000001, 2024-09-18T10:15:24.817 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.50859120400000002, 2024-09-18T10:15:24.817 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:24.817 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:24.817 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:24.817 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:24.817 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:24.818 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 348 2024-09-18T10:15:24.818 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.818 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:24.818 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.818 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:24.818 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.899423+0000", 2024-09-18T10:15:24.818 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:24.818 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.818 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.818 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:24.818 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.899423+0000", 2024-09-18T10:15:24.818 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:24.818 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.818 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.819 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:24.819 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.899422+0000", 2024-09-18T10:15:24.819 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:24.819 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.819 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.819 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:24.819 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.899431+0000", 2024-09-18T10:15:24.819 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.5960000000000004e-06 2024-09-18T10:15:24.819 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.819 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.819 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:24.819 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.899451+0000", 2024-09-18T10:15:24.819 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0089999999999999e-05 2024-09-18T10:15:24.819 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.820 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.820 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:24.820 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.899461+0000", 2024-09-18T10:15:24.820 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0373e-05 2024-09-18T10:15:24.820 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.820 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.820 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:24.820 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.106191+0000", 2024-09-18T10:15:24.820 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20672991700000001 2024-09-18T10:15:24.820 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.820 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.820 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:24.820 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.106370+0000", 2024-09-18T10:15:24.820 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00017880500000000001 2024-09-18T10:15:24.821 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.821 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.821 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:24.821 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.107175+0000", 2024-09-18T10:15:24.821 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00080489699999999995 2024-09-18T10:15:24.821 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.821 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.821 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:24.821 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.209383+0000", 2024-09-18T10:15:24.821 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10220804999999999 2024-09-18T10:15:24.821 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.821 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.821 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:24.821 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.407801+0000", 2024-09-18T10:15:24.822 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.19841824199999999 2024-09-18T10:15:24.822 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.822 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.822 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:24.822 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.408015+0000", 2024-09-18T10:15:24.822 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00021364100000000001 2024-09-18T10:15:24.822 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:24.822 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:24.822 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:24.822 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.822 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.822 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:349 3.es0 3:7255ddf4:::smithi18925896-39:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e47)", 2024-09-18T10:15:24.822 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:02.899474+0000", 2024-09-18T10:15:24.822 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 21.854696117, 2024-09-18T10:15:24.822 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.81294787599999996, 2024-09-18T10:15:24.823 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:24.823 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:15:24.823 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:24.823 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:24.823 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:24.823 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 349 2024-09-18T10:15:24.823 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.823 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:24.823 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.823 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:24.823 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.899474+0000", 2024-09-18T10:15:24.823 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:24.823 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.823 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.824 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:24.824 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.899474+0000", 2024-09-18T10:15:24.824 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:24.824 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.824 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.824 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:24.824 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.899472+0000", 2024-09-18T10:15:24.824 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:24.824 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.824 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.824 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:24.824 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.899480+0000", 2024-09-18T10:15:24.824 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.0129999999999993e-06 2024-09-18T10:15:24.825 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.825 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.825 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:24.825 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.899501+0000", 2024-09-18T10:15:24.825 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0959999999999999e-05 2024-09-18T10:15:24.825 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.825 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.825 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:24.825 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.899513+0000", 2024-09-18T10:15:24.825 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1736000000000001e-05 2024-09-18T10:15:24.825 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.825 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.825 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:24.825 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.208436+0000", 2024-09-18T10:15:24.826 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.30892317800000002 2024-09-18T10:15:24.826 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.826 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.826 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for rw locks", 2024-09-18T10:15:24.826 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.208762+0000", 2024-09-18T10:15:24.826 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00032566099999999998 2024-09-18T10:15:24.826 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.826 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.826 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:24.826 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.408786+0000", 2024-09-18T10:15:24.826 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.200024389 2024-09-18T10:15:24.826 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.826 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.826 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:24.826 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.409088+0000", 2024-09-18T10:15:24.827 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000301272 2024-09-18T10:15:24.827 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.827 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.827 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:24.827 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.712422+0000", 2024-09-18T10:15:24.827 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.303333937 2024-09-18T10:15:24.827 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:24.827 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:24.827 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:24.827 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.827 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.827 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.17s1 57/40 ECSubWrite(tid=106, reqid=client.4632.0:441, at_version=57'22, trim_to=0'0, roll_forward_to=53'18))", 2024-09-18T10:15:24.827 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:12.963189+0000", 2024-09-18T10:15:24.827 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 11.790980401000001, 2024-09-18T10:15:24.827 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.404840753, 2024-09-18T10:15:24.827 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:24.828 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:24.828 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:24.828 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.828 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:24.828 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:12.963189+0000", 2024-09-18T10:15:24.828 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:24.828 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.828 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.828 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:24.828 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:12.963189+0000", 2024-09-18T10:15:24.828 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:24.828 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.828 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.828 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:24.828 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:12.963189+0000", 2024-09-18T10:15:24.828 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:15:24.829 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.829 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.829 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:24.829 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:12.963511+0000", 2024-09-18T10:15:24.829 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00032195800000000001 2024-09-18T10:15:24.829 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.829 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.829 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:24.829 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:12.963536+0000", 2024-09-18T10:15:24.829 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5795e-05 2024-09-18T10:15:24.829 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.829 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.829 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:24.830 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:12.963548+0000", 2024-09-18T10:15:24.830 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1235e-05 2024-09-18T10:15:24.830 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.830 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.830 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:24.830 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.164194+0000", 2024-09-18T10:15:24.830 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.200646044 2024-09-18T10:15:24.830 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.830 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.830 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:24.830 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.164208+0000", 2024-09-18T10:15:24.830 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4107000000000001e-05 2024-09-18T10:15:24.830 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.830 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.830 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:24.831 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.367962+0000", 2024-09-18T10:15:24.831 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20375375800000001 2024-09-18T10:15:24.831 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.831 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.831 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:24.831 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.368030+0000", 2024-09-18T10:15:24.831 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.8664999999999994e-05 2024-09-18T10:15:24.831 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:24.831 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:24.831 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:24.831 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.831 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.831 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.17s1 57/40 ECSubWrite(tid=107, reqid=client.4632.0:442, at_version=57'23, trim_to=0'0, roll_forward_to=53'18))", 2024-09-18T10:15:24.832 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:12.963562+0000", 2024-09-18T10:15:24.832 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 11.790607681999999, 2024-09-18T10:15:24.832 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40594254800000001, 2024-09-18T10:15:24.832 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:24.832 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:24.832 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:24.832 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.832 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:24.832 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:12.963562+0000", 2024-09-18T10:15:24.832 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:24.832 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.832 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.833 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:24.833 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:12.963562+0000", 2024-09-18T10:15:24.833 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:24.833 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.833 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.833 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:24.833 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:12.963561+0000", 2024-09-18T10:15:24.833 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:24.833 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.833 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.833 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:24.833 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:12.963869+0000", 2024-09-18T10:15:24.833 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00030785799999999999 2024-09-18T10:15:24.833 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.834 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.834 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:24.834 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:12.963886+0000", 2024-09-18T10:15:24.834 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6582000000000001e-05 2024-09-18T10:15:24.834 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.834 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.834 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:24.834 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.063959+0000", 2024-09-18T10:15:24.834 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10007302699999999 2024-09-18T10:15:24.834 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.834 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.834 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:24.834 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.166049+0000", 2024-09-18T10:15:24.835 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.102089845 2024-09-18T10:15:24.835 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.835 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.835 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:24.835 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.166064+0000", 2024-09-18T10:15:24.835 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5287000000000001e-05 2024-09-18T10:15:24.835 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.835 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.835 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:24.835 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.369452+0000", 2024-09-18T10:15:24.835 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20338777 2024-09-18T10:15:24.835 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.835 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.835 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:24.836 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.369505+0000", 2024-09-18T10:15:24.836 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.2868000000000003e-05 2024-09-18T10:15:24.836 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:24.836 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:24.836 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:24.836 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.836 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.836 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.17s1 57/40 ECSubWrite(tid=108, reqid=client.4632.0:443, at_version=57'24, trim_to=0'0, roll_forward_to=53'18))", 2024-09-18T10:15:24.836 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:13.063971+0000", 2024-09-18T10:15:24.836 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 11.690198365000001, 2024-09-18T10:15:24.836 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.30555857800000003, 2024-09-18T10:15:24.836 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:24.836 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:24.836 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:24.836 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.837 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:24.837 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.063971+0000", 2024-09-18T10:15:24.837 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:24.837 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.837 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.837 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:24.837 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.063971+0000", 2024-09-18T10:15:24.837 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:24.837 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.837 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.837 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:24.837 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.063971+0000", 2024-09-18T10:15:24.837 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:24.837 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.838 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.838 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:24.838 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.064015+0000", 2024-09-18T10:15:24.838 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3918000000000002e-05 2024-09-18T10:15:24.838 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.838 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.838 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:24.838 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.064025+0000", 2024-09-18T10:15:24.838 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.9029999999999999e-06 2024-09-18T10:15:24.838 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.838 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.838 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:24.838 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.064030+0000", 2024-09-18T10:15:24.839 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.6910000000000002e-06 2024-09-18T10:15:24.839 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.839 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.839 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:24.839 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.166744+0000", 2024-09-18T10:15:24.839 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10271409300000001 2024-09-18T10:15:24.839 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.839 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.839 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:24.839 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.166755+0000", 2024-09-18T10:15:24.839 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0125e-05 2024-09-18T10:15:24.839 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.839 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.839 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:24.840 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.369511+0000", 2024-09-18T10:15:24.840 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20275652299999999 2024-09-18T10:15:24.840 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.840 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.840 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:24.840 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.369530+0000", 2024-09-18T10:15:24.840 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.893e-05 2024-09-18T10:15:24.840 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:24.840 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:24.840 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:24.840 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.840 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.840 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.17s1 57/40 ECSubWrite(tid=109, reqid=client.4632.0:444, at_version=57'25, trim_to=0'0, roll_forward_to=53'18))", 2024-09-18T10:15:24.840 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:13.064037+0000", 2024-09-18T10:15:24.841 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 11.690132855, 2024-09-18T10:15:24.841 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.30551024300000001, 2024-09-18T10:15:24.841 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:24.841 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:24.841 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:24.841 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.841 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:24.841 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.064037+0000", 2024-09-18T10:15:24.841 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:24.841 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.841 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.841 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:24.841 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.064037+0000", 2024-09-18T10:15:24.841 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:24.842 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.842 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.842 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:24.842 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.064037+0000", 2024-09-18T10:15:24.842 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:24.842 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.842 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.842 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:24.842 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.064053+0000", 2024-09-18T10:15:24.842 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5964000000000002e-05 2024-09-18T10:15:24.842 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.842 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.842 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:24.842 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.064062+0000", 2024-09-18T10:15:24.843 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.1479999999999995e-06 2024-09-18T10:15:24.843 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.843 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.843 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:24.843 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.064071+0000", 2024-09-18T10:15:24.843 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.5459999999999997e-06 2024-09-18T10:15:24.843 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.843 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.843 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:24.843 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.266847+0000", 2024-09-18T10:15:24.843 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20277574100000001 2024-09-18T10:15:24.843 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.843 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.843 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:24.843 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.266860+0000", 2024-09-18T10:15:24.844 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3264999999999999e-05 2024-09-18T10:15:24.844 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.844 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.844 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:24.844 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.369533+0000", 2024-09-18T10:15:24.844 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.102672676 2024-09-18T10:15:24.844 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.844 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.844 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:24.844 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.369547+0000", 2024-09-18T10:15:24.844 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4319e-05 2024-09-18T10:15:24.844 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:24.844 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:24.844 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:24.845 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.845 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.845 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:495 3.es0 3:7255ddf4:::smithi18925896-39:head [rmxattr R^`al]TqifatYOiOU]p`ZegfXmNsXVaZcdd]b]YX in=40b,rmxattr TNQYR^jgR`pPaYitmrcliV[dttobncTntcfTlQkt in=40b,rmxattr W_aq[RVsaUpQtX_RpV^Vpqrg`QgQViPeVaRqZSmf in=40b,rmxattr aYfgr^q[\\aPg_aZsakfUfsdhYbra^QXjlkrpdY\\\\ in=40b,rmxattr dgXPi in=5b,rmxattr p]OS_Znl[dNa^qTn`WW[j\\dtgbNac_bijYmg in=36b,rmxattr tZ]grhlaq]kUO[k^d^dr``XNio^^Oaebpj\\[Ytdq in=40b] snapc 19=[19,18,16,12,11,10,f,e] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T10:15:24.845 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:17.889335+0000", 2024-09-18T10:15:24.845 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 6.8648344489999999, 2024-09-18T10:15:24.845 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.42846536899999998, 2024-09-18T10:15:24.845 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:24.845 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:24.845 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:24.845 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:24.845 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:24.845 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 495 2024-09-18T10:15:24.845 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.845 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:24.846 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.846 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:24.846 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889335+0000", 2024-09-18T10:15:24.846 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:24.846 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.846 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.846 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:24.846 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889335+0000", 2024-09-18T10:15:24.846 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:24.846 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.846 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.846 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:24.846 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889333+0000", 2024-09-18T10:15:24.846 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999976 2024-09-18T10:15:24.847 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.847 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.847 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:24.847 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889351+0000", 2024-09-18T10:15:24.847 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7795000000000002e-05 2024-09-18T10:15:24.847 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.847 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.847 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:24.847 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889383+0000", 2024-09-18T10:15:24.847 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2740999999999997e-05 2024-09-18T10:15:24.847 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.847 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.847 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:24.847 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889396+0000", 2024-09-18T10:15:24.848 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2926e-05 2024-09-18T10:15:24.848 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.848 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.848 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:24.848 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.989725+0000", 2024-09-18T10:15:24.848 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10032845999999999 2024-09-18T10:15:24.848 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.848 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.848 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "op must wait for map", 2024-09-18T10:15:24.848 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.989751+0000", 2024-09-18T10:15:24.848 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5819000000000001e-05 2024-09-18T10:15:24.848 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.848 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.848 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:24.848 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.999619+0000", 2024-09-18T10:15:24.849 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0098683920000000001 2024-09-18T10:15:24.849 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.849 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.849 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:24.849 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.999776+0000", 2024-09-18T10:15:24.849 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000157336 2024-09-18T10:15:24.849 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.849 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.849 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:24.849 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.001374+0000", 2024-09-18T10:15:24.849 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0015982419999999999 2024-09-18T10:15:24.849 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.849 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.849 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:24.850 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.003908+0000", 2024-09-18T10:15:24.850 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0025331870000000001 2024-09-18T10:15:24.850 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.850 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.850 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:24.850 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.317708+0000", 2024-09-18T10:15:24.850 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.31380064800000002 2024-09-18T10:15:24.850 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:24.850 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:24.850 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:24.850 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.317801+0000", 2024-09-18T10:15:24.850 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.2427000000000001e-05 2024-09-18T10:15:24.850 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:24.850 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:24.850 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:24.851 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:24.851 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:24.851 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-18T10:15:24.852 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_ops_in_flight 2024-09-18T10:15:24.853 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-18T10:15:24.849+0000 7f578c52f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22134) UID: 0 2024-09-18T10:15:24.861 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T10:15:24.873 INFO:tasks.rados.rados.0.smithi189.stdout:228: finishing write tid 6 to smithi18925896-17 2024-09-18T10:15:24.874 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 17 v 92 (ObjNum 101 snap 19 seq_num 101) dirty exists 2024-09-18T10:15:24.874 INFO:tasks.rados.rados.0.smithi189.stdout:228: left oid 17 (ObjNum 101 snap 19 seq_num 101) 2024-09-18T10:15:24.942 INFO:tasks.rados.rados.0.smithi189.stdout:228: done (2 left) 2024-09-18T10:15:24.942 INFO:tasks.rados.rados.0.smithi189.stdout:229: done (1 left) 2024-09-18T10:15:24.942 INFO:tasks.rados.rados.0.smithi189.stdout:233: done (0 left) 2024-09-18T10:15:24.942 INFO:tasks.rados.rados.0.smithi189.stdout:234: copy_from oid 48 from oid 47 current snap is 20 2024-09-18T10:15:24.942 INFO:tasks.rados.rados.0.smithi189.stdout:235: read oid 34 snap -1 2024-09-18T10:15:24.942 INFO:tasks.rados.rados.0.smithi189.stdout:235: expect (ObjNum 33 snap 0 seq_num 33) 2024-09-18T10:15:24.943 INFO:tasks.rados.rados.0.smithi189.stdout:236: snap_create 2024-09-18T10:15:24.946 INFO:teuthology.orchestra.run.smithi029.stderr:reweighted osd.15 to 1 (10000) 2024-09-18T10:15:24.953 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-18T10:15:24.950+0000 7f14dcc5b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22398) UID: 0 2024-09-18T10:15:24.961 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-18T10:15:24.961 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T10:15:24.965 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-18T10:15:24.966 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-18T10:15:24.966 INFO:teuthology.orchestra.run.smithi189.stdout: "num_ops": 0 2024-09-18T10:15:24.966 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-18T10:15:24.974 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_blocked_ops 2024-09-18T10:15:25.009 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-18T10:15:25.033 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-18T10:15:25.039 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-18T10:15:25.053 INFO:tasks.ceph.osd.6.smithi154.stderr:2024-09-18T10:15:25.051+0000 7f0abc9e1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22089) UID: 0 2024-09-18T10:15:25.054 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-18T10:15:25.092 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-18T10:15:25.116 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-18T10:15:25.123 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-18T10:15:25.123 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-18T10:15:25.123 INFO:teuthology.orchestra.run.smithi189.stdout: "complaint_time": 30, 2024-09-18T10:15:25.123 INFO:teuthology.orchestra.run.smithi189.stdout: "num_blocked_ops": 0 2024-09-18T10:15:25.123 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-18T10:15:25.133 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_historic_ops 2024-09-18T10:15:25.137 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-18T10:15:25.154 INFO:tasks.ceph.osd.15.smithi189.stderr:2024-09-18T10:15:25.150+0000 7fd0e725e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 22401) UID: 0 2024-09-18T10:15:25.159 INFO:teuthology.orchestra.run.smithi029.stdout:osd.6: {} 2024-09-18T10:15:25.182 INFO:teuthology.orchestra.run.smithi029.stdout:osd.7: {} 2024-09-18T10:15:25.202 INFO:teuthology.orchestra.run.smithi029.stdout:osd.8: {} 2024-09-18T10:15:25.225 INFO:teuthology.orchestra.run.smithi029.stdout:osd.9: {} 2024-09-18T10:15:25.234 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-18T10:15:25.234 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 20, 2024-09-18T10:15:25.234 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 600, 2024-09-18T10:15:25.234 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [ 2024-09-18T10:15:25.234 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.234 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4257.0:19 1.0 e24/23 1:30306672:devicehealth::main.db.0000000000000000:head v 24'9, mlcod=24'7)", 2024-09-18T10:15:25.234 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:33.346192+0000", 2024-09-18T10:15:25.234 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 51.883840628999998, 2024-09-18T10:15:25.234 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20178458699999999, 2024-09-18T10:15:25.234 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:25.235 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:25.235 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:25.235 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.235 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:25.235 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:33.346192+0000", 2024-09-18T10:15:25.235 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:25.235 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.235 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.235 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:25.235 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:33.346192+0000", 2024-09-18T10:15:25.235 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:25.235 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.235 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.235 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:25.236 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:33.346190+0000", 2024-09-18T10:15:25.236 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:25.236 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.236 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.236 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:25.236 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:33.346206+0000", 2024-09-18T10:15:25.236 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6257e-05 2024-09-18T10:15:25.236 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.236 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.236 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:25.236 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:33.346237+0000", 2024-09-18T10:15:25.236 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0471999999999999e-05 2024-09-18T10:15:25.236 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.237 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.237 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:25.237 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:33.446377+0000", 2024-09-18T10:15:25.237 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10014044499999999 2024-09-18T10:15:25.237 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.237 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.237 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:25.237 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:33.546722+0000", 2024-09-18T10:15:25.238 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100344593 2024-09-18T10:15:25.238 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.238 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.238 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:25.238 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:33.546787+0000", 2024-09-18T10:15:25.238 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.5347999999999998e-05 2024-09-18T10:15:25.238 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.238 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.238 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:25.238 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:33.547891+0000", 2024-09-18T10:15:25.238 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001104379 2024-09-18T10:15:25.238 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.238 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.239 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:25.239 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:33.547976+0000", 2024-09-18T10:15:25.239 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.5023000000000002e-05 2024-09-18T10:15:25.239 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:25.239 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:25.239 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:25.239 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.239 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.239 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4257.0:48 1.0 e26/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 26'30, mlcod=26'22)", 2024-09-18T10:15:25.240 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:35.294144+0000", 2024-09-18T10:15:25.240 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 49.935888486000003, 2024-09-18T10:15:25.240 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20346468400000001, 2024-09-18T10:15:25.240 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:25.240 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:25.240 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:25.240 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.240 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:25.240 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:35.294144+0000", 2024-09-18T10:15:25.240 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:25.240 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.241 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.241 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:25.241 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:35.294144+0000", 2024-09-18T10:15:25.241 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:25.241 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.241 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.241 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:25.241 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:35.294142+0000", 2024-09-18T10:15:25.242 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:25.242 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.242 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.242 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:25.242 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:35.294156+0000", 2024-09-18T10:15:25.242 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4059e-05 2024-09-18T10:15:25.242 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.242 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.242 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:25.242 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:35.294209+0000", 2024-09-18T10:15:25.242 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.2164999999999999e-05 2024-09-18T10:15:25.242 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.242 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.243 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:25.243 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:35.394305+0000", 2024-09-18T10:15:25.243 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10009605000000001 2024-09-18T10:15:25.243 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.243 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.243 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:25.243 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:35.394422+0000", 2024-09-18T10:15:25.243 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000117093 2024-09-18T10:15:25.243 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.243 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.243 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:25.243 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:35.394460+0000", 2024-09-18T10:15:25.244 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8037999999999997e-05 2024-09-18T10:15:25.244 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.244 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.244 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:25.244 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:35.497437+0000", 2024-09-18T10:15:25.244 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.102977206 2024-09-18T10:15:25.244 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.244 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.244 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:25.244 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:35.497609+0000", 2024-09-18T10:15:25.245 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00017165299999999999 2024-09-18T10:15:25.245 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:25.245 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:25.245 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:25.245 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.245 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.245 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4257.0:96 1.0 e29/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 29'70, mlcod=29'68)", 2024-09-18T10:15:25.245 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:39.427232+0000", 2024-09-18T10:15:25.245 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 45.802800361999999, 2024-09-18T10:15:25.245 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20408627300000001, 2024-09-18T10:15:25.245 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:25.246 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:25.246 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:25.246 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.246 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:25.246 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.427232+0000", 2024-09-18T10:15:25.246 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:25.246 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.246 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.246 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:25.246 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.427232+0000", 2024-09-18T10:15:25.246 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:25.246 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.246 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.247 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:25.247 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.427230+0000", 2024-09-18T10:15:25.247 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:25.247 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.247 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.247 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:25.247 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.427399+0000", 2024-09-18T10:15:25.247 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000168787 2024-09-18T10:15:25.247 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.247 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.247 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:25.247 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.427431+0000", 2024-09-18T10:15:25.247 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2372000000000001e-05 2024-09-18T10:15:25.247 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.248 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.248 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:25.248 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.427444+0000", 2024-09-18T10:15:25.248 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2303000000000001e-05 2024-09-18T10:15:25.248 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.248 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.248 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:25.248 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.427580+0000", 2024-09-18T10:15:25.248 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000136019 2024-09-18T10:15:25.248 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.248 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.248 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:25.248 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.427622+0000", 2024-09-18T10:15:25.249 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.2551e-05 2024-09-18T10:15:25.249 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.249 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.249 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:25.249 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.631238+0000", 2024-09-18T10:15:25.249 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20361580100000001 2024-09-18T10:15:25.249 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.249 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.250 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:25.250 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.631318+0000", 2024-09-18T10:15:25.250 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.0370000000000005e-05 2024-09-18T10:15:25.250 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:25.250 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:25.250 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:25.250 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.250 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.250 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4257.0:97 1.0 e29/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 29'71, mlcod=29'68)", 2024-09-18T10:15:25.250 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:39.428800+0000", 2024-09-18T10:15:25.250 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 45.801232419999998, 2024-09-18T10:15:25.250 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.202566529, 2024-09-18T10:15:25.250 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:25.251 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:25.251 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:25.251 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.251 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:25.251 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.428800+0000", 2024-09-18T10:15:25.251 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:25.251 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.252 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.252 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:25.252 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.428800+0000", 2024-09-18T10:15:25.252 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:25.252 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.252 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.252 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:25.252 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.428798+0000", 2024-09-18T10:15:25.252 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:25.252 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.253 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.253 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:25.253 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.428814+0000", 2024-09-18T10:15:25.253 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5262999999999999e-05 2024-09-18T10:15:25.253 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.253 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.253 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:25.253 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.428836+0000", 2024-09-18T10:15:25.253 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.2096e-05 2024-09-18T10:15:25.253 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.253 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.253 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:25.254 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.428847+0000", 2024-09-18T10:15:25.254 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1272e-05 2024-09-18T10:15:25.254 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.254 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.254 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:25.254 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.529081+0000", 2024-09-18T10:15:25.254 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100233853 2024-09-18T10:15:25.255 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.255 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.255 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:25.255 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.529115+0000", 2024-09-18T10:15:25.255 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3862000000000002e-05 2024-09-18T10:15:25.255 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.255 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.255 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:25.255 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.631329+0000", 2024-09-18T10:15:25.255 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.102214465 2024-09-18T10:15:25.255 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.255 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.256 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:25.256 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.631367+0000", 2024-09-18T10:15:25.256 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7450999999999998e-05 2024-09-18T10:15:25.256 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:25.256 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:25.256 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:25.256 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.256 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.257 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4257.0:98 1.0 e29/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 29'72, mlcod=29'68)", 2024-09-18T10:15:25.257 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:39.429995+0000", 2024-09-18T10:15:25.257 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 45.800037197000002, 2024-09-18T10:15:25.257 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20141884500000001, 2024-09-18T10:15:25.257 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:25.257 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:25.257 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:25.257 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.257 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:25.257 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.429995+0000", 2024-09-18T10:15:25.257 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:25.257 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.257 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.258 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:25.258 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.429995+0000", 2024-09-18T10:15:25.258 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:25.258 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.258 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.258 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:25.258 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.429994+0000", 2024-09-18T10:15:25.258 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:25.258 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.258 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.258 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:25.258 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.430008+0000", 2024-09-18T10:15:25.258 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4185999999999999e-05 2024-09-18T10:15:25.259 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.259 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.259 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:25.259 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.430036+0000", 2024-09-18T10:15:25.259 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7878999999999999e-05 2024-09-18T10:15:25.259 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.259 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.259 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:25.259 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.430048+0000", 2024-09-18T10:15:25.259 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1725999999999999e-05 2024-09-18T10:15:25.259 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.259 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.259 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:25.260 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.629731+0000", 2024-09-18T10:15:25.260 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.19968341100000001 2024-09-18T10:15:25.260 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.260 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.260 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:25.260 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.629764+0000", 2024-09-18T10:15:25.260 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3572999999999999e-05 2024-09-18T10:15:25.260 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.260 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.260 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:25.260 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.631372+0000", 2024-09-18T10:15:25.260 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0016074279999999999 2024-09-18T10:15:25.261 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.261 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.261 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:25.261 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.631414+0000", 2024-09-18T10:15:25.261 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.2191999999999999e-05 2024-09-18T10:15:25.261 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:25.261 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:25.261 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:25.261 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.261 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.261 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4257.0:99 1.0 e29/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 29'73, mlcod=29'68)", 2024-09-18T10:15:25.261 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:39.431146+0000", 2024-09-18T10:15:25.261 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 45.798886473000003, 2024-09-18T10:15:25.262 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.200928938, 2024-09-18T10:15:25.262 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:25.262 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:25.262 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:25.262 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.262 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:25.262 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.431146+0000", 2024-09-18T10:15:25.262 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:25.262 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.262 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.262 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:25.262 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.431146+0000", 2024-09-18T10:15:25.262 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:25.263 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.263 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.263 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:25.263 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.431144+0000", 2024-09-18T10:15:25.263 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:25.263 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.263 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.263 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:25.263 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.431218+0000", 2024-09-18T10:15:25.263 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.3577000000000004e-05 2024-09-18T10:15:25.263 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.264 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.264 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:25.264 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.431250+0000", 2024-09-18T10:15:25.264 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2268999999999999e-05 2024-09-18T10:15:25.264 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.264 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.264 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:25.264 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.431262+0000", 2024-09-18T10:15:25.264 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2053e-05 2024-09-18T10:15:25.264 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.264 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.265 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:25.265 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.630328+0000", 2024-09-18T10:15:25.265 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.19906532900000001 2024-09-18T10:15:25.265 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.265 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.265 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:25.265 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.630359+0000", 2024-09-18T10:15:25.265 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1606000000000002e-05 2024-09-18T10:15:25.265 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.266 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.266 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:25.266 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.632015+0000", 2024-09-18T10:15:25.266 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0016557900000000001 2024-09-18T10:15:25.266 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.266 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.266 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:25.266 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.632075+0000", 2024-09-18T10:15:25.267 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9904000000000001e-05 2024-09-18T10:15:25.267 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:25.267 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:25.267 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:25.267 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.267 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.267 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4257.0:100 1.0 e29/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 29'74, mlcod=29'68)", 2024-09-18T10:15:25.267 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:39.432631+0000", 2024-09-18T10:15:25.268 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 45.797401383999997, 2024-09-18T10:15:25.268 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20014804899999999, 2024-09-18T10:15:25.268 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:25.269 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:25.269 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:25.269 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.269 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:25.269 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.432631+0000", 2024-09-18T10:15:25.270 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:25.270 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.270 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.270 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:25.270 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.432631+0000", 2024-09-18T10:15:25.270 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:25.270 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.270 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.270 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:25.270 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.432630+0000", 2024-09-18T10:15:25.270 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:25.270 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.270 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.271 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:25.271 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.432643+0000", 2024-09-18T10:15:25.271 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3525e-05 2024-09-18T10:15:25.271 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.271 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.271 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:25.271 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.432674+0000", 2024-09-18T10:15:25.271 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1183000000000003e-05 2024-09-18T10:15:25.271 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.272 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.272 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:25.272 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.432690+0000", 2024-09-18T10:15:25.272 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5665999999999999e-05 2024-09-18T10:15:25.272 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.272 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.272 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:25.272 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.631457+0000", 2024-09-18T10:15:25.272 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.19876732899999999 2024-09-18T10:15:25.272 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.273 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.273 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:25.273 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.631488+0000", 2024-09-18T10:15:25.273 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1059000000000002e-05 2024-09-18T10:15:25.273 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.273 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.273 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:25.273 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.632717+0000", 2024-09-18T10:15:25.273 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0012285219999999999 2024-09-18T10:15:25.273 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.273 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.274 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:25.274 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.632779+0000", 2024-09-18T10:15:25.274 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.2328000000000004e-05 2024-09-18T10:15:25.274 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:25.274 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:25.274 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:25.274 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.274 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.274 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4257.0:103 1.0 e29/23 1:30306672:devicehealth::main.db.0000000000000000:head v 29'77, mlcod=29'73)", 2024-09-18T10:15:25.274 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:40.036738+0000", 2024-09-18T10:15:25.274 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 45.193294414, 2024-09-18T10:15:25.274 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20132596899999999, 2024-09-18T10:15:25.274 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:25.275 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:25.275 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:25.275 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.275 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:25.275 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.036738+0000", 2024-09-18T10:15:25.275 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:25.275 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.275 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.275 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:25.275 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.036738+0000", 2024-09-18T10:15:25.275 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:25.276 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.276 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.276 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:25.276 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.036737+0000", 2024-09-18T10:15:25.276 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:25.276 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.276 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.276 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:25.276 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.036804+0000", 2024-09-18T10:15:25.276 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.7178e-05 2024-09-18T10:15:25.276 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.276 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.276 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:25.277 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.036821+0000", 2024-09-18T10:15:25.277 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7153e-05 2024-09-18T10:15:25.277 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.277 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.277 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:25.277 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.136954+0000", 2024-09-18T10:15:25.277 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100132572 2024-09-18T10:15:25.277 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.277 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.278 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:25.278 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.137090+0000", 2024-09-18T10:15:25.278 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000136332 2024-09-18T10:15:25.278 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.278 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.278 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:25.279 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.137124+0000", 2024-09-18T10:15:25.279 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3458999999999999e-05 2024-09-18T10:15:25.279 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.279 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.280 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:25.280 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.237999+0000", 2024-09-18T10:15:25.280 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100875385 2024-09-18T10:15:25.280 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.280 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.280 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:25.281 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.238064+0000", 2024-09-18T10:15:25.281 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.5086999999999995e-05 2024-09-18T10:15:25.281 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:25.281 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:25.281 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:25.281 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.281 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.282 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4257.0:113 1.0 e29/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 29'83, mlcod=29'80)", 2024-09-18T10:15:25.282 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:40.801786+0000", 2024-09-18T10:15:25.282 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 44.428246909999999, 2024-09-18T10:15:25.282 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.203408961, 2024-09-18T10:15:25.282 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:25.282 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:25.282 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:25.283 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.283 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:25.283 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.801786+0000", 2024-09-18T10:15:25.283 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:25.283 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.283 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.283 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:25.283 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.801786+0000", 2024-09-18T10:15:25.283 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:25.283 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.283 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.283 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:25.283 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.801784+0000", 2024-09-18T10:15:25.284 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:25.284 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.284 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.284 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:25.284 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.801801+0000", 2024-09-18T10:15:25.284 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7223000000000001e-05 2024-09-18T10:15:25.284 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.284 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.284 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:25.284 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.801823+0000", 2024-09-18T10:15:25.285 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1469000000000001e-05 2024-09-18T10:15:25.285 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.285 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.285 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:25.285 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.801835+0000", 2024-09-18T10:15:25.285 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2033e-05 2024-09-18T10:15:25.285 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.285 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.285 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:25.285 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.902049+0000", 2024-09-18T10:15:25.285 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100213659 2024-09-18T10:15:25.285 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.286 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.286 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:25.286 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.902082+0000", 2024-09-18T10:15:25.286 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3668999999999999e-05 2024-09-18T10:15:25.286 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.286 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.286 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:25.286 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.005131+0000", 2024-09-18T10:15:25.286 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.103048379 2024-09-18T10:15:25.286 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.286 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.286 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:25.286 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.005195+0000", 2024-09-18T10:15:25.286 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.3805000000000003e-05 2024-09-18T10:15:25.287 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:25.287 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:25.287 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:25.287 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.287 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.287 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4257.0:114 1.0 e29/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 29'84, mlcod=29'80)", 2024-09-18T10:15:25.287 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:40.802657+0000", 2024-09-18T10:15:25.287 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 44.427375775999998, 2024-09-18T10:15:25.287 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20256802600000001, 2024-09-18T10:15:25.287 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:25.287 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:25.287 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:25.287 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.287 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:25.287 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.802657+0000", 2024-09-18T10:15:25.288 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:25.288 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.288 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.288 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:25.288 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.802657+0000", 2024-09-18T10:15:25.288 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:25.288 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.288 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.288 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:25.288 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.802656+0000", 2024-09-18T10:15:25.288 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:15:25.288 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.288 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.288 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:25.289 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.802824+0000", 2024-09-18T10:15:25.289 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000168784 2024-09-18T10:15:25.289 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.289 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.289 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:25.289 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.802844+0000", 2024-09-18T10:15:25.289 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0058999999999999e-05 2024-09-18T10:15:25.289 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.289 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.289 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:25.289 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.802857+0000", 2024-09-18T10:15:25.289 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2163e-05 2024-09-18T10:15:25.290 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.290 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.290 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:25.290 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.902531+0000", 2024-09-18T10:15:25.290 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.099674591000000007 2024-09-18T10:15:25.290 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.290 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.290 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:25.290 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.902559+0000", 2024-09-18T10:15:25.290 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7483000000000001e-05 2024-09-18T10:15:25.291 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.291 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.291 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:25.291 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.005198+0000", 2024-09-18T10:15:25.291 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.102638853 2024-09-18T10:15:25.291 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.291 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.291 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:25.291 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.005225+0000", 2024-09-18T10:15:25.291 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7206e-05 2024-09-18T10:15:25.291 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:25.292 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:25.292 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:25.292 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.292 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.292 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4257.0:115 1.0 e29/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 29'85, mlcod=29'80)", 2024-09-18T10:15:25.292 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:40.803806+0000", 2024-09-18T10:15:25.293 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 44.426226389, 2024-09-18T10:15:25.293 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.201443711, 2024-09-18T10:15:25.293 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:25.293 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:25.293 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:25.293 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.293 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:25.294 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.803806+0000", 2024-09-18T10:15:25.294 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:25.294 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.294 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.294 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:25.294 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.803806+0000", 2024-09-18T10:15:25.295 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:25.295 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.295 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.295 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:25.295 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.803804+0000", 2024-09-18T10:15:25.295 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:25.296 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.296 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.296 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:25.296 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.803818+0000", 2024-09-18T10:15:25.296 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3477e-05 2024-09-18T10:15:25.296 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.297 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.297 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:25.297 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.803851+0000", 2024-09-18T10:15:25.297 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3086000000000001e-05 2024-09-18T10:15:25.297 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.297 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.298 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:25.298 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.803867+0000", 2024-09-18T10:15:25.298 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5885000000000001e-05 2024-09-18T10:15:25.298 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.298 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.298 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:25.299 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.903206+0000", 2024-09-18T10:15:25.299 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.099338990000000002 2024-09-18T10:15:25.299 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.299 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.299 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:25.299 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.903253+0000", 2024-09-18T10:15:25.300 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.7419000000000001e-05 2024-09-18T10:15:25.300 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.300 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.300 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:25.300 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.005227+0000", 2024-09-18T10:15:25.300 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.101973199 2024-09-18T10:15:25.301 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.301 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.301 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:25.301 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.005250+0000", 2024-09-18T10:15:25.301 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3261999999999999e-05 2024-09-18T10:15:25.301 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:25.302 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:25.302 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:25.302 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.302 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.302 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4257.0:116 1.0 e29/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 29'86, mlcod=29'80)", 2024-09-18T10:15:25.303 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:40.804730+0000", 2024-09-18T10:15:25.303 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 44.42530241, 2024-09-18T10:15:25.303 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20068456000000001, 2024-09-18T10:15:25.303 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:25.303 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:25.303 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:25.304 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.304 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:25.304 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.804730+0000", 2024-09-18T10:15:25.304 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:25.304 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.304 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.305 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:25.305 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.804730+0000", 2024-09-18T10:15:25.305 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:25.305 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.305 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.305 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:25.306 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.804729+0000", 2024-09-18T10:15:25.306 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:25.306 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.306 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.306 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:25.306 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.804739+0000", 2024-09-18T10:15:25.307 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0669000000000001e-05 2024-09-18T10:15:25.307 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.308 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.308 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:25.308 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.804762+0000", 2024-09-18T10:15:25.308 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3062999999999999e-05 2024-09-18T10:15:25.308 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.309 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.309 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:25.309 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.804776+0000", 2024-09-18T10:15:25.309 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3273e-05 2024-09-18T10:15:25.309 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.309 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.310 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:25.310 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.003889+0000", 2024-09-18T10:15:25.310 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.19911304099999999 2024-09-18T10:15:25.310 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.310 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.310 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:25.310 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.003922+0000", 2024-09-18T10:15:25.310 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3621999999999998e-05 2024-09-18T10:15:25.310 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.310 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.310 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:25.311 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.005347+0000", 2024-09-18T10:15:25.311 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0014251050000000001 2024-09-18T10:15:25.311 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.311 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.311 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:25.311 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.005415+0000", 2024-09-18T10:15:25.311 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.7154000000000001e-05 2024-09-18T10:15:25.311 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:25.311 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:25.311 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:25.311 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.312 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.312 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4257.0:117 1.0 e29/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 29'87, mlcod=29'80)", 2024-09-18T10:15:25.312 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:40.805727+0000", 2024-09-18T10:15:25.312 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 44.424305150999999, 2024-09-18T10:15:25.312 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.200946191, 2024-09-18T10:15:25.312 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:25.312 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:25.312 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:25.312 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.312 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:25.312 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.805727+0000", 2024-09-18T10:15:25.312 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:25.312 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.313 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.313 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:25.313 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.805727+0000", 2024-09-18T10:15:25.313 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:25.313 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.313 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.313 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:25.313 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.805726+0000", 2024-09-18T10:15:25.313 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:25.313 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.313 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.313 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:25.313 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.805783+0000", 2024-09-18T10:15:25.314 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.7068000000000003e-05 2024-09-18T10:15:25.314 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.314 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.314 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:25.314 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.805817+0000", 2024-09-18T10:15:25.314 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4471999999999998e-05 2024-09-18T10:15:25.314 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.314 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.314 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:25.314 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.805833+0000", 2024-09-18T10:15:25.314 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6313000000000001e-05 2024-09-18T10:15:25.314 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.314 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.314 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:25.315 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.004481+0000", 2024-09-18T10:15:25.315 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.198647823 2024-09-18T10:15:25.315 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.315 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.315 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:25.315 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.004520+0000", 2024-09-18T10:15:25.315 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8992000000000002e-05 2024-09-18T10:15:25.315 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.315 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.315 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:25.315 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.006585+0000", 2024-09-18T10:15:25.315 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0020647389999999999 2024-09-18T10:15:25.315 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.315 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.316 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:25.316 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.006673+0000", 2024-09-18T10:15:25.316 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.8443999999999995e-05 2024-09-18T10:15:25.316 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:25.316 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:25.316 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:25.316 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.316 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.317 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4257.0:128 1.0 e29/23 1:30306672:devicehealth::main.db.0000000000000000:head v 29'98, mlcod=29'95)", 2024-09-18T10:15:25.317 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:41.222806+0000", 2024-09-18T10:15:25.317 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 44.007226510000002, 2024-09-18T10:15:25.317 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20200015099999999, 2024-09-18T10:15:25.317 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:25.317 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:25.317 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:25.317 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.317 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:25.317 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.222806+0000", 2024-09-18T10:15:25.317 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:25.317 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.318 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.318 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:25.318 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.222806+0000", 2024-09-18T10:15:25.318 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:25.318 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.318 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.318 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:25.318 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.222804+0000", 2024-09-18T10:15:25.318 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:25.318 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.318 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.318 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:25.318 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.222885+0000", 2024-09-18T10:15:25.319 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.0140000000000002e-05 2024-09-18T10:15:25.319 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.319 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.319 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:25.319 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.222915+0000", 2024-09-18T10:15:25.319 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0168999999999999e-05 2024-09-18T10:15:25.319 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.319 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.319 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:25.319 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.323060+0000", 2024-09-18T10:15:25.319 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100145475 2024-09-18T10:15:25.319 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.319 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.319 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:25.320 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.423304+0000", 2024-09-18T10:15:25.320 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100243924 2024-09-18T10:15:25.320 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.320 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.320 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:25.320 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.423338+0000", 2024-09-18T10:15:25.320 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3819000000000003e-05 2024-09-18T10:15:25.320 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.320 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.320 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:25.320 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.424724+0000", 2024-09-18T10:15:25.320 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0013862690000000001 2024-09-18T10:15:25.320 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.321 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.321 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:25.321 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.424806+0000", 2024-09-18T10:15:25.321 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.1901000000000001e-05 2024-09-18T10:15:25.321 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:25.321 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:25.321 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:25.321 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.321 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.321 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.4s1 41/40 ECSubWrite(tid=17, reqid=osd.8.0:17, at_version=41'10, trim_to=0'0, roll_forward_to=0'0))", 2024-09-18T10:15:25.321 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:57.445149+0000", 2024-09-18T10:15:25.321 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 27.784883948000001, 2024-09-18T10:15:25.321 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20394425099999999, 2024-09-18T10:15:25.322 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:25.322 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:25.322 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:25.322 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.322 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:25.322 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.445149+0000", 2024-09-18T10:15:25.322 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:25.322 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.322 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.322 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:25.322 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.445149+0000", 2024-09-18T10:15:25.323 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:25.323 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.323 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.323 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:25.323 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.445147+0000", 2024-09-18T10:15:25.323 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:25.323 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.323 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.323 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:25.323 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.445221+0000", 2024-09-18T10:15:25.323 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.4140999999999994e-05 2024-09-18T10:15:25.323 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.323 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.324 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:25.324 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.445260+0000", 2024-09-18T10:15:25.324 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8998999999999997e-05 2024-09-18T10:15:25.324 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.324 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.324 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:25.324 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.545386+0000", 2024-09-18T10:15:25.324 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10012621300000001 2024-09-18T10:15:25.324 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.324 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.324 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:25.324 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.645963+0000", 2024-09-18T10:15:25.325 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10057662000000001 2024-09-18T10:15:25.325 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.325 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.325 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:25.325 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.645988+0000", 2024-09-18T10:15:25.325 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5449e-05 2024-09-18T10:15:25.325 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.325 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.325 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:25.325 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.649028+0000", 2024-09-18T10:15:25.325 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0030390840000000001 2024-09-18T10:15:25.325 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.326 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.326 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:25.326 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.649093+0000", 2024-09-18T10:15:25.326 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.5258000000000004e-05 2024-09-18T10:15:25.326 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:25.326 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:25.326 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:25.326 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.326 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.326 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.6s6 41/40 ECSubWrite(tid=31, reqid=client.4632.0:249, at_version=41'13, trim_to=0'0, roll_forward_to=0'0))", 2024-09-18T10:15:25.326 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:57.550336+0000", 2024-09-18T10:15:25.326 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 27.679696909, 2024-09-18T10:15:25.326 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.203590574, 2024-09-18T10:15:25.327 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:25.327 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:25.327 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:25.327 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.327 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:25.327 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.550336+0000", 2024-09-18T10:15:25.327 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:25.327 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.327 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.327 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:25.327 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.550336+0000", 2024-09-18T10:15:25.327 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:25.327 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.327 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.328 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:25.328 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.550329+0000", 2024-09-18T10:15:25.328 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999933 2024-09-18T10:15:25.328 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.328 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.328 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:25.328 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.550932+0000", 2024-09-18T10:15:25.328 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00060276999999999998 2024-09-18T10:15:25.328 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.328 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.328 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:25.329 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.550955+0000", 2024-09-18T10:15:25.329 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.2884999999999999e-05 2024-09-18T10:15:25.329 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.329 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.329 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:25.329 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.651052+0000", 2024-09-18T10:15:25.329 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100097174 2024-09-18T10:15:25.329 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.329 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.329 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:25.329 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.651160+0000", 2024-09-18T10:15:25.329 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000108483 2024-09-18T10:15:25.329 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.330 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.330 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:25.330 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.651172+0000", 2024-09-18T10:15:25.330 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1790999999999999e-05 2024-09-18T10:15:25.330 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.330 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.330 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:25.330 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.753875+0000", 2024-09-18T10:15:25.330 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10270337 2024-09-18T10:15:25.330 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.330 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.330 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:25.330 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.753926+0000", 2024-09-18T10:15:25.330 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0767999999999999e-05 2024-09-18T10:15:25.331 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:25.331 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:25.331 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:25.331 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.331 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.331 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.8s4 42/40 ECSubWrite(tid=133, reqid=osd.6.0:132, at_version=0'0, trim_to=0'0, roll_forward_to=41'13))", 2024-09-18T10:15:25.331 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:58.049763+0000", 2024-09-18T10:15:25.331 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 27.180269172999999, 2024-09-18T10:15:25.331 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.21446268399999999, 2024-09-18T10:15:25.331 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:25.331 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:25.331 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:25.331 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.332 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:25.332 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:58.049763+0000", 2024-09-18T10:15:25.332 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:25.332 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.332 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.332 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:25.332 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:58.049763+0000", 2024-09-18T10:15:25.332 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:25.333 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.333 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.333 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:25.333 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:58.049763+0000", 2024-09-18T10:15:25.333 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:25.333 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.333 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.333 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:25.333 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:58.049787+0000", 2024-09-18T10:15:25.333 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3819e-05 2024-09-18T10:15:25.333 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.334 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.334 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:25.334 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:58.049817+0000", 2024-09-18T10:15:25.334 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0749e-05 2024-09-18T10:15:25.334 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.334 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.334 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:25.334 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:58.049836+0000", 2024-09-18T10:15:25.334 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9182999999999999e-05 2024-09-18T10:15:25.334 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.334 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.334 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:25.334 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:58.163409+0000", 2024-09-18T10:15:25.335 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.11357204899999999 2024-09-18T10:15:25.335 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.335 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.335 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:25.335 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:58.163424+0000", 2024-09-18T10:15:25.335 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5403000000000001e-05 2024-09-18T10:15:25.335 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.335 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.335 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:25.335 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:58.264161+0000", 2024-09-18T10:15:25.335 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100736942 2024-09-18T10:15:25.335 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.335 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.336 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:25.336 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:58.264226+0000", 2024-09-18T10:15:25.336 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.5085000000000005e-05 2024-09-18T10:15:25.336 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:25.336 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:25.336 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:25.336 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.336 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.336 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.ds3 47/32 ECSubWrite(tid=56, reqid=client.4632.0:333, at_version=47'9, trim_to=0'0, roll_forward_to=41'6))", 2024-09-18T10:15:25.336 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:02.801124+0000", 2024-09-18T10:15:25.337 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 22.428908015000001, 2024-09-18T10:15:25.337 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20185083000000001, 2024-09-18T10:15:25.337 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:25.337 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:25.337 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:25.337 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.337 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:25.337 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.801124+0000", 2024-09-18T10:15:25.337 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:25.338 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.338 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.338 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:25.338 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.801124+0000", 2024-09-18T10:15:25.338 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:25.338 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.338 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.338 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:25.338 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.801103+0000", 2024-09-18T10:15:25.338 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999785 2024-09-18T10:15:25.338 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.338 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.339 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:25.339 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.801366+0000", 2024-09-18T10:15:25.339 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00026269299999999999 2024-09-18T10:15:25.339 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.339 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.339 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:25.339 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.801395+0000", 2024-09-18T10:15:25.339 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.9572000000000001e-05 2024-09-18T10:15:25.339 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.340 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.340 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:25.340 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.801412+0000", 2024-09-18T10:15:25.340 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6872999999999999e-05 2024-09-18T10:15:25.340 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.340 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.340 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:25.340 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.000543+0000", 2024-09-18T10:15:25.340 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.19913046700000001 2024-09-18T10:15:25.340 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.340 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.340 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:25.340 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.000579+0000", 2024-09-18T10:15:25.341 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6649000000000001e-05 2024-09-18T10:15:25.341 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.341 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.341 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:25.341 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.002922+0000", 2024-09-18T10:15:25.341 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0023426139999999998 2024-09-18T10:15:25.341 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.341 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.341 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:25.341 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.002975+0000", 2024-09-18T10:15:25.341 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3232000000000001e-05 2024-09-18T10:15:25.341 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:25.342 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:25.342 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:25.342 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.342 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.342 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.ds3 47/32 ECSubWrite(tid=57, reqid=client.4632.0:334, at_version=47'10, trim_to=0'0, roll_forward_to=41'6))", 2024-09-18T10:15:25.342 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:02.803374+0000", 2024-09-18T10:15:25.342 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 22.426658103000001, 2024-09-18T10:15:25.342 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20018619500000001, 2024-09-18T10:15:25.342 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:25.342 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:25.342 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:25.343 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.343 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:25.343 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.803374+0000", 2024-09-18T10:15:25.343 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:25.343 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.343 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.343 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:25.343 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.803374+0000", 2024-09-18T10:15:25.343 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:25.343 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.343 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.343 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:25.343 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.803373+0000", 2024-09-18T10:15:25.344 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:15:25.344 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.344 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.344 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:25.344 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.803399+0000", 2024-09-18T10:15:25.344 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5738e-05 2024-09-18T10:15:25.344 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.344 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.344 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:25.344 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.803419+0000", 2024-09-18T10:15:25.344 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9890000000000001e-05 2024-09-18T10:15:25.344 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.344 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.345 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:25.345 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.803430+0000", 2024-09-18T10:15:25.345 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.117e-05 2024-09-18T10:15:25.345 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.345 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.345 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:25.345 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.001843+0000", 2024-09-18T10:15:25.345 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.198412951 2024-09-18T10:15:25.345 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.345 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.345 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:25.345 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.001860+0000", 2024-09-18T10:15:25.345 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.658e-05 2024-09-18T10:15:25.346 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.346 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.346 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:25.346 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.003525+0000", 2024-09-18T10:15:25.346 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0016650160000000001 2024-09-18T10:15:25.346 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.346 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.346 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:25.346 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.003561+0000", 2024-09-18T10:15:25.346 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.5886000000000002e-05 2024-09-18T10:15:25.346 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:25.346 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:25.346 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:25.346 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.347 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.347 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.15s1 63/60 ECSubWrite(tid=54, reqid=client.4632.0:512, at_version=0'0, trim_to=0'0, roll_forward_to=63'4))", 2024-09-18T10:15:25.347 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:19.215356+0000", 2024-09-18T10:15:25.347 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 6.0146761529999999, 2024-09-18T10:15:25.347 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20187551500000001, 2024-09-18T10:15:25.347 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:25.347 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:25.347 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:25.347 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.347 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:25.347 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.215356+0000", 2024-09-18T10:15:25.347 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:25.347 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.347 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.348 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:25.348 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.215356+0000", 2024-09-18T10:15:25.348 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:25.348 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.348 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.348 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:25.348 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.215355+0000", 2024-09-18T10:15:25.348 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:25.348 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.348 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.348 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:25.348 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.215386+0000", 2024-09-18T10:15:25.348 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0809000000000003e-05 2024-09-18T10:15:25.349 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.349 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.349 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:25.349 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.215434+0000", 2024-09-18T10:15:25.349 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.8751999999999997e-05 2024-09-18T10:15:25.349 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.349 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.349 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:25.349 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.315532+0000", 2024-09-18T10:15:25.349 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100097281 2024-09-18T10:15:25.349 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.349 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.349 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:25.350 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.415795+0000", 2024-09-18T10:15:25.350 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10026387 2024-09-18T10:15:25.350 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.350 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.350 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:25.350 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.415823+0000", 2024-09-18T10:15:25.350 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7469000000000001e-05 2024-09-18T10:15:25.350 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.350 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.350 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:25.350 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.417173+0000", 2024-09-18T10:15:25.350 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0013505030000000001 2024-09-18T10:15:25.351 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:25.351 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:25.351 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:25.351 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.417232+0000", 2024-09-18T10:15:25.351 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8421000000000002e-05 2024-09-18T10:15:25.351 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:25.351 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:25.351 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:25.351 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:25.351 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:25.352 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-18T10:15:25.353 INFO:teuthology.orchestra.run.smithi029.stdout:osd.10: {} 2024-09-18T10:15:25.355 INFO:tasks.rados.rados.0.smithi189.stdout:235: expect (ObjNum 33 snap 0 seq_num 33) 2024-09-18T10:15:25.356 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T10:15:25.357 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-18T10:15:25.358 INFO:teuthology.orchestra.run.smithi029.stdout:osd.11: {} 2024-09-18T10:15:25.359 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-09-18T10:15:25.359 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":82,"stamp":"2024-09-18T10:15:25.210194+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":53715507,"num_objects":75,"num_object_clones":32,"num_object_copies":505,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":6,"num_read":264,"num_read_kb":33762,"num_write":463,"num_write_kb":46122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1638400,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":781,"ondisk_log_size":781,"up":200,"acting":200,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":208,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":16,"kb":1509949440,"kb_used":4870192,"kb_used_data":112432,"kb_used_omap":26,"kb_used_meta":4757477,"kb_avail":1505079248,"statfs":{"total":1546188226560,"available":1541201149952,"internally_reserved":0,"allocated":115130368,"data_stored":144936546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":27264,"internal_metadata":4871656832},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,1,2,5,14,1,8,3],"upper_bound":2048},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5062656,"num_objects":1,"num_object_clones":1,"num_object_copies":7,"num_objects_missing_on_primary":-2,"num_objects_missing":-2,"num_objects_degraded":-14,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":9,"num_read_kb":1673,"num_write":10,"num_write_kb":4944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":688128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001558"},"pg_stats":[{"pgid":"3.19","version":"63'64","reported_seq":171,"reported_epoch":64,"state":"active+clean+snaptrim","last_fresh":"2024-09-18T10:15:20.272186+0000","last_change":"2024-09-18T10:15:20.272257+0000","last_active":"2024-09-18T10:15:20.272186+0000","last_peered":"2024-09-18T10:15:20.272186+0000","last_clean":"2024-09-18T10:15:20.272186+0000","last_became_active":"2024-09-18T10:14:57.350904+0000","last_became_peered":"2024-09-18T10:14:57.350904+0000","last_unstale":"2024-09-18T10:15:20.272186+0000","last_undegraded":"2024-09-18T10:15:20.272186+0000","last_fullsized":"2024-09-18T10:15:20.272186+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:14:45.576168+0000","objects_scrubbed":0,"log_size":64,"log_dups_size":0,"ondisk_log_size":64,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.31209502,"stat_sum":{"num_bytes":4595712,"num_objects":8,"num_object_clones":5,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":22,"num_read_kb":7473,"num_write":44,"num_write_kb":5400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,10,11,5,8,3],"acting":[7,2,10,11,5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"13","length":"1"}]},{"pgid":"3.18","version":"60'30","reported_seq":140,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-18T10:15:19.905225+0000","last_change":"2024-09-18T10:15:19.907721+0000","last_active":"2024-09-18T10:15:19.905225+0000","last_peered":"2024-09-18T10:15:19.905225+0000","last_clean":"2024-09-18T10:15:19.905225+0000","last_became_active":"2024-09-18T10:14:57.647465+0000","last_became_peered":"2024-09-18T10:14:57.647465+0000","last_unstale":"2024-09-18T10:15:19.905225+0000","last_undegraded":"2024-09-18T10:15:19.905225+0000","last_fullsized":"2024-09-18T10:15:19.905225+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:14:45.576168+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030463600000000001,"stat_sum":{"num_bytes":3293184,"num_objects":4,"num_object_clones":2,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":11,"num_read_kb":2309,"num_write":24,"num_write_kb":3232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,11,3,15,1,7],"acting":[6,0,11,3,15,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"13","length":"1"},{"start":"19","length":"1"}]},{"pgid":"3.17","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"peering","last_fresh":"2024-09-18T10:15:19.908409+0000","last_change":"2024-09-18T10:15:19.908409+0000","last_active":"0.000000","last_peered":"0.000000","last_clean":"0.000000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-18T10:15:19.908409+0000","last_undegraded":"2024-09-18T10:15:19.908409+0000","last_fullsized":"2024-09-18T10:15:19.908409+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:14:45.576168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:15:53.934915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,13,3,14,2,11,6],"acting":[10,13,3,14,2,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"66'18","reported_seq":53,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-18T10:15:22.238585+0000","last_change":"2024-09-18T10:15:22.238585+0000","last_active":"2024-09-18T10:15:22.238585+0000","last_peered":"2024-09-18T10:15:22.238585+0000","last_clean":"2024-09-18T10:15:22.238585+0000","last_became_active":"2024-09-18T10:15:19.227471+0000","last_became_peered":"2024-09-18T10:15:19.227471+0000","last_unstale":"2024-09-18T10:15:22.238585+0000","last_undegraded":"2024-09-18T10:15:22.238585+0000","last_fullsized":"2024-09-18T10:15:22.238585+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":5,"last_scrub":"36'11","last_scrub_stamp":"2024-09-18T10:15:07.445786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:07.445786+0000","objects_scrubbed":1,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:23.145992+0000","scrub_duration":0.30645798400000002,"objects_trimmed":0,"snaptrim_duration":0.30014816300000002,"stat_sum":{"num_bytes":2621440,"num_objects":2,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":1888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":688128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,6,4,8,2,14,10],"acting":[9,6,4,8,2,14,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"e","length":"1"},{"start":"13","length":"1"},{"start":"18","length":"2"}]},{"pgid":"3.15","version":"65'8","reported_seq":49,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-18T10:15:21.109971+0000","last_change":"2024-09-18T10:15:21.112019+0000","last_active":"2024-09-18T10:15:21.109971+0000","last_peered":"2024-09-18T10:15:21.109971+0000","last_clean":"2024-09-18T10:15:21.109971+0000","last_became_active":"2024-09-18T10:15:17.127771+0000","last_became_peered":"2024-09-18T10:15:17.127771+0000","last_unstale":"2024-09-18T10:15:21.109971+0000","last_undegraded":"2024-09-18T10:15:21.109971+0000","last_fullsized":"2024-09-18T10:15:21.109971+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:14:45.576168+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.20870269999999999,"stat_sum":{"num_bytes":32768,"num_objects":2,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":1,"num_read":2,"num_read_kb":0,"num_write":2,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,15,13,10,1,7,9],"acting":[14,15,13,10,1,7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"13","length":"1"},{"start":"18","length":"2"}]},{"pgid":"3.14","version":"37'8","reported_seq":41,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-18T10:15:20.031743+0000","last_change":"2024-09-18T10:15:20.132377+0000","last_active":"2024-09-18T10:15:20.031743+0000","last_peered":"2024-09-18T10:15:20.031743+0000","last_clean":"2024-09-18T10:15:20.031743+0000","last_became_active":"2024-09-18T10:15:14.767770+0000","last_became_peered":"2024-09-18T10:15:14.767770+0000","last_unstale":"2024-09-18T10:15:20.031743+0000","last_undegraded":"2024-09-18T10:15:20.031743+0000","last_fullsized":"2024-09-18T10:15:20.031743+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":5,"last_scrub":"37'8","last_scrub_stamp":"2024-09-18T10:15:17.041590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:17.041590+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:32.741813+0000","scrub_duration":0.60018000800000004,"objects_trimmed":0,"snaptrim_duration":0.10088322600000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,14,9,10,5,13,4],"acting":[1,14,9,10,5,13,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"13","length":"1"},{"start":"19","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":50,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-18T10:15:19.910703+0000","last_change":"2024-09-18T10:14:37.427722+0000","last_active":"2024-09-18T10:15:19.910703+0000","last_peered":"2024-09-18T10:15:19.910703+0000","last_clean":"2024-09-18T10:15:19.910703+0000","last_became_active":"2024-09-18T10:14:35.348943+0000","last_became_peered":"2024-09-18T10:14:35.348943+0000","last_unstale":"2024-09-18T10:15:19.910703+0000","last_undegraded":"2024-09-18T10:15:19.910703+0000","last_fullsized":"2024-09-18T10:15:19.910703+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:14:34.288596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:03.161727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060442999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,10],"acting":[13,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"3.6","version":"57'32","reported_seq":167,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-18T10:15:23.069165+0000","last_change":"2024-09-18T10:15:21.935648+0000","last_active":"2024-09-18T10:15:23.069165+0000","last_peered":"2024-09-18T10:15:23.069165+0000","last_clean":"2024-09-18T10:15:23.069165+0000","last_became_active":"2024-09-18T10:14:57.139786+0000","last_became_peered":"2024-09-18T10:14:57.139786+0000","last_unstale":"2024-09-18T10:15:23.069165+0000","last_undegraded":"2024-09-18T10:15:23.069165+0000","last_fullsized":"2024-09-18T10:15:23.069165+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:14:45.576168+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010832179999999999,"stat_sum":{"num_bytes":868352,"num_objects":2,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":988,"num_write":11,"num_write_kb":1960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,14,6,2,5,9,15],"acting":[0,14,6,2,5,9,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"e","length":"1"},{"start":"13","length":"1"},{"start":"18","length":"2"}]},{"pgid":"2.6","version":"0'0","reported_seq":49,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-18T10:15:18.896714+0000","last_change":"2024-09-18T10:14:38.963300+0000","last_active":"2024-09-18T10:15:18.896714+0000","last_peered":"2024-09-18T10:15:18.896714+0000","last_clean":"2024-09-18T10:15:18.896714+0000","last_became_active":"2024-09-18T10:14:35.429859+0000","last_became_peered":"2024-09-18T10:14:35.429859+0000","last_unstale":"2024-09-18T10:15:18.896714+0000","last_undegraded":"2024-09-18T10:15:18.896714+0000","last_fullsized":"2024-09-18T10:15:18.896714+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:14:34.288596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:15:40.196292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037634499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11],"acting":[8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.7","version":"61'35","reported_seq":154,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-18T10:15:19.917451+0000","last_change":"2024-09-18T10:15:19.917959+0000","last_active":"2024-09-18T10:15:19.917451+0000","last_peered":"2024-09-18T10:15:19.917451+0000","last_clean":"2024-09-18T10:15:19.917451+0000","last_became_active":"2024-09-18T10:14:57.137776+0000","last_became_peered":"2024-09-18T10:14:57.137776+0000","last_unstale":"2024-09-18T10:15:19.917451+0000","last_undegraded":"2024-09-18T10:15:19.917451+0000","last_fullsized":"2024-09-18T10:15:19.917451+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"60'30","last_scrub_stamp":"2024-09-18T10:15:17.136535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:17.136535+0000","objects_scrubbed":3,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:35.499009+0000","scrub_duration":0.60680385400000003,"objects_trimmed":0,"snaptrim_duration":0.00090678099999999995,"stat_sum":{"num_bytes":5324800,"num_objects":4,"num_object_clones":2,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6,0,14,10,13],"acting":[4,11,6,0,14,10,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"13","length":"1"},{"start":"19","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":48,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-18T10:15:18.908180+0000","last_change":"2024-09-18T10:14:37.327877+0000","last_active":"2024-09-18T10:15:18.908180+0000","last_peered":"2024-09-18T10:15:18.908180+0000","last_clean":"2024-09-18T10:15:18.908180+0000","last_became_active":"2024-09-18T10:14:35.329198+0000","last_became_peered":"2024-09-18T10:14:35.329198+0000","last_unstale":"2024-09-18T10:15:18.908180+0000","last_undegraded":"2024-09-18T10:15:18.908180+0000","last_fullsized":"2024-09-18T10:15:18.908180+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:14:34.288596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:15:44.196046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062415699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"60'34","reported_seq":159,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-18T10:15:20.001468+0000","last_change":"2024-09-18T10:15:20.002774+0000","last_active":"2024-09-18T10:15:20.001468+0000","last_peered":"2024-09-18T10:15:20.001468+0000","last_clean":"2024-09-18T10:15:20.001468+0000","last_became_active":"2024-09-18T10:14:57.134641+0000","last_became_peered":"2024-09-18T10:14:57.134641+0000","last_unstale":"2024-09-18T10:15:20.001468+0000","last_undegraded":"2024-09-18T10:15:20.001468+0000","last_fullsized":"2024-09-18T10:15:20.001468+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"60'34","last_scrub_stamp":"2024-09-18T10:15:17.325003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:17.325003+0000","objects_scrubbed":6,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:18.565493+0000","scrub_duration":0.82233752999999998,"objects_trimmed":0,"snaptrim_duration":0.0016468870000000001,"stat_sum":{"num_bytes":4456448,"num_objects":6,"num_object_clones":3,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":4,"num_read_kb":1815,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,15,4,2,6,10,7],"acting":[8,15,4,2,6,10,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"13","length":"1"},{"start":"19","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":49,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-18T10:15:18.908634+0000","last_change":"2024-09-18T10:14:37.426192+0000","last_active":"2024-09-18T10:15:18.908634+0000","last_peered":"2024-09-18T10:15:18.908634+0000","last_clean":"2024-09-18T10:15:18.908634+0000","last_became_active":"2024-09-18T10:14:35.524102+0000","last_became_peered":"2024-09-18T10:14:35.524102+0000","last_unstale":"2024-09-18T10:15:18.908634+0000","last_undegraded":"2024-09-18T10:15:18.908634+0000","last_fullsized":"2024-09-18T10:15:18.908634+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:14:34.288596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:15:59.190171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.10077985,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.5","version":"35'2","reported_seq":113,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-18T10:15:20.033207+0000","last_change":"2024-09-18T10:15:20.034407+0000","last_active":"2024-09-18T10:15:20.033207+0000","last_peered":"2024-09-18T10:15:20.033207+0000","last_clean":"2024-09-18T10:15:20.033207+0000","last_became_active":"2024-09-18T10:14:57.135810+0000","last_became_peered":"2024-09-18T10:14:57.135810+0000","last_unstale":"2024-09-18T10:15:20.033207+0000","last_undegraded":"2024-09-18T10:15:20.033207+0000","last_fullsized":"2024-09-18T10:15:20.033207+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"35'2","last_scrub_stamp":"2024-09-18T10:15:03.886316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:03.886316+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:14.307127+0000","scrub_duration":0.35694589500000001,"objects_trimmed":0,"snaptrim_duration":0.001572323,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,4,9,12,0,11],"acting":[5,8,4,9,12,0,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"13","length":"1"},{"start":"19","length":"1"}]},{"pgid":"3.3","version":"60'22","reported_seq":127,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-18T10:15:19.906859+0000","last_change":"2024-09-18T10:15:20.008346+0000","last_active":"2024-09-18T10:15:19.906859+0000","last_peered":"2024-09-18T10:15:19.906859+0000","last_clean":"2024-09-18T10:15:19.906859+0000","last_became_active":"2024-09-18T10:14:57.138546+0000","last_became_peered":"2024-09-18T10:14:57.138546+0000","last_unstale":"2024-09-18T10:15:19.906859+0000","last_undegraded":"2024-09-18T10:15:19.906859+0000","last_fullsized":"2024-09-18T10:15:19.906859+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:14:45.576168+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.10157817700000001,"stat_sum":{"num_bytes":516096,"num_objects":2,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":1,"num_write":6,"num_write_kb":784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4,13,11,2,5],"acting":[10,9,4,13,11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"13","length":"1"},{"start":"19","length":"1"}]},{"pgid":"2.2","version":"29'2","reported_seq":52,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-18T10:15:19.909866+0000","last_change":"2024-09-18T10:14:37.429583+0000","last_active":"2024-09-18T10:15:19.909866+0000","last_peered":"2024-09-18T10:15:19.909866+0000","last_clean":"2024-09-18T10:15:19.909866+0000","last_became_active":"2024-09-18T10:14:35.715374+0000","last_became_peered":"2024-09-18T10:14:35.715374+0000","last_unstale":"2024-09-18T10:15:19.909866+0000","last_undegraded":"2024-09-18T10:15:19.909866+0000","last_fullsized":"2024-09-18T10:15:19.909866+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:14:34.288596+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:03.520789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002421224,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,12],"acting":[13,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"3.0","version":"64'14","reported_seq":132,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-18T10:15:20.211561+0000","last_change":"2024-09-18T10:15:20.213644+0000","last_active":"2024-09-18T10:15:20.211561+0000","last_peered":"2024-09-18T10:15:20.211561+0000","last_clean":"2024-09-18T10:15:20.211561+0000","last_became_active":"2024-09-18T10:14:57.444374+0000","last_became_peered":"2024-09-18T10:14:57.444374+0000","last_unstale":"2024-09-18T10:15:20.211561+0000","last_undegraded":"2024-09-18T10:15:20.211561+0000","last_fullsized":"2024-09-18T10:15:20.211561+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'4","last_scrub_stamp":"2024-09-18T10:15:05.668933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:05.668933+0000","objects_scrubbed":1,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:18.433780+0000","scrub_duration":2.6328342170000001,"objects_trimmed":0,"snaptrim_duration":0.20407072700000001,"stat_sum":{"num_bytes":212992,"num_objects":2,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":1,"num_read":2,"num_read_kb":0,"num_write":2,"num_write_kb":208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,9,2,6,15,4],"acting":[3,7,9,2,6,15,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"13","length":"1"},{"start":"19","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":51,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-18T10:15:20.001122+0000","last_change":"2024-09-18T10:14:37.601046+0000","last_active":"2024-09-18T10:15:20.001122+0000","last_peered":"2024-09-18T10:15:20.001122+0000","last_clean":"2024-09-18T10:15:20.001122+0000","last_became_active":"2024-09-18T10:14:35.348262+0000","last_became_peered":"2024-09-18T10:14:35.348262+0000","last_unstale":"2024-09-18T10:15:20.001122+0000","last_undegraded":"2024-09-18T10:15:20.001122+0000","last_fullsized":"2024-09-18T10:15:20.001122+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:14:34.288596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:03.520789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063430400000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,0],"acting":[14,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"3.1","version":"60'24","reported_seq":133,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-18T10:15:19.914240+0000","last_change":"2024-09-18T10:15:20.014989+0000","last_active":"2024-09-18T10:15:19.914240+0000","last_peered":"2024-09-18T10:15:19.914240+0000","last_clean":"2024-09-18T10:15:19.914240+0000","last_became_active":"2024-09-18T10:14:57.038503+0000","last_became_peered":"2024-09-18T10:14:57.038503+0000","last_unstale":"2024-09-18T10:15:19.914240+0000","last_undegraded":"2024-09-18T10:15:19.914240+0000","last_fullsized":"2024-09-18T10:15:19.914240+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:14:45.576168+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.101111137,"stat_sum":{"num_bytes":696320,"num_objects":2,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":1779,"num_write":6,"num_write_kb":680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,13,14,10,7,3,2],"acting":[4,13,14,10,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"13","length":"1"},{"start":"19","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":53,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-18T10:15:21.938310+0000","last_change":"2024-09-18T10:14:37.600953+0000","last_active":"2024-09-18T10:15:21.938310+0000","last_peered":"2024-09-18T10:15:21.938310+0000","last_clean":"2024-09-18T10:15:21.938310+0000","last_became_active":"2024-09-18T10:14:35.349203+0000","last_became_peered":"2024-09-18T10:14:35.349203+0000","last_unstale":"2024-09-18T10:15:21.938310+0000","last_undegraded":"2024-09-18T10:15:21.938310+0000","last_fullsized":"2024-09-18T10:15:21.938310+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:14:34.288596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:15:37.868112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060130600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3],"acting":[9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2","version":"65'41","reported_seq":168,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-18T10:15:21.009814+0000","last_change":"2024-09-18T10:15:21.012194+0000","last_active":"2024-09-18T10:15:21.009814+0000","last_peered":"2024-09-18T10:15:21.009814+0000","last_clean":"2024-09-18T10:15:21.009814+0000","last_became_active":"2024-09-18T10:14:57.227520+0000","last_became_peered":"2024-09-18T10:14:57.227520+0000","last_unstale":"2024-09-18T10:15:21.009814+0000","last_undegraded":"2024-09-18T10:15:21.009814+0000","last_fullsized":"2024-09-18T10:15:21.009814+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'18","last_scrub_stamp":"2024-09-18T10:15:03.202100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:03.202100+0000","objects_scrubbed":6,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:12.722351+0000","scrub_duration":0.63960403700000001,"objects_trimmed":0,"snaptrim_duration":0.10825246400000001,"stat_sum":{"num_bytes":3112960,"num_objects":6,"num_object_clones":2,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":1,"num_read":9,"num_read_kb":2307,"num_write":7,"num_write_kb":1552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,3,13,1,5,10,7],"acting":[14,3,13,1,5,10,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"13","length":"1"},{"start":"18","length":"2"}]},{"pgid":"2.3","version":"27'1","reported_seq":52,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-18T10:15:19.900982+0000","last_change":"2024-09-18T10:14:37.801252+0000","last_active":"2024-09-18T10:15:19.900982+0000","last_peered":"2024-09-18T10:15:19.900982+0000","last_clean":"2024-09-18T10:15:19.900982+0000","last_became_active":"2024-09-18T10:14:35.487746+0000","last_became_peered":"2024-09-18T10:14:35.487746+0000","last_unstale":"2024-09-18T10:15:19.900982+0000","last_undegraded":"2024-09-18T10:15:19.900982+0000","last_fullsized":"2024-09-18T10:15:19.900982+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:14:34.288596+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:03.161727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.10072764100000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,5],"acting":[14,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"1.0","version":"29'112","reported_seq":184,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-18T10:15:19.006535+0000","last_change":"2024-09-18T10:14:32.509942+0000","last_active":"2024-09-18T10:15:19.006535+0000","last_peered":"2024-09-18T10:15:19.006535+0000","last_clean":"2024-09-18T10:15:19.006535+0000","last_became_active":"2024-09-18T10:14:32.508037+0000","last_became_peered":"2024-09-18T10:14:32.508037+0000","last_unstale":"2024-09-18T10:15:19.006535+0000","last_undegraded":"2024-09-18T10:15:19.006535+0000","last_fullsized":"2024-09-18T10:15:19.006535+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:31.267741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:31.267741+0000","last_clean_scrub_stamp":"2024-09-18T10:14:31.267741+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:15:59.262610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,15],"acting":[6,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.8","version":"41'13","reported_seq":129,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-18T10:15:19.907153+0000","last_change":"2024-09-18T10:15:19.908199+0000","last_active":"2024-09-18T10:15:19.907153+0000","last_peered":"2024-09-18T10:15:19.907153+0000","last_clean":"2024-09-18T10:15:19.907153+0000","last_became_active":"2024-09-18T10:14:57.443754+0000","last_became_peered":"2024-09-18T10:14:57.443754+0000","last_unstale":"2024-09-18T10:15:19.907153+0000","last_undegraded":"2024-09-18T10:15:19.907153+0000","last_fullsized":"2024-09-18T10:15:19.907153+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"41'13","last_scrub_stamp":"2024-09-18T10:15:05.373444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:05.373444+0000","objects_scrubbed":2,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:31.932850+0000","scrub_duration":2.8745114210000002,"objects_trimmed":0,"snaptrim_duration":0.0031925629999999998,"stat_sum":{"num_bytes":966656,"num_objects":2,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":705,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,11,3,15,1,7],"acting":[6,0,11,3,15,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"13","length":"1"},{"start":"19","length":"1"}]},{"pgid":"3.9","version":"63'56","reported_seq":174,"reported_epoch":64,"state":"active+clean+snaptrim","last_fresh":"2024-09-18T10:15:20.172992+0000","last_change":"2024-09-18T10:15:20.173063+0000","last_active":"2024-09-18T10:15:20.172992+0000","last_peered":"2024-09-18T10:15:20.172992+0000","last_clean":"2024-09-18T10:15:20.172992+0000","last_became_active":"2024-09-18T10:14:56.840271+0000","last_became_peered":"2024-09-18T10:14:56.840271+0000","last_unstale":"2024-09-18T10:15:20.172992+0000","last_undegraded":"2024-09-18T10:15:20.172992+0000","last_fullsized":"2024-09-18T10:15:20.172992+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'35","last_scrub_stamp":"2024-09-18T10:15:07.077517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:07.077517+0000","objects_scrubbed":4,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:31.443309+0000","scrub_duration":0.71488955499999995,"objects_trimmed":0,"snaptrim_duration":0.31652517000000002,"stat_sum":{"num_bytes":6062080,"num_objects":6,"num_object_clones":4,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":11,"num_read_kb":5292,"num_write":8,"num_write_kb":4704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,10,11,5,8,3],"acting":[7,2,10,11,5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"13","length":"1"}]},{"pgid":"3.a","version":"60'24","reported_seq":127,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-18T10:15:20.912853+0000","last_change":"2024-09-18T10:15:20.913618+0000","last_active":"2024-09-18T10:15:20.912853+0000","last_peered":"2024-09-18T10:15:20.912853+0000","last_clean":"2024-09-18T10:15:20.912853+0000","last_became_active":"2024-09-18T10:14:46.765703+0000","last_became_peered":"2024-09-18T10:14:46.765703+0000","last_unstale":"2024-09-18T10:15:20.912853+0000","last_undegraded":"2024-09-18T10:15:20.912853+0000","last_fullsized":"2024-09-18T10:15:20.912853+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:14:45.576168+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:08.721290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090196500000000001,"stat_sum":{"num_bytes":770048,"num_objects":4,"num_object_clones":2,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":1,"num_read":23,"num_read_kb":3549,"num_write":29,"num_write_kb":1344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,1,6,5,3,15,4],"acting":[13,1,6,5,3,15,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"13","length":"1"},{"start":"18","length":"2"}]},{"pgid":"3.b","version":"60'21","reported_seq":105,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-18T10:15:19.915264+0000","last_change":"2024-09-18T10:15:19.917336+0000","last_active":"2024-09-18T10:15:19.915264+0000","last_peered":"2024-09-18T10:15:19.915264+0000","last_clean":"2024-09-18T10:15:19.915264+0000","last_became_active":"2024-09-18T10:14:46.802834+0000","last_became_peered":"2024-09-18T10:14:46.802834+0000","last_unstale":"2024-09-18T10:15:19.915264+0000","last_undegraded":"2024-09-18T10:15:19.915264+0000","last_fullsized":"2024-09-18T10:15:19.915264+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:14:45.576168+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:04.590511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024087140000000002,"stat_sum":{"num_bytes":671744,"num_objects":1,"num_object_clones":0,"num_object_copies":7,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":31,"num_write_kb":1600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,10,12,11,1,14],"acting":[4,0,10,12,11,1,14],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"13","length":"1"},{"start":"19","length":"1"}]},{"pgid":"3.c","version":"60'74","reported_seq":159,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-18T10:15:20.032938+0000","last_change":"2024-09-18T10:15:20.033707+0000","last_active":"2024-09-18T10:15:20.032938+0000","last_peered":"2024-09-18T10:15:20.032938+0000","last_clean":"2024-09-18T10:15:20.032938+0000","last_became_active":"2024-09-18T10:14:46.795995+0000","last_became_peered":"2024-09-18T10:14:46.795995+0000","last_unstale":"2024-09-18T10:15:20.032938+0000","last_undegraded":"2024-09-18T10:15:20.032938+0000","last_fullsized":"2024-09-18T10:15:20.032938+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:14:45.576168+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:14.450533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001040731,"stat_sum":{"num_bytes":6832128,"num_objects":7,"num_object_clones":2,"num_object_copies":49,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":23,"num_read_kb":3467,"num_write":73,"num_write_kb":7344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,13,3,7,12,14],"acting":[5,8,13,3,7,12,14],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"13","length":"1"},{"start":"19","length":"1"}]},{"pgid":"3.d","version":"65'23","reported_seq":118,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-18T10:15:21.018757+0000","last_change":"2024-09-18T10:15:21.020024+0000","last_active":"2024-09-18T10:15:21.018757+0000","last_peered":"2024-09-18T10:15:21.018757+0000","last_clean":"2024-09-18T10:15:21.018757+0000","last_became_active":"2024-09-18T10:14:46.793893+0000","last_became_peered":"2024-09-18T10:14:46.793893+0000","last_unstale":"2024-09-18T10:15:21.018757+0000","last_undegraded":"2024-09-18T10:15:21.018757+0000","last_fullsized":"2024-09-18T10:15:21.018757+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:14:45.576168+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:13.756262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.106940761,"stat_sum":{"num_bytes":2899968,"num_objects":2,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":9,"num_read_kb":532,"num_write":10,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,1,11,15,9,8,4],"acting":[13,1,11,15,9,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"13","length":"1"},{"start":"18","length":"2"}]},{"pgid":"3.e","version":"66'35","reported_seq":131,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-18T10:15:23.153759+0000","last_change":"2024-09-18T10:15:22.133069+0000","last_active":"2024-09-18T10:15:23.153759+0000","last_peered":"2024-09-18T10:15:23.153759+0000","last_clean":"2024-09-18T10:15:23.153759+0000","last_became_active":"2024-09-18T10:14:46.801571+0000","last_became_peered":"2024-09-18T10:14:46.801571+0000","last_unstale":"2024-09-18T10:15:23.153759+0000","last_undegraded":"2024-09-18T10:15:23.153759+0000","last_fullsized":"2024-09-18T10:15:23.153759+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:14:45.576168+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:15:59.433777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.20719484399999999,"stat_sum":{"num_bytes":3571712,"num_objects":4,"num_object_clones":2,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":1,"num_read":8,"num_read_kb":4,"num_write":39,"num_write_kb":3488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,14,6,13,8,3,12],"acting":[11,14,6,13,8,3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"e","length":"1"},{"start":"13","length":"1"},{"start":"18","length":"2"}]},{"pgid":"3.f","version":"65'23","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-18T10:15:21.016799+0000","last_change":"2024-09-18T10:15:21.017967+0000","last_active":"2024-09-18T10:15:21.016799+0000","last_peered":"2024-09-18T10:15:21.016799+0000","last_clean":"2024-09-18T10:15:21.016799+0000","last_became_active":"2024-09-18T10:14:46.803296+0000","last_became_peered":"2024-09-18T10:14:46.803296+0000","last_unstale":"2024-09-18T10:15:21.016799+0000","last_undegraded":"2024-09-18T10:15:21.016799+0000","last_fullsized":"2024-09-18T10:15:21.016799+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:14:45.576168+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:11.624416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.105075355,"stat_sum":{"num_bytes":2129920,"num_objects":2,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":9,"num_read_kb":1780,"num_write":11,"num_write_kb":2640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,3,15,14,2,11,0],"acting":[13,3,15,14,2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"13","length":"1"},{"start":"18","length":"2"}]},{"pgid":"3.10","version":"60'14","reported_seq":79,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-18T10:15:19.904608+0000","last_change":"2024-09-18T10:15:19.906976+0000","last_active":"2024-09-18T10:15:19.904608+0000","last_peered":"2024-09-18T10:15:19.904608+0000","last_clean":"2024-09-18T10:15:19.904608+0000","last_became_active":"2024-09-18T10:15:03.217299+0000","last_became_peered":"2024-09-18T10:15:03.217299+0000","last_unstale":"2024-09-18T10:15:19.904608+0000","last_undegraded":"2024-09-18T10:15:19.904608+0000","last_fullsized":"2024-09-18T10:15:19.904608+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:14:45.576168+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025989540000000001,"stat_sum":{"num_bytes":1769472,"num_objects":1,"num_object_clones":0,"num_object_copies":7,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":9,"num_read_kb":1684,"num_write":6,"num_write_kb":2048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":884736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,5,4,8,0,9],"acting":[6,7,5,4,8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"13","length":"1"},{"start":"19","length":"1"}]},{"pgid":"3.11","version":"36'6","reported_seq":61,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-18T10:15:19.913609+0000","last_change":"2024-09-18T10:15:20.014304+0000","last_active":"2024-09-18T10:15:19.913609+0000","last_peered":"2024-09-18T10:15:19.913609+0000","last_clean":"2024-09-18T10:15:19.913609+0000","last_became_active":"2024-09-18T10:15:04.955869+0000","last_became_peered":"2024-09-18T10:15:04.955869+0000","last_unstale":"2024-09-18T10:15:19.913609+0000","last_undegraded":"2024-09-18T10:15:19.913609+0000","last_fullsized":"2024-09-18T10:15:19.913609+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":5,"last_scrub":"36'6","last_scrub_stamp":"2024-09-18T10:15:07.755163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:07.755163+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:13.113623+0000","scrub_duration":0.60886176599999997,"objects_trimmed":0,"snaptrim_duration":0.10113008699999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,13,10,0,14,2],"acting":[3,5,13,10,0,14,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"13","length":"1"},{"start":"19","length":"1"}]},{"pgid":"3.12","version":"36'8","reported_seq":141,"reported_epoch":65,"state":"active+clean+snaptrim","last_fresh":"2024-09-18T10:15:21.012250+0000","last_change":"2024-09-18T10:15:21.012388+0000","last_active":"2024-09-18T10:15:21.012250+0000","last_peered":"2024-09-18T10:15:21.012250+0000","last_clean":"2024-09-18T10:15:21.012250+0000","last_became_active":"2024-09-18T10:15:06.903185+0000","last_became_peered":"2024-09-18T10:15:06.903185+0000","last_unstale":"2024-09-18T10:15:21.012250+0000","last_undegraded":"2024-09-18T10:15:21.012250+0000","last_fullsized":"2024-09-18T10:15:21.012250+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":5,"last_scrub":"36'8","last_scrub_stamp":"2024-09-18T10:15:04.353687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:04.353687+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:28.531412+0000","scrub_duration":0.76511709400000005,"objects_trimmed":0,"snaptrim_duration":0.0052443999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,5,8,3,4,11,1],"acting":[14,5,8,3,4,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"13","length":"1"},{"start":"19","length":"1"}]},{"pgid":"3.13","version":"64'37","reported_seq":115,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-18T10:15:20.235626+0000","last_change":"2024-09-18T10:15:20.238036+0000","last_active":"2024-09-18T10:15:20.235626+0000","last_peered":"2024-09-18T10:15:20.235626+0000","last_clean":"2024-09-18T10:15:20.235626+0000","last_became_active":"2024-09-18T10:15:13.263306+0000","last_became_peered":"2024-09-18T10:15:13.263306+0000","last_unstale":"2024-09-18T10:15:20.235626+0000","last_undegraded":"2024-09-18T10:15:20.235626+0000","last_fullsized":"2024-09-18T10:15:20.235626+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":5,"last_scrub":"60'33","last_scrub_stamp":"2024-09-18T10:15:19.140411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:19.140411+0000","objects_scrubbed":1,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:39.399176+0000","scrub_duration":0.706777345,"objects_trimmed":0,"snaptrim_duration":0.20629254,"stat_sum":{"num_bytes":1720320,"num_objects":2,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":1,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,13,2,6,0,4,15],"acting":[1,13,2,6,0,4,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"13","length":"1"},{"start":"19","length":"1"}]}],"pool_stats":[{"poolid":3,"num_pg":26,"stat_sum":{"num_bytes":53125120,"num_objects":71,"num_object_clones":32,"num_object_copies":497,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":6,"num_read":178,"num_read_kb":33689,"num_write":316,"num_write_kb":43448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1638400,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":103034880,"data_stored":137261056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":666,"ondisk_log_size":666,"up":182,"acting":182,"num_store_stats":16},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":112,"ondisk_log_size":112,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":15,"up_from":22,"seq":94489280524,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303200,"kb_used_data":5836,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94068640,"statfs":{"total":96636764160,"available":96326287360,"internally_reserved":0,"allocated":5976064,"data_stored":7044415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6,10,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66400000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86199999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82699999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2170000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1830000000000001}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4490000000000001}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.296}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.474}]}]},{"osd":14,"up_from":22,"seq":94489280524,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":305620,"kb_used_data":8260,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94066220,"statfs":{"total":96636764160,"available":96323809280,"internally_reserved":0,"allocated":8458240,"data_stored":10263327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,7,8,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,0,0,0,0,5],"upper_bound":1024},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.624}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.417}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.03}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84499999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.109}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2010000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3700000000000001}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1619999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":91.111999999999995}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91100000000000003}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":78.804000000000002}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4119999999999999}]}]},{"osd":13,"up_from":22,"seq":94489280525,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":305720,"kb_used_data":8360,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94066120,"statfs":{"total":96636764160,"available":96323706880,"internally_reserved":0,"allocated":8560640,"data_stored":10824498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6,8,9,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,3,0,0,1],"upper_bound":2048},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89400000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0660000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3779999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0920000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.046}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.222}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.444}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.278}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6779999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.784}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.99}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6930000000000001}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.718}]}]},{"osd":0,"up_from":21,"seq":90194313229,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303332,"kb_used_data":5968,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94068508,"statfs":{"total":96636764160,"available":96326152192,"internally_reserved":0,"allocated":6111232,"data_stored":6892379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,4,5,6,9,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84699999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1080000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.196}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95099999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2869999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.349}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.518}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5349999999999999}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.486}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3839999999999999}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6080000000000001}]}]},{"osd":4,"up_from":21,"seq":90194313230,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303184,"kb_used_data":5828,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94068656,"statfs":{"total":96636764160,"available":96326303744,"internally_reserved":0,"allocated":5967872,"data_stored":7127789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":58.866999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":164.21199999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":59.024000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97099999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":59.292999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":59.383000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":164.65899999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5289999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":164.43899999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.103}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":59.463999999999999}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2789999999999999}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1890000000000001}]}]},{"osd":9,"up_from":21,"seq":90194313229,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":300352,"kb_used_data":2992,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94071488,"statfs":{"total":96636764160,"available":96329203712,"internally_reserved":0,"allocated":3063808,"data_stored":2966374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,8,10,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94899999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.626}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52000000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1910000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89200000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.218}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2609999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1200000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.669}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.446}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":96}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1439999999999999}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98299999999999998}]}]},{"osd":11,"up_from":21,"seq":90194313229,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308524,"kb_used_data":11160,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94063316,"statfs":{"total":96636764160,"available":96320835584,"internally_reserved":0,"allocated":11427840,"data_stored":15297371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[2,3,6,8,9,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0049999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84299999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95999999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80100000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.03}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.335}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4590000000000001}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2030000000000001}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3109999999999999}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":123.80500000000001}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6379999999999999}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5069999999999999}]}]},{"osd":12,"up_from":21,"seq":90194313230,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301396,"kb_used_data":4036,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94070444,"statfs":{"total":96636764160,"available":96328134656,"internally_reserved":0,"allocated":4132864,"data_stored":5305209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,9,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67800000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95399999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.393}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.159}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3160000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8460000000000001}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3600000000000001}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6339999999999999}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9810000000000001}]}]},{"osd":1,"up_from":22,"seq":94489280524,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301524,"kb_used_data":4164,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94070316,"statfs":{"total":96636764160,"available":96328003584,"internally_reserved":0,"allocated":4263936,"data_stored":4799263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,9,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95199999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84999999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1830000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.276}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2410000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":118.43600000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7789999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.538}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6659999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.825}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.589}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7150000000000001}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.907}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7370000000000001}]}]},{"osd":2,"up_from":22,"seq":94489280524,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303748,"kb_used_data":6392,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94068092,"statfs":{"total":96636764160,"available":96325726208,"internally_reserved":0,"allocated":6545408,"data_stored":7348973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,3,4,5,6,7,8,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":419.05700000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0469999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90300000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93300000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0880000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.111}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.486}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4079999999999999}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4410000000000001}]}]},{"osd":3,"up_from":22,"seq":94489280524,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306424,"kb_used_data":9064,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94065416,"statfs":{"total":96636764160,"available":96322985984,"internally_reserved":0,"allocated":9281536,"data_stored":13409055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,4,5,6,7,9,10,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79000000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81599999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1539999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90600000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.121}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0620000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.026}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97299999999999998}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1060000000000001}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2330000000000001}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1850000000000001}]}]},{"osd":5,"up_from":22,"seq":94489280524,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304032,"kb_used_data":6676,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94067808,"statfs":{"total":96636764160,"available":96325435392,"internally_reserved":0,"allocated":6836224,"data_stored":7557869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,6,7,8,9,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96999999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73999999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88800000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94499999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.298}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0940000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":80.272000000000006}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6479999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.345}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0009999999999999}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4430000000000001}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":101.733}]}]},{"osd":6,"up_from":22,"seq":94489280524,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":305284,"kb_used_data":7920,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94066556,"statfs":{"total":96636764160,"available":96324153344,"internally_reserved":0,"allocated":8110080,"data_stored":11097357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,7,8,9,11,12,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80500000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92300000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.343}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93899999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.343}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0429999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.294}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":95.691999999999993}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.329}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5680000000000001}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2410000000000001}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2050000000000001}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2669999999999999}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.613}]}]},{"osd":7,"up_from":22,"seq":94489280524,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306004,"kb_used_data":8648,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94065836,"statfs":{"total":96636764160,"available":96323416064,"internally_reserved":0,"allocated":8855552,"data_stored":11694829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,8,10,11,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,5,6,1,3,2],"upper_bound":2048},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":139.58600000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":139.36799999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":139.27199999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":139.51599999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.018}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":139.72800000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0860000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.861}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94299999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.127}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.885}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":139.655}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5900000000000001}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5509999999999999}]}]},{"osd":8,"up_from":22,"seq":94489280524,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306436,"kb_used_data":9076,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94065404,"statfs":{"total":96636764160,"available":96322973696,"internally_reserved":0,"allocated":9293824,"data_stored":11975455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[2,4,6,7,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93300000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.083}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94499999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85699999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0569999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0169999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0349999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.212}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.284}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.75}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":33.423000000000002}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3260000000000001}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.306}]}]},{"osd":10,"up_from":22,"seq":94489280525,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":305412,"kb_used_data":8052,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94066428,"statfs":{"total":96636764160,"available":96324022272,"internally_reserved":0,"allocated":8245248,"data_stored":11332383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,4,5,7,9,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,0,0,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79800000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0860000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0429999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1970000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1180000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4410000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7170000000000001}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.232}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":35.698999999999998}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66200000000000003}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5800000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4378624,"data_stored":5226496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":3375104,"data_stored":4313088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":6381568,"data_stored":8269824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8392704,"data_stored":11513856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":6152192,"data_stored":8032256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":6414336,"data_stored":8241152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":7360512,"data_stored":10059776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8327168,"data_stored":10874880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8404992,"data_stored":11489280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":2125824,"data_stored":2449408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":8011776,"data_stored":10780672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":10899456,"data_stored":15249408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":3190784,"data_stored":4788224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":7667712,"data_stored":10338304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":7397376,"data_stored":9605120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":4554752,"data_stored":6029312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T10:15:25.363 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T10:15:25.364 INFO:teuthology.orchestra.run.smithi029.stdout:osd.12: {} 2024-09-18T10:15:25.364 INFO:teuthology.orchestra.run.smithi029.stdout:osd.13: {} 2024-09-18T10:15:25.364 INFO:teuthology.orchestra.run.smithi029.stdout:osd.14: {} 2024-09-18T10:15:25.364 INFO:teuthology.orchestra.run.smithi029.stdout:osd.15: {} 2024-09-18T10:15:25.365 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T10:15:25.365 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T10:15:25.365 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T10:15:25.365 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T10:15:25.365 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T10:15:25.365 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T10:15:25.365 INFO:teuthology.orchestra.run.smithi029.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T10:15:25.365 INFO:teuthology.orchestra.run.smithi029.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T10:15:25.365 INFO:teuthology.orchestra.run.smithi029.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-18T10:15:25.365 INFO:teuthology.orchestra.run.smithi029.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-18T10:15:25.366 INFO:teuthology.orchestra.run.smithi029.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-18T10:15:25.366 INFO:teuthology.orchestra.run.smithi029.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-18T10:15:25.366 INFO:teuthology.orchestra.run.smithi029.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-18T10:15:25.366 INFO:teuthology.orchestra.run.smithi029.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-18T10:15:25.366 INFO:teuthology.orchestra.run.smithi029.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-18T10:15:25.366 INFO:teuthology.orchestra.run.smithi029.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-18T10:15:25.367 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-18T10:15:25.346+0000 7f302f2c3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22187) UID: 0 2024-09-18T10:15:25.370 INFO:tasks.rados.rados.0.smithi189.stdout:234: finishing copy_from racing read to smithi18925896-48 2024-09-18T10:15:25.454 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-18T10:15:25.450+0000 7f302f2c3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22187) UID: 0 2024-09-18T10:15:25.468 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:25.468 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:25.477 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T10:15:25.555 INFO:tasks.ceph.osd.14.smithi154.stderr:2024-09-18T10:15:25.551+0000 7f75a19e7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 22093) UID: 0 2024-09-18T10:15:25.566 INFO:tasks.rados.rados.0.smithi189.stdout:234: finishing copy_from to smithi18925896-48 2024-09-18T10:15:25.566 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 48 v 8 (ObjNum 41 snap 0 seq_num 41) dirty exists 2024-09-18T10:15:25.576 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:25.576 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:25.586 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T10:15:25.656 INFO:tasks.ceph.osd.10.smithi154.stderr:2024-09-18T10:15:25.655+0000 7fb73221a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 22091) UID: 0 2024-09-18T10:15:25.686 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:25.686 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:25.696 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T10:15:25.740 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-18T10:15:25.743 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-09-18T10:15:25.755 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":82,"stamp":"2024-09-18T10:15:25.210194+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":53715507,"num_objects":75,"num_object_clones":32,"num_object_copies":505,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":6,"num_read":264,"num_read_kb":33762,"num_write":463,"num_write_kb":46122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1638400,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":781,"ondisk_log_size":781,"up":200,"acting":200,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":208,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":16,"kb":1509949440,"kb_used":4870192,"kb_used_data":112432,"kb_used_omap":26,"kb_used_meta":4757477,"kb_avail":1505079248,"statfs":{"total":1546188226560,"available":1541201149952,"internally_reserved":0,"allocated":115130368,"data_stored":144936546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":27264,"internal_metadata":4871656832},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,1,2,5,14,1,8,3],"upper_bound":2048},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5062656,"num_objects":1,"num_object_clones":1,"num_object_copies":7,"num_objects_missing_on_primary":-2,"num_objects_missing":-2,"num_objects_degraded":-14,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":9,"num_read_kb":1673,"num_write":10,"num_write_kb":4944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":688128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001558"},"pg_stats":[{"pgid":"3.19","version":"63'64","reported_seq":171,"reported_epoch":64,"state":"active+clean+snaptrim","last_fresh":"2024-09-18T10:15:20.272186+0000","last_change":"2024-09-18T10:15:20.272257+0000","last_active":"2024-09-18T10:15:20.272186+0000","last_peered":"2024-09-18T10:15:20.272186+0000","last_clean":"2024-09-18T10:15:20.272186+0000","last_became_active":"2024-09-18T10:14:57.350904+0000","last_became_peered":"2024-09-18T10:14:57.350904+0000","last_unstale":"2024-09-18T10:15:20.272186+0000","last_undegraded":"2024-09-18T10:15:20.272186+0000","last_fullsized":"2024-09-18T10:15:20.272186+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:14:45.576168+0000","objects_scrubbed":0,"log_size":64,"log_dups_size":0,"ondisk_log_size":64,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.31209502,"stat_sum":{"num_bytes":4595712,"num_objects":8,"num_object_clones":5,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":22,"num_read_kb":7473,"num_write":44,"num_write_kb":5400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,10,11,5,8,3],"acting":[7,2,10,11,5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"13","length":"1"}]},{"pgid":"3.18","version":"60'30","reported_seq":140,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-18T10:15:19.905225+0000","last_change":"2024-09-18T10:15:19.907721+0000","last_active":"2024-09-18T10:15:19.905225+0000","last_peered":"2024-09-18T10:15:19.905225+0000","last_clean":"2024-09-18T10:15:19.905225+0000","last_became_active":"2024-09-18T10:14:57.647465+0000","last_became_peered":"2024-09-18T10:14:57.647465+0000","last_unstale":"2024-09-18T10:15:19.905225+0000","last_undegraded":"2024-09-18T10:15:19.905225+0000","last_fullsized":"2024-09-18T10:15:19.905225+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:14:45.576168+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030463600000000001,"stat_sum":{"num_bytes":3293184,"num_objects":4,"num_object_clones":2,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":11,"num_read_kb":2309,"num_write":24,"num_write_kb":3232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,11,3,15,1,7],"acting":[6,0,11,3,15,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"13","length":"1"},{"start":"19","length":"1"}]},{"pgid":"3.17","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"peering","last_fresh":"2024-09-18T10:15:19.908409+0000","last_change":"2024-09-18T10:15:19.908409+0000","last_active":"0.000000","last_peered":"0.000000","last_clean":"0.000000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-18T10:15:19.908409+0000","last_undegraded":"2024-09-18T10:15:19.908409+0000","last_fullsized":"2024-09-18T10:15:19.908409+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:14:45.576168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:15:53.934915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,13,3,14,2,11,6],"acting":[10,13,3,14,2,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"66'18","reported_seq":53,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-18T10:15:22.238585+0000","last_change":"2024-09-18T10:15:22.238585+0000","last_active":"2024-09-18T10:15:22.238585+0000","last_peered":"2024-09-18T10:15:22.238585+0000","last_clean":"2024-09-18T10:15:22.238585+0000","last_became_active":"2024-09-18T10:15:19.227471+0000","last_became_peered":"2024-09-18T10:15:19.227471+0000","last_unstale":"2024-09-18T10:15:22.238585+0000","last_undegraded":"2024-09-18T10:15:22.238585+0000","last_fullsized":"2024-09-18T10:15:22.238585+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":5,"last_scrub":"36'11","last_scrub_stamp":"2024-09-18T10:15:07.445786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:07.445786+0000","objects_scrubbed":1,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:23.145992+0000","scrub_duration":0.30645798400000002,"objects_trimmed":0,"snaptrim_duration":0.30014816300000002,"stat_sum":{"num_bytes":2621440,"num_objects":2,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":1888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":688128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,6,4,8,2,14,10],"acting":[9,6,4,8,2,14,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"e","length":"1"},{"start":"13","length":"1"},{"start":"18","length":"2"}]},{"pgid":"3.15","version":"65'8","reported_seq":49,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-18T10:15:21.109971+0000","last_change":"2024-09-18T10:15:21.112019+0000","last_active":"2024-09-18T10:15:21.109971+0000","last_peered":"2024-09-18T10:15:21.109971+0000","last_clean":"2024-09-18T10:15:21.109971+0000","last_became_active":"2024-09-18T10:15:17.127771+0000","last_became_peered":"2024-09-18T10:15:17.127771+0000","last_unstale":"2024-09-18T10:15:21.109971+0000","last_undegraded":"2024-09-18T10:15:21.109971+0000","last_fullsized":"2024-09-18T10:15:21.109971+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:14:45.576168+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.20870269999999999,"stat_sum":{"num_bytes":32768,"num_objects":2,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":1,"num_read":2,"num_read_kb":0,"num_write":2,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,15,13,10,1,7,9],"acting":[14,15,13,10,1,7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"13","length":"1"},{"start":"18","length":"2"}]},{"pgid":"3.14","version":"37'8","reported_seq":41,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-18T10:15:20.031743+0000","last_change":"2024-09-18T10:15:20.132377+0000","last_active":"2024-09-18T10:15:20.031743+0000","last_peered":"2024-09-18T10:15:20.031743+0000","last_clean":"2024-09-18T10:15:20.031743+0000","last_became_active":"2024-09-18T10:15:14.767770+0000","last_became_peered":"2024-09-18T10:15:14.767770+0000","last_unstale":"2024-09-18T10:15:20.031743+0000","last_undegraded":"2024-09-18T10:15:20.031743+0000","last_fullsized":"2024-09-18T10:15:20.031743+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":5,"last_scrub":"37'8","last_scrub_stamp":"2024-09-18T10:15:17.041590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:17.041590+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:32.741813+0000","scrub_duration":0.60018000800000004,"objects_trimmed":0,"snaptrim_duration":0.10088322600000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,14,9,10,5,13,4],"acting":[1,14,9,10,5,13,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"13","length":"1"},{"start":"19","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":50,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-18T10:15:19.910703+0000","last_change":"2024-09-18T10:14:37.427722+0000","last_active":"2024-09-18T10:15:19.910703+0000","last_peered":"2024-09-18T10:15:19.910703+0000","last_clean":"2024-09-18T10:15:19.910703+0000","last_became_active":"2024-09-18T10:14:35.348943+0000","last_became_peered":"2024-09-18T10:14:35.348943+0000","last_unstale":"2024-09-18T10:15:19.910703+0000","last_undegraded":"2024-09-18T10:15:19.910703+0000","last_fullsized":"2024-09-18T10:15:19.910703+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:14:34.288596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:03.161727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060442999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,10],"acting":[13,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"3.6","version":"57'32","reported_seq":167,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-18T10:15:23.069165+0000","last_change":"2024-09-18T10:15:21.935648+0000","last_active":"2024-09-18T10:15:23.069165+0000","last_peered":"2024-09-18T10:15:23.069165+0000","last_clean":"2024-09-18T10:15:23.069165+0000","last_became_active":"2024-09-18T10:14:57.139786+0000","last_became_peered":"2024-09-18T10:14:57.139786+0000","last_unstale":"2024-09-18T10:15:23.069165+0000","last_undegraded":"2024-09-18T10:15:23.069165+0000","last_fullsized":"2024-09-18T10:15:23.069165+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:14:45.576168+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010832179999999999,"stat_sum":{"num_bytes":868352,"num_objects":2,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":988,"num_write":11,"num_write_kb":1960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,14,6,2,5,9,15],"acting":[0,14,6,2,5,9,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"e","length":"1"},{"start":"13","length":"1"},{"start":"18","length":"2"}]},{"pgid":"2.6","version":"0'0","reported_seq":49,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-18T10:15:18.896714+0000","last_change":"2024-09-18T10:14:38.963300+0000","last_active":"2024-09-18T10:15:18.896714+0000","last_peered":"2024-09-18T10:15:18.896714+0000","last_clean":"2024-09-18T10:15:18.896714+0000","last_became_active":"2024-09-18T10:14:35.429859+0000","last_became_peered":"2024-09-18T10:14:35.429859+0000","last_unstale":"2024-09-18T10:15:18.896714+0000","last_undegraded":"2024-09-18T10:15:18.896714+0000","last_fullsized":"2024-09-18T10:15:18.896714+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:14:34.288596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:15:40.196292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037634499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11],"acting":[8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.7","version":"61'35","reported_seq":154,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-18T10:15:19.917451+0000","last_change":"2024-09-18T10:15:19.917959+0000","last_active":"2024-09-18T10:15:19.917451+0000","last_peered":"2024-09-18T10:15:19.917451+0000","last_clean":"2024-09-18T10:15:19.917451+0000","last_became_active":"2024-09-18T10:14:57.137776+0000","last_became_peered":"2024-09-18T10:14:57.137776+0000","last_unstale":"2024-09-18T10:15:19.917451+0000","last_undegraded":"2024-09-18T10:15:19.917451+0000","last_fullsized":"2024-09-18T10:15:19.917451+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"60'30","last_scrub_stamp":"2024-09-18T10:15:17.136535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:17.136535+0000","objects_scrubbed":3,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:35.499009+0000","scrub_duration":0.60680385400000003,"objects_trimmed":0,"snaptrim_duration":0.00090678099999999995,"stat_sum":{"num_bytes":5324800,"num_objects":4,"num_object_clones":2,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6,0,14,10,13],"acting":[4,11,6,0,14,10,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"13","length":"1"},{"start":"19","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":48,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-18T10:15:18.908180+0000","last_change":"2024-09-18T10:14:37.327877+0000","last_active":"2024-09-18T10:15:18.908180+0000","last_peered":"2024-09-18T10:15:18.908180+0000","last_clean":"2024-09-18T10:15:18.908180+0000","last_became_active":"2024-09-18T10:14:35.329198+0000","last_became_peered":"2024-09-18T10:14:35.329198+0000","last_unstale":"2024-09-18T10:15:18.908180+0000","last_undegraded":"2024-09-18T10:15:18.908180+0000","last_fullsized":"2024-09-18T10:15:18.908180+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:14:34.288596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:15:44.196046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062415699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"60'34","reported_seq":159,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-18T10:15:20.001468+0000","last_change":"2024-09-18T10:15:20.002774+0000","last_active":"2024-09-18T10:15:20.001468+0000","last_peered":"2024-09-18T10:15:20.001468+0000","last_clean":"2024-09-18T10:15:20.001468+0000","last_became_active":"2024-09-18T10:14:57.134641+0000","last_became_peered":"2024-09-18T10:14:57.134641+0000","last_unstale":"2024-09-18T10:15:20.001468+0000","last_undegraded":"2024-09-18T10:15:20.001468+0000","last_fullsized":"2024-09-18T10:15:20.001468+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"60'34","last_scrub_stamp":"2024-09-18T10:15:17.325003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:17.325003+0000","objects_scrubbed":6,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:18.565493+0000","scrub_duration":0.82233752999999998,"objects_trimmed":0,"snaptrim_duration":0.0016468870000000001,"stat_sum":{"num_bytes":4456448,"num_objects":6,"num_object_clones":3,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":4,"num_read_kb":1815,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,15,4,2,6,10,7],"acting":[8,15,4,2,6,10,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"13","length":"1"},{"start":"19","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":49,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-18T10:15:18.908634+0000","last_change":"2024-09-18T10:14:37.426192+0000","last_active":"2024-09-18T10:15:18.908634+0000","last_peered":"2024-09-18T10:15:18.908634+0000","last_clean":"2024-09-18T10:15:18.908634+0000","last_became_active":"2024-09-18T10:14:35.524102+0000","last_became_peered":"2024-09-18T10:14:35.524102+0000","last_unstale":"2024-09-18T10:15:18.908634+0000","last_undegraded":"2024-09-18T10:15:18.908634+0000","last_fullsized":"2024-09-18T10:15:18.908634+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:14:34.288596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:15:59.190171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.10077985,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.5","version":"35'2","reported_seq":113,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-18T10:15:20.033207+0000","last_change":"2024-09-18T10:15:20.034407+0000","last_active":"2024-09-18T10:15:20.033207+0000","last_peered":"2024-09-18T10:15:20.033207+0000","last_clean":"2024-09-18T10:15:20.033207+0000","last_became_active":"2024-09-18T10:14:57.135810+0000","last_became_peered":"2024-09-18T10:14:57.135810+0000","last_unstale":"2024-09-18T10:15:20.033207+0000","last_undegraded":"2024-09-18T10:15:20.033207+0000","last_fullsized":"2024-09-18T10:15:20.033207+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"35'2","last_scrub_stamp":"2024-09-18T10:15:03.886316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:03.886316+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:14.307127+0000","scrub_duration":0.35694589500000001,"objects_trimmed":0,"snaptrim_duration":0.001572323,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,4,9,12,0,11],"acting":[5,8,4,9,12,0,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"13","length":"1"},{"start":"19","length":"1"}]},{"pgid":"3.3","version":"60'22","reported_seq":127,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-18T10:15:19.906859+0000","last_change":"2024-09-18T10:15:20.008346+0000","last_active":"2024-09-18T10:15:19.906859+0000","last_peered":"2024-09-18T10:15:19.906859+0000","last_clean":"2024-09-18T10:15:19.906859+0000","last_became_active":"2024-09-18T10:14:57.138546+0000","last_became_peered":"2024-09-18T10:14:57.138546+0000","last_unstale":"2024-09-18T10:15:19.906859+0000","last_undegraded":"2024-09-18T10:15:19.906859+0000","last_fullsized":"2024-09-18T10:15:19.906859+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:14:45.576168+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.10157817700000001,"stat_sum":{"num_bytes":516096,"num_objects":2,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":1,"num_write":6,"num_write_kb":784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4,13,11,2,5],"acting":[10,9,4,13,11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"13","length":"1"},{"start":"19","length":"1"}]},{"pgid":"2.2","version":"29'2","reported_seq":52,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-18T10:15:19.909866+0000","last_change":"2024-09-18T10:14:37.429583+0000","last_active":"2024-09-18T10:15:19.909866+0000","last_peered":"2024-09-18T10:15:19.909866+0000","last_clean":"2024-09-18T10:15:19.909866+0000","last_became_active":"2024-09-18T10:14:35.715374+0000","last_became_peered":"2024-09-18T10:14:35.715374+0000","last_unstale":"2024-09-18T10:15:19.909866+0000","last_undegraded":"2024-09-18T10:15:19.909866+0000","last_fullsized":"2024-09-18T10:15:19.909866+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:14:34.288596+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:03.520789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002421224,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,12],"acting":[13,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"3.0","version":"64'14","reported_seq":132,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-18T10:15:20.211561+0000","last_change":"2024-09-18T10:15:20.213644+0000","last_active":"2024-09-18T10:15:20.211561+0000","last_peered":"2024-09-18T10:15:20.211561+0000","last_clean":"2024-09-18T10:15:20.211561+0000","last_became_active":"2024-09-18T10:14:57.444374+0000","last_became_peered":"2024-09-18T10:14:57.444374+0000","last_unstale":"2024-09-18T10:15:20.211561+0000","last_undegraded":"2024-09-18T10:15:20.211561+0000","last_fullsized":"2024-09-18T10:15:20.211561+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'4","last_scrub_stamp":"2024-09-18T10:15:05.668933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:05.668933+0000","objects_scrubbed":1,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:18.433780+0000","scrub_duration":2.6328342170000001,"objects_trimmed":0,"snaptrim_duration":0.20407072700000001,"stat_sum":{"num_bytes":212992,"num_objects":2,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":1,"num_read":2,"num_read_kb":0,"num_write":2,"num_write_kb":208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,9,2,6,15,4],"acting":[3,7,9,2,6,15,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"13","length":"1"},{"start":"19","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":51,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-18T10:15:20.001122+0000","last_change":"2024-09-18T10:14:37.601046+0000","last_active":"2024-09-18T10:15:20.001122+0000","last_peered":"2024-09-18T10:15:20.001122+0000","last_clean":"2024-09-18T10:15:20.001122+0000","last_became_active":"2024-09-18T10:14:35.348262+0000","last_became_peered":"2024-09-18T10:14:35.348262+0000","last_unstale":"2024-09-18T10:15:20.001122+0000","last_undegraded":"2024-09-18T10:15:20.001122+0000","last_fullsized":"2024-09-18T10:15:20.001122+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:14:34.288596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:03.520789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063430400000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,0],"acting":[14,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"3.1","version":"60'24","reported_seq":133,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-18T10:15:19.914240+0000","last_change":"2024-09-18T10:15:20.014989+0000","last_active":"2024-09-18T10:15:19.914240+0000","last_peered":"2024-09-18T10:15:19.914240+0000","last_clean":"2024-09-18T10:15:19.914240+0000","last_became_active":"2024-09-18T10:14:57.038503+0000","last_became_peered":"2024-09-18T10:14:57.038503+0000","last_unstale":"2024-09-18T10:15:19.914240+0000","last_undegraded":"2024-09-18T10:15:19.914240+0000","last_fullsized":"2024-09-18T10:15:19.914240+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:14:45.576168+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.101111137,"stat_sum":{"num_bytes":696320,"num_objects":2,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":1779,"num_write":6,"num_write_kb":680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,13,14,10,7,3,2],"acting":[4,13,14,10,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"13","length":"1"},{"start":"19","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":53,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-18T10:15:21.938310+0000","last_change":"2024-09-18T10:14:37.600953+0000","last_active":"2024-09-18T10:15:21.938310+0000","last_peered":"2024-09-18T10:15:21.938310+0000","last_clean":"2024-09-18T10:15:21.938310+0000","last_became_active":"2024-09-18T10:14:35.349203+0000","last_became_peered":"2024-09-18T10:14:35.349203+0000","last_unstale":"2024-09-18T10:15:21.938310+0000","last_undegraded":"2024-09-18T10:15:21.938310+0000","last_fullsized":"2024-09-18T10:15:21.938310+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:14:34.288596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:15:37.868112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060130600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3],"acting":[9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2","version":"65'41","reported_seq":168,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-18T10:15:21.009814+0000","last_change":"2024-09-18T10:15:21.012194+0000","last_active":"2024-09-18T10:15:21.009814+0000","last_peered":"2024-09-18T10:15:21.009814+0000","last_clean":"2024-09-18T10:15:21.009814+0000","last_became_active":"2024-09-18T10:14:57.227520+0000","last_became_peered":"2024-09-18T10:14:57.227520+0000","last_unstale":"2024-09-18T10:15:21.009814+0000","last_undegraded":"2024-09-18T10:15:21.009814+0000","last_fullsized":"2024-09-18T10:15:21.009814+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'18","last_scrub_stamp":"2024-09-18T10:15:03.202100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:03.202100+0000","objects_scrubbed":6,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:12.722351+0000","scrub_duration":0.63960403700000001,"objects_trimmed":0,"snaptrim_duration":0.10825246400000001,"stat_sum":{"num_bytes":3112960,"num_objects":6,"num_object_clones":2,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":1,"num_read":9,"num_read_kb":2307,"num_write":7,"num_write_kb":1552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,3,13,1,5,10,7],"acting":[14,3,13,1,5,10,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"13","length":"1"},{"start":"18","length":"2"}]},{"pgid":"2.3","version":"27'1","reported_seq":52,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-18T10:15:19.900982+0000","last_change":"2024-09-18T10:14:37.801252+0000","last_active":"2024-09-18T10:15:19.900982+0000","last_peered":"2024-09-18T10:15:19.900982+0000","last_clean":"2024-09-18T10:15:19.900982+0000","last_became_active":"2024-09-18T10:14:35.487746+0000","last_became_peered":"2024-09-18T10:14:35.487746+0000","last_unstale":"2024-09-18T10:15:19.900982+0000","last_undegraded":"2024-09-18T10:15:19.900982+0000","last_fullsized":"2024-09-18T10:15:19.900982+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:14:34.288596+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:03.161727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.10072764100000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,5],"acting":[14,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"1.0","version":"29'112","reported_seq":184,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-18T10:15:19.006535+0000","last_change":"2024-09-18T10:14:32.509942+0000","last_active":"2024-09-18T10:15:19.006535+0000","last_peered":"2024-09-18T10:15:19.006535+0000","last_clean":"2024-09-18T10:15:19.006535+0000","last_became_active":"2024-09-18T10:14:32.508037+0000","last_became_peered":"2024-09-18T10:14:32.508037+0000","last_unstale":"2024-09-18T10:15:19.006535+0000","last_undegraded":"2024-09-18T10:15:19.006535+0000","last_fullsized":"2024-09-18T10:15:19.006535+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:31.267741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:31.267741+0000","last_clean_scrub_stamp":"2024-09-18T10:14:31.267741+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:15:59.262610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,15],"acting":[6,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.8","version":"41'13","reported_seq":129,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-18T10:15:19.907153+0000","last_change":"2024-09-18T10:15:19.908199+0000","last_active":"2024-09-18T10:15:19.907153+0000","last_peered":"2024-09-18T10:15:19.907153+0000","last_clean":"2024-09-18T10:15:19.907153+0000","last_became_active":"2024-09-18T10:14:57.443754+0000","last_became_peered":"2024-09-18T10:14:57.443754+0000","last_unstale":"2024-09-18T10:15:19.907153+0000","last_undegraded":"2024-09-18T10:15:19.907153+0000","last_fullsized":"2024-09-18T10:15:19.907153+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"41'13","last_scrub_stamp":"2024-09-18T10:15:05.373444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:05.373444+0000","objects_scrubbed":2,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:31.932850+0000","scrub_duration":2.8745114210000002,"objects_trimmed":0,"snaptrim_duration":0.0031925629999999998,"stat_sum":{"num_bytes":966656,"num_objects":2,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":705,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,11,3,15,1,7],"acting":[6,0,11,3,15,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"13","length":"1"},{"start":"19","length":"1"}]},{"pgid":"3.9","version":"63'56","reported_seq":174,"reported_epoch":64,"state":"active+clean+snaptrim","last_fresh":"2024-09-18T10:15:20.172992+0000","last_change":"2024-09-18T10:15:20.173063+0000","last_active":"2024-09-18T10:15:20.172992+0000","last_peered":"2024-09-18T10:15:20.172992+0000","last_clean":"2024-09-18T10:15:20.172992+0000","last_became_active":"2024-09-18T10:14:56.840271+0000","last_became_peered":"2024-09-18T10:14:56.840271+0000","last_unstale":"2024-09-18T10:15:20.172992+0000","last_undegraded":"2024-09-18T10:15:20.172992+0000","last_fullsized":"2024-09-18T10:15:20.172992+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'35","last_scrub_stamp":"2024-09-18T10:15:07.077517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:07.077517+0000","objects_scrubbed":4,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:31.443309+0000","scrub_duration":0.71488955499999995,"objects_trimmed":0,"snaptrim_duration":0.31652517000000002,"stat_sum":{"num_bytes":6062080,"num_objects":6,"num_object_clones":4,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":11,"num_read_kb":5292,"num_write":8,"num_write_kb":4704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,10,11,5,8,3],"acting":[7,2,10,11,5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"13","length":"1"}]},{"pgid":"3.a","version":"60'24","reported_seq":127,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-18T10:15:20.912853+0000","last_change":"2024-09-18T10:15:20.913618+0000","last_active":"2024-09-18T10:15:20.912853+0000","last_peered":"2024-09-18T10:15:20.912853+0000","last_clean":"2024-09-18T10:15:20.912853+0000","last_became_active":"2024-09-18T10:14:46.765703+0000","last_became_peered":"2024-09-18T10:14:46.765703+0000","last_unstale":"2024-09-18T10:15:20.912853+0000","last_undegraded":"2024-09-18T10:15:20.912853+0000","last_fullsized":"2024-09-18T10:15:20.912853+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:14:45.576168+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:08.721290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090196500000000001,"stat_sum":{"num_bytes":770048,"num_objects":4,"num_object_clones":2,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":1,"num_read":23,"num_read_kb":3549,"num_write":29,"num_write_kb":1344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,1,6,5,3,15,4],"acting":[13,1,6,5,3,15,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"13","length":"1"},{"start":"18","length":"2"}]},{"pgid":"3.b","version":"60'21","reported_seq":105,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-18T10:15:19.915264+0000","last_change":"2024-09-18T10:15:19.917336+0000","last_active":"2024-09-18T10:15:19.915264+0000","last_peered":"2024-09-18T10:15:19.915264+0000","last_clean":"2024-09-18T10:15:19.915264+0000","last_became_active":"2024-09-18T10:14:46.802834+0000","last_became_peered":"2024-09-18T10:14:46.802834+0000","last_unstale":"2024-09-18T10:15:19.915264+0000","last_undegraded":"2024-09-18T10:15:19.915264+0000","last_fullsized":"2024-09-18T10:15:19.915264+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:14:45.576168+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:04.590511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024087140000000002,"stat_sum":{"num_bytes":671744,"num_objects":1,"num_object_clones":0,"num_object_copies":7,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":31,"num_write_kb":1600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,10,12,11,1,14],"acting":[4,0,10,12,11,1,14],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"13","length":"1"},{"start":"19","length":"1"}]},{"pgid":"3.c","version":"60'74","reported_seq":159,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-18T10:15:20.032938+0000","last_change":"2024-09-18T10:15:20.033707+0000","last_active":"2024-09-18T10:15:20.032938+0000","last_peered":"2024-09-18T10:15:20.032938+0000","last_clean":"2024-09-18T10:15:20.032938+0000","last_became_active":"2024-09-18T10:14:46.795995+0000","last_became_peered":"2024-09-18T10:14:46.795995+0000","last_unstale":"2024-09-18T10:15:20.032938+0000","last_undegraded":"2024-09-18T10:15:20.032938+0000","last_fullsized":"2024-09-18T10:15:20.032938+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:14:45.576168+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:14.450533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001040731,"stat_sum":{"num_bytes":6832128,"num_objects":7,"num_object_clones":2,"num_object_copies":49,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":23,"num_read_kb":3467,"num_write":73,"num_write_kb":7344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,13,3,7,12,14],"acting":[5,8,13,3,7,12,14],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"13","length":"1"},{"start":"19","length":"1"}]},{"pgid":"3.d","version":"65'23","reported_seq":118,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-18T10:15:21.018757+0000","last_change":"2024-09-18T10:15:21.020024+0000","last_active":"2024-09-18T10:15:21.018757+0000","last_peered":"2024-09-18T10:15:21.018757+0000","last_clean":"2024-09-18T10:15:21.018757+0000","last_became_active":"2024-09-18T10:14:46.793893+0000","last_became_peered":"2024-09-18T10:14:46.793893+0000","last_unstale":"2024-09-18T10:15:21.018757+0000","last_undegraded":"2024-09-18T10:15:21.018757+0000","last_fullsized":"2024-09-18T10:15:21.018757+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:14:45.576168+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:13.756262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.106940761,"stat_sum":{"num_bytes":2899968,"num_objects":2,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":9,"num_read_kb":532,"num_write":10,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,1,11,15,9,8,4],"acting":[13,1,11,15,9,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"13","length":"1"},{"start":"18","length":"2"}]},{"pgid":"3.e","version":"66'35","reported_seq":131,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-18T10:15:23.153759+0000","last_change":"2024-09-18T10:15:22.133069+0000","last_active":"2024-09-18T10:15:23.153759+0000","last_peered":"2024-09-18T10:15:23.153759+0000","last_clean":"2024-09-18T10:15:23.153759+0000","last_became_active":"2024-09-18T10:14:46.801571+0000","last_became_peered":"2024-09-18T10:14:46.801571+0000","last_unstale":"2024-09-18T10:15:23.153759+0000","last_undegraded":"2024-09-18T10:15:23.153759+0000","last_fullsized":"2024-09-18T10:15:23.153759+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:14:45.576168+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:15:59.433777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.20719484399999999,"stat_sum":{"num_bytes":3571712,"num_objects":4,"num_object_clones":2,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":1,"num_read":8,"num_read_kb":4,"num_write":39,"num_write_kb":3488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,14,6,13,8,3,12],"acting":[11,14,6,13,8,3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"e","length":"1"},{"start":"13","length":"1"},{"start":"18","length":"2"}]},{"pgid":"3.f","version":"65'23","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-18T10:15:21.016799+0000","last_change":"2024-09-18T10:15:21.017967+0000","last_active":"2024-09-18T10:15:21.016799+0000","last_peered":"2024-09-18T10:15:21.016799+0000","last_clean":"2024-09-18T10:15:21.016799+0000","last_became_active":"2024-09-18T10:14:46.803296+0000","last_became_peered":"2024-09-18T10:14:46.803296+0000","last_unstale":"2024-09-18T10:15:21.016799+0000","last_undegraded":"2024-09-18T10:15:21.016799+0000","last_fullsized":"2024-09-18T10:15:21.016799+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:14:45.576168+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:11.624416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.105075355,"stat_sum":{"num_bytes":2129920,"num_objects":2,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":9,"num_read_kb":1780,"num_write":11,"num_write_kb":2640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,3,15,14,2,11,0],"acting":[13,3,15,14,2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"13","length":"1"},{"start":"18","length":"2"}]},{"pgid":"3.10","version":"60'14","reported_seq":79,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-18T10:15:19.904608+0000","last_change":"2024-09-18T10:15:19.906976+0000","last_active":"2024-09-18T10:15:19.904608+0000","last_peered":"2024-09-18T10:15:19.904608+0000","last_clean":"2024-09-18T10:15:19.904608+0000","last_became_active":"2024-09-18T10:15:03.217299+0000","last_became_peered":"2024-09-18T10:15:03.217299+0000","last_unstale":"2024-09-18T10:15:19.904608+0000","last_undegraded":"2024-09-18T10:15:19.904608+0000","last_fullsized":"2024-09-18T10:15:19.904608+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:14:45.576168+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025989540000000001,"stat_sum":{"num_bytes":1769472,"num_objects":1,"num_object_clones":0,"num_object_copies":7,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":9,"num_read_kb":1684,"num_write":6,"num_write_kb":2048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":884736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,5,4,8,0,9],"acting":[6,7,5,4,8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"13","length":"1"},{"start":"19","length":"1"}]},{"pgid":"3.11","version":"36'6","reported_seq":61,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-18T10:15:19.913609+0000","last_change":"2024-09-18T10:15:20.014304+0000","last_active":"2024-09-18T10:15:19.913609+0000","last_peered":"2024-09-18T10:15:19.913609+0000","last_clean":"2024-09-18T10:15:19.913609+0000","last_became_active":"2024-09-18T10:15:04.955869+0000","last_became_peered":"2024-09-18T10:15:04.955869+0000","last_unstale":"2024-09-18T10:15:19.913609+0000","last_undegraded":"2024-09-18T10:15:19.913609+0000","last_fullsized":"2024-09-18T10:15:19.913609+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":5,"last_scrub":"36'6","last_scrub_stamp":"2024-09-18T10:15:07.755163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:07.755163+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:13.113623+0000","scrub_duration":0.60886176599999997,"objects_trimmed":0,"snaptrim_duration":0.10113008699999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,13,10,0,14,2],"acting":[3,5,13,10,0,14,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"13","length":"1"},{"start":"19","length":"1"}]},{"pgid":"3.12","version":"36'8","reported_seq":141,"reported_epoch":65,"state":"active+clean+snaptrim","last_fresh":"2024-09-18T10:15:21.012250+0000","last_change":"2024-09-18T10:15:21.012388+0000","last_active":"2024-09-18T10:15:21.012250+0000","last_peered":"2024-09-18T10:15:21.012250+0000","last_clean":"2024-09-18T10:15:21.012250+0000","last_became_active":"2024-09-18T10:15:06.903185+0000","last_became_peered":"2024-09-18T10:15:06.903185+0000","last_unstale":"2024-09-18T10:15:21.012250+0000","last_undegraded":"2024-09-18T10:15:21.012250+0000","last_fullsized":"2024-09-18T10:15:21.012250+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":5,"last_scrub":"36'8","last_scrub_stamp":"2024-09-18T10:15:04.353687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:04.353687+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:28.531412+0000","scrub_duration":0.76511709400000005,"objects_trimmed":0,"snaptrim_duration":0.0052443999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,5,8,3,4,11,1],"acting":[14,5,8,3,4,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"13","length":"1"},{"start":"19","length":"1"}]},{"pgid":"3.13","version":"64'37","reported_seq":115,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-18T10:15:20.235626+0000","last_change":"2024-09-18T10:15:20.238036+0000","last_active":"2024-09-18T10:15:20.235626+0000","last_peered":"2024-09-18T10:15:20.235626+0000","last_clean":"2024-09-18T10:15:20.235626+0000","last_became_active":"2024-09-18T10:15:13.263306+0000","last_became_peered":"2024-09-18T10:15:13.263306+0000","last_unstale":"2024-09-18T10:15:20.235626+0000","last_undegraded":"2024-09-18T10:15:20.235626+0000","last_fullsized":"2024-09-18T10:15:20.235626+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":5,"last_scrub":"60'33","last_scrub_stamp":"2024-09-18T10:15:19.140411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:19.140411+0000","objects_scrubbed":1,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:39.399176+0000","scrub_duration":0.706777345,"objects_trimmed":0,"snaptrim_duration":0.20629254,"stat_sum":{"num_bytes":1720320,"num_objects":2,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":1,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,13,2,6,0,4,15],"acting":[1,13,2,6,0,4,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"13","length":"1"},{"start":"19","length":"1"}]}],"pool_stats":[{"poolid":3,"num_pg":26,"stat_sum":{"num_bytes":53125120,"num_objects":71,"num_object_clones":32,"num_object_copies":497,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":6,"num_read":178,"num_read_kb":33689,"num_write":316,"num_write_kb":43448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1638400,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":103034880,"data_stored":137261056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":666,"ondisk_log_size":666,"up":182,"acting":182,"num_store_stats":16},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":112,"ondisk_log_size":112,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":15,"up_from":22,"seq":94489280524,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303200,"kb_used_data":5836,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94068640,"statfs":{"total":96636764160,"available":96326287360,"internally_reserved":0,"allocated":5976064,"data_stored":7044415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6,10,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66400000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86199999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82699999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2170000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1830000000000001}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4490000000000001}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.296}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.474}]}]},{"osd":14,"up_from":22,"seq":94489280524,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":305620,"kb_used_data":8260,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94066220,"statfs":{"total":96636764160,"available":96323809280,"internally_reserved":0,"allocated":8458240,"data_stored":10263327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,7,8,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,0,0,0,0,5],"upper_bound":1024},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.624}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.417}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.03}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84499999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.109}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2010000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3700000000000001}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1619999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":91.111999999999995}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91100000000000003}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":78.804000000000002}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4119999999999999}]}]},{"osd":13,"up_from":22,"seq":94489280525,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":305720,"kb_used_data":8360,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94066120,"statfs":{"total":96636764160,"available":96323706880,"internally_reserved":0,"allocated":8560640,"data_stored":10824498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6,8,9,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,3,0,0,1],"upper_bound":2048},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89400000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0660000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3779999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0920000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.046}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.222}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.444}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.278}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6779999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.784}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.99}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6930000000000001}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.718}]}]},{"osd":0,"up_from":21,"seq":90194313229,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303332,"kb_used_data":5968,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94068508,"statfs":{"total":96636764160,"available":96326152192,"internally_reserved":0,"allocated":6111232,"data_stored":6892379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,4,5,6,9,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84699999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1080000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.196}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95099999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2869999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.349}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.518}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5349999999999999}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.486}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3839999999999999}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6080000000000001}]}]},{"osd":4,"up_from":21,"seq":90194313230,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303184,"kb_used_data":5828,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94068656,"statfs":{"total":96636764160,"available":96326303744,"internally_reserved":0,"allocated":5967872,"data_stored":7127789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":58.866999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":164.21199999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":59.024000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97099999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":59.292999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":59.383000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":164.65899999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5289999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":164.43899999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.103}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":59.463999999999999}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2789999999999999}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1890000000000001}]}]},{"osd":9,"up_from":21,"seq":90194313229,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":300352,"kb_used_data":2992,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94071488,"statfs":{"total":96636764160,"available":96329203712,"internally_reserved":0,"allocated":3063808,"data_stored":2966374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,8,10,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94899999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.626}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52000000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1910000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89200000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.218}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2609999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1200000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.669}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.446}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":96}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1439999999999999}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98299999999999998}]}]},{"osd":11,"up_from":21,"seq":90194313229,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308524,"kb_used_data":11160,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94063316,"statfs":{"total":96636764160,"available":96320835584,"internally_reserved":0,"allocated":11427840,"data_stored":15297371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[2,3,6,8,9,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0049999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84299999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95999999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80100000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.03}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.335}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4590000000000001}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2030000000000001}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3109999999999999}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":123.80500000000001}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6379999999999999}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5069999999999999}]}]},{"osd":12,"up_from":21,"seq":90194313230,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301396,"kb_used_data":4036,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94070444,"statfs":{"total":96636764160,"available":96328134656,"internally_reserved":0,"allocated":4132864,"data_stored":5305209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,9,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67800000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95399999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.393}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.159}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3160000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8460000000000001}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3600000000000001}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6339999999999999}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9810000000000001}]}]},{"osd":1,"up_from":22,"seq":94489280524,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301524,"kb_used_data":4164,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94070316,"statfs":{"total":96636764160,"available":96328003584,"internally_reserved":0,"allocated":4263936,"data_stored":4799263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,9,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95199999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84999999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1830000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.276}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2410000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":118.43600000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7789999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.538}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6659999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.825}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.589}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7150000000000001}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.907}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7370000000000001}]}]},{"osd":2,"up_from":22,"seq":94489280524,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303748,"kb_used_data":6392,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94068092,"statfs":{"total":96636764160,"available":96325726208,"internally_reserved":0,"allocated":6545408,"data_stored":7348973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,3,4,5,6,7,8,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":419.05700000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0469999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90300000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93300000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0880000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.111}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.486}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4079999999999999}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4410000000000001}]}]},{"osd":3,"up_from":22,"seq":94489280524,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306424,"kb_used_data":9064,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94065416,"statfs":{"total":96636764160,"available":96322985984,"internally_reserved":0,"allocated":9281536,"data_stored":13409055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,4,5,6,7,9,10,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79000000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81599999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1539999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90600000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.121}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0620000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.026}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97299999999999998}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1060000000000001}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2330000000000001}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1850000000000001}]}]},{"osd":5,"up_from":22,"seq":94489280524,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304032,"kb_used_data":6676,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94067808,"statfs":{"total":96636764160,"available":96325435392,"internally_reserved":0,"allocated":6836224,"data_stored":7557869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,6,7,8,9,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96999999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73999999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88800000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94499999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.298}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0940000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":80.272000000000006}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6479999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.345}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0009999999999999}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4430000000000001}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":101.733}]}]},{"osd":6,"up_from":22,"seq":94489280524,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":305284,"kb_used_data":7920,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94066556,"statfs":{"total":96636764160,"available":96324153344,"internally_reserved":0,"allocated":8110080,"data_stored":11097357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,7,8,9,11,12,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80500000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92300000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.343}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93899999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.343}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0429999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.294}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":95.691999999999993}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.329}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5680000000000001}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2410000000000001}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2050000000000001}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2669999999999999}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.613}]}]},{"osd":7,"up_from":22,"seq":94489280524,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306004,"kb_used_data":8648,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94065836,"statfs":{"total":96636764160,"available":96323416064,"internally_reserved":0,"allocated":8855552,"data_stored":11694829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,8,10,11,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,5,6,1,3,2],"upper_bound":2048},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":139.58600000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":139.36799999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":139.27199999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":139.51599999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.018}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":139.72800000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0860000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.861}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94299999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.127}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.885}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":139.655}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5900000000000001}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5509999999999999}]}]},{"osd":8,"up_from":22,"seq":94489280524,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306436,"kb_used_data":9076,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94065404,"statfs":{"total":96636764160,"available":96322973696,"internally_reserved":0,"allocated":9293824,"data_stored":11975455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[2,4,6,7,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93300000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.083}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94499999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85699999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0569999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0169999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0349999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.212}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.284}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.75}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":33.423000000000002}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3260000000000001}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.306}]}]},{"osd":10,"up_from":22,"seq":94489280525,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":305412,"kb_used_data":8052,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94066428,"statfs":{"total":96636764160,"available":96324022272,"internally_reserved":0,"allocated":8245248,"data_stored":11332383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,4,5,7,9,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,0,0,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79800000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0860000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0429999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1970000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1180000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4410000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7170000000000001}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.232}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":35.698999999999998}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66200000000000003}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5800000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4378624,"data_stored":5226496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":3375104,"data_stored":4313088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":6381568,"data_stored":8269824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8392704,"data_stored":11513856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":6152192,"data_stored":8032256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":6414336,"data_stored":8241152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":7360512,"data_stored":10059776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8327168,"data_stored":10874880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8404992,"data_stored":11489280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":2125824,"data_stored":2449408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":8011776,"data_stored":10780672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":10899456,"data_stored":15249408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":3190784,"data_stored":4788224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":7667712,"data_stored":10338304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":7397376,"data_stored":9605120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":4554752,"data_stored":6029312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T10:15:25.759 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-18T10:15:25.761 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-18T10:15:25.757+0000 7f14c6a1a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 22138) UID: 0 2024-09-18T10:15:25.794 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:25.794 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:25.804 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T10:15:25.861 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-18T10:15:25.858+0000 7f302f2c3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22187) UID: 0 2024-09-18T10:15:25.898 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:25.898 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:25.908 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T10:15:25.958 INFO:tasks.rados.rados.0.smithi189.stdout:234: done (2 left) 2024-09-18T10:15:25.958 INFO:tasks.rados.rados.0.smithi189.stdout:235: done (1 left) 2024-09-18T10:15:25.958 INFO:tasks.rados.rados.0.smithi189.stdout:236: done (0 left) 2024-09-18T10:15:25.958 INFO:tasks.rados.rados.0.smithi189.stdout:237: read oid 47 snap -1 2024-09-18T10:15:25.959 INFO:tasks.rados.rados.0.smithi189.stdout:237: expect (ObjNum 41 snap 0 seq_num 41) 2024-09-18T10:15:25.959 INFO:tasks.rados.rados.0.smithi189.stdout:append oid 45 current snap is 21 2024-09-18T10:15:25.959 INFO:tasks.rados.rados.0.smithi189.stdout:238: seq_num 103 ranges {2146304=573440,2719744=786432,3506176=376832} 2024-09-18T10:15:25.962 INFO:tasks.ceph.osd.11.smithi189.stderr:2024-09-18T10:15:25.958+0000 7f718b60a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 22400) UID: 0 2024-09-18T10:15:25.963 INFO:teuthology.orchestra.run.smithi029.stderr:noscrub is unset 2024-09-18T10:15:25.980 INFO:tasks.rados.rados.0.smithi189.stdout:238: writing smithi18925896-45 from 2146304 to 2719744 tid 1 2024-09-18T10:15:25.984 INFO:tasks.rados.rados.0.smithi189.stdout:238: writing smithi18925896-45 from 2719744 to 3506176 tid 2 2024-09-18T10:15:25.986 INFO:tasks.rados.rados.0.smithi189.stdout:238: writing smithi18925896-45 from 3506176 to 3883008 tid 3 2024-09-18T10:15:25.986 INFO:tasks.rados.rados.0.smithi189.stdout:239: read oid 49 snap 13 2024-09-18T10:15:25.987 INFO:tasks.rados.rados.0.smithi189.stdout:239: expect (ObjNum 48 snap 0 seq_num 48) 2024-09-18T10:15:25.987 INFO:tasks.rados.rados.0.smithi189.stdout:240: read oid 28 snap -1 2024-09-18T10:15:25.987 INFO:tasks.rados.rados.0.smithi189.stdout:240: expect (ObjNum 33 snap 0 seq_num 33) 2024-09-18T10:15:25.987 INFO:tasks.rados.rados.0.smithi189.stdout:241: read oid 11 snap -1 2024-09-18T10:15:25.987 INFO:tasks.rados.rados.0.smithi189.stdout:241: expect (ObjNum 99 snap 17 seq_num 99) 2024-09-18T10:15:25.987 INFO:tasks.rados.rados.0.smithi189.stdout:242: read oid 5 snap 9 2024-09-18T10:15:25.987 INFO:tasks.rados.rados.0.smithi189.stdout:242: expect (ObjNum 4 snap 0 seq_num 4) 2024-09-18T10:15:25.987 INFO:tasks.rados.rados.0.smithi189.stdout:243: setattr oid 48 current snap is 21 2024-09-18T10:15:25.988 INFO:tasks.rados.rados.0.smithi189.stdout:244: rollback oid 14 current snap is 21 2024-09-18T10:15:25.988 INFO:tasks.rados.rados.0.smithi189.stdout:rollback oid 14 to 16 2024-09-18T10:15:25.989 INFO:tasks.rados.rados.0.smithi189.stdout:245: delete oid 18 current snap is 21 2024-09-18T10:15:26.056 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:26.056 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:26.063 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-18T10:15:26.058+0000 7f14dcc5b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22398) UID: 0 2024-09-18T10:15:26.065 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-09-18T10:15:26.098 INFO:tasks.rados.rados.0.smithi189.stdout:245: done (8 left) 2024-09-18T10:15:26.099 INFO:tasks.rados.rados.0.smithi189.stdout:append oid 30 current snap is 21 2024-09-18T10:15:26.099 INFO:tasks.rados.rados.0.smithi189.stdout:246: seq_num 105 ranges {0=606208,606208=688128,1294336=475136} 2024-09-18T10:15:26.103 INFO:tasks.rados.rados.0.smithi189.stdout:246: writing smithi18925896-30 from 0 to 606208 tid 1 2024-09-18T10:15:26.107 INFO:tasks.rados.rados.0.smithi189.stdout:246: writing smithi18925896-30 from 606208 to 1294336 tid 2 2024-09-18T10:15:26.110 INFO:tasks.rados.rados.0.smithi189.stdout:246: writing smithi18925896-30 from 1294336 to 1769472 tid 3 2024-09-18T10:15:26.110 INFO:tasks.rados.rados.0.smithi189.stdout:247: setattr oid 24 current snap is 21 2024-09-18T10:15:26.112 INFO:tasks.rados.rados.0.smithi189.stdout:248: setattr oid 34 current snap is 21 2024-09-18T10:15:26.114 INFO:tasks.rados.rados.0.smithi189.stdout:249: snap_remove snap 8 2024-09-18T10:15:26.125 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-18T10:15:26.125 INFO:teuthology.orchestra.run.smithi029.stdout:{"fsid":"ab6d6beb-0627-4b7f-8cdb-4e0a44970f10","health":{"status":"HEALTH_WARN","checks":{"OSDMAP_FLAGS":{"severity":"HEALTH_WARN","summary":{"message":"nodeep-scrub flag(s) set","count":12},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":67,"monmap":{"epoch":1,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":70,"num_osds":16,"num_up_osds":16,"osd_up_since":1726654470,"num_in_osds":16,"osd_in_since":1726654462,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":31},{"state_name":"active+clean+snaptrim","count":3},{"state_name":"peering","count":1}],"num_pgs":35,"num_pools":3,"num_objects":75,"data_bytes":53715507,"bytes_used":4987076608,"bytes_avail":1541201149952,"bytes_total":1546188226560,"inactive_pgs_ratio":0.02857142873108387,"recovering_objects_per_sec":0,"recovering_bytes_per_sec":171964,"recovering_keys_per_sec":0,"num_objects_recovered":2,"num_bytes_recovered":688128,"num_keys_recovered":0,"read_bytes_sec":428121,"write_bytes_sec":1265171,"read_op_per_sec":2,"write_op_per_sec":2},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":2,"modified":"2024-09-18T10:15:25.210282+0000","services":{"osd":{"daemons":{"summary":"","0":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"10":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"11":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"12":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"13":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"14":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"15":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"2":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"3":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"4":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"5":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"6":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"7":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"8":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"9":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-18T10:15:26.139 INFO:tasks.ceph.ceph_manager.ceph:making progress, resetting timeout 2024-09-18T10:15:26.140 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T10:15:26.164 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:26.165 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:26.165 INFO:tasks.ceph.osd.8.smithi029.stderr:2024-09-18T10:15:26.158+0000 7f2555c27700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 22192) UID: 0 2024-09-18T10:15:26.174 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-09-18T10:15:26.265 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-18T10:15:26.258+0000 7f302f2c3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22187) UID: 0 2024-09-18T10:15:26.322 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:26.322 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:26.331 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-09-18T10:15:26.366 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-18T10:15:26.362+0000 7f302f2c3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22187) UID: 0 2024-09-18T10:15:26.426 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:26.426 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:26.435 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_ops_in_flight 2024-09-18T10:15:26.440 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-18T10:15:26.443 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-09-18T10:15:26.455 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":83,"stamp":"2024-09-18T10:15:25.950882+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":64692787,"num_objects":86,"num_object_clones":41,"num_object_copies":582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86,"num_whiteouts":8,"num_read":284,"num_read_kb":38523,"num_write":518,"num_write_kb":57050,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":6012928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":896,"ondisk_log_size":896,"up":200,"acting":200,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":207,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":16,"kb":1509949440,"kb_used":4886832,"kb_used_data":129060,"kb_used_omap":26,"kb_used_meta":4757477,"kb_avail":1505062608,"statfs":{"total":1546188226560,"available":1541184110592,"internally_reserved":0,"allocated":132157440,"data_stored":164675987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":27264,"internal_metadata":4871656832},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,1,0,0,3,0,5,1],"upper_bound":2048},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":12910592,"num_objects":11,"num_object_clones":9,"num_object_copies":77,"num_objects_missing_on_primary":-2,"num_objects_missing":-2,"num_objects_degraded":-14,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":2,"num_read":23,"num_read_kb":4761,"num_write":57,"num_write_kb":12816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":5062656,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.030016"},"pg_stats":[{"pgid":"3.19","version":"68'92","reported_seq":210,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-18T10:15:24.873328+0000","last_change":"2024-09-18T10:15:23.459301+0000","last_active":"2024-09-18T10:15:24.873328+0000","last_peered":"2024-09-18T10:15:24.873328+0000","last_clean":"2024-09-18T10:15:24.873328+0000","last_became_active":"2024-09-18T10:14:57.350904+0000","last_became_peered":"2024-09-18T10:14:57.350904+0000","last_unstale":"2024-09-18T10:15:24.873328+0000","last_undegraded":"2024-09-18T10:15:24.873328+0000","last_fullsized":"2024-09-18T10:15:24.873328+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:14:45.576168+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.41041204199999998,"stat_sum":{"num_bytes":9592832,"num_objects":10,"num_object_clones":7,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":25,"num_read_kb":7476,"num_write":57,"num_write_kb":10280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,10,11,5,8,3],"acting":[7,2,10,11,5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"e","length":"1"},{"start":"13","length":"1"},{"start":"18","length":"2"}]},{"pgid":"3.18","version":"68'36","reported_seq":159,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-18T10:15:25.252708+0000","last_change":"2024-09-18T10:15:22.137237+0000","last_active":"2024-09-18T10:15:25.252708+0000","last_peered":"2024-09-18T10:15:25.252708+0000","last_clean":"2024-09-18T10:15:25.252708+0000","last_became_active":"2024-09-18T10:14:57.647465+0000","last_became_peered":"2024-09-18T10:14:57.647465+0000","last_unstale":"2024-09-18T10:15:25.252708+0000","last_undegraded":"2024-09-18T10:15:25.252708+0000","last_fullsized":"2024-09-18T10:15:25.252708+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:14:45.576168+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.116355192,"stat_sum":{"num_bytes":3997696,"num_objects":5,"num_object_clones":3,"num_object_copies":35,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":1,"num_read":13,"num_read_kb":2309,"num_write":26,"num_write_kb":3920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,11,3,15,1,7],"acting":[6,0,11,3,15,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"e","length":"1"},{"start":"13","length":"1"},{"start":"18","length":"2"}]},{"pgid":"3.17","version":"68'35","reported_seq":76,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-18T10:15:25.001073+0000","last_change":"2024-09-18T10:15:25.001073+0000","last_active":"2024-09-18T10:15:25.001073+0000","last_peered":"2024-09-18T10:15:25.001073+0000","last_clean":"2024-09-18T10:15:25.001073+0000","last_became_active":"2024-09-18T10:15:21.323733+0000","last_became_peered":"2024-09-18T10:15:21.323733+0000","last_unstale":"2024-09-18T10:15:25.001073+0000","last_undegraded":"2024-09-18T10:15:25.001073+0000","last_fullsized":"2024-09-18T10:15:25.001073+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:14:45.576168+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.41336515200000001,"stat_sum":{"num_bytes":2883584,"num_objects":4,"num_object_clones":2,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":1,"num_read":13,"num_read_kb":4757,"num_write":13,"num_write_kb":3024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":4374528,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,13,3,14,2,11,6],"acting":[10,13,3,14,2,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"e","length":"1"},{"start":"13","length":"1"},{"start":"18","length":"2"}]},{"pgid":"3.16","version":"66'18","reported_seq":53,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-18T10:15:22.238585+0000","last_change":"2024-09-18T10:15:22.238585+0000","last_active":"2024-09-18T10:15:22.238585+0000","last_peered":"2024-09-18T10:15:22.238585+0000","last_clean":"2024-09-18T10:15:22.238585+0000","last_became_active":"2024-09-18T10:15:19.227471+0000","last_became_peered":"2024-09-18T10:15:19.227471+0000","last_unstale":"2024-09-18T10:15:22.238585+0000","last_undegraded":"2024-09-18T10:15:22.238585+0000","last_fullsized":"2024-09-18T10:15:22.238585+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":5,"last_scrub":"36'11","last_scrub_stamp":"2024-09-18T10:15:07.445786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:07.445786+0000","objects_scrubbed":1,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:23.145992+0000","scrub_duration":0.30645798400000002,"objects_trimmed":0,"snaptrim_duration":0.30014816300000002,"stat_sum":{"num_bytes":2621440,"num_objects":2,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":1888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":688128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,6,4,8,2,14,10],"acting":[9,6,4,8,2,14,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"e","length":"1"},{"start":"13","length":"1"},{"start":"18","length":"2"}]},{"pgid":"3.15","version":"65'8","reported_seq":49,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-18T10:15:21.109971+0000","last_change":"2024-09-18T10:15:21.112019+0000","last_active":"2024-09-18T10:15:21.109971+0000","last_peered":"2024-09-18T10:15:21.109971+0000","last_clean":"2024-09-18T10:15:21.109971+0000","last_became_active":"2024-09-18T10:15:17.127771+0000","last_became_peered":"2024-09-18T10:15:17.127771+0000","last_unstale":"2024-09-18T10:15:21.109971+0000","last_undegraded":"2024-09-18T10:15:21.109971+0000","last_fullsized":"2024-09-18T10:15:21.109971+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:14:45.576168+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.20870269999999999,"stat_sum":{"num_bytes":32768,"num_objects":2,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":1,"num_read":2,"num_read_kb":0,"num_write":2,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,15,13,10,1,7,9],"acting":[14,15,13,10,1,7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"13","length":"1"},{"start":"18","length":"2"}]},{"pgid":"3.14","version":"37'8","reported_seq":54,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-18T10:15:23.962750+0000","last_change":"2024-09-18T10:15:21.930993+0000","last_active":"2024-09-18T10:15:23.962750+0000","last_peered":"2024-09-18T10:15:23.962750+0000","last_clean":"2024-09-18T10:15:23.962750+0000","last_became_active":"2024-09-18T10:15:14.767770+0000","last_became_peered":"2024-09-18T10:15:14.767770+0000","last_unstale":"2024-09-18T10:15:23.962750+0000","last_undegraded":"2024-09-18T10:15:23.962750+0000","last_fullsized":"2024-09-18T10:15:23.962750+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":5,"last_scrub":"37'8","last_scrub_stamp":"2024-09-18T10:15:17.041590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:17.041590+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:32.741813+0000","scrub_duration":0.60018000800000004,"objects_trimmed":0,"snaptrim_duration":0.0014793740000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,14,9,10,5,13,4],"acting":[1,14,9,10,5,13,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"e","length":"1"},{"start":"13","length":"1"},{"start":"18","length":"2"}]},{"pgid":"2.7","version":"0'0","reported_seq":50,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-18T10:15:19.910703+0000","last_change":"2024-09-18T10:14:37.427722+0000","last_active":"2024-09-18T10:15:19.910703+0000","last_peered":"2024-09-18T10:15:19.910703+0000","last_clean":"2024-09-18T10:15:19.910703+0000","last_became_active":"2024-09-18T10:14:35.348943+0000","last_became_peered":"2024-09-18T10:14:35.348943+0000","last_unstale":"2024-09-18T10:15:19.910703+0000","last_undegraded":"2024-09-18T10:15:19.910703+0000","last_fullsized":"2024-09-18T10:15:19.910703+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:14:34.288596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:03.161727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060442999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,10],"acting":[13,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"3.6","version":"57'32","reported_seq":167,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-18T10:15:23.069165+0000","last_change":"2024-09-18T10:15:21.935648+0000","last_active":"2024-09-18T10:15:23.069165+0000","last_peered":"2024-09-18T10:15:23.069165+0000","last_clean":"2024-09-18T10:15:23.069165+0000","last_became_active":"2024-09-18T10:14:57.139786+0000","last_became_peered":"2024-09-18T10:14:57.139786+0000","last_unstale":"2024-09-18T10:15:23.069165+0000","last_undegraded":"2024-09-18T10:15:23.069165+0000","last_fullsized":"2024-09-18T10:15:23.069165+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:14:45.576168+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010832179999999999,"stat_sum":{"num_bytes":868352,"num_objects":2,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":988,"num_write":11,"num_write_kb":1960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,14,6,2,5,9,15],"acting":[0,14,6,2,5,9,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"e","length":"1"},{"start":"13","length":"1"},{"start":"18","length":"2"}]},{"pgid":"2.6","version":"0'0","reported_seq":54,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-18T10:15:24.160814+0000","last_change":"2024-09-18T10:14:38.963300+0000","last_active":"2024-09-18T10:15:24.160814+0000","last_peered":"2024-09-18T10:15:24.160814+0000","last_clean":"2024-09-18T10:15:24.160814+0000","last_became_active":"2024-09-18T10:14:35.429859+0000","last_became_peered":"2024-09-18T10:14:35.429859+0000","last_unstale":"2024-09-18T10:15:24.160814+0000","last_undegraded":"2024-09-18T10:15:24.160814+0000","last_fullsized":"2024-09-18T10:15:24.160814+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:14:34.288596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:15:40.196292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037634499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11],"acting":[8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.7","version":"66'39","reported_seq":171,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-18T10:15:23.959984+0000","last_change":"2024-09-18T10:15:22.345767+0000","last_active":"2024-09-18T10:15:23.959984+0000","last_peered":"2024-09-18T10:15:23.959984+0000","last_clean":"2024-09-18T10:15:23.959984+0000","last_became_active":"2024-09-18T10:14:57.137776+0000","last_became_peered":"2024-09-18T10:14:57.137776+0000","last_unstale":"2024-09-18T10:15:23.959984+0000","last_undegraded":"2024-09-18T10:15:23.959984+0000","last_fullsized":"2024-09-18T10:15:23.959984+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"60'30","last_scrub_stamp":"2024-09-18T10:15:17.136535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:17.136535+0000","objects_scrubbed":3,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:35.499009+0000","scrub_duration":0.60680385400000003,"objects_trimmed":0,"snaptrim_duration":0.41230857199999998,"stat_sum":{"num_bytes":5324800,"num_objects":4,"num_object_clones":2,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6,0,14,10,13],"acting":[4,11,6,0,14,10,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"e","length":"1"},{"start":"13","length":"1"},{"start":"18","length":"2"}]},{"pgid":"2.5","version":"0'0","reported_seq":53,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-18T10:15:24.153802+0000","last_change":"2024-09-18T10:14:37.327877+0000","last_active":"2024-09-18T10:15:24.153802+0000","last_peered":"2024-09-18T10:15:24.153802+0000","last_clean":"2024-09-18T10:15:24.153802+0000","last_became_active":"2024-09-18T10:14:35.329198+0000","last_became_peered":"2024-09-18T10:14:35.329198+0000","last_unstale":"2024-09-18T10:15:24.153802+0000","last_undegraded":"2024-09-18T10:15:24.153802+0000","last_fullsized":"2024-09-18T10:15:24.153802+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:14:34.288596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:15:44.196046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062415699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"66'40","reported_seq":177,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-18T10:15:24.160601+0000","last_change":"2024-09-18T10:15:22.747902+0000","last_active":"2024-09-18T10:15:24.160601+0000","last_peered":"2024-09-18T10:15:24.160601+0000","last_clean":"2024-09-18T10:15:24.160601+0000","last_became_active":"2024-09-18T10:14:57.134641+0000","last_became_peered":"2024-09-18T10:14:57.134641+0000","last_unstale":"2024-09-18T10:15:24.160601+0000","last_undegraded":"2024-09-18T10:15:24.160601+0000","last_fullsized":"2024-09-18T10:15:24.160601+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"60'34","last_scrub_stamp":"2024-09-18T10:15:17.325003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:17.325003+0000","objects_scrubbed":6,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:18.565493+0000","scrub_duration":0.82233752999999998,"objects_trimmed":0,"snaptrim_duration":0.81593390099999996,"stat_sum":{"num_bytes":4456448,"num_objects":6,"num_object_clones":3,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":4,"num_read_kb":1815,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,15,4,2,6,10,7],"acting":[8,15,4,2,6,10,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"e","length":"1"},{"start":"13","length":"1"},{"start":"18","length":"2"}]},{"pgid":"2.4","version":"0'0","reported_seq":54,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-18T10:15:23.959787+0000","last_change":"2024-09-18T10:14:37.426192+0000","last_active":"2024-09-18T10:15:23.959787+0000","last_peered":"2024-09-18T10:15:23.959787+0000","last_clean":"2024-09-18T10:15:23.959787+0000","last_became_active":"2024-09-18T10:14:35.524102+0000","last_became_peered":"2024-09-18T10:14:35.524102+0000","last_unstale":"2024-09-18T10:15:23.959787+0000","last_undegraded":"2024-09-18T10:15:23.959787+0000","last_fullsized":"2024-09-18T10:15:23.959787+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:14:34.288596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:15:59.190171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.10077985,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.5","version":"35'2","reported_seq":126,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-18T10:15:23.936668+0000","last_change":"2024-09-18T10:15:22.136251+0000","last_active":"2024-09-18T10:15:23.936668+0000","last_peered":"2024-09-18T10:15:23.936668+0000","last_clean":"2024-09-18T10:15:23.936668+0000","last_became_active":"2024-09-18T10:14:57.135810+0000","last_became_peered":"2024-09-18T10:14:57.135810+0000","last_unstale":"2024-09-18T10:15:23.936668+0000","last_undegraded":"2024-09-18T10:15:23.936668+0000","last_fullsized":"2024-09-18T10:15:23.936668+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"35'2","last_scrub_stamp":"2024-09-18T10:15:03.886316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:03.886316+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:14.307127+0000","scrub_duration":0.35694589500000001,"objects_trimmed":0,"snaptrim_duration":0.101978342,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,4,9,12,0,11],"acting":[5,8,4,9,12,0,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"e","length":"1"},{"start":"13","length":"1"},{"start":"18","length":"2"}]},{"pgid":"3.3","version":"65'23","reported_seq":141,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-18T10:15:24.064082+0000","last_change":"2024-09-18T10:15:22.033003+0000","last_active":"2024-09-18T10:15:24.064082+0000","last_peered":"2024-09-18T10:15:24.064082+0000","last_clean":"2024-09-18T10:15:24.064082+0000","last_became_active":"2024-09-18T10:14:57.138546+0000","last_became_peered":"2024-09-18T10:14:57.138546+0000","last_unstale":"2024-09-18T10:15:24.064082+0000","last_undegraded":"2024-09-18T10:15:24.064082+0000","last_fullsized":"2024-09-18T10:15:24.064082+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:14:45.576168+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001309478,"stat_sum":{"num_bytes":516096,"num_objects":2,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":1,"num_write":6,"num_write_kb":784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4,13,11,2,5],"acting":[10,9,4,13,11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"e","length":"1"},{"start":"13","length":"1"},{"start":"18","length":"2"}]},{"pgid":"2.2","version":"29'2","reported_seq":52,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-18T10:15:19.909866+0000","last_change":"2024-09-18T10:14:37.429583+0000","last_active":"2024-09-18T10:15:19.909866+0000","last_peered":"2024-09-18T10:15:19.909866+0000","last_clean":"2024-09-18T10:15:19.909866+0000","last_became_active":"2024-09-18T10:14:35.715374+0000","last_became_peered":"2024-09-18T10:14:35.715374+0000","last_unstale":"2024-09-18T10:15:19.909866+0000","last_undegraded":"2024-09-18T10:15:19.909866+0000","last_fullsized":"2024-09-18T10:15:19.909866+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:14:34.288596+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:03.520789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002421224,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,12],"acting":[13,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"3.0","version":"66'18","reported_seq":149,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-18T10:15:24.058769+0000","last_change":"2024-09-18T10:15:22.138999+0000","last_active":"2024-09-18T10:15:24.058769+0000","last_peered":"2024-09-18T10:15:24.058769+0000","last_clean":"2024-09-18T10:15:24.058769+0000","last_became_active":"2024-09-18T10:14:57.444374+0000","last_became_peered":"2024-09-18T10:14:57.444374+0000","last_unstale":"2024-09-18T10:15:24.058769+0000","last_undegraded":"2024-09-18T10:15:24.058769+0000","last_fullsized":"2024-09-18T10:15:24.058769+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'4","last_scrub_stamp":"2024-09-18T10:15:05.668933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:05.668933+0000","objects_scrubbed":1,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:18.433780+0000","scrub_duration":2.6328342170000001,"objects_trimmed":0,"snaptrim_duration":0.20786020399999999,"stat_sum":{"num_bytes":212992,"num_objects":2,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":1,"num_read":2,"num_read_kb":0,"num_write":2,"num_write_kb":208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,9,2,6,15,4],"acting":[3,7,9,2,6,15,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"e","length":"1"},{"start":"13","length":"1"},{"start":"18","length":"2"}]},{"pgid":"2.1","version":"0'0","reported_seq":51,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-18T10:15:20.001122+0000","last_change":"2024-09-18T10:14:37.601046+0000","last_active":"2024-09-18T10:15:20.001122+0000","last_peered":"2024-09-18T10:15:20.001122+0000","last_clean":"2024-09-18T10:15:20.001122+0000","last_became_active":"2024-09-18T10:14:35.348262+0000","last_became_peered":"2024-09-18T10:14:35.348262+0000","last_unstale":"2024-09-18T10:15:20.001122+0000","last_undegraded":"2024-09-18T10:15:20.001122+0000","last_fullsized":"2024-09-18T10:15:20.001122+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:14:34.288596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:03.520789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063430400000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,0],"acting":[14,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"3.1","version":"68'30","reported_seq":151,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-18T10:15:24.166023+0000","last_change":"2024-09-18T10:15:22.436295+0000","last_active":"2024-09-18T10:15:24.166023+0000","last_peered":"2024-09-18T10:15:24.166023+0000","last_clean":"2024-09-18T10:15:24.166023+0000","last_became_active":"2024-09-18T10:14:57.038503+0000","last_became_peered":"2024-09-18T10:14:57.038503+0000","last_unstale":"2024-09-18T10:15:24.166023+0000","last_undegraded":"2024-09-18T10:15:24.166023+0000","last_fullsized":"2024-09-18T10:15:24.166023+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:14:45.576168+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.50270893999999999,"stat_sum":{"num_bytes":696320,"num_objects":3,"num_object_clones":2,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":10,"num_read_kb":1779,"num_write":8,"num_write_kb":680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,13,14,10,7,3,2],"acting":[4,13,14,10,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"e","length":"1"},{"start":"13","length":"1"},{"start":"18","length":"2"}]},{"pgid":"2.0","version":"0'0","reported_seq":53,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-18T10:15:21.938310+0000","last_change":"2024-09-18T10:14:37.600953+0000","last_active":"2024-09-18T10:15:21.938310+0000","last_peered":"2024-09-18T10:15:21.938310+0000","last_clean":"2024-09-18T10:15:21.938310+0000","last_became_active":"2024-09-18T10:14:35.349203+0000","last_became_peered":"2024-09-18T10:14:35.349203+0000","last_unstale":"2024-09-18T10:15:21.938310+0000","last_undegraded":"2024-09-18T10:15:21.938310+0000","last_fullsized":"2024-09-18T10:15:21.938310+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:14:34.288596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:15:37.868112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060130600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3],"acting":[9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2","version":"65'41","reported_seq":168,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-18T10:15:21.009814+0000","last_change":"2024-09-18T10:15:21.012194+0000","last_active":"2024-09-18T10:15:21.009814+0000","last_peered":"2024-09-18T10:15:21.009814+0000","last_clean":"2024-09-18T10:15:21.009814+0000","last_became_active":"2024-09-18T10:14:57.227520+0000","last_became_peered":"2024-09-18T10:14:57.227520+0000","last_unstale":"2024-09-18T10:15:21.009814+0000","last_undegraded":"2024-09-18T10:15:21.009814+0000","last_fullsized":"2024-09-18T10:15:21.009814+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'18","last_scrub_stamp":"2024-09-18T10:15:03.202100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:03.202100+0000","objects_scrubbed":6,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:12.722351+0000","scrub_duration":0.63960403700000001,"objects_trimmed":0,"snaptrim_duration":0.10825246400000001,"stat_sum":{"num_bytes":3112960,"num_objects":6,"num_object_clones":2,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":1,"num_read":9,"num_read_kb":2307,"num_write":7,"num_write_kb":1552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,3,13,1,5,10,7],"acting":[14,3,13,1,5,10,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"13","length":"1"},{"start":"18","length":"2"}]},{"pgid":"2.3","version":"27'1","reported_seq":52,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-18T10:15:19.900982+0000","last_change":"2024-09-18T10:14:37.801252+0000","last_active":"2024-09-18T10:15:19.900982+0000","last_peered":"2024-09-18T10:15:19.900982+0000","last_clean":"2024-09-18T10:15:19.900982+0000","last_became_active":"2024-09-18T10:14:35.487746+0000","last_became_peered":"2024-09-18T10:14:35.487746+0000","last_unstale":"2024-09-18T10:15:19.900982+0000","last_undegraded":"2024-09-18T10:15:19.900982+0000","last_fullsized":"2024-09-18T10:15:19.900982+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:14:34.288596+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:03.161727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.10072764100000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,5],"acting":[14,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"1.0","version":"29'112","reported_seq":189,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-18T10:15:23.939109+0000","last_change":"2024-09-18T10:14:32.509942+0000","last_active":"2024-09-18T10:15:23.939109+0000","last_peered":"2024-09-18T10:15:23.939109+0000","last_clean":"2024-09-18T10:15:23.939109+0000","last_became_active":"2024-09-18T10:14:32.508037+0000","last_became_peered":"2024-09-18T10:14:32.508037+0000","last_unstale":"2024-09-18T10:15:23.939109+0000","last_undegraded":"2024-09-18T10:15:23.939109+0000","last_fullsized":"2024-09-18T10:15:23.939109+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:31.267741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:31.267741+0000","last_clean_scrub_stamp":"2024-09-18T10:14:31.267741+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:15:59.262610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,15],"acting":[6,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.8","version":"41'13","reported_seq":142,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-18T10:15:24.039045+0000","last_change":"2024-09-18T10:15:22.123091+0000","last_active":"2024-09-18T10:15:24.039045+0000","last_peered":"2024-09-18T10:15:24.039045+0000","last_clean":"2024-09-18T10:15:24.039045+0000","last_became_active":"2024-09-18T10:14:57.443754+0000","last_became_peered":"2024-09-18T10:14:57.443754+0000","last_unstale":"2024-09-18T10:15:24.039045+0000","last_undegraded":"2024-09-18T10:15:24.039045+0000","last_fullsized":"2024-09-18T10:15:24.039045+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"41'13","last_scrub_stamp":"2024-09-18T10:15:05.373444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:05.373444+0000","objects_scrubbed":2,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:31.932850+0000","scrub_duration":2.8745114210000002,"objects_trimmed":0,"snaptrim_duration":0.0012354600000000001,"stat_sum":{"num_bytes":966656,"num_objects":2,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":705,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,11,3,15,1,7],"acting":[6,0,11,3,15,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"e","length":"1"},{"start":"13","length":"1"},{"start":"18","length":"2"}]},{"pgid":"3.9","version":"69'68","reported_seq":197,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-18T10:15:25.146780+0000","last_change":"2024-09-18T10:15:23.152029+0000","last_active":"2024-09-18T10:15:25.146780+0000","last_peered":"2024-09-18T10:15:25.146780+0000","last_clean":"2024-09-18T10:15:25.146780+0000","last_became_active":"2024-09-18T10:14:56.840271+0000","last_became_peered":"2024-09-18T10:14:56.840271+0000","last_unstale":"2024-09-18T10:15:25.146780+0000","last_undegraded":"2024-09-18T10:15:25.146780+0000","last_fullsized":"2024-09-18T10:15:25.146780+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'35","last_scrub_stamp":"2024-09-18T10:15:07.077517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:07.077517+0000","objects_scrubbed":4,"log_size":68,"log_dups_size":0,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:31.443309+0000","scrub_duration":0.71488955499999995,"objects_trimmed":0,"snaptrim_duration":0.0014440639999999999,"stat_sum":{"num_bytes":6766592,"num_objects":8,"num_object_clones":6,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":12,"num_read_kb":5292,"num_write":29,"num_write_kb":5392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,10,11,5,8,3],"acting":[7,2,10,11,5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"e","length":"1"},{"start":"13","length":"1"},{"start":"18","length":"2"}]},{"pgid":"3.a","version":"60'24","reported_seq":127,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-18T10:15:20.912853+0000","last_change":"2024-09-18T10:15:20.913618+0000","last_active":"2024-09-18T10:15:20.912853+0000","last_peered":"2024-09-18T10:15:20.912853+0000","last_clean":"2024-09-18T10:15:20.912853+0000","last_became_active":"2024-09-18T10:14:46.765703+0000","last_became_peered":"2024-09-18T10:14:46.765703+0000","last_unstale":"2024-09-18T10:15:20.912853+0000","last_undegraded":"2024-09-18T10:15:20.912853+0000","last_fullsized":"2024-09-18T10:15:20.912853+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:14:45.576168+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:08.721290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090196500000000001,"stat_sum":{"num_bytes":770048,"num_objects":4,"num_object_clones":2,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":1,"num_read":23,"num_read_kb":3549,"num_write":29,"num_write_kb":1344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,1,6,5,3,15,4],"acting":[13,1,6,5,3,15,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"13","length":"1"},{"start":"18","length":"2"}]},{"pgid":"3.b","version":"60'21","reported_seq":118,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-18T10:15:23.959955+0000","last_change":"2024-09-18T10:15:22.446437+0000","last_active":"2024-09-18T10:15:23.959955+0000","last_peered":"2024-09-18T10:15:23.959955+0000","last_clean":"2024-09-18T10:15:23.959955+0000","last_became_active":"2024-09-18T10:14:46.802834+0000","last_became_peered":"2024-09-18T10:14:46.802834+0000","last_unstale":"2024-09-18T10:15:23.959955+0000","last_undegraded":"2024-09-18T10:15:23.959955+0000","last_fullsized":"2024-09-18T10:15:23.959955+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:14:45.576168+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:04.590511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.41299139200000001,"stat_sum":{"num_bytes":671744,"num_objects":1,"num_object_clones":0,"num_object_copies":7,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":31,"num_write_kb":1600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,10,12,11,1,14],"acting":[4,0,10,12,11,1,14],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"e","length":"1"},{"start":"13","length":"1"},{"start":"18","length":"2"}]},{"pgid":"3.c","version":"66'83","reported_seq":181,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-18T10:15:23.936682+0000","last_change":"2024-09-18T10:15:22.846529+0000","last_active":"2024-09-18T10:15:23.936682+0000","last_peered":"2024-09-18T10:15:23.936682+0000","last_clean":"2024-09-18T10:15:23.936682+0000","last_became_active":"2024-09-18T10:14:46.795995+0000","last_became_peered":"2024-09-18T10:14:46.795995+0000","last_unstale":"2024-09-18T10:15:23.936682+0000","last_undegraded":"2024-09-18T10:15:23.936682+0000","last_fullsized":"2024-09-18T10:15:23.936682+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:14:45.576168+0000","objects_scrubbed":0,"log_size":83,"log_dups_size":0,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:14.450533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.77562220400000004,"stat_sum":{"num_bytes":8519680,"num_objects":8,"num_object_clones":3,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":24,"num_read_kb":3468,"num_write":77,"num_write_kb":8992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,13,3,7,12,14],"acting":[5,8,13,3,7,12,14],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"e","length":"1"},{"start":"13","length":"1"},{"start":"18","length":"2"}]},{"pgid":"3.d","version":"65'23","reported_seq":118,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-18T10:15:21.018757+0000","last_change":"2024-09-18T10:15:21.020024+0000","last_active":"2024-09-18T10:15:21.018757+0000","last_peered":"2024-09-18T10:15:21.018757+0000","last_clean":"2024-09-18T10:15:21.018757+0000","last_became_active":"2024-09-18T10:14:46.793893+0000","last_became_peered":"2024-09-18T10:14:46.793893+0000","last_unstale":"2024-09-18T10:15:21.018757+0000","last_undegraded":"2024-09-18T10:15:21.018757+0000","last_fullsized":"2024-09-18T10:15:21.018757+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:14:45.576168+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:13.756262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.106940761,"stat_sum":{"num_bytes":2899968,"num_objects":2,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":9,"num_read_kb":532,"num_write":10,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,1,11,15,9,8,4],"acting":[13,1,11,15,9,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"13","length":"1"},{"start":"18","length":"2"}]},{"pgid":"3.e","version":"66'35","reported_seq":131,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-18T10:15:23.153759+0000","last_change":"2024-09-18T10:15:22.133069+0000","last_active":"2024-09-18T10:15:23.153759+0000","last_peered":"2024-09-18T10:15:23.153759+0000","last_clean":"2024-09-18T10:15:23.153759+0000","last_became_active":"2024-09-18T10:14:46.801571+0000","last_became_peered":"2024-09-18T10:14:46.801571+0000","last_unstale":"2024-09-18T10:15:23.153759+0000","last_undegraded":"2024-09-18T10:15:23.153759+0000","last_fullsized":"2024-09-18T10:15:23.153759+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:14:45.576168+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:15:59.433777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.20719484399999999,"stat_sum":{"num_bytes":3571712,"num_objects":4,"num_object_clones":2,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":1,"num_read":8,"num_read_kb":4,"num_write":39,"num_write_kb":3488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,14,6,13,8,3,12],"acting":[11,14,6,13,8,3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"e","length":"1"},{"start":"13","length":"1"},{"start":"18","length":"2"}]},{"pgid":"3.f","version":"65'23","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-18T10:15:21.016799+0000","last_change":"2024-09-18T10:15:21.017967+0000","last_active":"2024-09-18T10:15:21.016799+0000","last_peered":"2024-09-18T10:15:21.016799+0000","last_clean":"2024-09-18T10:15:21.016799+0000","last_became_active":"2024-09-18T10:14:46.803296+0000","last_became_peered":"2024-09-18T10:14:46.803296+0000","last_unstale":"2024-09-18T10:15:21.016799+0000","last_undegraded":"2024-09-18T10:15:21.016799+0000","last_fullsized":"2024-09-18T10:15:21.016799+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:14:45.576168+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:11.624416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.105075355,"stat_sum":{"num_bytes":2129920,"num_objects":2,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":9,"num_read_kb":1780,"num_write":11,"num_write_kb":2640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,3,15,14,2,11,0],"acting":[13,3,15,14,2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"13","length":"1"},{"start":"18","length":"2"}]},{"pgid":"3.10","version":"60'14","reported_seq":92,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-18T10:15:23.938487+0000","last_change":"2024-09-18T10:15:22.034432+0000","last_active":"2024-09-18T10:15:23.938487+0000","last_peered":"2024-09-18T10:15:23.938487+0000","last_clean":"2024-09-18T10:15:23.938487+0000","last_became_active":"2024-09-18T10:15:03.217299+0000","last_became_peered":"2024-09-18T10:15:03.217299+0000","last_unstale":"2024-09-18T10:15:23.938487+0000","last_undegraded":"2024-09-18T10:15:23.938487+0000","last_fullsized":"2024-09-18T10:15:23.938487+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:14:45.576168+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.006433032,"stat_sum":{"num_bytes":1769472,"num_objects":1,"num_object_clones":0,"num_object_copies":7,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":9,"num_read_kb":1684,"num_write":6,"num_write_kb":2048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":884736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,5,4,8,0,9],"acting":[6,7,5,4,8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"e","length":"1"},{"start":"13","length":"1"},{"start":"18","length":"2"}]},{"pgid":"3.11","version":"36'6","reported_seq":74,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-18T10:15:23.960513+0000","last_change":"2024-09-18T10:15:21.933314+0000","last_active":"2024-09-18T10:15:23.960513+0000","last_peered":"2024-09-18T10:15:23.960513+0000","last_clean":"2024-09-18T10:15:23.960513+0000","last_became_active":"2024-09-18T10:15:04.955869+0000","last_became_peered":"2024-09-18T10:15:04.955869+0000","last_unstale":"2024-09-18T10:15:23.960513+0000","last_undegraded":"2024-09-18T10:15:23.960513+0000","last_fullsized":"2024-09-18T10:15:23.960513+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":5,"last_scrub":"36'6","last_scrub_stamp":"2024-09-18T10:15:07.755163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:07.755163+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:13.113623+0000","scrub_duration":0.60886176599999997,"objects_trimmed":0,"snaptrim_duration":0.002492476,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,13,10,0,14,2],"acting":[3,5,13,10,0,14,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"e","length":"1"},{"start":"13","length":"1"},{"start":"18","length":"2"}]},{"pgid":"3.12","version":"36'8","reported_seq":141,"reported_epoch":65,"state":"active+clean+snaptrim","last_fresh":"2024-09-18T10:15:21.012250+0000","last_change":"2024-09-18T10:15:21.012388+0000","last_active":"2024-09-18T10:15:21.012250+0000","last_peered":"2024-09-18T10:15:21.012250+0000","last_clean":"2024-09-18T10:15:21.012250+0000","last_became_active":"2024-09-18T10:15:06.903185+0000","last_became_peered":"2024-09-18T10:15:06.903185+0000","last_unstale":"2024-09-18T10:15:21.012250+0000","last_undegraded":"2024-09-18T10:15:21.012250+0000","last_fullsized":"2024-09-18T10:15:21.012250+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":5,"last_scrub":"36'8","last_scrub_stamp":"2024-09-18T10:15:04.353687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:04.353687+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:28.531412+0000","scrub_duration":0.76511709400000005,"objects_trimmed":0,"snaptrim_duration":0.0052443999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,5,8,3,4,11,1],"acting":[14,5,8,3,4,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"13","length":"1"},{"start":"19","length":"1"}]},{"pgid":"3.13","version":"66'41","reported_seq":132,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-18T10:15:23.962792+0000","last_change":"2024-09-18T10:15:22.039043+0000","last_active":"2024-09-18T10:15:23.962792+0000","last_peered":"2024-09-18T10:15:23.962792+0000","last_clean":"2024-09-18T10:15:23.962792+0000","last_became_active":"2024-09-18T10:15:13.263306+0000","last_became_peered":"2024-09-18T10:15:13.263306+0000","last_unstale":"2024-09-18T10:15:23.962792+0000","last_undegraded":"2024-09-18T10:15:23.962792+0000","last_fullsized":"2024-09-18T10:15:23.962792+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":5,"last_scrub":"60'33","last_scrub_stamp":"2024-09-18T10:15:19.140411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:19.140411+0000","objects_scrubbed":1,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:39.399176+0000","scrub_duration":0.706777345,"objects_trimmed":0,"snaptrim_duration":0.109764865,"stat_sum":{"num_bytes":1720320,"num_objects":2,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":1,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,13,2,6,0,4,15],"acting":[1,13,2,6,0,4,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"e","length":"1"},{"start":"13","length":"1"},{"start":"18","length":"2"}]}],"pool_stats":[{"poolid":3,"num_pg":26,"stat_sum":{"num_bytes":64102400,"num_objects":82,"num_object_clones":41,"num_object_copies":574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":8,"num_read":198,"num_read_kb":38450,"num_write":371,"num_write_kb":54376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":6012928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":115507200,"data_stored":155090944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":781,"ondisk_log_size":781,"up":182,"acting":182,"num_store_stats":16},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":112,"ondisk_log_size":112,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":15,"up_from":22,"seq":94489280525,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303260,"kb_used_data":5904,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94068580,"statfs":{"total":96636764160,"available":96326225920,"internally_reserved":0,"allocated":6045696,"data_stored":6634875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6,10,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.024}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96199999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79600000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92000000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.177}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2110000000000001}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":86.546999999999997}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3400000000000001}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.151}]}]},{"osd":14,"up_from":22,"seq":94489280524,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":305620,"kb_used_data":8260,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94066220,"statfs":{"total":96636764160,"available":96323809280,"internally_reserved":0,"allocated":8458240,"data_stored":10263327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,7,8,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,0,0,0,0,5],"upper_bound":1024},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.624}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.417}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.03}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84499999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.109}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2010000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3700000000000001}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1619999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":91.111999999999995}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91100000000000003}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":78.804000000000002}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4119999999999999}]}]},{"osd":13,"up_from":22,"seq":94489280525,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":305720,"kb_used_data":8360,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94066120,"statfs":{"total":96636764160,"available":96323706880,"internally_reserved":0,"allocated":8560640,"data_stored":10824498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6,8,9,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,3,0,0,1],"upper_bound":2048},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89400000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0660000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3779999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0920000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.046}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.222}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.444}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.278}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6779999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.784}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.99}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6930000000000001}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.718}]}]},{"osd":0,"up_from":21,"seq":90194313229,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303332,"kb_used_data":5968,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94068508,"statfs":{"total":96636764160,"available":96326152192,"internally_reserved":0,"allocated":6111232,"data_stored":6892379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,4,5,6,9,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84699999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1080000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.196}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95099999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2869999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.349}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.518}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5349999999999999}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.486}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3839999999999999}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6080000000000001}]}]},{"osd":4,"up_from":21,"seq":90194313231,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304304,"kb_used_data":6940,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94067536,"statfs":{"total":96636764160,"available":96325156864,"internally_reserved":0,"allocated":7106560,"data_stored":8592219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93100000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.04}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.421}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6579999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.702}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.52}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.294}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5289999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.395}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9399999999999999}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9690000000000001}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8560000000000001}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.081}]}]},{"osd":9,"up_from":21,"seq":90194313229,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":300352,"kb_used_data":2992,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94071488,"statfs":{"total":96636764160,"available":96329203712,"internally_reserved":0,"allocated":3063808,"data_stored":2966374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,8,10,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94899999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.626}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52000000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1910000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89200000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.218}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2609999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1200000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.669}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.446}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":96}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1439999999999999}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98299999999999998}]}]},{"osd":11,"up_from":21,"seq":90194313229,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308524,"kb_used_data":11160,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94063316,"statfs":{"total":96636764160,"available":96320835584,"internally_reserved":0,"allocated":11427840,"data_stored":15297371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[2,3,6,8,9,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0049999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84299999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95999999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80100000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.03}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.335}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4590000000000001}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2030000000000001}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3109999999999999}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":123.80500000000001}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6379999999999999}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5069999999999999}]}]},{"osd":12,"up_from":21,"seq":90194313230,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301396,"kb_used_data":4036,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94070444,"statfs":{"total":96636764160,"available":96328134656,"internally_reserved":0,"allocated":4132864,"data_stored":5305209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,9,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67800000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95399999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.393}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.159}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3160000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8460000000000001}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3600000000000001}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6339999999999999}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9810000000000001}]}]},{"osd":1,"up_from":22,"seq":94489280525,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301764,"kb_used_data":4400,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94070076,"statfs":{"total":96636764160,"available":96327757824,"internally_reserved":0,"allocated":4505600,"data_stored":4561755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,9,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67800000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0269999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1659999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5029999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82299999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":118.43600000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7789999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94399999999999995}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6850000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.825}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.339}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7529999999999999}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.851}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7250000000000001}]}]},{"osd":2,"up_from":22,"seq":94489280525,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306648,"kb_used_data":9284,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94065192,"statfs":{"total":96636764160,"available":96322756608,"internally_reserved":0,"allocated":9506816,"data_stored":11844443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,3,4,5,6,7,8,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.02}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1990000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92300000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.226}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4199999999999999}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5509999999999999}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3460000000000001}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.486}]}]},{"osd":3,"up_from":22,"seq":94489280525,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":309372,"kb_used_data":12016,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94062468,"statfs":{"total":96636764160,"available":96319967232,"internally_reserved":0,"allocated":12304384,"data_stored":15987536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,4,5,6,7,9,10,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1240000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.248}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4330000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5349999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4570000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.857}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0620000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9990000000000001}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.915}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6399999999999999}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0209999999999999}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.948}]}]},{"osd":5,"up_from":22,"seq":94489280525,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306024,"kb_used_data":8660,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94065816,"statfs":{"total":96636764160,"available":96323395584,"internally_reserved":0,"allocated":8867840,"data_stored":11684699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,6,7,8,9,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.874}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.012}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68200000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93700000000000006}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90800000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.222}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2509999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98199999999999998}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6479999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":12.967000000000001}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1240000000000001}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.365}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1930000000000001}]}]},{"osd":6,"up_from":22,"seq":94489280525,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306332,"kb_used_data":8968,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94065508,"statfs":{"total":96636764160,"available":96323080192,"internally_reserved":0,"allocated":9183232,"data_stored":10675568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,7,8,9,11,12,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80100000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.343}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92100000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.091}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1200000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.29}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3460000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.458}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.329}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6719999999999999}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4810000000000001}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2050000000000001}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2669999999999999}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5049999999999999}]}]},{"osd":7,"up_from":22,"seq":94489280525,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308232,"kb_used_data":10868,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94063608,"statfs":{"total":96636764160,"available":96321134592,"internally_reserved":0,"allocated":11128832,"data_stored":14244699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,8,10,11,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83999999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":11.805}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.022}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.077}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.177}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.373}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.399}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.861}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":21.081}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.8639999999999999}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.885}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":105.13}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5900000000000001}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5509999999999999}]}]},{"osd":8,"up_from":22,"seq":94489280525,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308316,"kb_used_data":10960,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94063524,"statfs":{"total":96636764160,"available":96321048576,"internally_reserved":0,"allocated":11223040,"data_stored":15467344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[2,4,6,7,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93300000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.083}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66200000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90800000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83299999999999996}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95699999999999996}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80900000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0800000000000001}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.75}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.274}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0349999999999999}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.226}]}]},{"osd":10,"up_from":22,"seq":94489280526,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307636,"kb_used_data":10284,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94064204,"statfs":{"total":96636764160,"available":96321744896,"internally_reserved":0,"allocated":10530816,"data_stored":13433691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6,7,9,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":79.319000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":79.353999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":79.171999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83999999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96099999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":79.394999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.046}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.087}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.22}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":79.495000000000005}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.198}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":66.772000000000006}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":79.284000000000006}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5800000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4378624,"data_stored":5226496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":3551232,"data_stored":4034560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8728576,"data_stored":11669504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":11509760,"data_stored":15802368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":5373952,"data_stored":6971392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8089600,"data_stored":11509760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":7680000,"data_stored":9609216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":10350592,"data_stored":14069760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":10252288,"data_stored":14929920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":2125824,"data_stored":2449408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":9752576,"data_stored":13258752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":10899456,"data_stored":15249408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":3190784,"data_stored":4788224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":7667712,"data_stored":10338304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":7397376,"data_stored":9605120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":4558848,"data_stored":5578752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T10:15:26.467 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-09-18T10:15:26.467+0000 7fdfa8698700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22087) UID: 0 2024-09-18T10:15:26.532 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:26.532 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:26.542 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_blocked_ops 2024-09-18T10:15:26.569 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-18T10:15:26.562+0000 7f302f2c3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22187) UID: 0 2024-09-18T10:15:26.640 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:26.640 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:26.651 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_historic_ops 2024-09-18T10:15:26.670 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-18T10:15:26.666+0000 7f14dcc5b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22398) UID: 0 2024-09-18T10:15:26.750 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:26.751 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:26.761 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T10:15:26.770 INFO:tasks.ceph.osd.3.smithi189.stderr:2024-09-18T10:15:26.766+0000 7f2440fc7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22396) UID: 0 2024-09-18T10:15:26.871 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-09-18T10:15:26.871+0000 7fdfa8698700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22087) UID: 0 2024-09-18T10:15:26.875 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:26.875 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:26.885 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T10:15:26.960 INFO:tasks.rados.rados.0.smithi189.stdout:249: done (11 left) 2024-09-18T10:15:26.961 INFO:tasks.rados.rados.0.smithi189.stdout:250: read oid 6 snap -1 2024-09-18T10:15:26.961 INFO:tasks.rados.rados.0.smithi189.stdout:250: expect (ObjNum 75 snap 7 seq_num 75) 2024-09-18T10:15:26.961 INFO:tasks.rados.rados.0.smithi189.stdout:242: expect (ObjNum 4 snap 0 seq_num 4) 2024-09-18T10:15:26.968 INFO:tasks.rados.rados.0.smithi189.stdout:240: expect (ObjNum 33 snap 0 seq_num 33) 2024-09-18T10:15:26.973 INFO:tasks.ceph.osd.12.smithi029.stderr:2024-09-18T10:15:26.966+0000 7f8e4e627700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 22190) UID: 0 2024-09-18T10:15:26.987 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:26.987 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:26.987 INFO:tasks.rados.rados.0.smithi189.stdout:239: expect (ObjNum 48 snap 0 seq_num 48) 2024-09-18T10:15:26.995 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T10:15:27.009 INFO:tasks.rados.rados.0.smithi189.stdout:239: done (11 left) 2024-09-18T10:15:27.009 INFO:tasks.rados.rados.0.smithi189.stdout:240: done (10 left) 2024-09-18T10:15:27.009 INFO:tasks.rados.rados.0.smithi189.stdout:242: done (9 left) 2024-09-18T10:15:27.009 INFO:tasks.rados.rados.0.smithi189.stdout:251: read oid 9 snap -1 2024-09-18T10:15:27.009 INFO:tasks.rados.rados.0.smithi189.stdout:251: expect deleted 2024-09-18T10:15:27.010 INFO:tasks.rados.rados.0.smithi189.stdout:252: read oid 46 snap -1 2024-09-18T10:15:27.010 INFO:tasks.rados.rados.0.smithi189.stdout:252: expect (ObjNum 96 snap 17 seq_num 96) 2024-09-18T10:15:27.010 INFO:tasks.rados.rados.0.smithi189.stdout:253: snap_remove snap 20 2024-09-18T10:15:27.073 INFO:tasks.ceph.osd.15.smithi189.stderr:2024-09-18T10:15:27.070+0000 7fd0e725e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 22401) UID: 0 2024-09-18T10:15:27.094 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:27.095 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:27.104 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T10:15:27.174 INFO:tasks.ceph.osd.4.smithi029.stderr:2024-09-18T10:15:27.170+0000 7f550cf6e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22189) UID: 0 2024-09-18T10:15:27.199 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:27.199 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:27.209 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T10:15:27.275 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-18T10:15:27.270+0000 7f302f2c3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22187) UID: 0 2024-09-18T10:15:27.302 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:27.302 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:27.311 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T10:15:27.370 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T10:15:27.376 INFO:tasks.ceph.osd.3.smithi189.stderr:2024-09-18T10:15:27.370+0000 7f2440fc7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22396) UID: 0 2024-09-18T10:15:27.413 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:27.413 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:27.422 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-09-18T10:15:27.477 INFO:tasks.ceph.osd.14.smithi154.stderr:2024-09-18T10:15:27.475+0000 7f75a19e7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 22093) UID: 0 2024-09-18T10:15:27.521 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:27.521 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:27.521 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-18T10:15:27.530 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-09-18T10:15:27.542 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-18T10:15:27.578 INFO:tasks.ceph.osd.4.smithi029.stderr:2024-09-18T10:15:27.574+0000 7f550cf6e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22189) UID: 0 2024-09-18T10:15:27.615 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-18T10:15:27.625 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:27.626 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:27.635 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-09-18T10:15:27.639 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-18T10:15:27.661 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-18T10:15:27.679 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-18T10:15:27.674+0000 7f302f2c3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22187) UID: 0 2024-09-18T10:15:27.685 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-18T10:15:27.734 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:27.734 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:27.743 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_ops_in_flight 2024-09-18T10:15:27.780 INFO:tasks.ceph.osd.4.smithi029.stderr:2024-09-18T10:15:27.774+0000 7f550cf6e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22189) UID: 0 2024-09-18T10:15:27.807 INFO:teuthology.orchestra.run.smithi029.stdout:osd.6: {} 2024-09-18T10:15:27.830 INFO:teuthology.orchestra.run.smithi029.stdout:osd.7: {} 2024-09-18T10:15:27.840 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:27.840 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:27.849 INFO:teuthology.orchestra.run.smithi029.stdout:osd.8: {} 2024-09-18T10:15:27.850 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_blocked_ops 2024-09-18T10:15:27.868 INFO:teuthology.orchestra.run.smithi029.stdout:osd.9: {} 2024-09-18T10:15:27.881 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-18T10:15:27.876+0000 7f00046a8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 22136) UID: 0 2024-09-18T10:15:27.891 INFO:teuthology.orchestra.run.smithi029.stdout:osd.10: {} 2024-09-18T10:15:27.912 INFO:teuthology.orchestra.run.smithi029.stdout:osd.11: {} 2024-09-18T10:15:27.934 INFO:teuthology.orchestra.run.smithi029.stdout:osd.12: {} 2024-09-18T10:15:27.945 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:27.945 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:27.955 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_historic_ops 2024-09-18T10:15:27.957 INFO:teuthology.orchestra.run.smithi029.stdout:osd.13: {} 2024-09-18T10:15:27.977 INFO:tasks.rados.rados.0.smithi189.stdout:253: done (11 left) 2024-09-18T10:15:27.977 INFO:tasks.rados.rados.0.smithi189.stdout:254: copy_from oid 44 from oid 28 current snap is 21 2024-09-18T10:15:27.978 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 48 v 70 (ObjNum 41 snap 0 seq_num 41) dirty exists 2024-09-18T10:15:27.978 INFO:tasks.rados.rados.0.smithi189.stdout:238: finishing write tid 1 to smithi18925896-45 2024-09-18T10:15:27.978 INFO:tasks.rados.rados.0.smithi189.stdout:238: finishing write tid 2 to smithi18925896-45 2024-09-18T10:15:27.978 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 34 v 47 (ObjNum 33 snap 0 seq_num 33) dirty exists 2024-09-18T10:15:27.978 INFO:tasks.rados.rados.0.smithi189.stdout:241: expect (ObjNum 99 snap 17 seq_num 99) 2024-09-18T10:15:27.978 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-18T10:15:27.981 INFO:tasks.ceph.osd.14.smithi154.stderr:2024-09-18T10:15:27.979+0000 7f75a19e7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 22093) UID: 0 2024-09-18T10:15:27.984 INFO:teuthology.orchestra.run.smithi029.stdout:osd.14: {} 2024-09-18T10:15:28.020 INFO:teuthology.orchestra.run.smithi029.stdout:osd.15: {} 2024-09-18T10:15:28.023 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T10:15:28.023 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T10:15:28.024 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T10:15:28.024 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T10:15:28.024 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T10:15:28.024 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T10:15:28.024 INFO:teuthology.orchestra.run.smithi029.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T10:15:28.024 INFO:teuthology.orchestra.run.smithi029.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T10:15:28.024 INFO:teuthology.orchestra.run.smithi029.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-18T10:15:28.024 INFO:teuthology.orchestra.run.smithi029.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-18T10:15:28.024 INFO:teuthology.orchestra.run.smithi029.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-18T10:15:28.024 INFO:teuthology.orchestra.run.smithi029.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-18T10:15:28.024 INFO:teuthology.orchestra.run.smithi029.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-18T10:15:28.024 INFO:teuthology.orchestra.run.smithi029.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-18T10:15:28.025 INFO:teuthology.orchestra.run.smithi029.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-18T10:15:28.025 INFO:teuthology.orchestra.run.smithi029.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-18T10:15:28.082 INFO:tasks.rados.rados.0.smithi189.stdout:237: expect (ObjNum 41 snap 0 seq_num 41) 2024-09-18T10:15:28.083 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-18T10:15:28.080+0000 7f1981873700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22132) UID: 0 2024-09-18T10:15:28.104 INFO:tasks.rados.rados.0.smithi189.stdout:238: finishing write tid 3 to smithi18925896-45 2024-09-18T10:15:28.104 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 24 v 42 (ObjNum 33 snap 0 seq_num 3858653808) dirty exists 2024-09-18T10:15:28.104 INFO:tasks.rados.rados.0.smithi189.stdout:246: finishing write tid 1 to smithi18925896-30 2024-09-18T10:15:28.104 INFO:tasks.rados.rados.0.smithi189.stdout:246: finishing write tid 2 to smithi18925896-30 2024-09-18T10:15:28.104 INFO:tasks.rados.rados.0.smithi189.stdout:246: finishing write tid 3 to smithi18925896-30 2024-09-18T10:15:28.104 INFO:tasks.rados.rados.0.smithi189.stdout:246: finishing write tid 4 to smithi18925896-30 2024-09-18T10:15:28.104 INFO:tasks.rados.rados.0.smithi189.stdout:238: finishing write tid 4 to smithi18925896-45 2024-09-18T10:15:28.104 INFO:tasks.rados.rados.0.smithi189.stdout:246: finishing write tid 5 to smithi18925896-30 2024-09-18T10:15:28.104 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 30 v 22 (ObjNum 105 snap 21 seq_num 105) dirty exists 2024-09-18T10:15:28.104 INFO:tasks.rados.rados.0.smithi189.stdout:246: left oid 30 (ObjNum 105 snap 21 seq_num 105) 2024-09-18T10:15:28.104 INFO:tasks.rados.rados.0.smithi189.stdout:244: finishing rollback tid 0 to smithi18925896-14 2024-09-18T10:15:28.105 INFO:tasks.rados.rados.0.smithi189.stdout:244: finishing rollback tid 1 to smithi18925896-14 2024-09-18T10:15:28.105 INFO:tasks.rados.rados.0.smithi189.stdout:237: done (11 left) 2024-09-18T10:15:28.105 INFO:tasks.rados.rados.0.smithi189.stdout:241: done (10 left) 2024-09-18T10:15:28.105 INFO:tasks.rados.rados.0.smithi189.stdout:243: done (9 left) 2024-09-18T10:15:28.105 INFO:tasks.rados.rados.0.smithi189.stdout:246: done (8 left) 2024-09-18T10:15:28.105 INFO:tasks.rados.rados.0.smithi189.stdout:247: done (7 left) 2024-09-18T10:15:28.105 INFO:tasks.rados.rados.0.smithi189.stdout:248: done (6 left) 2024-09-18T10:15:28.105 INFO:tasks.rados.rados.0.smithi189.stdout:255: read oid 32 snap -1 2024-09-18T10:15:28.105 INFO:tasks.rados.rados.0.smithi189.stdout:255: expect (ObjNum 36 snap 0 seq_num 36) 2024-09-18T10:15:28.105 INFO:tasks.rados.rados.0.smithi189.stdout:256: delete oid 40 current snap is 21 2024-09-18T10:15:28.105 INFO:tasks.rados.rados.0.smithi189.stdout:238: finishing write tid 5 to smithi18925896-45 2024-09-18T10:15:28.105 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 45 v 45 (ObjNum 103 snap 21 seq_num 103) dirty exists 2024-09-18T10:15:28.105 INFO:tasks.rados.rados.0.smithi189.stdout:238: left oid 45 (ObjNum 103 snap 21 seq_num 103) 2024-09-18T10:15:28.105 INFO:tasks.rados.rados.0.smithi189.stdout:244: finishing rollback tid 2 to smithi18925896-14 2024-09-18T10:15:28.106 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 14 v 17 (ObjNum 13 snap 0 seq_num 13) dirty exists 2024-09-18T10:15:28.106 INFO:tasks.rados.rados.0.smithi189.stdout:252: expect (ObjNum 96 snap 17 seq_num 96) 2024-09-18T10:15:28.113 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-18T10:15:28.113 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-18T10:15:28.113 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-18T10:15:28.113 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-18T10:15:28.113 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.113 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:176 3.ds0 3:b1a14543:::smithi18925896-15:head [copy-from ver 4] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:28.113 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:52.680518+0000", 2024-09-18T10:15:28.113 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 35.429083736000003, 2024-09-18T10:15:28.113 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.42607371300000002, 2024-09-18T10:15:28.113 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:28.113 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:28.113 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:28.114 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:28.114 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:28.114 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 176 2024-09-18T10:15:28.114 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.114 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:28.114 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.114 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:28.114 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.680518+0000", 2024-09-18T10:15:28.114 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:28.114 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.114 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.114 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:28.115 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.680518+0000", 2024-09-18T10:15:28.115 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:28.115 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.115 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.115 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:28.115 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.680515+0000", 2024-09-18T10:15:28.115 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999971 2024-09-18T10:15:28.115 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.115 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.115 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:28.115 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.680533+0000", 2024-09-18T10:15:28.115 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8756000000000001e-05 2024-09-18T10:15:28.115 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.116 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.116 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:28.116 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.680573+0000", 2024-09-18T10:15:28.116 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9431999999999998e-05 2024-09-18T10:15:28.116 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.116 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.116 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:28.116 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.680587+0000", 2024-09-18T10:15:28.116 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.447e-05 2024-09-18T10:15:28.116 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.116 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.116 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:28.117 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.680755+0000", 2024-09-18T10:15:28.117 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000167584 2024-09-18T10:15:28.117 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.117 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.117 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-09-18T10:15:28.117 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.680784+0000", 2024-09-18T10:15:28.117 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9439000000000001e-05 2024-09-18T10:15:28.117 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.117 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.118 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:28.118 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.685398+0000", 2024-09-18T10:15:28.118 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0046140060000000004 2024-09-18T10:15:28.118 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.118 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.118 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:28.118 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.685502+0000", 2024-09-18T10:15:28.118 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000103702 2024-09-18T10:15:28.118 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.118 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.118 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:28.118 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.003083+0000", 2024-09-18T10:15:28.119 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.31758088600000001 2024-09-18T10:15:28.119 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.119 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.119 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:28.119 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.006087+0000", 2024-09-18T10:15:28.119 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0030044820000000002 2024-09-18T10:15:28.119 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.119 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.119 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:15:28.119 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.106397+0000", 2024-09-18T10:15:28.119 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100309469 2024-09-18T10:15:28.119 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.120 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.120 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:28.120 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.106591+0000", 2024-09-18T10:15:28.120 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000194513 2024-09-18T10:15:28.120 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:28.120 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:28.120 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:28.120 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.120 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.120 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:177 3.ds0 3:b1a14543:::smithi18925896-15:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:28.120 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:52.680610+0000", 2024-09-18T10:15:28.120 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 35.428991015000001, 2024-09-18T10:15:28.120 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.32474139800000001, 2024-09-18T10:15:28.121 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:28.121 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-18T10:15:28.121 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:28.121 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:28.121 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:28.121 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 177 2024-09-18T10:15:28.121 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.121 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:28.121 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.121 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:28.121 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.680610+0000", 2024-09-18T10:15:28.121 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:28.121 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.122 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.122 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:28.122 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.680610+0000", 2024-09-18T10:15:28.122 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:28.122 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.122 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.122 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:28.122 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.680609+0000", 2024-09-18T10:15:28.122 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:15:28.122 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.123 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.123 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:28.123 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.680614+0000", 2024-09-18T10:15:28.123 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6840000000000004e-06 2024-09-18T10:15:28.123 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.123 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.123 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:28.123 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.680625+0000", 2024-09-18T10:15:28.123 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0739e-05 2024-09-18T10:15:28.123 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.123 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.123 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:28.123 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.680655+0000", 2024-09-18T10:15:28.124 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0022000000000002e-05 2024-09-18T10:15:28.124 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.124 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.124 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:28.125 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.680965+0000", 2024-09-18T10:15:28.125 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00031078300000000002 2024-09-18T10:15:28.125 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.125 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.125 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:28.125 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.680976+0000", 2024-09-18T10:15:28.125 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0589e-05 2024-09-18T10:15:28.125 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.125 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.126 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:28.126 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.785862+0000", 2024-09-18T10:15:28.126 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10488597500000001 2024-09-18T10:15:28.126 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.126 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.126 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2024-09-18T10:15:28.126 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.785963+0000", 2024-09-18T10:15:28.126 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000100523 2024-09-18T10:15:28.126 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.126 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.126 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:28.126 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.005029+0000", 2024-09-18T10:15:28.126 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.219066329 2024-09-18T10:15:28.127 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.127 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.127 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:28.127 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.005193+0000", 2024-09-18T10:15:28.127 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00016436200000000001 2024-09-18T10:15:28.127 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.127 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.127 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:28.127 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.005352+0000", 2024-09-18T10:15:28.127 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000158486 2024-09-18T10:15:28.127 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:28.127 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:28.127 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:28.127 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.128 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.128 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:200 3.as0 3:5dc9cbed:::smithi18925896-13:2 [read 0~212992,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:28.128 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:52.694751+0000", 2024-09-18T10:15:28.128 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 35.414850585000003, 2024-09-18T10:15:28.129 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.40590051599999999, 2024-09-18T10:15:28.129 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:28.129 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-18T10:15:28.129 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:28.129 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:28.129 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:28.129 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 200 2024-09-18T10:15:28.129 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.129 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:28.129 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.129 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:28.129 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694751+0000", 2024-09-18T10:15:28.129 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:28.130 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.130 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.130 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:28.130 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694751+0000", 2024-09-18T10:15:28.130 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:28.130 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.130 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.130 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:28.130 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694750+0000", 2024-09-18T10:15:28.130 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:28.130 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.130 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.130 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:28.131 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694756+0000", 2024-09-18T10:15:28.131 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8119999999999996e-06 2024-09-18T10:15:28.131 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.131 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.131 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:28.131 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694762+0000", 2024-09-18T10:15:28.131 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8050000000000003e-06 2024-09-18T10:15:28.131 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.131 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.131 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:28.131 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694768+0000", 2024-09-18T10:15:28.131 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5549999999999999e-06 2024-09-18T10:15:28.131 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.131 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.132 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:28.132 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.695694+0000", 2024-09-18T10:15:28.132 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00092562999999999999 2024-09-18T10:15:28.132 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.132 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.132 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:28.132 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.695903+0000", 2024-09-18T10:15:28.132 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00020888699999999999 2024-09-18T10:15:28.132 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.133 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.133 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:28.133 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.100651+0000", 2024-09-18T10:15:28.133 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.40474845599999998 2024-09-18T10:15:28.133 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:28.133 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:28.133 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:28.133 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.133 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.133 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:201 3.as0 3:5dc9cbed:::smithi18925896-13:2 [read 0~212992,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:28.133 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:52.694813+0000", 2024-09-18T10:15:28.133 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 35.414788780999999, 2024-09-18T10:15:28.133 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.50652904899999995, 2024-09-18T10:15:28.134 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:28.134 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-18T10:15:28.134 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:28.134 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:28.134 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:28.134 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 201 2024-09-18T10:15:28.134 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.134 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:28.134 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.134 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:28.134 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694813+0000", 2024-09-18T10:15:28.134 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:28.134 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.134 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.135 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:28.135 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694813+0000", 2024-09-18T10:15:28.135 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:28.135 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.135 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.135 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:28.135 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694812+0000", 2024-09-18T10:15:28.135 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:28.135 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.135 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.135 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:28.136 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694816+0000", 2024-09-18T10:15:28.136 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7720000000000001e-06 2024-09-18T10:15:28.136 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.136 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.136 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:28.136 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694822+0000", 2024-09-18T10:15:28.136 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5529999999999998e-06 2024-09-18T10:15:28.136 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.136 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.136 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:28.136 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694827+0000", 2024-09-18T10:15:28.136 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1500000000000001e-06 2024-09-18T10:15:28.136 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.137 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.137 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:28.137 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.797100+0000", 2024-09-18T10:15:28.137 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.102273424 2024-09-18T10:15:28.137 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.137 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.137 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:28.137 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.797329+0000", 2024-09-18T10:15:28.137 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00022870200000000001 2024-09-18T10:15:28.137 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.137 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.137 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:28.137 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.201342+0000", 2024-09-18T10:15:28.137 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.40401293799999999 2024-09-18T10:15:28.138 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:28.138 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:28.138 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:28.138 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.138 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.138 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:207 3.as0 3:5c566071:::smithi18925896-28:head [copy-from ver 6] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:28.138 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:52.694855+0000", 2024-09-18T10:15:28.138 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 35.414745979000003, 2024-09-18T10:15:28.138 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.81394701899999999, 2024-09-18T10:15:28.138 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:28.138 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:28.138 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:28.138 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:28.138 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:28.139 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 207 2024-09-18T10:15:28.139 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.139 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:28.139 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.139 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:28.139 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694855+0000", 2024-09-18T10:15:28.139 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:28.139 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.139 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.139 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:28.139 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694855+0000", 2024-09-18T10:15:28.139 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:28.139 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.139 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.140 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:28.140 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694852+0000", 2024-09-18T10:15:28.140 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999967 2024-09-18T10:15:28.140 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.140 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.140 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:28.140 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694859+0000", 2024-09-18T10:15:28.140 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.08e-06 2024-09-18T10:15:28.140 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.140 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.140 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:28.141 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694866+0000", 2024-09-18T10:15:28.141 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9249999999999998e-06 2024-09-18T10:15:28.141 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.141 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.141 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:28.141 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694870+0000", 2024-09-18T10:15:28.141 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0330000000000002e-06 2024-09-18T10:15:28.141 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.141 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.141 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:28.141 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.900743+0000", 2024-09-18T10:15:28.141 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20587325400000001 2024-09-18T10:15:28.141 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.142 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.142 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:28.142 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.901026+0000", 2024-09-18T10:15:28.142 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00028313600000000002 2024-09-18T10:15:28.142 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.142 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.142 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:28.142 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.204074+0000", 2024-09-18T10:15:28.142 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30304722299999998 2024-09-18T10:15:28.142 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.142 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.143 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:28.143 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.207280+0000", 2024-09-18T10:15:28.143 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0032067049999999998 2024-09-18T10:15:28.143 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.143 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.143 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:15:28.143 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.508672+0000", 2024-09-18T10:15:28.143 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30139174800000001 2024-09-18T10:15:28.143 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.143 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.143 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:28.143 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.508802+0000", 2024-09-18T10:15:28.143 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013037500000000001 2024-09-18T10:15:28.144 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:28.144 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:28.144 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:28.144 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.144 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.144 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:208 3.as0 3:5c566071:::smithi18925896-28:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:28.144 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:52.694875+0000", 2024-09-18T10:15:28.144 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 35.414726817999998, 2024-09-18T10:15:28.144 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.51199464299999997, 2024-09-18T10:15:28.144 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:28.144 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-18T10:15:28.144 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:28.144 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:28.145 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:28.145 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 208 2024-09-18T10:15:28.145 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.145 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:28.145 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.145 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:28.145 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694875+0000", 2024-09-18T10:15:28.145 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:28.145 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.145 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.145 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:28.145 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694875+0000", 2024-09-18T10:15:28.145 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:28.145 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.146 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.146 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:28.146 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694874+0000", 2024-09-18T10:15:28.146 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:28.146 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.146 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.146 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:28.146 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694876+0000", 2024-09-18T10:15:28.146 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2369999999999999e-06 2024-09-18T10:15:28.146 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.147 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.147 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:28.147 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694881+0000", 2024-09-18T10:15:28.147 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0950000000000003e-06 2024-09-18T10:15:28.147 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.147 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.147 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:28.147 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694887+0000", 2024-09-18T10:15:28.147 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5690000000000004e-06 2024-09-18T10:15:28.147 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.147 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.147 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:28.147 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.901475+0000", 2024-09-18T10:15:28.147 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20658803000000001 2024-09-18T10:15:28.148 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.148 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.148 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2024-09-18T10:15:28.148 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.901550+0000", 2024-09-18T10:15:28.148 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5020999999999999e-05 2024-09-18T10:15:28.148 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.148 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.148 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:28.148 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.206415+0000", 2024-09-18T10:15:28.148 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30486486000000002 2024-09-18T10:15:28.148 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.148 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.148 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:28.149 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.206634+0000", 2024-09-18T10:15:28.149 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00021925499999999999 2024-09-18T10:15:28.149 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.149 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.149 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:28.149 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.206869+0000", 2024-09-18T10:15:28.149 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00023503599999999999 2024-09-18T10:15:28.149 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:28.149 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:28.149 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:28.149 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.149 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.149 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:332 3.ds0 3:b1a14543:::smithi18925896-15:head [append 0~737280 in=737280b] snapc b=[b,4] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-18T10:15:28.149 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:02.789344+0000", 2024-09-18T10:15:28.150 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 25.320257547000001, 2024-09-18T10:15:28.150 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.31055802500000002, 2024-09-18T10:15:28.150 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:28.150 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:28.150 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:28.150 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:28.150 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:28.150 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 332 2024-09-18T10:15:28.150 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.150 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:28.150 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.151 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:28.151 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.789344+0000", 2024-09-18T10:15:28.151 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:28.151 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.151 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.151 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:28.151 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.789344+0000", 2024-09-18T10:15:28.151 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:28.152 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.152 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.152 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:28.152 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.789342+0000", 2024-09-18T10:15:28.152 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999976 2024-09-18T10:15:28.152 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.152 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.152 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:28.152 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.790370+0000", 2024-09-18T10:15:28.152 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010286150000000001 2024-09-18T10:15:28.152 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.152 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.152 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:28.153 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.790403+0000", 2024-09-18T10:15:28.153 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2549000000000003e-05 2024-09-18T10:15:28.153 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.153 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.153 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:28.153 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.790414+0000", 2024-09-18T10:15:28.153 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1279e-05 2024-09-18T10:15:28.153 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.153 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.153 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:28.153 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.790526+0000", 2024-09-18T10:15:28.153 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000111994 2024-09-18T10:15:28.153 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.154 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.154 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-09-18T10:15:28.154 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.790540+0000", 2024-09-18T10:15:28.154 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4296e-05 2024-09-18T10:15:28.154 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.154 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.154 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:28.154 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.794331+0000", 2024-09-18T10:15:28.154 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0037907029999999999 2024-09-18T10:15:28.154 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.154 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.154 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:28.154 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.794483+0000", 2024-09-18T10:15:28.155 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000151772 2024-09-18T10:15:28.155 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.155 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.155 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:28.155 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.796730+0000", 2024-09-18T10:15:28.155 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0022475009999999998 2024-09-18T10:15:28.155 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.155 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.155 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:28.156 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.804001+0000", 2024-09-18T10:15:28.156 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0072711420000000004 2024-09-18T10:15:28.156 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.156 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.156 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:15:28.156 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:03.099777+0000", 2024-09-18T10:15:28.156 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.295775552 2024-09-18T10:15:28.156 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.156 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.156 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:28.156 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:03.099902+0000", 2024-09-18T10:15:28.157 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012481999999999999 2024-09-18T10:15:28.157 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:28.157 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:28.157 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:28.157 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.157 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.157 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:333 3.ds0 3:b1a14543:::smithi18925896-15:head [append 0~376832 in=376832b] snapc b=[b,4] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-18T10:15:28.157 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:02.792703+0000", 2024-09-18T10:15:28.157 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 25.316898152, 2024-09-18T10:15:28.157 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.40779418200000001, 2024-09-18T10:15:28.157 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:28.157 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:28.157 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:28.158 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:28.158 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:28.158 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 333 2024-09-18T10:15:28.158 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.158 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:28.158 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.158 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:28.158 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.792703+0000", 2024-09-18T10:15:28.158 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:28.158 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.158 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.158 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:28.158 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.792703+0000", 2024-09-18T10:15:28.158 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:28.159 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.159 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.159 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:28.159 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.792702+0000", 2024-09-18T10:15:28.159 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:28.159 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.159 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.159 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:28.159 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.793113+0000", 2024-09-18T10:15:28.159 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00041176199999999998 2024-09-18T10:15:28.159 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.159 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.159 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:28.159 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.793131+0000", 2024-09-18T10:15:28.160 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7802e-05 2024-09-18T10:15:28.160 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.160 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.160 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:28.160 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.793140+0000", 2024-09-18T10:15:28.160 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0790000000000007e-06 2024-09-18T10:15:28.160 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.160 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.160 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:28.160 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.793223+0000", 2024-09-18T10:15:28.160 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2414000000000001e-05 2024-09-18T10:15:28.160 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.160 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.160 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:28.161 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.793233+0000", 2024-09-18T10:15:28.161 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.076e-05 2024-09-18T10:15:28.161 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.161 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.161 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:28.161 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.799019+0000", 2024-09-18T10:15:28.161 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0057857109999999998 2024-09-18T10:15:28.161 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.161 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.161 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:28.161 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.799289+0000", 2024-09-18T10:15:28.161 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00026981800000000001 2024-09-18T10:15:28.161 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.162 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.162 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:28.162 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.801174+0000", 2024-09-18T10:15:28.162 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00188511 2024-09-18T10:15:28.162 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.162 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.162 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:28.162 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.804009+0000", 2024-09-18T10:15:28.162 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002835181 2024-09-18T10:15:28.162 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.162 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.162 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:15:28.162 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:03.200390+0000", 2024-09-18T10:15:28.162 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.39638093699999999 2024-09-18T10:15:28.163 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.163 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.163 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:28.163 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:03.200497+0000", 2024-09-18T10:15:28.163 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000107243 2024-09-18T10:15:28.163 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:28.163 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:28.163 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:28.163 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.163 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.163 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:334 3.ds0 3:b1a14543:::smithi18925896-15:head [setxattr _header (56) in=63b] snapc b=[b,4] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-18T10:15:28.163 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:02.793164+0000", 2024-09-18T10:15:28.163 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 25.316437007000001, 2024-09-18T10:15:28.164 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.408134104, 2024-09-18T10:15:28.164 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:28.164 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:28.164 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:28.164 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:28.164 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:28.164 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 334 2024-09-18T10:15:28.164 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.164 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:28.164 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.164 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:28.164 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.793164+0000", 2024-09-18T10:15:28.164 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:28.165 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.165 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.165 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:28.165 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.793164+0000", 2024-09-18T10:15:28.165 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:28.165 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.165 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.165 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:28.165 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.793163+0000", 2024-09-18T10:15:28.165 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:28.165 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.165 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.165 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:28.166 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.793170+0000", 2024-09-18T10:15:28.166 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3050000000000004e-06 2024-09-18T10:15:28.166 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.166 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.166 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:28.166 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.793180+0000", 2024-09-18T10:15:28.166 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8069999999999994e-06 2024-09-18T10:15:28.166 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.166 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.166 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:28.166 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.793187+0000", 2024-09-18T10:15:28.166 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6879999999999999e-06 2024-09-18T10:15:28.166 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.166 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.167 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:28.167 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.793293+0000", 2024-09-18T10:15:28.167 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000105441 2024-09-18T10:15:28.167 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.167 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.167 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:28.167 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.793302+0000", 2024-09-18T10:15:28.167 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.5769999999999994e-06 2024-09-18T10:15:28.167 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.167 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.167 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:28.167 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.802703+0000", 2024-09-18T10:15:28.167 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0094011400000000005 2024-09-18T10:15:28.168 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.168 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.168 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:28.168 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.802949+0000", 2024-09-18T10:15:28.168 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00024604500000000002 2024-09-18T10:15:28.168 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.168 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.168 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:28.168 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.803628+0000", 2024-09-18T10:15:28.168 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000679391 2024-09-18T10:15:28.168 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.168 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.168 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:28.169 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.804224+0000", 2024-09-18T10:15:28.169 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00059597000000000003 2024-09-18T10:15:28.169 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.169 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.169 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:15:28.169 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:03.201019+0000", 2024-09-18T10:15:28.169 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.39679527799999997 2024-09-18T10:15:28.169 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.169 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.169 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:28.169 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:03.201298+0000", 2024-09-18T10:15:28.169 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00027909199999999999 2024-09-18T10:15:28.169 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:28.169 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:28.170 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:28.170 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.170 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.170 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:335 3.ds0 3:b1a14543:::smithi18925896-15:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e47)", 2024-09-18T10:15:28.170 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:02.793201+0000", 2024-09-18T10:15:28.170 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 25.316400546000001, 2024-09-18T10:15:28.170 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.51366298600000004, 2024-09-18T10:15:28.170 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:28.170 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-18T10:15:28.170 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:28.170 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:28.170 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:28.170 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 335 2024-09-18T10:15:28.170 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.171 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:28.171 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.171 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:28.171 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.793201+0000", 2024-09-18T10:15:28.171 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:28.171 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.171 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.171 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:28.171 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.793201+0000", 2024-09-18T10:15:28.171 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:28.171 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.171 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.171 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:28.171 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.793200+0000", 2024-09-18T10:15:28.172 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:28.172 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.172 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.172 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:28.172 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.793206+0000", 2024-09-18T10:15:28.172 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2199999999999997e-06 2024-09-18T10:15:28.172 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.172 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.172 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:28.172 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.793221+0000", 2024-09-18T10:15:28.172 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5092000000000001e-05 2024-09-18T10:15:28.172 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.173 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.173 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:28.173 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.793229+0000", 2024-09-18T10:15:28.173 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1820000000000006e-06 2024-09-18T10:15:28.173 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.173 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.173 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:28.173 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.793367+0000", 2024-09-18T10:15:28.173 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000138057 2024-09-18T10:15:28.173 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.173 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.173 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:28.173 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.793384+0000", 2024-09-18T10:15:28.173 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6731e-05 2024-09-18T10:15:28.174 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.174 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.174 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:28.174 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.804064+0000", 2024-09-18T10:15:28.174 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010680185 2024-09-18T10:15:28.174 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.174 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.174 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2024-09-18T10:15:28.174 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.804199+0000", 2024-09-18T10:15:28.174 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013532999999999999 2024-09-18T10:15:28.174 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.174 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.174 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:28.174 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:03.202193+0000", 2024-09-18T10:15:28.175 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.39799325099999999 2024-09-18T10:15:28.175 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.175 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.175 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:28.175 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:03.202292+0000", 2024-09-18T10:15:28.175 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9628000000000002e-05 2024-09-18T10:15:28.175 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.175 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.175 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:28.175 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:03.306864+0000", 2024-09-18T10:15:28.175 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.104571572 2024-09-18T10:15:28.175 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:28.175 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:28.176 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:28.176 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.176 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.176 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:503 3.fs0 3:feb35297:::smithi18925896-37:head [append 0~753664] snapc 19=[19,18,16,12,11,10,f,e] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T10:15:28.176 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:17.908895+0000", 2024-09-18T10:15:28.176 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.200706565000001, 2024-09-18T10:15:28.176 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.51525800200000005, 2024-09-18T10:15:28.176 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:28.176 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:28.176 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:28.176 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:28.176 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:28.176 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 503 2024-09-18T10:15:28.177 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.177 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:28.177 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.177 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:28.177 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.908895+0000", 2024-09-18T10:15:28.177 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:28.177 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.177 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.177 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:28.177 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.908895+0000", 2024-09-18T10:15:28.178 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:28.178 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.178 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.178 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:28.178 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.908892+0000", 2024-09-18T10:15:28.178 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999976 2024-09-18T10:15:28.178 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.178 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.178 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:28.178 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.910143+0000", 2024-09-18T10:15:28.178 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0012507040000000001 2024-09-18T10:15:28.178 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.178 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.179 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:28.179 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.910167+0000", 2024-09-18T10:15:28.179 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4176999999999999e-05 2024-09-18T10:15:28.179 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.179 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.179 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:28.179 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.910178+0000", 2024-09-18T10:15:28.179 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0669999999999999e-05 2024-09-18T10:15:28.179 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.179 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.179 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:28.179 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.910329+0000", 2024-09-18T10:15:28.179 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000150746 2024-09-18T10:15:28.179 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.180 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.180 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-09-18T10:15:28.180 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.910346+0000", 2024-09-18T10:15:28.180 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7680999999999999e-05 2024-09-18T10:15:28.180 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.180 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.180 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:28.180 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.015009+0000", 2024-09-18T10:15:28.180 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.104662914 2024-09-18T10:15:28.180 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.180 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.180 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:28.181 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.015304+0000", 2024-09-18T10:15:28.181 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00029439899999999998 2024-09-18T10:15:28.181 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.181 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.181 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:28.181 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.017669+0000", 2024-09-18T10:15:28.181 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0023647910000000002 2024-09-18T10:15:28.181 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.181 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.181 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:28.181 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.126654+0000", 2024-09-18T10:15:28.181 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.108985055 2024-09-18T10:15:28.181 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.181 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.182 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:15:28.182 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.423961+0000", 2024-09-18T10:15:28.182 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.29730786399999998 2024-09-18T10:15:28.182 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.182 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.182 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:28.182 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.424153+0000", 2024-09-18T10:15:28.182 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019128099999999999 2024-09-18T10:15:28.182 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:28.182 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:28.182 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:28.182 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.182 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.183 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:504 3.fs0 3:feb35297:::smithi18925896-37:head [append 0~720896] snapc 19=[19,18,16,12,11,10,f,e] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T10:15:28.183 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:17.912996+0000", 2024-09-18T10:15:28.183 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.196605529999999, 2024-09-18T10:15:28.183 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.61195014999999997, 2024-09-18T10:15:28.183 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:28.183 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:28.183 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:28.183 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:28.183 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:28.183 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 504 2024-09-18T10:15:28.183 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.183 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:28.183 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.183 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:28.184 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.912996+0000", 2024-09-18T10:15:28.184 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:28.184 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.184 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.184 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:28.184 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.912996+0000", 2024-09-18T10:15:28.184 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:28.184 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.184 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.184 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:28.184 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.912994+0000", 2024-09-18T10:15:28.184 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:28.184 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.185 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.185 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:28.185 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.913907+0000", 2024-09-18T10:15:28.185 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00091254300000000003 2024-09-18T10:15:28.185 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.185 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.185 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:28.185 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.913950+0000", 2024-09-18T10:15:28.185 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3019000000000003e-05 2024-09-18T10:15:28.185 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.185 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.185 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:28.185 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.913960+0000", 2024-09-18T10:15:28.185 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0305e-05 2024-09-18T10:15:28.186 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.186 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.186 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:28.186 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.914067+0000", 2024-09-18T10:15:28.186 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000106696 2024-09-18T10:15:28.186 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.186 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.186 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:28.186 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.914079+0000", 2024-09-18T10:15:28.186 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2330000000000001e-05 2024-09-18T10:15:28.186 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.186 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.186 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:28.186 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.119322+0000", 2024-09-18T10:15:28.187 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20524277899999999 2024-09-18T10:15:28.187 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.187 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.187 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:28.187 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.119508+0000", 2024-09-18T10:15:28.187 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00018648800000000001 2024-09-18T10:15:28.187 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.187 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.187 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:28.187 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.121315+0000", 2024-09-18T10:15:28.187 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001806483 2024-09-18T10:15:28.187 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.187 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.188 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:28.188 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.127304+0000", 2024-09-18T10:15:28.188 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0059895599999999997 2024-09-18T10:15:28.188 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.188 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.188 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:15:28.188 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.524828+0000", 2024-09-18T10:15:28.188 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.39752343600000001 2024-09-18T10:15:28.188 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.188 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.188 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:28.188 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.524946+0000", 2024-09-18T10:15:28.188 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000118193 2024-09-18T10:15:28.188 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:28.189 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:28.189 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:28.189 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.189 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.189 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:505 3.fs0 3:feb35297:::smithi18925896-37:head [append 0~49152] snapc 19=[19,18,16,12,11,10,f,e] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T10:15:28.189 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:17.913979+0000", 2024-09-18T10:15:28.189 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.195622051000001, 2024-09-18T10:15:28.189 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.71150136900000005, 2024-09-18T10:15:28.189 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:28.189 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:28.189 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:28.189 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:28.189 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:28.190 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 505 2024-09-18T10:15:28.190 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.190 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:28.190 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.190 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:28.190 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.913979+0000", 2024-09-18T10:15:28.190 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:28.190 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.190 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.190 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:28.190 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.913979+0000", 2024-09-18T10:15:28.190 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:28.190 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.191 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.191 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:28.191 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.913978+0000", 2024-09-18T10:15:28.191 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:28.191 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.191 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.191 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:28.191 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.914061+0000", 2024-09-18T10:15:28.191 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2533000000000004e-05 2024-09-18T10:15:28.191 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.191 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.191 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:28.191 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.914077+0000", 2024-09-18T10:15:28.191 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6277e-05 2024-09-18T10:15:28.192 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.192 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.192 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:28.192 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.914087+0000", 2024-09-18T10:15:28.192 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0282000000000001e-05 2024-09-18T10:15:28.192 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.192 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.192 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:28.192 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.914144+0000", 2024-09-18T10:15:28.192 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6784999999999999e-05 2024-09-18T10:15:28.192 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.192 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.192 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:28.193 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.914151+0000", 2024-09-18T10:15:28.193 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.968e-06 2024-09-18T10:15:28.193 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.193 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.193 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:28.193 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.122495+0000", 2024-09-18T10:15:28.193 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.208343742 2024-09-18T10:15:28.193 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.193 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.193 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:28.193 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.122788+0000", 2024-09-18T10:15:28.193 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00029334199999999998 2024-09-18T10:15:28.196 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.196 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.196 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:28.196 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.124367+0000", 2024-09-18T10:15:28.197 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0015794699999999999 2024-09-18T10:15:28.197 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.197 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.197 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:28.197 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.127307+0000", 2024-09-18T10:15:28.197 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0029394780000000001 2024-09-18T10:15:28.197 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.197 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.197 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:15:28.197 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.625281+0000", 2024-09-18T10:15:28.197 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.497974166 2024-09-18T10:15:28.197 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.197 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.198 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:28.198 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.625481+0000", 2024-09-18T10:15:28.198 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019956100000000001 2024-09-18T10:15:28.198 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:28.198 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:28.198 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:28.198 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.198 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.198 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:506 3.fs0 3:feb35297:::smithi18925896-37:head [setxattr _header (57) in=7b] snapc 19=[19,18,16,12,11,10,f,e] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T10:15:28.198 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:17.914106+0000", 2024-09-18T10:15:28.198 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.195495784, 2024-09-18T10:15:28.198 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.71184716599999998, 2024-09-18T10:15:28.198 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:28.198 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:28.199 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:28.199 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:28.199 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:28.199 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 506 2024-09-18T10:15:28.199 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.199 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:28.199 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.199 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:28.199 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.914106+0000", 2024-09-18T10:15:28.199 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:28.199 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.199 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.199 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:28.199 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.914106+0000", 2024-09-18T10:15:28.200 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:28.200 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.200 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.200 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:28.200 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.914105+0000", 2024-09-18T10:15:28.200 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:15:28.200 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.200 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.200 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:28.200 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.914110+0000", 2024-09-18T10:15:28.200 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.045e-06 2024-09-18T10:15:28.200 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.200 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.201 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:28.201 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.914117+0000", 2024-09-18T10:15:28.201 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6510000000000001e-06 2024-09-18T10:15:28.201 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.201 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.201 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:28.201 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.914122+0000", 2024-09-18T10:15:28.201 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5839999999999996e-06 2024-09-18T10:15:28.201 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.201 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.201 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:28.201 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.014305+0000", 2024-09-18T10:15:28.201 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100183091 2024-09-18T10:15:28.201 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.202 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.202 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:28.202 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.014325+0000", 2024-09-18T10:15:28.202 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0032000000000001e-05 2024-09-18T10:15:28.202 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.202 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.202 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:28.202 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.125511+0000", 2024-09-18T10:15:28.202 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.111185942 2024-09-18T10:15:28.202 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.202 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.202 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:28.202 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.125760+0000", 2024-09-18T10:15:28.202 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00024864100000000002 2024-09-18T10:15:28.203 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.203 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.203 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:28.203 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.126375+0000", 2024-09-18T10:15:28.203 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00061536299999999996 2024-09-18T10:15:28.203 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.203 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.203 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:28.203 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.227656+0000", 2024-09-18T10:15:28.203 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.101281097 2024-09-18T10:15:28.203 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.203 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.203 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:15:28.203 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.625791+0000", 2024-09-18T10:15:28.204 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.39813470299999998 2024-09-18T10:15:28.204 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.204 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.204 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:28.204 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.625953+0000", 2024-09-18T10:15:28.204 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00016192999999999999 2024-09-18T10:15:28.204 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:28.204 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:28.205 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:28.205 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.205 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.205 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:507 3.fs0 3:feb35297:::smithi18925896-37:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e62)", 2024-09-18T10:15:28.206 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:17.914159+0000", 2024-09-18T10:15:28.206 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.195442668, 2024-09-18T10:15:28.206 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0151776809999999, 2024-09-18T10:15:28.206 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:28.206 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-18T10:15:28.206 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:28.206 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:28.206 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:28.206 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 507 2024-09-18T10:15:28.206 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.207 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:28.207 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.208 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:28.208 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.914159+0000", 2024-09-18T10:15:28.208 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:28.208 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.208 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.208 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:28.208 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.914159+0000", 2024-09-18T10:15:28.208 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:28.209 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.209 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.209 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:28.209 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.914158+0000", 2024-09-18T10:15:28.209 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:15:28.209 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.209 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.209 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:28.209 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.914162+0000", 2024-09-18T10:15:28.209 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0729999999999998e-06 2024-09-18T10:15:28.210 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.210 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.210 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:28.210 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.914176+0000", 2024-09-18T10:15:28.210 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4258e-05 2024-09-18T10:15:28.210 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.210 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.210 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:28.210 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.014273+0000", 2024-09-18T10:15:28.210 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10009702400000001 2024-09-18T10:15:28.210 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.210 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.211 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:28.211 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.126754+0000", 2024-09-18T10:15:28.211 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.112480581 2024-09-18T10:15:28.211 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.211 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.211 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2024-09-18T10:15:28.211 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.127051+0000", 2024-09-18T10:15:28.211 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00029721199999999999 2024-09-18T10:15:28.211 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.211 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.211 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:28.211 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.626655+0000", 2024-09-18T10:15:28.211 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.49960442399999999 2024-09-18T10:15:28.212 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.212 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.212 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:28.212 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.626826+0000", 2024-09-18T10:15:28.212 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000170459 2024-09-18T10:15:28.212 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.212 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.212 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:28.212 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.929336+0000", 2024-09-18T10:15:28.212 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30251042900000003 2024-09-18T10:15:28.212 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:28.212 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:28.212 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:28.213 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.213 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.213 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:523 3.as0 3:5c566071:::smithi18925896-28:head [read 0~557056,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T10:15:28.213 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:19.442344+0000", 2024-09-18T10:15:28.213 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 8.6672574620000002, 2024-09-18T10:15:28.213 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.385121027, 2024-09-18T10:15:28.213 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:28.213 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-18T10:15:28.213 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:28.213 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:28.214 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:28.214 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 523 2024-09-18T10:15:28.214 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.214 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:28.214 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.214 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:28.214 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.442344+0000", 2024-09-18T10:15:28.214 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:28.214 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.214 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.214 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:28.214 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.442344+0000", 2024-09-18T10:15:28.214 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:28.215 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.215 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.215 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:28.215 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.442343+0000", 2024-09-18T10:15:28.215 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:15:28.215 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.215 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.215 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:28.215 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.442350+0000", 2024-09-18T10:15:28.215 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7939999999999997e-06 2024-09-18T10:15:28.215 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.215 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.215 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:28.215 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.442364+0000", 2024-09-18T10:15:28.216 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3909e-05 2024-09-18T10:15:28.216 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.216 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.216 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:28.216 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.442373+0000", 2024-09-18T10:15:28.216 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3659999999999998e-06 2024-09-18T10:15:28.216 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.216 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.216 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:28.216 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.444188+0000", 2024-09-18T10:15:28.216 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001814735 2024-09-18T10:15:28.216 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.216 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.217 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:28.217 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.444337+0000", 2024-09-18T10:15:28.217 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014881599999999999 2024-09-18T10:15:28.217 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.217 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.217 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:28.217 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.827465+0000", 2024-09-18T10:15:28.217 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.38312838100000002 2024-09-18T10:15:28.217 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:28.217 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:28.217 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:28.217 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.217 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.217 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:524 3.ds0 3:b1a14543:::smithi18925896-15:head [append 0~442368] snapc 1a=[1a,19,18,16,12,11,10,f,e] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-18T10:15:28.218 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:19.452015+0000", 2024-09-18T10:15:28.218 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 8.6575866860000001, 2024-09-18T10:15:28.218 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.55248773200000001, 2024-09-18T10:15:28.218 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:28.218 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:28.218 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:28.218 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:28.218 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:28.218 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 524 2024-09-18T10:15:28.218 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.218 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:28.218 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.218 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:28.219 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.452015+0000", 2024-09-18T10:15:28.219 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:28.219 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.219 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.219 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:28.219 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.452015+0000", 2024-09-18T10:15:28.219 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:28.219 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.219 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.219 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:28.219 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.452012+0000", 2024-09-18T10:15:28.219 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999976 2024-09-18T10:15:28.219 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.220 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.220 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:28.220 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.452793+0000", 2024-09-18T10:15:28.220 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00078048600000000005 2024-09-18T10:15:28.220 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.220 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.220 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:28.220 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.452825+0000", 2024-09-18T10:15:28.220 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2672999999999997e-05 2024-09-18T10:15:28.220 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.220 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.220 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:28.220 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.552970+0000", 2024-09-18T10:15:28.221 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100144728 2024-09-18T10:15:28.221 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.221 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.221 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:28.221 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.553127+0000", 2024-09-18T10:15:28.221 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000157202 2024-09-18T10:15:28.221 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.221 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.221 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:28.221 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.553439+0000", 2024-09-18T10:15:28.221 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00031139500000000001 2024-09-18T10:15:28.221 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.221 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.221 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:28.222 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.556240+0000", 2024-09-18T10:15:28.222 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0028012509999999998 2024-09-18T10:15:28.222 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.222 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.222 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:28.222 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.665529+0000", 2024-09-18T10:15:28.222 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10928916700000001 2024-09-18T10:15:28.222 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.222 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.222 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:15:28.222 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:20.004378+0000", 2024-09-18T10:15:28.222 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.33884916300000001 2024-09-18T10:15:28.222 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.223 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.223 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:28.223 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:20.004502+0000", 2024-09-18T10:15:28.223 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012407999999999999 2024-09-18T10:15:28.223 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:28.223 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:28.223 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:28.223 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.223 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.223 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:525 3.ds0 3:b1a14543:::smithi18925896-15:head [append 0~540672] snapc 1a=[1a,19,18,16,12,11,10,f,e] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-18T10:15:28.223 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:19.553028+0000", 2024-09-18T10:15:28.223 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 8.556572847, 2024-09-18T10:15:28.223 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.45756244000000001, 2024-09-18T10:15:28.223 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:28.224 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:28.224 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:28.224 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:28.224 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:28.224 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 525 2024-09-18T10:15:28.224 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.224 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:28.224 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.224 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:28.224 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.553028+0000", 2024-09-18T10:15:28.224 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:28.224 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.224 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.225 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:28.225 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.553028+0000", 2024-09-18T10:15:28.225 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:28.225 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.225 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.225 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:28.225 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.553026+0000", 2024-09-18T10:15:28.225 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:28.225 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.225 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.225 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:28.225 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.553796+0000", 2024-09-18T10:15:28.226 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00076915200000000005 2024-09-18T10:15:28.226 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.226 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.226 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:28.226 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.553819+0000", 2024-09-18T10:15:28.226 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3008999999999999e-05 2024-09-18T10:15:28.226 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.226 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.226 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:28.226 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.553829+0000", 2024-09-18T10:15:28.226 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0543999999999999e-05 2024-09-18T10:15:28.226 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.226 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.226 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:28.227 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.558075+0000", 2024-09-18T10:15:28.227 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.004245932 2024-09-18T10:15:28.227 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.227 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.227 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:28.227 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.558384+0000", 2024-09-18T10:15:28.227 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00030926100000000002 2024-09-18T10:15:28.227 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.227 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.227 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:28.228 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.559950+0000", 2024-09-18T10:15:28.228 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001565615 2024-09-18T10:15:28.228 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.228 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.228 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:28.228 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.665535+0000", 2024-09-18T10:15:28.228 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.105584707 2024-09-18T10:15:28.228 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.228 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.228 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:15:28.228 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:20.010428+0000", 2024-09-18T10:15:28.228 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.34489341800000001 2024-09-18T10:15:28.229 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.229 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.229 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:28.229 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:20.010591+0000", 2024-09-18T10:15:28.229 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000162845 2024-09-18T10:15:28.229 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:28.229 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:28.229 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:28.229 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.229 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.229 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:526 3.ds0 3:b1a14543:::smithi18925896-15:head [append 0~622592] snapc 1a=[1a,19,18,16,12,11,10,f,e] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-18T10:15:28.229 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:19.553845+0000", 2024-09-18T10:15:28.229 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 8.5557561319999991, 2024-09-18T10:15:28.230 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.55723060999999996, 2024-09-18T10:15:28.230 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:28.230 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:28.230 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:28.230 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:28.230 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:28.230 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 526 2024-09-18T10:15:28.230 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.230 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:28.230 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.230 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:28.230 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.553845+0000", 2024-09-18T10:15:28.230 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:28.230 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.231 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.231 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:28.231 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.553845+0000", 2024-09-18T10:15:28.231 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:28.231 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.231 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.231 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:28.231 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.553844+0000", 2024-09-18T10:15:28.231 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:28.231 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.231 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.231 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:28.231 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.554666+0000", 2024-09-18T10:15:28.231 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00082239100000000001 2024-09-18T10:15:28.232 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.232 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.232 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:28.232 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.554679+0000", 2024-09-18T10:15:28.232 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.271e-05 2024-09-18T10:15:28.232 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.232 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.232 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:28.232 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.554685+0000", 2024-09-18T10:15:28.232 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4230000000000002e-06 2024-09-18T10:15:28.232 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.232 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.232 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:28.233 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.561235+0000", 2024-09-18T10:15:28.233 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0065498179999999998 2024-09-18T10:15:28.233 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.233 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.233 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:28.233 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.561523+0000", 2024-09-18T10:15:28.233 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000287862 2024-09-18T10:15:28.233 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.233 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.233 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:28.233 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.562904+0000", 2024-09-18T10:15:28.233 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0013810090000000001 2024-09-18T10:15:28.233 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.234 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.234 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:28.234 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.666083+0000", 2024-09-18T10:15:28.234 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.103178495 2024-09-18T10:15:28.234 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.234 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.234 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:15:28.234 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:20.110956+0000", 2024-09-18T10:15:28.234 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.44487371999999997 2024-09-18T10:15:28.234 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.234 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.234 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:28.234 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:20.111076+0000", 2024-09-18T10:15:28.234 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000119436 2024-09-18T10:15:28.235 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:28.235 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:28.235 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:28.235 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.235 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.235 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:527 3.ds0 3:b1a14543:::smithi18925896-15:head [setxattr _header (57) in=7b] snapc 1a=[1a,19,18,16,12,11,10,f,e] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-18T10:15:28.235 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:19.554694+0000", 2024-09-18T10:15:28.235 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 8.5549077649999994, 2024-09-18T10:15:28.235 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.55695282099999999, 2024-09-18T10:15:28.235 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:28.235 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:28.235 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:28.235 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:28.236 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:28.236 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 527 2024-09-18T10:15:28.236 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.236 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:28.236 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.236 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:28.236 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.554694+0000", 2024-09-18T10:15:28.236 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:28.236 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.236 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.236 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:28.236 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.554694+0000", 2024-09-18T10:15:28.236 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:28.236 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.237 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.237 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:28.237 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.554692+0000", 2024-09-18T10:15:28.237 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:15:28.237 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.237 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.237 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:28.237 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.554698+0000", 2024-09-18T10:15:28.237 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8599999999999998e-06 2024-09-18T10:15:28.237 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.237 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.237 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:28.237 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.554707+0000", 2024-09-18T10:15:28.238 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4090000000000003e-06 2024-09-18T10:15:28.238 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.238 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.238 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:28.238 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.554712+0000", 2024-09-18T10:15:28.238 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4500000000000003e-06 2024-09-18T10:15:28.238 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.238 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.238 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:28.238 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.663747+0000", 2024-09-18T10:15:28.238 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10903526400000001 2024-09-18T10:15:28.238 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.238 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.238 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:28.238 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.664066+0000", 2024-09-18T10:15:28.239 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00031885099999999999 2024-09-18T10:15:28.239 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.239 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.239 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:28.239 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.665080+0000", 2024-09-18T10:15:28.239 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010141200000000001 2024-09-18T10:15:28.239 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.239 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.239 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:28.239 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.666476+0000", 2024-09-18T10:15:28.239 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001395456 2024-09-18T10:15:28.239 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.239 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.240 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:15:28.240 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:20.111512+0000", 2024-09-18T10:15:28.240 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.445035929 2024-09-18T10:15:28.240 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:28.240 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:28.240 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:28.240 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:20.111646+0000", 2024-09-18T10:15:28.240 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000134542 2024-09-18T10:15:28.240 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:28.240 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:28.240 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:28.240 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:28.240 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:28.240 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-18T10:15:28.242 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-18T10:15:28.238+0000 7f14dcc5b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22398) UID: 0 2024-09-18T10:15:28.242 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T10:15:28.246 INFO:tasks.rados.rados.0.smithi189.stdout:250: expect (ObjNum 75 snap 7 seq_num 75) 2024-09-18T10:15:28.343 INFO:tasks.ceph.osd.6.smithi154.stderr:2024-09-18T10:15:28.343+0000 7f0abc9e1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22089) UID: 0 2024-09-18T10:15:28.355 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-09-18T10:15:28.356 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-09-18T10:15:28.356 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.356 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.6s3 72/40 ECSubWrite(tid=62, reqid=osd.0.0:62, at_version=72'38, trim_to=0'0, roll_forward_to=71'36))", 2024-09-18T10:15:28.356 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:28.189402+0000", 2024-09-18T10:15:28.356 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 0.165115655, 2024-09-18T10:15:28.356 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.16515384399999999, 2024-09-18T10:15:28.356 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:28.356 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "queued for pg", 2024-09-18T10:15:28.356 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:28.356 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.356 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:28.356 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.189402+0000", 2024-09-18T10:15:28.356 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:28.357 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.357 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.357 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:28.357 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.189402+0000", 2024-09-18T10:15:28.357 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:28.357 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.357 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.357 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:28.357 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.189402+0000", 2024-09-18T10:15:28.357 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:28.357 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.357 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.357 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:28.358 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.189461+0000", 2024-09-18T10:15:28.358 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.9077999999999999e-05 2024-09-18T10:15:28.358 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.358 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.358 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:28.358 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.189484+0000", 2024-09-18T10:15:28.358 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.3353000000000002e-05 2024-09-18T10:15:28.358 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.358 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.358 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:28.358 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.289578+0000", 2024-09-18T10:15:28.358 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100094161 2024-09-18T10:15:28.358 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:28.358 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:28.359 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:28.359 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.359 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.359 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpRead(3.4s3 72/40 ECSubRead(tid=71, to_read={3:213f864e:::smithi18925896-32:head=0,151552,0}, subchunks={3:213f864e:::smithi18925896-32:head=[0,1]}, attrs_to_read=))", 2024-09-18T10:15:28.359 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:28.330519+0000", 2024-09-18T10:15:28.359 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 0.023999353000000001, 2024-09-18T10:15:28.359 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.024135693, 2024-09-18T10:15:28.359 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:28.359 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "queued for pg", 2024-09-18T10:15:28.359 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:28.359 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.359 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:28.359 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.330519+0000", 2024-09-18T10:15:28.359 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:28.360 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.360 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.360 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:28.360 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.330519+0000", 2024-09-18T10:15:28.360 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:28.360 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.360 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.360 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:28.360 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.330517+0000", 2024-09-18T10:15:28.360 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:28.360 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.360 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.360 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:28.360 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.330540+0000", 2024-09-18T10:15:28.361 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.3125999999999998e-05 2024-09-18T10:15:28.361 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.361 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.361 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:28.361 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.330565+0000", 2024-09-18T10:15:28.361 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4606000000000001e-05 2024-09-18T10:15:28.361 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.361 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.361 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:28.361 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.330577+0000", 2024-09-18T10:15:28.361 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2017e-05 2024-09-18T10:15:28.361 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:28.361 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:28.362 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:28.362 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.362 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.362 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.4s3 72/40 ECSubWrite(tid=72, reqid=osd.8.0:72, at_version=72'53, trim_to=0'0, roll_forward_to=71'51))", 2024-09-18T10:15:28.362 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:28.330662+0000", 2024-09-18T10:15:28.362 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 0.023855399999999999, 2024-09-18T10:15:28.362 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.024070744000000002, 2024-09-18T10:15:28.362 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:28.362 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "queued for pg", 2024-09-18T10:15:28.362 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:28.362 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.362 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:28.362 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.330662+0000", 2024-09-18T10:15:28.362 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:28.363 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.363 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.363 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:28.363 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.330662+0000", 2024-09-18T10:15:28.363 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:28.363 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.363 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.363 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:28.363 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.330662+0000", 2024-09-18T10:15:28.363 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:28.363 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.363 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.363 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:28.363 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.330725+0000", 2024-09-18T10:15:28.364 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.3264999999999998e-05 2024-09-18T10:15:28.364 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.364 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.364 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:28.364 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.330745+0000", 2024-09-18T10:15:28.364 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.0035999999999999e-05 2024-09-18T10:15:28.364 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.364 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.364 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:28.364 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.330754+0000", 2024-09-18T10:15:28.364 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.1130000000000007e-06 2024-09-18T10:15:28.364 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:28.364 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:28.365 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:28.365 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.365 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.365 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpRead(3.4s3 72/40 ECSubRead(tid=73, to_read={3:213f864e:::smithi18925896-32:head=0,151552,0}, subchunks={3:213f864e:::smithi18925896-32:head=[0,1]}, attrs_to_read=))", 2024-09-18T10:15:28.365 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:28.330763+0000", 2024-09-18T10:15:28.365 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 0.023754715999999999, 2024-09-18T10:15:28.365 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.024051108000000002, 2024-09-18T10:15:28.365 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:28.365 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "queued for pg", 2024-09-18T10:15:28.365 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:28.365 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.365 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:28.365 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.330763+0000", 2024-09-18T10:15:28.366 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:28.366 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.366 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.366 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:28.366 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.330763+0000", 2024-09-18T10:15:28.366 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:28.366 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.366 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.366 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:28.366 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.330763+0000", 2024-09-18T10:15:28.366 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:28.366 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.366 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.367 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:28.367 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.330770+0000", 2024-09-18T10:15:28.367 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.6699999999999994e-06 2024-09-18T10:15:28.367 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.367 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.367 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:28.367 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.330789+0000", 2024-09-18T10:15:28.367 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8315999999999999e-05 2024-09-18T10:15:28.367 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.367 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.367 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:28.368 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.330796+0000", 2024-09-18T10:15:28.368 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.5170000000000002e-06 2024-09-18T10:15:28.368 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:28.368 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:28.368 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:28.368 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:28.368 INFO:teuthology.orchestra.run.smithi154.stdout: ], 2024-09-18T10:15:28.368 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 4 2024-09-18T10:15:28.368 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-09-18T10:15:28.369 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T10:15:28.436 INFO:tasks.rados.rados.0.smithi189.stdout:255: expect (ObjNum 36 snap 0 seq_num 36) 2024-09-18T10:15:28.444 INFO:tasks.ceph.osd.14.smithi154.stderr:2024-09-18T10:15:28.443+0000 7f75a19e7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 22093) UID: 0 2024-09-18T10:15:28.467 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-09-18T10:15:28.468 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-09-18T10:15:28.468 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-09-18T10:15:28.468 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-09-18T10:15:28.468 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-09-18T10:15:28.476 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T10:15:28.486 INFO:tasks.rados.rados.0.smithi189.stdout:254: finishing copy_from racing read to smithi18925896-44 2024-09-18T10:15:28.545 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-18T10:15:28.542+0000 7f302f2c3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22187) UID: 0 2024-09-18T10:15:28.559 INFO:teuthology.orchestra.run.smithi029.stderr:nodeep-scrub is unset 2024-09-18T10:15:28.627 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-09-18T10:15:28.627 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-09-18T10:15:28.627 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-09-18T10:15:28.627 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-09-18T10:15:28.628 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.628 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.6s3 36/32 ECSubWrite(tid=15, reqid=client.4632.0:122, at_version=36'10, trim_to=0'0, roll_forward_to=36'7))", 2024-09-18T10:15:28.628 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:50.804653+0000", 2024-09-18T10:15:28.628 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 37.818918392999997, 2024-09-18T10:15:28.628 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.108947892, 2024-09-18T10:15:28.628 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:28.628 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:15:28.628 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:28.628 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.628 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:28.628 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.804653+0000", 2024-09-18T10:15:28.629 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:28.629 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.629 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.629 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:28.629 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.804653+0000", 2024-09-18T10:15:28.629 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:28.629 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.629 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.630 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:28.630 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.804653+0000", 2024-09-18T10:15:28.630 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:15:28.630 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.630 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.631 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:28.631 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.804677+0000", 2024-09-18T10:15:28.631 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4043e-05 2024-09-18T10:15:28.631 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.632 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.632 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:28.632 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.804691+0000", 2024-09-18T10:15:28.632 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.402e-05 2024-09-18T10:15:28.632 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.633 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.633 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:28.633 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.906029+0000", 2024-09-18T10:15:28.633 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.101337911 2024-09-18T10:15:28.633 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.633 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.633 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:28.634 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.906208+0000", 2024-09-18T10:15:28.634 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000179292 2024-09-18T10:15:28.634 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.634 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.634 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:28.634 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.906232+0000", 2024-09-18T10:15:28.635 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.3739000000000001e-05 2024-09-18T10:15:28.635 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.635 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.635 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:28.635 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.913534+0000", 2024-09-18T10:15:28.635 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0073023029999999996 2024-09-18T10:15:28.636 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.636 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.636 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:28.636 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.913601+0000", 2024-09-18T10:15:28.636 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.7314e-05 2024-09-18T10:15:28.636 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:28.637 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:28.637 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:28.637 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.637 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.637 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.9s1 36/32 ECSubWrite(tid=16, reqid=client.4632.0:143, at_version=36'12, trim_to=0'0, roll_forward_to=36'9))", 2024-09-18T10:15:28.638 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:51.130949+0000", 2024-09-18T10:15:28.638 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 37.492622629000003, 2024-09-18T10:15:28.638 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.202020333, 2024-09-18T10:15:28.638 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:28.638 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:15:28.638 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:28.638 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.639 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:28.639 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.130949+0000", 2024-09-18T10:15:28.639 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:28.639 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.639 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.639 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:28.639 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.130949+0000", 2024-09-18T10:15:28.639 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:28.640 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.640 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.640 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:28.640 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.130947+0000", 2024-09-18T10:15:28.640 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:28.640 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.641 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.641 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:28.641 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.131149+0000", 2024-09-18T10:15:28.641 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00020114800000000001 2024-09-18T10:15:28.641 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.641 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.642 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:28.642 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.131182+0000", 2024-09-18T10:15:28.642 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.3844000000000003e-05 2024-09-18T10:15:28.642 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.642 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.643 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:28.643 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.131204+0000", 2024-09-18T10:15:28.643 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.1753999999999999e-05 2024-09-18T10:15:28.643 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.643 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.644 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:28.644 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.231367+0000", 2024-09-18T10:15:28.644 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100162472 2024-09-18T10:15:28.644 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.644 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.645 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:28.645 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.231380+0000", 2024-09-18T10:15:28.645 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3624e-05 2024-09-18T10:15:28.645 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.645 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.646 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:28.646 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.332934+0000", 2024-09-18T10:15:28.646 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.101553714 2024-09-18T10:15:28.646 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.646 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.646 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:28.646 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.332969+0000", 2024-09-18T10:15:28.646 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5525999999999998e-05 2024-09-18T10:15:28.647 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:28.647 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:28.647 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:28.647 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.647 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.647 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.9s1 36/32 ECSubWrite(tid=17, reqid=client.4632.0:144, at_version=36'13, trim_to=0'0, roll_forward_to=36'9))", 2024-09-18T10:15:28.647 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:51.131916+0000", 2024-09-18T10:15:28.647 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 37.491655801999997, 2024-09-18T10:15:28.648 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20110955899999999, 2024-09-18T10:15:28.648 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:28.648 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:15:28.648 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:28.648 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.648 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:28.648 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.131916+0000", 2024-09-18T10:15:28.648 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:28.649 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.649 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.649 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:28.649 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.131916+0000", 2024-09-18T10:15:28.649 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:28.649 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.650 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.650 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:28.650 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.131915+0000", 2024-09-18T10:15:28.650 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:28.650 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.650 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.650 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:28.651 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.131949+0000", 2024-09-18T10:15:28.651 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.4576999999999998e-05 2024-09-18T10:15:28.651 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.651 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.651 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:28.651 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.131979+0000", 2024-09-18T10:15:28.651 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9454e-05 2024-09-18T10:15:28.651 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.651 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.651 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:28.651 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.131992+0000", 2024-09-18T10:15:28.651 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3029e-05 2024-09-18T10:15:28.652 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.652 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.652 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:28.652 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.332112+0000", 2024-09-18T10:15:28.652 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.200120295 2024-09-18T10:15:28.652 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.652 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.652 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:28.652 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.332126+0000", 2024-09-18T10:15:28.652 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4494999999999999e-05 2024-09-18T10:15:28.652 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.652 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.653 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:28.653 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.332996+0000", 2024-09-18T10:15:28.653 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00086989000000000001 2024-09-18T10:15:28.653 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.653 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.653 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:28.653 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.333026+0000", 2024-09-18T10:15:28.653 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9159999999999999e-05 2024-09-18T10:15:28.653 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:28.653 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:28.653 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:28.653 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.654 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.654 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpRead(3.4s3 36/32 ECSubRead(tid=11, to_read={3:2650be24:::smithi18925896-45:head=0,4096,0}, subchunks={3:2650be24:::smithi18925896-45:head=[0,1]}, attrs_to_read=))", 2024-09-18T10:15:28.654 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:51.425236+0000", 2024-09-18T10:15:28.654 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 37.198335714999999, 2024-09-18T10:15:28.654 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.200812289, 2024-09-18T10:15:28.654 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:28.654 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:15:28.654 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:28.654 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.654 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:28.654 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.425236+0000", 2024-09-18T10:15:28.654 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:28.654 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.654 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.655 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:28.655 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.425236+0000", 2024-09-18T10:15:28.655 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:28.655 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.655 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.655 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:28.655 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.425234+0000", 2024-09-18T10:15:28.655 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:28.655 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.655 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.655 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:28.655 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.425253+0000", 2024-09-18T10:15:28.655 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8468999999999999e-05 2024-09-18T10:15:28.656 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.656 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.656 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:28.656 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.425289+0000", 2024-09-18T10:15:28.656 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.6514999999999999e-05 2024-09-18T10:15:28.656 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.656 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.656 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:28.656 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.525424+0000", 2024-09-18T10:15:28.657 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100135157 2024-09-18T10:15:28.657 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.657 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.657 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:28.657 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.625672+0000", 2024-09-18T10:15:28.657 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100247555 2024-09-18T10:15:28.657 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.657 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.657 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:28.657 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.626048+0000", 2024-09-18T10:15:28.657 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00037635000000000001 2024-09-18T10:15:28.658 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:28.658 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:28.658 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:28.658 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.658 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.658 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.9s1 37/32 ECSubWrite(tid=30, reqid=client.4632.0:187, at_version=0'0, trim_to=0'0, roll_forward_to=37'17))", 2024-09-18T10:15:28.658 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:53.090512+0000", 2024-09-18T10:15:28.658 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 35.533059860999998, 2024-09-18T10:15:28.658 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.21478031, 2024-09-18T10:15:28.659 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:28.659 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:15:28.659 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:28.659 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.659 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:28.659 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.090512+0000", 2024-09-18T10:15:28.659 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:28.659 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.659 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.659 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:28.659 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.090512+0000", 2024-09-18T10:15:28.659 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:28.659 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.660 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.660 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:28.660 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.090510+0000", 2024-09-18T10:15:28.660 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:28.660 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.660 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.660 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:28.660 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.090535+0000", 2024-09-18T10:15:28.660 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4848999999999999e-05 2024-09-18T10:15:28.661 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.661 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.661 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:28.661 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.090566+0000", 2024-09-18T10:15:28.661 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1096000000000002e-05 2024-09-18T10:15:28.661 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.661 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.662 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:28.662 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.190713+0000", 2024-09-18T10:15:28.662 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10014709300000001 2024-09-18T10:15:28.662 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.662 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.662 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:28.663 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.290966+0000", 2024-09-18T10:15:28.663 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10025297399999999 2024-09-18T10:15:28.663 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.663 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.663 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:28.663 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.290981+0000", 2024-09-18T10:15:28.663 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4960000000000001e-05 2024-09-18T10:15:28.663 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.663 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.663 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:28.663 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.305252+0000", 2024-09-18T10:15:28.663 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.014270738999999999 2024-09-18T10:15:28.663 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.663 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.664 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:28.664 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.305292+0000", 2024-09-18T10:15:28.664 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.0092000000000002e-05 2024-09-18T10:15:28.664 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:28.664 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:28.664 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:28.664 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.664 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.664 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.6s3 41/40 ECSubWrite(tid=32, reqid=client.4632.0:250, at_version=41'14, trim_to=0'0, roll_forward_to=0'0))", 2024-09-18T10:15:28.664 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:57.552410+0000", 2024-09-18T10:15:28.664 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 31.071161755999999, 2024-09-18T10:15:28.664 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20275494299999999, 2024-09-18T10:15:28.665 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:28.665 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:15:28.665 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:28.665 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.665 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:28.665 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.552410+0000", 2024-09-18T10:15:28.665 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:28.665 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.665 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.665 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:28.665 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.552410+0000", 2024-09-18T10:15:28.665 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:28.666 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.666 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.666 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:28.666 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.552409+0000", 2024-09-18T10:15:28.666 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:28.666 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.666 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.666 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:28.666 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.552478+0000", 2024-09-18T10:15:28.666 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.9084999999999993e-05 2024-09-18T10:15:28.666 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.666 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.666 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:28.667 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.552488+0000", 2024-09-18T10:15:28.667 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.7990000000000005e-06 2024-09-18T10:15:28.667 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.667 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.667 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:28.667 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.552496+0000", 2024-09-18T10:15:28.667 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.3200000000000002e-06 2024-09-18T10:15:28.667 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.667 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.668 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:28.668 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.653130+0000", 2024-09-18T10:15:28.668 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100634549 2024-09-18T10:15:28.668 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.668 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.669 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:28.669 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.653144+0000", 2024-09-18T10:15:28.669 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3619e-05 2024-09-18T10:15:28.669 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.669 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.670 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:28.670 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.755128+0000", 2024-09-18T10:15:28.670 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.101983924 2024-09-18T10:15:28.670 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.670 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.670 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:28.671 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.755165+0000", 2024-09-18T10:15:28.671 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.7361999999999999e-05 2024-09-18T10:15:28.671 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:28.671 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:28.671 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:28.672 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.672 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.672 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.6s3 41/40 ECSubWrite(tid=33, reqid=client.4632.0:251, at_version=41'15, trim_to=0'0, roll_forward_to=0'0))", 2024-09-18T10:15:28.672 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:57.553805+0000", 2024-09-18T10:15:28.672 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 31.069767175999999, 2024-09-18T10:15:28.673 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20199056600000001, 2024-09-18T10:15:28.673 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:28.673 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:15:28.673 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:28.673 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.673 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:28.674 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.553805+0000", 2024-09-18T10:15:28.674 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:28.674 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.674 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.675 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:28.675 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.553805+0000", 2024-09-18T10:15:28.675 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:28.675 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.675 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.675 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:28.676 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.553804+0000", 2024-09-18T10:15:28.676 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:15:28.676 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.676 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.676 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:28.677 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.553827+0000", 2024-09-18T10:15:28.677 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.3516e-05 2024-09-18T10:15:28.677 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.677 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.677 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:28.678 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.553845+0000", 2024-09-18T10:15:28.678 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7881999999999999e-05 2024-09-18T10:15:28.678 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.678 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.678 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:28.679 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.553853+0000", 2024-09-18T10:15:28.679 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.7109999999999995e-06 2024-09-18T10:15:28.679 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.679 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.679 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:28.680 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.754527+0000", 2024-09-18T10:15:28.680 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.200674198 2024-09-18T10:15:28.680 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.680 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.680 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:28.680 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.754543+0000", 2024-09-18T10:15:28.681 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.5682e-05 2024-09-18T10:15:28.681 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.681 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.681 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:28.681 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.755758+0000", 2024-09-18T10:15:28.682 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0012155860000000001 2024-09-18T10:15:28.682 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.682 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.682 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:28.683 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.755795+0000", 2024-09-18T10:15:28.683 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.6808999999999999e-05 2024-09-18T10:15:28.683 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:28.683 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:28.683 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:28.684 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.684 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.684 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.19s1 51/40 ECSubWrite(tid=77, reqid=client.4632.0:407, at_version=51'45, trim_to=0'0, roll_forward_to=50'39))", 2024-09-18T10:15:28.684 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:06.877639+0000", 2024-09-18T10:15:28.684 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 21.745932551999999, 2024-09-18T10:15:28.684 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.301778663, 2024-09-18T10:15:28.685 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:28.685 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:15:28.685 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:28.685 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.685 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:28.686 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:06.877639+0000", 2024-09-18T10:15:28.686 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:28.686 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.686 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.686 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:28.687 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:06.877639+0000", 2024-09-18T10:15:28.687 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:28.687 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.687 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.687 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:28.687 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:06.877638+0000", 2024-09-18T10:15:28.688 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:28.688 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.688 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.688 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:28.688 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:06.877679+0000", 2024-09-18T10:15:28.689 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.0908000000000002e-05 2024-09-18T10:15:28.689 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.689 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.689 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:28.689 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:06.877717+0000", 2024-09-18T10:15:28.689 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.7988999999999998e-05 2024-09-18T10:15:28.690 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.690 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.690 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:28.690 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:06.977811+0000", 2024-09-18T10:15:28.690 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100094533 2024-09-18T10:15:28.691 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.691 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.691 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:28.691 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:06.977916+0000", 2024-09-18T10:15:28.691 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000104823 2024-09-18T10:15:28.691 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.692 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.692 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:28.692 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:06.977932+0000", 2024-09-18T10:15:28.692 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.6099999999999998e-05 2024-09-18T10:15:28.692 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.693 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.693 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:28.693 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:07.179377+0000", 2024-09-18T10:15:28.693 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.201444861 2024-09-18T10:15:28.693 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.694 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.694 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:28.694 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:07.179418+0000", 2024-09-18T10:15:28.694 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.0645999999999997e-05 2024-09-18T10:15:28.694 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:28.694 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:28.695 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:28.695 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.695 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.695 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.19s1 51/40 ECSubWrite(tid=78, reqid=client.4632.0:408, at_version=51'46, trim_to=0'0, roll_forward_to=50'39))", 2024-09-18T10:15:28.695 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:06.977840+0000", 2024-09-18T10:15:28.696 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 21.645731847, 2024-09-18T10:15:28.696 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.201619306, 2024-09-18T10:15:28.696 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:28.696 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:15:28.696 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:28.697 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.697 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:28.697 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:06.977840+0000", 2024-09-18T10:15:28.697 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:28.697 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.697 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.697 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:28.697 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:06.977840+0000", 2024-09-18T10:15:28.697 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:28.697 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.697 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.697 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:28.698 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:06.977839+0000", 2024-09-18T10:15:28.698 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:15:28.698 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.698 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.698 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:28.698 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:06.977887+0000", 2024-09-18T10:15:28.698 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.8470999999999997e-05 2024-09-18T10:15:28.698 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.698 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.698 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:28.698 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:06.977924+0000", 2024-09-18T10:15:28.698 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.6729e-05 2024-09-18T10:15:28.698 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.699 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.699 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:28.699 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:06.977937+0000", 2024-09-18T10:15:28.699 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2476e-05 2024-09-18T10:15:28.699 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.699 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.699 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:28.699 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:07.078614+0000", 2024-09-18T10:15:28.699 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100677007 2024-09-18T10:15:28.699 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.699 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.699 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:28.699 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:07.078634+0000", 2024-09-18T10:15:28.699 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.0134000000000001e-05 2024-09-18T10:15:28.700 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.700 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.700 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:28.700 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:07.179429+0000", 2024-09-18T10:15:28.700 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100795736 2024-09-18T10:15:28.700 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.700 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.700 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:28.700 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:07.179459+0000", 2024-09-18T10:15:28.700 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9788999999999999e-05 2024-09-18T10:15:28.700 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:28.700 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:28.700 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:28.701 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.701 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.701 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDScrubReserve(3.11s6 REQUEST e51)", 2024-09-18T10:15:28.701 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:07.146316+0000", 2024-09-18T10:15:28.701 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 21.477255294999999, 2024-09-18T10:15:28.701 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.200408688, 2024-09-18T10:15:28.701 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:28.701 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "started", 2024-09-18T10:15:28.701 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:28.701 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.701 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:28.701 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:07.146316+0000", 2024-09-18T10:15:28.701 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:28.701 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.702 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.702 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:28.702 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:07.146316+0000", 2024-09-18T10:15:28.702 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:28.702 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.702 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.702 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:28.702 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:07.146316+0000", 2024-09-18T10:15:28.702 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:28.702 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.702 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.702 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:28.702 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:07.146320+0000", 2024-09-18T10:15:28.703 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.3340000000000003e-06 2024-09-18T10:15:28.703 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.703 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.703 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:28.703 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:07.146347+0000", 2024-09-18T10:15:28.703 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6899000000000001e-05 2024-09-18T10:15:28.703 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.703 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.703 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:28.703 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:07.246429+0000", 2024-09-18T10:15:28.703 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10008152100000001 2024-09-18T10:15:28.703 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.703 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.703 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:28.704 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:07.346637+0000", 2024-09-18T10:15:28.704 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10020834100000001 2024-09-18T10:15:28.704 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.704 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.704 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:28.704 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:07.346666+0000", 2024-09-18T10:15:28.704 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9269e-05 2024-09-18T10:15:28.704 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.704 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.704 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:28.704 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:07.346725+0000", 2024-09-18T10:15:28.704 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.8748000000000001e-05 2024-09-18T10:15:28.704 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:28.705 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:28.705 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:28.705 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.705 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.705 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.1s6 57/40 ECSubWrite(tid=111, reqid=osd.4.0:111, at_version=57'18, trim_to=0'0, roll_forward_to=56'16))", 2024-09-18T10:15:28.705 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:13.054699+0000", 2024-09-18T10:15:28.705 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 15.568872804, 2024-09-18T10:15:28.705 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10894221699999999, 2024-09-18T10:15:28.705 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:28.705 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:15:28.705 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:28.705 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.706 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:28.706 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.054699+0000", 2024-09-18T10:15:28.706 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:28.706 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.706 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.706 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:28.706 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.054699+0000", 2024-09-18T10:15:28.706 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:28.706 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.706 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.706 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:28.707 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.054699+0000", 2024-09-18T10:15:28.707 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:28.707 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.707 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.707 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:28.707 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.054729+0000", 2024-09-18T10:15:28.707 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0781000000000002e-05 2024-09-18T10:15:28.707 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.707 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.707 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:28.707 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.054746+0000", 2024-09-18T10:15:28.707 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.6626e-05 2024-09-18T10:15:28.707 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.707 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.708 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:28.708 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.054759+0000", 2024-09-18T10:15:28.708 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3037000000000001e-05 2024-09-18T10:15:28.708 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.708 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.708 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:28.708 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.055058+0000", 2024-09-18T10:15:28.708 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000298728 2024-09-18T10:15:28.708 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.708 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.708 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:28.708 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.055080+0000", 2024-09-18T10:15:28.709 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.2796e-05 2024-09-18T10:15:28.709 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.709 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.709 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:28.709 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.163616+0000", 2024-09-18T10:15:28.709 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.108535338 2024-09-18T10:15:28.709 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.709 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.709 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:28.709 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.163641+0000", 2024-09-18T10:15:28.709 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5371000000000002e-05 2024-09-18T10:15:28.709 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:28.709 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:28.709 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:28.710 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.710 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.710 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.1s6 57/40 ECSubWrite(tid=112, reqid=osd.4.0:112, at_version=57'20, trim_to=0'0, roll_forward_to=56'16))", 2024-09-18T10:15:28.710 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:13.056191+0000", 2024-09-18T10:15:28.710 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 15.567380462999999, 2024-09-18T10:15:28.710 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.107487746, 2024-09-18T10:15:28.710 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:28.710 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:15:28.710 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:28.710 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.710 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:28.710 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.056191+0000", 2024-09-18T10:15:28.710 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:28.710 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.711 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.711 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:28.711 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.056191+0000", 2024-09-18T10:15:28.711 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:28.711 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.711 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.711 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:28.711 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.056190+0000", 2024-09-18T10:15:28.711 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:15:28.711 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.711 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.711 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:28.711 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.056221+0000", 2024-09-18T10:15:28.712 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1075999999999999e-05 2024-09-18T10:15:28.712 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.712 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.712 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:28.712 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.056238+0000", 2024-09-18T10:15:28.712 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.6231000000000001e-05 2024-09-18T10:15:28.712 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.712 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.712 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:28.712 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.056246+0000", 2024-09-18T10:15:28.712 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.3480000000000004e-06 2024-09-18T10:15:28.712 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.712 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.712 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:28.713 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.056672+0000", 2024-09-18T10:15:28.713 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00042641800000000002 2024-09-18T10:15:28.713 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.713 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.713 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:28.713 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.056688+0000", 2024-09-18T10:15:28.713 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.5274e-05 2024-09-18T10:15:28.713 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.713 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.713 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:28.713 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.163649+0000", 2024-09-18T10:15:28.713 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10696096200000001 2024-09-18T10:15:28.713 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.714 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.714 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:28.714 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.163679+0000", 2024-09-18T10:15:28.714 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0385000000000001e-05 2024-09-18T10:15:28.714 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:28.714 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:28.714 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:28.714 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.714 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.714 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.19s1 57/40 ECSubWrite(tid=94, reqid=osd.7.0:92, at_version=0'0, trim_to=0'0, roll_forward_to=57'50))", 2024-09-18T10:15:28.714 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:13.157902+0000", 2024-09-18T10:15:28.714 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 15.465670055, 2024-09-18T10:15:28.714 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.111878063, 2024-09-18T10:15:28.715 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:28.715 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:15:28.715 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:28.715 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.715 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:28.715 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.157902+0000", 2024-09-18T10:15:28.715 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:28.715 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.715 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.715 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:28.715 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.157902+0000", 2024-09-18T10:15:28.715 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:28.715 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.715 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.716 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:28.716 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.157901+0000", 2024-09-18T10:15:28.716 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:15:28.716 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.716 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.716 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:28.716 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.157919+0000", 2024-09-18T10:15:28.716 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8241999999999999e-05 2024-09-18T10:15:28.716 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.716 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.716 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:28.716 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.157946+0000", 2024-09-18T10:15:28.716 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.7025999999999998e-05 2024-09-18T10:15:28.716 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.717 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.717 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:28.717 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.157957+0000", 2024-09-18T10:15:28.717 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1422e-05 2024-09-18T10:15:28.717 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.717 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.717 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:28.717 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.263418+0000", 2024-09-18T10:15:28.717 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.105460525 2024-09-18T10:15:28.717 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.717 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.717 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:28.717 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.263436+0000", 2024-09-18T10:15:28.717 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8451e-05 2024-09-18T10:15:28.718 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.718 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.718 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:28.718 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.269729+0000", 2024-09-18T10:15:28.718 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0062925519999999999 2024-09-18T10:15:28.718 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.718 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.718 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:28.718 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.269780+0000", 2024-09-18T10:15:28.718 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.0926e-05 2024-09-18T10:15:28.718 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:28.718 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:28.718 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:28.718 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.719 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.719 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.1s6 58/40 ECSubWrite(tid=113, reqid=osd.4.0:112, at_version=0'0, trim_to=0'0, roll_forward_to=57'20))", 2024-09-18T10:15:28.719 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:13.649564+0000", 2024-09-18T10:15:28.719 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 14.974007388, 2024-09-18T10:15:28.719 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.204868989, 2024-09-18T10:15:28.719 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:28.719 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:15:28.719 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:28.719 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.719 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:28.719 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.649564+0000", 2024-09-18T10:15:28.719 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:28.719 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.720 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.720 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:28.720 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.649564+0000", 2024-09-18T10:15:28.720 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:28.720 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.720 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.720 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:28.720 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.649563+0000", 2024-09-18T10:15:28.720 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:28.720 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.720 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.720 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:28.720 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.649600+0000", 2024-09-18T10:15:28.721 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.7224999999999997e-05 2024-09-18T10:15:28.721 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.721 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.721 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:28.721 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.649651+0000", 2024-09-18T10:15:28.721 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.0905000000000001e-05 2024-09-18T10:15:28.721 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.721 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.721 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:28.721 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.649669+0000", 2024-09-18T10:15:28.721 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8522999999999999e-05 2024-09-18T10:15:28.721 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.721 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.722 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:28.722 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.750460+0000", 2024-09-18T10:15:28.722 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100791013 2024-09-18T10:15:28.722 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.722 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.722 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:28.722 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.750487+0000", 2024-09-18T10:15:28.722 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6389000000000001e-05 2024-09-18T10:15:28.722 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.722 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.722 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:28.723 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.854315+0000", 2024-09-18T10:15:28.723 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.103828238 2024-09-18T10:15:28.723 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.723 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.723 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:28.723 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.854433+0000", 2024-09-18T10:15:28.723 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000118449 2024-09-18T10:15:28.723 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:28.723 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:28.723 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:28.723 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.723 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.723 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.19s1 58/40 ECSubWrite(tid=97, reqid=osd.7.0:96, at_version=0'0, trim_to=0'0, roll_forward_to=57'54))", 2024-09-18T10:15:28.724 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:13.672411+0000", 2024-09-18T10:15:28.724 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 14.951160958999999, 2024-09-18T10:15:28.724 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.28289547100000001, 2024-09-18T10:15:28.724 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:28.724 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:15:28.724 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:28.724 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.724 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:28.724 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.672411+0000", 2024-09-18T10:15:28.724 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:28.724 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.724 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.724 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:28.725 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.672411+0000", 2024-09-18T10:15:28.725 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:28.725 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.725 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.725 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:28.725 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.672410+0000", 2024-09-18T10:15:28.725 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:28.725 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.725 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.725 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:28.725 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.672428+0000", 2024-09-18T10:15:28.725 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7116e-05 2024-09-18T10:15:28.725 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.725 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.726 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:28.726 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.672451+0000", 2024-09-18T10:15:28.726 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.3611000000000001e-05 2024-09-18T10:15:28.726 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.726 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.726 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:28.726 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.672465+0000", 2024-09-18T10:15:28.726 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3818999999999999e-05 2024-09-18T10:15:28.726 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.726 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.726 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:28.726 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.853423+0000", 2024-09-18T10:15:28.726 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.180957599 2024-09-18T10:15:28.726 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.727 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.727 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:28.727 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.853438+0000", 2024-09-18T10:15:28.727 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.5446000000000001e-05 2024-09-18T10:15:28.727 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.727 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.727 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:28.727 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.955241+0000", 2024-09-18T10:15:28.727 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.101803053 2024-09-18T10:15:28.727 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.727 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.727 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:28.727 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.955306+0000", 2024-09-18T10:15:28.728 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.5215000000000005e-05 2024-09-18T10:15:28.728 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:28.728 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:28.728 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:28.728 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.728 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.728 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.1s6 58/40 ECSubWrite(tid=120, reqid=osd.4.0:117, at_version=0'0, trim_to=0'0, roll_forward_to=58'22))", 2024-09-18T10:15:28.728 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:14.170897+0000", 2024-09-18T10:15:28.728 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 14.452674829999999, 2024-09-18T10:15:28.728 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20147283599999999, 2024-09-18T10:15:28.728 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:28.728 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:15:28.728 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:28.728 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.729 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:28.729 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:14.170897+0000", 2024-09-18T10:15:28.729 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:28.729 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.729 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.729 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:28.729 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:14.170897+0000", 2024-09-18T10:15:28.729 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:28.729 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.729 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.729 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:28.729 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:14.170895+0000", 2024-09-18T10:15:28.729 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:28.730 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.730 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.730 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:28.730 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:14.170921+0000", 2024-09-18T10:15:28.730 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5848999999999999e-05 2024-09-18T10:15:28.730 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.730 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.730 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:28.730 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:14.170998+0000", 2024-09-18T10:15:28.730 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.7341000000000004e-05 2024-09-18T10:15:28.730 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.730 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.730 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:28.730 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:14.171019+0000", 2024-09-18T10:15:28.731 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.0443000000000001e-05 2024-09-18T10:15:28.731 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.731 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.731 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:28.731 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:14.271283+0000", 2024-09-18T10:15:28.731 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100263848 2024-09-18T10:15:28.731 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.731 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.731 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:28.731 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:14.271316+0000", 2024-09-18T10:15:28.731 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.3516000000000002e-05 2024-09-18T10:15:28.731 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.731 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.731 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:28.732 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:14.372304+0000", 2024-09-18T10:15:28.732 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100987524 2024-09-18T10:15:28.732 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.732 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.732 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:28.732 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:14.372370+0000", 2024-09-18T10:15:28.732 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.6107999999999998e-05 2024-09-18T10:15:28.732 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:28.732 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:28.732 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:28.732 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.732 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.732 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.9s1 63/40 ECSubWrite(tid=118, reqid=client.4632.0:531, at_version=63'56, trim_to=0'0, roll_forward_to=60'52))", 2024-09-18T10:15:28.733 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:19.565882+0000", 2024-09-18T10:15:28.733 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 9.0576898860000004, 2024-09-18T10:15:28.733 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.201669185, 2024-09-18T10:15:28.733 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:28.733 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:15:28.733 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:28.733 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.733 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:28.733 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.565882+0000", 2024-09-18T10:15:28.733 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:28.733 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.733 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.733 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:28.733 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.565882+0000", 2024-09-18T10:15:28.734 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:28.734 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.734 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.734 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:28.734 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.565881+0000", 2024-09-18T10:15:28.734 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:28.734 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.734 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.734 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:28.734 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.565919+0000", 2024-09-18T10:15:28.734 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8254999999999999e-05 2024-09-18T10:15:28.734 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.734 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.735 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:28.735 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.565952+0000", 2024-09-18T10:15:28.735 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.3219000000000002e-05 2024-09-18T10:15:28.735 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.735 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.735 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:28.735 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.565965+0000", 2024-09-18T10:15:28.735 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.272e-05 2024-09-18T10:15:28.735 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.735 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.735 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:28.735 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.766254+0000", 2024-09-18T10:15:28.735 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.200289512 2024-09-18T10:15:28.735 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.736 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.736 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:28.736 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.766272+0000", 2024-09-18T10:15:28.736 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7306e-05 2024-09-18T10:15:28.736 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.736 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.736 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:28.736 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.767501+0000", 2024-09-18T10:15:28.736 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001229794 2024-09-18T10:15:28.736 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.736 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.736 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:28.736 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.767551+0000", 2024-09-18T10:15:28.737 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9662e-05 2024-09-18T10:15:28.737 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:28.737 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:28.737 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:28.737 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.737 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.737 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.4s3 66/40 ECSubWrite(tid=57, reqid=osd.8.0:57, at_version=66'40, trim_to=0'0, roll_forward_to=66'38))", 2024-09-18T10:15:28.737 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:22.543395+0000", 2024-09-18T10:15:28.737 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 6.0801771120000003, 2024-09-18T10:15:28.737 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20190205, 2024-09-18T10:15:28.737 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:28.737 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:15:28.737 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:28.737 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.738 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:28.738 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:22.543395+0000", 2024-09-18T10:15:28.738 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:28.738 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.738 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.738 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:28.738 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:22.543395+0000", 2024-09-18T10:15:28.738 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:28.738 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.738 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.738 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:28.738 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:22.543393+0000", 2024-09-18T10:15:28.738 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:28.739 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.739 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.739 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:28.739 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:22.543438+0000", 2024-09-18T10:15:28.739 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.5504999999999999e-05 2024-09-18T10:15:28.739 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.739 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.739 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:28.739 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:22.543471+0000", 2024-09-18T10:15:28.739 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2969000000000002e-05 2024-09-18T10:15:28.739 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.739 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.739 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:28.740 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:22.643568+0000", 2024-09-18T10:15:28.740 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100096327 2024-09-18T10:15:28.740 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.740 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.740 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:28.740 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:22.743796+0000", 2024-09-18T10:15:28.740 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100228206 2024-09-18T10:15:28.740 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.740 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.740 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:28.740 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:22.743818+0000", 2024-09-18T10:15:28.740 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.1778999999999999e-05 2024-09-18T10:15:28.740 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.740 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.741 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:28.741 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:22.745263+0000", 2024-09-18T10:15:28.741 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0014448729999999999 2024-09-18T10:15:28.741 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.741 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.741 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:28.741 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:22.745297+0000", 2024-09-18T10:15:28.741 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.4190999999999998e-05 2024-09-18T10:15:28.741 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:28.741 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:28.741 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:28.741 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.741 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.741 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.9s1 71/40 ECSubWrite(tid=171, reqid=osd.7.0:168, at_version=0'0, trim_to=0'0, roll_forward_to=71'74))", 2024-09-18T10:15:28.742 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:27.693404+0000", 2024-09-18T10:15:28.742 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 0.93016759299999996, 2024-09-18T10:15:28.742 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.196191646, 2024-09-18T10:15:28.742 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:28.742 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:15:28.742 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:28.742 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.742 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:28.742 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:27.693404+0000", 2024-09-18T10:15:28.742 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:28.742 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.742 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.742 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:28.743 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:27.693404+0000", 2024-09-18T10:15:28.743 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:28.743 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.743 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.743 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:28.743 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:27.693402+0000", 2024-09-18T10:15:28.743 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:28.743 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.743 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.743 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:28.743 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:27.693429+0000", 2024-09-18T10:15:28.743 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6733e-05 2024-09-18T10:15:28.743 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.743 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.744 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:28.744 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:27.693471+0000", 2024-09-18T10:15:28.744 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.2024999999999998e-05 2024-09-18T10:15:28.744 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.744 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.744 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:28.744 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:27.693538+0000", 2024-09-18T10:15:28.744 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.7231999999999996e-05 2024-09-18T10:15:28.744 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.744 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.744 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:28.744 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:27.888225+0000", 2024-09-18T10:15:28.744 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.19468701499999999 2024-09-18T10:15:28.744 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.745 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.745 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:28.745 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:27.888252+0000", 2024-09-18T10:15:28.745 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6699999999999998e-05 2024-09-18T10:15:28.745 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.745 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.745 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:28.745 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:27.889524+0000", 2024-09-18T10:15:28.745 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001271933 2024-09-18T10:15:28.745 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.745 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.745 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:28.745 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:27.889596+0000", 2024-09-18T10:15:28.745 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.1967999999999999e-05 2024-09-18T10:15:28.746 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:28.746 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:28.746 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:28.746 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.746 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.746 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.6s3 72/40 ECSubWrite(tid=62, reqid=osd.0.0:62, at_version=72'38, trim_to=0'0, roll_forward_to=71'36))", 2024-09-18T10:15:28.746 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:28.189402+0000", 2024-09-18T10:15:28.746 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 0.43416957, 2024-09-18T10:15:28.746 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.201226193, 2024-09-18T10:15:28.746 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:28.746 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:15:28.746 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:28.746 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.747 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:28.747 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.189402+0000", 2024-09-18T10:15:28.747 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:28.747 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.747 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.747 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:28.747 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.189402+0000", 2024-09-18T10:15:28.747 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:28.747 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.747 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.747 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:28.747 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.189402+0000", 2024-09-18T10:15:28.747 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:28.747 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.747 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.748 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:28.748 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.189461+0000", 2024-09-18T10:15:28.748 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.9077999999999999e-05 2024-09-18T10:15:28.748 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.748 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.748 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:28.748 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.189484+0000", 2024-09-18T10:15:28.748 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.3353000000000002e-05 2024-09-18T10:15:28.748 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.748 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.748 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:28.748 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.289578+0000", 2024-09-18T10:15:28.748 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100094161 2024-09-18T10:15:28.749 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.749 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.749 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:28.749 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.389741+0000", 2024-09-18T10:15:28.749 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100162923 2024-09-18T10:15:28.749 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.749 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.749 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:28.749 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.389749+0000", 2024-09-18T10:15:28.749 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.2889999999999998e-06 2024-09-18T10:15:28.749 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.749 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.749 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:28.749 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.390593+0000", 2024-09-18T10:15:28.750 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00084384499999999997 2024-09-18T10:15:28.750 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.750 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.750 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:28.750 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.390628+0000", 2024-09-18T10:15:28.750 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5120999999999998e-05 2024-09-18T10:15:28.750 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:28.750 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:28.750 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:28.750 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:28.750 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:28.750 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-09-18T10:15:28.753 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T10:15:28.754 INFO:tasks.rados.rados.0.smithi189.stdout:238: done (7 left) 2024-09-18T10:15:28.754 INFO:tasks.rados.rados.0.smithi189.stdout:244: done (6 left) 2024-09-18T10:15:28.754 INFO:tasks.rados.rados.0.smithi189.stdout:250: done (5 left) 2024-09-18T10:15:28.754 INFO:tasks.rados.rados.0.smithi189.stdout:251: done (4 left) 2024-09-18T10:15:28.754 INFO:tasks.rados.rados.0.smithi189.stdout:252: done (3 left) 2024-09-18T10:15:28.754 INFO:tasks.rados.rados.0.smithi189.stdout:255: done (2 left) 2024-09-18T10:15:28.754 INFO:tasks.rados.rados.0.smithi189.stdout:256: done (1 left) 2024-09-18T10:15:28.754 INFO:tasks.rados.rados.0.smithi189.stdout:257: rollback oid 7 current snap is 21 2024-09-18T10:15:28.754 INFO:tasks.rados.rados.0.smithi189.stdout:rollback oid 7 to 16 2024-09-18T10:15:28.754 INFO:tasks.rados.rados.0.smithi189.stdout:254: finishing copy_from to smithi18925896-44 2024-09-18T10:15:28.754 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 44 v 14 (ObjNum 33 snap 0 seq_num 33) dirty exists 2024-09-18T10:15:28.754 INFO:tasks.rados.rados.0.smithi189.stdout:254: done (1 left) 2024-09-18T10:15:28.754 INFO:tasks.rados.rados.0.smithi189.stdout:append oid 20 current snap is 21 2024-09-18T10:15:28.755 INFO:tasks.rados.rados.0.smithi189.stdout:258: seq_num 107 ranges {1687552=638976,2326528=458752,2785280=704512} 2024-09-18T10:15:28.755 INFO:tasks.rados.rados.0.smithi189.stdout:258: writing smithi18925896-20 from 1687552 to 2326528 tid 1 2024-09-18T10:15:28.755 INFO:tasks.rados.rados.0.smithi189.stdout:258: writing smithi18925896-20 from 2326528 to 2785280 tid 2 2024-09-18T10:15:28.755 INFO:tasks.rados.rados.0.smithi189.stdout:258: writing smithi18925896-20 from 2785280 to 3489792 tid 3 2024-09-18T10:15:28.755 INFO:tasks.rados.rados.0.smithi189.stdout:259: read oid 50 snap -1 2024-09-18T10:15:28.755 INFO:tasks.rados.rados.0.smithi189.stdout:259: expect (ObjNum 100 snap 17 seq_num 100) 2024-09-18T10:15:28.755 INFO:tasks.rados.rados.0.smithi189.stdout:260: snap_remove snap 11 2024-09-18T10:15:28.756 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-18T10:15:28.748+0000 7f14c6a1a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 22138) UID: 0 2024-09-18T10:15:28.852 INFO:tasks.ceph.osd.15.smithi189.stderr:2024-09-18T10:15:28.846+0000 7fd0e725e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 22401) UID: 0 2024-09-18T10:15:28.863 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-09-18T10:15:28.863 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-09-18T10:15:28.863 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.863 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWriteReply(3.8s0 73/40 ECSubWriteReply(tid=193, last_complete=72'23, committed=1, applied=1))", 2024-09-18T10:15:28.863 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:28.790894+0000", 2024-09-18T10:15:28.863 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 0.071189949000000002, 2024-09-18T10:15:28.863 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.071229297999999996, 2024-09-18T10:15:28.863 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:28.863 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "queued for pg", 2024-09-18T10:15:28.863 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:28.864 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.864 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:28.864 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.790894+0000", 2024-09-18T10:15:28.864 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:28.864 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.864 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.864 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:28.864 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.790894+0000", 2024-09-18T10:15:28.864 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:28.864 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.864 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.864 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:28.864 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.790893+0000", 2024-09-18T10:15:28.864 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:28.865 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.865 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.865 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:28.865 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.790906+0000", 2024-09-18T10:15:28.865 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2653000000000001e-05 2024-09-18T10:15:28.865 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.865 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.865 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:28.865 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.790925+0000", 2024-09-18T10:15:28.865 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.9179999999999999e-05 2024-09-18T10:15:28.865 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.865 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.865 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:28.866 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.790939+0000", 2024-09-18T10:15:28.866 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3594000000000001e-05 2024-09-18T10:15:28.866 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:28.866 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:28.866 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:28.866 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.866 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.866 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWriteReply(3.8s0 73/40 ECSubWriteReply(tid=193, last_complete=72'23, committed=1, applied=1))", 2024-09-18T10:15:28.866 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:28.791746+0000", 2024-09-18T10:15:28.866 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 0.070337983000000007, 2024-09-18T10:15:28.866 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.070459003000000006, 2024-09-18T10:15:28.866 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:28.866 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "queued for pg", 2024-09-18T10:15:28.867 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:28.867 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.867 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:28.867 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.791746+0000", 2024-09-18T10:15:28.867 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:28.867 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.867 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.867 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:28.867 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.791746+0000", 2024-09-18T10:15:28.867 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:28.867 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.867 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.867 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:28.867 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.791745+0000", 2024-09-18T10:15:28.868 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:15:28.868 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.868 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.868 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:28.868 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.791754+0000", 2024-09-18T10:15:28.868 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.7979999999999995e-06 2024-09-18T10:15:28.868 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.868 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.868 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:28.868 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.791782+0000", 2024-09-18T10:15:28.868 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.7543e-05 2024-09-18T10:15:28.868 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.869 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.869 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:28.869 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.791792+0000", 2024-09-18T10:15:28.869 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0774000000000001e-05 2024-09-18T10:15:28.869 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:28.869 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:28.869 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:28.869 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.869 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.870 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:627 3.10s0 3.cd6bc150 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-18T10:15:28.870 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:28.809382+0000", 2024-09-18T10:15:28.870 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 0.052701927000000003, 2024-09-18T10:15:28.870 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.052900114999999998, 2024-09-18T10:15:28.870 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:28.870 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "no flag points reached", 2024-09-18T10:15:28.870 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:28.870 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:15:28.870 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:28.870 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 627 2024-09-18T10:15:28.870 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.870 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:28.870 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.871 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:28.871 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.809382+0000", 2024-09-18T10:15:28.871 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:28.871 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.871 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.871 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:28.871 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.809382+0000", 2024-09-18T10:15:28.871 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:28.871 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.871 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.871 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:28.871 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.809380+0000", 2024-09-18T10:15:28.871 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999971 2024-09-18T10:15:28.872 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.872 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.872 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:28.872 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.809798+0000", 2024-09-18T10:15:28.872 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00041867400000000001 2024-09-18T10:15:28.872 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.872 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.872 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:28.872 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.809836+0000", 2024-09-18T10:15:28.872 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.7453999999999997e-05 2024-09-18T10:15:28.872 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:28.872 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:28.872 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:28.872 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.873 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.873 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWriteReply(3.8s0 72/40 ECSubWriteReply(tid=193, last_complete=72'23, committed=1, applied=1))", 2024-09-18T10:15:28.873 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:28.691434+0000", 2024-09-18T10:15:28.873 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 0.17065055200000001, 2024-09-18T10:15:28.873 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.170927774, 2024-09-18T10:15:28.873 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:28.873 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "queued for pg", 2024-09-18T10:15:28.873 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:28.873 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.873 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:28.873 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.691434+0000", 2024-09-18T10:15:28.873 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:28.873 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.874 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.874 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:28.874 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.691434+0000", 2024-09-18T10:15:28.874 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:28.874 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.874 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.874 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:28.874 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.691433+0000", 2024-09-18T10:15:28.874 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:28.874 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.874 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.874 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:28.874 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.691439+0000", 2024-09-18T10:15:28.874 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.3700000000000003e-06 2024-09-18T10:15:28.875 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.875 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.875 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:28.875 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.691459+0000", 2024-09-18T10:15:28.875 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.0703000000000002e-05 2024-09-18T10:15:28.875 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.875 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.875 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:28.875 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.791537+0000", 2024-09-18T10:15:28.875 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10007782699999999 2024-09-18T10:15:28.875 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:28.875 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:28.876 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:28.876 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.876 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.876 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:626 3.10s0 3.cd6bc150 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-18T10:15:28.876 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:28.708261+0000", 2024-09-18T10:15:28.876 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 0.15382290000000001, 2024-09-18T10:15:28.876 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.15417104400000001, 2024-09-18T10:15:28.876 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:28.877 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "queued for pg", 2024-09-18T10:15:28.877 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:28.877 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:15:28.877 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:28.877 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 626 2024-09-18T10:15:28.877 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.877 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:28.877 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.877 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:28.877 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.708261+0000", 2024-09-18T10:15:28.877 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:28.877 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.877 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.877 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:28.878 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.708261+0000", 2024-09-18T10:15:28.878 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:28.878 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.878 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.878 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:28.878 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.708259+0000", 2024-09-18T10:15:28.878 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:28.878 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.878 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.878 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:28.878 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.709175+0000", 2024-09-18T10:15:28.878 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00091571199999999995 2024-09-18T10:15:28.878 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.879 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.879 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:28.879 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.709199+0000", 2024-09-18T10:15:28.879 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.3785e-05 2024-09-18T10:15:28.879 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:28.879 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:28.879 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:28.879 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.809328+0000", 2024-09-18T10:15:28.879 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100129494 2024-09-18T10:15:28.879 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:28.879 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:28.879 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:28.879 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:28.880 INFO:teuthology.orchestra.run.smithi154.stdout: ], 2024-09-18T10:15:28.880 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 5 2024-09-18T10:15:28.880 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-09-18T10:15:28.881 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T10:15:28.953 INFO:tasks.ceph.osd.6.smithi154.stderr:2024-09-18T10:15:28.951+0000 7f0abc9e1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22089) UID: 0 2024-09-18T10:15:28.983 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-09-18T10:15:28.983 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-09-18T10:15:28.984 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-09-18T10:15:28.984 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-09-18T10:15:28.984 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-09-18T10:15:28.993 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T10:15:29.054 INFO:tasks.ceph.osd.11.smithi189.stderr:2024-09-18T10:15:29.050+0000 7f718b60a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 22400) UID: 0 2024-09-18T10:15:29.097 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-09-18T10:15:29.097 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-09-18T10:15:29.097 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-09-18T10:15:29.097 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-09-18T10:15:29.097 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.097 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-18T10:15:29.097 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:35.293815+0000", 2024-09-18T10:15:29.098 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 53.795431407000002, 2024-09-18T10:15:29.098 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.513288525, 2024-09-18T10:15:29.098 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:29.098 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:29.098 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:29.098 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:15:29.098 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:15:29.098 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 52 2024-09-18T10:15:29.098 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.098 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:29.099 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.099 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:29.099 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.293815+0000", 2024-09-18T10:15:29.099 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:29.099 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.099 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.099 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:29.099 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.293815+0000", 2024-09-18T10:15:29.099 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:29.099 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.099 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.099 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:29.099 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.293814+0000", 2024-09-18T10:15:29.099 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:15:29.100 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.100 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.100 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:29.100 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.293822+0000", 2024-09-18T10:15:29.100 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.8089999999999993e-06 2024-09-18T10:15:29.100 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.100 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.100 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:29.100 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.293833+0000", 2024-09-18T10:15:29.100 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1389000000000001e-05 2024-09-18T10:15:29.100 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.100 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.100 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:29.101 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.393937+0000", 2024-09-18T10:15:29.101 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10010415 2024-09-18T10:15:29.101 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.101 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.101 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:29.101 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.704491+0000", 2024-09-18T10:15:29.101 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.310554202 2024-09-18T10:15:29.101 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.101 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.101 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:29.101 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.704647+0000", 2024-09-18T10:15:29.101 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00015524399999999999 2024-09-18T10:15:29.102 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.102 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.102 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:15:29.102 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.704859+0000", 2024-09-18T10:15:29.102 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000212664 2024-09-18T10:15:29.102 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.102 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.102 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:15:29.102 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.705935+0000", 2024-09-18T10:15:29.102 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001075505 2024-09-18T10:15:29.102 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.102 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.102 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:15:29.102 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.806952+0000", 2024-09-18T10:15:29.103 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10101700299999999 2024-09-18T10:15:29.103 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.103 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.103 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:15:29.103 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.807041+0000", 2024-09-18T10:15:29.103 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.8811000000000001e-05 2024-09-18T10:15:29.103 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.103 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.103 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:29.103 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.807103+0000", 2024-09-18T10:15:29.103 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.2596999999999997e-05 2024-09-18T10:15:29.103 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:29.103 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:29.103 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:29.104 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.104 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.104 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:71 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T10:15:29.104 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:36.786902+0000", 2024-09-18T10:15:29.104 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 52.302344642999998, 2024-09-18T10:15:29.104 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.51197160200000003, 2024-09-18T10:15:29.104 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:29.104 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:29.104 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:29.104 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:15:29.104 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:15:29.104 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 71 2024-09-18T10:15:29.104 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.105 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:29.105 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.105 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:29.105 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.786902+0000", 2024-09-18T10:15:29.105 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:29.105 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.105 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.105 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:29.105 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.786902+0000", 2024-09-18T10:15:29.105 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:29.105 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.106 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.106 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:29.106 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.786900+0000", 2024-09-18T10:15:29.106 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:15:29.106 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.106 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.106 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:29.106 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.786909+0000", 2024-09-18T10:15:29.106 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.9430000000000006e-06 2024-09-18T10:15:29.106 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.106 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.106 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:29.107 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.786921+0000", 2024-09-18T10:15:29.107 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1789e-05 2024-09-18T10:15:29.107 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.107 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.107 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:29.107 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.786927+0000", 2024-09-18T10:15:29.107 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5770000000000001e-06 2024-09-18T10:15:29.107 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.107 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.107 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:29.107 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.293768+0000", 2024-09-18T10:15:29.108 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.50684175499999995 2024-09-18T10:15:29.108 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.108 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.108 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:29.108 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.293894+0000", 2024-09-18T10:15:29.108 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00012588899999999999 2024-09-18T10:15:29.108 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.108 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.108 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:15:29.108 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.294094+0000", 2024-09-18T10:15:29.108 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000199896 2024-09-18T10:15:29.108 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.108 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.109 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:15:29.109 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.298660+0000", 2024-09-18T10:15:29.109 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.004566082 2024-09-18T10:15:29.109 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.109 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.109 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:15:29.109 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.298739+0000", 2024-09-18T10:15:29.109 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.8269000000000006e-05 2024-09-18T10:15:29.109 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.109 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.109 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:15:29.109 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.298814+0000", 2024-09-18T10:15:29.109 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.5582999999999998e-05 2024-09-18T10:15:29.109 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.110 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.110 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:29.110 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.298873+0000", 2024-09-18T10:15:29.110 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.8989000000000001e-05 2024-09-18T10:15:29.110 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:29.110 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:29.110 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:29.110 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.110 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.110 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T10:15:29.110 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:36.786936+0000", 2024-09-18T10:15:29.110 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 52.302310445000003, 2024-09-18T10:15:29.110 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.71130157999999999, 2024-09-18T10:15:29.110 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:29.111 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:29.111 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:29.111 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:15:29.111 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:15:29.111 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 72 2024-09-18T10:15:29.111 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.111 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:29.111 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.111 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:29.111 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.786936+0000", 2024-09-18T10:15:29.111 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:29.111 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.111 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.111 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:29.112 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.786936+0000", 2024-09-18T10:15:29.112 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:29.112 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.112 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.112 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:29.112 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.786935+0000", 2024-09-18T10:15:29.112 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:29.112 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.112 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.112 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:29.112 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787018+0000", 2024-09-18T10:15:29.113 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.2511999999999999e-05 2024-09-18T10:15:29.113 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.113 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.113 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:29.113 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787028+0000", 2024-09-18T10:15:29.113 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0709e-05 2024-09-18T10:15:29.113 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.113 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.113 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:29.113 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787034+0000", 2024-09-18T10:15:29.113 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.7769999999999999e-06 2024-09-18T10:15:29.113 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.114 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.114 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:29.114 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.295095+0000", 2024-09-18T10:15:29.114 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.50806109099999996 2024-09-18T10:15:29.114 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.114 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.114 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:29.114 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.295221+0000", 2024-09-18T10:15:29.114 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00012563599999999999 2024-09-18T10:15:29.114 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.114 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.114 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:15:29.114 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.295453+0000", 2024-09-18T10:15:29.114 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000232387 2024-09-18T10:15:29.115 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.115 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.115 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:15:29.115 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.301373+0000", 2024-09-18T10:15:29.115 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0059202279999999996 2024-09-18T10:15:29.115 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.115 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.115 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:15:29.115 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.498044+0000", 2024-09-18T10:15:29.115 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.19667104699999999 2024-09-18T10:15:29.115 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.115 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.115 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:15:29.115 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.498133+0000", 2024-09-18T10:15:29.116 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.9044999999999999e-05 2024-09-18T10:15:29.116 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.116 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.116 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:29.116 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.498237+0000", 2024-09-18T10:15:29.116 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00010385100000000001 2024-09-18T10:15:29.116 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:29.116 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:29.116 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:29.116 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.116 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.116 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T10:15:29.116 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:36.787042+0000", 2024-09-18T10:15:29.117 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 52.302203738000003, 2024-09-18T10:15:29.117 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.71152325100000002, 2024-09-18T10:15:29.117 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:29.117 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:29.117 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:29.117 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:15:29.117 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:15:29.117 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 73 2024-09-18T10:15:29.117 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.117 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:29.118 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.118 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:29.118 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787042+0000", 2024-09-18T10:15:29.118 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:29.118 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.118 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.118 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:29.118 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787042+0000", 2024-09-18T10:15:29.118 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:29.118 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.118 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.118 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:29.118 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787042+0000", 2024-09-18T10:15:29.119 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:15:29.119 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.119 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.119 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:29.119 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787048+0000", 2024-09-18T10:15:29.119 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.8900000000000001e-06 2024-09-18T10:15:29.119 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.119 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.119 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:29.119 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787057+0000", 2024-09-18T10:15:29.119 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.4430000000000003e-06 2024-09-18T10:15:29.119 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.119 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.119 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:29.120 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787063+0000", 2024-09-18T10:15:29.120 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.6459999999999998e-06 2024-09-18T10:15:29.120 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.120 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.120 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:29.120 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.297548+0000", 2024-09-18T10:15:29.120 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.51048589499999997 2024-09-18T10:15:29.120 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.120 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.120 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:29.120 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.297668+0000", 2024-09-18T10:15:29.120 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00011917 2024-09-18T10:15:29.120 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.120 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.121 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:15:29.121 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.297891+0000", 2024-09-18T10:15:29.121 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00022301999999999999 2024-09-18T10:15:29.121 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.121 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.121 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:15:29.121 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.301383+0000", 2024-09-18T10:15:29.121 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.003492762 2024-09-18T10:15:29.121 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.121 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.121 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:15:29.121 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.498374+0000", 2024-09-18T10:15:29.121 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.19699053899999999 2024-09-18T10:15:29.122 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.122 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.122 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:15:29.122 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.498506+0000", 2024-09-18T10:15:29.122 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00013246899999999999 2024-09-18T10:15:29.122 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.122 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.122 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:29.122 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.498566+0000", 2024-09-18T10:15:29.122 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.9274000000000002e-05 2024-09-18T10:15:29.122 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:29.122 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:29.122 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:29.122 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.123 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.123 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T10:15:29.123 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:36.787128+0000", 2024-09-18T10:15:29.123 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 52.30211843, 2024-09-18T10:15:29.123 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.71169541800000002, 2024-09-18T10:15:29.123 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:29.123 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:29.123 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:29.123 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:15:29.123 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:15:29.123 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 74 2024-09-18T10:15:29.123 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.123 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:29.123 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.124 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:29.124 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787128+0000", 2024-09-18T10:15:29.124 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:29.124 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.124 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.124 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:29.124 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787128+0000", 2024-09-18T10:15:29.124 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:29.124 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.124 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.124 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:29.124 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787127+0000", 2024-09-18T10:15:29.124 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:15:29.124 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.125 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.125 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:29.125 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787133+0000", 2024-09-18T10:15:29.125 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.6930000000000004e-06 2024-09-18T10:15:29.125 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.125 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.125 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:29.125 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787142+0000", 2024-09-18T10:15:29.125 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.1959999999999997e-06 2024-09-18T10:15:29.125 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.125 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.125 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:29.125 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787150+0000", 2024-09-18T10:15:29.125 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.6399999999999997e-06 2024-09-18T10:15:29.126 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.126 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.126 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:29.126 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.298925+0000", 2024-09-18T10:15:29.126 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.51177523599999997 2024-09-18T10:15:29.126 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.126 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.126 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:29.126 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.299064+0000", 2024-09-18T10:15:29.126 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000139622 2024-09-18T10:15:29.126 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.126 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.126 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:15:29.127 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.299287+0000", 2024-09-18T10:15:29.127 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00022309700000000001 2024-09-18T10:15:29.127 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.127 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.127 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:15:29.127 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.301389+0000", 2024-09-18T10:15:29.127 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0021018759999999999 2024-09-18T10:15:29.127 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.127 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.127 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:15:29.127 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.498648+0000", 2024-09-18T10:15:29.127 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.19725873999999999 2024-09-18T10:15:29.127 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.128 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.128 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:15:29.128 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.498731+0000", 2024-09-18T10:15:29.128 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.2429000000000006e-05 2024-09-18T10:15:29.128 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.128 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.128 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:29.128 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.498823+0000", 2024-09-18T10:15:29.128 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.2655e-05 2024-09-18T10:15:29.128 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:29.128 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:29.128 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:29.128 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.129 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.129 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T10:15:29.129 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:36.787157+0000", 2024-09-18T10:15:29.129 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 52.302089205000001, 2024-09-18T10:15:29.129 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.81204889999999996, 2024-09-18T10:15:29.129 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:29.129 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:29.129 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:29.129 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:15:29.129 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:15:29.129 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 75 2024-09-18T10:15:29.129 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.129 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:29.129 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.130 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:29.130 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787157+0000", 2024-09-18T10:15:29.130 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:29.130 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.130 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.130 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:29.130 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787157+0000", 2024-09-18T10:15:29.130 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:29.130 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.130 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.130 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:29.130 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787156+0000", 2024-09-18T10:15:29.130 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:15:29.130 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.131 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.131 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:29.131 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787236+0000", 2024-09-18T10:15:29.131 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.0175999999999999e-05 2024-09-18T10:15:29.131 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.131 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.131 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:29.131 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787247+0000", 2024-09-18T10:15:29.131 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0448999999999999e-05 2024-09-18T10:15:29.131 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.131 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.131 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:29.131 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787253+0000", 2024-09-18T10:15:29.132 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.2469999999999997e-06 2024-09-18T10:15:29.132 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.132 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.132 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:29.132 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.299974+0000", 2024-09-18T10:15:29.132 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.51272114499999999 2024-09-18T10:15:29.132 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.132 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.132 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:29.132 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.300115+0000", 2024-09-18T10:15:29.132 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000141452 2024-09-18T10:15:29.132 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.132 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.132 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:15:29.133 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.300349+0000", 2024-09-18T10:15:29.133 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00023363999999999999 2024-09-18T10:15:29.133 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.133 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.133 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:15:29.133 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.301947+0000", 2024-09-18T10:15:29.133 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0015976510000000001 2024-09-18T10:15:29.133 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.133 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.133 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:15:29.133 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.599027+0000", 2024-09-18T10:15:29.133 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.29708051600000002 2024-09-18T10:15:29.133 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.134 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.134 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:15:29.134 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.599106+0000", 2024-09-18T10:15:29.134 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.9033e-05 2024-09-18T10:15:29.134 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.134 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.134 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:29.134 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.599206+0000", 2024-09-18T10:15:29.134 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.9561000000000004e-05 2024-09-18T10:15:29.134 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:29.134 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:29.134 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:29.134 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.134 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.135 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T10:15:29.135 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:36.787261+0000", 2024-09-18T10:15:29.135 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 52.301985553999998, 2024-09-18T10:15:29.135 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.81223649499999995, 2024-09-18T10:15:29.135 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:29.135 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:29.135 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:29.135 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:15:29.135 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:15:29.135 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 76 2024-09-18T10:15:29.135 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.135 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:29.135 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.135 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:29.136 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787261+0000", 2024-09-18T10:15:29.136 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:29.136 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.136 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.136 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:29.136 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787261+0000", 2024-09-18T10:15:29.136 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:29.136 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.136 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.136 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:29.136 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787260+0000", 2024-09-18T10:15:29.136 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:15:29.136 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.136 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.137 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:29.137 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787265+0000", 2024-09-18T10:15:29.137 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1590000000000001e-06 2024-09-18T10:15:29.137 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.137 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.137 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:29.137 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787273+0000", 2024-09-18T10:15:29.137 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.4100000000000008e-06 2024-09-18T10:15:29.137 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.137 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.137 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:29.137 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787279+0000", 2024-09-18T10:15:29.137 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.3630000000000002e-06 2024-09-18T10:15:29.137 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.138 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.138 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:29.138 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.496955+0000", 2024-09-18T10:15:29.138 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.70967585099999997 2024-09-18T10:15:29.138 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.138 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.138 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:29.138 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.497107+0000", 2024-09-18T10:15:29.138 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000152044 2024-09-18T10:15:29.138 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.138 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.138 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:15:29.138 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.497345+0000", 2024-09-18T10:15:29.138 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000238693 2024-09-18T10:15:29.139 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.139 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.139 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:15:29.139 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.599248+0000", 2024-09-18T10:15:29.139 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.101902836 2024-09-18T10:15:29.139 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.139 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.139 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:15:29.139 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.599329+0000", 2024-09-18T10:15:29.139 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.1064999999999997e-05 2024-09-18T10:15:29.139 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.139 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.139 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:15:29.139 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.599441+0000", 2024-09-18T10:15:29.139 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000111797 2024-09-18T10:15:29.140 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.140 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.140 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:29.140 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.599497+0000", 2024-09-18T10:15:29.140 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.6119999999999998e-05 2024-09-18T10:15:29.140 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:29.140 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:29.140 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:29.140 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.140 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.140 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:93 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T10:15:29.140 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:38.616472+0000", 2024-09-18T10:15:29.140 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 50.472774270999999, 2024-09-18T10:15:29.141 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.60940747100000003, 2024-09-18T10:15:29.141 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:29.141 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:29.141 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:29.141 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:15:29.141 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:15:29.141 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 93 2024-09-18T10:15:29.141 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.141 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:29.141 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.141 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:29.141 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.616472+0000", 2024-09-18T10:15:29.141 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:29.141 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.142 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.142 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:29.142 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.616472+0000", 2024-09-18T10:15:29.142 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:29.142 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.142 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.142 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:29.142 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.616470+0000", 2024-09-18T10:15:29.142 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:29.142 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.142 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.142 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:29.142 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.616644+0000", 2024-09-18T10:15:29.142 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00017372000000000001 2024-09-18T10:15:29.143 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.143 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.143 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:29.143 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.616695+0000", 2024-09-18T10:15:29.143 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.0991e-05 2024-09-18T10:15:29.143 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.143 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.143 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:29.143 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.616709+0000", 2024-09-18T10:15:29.143 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4008999999999999e-05 2024-09-18T10:15:29.143 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.143 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.143 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:29.143 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.720960+0000", 2024-09-18T10:15:29.144 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.104250624 2024-09-18T10:15:29.144 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.144 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.144 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:29.144 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.721068+0000", 2024-09-18T10:15:29.144 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00010814700000000001 2024-09-18T10:15:29.144 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.144 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.144 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:15:29.144 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.721270+0000", 2024-09-18T10:15:29.144 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00020184200000000001 2024-09-18T10:15:29.144 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.144 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.145 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:15:29.145 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.024691+0000", 2024-09-18T10:15:29.145 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.30342105600000002 2024-09-18T10:15:29.145 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.145 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.145 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:15:29.145 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.225725+0000", 2024-09-18T10:15:29.145 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20103395099999999 2024-09-18T10:15:29.145 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.145 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.145 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:15:29.145 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.225816+0000", 2024-09-18T10:15:29.145 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.1928999999999993e-05 2024-09-18T10:15:29.145 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.146 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.146 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:29.146 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.225879+0000", 2024-09-18T10:15:29.146 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.2918999999999998e-05 2024-09-18T10:15:29.146 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:29.146 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:29.146 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:29.146 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.146 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.146 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:94 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T10:15:29.146 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:38.616729+0000", 2024-09-18T10:15:29.146 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 50.472517246000002, 2024-09-18T10:15:29.146 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.60943170999999996, 2024-09-18T10:15:29.146 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:29.147 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:29.147 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:29.147 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:15:29.147 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:15:29.147 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 94 2024-09-18T10:15:29.147 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.147 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:29.147 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.147 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:29.147 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.616729+0000", 2024-09-18T10:15:29.147 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:29.147 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.147 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.147 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:29.148 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.616729+0000", 2024-09-18T10:15:29.148 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:29.148 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.148 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.148 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:29.148 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.616727+0000", 2024-09-18T10:15:29.148 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:29.148 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.148 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.148 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:29.148 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.616738+0000", 2024-09-18T10:15:29.148 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0776e-05 2024-09-18T10:15:29.148 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.148 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.149 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:29.149 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.616756+0000", 2024-09-18T10:15:29.149 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8079000000000001e-05 2024-09-18T10:15:29.149 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.149 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.149 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:29.149 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.716882+0000", 2024-09-18T10:15:29.149 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10012612999999999 2024-09-18T10:15:29.149 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.149 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.149 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:29.149 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.822315+0000", 2024-09-18T10:15:29.149 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.105433162 2024-09-18T10:15:29.150 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.150 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.150 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:29.150 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.822453+0000", 2024-09-18T10:15:29.150 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000137396 2024-09-18T10:15:29.150 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.150 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.150 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:15:29.150 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.822674+0000", 2024-09-18T10:15:29.150 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00022154400000000001 2024-09-18T10:15:29.150 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.150 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.150 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:15:29.150 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.024888+0000", 2024-09-18T10:15:29.151 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20221397199999999 2024-09-18T10:15:29.151 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.151 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.151 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:15:29.151 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.226011+0000", 2024-09-18T10:15:29.151 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20112316299999999 2024-09-18T10:15:29.151 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.151 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.151 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:15:29.151 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.226094+0000", 2024-09-18T10:15:29.151 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.2155000000000002e-05 2024-09-18T10:15:29.151 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.151 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.151 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:29.152 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.226161+0000", 2024-09-18T10:15:29.152 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.7138999999999995e-05 2024-09-18T10:15:29.152 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:29.152 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:29.152 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:29.152 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.152 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.152 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:95 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T10:15:29.152 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:38.716909+0000", 2024-09-18T10:15:29.152 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 50.372337532000003, 2024-09-18T10:15:29.152 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.60960763600000001, 2024-09-18T10:15:29.152 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:29.152 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:29.152 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:29.153 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:15:29.153 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:15:29.153 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 95 2024-09-18T10:15:29.153 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.153 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:29.153 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.153 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:29.153 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.716909+0000", 2024-09-18T10:15:29.153 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:29.153 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.153 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.153 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:29.153 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.716909+0000", 2024-09-18T10:15:29.153 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:29.154 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.154 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.154 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:29.154 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.716907+0000", 2024-09-18T10:15:29.154 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:29.154 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.154 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.154 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:29.154 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.716921+0000", 2024-09-18T10:15:29.154 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4572e-05 2024-09-18T10:15:29.154 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.154 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.154 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:29.155 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.716939+0000", 2024-09-18T10:15:29.155 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7149000000000001e-05 2024-09-18T10:15:29.155 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.155 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.155 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:29.155 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.716948+0000", 2024-09-18T10:15:29.155 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.7000000000000003e-06 2024-09-18T10:15:29.156 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.156 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.156 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:29.156 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.923430+0000", 2024-09-18T10:15:29.156 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20648133299999999 2024-09-18T10:15:29.156 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.156 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.156 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:29.156 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.923566+0000", 2024-09-18T10:15:29.156 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00013657599999999999 2024-09-18T10:15:29.156 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.156 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.156 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:15:29.156 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.923782+0000", 2024-09-18T10:15:29.157 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00021599400000000001 2024-09-18T10:15:29.157 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.157 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.157 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:15:29.157 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.024899+0000", 2024-09-18T10:15:29.157 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10111666800000001 2024-09-18T10:15:29.157 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.157 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.157 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:15:29.157 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.326379+0000", 2024-09-18T10:15:29.158 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.30148048799999999 2024-09-18T10:15:29.158 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.158 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.158 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:15:29.158 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.326454+0000", 2024-09-18T10:15:29.158 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.4988999999999996e-05 2024-09-18T10:15:29.158 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.158 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.158 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:29.158 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.326516+0000", 2024-09-18T10:15:29.159 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.1902999999999993e-05 2024-09-18T10:15:29.159 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:29.159 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:29.159 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:29.159 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.159 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.159 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:96 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T10:15:29.159 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:38.716972+0000", 2024-09-18T10:15:29.159 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 50.372274542, 2024-09-18T10:15:29.159 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.91539460800000005, 2024-09-18T10:15:29.159 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:29.159 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:29.159 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:29.160 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:15:29.160 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:15:29.160 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 96 2024-09-18T10:15:29.160 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.160 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:29.160 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.160 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:29.160 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.716972+0000", 2024-09-18T10:15:29.160 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:29.160 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.160 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.160 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:29.160 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.716972+0000", 2024-09-18T10:15:29.160 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:29.161 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.161 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.161 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:29.161 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.716971+0000", 2024-09-18T10:15:29.161 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:15:29.161 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.161 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.161 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:29.161 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717061+0000", 2024-09-18T10:15:29.161 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.0744999999999999e-05 2024-09-18T10:15:29.161 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.161 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.161 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:29.161 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717073+0000", 2024-09-18T10:15:29.162 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1943000000000001e-05 2024-09-18T10:15:29.162 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.162 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.162 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:29.162 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717080+0000", 2024-09-18T10:15:29.162 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0060000000000003e-06 2024-09-18T10:15:29.162 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.162 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.162 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:29.162 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.426708+0000", 2024-09-18T10:15:29.162 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.70962730399999996 2024-09-18T10:15:29.162 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.162 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.163 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:29.163 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.426854+0000", 2024-09-18T10:15:29.163 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00014654199999999999 2024-09-18T10:15:29.163 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.163 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.163 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:15:29.163 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.427106+0000", 2024-09-18T10:15:29.163 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00025226600000000001 2024-09-18T10:15:29.163 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.163 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.163 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:15:29.163 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.433087+0000", 2024-09-18T10:15:29.163 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0059804860000000001 2024-09-18T10:15:29.163 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.164 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.164 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:15:29.164 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.632096+0000", 2024-09-18T10:15:29.164 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.19900922099999999 2024-09-18T10:15:29.164 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.164 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.164 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:15:29.164 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.632257+0000", 2024-09-18T10:15:29.164 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00016058399999999999 2024-09-18T10:15:29.164 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.164 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.164 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:29.164 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.632366+0000", 2024-09-18T10:15:29.165 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000109601 2024-09-18T10:15:29.165 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:29.165 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:29.165 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:29.165 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.165 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.165 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:97 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T10:15:29.165 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:38.717093+0000", 2024-09-18T10:15:29.165 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 50.372152732000004, 2024-09-18T10:15:29.165 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.91561460900000002, 2024-09-18T10:15:29.165 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:29.165 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:29.165 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:29.165 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:15:29.166 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:15:29.166 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 97 2024-09-18T10:15:29.166 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.166 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:29.166 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.166 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:29.166 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717093+0000", 2024-09-18T10:15:29.166 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:29.166 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.166 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.166 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:29.166 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717093+0000", 2024-09-18T10:15:29.166 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:29.166 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.167 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.167 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:29.167 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717092+0000", 2024-09-18T10:15:29.167 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:15:29.167 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.167 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.167 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:29.167 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717099+0000", 2024-09-18T10:15:29.167 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.246e-06 2024-09-18T10:15:29.167 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.167 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.167 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:29.167 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717109+0000", 2024-09-18T10:15:29.168 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0499999999999999e-05 2024-09-18T10:15:29.168 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.168 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.168 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:29.168 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717115+0000", 2024-09-18T10:15:29.168 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.7690000000000002e-06 2024-09-18T10:15:29.168 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.168 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.168 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:29.168 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.428509+0000", 2024-09-18T10:15:29.168 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.71139444299999999 2024-09-18T10:15:29.168 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.168 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.168 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:29.169 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.428621+0000", 2024-09-18T10:15:29.169 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00011156 2024-09-18T10:15:29.169 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.169 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.169 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:15:29.169 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.428805+0000", 2024-09-18T10:15:29.169 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000183597 2024-09-18T10:15:29.169 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.169 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.169 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:15:29.169 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.433106+0000", 2024-09-18T10:15:29.169 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0043015019999999996 2024-09-18T10:15:29.169 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.170 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.170 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:15:29.170 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.632474+0000", 2024-09-18T10:15:29.170 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.199368092 2024-09-18T10:15:29.170 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.170 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.170 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:15:29.170 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.632605+0000", 2024-09-18T10:15:29.170 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00013110899999999999 2024-09-18T10:15:29.170 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.170 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.170 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:29.170 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.632708+0000", 2024-09-18T10:15:29.170 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000102748 2024-09-18T10:15:29.171 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:29.171 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:29.171 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:29.171 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.171 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.171 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:98 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T10:15:29.171 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:38.717126+0000", 2024-09-18T10:15:29.171 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 50.372120469999999, 2024-09-18T10:15:29.171 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.015084141, 2024-09-18T10:15:29.171 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:29.171 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:29.171 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:29.171 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:15:29.172 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:15:29.172 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 98 2024-09-18T10:15:29.172 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.172 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:29.172 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.172 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:29.172 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717126+0000", 2024-09-18T10:15:29.172 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:29.172 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.172 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.172 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:29.172 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717126+0000", 2024-09-18T10:15:29.172 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:29.172 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.173 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.173 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:29.173 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717125+0000", 2024-09-18T10:15:29.173 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:15:29.173 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.173 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.173 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:29.173 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717130+0000", 2024-09-18T10:15:29.173 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.4160000000000003e-06 2024-09-18T10:15:29.173 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.173 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.173 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:29.173 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717139+0000", 2024-09-18T10:15:29.173 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.6230000000000003e-06 2024-09-18T10:15:29.174 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.174 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.174 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:29.174 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717144+0000", 2024-09-18T10:15:29.174 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5330000000000002e-06 2024-09-18T10:15:29.174 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.174 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.174 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:29.174 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.429491+0000", 2024-09-18T10:15:29.174 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.71234629500000002 2024-09-18T10:15:29.174 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.174 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.174 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:29.174 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.429741+0000", 2024-09-18T10:15:29.175 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000250516 2024-09-18T10:15:29.175 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.175 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.175 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:15:29.175 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.429989+0000", 2024-09-18T10:15:29.175 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00024811599999999999 2024-09-18T10:15:29.175 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.175 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.175 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:15:29.175 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.435056+0000", 2024-09-18T10:15:29.175 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0050669670000000003 2024-09-18T10:15:29.175 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.175 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.176 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:15:29.176 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.732068+0000", 2024-09-18T10:15:29.176 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.29701180900000002 2024-09-18T10:15:29.176 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.176 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.176 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:15:29.176 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.732145+0000", 2024-09-18T10:15:29.176 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.7152999999999998e-05 2024-09-18T10:15:29.176 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.176 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.176 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:29.176 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.732210+0000", 2024-09-18T10:15:29.176 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.4503000000000002e-05 2024-09-18T10:15:29.176 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:29.177 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:29.177 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:29.177 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.177 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.177 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:99 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T10:15:29.177 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:38.717230+0000", 2024-09-18T10:15:29.177 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 50.372015910000002, 2024-09-18T10:15:29.177 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0157905819999999, 2024-09-18T10:15:29.177 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:29.177 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:29.177 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:29.177 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:15:29.177 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:15:29.178 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 99 2024-09-18T10:15:29.178 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.178 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:29.178 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.178 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:29.178 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717230+0000", 2024-09-18T10:15:29.178 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:29.178 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.178 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.178 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:29.178 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717230+0000", 2024-09-18T10:15:29.178 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:29.178 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.178 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.179 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:29.179 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717229+0000", 2024-09-18T10:15:29.179 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:15:29.179 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.179 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.179 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:29.179 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717318+0000", 2024-09-18T10:15:29.179 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.8197999999999998e-05 2024-09-18T10:15:29.179 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.179 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.179 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:29.179 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717342+0000", 2024-09-18T10:15:29.179 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4445000000000001e-05 2024-09-18T10:15:29.180 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.180 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.180 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:29.180 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717360+0000", 2024-09-18T10:15:29.180 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7782999999999999e-05 2024-09-18T10:15:29.180 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.180 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.180 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:29.180 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.430650+0000", 2024-09-18T10:15:29.180 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.71329020200000004 2024-09-18T10:15:29.180 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.180 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.180 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:29.181 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.430805+0000", 2024-09-18T10:15:29.181 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00015474900000000001 2024-09-18T10:15:29.181 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.181 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.181 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:15:29.181 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.431021+0000", 2024-09-18T10:15:29.181 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000215911 2024-09-18T10:15:29.181 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.181 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.181 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:15:29.181 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.435099+0000", 2024-09-18T10:15:29.181 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0040787439999999996 2024-09-18T10:15:29.181 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.181 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.181 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:15:29.182 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.732758+0000", 2024-09-18T10:15:29.182 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.297658637 2024-09-18T10:15:29.182 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.182 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.182 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:15:29.182 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.732909+0000", 2024-09-18T10:15:29.182 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00015069899999999999 2024-09-18T10:15:29.182 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.182 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.182 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:29.182 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.733021+0000", 2024-09-18T10:15:29.182 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00011207 2024-09-18T10:15:29.182 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:29.182 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:29.183 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:29.183 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.183 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.183 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:100 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T10:15:29.183 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:38.717370+0000", 2024-09-18T10:15:29.183 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 50.371876348000001, 2024-09-18T10:15:29.183 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1158324850000001, 2024-09-18T10:15:29.183 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:29.183 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:29.183 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:29.183 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:15:29.183 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:15:29.183 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 100 2024-09-18T10:15:29.184 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.184 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:29.184 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.184 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:29.184 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717370+0000", 2024-09-18T10:15:29.184 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:29.184 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.184 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.184 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:29.184 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717370+0000", 2024-09-18T10:15:29.184 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:29.184 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.184 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.185 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:29.185 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717369+0000", 2024-09-18T10:15:29.185 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:15:29.185 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.185 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.185 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:29.185 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717378+0000", 2024-09-18T10:15:29.185 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.3559999999999999e-06 2024-09-18T10:15:29.185 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.185 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.185 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:29.185 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717398+0000", 2024-09-18T10:15:29.185 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.9962e-05 2024-09-18T10:15:29.185 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.186 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.186 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:29.186 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717407+0000", 2024-09-18T10:15:29.186 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.2699999999999993e-06 2024-09-18T10:15:29.186 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.186 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.186 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:29.186 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.432184+0000", 2024-09-18T10:15:29.186 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.71477630299999995 2024-09-18T10:15:29.186 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.186 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.186 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:29.186 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.432316+0000", 2024-09-18T10:15:29.186 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000132082 2024-09-18T10:15:29.187 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.187 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.187 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:15:29.187 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.432526+0000", 2024-09-18T10:15:29.187 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000210659 2024-09-18T10:15:29.187 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.187 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.187 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:15:29.187 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.435651+0000", 2024-09-18T10:15:29.187 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.003124169 2024-09-18T10:15:29.187 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.187 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.187 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:15:29.188 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.832931+0000", 2024-09-18T10:15:29.188 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.39728084899999999 2024-09-18T10:15:29.188 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.188 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.188 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:15:29.188 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.833058+0000", 2024-09-18T10:15:29.188 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000126969 2024-09-18T10:15:29.188 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.188 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.188 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:29.188 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.833202+0000", 2024-09-18T10:15:29.188 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000143922 2024-09-18T10:15:29.188 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:29.189 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:29.189 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:29.189 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.189 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.189 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:132 3.8s0 3:1162852a:::smithi18925896-44:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:29.189 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:50.819929+0000", 2024-09-18T10:15:29.189 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 38.269317156, 2024-09-18T10:15:29.189 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.59839508200000002, 2024-09-18T10:15:29.189 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:29.189 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "started", 2024-09-18T10:15:29.189 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:29.189 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:15:29.189 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:29.190 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 132 2024-09-18T10:15:29.190 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.190 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:29.190 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.190 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:29.190 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.819929+0000", 2024-09-18T10:15:29.190 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:29.190 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.190 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.190 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:29.190 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.819929+0000", 2024-09-18T10:15:29.190 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:29.190 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.190 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.191 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:29.191 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.819927+0000", 2024-09-18T10:15:29.191 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:29.191 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.191 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.191 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:29.191 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.819938+0000", 2024-09-18T10:15:29.191 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0502999999999999e-05 2024-09-18T10:15:29.191 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.191 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.191 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:29.191 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.819962+0000", 2024-09-18T10:15:29.191 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4241999999999999e-05 2024-09-18T10:15:29.191 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.192 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.192 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:29.192 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.819973+0000", 2024-09-18T10:15:29.192 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0730000000000001e-05 2024-09-18T10:15:29.192 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.192 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.192 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:29.192 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.908665+0000", 2024-09-18T10:15:29.192 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.088691995999999995 2024-09-18T10:15:29.192 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.192 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.192 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for rw locks", 2024-09-18T10:15:29.192 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.908807+0000", 2024-09-18T10:15:29.192 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00014202400000000001 2024-09-18T10:15:29.193 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.193 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.193 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:29.193 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.214385+0000", 2024-09-18T10:15:29.193 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.30557793100000002 2024-09-18T10:15:29.193 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.193 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.193 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:29.193 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.214515+0000", 2024-09-18T10:15:29.193 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00013058699999999999 2024-09-18T10:15:29.193 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.193 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.193 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:29.194 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.418324+0000", 2024-09-18T10:15:29.194 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20380895600000001 2024-09-18T10:15:29.194 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:29.194 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:29.194 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:29.194 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.194 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.194 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:141 3.8s0 3:1c02e63c:::smithi18925896-47:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:29.194 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:51.119430+0000", 2024-09-18T10:15:29.194 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 37.969816143999999, 2024-09-18T10:15:29.194 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.60306919000000003, 2024-09-18T10:15:29.195 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:29.195 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "started", 2024-09-18T10:15:29.195 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:29.195 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:15:29.195 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:29.195 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 141 2024-09-18T10:15:29.195 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.195 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:29.195 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.195 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:29.195 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.119430+0000", 2024-09-18T10:15:29.195 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:29.195 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.196 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.196 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:29.196 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.119430+0000", 2024-09-18T10:15:29.196 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:29.196 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.196 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.196 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:29.196 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.119430+0000", 2024-09-18T10:15:29.196 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:29.196 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.196 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.196 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:29.196 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.119432+0000", 2024-09-18T10:15:29.197 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6240000000000002e-06 2024-09-18T10:15:29.197 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.197 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.197 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:29.197 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.119439+0000", 2024-09-18T10:15:29.197 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.8329999999999996e-06 2024-09-18T10:15:29.197 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.197 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.197 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:29.197 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.119444+0000", 2024-09-18T10:15:29.197 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.485e-06 2024-09-18T10:15:29.197 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.197 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.197 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:29.197 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.620626+0000", 2024-09-18T10:15:29.198 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.50118172299999997 2024-09-18T10:15:29.198 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.198 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.198 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:29.198 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.620781+0000", 2024-09-18T10:15:29.198 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00015495 2024-09-18T10:15:29.198 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.198 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.198 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:29.198 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.722499+0000", 2024-09-18T10:15:29.198 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.101718085 2024-09-18T10:15:29.198 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:29.198 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:29.199 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:29.199 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.199 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.199 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:154 3.8s0 3:1d386431:::smithi18925896-2:head [copy-from ver 4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:29.199 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:51.211083+0000", 2024-09-18T10:15:29.199 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 37.878162918999998, 2024-09-18T10:15:29.199 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.73193688499999998, 2024-09-18T10:15:29.199 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:29.199 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:29.199 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:29.199 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:15:29.199 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:29.199 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 154 2024-09-18T10:15:29.200 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.200 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:29.200 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.200 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:29.200 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.211083+0000", 2024-09-18T10:15:29.200 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:29.200 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.200 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.200 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:29.200 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.211083+0000", 2024-09-18T10:15:29.200 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:29.200 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.200 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.200 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:29.201 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.211080+0000", 2024-09-18T10:15:29.201 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999971 2024-09-18T10:15:29.201 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.201 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.201 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:29.201 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.211096+0000", 2024-09-18T10:15:29.201 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.5699999999999999e-05 2024-09-18T10:15:29.201 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.201 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.201 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:29.201 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.211134+0000", 2024-09-18T10:15:29.201 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.7694000000000002e-05 2024-09-18T10:15:29.201 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.202 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.202 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:29.202 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.211149+0000", 2024-09-18T10:15:29.202 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4746000000000001e-05 2024-09-18T10:15:29.202 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.202 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.202 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:29.202 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.621341+0000", 2024-09-18T10:15:29.202 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.41019203799999998 2024-09-18T10:15:29.202 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.202 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.202 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:29.202 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.621505+0000", 2024-09-18T10:15:29.202 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00016443299999999999 2024-09-18T10:15:29.203 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.203 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.203 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:29.203 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.840064+0000", 2024-09-18T10:15:29.203 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.21855904700000001 2024-09-18T10:15:29.203 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.203 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.203 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:29.203 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.842430+0000", 2024-09-18T10:15:29.203 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0023664100000000002 2024-09-18T10:15:29.203 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.203 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.203 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:15:29.203 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.942864+0000", 2024-09-18T10:15:29.204 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10043373 2024-09-18T10:15:29.204 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.204 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.204 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:29.204 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.943020+0000", 2024-09-18T10:15:29.204 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00015598000000000001 2024-09-18T10:15:29.204 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:29.204 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:29.204 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:29.204 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.204 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.204 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:155 3.8s0 3:1d386431:::smithi18925896-2:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:29.204 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:51.211164+0000", 2024-09-18T10:15:29.205 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 37.878082548999998, 2024-09-18T10:15:29.205 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.630957349, 2024-09-18T10:15:29.205 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:29.205 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "started", 2024-09-18T10:15:29.205 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:29.205 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:15:29.205 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:29.205 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 155 2024-09-18T10:15:29.205 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.205 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:29.205 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.205 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:29.206 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.211164+0000", 2024-09-18T10:15:29.206 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:29.206 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.206 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.206 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:29.206 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.211164+0000", 2024-09-18T10:15:29.206 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:29.206 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.206 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.206 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:29.206 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.211163+0000", 2024-09-18T10:15:29.206 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:15:29.206 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.207 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.207 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:29.207 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.211167+0000", 2024-09-18T10:15:29.207 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.6299999999999997e-06 2024-09-18T10:15:29.207 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.207 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.207 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:29.207 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.211178+0000", 2024-09-18T10:15:29.207 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0798999999999999e-05 2024-09-18T10:15:29.207 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.207 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.207 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:29.207 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.211186+0000", 2024-09-18T10:15:29.207 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.5669999999999997e-06 2024-09-18T10:15:29.208 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.208 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.208 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:29.208 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.623024+0000", 2024-09-18T10:15:29.208 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.411838433 2024-09-18T10:15:29.208 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.208 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.208 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for blocked object", 2024-09-18T10:15:29.208 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.623104+0000", 2024-09-18T10:15:29.208 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.0136000000000007e-05 2024-09-18T10:15:29.208 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.208 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.208 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:29.208 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.841600+0000", 2024-09-18T10:15:29.209 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.21849595399999999 2024-09-18T10:15:29.209 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.209 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.209 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:29.209 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.841868+0000", 2024-09-18T10:15:29.209 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00026825800000000001 2024-09-18T10:15:29.209 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.209 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.209 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:29.209 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.842121+0000", 2024-09-18T10:15:29.209 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00025266500000000001 2024-09-18T10:15:29.209 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:29.209 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:29.209 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:29.210 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.210 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.210 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:619 3.8s0 3:1162852a:::smithi18925896-44:head [copy-from ver 14] snapc 21=[20,1f,1e,1a,16,12,11,10] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-18T10:15:29.210 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:27.977392+0000", 2024-09-18T10:15:29.210 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 1.1118542950000001, 2024-09-18T10:15:29.210 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.71139311599999999, 2024-09-18T10:15:29.210 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:29.210 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:29.210 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:29.210 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:15:29.210 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:29.210 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 619 2024-09-18T10:15:29.210 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.210 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:29.211 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.211 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:29.211 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:27.977392+0000", 2024-09-18T10:15:29.211 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:29.211 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.211 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.211 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:29.211 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:27.977392+0000", 2024-09-18T10:15:29.211 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:29.211 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.211 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.211 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:29.211 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:27.977390+0000", 2024-09-18T10:15:29.211 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:29.211 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.212 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.212 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:29.212 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:27.977399+0000", 2024-09-18T10:15:29.212 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.1239999999999994e-06 2024-09-18T10:15:29.212 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.212 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.212 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:29.212 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:27.977422+0000", 2024-09-18T10:15:29.212 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.2734999999999998e-05 2024-09-18T10:15:29.212 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.212 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.212 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:29.212 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.077547+0000", 2024-09-18T10:15:29.213 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100124728 2024-09-18T10:15:29.213 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.213 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.213 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:29.213 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.077713+0000", 2024-09-18T10:15:29.213 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000166515 2024-09-18T10:15:29.213 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.213 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.213 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op must wait for map", 2024-09-18T10:15:29.213 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.077728+0000", 2024-09-18T10:15:29.213 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.5036e-05 2024-09-18T10:15:29.214 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.214 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.214 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:29.214 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.275819+0000", 2024-09-18T10:15:29.214 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.19809117600000001 2024-09-18T10:15:29.214 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.214 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.214 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:29.214 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.276220+0000", 2024-09-18T10:15:29.214 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000400426 2024-09-18T10:15:29.214 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.214 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.214 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:29.214 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.383764+0000", 2024-09-18T10:15:29.215 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.107543814 2024-09-18T10:15:29.215 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.215 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.215 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:29.215 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.486402+0000", 2024-09-18T10:15:29.215 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.102638484 2024-09-18T10:15:29.215 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.215 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.215 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:15:29.215 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.688521+0000", 2024-09-18T10:15:29.215 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.202118873 2024-09-18T10:15:29.215 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.215 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.215 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:29.215 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.688785+0000", 2024-09-18T10:15:29.216 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000264133 2024-09-18T10:15:29.216 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:29.216 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:29.216 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:29.216 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:29.216 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:29.216 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-09-18T10:15:29.218 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-09-18T10:15:29.218 INFO:tasks.ceph.osd.4.smithi029.stderr:2024-09-18T10:15:29.214+0000 7f550cf6e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22189) UID: 0 2024-09-18T10:15:29.319 INFO:tasks.ceph.osd.4.smithi029.stderr:2024-09-18T10:15:29.314+0000 7f550cf6e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22189) UID: 0 2024-09-18T10:15:29.329 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-09-18T10:15:29.329 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-09-18T10:15:29.329 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-09-18T10:15:29.329 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-09-18T10:15:29.338 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-09-18T10:15:29.422 INFO:tasks.ceph.osd.11.smithi189.stderr:2024-09-18T10:15:29.418+0000 7f718b60a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 22400) UID: 0 2024-09-18T10:15:29.448 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-09-18T10:15:29.448 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-09-18T10:15:29.448 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-09-18T10:15:29.448 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-09-18T10:15:29.448 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-09-18T10:15:29.457 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-09-18T10:15:29.461 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T10:15:29.523 INFO:tasks.ceph.osd.4.smithi029.stderr:2024-09-18T10:15:29.518+0000 7f550cf6e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22189) UID: 0 2024-09-18T10:15:29.560 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-09-18T10:15:29.560 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-09-18T10:15:29.560 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-09-18T10:15:29.560 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-09-18T10:15:29.560 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.560 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:189 3.3s0 3:ce8164ce:::smithi18925896-7:head [copy-from ver 6] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:29.560 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:52.694279+0000", 2024-09-18T10:15:29.560 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 36.859657576000004, 2024-09-18T10:15:29.561 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.51539283599999997, 2024-09-18T10:15:29.561 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:29.561 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:29.561 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:29.561 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:15:29.561 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:29.561 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 189 2024-09-18T10:15:29.561 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.561 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:29.561 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.561 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:29.561 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694279+0000", 2024-09-18T10:15:29.561 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:29.561 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.562 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.562 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:29.562 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694279+0000", 2024-09-18T10:15:29.562 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:29.562 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.562 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.562 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:29.562 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694277+0000", 2024-09-18T10:15:29.562 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999971 2024-09-18T10:15:29.562 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.562 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.562 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:29.562 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694290+0000", 2024-09-18T10:15:29.563 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3862e-05 2024-09-18T10:15:29.563 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.563 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.563 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:29.563 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694311+0000", 2024-09-18T10:15:29.563 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.0273999999999999e-05 2024-09-18T10:15:29.563 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.563 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.563 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:29.563 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694322+0000", 2024-09-18T10:15:29.563 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1722999999999999e-05 2024-09-18T10:15:29.563 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.563 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.563 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:29.564 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.794494+0000", 2024-09-18T10:15:29.564 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100171653 2024-09-18T10:15:29.564 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.564 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.564 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:29.564 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.794705+0000", 2024-09-18T10:15:29.564 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00021126000000000001 2024-09-18T10:15:29.564 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.564 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.564 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:29.564 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.096113+0000", 2024-09-18T10:15:29.564 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.30140739100000002 2024-09-18T10:15:29.564 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.565 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.565 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:29.565 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.098602+0000", 2024-09-18T10:15:29.565 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.002489593 2024-09-18T10:15:29.565 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.565 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.565 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:15:29.565 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.209538+0000", 2024-09-18T10:15:29.565 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.110935482 2024-09-18T10:15:29.565 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.565 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.565 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:29.565 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.209672+0000", 2024-09-18T10:15:29.565 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00013435399999999999 2024-09-18T10:15:29.566 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:29.566 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:29.566 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:29.566 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.566 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.566 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:190 3.3s0 3:ce8164ce:::smithi18925896-7:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:29.566 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:52.694343+0000", 2024-09-18T10:15:29.566 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 36.859593986, 2024-09-18T10:15:29.566 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.40407532499999999, 2024-09-18T10:15:29.566 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:29.566 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "started", 2024-09-18T10:15:29.566 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:29.566 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:15:29.566 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:29.567 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 190 2024-09-18T10:15:29.567 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.567 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:29.567 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.567 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:29.567 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694343+0000", 2024-09-18T10:15:29.567 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:29.567 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.567 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.567 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:29.567 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694343+0000", 2024-09-18T10:15:29.567 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:29.567 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.568 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.568 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:29.568 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694342+0000", 2024-09-18T10:15:29.568 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:15:29.568 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.568 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.568 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:29.568 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694349+0000", 2024-09-18T10:15:29.568 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.5969999999999997e-06 2024-09-18T10:15:29.568 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.568 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.568 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:29.568 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694367+0000", 2024-09-18T10:15:29.568 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8323000000000001e-05 2024-09-18T10:15:29.569 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.569 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.569 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:29.569 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694376+0000", 2024-09-18T10:15:29.569 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.9740000000000008e-06 2024-09-18T10:15:29.569 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.569 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.569 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:29.569 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.795028+0000", 2024-09-18T10:15:29.569 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10065176000000001 2024-09-18T10:15:29.569 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.569 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.569 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for blocked object", 2024-09-18T10:15:29.569 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.795171+0000", 2024-09-18T10:15:29.570 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00014321399999999999 2024-09-18T10:15:29.570 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.570 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.570 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:29.570 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.098069+0000", 2024-09-18T10:15:29.570 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.30289780700000002 2024-09-18T10:15:29.570 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.570 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.570 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:29.570 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.098247+0000", 2024-09-18T10:15:29.570 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00017831300000000001 2024-09-18T10:15:29.570 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.570 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.571 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:29.571 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.098418+0000", 2024-09-18T10:15:29.571 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00017122599999999999 2024-09-18T10:15:29.571 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:29.571 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:29.571 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:29.571 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.571 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.571 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:194 3.3s0 3:c9ce6440:::smithi18925896-24:head [copy-from ver 6] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:29.571 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:52.694388+0000", 2024-09-18T10:15:29.571 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 36.859548713000002, 2024-09-18T10:15:29.571 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.92118689200000003, 2024-09-18T10:15:29.571 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:29.571 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:29.572 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:29.572 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:15:29.572 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:29.572 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 194 2024-09-18T10:15:29.572 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.572 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:29.572 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.572 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:29.572 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694388+0000", 2024-09-18T10:15:29.572 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:29.572 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.572 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.572 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:29.573 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694388+0000", 2024-09-18T10:15:29.573 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:29.573 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.573 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.573 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:29.573 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694388+0000", 2024-09-18T10:15:29.573 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:15:29.573 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.573 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.573 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:29.573 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694393+0000", 2024-09-18T10:15:29.573 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.2299999999999999e-06 2024-09-18T10:15:29.573 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.574 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.574 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:29.574 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694411+0000", 2024-09-18T10:15:29.574 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7802999999999999e-05 2024-09-18T10:15:29.574 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.574 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.574 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:29.574 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694417+0000", 2024-09-18T10:15:29.574 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.4640000000000003e-06 2024-09-18T10:15:29.574 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.574 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.574 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:29.574 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.795289+0000", 2024-09-18T10:15:29.574 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100872288 2024-09-18T10:15:29.575 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.575 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.575 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:29.575 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.795548+0000", 2024-09-18T10:15:29.575 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00025869199999999998 2024-09-18T10:15:29.575 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.575 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.575 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:29.575 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.412215+0000", 2024-09-18T10:15:29.575 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.61666667100000006 2024-09-18T10:15:29.575 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.575 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.575 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:29.575 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.514838+0000", 2024-09-18T10:15:29.576 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10262323399999999 2024-09-18T10:15:29.576 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.576 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.576 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:15:29.576 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.615386+0000", 2024-09-18T10:15:29.576 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100548106 2024-09-18T10:15:29.576 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.576 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.576 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:29.576 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.615575+0000", 2024-09-18T10:15:29.576 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000189128 2024-09-18T10:15:29.576 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:29.576 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:29.577 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:29.577 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.577 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.577 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:195 3.3s0 3:c9ce6440:::smithi18925896-24:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:29.577 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:52.694424+0000", 2024-09-18T10:15:29.577 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 36.859512977000001, 2024-09-18T10:15:29.577 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.82009685499999996, 2024-09-18T10:15:29.577 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:29.577 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "started", 2024-09-18T10:15:29.577 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:29.577 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:15:29.577 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:29.577 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 195 2024-09-18T10:15:29.578 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.578 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:29.578 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.578 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:29.578 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694424+0000", 2024-09-18T10:15:29.578 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:29.578 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.578 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.578 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:29.578 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694424+0000", 2024-09-18T10:15:29.578 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:29.578 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.578 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.578 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:29.578 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694423+0000", 2024-09-18T10:15:29.579 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:29.579 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.579 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.579 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:29.579 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694427+0000", 2024-09-18T10:15:29.579 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2650000000000001e-06 2024-09-18T10:15:29.579 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.579 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.579 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:29.579 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694434+0000", 2024-09-18T10:15:29.579 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.5410000000000003e-06 2024-09-18T10:15:29.579 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.579 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.579 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:29.579 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694441+0000", 2024-09-18T10:15:29.580 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.187e-06 2024-09-18T10:15:29.580 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.580 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.580 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:29.580 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.795870+0000", 2024-09-18T10:15:29.580 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.101428617 2024-09-18T10:15:29.580 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.580 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.580 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for blocked object", 2024-09-18T10:15:29.580 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.796040+0000", 2024-09-18T10:15:29.580 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00017008 2024-09-18T10:15:29.580 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.580 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.580 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:29.580 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.514130+0000", 2024-09-18T10:15:29.581 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.71809022199999994 2024-09-18T10:15:29.581 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.581 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.581 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:29.581 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.514348+0000", 2024-09-18T10:15:29.581 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000217488 2024-09-18T10:15:29.581 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.581 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.581 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:29.581 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.514521+0000", 2024-09-18T10:15:29.581 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00017300900000000001 2024-09-18T10:15:29.581 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:29.581 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:29.581 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:29.582 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.582 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.582 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.bs2 41/32 ECSubWrite(tid=58, reqid=osd.4.0:58, at_version=41'14, trim_to=0'0, roll_forward_to=39'12))", 2024-09-18T10:15:29.582 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:56.829698+0000", 2024-09-18T10:15:29.582 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 32.724238882999998, 2024-09-18T10:15:29.582 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.71711602600000002, 2024-09-18T10:15:29.582 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:29.582 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:15:29.582 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:29.582 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.582 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:29.582 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.829698+0000", 2024-09-18T10:15:29.582 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:29.582 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.583 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.583 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:29.583 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.829698+0000", 2024-09-18T10:15:29.583 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:29.583 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.583 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.583 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:29.583 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.829697+0000", 2024-09-18T10:15:29.583 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:29.583 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.583 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.583 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:29.583 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.829733+0000", 2024-09-18T10:15:29.583 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.6217999999999998e-05 2024-09-18T10:15:29.583 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.584 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.584 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:29.584 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.829757+0000", 2024-09-18T10:15:29.584 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.3915000000000001e-05 2024-09-18T10:15:29.584 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.584 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.584 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:29.584 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.829769+0000", 2024-09-18T10:15:29.584 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1754e-05 2024-09-18T10:15:29.584 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.584 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.584 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:29.584 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.342965+0000", 2024-09-18T10:15:29.584 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.51319593799999996 2024-09-18T10:15:29.584 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.585 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.585 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:29.585 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.343020+0000", 2024-09-18T10:15:29.585 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5367e-05 2024-09-18T10:15:29.585 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.585 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.585 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:29.585 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.546772+0000", 2024-09-18T10:15:29.585 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.203751933 2024-09-18T10:15:29.585 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.585 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.585 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:29.585 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.546814+0000", 2024-09-18T10:15:29.586 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.2196000000000001e-05 2024-09-18T10:15:29.586 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:29.586 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:29.586 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:29.586 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.586 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.586 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:322 3.13s0 3:c9ce6440:::smithi18925896-24:4 [read 0~475136,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T10:15:29.586 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:01.795716+0000", 2024-09-18T10:15:29.586 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 27.758220797, 2024-09-18T10:15:29.586 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.30787060799999999, 2024-09-18T10:15:29.586 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:29.586 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "started", 2024-09-18T10:15:29.586 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:29.586 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:15:29.587 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:29.587 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 322 2024-09-18T10:15:29.587 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.587 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:29.587 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.587 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:29.587 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:01.795716+0000", 2024-09-18T10:15:29.587 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:29.587 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.587 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.587 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:29.587 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:01.795716+0000", 2024-09-18T10:15:29.587 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:29.587 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.588 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.588 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:29.588 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:01.795715+0000", 2024-09-18T10:15:29.588 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:15:29.588 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.588 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.588 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:29.588 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:01.795723+0000", 2024-09-18T10:15:29.588 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.5730000000000002e-06 2024-09-18T10:15:29.588 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.588 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.588 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:29.588 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:01.795737+0000", 2024-09-18T10:15:29.588 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4158999999999999e-05 2024-09-18T10:15:29.589 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.589 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.589 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:29.589 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:01.895874+0000", 2024-09-18T10:15:29.589 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100137485 2024-09-18T10:15:29.589 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.589 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.589 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:29.589 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:01.999648+0000", 2024-09-18T10:15:29.589 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10377353 2024-09-18T10:15:29.589 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.589 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.589 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:29.589 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:01.999827+0000", 2024-09-18T10:15:29.589 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0001794 2024-09-18T10:15:29.590 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.590 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.590 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:29.590 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:02.103587+0000", 2024-09-18T10:15:29.590 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.103759648 2024-09-18T10:15:29.590 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:29.590 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:29.590 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:29.590 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.590 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.590 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.19s2 45/40 ECSubWrite(tid=58, reqid=client.4632.0:297, at_version=0'0, trim_to=0'0, roll_forward_to=45'35))", 2024-09-18T10:15:29.590 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:01.915741+0000", 2024-09-18T10:15:29.590 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 27.638195903, 2024-09-18T10:15:29.590 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.28605987500000002, 2024-09-18T10:15:29.591 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:29.591 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:15:29.591 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:29.591 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.591 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:29.591 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:01.915741+0000", 2024-09-18T10:15:29.591 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:29.591 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.591 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.591 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:29.591 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:01.915741+0000", 2024-09-18T10:15:29.591 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:29.591 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.591 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.591 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:29.591 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:01.915739+0000", 2024-09-18T10:15:29.592 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:29.592 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.592 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.592 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:29.592 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:01.915763+0000", 2024-09-18T10:15:29.592 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.3938999999999999e-05 2024-09-18T10:15:29.592 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.592 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.592 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:29.592 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:01.915792+0000", 2024-09-18T10:15:29.592 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8232999999999999e-05 2024-09-18T10:15:29.592 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.592 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.592 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:29.593 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:01.915807+0000", 2024-09-18T10:15:29.593 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.5682e-05 2024-09-18T10:15:29.593 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.593 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.593 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:29.593 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:02.100734+0000", 2024-09-18T10:15:29.593 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.18492666999999999 2024-09-18T10:15:29.593 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.593 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.593 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:29.593 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:02.100766+0000", 2024-09-18T10:15:29.593 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2132000000000003e-05 2024-09-18T10:15:29.593 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.593 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.593 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:29.594 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:02.201732+0000", 2024-09-18T10:15:29.594 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10096622700000001 2024-09-18T10:15:29.594 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.594 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.594 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:29.594 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:02.201801+0000", 2024-09-18T10:15:29.594 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.8694000000000003e-05 2024-09-18T10:15:29.594 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:29.594 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:29.594 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:29.594 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.594 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.594 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:345 3.13s0 3:c9ce6440:::smithi18925896-24:head [read 0~475136,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-18T10:15:29.594 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:02.803095+0000", 2024-09-18T10:15:29.594 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 26.750841792999999, 2024-09-18T10:15:29.595 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.21197828799999999, 2024-09-18T10:15:29.595 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:29.595 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "started", 2024-09-18T10:15:29.595 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:29.595 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:15:29.595 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:29.595 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 345 2024-09-18T10:15:29.595 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.595 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:29.595 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.595 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:29.595 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:02.803095+0000", 2024-09-18T10:15:29.595 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:29.595 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.595 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.596 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:29.596 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:02.803095+0000", 2024-09-18T10:15:29.596 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:29.596 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.596 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.596 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:29.596 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:02.803094+0000", 2024-09-18T10:15:29.596 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:15:29.596 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.596 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.596 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:29.596 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:02.803104+0000", 2024-09-18T10:15:29.596 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.6430000000000007e-06 2024-09-18T10:15:29.597 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.597 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.597 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:29.597 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:02.803142+0000", 2024-09-18T10:15:29.597 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8591999999999999e-05 2024-09-18T10:15:29.597 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.597 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.597 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:29.597 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:02.803155+0000", 2024-09-18T10:15:29.597 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2466000000000001e-05 2024-09-18T10:15:29.597 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.597 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.597 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:29.597 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:02.803466+0000", 2024-09-18T10:15:29.598 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00031142499999999999 2024-09-18T10:15:29.598 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.598 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.598 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:29.598 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:02.803487+0000", 2024-09-18T10:15:29.598 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.0305999999999998e-05 2024-09-18T10:15:29.598 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.598 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.598 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:29.598 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:02.808581+0000", 2024-09-18T10:15:29.598 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0050947400000000004 2024-09-18T10:15:29.598 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.598 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.599 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:29.599 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:02.808678+0000", 2024-09-18T10:15:29.599 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.6797000000000002e-05 2024-09-18T10:15:29.599 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.599 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.599 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:29.599 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:03.015073+0000", 2024-09-18T10:15:29.599 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20639537599999999 2024-09-18T10:15:29.599 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:29.599 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:29.599 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:29.599 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.599 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.599 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDPGPushReply(3.17s0 67/64 [PushReplyOp(3:ee962131:::smithi18925896-29:10)])", 2024-09-18T10:15:29.600 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:23.671965+0000", 2024-09-18T10:15:29.600 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 5.8819724329999996, 2024-09-18T10:15:29.600 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.60673808500000004, 2024-09-18T10:15:29.600 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:29.600 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:15:29.600 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:29.600 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.600 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:29.600 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.671965+0000", 2024-09-18T10:15:29.600 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:29.601 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.601 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.601 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:29.601 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.671965+0000", 2024-09-18T10:15:29.601 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:29.601 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.601 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.601 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:29.601 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.671963+0000", 2024-09-18T10:15:29.601 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:29.601 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.601 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.601 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:29.602 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.671977+0000", 2024-09-18T10:15:29.602 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4093e-05 2024-09-18T10:15:29.602 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.602 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.602 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:29.602 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.672009+0000", 2024-09-18T10:15:29.602 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1939e-05 2024-09-18T10:15:29.602 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.602 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.602 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:29.602 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.772102+0000", 2024-09-18T10:15:29.602 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10009319 2024-09-18T10:15:29.602 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.602 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.603 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:29.603 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.278591+0000", 2024-09-18T10:15:29.603 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.50648915999999999 2024-09-18T10:15:29.603 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.603 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.603 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:29.603 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.278703+0000", 2024-09-18T10:15:29.603 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000111433 2024-09-18T10:15:29.603 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:29.603 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:29.603 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:29.603 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.603 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.604 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDPGPushReply(3.17s0 67/64 [PushReplyOp(3:ee962131:::smithi18925896-29:10)])", 2024-09-18T10:15:29.604 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:23.672359+0000", 2024-09-18T10:15:29.604 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 5.8815779770000001, 2024-09-18T10:15:29.604 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.50581707200000003, 2024-09-18T10:15:29.604 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:29.604 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:15:29.604 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:29.604 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.604 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:29.604 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.672359+0000", 2024-09-18T10:15:29.604 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:29.604 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.604 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.605 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:29.605 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.672359+0000", 2024-09-18T10:15:29.605 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:29.605 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.605 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.605 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:29.605 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.672358+0000", 2024-09-18T10:15:29.605 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:29.605 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.605 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.605 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:29.605 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.672367+0000", 2024-09-18T10:15:29.605 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.2529999999999993e-06 2024-09-18T10:15:29.606 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.606 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.606 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:29.606 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.672388+0000", 2024-09-18T10:15:29.606 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.0999e-05 2024-09-18T10:15:29.606 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.606 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.606 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:29.606 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.672397+0000", 2024-09-18T10:15:29.606 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.8300000000000002e-06 2024-09-18T10:15:29.606 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.606 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.606 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:29.607 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.178110+0000", 2024-09-18T10:15:29.607 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.50571351399999998 2024-09-18T10:15:29.607 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.607 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.607 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:29.607 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.178176+0000", 2024-09-18T10:15:29.607 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.5703000000000004e-05 2024-09-18T10:15:29.607 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:29.607 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:29.607 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:29.607 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.607 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.607 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDPGPushReply(3.17s0 67/64 [PushReplyOp(3:ee962131:::smithi18925896-29:10)])", 2024-09-18T10:15:29.608 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:23.770889+0000", 2024-09-18T10:15:29.608 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 5.7830475329999995, 2024-09-18T10:15:29.608 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.40748533399999998, 2024-09-18T10:15:29.608 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:29.608 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:15:29.608 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:29.608 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.608 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:29.608 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.770889+0000", 2024-09-18T10:15:29.608 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:29.608 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.608 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.608 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:29.609 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.770889+0000", 2024-09-18T10:15:29.609 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:29.609 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.609 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.609 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:29.609 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.770888+0000", 2024-09-18T10:15:29.609 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:29.609 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.609 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.609 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:29.609 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.770901+0000", 2024-09-18T10:15:29.609 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3216e-05 2024-09-18T10:15:29.610 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.610 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.610 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:29.610 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.770933+0000", 2024-09-18T10:15:29.610 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1955999999999997e-05 2024-09-18T10:15:29.610 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.610 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.610 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:29.610 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.770946+0000", 2024-09-18T10:15:29.610 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.289e-05 2024-09-18T10:15:29.610 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.610 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.610 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:29.610 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.178302+0000", 2024-09-18T10:15:29.610 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.40735659000000002 2024-09-18T10:15:29.611 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.611 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.611 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:29.611 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.178375+0000", 2024-09-18T10:15:29.611 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.2335000000000005e-05 2024-09-18T10:15:29.611 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:29.611 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:29.611 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:29.611 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.611 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.611 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDPGPushReply(3.17s0 67/64 [PushReplyOp(3:ee962131:::smithi18925896-29:10)])", 2024-09-18T10:15:29.611 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:23.772436+0000", 2024-09-18T10:15:29.611 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 5.7815011969999999, 2024-09-18T10:15:29.611 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.50655292500000004, 2024-09-18T10:15:29.611 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:29.612 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:15:29.612 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:29.612 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.612 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:29.612 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.772436+0000", 2024-09-18T10:15:29.612 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:29.612 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.612 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.612 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:29.612 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.772436+0000", 2024-09-18T10:15:29.612 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:29.612 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.612 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.612 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:29.613 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.772434+0000", 2024-09-18T10:15:29.613 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:29.613 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.613 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.613 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:29.613 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.772448+0000", 2024-09-18T10:15:29.613 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3682e-05 2024-09-18T10:15:29.613 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.613 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.613 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:29.613 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.772477+0000", 2024-09-18T10:15:29.613 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9255999999999999e-05 2024-09-18T10:15:29.613 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.613 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.614 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:29.614 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.772490+0000", 2024-09-18T10:15:29.614 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2962999999999999e-05 2024-09-18T10:15:29.614 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.614 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.614 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:29.614 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.278888+0000", 2024-09-18T10:15:29.614 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.50639794699999996 2024-09-18T10:15:29.614 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.614 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.614 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:29.614 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.278989+0000", 2024-09-18T10:15:29.614 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000100777 2024-09-18T10:15:29.615 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:29.615 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:29.615 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:29.615 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.615 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.615 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDPGPushReply(3.17s0 67/64 [PushReplyOp(3:ee962131:::smithi18925896-29:10)])", 2024-09-18T10:15:29.615 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:23.772911+0000", 2024-09-18T10:15:29.615 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 5.7810259180000001, 2024-09-18T10:15:29.615 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.50636473400000004, 2024-09-18T10:15:29.615 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:29.615 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:15:29.615 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:29.615 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.615 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:29.616 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.772911+0000", 2024-09-18T10:15:29.616 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:29.616 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.616 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.616 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:29.616 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.772911+0000", 2024-09-18T10:15:29.616 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:29.616 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.616 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.616 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:29.617 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.772909+0000", 2024-09-18T10:15:29.617 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:29.617 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.617 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.617 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:29.617 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.772923+0000", 2024-09-18T10:15:29.617 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3957e-05 2024-09-18T10:15:29.617 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.617 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.617 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:29.617 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.772956+0000", 2024-09-18T10:15:29.617 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2925000000000001e-05 2024-09-18T10:15:29.617 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.617 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.617 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:29.618 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.772969+0000", 2024-09-18T10:15:29.618 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2516e-05 2024-09-18T10:15:29.618 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.618 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.618 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:29.618 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.279192+0000", 2024-09-18T10:15:29.618 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.50622349799999999 2024-09-18T10:15:29.618 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.618 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.618 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:29.618 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.279276+0000", 2024-09-18T10:15:29.618 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.3498000000000006e-05 2024-09-18T10:15:29.618 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:29.618 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:29.619 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:29.619 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.619 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.619 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDPGPush(3.17s0 67/64 [PushOp(3:ead53b0f:::smithi18925896-25:13, version: 60'28, data_included: [262144~53248], data_size: 53248, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:ead53b0f:::smithi18925896-25:13@60'28, size: 1261568, copy_subset: [], clone_subset: {}, snapset: 13=[]:{13=[12,11,10,f,e]}, object_exist: 1), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:1261568, data_complete:true, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to:1048576, data_complete:false, omap_recovered_to:, omap_complete:true, error:false))])", 2024-09-18T10:15:29.619 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:23.871838+0000", 2024-09-18T10:15:29.619 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 5.6820987039999995, 2024-09-18T10:15:29.619 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.409266511, 2024-09-18T10:15:29.619 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:29.619 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:15:29.619 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:29.619 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.619 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:29.619 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.871838+0000", 2024-09-18T10:15:29.619 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:29.619 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.619 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.620 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:29.620 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.871838+0000", 2024-09-18T10:15:29.620 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:29.620 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.620 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.620 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:29.620 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.871838+0000", 2024-09-18T10:15:29.620 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:29.620 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.620 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.620 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:29.620 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.871838+0000", 2024-09-18T10:15:29.620 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:29.620 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.621 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.621 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:29.621 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.871884+0000", 2024-09-18T10:15:29.621 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.5512000000000001e-05 2024-09-18T10:15:29.621 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.621 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.621 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:29.621 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.871901+0000", 2024-09-18T10:15:29.621 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.6906e-05 2024-09-18T10:15:29.621 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.621 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.621 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:29.622 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.279416+0000", 2024-09-18T10:15:29.622 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.407515605 2024-09-18T10:15:29.622 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.622 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.622 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:29.622 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.281105+0000", 2024-09-18T10:15:29.622 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0016884879999999999 2024-09-18T10:15:29.622 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:29.622 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:29.622 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:29.622 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.622 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.622 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDPGPushReply(3.17s0 67/64 [PushReplyOp(3:ead53b0f:::smithi18925896-25:13)])", 2024-09-18T10:15:29.623 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:23.874483+0000", 2024-09-18T10:15:29.623 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 5.6794538770000003, 2024-09-18T10:15:29.623 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.40688253899999999, 2024-09-18T10:15:29.623 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:29.623 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:15:29.623 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:29.623 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.623 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:29.623 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.874483+0000", 2024-09-18T10:15:29.623 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:29.623 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.623 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.623 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:29.623 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.874483+0000", 2024-09-18T10:15:29.624 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:29.624 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.624 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.624 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:29.624 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.874483+0000", 2024-09-18T10:15:29.624 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:29.624 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.624 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.624 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:29.624 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.874489+0000", 2024-09-18T10:15:29.624 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.4869999999999999e-06 2024-09-18T10:15:29.624 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.624 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.625 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:29.625 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.874501+0000", 2024-09-18T10:15:29.625 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1803e-05 2024-09-18T10:15:29.625 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.625 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.628 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:29.628 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.874511+0000", 2024-09-18T10:15:29.628 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0363000000000001e-05 2024-09-18T10:15:29.628 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.629 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.629 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:29.629 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.281278+0000", 2024-09-18T10:15:29.629 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.40676716899999998 2024-09-18T10:15:29.629 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.629 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.629 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:29.629 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.281366+0000", 2024-09-18T10:15:29.629 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.7233999999999999e-05 2024-09-18T10:15:29.629 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:29.629 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:29.629 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:29.629 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.630 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.630 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDPGPushReply(3.17s0 67/64 [PushReplyOp(3:ead53b0f:::smithi18925896-25:13)])", 2024-09-18T10:15:29.630 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:23.874889+0000", 2024-09-18T10:15:29.630 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 5.6790475780000005, 2024-09-18T10:15:29.630 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.40661987900000002, 2024-09-18T10:15:29.630 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:29.630 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:15:29.630 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:29.630 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.630 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:29.630 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.874889+0000", 2024-09-18T10:15:29.630 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:29.630 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.630 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.631 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:29.631 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.874889+0000", 2024-09-18T10:15:29.631 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:29.631 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.631 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.631 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:29.631 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.874889+0000", 2024-09-18T10:15:29.631 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:29.631 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.631 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.631 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:29.631 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.874895+0000", 2024-09-18T10:15:29.631 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.6459999999999998e-06 2024-09-18T10:15:29.632 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.632 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.632 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:29.632 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.874905+0000", 2024-09-18T10:15:29.632 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.005e-05 2024-09-18T10:15:29.632 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.632 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.632 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:29.632 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.874917+0000", 2024-09-18T10:15:29.632 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.204e-05 2024-09-18T10:15:29.632 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.632 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.632 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:29.632 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.281419+0000", 2024-09-18T10:15:29.633 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.40650187300000001 2024-09-18T10:15:29.633 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.633 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.633 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:29.633 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.281509+0000", 2024-09-18T10:15:29.633 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.0729999999999994e-05 2024-09-18T10:15:29.633 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:29.633 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:29.633 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:29.633 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.633 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.633 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDPGPushReply(3.17s0 67/64 [PushReplyOp(3:ead53b0f:::smithi18925896-25:13)])", 2024-09-18T10:15:29.633 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:23.875445+0000", 2024-09-18T10:15:29.633 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 5.6784923540000003, 2024-09-18T10:15:29.634 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.4061514, 2024-09-18T10:15:29.634 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:29.634 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:15:29.634 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:29.634 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.634 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:29.634 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.875445+0000", 2024-09-18T10:15:29.634 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:29.634 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.634 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.634 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:29.634 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.875445+0000", 2024-09-18T10:15:29.634 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:29.635 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.635 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.635 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:29.635 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.875444+0000", 2024-09-18T10:15:29.635 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:15:29.635 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.635 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.635 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:29.635 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.875454+0000", 2024-09-18T10:15:29.635 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0733e-05 2024-09-18T10:15:29.635 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.635 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.635 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:29.635 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.875469+0000", 2024-09-18T10:15:29.636 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4966e-05 2024-09-18T10:15:29.636 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.636 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.636 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:29.636 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.875480+0000", 2024-09-18T10:15:29.636 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.077e-05 2024-09-18T10:15:29.636 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.636 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.636 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:29.636 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.281552+0000", 2024-09-18T10:15:29.636 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.406071982 2024-09-18T10:15:29.636 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.636 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.636 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:29.637 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.281596+0000", 2024-09-18T10:15:29.637 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4029000000000001e-05 2024-09-18T10:15:29.637 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:29.637 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:29.637 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:29.637 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.637 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.637 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDPGPushReply(3.17s0 67/64 [PushReplyOp(3:ead53b0f:::smithi18925896-25:13)])", 2024-09-18T10:15:29.637 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:23.875511+0000", 2024-09-18T10:15:29.637 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 5.6784262930000002, 2024-09-18T10:15:29.637 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.50638906400000006, 2024-09-18T10:15:29.637 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:29.637 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:15:29.638 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:29.638 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.638 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:29.638 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.875511+0000", 2024-09-18T10:15:29.638 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:29.638 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.638 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.638 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:29.638 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.875511+0000", 2024-09-18T10:15:29.638 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:29.638 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.638 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.638 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:29.638 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.875510+0000", 2024-09-18T10:15:29.639 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:29.639 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.639 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.639 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:29.639 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.875516+0000", 2024-09-18T10:15:29.639 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.8930000000000002e-06 2024-09-18T10:15:29.639 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.639 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.639 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:29.639 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.875526+0000", 2024-09-18T10:15:29.639 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0183000000000001e-05 2024-09-18T10:15:29.639 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.639 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.639 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:29.640 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.975617+0000", 2024-09-18T10:15:29.640 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10009069299999999 2024-09-18T10:15:29.640 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.640 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.640 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:29.640 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.381839+0000", 2024-09-18T10:15:29.640 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.40622223600000001 2024-09-18T10:15:29.640 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.640 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.640 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:29.640 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.381900+0000", 2024-09-18T10:15:29.640 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.0572000000000002e-05 2024-09-18T10:15:29.640 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:29.640 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:29.641 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:29.641 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.641 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.641 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDPGPushReply(3.17s0 67/64 [PushReplyOp(3:ead53b0f:::smithi18925896-25:13)])", 2024-09-18T10:15:29.641 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:23.975021+0000", 2024-09-18T10:15:29.641 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 5.5789159499999998, 2024-09-18T10:15:29.641 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.30665447400000001, 2024-09-18T10:15:29.641 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:29.641 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:15:29.641 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:29.641 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.641 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:29.641 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.975021+0000", 2024-09-18T10:15:29.642 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:29.642 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.642 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.642 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:29.642 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.975021+0000", 2024-09-18T10:15:29.642 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:29.642 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.642 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.642 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:29.642 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.975020+0000", 2024-09-18T10:15:29.642 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:29.642 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.642 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.642 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:29.643 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.975028+0000", 2024-09-18T10:15:29.643 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.5340000000000002e-06 2024-09-18T10:15:29.643 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.643 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.643 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:29.643 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.975045+0000", 2024-09-18T10:15:29.643 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7365999999999999e-05 2024-09-18T10:15:29.643 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.643 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.643 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:29.643 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.975058+0000", 2024-09-18T10:15:29.643 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2166e-05 2024-09-18T10:15:29.643 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.643 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.643 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:29.644 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.281634+0000", 2024-09-18T10:15:29.644 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.30657623899999997 2024-09-18T10:15:29.644 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.644 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.644 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:29.644 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.281675+0000", 2024-09-18T10:15:29.644 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.1689e-05 2024-09-18T10:15:29.644 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:29.644 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:29.644 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:29.644 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.644 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.644 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDPGPushReply(3.17s0 67/64 [PushReplyOp(3:ead53b0f:::smithi18925896-25:13)])", 2024-09-18T10:15:29.644 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:23.975748+0000", 2024-09-18T10:15:29.645 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 5.5781889939999996, 2024-09-18T10:15:29.645 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.406336629, 2024-09-18T10:15:29.645 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:29.645 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:15:29.645 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:29.645 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.645 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:29.645 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.975748+0000", 2024-09-18T10:15:29.645 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:29.645 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.645 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.645 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:29.645 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.975748+0000", 2024-09-18T10:15:29.646 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:29.646 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.646 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.646 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:29.646 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.975748+0000", 2024-09-18T10:15:29.646 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:29.646 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.646 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.646 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:29.646 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.975757+0000", 2024-09-18T10:15:29.646 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.9369999999999993e-06 2024-09-18T10:15:29.646 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.646 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.646 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:29.646 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.975771+0000", 2024-09-18T10:15:29.647 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4722999999999999e-05 2024-09-18T10:15:29.647 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.647 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.647 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:29.647 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.975779+0000", 2024-09-18T10:15:29.647 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.2029999999999992e-06 2024-09-18T10:15:29.647 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.647 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.647 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:29.647 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.381963+0000", 2024-09-18T10:15:29.647 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.40618386400000001 2024-09-18T10:15:29.647 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.647 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.647 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:29.648 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.382085+0000", 2024-09-18T10:15:29.648 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000121279 2024-09-18T10:15:29.648 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:29.648 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:29.648 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:29.648 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:29.648 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:29.648 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-09-18T10:15:29.650 INFO:tasks.rados.rados.0.smithi189.stdout:260: done (3 left) 2024-09-18T10:15:29.650 INFO:tasks.rados.rados.0.smithi189.stdout:append oid 25 current snap is 21 2024-09-18T10:15:29.650 INFO:tasks.rados.rados.0.smithi189.stdout:257: finishing rollback tid 1 to smithi18925896-7 2024-09-18T10:15:29.650 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 7 v 0 (ObjNum 129 snap 0 seq_num 3858943912) dirty dne 2024-09-18T10:15:29.650 INFO:tasks.rados.rados.0.smithi189.stdout:259: expect (ObjNum 100 snap 17 seq_num 100) 2024-09-18T10:15:29.651 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_ops_in_flight 2024-09-18T10:15:29.651 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-18T10:15:29.648+0000 7f14c6a1a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 22138) UID: 0 2024-09-18T10:15:29.730 INFO:tasks.rados.rados.0.smithi189.stdout:258: finishing write tid 1 to smithi18925896-20 2024-09-18T10:15:29.730 INFO:tasks.rados.rados.0.smithi189.stdout:258: finishing write tid 2 to smithi18925896-20 2024-09-18T10:15:29.730 INFO:tasks.rados.rados.0.smithi189.stdout:258: finishing write tid 3 to smithi18925896-20 2024-09-18T10:15:29.730 INFO:tasks.rados.rados.0.smithi189.stdout:261: seq_num 108 ranges {2637824=655360,3293184=606208,3899392=475136,4374528=81920} 2024-09-18T10:15:29.750 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-09-18T10:15:29.747+0000 7fdfa8698700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22087) UID: 0 2024-09-18T10:15:29.754 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-09-18T10:15:29.754 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-09-18T10:15:29.754 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.754 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWriteReply(3.2s0 74/40 ECSubWriteReply(tid=95, last_complete=74'59, committed=1, applied=1))", 2024-09-18T10:15:29.754 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:29.564678+0000", 2024-09-18T10:15:29.754 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 0.18912056399999999, 2024-09-18T10:15:29.755 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.18915407100000001, 2024-09-18T10:15:29.755 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:29.755 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "queued for pg", 2024-09-18T10:15:29.755 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:29.755 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.755 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:29.755 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.564678+0000", 2024-09-18T10:15:29.755 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:29.755 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.755 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.755 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:29.755 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.564678+0000", 2024-09-18T10:15:29.756 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:29.756 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.756 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.756 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:29.756 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.564677+0000", 2024-09-18T10:15:29.756 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:29.756 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.756 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.756 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:29.756 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.564682+0000", 2024-09-18T10:15:29.756 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.5000000000000001e-06 2024-09-18T10:15:29.756 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.757 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.757 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:29.757 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.564692+0000", 2024-09-18T10:15:29.757 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0618999999999999e-05 2024-09-18T10:15:29.757 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.757 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.757 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:29.757 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.664871+0000", 2024-09-18T10:15:29.757 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100178509 2024-09-18T10:15:29.757 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:29.757 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:29.757 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:29.757 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.758 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.758 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.es1 74/32 ECSubWrite(tid=48, reqid=osd.11.0:48, at_version=74'39, trim_to=0'0, roll_forward_to=71'37))", 2024-09-18T10:15:29.758 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:29.667570+0000", 2024-09-18T10:15:29.758 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 0.086228363000000002, 2024-09-18T10:15:29.758 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.086336473999999996, 2024-09-18T10:15:29.758 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:29.758 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "queued for pg", 2024-09-18T10:15:29.758 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:29.758 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.758 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:29.758 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.667570+0000", 2024-09-18T10:15:29.758 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:29.758 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.758 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.759 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:29.759 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.667570+0000", 2024-09-18T10:15:29.759 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:29.759 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.759 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.759 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:29.759 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.667570+0000", 2024-09-18T10:15:29.759 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:29.759 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.759 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.759 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:29.759 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.667601+0000", 2024-09-18T10:15:29.759 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1242999999999999e-05 2024-09-18T10:15:29.759 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.759 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.759 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:29.760 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.667612+0000", 2024-09-18T10:15:29.760 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0943e-05 2024-09-18T10:15:29.760 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.760 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.760 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:29.760 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.667620+0000", 2024-09-18T10:15:29.760 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.4970000000000001e-06 2024-09-18T10:15:29.760 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:29.760 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:29.760 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:29.760 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.760 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.760 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.6s1 74/40 ECSubWrite(tid=64, reqid=osd.0.0:64, at_version=74'40, trim_to=0'0, roll_forward_to=72'38))", 2024-09-18T10:15:29.760 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:29.670719+0000", 2024-09-18T10:15:29.760 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 0.083079182000000001, 2024-09-18T10:15:29.761 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.083249330999999996, 2024-09-18T10:15:29.761 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:29.761 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:15:29.761 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:29.761 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.761 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:29.761 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.670719+0000", 2024-09-18T10:15:29.761 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:29.761 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.761 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.761 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:29.761 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.670719+0000", 2024-09-18T10:15:29.761 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:29.762 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.762 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.762 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:29.762 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.670718+0000", 2024-09-18T10:15:29.762 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:15:29.762 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.762 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.762 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:29.762 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.670769+0000", 2024-09-18T10:15:29.762 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.0685e-05 2024-09-18T10:15:29.762 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.762 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.762 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:29.762 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.670784+0000", 2024-09-18T10:15:29.763 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.5143999999999999e-05 2024-09-18T10:15:29.763 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.763 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.763 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:29.763 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.670801+0000", 2024-09-18T10:15:29.763 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.6725e-05 2024-09-18T10:15:29.763 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.763 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.763 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:29.763 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.670863+0000", 2024-09-18T10:15:29.763 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.1533999999999997e-05 2024-09-18T10:15:29.763 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:29.763 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:29.764 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:29.764 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.670881+0000", 2024-09-18T10:15:29.764 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8258999999999999e-05 2024-09-18T10:15:29.764 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:29.764 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:29.764 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:29.764 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:29.764 INFO:teuthology.orchestra.run.smithi154.stdout: ], 2024-09-18T10:15:29.764 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 3 2024-09-18T10:15:29.764 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-09-18T10:15:29.766 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_blocked_ops 2024-09-18T10:15:29.766 INFO:tasks.rados.rados.0.smithi189.stdout:261: writing smithi18925896-25 from 2637824 to 3293184 tid 1 2024-09-18T10:15:29.766 INFO:tasks.rados.rados.0.smithi189.stdout:261: writing smithi18925896-25 from 3293184 to 3899392 tid 2 2024-09-18T10:15:29.766 INFO:tasks.rados.rados.0.smithi189.stdout:261: writing smithi18925896-25 from 3899392 to 4374528 tid 3 2024-09-18T10:15:29.767 INFO:tasks.rados.rados.0.smithi189.stdout:261: writing smithi18925896-25 from 4374528 to 4456448 tid 4 2024-09-18T10:15:29.767 INFO:tasks.rados.rados.0.smithi189.stdout:257: done (3 left) 2024-09-18T10:15:29.767 INFO:tasks.rados.rados.0.smithi189.stdout:259: done (2 left) 2024-09-18T10:15:29.767 INFO:tasks.rados.rados.0.smithi189.stdout:append oid 17 current snap is 21 2024-09-18T10:15:29.767 INFO:tasks.rados.rados.0.smithi189.stdout:262: seq_num 109 ranges {1916928=671744,2588672=753664,3342336=409600} 2024-09-18T10:15:29.770 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-18T10:15:29.774 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-09-18T10:15:29.777 INFO:tasks.rados.rados.0.smithi189.stdout:262: writing smithi18925896-17 from 1916928 to 2588672 tid 1 2024-09-18T10:15:29.781 INFO:tasks.rados.rados.0.smithi189.stdout:262: writing smithi18925896-17 from 2588672 to 3342336 tid 2 2024-09-18T10:15:29.785 INFO:tasks.rados.rados.0.smithi189.stdout:262: writing smithi18925896-17 from 3342336 to 3751936 tid 3 2024-09-18T10:15:29.785 INFO:tasks.rados.rados.0.smithi189.stdout:263: rollback oid 49 current snap is 21 2024-09-18T10:15:29.785 INFO:tasks.rados.rados.0.smithi189.stdout:rollback oid 49 to 9 2024-09-18T10:15:29.785 INFO:tasks.rados.rados.0.smithi189.stdout:258: finishing write tid 4 to smithi18925896-20 2024-09-18T10:15:29.785 INFO:tasks.rados.rados.0.smithi189.stdout:264: rollback oid 40 current snap is 21 2024-09-18T10:15:29.785 INFO:tasks.rados.rados.0.smithi189.stdout:rollback oid 40 to 10 2024-09-18T10:15:29.785 INFO:tasks.rados.rados.0.smithi189.stdout:265: rmattr oid 23 current snap is 21 2024-09-18T10:15:29.785 INFO:tasks.rados.rados.0.smithi189.stdout:266: rollback oid 33 current snap is 21 2024-09-18T10:15:29.785 INFO:tasks.rados.rados.0.smithi189.stdout:rollback oid 33 to 16 2024-09-18T10:15:29.786 INFO:tasks.rados.rados.0.smithi189.stdout:267: rmattr oid 10 current snap is 21 2024-09-18T10:15:29.786 INFO:tasks.rados.rados.0.smithi189.stdout:267: done (7 left) 2024-09-18T10:15:29.786 INFO:tasks.rados.rados.0.smithi189.stdout:268: copy_from oid 34 from oid 7 current snap is 21 2024-09-18T10:15:29.786 INFO:tasks.rados.rados.0.smithi189.stdout:269: rollback oid 43 current snap is 21 2024-09-18T10:15:29.786 INFO:tasks.rados.rados.0.smithi189.stdout:rollback oid 43 to 10 2024-09-18T10:15:29.786 INFO:tasks.rados.rados.0.smithi189.stdout:270: snap_remove snap 13 2024-09-18T10:15:29.789 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":89,"stamp":"2024-09-18T10:15:29.552554+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":64692787,"num_objects":86,"num_object_clones":41,"num_object_copies":582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86,"num_whiteouts":8,"num_read":298,"num_read_kb":44309,"num_write":518,"num_write_kb":57050,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":6012928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":914,"ondisk_log_size":914,"up":200,"acting":200,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":206,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":16,"kb":1509949440,"kb_used":4893080,"kb_used_data":131160,"kb_used_omap":26,"kb_used_meta":4761637,"kb_avail":1505056360,"statfs":{"total":1546188226560,"available":1541177712640,"internally_reserved":0,"allocated":134307840,"data_stored":167036909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":27264,"internal_metadata":4875916672},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14,"num_read_kb":5786,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.601671"},"pg_stats":[{"pgid":"3.19","version":"68'92","reported_seq":210,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-18T10:15:24.873328+0000","last_change":"2024-09-18T10:15:23.459301+0000","last_active":"2024-09-18T10:15:24.873328+0000","last_peered":"2024-09-18T10:15:24.873328+0000","last_clean":"2024-09-18T10:15:24.873328+0000","last_became_active":"2024-09-18T10:14:57.350904+0000","last_became_peered":"2024-09-18T10:14:57.350904+0000","last_unstale":"2024-09-18T10:15:24.873328+0000","last_undegraded":"2024-09-18T10:15:24.873328+0000","last_fullsized":"2024-09-18T10:15:24.873328+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:14:45.576168+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.41041204199999998,"stat_sum":{"num_bytes":9592832,"num_objects":10,"num_object_clones":7,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":25,"num_read_kb":7476,"num_write":57,"num_write_kb":10280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,10,11,5,8,3],"acting":[7,2,10,11,5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"e","length":"1"},{"start":"13","length":"1"},{"start":"18","length":"2"}]},{"pgid":"3.18","version":"68'36","reported_seq":159,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-18T10:15:25.252708+0000","last_change":"2024-09-18T10:15:22.137237+0000","last_active":"2024-09-18T10:15:25.252708+0000","last_peered":"2024-09-18T10:15:25.252708+0000","last_clean":"2024-09-18T10:15:25.252708+0000","last_became_active":"2024-09-18T10:14:57.647465+0000","last_became_peered":"2024-09-18T10:14:57.647465+0000","last_unstale":"2024-09-18T10:15:25.252708+0000","last_undegraded":"2024-09-18T10:15:25.252708+0000","last_fullsized":"2024-09-18T10:15:25.252708+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:14:45.576168+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.116355192,"stat_sum":{"num_bytes":3997696,"num_objects":5,"num_object_clones":3,"num_object_copies":35,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":1,"num_read":13,"num_read_kb":2309,"num_write":26,"num_write_kb":3920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,11,3,15,1,7],"acting":[6,0,11,3,15,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"e","length":"1"},{"start":"13","length":"1"},{"start":"18","length":"2"}]},{"pgid":"3.17","version":"68'35","reported_seq":76,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-18T10:15:25.001073+0000","last_change":"2024-09-18T10:15:25.001073+0000","last_active":"2024-09-18T10:15:25.001073+0000","last_peered":"2024-09-18T10:15:25.001073+0000","last_clean":"2024-09-18T10:15:25.001073+0000","last_became_active":"2024-09-18T10:15:21.323733+0000","last_became_peered":"2024-09-18T10:15:21.323733+0000","last_unstale":"2024-09-18T10:15:25.001073+0000","last_undegraded":"2024-09-18T10:15:25.001073+0000","last_fullsized":"2024-09-18T10:15:25.001073+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:14:45.576168+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.41336515200000001,"stat_sum":{"num_bytes":2883584,"num_objects":4,"num_object_clones":2,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":1,"num_read":13,"num_read_kb":4757,"num_write":13,"num_write_kb":3024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":4374528,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,13,3,14,2,11,6],"acting":[10,13,3,14,2,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"e","length":"1"},{"start":"13","length":"1"},{"start":"18","length":"2"}]},{"pgid":"3.16","version":"71'20","reported_seq":70,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-18T10:15:28.966389+0000","last_change":"2024-09-18T10:15:28.186914+0000","last_active":"2024-09-18T10:15:28.966389+0000","last_peered":"2024-09-18T10:15:28.966389+0000","last_clean":"2024-09-18T10:15:28.966389+0000","last_became_active":"2024-09-18T10:15:19.227471+0000","last_became_peered":"2024-09-18T10:15:19.227471+0000","last_unstale":"2024-09-18T10:15:28.966389+0000","last_undegraded":"2024-09-18T10:15:28.966389+0000","last_fullsized":"2024-09-18T10:15:28.966389+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":5,"last_scrub":"36'11","last_scrub_stamp":"2024-09-18T10:15:07.445786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:07.445786+0000","objects_scrubbed":1,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:23.145992+0000","scrub_duration":0.30645798400000002,"objects_trimmed":0,"snaptrim_duration":0.001563779,"stat_sum":{"num_bytes":2621440,"num_objects":2,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":1888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":688128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,6,4,8,2,14,10],"acting":[9,6,4,8,2,14,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[{"start":"f","length":"1"},{"start":"21","length":"1"}]},{"pgid":"3.15","version":"66'10","reported_seq":60,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-18T10:15:25.100987+0000","last_change":"2024-09-18T10:15:22.238714+0000","last_active":"2024-09-18T10:15:25.100987+0000","last_peered":"2024-09-18T10:15:25.100987+0000","last_clean":"2024-09-18T10:15:25.100987+0000","last_became_active":"2024-09-18T10:15:17.127771+0000","last_became_peered":"2024-09-18T10:15:17.127771+0000","last_unstale":"2024-09-18T10:15:25.100987+0000","last_undegraded":"2024-09-18T10:15:25.100987+0000","last_fullsized":"2024-09-18T10:15:25.100987+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:14:45.576168+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.21260638700000001,"stat_sum":{"num_bytes":32768,"num_objects":2,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":1,"num_read":2,"num_read_kb":0,"num_write":2,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,15,13,10,1,7,9],"acting":[14,15,13,10,1,7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"e","length":"1"},{"start":"13","length":"1"},{"start":"18","length":"2"}]},{"pgid":"3.14","version":"37'8","reported_seq":54,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-18T10:15:23.962750+0000","last_change":"2024-09-18T10:15:21.930993+0000","last_active":"2024-09-18T10:15:23.962750+0000","last_peered":"2024-09-18T10:15:23.962750+0000","last_clean":"2024-09-18T10:15:23.962750+0000","last_became_active":"2024-09-18T10:15:14.767770+0000","last_became_peered":"2024-09-18T10:15:14.767770+0000","last_unstale":"2024-09-18T10:15:23.962750+0000","last_undegraded":"2024-09-18T10:15:23.962750+0000","last_fullsized":"2024-09-18T10:15:23.962750+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":5,"last_scrub":"37'8","last_scrub_stamp":"2024-09-18T10:15:17.041590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:17.041590+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:32.741813+0000","scrub_duration":0.60018000800000004,"objects_trimmed":0,"snaptrim_duration":0.0014793740000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,14,9,10,5,13,4],"acting":[1,14,9,10,5,13,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"e","length":"1"},{"start":"13","length":"1"},{"start":"18","length":"2"}]},{"pgid":"2.7","version":"0'0","reported_seq":55,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-18T10:15:25.206095+0000","last_change":"2024-09-18T10:14:37.427722+0000","last_active":"2024-09-18T10:15:25.206095+0000","last_peered":"2024-09-18T10:15:25.206095+0000","last_clean":"2024-09-18T10:15:25.206095+0000","last_became_active":"2024-09-18T10:14:35.348943+0000","last_became_peered":"2024-09-18T10:14:35.348943+0000","last_unstale":"2024-09-18T10:15:25.206095+0000","last_undegraded":"2024-09-18T10:15:25.206095+0000","last_fullsized":"2024-09-18T10:15:25.206095+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:14:34.288596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:03.161727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060442999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,10],"acting":[13,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"3.6","version":"57'32","reported_seq":167,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-18T10:15:23.069165+0000","last_change":"2024-09-18T10:15:21.935648+0000","last_active":"2024-09-18T10:15:23.069165+0000","last_peered":"2024-09-18T10:15:23.069165+0000","last_clean":"2024-09-18T10:15:23.069165+0000","last_became_active":"2024-09-18T10:14:57.139786+0000","last_became_peered":"2024-09-18T10:14:57.139786+0000","last_unstale":"2024-09-18T10:15:23.069165+0000","last_undegraded":"2024-09-18T10:15:23.069165+0000","last_fullsized":"2024-09-18T10:15:23.069165+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:14:45.576168+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010832179999999999,"stat_sum":{"num_bytes":868352,"num_objects":2,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":988,"num_write":11,"num_write_kb":1960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,14,6,2,5,9,15],"acting":[0,14,6,2,5,9,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"e","length":"1"},{"start":"13","length":"1"},{"start":"18","length":"2"}]},{"pgid":"2.6","version":"0'0","reported_seq":54,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-18T10:15:24.160814+0000","last_change":"2024-09-18T10:14:38.963300+0000","last_active":"2024-09-18T10:15:24.160814+0000","last_peered":"2024-09-18T10:15:24.160814+0000","last_clean":"2024-09-18T10:15:24.160814+0000","last_became_active":"2024-09-18T10:14:35.429859+0000","last_became_peered":"2024-09-18T10:14:35.429859+0000","last_unstale":"2024-09-18T10:15:24.160814+0000","last_undegraded":"2024-09-18T10:15:24.160814+0000","last_fullsized":"2024-09-18T10:15:24.160814+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:14:34.288596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:15:40.196292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037634499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11],"acting":[8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.7","version":"66'39","reported_seq":171,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-18T10:15:23.959984+0000","last_change":"2024-09-18T10:15:22.345767+0000","last_active":"2024-09-18T10:15:23.959984+0000","last_peered":"2024-09-18T10:15:23.959984+0000","last_clean":"2024-09-18T10:15:23.959984+0000","last_became_active":"2024-09-18T10:14:57.137776+0000","last_became_peered":"2024-09-18T10:14:57.137776+0000","last_unstale":"2024-09-18T10:15:23.959984+0000","last_undegraded":"2024-09-18T10:15:23.959984+0000","last_fullsized":"2024-09-18T10:15:23.959984+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"60'30","last_scrub_stamp":"2024-09-18T10:15:17.136535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:17.136535+0000","objects_scrubbed":3,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:35.499009+0000","scrub_duration":0.60680385400000003,"objects_trimmed":0,"snaptrim_duration":0.41230857199999998,"stat_sum":{"num_bytes":5324800,"num_objects":4,"num_object_clones":2,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6,0,14,10,13],"acting":[4,11,6,0,14,10,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"e","length":"1"},{"start":"13","length":"1"},{"start":"18","length":"2"}]},{"pgid":"2.5","version":"0'0","reported_seq":53,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-18T10:15:24.153802+0000","last_change":"2024-09-18T10:14:37.327877+0000","last_active":"2024-09-18T10:15:24.153802+0000","last_peered":"2024-09-18T10:15:24.153802+0000","last_clean":"2024-09-18T10:15:24.153802+0000","last_became_active":"2024-09-18T10:14:35.329198+0000","last_became_peered":"2024-09-18T10:14:35.329198+0000","last_unstale":"2024-09-18T10:15:24.153802+0000","last_undegraded":"2024-09-18T10:15:24.153802+0000","last_fullsized":"2024-09-18T10:15:24.153802+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:14:34.288596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:15:44.196046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062415699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"66'40","reported_seq":177,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-18T10:15:24.160601+0000","last_change":"2024-09-18T10:15:22.747902+0000","last_active":"2024-09-18T10:15:24.160601+0000","last_peered":"2024-09-18T10:15:24.160601+0000","last_clean":"2024-09-18T10:15:24.160601+0000","last_became_active":"2024-09-18T10:14:57.134641+0000","last_became_peered":"2024-09-18T10:14:57.134641+0000","last_unstale":"2024-09-18T10:15:24.160601+0000","last_undegraded":"2024-09-18T10:15:24.160601+0000","last_fullsized":"2024-09-18T10:15:24.160601+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"60'34","last_scrub_stamp":"2024-09-18T10:15:17.325003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:17.325003+0000","objects_scrubbed":6,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:18.565493+0000","scrub_duration":0.82233752999999998,"objects_trimmed":0,"snaptrim_duration":0.81593390099999996,"stat_sum":{"num_bytes":4456448,"num_objects":6,"num_object_clones":3,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":4,"num_read_kb":1815,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,15,4,2,6,10,7],"acting":[8,15,4,2,6,10,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"e","length":"1"},{"start":"13","length":"1"},{"start":"18","length":"2"}]},{"pgid":"2.4","version":"0'0","reported_seq":54,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-18T10:15:23.959787+0000","last_change":"2024-09-18T10:14:37.426192+0000","last_active":"2024-09-18T10:15:23.959787+0000","last_peered":"2024-09-18T10:15:23.959787+0000","last_clean":"2024-09-18T10:15:23.959787+0000","last_became_active":"2024-09-18T10:14:35.524102+0000","last_became_peered":"2024-09-18T10:14:35.524102+0000","last_unstale":"2024-09-18T10:15:23.959787+0000","last_undegraded":"2024-09-18T10:15:23.959787+0000","last_fullsized":"2024-09-18T10:15:23.959787+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:14:34.288596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:15:59.190171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.10077985,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.5","version":"35'2","reported_seq":126,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-18T10:15:23.936668+0000","last_change":"2024-09-18T10:15:22.136251+0000","last_active":"2024-09-18T10:15:23.936668+0000","last_peered":"2024-09-18T10:15:23.936668+0000","last_clean":"2024-09-18T10:15:23.936668+0000","last_became_active":"2024-09-18T10:14:57.135810+0000","last_became_peered":"2024-09-18T10:14:57.135810+0000","last_unstale":"2024-09-18T10:15:23.936668+0000","last_undegraded":"2024-09-18T10:15:23.936668+0000","last_fullsized":"2024-09-18T10:15:23.936668+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"35'2","last_scrub_stamp":"2024-09-18T10:15:03.886316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:03.886316+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:14.307127+0000","scrub_duration":0.35694589500000001,"objects_trimmed":0,"snaptrim_duration":0.101978342,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,4,9,12,0,11],"acting":[5,8,4,9,12,0,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"e","length":"1"},{"start":"13","length":"1"},{"start":"18","length":"2"}]},{"pgid":"3.3","version":"65'23","reported_seq":141,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-18T10:15:24.064082+0000","last_change":"2024-09-18T10:15:22.033003+0000","last_active":"2024-09-18T10:15:24.064082+0000","last_peered":"2024-09-18T10:15:24.064082+0000","last_clean":"2024-09-18T10:15:24.064082+0000","last_became_active":"2024-09-18T10:14:57.138546+0000","last_became_peered":"2024-09-18T10:14:57.138546+0000","last_unstale":"2024-09-18T10:15:24.064082+0000","last_undegraded":"2024-09-18T10:15:24.064082+0000","last_fullsized":"2024-09-18T10:15:24.064082+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:14:45.576168+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001309478,"stat_sum":{"num_bytes":516096,"num_objects":2,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":1,"num_write":6,"num_write_kb":784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4,13,11,2,5],"acting":[10,9,4,13,11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"e","length":"1"},{"start":"13","length":"1"},{"start":"18","length":"2"}]},{"pgid":"2.2","version":"29'2","reported_seq":57,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-18T10:15:25.105768+0000","last_change":"2024-09-18T10:14:37.429583+0000","last_active":"2024-09-18T10:15:25.105768+0000","last_peered":"2024-09-18T10:15:25.105768+0000","last_clean":"2024-09-18T10:15:25.105768+0000","last_became_active":"2024-09-18T10:14:35.715374+0000","last_became_peered":"2024-09-18T10:14:35.715374+0000","last_unstale":"2024-09-18T10:15:25.105768+0000","last_undegraded":"2024-09-18T10:15:25.105768+0000","last_fullsized":"2024-09-18T10:15:25.105768+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:14:34.288596+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:03.520789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002421224,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,12],"acting":[13,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"3.0","version":"66'18","reported_seq":149,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-18T10:15:24.058769+0000","last_change":"2024-09-18T10:15:22.138999+0000","last_active":"2024-09-18T10:15:24.058769+0000","last_peered":"2024-09-18T10:15:24.058769+0000","last_clean":"2024-09-18T10:15:24.058769+0000","last_became_active":"2024-09-18T10:14:57.444374+0000","last_became_peered":"2024-09-18T10:14:57.444374+0000","last_unstale":"2024-09-18T10:15:24.058769+0000","last_undegraded":"2024-09-18T10:15:24.058769+0000","last_fullsized":"2024-09-18T10:15:24.058769+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'4","last_scrub_stamp":"2024-09-18T10:15:05.668933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:05.668933+0000","objects_scrubbed":1,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:18.433780+0000","scrub_duration":2.6328342170000001,"objects_trimmed":0,"snaptrim_duration":0.20786020399999999,"stat_sum":{"num_bytes":212992,"num_objects":2,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":1,"num_read":2,"num_read_kb":0,"num_write":2,"num_write_kb":208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,9,2,6,15,4],"acting":[3,7,9,2,6,15,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"e","length":"1"},{"start":"13","length":"1"},{"start":"18","length":"2"}]},{"pgid":"2.1","version":"0'0","reported_seq":56,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-18T10:15:25.100469+0000","last_change":"2024-09-18T10:14:37.601046+0000","last_active":"2024-09-18T10:15:25.100469+0000","last_peered":"2024-09-18T10:15:25.100469+0000","last_clean":"2024-09-18T10:15:25.100469+0000","last_became_active":"2024-09-18T10:14:35.348262+0000","last_became_peered":"2024-09-18T10:14:35.348262+0000","last_unstale":"2024-09-18T10:15:25.100469+0000","last_undegraded":"2024-09-18T10:15:25.100469+0000","last_fullsized":"2024-09-18T10:15:25.100469+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:14:34.288596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:03.520789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063430400000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,0],"acting":[14,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"3.1","version":"68'30","reported_seq":151,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-18T10:15:24.166023+0000","last_change":"2024-09-18T10:15:22.436295+0000","last_active":"2024-09-18T10:15:24.166023+0000","last_peered":"2024-09-18T10:15:24.166023+0000","last_clean":"2024-09-18T10:15:24.166023+0000","last_became_active":"2024-09-18T10:14:57.038503+0000","last_became_peered":"2024-09-18T10:14:57.038503+0000","last_unstale":"2024-09-18T10:15:24.166023+0000","last_undegraded":"2024-09-18T10:15:24.166023+0000","last_fullsized":"2024-09-18T10:15:24.166023+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:14:45.576168+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.50270893999999999,"stat_sum":{"num_bytes":696320,"num_objects":3,"num_object_clones":2,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":10,"num_read_kb":1779,"num_write":8,"num_write_kb":680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,13,14,10,7,3,2],"acting":[4,13,14,10,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"e","length":"1"},{"start":"13","length":"1"},{"start":"18","length":"2"}]},{"pgid":"2.0","version":"0'0","reported_seq":59,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-18T10:15:28.084815+0000","last_change":"2024-09-18T10:14:37.600953+0000","last_active":"2024-09-18T10:15:28.084815+0000","last_peered":"2024-09-18T10:15:28.084815+0000","last_clean":"2024-09-18T10:15:28.084815+0000","last_became_active":"2024-09-18T10:14:35.349203+0000","last_became_peered":"2024-09-18T10:14:35.349203+0000","last_unstale":"2024-09-18T10:15:28.084815+0000","last_undegraded":"2024-09-18T10:15:28.084815+0000","last_fullsized":"2024-09-18T10:15:28.084815+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:14:34.288596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:15:37.868112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060130600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3],"acting":[9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2","version":"66'45","reported_seq":185,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-18T10:15:25.953811+0000","last_change":"2024-09-18T10:15:22.232917+0000","last_active":"2024-09-18T10:15:25.953811+0000","last_peered":"2024-09-18T10:15:25.953811+0000","last_clean":"2024-09-18T10:15:25.953811+0000","last_became_active":"2024-09-18T10:14:57.227520+0000","last_became_peered":"2024-09-18T10:14:57.227520+0000","last_unstale":"2024-09-18T10:15:25.953811+0000","last_undegraded":"2024-09-18T10:15:25.953811+0000","last_fullsized":"2024-09-18T10:15:25.953811+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'18","last_scrub_stamp":"2024-09-18T10:15:03.202100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:03.202100+0000","objects_scrubbed":6,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:12.722351+0000","scrub_duration":0.63960403700000001,"objects_trimmed":0,"snaptrim_duration":0.30826759100000001,"stat_sum":{"num_bytes":3112960,"num_objects":6,"num_object_clones":2,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":1,"num_read":15,"num_read_kb":4021,"num_write":7,"num_write_kb":1552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,3,13,1,5,10,7],"acting":[14,3,13,1,5,10,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"e","length":"1"},{"start":"13","length":"1"},{"start":"18","length":"2"}]},{"pgid":"2.3","version":"27'1","reported_seq":57,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-18T10:15:25.100599+0000","last_change":"2024-09-18T10:14:37.801252+0000","last_active":"2024-09-18T10:15:25.100599+0000","last_peered":"2024-09-18T10:15:25.100599+0000","last_clean":"2024-09-18T10:15:25.100599+0000","last_became_active":"2024-09-18T10:14:35.487746+0000","last_became_peered":"2024-09-18T10:14:35.487746+0000","last_unstale":"2024-09-18T10:15:25.100599+0000","last_undegraded":"2024-09-18T10:15:25.100599+0000","last_fullsized":"2024-09-18T10:15:25.100599+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:14:34.288596+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:03.161727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.10072764100000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,5],"acting":[14,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"1.0","version":"29'112","reported_seq":189,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-18T10:15:23.939109+0000","last_change":"2024-09-18T10:14:32.509942+0000","last_active":"2024-09-18T10:15:23.939109+0000","last_peered":"2024-09-18T10:15:23.939109+0000","last_clean":"2024-09-18T10:15:23.939109+0000","last_became_active":"2024-09-18T10:14:32.508037+0000","last_became_peered":"2024-09-18T10:14:32.508037+0000","last_unstale":"2024-09-18T10:15:23.939109+0000","last_undegraded":"2024-09-18T10:15:23.939109+0000","last_fullsized":"2024-09-18T10:15:23.939109+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:31.267741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:31.267741+0000","last_clean_scrub_stamp":"2024-09-18T10:14:31.267741+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:15:59.262610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,15],"acting":[6,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.8","version":"41'13","reported_seq":142,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-18T10:15:24.039045+0000","last_change":"2024-09-18T10:15:22.123091+0000","last_active":"2024-09-18T10:15:24.039045+0000","last_peered":"2024-09-18T10:15:24.039045+0000","last_clean":"2024-09-18T10:15:24.039045+0000","last_became_active":"2024-09-18T10:14:57.443754+0000","last_became_peered":"2024-09-18T10:14:57.443754+0000","last_unstale":"2024-09-18T10:15:24.039045+0000","last_undegraded":"2024-09-18T10:15:24.039045+0000","last_fullsized":"2024-09-18T10:15:24.039045+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"41'13","last_scrub_stamp":"2024-09-18T10:15:05.373444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:05.373444+0000","objects_scrubbed":2,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:31.932850+0000","scrub_duration":2.8745114210000002,"objects_trimmed":0,"snaptrim_duration":0.0012354600000000001,"stat_sum":{"num_bytes":966656,"num_objects":2,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":705,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,11,3,15,1,7],"acting":[6,0,11,3,15,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"e","length":"1"},{"start":"13","length":"1"},{"start":"18","length":"2"}]},{"pgid":"3.9","version":"69'68","reported_seq":197,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-18T10:15:25.146780+0000","last_change":"2024-09-18T10:15:23.152029+0000","last_active":"2024-09-18T10:15:25.146780+0000","last_peered":"2024-09-18T10:15:25.146780+0000","last_clean":"2024-09-18T10:15:25.146780+0000","last_became_active":"2024-09-18T10:14:56.840271+0000","last_became_peered":"2024-09-18T10:14:56.840271+0000","last_unstale":"2024-09-18T10:15:25.146780+0000","last_undegraded":"2024-09-18T10:15:25.146780+0000","last_fullsized":"2024-09-18T10:15:25.146780+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'35","last_scrub_stamp":"2024-09-18T10:15:07.077517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:07.077517+0000","objects_scrubbed":4,"log_size":68,"log_dups_size":0,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:31.443309+0000","scrub_duration":0.71488955499999995,"objects_trimmed":0,"snaptrim_duration":0.0014440639999999999,"stat_sum":{"num_bytes":6766592,"num_objects":8,"num_object_clones":6,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":12,"num_read_kb":5292,"num_write":29,"num_write_kb":5392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,10,11,5,8,3],"acting":[7,2,10,11,5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"e","length":"1"},{"start":"13","length":"1"},{"start":"18","length":"2"}]},{"pgid":"3.a","version":"66'28","reported_seq":142,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-18T10:15:26.093878+0000","last_change":"2024-09-18T10:15:22.536134+0000","last_active":"2024-09-18T10:15:26.093878+0000","last_peered":"2024-09-18T10:15:26.093878+0000","last_clean":"2024-09-18T10:15:26.093878+0000","last_became_active":"2024-09-18T10:14:46.765703+0000","last_became_peered":"2024-09-18T10:14:46.765703+0000","last_unstale":"2024-09-18T10:15:26.093878+0000","last_undegraded":"2024-09-18T10:15:26.093878+0000","last_fullsized":"2024-09-18T10:15:26.093878+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:14:45.576168+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:08.721290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.60505593199999996,"stat_sum":{"num_bytes":770048,"num_objects":4,"num_object_clones":2,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":1,"num_read":27,"num_read_kb":5220,"num_write":29,"num_write_kb":1344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,1,6,5,3,15,4],"acting":[13,1,6,5,3,15,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"e","length":"1"},{"start":"13","length":"1"},{"start":"18","length":"2"}]},{"pgid":"3.b","version":"60'21","reported_seq":118,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-18T10:15:23.959955+0000","last_change":"2024-09-18T10:15:22.446437+0000","last_active":"2024-09-18T10:15:23.959955+0000","last_peered":"2024-09-18T10:15:23.959955+0000","last_clean":"2024-09-18T10:15:23.959955+0000","last_became_active":"2024-09-18T10:14:46.802834+0000","last_became_peered":"2024-09-18T10:14:46.802834+0000","last_unstale":"2024-09-18T10:15:23.959955+0000","last_undegraded":"2024-09-18T10:15:23.959955+0000","last_fullsized":"2024-09-18T10:15:23.959955+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:14:45.576168+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:04.590511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.41299139200000001,"stat_sum":{"num_bytes":671744,"num_objects":1,"num_object_clones":0,"num_object_copies":7,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":31,"num_write_kb":1600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,10,12,11,1,14],"acting":[4,0,10,12,11,1,14],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"e","length":"1"},{"start":"13","length":"1"},{"start":"18","length":"2"}]},{"pgid":"3.c","version":"66'83","reported_seq":181,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-18T10:15:23.936682+0000","last_change":"2024-09-18T10:15:22.846529+0000","last_active":"2024-09-18T10:15:23.936682+0000","last_peered":"2024-09-18T10:15:23.936682+0000","last_clean":"2024-09-18T10:15:23.936682+0000","last_became_active":"2024-09-18T10:14:46.795995+0000","last_became_peered":"2024-09-18T10:14:46.795995+0000","last_unstale":"2024-09-18T10:15:23.936682+0000","last_undegraded":"2024-09-18T10:15:23.936682+0000","last_fullsized":"2024-09-18T10:15:23.936682+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:14:45.576168+0000","objects_scrubbed":0,"log_size":83,"log_dups_size":0,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:14.450533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.77562220400000004,"stat_sum":{"num_bytes":8519680,"num_objects":8,"num_object_clones":3,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":24,"num_read_kb":3468,"num_write":77,"num_write_kb":8992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,13,3,7,12,14],"acting":[5,8,13,3,7,12,14],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"e","length":"1"},{"start":"13","length":"1"},{"start":"18","length":"2"}]},{"pgid":"3.d","version":"66'25","reported_seq":129,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-18T10:15:25.105610+0000","last_change":"2024-09-18T10:15:22.137283+0000","last_active":"2024-09-18T10:15:25.105610+0000","last_peered":"2024-09-18T10:15:25.105610+0000","last_clean":"2024-09-18T10:15:25.105610+0000","last_became_active":"2024-09-18T10:14:46.793893+0000","last_became_peered":"2024-09-18T10:14:46.793893+0000","last_unstale":"2024-09-18T10:15:25.105610+0000","last_undegraded":"2024-09-18T10:15:25.105610+0000","last_fullsized":"2024-09-18T10:15:25.105610+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:14:45.576168+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:13.756262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.20716743200000001,"stat_sum":{"num_bytes":2899968,"num_objects":2,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":9,"num_read_kb":532,"num_write":10,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,1,11,15,9,8,4],"acting":[13,1,11,15,9,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"e","length":"1"},{"start":"13","length":"1"},{"start":"18","length":"2"}]},{"pgid":"3.e","version":"71'37","reported_seq":150,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-18T10:15:28.748183+0000","last_change":"2024-09-18T10:15:28.275984+0000","last_active":"2024-09-18T10:15:28.748183+0000","last_peered":"2024-09-18T10:15:28.748183+0000","last_clean":"2024-09-18T10:15:28.748183+0000","last_became_active":"2024-09-18T10:14:46.801571+0000","last_became_peered":"2024-09-18T10:14:46.801571+0000","last_unstale":"2024-09-18T10:15:28.748183+0000","last_undegraded":"2024-09-18T10:15:28.748183+0000","last_fullsized":"2024-09-18T10:15:28.748183+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:14:45.576168+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:15:59.433777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.10115629800000001,"stat_sum":{"num_bytes":3571712,"num_objects":4,"num_object_clones":2,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":1,"num_read":12,"num_read_kb":2405,"num_write":39,"num_write_kb":3488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,14,6,13,8,3,12],"acting":[11,14,6,13,8,3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"f","length":"1"},{"start":"21","length":"1"}]},{"pgid":"3.f","version":"66'25","reported_seq":125,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-18T10:15:25.105869+0000","last_change":"2024-09-18T10:15:22.237333+0000","last_active":"2024-09-18T10:15:25.105869+0000","last_peered":"2024-09-18T10:15:25.105869+0000","last_clean":"2024-09-18T10:15:25.105869+0000","last_became_active":"2024-09-18T10:14:46.803296+0000","last_became_peered":"2024-09-18T10:14:46.803296+0000","last_unstale":"2024-09-18T10:15:25.105869+0000","last_undegraded":"2024-09-18T10:15:25.105869+0000","last_fullsized":"2024-09-18T10:15:25.105869+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:14:45.576168+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:11.624416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.30690593500000002,"stat_sum":{"num_bytes":2129920,"num_objects":2,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":9,"num_read_kb":1780,"num_write":11,"num_write_kb":2640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,3,15,14,2,11,0],"acting":[13,3,15,14,2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"e","length":"1"},{"start":"13","length":"1"},{"start":"18","length":"2"}]},{"pgid":"3.10","version":"60'14","reported_seq":92,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-18T10:15:23.938487+0000","last_change":"2024-09-18T10:15:22.034432+0000","last_active":"2024-09-18T10:15:23.938487+0000","last_peered":"2024-09-18T10:15:23.938487+0000","last_clean":"2024-09-18T10:15:23.938487+0000","last_became_active":"2024-09-18T10:15:03.217299+0000","last_became_peered":"2024-09-18T10:15:03.217299+0000","last_unstale":"2024-09-18T10:15:23.938487+0000","last_undegraded":"2024-09-18T10:15:23.938487+0000","last_fullsized":"2024-09-18T10:15:23.938487+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:14:45.576168+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.006433032,"stat_sum":{"num_bytes":1769472,"num_objects":1,"num_object_clones":0,"num_object_copies":7,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":9,"num_read_kb":1684,"num_write":6,"num_write_kb":2048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":884736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,5,4,8,0,9],"acting":[6,7,5,4,8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"e","length":"1"},{"start":"13","length":"1"},{"start":"18","length":"2"}]},{"pgid":"3.11","version":"36'6","reported_seq":74,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-18T10:15:23.960513+0000","last_change":"2024-09-18T10:15:21.933314+0000","last_active":"2024-09-18T10:15:23.960513+0000","last_peered":"2024-09-18T10:15:23.960513+0000","last_clean":"2024-09-18T10:15:23.960513+0000","last_became_active":"2024-09-18T10:15:04.955869+0000","last_became_peered":"2024-09-18T10:15:04.955869+0000","last_unstale":"2024-09-18T10:15:23.960513+0000","last_undegraded":"2024-09-18T10:15:23.960513+0000","last_fullsized":"2024-09-18T10:15:23.960513+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":5,"last_scrub":"36'6","last_scrub_stamp":"2024-09-18T10:15:07.755163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:07.755163+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:13.113623+0000","scrub_duration":0.60886176599999997,"objects_trimmed":0,"snaptrim_duration":0.002492476,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,13,10,0,14,2],"acting":[3,5,13,10,0,14,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"e","length":"1"},{"start":"13","length":"1"},{"start":"18","length":"2"}]},{"pgid":"3.12","version":"36'8","reported_seq":151,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-18T10:15:25.100538+0000","last_change":"2024-09-18T10:15:21.925801+0000","last_active":"2024-09-18T10:15:25.100538+0000","last_peered":"2024-09-18T10:15:25.100538+0000","last_clean":"2024-09-18T10:15:25.100538+0000","last_became_active":"2024-09-18T10:15:06.903185+0000","last_became_peered":"2024-09-18T10:15:06.903185+0000","last_unstale":"2024-09-18T10:15:25.100538+0000","last_undegraded":"2024-09-18T10:15:25.100538+0000","last_fullsized":"2024-09-18T10:15:25.100538+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":5,"last_scrub":"36'8","last_scrub_stamp":"2024-09-18T10:15:04.353687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:04.353687+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:28.531412+0000","scrub_duration":0.76511709400000005,"objects_trimmed":0,"snaptrim_duration":0.0015708359999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,5,8,3,4,11,1],"acting":[14,5,8,3,4,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"e","length":"1"},{"start":"13","length":"1"},{"start":"18","length":"2"}]},{"pgid":"3.13","version":"66'41","reported_seq":132,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-18T10:15:23.962792+0000","last_change":"2024-09-18T10:15:22.039043+0000","last_active":"2024-09-18T10:15:23.962792+0000","last_peered":"2024-09-18T10:15:23.962792+0000","last_clean":"2024-09-18T10:15:23.962792+0000","last_became_active":"2024-09-18T10:15:13.263306+0000","last_became_peered":"2024-09-18T10:15:13.263306+0000","last_unstale":"2024-09-18T10:15:23.962792+0000","last_undegraded":"2024-09-18T10:15:23.962792+0000","last_fullsized":"2024-09-18T10:15:23.962792+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":5,"last_scrub":"60'33","last_scrub_stamp":"2024-09-18T10:15:19.140411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:19.140411+0000","objects_scrubbed":1,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:39.399176+0000","scrub_duration":0.706777345,"objects_trimmed":0,"snaptrim_duration":0.109764865,"stat_sum":{"num_bytes":1720320,"num_objects":2,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":1,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,13,2,6,0,4,15],"acting":[1,13,2,6,0,4,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"e","length":"1"},{"start":"13","length":"1"},{"start":"18","length":"2"}]}],"pool_stats":[{"poolid":3,"num_pg":26,"stat_sum":{"num_bytes":64102400,"num_objects":82,"num_object_clones":41,"num_object_copies":574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":8,"num_read":212,"num_read_kb":44236,"num_write":371,"num_write_kb":54376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":6012928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":117411840,"data_stored":157880320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":799,"ondisk_log_size":799,"up":182,"acting":182,"num_store_stats":16},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":112,"ondisk_log_size":112,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":15,"up_from":22,"seq":94489280525,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303260,"kb_used_data":5904,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94068580,"statfs":{"total":96636764160,"available":96326225920,"internally_reserved":0,"allocated":6045696,"data_stored":6634875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6,10,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.024}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96199999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79600000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92000000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.177}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2110000000000001}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":86.546999999999997}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3400000000000001}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.151}]}]},{"osd":14,"up_from":22,"seq":94489280525,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306248,"kb_used_data":8892,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94065592,"statfs":{"total":96636764160,"available":96323166208,"internally_reserved":0,"allocated":9105408,"data_stored":11170640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,7,8,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3149999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88600000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.417}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.101}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3999999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1299999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.427}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5489999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4770000000000001}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.667}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4990000000000001}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6930000000000001}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5720000000000001}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.863}]}]},{"osd":13,"up_from":22,"seq":94489280526,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306516,"kb_used_data":9160,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94065324,"statfs":{"total":96636764160,"available":96322891776,"internally_reserved":0,"allocated":9379840,"data_stored":11903843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6,8,9,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92100000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0489999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.405}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.101}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":22.795999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.87}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99099999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2210000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.133}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4299999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.508}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6439999999999999}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6890000000000001}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.484}]}]},{"osd":0,"up_from":21,"seq":90194313229,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303332,"kb_used_data":5968,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94068508,"statfs":{"total":96636764160,"available":96326152192,"internally_reserved":0,"allocated":6111232,"data_stored":6892379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,4,5,6,9,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84699999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1080000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.196}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95099999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2869999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.349}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.518}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5349999999999999}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.486}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3839999999999999}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6080000000000001}]}]},{"osd":4,"up_from":21,"seq":90194313231,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304304,"kb_used_data":6940,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94067536,"statfs":{"total":96636764160,"available":96325156864,"internally_reserved":0,"allocated":7106560,"data_stored":8592219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93100000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.04}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.421}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6579999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.702}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.52}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.294}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5289999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.395}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9399999999999999}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9690000000000001}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8560000000000001}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.081}]}]},{"osd":9,"up_from":21,"seq":90194313230,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":300880,"kb_used_data":3520,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94070960,"statfs":{"total":96636764160,"available":96328663040,"internally_reserved":0,"allocated":3604480,"data_stored":3470005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,8,10,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66500000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73199999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85199999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98599999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2430000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1990000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.089}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2889999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.669}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.446}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":96}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":46.162999999999997}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3360000000000001}]}]},{"osd":11,"up_from":21,"seq":90194313230,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312724,"kb_used_data":11204,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94059116,"statfs":{"total":96636764160,"available":96316534784,"internally_reserved":0,"allocated":11472896,"data_stored":15106741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[2,3,6,8,9,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0049999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":30.919}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":85.963999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80100000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":85.900000000000006}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":85.838999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68500000000000005}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84199999999999997}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":83.988}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":86.109999999999999}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88400000000000001}]}]},{"osd":12,"up_from":21,"seq":90194313231,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301492,"kb_used_data":4132,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94070348,"statfs":{"total":96636764160,"available":96328036352,"internally_reserved":0,"allocated":4231168,"data_stored":5366472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,9,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71399999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92900000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0549999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0820000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3220000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.302}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4490000000000001}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.278}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3999999999999999}]}]},{"osd":1,"up_from":22,"seq":94489280525,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301764,"kb_used_data":4400,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94070076,"statfs":{"total":96636764160,"available":96327757824,"internally_reserved":0,"allocated":4505600,"data_stored":4561755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,9,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67800000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0269999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1659999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5029999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82299999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":118.43600000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7789999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94399999999999995}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6850000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.825}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.339}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7529999999999999}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.851}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7250000000000001}]}]},{"osd":2,"up_from":22,"seq":94489280525,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306648,"kb_used_data":9284,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94065192,"statfs":{"total":96636764160,"available":96322756608,"internally_reserved":0,"allocated":9506816,"data_stored":11844443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,3,4,5,6,7,8,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.02}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1990000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92300000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.226}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4199999999999999}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5509999999999999}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3460000000000001}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.486}]}]},{"osd":3,"up_from":22,"seq":94489280525,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":309372,"kb_used_data":12016,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94062468,"statfs":{"total":96636764160,"available":96319967232,"internally_reserved":0,"allocated":12304384,"data_stored":15987536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,4,5,6,7,9,10,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1240000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.248}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4330000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5349999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4570000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.857}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0620000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9990000000000001}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.915}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6399999999999999}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0209999999999999}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.948}]}]},{"osd":5,"up_from":22,"seq":94489280525,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306024,"kb_used_data":8660,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94065816,"statfs":{"total":96636764160,"available":96323395584,"internally_reserved":0,"allocated":8867840,"data_stored":11684699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,6,7,8,9,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.874}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.012}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68200000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93700000000000006}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90800000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.222}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2509999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98199999999999998}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6479999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":12.967000000000001}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1240000000000001}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.365}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1930000000000001}]}]},{"osd":6,"up_from":22,"seq":94489280525,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306332,"kb_used_data":8968,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94065508,"statfs":{"total":96636764160,"available":96323080192,"internally_reserved":0,"allocated":9183232,"data_stored":10675568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,7,8,9,11,12,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80100000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.343}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92100000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.091}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1200000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.29}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3460000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.458}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.329}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6719999999999999}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4810000000000001}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2050000000000001}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2669999999999999}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5049999999999999}]}]},{"osd":7,"up_from":22,"seq":94489280525,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308232,"kb_used_data":10868,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94063608,"statfs":{"total":96636764160,"available":96321134592,"internally_reserved":0,"allocated":11128832,"data_stored":14244699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,8,10,11,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83999999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":11.805}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.022}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.077}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.177}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.373}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.399}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.861}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":21.081}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.8639999999999999}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.885}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":105.13}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5900000000000001}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5509999999999999}]}]},{"osd":8,"up_from":22,"seq":94489280525,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308316,"kb_used_data":10960,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94063524,"statfs":{"total":96636764160,"available":96321048576,"internally_reserved":0,"allocated":11223040,"data_stored":15467344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[2,4,6,7,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93300000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.083}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66200000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90800000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83299999999999996}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95699999999999996}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80900000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0800000000000001}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.75}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.274}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0349999999999999}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.226}]}]},{"osd":10,"up_from":22,"seq":94489280526,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307636,"kb_used_data":10284,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94064204,"statfs":{"total":96636764160,"available":96321744896,"internally_reserved":0,"allocated":10530816,"data_stored":13433691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6,7,9,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":79.319000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":79.353999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":79.171999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83999999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96099999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":79.394999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.046}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.087}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.22}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":79.495000000000005}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.198}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":66.772000000000006}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":79.284000000000006}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5800000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4378624,"data_stored":5226496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":3551232,"data_stored":4034560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8728576,"data_stored":11669504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":11509760,"data_stored":15802368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":5373952,"data_stored":6971392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8089600,"data_stored":11509760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":7680000,"data_stored":9609216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":10350592,"data_stored":14069760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":10252288,"data_stored":14929920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":3018752,"data_stored":3764224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":9752576,"data_stored":13258752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":10436608,"data_stored":14528512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":3190784,"data_stored":4788224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":8404992,"data_stored":11505664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":8134656,"data_stored":10633216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":4558848,"data_stored":5578752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T10:15:29.793 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-18T10:15:29.793 INFO:tasks.thrashosds.thrasher:choose_action: min_in 8 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T10:15:29.793 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-18T10:15:29.794 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-18T10:15:29.852 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-09-18T10:15:29.851+0000 7fdfa8698700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22087) UID: 0 2024-09-18T10:15:29.912 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-09-18T10:15:29.912 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-09-18T10:15:29.912 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-09-18T10:15:29.912 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-09-18T10:15:29.912 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-09-18T10:15:29.923 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_historic_ops 2024-09-18T10:15:29.953 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-18T10:15:29.946+0000 7f302f2c3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22187) UID: 0 2024-09-18T10:15:30.023 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-09-18T10:15:30.023 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-09-18T10:15:30.023 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-09-18T10:15:30.023 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-09-18T10:15:30.023 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.023 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4341.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-18T10:15:30.024 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:35.324351+0000", 2024-09-18T10:15:30.024 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 54.695353634, 2024-09-18T10:15:30.024 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.36640713699999999, 2024-09-18T10:15:30.024 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:30.024 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:30.024 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:30.024 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4341", 2024-09-18T10:15:30.024 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.29:0/3001591505", 2024-09-18T10:15:30.024 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 1 2024-09-18T10:15:30.024 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.024 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:30.024 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.024 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:30.025 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.324351+0000", 2024-09-18T10:15:30.025 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:30.025 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.025 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.025 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:30.025 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.324351+0000", 2024-09-18T10:15:30.025 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:30.025 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.025 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.025 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:30.025 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.324348+0000", 2024-09-18T10:15:30.025 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999976 2024-09-18T10:15:30.025 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.025 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.026 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:30.026 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.324359+0000", 2024-09-18T10:15:30.026 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1048e-05 2024-09-18T10:15:30.026 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.026 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.026 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:30.026 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.324383+0000", 2024-09-18T10:15:30.026 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.3813000000000001e-05 2024-09-18T10:15:30.026 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.026 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.026 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:30.026 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.324421+0000", 2024-09-18T10:15:30.026 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8012000000000001e-05 2024-09-18T10:15:30.027 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.027 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.027 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:30.027 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.324548+0000", 2024-09-18T10:15:30.027 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00012702199999999999 2024-09-18T10:15:30.027 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.027 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.027 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for peered", 2024-09-18T10:15:30.027 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.324565+0000", 2024-09-18T10:15:30.027 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.6325e-05 2024-09-18T10:15:30.027 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.027 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.027 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:30.027 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.588380+0000", 2024-09-18T10:15:30.028 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.263815458 2024-09-18T10:15:30.028 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.028 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.028 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:30.028 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.588626+0000", 2024-09-18T10:15:30.028 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00024633499999999998 2024-09-18T10:15:30.028 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.028 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.028 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 5", 2024-09-18T10:15:30.028 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.588803+0000", 2024-09-18T10:15:30.028 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00017640599999999999 2024-09-18T10:15:30.028 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.028 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.029 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:15:30.029 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.589943+0000", 2024-09-18T10:15:30.029 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0011400049999999999 2024-09-18T10:15:30.029 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.029 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.029 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:15:30.029 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.690623+0000", 2024-09-18T10:15:30.029 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100680032 2024-09-18T10:15:30.029 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.029 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.029 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:15:30.029 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.690702+0000", 2024-09-18T10:15:30.029 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.9342e-05 2024-09-18T10:15:30.030 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.030 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.030 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:30.033 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.690758+0000", 2024-09-18T10:15:30.033 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5639e-05 2024-09-18T10:15:30.033 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:30.033 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:30.033 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:30.033 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.033 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.033 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(osd.4.21:1 3.2s0 3:46b2871c:::smithi18925896-34:head [assert-version v6,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:30.033 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:51.536666+0000", 2024-09-18T10:15:30.034 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 38.483038254999997, 2024-09-18T10:15:30.034 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.30016440500000002, 2024-09-18T10:15:30.034 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:30.034 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "started", 2024-09-18T10:15:30.034 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:30.034 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.034 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:30.034 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.536666+0000", 2024-09-18T10:15:30.034 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:30.034 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.034 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.034 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:30.034 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.536666+0000", 2024-09-18T10:15:30.035 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:30.035 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.035 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.035 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:30.035 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.536664+0000", 2024-09-18T10:15:30.035 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:30.035 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.035 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.035 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:30.035 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.536673+0000", 2024-09-18T10:15:30.035 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.6319999999999997e-06 2024-09-18T10:15:30.035 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.035 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.035 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:30.036 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.536688+0000", 2024-09-18T10:15:30.036 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.5058999999999999e-05 2024-09-18T10:15:30.036 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.036 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.036 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:30.036 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.536696+0000", 2024-09-18T10:15:30.036 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.8860000000000004e-06 2024-09-18T10:15:30.036 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.036 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.036 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:30.036 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.536868+0000", 2024-09-18T10:15:30.036 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000172475 2024-09-18T10:15:30.036 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.036 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.037 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:30.037 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.537125+0000", 2024-09-18T10:15:30.037 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000256906 2024-09-18T10:15:30.037 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.037 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.037 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:30.037 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.836830+0000", 2024-09-18T10:15:30.037 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.299705414 2024-09-18T10:15:30.037 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:30.037 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:30.037 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:30.037 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.037 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.038 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:264 3.2s0 3:46b2871c:::smithi18925896-34:head [append 0~606208] snapc 5=[5,4,3] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-18T10:15:30.038 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:56.739704+0000", 2024-09-18T10:15:30.038 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 33.280000649000002, 2024-09-18T10:15:30.038 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.79624036899999995, 2024-09-18T10:15:30.038 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:30.038 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:30.038 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:30.038 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:15:30.038 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:30.038 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 264 2024-09-18T10:15:30.038 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.038 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:30.038 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.038 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:30.039 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.739704+0000", 2024-09-18T10:15:30.039 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:30.039 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.039 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.039 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:30.039 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.739704+0000", 2024-09-18T10:15:30.039 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:30.039 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.039 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.039 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:30.039 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.739702+0000", 2024-09-18T10:15:30.039 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:30.039 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.039 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.039 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:30.040 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.740467+0000", 2024-09-18T10:15:30.040 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00076493100000000005 2024-09-18T10:15:30.040 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.040 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.040 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:30.040 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.740485+0000", 2024-09-18T10:15:30.040 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8516e-05 2024-09-18T10:15:30.040 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.040 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.040 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:30.040 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.740495+0000", 2024-09-18T10:15:30.040 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.2669999999999995e-06 2024-09-18T10:15:30.040 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.041 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.041 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:30.041 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.028948+0000", 2024-09-18T10:15:30.041 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.28845313500000003 2024-09-18T10:15:30.041 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.041 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.041 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for peered", 2024-09-18T10:15:30.041 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.028969+0000", 2024-09-18T10:15:30.041 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.1369000000000002e-05 2024-09-18T10:15:30.041 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.041 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.041 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:30.041 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.228324+0000", 2024-09-18T10:15:30.042 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.19935476699999999 2024-09-18T10:15:30.042 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.042 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.042 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:30.042 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.228558+0000", 2024-09-18T10:15:30.042 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00023393099999999999 2024-09-18T10:15:30.042 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.042 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.042 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:30.042 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.230390+0000", 2024-09-18T10:15:30.042 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018326970000000001 2024-09-18T10:15:30.042 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.042 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.043 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:30.043 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.239491+0000", 2024-09-18T10:15:30.043 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0091007950000000001 2024-09-18T10:15:30.043 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.043 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.043 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:15:30.043 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.535834+0000", 2024-09-18T10:15:30.043 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.29634234100000001 2024-09-18T10:15:30.043 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.043 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.043 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:30.043 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.535944+0000", 2024-09-18T10:15:30.043 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000110401 2024-09-18T10:15:30.043 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:30.043 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:30.044 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:30.044 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.044 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.044 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:265 3.2s0 3:46b2871c:::smithi18925896-34:head [append 0~376832] snapc 5=[5,4,3] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-18T10:15:30.044 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:56.741882+0000", 2024-09-18T10:15:30.044 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 33.277822626999999, 2024-09-18T10:15:30.044 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.80071129500000005, 2024-09-18T10:15:30.044 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:30.044 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:30.045 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:30.045 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:15:30.045 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:30.045 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 265 2024-09-18T10:15:30.045 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.045 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:30.045 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.045 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:30.045 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.741882+0000", 2024-09-18T10:15:30.045 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:30.045 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.045 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.045 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:30.046 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.741882+0000", 2024-09-18T10:15:30.046 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:30.046 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.046 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.046 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:30.046 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.741880+0000", 2024-09-18T10:15:30.046 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:30.046 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.046 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.046 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:30.046 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742251+0000", 2024-09-18T10:15:30.046 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00037050499999999998 2024-09-18T10:15:30.046 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.046 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.047 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:30.047 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742264+0000", 2024-09-18T10:15:30.047 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2787e-05 2024-09-18T10:15:30.047 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.047 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.047 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:30.047 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742269+0000", 2024-09-18T10:15:30.047 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.0509999999999996e-06 2024-09-18T10:15:30.047 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.047 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.047 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:30.047 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.234342+0000", 2024-09-18T10:15:30.047 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.492073712 2024-09-18T10:15:30.048 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.048 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.048 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:30.048 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.234556+0000", 2024-09-18T10:15:30.048 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00021380699999999999 2024-09-18T10:15:30.048 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.048 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.048 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:30.048 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.236316+0000", 2024-09-18T10:15:30.048 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0017597100000000001 2024-09-18T10:15:30.048 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.048 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.048 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:30.049 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.241476+0000", 2024-09-18T10:15:30.049 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0051604449999999996 2024-09-18T10:15:30.049 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.049 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.049 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:15:30.049 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.542502+0000", 2024-09-18T10:15:30.049 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.30102573700000002 2024-09-18T10:15:30.049 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.049 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.049 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:30.049 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.542593+0000", 2024-09-18T10:15:30.049 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.0859000000000005e-05 2024-09-18T10:15:30.049 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:30.050 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:30.050 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:30.050 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.050 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.050 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:266 3.2s0 3:46b2871c:::smithi18925896-34:head [setxattr _header (56) in=7b] snapc 5=[5,4,3] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-18T10:15:30.050 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:56.742274+0000", 2024-09-18T10:15:30.050 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 33.277429996000002, 2024-09-18T10:15:30.050 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.80056005299999999, 2024-09-18T10:15:30.050 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:30.050 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:30.050 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:30.050 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:15:30.050 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:30.051 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 266 2024-09-18T10:15:30.051 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.051 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:30.051 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.051 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:30.051 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742274+0000", 2024-09-18T10:15:30.051 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:30.051 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.051 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.051 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:30.051 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742274+0000", 2024-09-18T10:15:30.051 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:30.051 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.051 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.051 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:30.052 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742274+0000", 2024-09-18T10:15:30.052 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:30.052 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.052 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.052 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:30.052 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742278+0000", 2024-09-18T10:15:30.052 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.0840000000000002e-06 2024-09-18T10:15:30.052 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.052 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.052 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:30.052 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742284+0000", 2024-09-18T10:15:30.052 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.1090000000000003e-06 2024-09-18T10:15:30.052 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.053 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.053 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:30.053 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742289+0000", 2024-09-18T10:15:30.053 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.4890000000000003e-06 2024-09-18T10:15:30.053 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.053 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.053 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:30.053 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.237660+0000", 2024-09-18T10:15:30.053 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.49537068299999998 2024-09-18T10:15:30.053 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.053 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.053 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:30.053 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.237924+0000", 2024-09-18T10:15:30.053 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00026387300000000003 2024-09-18T10:15:30.054 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.054 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.054 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:30.054 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.238979+0000", 2024-09-18T10:15:30.054 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0010549380000000001 2024-09-18T10:15:30.054 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.054 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.054 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:30.054 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.243571+0000", 2024-09-18T10:15:30.054 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0045919749999999999 2024-09-18T10:15:30.054 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.054 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.054 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:15:30.054 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.542742+0000", 2024-09-18T10:15:30.054 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.29917095999999999 2024-09-18T10:15:30.055 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.055 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.055 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:30.055 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.542834+0000", 2024-09-18T10:15:30.055 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.2536999999999998e-05 2024-09-18T10:15:30.055 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:30.055 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:30.055 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:30.055 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.055 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.055 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:267 3.2s0 3:46b2871c:::smithi18925896-34:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e41)", 2024-09-18T10:15:30.055 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:56.742292+0000", 2024-09-18T10:15:30.055 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 33.277412157999997, 2024-09-18T10:15:30.056 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.98973012599999999, 2024-09-18T10:15:30.056 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:30.056 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "started", 2024-09-18T10:15:30.056 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:30.056 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:15:30.056 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:30.056 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 267 2024-09-18T10:15:30.056 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.056 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:30.056 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.056 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:30.056 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742292+0000", 2024-09-18T10:15:30.056 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:30.057 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.057 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.057 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:30.057 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742292+0000", 2024-09-18T10:15:30.057 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:30.057 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.057 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.057 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:30.057 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742292+0000", 2024-09-18T10:15:30.057 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:30.057 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.057 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.057 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:30.057 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742294+0000", 2024-09-18T10:15:30.058 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.103e-06 2024-09-18T10:15:30.058 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.058 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.058 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:30.058 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742298+0000", 2024-09-18T10:15:30.058 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8070000000000001e-06 2024-09-18T10:15:30.058 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.058 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.058 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:30.058 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742300+0000", 2024-09-18T10:15:30.058 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5890000000000001e-06 2024-09-18T10:15:30.058 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.058 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.058 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:30.059 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.239711+0000", 2024-09-18T10:15:30.059 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.497410625 2024-09-18T10:15:30.059 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.059 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.059 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for rw locks", 2024-09-18T10:15:30.059 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.239786+0000", 2024-09-18T10:15:30.059 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.4827999999999996e-05 2024-09-18T10:15:30.059 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.059 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.059 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:30.059 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.543234+0000", 2024-09-18T10:15:30.059 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.30344799099999997 2024-09-18T10:15:30.059 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.059 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.059 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:30.060 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.543335+0000", 2024-09-18T10:15:30.060 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000101625 2024-09-18T10:15:30.060 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.060 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.060 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:30.060 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.732022+0000", 2024-09-18T10:15:30.060 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.18868686700000001 2024-09-18T10:15:30.060 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:30.060 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:30.060 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:30.060 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.060 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.060 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.es1 41/32 ECSubWrite(tid=16, reqid=osd.11.0:15, at_version=0'0, trim_to=0'0, roll_forward_to=41'11))", 2024-09-18T10:15:30.061 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:56.930285+0000", 2024-09-18T10:15:30.061 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 33.089419245000002, 2024-09-18T10:15:30.061 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.211817538, 2024-09-18T10:15:30.061 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:30.061 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:15:30.061 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:30.061 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.061 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:30.061 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.930285+0000", 2024-09-18T10:15:30.061 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:30.061 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.061 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.061 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:30.062 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.930285+0000", 2024-09-18T10:15:30.062 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:30.062 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.062 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.062 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:30.062 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.930284+0000", 2024-09-18T10:15:30.062 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:30.062 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.062 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.062 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:30.062 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.930313+0000", 2024-09-18T10:15:30.062 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9394999999999999e-05 2024-09-18T10:15:30.062 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.062 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.063 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:30.063 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.930337+0000", 2024-09-18T10:15:30.063 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4239e-05 2024-09-18T10:15:30.063 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.063 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.063 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:30.063 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.930352+0000", 2024-09-18T10:15:30.063 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4577e-05 2024-09-18T10:15:30.063 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.063 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.063 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:30.063 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.030572+0000", 2024-09-18T10:15:30.063 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100219771 2024-09-18T10:15:30.063 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.064 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.064 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:30.064 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.030604+0000", 2024-09-18T10:15:30.064 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2654999999999998e-05 2024-09-18T10:15:30.064 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.064 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.064 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:30.064 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.142047+0000", 2024-09-18T10:15:30.064 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.111442992 2024-09-18T10:15:30.064 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.064 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.064 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:30.064 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.142103+0000", 2024-09-18T10:15:30.064 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5291999999999998e-05 2024-09-18T10:15:30.065 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:30.065 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:30.065 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:30.065 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.065 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.065 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(osd.7.22:2 3.2s0 3:427b2c0b:::smithi18925896-42:head [assert-version v8,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-18T10:15:30.065 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:08.756861+0000", 2024-09-18T10:15:30.065 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 21.262843432, 2024-09-18T10:15:30.065 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.27903143600000002, 2024-09-18T10:15:30.065 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:30.065 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "started", 2024-09-18T10:15:30.065 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:30.065 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.065 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:30.066 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:08.756861+0000", 2024-09-18T10:15:30.066 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:30.066 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.066 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.066 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:30.066 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:08.756861+0000", 2024-09-18T10:15:30.066 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:30.066 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.066 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.066 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:30.066 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:08.756859+0000", 2024-09-18T10:15:30.066 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999976 2024-09-18T10:15:30.066 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.066 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.067 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:30.067 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:08.756869+0000", 2024-09-18T10:15:30.067 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0389e-05 2024-09-18T10:15:30.067 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.067 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.067 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:30.067 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:08.756887+0000", 2024-09-18T10:15:30.067 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8094e-05 2024-09-18T10:15:30.067 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.067 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.067 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:30.067 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:08.756896+0000", 2024-09-18T10:15:30.067 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.9290000000000004e-06 2024-09-18T10:15:30.067 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.068 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.068 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:30.068 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:08.757008+0000", 2024-09-18T10:15:30.068 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00011184 2024-09-18T10:15:30.068 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.068 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.068 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op must wait for map", 2024-09-18T10:15:30.068 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:08.757025+0000", 2024-09-18T10:15:30.068 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.6912999999999999e-05 2024-09-18T10:15:30.068 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.068 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.068 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:30.068 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:08.931998+0000", 2024-09-18T10:15:30.068 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.174973289 2024-09-18T10:15:30.069 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.069 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.069 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:30.069 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:08.932274+0000", 2024-09-18T10:15:30.069 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00027573799999999998 2024-09-18T10:15:30.069 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.069 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.069 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:30.069 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:09.035892+0000", 2024-09-18T10:15:30.069 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10361846700000001 2024-09-18T10:15:30.069 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:30.069 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:30.069 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:30.070 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.070 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.070 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:452 3.2s0 3:47202842:::smithi18925896-22:head [delete] snapc 13=[12,11,10,f,e,b] ondisk+write+known_if_redirected+full_force+supports_pool_eio e58)", 2024-09-18T10:15:30.070 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:13.551955+0000", 2024-09-18T10:15:30.070 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 16.467749144999999, 2024-09-18T10:15:30.070 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.31127974800000002, 2024-09-18T10:15:30.070 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:30.070 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:30.070 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:30.070 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:15:30.070 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:30.070 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 452 2024-09-18T10:15:30.070 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.070 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:30.071 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.071 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:30.071 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.551955+0000", 2024-09-18T10:15:30.071 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:30.071 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.071 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.071 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:30.071 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.551955+0000", 2024-09-18T10:15:30.071 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:30.071 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.071 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.071 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:30.071 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.551952+0000", 2024-09-18T10:15:30.071 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999967 2024-09-18T10:15:30.072 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.072 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.072 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:30.072 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.551970+0000", 2024-09-18T10:15:30.072 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8366999999999999e-05 2024-09-18T10:15:30.072 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.072 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.072 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:30.072 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.552016+0000", 2024-09-18T10:15:30.072 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.5868000000000002e-05 2024-09-18T10:15:30.072 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.072 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.072 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:30.073 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.552037+0000", 2024-09-18T10:15:30.073 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.1311999999999999e-05 2024-09-18T10:15:30.073 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.073 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.073 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:30.073 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.560400+0000", 2024-09-18T10:15:30.073 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0083625920000000003 2024-09-18T10:15:30.073 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.073 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.073 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op must wait for map", 2024-09-18T10:15:30.073 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.560411+0000", 2024-09-18T10:15:30.073 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1043000000000001e-05 2024-09-18T10:15:30.073 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.073 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.073 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:30.074 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.562108+0000", 2024-09-18T10:15:30.074 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0016968479999999999 2024-09-18T10:15:30.074 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.074 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.074 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:30.074 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.562286+0000", 2024-09-18T10:15:30.074 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00017818599999999999 2024-09-18T10:15:30.074 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.074 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.074 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:30.074 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.563336+0000", 2024-09-18T10:15:30.074 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001050051 2024-09-18T10:15:30.074 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.075 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.075 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:30.075 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.565490+0000", 2024-09-18T10:15:30.075 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0021545119999999999 2024-09-18T10:15:30.075 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.075 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.075 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:15:30.075 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.863047+0000", 2024-09-18T10:15:30.075 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.29755694199999999 2024-09-18T10:15:30.075 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.075 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.075 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:30.075 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.863235+0000", 2024-09-18T10:15:30.075 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000187524 2024-09-18T10:15:30.076 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:30.076 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:30.076 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:30.076 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.076 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.076 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDPGPushReply(3.15s0 61/60 [PushReplyOp(3:acecabac:::smithi18925896-4:head)])", 2024-09-18T10:15:30.076 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:17.756036+0000", 2024-09-18T10:15:30.076 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 12.263668443, 2024-09-18T10:15:30.076 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.30209286600000002, 2024-09-18T10:15:30.076 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:30.076 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:15:30.076 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:30.076 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.077 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:30.077 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.756036+0000", 2024-09-18T10:15:30.077 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:30.077 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.077 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.077 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:30.077 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.756036+0000", 2024-09-18T10:15:30.077 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:30.077 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.077 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.077 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:30.077 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.756035+0000", 2024-09-18T10:15:30.077 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:15:30.077 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.078 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.078 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:30.078 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.756042+0000", 2024-09-18T10:15:30.078 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.6630000000000004e-06 2024-09-18T10:15:30.078 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.078 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.078 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:30.078 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.756057+0000", 2024-09-18T10:15:30.078 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.5719e-05 2024-09-18T10:15:30.078 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.078 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.078 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:30.078 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.756068+0000", 2024-09-18T10:15:30.078 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0207000000000001e-05 2024-09-18T10:15:30.079 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.079 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.079 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:30.079 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:18.058067+0000", 2024-09-18T10:15:30.079 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.30199969599999998 2024-09-18T10:15:30.079 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.079 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.079 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:30.079 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:18.058129+0000", 2024-09-18T10:15:30.079 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.1303999999999994e-05 2024-09-18T10:15:30.079 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:30.079 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:30.079 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:30.079 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.080 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.080 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:496 3.2s0 3:44cb9e27:::smithi18925896-5:head [read 0~81920,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T10:15:30.080 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:17.889532+0000", 2024-09-18T10:15:30.080 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 12.130172539, 2024-09-18T10:15:30.080 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.32688004999999998, 2024-09-18T10:15:30.080 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:30.080 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "started", 2024-09-18T10:15:30.080 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:30.080 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:15:30.080 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:30.080 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 496 2024-09-18T10:15:30.080 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.080 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:30.080 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.081 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:30.081 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889532+0000", 2024-09-18T10:15:30.081 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:30.081 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.081 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.081 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:30.081 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889532+0000", 2024-09-18T10:15:30.081 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:30.081 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.081 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.081 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:30.081 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889529+0000", 2024-09-18T10:15:30.081 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999971 2024-09-18T10:15:30.081 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.082 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.082 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:30.082 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889546+0000", 2024-09-18T10:15:30.082 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.6690000000000001e-05 2024-09-18T10:15:30.082 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.082 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.082 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:30.082 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889580+0000", 2024-09-18T10:15:30.082 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.4638999999999998e-05 2024-09-18T10:15:30.082 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.082 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.082 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:30.082 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889596+0000", 2024-09-18T10:15:30.083 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.5571999999999999e-05 2024-09-18T10:15:30.083 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.083 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.083 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:30.083 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.989817+0000", 2024-09-18T10:15:30.083 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100221713 2024-09-18T10:15:30.083 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.083 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.083 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op must wait for map", 2024-09-18T10:15:30.083 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.989836+0000", 2024-09-18T10:15:30.083 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8516e-05 2024-09-18T10:15:30.083 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.083 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.083 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:30.084 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.990420+0000", 2024-09-18T10:15:30.084 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000583918 2024-09-18T10:15:30.084 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.084 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.084 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:30.084 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.990693+0000", 2024-09-18T10:15:30.084 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00027301499999999999 2024-09-18T10:15:30.084 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.084 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.084 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:30.084 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:18.216412+0000", 2024-09-18T10:15:30.084 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.22571880699999999 2024-09-18T10:15:30.084 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:30.085 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:30.085 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:30.085 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.085 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.085 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:497 3.2s0 3:44cb9e27:::smithi18925896-5:head [read 0~81920,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T10:15:30.085 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:17.889627+0000", 2024-09-18T10:15:30.085 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 12.130077622, 2024-09-18T10:15:30.085 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.32738385800000003, 2024-09-18T10:15:30.085 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:30.085 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "started", 2024-09-18T10:15:30.085 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:30.085 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:15:30.085 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:30.085 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 497 2024-09-18T10:15:30.086 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.086 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:30.086 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.086 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:30.086 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889627+0000", 2024-09-18T10:15:30.086 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:30.086 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.086 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.086 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:30.086 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889627+0000", 2024-09-18T10:15:30.086 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:30.086 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.086 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.086 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:30.087 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889625+0000", 2024-09-18T10:15:30.087 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:15:30.087 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.087 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.087 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:30.087 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889633+0000", 2024-09-18T10:15:30.087 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.4170000000000003e-06 2024-09-18T10:15:30.087 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.087 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.087 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:30.087 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889664+0000", 2024-09-18T10:15:30.087 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0589000000000001e-05 2024-09-18T10:15:30.087 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.088 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.088 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:30.088 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889672+0000", 2024-09-18T10:15:30.088 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.3729999999999993e-06 2024-09-18T10:15:30.088 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.088 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.088 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:30.088 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.990017+0000", 2024-09-18T10:15:30.088 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100345608 2024-09-18T10:15:30.088 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.088 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.088 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:30.088 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.990046+0000", 2024-09-18T10:15:30.088 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8912999999999999e-05 2024-09-18T10:15:30.089 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.089 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.089 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:30.089 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.991459+0000", 2024-09-18T10:15:30.089 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0014122340000000001 2024-09-18T10:15:30.089 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.089 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.089 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:30.089 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.991631+0000", 2024-09-18T10:15:30.089 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000172029 2024-09-18T10:15:30.089 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.089 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.089 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:30.089 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:18.217010+0000", 2024-09-18T10:15:30.090 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.22537980499999999 2024-09-18T10:15:30.090 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:30.090 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:30.090 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:30.090 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.090 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.090 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:498 3.2s0 3:44cb9e27:::smithi18925896-5:head [read 0~81920,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T10:15:30.090 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:17.889709+0000", 2024-09-18T10:15:30.090 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 12.129995264, 2024-09-18T10:15:30.090 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.32775850600000001, 2024-09-18T10:15:30.090 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:30.090 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "started", 2024-09-18T10:15:30.090 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:30.091 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:15:30.091 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:30.091 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 498 2024-09-18T10:15:30.091 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.091 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:30.091 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.091 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:30.091 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889709+0000", 2024-09-18T10:15:30.091 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:30.091 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.091 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.091 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:30.091 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889709+0000", 2024-09-18T10:15:30.091 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:30.092 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.092 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.092 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:30.092 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889708+0000", 2024-09-18T10:15:30.092 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:15:30.092 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.092 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.092 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:30.092 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889716+0000", 2024-09-18T10:15:30.092 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.0989999999999999e-06 2024-09-18T10:15:30.092 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.092 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.092 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:30.092 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889732+0000", 2024-09-18T10:15:30.093 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.6093e-05 2024-09-18T10:15:30.093 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.093 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.093 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:30.093 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889742+0000", 2024-09-18T10:15:30.093 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.4599999999999992e-06 2024-09-18T10:15:30.093 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.093 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.093 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:30.094 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.992175+0000", 2024-09-18T10:15:30.094 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10243377200000001 2024-09-18T10:15:30.094 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.094 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.094 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:30.094 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.992333+0000", 2024-09-18T10:15:30.094 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00015753499999999999 2024-09-18T10:15:30.094 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.094 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.094 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:30.094 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:18.217467+0000", 2024-09-18T10:15:30.094 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.22513459999999999 2024-09-18T10:15:30.094 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:30.094 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:30.095 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:30.095 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.095 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.095 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:516 3.2s0 3:44cb9e27:::smithi18925896-5:head [append 0~425984] snapc 1a=[1a,19,18,16,12,11,10,f,e] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-18T10:15:30.095 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:19.430973+0000", 2024-09-18T10:15:30.095 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 10.588730875, 2024-09-18T10:15:30.095 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.59812562800000002, 2024-09-18T10:15:30.095 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:30.095 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:30.095 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:30.095 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:15:30.095 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:30.095 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 516 2024-09-18T10:15:30.095 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.095 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:30.096 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.096 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:30.096 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.430973+0000", 2024-09-18T10:15:30.096 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:30.096 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.096 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.096 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:30.096 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.430973+0000", 2024-09-18T10:15:30.096 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:30.096 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.096 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.096 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:30.096 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.430970+0000", 2024-09-18T10:15:30.096 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999967 2024-09-18T10:15:30.097 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.097 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.097 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:30.097 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.431995+0000", 2024-09-18T10:15:30.097 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001024896 2024-09-18T10:15:30.097 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.097 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.097 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:30.097 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.432030+0000", 2024-09-18T10:15:30.097 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5241999999999999e-05 2024-09-18T10:15:30.097 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.097 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.097 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:30.097 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.432044+0000", 2024-09-18T10:15:30.098 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.411e-05 2024-09-18T10:15:30.098 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.098 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.098 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:30.098 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.432226+0000", 2024-09-18T10:15:30.098 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000181841 2024-09-18T10:15:30.098 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.098 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.098 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:30.098 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.432421+0000", 2024-09-18T10:15:30.098 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000194527 2024-09-18T10:15:30.098 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.098 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.098 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:30.099 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.434150+0000", 2024-09-18T10:15:30.099 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001729742 2024-09-18T10:15:30.099 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.099 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.099 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:30.099 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.436513+0000", 2024-09-18T10:15:30.099 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0023622299999999999 2024-09-18T10:15:30.099 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.099 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.099 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:15:30.099 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:20.029005+0000", 2024-09-18T10:15:30.099 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.59249230100000005 2024-09-18T10:15:30.099 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.099 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.100 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:30.100 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:20.029099+0000", 2024-09-18T10:15:30.100 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.4042000000000005e-05 2024-09-18T10:15:30.100 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:30.100 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:30.100 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:30.100 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.100 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.100 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:517 3.2s0 3:44cb9e27:::smithi18925896-5:head [append 0~786432] snapc 1a=[1a,19,18,16,12,11,10,f,e] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-18T10:15:30.101 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:19.439831+0000", 2024-09-18T10:15:30.101 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 10.579873336, 2024-09-18T10:15:30.101 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.58958403800000003, 2024-09-18T10:15:30.101 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:30.101 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:30.101 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:30.101 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:15:30.101 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:30.101 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 517 2024-09-18T10:15:30.101 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.101 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:30.101 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.101 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:30.101 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.439831+0000", 2024-09-18T10:15:30.102 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:30.102 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.102 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.102 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:30.102 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.439831+0000", 2024-09-18T10:15:30.102 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:30.102 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.102 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.102 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:30.102 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.439828+0000", 2024-09-18T10:15:30.102 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999976 2024-09-18T10:15:30.102 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.102 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.102 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:30.103 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.441109+0000", 2024-09-18T10:15:30.103 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001280988 2024-09-18T10:15:30.103 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.103 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.103 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:30.103 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.441168+0000", 2024-09-18T10:15:30.103 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.8869000000000002e-05 2024-09-18T10:15:30.103 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.103 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.103 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:30.103 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.541302+0000", 2024-09-18T10:15:30.103 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100134055 2024-09-18T10:15:30.103 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.103 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.104 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:30.104 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.637962+0000", 2024-09-18T10:15:30.104 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.096659682999999996 2024-09-18T10:15:30.104 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.104 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.104 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:30.104 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.638164+0000", 2024-09-18T10:15:30.104 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00020226700000000001 2024-09-18T10:15:30.104 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.104 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.104 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:30.104 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.639933+0000", 2024-09-18T10:15:30.104 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001768501 2024-09-18T10:15:30.104 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.105 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.105 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:30.105 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.646572+0000", 2024-09-18T10:15:30.105 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.006639069 2024-09-18T10:15:30.105 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.105 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.105 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:15:30.105 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:20.029341+0000", 2024-09-18T10:15:30.105 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.38276939599999998 2024-09-18T10:15:30.105 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.105 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.105 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:30.105 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:20.029415+0000", 2024-09-18T10:15:30.105 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.3726000000000005e-05 2024-09-18T10:15:30.106 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:30.106 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:30.106 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:30.106 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.106 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.106 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:518 3.2s0 3:44cb9e27:::smithi18925896-5:head [append 0~376832] snapc 1a=[1a,19,18,16,12,11,10,f,e] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-18T10:15:30.106 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:19.541352+0000", 2024-09-18T10:15:30.106 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 10.478352298000001, 2024-09-18T10:15:30.106 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.48828847600000003, 2024-09-18T10:15:30.106 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:30.106 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:30.106 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:30.106 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:15:30.106 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:30.107 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 518 2024-09-18T10:15:30.107 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.107 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:30.107 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.107 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:30.107 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.541352+0000", 2024-09-18T10:15:30.107 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:30.107 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.107 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.107 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:30.107 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.541352+0000", 2024-09-18T10:15:30.107 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:30.107 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.107 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.108 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:30.108 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.541350+0000", 2024-09-18T10:15:30.108 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:30.108 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.108 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.108 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:30.108 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.541834+0000", 2024-09-18T10:15:30.108 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00048402500000000002 2024-09-18T10:15:30.108 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.108 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.108 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:30.108 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.541861+0000", 2024-09-18T10:15:30.108 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.7333000000000001e-05 2024-09-18T10:15:30.108 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.109 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.109 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:30.109 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.541875+0000", 2024-09-18T10:15:30.109 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4025999999999999e-05 2024-09-18T10:15:30.109 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.109 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.109 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:30.109 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.641684+0000", 2024-09-18T10:15:30.109 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.099808872000000007 2024-09-18T10:15:30.109 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.109 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.109 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:30.109 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.641873+0000", 2024-09-18T10:15:30.110 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00018933200000000001 2024-09-18T10:15:30.110 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.110 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.110 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:30.110 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.643401+0000", 2024-09-18T10:15:30.110 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001527104 2024-09-18T10:15:30.110 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.110 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.110 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:30.110 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.646854+0000", 2024-09-18T10:15:30.110 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0034532069999999998 2024-09-18T10:15:30.110 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.110 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.110 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:15:30.111 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:20.029569+0000", 2024-09-18T10:15:30.111 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.38271509599999998 2024-09-18T10:15:30.111 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.111 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.111 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:30.111 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:20.029640+0000", 2024-09-18T10:15:30.111 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.1594000000000006e-05 2024-09-18T10:15:30.111 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:30.111 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:30.111 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:30.111 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.111 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.111 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:519 3.2s0 3:44cb9e27:::smithi18925896-5:head [setxattr _header (56) in=7b] snapc 1a=[1a,19,18,16,12,11,10,f,e] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-18T10:15:30.111 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:19.541886+0000", 2024-09-18T10:15:30.112 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 10.477818611, 2024-09-18T10:15:30.112 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.488165768, 2024-09-18T10:15:30.112 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:30.112 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:30.112 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:30.112 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:15:30.112 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:30.112 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 519 2024-09-18T10:15:30.112 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.113 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:30.113 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.113 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:30.113 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.541886+0000", 2024-09-18T10:15:30.113 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:30.113 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.113 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.113 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:30.113 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.541886+0000", 2024-09-18T10:15:30.113 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:30.113 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.113 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.113 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:30.113 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.541884+0000", 2024-09-18T10:15:30.114 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:30.114 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.114 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.114 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:30.114 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.541890+0000", 2024-09-18T10:15:30.114 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.0589999999999999e-06 2024-09-18T10:15:30.114 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.114 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.114 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:30.114 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.541899+0000", 2024-09-18T10:15:30.114 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.02e-06 2024-09-18T10:15:30.114 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.114 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.114 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:30.115 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.541905+0000", 2024-09-18T10:15:30.115 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.9399999999999999e-06 2024-09-18T10:15:30.115 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.115 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.115 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:30.115 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.644561+0000", 2024-09-18T10:15:30.115 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.102655408 2024-09-18T10:15:30.115 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.115 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.115 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:30.115 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.644869+0000", 2024-09-18T10:15:30.115 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00030766700000000002 2024-09-18T10:15:30.115 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.115 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.116 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:30.116 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.645745+0000", 2024-09-18T10:15:30.116 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000876314 2024-09-18T10:15:30.116 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.116 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.116 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:30.116 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.647092+0000", 2024-09-18T10:15:30.116 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0013469269999999999 2024-09-18T10:15:30.116 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.116 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.116 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:15:30.116 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:20.029901+0000", 2024-09-18T10:15:30.116 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.38280895199999998 2024-09-18T10:15:30.116 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.117 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.117 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:30.117 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:20.030051+0000", 2024-09-18T10:15:30.117 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00015067699999999999 2024-09-18T10:15:30.117 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:30.117 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:30.117 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:30.117 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.117 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.117 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:572 3.2s0 3:46b2871c:::smithi18925896-34:head [read 0~557056,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-18T10:15:30.117 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:24.942246+0000", 2024-09-18T10:15:30.117 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 5.0774578540000004, 2024-09-18T10:15:30.117 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.262248706, 2024-09-18T10:15:30.118 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:30.118 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "started", 2024-09-18T10:15:30.118 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:30.118 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:15:30.118 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:30.118 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 572 2024-09-18T10:15:30.118 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.118 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:30.118 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.118 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:30.118 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.942246+0000", 2024-09-18T10:15:30.118 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:30.118 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.118 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.119 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:30.119 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.942246+0000", 2024-09-18T10:15:30.119 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:30.119 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.119 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.119 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:30.119 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.942243+0000", 2024-09-18T10:15:30.119 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999971 2024-09-18T10:15:30.119 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.119 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.119 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:30.119 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.942262+0000", 2024-09-18T10:15:30.119 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8303000000000001e-05 2024-09-18T10:15:30.120 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.120 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.120 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:30.120 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.942298+0000", 2024-09-18T10:15:30.120 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.6396000000000002e-05 2024-09-18T10:15:30.120 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.120 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.120 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:30.120 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.942313+0000", 2024-09-18T10:15:30.120 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.5421999999999999e-05 2024-09-18T10:15:30.120 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.120 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.120 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:30.121 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.942453+0000", 2024-09-18T10:15:30.121 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00014003900000000001 2024-09-18T10:15:30.121 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.121 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.121 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op must wait for map", 2024-09-18T10:15:30.121 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.942471+0000", 2024-09-18T10:15:30.121 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7859999999999998e-05 2024-09-18T10:15:30.121 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.121 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.121 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:30.121 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:25.100684+0000", 2024-09-18T10:15:30.121 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.15821273599999999 2024-09-18T10:15:30.121 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.121 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.122 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:30.122 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:25.100921+0000", 2024-09-18T10:15:30.122 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000236963 2024-09-18T10:15:30.122 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.122 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.122 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:30.122 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:25.204495+0000", 2024-09-18T10:15:30.122 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10357403 2024-09-18T10:15:30.122 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:30.122 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:30.122 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:30.122 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.122 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.122 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:573 3.2s0 3:46b2871c:::smithi18925896-34:head [read 0~557056,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-18T10:15:30.123 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:24.942342+0000", 2024-09-18T10:15:30.123 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 5.0773618970000003, 2024-09-18T10:15:30.123 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.36312128700000001, 2024-09-18T10:15:30.123 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:30.123 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "started", 2024-09-18T10:15:30.123 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:30.123 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:15:30.123 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:30.123 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 573 2024-09-18T10:15:30.123 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.123 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:30.123 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.123 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:30.123 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.942342+0000", 2024-09-18T10:15:30.124 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:30.124 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.124 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.124 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:30.124 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.942342+0000", 2024-09-18T10:15:30.124 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:30.124 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.124 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.124 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:30.124 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.942341+0000", 2024-09-18T10:15:30.124 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:15:30.124 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.124 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.124 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:30.125 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.942363+0000", 2024-09-18T10:15:30.125 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.1406000000000001e-05 2024-09-18T10:15:30.125 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.125 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.125 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:30.125 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.942374+0000", 2024-09-18T10:15:30.125 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1466e-05 2024-09-18T10:15:30.125 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.125 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.125 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:30.125 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.942381+0000", 2024-09-18T10:15:30.125 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.9530000000000002e-06 2024-09-18T10:15:30.125 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.125 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.126 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:30.126 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.942654+0000", 2024-09-18T10:15:30.126 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000272692 2024-09-18T10:15:30.126 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.126 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.126 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:30.126 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.942672+0000", 2024-09-18T10:15:30.126 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7643e-05 2024-09-18T10:15:30.126 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.126 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.126 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:30.126 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:25.101730+0000", 2024-09-18T10:15:30.126 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.15905815800000001 2024-09-18T10:15:30.126 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.127 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.127 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:30.127 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:25.101881+0000", 2024-09-18T10:15:30.127 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000151299 2024-09-18T10:15:30.127 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.127 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.127 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:30.127 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:25.305464+0000", 2024-09-18T10:15:30.127 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.203582656 2024-09-18T10:15:30.127 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:30.127 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:30.127 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:30.127 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.127 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.128 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:574 3.2s0 3:46b2871c:::smithi18925896-34:head [read 0~557056,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-18T10:15:30.128 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:24.942392+0000", 2024-09-18T10:15:30.128 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 5.0773121010000004, 2024-09-18T10:15:30.128 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.36415850300000002, 2024-09-18T10:15:30.128 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:30.128 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "started", 2024-09-18T10:15:30.128 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:30.128 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:15:30.128 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:30.128 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 574 2024-09-18T10:15:30.128 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.128 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:30.128 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.129 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:30.129 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.942392+0000", 2024-09-18T10:15:30.129 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:30.129 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.129 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.129 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:30.129 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.942392+0000", 2024-09-18T10:15:30.129 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:30.129 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.129 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.129 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:30.129 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.942391+0000", 2024-09-18T10:15:30.129 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:15:30.129 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.129 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.130 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:30.130 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.942396+0000", 2024-09-18T10:15:30.130 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.6759999999999998e-06 2024-09-18T10:15:30.130 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.130 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.130 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:30.130 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.942407+0000", 2024-09-18T10:15:30.130 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0686999999999999e-05 2024-09-18T10:15:30.130 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.130 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.130 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:30.130 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.942413+0000", 2024-09-18T10:15:30.130 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.1659999999999999e-06 2024-09-18T10:15:30.130 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.131 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.131 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:30.131 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.942750+0000", 2024-09-18T10:15:30.131 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00033731700000000001 2024-09-18T10:15:30.131 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.131 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.131 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:30.131 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.942776+0000", 2024-09-18T10:15:30.131 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6342999999999998e-05 2024-09-18T10:15:30.131 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.131 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.131 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:30.131 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:25.102396+0000", 2024-09-18T10:15:30.131 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.15961924199999999 2024-09-18T10:15:30.132 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.132 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.132 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:30.132 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:25.102659+0000", 2024-09-18T10:15:30.132 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00026355500000000001 2024-09-18T10:15:30.132 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:30.132 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:30.132 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:30.132 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:25.306551+0000", 2024-09-18T10:15:30.132 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20389143700000001 2024-09-18T10:15:30.132 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:30.132 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:30.132 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:30.132 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:30.132 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:30.133 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-09-18T10:15:30.133 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T10:15:30.136 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T10:15:30.136 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-18T10:15:30.136 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-18T10:15:30.136 INFO:teuthology.orchestra.run.smithi029.stdout: "epoch": 74, 2024-09-18T10:15:30.137 INFO:teuthology.orchestra.run.smithi029.stdout: "fsid": "ab6d6beb-0627-4b7f-8cdb-4e0a44970f10", 2024-09-18T10:15:30.137 INFO:teuthology.orchestra.run.smithi029.stdout: "created": "2024-09-18T10:14:18.827644+0000", 2024-09-18T10:15:30.137 INFO:teuthology.orchestra.run.smithi029.stdout: "modified": "2024-09-18T10:15:29.548368+0000", 2024-09-18T10:15:30.137 INFO:teuthology.orchestra.run.smithi029.stdout: "last_up_change": "2024-09-18T10:14:30.228591+0000", 2024-09-18T10:15:30.137 INFO:teuthology.orchestra.run.smithi029.stdout: "last_in_change": "2024-09-18T10:14:22.927003+0000", 2024-09-18T10:15:30.137 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-18T10:15:30.137 INFO:teuthology.orchestra.run.smithi029.stdout: "flags_num": 5799936, 2024-09-18T10:15:30.137 INFO:teuthology.orchestra.run.smithi029.stdout: "flags_set": [ 2024-09-18T10:15:30.137 INFO:teuthology.orchestra.run.smithi029.stdout: "pglog_hardlimit", 2024-09-18T10:15:30.137 INFO:teuthology.orchestra.run.smithi029.stdout: "purged_snapdirs", 2024-09-18T10:15:30.137 INFO:teuthology.orchestra.run.smithi029.stdout: "recovery_deletes", 2024-09-18T10:15:30.137 INFO:teuthology.orchestra.run.smithi029.stdout: "sortbitwise" 2024-09-18T10:15:30.137 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-18T10:15:30.138 INFO:teuthology.orchestra.run.smithi029.stdout: "crush_version": 6, 2024-09-18T10:15:30.138 INFO:teuthology.orchestra.run.smithi029.stdout: "full_ratio": 0.94999998807907104, 2024-09-18T10:15:30.138 INFO:teuthology.orchestra.run.smithi029.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-18T10:15:30.138 INFO:teuthology.orchestra.run.smithi029.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-18T10:15:30.138 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_snapshot": "", 2024-09-18T10:15:30.138 INFO:teuthology.orchestra.run.smithi029.stdout: "pool_max": 3, 2024-09-18T10:15:30.138 INFO:teuthology.orchestra.run.smithi029.stdout: "max_osd": 16, 2024-09-18T10:15:30.138 INFO:teuthology.orchestra.run.smithi029.stdout: "require_min_compat_client": "luminous", 2024-09-18T10:15:30.138 INFO:teuthology.orchestra.run.smithi029.stdout: "min_compat_client": "jewel", 2024-09-18T10:15:30.138 INFO:teuthology.orchestra.run.smithi029.stdout: "require_osd_release": "quincy", 2024-09-18T10:15:30.138 INFO:teuthology.orchestra.run.smithi029.stdout: "pools": [ 2024-09-18T10:15:30.138 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.138 INFO:teuthology.orchestra.run.smithi029.stdout: "pool": 1, 2024-09-18T10:15:30.138 INFO:teuthology.orchestra.run.smithi029.stdout: "pool_name": ".mgr", 2024-09-18T10:15:30.139 INFO:teuthology.orchestra.run.smithi029.stdout: "create_time": "2024-09-18T10:14:31.266113+0000", 2024-09-18T10:15:30.139 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-18T10:15:30.139 INFO:teuthology.orchestra.run.smithi029.stdout: "flags_names": "hashpspool", 2024-09-18T10:15:30.139 INFO:teuthology.orchestra.run.smithi029.stdout: "type": 1, 2024-09-18T10:15:30.139 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 2, 2024-09-18T10:15:30.139 INFO:teuthology.orchestra.run.smithi029.stdout: "min_size": 1, 2024-09-18T10:15:30.139 INFO:teuthology.orchestra.run.smithi029.stdout: "crush_rule": 0, 2024-09-18T10:15:30.139 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_count": 0, 2024-09-18T10:15:30.139 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_target": 0, 2024-09-18T10:15:30.139 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_barrier": 0, 2024-09-18T10:15:30.139 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-18T10:15:30.139 INFO:teuthology.orchestra.run.smithi029.stdout: "object_hash": 2, 2024-09-18T10:15:30.139 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_autoscale_mode": "off", 2024-09-18T10:15:30.139 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num": 1, 2024-09-18T10:15:30.140 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_placement_num": 1, 2024-09-18T10:15:30.140 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_placement_num_target": 1, 2024-09-18T10:15:30.140 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num_target": 1, 2024-09-18T10:15:30.140 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num_pending": 1, 2024-09-18T10:15:30.140 INFO:teuthology.orchestra.run.smithi029.stdout: "last_pg_merge_meta": { 2024-09-18T10:15:30.140 INFO:teuthology.orchestra.run.smithi029.stdout: "source_pgid": "0.0", 2024-09-18T10:15:30.140 INFO:teuthology.orchestra.run.smithi029.stdout: "ready_epoch": 0, 2024-09-18T10:15:30.140 INFO:teuthology.orchestra.run.smithi029.stdout: "last_epoch_started": 0, 2024-09-18T10:15:30.140 INFO:teuthology.orchestra.run.smithi029.stdout: "last_epoch_clean": 0, 2024-09-18T10:15:30.140 INFO:teuthology.orchestra.run.smithi029.stdout: "source_version": "0'0", 2024-09-18T10:15:30.140 INFO:teuthology.orchestra.run.smithi029.stdout: "target_version": "0'0" 2024-09-18T10:15:30.140 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.140 INFO:teuthology.orchestra.run.smithi029.stdout: "last_change": "25", 2024-09-18T10:15:30.141 INFO:teuthology.orchestra.run.smithi029.stdout: "last_force_op_resend": "0", 2024-09-18T10:15:30.141 INFO:teuthology.orchestra.run.smithi029.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-18T10:15:30.141 INFO:teuthology.orchestra.run.smithi029.stdout: "last_force_op_resend_preluminous": "0", 2024-09-18T10:15:30.141 INFO:teuthology.orchestra.run.smithi029.stdout: "auid": 0, 2024-09-18T10:15:30.141 INFO:teuthology.orchestra.run.smithi029.stdout: "snap_mode": "selfmanaged", 2024-09-18T10:15:30.141 INFO:teuthology.orchestra.run.smithi029.stdout: "snap_seq": 0, 2024-09-18T10:15:30.141 INFO:teuthology.orchestra.run.smithi029.stdout: "snap_epoch": 0, 2024-09-18T10:15:30.141 INFO:teuthology.orchestra.run.smithi029.stdout: "pool_snaps": [], 2024-09-18T10:15:30.141 INFO:teuthology.orchestra.run.smithi029.stdout: "removed_snaps": "[]", 2024-09-18T10:15:30.141 INFO:teuthology.orchestra.run.smithi029.stdout: "quota_max_bytes": 0, 2024-09-18T10:15:30.141 INFO:teuthology.orchestra.run.smithi029.stdout: "quota_max_objects": 0, 2024-09-18T10:15:30.141 INFO:teuthology.orchestra.run.smithi029.stdout: "tiers": [], 2024-09-18T10:15:30.141 INFO:teuthology.orchestra.run.smithi029.stdout: "tier_of": -1, 2024-09-18T10:15:30.141 INFO:teuthology.orchestra.run.smithi029.stdout: "read_tier": -1, 2024-09-18T10:15:30.142 INFO:teuthology.orchestra.run.smithi029.stdout: "write_tier": -1, 2024-09-18T10:15:30.142 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_mode": "none", 2024-09-18T10:15:30.142 INFO:teuthology.orchestra.run.smithi029.stdout: "target_max_bytes": 0, 2024-09-18T10:15:30.142 INFO:teuthology.orchestra.run.smithi029.stdout: "target_max_objects": 0, 2024-09-18T10:15:30.142 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-18T10:15:30.142 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-18T10:15:30.142 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-18T10:15:30.142 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_min_flush_age": 0, 2024-09-18T10:15:30.142 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_min_evict_age": 0, 2024-09-18T10:15:30.142 INFO:teuthology.orchestra.run.smithi029.stdout: "erasure_code_profile": "", 2024-09-18T10:15:30.142 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_params": { 2024-09-18T10:15:30.142 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "none" 2024-09-18T10:15:30.142 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.143 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_period": 0, 2024-09-18T10:15:30.143 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_count": 0, 2024-09-18T10:15:30.143 INFO:teuthology.orchestra.run.smithi029.stdout: "use_gmt_hitset": true, 2024-09-18T10:15:30.143 INFO:teuthology.orchestra.run.smithi029.stdout: "min_read_recency_for_promote": 0, 2024-09-18T10:15:30.143 INFO:teuthology.orchestra.run.smithi029.stdout: "min_write_recency_for_promote": 0, 2024-09-18T10:15:30.143 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_grade_decay_rate": 0, 2024-09-18T10:15:30.143 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_search_last_n": 0, 2024-09-18T10:15:30.143 INFO:teuthology.orchestra.run.smithi029.stdout: "grade_table": [], 2024-09-18T10:15:30.143 INFO:teuthology.orchestra.run.smithi029.stdout: "stripe_width": 0, 2024-09-18T10:15:30.143 INFO:teuthology.orchestra.run.smithi029.stdout: "expected_num_objects": 0, 2024-09-18T10:15:30.143 INFO:teuthology.orchestra.run.smithi029.stdout: "fast_read": false, 2024-09-18T10:15:30.143 INFO:teuthology.orchestra.run.smithi029.stdout: "options": { 2024-09-18T10:15:30.143 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num_max": 32, 2024-09-18T10:15:30.143 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num_min": 1 2024-09-18T10:15:30.144 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.144 INFO:teuthology.orchestra.run.smithi029.stdout: "application_metadata": { 2024-09-18T10:15:30.144 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": {} 2024-09-18T10:15:30.144 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:30.144 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.144 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.144 INFO:teuthology.orchestra.run.smithi029.stdout: "pool": 2, 2024-09-18T10:15:30.144 INFO:teuthology.orchestra.run.smithi029.stdout: "pool_name": "rbd", 2024-09-18T10:15:30.144 INFO:teuthology.orchestra.run.smithi029.stdout: "create_time": "2024-09-18T10:14:33.358485+0000", 2024-09-18T10:15:30.144 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 8193, 2024-09-18T10:15:30.144 INFO:teuthology.orchestra.run.smithi029.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-18T10:15:30.144 INFO:teuthology.orchestra.run.smithi029.stdout: "type": 1, 2024-09-18T10:15:30.144 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 2, 2024-09-18T10:15:30.145 INFO:teuthology.orchestra.run.smithi029.stdout: "min_size": 1, 2024-09-18T10:15:30.145 INFO:teuthology.orchestra.run.smithi029.stdout: "crush_rule": 0, 2024-09-18T10:15:30.145 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_count": 0, 2024-09-18T10:15:30.145 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_target": 0, 2024-09-18T10:15:30.145 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_barrier": 0, 2024-09-18T10:15:30.145 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-18T10:15:30.145 INFO:teuthology.orchestra.run.smithi029.stdout: "object_hash": 2, 2024-09-18T10:15:30.145 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_autoscale_mode": "off", 2024-09-18T10:15:30.145 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num": 8, 2024-09-18T10:15:30.145 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_placement_num": 8, 2024-09-18T10:15:30.145 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_placement_num_target": 8, 2024-09-18T10:15:30.145 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num_target": 8, 2024-09-18T10:15:30.145 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num_pending": 8, 2024-09-18T10:15:30.146 INFO:teuthology.orchestra.run.smithi029.stdout: "last_pg_merge_meta": { 2024-09-18T10:15:30.146 INFO:teuthology.orchestra.run.smithi029.stdout: "source_pgid": "0.0", 2024-09-18T10:15:30.146 INFO:teuthology.orchestra.run.smithi029.stdout: "ready_epoch": 0, 2024-09-18T10:15:30.146 INFO:teuthology.orchestra.run.smithi029.stdout: "last_epoch_started": 0, 2024-09-18T10:15:30.146 INFO:teuthology.orchestra.run.smithi029.stdout: "last_epoch_clean": 0, 2024-09-18T10:15:30.146 INFO:teuthology.orchestra.run.smithi029.stdout: "source_version": "0'0", 2024-09-18T10:15:30.146 INFO:teuthology.orchestra.run.smithi029.stdout: "target_version": "0'0" 2024-09-18T10:15:30.146 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.146 INFO:teuthology.orchestra.run.smithi029.stdout: "last_change": "29", 2024-09-18T10:15:30.146 INFO:teuthology.orchestra.run.smithi029.stdout: "last_force_op_resend": "0", 2024-09-18T10:15:30.146 INFO:teuthology.orchestra.run.smithi029.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-18T10:15:30.146 INFO:teuthology.orchestra.run.smithi029.stdout: "last_force_op_resend_preluminous": "0", 2024-09-18T10:15:30.146 INFO:teuthology.orchestra.run.smithi029.stdout: "auid": 0, 2024-09-18T10:15:30.146 INFO:teuthology.orchestra.run.smithi029.stdout: "snap_mode": "selfmanaged", 2024-09-18T10:15:30.147 INFO:teuthology.orchestra.run.smithi029.stdout: "snap_seq": 3, 2024-09-18T10:15:30.147 INFO:teuthology.orchestra.run.smithi029.stdout: "snap_epoch": 29, 2024-09-18T10:15:30.147 INFO:teuthology.orchestra.run.smithi029.stdout: "pool_snaps": [], 2024-09-18T10:15:30.147 INFO:teuthology.orchestra.run.smithi029.stdout: "removed_snaps": "[]", 2024-09-18T10:15:30.147 INFO:teuthology.orchestra.run.smithi029.stdout: "quota_max_bytes": 0, 2024-09-18T10:15:30.147 INFO:teuthology.orchestra.run.smithi029.stdout: "quota_max_objects": 0, 2024-09-18T10:15:30.147 INFO:teuthology.orchestra.run.smithi029.stdout: "tiers": [], 2024-09-18T10:15:30.147 INFO:teuthology.orchestra.run.smithi029.stdout: "tier_of": -1, 2024-09-18T10:15:30.147 INFO:teuthology.orchestra.run.smithi029.stdout: "read_tier": -1, 2024-09-18T10:15:30.147 INFO:teuthology.orchestra.run.smithi029.stdout: "write_tier": -1, 2024-09-18T10:15:30.147 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_mode": "none", 2024-09-18T10:15:30.147 INFO:teuthology.orchestra.run.smithi029.stdout: "target_max_bytes": 0, 2024-09-18T10:15:30.147 INFO:teuthology.orchestra.run.smithi029.stdout: "target_max_objects": 0, 2024-09-18T10:15:30.147 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-18T10:15:30.148 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-18T10:15:30.148 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-18T10:15:30.148 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_min_flush_age": 0, 2024-09-18T10:15:30.148 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_min_evict_age": 0, 2024-09-18T10:15:30.148 INFO:teuthology.orchestra.run.smithi029.stdout: "erasure_code_profile": "", 2024-09-18T10:15:30.148 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_params": { 2024-09-18T10:15:30.148 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "none" 2024-09-18T10:15:30.148 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.148 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_period": 0, 2024-09-18T10:15:30.148 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_count": 0, 2024-09-18T10:15:30.148 INFO:teuthology.orchestra.run.smithi029.stdout: "use_gmt_hitset": true, 2024-09-18T10:15:30.148 INFO:teuthology.orchestra.run.smithi029.stdout: "min_read_recency_for_promote": 0, 2024-09-18T10:15:30.148 INFO:teuthology.orchestra.run.smithi029.stdout: "min_write_recency_for_promote": 0, 2024-09-18T10:15:30.148 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_grade_decay_rate": 0, 2024-09-18T10:15:30.149 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_search_last_n": 0, 2024-09-18T10:15:30.149 INFO:teuthology.orchestra.run.smithi029.stdout: "grade_table": [], 2024-09-18T10:15:30.149 INFO:teuthology.orchestra.run.smithi029.stdout: "stripe_width": 0, 2024-09-18T10:15:30.149 INFO:teuthology.orchestra.run.smithi029.stdout: "expected_num_objects": 0, 2024-09-18T10:15:30.149 INFO:teuthology.orchestra.run.smithi029.stdout: "fast_read": false, 2024-09-18T10:15:30.149 INFO:teuthology.orchestra.run.smithi029.stdout: "options": {}, 2024-09-18T10:15:30.149 INFO:teuthology.orchestra.run.smithi029.stdout: "application_metadata": { 2024-09-18T10:15:30.149 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd": {} 2024-09-18T10:15:30.149 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:30.149 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.149 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.149 INFO:teuthology.orchestra.run.smithi029.stdout: "pool": 3, 2024-09-18T10:15:30.149 INFO:teuthology.orchestra.run.smithi029.stdout: "pool_name": "unique_pool_0", 2024-09-18T10:15:30.150 INFO:teuthology.orchestra.run.smithi029.stdout: "create_time": "2024-09-18T10:14:44.624320+0000", 2024-09-18T10:15:30.150 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 8193, 2024-09-18T10:15:30.150 INFO:teuthology.orchestra.run.smithi029.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-18T10:15:30.150 INFO:teuthology.orchestra.run.smithi029.stdout: "type": 3, 2024-09-18T10:15:30.150 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 7, 2024-09-18T10:15:30.150 INFO:teuthology.orchestra.run.smithi029.stdout: "min_size": 5, 2024-09-18T10:15:30.150 INFO:teuthology.orchestra.run.smithi029.stdout: "crush_rule": 1, 2024-09-18T10:15:30.150 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_count": 0, 2024-09-18T10:15:30.150 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_target": 0, 2024-09-18T10:15:30.150 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_barrier": 0, 2024-09-18T10:15:30.150 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-18T10:15:30.150 INFO:teuthology.orchestra.run.smithi029.stdout: "object_hash": 2, 2024-09-18T10:15:30.150 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_autoscale_mode": "off", 2024-09-18T10:15:30.150 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num": 26, 2024-09-18T10:15:30.151 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_placement_num": 26, 2024-09-18T10:15:30.151 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_placement_num_target": 26, 2024-09-18T10:15:30.151 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num_target": 26, 2024-09-18T10:15:30.151 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num_pending": 26, 2024-09-18T10:15:30.151 INFO:teuthology.orchestra.run.smithi029.stdout: "last_pg_merge_meta": { 2024-09-18T10:15:30.151 INFO:teuthology.orchestra.run.smithi029.stdout: "source_pgid": "0.0", 2024-09-18T10:15:30.151 INFO:teuthology.orchestra.run.smithi029.stdout: "ready_epoch": 0, 2024-09-18T10:15:30.151 INFO:teuthology.orchestra.run.smithi029.stdout: "last_epoch_started": 0, 2024-09-18T10:15:30.151 INFO:teuthology.orchestra.run.smithi029.stdout: "last_epoch_clean": 0, 2024-09-18T10:15:30.151 INFO:teuthology.orchestra.run.smithi029.stdout: "source_version": "0'0", 2024-09-18T10:15:30.151 INFO:teuthology.orchestra.run.smithi029.stdout: "target_version": "0'0" 2024-09-18T10:15:30.151 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.151 INFO:teuthology.orchestra.run.smithi029.stdout: "last_change": "74", 2024-09-18T10:15:30.151 INFO:teuthology.orchestra.run.smithi029.stdout: "last_force_op_resend": "0", 2024-09-18T10:15:30.151 INFO:teuthology.orchestra.run.smithi029.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-18T10:15:30.152 INFO:teuthology.orchestra.run.smithi029.stdout: "last_force_op_resend_preluminous": "40", 2024-09-18T10:15:30.152 INFO:teuthology.orchestra.run.smithi029.stdout: "auid": 0, 2024-09-18T10:15:30.152 INFO:teuthology.orchestra.run.smithi029.stdout: "snap_mode": "selfmanaged", 2024-09-18T10:15:30.152 INFO:teuthology.orchestra.run.smithi029.stdout: "snap_seq": 36, 2024-09-18T10:15:30.152 INFO:teuthology.orchestra.run.smithi029.stdout: "snap_epoch": 74, 2024-09-18T10:15:30.152 INFO:teuthology.orchestra.run.smithi029.stdout: "pool_snaps": [], 2024-09-18T10:15:30.152 INFO:teuthology.orchestra.run.smithi029.stdout: "removed_snaps": "[]", 2024-09-18T10:15:30.152 INFO:teuthology.orchestra.run.smithi029.stdout: "quota_max_bytes": 0, 2024-09-18T10:15:30.152 INFO:teuthology.orchestra.run.smithi029.stdout: "quota_max_objects": 0, 2024-09-18T10:15:30.152 INFO:teuthology.orchestra.run.smithi029.stdout: "tiers": [], 2024-09-18T10:15:30.152 INFO:teuthology.orchestra.run.smithi029.stdout: "tier_of": -1, 2024-09-18T10:15:30.152 INFO:teuthology.orchestra.run.smithi029.stdout: "read_tier": -1, 2024-09-18T10:15:30.152 INFO:teuthology.orchestra.run.smithi029.stdout: "write_tier": -1, 2024-09-18T10:15:30.152 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_mode": "none", 2024-09-18T10:15:30.153 INFO:teuthology.orchestra.run.smithi029.stdout: "target_max_bytes": 0, 2024-09-18T10:15:30.153 INFO:teuthology.orchestra.run.smithi029.stdout: "target_max_objects": 0, 2024-09-18T10:15:30.153 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-18T10:15:30.153 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-18T10:15:30.153 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-18T10:15:30.153 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_min_flush_age": 0, 2024-09-18T10:15:30.153 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_min_evict_age": 0, 2024-09-18T10:15:30.153 INFO:teuthology.orchestra.run.smithi029.stdout: "erasure_code_profile": "shecprofile", 2024-09-18T10:15:30.153 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_params": { 2024-09-18T10:15:30.153 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "none" 2024-09-18T10:15:30.153 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.153 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_period": 0, 2024-09-18T10:15:30.153 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_count": 0, 2024-09-18T10:15:30.154 INFO:teuthology.orchestra.run.smithi029.stdout: "use_gmt_hitset": true, 2024-09-18T10:15:30.154 INFO:teuthology.orchestra.run.smithi029.stdout: "min_read_recency_for_promote": 0, 2024-09-18T10:15:30.154 INFO:teuthology.orchestra.run.smithi029.stdout: "min_write_recency_for_promote": 0, 2024-09-18T10:15:30.154 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_grade_decay_rate": 0, 2024-09-18T10:15:30.154 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_search_last_n": 0, 2024-09-18T10:15:30.154 INFO:teuthology.orchestra.run.smithi029.stdout: "grade_table": [], 2024-09-18T10:15:30.154 INFO:teuthology.orchestra.run.smithi029.stdout: "stripe_width": 16384, 2024-09-18T10:15:30.154 INFO:teuthology.orchestra.run.smithi029.stdout: "expected_num_objects": 0, 2024-09-18T10:15:30.154 INFO:teuthology.orchestra.run.smithi029.stdout: "fast_read": false, 2024-09-18T10:15:30.154 INFO:teuthology.orchestra.run.smithi029.stdout: "options": {}, 2024-09-18T10:15:30.154 INFO:teuthology.orchestra.run.smithi029.stdout: "application_metadata": { 2024-09-18T10:15:30.154 INFO:teuthology.orchestra.run.smithi029.stdout: "rados": {} 2024-09-18T10:15:30.154 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:30.154 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:30.155 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-18T10:15:30.155 INFO:teuthology.orchestra.run.smithi029.stdout: "osds": [ 2024-09-18T10:15:30.155 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.155 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 0, 2024-09-18T10:15:30.155 INFO:teuthology.orchestra.run.smithi029.stdout: "uuid": "39cf0511-0009-433f-92f2-e5686f33d713", 2024-09-18T10:15:30.155 INFO:teuthology.orchestra.run.smithi029.stdout: "up": 1, 2024-09-18T10:15:30.155 INFO:teuthology.orchestra.run.smithi029.stdout: "in": 1, 2024-09-18T10:15:30.155 INFO:teuthology.orchestra.run.smithi029.stdout: "weight": 1, 2024-09-18T10:15:30.155 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_affinity": 1, 2024-09-18T10:15:30.155 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_begin": 0, 2024-09-18T10:15:30.155 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_end": 0, 2024-09-18T10:15:30.155 INFO:teuthology.orchestra.run.smithi029.stdout: "up_from": 21, 2024-09-18T10:15:30.155 INFO:teuthology.orchestra.run.smithi029.stdout: "up_thru": 40, 2024-09-18T10:15:30.155 INFO:teuthology.orchestra.run.smithi029.stdout: "down_at": 0, 2024-09-18T10:15:30.155 INFO:teuthology.orchestra.run.smithi029.stdout: "lost_at": 0, 2024-09-18T10:15:30.156 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addrs": { 2024-09-18T10:15:30.156 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:30.156 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.156 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:30.156 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6808", 2024-09-18T10:15:30.156 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22197 2024-09-18T10:15:30.156 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.156 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.156 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:30.156 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6809", 2024-09-18T10:15:30.156 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22197 2024-09-18T10:15:30.156 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:30.156 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:30.157 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.157 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addrs": { 2024-09-18T10:15:30.157 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:30.157 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.157 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:30.157 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6810", 2024-09-18T10:15:30.157 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22197 2024-09-18T10:15:30.157 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.157 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.157 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:30.157 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6811", 2024-09-18T10:15:30.157 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22197 2024-09-18T10:15:30.157 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:30.157 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:30.158 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.158 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addrs": { 2024-09-18T10:15:30.158 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:30.158 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.158 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:30.158 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6814", 2024-09-18T10:15:30.158 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22197 2024-09-18T10:15:30.158 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.158 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.158 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:30.158 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6815", 2024-09-18T10:15:30.158 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22197 2024-09-18T10:15:30.158 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:30.158 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:30.159 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.159 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addrs": { 2024-09-18T10:15:30.159 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:30.159 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.159 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:30.159 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6812", 2024-09-18T10:15:30.159 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22197 2024-09-18T10:15:30.159 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.159 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.159 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:30.159 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6813", 2024-09-18T10:15:30.159 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22197 2024-09-18T10:15:30.159 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:30.159 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:30.160 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.160 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addr": "172.21.15.29:6809/22197", 2024-09-18T10:15:30.160 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addr": "172.21.15.29:6811/22197", 2024-09-18T10:15:30.160 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addr": "172.21.15.29:6815/22197", 2024-09-18T10:15:30.160 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addr": "172.21.15.29:6813/22197", 2024-09-18T10:15:30.160 INFO:teuthology.orchestra.run.smithi029.stdout: "state": [ 2024-09-18T10:15:30.160 INFO:teuthology.orchestra.run.smithi029.stdout: "exists", 2024-09-18T10:15:30.160 INFO:teuthology.orchestra.run.smithi029.stdout: "up" 2024-09-18T10:15:30.160 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:30.160 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.160 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.160 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 1, 2024-09-18T10:15:30.160 INFO:teuthology.orchestra.run.smithi029.stdout: "uuid": "a0bf06ab-b98e-4130-91d2-7d4283540c08", 2024-09-18T10:15:30.160 INFO:teuthology.orchestra.run.smithi029.stdout: "up": 1, 2024-09-18T10:15:30.161 INFO:teuthology.orchestra.run.smithi029.stdout: "in": 1, 2024-09-18T10:15:30.161 INFO:teuthology.orchestra.run.smithi029.stdout: "weight": 1, 2024-09-18T10:15:30.161 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_affinity": 1, 2024-09-18T10:15:30.161 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_begin": 0, 2024-09-18T10:15:30.161 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_end": 0, 2024-09-18T10:15:30.161 INFO:teuthology.orchestra.run.smithi029.stdout: "up_from": 22, 2024-09-18T10:15:30.161 INFO:teuthology.orchestra.run.smithi029.stdout: "up_thru": 58, 2024-09-18T10:15:30.161 INFO:teuthology.orchestra.run.smithi029.stdout: "down_at": 0, 2024-09-18T10:15:30.161 INFO:teuthology.orchestra.run.smithi029.stdout: "lost_at": 0, 2024-09-18T10:15:30.161 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addrs": { 2024-09-18T10:15:30.161 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:30.161 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.161 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:30.161 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.136:6808", 2024-09-18T10:15:30.162 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22140 2024-09-18T10:15:30.162 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.162 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.162 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:30.162 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.136:6809", 2024-09-18T10:15:30.162 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22140 2024-09-18T10:15:30.162 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:30.162 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:30.162 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.162 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addrs": { 2024-09-18T10:15:30.162 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:30.162 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.162 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:30.162 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.136:6810", 2024-09-18T10:15:30.163 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22140 2024-09-18T10:15:30.163 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.163 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.163 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:30.163 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.136:6811", 2024-09-18T10:15:30.163 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22140 2024-09-18T10:15:30.163 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:30.163 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:30.163 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.163 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addrs": { 2024-09-18T10:15:30.163 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:30.163 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.163 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:30.163 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.136:6814", 2024-09-18T10:15:30.164 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22140 2024-09-18T10:15:30.164 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.164 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.164 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:30.164 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.136:6815", 2024-09-18T10:15:30.164 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22140 2024-09-18T10:15:30.164 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:30.164 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:30.164 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.164 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addrs": { 2024-09-18T10:15:30.164 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:30.164 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.164 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:30.164 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.136:6812", 2024-09-18T10:15:30.165 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22140 2024-09-18T10:15:30.165 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.165 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.165 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:30.165 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.136:6813", 2024-09-18T10:15:30.165 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22140 2024-09-18T10:15:30.165 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:30.165 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:30.165 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.165 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addr": "172.21.15.136:6809/22140", 2024-09-18T10:15:30.165 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addr": "172.21.15.136:6811/22140", 2024-09-18T10:15:30.165 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addr": "172.21.15.136:6815/22140", 2024-09-18T10:15:30.165 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addr": "172.21.15.136:6813/22140", 2024-09-18T10:15:30.165 INFO:teuthology.orchestra.run.smithi029.stdout: "state": [ 2024-09-18T10:15:30.166 INFO:teuthology.orchestra.run.smithi029.stdout: "exists", 2024-09-18T10:15:30.166 INFO:teuthology.orchestra.run.smithi029.stdout: "up" 2024-09-18T10:15:30.166 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:30.166 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.166 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.166 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 2, 2024-09-18T10:15:30.166 INFO:teuthology.orchestra.run.smithi029.stdout: "uuid": "742a786e-6129-4fbc-bb5b-e2e3c03c2d77", 2024-09-18T10:15:30.166 INFO:teuthology.orchestra.run.smithi029.stdout: "up": 1, 2024-09-18T10:15:30.166 INFO:teuthology.orchestra.run.smithi029.stdout: "in": 1, 2024-09-18T10:15:30.166 INFO:teuthology.orchestra.run.smithi029.stdout: "weight": 1, 2024-09-18T10:15:30.166 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_affinity": 1, 2024-09-18T10:15:30.166 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_begin": 0, 2024-09-18T10:15:30.166 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_end": 0, 2024-09-18T10:15:30.166 INFO:teuthology.orchestra.run.smithi029.stdout: "up_from": 22, 2024-09-18T10:15:30.167 INFO:teuthology.orchestra.run.smithi029.stdout: "up_thru": 0, 2024-09-18T10:15:30.167 INFO:teuthology.orchestra.run.smithi029.stdout: "down_at": 0, 2024-09-18T10:15:30.167 INFO:teuthology.orchestra.run.smithi029.stdout: "lost_at": 0, 2024-09-18T10:15:30.167 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addrs": { 2024-09-18T10:15:30.167 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:30.167 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.167 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:30.167 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.154:6824", 2024-09-18T10:15:30.167 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22095 2024-09-18T10:15:30.167 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.167 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.167 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:30.167 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.154:6825", 2024-09-18T10:15:30.167 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22095 2024-09-18T10:15:30.168 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:30.168 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:30.168 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.168 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addrs": { 2024-09-18T10:15:30.168 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:30.168 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.168 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:30.168 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.154:6826", 2024-09-18T10:15:30.168 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22095 2024-09-18T10:15:30.168 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.168 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.168 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:30.168 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.154:6827", 2024-09-18T10:15:30.168 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22095 2024-09-18T10:15:30.169 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:30.169 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:30.169 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.169 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addrs": { 2024-09-18T10:15:30.169 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:30.169 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.169 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:30.169 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.154:6830", 2024-09-18T10:15:30.169 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22095 2024-09-18T10:15:30.169 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.169 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.169 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:30.169 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.154:6831", 2024-09-18T10:15:30.169 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22095 2024-09-18T10:15:30.170 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:30.170 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:30.170 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.170 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addrs": { 2024-09-18T10:15:30.170 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:30.170 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.170 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:30.170 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.154:6828", 2024-09-18T10:15:30.170 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22095 2024-09-18T10:15:30.170 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.170 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.170 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:30.170 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.154:6829", 2024-09-18T10:15:30.170 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22095 2024-09-18T10:15:30.171 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:30.171 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:30.171 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.171 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addr": "172.21.15.154:6825/22095", 2024-09-18T10:15:30.171 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addr": "172.21.15.154:6827/22095", 2024-09-18T10:15:30.171 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addr": "172.21.15.154:6831/22095", 2024-09-18T10:15:30.171 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addr": "172.21.15.154:6829/22095", 2024-09-18T10:15:30.171 INFO:teuthology.orchestra.run.smithi029.stdout: "state": [ 2024-09-18T10:15:30.171 INFO:teuthology.orchestra.run.smithi029.stdout: "exists", 2024-09-18T10:15:30.171 INFO:teuthology.orchestra.run.smithi029.stdout: "up" 2024-09-18T10:15:30.171 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:30.171 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.171 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.171 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 3, 2024-09-18T10:15:30.171 INFO:teuthology.orchestra.run.smithi029.stdout: "uuid": "3aa77114-24ed-49ab-86ff-a0600794cbf8", 2024-09-18T10:15:30.172 INFO:teuthology.orchestra.run.smithi029.stdout: "up": 1, 2024-09-18T10:15:30.172 INFO:teuthology.orchestra.run.smithi029.stdout: "in": 1, 2024-09-18T10:15:30.172 INFO:teuthology.orchestra.run.smithi029.stdout: "weight": 1, 2024-09-18T10:15:30.172 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_affinity": 1, 2024-09-18T10:15:30.172 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_begin": 0, 2024-09-18T10:15:30.172 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_end": 0, 2024-09-18T10:15:30.172 INFO:teuthology.orchestra.run.smithi029.stdout: "up_from": 22, 2024-09-18T10:15:30.172 INFO:teuthology.orchestra.run.smithi029.stdout: "up_thru": 48, 2024-09-18T10:15:30.172 INFO:teuthology.orchestra.run.smithi029.stdout: "down_at": 0, 2024-09-18T10:15:30.172 INFO:teuthology.orchestra.run.smithi029.stdout: "lost_at": 0, 2024-09-18T10:15:30.172 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addrs": { 2024-09-18T10:15:30.172 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:30.172 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.173 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:30.173 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.189:6802", 2024-09-18T10:15:30.173 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22404 2024-09-18T10:15:30.173 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.173 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.173 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:30.173 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.189:6803", 2024-09-18T10:15:30.173 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22404 2024-09-18T10:15:30.173 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:30.173 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:30.173 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.173 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addrs": { 2024-09-18T10:15:30.173 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:30.173 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.174 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:30.174 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.189:6804", 2024-09-18T10:15:30.174 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22404 2024-09-18T10:15:30.174 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.174 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.174 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:30.174 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.189:6805", 2024-09-18T10:15:30.174 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22404 2024-09-18T10:15:30.174 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:30.174 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:30.174 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.174 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addrs": { 2024-09-18T10:15:30.174 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:30.174 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.175 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:30.175 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.189:6808", 2024-09-18T10:15:30.175 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22404 2024-09-18T10:15:30.175 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.175 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.175 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:30.175 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.189:6809", 2024-09-18T10:15:30.175 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22404 2024-09-18T10:15:30.175 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:30.175 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:30.175 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.175 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addrs": { 2024-09-18T10:15:30.175 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:30.175 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.176 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:30.176 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.189:6806", 2024-09-18T10:15:30.176 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22404 2024-09-18T10:15:30.176 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.176 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.176 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:30.176 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.189:6807", 2024-09-18T10:15:30.176 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22404 2024-09-18T10:15:30.176 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:30.176 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:30.176 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.176 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addr": "172.21.15.189:6803/22404", 2024-09-18T10:15:30.176 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addr": "172.21.15.189:6805/22404", 2024-09-18T10:15:30.176 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addr": "172.21.15.189:6809/22404", 2024-09-18T10:15:30.177 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addr": "172.21.15.189:6807/22404", 2024-09-18T10:15:30.177 INFO:teuthology.orchestra.run.smithi029.stdout: "state": [ 2024-09-18T10:15:30.177 INFO:teuthology.orchestra.run.smithi029.stdout: "exists", 2024-09-18T10:15:30.177 INFO:teuthology.orchestra.run.smithi029.stdout: "up" 2024-09-18T10:15:30.177 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:30.177 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.177 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.177 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 4, 2024-09-18T10:15:30.177 INFO:teuthology.orchestra.run.smithi029.stdout: "uuid": "426567ad-b0be-41d4-afcf-dc56ec3974cd", 2024-09-18T10:15:30.177 INFO:teuthology.orchestra.run.smithi029.stdout: "up": 1, 2024-09-18T10:15:30.177 INFO:teuthology.orchestra.run.smithi029.stdout: "in": 1, 2024-09-18T10:15:30.177 INFO:teuthology.orchestra.run.smithi029.stdout: "weight": 1, 2024-09-18T10:15:30.177 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_affinity": 1, 2024-09-18T10:15:30.177 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_begin": 0, 2024-09-18T10:15:30.178 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_end": 0, 2024-09-18T10:15:30.178 INFO:teuthology.orchestra.run.smithi029.stdout: "up_from": 21, 2024-09-18T10:15:30.178 INFO:teuthology.orchestra.run.smithi029.stdout: "up_thru": 40, 2024-09-18T10:15:30.178 INFO:teuthology.orchestra.run.smithi029.stdout: "down_at": 0, 2024-09-18T10:15:30.178 INFO:teuthology.orchestra.run.smithi029.stdout: "lost_at": 0, 2024-09-18T10:15:30.178 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addrs": { 2024-09-18T10:15:30.178 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:30.178 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.178 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:30.178 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6800", 2024-09-18T10:15:30.178 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22195 2024-09-18T10:15:30.178 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.178 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.178 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:30.179 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6801", 2024-09-18T10:15:30.179 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22195 2024-09-18T10:15:30.179 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:30.179 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:30.179 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.179 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addrs": { 2024-09-18T10:15:30.179 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:30.179 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.179 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:30.179 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6802", 2024-09-18T10:15:30.179 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22195 2024-09-18T10:15:30.179 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.179 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.180 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:30.180 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6803", 2024-09-18T10:15:30.180 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22195 2024-09-18T10:15:30.180 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:30.180 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:30.180 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.180 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addrs": { 2024-09-18T10:15:30.180 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:30.180 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.180 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:30.180 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6806", 2024-09-18T10:15:30.180 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22195 2024-09-18T10:15:30.180 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.180 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.181 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:30.181 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6807", 2024-09-18T10:15:30.181 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22195 2024-09-18T10:15:30.181 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:30.181 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:30.181 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.181 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addrs": { 2024-09-18T10:15:30.181 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:30.181 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.181 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:30.181 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6804", 2024-09-18T10:15:30.181 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22195 2024-09-18T10:15:30.181 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.181 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.182 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:30.182 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6805", 2024-09-18T10:15:30.182 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22195 2024-09-18T10:15:30.182 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:30.182 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:30.182 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.182 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addr": "172.21.15.29:6801/22195", 2024-09-18T10:15:30.182 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addr": "172.21.15.29:6803/22195", 2024-09-18T10:15:30.182 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addr": "172.21.15.29:6807/22195", 2024-09-18T10:15:30.182 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addr": "172.21.15.29:6805/22195", 2024-09-18T10:15:30.182 INFO:teuthology.orchestra.run.smithi029.stdout: "state": [ 2024-09-18T10:15:30.182 INFO:teuthology.orchestra.run.smithi029.stdout: "exists", 2024-09-18T10:15:30.182 INFO:teuthology.orchestra.run.smithi029.stdout: "up" 2024-09-18T10:15:30.183 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:30.183 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.183 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.183 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 5, 2024-09-18T10:15:30.183 INFO:teuthology.orchestra.run.smithi029.stdout: "uuid": "d9559e42-1f4e-4373-8392-a536a0afd871", 2024-09-18T10:15:30.183 INFO:teuthology.orchestra.run.smithi029.stdout: "up": 1, 2024-09-18T10:15:30.183 INFO:teuthology.orchestra.run.smithi029.stdout: "in": 1, 2024-09-18T10:15:30.183 INFO:teuthology.orchestra.run.smithi029.stdout: "weight": 1, 2024-09-18T10:15:30.183 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_affinity": 1, 2024-09-18T10:15:30.183 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_begin": 0, 2024-09-18T10:15:30.183 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_end": 0, 2024-09-18T10:15:30.183 INFO:teuthology.orchestra.run.smithi029.stdout: "up_from": 22, 2024-09-18T10:15:30.183 INFO:teuthology.orchestra.run.smithi029.stdout: "up_thru": 40, 2024-09-18T10:15:30.183 INFO:teuthology.orchestra.run.smithi029.stdout: "down_at": 0, 2024-09-18T10:15:30.184 INFO:teuthology.orchestra.run.smithi029.stdout: "lost_at": 0, 2024-09-18T10:15:30.184 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addrs": { 2024-09-18T10:15:30.184 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:30.184 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.184 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:30.184 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.136:6817", 2024-09-18T10:15:30.184 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22141 2024-09-18T10:15:30.184 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.184 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.184 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:30.184 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.136:6819", 2024-09-18T10:15:30.184 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22141 2024-09-18T10:15:30.184 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:30.184 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:30.185 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.185 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addrs": { 2024-09-18T10:15:30.185 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:30.185 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.185 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:30.185 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.136:6821", 2024-09-18T10:15:30.185 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22141 2024-09-18T10:15:30.185 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.185 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.185 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:30.185 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.136:6823", 2024-09-18T10:15:30.185 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22141 2024-09-18T10:15:30.185 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:30.185 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:30.186 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.186 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addrs": { 2024-09-18T10:15:30.186 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:30.186 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.186 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:30.186 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.136:6829", 2024-09-18T10:15:30.186 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22141 2024-09-18T10:15:30.186 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.186 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.186 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:30.186 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.136:6831", 2024-09-18T10:15:30.186 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22141 2024-09-18T10:15:30.186 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:30.186 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:30.187 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.187 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addrs": { 2024-09-18T10:15:30.187 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:30.187 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.187 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:30.187 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.136:6825", 2024-09-18T10:15:30.187 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22141 2024-09-18T10:15:30.187 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.187 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.187 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:30.187 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.136:6827", 2024-09-18T10:15:30.187 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22141 2024-09-18T10:15:30.187 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:30.187 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:30.188 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.188 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addr": "172.21.15.136:6819/22141", 2024-09-18T10:15:30.188 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addr": "172.21.15.136:6823/22141", 2024-09-18T10:15:30.188 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addr": "172.21.15.136:6831/22141", 2024-09-18T10:15:30.188 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addr": "172.21.15.136:6827/22141", 2024-09-18T10:15:30.188 INFO:teuthology.orchestra.run.smithi029.stdout: "state": [ 2024-09-18T10:15:30.188 INFO:teuthology.orchestra.run.smithi029.stdout: "exists", 2024-09-18T10:15:30.188 INFO:teuthology.orchestra.run.smithi029.stdout: "up" 2024-09-18T10:15:30.188 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:30.188 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.188 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.188 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 6, 2024-09-18T10:15:30.188 INFO:teuthology.orchestra.run.smithi029.stdout: "uuid": "d50ef1e2-8bc5-4a90-9be3-365a0db2be6f", 2024-09-18T10:15:30.188 INFO:teuthology.orchestra.run.smithi029.stdout: "up": 1, 2024-09-18T10:15:30.189 INFO:teuthology.orchestra.run.smithi029.stdout: "in": 1, 2024-09-18T10:15:30.189 INFO:teuthology.orchestra.run.smithi029.stdout: "weight": 1, 2024-09-18T10:15:30.189 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_affinity": 1, 2024-09-18T10:15:30.189 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_begin": 0, 2024-09-18T10:15:30.189 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_end": 0, 2024-09-18T10:15:30.189 INFO:teuthology.orchestra.run.smithi029.stdout: "up_from": 22, 2024-09-18T10:15:30.189 INFO:teuthology.orchestra.run.smithi029.stdout: "up_thru": 46, 2024-09-18T10:15:30.189 INFO:teuthology.orchestra.run.smithi029.stdout: "down_at": 0, 2024-09-18T10:15:30.189 INFO:teuthology.orchestra.run.smithi029.stdout: "lost_at": 0, 2024-09-18T10:15:30.189 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addrs": { 2024-09-18T10:15:30.189 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:30.189 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.189 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:30.189 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.154:6800", 2024-09-18T10:15:30.190 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22096 2024-09-18T10:15:30.190 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.190 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.190 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:30.190 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.154:6801", 2024-09-18T10:15:30.190 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22096 2024-09-18T10:15:30.190 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:30.190 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:30.190 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.190 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addrs": { 2024-09-18T10:15:30.190 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:30.190 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.190 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:30.190 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.154:6802", 2024-09-18T10:15:30.191 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22096 2024-09-18T10:15:30.191 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.191 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.191 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:30.191 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.154:6803", 2024-09-18T10:15:30.191 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22096 2024-09-18T10:15:30.191 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:30.191 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:30.191 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.191 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addrs": { 2024-09-18T10:15:30.191 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:30.191 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.191 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:30.191 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.154:6806", 2024-09-18T10:15:30.192 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22096 2024-09-18T10:15:30.192 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.192 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.192 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:30.192 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.154:6807", 2024-09-18T10:15:30.192 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22096 2024-09-18T10:15:30.192 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:30.192 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:30.192 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.192 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addrs": { 2024-09-18T10:15:30.192 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:30.192 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.192 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:30.192 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.154:6804", 2024-09-18T10:15:30.192 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22096 2024-09-18T10:15:30.193 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.193 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.193 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:30.193 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.154:6805", 2024-09-18T10:15:30.193 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22096 2024-09-18T10:15:30.193 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:30.193 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:30.193 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.193 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addr": "172.21.15.154:6801/22096", 2024-09-18T10:15:30.194 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addr": "172.21.15.154:6803/22096", 2024-09-18T10:15:30.194 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addr": "172.21.15.154:6807/22096", 2024-09-18T10:15:30.194 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addr": "172.21.15.154:6805/22096", 2024-09-18T10:15:30.194 INFO:teuthology.orchestra.run.smithi029.stdout: "state": [ 2024-09-18T10:15:30.194 INFO:teuthology.orchestra.run.smithi029.stdout: "exists", 2024-09-18T10:15:30.194 INFO:teuthology.orchestra.run.smithi029.stdout: "up" 2024-09-18T10:15:30.194 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:30.194 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.194 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.194 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 7, 2024-09-18T10:15:30.194 INFO:teuthology.orchestra.run.smithi029.stdout: "uuid": "bd0b85e4-cfee-4d19-90a5-79854794b48b", 2024-09-18T10:15:30.194 INFO:teuthology.orchestra.run.smithi029.stdout: "up": 1, 2024-09-18T10:15:30.194 INFO:teuthology.orchestra.run.smithi029.stdout: "in": 1, 2024-09-18T10:15:30.194 INFO:teuthology.orchestra.run.smithi029.stdout: "weight": 1, 2024-09-18T10:15:30.195 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_affinity": 1, 2024-09-18T10:15:30.195 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_begin": 0, 2024-09-18T10:15:30.195 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_end": 0, 2024-09-18T10:15:30.195 INFO:teuthology.orchestra.run.smithi029.stdout: "up_from": 22, 2024-09-18T10:15:30.195 INFO:teuthology.orchestra.run.smithi029.stdout: "up_thru": 40, 2024-09-18T10:15:30.195 INFO:teuthology.orchestra.run.smithi029.stdout: "down_at": 0, 2024-09-18T10:15:30.195 INFO:teuthology.orchestra.run.smithi029.stdout: "lost_at": 0, 2024-09-18T10:15:30.195 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addrs": { 2024-09-18T10:15:30.195 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:30.195 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.195 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:30.195 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.189:6818", 2024-09-18T10:15:30.195 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22405 2024-09-18T10:15:30.195 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.196 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.196 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:30.196 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.189:6819", 2024-09-18T10:15:30.196 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22405 2024-09-18T10:15:30.196 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:30.196 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:30.196 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.196 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addrs": { 2024-09-18T10:15:30.196 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:30.196 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.196 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:30.196 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.189:6820", 2024-09-18T10:15:30.196 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22405 2024-09-18T10:15:30.197 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.197 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.197 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:30.197 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.189:6821", 2024-09-18T10:15:30.197 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22405 2024-09-18T10:15:30.197 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:30.197 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:30.197 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.197 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addrs": { 2024-09-18T10:15:30.197 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:30.197 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.197 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:30.197 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.189:6824", 2024-09-18T10:15:30.197 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22405 2024-09-18T10:15:30.198 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.198 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.198 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:30.198 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.189:6825", 2024-09-18T10:15:30.198 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22405 2024-09-18T10:15:30.198 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:30.198 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:30.198 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.198 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addrs": { 2024-09-18T10:15:30.198 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:30.198 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.198 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:30.198 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.189:6822", 2024-09-18T10:15:30.198 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22405 2024-09-18T10:15:30.199 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.199 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.199 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:30.199 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.189:6823", 2024-09-18T10:15:30.199 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22405 2024-09-18T10:15:30.199 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:30.199 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:30.199 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.199 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addr": "172.21.15.189:6819/22405", 2024-09-18T10:15:30.199 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addr": "172.21.15.189:6821/22405", 2024-09-18T10:15:30.199 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addr": "172.21.15.189:6825/22405", 2024-09-18T10:15:30.199 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addr": "172.21.15.189:6823/22405", 2024-09-18T10:15:30.199 INFO:teuthology.orchestra.run.smithi029.stdout: "state": [ 2024-09-18T10:15:30.199 INFO:teuthology.orchestra.run.smithi029.stdout: "exists", 2024-09-18T10:15:30.200 INFO:teuthology.orchestra.run.smithi029.stdout: "up" 2024-09-18T10:15:30.200 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:30.200 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.200 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.200 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 8, 2024-09-18T10:15:30.200 INFO:teuthology.orchestra.run.smithi029.stdout: "uuid": "cd333f56-c165-45d7-8c8b-48d34bc4ddc5", 2024-09-18T10:15:30.200 INFO:teuthology.orchestra.run.smithi029.stdout: "up": 1, 2024-09-18T10:15:30.200 INFO:teuthology.orchestra.run.smithi029.stdout: "in": 1, 2024-09-18T10:15:30.200 INFO:teuthology.orchestra.run.smithi029.stdout: "weight": 1, 2024-09-18T10:15:30.200 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_affinity": 1, 2024-09-18T10:15:30.200 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_begin": 0, 2024-09-18T10:15:30.200 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_end": 0, 2024-09-18T10:15:30.200 INFO:teuthology.orchestra.run.smithi029.stdout: "up_from": 22, 2024-09-18T10:15:30.201 INFO:teuthology.orchestra.run.smithi029.stdout: "up_thru": 40, 2024-09-18T10:15:30.201 INFO:teuthology.orchestra.run.smithi029.stdout: "down_at": 0, 2024-09-18T10:15:30.201 INFO:teuthology.orchestra.run.smithi029.stdout: "lost_at": 0, 2024-09-18T10:15:30.201 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addrs": { 2024-09-18T10:15:30.201 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:30.201 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.201 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:30.201 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6824", 2024-09-18T10:15:30.201 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22200 2024-09-18T10:15:30.201 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.201 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.201 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:30.201 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6825", 2024-09-18T10:15:30.201 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22200 2024-09-18T10:15:30.202 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:30.202 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:30.202 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.202 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addrs": { 2024-09-18T10:15:30.202 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:30.202 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.202 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:30.202 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6826", 2024-09-18T10:15:30.202 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22200 2024-09-18T10:15:30.202 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.202 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.202 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:30.202 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6827", 2024-09-18T10:15:30.202 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22200 2024-09-18T10:15:30.203 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:30.203 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:30.203 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.203 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addrs": { 2024-09-18T10:15:30.203 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:30.203 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.203 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:30.203 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6830", 2024-09-18T10:15:30.203 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22200 2024-09-18T10:15:30.203 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.203 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.203 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:30.203 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6831", 2024-09-18T10:15:30.203 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22200 2024-09-18T10:15:30.204 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:30.204 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:30.204 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.204 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addrs": { 2024-09-18T10:15:30.204 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:30.204 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.204 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:30.204 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6828", 2024-09-18T10:15:30.204 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22200 2024-09-18T10:15:30.204 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.204 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.204 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:30.204 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6829", 2024-09-18T10:15:30.204 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22200 2024-09-18T10:15:30.205 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:30.205 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:30.205 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.205 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addr": "172.21.15.29:6825/22200", 2024-09-18T10:15:30.205 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addr": "172.21.15.29:6827/22200", 2024-09-18T10:15:30.205 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addr": "172.21.15.29:6831/22200", 2024-09-18T10:15:30.205 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addr": "172.21.15.29:6829/22200", 2024-09-18T10:15:30.205 INFO:teuthology.orchestra.run.smithi029.stdout: "state": [ 2024-09-18T10:15:30.205 INFO:teuthology.orchestra.run.smithi029.stdout: "exists", 2024-09-18T10:15:30.205 INFO:teuthology.orchestra.run.smithi029.stdout: "up" 2024-09-18T10:15:30.205 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:30.205 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.205 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.205 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 9, 2024-09-18T10:15:30.206 INFO:teuthology.orchestra.run.smithi029.stdout: "uuid": "1e079e4c-8801-4754-9d0d-d0b68695d32e", 2024-09-18T10:15:30.206 INFO:teuthology.orchestra.run.smithi029.stdout: "up": 1, 2024-09-18T10:15:30.206 INFO:teuthology.orchestra.run.smithi029.stdout: "in": 1, 2024-09-18T10:15:30.206 INFO:teuthology.orchestra.run.smithi029.stdout: "weight": 1, 2024-09-18T10:15:30.206 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_affinity": 1, 2024-09-18T10:15:30.206 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_begin": 0, 2024-09-18T10:15:30.206 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_end": 0, 2024-09-18T10:15:30.206 INFO:teuthology.orchestra.run.smithi029.stdout: "up_from": 21, 2024-09-18T10:15:30.206 INFO:teuthology.orchestra.run.smithi029.stdout: "up_thru": 62, 2024-09-18T10:15:30.206 INFO:teuthology.orchestra.run.smithi029.stdout: "down_at": 0, 2024-09-18T10:15:30.206 INFO:teuthology.orchestra.run.smithi029.stdout: "lost_at": 0, 2024-09-18T10:15:30.206 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addrs": { 2024-09-18T10:15:30.206 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:30.206 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.206 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:30.207 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.136:6800", 2024-09-18T10:15:30.207 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22142 2024-09-18T10:15:30.207 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.207 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.207 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:30.207 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.136:6801", 2024-09-18T10:15:30.207 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22142 2024-09-18T10:15:30.207 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:30.207 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:30.207 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.207 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addrs": { 2024-09-18T10:15:30.207 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:30.207 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.207 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:30.208 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.136:6802", 2024-09-18T10:15:30.208 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22142 2024-09-18T10:15:30.208 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.208 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.208 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:30.208 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.136:6803", 2024-09-18T10:15:30.208 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22142 2024-09-18T10:15:30.208 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:30.208 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:30.208 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.208 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addrs": { 2024-09-18T10:15:30.208 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:30.208 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.208 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:30.209 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.136:6806", 2024-09-18T10:15:30.209 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22142 2024-09-18T10:15:30.209 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.209 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.209 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:30.209 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.136:6807", 2024-09-18T10:15:30.209 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22142 2024-09-18T10:15:30.209 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:30.209 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:30.209 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.209 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addrs": { 2024-09-18T10:15:30.209 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:30.209 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.210 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:30.210 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.136:6804", 2024-09-18T10:15:30.210 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22142 2024-09-18T10:15:30.210 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.210 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.210 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:30.210 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.136:6805", 2024-09-18T10:15:30.210 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22142 2024-09-18T10:15:30.210 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:30.210 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:30.210 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.210 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addr": "172.21.15.136:6801/22142", 2024-09-18T10:15:30.210 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addr": "172.21.15.136:6803/22142", 2024-09-18T10:15:30.210 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addr": "172.21.15.136:6807/22142", 2024-09-18T10:15:30.211 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addr": "172.21.15.136:6805/22142", 2024-09-18T10:15:30.211 INFO:teuthology.orchestra.run.smithi029.stdout: "state": [ 2024-09-18T10:15:30.211 INFO:teuthology.orchestra.run.smithi029.stdout: "exists", 2024-09-18T10:15:30.211 INFO:teuthology.orchestra.run.smithi029.stdout: "up" 2024-09-18T10:15:30.211 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:30.211 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.211 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.211 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 10, 2024-09-18T10:15:30.211 INFO:teuthology.orchestra.run.smithi029.stdout: "uuid": "92ee68d4-371c-4849-930f-a8fa18d64b99", 2024-09-18T10:15:30.211 INFO:teuthology.orchestra.run.smithi029.stdout: "up": 1, 2024-09-18T10:15:30.211 INFO:teuthology.orchestra.run.smithi029.stdout: "in": 1, 2024-09-18T10:15:30.211 INFO:teuthology.orchestra.run.smithi029.stdout: "weight": 1, 2024-09-18T10:15:30.211 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_affinity": 1, 2024-09-18T10:15:30.211 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_begin": 0, 2024-09-18T10:15:30.212 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_end": 0, 2024-09-18T10:15:30.212 INFO:teuthology.orchestra.run.smithi029.stdout: "up_from": 22, 2024-09-18T10:15:30.212 INFO:teuthology.orchestra.run.smithi029.stdout: "up_thru": 72, 2024-09-18T10:15:30.212 INFO:teuthology.orchestra.run.smithi029.stdout: "down_at": 0, 2024-09-18T10:15:30.212 INFO:teuthology.orchestra.run.smithi029.stdout: "lost_at": 0, 2024-09-18T10:15:30.212 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addrs": { 2024-09-18T10:15:30.212 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:30.212 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.212 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:30.212 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.154:6808", 2024-09-18T10:15:30.212 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22097 2024-09-18T10:15:30.212 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.212 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.212 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:30.213 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.154:6809", 2024-09-18T10:15:30.213 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22097 2024-09-18T10:15:30.213 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:30.213 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:30.213 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.213 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addrs": { 2024-09-18T10:15:30.213 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:30.213 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.213 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:30.213 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.154:6810", 2024-09-18T10:15:30.213 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22097 2024-09-18T10:15:30.213 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.213 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.214 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:30.214 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.154:6811", 2024-09-18T10:15:30.214 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22097 2024-09-18T10:15:30.214 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:30.214 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:30.214 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.214 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addrs": { 2024-09-18T10:15:30.214 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:30.214 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.214 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:30.214 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.154:6814", 2024-09-18T10:15:30.214 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22097 2024-09-18T10:15:30.214 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.214 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.215 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:30.215 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.154:6815", 2024-09-18T10:15:30.215 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22097 2024-09-18T10:15:30.215 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:30.215 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:30.215 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.215 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addrs": { 2024-09-18T10:15:30.215 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:30.215 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.215 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:30.215 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.154:6812", 2024-09-18T10:15:30.215 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22097 2024-09-18T10:15:30.215 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.215 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.216 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:30.216 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.154:6813", 2024-09-18T10:15:30.216 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22097 2024-09-18T10:15:30.216 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:30.216 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:30.216 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.216 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addr": "172.21.15.154:6809/22097", 2024-09-18T10:15:30.216 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addr": "172.21.15.154:6811/22097", 2024-09-18T10:15:30.216 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addr": "172.21.15.154:6815/22097", 2024-09-18T10:15:30.216 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addr": "172.21.15.154:6813/22097", 2024-09-18T10:15:30.216 INFO:teuthology.orchestra.run.smithi029.stdout: "state": [ 2024-09-18T10:15:30.216 INFO:teuthology.orchestra.run.smithi029.stdout: "exists", 2024-09-18T10:15:30.216 INFO:teuthology.orchestra.run.smithi029.stdout: "up" 2024-09-18T10:15:30.217 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:30.217 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.217 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.217 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 11, 2024-09-18T10:15:30.217 INFO:teuthology.orchestra.run.smithi029.stdout: "uuid": "634ddaf5-f7b8-4b27-a46a-04e21b73fcf0", 2024-09-18T10:15:30.217 INFO:teuthology.orchestra.run.smithi029.stdout: "up": 1, 2024-09-18T10:15:30.217 INFO:teuthology.orchestra.run.smithi029.stdout: "in": 1, 2024-09-18T10:15:30.217 INFO:teuthology.orchestra.run.smithi029.stdout: "weight": 1, 2024-09-18T10:15:30.217 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_affinity": 1, 2024-09-18T10:15:30.217 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_begin": 0, 2024-09-18T10:15:30.217 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_end": 0, 2024-09-18T10:15:30.217 INFO:teuthology.orchestra.run.smithi029.stdout: "up_from": 21, 2024-09-18T10:15:30.217 INFO:teuthology.orchestra.run.smithi029.stdout: "up_thru": 32, 2024-09-18T10:15:30.217 INFO:teuthology.orchestra.run.smithi029.stdout: "down_at": 0, 2024-09-18T10:15:30.218 INFO:teuthology.orchestra.run.smithi029.stdout: "lost_at": 0, 2024-09-18T10:15:30.218 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addrs": { 2024-09-18T10:15:30.218 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:30.218 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.218 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:30.218 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.189:6826", 2024-09-18T10:15:30.218 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22416 2024-09-18T10:15:30.218 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.218 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.218 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:30.218 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.189:6827", 2024-09-18T10:15:30.218 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22416 2024-09-18T10:15:30.218 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:30.218 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:30.219 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.219 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addrs": { 2024-09-18T10:15:30.219 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:30.219 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.219 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:30.219 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.189:6828", 2024-09-18T10:15:30.219 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22416 2024-09-18T10:15:30.219 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.219 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.219 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:30.219 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.189:6829", 2024-09-18T10:15:30.219 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22416 2024-09-18T10:15:30.219 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:30.220 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:30.220 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.220 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addrs": { 2024-09-18T10:15:30.220 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:30.220 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.220 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:30.220 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.189:6832", 2024-09-18T10:15:30.220 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22416 2024-09-18T10:15:30.220 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.220 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.220 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:30.220 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.189:6833", 2024-09-18T10:15:30.220 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22416 2024-09-18T10:15:30.220 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:30.221 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:30.221 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.221 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addrs": { 2024-09-18T10:15:30.221 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:30.221 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.221 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:30.221 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.189:6830", 2024-09-18T10:15:30.221 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22416 2024-09-18T10:15:30.221 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.221 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.222 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:30.222 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.189:6831", 2024-09-18T10:15:30.222 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22416 2024-09-18T10:15:30.222 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:30.222 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:30.222 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.222 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addr": "172.21.15.189:6827/22416", 2024-09-18T10:15:30.222 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addr": "172.21.15.189:6829/22416", 2024-09-18T10:15:30.222 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addr": "172.21.15.189:6833/22416", 2024-09-18T10:15:30.222 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addr": "172.21.15.189:6831/22416", 2024-09-18T10:15:30.222 INFO:teuthology.orchestra.run.smithi029.stdout: "state": [ 2024-09-18T10:15:30.222 INFO:teuthology.orchestra.run.smithi029.stdout: "exists", 2024-09-18T10:15:30.222 INFO:teuthology.orchestra.run.smithi029.stdout: "up" 2024-09-18T10:15:30.222 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:30.223 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.223 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.223 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 12, 2024-09-18T10:15:30.223 INFO:teuthology.orchestra.run.smithi029.stdout: "uuid": "bc58137b-5948-400d-b377-193b15824d8d", 2024-09-18T10:15:30.223 INFO:teuthology.orchestra.run.smithi029.stdout: "up": 1, 2024-09-18T10:15:30.223 INFO:teuthology.orchestra.run.smithi029.stdout: "in": 1, 2024-09-18T10:15:30.223 INFO:teuthology.orchestra.run.smithi029.stdout: "weight": 1, 2024-09-18T10:15:30.223 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_affinity": 1, 2024-09-18T10:15:30.223 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_begin": 0, 2024-09-18T10:15:30.223 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_end": 0, 2024-09-18T10:15:30.223 INFO:teuthology.orchestra.run.smithi029.stdout: "up_from": 21, 2024-09-18T10:15:30.223 INFO:teuthology.orchestra.run.smithi029.stdout: "up_thru": 0, 2024-09-18T10:15:30.223 INFO:teuthology.orchestra.run.smithi029.stdout: "down_at": 0, 2024-09-18T10:15:30.223 INFO:teuthology.orchestra.run.smithi029.stdout: "lost_at": 0, 2024-09-18T10:15:30.224 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addrs": { 2024-09-18T10:15:30.224 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:30.224 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.224 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:30.224 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6816", 2024-09-18T10:15:30.224 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22210 2024-09-18T10:15:30.224 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.224 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.224 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:30.224 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6817", 2024-09-18T10:15:30.224 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22210 2024-09-18T10:15:30.224 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:30.224 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:30.224 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.225 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addrs": { 2024-09-18T10:15:30.225 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:30.225 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.225 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:30.225 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6818", 2024-09-18T10:15:30.225 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22210 2024-09-18T10:15:30.225 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.225 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.225 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:30.225 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6819", 2024-09-18T10:15:30.225 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22210 2024-09-18T10:15:30.225 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:30.225 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:30.226 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.226 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addrs": { 2024-09-18T10:15:30.226 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:30.226 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.226 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:30.226 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6822", 2024-09-18T10:15:30.226 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22210 2024-09-18T10:15:30.226 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.226 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.226 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:30.226 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6823", 2024-09-18T10:15:30.226 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22210 2024-09-18T10:15:30.226 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:30.226 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:30.227 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.227 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addrs": { 2024-09-18T10:15:30.227 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:30.227 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.227 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:30.227 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6820", 2024-09-18T10:15:30.227 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22210 2024-09-18T10:15:30.227 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.227 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.227 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:30.227 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6821", 2024-09-18T10:15:30.227 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22210 2024-09-18T10:15:30.227 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:30.227 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:30.227 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.228 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addr": "172.21.15.29:6817/22210", 2024-09-18T10:15:30.228 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addr": "172.21.15.29:6819/22210", 2024-09-18T10:15:30.228 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addr": "172.21.15.29:6823/22210", 2024-09-18T10:15:30.228 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addr": "172.21.15.29:6821/22210", 2024-09-18T10:15:30.228 INFO:teuthology.orchestra.run.smithi029.stdout: "state": [ 2024-09-18T10:15:30.228 INFO:teuthology.orchestra.run.smithi029.stdout: "exists", 2024-09-18T10:15:30.228 INFO:teuthology.orchestra.run.smithi029.stdout: "up" 2024-09-18T10:15:30.228 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:30.228 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.228 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.228 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 13, 2024-09-18T10:15:30.228 INFO:teuthology.orchestra.run.smithi029.stdout: "uuid": "a8b753a7-ab69-4853-ac36-66d9b85c2e20", 2024-09-18T10:15:30.228 INFO:teuthology.orchestra.run.smithi029.stdout: "up": 1, 2024-09-18T10:15:30.228 INFO:teuthology.orchestra.run.smithi029.stdout: "in": 1, 2024-09-18T10:15:30.229 INFO:teuthology.orchestra.run.smithi029.stdout: "weight": 1, 2024-09-18T10:15:30.229 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_affinity": 1, 2024-09-18T10:15:30.229 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_begin": 0, 2024-09-18T10:15:30.229 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_end": 0, 2024-09-18T10:15:30.229 INFO:teuthology.orchestra.run.smithi029.stdout: "up_from": 22, 2024-09-18T10:15:30.229 INFO:teuthology.orchestra.run.smithi029.stdout: "up_thru": 32, 2024-09-18T10:15:30.229 INFO:teuthology.orchestra.run.smithi029.stdout: "down_at": 0, 2024-09-18T10:15:30.229 INFO:teuthology.orchestra.run.smithi029.stdout: "lost_at": 0, 2024-09-18T10:15:30.229 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addrs": { 2024-09-18T10:15:30.229 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:30.229 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.229 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:30.229 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.136:6816", 2024-09-18T10:15:30.229 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22143 2024-09-18T10:15:30.230 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.230 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.230 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:30.230 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.136:6818", 2024-09-18T10:15:30.230 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22143 2024-09-18T10:15:30.230 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:30.230 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:30.230 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.230 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addrs": { 2024-09-18T10:15:30.230 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:30.230 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.230 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:30.230 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.136:6820", 2024-09-18T10:15:30.230 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22143 2024-09-18T10:15:30.231 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.231 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.231 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:30.231 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.136:6822", 2024-09-18T10:15:30.231 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22143 2024-09-18T10:15:30.231 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:30.231 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:30.231 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.231 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addrs": { 2024-09-18T10:15:30.231 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:30.231 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.231 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:30.231 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.136:6828", 2024-09-18T10:15:30.231 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22143 2024-09-18T10:15:30.232 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.232 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.232 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:30.232 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.136:6830", 2024-09-18T10:15:30.232 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22143 2024-09-18T10:15:30.232 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:30.232 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:30.232 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.232 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addrs": { 2024-09-18T10:15:30.232 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:30.232 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.232 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:30.232 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.136:6824", 2024-09-18T10:15:30.232 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22143 2024-09-18T10:15:30.233 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.233 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.233 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:30.233 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.136:6826", 2024-09-18T10:15:30.233 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22143 2024-09-18T10:15:30.233 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:30.233 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:30.233 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.233 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addr": "172.21.15.136:6818/22143", 2024-09-18T10:15:30.233 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addr": "172.21.15.136:6822/22143", 2024-09-18T10:15:30.233 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addr": "172.21.15.136:6830/22143", 2024-09-18T10:15:30.233 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addr": "172.21.15.136:6826/22143", 2024-09-18T10:15:30.233 INFO:teuthology.orchestra.run.smithi029.stdout: "state": [ 2024-09-18T10:15:30.233 INFO:teuthology.orchestra.run.smithi029.stdout: "exists", 2024-09-18T10:15:30.234 INFO:teuthology.orchestra.run.smithi029.stdout: "up" 2024-09-18T10:15:30.234 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:30.234 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.234 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.234 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 14, 2024-09-18T10:15:30.234 INFO:teuthology.orchestra.run.smithi029.stdout: "uuid": "8ad0848f-0de4-4a75-9c63-8c4a4a7b8524", 2024-09-18T10:15:30.234 INFO:teuthology.orchestra.run.smithi029.stdout: "up": 1, 2024-09-18T10:15:30.234 INFO:teuthology.orchestra.run.smithi029.stdout: "in": 1, 2024-09-18T10:15:30.234 INFO:teuthology.orchestra.run.smithi029.stdout: "weight": 1, 2024-09-18T10:15:30.234 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_affinity": 1, 2024-09-18T10:15:30.234 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_begin": 0, 2024-09-18T10:15:30.234 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_end": 0, 2024-09-18T10:15:30.234 INFO:teuthology.orchestra.run.smithi029.stdout: "up_from": 22, 2024-09-18T10:15:30.234 INFO:teuthology.orchestra.run.smithi029.stdout: "up_thru": 60, 2024-09-18T10:15:30.235 INFO:teuthology.orchestra.run.smithi029.stdout: "down_at": 0, 2024-09-18T10:15:30.235 INFO:teuthology.orchestra.run.smithi029.stdout: "lost_at": 0, 2024-09-18T10:15:30.235 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addrs": { 2024-09-18T10:15:30.235 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:30.235 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.235 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:30.235 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.154:6816", 2024-09-18T10:15:30.235 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22098 2024-09-18T10:15:30.235 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.235 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.235 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:30.235 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.154:6817", 2024-09-18T10:15:30.235 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22098 2024-09-18T10:15:30.235 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:30.236 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:30.236 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.236 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addrs": { 2024-09-18T10:15:30.236 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:30.236 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.236 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:30.236 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.154:6818", 2024-09-18T10:15:30.236 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22098 2024-09-18T10:15:30.236 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.236 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.236 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:30.236 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.154:6819", 2024-09-18T10:15:30.236 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22098 2024-09-18T10:15:30.236 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:30.237 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:30.237 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.237 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addrs": { 2024-09-18T10:15:30.237 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:30.237 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.237 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:30.237 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.154:6822", 2024-09-18T10:15:30.237 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22098 2024-09-18T10:15:30.237 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.237 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.237 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:30.237 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.154:6823", 2024-09-18T10:15:30.237 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22098 2024-09-18T10:15:30.238 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:30.238 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:30.238 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.238 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addrs": { 2024-09-18T10:15:30.238 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:30.238 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.238 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:30.238 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.154:6820", 2024-09-18T10:15:30.238 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22098 2024-09-18T10:15:30.238 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.238 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.238 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:30.238 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.154:6821", 2024-09-18T10:15:30.238 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22098 2024-09-18T10:15:30.239 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:30.239 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:30.239 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.239 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addr": "172.21.15.154:6817/22098", 2024-09-18T10:15:30.239 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addr": "172.21.15.154:6819/22098", 2024-09-18T10:15:30.239 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addr": "172.21.15.154:6823/22098", 2024-09-18T10:15:30.239 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addr": "172.21.15.154:6821/22098", 2024-09-18T10:15:30.239 INFO:teuthology.orchestra.run.smithi029.stdout: "state": [ 2024-09-18T10:15:30.239 INFO:teuthology.orchestra.run.smithi029.stdout: "exists", 2024-09-18T10:15:30.239 INFO:teuthology.orchestra.run.smithi029.stdout: "up" 2024-09-18T10:15:30.239 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:30.239 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.239 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.239 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 15, 2024-09-18T10:15:30.240 INFO:teuthology.orchestra.run.smithi029.stdout: "uuid": "069099b3-ec1a-483b-b967-98cddbf2f678", 2024-09-18T10:15:30.240 INFO:teuthology.orchestra.run.smithi029.stdout: "up": 1, 2024-09-18T10:15:30.240 INFO:teuthology.orchestra.run.smithi029.stdout: "in": 1, 2024-09-18T10:15:30.240 INFO:teuthology.orchestra.run.smithi029.stdout: "weight": 1, 2024-09-18T10:15:30.240 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_affinity": 1, 2024-09-18T10:15:30.240 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_begin": 0, 2024-09-18T10:15:30.240 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_end": 0, 2024-09-18T10:15:30.240 INFO:teuthology.orchestra.run.smithi029.stdout: "up_from": 22, 2024-09-18T10:15:30.240 INFO:teuthology.orchestra.run.smithi029.stdout: "up_thru": 0, 2024-09-18T10:15:30.240 INFO:teuthology.orchestra.run.smithi029.stdout: "down_at": 0, 2024-09-18T10:15:30.240 INFO:teuthology.orchestra.run.smithi029.stdout: "lost_at": 0, 2024-09-18T10:15:30.240 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addrs": { 2024-09-18T10:15:30.240 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:30.240 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.241 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:30.241 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.189:6810", 2024-09-18T10:15:30.241 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22415 2024-09-18T10:15:30.241 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.241 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.241 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:30.241 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.189:6811", 2024-09-18T10:15:30.241 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22415 2024-09-18T10:15:30.241 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:30.241 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:30.241 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.241 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addrs": { 2024-09-18T10:15:30.241 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:30.241 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.242 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:30.242 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.189:6812", 2024-09-18T10:15:30.242 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22415 2024-09-18T10:15:30.242 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.242 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.242 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:30.242 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.189:6813", 2024-09-18T10:15:30.242 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22415 2024-09-18T10:15:30.242 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:30.242 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:30.242 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.242 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addrs": { 2024-09-18T10:15:30.242 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:30.243 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.243 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:30.243 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.189:6816", 2024-09-18T10:15:30.243 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22415 2024-09-18T10:15:30.243 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.243 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.243 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:30.243 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.189:6817", 2024-09-18T10:15:30.243 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22415 2024-09-18T10:15:30.243 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:30.243 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:30.243 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.243 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addrs": { 2024-09-18T10:15:30.243 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:30.244 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.244 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:30.244 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.189:6814", 2024-09-18T10:15:30.244 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22415 2024-09-18T10:15:30.244 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.244 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.244 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:30.244 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.189:6815", 2024-09-18T10:15:30.244 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22415 2024-09-18T10:15:30.244 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:30.244 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:30.244 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.244 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addr": "172.21.15.189:6811/22415", 2024-09-18T10:15:30.244 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addr": "172.21.15.189:6813/22415", 2024-09-18T10:15:30.245 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addr": "172.21.15.189:6817/22415", 2024-09-18T10:15:30.245 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addr": "172.21.15.189:6815/22415", 2024-09-18T10:15:30.245 INFO:teuthology.orchestra.run.smithi029.stdout: "state": [ 2024-09-18T10:15:30.245 INFO:teuthology.orchestra.run.smithi029.stdout: "exists", 2024-09-18T10:15:30.245 INFO:teuthology.orchestra.run.smithi029.stdout: "up" 2024-09-18T10:15:30.245 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:30.245 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:30.245 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-18T10:15:30.245 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_xinfo": [ 2024-09-18T10:15:30.245 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.245 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 0, 2024-09-18T10:15:30.245 INFO:teuthology.orchestra.run.smithi029.stdout: "down_stamp": "0.000000", 2024-09-18T10:15:30.245 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_probability": 0, 2024-09-18T10:15:30.245 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_interval": 0, 2024-09-18T10:15:30.246 INFO:teuthology.orchestra.run.smithi029.stdout: "features": 4540138320759226367, 2024-09-18T10:15:30.246 INFO:teuthology.orchestra.run.smithi029.stdout: "old_weight": 0, 2024-09-18T10:15:30.246 INFO:teuthology.orchestra.run.smithi029.stdout: "last_purged_snaps_scrub": "2024-09-18T10:14:28.132367+0000", 2024-09-18T10:15:30.246 INFO:teuthology.orchestra.run.smithi029.stdout: "dead_epoch": 0 2024-09-18T10:15:30.246 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.246 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.246 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 1, 2024-09-18T10:15:30.246 INFO:teuthology.orchestra.run.smithi029.stdout: "down_stamp": "0.000000", 2024-09-18T10:15:30.246 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_probability": 0, 2024-09-18T10:15:30.246 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_interval": 0, 2024-09-18T10:15:30.246 INFO:teuthology.orchestra.run.smithi029.stdout: "features": 4540138320759226367, 2024-09-18T10:15:30.246 INFO:teuthology.orchestra.run.smithi029.stdout: "old_weight": 0, 2024-09-18T10:15:30.246 INFO:teuthology.orchestra.run.smithi029.stdout: "last_purged_snaps_scrub": "2024-09-18T10:14:28.342192+0000", 2024-09-18T10:15:30.246 INFO:teuthology.orchestra.run.smithi029.stdout: "dead_epoch": 0 2024-09-18T10:15:30.247 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.247 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.247 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 2, 2024-09-18T10:15:30.247 INFO:teuthology.orchestra.run.smithi029.stdout: "down_stamp": "0.000000", 2024-09-18T10:15:30.247 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_probability": 0, 2024-09-18T10:15:30.247 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_interval": 0, 2024-09-18T10:15:30.247 INFO:teuthology.orchestra.run.smithi029.stdout: "features": 4540138320759226367, 2024-09-18T10:15:30.247 INFO:teuthology.orchestra.run.smithi029.stdout: "old_weight": 0, 2024-09-18T10:15:30.247 INFO:teuthology.orchestra.run.smithi029.stdout: "last_purged_snaps_scrub": "2024-09-18T10:14:28.486853+0000", 2024-09-18T10:15:30.247 INFO:teuthology.orchestra.run.smithi029.stdout: "dead_epoch": 0 2024-09-18T10:15:30.247 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.247 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.247 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 3, 2024-09-18T10:15:30.247 INFO:teuthology.orchestra.run.smithi029.stdout: "down_stamp": "0.000000", 2024-09-18T10:15:30.248 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_probability": 0, 2024-09-18T10:15:30.248 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_interval": 0, 2024-09-18T10:15:30.248 INFO:teuthology.orchestra.run.smithi029.stdout: "features": 4540138320759226367, 2024-09-18T10:15:30.248 INFO:teuthology.orchestra.run.smithi029.stdout: "old_weight": 0, 2024-09-18T10:15:30.248 INFO:teuthology.orchestra.run.smithi029.stdout: "last_purged_snaps_scrub": "2024-09-18T10:14:28.299130+0000", 2024-09-18T10:15:30.248 INFO:teuthology.orchestra.run.smithi029.stdout: "dead_epoch": 0 2024-09-18T10:15:30.248 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.248 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.248 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 4, 2024-09-18T10:15:30.248 INFO:teuthology.orchestra.run.smithi029.stdout: "down_stamp": "0.000000", 2024-09-18T10:15:30.248 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_probability": 0, 2024-09-18T10:15:30.248 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_interval": 0, 2024-09-18T10:15:30.248 INFO:teuthology.orchestra.run.smithi029.stdout: "features": 4540138320759226367, 2024-09-18T10:15:30.248 INFO:teuthology.orchestra.run.smithi029.stdout: "old_weight": 0, 2024-09-18T10:15:30.249 INFO:teuthology.orchestra.run.smithi029.stdout: "last_purged_snaps_scrub": "2024-09-18T10:14:28.027740+0000", 2024-09-18T10:15:30.249 INFO:teuthology.orchestra.run.smithi029.stdout: "dead_epoch": 0 2024-09-18T10:15:30.249 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.249 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.249 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 5, 2024-09-18T10:15:30.249 INFO:teuthology.orchestra.run.smithi029.stdout: "down_stamp": "0.000000", 2024-09-18T10:15:30.249 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_probability": 0, 2024-09-18T10:15:30.249 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_interval": 0, 2024-09-18T10:15:30.249 INFO:teuthology.orchestra.run.smithi029.stdout: "features": 4540138320759226367, 2024-09-18T10:15:30.249 INFO:teuthology.orchestra.run.smithi029.stdout: "old_weight": 0, 2024-09-18T10:15:30.249 INFO:teuthology.orchestra.run.smithi029.stdout: "last_purged_snaps_scrub": "2024-09-18T10:14:28.450120+0000", 2024-09-18T10:15:30.249 INFO:teuthology.orchestra.run.smithi029.stdout: "dead_epoch": 0 2024-09-18T10:15:30.249 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.249 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.250 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 6, 2024-09-18T10:15:30.250 INFO:teuthology.orchestra.run.smithi029.stdout: "down_stamp": "0.000000", 2024-09-18T10:15:30.250 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_probability": 0, 2024-09-18T10:15:30.250 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_interval": 0, 2024-09-18T10:15:30.250 INFO:teuthology.orchestra.run.smithi029.stdout: "features": 4540138320759226367, 2024-09-18T10:15:30.250 INFO:teuthology.orchestra.run.smithi029.stdout: "old_weight": 0, 2024-09-18T10:15:30.250 INFO:teuthology.orchestra.run.smithi029.stdout: "last_purged_snaps_scrub": "2024-09-18T10:14:28.412452+0000", 2024-09-18T10:15:30.250 INFO:teuthology.orchestra.run.smithi029.stdout: "dead_epoch": 0 2024-09-18T10:15:30.250 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.250 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.250 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 7, 2024-09-18T10:15:30.250 INFO:teuthology.orchestra.run.smithi029.stdout: "down_stamp": "0.000000", 2024-09-18T10:15:30.250 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_probability": 0, 2024-09-18T10:15:30.250 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_interval": 0, 2024-09-18T10:15:30.251 INFO:teuthology.orchestra.run.smithi029.stdout: "features": 4540138320759226367, 2024-09-18T10:15:30.251 INFO:teuthology.orchestra.run.smithi029.stdout: "old_weight": 0, 2024-09-18T10:15:30.251 INFO:teuthology.orchestra.run.smithi029.stdout: "last_purged_snaps_scrub": "2024-09-18T10:14:28.368142+0000", 2024-09-18T10:15:30.251 INFO:teuthology.orchestra.run.smithi029.stdout: "dead_epoch": 0 2024-09-18T10:15:30.251 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.251 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.251 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 8, 2024-09-18T10:15:30.251 INFO:teuthology.orchestra.run.smithi029.stdout: "down_stamp": "0.000000", 2024-09-18T10:15:30.251 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_probability": 0, 2024-09-18T10:15:30.251 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_interval": 0, 2024-09-18T10:15:30.251 INFO:teuthology.orchestra.run.smithi029.stdout: "features": 4540138320759226367, 2024-09-18T10:15:30.251 INFO:teuthology.orchestra.run.smithi029.stdout: "old_weight": 0, 2024-09-18T10:15:30.251 INFO:teuthology.orchestra.run.smithi029.stdout: "last_purged_snaps_scrub": "2024-09-18T10:14:28.238445+0000", 2024-09-18T10:15:30.252 INFO:teuthology.orchestra.run.smithi029.stdout: "dead_epoch": 0 2024-09-18T10:15:30.252 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.252 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.252 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 9, 2024-09-18T10:15:30.252 INFO:teuthology.orchestra.run.smithi029.stdout: "down_stamp": "0.000000", 2024-09-18T10:15:30.252 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_probability": 0, 2024-09-18T10:15:30.252 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_interval": 0, 2024-09-18T10:15:30.252 INFO:teuthology.orchestra.run.smithi029.stdout: "features": 4540138320759226367, 2024-09-18T10:15:30.252 INFO:teuthology.orchestra.run.smithi029.stdout: "old_weight": 0, 2024-09-18T10:15:30.252 INFO:teuthology.orchestra.run.smithi029.stdout: "last_purged_snaps_scrub": "2024-09-18T10:14:28.096605+0000", 2024-09-18T10:15:30.252 INFO:teuthology.orchestra.run.smithi029.stdout: "dead_epoch": 0 2024-09-18T10:15:30.252 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.252 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.253 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 10, 2024-09-18T10:15:30.253 INFO:teuthology.orchestra.run.smithi029.stdout: "down_stamp": "0.000000", 2024-09-18T10:15:30.253 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_probability": 0, 2024-09-18T10:15:30.253 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_interval": 0, 2024-09-18T10:15:30.253 INFO:teuthology.orchestra.run.smithi029.stdout: "features": 4540138320759226367, 2024-09-18T10:15:30.253 INFO:teuthology.orchestra.run.smithi029.stdout: "old_weight": 0, 2024-09-18T10:15:30.253 INFO:teuthology.orchestra.run.smithi029.stdout: "last_purged_snaps_scrub": "2024-09-18T10:14:28.345864+0000", 2024-09-18T10:15:30.253 INFO:teuthology.orchestra.run.smithi029.stdout: "dead_epoch": 0 2024-09-18T10:15:30.253 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.253 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.253 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 11, 2024-09-18T10:15:30.253 INFO:teuthology.orchestra.run.smithi029.stdout: "down_stamp": "0.000000", 2024-09-18T10:15:30.253 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_probability": 0, 2024-09-18T10:15:30.253 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_interval": 0, 2024-09-18T10:15:30.254 INFO:teuthology.orchestra.run.smithi029.stdout: "features": 4540138320759226367, 2024-09-18T10:15:30.254 INFO:teuthology.orchestra.run.smithi029.stdout: "old_weight": 0, 2024-09-18T10:15:30.254 INFO:teuthology.orchestra.run.smithi029.stdout: "last_purged_snaps_scrub": "2024-09-18T10:14:28.131654+0000", 2024-09-18T10:15:30.254 INFO:teuthology.orchestra.run.smithi029.stdout: "dead_epoch": 0 2024-09-18T10:15:30.254 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.254 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.254 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 12, 2024-09-18T10:15:30.254 INFO:teuthology.orchestra.run.smithi029.stdout: "down_stamp": "0.000000", 2024-09-18T10:15:30.254 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_probability": 0, 2024-09-18T10:15:30.254 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_interval": 0, 2024-09-18T10:15:30.254 INFO:teuthology.orchestra.run.smithi029.stdout: "features": 4540138320759226367, 2024-09-18T10:15:30.254 INFO:teuthology.orchestra.run.smithi029.stdout: "old_weight": 0, 2024-09-18T10:15:30.254 INFO:teuthology.orchestra.run.smithi029.stdout: "last_purged_snaps_scrub": "2024-09-18T10:14:28.311441+0000", 2024-09-18T10:15:30.255 INFO:teuthology.orchestra.run.smithi029.stdout: "dead_epoch": 0 2024-09-18T10:15:30.255 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.255 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.255 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 13, 2024-09-18T10:15:30.255 INFO:teuthology.orchestra.run.smithi029.stdout: "down_stamp": "0.000000", 2024-09-18T10:15:30.255 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_probability": 0, 2024-09-18T10:15:30.255 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_interval": 0, 2024-09-18T10:15:30.255 INFO:teuthology.orchestra.run.smithi029.stdout: "features": 4540138320759226367, 2024-09-18T10:15:30.255 INFO:teuthology.orchestra.run.smithi029.stdout: "old_weight": 0, 2024-09-18T10:15:30.255 INFO:teuthology.orchestra.run.smithi029.stdout: "last_purged_snaps_scrub": "2024-09-18T10:14:28.483117+0000", 2024-09-18T10:15:30.255 INFO:teuthology.orchestra.run.smithi029.stdout: "dead_epoch": 0 2024-09-18T10:15:30.255 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.255 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.255 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 14, 2024-09-18T10:15:30.256 INFO:teuthology.orchestra.run.smithi029.stdout: "down_stamp": "0.000000", 2024-09-18T10:15:30.256 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_probability": 0, 2024-09-18T10:15:30.256 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_interval": 0, 2024-09-18T10:15:30.256 INFO:teuthology.orchestra.run.smithi029.stdout: "features": 4540138320759226367, 2024-09-18T10:15:30.256 INFO:teuthology.orchestra.run.smithi029.stdout: "old_weight": 0, 2024-09-18T10:15:30.256 INFO:teuthology.orchestra.run.smithi029.stdout: "last_purged_snaps_scrub": "2024-09-18T10:14:28.510528+0000", 2024-09-18T10:15:30.256 INFO:teuthology.orchestra.run.smithi029.stdout: "dead_epoch": 0 2024-09-18T10:15:30.256 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.256 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.256 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 15, 2024-09-18T10:15:30.256 INFO:teuthology.orchestra.run.smithi029.stdout: "down_stamp": "0.000000", 2024-09-18T10:15:30.256 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_probability": 0, 2024-09-18T10:15:30.256 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_interval": 0, 2024-09-18T10:15:30.256 INFO:teuthology.orchestra.run.smithi029.stdout: "features": 4540138320759226367, 2024-09-18T10:15:30.257 INFO:teuthology.orchestra.run.smithi029.stdout: "old_weight": 0, 2024-09-18T10:15:30.257 INFO:teuthology.orchestra.run.smithi029.stdout: "last_purged_snaps_scrub": "2024-09-18T10:14:28.398477+0000", 2024-09-18T10:15:30.257 INFO:teuthology.orchestra.run.smithi029.stdout: "dead_epoch": 0 2024-09-18T10:15:30.257 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:30.257 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-18T10:15:30.257 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_upmap": [], 2024-09-18T10:15:30.257 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_upmap_items": [], 2024-09-18T10:15:30.257 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_temp": [], 2024-09-18T10:15:30.257 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_temp": [], 2024-09-18T10:15:30.257 INFO:teuthology.orchestra.run.smithi029.stdout: "blocklist": {}, 2024-09-18T10:15:30.257 INFO:teuthology.orchestra.run.smithi029.stdout: "range_blocklist": {}, 2024-09-18T10:15:30.257 INFO:teuthology.orchestra.run.smithi029.stdout: "erasure_code_profiles": { 2024-09-18T10:15:30.257 INFO:teuthology.orchestra.run.smithi029.stdout: "default": { 2024-09-18T10:15:30.257 INFO:teuthology.orchestra.run.smithi029.stdout: "crush-failure-domain": "osd", 2024-09-18T10:15:30.258 INFO:teuthology.orchestra.run.smithi029.stdout: "k": "2", 2024-09-18T10:15:30.258 INFO:teuthology.orchestra.run.smithi029.stdout: "m": "1", 2024-09-18T10:15:30.258 INFO:teuthology.orchestra.run.smithi029.stdout: "plugin": "jerasure", 2024-09-18T10:15:30.258 INFO:teuthology.orchestra.run.smithi029.stdout: "technique": "reed_sol_van" 2024-09-18T10:15:30.258 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.258 INFO:teuthology.orchestra.run.smithi029.stdout: "shecprofile": { 2024-09-18T10:15:30.258 INFO:teuthology.orchestra.run.smithi029.stdout: "c": "2", 2024-09-18T10:15:30.258 INFO:teuthology.orchestra.run.smithi029.stdout: "crush-device-class": "", 2024-09-18T10:15:30.258 INFO:teuthology.orchestra.run.smithi029.stdout: "crush-failure-domain": "osd", 2024-09-18T10:15:30.258 INFO:teuthology.orchestra.run.smithi029.stdout: "crush-root": "default", 2024-09-18T10:15:30.258 INFO:teuthology.orchestra.run.smithi029.stdout: "k": "4", 2024-09-18T10:15:30.258 INFO:teuthology.orchestra.run.smithi029.stdout: "m": "3", 2024-09-18T10:15:30.258 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "shecprofile", 2024-09-18T10:15:30.259 INFO:teuthology.orchestra.run.smithi029.stdout: "plugin": "shec", 2024-09-18T10:15:30.259 INFO:teuthology.orchestra.run.smithi029.stdout: "technique": "multiple" 2024-09-18T10:15:30.259 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:30.259 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.259 INFO:teuthology.orchestra.run.smithi029.stdout: "removed_snaps_queue": [ 2024-09-18T10:15:30.259 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.259 INFO:teuthology.orchestra.run.smithi029.stdout: "pool": 3, 2024-09-18T10:15:30.259 INFO:teuthology.orchestra.run.smithi029.stdout: "snaps": [ 2024-09-18T10:15:30.259 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.259 INFO:teuthology.orchestra.run.smithi029.stdout: "begin": 15, 2024-09-18T10:15:30.259 INFO:teuthology.orchestra.run.smithi029.stdout: "length": 1 2024-09-18T10:15:30.259 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.259 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.259 INFO:teuthology.orchestra.run.smithi029.stdout: "begin": 18, 2024-09-18T10:15:30.260 INFO:teuthology.orchestra.run.smithi029.stdout: "length": 1 2024-09-18T10:15:30.260 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:30.260 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.260 INFO:teuthology.orchestra.run.smithi029.stdout: "begin": 33, 2024-09-18T10:15:30.260 INFO:teuthology.orchestra.run.smithi029.stdout: "length": 1 2024-09-18T10:15:30.260 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:30.260 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:30.260 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:30.260 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-18T10:15:30.260 INFO:teuthology.orchestra.run.smithi029.stdout: "new_removed_snaps": [ 2024-09-18T10:15:30.260 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.260 INFO:teuthology.orchestra.run.smithi029.stdout: "pool": 3, 2024-09-18T10:15:30.260 INFO:teuthology.orchestra.run.smithi029.stdout: "snaps": [ 2024-09-18T10:15:30.260 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:30.261 INFO:teuthology.orchestra.run.smithi029.stdout: "begin": 18, 2024-09-18T10:15:30.261 INFO:teuthology.orchestra.run.smithi029.stdout: "length": 1 2024-09-18T10:15:30.261 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:30.261 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:30.261 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:30.261 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-18T10:15:30.261 INFO:teuthology.orchestra.run.smithi029.stdout: "new_purged_snaps": [], 2024-09-18T10:15:30.261 INFO:teuthology.orchestra.run.smithi029.stdout: "crush_node_flags": {}, 2024-09-18T10:15:30.261 INFO:teuthology.orchestra.run.smithi029.stdout: "device_class_flags": {}, 2024-09-18T10:15:30.261 INFO:teuthology.orchestra.run.smithi029.stdout: "stretch_mode": { 2024-09-18T10:15:30.261 INFO:teuthology.orchestra.run.smithi029.stdout: "stretch_mode_enabled": false, 2024-09-18T10:15:30.261 INFO:teuthology.orchestra.run.smithi029.stdout: "stretch_bucket_count": 0, 2024-09-18T10:15:30.261 INFO:teuthology.orchestra.run.smithi029.stdout: "degraded_stretch_mode": 0, 2024-09-18T10:15:30.261 INFO:teuthology.orchestra.run.smithi029.stdout: "recovering_stretch_mode": 0, 2024-09-18T10:15:30.262 INFO:teuthology.orchestra.run.smithi029.stdout: "stretch_mode_bucket": 0 2024-09-18T10:15:30.262 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:30.262 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-18T10:15:30.263 INFO:tasks.thrashosds.thrasher:j is {'epoch': 74, 'fsid': 'ab6d6beb-0627-4b7f-8cdb-4e0a44970f10', 'created': '2024-09-18T10:14:18.827644+0000', 'modified': '2024-09-18T10:15:29.548368+0000', 'last_up_change': '2024-09-18T10:14:30.228591+0000', 'last_in_change': '2024-09-18T10:14:22.927003+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 6, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 16, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'quincy', 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-18T10:14:31.266113+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-18T10:14:33.358485+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '29', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 29, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'unique_pool_0', 'create_time': '2024-09-18T10:14:44.624320+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 3, 'size': 7, 'min_size': 5, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 26, 'pg_placement_num': 26, 'pg_placement_num_target': 26, 'pg_num_target': 26, 'pg_num_pending': 26, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '74', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '40', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 36, 'snap_epoch': 74, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'shecprofile', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 16384, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rados': {}}}], 'osds': [{'osd': 0, 'uuid': '39cf0511-0009-433f-92f2-e5686f33d713', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 40, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6808', 'nonce': 22197}, {'type': 'v1', 'addr': '172.21.15.29:6809', 'nonce': 22197}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6810', 'nonce': 22197}, {'type': 'v1', 'addr': '172.21.15.29:6811', 'nonce': 22197}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6814', 'nonce': 22197}, {'type': 'v1', 'addr': '172.21.15.29:6815', 'nonce': 22197}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6812', 'nonce': 22197}, {'type': 'v1', 'addr': '172.21.15.29:6813', 'nonce': 22197}]}, 'public_addr': '172.21.15.29:6809/22197', 'cluster_addr': '172.21.15.29:6811/22197', 'heartbeat_back_addr': '172.21.15.29:6815/22197', 'heartbeat_front_addr': '172.21.15.29:6813/22197', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'a0bf06ab-b98e-4130-91d2-7d4283540c08', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 58, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6808', 'nonce': 22140}, {'type': 'v1', 'addr': '172.21.15.136:6809', 'nonce': 22140}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6810', 'nonce': 22140}, {'type': 'v1', 'addr': '172.21.15.136:6811', 'nonce': 22140}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6814', 'nonce': 22140}, {'type': 'v1', 'addr': '172.21.15.136:6815', 'nonce': 22140}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6812', 'nonce': 22140}, {'type': 'v1', 'addr': '172.21.15.136:6813', 'nonce': 22140}]}, 'public_addr': '172.21.15.136:6809/22140', 'cluster_addr': '172.21.15.136:6811/22140', 'heartbeat_back_addr': '172.21.15.136:6815/22140', 'heartbeat_front_addr': '172.21.15.136:6813/22140', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '742a786e-6129-4fbc-bb5b-e2e3c03c2d77', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6824', 'nonce': 22095}, {'type': 'v1', 'addr': '172.21.15.154:6825', 'nonce': 22095}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6826', 'nonce': 22095}, {'type': 'v1', 'addr': '172.21.15.154:6827', 'nonce': 22095}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6830', 'nonce': 22095}, {'type': 'v1', 'addr': '172.21.15.154:6831', 'nonce': 22095}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6828', 'nonce': 22095}, {'type': 'v1', 'addr': '172.21.15.154:6829', 'nonce': 22095}]}, 'public_addr': '172.21.15.154:6825/22095', 'cluster_addr': '172.21.15.154:6827/22095', 'heartbeat_back_addr': '172.21.15.154:6831/22095', 'heartbeat_front_addr': '172.21.15.154:6829/22095', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '3aa77114-24ed-49ab-86ff-a0600794cbf8', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 48, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6802', 'nonce': 22404}, {'type': 'v1', 'addr': '172.21.15.189:6803', 'nonce': 22404}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6804', 'nonce': 22404}, {'type': 'v1', 'addr': '172.21.15.189:6805', 'nonce': 22404}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6808', 'nonce': 22404}, {'type': 'v1', 'addr': '172.21.15.189:6809', 'nonce': 22404}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6806', 'nonce': 22404}, {'type': 'v1', 'addr': '172.21.15.189:6807', 'nonce': 22404}]}, 'public_addr': '172.21.15.189:6803/22404', 'cluster_addr': '172.21.15.189:6805/22404', 'heartbeat_back_addr': '172.21.15.189:6809/22404', 'heartbeat_front_addr': '172.21.15.189:6807/22404', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '426567ad-b0be-41d4-afcf-dc56ec3974cd', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 40, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6800', 'nonce': 22195}, {'type': 'v1', 'addr': '172.21.15.29:6801', 'nonce': 22195}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6802', 'nonce': 22195}, {'type': 'v1', 'addr': '172.21.15.29:6803', 'nonce': 22195}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6806', 'nonce': 22195}, {'type': 'v1', 'addr': '172.21.15.29:6807', 'nonce': 22195}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6804', 'nonce': 22195}, {'type': 'v1', 'addr': '172.21.15.29:6805', 'nonce': 22195}]}, 'public_addr': '172.21.15.29:6801/22195', 'cluster_addr': '172.21.15.29:6803/22195', 'heartbeat_back_addr': '172.21.15.29:6807/22195', 'heartbeat_front_addr': '172.21.15.29:6805/22195', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'd9559e42-1f4e-4373-8392-a536a0afd871', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 40, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6817', 'nonce': 22141}, {'type': 'v1', 'addr': '172.21.15.136:6819', 'nonce': 22141}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6821', 'nonce': 22141}, {'type': 'v1', 'addr': '172.21.15.136:6823', 'nonce': 22141}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6829', 'nonce': 22141}, {'type': 'v1', 'addr': '172.21.15.136:6831', 'nonce': 22141}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6825', 'nonce': 22141}, {'type': 'v1', 'addr': '172.21.15.136:6827', 'nonce': 22141}]}, 'public_addr': '172.21.15.136:6819/22141', 'cluster_addr': '172.21.15.136:6823/22141', 'heartbeat_back_addr': '172.21.15.136:6831/22141', 'heartbeat_front_addr': '172.21.15.136:6827/22141', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'd50ef1e2-8bc5-4a90-9be3-365a0db2be6f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 46, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6800', 'nonce': 22096}, {'type': 'v1', 'addr': '172.21.15.154:6801', 'nonce': 22096}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6802', 'nonce': 22096}, {'type': 'v1', 'addr': '172.21.15.154:6803', 'nonce': 22096}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6806', 'nonce': 22096}, {'type': 'v1', 'addr': '172.21.15.154:6807', 'nonce': 22096}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6804', 'nonce': 22096}, {'type': 'v1', 'addr': '172.21.15.154:6805', 'nonce': 22096}]}, 'public_addr': '172.21.15.154:6801/22096', 'cluster_addr': '172.21.15.154:6803/22096', 'heartbeat_back_addr': '172.21.15.154:6807/22096', 'heartbeat_front_addr': '172.21.15.154:6805/22096', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'bd0b85e4-cfee-4d19-90a5-79854794b48b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 40, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6818', 'nonce': 22405}, {'type': 'v1', 'addr': '172.21.15.189:6819', 'nonce': 22405}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6820', 'nonce': 22405}, {'type': 'v1', 'addr': '172.21.15.189:6821', 'nonce': 22405}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6824', 'nonce': 22405}, {'type': 'v1', 'addr': '172.21.15.189:6825', 'nonce': 22405}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6822', 'nonce': 22405}, {'type': 'v1', 'addr': '172.21.15.189:6823', 'nonce': 22405}]}, 'public_addr': '172.21.15.189:6819/22405', 'cluster_addr': '172.21.15.189:6821/22405', 'heartbeat_back_addr': '172.21.15.189:6825/22405', 'heartbeat_front_addr': '172.21.15.189:6823/22405', 'state': ['exists', 'up']}, {'osd': 8, 'uuid': 'cd333f56-c165-45d7-8c8b-48d34bc4ddc5', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 40, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6824', 'nonce': 22200}, {'type': 'v1', 'addr': '172.21.15.29:6825', 'nonce': 22200}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6826', 'nonce': 22200}, {'type': 'v1', 'addr': '172.21.15.29:6827', 'nonce': 22200}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6830', 'nonce': 22200}, {'type': 'v1', 'addr': '172.21.15.29:6831', 'nonce': 22200}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6828', 'nonce': 22200}, {'type': 'v1', 'addr': '172.21.15.29:6829', 'nonce': 22200}]}, 'public_addr': '172.21.15.29:6825/22200', 'cluster_addr': '172.21.15.29:6827/22200', 'heartbeat_back_addr': '172.21.15.29:6831/22200', 'heartbeat_front_addr': '172.21.15.29:6829/22200', 'state': ['exists', 'up']}, {'osd': 9, 'uuid': '1e079e4c-8801-4754-9d0d-d0b68695d32e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 62, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6800', 'nonce': 22142}, {'type': 'v1', 'addr': '172.21.15.136:6801', 'nonce': 22142}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6802', 'nonce': 22142}, {'type': 'v1', 'addr': '172.21.15.136:6803', 'nonce': 22142}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6806', 'nonce': 22142}, {'type': 'v1', 'addr': '172.21.15.136:6807', 'nonce': 22142}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6804', 'nonce': 22142}, {'type': 'v1', 'addr': '172.21.15.136:6805', 'nonce': 22142}]}, 'public_addr': '172.21.15.136:6801/22142', 'cluster_addr': '172.21.15.136:6803/22142', 'heartbeat_back_addr': '172.21.15.136:6807/22142', 'heartbeat_front_addr': '172.21.15.136:6805/22142', 'state': ['exists', 'up']}, {'osd': 10, 'uuid': '92ee68d4-371c-4849-930f-a8fa18d64b99', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 72, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6808', 'nonce': 22097}, {'type': 'v1', 'addr': '172.21.15.154:6809', 'nonce': 22097}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6810', 'nonce': 22097}, {'type': 'v1', 'addr': '172.21.15.154:6811', 'nonce': 22097}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6814', 'nonce': 22097}, {'type': 'v1', 'addr': '172.21.15.154:6815', 'nonce': 22097}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6812', 'nonce': 22097}, {'type': 'v1', 'addr': '172.21.15.154:6813', 'nonce': 22097}]}, 'public_addr': '172.21.15.154:6809/22097', 'cluster_addr': '172.21.15.154:6811/22097', 'heartbeat_back_addr': '172.21.15.154:6815/22097', 'heartbeat_front_addr': '172.21.15.154:6813/22097', 'state': ['exists', 'up']}, {'osd': 11, 'uuid': '634ddaf5-f7b8-4b27-a46a-04e21b73fcf0', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 32, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6826', 'nonce': 22416}, {'type': 'v1', 'addr': '172.21.15.189:6827', 'nonce': 22416}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6828', 'nonce': 22416}, {'type': 'v1', 'addr': '172.21.15.189:6829', 'nonce': 22416}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6832', 'nonce': 22416}, {'type': 'v1', 'addr': '172.21.15.189:6833', 'nonce': 22416}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6830', 'nonce': 22416}, {'type': 'v1', 'addr': '172.21.15.189:6831', 'nonce': 22416}]}, 'public_addr': '172.21.15.189:6827/22416', 'cluster_addr': '172.21.15.189:6829/22416', 'heartbeat_back_addr': '172.21.15.189:6833/22416', 'heartbeat_front_addr': '172.21.15.189:6831/22416', 'state': ['exists', 'up']}, {'osd': 12, 'uuid': 'bc58137b-5948-400d-b377-193b15824d8d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6816', 'nonce': 22210}, {'type': 'v1', 'addr': '172.21.15.29:6817', 'nonce': 22210}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6818', 'nonce': 22210}, {'type': 'v1', 'addr': '172.21.15.29:6819', 'nonce': 22210}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6822', 'nonce': 22210}, {'type': 'v1', 'addr': '172.21.15.29:6823', 'nonce': 22210}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6820', 'nonce': 22210}, {'type': 'v1', 'addr': '172.21.15.29:6821', 'nonce': 22210}]}, 'public_addr': '172.21.15.29:6817/22210', 'cluster_addr': '172.21.15.29:6819/22210', 'heartbeat_back_addr': '172.21.15.29:6823/22210', 'heartbeat_front_addr': '172.21.15.29:6821/22210', 'state': ['exists', 'up']}, {'osd': 13, 'uuid': 'a8b753a7-ab69-4853-ac36-66d9b85c2e20', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 32, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6816', 'nonce': 22143}, {'type': 'v1', 'addr': '172.21.15.136:6818', 'nonce': 22143}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6820', 'nonce': 22143}, {'type': 'v1', 'addr': '172.21.15.136:6822', 'nonce': 22143}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6828', 'nonce': 22143}, {'type': 'v1', 'addr': '172.21.15.136:6830', 'nonce': 22143}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6824', 'nonce': 22143}, {'type': 'v1', 'addr': '172.21.15.136:6826', 'nonce': 22143}]}, 'public_addr': '172.21.15.136:6818/22143', 'cluster_addr': '172.21.15.136:6822/22143', 'heartbeat_back_addr': '172.21.15.136:6830/22143', 'heartbeat_front_addr': '172.21.15.136:6826/22143', 'state': ['exists', 'up']}, {'osd': 14, 'uuid': '8ad0848f-0de4-4a75-9c63-8c4a4a7b8524', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 60, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6816', 'nonce': 22098}, {'type': 'v1', 'addr': '172.21.15.154:6817', 'nonce': 22098}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6818', 'nonce': 22098}, {'type': 'v1', 'addr': '172.21.15.154:6819', 'nonce': 22098}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6822', 'nonce': 22098}, {'type': 'v1', 'addr': '172.21.15.154:6823', 'nonce': 22098}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6820', 'nonce': 22098}, {'type': 'v1', 'addr': '172.21.15.154:6821', 'nonce': 22098}]}, 'public_addr': '172.21.15.154:6817/22098', 'cluster_addr': '172.21.15.154:6819/22098', 'heartbeat_back_addr': '172.21.15.154:6823/22098', 'heartbeat_front_addr': '172.21.15.154:6821/22098', 'state': ['exists', 'up']}, {'osd': 15, 'uuid': '069099b3-ec1a-483b-b967-98cddbf2f678', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6810', 'nonce': 22415}, {'type': 'v1', 'addr': '172.21.15.189:6811', 'nonce': 22415}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6812', 'nonce': 22415}, {'type': 'v1', 'addr': '172.21.15.189:6813', 'nonce': 22415}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6816', 'nonce': 22415}, {'type': 'v1', 'addr': '172.21.15.189:6817', 'nonce': 22415}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6814', 'nonce': 22415}, {'type': 'v1', 'addr': '172.21.15.189:6815', 'nonce': 22415}]}, 'public_addr': '172.21.15.189:6811/22415', 'cluster_addr': '172.21.15.189:6813/22415', 'heartbeat_back_addr': '172.21.15.189:6817/22415', 'heartbeat_front_addr': '172.21.15.189:6815/22415', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T10:14:28.132367+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T10:14:28.342192+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T10:14:28.486853+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T10:14:28.299130+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T10:14:28.027740+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T10:14:28.450120+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T10:14:28.412452+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T10:14:28.368142+0000', 'dead_epoch': 0}, {'osd': 8, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T10:14:28.238445+0000', 'dead_epoch': 0}, {'osd': 9, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T10:14:28.096605+0000', 'dead_epoch': 0}, {'osd': 10, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T10:14:28.345864+0000', 'dead_epoch': 0}, {'osd': 11, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T10:14:28.131654+0000', 'dead_epoch': 0}, {'osd': 12, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T10:14:28.311441+0000', 'dead_epoch': 0}, {'osd': 13, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T10:14:28.483117+0000', 'dead_epoch': 0}, {'osd': 14, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T10:14:28.510528+0000', 'dead_epoch': 0}, {'osd': 15, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T10:14:28.398477+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}, 'shecprofile': {'c': '2', 'crush-device-class': '', 'crush-failure-domain': 'osd', 'crush-root': 'default', 'k': '4', 'm': '3', 'name': 'shecprofile', 'plugin': 'shec', 'technique': 'multiple'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 15, 'length': 1}, {'begin': 18, 'length': 1}, {'begin': 33, 'length': 1}]}], 'new_removed_snaps': [{'pool': 3, 'snaps': [{'begin': 18, 'length': 1}]}], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-18T10:15:30.263 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T10:15:30.264 INFO:tasks.ceph.osd.12.smithi029.stderr:2024-09-18T10:15:30.130+0000 7f8e4e627700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 22190) UID: 0 2024-09-18T10:15:30.266 INFO:tasks.ceph.osd.15.smithi189.stderr:2024-09-18T10:15:30.262+0000 7fd0e725e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 22401) UID: 0 2024-09-18T10:15:30.366 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-18T10:15:30.366 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-18T10:15:30.366 INFO:teuthology.orchestra.run.smithi189.stdout: "num_ops": 0 2024-09-18T10:15:30.366 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-18T10:15:30.367 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-09-18T10:15:30.363+0000 7fdfa8698700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22087) UID: 0 2024-09-18T10:15:30.375 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T10:15:30.406 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-18T10:15:30.428 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-18T10:15:30.452 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-18T10:15:30.468 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-18T10:15:30.464+0000 7f14c6a1a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 22138) UID: 0 2024-09-18T10:15:30.472 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-18T10:15:30.473 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-18T10:15:30.473 INFO:teuthology.orchestra.run.smithi189.stdout: "complaint_time": 30, 2024-09-18T10:15:30.473 INFO:teuthology.orchestra.run.smithi189.stdout: "num_blocked_ops": 0 2024-09-18T10:15:30.473 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-18T10:15:30.482 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T10:15:30.502 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-18T10:15:30.527 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-18T10:15:30.558 INFO:tasks.rados.rados.0.smithi189.stdout:270: done (9 left) 2024-09-18T10:15:30.559 INFO:tasks.rados.rados.0.smithi189.stdout:271: snap_create 2024-09-18T10:15:30.559 INFO:tasks.rados.rados.0.smithi189.stdout:268: finishing copy_from to smithi18925896-34 2024-09-18T10:15:30.559 INFO:tasks.rados.rados.0.smithi189.stdout:268: got expected ENOENT (src dne) 2024-09-18T10:15:30.559 INFO:tasks.rados.rados.0.smithi189.stdout:268: finishing copy_from racing read to smithi18925896-34 2024-09-18T10:15:30.559 INFO:tasks.rados.rados.0.smithi189.stdout:258: finishing write tid 5 to smithi18925896-20 2024-09-18T10:15:30.559 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 20 v 19 (ObjNum 107 snap 21 seq_num 107) dirty exists 2024-09-18T10:15:30.559 INFO:tasks.rados.rados.0.smithi189.stdout:258: left oid 20 (ObjNum 107 snap 21 seq_num 107) 2024-09-18T10:15:30.559 INFO:tasks.rados.rados.0.smithi189.stdout:263: finishing rollback tid 0 to smithi18925896-49 2024-09-18T10:15:30.559 INFO:tasks.rados.rados.0.smithi189.stdout:263: finishing rollback tid 1 to smithi18925896-49 2024-09-18T10:15:30.559 INFO:tasks.rados.rados.0.smithi189.stdout:266: finishing rollback tid 0 to smithi18925896-33 2024-09-18T10:15:30.559 INFO:tasks.rados.rados.0.smithi189.stdout:266: finishing rollback tid 1 to smithi18925896-33 2024-09-18T10:15:30.560 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 23 v 93 (ObjNum 53 snap 1 seq_num 53) dirty exists 2024-09-18T10:15:30.560 INFO:tasks.rados.rados.0.smithi189.stdout:266: finishing rollback tid 2 to smithi18925896-33 2024-09-18T10:15:30.560 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 33 v 42 (ObjNum 15 snap 0 seq_num 15) dirty exists 2024-09-18T10:15:30.560 INFO:tasks.rados.rados.0.smithi189.stdout:269: finishing rollback tid 0 to smithi18925896-43 2024-09-18T10:15:30.560 INFO:tasks.rados.rados.0.smithi189.stdout:264: finishing rollback tid 1 to smithi18925896-40 2024-09-18T10:15:30.560 INFO:tasks.rados.rados.0.smithi189.stdout:269: finishing rollback tid 1 to smithi18925896-43 2024-09-18T10:15:30.560 INFO:tasks.rados.rados.0.smithi189.stdout:269: finishing rollback tid 2 to smithi18925896-43 2024-09-18T10:15:30.560 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 43 v 45 (ObjNum 42 snap 0 seq_num 42) dirty exists 2024-09-18T10:15:30.560 INFO:tasks.rados.rados.0.smithi189.stdout:261: finishing write tid 1 to smithi18925896-25 2024-09-18T10:15:30.560 INFO:tasks.rados.rados.0.smithi189.stdout:261: finishing write tid 2 to smithi18925896-25 2024-09-18T10:15:30.560 INFO:tasks.rados.rados.0.smithi189.stdout:261: finishing write tid 3 to smithi18925896-25 2024-09-18T10:15:30.560 INFO:tasks.rados.rados.0.smithi189.stdout:261: finishing write tid 4 to smithi18925896-25 2024-09-18T10:15:30.560 INFO:tasks.rados.rados.0.smithi189.stdout:261: finishing write tid 5 to smithi18925896-25 2024-09-18T10:15:30.560 INFO:tasks.rados.rados.0.smithi189.stdout:264: finishing rollback tid 2 to smithi18925896-40 2024-09-18T10:15:30.561 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 40 v 27 (ObjNum 39 snap 0 seq_num 39) dirty exists 2024-09-18T10:15:30.567 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-18T10:15:30.568 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-18T10:15:30.564+0000 7f578c52f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22134) UID: 0 2024-09-18T10:15:30.572 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-09-18T10:15:30.573 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-18T10:15:30.577 INFO:tasks.rados.rados.0.smithi189.stdout:263: finishing rollback tid 2 to smithi18925896-49 2024-09-18T10:15:30.577 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 49 v 43 (ObjNum 48 snap 0 seq_num 48) dirty exists 2024-09-18T10:15:30.581 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:30.582 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:30.585 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":89,"stamp":"2024-09-18T10:15:29.552554+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":64692787,"num_objects":86,"num_object_clones":41,"num_object_copies":582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86,"num_whiteouts":8,"num_read":298,"num_read_kb":44309,"num_write":518,"num_write_kb":57050,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":6012928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":914,"ondisk_log_size":914,"up":200,"acting":200,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":206,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":16,"kb":1509949440,"kb_used":4893080,"kb_used_data":131160,"kb_used_omap":26,"kb_used_meta":4761637,"kb_avail":1505056360,"statfs":{"total":1546188226560,"available":1541177712640,"internally_reserved":0,"allocated":134307840,"data_stored":167036909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":27264,"internal_metadata":4875916672},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14,"num_read_kb":5786,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.601671"},"pg_stats":[{"pgid":"3.19","version":"68'92","reported_seq":210,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-18T10:15:24.873328+0000","last_change":"2024-09-18T10:15:23.459301+0000","last_active":"2024-09-18T10:15:24.873328+0000","last_peered":"2024-09-18T10:15:24.873328+0000","last_clean":"2024-09-18T10:15:24.873328+0000","last_became_active":"2024-09-18T10:14:57.350904+0000","last_became_peered":"2024-09-18T10:14:57.350904+0000","last_unstale":"2024-09-18T10:15:24.873328+0000","last_undegraded":"2024-09-18T10:15:24.873328+0000","last_fullsized":"2024-09-18T10:15:24.873328+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:14:45.576168+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.41041204199999998,"stat_sum":{"num_bytes":9592832,"num_objects":10,"num_object_clones":7,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":25,"num_read_kb":7476,"num_write":57,"num_write_kb":10280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,10,11,5,8,3],"acting":[7,2,10,11,5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"e","length":"1"},{"start":"13","length":"1"},{"start":"18","length":"2"}]},{"pgid":"3.18","version":"68'36","reported_seq":159,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-18T10:15:25.252708+0000","last_change":"2024-09-18T10:15:22.137237+0000","last_active":"2024-09-18T10:15:25.252708+0000","last_peered":"2024-09-18T10:15:25.252708+0000","last_clean":"2024-09-18T10:15:25.252708+0000","last_became_active":"2024-09-18T10:14:57.647465+0000","last_became_peered":"2024-09-18T10:14:57.647465+0000","last_unstale":"2024-09-18T10:15:25.252708+0000","last_undegraded":"2024-09-18T10:15:25.252708+0000","last_fullsized":"2024-09-18T10:15:25.252708+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:14:45.576168+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.116355192,"stat_sum":{"num_bytes":3997696,"num_objects":5,"num_object_clones":3,"num_object_copies":35,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":1,"num_read":13,"num_read_kb":2309,"num_write":26,"num_write_kb":3920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,11,3,15,1,7],"acting":[6,0,11,3,15,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"e","length":"1"},{"start":"13","length":"1"},{"start":"18","length":"2"}]},{"pgid":"3.17","version":"68'35","reported_seq":76,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-18T10:15:25.001073+0000","last_change":"2024-09-18T10:15:25.001073+0000","last_active":"2024-09-18T10:15:25.001073+0000","last_peered":"2024-09-18T10:15:25.001073+0000","last_clean":"2024-09-18T10:15:25.001073+0000","last_became_active":"2024-09-18T10:15:21.323733+0000","last_became_peered":"2024-09-18T10:15:21.323733+0000","last_unstale":"2024-09-18T10:15:25.001073+0000","last_undegraded":"2024-09-18T10:15:25.001073+0000","last_fullsized":"2024-09-18T10:15:25.001073+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:14:45.576168+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.41336515200000001,"stat_sum":{"num_bytes":2883584,"num_objects":4,"num_object_clones":2,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":1,"num_read":13,"num_read_kb":4757,"num_write":13,"num_write_kb":3024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":4374528,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,13,3,14,2,11,6],"acting":[10,13,3,14,2,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"e","length":"1"},{"start":"13","length":"1"},{"start":"18","length":"2"}]},{"pgid":"3.16","version":"71'20","reported_seq":70,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-18T10:15:28.966389+0000","last_change":"2024-09-18T10:15:28.186914+0000","last_active":"2024-09-18T10:15:28.966389+0000","last_peered":"2024-09-18T10:15:28.966389+0000","last_clean":"2024-09-18T10:15:28.966389+0000","last_became_active":"2024-09-18T10:15:19.227471+0000","last_became_peered":"2024-09-18T10:15:19.227471+0000","last_unstale":"2024-09-18T10:15:28.966389+0000","last_undegraded":"2024-09-18T10:15:28.966389+0000","last_fullsized":"2024-09-18T10:15:28.966389+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":5,"last_scrub":"36'11","last_scrub_stamp":"2024-09-18T10:15:07.445786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:07.445786+0000","objects_scrubbed":1,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:23.145992+0000","scrub_duration":0.30645798400000002,"objects_trimmed":0,"snaptrim_duration":0.001563779,"stat_sum":{"num_bytes":2621440,"num_objects":2,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":1888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":688128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,6,4,8,2,14,10],"acting":[9,6,4,8,2,14,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[{"start":"f","length":"1"},{"start":"21","length":"1"}]},{"pgid":"3.15","version":"66'10","reported_seq":60,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-18T10:15:25.100987+0000","last_change":"2024-09-18T10:15:22.238714+0000","last_active":"2024-09-18T10:15:25.100987+0000","last_peered":"2024-09-18T10:15:25.100987+0000","last_clean":"2024-09-18T10:15:25.100987+0000","last_became_active":"2024-09-18T10:15:17.127771+0000","last_became_peered":"2024-09-18T10:15:17.127771+0000","last_unstale":"2024-09-18T10:15:25.100987+0000","last_undegraded":"2024-09-18T10:15:25.100987+0000","last_fullsized":"2024-09-18T10:15:25.100987+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:14:45.576168+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.21260638700000001,"stat_sum":{"num_bytes":32768,"num_objects":2,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":1,"num_read":2,"num_read_kb":0,"num_write":2,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,15,13,10,1,7,9],"acting":[14,15,13,10,1,7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"e","length":"1"},{"start":"13","length":"1"},{"start":"18","length":"2"}]},{"pgid":"3.14","version":"37'8","reported_seq":54,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-18T10:15:23.962750+0000","last_change":"2024-09-18T10:15:21.930993+0000","last_active":"2024-09-18T10:15:23.962750+0000","last_peered":"2024-09-18T10:15:23.962750+0000","last_clean":"2024-09-18T10:15:23.962750+0000","last_became_active":"2024-09-18T10:15:14.767770+0000","last_became_peered":"2024-09-18T10:15:14.767770+0000","last_unstale":"2024-09-18T10:15:23.962750+0000","last_undegraded":"2024-09-18T10:15:23.962750+0000","last_fullsized":"2024-09-18T10:15:23.962750+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":5,"last_scrub":"37'8","last_scrub_stamp":"2024-09-18T10:15:17.041590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:17.041590+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:32.741813+0000","scrub_duration":0.60018000800000004,"objects_trimmed":0,"snaptrim_duration":0.0014793740000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,14,9,10,5,13,4],"acting":[1,14,9,10,5,13,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"e","length":"1"},{"start":"13","length":"1"},{"start":"18","length":"2"}]},{"pgid":"2.7","version":"0'0","reported_seq":55,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-18T10:15:25.206095+0000","last_change":"2024-09-18T10:14:37.427722+0000","last_active":"2024-09-18T10:15:25.206095+0000","last_peered":"2024-09-18T10:15:25.206095+0000","last_clean":"2024-09-18T10:15:25.206095+0000","last_became_active":"2024-09-18T10:14:35.348943+0000","last_became_peered":"2024-09-18T10:14:35.348943+0000","last_unstale":"2024-09-18T10:15:25.206095+0000","last_undegraded":"2024-09-18T10:15:25.206095+0000","last_fullsized":"2024-09-18T10:15:25.206095+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:14:34.288596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:03.161727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060442999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,10],"acting":[13,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"3.6","version":"57'32","reported_seq":167,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-18T10:15:23.069165+0000","last_change":"2024-09-18T10:15:21.935648+0000","last_active":"2024-09-18T10:15:23.069165+0000","last_peered":"2024-09-18T10:15:23.069165+0000","last_clean":"2024-09-18T10:15:23.069165+0000","last_became_active":"2024-09-18T10:14:57.139786+0000","last_became_peered":"2024-09-18T10:14:57.139786+0000","last_unstale":"2024-09-18T10:15:23.069165+0000","last_undegraded":"2024-09-18T10:15:23.069165+0000","last_fullsized":"2024-09-18T10:15:23.069165+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:14:45.576168+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010832179999999999,"stat_sum":{"num_bytes":868352,"num_objects":2,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":988,"num_write":11,"num_write_kb":1960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,14,6,2,5,9,15],"acting":[0,14,6,2,5,9,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"e","length":"1"},{"start":"13","length":"1"},{"start":"18","length":"2"}]},{"pgid":"2.6","version":"0'0","reported_seq":54,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-18T10:15:24.160814+0000","last_change":"2024-09-18T10:14:38.963300+0000","last_active":"2024-09-18T10:15:24.160814+0000","last_peered":"2024-09-18T10:15:24.160814+0000","last_clean":"2024-09-18T10:15:24.160814+0000","last_became_active":"2024-09-18T10:14:35.429859+0000","last_became_peered":"2024-09-18T10:14:35.429859+0000","last_unstale":"2024-09-18T10:15:24.160814+0000","last_undegraded":"2024-09-18T10:15:24.160814+0000","last_fullsized":"2024-09-18T10:15:24.160814+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:14:34.288596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:15:40.196292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037634499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11],"acting":[8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.7","version":"66'39","reported_seq":171,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-18T10:15:23.959984+0000","last_change":"2024-09-18T10:15:22.345767+0000","last_active":"2024-09-18T10:15:23.959984+0000","last_peered":"2024-09-18T10:15:23.959984+0000","last_clean":"2024-09-18T10:15:23.959984+0000","last_became_active":"2024-09-18T10:14:57.137776+0000","last_became_peered":"2024-09-18T10:14:57.137776+0000","last_unstale":"2024-09-18T10:15:23.959984+0000","last_undegraded":"2024-09-18T10:15:23.959984+0000","last_fullsized":"2024-09-18T10:15:23.959984+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"60'30","last_scrub_stamp":"2024-09-18T10:15:17.136535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:17.136535+0000","objects_scrubbed":3,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:35.499009+0000","scrub_duration":0.60680385400000003,"objects_trimmed":0,"snaptrim_duration":0.41230857199999998,"stat_sum":{"num_bytes":5324800,"num_objects":4,"num_object_clones":2,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6,0,14,10,13],"acting":[4,11,6,0,14,10,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"e","length":"1"},{"start":"13","length":"1"},{"start":"18","length":"2"}]},{"pgid":"2.5","version":"0'0","reported_seq":53,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-18T10:15:24.153802+0000","last_change":"2024-09-18T10:14:37.327877+0000","last_active":"2024-09-18T10:15:24.153802+0000","last_peered":"2024-09-18T10:15:24.153802+0000","last_clean":"2024-09-18T10:15:24.153802+0000","last_became_active":"2024-09-18T10:14:35.329198+0000","last_became_peered":"2024-09-18T10:14:35.329198+0000","last_unstale":"2024-09-18T10:15:24.153802+0000","last_undegraded":"2024-09-18T10:15:24.153802+0000","last_fullsized":"2024-09-18T10:15:24.153802+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:14:34.288596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:15:44.196046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062415699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"66'40","reported_seq":177,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-18T10:15:24.160601+0000","last_change":"2024-09-18T10:15:22.747902+0000","last_active":"2024-09-18T10:15:24.160601+0000","last_peered":"2024-09-18T10:15:24.160601+0000","last_clean":"2024-09-18T10:15:24.160601+0000","last_became_active":"2024-09-18T10:14:57.134641+0000","last_became_peered":"2024-09-18T10:14:57.134641+0000","last_unstale":"2024-09-18T10:15:24.160601+0000","last_undegraded":"2024-09-18T10:15:24.160601+0000","last_fullsized":"2024-09-18T10:15:24.160601+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"60'34","last_scrub_stamp":"2024-09-18T10:15:17.325003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:17.325003+0000","objects_scrubbed":6,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:18.565493+0000","scrub_duration":0.82233752999999998,"objects_trimmed":0,"snaptrim_duration":0.81593390099999996,"stat_sum":{"num_bytes":4456448,"num_objects":6,"num_object_clones":3,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":4,"num_read_kb":1815,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,15,4,2,6,10,7],"acting":[8,15,4,2,6,10,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"e","length":"1"},{"start":"13","length":"1"},{"start":"18","length":"2"}]},{"pgid":"2.4","version":"0'0","reported_seq":54,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-18T10:15:23.959787+0000","last_change":"2024-09-18T10:14:37.426192+0000","last_active":"2024-09-18T10:15:23.959787+0000","last_peered":"2024-09-18T10:15:23.959787+0000","last_clean":"2024-09-18T10:15:23.959787+0000","last_became_active":"2024-09-18T10:14:35.524102+0000","last_became_peered":"2024-09-18T10:14:35.524102+0000","last_unstale":"2024-09-18T10:15:23.959787+0000","last_undegraded":"2024-09-18T10:15:23.959787+0000","last_fullsized":"2024-09-18T10:15:23.959787+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:14:34.288596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:15:59.190171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.10077985,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.5","version":"35'2","reported_seq":126,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-18T10:15:23.936668+0000","last_change":"2024-09-18T10:15:22.136251+0000","last_active":"2024-09-18T10:15:23.936668+0000","last_peered":"2024-09-18T10:15:23.936668+0000","last_clean":"2024-09-18T10:15:23.936668+0000","last_became_active":"2024-09-18T10:14:57.135810+0000","last_became_peered":"2024-09-18T10:14:57.135810+0000","last_unstale":"2024-09-18T10:15:23.936668+0000","last_undegraded":"2024-09-18T10:15:23.936668+0000","last_fullsized":"2024-09-18T10:15:23.936668+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"35'2","last_scrub_stamp":"2024-09-18T10:15:03.886316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:03.886316+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:14.307127+0000","scrub_duration":0.35694589500000001,"objects_trimmed":0,"snaptrim_duration":0.101978342,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,4,9,12,0,11],"acting":[5,8,4,9,12,0,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"e","length":"1"},{"start":"13","length":"1"},{"start":"18","length":"2"}]},{"pgid":"3.3","version":"65'23","reported_seq":141,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-18T10:15:24.064082+0000","last_change":"2024-09-18T10:15:22.033003+0000","last_active":"2024-09-18T10:15:24.064082+0000","last_peered":"2024-09-18T10:15:24.064082+0000","last_clean":"2024-09-18T10:15:24.064082+0000","last_became_active":"2024-09-18T10:14:57.138546+0000","last_became_peered":"2024-09-18T10:14:57.138546+0000","last_unstale":"2024-09-18T10:15:24.064082+0000","last_undegraded":"2024-09-18T10:15:24.064082+0000","last_fullsized":"2024-09-18T10:15:24.064082+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:14:45.576168+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001309478,"stat_sum":{"num_bytes":516096,"num_objects":2,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":1,"num_write":6,"num_write_kb":784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4,13,11,2,5],"acting":[10,9,4,13,11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"e","length":"1"},{"start":"13","length":"1"},{"start":"18","length":"2"}]},{"pgid":"2.2","version":"29'2","reported_seq":57,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-18T10:15:25.105768+0000","last_change":"2024-09-18T10:14:37.429583+0000","last_active":"2024-09-18T10:15:25.105768+0000","last_peered":"2024-09-18T10:15:25.105768+0000","last_clean":"2024-09-18T10:15:25.105768+0000","last_became_active":"2024-09-18T10:14:35.715374+0000","last_became_peered":"2024-09-18T10:14:35.715374+0000","last_unstale":"2024-09-18T10:15:25.105768+0000","last_undegraded":"2024-09-18T10:15:25.105768+0000","last_fullsized":"2024-09-18T10:15:25.105768+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:14:34.288596+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:03.520789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002421224,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,12],"acting":[13,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"3.0","version":"66'18","reported_seq":149,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-18T10:15:24.058769+0000","last_change":"2024-09-18T10:15:22.138999+0000","last_active":"2024-09-18T10:15:24.058769+0000","last_peered":"2024-09-18T10:15:24.058769+0000","last_clean":"2024-09-18T10:15:24.058769+0000","last_became_active":"2024-09-18T10:14:57.444374+0000","last_became_peered":"2024-09-18T10:14:57.444374+0000","last_unstale":"2024-09-18T10:15:24.058769+0000","last_undegraded":"2024-09-18T10:15:24.058769+0000","last_fullsized":"2024-09-18T10:15:24.058769+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'4","last_scrub_stamp":"2024-09-18T10:15:05.668933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:05.668933+0000","objects_scrubbed":1,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:18.433780+0000","scrub_duration":2.6328342170000001,"objects_trimmed":0,"snaptrim_duration":0.20786020399999999,"stat_sum":{"num_bytes":212992,"num_objects":2,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":1,"num_read":2,"num_read_kb":0,"num_write":2,"num_write_kb":208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,9,2,6,15,4],"acting":[3,7,9,2,6,15,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"e","length":"1"},{"start":"13","length":"1"},{"start":"18","length":"2"}]},{"pgid":"2.1","version":"0'0","reported_seq":56,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-18T10:15:25.100469+0000","last_change":"2024-09-18T10:14:37.601046+0000","last_active":"2024-09-18T10:15:25.100469+0000","last_peered":"2024-09-18T10:15:25.100469+0000","last_clean":"2024-09-18T10:15:25.100469+0000","last_became_active":"2024-09-18T10:14:35.348262+0000","last_became_peered":"2024-09-18T10:14:35.348262+0000","last_unstale":"2024-09-18T10:15:25.100469+0000","last_undegraded":"2024-09-18T10:15:25.100469+0000","last_fullsized":"2024-09-18T10:15:25.100469+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:14:34.288596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:03.520789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063430400000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,0],"acting":[14,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"3.1","version":"68'30","reported_seq":151,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-18T10:15:24.166023+0000","last_change":"2024-09-18T10:15:22.436295+0000","last_active":"2024-09-18T10:15:24.166023+0000","last_peered":"2024-09-18T10:15:24.166023+0000","last_clean":"2024-09-18T10:15:24.166023+0000","last_became_active":"2024-09-18T10:14:57.038503+0000","last_became_peered":"2024-09-18T10:14:57.038503+0000","last_unstale":"2024-09-18T10:15:24.166023+0000","last_undegraded":"2024-09-18T10:15:24.166023+0000","last_fullsized":"2024-09-18T10:15:24.166023+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:14:45.576168+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.50270893999999999,"stat_sum":{"num_bytes":696320,"num_objects":3,"num_object_clones":2,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":10,"num_read_kb":1779,"num_write":8,"num_write_kb":680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,13,14,10,7,3,2],"acting":[4,13,14,10,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"e","length":"1"},{"start":"13","length":"1"},{"start":"18","length":"2"}]},{"pgid":"2.0","version":"0'0","reported_seq":59,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-18T10:15:28.084815+0000","last_change":"2024-09-18T10:14:37.600953+0000","last_active":"2024-09-18T10:15:28.084815+0000","last_peered":"2024-09-18T10:15:28.084815+0000","last_clean":"2024-09-18T10:15:28.084815+0000","last_became_active":"2024-09-18T10:14:35.349203+0000","last_became_peered":"2024-09-18T10:14:35.349203+0000","last_unstale":"2024-09-18T10:15:28.084815+0000","last_undegraded":"2024-09-18T10:15:28.084815+0000","last_fullsized":"2024-09-18T10:15:28.084815+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:14:34.288596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:15:37.868112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060130600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3],"acting":[9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2","version":"66'45","reported_seq":185,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-18T10:15:25.953811+0000","last_change":"2024-09-18T10:15:22.232917+0000","last_active":"2024-09-18T10:15:25.953811+0000","last_peered":"2024-09-18T10:15:25.953811+0000","last_clean":"2024-09-18T10:15:25.953811+0000","last_became_active":"2024-09-18T10:14:57.227520+0000","last_became_peered":"2024-09-18T10:14:57.227520+0000","last_unstale":"2024-09-18T10:15:25.953811+0000","last_undegraded":"2024-09-18T10:15:25.953811+0000","last_fullsized":"2024-09-18T10:15:25.953811+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'18","last_scrub_stamp":"2024-09-18T10:15:03.202100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:03.202100+0000","objects_scrubbed":6,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:12.722351+0000","scrub_duration":0.63960403700000001,"objects_trimmed":0,"snaptrim_duration":0.30826759100000001,"stat_sum":{"num_bytes":3112960,"num_objects":6,"num_object_clones":2,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":1,"num_read":15,"num_read_kb":4021,"num_write":7,"num_write_kb":1552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,3,13,1,5,10,7],"acting":[14,3,13,1,5,10,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"e","length":"1"},{"start":"13","length":"1"},{"start":"18","length":"2"}]},{"pgid":"2.3","version":"27'1","reported_seq":57,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-18T10:15:25.100599+0000","last_change":"2024-09-18T10:14:37.801252+0000","last_active":"2024-09-18T10:15:25.100599+0000","last_peered":"2024-09-18T10:15:25.100599+0000","last_clean":"2024-09-18T10:15:25.100599+0000","last_became_active":"2024-09-18T10:14:35.487746+0000","last_became_peered":"2024-09-18T10:14:35.487746+0000","last_unstale":"2024-09-18T10:15:25.100599+0000","last_undegraded":"2024-09-18T10:15:25.100599+0000","last_fullsized":"2024-09-18T10:15:25.100599+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:14:34.288596+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:03.161727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.10072764100000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,5],"acting":[14,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"1.0","version":"29'112","reported_seq":189,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-18T10:15:23.939109+0000","last_change":"2024-09-18T10:14:32.509942+0000","last_active":"2024-09-18T10:15:23.939109+0000","last_peered":"2024-09-18T10:15:23.939109+0000","last_clean":"2024-09-18T10:15:23.939109+0000","last_became_active":"2024-09-18T10:14:32.508037+0000","last_became_peered":"2024-09-18T10:14:32.508037+0000","last_unstale":"2024-09-18T10:15:23.939109+0000","last_undegraded":"2024-09-18T10:15:23.939109+0000","last_fullsized":"2024-09-18T10:15:23.939109+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:31.267741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:31.267741+0000","last_clean_scrub_stamp":"2024-09-18T10:14:31.267741+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:15:59.262610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,15],"acting":[6,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.8","version":"41'13","reported_seq":142,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-18T10:15:24.039045+0000","last_change":"2024-09-18T10:15:22.123091+0000","last_active":"2024-09-18T10:15:24.039045+0000","last_peered":"2024-09-18T10:15:24.039045+0000","last_clean":"2024-09-18T10:15:24.039045+0000","last_became_active":"2024-09-18T10:14:57.443754+0000","last_became_peered":"2024-09-18T10:14:57.443754+0000","last_unstale":"2024-09-18T10:15:24.039045+0000","last_undegraded":"2024-09-18T10:15:24.039045+0000","last_fullsized":"2024-09-18T10:15:24.039045+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"41'13","last_scrub_stamp":"2024-09-18T10:15:05.373444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:05.373444+0000","objects_scrubbed":2,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:31.932850+0000","scrub_duration":2.8745114210000002,"objects_trimmed":0,"snaptrim_duration":0.0012354600000000001,"stat_sum":{"num_bytes":966656,"num_objects":2,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":705,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,11,3,15,1,7],"acting":[6,0,11,3,15,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"e","length":"1"},{"start":"13","length":"1"},{"start":"18","length":"2"}]},{"pgid":"3.9","version":"69'68","reported_seq":197,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-18T10:15:25.146780+0000","last_change":"2024-09-18T10:15:23.152029+0000","last_active":"2024-09-18T10:15:25.146780+0000","last_peered":"2024-09-18T10:15:25.146780+0000","last_clean":"2024-09-18T10:15:25.146780+0000","last_became_active":"2024-09-18T10:14:56.840271+0000","last_became_peered":"2024-09-18T10:14:56.840271+0000","last_unstale":"2024-09-18T10:15:25.146780+0000","last_undegraded":"2024-09-18T10:15:25.146780+0000","last_fullsized":"2024-09-18T10:15:25.146780+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'35","last_scrub_stamp":"2024-09-18T10:15:07.077517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:07.077517+0000","objects_scrubbed":4,"log_size":68,"log_dups_size":0,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:31.443309+0000","scrub_duration":0.71488955499999995,"objects_trimmed":0,"snaptrim_duration":0.0014440639999999999,"stat_sum":{"num_bytes":6766592,"num_objects":8,"num_object_clones":6,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":12,"num_read_kb":5292,"num_write":29,"num_write_kb":5392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,10,11,5,8,3],"acting":[7,2,10,11,5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"e","length":"1"},{"start":"13","length":"1"},{"start":"18","length":"2"}]},{"pgid":"3.a","version":"66'28","reported_seq":142,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-18T10:15:26.093878+0000","last_change":"2024-09-18T10:15:22.536134+0000","last_active":"2024-09-18T10:15:26.093878+0000","last_peered":"2024-09-18T10:15:26.093878+0000","last_clean":"2024-09-18T10:15:26.093878+0000","last_became_active":"2024-09-18T10:14:46.765703+0000","last_became_peered":"2024-09-18T10:14:46.765703+0000","last_unstale":"2024-09-18T10:15:26.093878+0000","last_undegraded":"2024-09-18T10:15:26.093878+0000","last_fullsized":"2024-09-18T10:15:26.093878+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:14:45.576168+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:08.721290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.60505593199999996,"stat_sum":{"num_bytes":770048,"num_objects":4,"num_object_clones":2,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":1,"num_read":27,"num_read_kb":5220,"num_write":29,"num_write_kb":1344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,1,6,5,3,15,4],"acting":[13,1,6,5,3,15,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"e","length":"1"},{"start":"13","length":"1"},{"start":"18","length":"2"}]},{"pgid":"3.b","version":"60'21","reported_seq":118,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-18T10:15:23.959955+0000","last_change":"2024-09-18T10:15:22.446437+0000","last_active":"2024-09-18T10:15:23.959955+0000","last_peered":"2024-09-18T10:15:23.959955+0000","last_clean":"2024-09-18T10:15:23.959955+0000","last_became_active":"2024-09-18T10:14:46.802834+0000","last_became_peered":"2024-09-18T10:14:46.802834+0000","last_unstale":"2024-09-18T10:15:23.959955+0000","last_undegraded":"2024-09-18T10:15:23.959955+0000","last_fullsized":"2024-09-18T10:15:23.959955+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:14:45.576168+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:04.590511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.41299139200000001,"stat_sum":{"num_bytes":671744,"num_objects":1,"num_object_clones":0,"num_object_copies":7,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":31,"num_write_kb":1600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,10,12,11,1,14],"acting":[4,0,10,12,11,1,14],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"e","length":"1"},{"start":"13","length":"1"},{"start":"18","length":"2"}]},{"pgid":"3.c","version":"66'83","reported_seq":181,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-18T10:15:23.936682+0000","last_change":"2024-09-18T10:15:22.846529+0000","last_active":"2024-09-18T10:15:23.936682+0000","last_peered":"2024-09-18T10:15:23.936682+0000","last_clean":"2024-09-18T10:15:23.936682+0000","last_became_active":"2024-09-18T10:14:46.795995+0000","last_became_peered":"2024-09-18T10:14:46.795995+0000","last_unstale":"2024-09-18T10:15:23.936682+0000","last_undegraded":"2024-09-18T10:15:23.936682+0000","last_fullsized":"2024-09-18T10:15:23.936682+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:14:45.576168+0000","objects_scrubbed":0,"log_size":83,"log_dups_size":0,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:14.450533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.77562220400000004,"stat_sum":{"num_bytes":8519680,"num_objects":8,"num_object_clones":3,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":24,"num_read_kb":3468,"num_write":77,"num_write_kb":8992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,13,3,7,12,14],"acting":[5,8,13,3,7,12,14],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"e","length":"1"},{"start":"13","length":"1"},{"start":"18","length":"2"}]},{"pgid":"3.d","version":"66'25","reported_seq":129,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-18T10:15:25.105610+0000","last_change":"2024-09-18T10:15:22.137283+0000","last_active":"2024-09-18T10:15:25.105610+0000","last_peered":"2024-09-18T10:15:25.105610+0000","last_clean":"2024-09-18T10:15:25.105610+0000","last_became_active":"2024-09-18T10:14:46.793893+0000","last_became_peered":"2024-09-18T10:14:46.793893+0000","last_unstale":"2024-09-18T10:15:25.105610+0000","last_undegraded":"2024-09-18T10:15:25.105610+0000","last_fullsized":"2024-09-18T10:15:25.105610+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:14:45.576168+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:13.756262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.20716743200000001,"stat_sum":{"num_bytes":2899968,"num_objects":2,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":9,"num_read_kb":532,"num_write":10,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,1,11,15,9,8,4],"acting":[13,1,11,15,9,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"e","length":"1"},{"start":"13","length":"1"},{"start":"18","length":"2"}]},{"pgid":"3.e","version":"71'37","reported_seq":150,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-18T10:15:28.748183+0000","last_change":"2024-09-18T10:15:28.275984+0000","last_active":"2024-09-18T10:15:28.748183+0000","last_peered":"2024-09-18T10:15:28.748183+0000","last_clean":"2024-09-18T10:15:28.748183+0000","last_became_active":"2024-09-18T10:14:46.801571+0000","last_became_peered":"2024-09-18T10:14:46.801571+0000","last_unstale":"2024-09-18T10:15:28.748183+0000","last_undegraded":"2024-09-18T10:15:28.748183+0000","last_fullsized":"2024-09-18T10:15:28.748183+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:14:45.576168+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:15:59.433777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.10115629800000001,"stat_sum":{"num_bytes":3571712,"num_objects":4,"num_object_clones":2,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":1,"num_read":12,"num_read_kb":2405,"num_write":39,"num_write_kb":3488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,14,6,13,8,3,12],"acting":[11,14,6,13,8,3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"f","length":"1"},{"start":"21","length":"1"}]},{"pgid":"3.f","version":"66'25","reported_seq":125,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-18T10:15:25.105869+0000","last_change":"2024-09-18T10:15:22.237333+0000","last_active":"2024-09-18T10:15:25.105869+0000","last_peered":"2024-09-18T10:15:25.105869+0000","last_clean":"2024-09-18T10:15:25.105869+0000","last_became_active":"2024-09-18T10:14:46.803296+0000","last_became_peered":"2024-09-18T10:14:46.803296+0000","last_unstale":"2024-09-18T10:15:25.105869+0000","last_undegraded":"2024-09-18T10:15:25.105869+0000","last_fullsized":"2024-09-18T10:15:25.105869+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:14:45.576168+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:11.624416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.30690593500000002,"stat_sum":{"num_bytes":2129920,"num_objects":2,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":9,"num_read_kb":1780,"num_write":11,"num_write_kb":2640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,3,15,14,2,11,0],"acting":[13,3,15,14,2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"e","length":"1"},{"start":"13","length":"1"},{"start":"18","length":"2"}]},{"pgid":"3.10","version":"60'14","reported_seq":92,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-18T10:15:23.938487+0000","last_change":"2024-09-18T10:15:22.034432+0000","last_active":"2024-09-18T10:15:23.938487+0000","last_peered":"2024-09-18T10:15:23.938487+0000","last_clean":"2024-09-18T10:15:23.938487+0000","last_became_active":"2024-09-18T10:15:03.217299+0000","last_became_peered":"2024-09-18T10:15:03.217299+0000","last_unstale":"2024-09-18T10:15:23.938487+0000","last_undegraded":"2024-09-18T10:15:23.938487+0000","last_fullsized":"2024-09-18T10:15:23.938487+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:14:45.576168+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.006433032,"stat_sum":{"num_bytes":1769472,"num_objects":1,"num_object_clones":0,"num_object_copies":7,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":9,"num_read_kb":1684,"num_write":6,"num_write_kb":2048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":884736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,5,4,8,0,9],"acting":[6,7,5,4,8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"e","length":"1"},{"start":"13","length":"1"},{"start":"18","length":"2"}]},{"pgid":"3.11","version":"36'6","reported_seq":74,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-18T10:15:23.960513+0000","last_change":"2024-09-18T10:15:21.933314+0000","last_active":"2024-09-18T10:15:23.960513+0000","last_peered":"2024-09-18T10:15:23.960513+0000","last_clean":"2024-09-18T10:15:23.960513+0000","last_became_active":"2024-09-18T10:15:04.955869+0000","last_became_peered":"2024-09-18T10:15:04.955869+0000","last_unstale":"2024-09-18T10:15:23.960513+0000","last_undegraded":"2024-09-18T10:15:23.960513+0000","last_fullsized":"2024-09-18T10:15:23.960513+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":5,"last_scrub":"36'6","last_scrub_stamp":"2024-09-18T10:15:07.755163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:07.755163+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:13.113623+0000","scrub_duration":0.60886176599999997,"objects_trimmed":0,"snaptrim_duration":0.002492476,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,13,10,0,14,2],"acting":[3,5,13,10,0,14,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"e","length":"1"},{"start":"13","length":"1"},{"start":"18","length":"2"}]},{"pgid":"3.12","version":"36'8","reported_seq":151,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-18T10:15:25.100538+0000","last_change":"2024-09-18T10:15:21.925801+0000","last_active":"2024-09-18T10:15:25.100538+0000","last_peered":"2024-09-18T10:15:25.100538+0000","last_clean":"2024-09-18T10:15:25.100538+0000","last_became_active":"2024-09-18T10:15:06.903185+0000","last_became_peered":"2024-09-18T10:15:06.903185+0000","last_unstale":"2024-09-18T10:15:25.100538+0000","last_undegraded":"2024-09-18T10:15:25.100538+0000","last_fullsized":"2024-09-18T10:15:25.100538+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":5,"last_scrub":"36'8","last_scrub_stamp":"2024-09-18T10:15:04.353687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:04.353687+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:28.531412+0000","scrub_duration":0.76511709400000005,"objects_trimmed":0,"snaptrim_duration":0.0015708359999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,5,8,3,4,11,1],"acting":[14,5,8,3,4,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"e","length":"1"},{"start":"13","length":"1"},{"start":"18","length":"2"}]},{"pgid":"3.13","version":"66'41","reported_seq":132,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-18T10:15:23.962792+0000","last_change":"2024-09-18T10:15:22.039043+0000","last_active":"2024-09-18T10:15:23.962792+0000","last_peered":"2024-09-18T10:15:23.962792+0000","last_clean":"2024-09-18T10:15:23.962792+0000","last_became_active":"2024-09-18T10:15:13.263306+0000","last_became_peered":"2024-09-18T10:15:13.263306+0000","last_unstale":"2024-09-18T10:15:23.962792+0000","last_undegraded":"2024-09-18T10:15:23.962792+0000","last_fullsized":"2024-09-18T10:15:23.962792+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":5,"last_scrub":"60'33","last_scrub_stamp":"2024-09-18T10:15:19.140411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:19.140411+0000","objects_scrubbed":1,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:39.399176+0000","scrub_duration":0.706777345,"objects_trimmed":0,"snaptrim_duration":0.109764865,"stat_sum":{"num_bytes":1720320,"num_objects":2,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":1,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,13,2,6,0,4,15],"acting":[1,13,2,6,0,4,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"4","length":"1"},{"start":"b","length":"1"},{"start":"e","length":"1"},{"start":"13","length":"1"},{"start":"18","length":"2"}]}],"pool_stats":[{"poolid":3,"num_pg":26,"stat_sum":{"num_bytes":64102400,"num_objects":82,"num_object_clones":41,"num_object_copies":574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":8,"num_read":212,"num_read_kb":44236,"num_write":371,"num_write_kb":54376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":6012928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":117411840,"data_stored":157880320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":799,"ondisk_log_size":799,"up":182,"acting":182,"num_store_stats":16},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":112,"ondisk_log_size":112,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":15,"up_from":22,"seq":94489280525,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303260,"kb_used_data":5904,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94068580,"statfs":{"total":96636764160,"available":96326225920,"internally_reserved":0,"allocated":6045696,"data_stored":6634875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6,10,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.024}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96199999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79600000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92000000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.177}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2110000000000001}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":86.546999999999997}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3400000000000001}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.151}]}]},{"osd":14,"up_from":22,"seq":94489280525,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306248,"kb_used_data":8892,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94065592,"statfs":{"total":96636764160,"available":96323166208,"internally_reserved":0,"allocated":9105408,"data_stored":11170640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,7,8,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3149999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88600000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.417}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.101}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3999999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1299999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.427}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5489999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4770000000000001}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.667}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4990000000000001}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6930000000000001}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5720000000000001}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.863}]}]},{"osd":13,"up_from":22,"seq":94489280526,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306516,"kb_used_data":9160,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94065324,"statfs":{"total":96636764160,"available":96322891776,"internally_reserved":0,"allocated":9379840,"data_stored":11903843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6,8,9,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92100000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0489999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.405}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.101}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":22.795999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.87}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99099999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2210000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.133}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4299999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.508}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6439999999999999}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6890000000000001}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.484}]}]},{"osd":0,"up_from":21,"seq":90194313229,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303332,"kb_used_data":5968,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94068508,"statfs":{"total":96636764160,"available":96326152192,"internally_reserved":0,"allocated":6111232,"data_stored":6892379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,4,5,6,9,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84699999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1080000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.196}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95099999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2869999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.349}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.518}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5349999999999999}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.486}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3839999999999999}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6080000000000001}]}]},{"osd":4,"up_from":21,"seq":90194313231,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304304,"kb_used_data":6940,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94067536,"statfs":{"total":96636764160,"available":96325156864,"internally_reserved":0,"allocated":7106560,"data_stored":8592219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93100000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.04}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.421}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6579999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.702}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.52}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.294}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5289999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.395}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9399999999999999}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9690000000000001}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8560000000000001}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.081}]}]},{"osd":9,"up_from":21,"seq":90194313230,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":300880,"kb_used_data":3520,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94070960,"statfs":{"total":96636764160,"available":96328663040,"internally_reserved":0,"allocated":3604480,"data_stored":3470005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,8,10,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66500000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73199999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85199999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98599999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2430000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1990000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.089}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2889999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.669}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.446}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":96}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":46.162999999999997}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3360000000000001}]}]},{"osd":11,"up_from":21,"seq":90194313230,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312724,"kb_used_data":11204,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94059116,"statfs":{"total":96636764160,"available":96316534784,"internally_reserved":0,"allocated":11472896,"data_stored":15106741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[2,3,6,8,9,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0049999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":30.919}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":85.963999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80100000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":85.900000000000006}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":85.838999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68500000000000005}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84199999999999997}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":83.988}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":86.109999999999999}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88400000000000001}]}]},{"osd":12,"up_from":21,"seq":90194313231,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301492,"kb_used_data":4132,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94070348,"statfs":{"total":96636764160,"available":96328036352,"internally_reserved":0,"allocated":4231168,"data_stored":5366472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,9,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71399999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92900000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0549999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0820000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3220000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.302}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4490000000000001}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.278}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3999999999999999}]}]},{"osd":1,"up_from":22,"seq":94489280525,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301764,"kb_used_data":4400,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94070076,"statfs":{"total":96636764160,"available":96327757824,"internally_reserved":0,"allocated":4505600,"data_stored":4561755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,9,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67800000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0269999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1659999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5029999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82299999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":118.43600000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7789999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94399999999999995}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6850000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.825}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.339}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7529999999999999}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.851}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7250000000000001}]}]},{"osd":2,"up_from":22,"seq":94489280525,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306648,"kb_used_data":9284,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94065192,"statfs":{"total":96636764160,"available":96322756608,"internally_reserved":0,"allocated":9506816,"data_stored":11844443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,3,4,5,6,7,8,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.02}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1990000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92300000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.226}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4199999999999999}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5509999999999999}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3460000000000001}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.486}]}]},{"osd":3,"up_from":22,"seq":94489280525,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":309372,"kb_used_data":12016,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94062468,"statfs":{"total":96636764160,"available":96319967232,"internally_reserved":0,"allocated":12304384,"data_stored":15987536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,4,5,6,7,9,10,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1240000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.248}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4330000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5349999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4570000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.857}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0620000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9990000000000001}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.915}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6399999999999999}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0209999999999999}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.948}]}]},{"osd":5,"up_from":22,"seq":94489280525,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306024,"kb_used_data":8660,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94065816,"statfs":{"total":96636764160,"available":96323395584,"internally_reserved":0,"allocated":8867840,"data_stored":11684699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,6,7,8,9,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.874}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.012}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68200000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93700000000000006}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90800000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.222}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2509999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98199999999999998}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6479999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":12.967000000000001}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1240000000000001}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.365}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1930000000000001}]}]},{"osd":6,"up_from":22,"seq":94489280525,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306332,"kb_used_data":8968,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94065508,"statfs":{"total":96636764160,"available":96323080192,"internally_reserved":0,"allocated":9183232,"data_stored":10675568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,7,8,9,11,12,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80100000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.343}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92100000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.091}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1200000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.29}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3460000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.458}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.329}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6719999999999999}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4810000000000001}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2050000000000001}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2669999999999999}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5049999999999999}]}]},{"osd":7,"up_from":22,"seq":94489280525,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308232,"kb_used_data":10868,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94063608,"statfs":{"total":96636764160,"available":96321134592,"internally_reserved":0,"allocated":11128832,"data_stored":14244699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,8,10,11,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83999999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":11.805}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.022}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.077}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.177}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.373}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.399}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.861}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":21.081}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.8639999999999999}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.885}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":105.13}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5900000000000001}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5509999999999999}]}]},{"osd":8,"up_from":22,"seq":94489280525,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308316,"kb_used_data":10960,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94063524,"statfs":{"total":96636764160,"available":96321048576,"internally_reserved":0,"allocated":11223040,"data_stored":15467344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[2,4,6,7,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93300000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.083}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66200000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90800000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83299999999999996}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95699999999999996}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80900000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0800000000000001}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.75}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.274}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0349999999999999}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.226}]}]},{"osd":10,"up_from":22,"seq":94489280526,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307636,"kb_used_data":10284,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94064204,"statfs":{"total":96636764160,"available":96321744896,"internally_reserved":0,"allocated":10530816,"data_stored":13433691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6,7,9,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":79.319000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":79.353999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":79.171999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83999999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96099999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":79.394999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.046}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.087}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.22}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":79.495000000000005}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.198}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":66.772000000000006}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":79.284000000000006}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5800000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4378624,"data_stored":5226496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":3551232,"data_stored":4034560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8728576,"data_stored":11669504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":11509760,"data_stored":15802368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":5373952,"data_stored":6971392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8089600,"data_stored":11509760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":7680000,"data_stored":9609216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":10350592,"data_stored":14069760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":10252288,"data_stored":14929920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":3018752,"data_stored":3764224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":9752576,"data_stored":13258752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":10436608,"data_stored":14528512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":3190784,"data_stored":4788224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":8404992,"data_stored":11505664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":8134656,"data_stored":10633216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":4558848,"data_stored":5578752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T10:15:30.588 INFO:tasks.thrashosds.thrasher:Setting 3.15 to [7, 14, 11, 6, 5, 4, 2, 13, 10, 15, 12, 3, 0, 9] 2024-09-18T10:15:30.589 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '3.15', '7', '14', '11', '6', '5', '4', '2', '13', '10', '15', '12', '3', '0', '9'] 2024-09-18T10:15:30.589 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 3.15 7 14 11 6 5 4 2 13 10 15 12 3 0 9 2024-09-18T10:15:30.591 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T10:15:30.597 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-18T10:15:30.621 INFO:teuthology.orchestra.run.smithi029.stdout:osd.6: {} 2024-09-18T10:15:30.669 INFO:tasks.ceph.osd.11.smithi189.stderr:2024-09-18T10:15:30.666+0000 7f718b60a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 22400) UID: 0 2024-09-18T10:15:30.693 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-18T10:15:30.694 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [ 2024-09-18T10:15:30.694 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:30.694 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.1s4 75/40 ECSubWrite(tid=170, reqid=osd.4.0:170, at_version=75'44, trim_to=0'0, roll_forward_to=74'42))", 2024-09-18T10:15:30.694 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:30.657483+0000", 2024-09-18T10:15:30.694 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.035254016999999999, 2024-09-18T10:15:30.694 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.035276773999999997, 2024-09-18T10:15:30.694 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:30.694 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:30.694 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:30.694 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:30.694 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:30.695 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.657483+0000", 2024-09-18T10:15:30.695 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:30.695 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:30.695 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:30.695 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:30.695 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.657483+0000", 2024-09-18T10:15:30.695 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:30.695 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:30.695 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:30.695 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:30.695 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.657483+0000", 2024-09-18T10:15:30.695 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:30.695 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:30.696 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:30.696 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:30.696 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.657527+0000", 2024-09-18T10:15:30.696 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4150999999999998e-05 2024-09-18T10:15:30.696 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:30.696 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:30.696 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:30.696 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.657547+0000", 2024-09-18T10:15:30.696 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9765000000000001e-05 2024-09-18T10:15:30.696 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:30.696 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:30.696 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:30.696 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.657560+0000", 2024-09-18T10:15:30.697 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3008e-05 2024-09-18T10:15:30.697 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:30.697 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:30.697 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:30.697 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.657722+0000", 2024-09-18T10:15:30.697 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00016288399999999999 2024-09-18T10:15:30.697 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:30.697 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:30.697 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:30.697 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.657757+0000", 2024-09-18T10:15:30.697 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4069000000000002e-05 2024-09-18T10:15:30.697 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:30.697 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:30.698 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:30.698 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:30.698 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:30.698 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.1s4 75/40 ECSubWrite(tid=171, reqid=osd.4.0:171, at_version=75'46, trim_to=0'0, roll_forward_to=74'42))", 2024-09-18T10:15:30.698 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:30.658077+0000", 2024-09-18T10:15:30.698 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.034660241000000001, 2024-09-18T10:15:30.698 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.034736783, 2024-09-18T10:15:30.698 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:30.698 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "queued for pg", 2024-09-18T10:15:30.698 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:30.698 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:30.698 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:30.698 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.658077+0000", 2024-09-18T10:15:30.699 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:30.699 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:30.699 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:30.699 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:30.699 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.658077+0000", 2024-09-18T10:15:30.699 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:30.699 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:30.699 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:30.699 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:30.699 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.658076+0000", 2024-09-18T10:15:30.699 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:30.699 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:30.699 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:30.700 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:30.700 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.658111+0000", 2024-09-18T10:15:30.700 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.5462e-05 2024-09-18T10:15:30.700 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:30.700 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:30.700 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:30.700 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.658137+0000", 2024-09-18T10:15:30.700 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5582e-05 2024-09-18T10:15:30.700 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:30.700 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:30.700 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:30.700 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.658165+0000", 2024-09-18T10:15:30.700 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.8464999999999999e-05 2024-09-18T10:15:30.701 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:30.701 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:30.701 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:30.701 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:30.701 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:30.701 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.9s0 75/40 ECSubWriteReply(tid=181, last_complete=75'82, committed=1, applied=1))", 2024-09-18T10:15:30.701 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:30.676223+0000", 2024-09-18T10:15:30.701 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.016514056999999999, 2024-09-18T10:15:30.701 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.016628659, 2024-09-18T10:15:30.701 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:30.701 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "queued for pg", 2024-09-18T10:15:30.701 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:30.701 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:30.702 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:30.702 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.676223+0000", 2024-09-18T10:15:30.702 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:30.702 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:30.702 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:30.702 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:30.702 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.676223+0000", 2024-09-18T10:15:30.702 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:30.702 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:30.702 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:30.702 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:30.702 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.676222+0000", 2024-09-18T10:15:30.702 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:15:30.703 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:30.703 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:30.703 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:30.703 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.676229+0000", 2024-09-18T10:15:30.703 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.0219999999999999e-06 2024-09-18T10:15:30.703 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:30.703 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:30.703 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:30.703 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.676246+0000", 2024-09-18T10:15:30.703 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6858999999999999e-05 2024-09-18T10:15:30.703 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:30.703 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:30.703 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:30.704 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.676253+0000", 2024-09-18T10:15:30.704 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.5530000000000004e-06 2024-09-18T10:15:30.704 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:30.704 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:30.704 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:30.704 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:30.704 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:30.704 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.9s0 75/40 ECSubWriteReply(tid=182, last_complete=75'84, committed=1, applied=1))", 2024-09-18T10:15:30.704 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:30.676983+0000", 2024-09-18T10:15:30.704 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.015753752999999999, 2024-09-18T10:15:30.704 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.015905611, 2024-09-18T10:15:30.704 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:30.705 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "queued for pg", 2024-09-18T10:15:30.705 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:30.705 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:30.705 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:30.705 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.676983+0000", 2024-09-18T10:15:30.705 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:30.705 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:30.705 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:30.705 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:30.705 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.676983+0000", 2024-09-18T10:15:30.705 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:30.705 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:30.705 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:30.705 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:30.706 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.676983+0000", 2024-09-18T10:15:30.706 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:15:30.706 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:30.706 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:30.706 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:30.706 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.676988+0000", 2024-09-18T10:15:30.706 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.925e-06 2024-09-18T10:15:30.706 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:30.706 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:30.706 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:30.706 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.677002+0000", 2024-09-18T10:15:30.706 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3508e-05 2024-09-18T10:15:30.706 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:30.706 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:30.707 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:30.707 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.677010+0000", 2024-09-18T10:15:30.707 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.1389999999999995e-06 2024-09-18T10:15:30.707 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:30.707 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:30.707 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:30.707 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:30.707 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:30.707 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.10s1 75/46 ECSubWrite(tid=207, reqid=osd.6.0:207, at_version=75'23, trim_to=0'0, roll_forward_to=74'21))", 2024-09-18T10:15:30.707 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:30.681533+0000", 2024-09-18T10:15:30.707 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.011204383, 2024-09-18T10:15:30.707 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.011408046999999999, 2024-09-18T10:15:30.707 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:30.707 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "no flag points reached", 2024-09-18T10:15:30.708 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:30.708 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:30.708 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:30.708 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.681533+0000", 2024-09-18T10:15:30.708 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:30.708 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:30.708 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:30.708 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:30.708 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.681533+0000", 2024-09-18T10:15:30.708 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:30.708 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:30.708 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:30.708 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:30.709 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.681532+0000", 2024-09-18T10:15:30.709 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:30.709 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:30.709 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:30.709 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:30.709 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.681558+0000", 2024-09-18T10:15:30.709 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5942999999999999e-05 2024-09-18T10:15:30.709 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:30.709 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:30.709 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:30.709 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.681571+0000", 2024-09-18T10:15:30.709 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.239e-05 2024-09-18T10:15:30.709 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:30.710 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:30.710 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:30.710 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:30.710 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:30.710 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.9s0 75/40 ECSubWriteReply(tid=181, last_complete=75'82, committed=1, applied=1))", 2024-09-18T10:15:30.710 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:30.681662+0000", 2024-09-18T10:15:30.710 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.011074897, 2024-09-18T10:15:30.710 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.011313597, 2024-09-18T10:15:30.710 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:30.710 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "queued for pg", 2024-09-18T10:15:30.710 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:30.710 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:30.710 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:30.711 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.681662+0000", 2024-09-18T10:15:30.711 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:30.711 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:30.711 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:30.711 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:30.711 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.681662+0000", 2024-09-18T10:15:30.711 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:30.711 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:30.711 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:30.711 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:30.711 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.681661+0000", 2024-09-18T10:15:30.711 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:15:30.711 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:30.712 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:30.712 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:30.712 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.681670+0000", 2024-09-18T10:15:30.712 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.9139999999999997e-06 2024-09-18T10:15:30.712 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:30.712 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:30.712 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:30.712 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.681685+0000", 2024-09-18T10:15:30.712 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4341e-05 2024-09-18T10:15:30.712 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:30.712 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:30.712 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:30.712 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.681692+0000", 2024-09-18T10:15:30.712 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.4070000000000004e-06 2024-09-18T10:15:30.713 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:30.713 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:30.713 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:30.713 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:30.713 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:30.713 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.9s0 75/40 ECSubWriteReply(tid=182, last_complete=75'84, committed=1, applied=1))", 2024-09-18T10:15:30.713 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:30.681700+0000", 2024-09-18T10:15:30.713 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.011037261, 2024-09-18T10:15:30.713 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.011313155, 2024-09-18T10:15:30.713 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:30.713 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "queued for pg", 2024-09-18T10:15:30.713 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:30.713 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:30.714 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:30.714 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.681700+0000", 2024-09-18T10:15:30.714 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:30.714 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:30.714 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:30.714 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:30.714 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.681700+0000", 2024-09-18T10:15:30.714 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:30.714 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:30.714 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:30.714 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:30.714 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.681699+0000", 2024-09-18T10:15:30.714 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:30.714 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:30.715 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:30.715 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:30.715 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.681703+0000", 2024-09-18T10:15:30.715 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.366e-06 2024-09-18T10:15:30.715 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:30.715 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:30.715 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:30.715 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.681710+0000", 2024-09-18T10:15:30.715 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.1960000000000002e-06 2024-09-18T10:15:30.715 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:30.715 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:30.715 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:30.715 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.681715+0000", 2024-09-18T10:15:30.716 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.8729999999999998e-06 2024-09-18T10:15:30.716 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:30.716 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:30.716 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:30.716 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:30.716 INFO:teuthology.orchestra.run.smithi189.stdout: ], 2024-09-18T10:15:30.716 INFO:teuthology.orchestra.run.smithi189.stdout: "num_ops": 7 2024-09-18T10:15:30.716 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-18T10:15:30.717 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T10:15:30.770 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-18T10:15:30.766+0000 7f14dcc5b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22398) UID: 0 2024-09-18T10:15:30.820 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-18T10:15:30.820 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-18T10:15:30.820 INFO:teuthology.orchestra.run.smithi189.stdout: "complaint_time": 30, 2024-09-18T10:15:30.820 INFO:teuthology.orchestra.run.smithi189.stdout: "num_blocked_ops": 0 2024-09-18T10:15:30.820 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-18T10:15:30.829 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T10:15:30.871 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-18T10:15:30.866+0000 7f302f2c3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22187) UID: 0 2024-09-18T10:15:30.909 INFO:teuthology.orchestra.run.smithi029.stdout:osd.7: {} 2024-09-18T10:15:30.929 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:30.930 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:30.930 INFO:teuthology.orchestra.run.smithi029.stdout:osd.8: {} 2024-09-18T10:15:30.939 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-09-18T10:15:30.973 INFO:tasks.ceph.osd.10.smithi154.stderr:2024-09-18T10:15:30.971+0000 7fb73221a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 22091) UID: 0 2024-09-18T10:15:31.004 INFO:teuthology.orchestra.run.smithi029.stdout:osd.9: {} 2024-09-18T10:15:31.025 INFO:teuthology.orchestra.run.smithi029.stdout:osd.10: {} 2024-09-18T10:15:31.036 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-18T10:15:31.036 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-18T10:15:31.036 INFO:teuthology.orchestra.run.smithi189.stdout: "num_ops": 0 2024-09-18T10:15:31.037 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-18T10:15:31.045 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-09-18T10:15:31.048 INFO:teuthology.orchestra.run.smithi029.stdout:osd.11: {} 2024-09-18T10:15:31.074 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-18T10:15:31.072+0000 7f00046a8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 22136) UID: 0 2024-09-18T10:15:31.075 INFO:teuthology.orchestra.run.smithi029.stdout:osd.12: {} 2024-09-18T10:15:31.101 INFO:teuthology.orchestra.run.smithi029.stdout:osd.13: {} 2024-09-18T10:15:31.145 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:31.146 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:31.155 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-09-18T10:15:31.169 INFO:tasks.rados.rados.0.smithi189.stdout:261: finishing write tid 6 to smithi18925896-25 2024-09-18T10:15:31.169 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 25 v 45 (ObjNum 108 snap 21 seq_num 108) dirty exists 2024-09-18T10:15:31.169 INFO:tasks.rados.rados.0.smithi189.stdout:261: left oid 25 (ObjNum 108 snap 21 seq_num 108) 2024-09-18T10:15:31.175 INFO:tasks.ceph.osd.15.smithi189.stderr:2024-09-18T10:15:31.170+0000 7fd0e725e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 22401) UID: 0 2024-09-18T10:15:31.178 INFO:teuthology.orchestra.run.smithi029.stdout:osd.14: {} 2024-09-18T10:15:31.206 INFO:teuthology.orchestra.run.smithi029.stdout:osd.15: {} 2024-09-18T10:15:31.209 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T10:15:31.209 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T10:15:31.209 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T10:15:31.209 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T10:15:31.209 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T10:15:31.209 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T10:15:31.209 INFO:teuthology.orchestra.run.smithi029.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T10:15:31.209 INFO:teuthology.orchestra.run.smithi029.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T10:15:31.210 INFO:teuthology.orchestra.run.smithi029.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-18T10:15:31.210 INFO:teuthology.orchestra.run.smithi029.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-18T10:15:31.210 INFO:teuthology.orchestra.run.smithi029.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-18T10:15:31.210 INFO:teuthology.orchestra.run.smithi029.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-18T10:15:31.210 INFO:teuthology.orchestra.run.smithi029.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-18T10:15:31.210 INFO:teuthology.orchestra.run.smithi029.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-18T10:15:31.210 INFO:teuthology.orchestra.run.smithi029.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-18T10:15:31.210 INFO:teuthology.orchestra.run.smithi029.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-18T10:15:31.251 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:31.251 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:31.260 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_ops_in_flight 2024-09-18T10:15:31.276 INFO:tasks.ceph.osd.14.smithi154.stderr:2024-09-18T10:15:31.275+0000 7f75a19e7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 22093) UID: 0 2024-09-18T10:15:31.377 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-18T10:15:31.372+0000 7f14c6a1a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 22138) UID: 0 2024-09-18T10:15:31.403 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:31.403 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:31.412 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_blocked_ops 2024-09-18T10:15:31.477 INFO:tasks.ceph.osd.11.smithi189.stderr:2024-09-18T10:15:31.474+0000 7f718b60a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 22400) UID: 0 2024-09-18T10:15:31.509 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:31.509 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:31.518 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_historic_ops 2024-09-18T10:15:31.564 INFO:tasks.rados.rados.0.smithi189.stdout:258: done (9 left) 2024-09-18T10:15:31.564 INFO:tasks.rados.rados.0.smithi189.stdout:261: done (8 left) 2024-09-18T10:15:31.564 INFO:tasks.rados.rados.0.smithi189.stdout:263: done (7 left) 2024-09-18T10:15:31.564 INFO:tasks.rados.rados.0.smithi189.stdout:264: done (6 left) 2024-09-18T10:15:31.564 INFO:tasks.rados.rados.0.smithi189.stdout:265: done (5 left) 2024-09-18T10:15:31.564 INFO:tasks.rados.rados.0.smithi189.stdout:266: done (4 left) 2024-09-18T10:15:31.564 INFO:tasks.rados.rados.0.smithi189.stdout:268: done (3 left) 2024-09-18T10:15:31.564 INFO:tasks.rados.rados.0.smithi189.stdout:269: done (2 left) 2024-09-18T10:15:31.564 INFO:tasks.rados.rados.0.smithi189.stdout:271: done (1 left) 2024-09-18T10:15:31.564 INFO:tasks.rados.rados.0.smithi189.stdout:append oid 49 current snap is 22 2024-09-18T10:15:31.564 INFO:tasks.rados.rados.0.smithi189.stdout:272: seq_num 110 ranges {819200=688128,1507328=507904,2015232=655360} 2024-09-18T10:15:31.569 INFO:teuthology.orchestra.run.smithi029.stderr:noscrub is set 2024-09-18T10:15:31.569 INFO:teuthology.orchestra.run.smithi029.stderr:set 3.15 pg_upmap_items mapping to [7->14,11->6,5->4,2->13,10->15,12->3,0->9] 2024-09-18T10:15:31.578 INFO:tasks.rados.rados.0.smithi189.stdout:272: writing smithi18925896-49 from 819200 to 1507328 tid 1 2024-09-18T10:15:31.578 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-18T10:15:31.576+0000 7f1981873700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22132) UID: 0 2024-09-18T10:15:31.582 INFO:tasks.rados.rados.0.smithi189.stdout:272: writing smithi18925896-49 from 1507328 to 2015232 tid 2 2024-09-18T10:15:31.586 INFO:tasks.rados.rados.0.smithi189.stdout:272: writing smithi18925896-49 from 2015232 to 2670592 tid 3 2024-09-18T10:15:31.586 INFO:tasks.rados.rados.0.smithi189.stdout:append oid 32 current snap is 22 2024-09-18T10:15:31.586 INFO:tasks.rados.rados.0.smithi189.stdout:273: seq_num 111 ranges {606208=704512,1310720=655360,1966080=507904} 2024-09-18T10:15:31.594 INFO:tasks.rados.rados.0.smithi189.stdout:273: writing smithi18925896-32 from 606208 to 1310720 tid 1 2024-09-18T10:15:31.598 INFO:tasks.rados.rados.0.smithi189.stdout:273: writing smithi18925896-32 from 1310720 to 1966080 tid 2 2024-09-18T10:15:31.601 INFO:tasks.rados.rados.0.smithi189.stdout:273: writing smithi18925896-32 from 1966080 to 2473984 tid 3 2024-09-18T10:15:31.601 INFO:tasks.rados.rados.0.smithi189.stdout:274: setattr oid 47 current snap is 22 2024-09-18T10:15:31.603 INFO:tasks.rados.rados.0.smithi189.stdout:275: snap_remove snap 9 2024-09-18T10:15:31.669 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:31.669 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:31.674 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T10:15:31.679 INFO:tasks.ceph.osd.14.smithi154.stderr:2024-09-18T10:15:31.675+0000 7f75a19e7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 22093) UID: 0 2024-09-18T10:15:31.780 INFO:tasks.ceph.osd.10.smithi154.stderr:2024-09-18T10:15:31.779+0000 7fb73221a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 22091) UID: 0 2024-09-18T10:15:31.784 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:31.784 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:31.794 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T10:15:31.881 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-18T10:15:31.880+0000 7f14c6a1a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 22138) UID: 0 2024-09-18T10:15:31.894 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:31.894 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:31.904 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T10:15:31.983 INFO:tasks.ceph.osd.10.smithi154.stderr:2024-09-18T10:15:31.979+0000 7fb73221a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 22091) UID: 0 2024-09-18T10:15:32.001 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:32.002 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:32.011 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T10:15:32.083 INFO:tasks.ceph.osd.12.smithi029.stderr:2024-09-18T10:15:32.077+0000 7f8e4e627700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 22190) UID: 0 2024-09-18T10:15:32.110 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:32.110 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:32.121 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T10:15:32.184 INFO:tasks.ceph.osd.15.smithi189.stderr:2024-09-18T10:15:32.178+0000 7fd0e725e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 22401) UID: 0 2024-09-18T10:15:32.262 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:32.262 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:32.272 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T10:15:32.285 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-18T10:15:32.284+0000 7f578c52f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22134) UID: 0 2024-09-18T10:15:32.369 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:32.369 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:32.379 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-09-18T10:15:32.386 INFO:tasks.ceph.osd.8.smithi029.stderr:2024-09-18T10:15:32.381+0000 7f2555c27700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 22192) UID: 0 2024-09-18T10:15:32.484 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:32.484 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:32.490 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-09-18T10:15:32.487+0000 7fdfa8698700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22087) UID: 0 2024-09-18T10:15:32.493 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-09-18T10:15:32.565 INFO:tasks.rados.rados.0.smithi189.stdout:275: done (4 left) 2024-09-18T10:15:32.565 INFO:tasks.rados.rados.0.smithi189.stdout:276: setattr oid 37 current snap is 22 2024-09-18T10:15:32.565 INFO:tasks.rados.rados.0.smithi189.stdout:272: finishing write tid 1 to smithi18925896-49 2024-09-18T10:15:32.565 INFO:tasks.rados.rados.0.smithi189.stdout:272: finishing write tid 2 to smithi18925896-49 2024-09-18T10:15:32.565 INFO:tasks.rados.rados.0.smithi189.stdout:272: finishing write tid 3 to smithi18925896-49 2024-09-18T10:15:32.566 INFO:tasks.rados.rados.0.smithi189.stdout:273: finishing write tid 1 to smithi18925896-32 2024-09-18T10:15:32.566 INFO:tasks.rados.rados.0.smithi189.stdout:273: finishing write tid 2 to smithi18925896-32 2024-09-18T10:15:32.566 INFO:tasks.rados.rados.0.smithi189.stdout:273: finishing write tid 3 to smithi18925896-32 2024-09-18T10:15:32.566 INFO:tasks.rados.rados.0.smithi189.stdout:273: finishing write tid 4 to smithi18925896-32 2024-09-18T10:15:32.566 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 47 v 42 (ObjNum 41 snap 0 seq_num 41) dirty exists 2024-09-18T10:15:32.566 INFO:tasks.rados.rados.0.smithi189.stdout:262: finishing write tid 1 to smithi18925896-17 2024-09-18T10:15:32.566 INFO:tasks.rados.rados.0.smithi189.stdout:272: finishing write tid 4 to smithi18925896-49 2024-09-18T10:15:32.566 INFO:tasks.rados.rados.0.smithi189.stdout:262: finishing write tid 2 to smithi18925896-17 2024-09-18T10:15:32.566 INFO:tasks.rados.rados.0.smithi189.stdout:262: finishing write tid 3 to smithi18925896-17 2024-09-18T10:15:32.566 INFO:tasks.rados.rados.0.smithi189.stdout:262: finishing write tid 4 to smithi18925896-17 2024-09-18T10:15:32.566 INFO:tasks.rados.rados.0.smithi189.stdout:273: finishing write tid 5 to smithi18925896-32 2024-09-18T10:15:32.566 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 32 v 68 (ObjNum 111 snap 22 seq_num 111) dirty exists 2024-09-18T10:15:32.566 INFO:tasks.rados.rados.0.smithi189.stdout:273: left oid 32 (ObjNum 111 snap 22 seq_num 111) 2024-09-18T10:15:32.566 INFO:tasks.rados.rados.0.smithi189.stdout:262: finishing write tid 5 to smithi18925896-17 2024-09-18T10:15:32.567 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 17 v 105 (ObjNum 109 snap 21 seq_num 109) dirty exists 2024-09-18T10:15:32.567 INFO:tasks.rados.rados.0.smithi189.stdout:262: left oid 17 (ObjNum 109 snap 21 seq_num 109) 2024-09-18T10:15:32.567 INFO:tasks.rados.rados.0.smithi189.stdout:272: finishing write tid 5 to smithi18925896-49 2024-09-18T10:15:32.567 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 49 v 52 (ObjNum 110 snap 22 seq_num 110) dirty exists 2024-09-18T10:15:32.567 INFO:tasks.rados.rados.0.smithi189.stdout:272: left oid 49 (ObjNum 110 snap 22 seq_num 110) 2024-09-18T10:15:32.571 INFO:tasks.rados.rados.0.smithi189.stdout:262: done (4 left) 2024-09-18T10:15:32.571 INFO:tasks.rados.rados.0.smithi189.stdout:272: done (3 left) 2024-09-18T10:15:32.571 INFO:tasks.rados.rados.0.smithi189.stdout:273: done (2 left) 2024-09-18T10:15:32.571 INFO:tasks.rados.rados.0.smithi189.stdout:274: done (1 left) 2024-09-18T10:15:32.571 INFO:tasks.rados.rados.0.smithi189.stdout:277: snap_create 2024-09-18T10:15:32.590 INFO:tasks.ceph.osd.3.smithi189.stderr:2024-09-18T10:15:32.586+0000 7f2440fc7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22396) UID: 0 2024-09-18T10:15:32.596 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:32.596 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:32.606 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-09-18T10:15:32.692 INFO:tasks.ceph.osd.15.smithi189.stderr:2024-09-18T10:15:32.686+0000 7fd0e725e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 22401) UID: 0 2024-09-18T10:15:32.711 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:32.711 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:32.721 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_ops_in_flight 2024-09-18T10:15:32.793 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-18T10:15:32.788+0000 7f578c52f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22134) UID: 0 2024-09-18T10:15:32.816 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:32.817 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:32.826 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_blocked_ops 2024-09-18T10:15:32.894 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-09-18T10:15:32.891+0000 7fdfa8698700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22087) UID: 0 2024-09-18T10:15:32.926 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:32.926 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:32.935 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_historic_ops 2024-09-18T10:15:32.986 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 37 v 33 (ObjNum 90 snap 16 seq_num 90) dirty exists 2024-09-18T10:15:32.996 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-18T10:15:32.992+0000 7f14c6a1a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 22138) UID: 0 2024-09-18T10:15:33.035 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:33.036 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:33.045 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T10:15:33.097 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-18T10:15:33.092+0000 7f00046a8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 22136) UID: 0 2024-09-18T10:15:33.158 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:33.158 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:33.168 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T10:15:33.198 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-18T10:15:33.196+0000 7f1981873700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22132) UID: 0 2024-09-18T10:15:33.223 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T10:15:33.263 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:33.263 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:33.272 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T10:15:33.299 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-09-18T10:15:33.298+0000 7fdfa8698700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22087) UID: 0 2024-09-18T10:15:33.369 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-18T10:15:33.391 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-18T10:15:33.400 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-09-18T10:15:33.398+0000 7fdfa8698700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22087) UID: 0 2024-09-18T10:15:33.414 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-18T10:15:33.432 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-18T10:15:33.432 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-18T10:15:33.432 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-18T10:15:33.432 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-18T10:15:33.432 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.432 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.8s5 36/32 ECSubWrite(tid=119, reqid=client.4632.0:130, at_version=36'5, trim_to=0'0, roll_forward_to=35'4))", 2024-09-18T10:15:33.432 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:50.722395+0000", 2024-09-18T10:15:33.432 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 42.706423073000003, 2024-09-18T10:15:33.432 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10448325999999999, 2024-09-18T10:15:33.432 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:33.433 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:33.433 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:33.433 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.433 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:33.433 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.722395+0000", 2024-09-18T10:15:33.433 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:33.433 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.433 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.433 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:33.433 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.722395+0000", 2024-09-18T10:15:33.433 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:33.433 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.433 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.433 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:33.434 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.722393+0000", 2024-09-18T10:15:33.434 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:33.434 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.434 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.434 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:33.434 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.722993+0000", 2024-09-18T10:15:33.434 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00059969600000000004 2024-09-18T10:15:33.434 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.434 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.434 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:33.434 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.723025+0000", 2024-09-18T10:15:33.434 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2364999999999999e-05 2024-09-18T10:15:33.434 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.434 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.435 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:33.435 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.723039+0000", 2024-09-18T10:15:33.435 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3709999999999999e-05 2024-09-18T10:15:33.435 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.435 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.435 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:33.435 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.823280+0000", 2024-09-18T10:15:33.435 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100241126 2024-09-18T10:15:33.435 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.435 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.435 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:33.435 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.823298+0000", 2024-09-18T10:15:33.435 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7745999999999999e-05 2024-09-18T10:15:33.436 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.436 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.436 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:33.436 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.826820+0000", 2024-09-18T10:15:33.436 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0035221470000000002 2024-09-18T10:15:33.436 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.436 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.436 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:33.436 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.826878+0000", 2024-09-18T10:15:33.436 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8344000000000002e-05 2024-09-18T10:15:33.436 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:33.436 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:33.436 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:33.437 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.437 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.437 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.bs5 36/32 ECSubWrite(tid=26, reqid=client.4632.0:119, at_version=36'4, trim_to=0'0, roll_forward_to=35'2))", 2024-09-18T10:15:33.437 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:50.812131+0000", 2024-09-18T10:15:33.437 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 42.616687267000003, 2024-09-18T10:15:33.437 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.201739738, 2024-09-18T10:15:33.437 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:33.437 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:33.437 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:33.437 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.437 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:33.437 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.812131+0000", 2024-09-18T10:15:33.437 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:33.438 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.438 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.438 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:33.438 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.812131+0000", 2024-09-18T10:15:33.438 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:33.438 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.438 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.438 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:33.438 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.812129+0000", 2024-09-18T10:15:33.438 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:33.438 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.438 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.438 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:33.439 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.812166+0000", 2024-09-18T10:15:33.439 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6309000000000001e-05 2024-09-18T10:15:33.439 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.439 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.439 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:33.439 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.812199+0000", 2024-09-18T10:15:33.439 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3219000000000002e-05 2024-09-18T10:15:33.439 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.439 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.439 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:33.439 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.812225+0000", 2024-09-18T10:15:33.439 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6302999999999999e-05 2024-09-18T10:15:33.439 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.439 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.440 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:33.440 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.012537+0000", 2024-09-18T10:15:33.440 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20031156899999999 2024-09-18T10:15:33.440 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.440 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.440 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:33.440 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.012564+0000", 2024-09-18T10:15:33.440 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7165000000000002e-05 2024-09-18T10:15:33.440 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.440 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.440 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:33.440 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.013814+0000", 2024-09-18T10:15:33.440 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001249836 2024-09-18T10:15:33.441 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.441 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.441 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:33.441 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.013871+0000", 2024-09-18T10:15:33.441 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6894000000000001e-05 2024-09-18T10:15:33.441 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:33.441 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:33.441 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:33.441 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.441 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.441 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.as1 37/32 ECSubWrite(tid=33, reqid=client.4632.0:207, at_version=37'9, trim_to=0'0, roll_forward_to=36'7))", 2024-09-18T10:15:33.441 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:53.209503+0000", 2024-09-18T10:15:33.441 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 40.219314934000003, 2024-09-18T10:15:33.441 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.103337639, 2024-09-18T10:15:33.442 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:33.442 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:33.442 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:33.442 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.442 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:33.442 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.209503+0000", 2024-09-18T10:15:33.442 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:33.442 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.442 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.442 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:33.442 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.209503+0000", 2024-09-18T10:15:33.442 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:33.443 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.443 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.443 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:33.443 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.209501+0000", 2024-09-18T10:15:33.443 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:33.443 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.443 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.443 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:33.443 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.209748+0000", 2024-09-18T10:15:33.443 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00024739800000000002 2024-09-18T10:15:33.443 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.443 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.443 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:33.443 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.209784+0000", 2024-09-18T10:15:33.444 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5615999999999999e-05 2024-09-18T10:15:33.444 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.444 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.444 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:33.444 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.209799+0000", 2024-09-18T10:15:33.444 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4796e-05 2024-09-18T10:15:33.444 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.444 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.444 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:33.444 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.309992+0000", 2024-09-18T10:15:33.444 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10019278299999999 2024-09-18T10:15:33.444 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.444 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.444 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:33.445 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.310018+0000", 2024-09-18T10:15:33.445 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6392000000000001e-05 2024-09-18T10:15:33.445 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.446 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.446 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:33.446 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.312776+0000", 2024-09-18T10:15:33.446 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0027584200000000001 2024-09-18T10:15:33.446 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.446 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.446 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:33.446 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.312841+0000", 2024-09-18T10:15:33.446 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4348000000000001e-05 2024-09-18T10:15:33.446 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:33.446 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:33.446 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:33.446 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.447 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.447 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.2s3 41/40 ECSubWrite(tid=23, reqid=client.4632.0:265, at_version=41'11, trim_to=0'0, roll_forward_to=0'0))", 2024-09-18T10:15:33.447 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:57.236544+0000", 2024-09-18T10:15:33.447 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 36.192274015000002, 2024-09-18T10:15:33.447 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20209979, 2024-09-18T10:15:33.447 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:33.447 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:33.447 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:33.447 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.447 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:33.447 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.236544+0000", 2024-09-18T10:15:33.447 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:33.448 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.448 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.448 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:33.448 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.236544+0000", 2024-09-18T10:15:33.448 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:33.448 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.448 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.448 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:33.448 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.236543+0000", 2024-09-18T10:15:33.448 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:33.448 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.448 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.448 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:33.449 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.236820+0000", 2024-09-18T10:15:33.449 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000277068 2024-09-18T10:15:33.449 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.449 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.449 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:33.449 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.236876+0000", 2024-09-18T10:15:33.449 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6555000000000003e-05 2024-09-18T10:15:33.449 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.449 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.449 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:33.449 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.336982+0000", 2024-09-18T10:15:33.449 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100105679 2024-09-18T10:15:33.450 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.450 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.450 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:33.450 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.337122+0000", 2024-09-18T10:15:33.450 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000139777 2024-09-18T10:15:33.450 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.450 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.450 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:33.450 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.337135+0000", 2024-09-18T10:15:33.450 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3559e-05 2024-09-18T10:15:33.450 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.450 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.450 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:33.450 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.438600+0000", 2024-09-18T10:15:33.451 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.101464572 2024-09-18T10:15:33.451 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.451 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.451 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:33.451 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.438644+0000", 2024-09-18T10:15:33.451 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4085000000000002e-05 2024-09-18T10:15:33.451 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:33.451 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:33.451 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:33.451 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.451 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.451 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.2s3 41/40 ECSubWrite(tid=25, reqid=client.4632.0:266, at_version=0'0, trim_to=0'0, roll_forward_to=41'12))", 2024-09-18T10:15:33.451 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:57.543345+0000", 2024-09-18T10:15:33.451 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 35.885473605000001, 2024-09-18T10:15:33.452 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.18858536100000001, 2024-09-18T10:15:33.452 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:33.452 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:33.452 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:33.452 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.452 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:33.452 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.543345+0000", 2024-09-18T10:15:33.452 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:33.452 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.452 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.452 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:33.453 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.543345+0000", 2024-09-18T10:15:33.453 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:33.453 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.453 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.453 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:33.454 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.543343+0000", 2024-09-18T10:15:33.454 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:33.454 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.454 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.454 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:33.454 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.543385+0000", 2024-09-18T10:15:33.455 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2487999999999997e-05 2024-09-18T10:15:33.455 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.455 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.455 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:33.455 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.543428+0000", 2024-09-18T10:15:33.455 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2315999999999999e-05 2024-09-18T10:15:33.456 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.456 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.456 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:33.456 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.543446+0000", 2024-09-18T10:15:33.456 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8189000000000002e-05 2024-09-18T10:15:33.456 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.456 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.456 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:33.456 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.730560+0000", 2024-09-18T10:15:33.456 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.187114473 2024-09-18T10:15:33.456 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.456 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.456 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:33.457 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.730585+0000", 2024-09-18T10:15:33.457 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4652e-05 2024-09-18T10:15:33.457 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.457 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.457 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:33.457 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.731859+0000", 2024-09-18T10:15:33.458 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001274425 2024-09-18T10:15:33.458 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.458 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.458 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:33.458 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.731930+0000", 2024-09-18T10:15:33.458 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0474000000000003e-05 2024-09-18T10:15:33.458 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:33.458 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:33.458 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:33.458 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.458 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.458 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpRead(3.2s3 41/40 ECSubRead(tid=26, to_read={3:46b2871c:::smithi18925896-34:head=0,4096,0}, subchunks={3:46b2871c:::smithi18925896-34:head=[0,1]}, attrs_to_read=))", 2024-09-18T10:15:33.459 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:57.543972+0000", 2024-09-18T10:15:33.459 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 35.884845978999998, 2024-09-18T10:15:33.459 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.18721849300000001, 2024-09-18T10:15:33.459 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:33.459 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:33.459 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:33.459 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.459 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:33.459 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.543972+0000", 2024-09-18T10:15:33.459 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:33.459 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.459 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.459 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:33.459 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.543972+0000", 2024-09-18T10:15:33.460 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:33.460 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.460 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.460 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:33.460 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.543970+0000", 2024-09-18T10:15:33.460 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:33.460 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.460 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.460 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:33.460 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.543996+0000", 2024-09-18T10:15:33.460 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5069e-05 2024-09-18T10:15:33.460 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.460 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.460 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:33.461 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.544056+0000", 2024-09-18T10:15:33.461 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0355000000000001e-05 2024-09-18T10:15:33.461 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.461 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.461 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:33.461 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.544079+0000", 2024-09-18T10:15:33.461 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3396000000000001e-05 2024-09-18T10:15:33.461 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.461 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.461 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:33.462 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.730946+0000", 2024-09-18T10:15:33.462 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.186866644 2024-09-18T10:15:33.462 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.462 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.462 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:33.462 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.731191+0000", 2024-09-18T10:15:33.462 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00024467899999999998 2024-09-18T10:15:33.462 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:33.462 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:33.462 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:33.462 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.462 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.463 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.ds1 47/32 ECSubWrite(tid=55, reqid=client.4632.0:332, at_version=47'8, trim_to=0'0, roll_forward_to=41'6))", 2024-09-18T10:15:33.463 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:02.796404+0000", 2024-09-18T10:15:33.463 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 30.632414383, 2024-09-18T10:15:33.463 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.103838836, 2024-09-18T10:15:33.463 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:33.463 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:33.463 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:33.463 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.463 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:33.463 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.796404+0000", 2024-09-18T10:15:33.463 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:33.463 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.464 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.464 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:33.464 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.796404+0000", 2024-09-18T10:15:33.464 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:33.464 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.464 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.464 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:33.464 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.796403+0000", 2024-09-18T10:15:33.464 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:15:33.464 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.464 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.464 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:33.464 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.796757+0000", 2024-09-18T10:15:33.465 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00035467799999999998 2024-09-18T10:15:33.465 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.465 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.465 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:33.465 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.796832+0000", 2024-09-18T10:15:33.465 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4661999999999998e-05 2024-09-18T10:15:33.465 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.466 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.466 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:33.466 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.896954+0000", 2024-09-18T10:15:33.466 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10012193799999999 2024-09-18T10:15:33.466 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.467 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.467 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:33.467 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.897127+0000", 2024-09-18T10:15:33.467 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017293500000000001 2024-09-18T10:15:33.467 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.467 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.467 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:33.467 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.897149+0000", 2024-09-18T10:15:33.467 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2262000000000001e-05 2024-09-18T10:15:33.467 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.467 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.468 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:33.468 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.900201+0000", 2024-09-18T10:15:33.468 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0030515379999999999 2024-09-18T10:15:33.468 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.468 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.468 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:33.468 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.900243+0000", 2024-09-18T10:15:33.468 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1758999999999998e-05 2024-09-18T10:15:33.468 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:33.468 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:33.468 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:33.468 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.468 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.469 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.as1 57/32 ECSubWrite(tid=68, reqid=osd.13.0:68, at_version=57'16, trim_to=0'0, roll_forward_to=56'14))", 2024-09-18T10:15:33.469 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:12.949505+0000", 2024-09-18T10:15:33.469 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 20.479313418, 2024-09-18T10:15:33.469 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.29432398300000001, 2024-09-18T10:15:33.469 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:33.469 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:33.469 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:33.469 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.469 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:33.469 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:12.949505+0000", 2024-09-18T10:15:33.469 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:33.469 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.469 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.470 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:33.470 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:12.949505+0000", 2024-09-18T10:15:33.470 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:33.470 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.470 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.470 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:33.470 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:12.949503+0000", 2024-09-18T10:15:33.470 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:33.470 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.470 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.470 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:33.470 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:12.949684+0000", 2024-09-18T10:15:33.470 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00018156800000000001 2024-09-18T10:15:33.470 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.471 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.471 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:33.471 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:12.950018+0000", 2024-09-18T10:15:33.471 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00033348699999999999 2024-09-18T10:15:33.471 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.471 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.471 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:33.471 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:12.950036+0000", 2024-09-18T10:15:33.471 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7669e-05 2024-09-18T10:15:33.471 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.471 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.471 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:33.471 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:13.143164+0000", 2024-09-18T10:15:33.471 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.19312819000000001 2024-09-18T10:15:33.472 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.472 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.472 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:33.472 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:13.143178+0000", 2024-09-18T10:15:33.472 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3974999999999999e-05 2024-09-18T10:15:33.472 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.472 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.472 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:33.472 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:13.243785+0000", 2024-09-18T10:15:33.472 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10060730499999999 2024-09-18T10:15:33.472 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.472 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.472 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:33.472 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:13.243829+0000", 2024-09-18T10:15:33.473 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3634999999999998e-05 2024-09-18T10:15:33.473 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:33.473 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:33.473 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:33.473 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.473 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.473 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.2s3 57/40 ECSubWrite(tid=43, reqid=osd.14.0:42, at_version=0'0, trim_to=0'0, roll_forward_to=57'26))", 2024-09-18T10:15:33.473 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:13.057362+0000", 2024-09-18T10:15:33.473 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 20.371456537, 2024-09-18T10:15:33.473 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.286917694, 2024-09-18T10:15:33.473 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:33.474 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:33.474 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:33.474 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.474 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:33.474 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:13.057362+0000", 2024-09-18T10:15:33.474 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:33.474 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.474 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.474 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:33.474 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:13.057362+0000", 2024-09-18T10:15:33.474 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:33.474 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.475 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.475 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:33.475 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:13.057359+0000", 2024-09-18T10:15:33.475 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999976 2024-09-18T10:15:33.475 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.475 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.475 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:33.475 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:13.057392+0000", 2024-09-18T10:15:33.475 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2836999999999997e-05 2024-09-18T10:15:33.476 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.476 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.476 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:33.476 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:13.057432+0000", 2024-09-18T10:15:33.476 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0195999999999999e-05 2024-09-18T10:15:33.476 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.476 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.476 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:33.476 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:13.057450+0000", 2024-09-18T10:15:33.476 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8156e-05 2024-09-18T10:15:33.476 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.476 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.476 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:33.476 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:13.243225+0000", 2024-09-18T10:15:33.477 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.18577438499999999 2024-09-18T10:15:33.477 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.477 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.477 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:33.477 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:13.243241+0000", 2024-09-18T10:15:33.477 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.643e-05 2024-09-18T10:15:33.477 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.477 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.477 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:33.477 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:13.344203+0000", 2024-09-18T10:15:33.477 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100961541 2024-09-18T10:15:33.477 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.477 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.478 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:33.478 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:13.344279+0000", 2024-09-18T10:15:33.478 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6487000000000001e-05 2024-09-18T10:15:33.478 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:33.478 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:33.478 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:33.478 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.478 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.478 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.2s3 58/40 ECSubWrite(tid=44, reqid=client.4632.0:452, at_version=58'28, trim_to=0'0, roll_forward_to=57'26))", 2024-09-18T10:15:33.479 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:13.563698+0000", 2024-09-18T10:15:33.479 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 19.865119996000001, 2024-09-18T10:15:33.479 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20210718, 2024-09-18T10:15:33.479 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:33.479 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:33.479 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:33.479 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.479 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:33.479 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:13.563698+0000", 2024-09-18T10:15:33.479 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:33.479 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.479 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.480 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:33.480 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:13.563698+0000", 2024-09-18T10:15:33.480 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:33.480 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.480 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.480 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:33.480 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:13.563697+0000", 2024-09-18T10:15:33.480 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:33.480 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.480 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.480 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:33.480 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:13.563729+0000", 2024-09-18T10:15:33.480 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1358e-05 2024-09-18T10:15:33.480 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.481 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.481 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:33.481 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:13.563740+0000", 2024-09-18T10:15:33.481 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.083e-05 2024-09-18T10:15:33.481 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.481 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.481 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:33.481 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:13.563747+0000", 2024-09-18T10:15:33.481 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6520000000000006e-06 2024-09-18T10:15:33.481 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.481 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.481 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:33.481 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:13.762333+0000", 2024-09-18T10:15:33.481 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.19858611700000001 2024-09-18T10:15:33.482 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.482 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.482 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:33.482 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:13.762350+0000", 2024-09-18T10:15:33.482 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6796e-05 2024-09-18T10:15:33.482 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.482 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.482 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:33.482 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:13.765695+0000", 2024-09-18T10:15:33.482 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0033452540000000002 2024-09-18T10:15:33.482 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.482 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.482 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:33.482 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:13.765805+0000", 2024-09-18T10:15:33.483 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000109843 2024-09-18T10:15:33.483 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:33.483 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:33.483 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:33.483 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.483 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.483 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDPGRecoveryDeleteReply(3.13s0 e58,56 3:c9ce6440:::smithi18925896-24:2,41'17)", 2024-09-18T10:15:33.483 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:13.987390+0000", 2024-09-18T10:15:33.483 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 19.441427715, 2024-09-18T10:15:33.483 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10444218, 2024-09-18T10:15:33.483 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:33.483 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:33.483 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:33.484 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.484 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:33.484 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:13.987390+0000", 2024-09-18T10:15:33.484 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:33.484 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.484 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.484 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:33.484 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:13.987390+0000", 2024-09-18T10:15:33.484 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:33.484 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.484 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.484 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:33.484 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:13.987388+0000", 2024-09-18T10:15:33.484 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:33.485 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.485 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.485 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:33.485 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:13.987402+0000", 2024-09-18T10:15:33.485 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4083000000000001e-05 2024-09-18T10:15:33.485 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.485 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.485 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:33.485 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:13.987449+0000", 2024-09-18T10:15:33.485 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6514999999999998e-05 2024-09-18T10:15:33.485 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.485 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.485 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:33.486 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:13.987463+0000", 2024-09-18T10:15:33.486 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4443e-05 2024-09-18T10:15:33.486 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.486 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.486 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:33.486 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:14.089024+0000", 2024-09-18T10:15:33.486 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.101560597 2024-09-18T10:15:33.486 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.486 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.486 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:33.486 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:14.091833+0000", 2024-09-18T10:15:33.486 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0028085720000000001 2024-09-18T10:15:33.486 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:33.486 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:33.486 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:33.487 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.487 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.487 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDPGRecoveryDeleteReply(3.13s0 e58,56 3:c9ce6440:::smithi18925896-24:2,41'17)", 2024-09-18T10:15:33.487 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:14.163960+0000", 2024-09-18T10:15:33.487 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 19.264858014000001, 2024-09-18T10:15:33.487 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.105232166, 2024-09-18T10:15:33.487 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:33.487 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:33.487 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:33.487 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.487 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:33.487 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:14.163960+0000", 2024-09-18T10:15:33.487 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:33.488 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.488 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.488 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:33.488 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:14.163960+0000", 2024-09-18T10:15:33.488 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:33.488 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.488 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.488 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:33.488 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:14.163958+0000", 2024-09-18T10:15:33.488 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:33.488 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.488 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.488 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:33.488 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:14.163975+0000", 2024-09-18T10:15:33.489 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6552999999999999e-05 2024-09-18T10:15:33.489 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.489 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.489 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:33.489 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:14.164009+0000", 2024-09-18T10:15:33.489 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4476e-05 2024-09-18T10:15:33.489 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.489 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.489 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:33.489 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:14.264163+0000", 2024-09-18T10:15:33.489 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100153834 2024-09-18T10:15:33.489 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.489 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.489 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:33.490 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:14.264384+0000", 2024-09-18T10:15:33.490 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000221301 2024-09-18T10:15:33.490 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.490 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.490 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:33.490 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:14.269192+0000", 2024-09-18T10:15:33.490 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0048078749999999996 2024-09-18T10:15:33.490 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:33.490 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:33.490 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:33.490 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.490 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.490 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDScrubReserve(3.18s5 REQUEST e61)", 2024-09-18T10:15:33.491 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:17.424550+0000", 2024-09-18T10:15:33.491 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 16.004268458999999, 2024-09-18T10:15:33.491 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.200564715, 2024-09-18T10:15:33.491 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:33.491 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-18T10:15:33.491 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:33.491 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.491 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:33.491 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.424550+0000", 2024-09-18T10:15:33.491 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:33.491 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.491 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.491 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:33.491 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.424550+0000", 2024-09-18T10:15:33.492 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:33.492 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.492 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.492 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:33.492 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.424548+0000", 2024-09-18T10:15:33.492 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:33.492 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.492 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.492 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:33.492 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.424560+0000", 2024-09-18T10:15:33.492 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2859e-05 2024-09-18T10:15:33.492 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.492 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.492 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:33.493 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.424595+0000", 2024-09-18T10:15:33.493 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4576000000000002e-05 2024-09-18T10:15:33.493 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.493 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.493 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:33.493 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.524738+0000", 2024-09-18T10:15:33.493 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100143278 2024-09-18T10:15:33.493 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.493 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.493 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:33.493 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.625032+0000", 2024-09-18T10:15:33.493 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10029397399999999 2024-09-18T10:15:33.493 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.493 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.494 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:33.494 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.625064+0000", 2024-09-18T10:15:33.494 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1848999999999999e-05 2024-09-18T10:15:33.494 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.494 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.494 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:33.494 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.625114+0000", 2024-09-18T10:15:33.494 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0305e-05 2024-09-18T10:15:33.494 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:33.494 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:33.494 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:33.494 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.494 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.494 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:515 3.13s0 3:c9ce6440:::smithi18925896-24:head [delete] snapc 1a=[1a,19,18,16,12,11,10,f,e] ondisk+write+known_if_redirected+full_force+supports_pool_eio e63)", 2024-09-18T10:15:33.495 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:19.217452+0000", 2024-09-18T10:15:33.495 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 14.211366527999999, 2024-09-18T10:15:33.495 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20492691099999999, 2024-09-18T10:15:33.495 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:33.495 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:33.495 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:33.495 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:33.495 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:33.495 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 515 2024-09-18T10:15:33.495 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.495 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:33.495 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.495 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:33.495 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.217452+0000", 2024-09-18T10:15:33.495 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:33.496 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.496 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.496 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:33.496 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.217452+0000", 2024-09-18T10:15:33.496 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:33.496 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.496 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.496 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:33.496 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.217448+0000", 2024-09-18T10:15:33.496 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999967 2024-09-18T10:15:33.496 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.496 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.496 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:33.497 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.217464+0000", 2024-09-18T10:15:33.497 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5763000000000001e-05 2024-09-18T10:15:33.497 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.497 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.497 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:33.497 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.217500+0000", 2024-09-18T10:15:33.497 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5536e-05 2024-09-18T10:15:33.497 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.497 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.497 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:33.497 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.217528+0000", 2024-09-18T10:15:33.497 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8609e-05 2024-09-18T10:15:33.497 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.497 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.498 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:33.498 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.217647+0000", 2024-09-18T10:15:33.498 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000118917 2024-09-18T10:15:33.498 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.498 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.498 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:33.498 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.217866+0000", 2024-09-18T10:15:33.498 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00021885399999999999 2024-09-18T10:15:33.498 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.498 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.498 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:33.498 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.219038+0000", 2024-09-18T10:15:33.498 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001172125 2024-09-18T10:15:33.498 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.499 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.499 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:33.499 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.220845+0000", 2024-09-18T10:15:33.499 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001806385 2024-09-18T10:15:33.499 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.499 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.499 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:15:33.499 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.422256+0000", 2024-09-18T10:15:33.499 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20141129799999999 2024-09-18T10:15:33.499 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.499 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.499 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:33.499 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.422379+0000", 2024-09-18T10:15:33.499 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012262699999999999 2024-09-18T10:15:33.500 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:33.500 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:33.500 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:33.500 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.500 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.500 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.bs5 72/32 ECSubWrite(tid=153, reqid=client.4632.0:624, at_version=72'23, trim_to=0'0, roll_forward_to=60'21))", 2024-09-18T10:15:33.500 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:28.105186+0000", 2024-09-18T10:15:33.500 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.3236320150000003, 2024-09-18T10:15:33.500 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.38192188100000002, 2024-09-18T10:15:33.500 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:33.500 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:33.500 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:33.500 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.501 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:33.501 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:28.105186+0000", 2024-09-18T10:15:33.501 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:33.501 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.501 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.501 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:33.501 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:28.105186+0000", 2024-09-18T10:15:33.501 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:33.501 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.501 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.501 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:33.501 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:28.105186+0000", 2024-09-18T10:15:33.501 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:33.501 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.502 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.502 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:33.502 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:28.105214+0000", 2024-09-18T10:15:33.502 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7985000000000001e-05 2024-09-18T10:15:33.502 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.502 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.502 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:33.502 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:28.105228+0000", 2024-09-18T10:15:33.502 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3930999999999999e-05 2024-09-18T10:15:33.502 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.502 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.502 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:33.502 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:28.205370+0000", 2024-09-18T10:15:33.502 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10014197800000001 2024-09-18T10:15:33.503 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.503 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.503 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:33.503 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:28.305658+0000", 2024-09-18T10:15:33.503 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.1002884 2024-09-18T10:15:33.503 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.503 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.503 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:33.503 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:28.305679+0000", 2024-09-18T10:15:33.503 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0812e-05 2024-09-18T10:15:33.503 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.503 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.503 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:33.503 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:28.487015+0000", 2024-09-18T10:15:33.504 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.18133600499999999 2024-09-18T10:15:33.504 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.504 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.504 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:33.504 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:28.487108+0000", 2024-09-18T10:15:33.504 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3011999999999996e-05 2024-09-18T10:15:33.504 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:33.504 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:33.504 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:33.504 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.505 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.505 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.2s3 72/40 ECSubWrite(tid=93, reqid=osd.14.0:92, at_version=0'0, trim_to=0'0, roll_forward_to=72'55))", 2024-09-18T10:15:33.505 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:28.389120+0000", 2024-09-18T10:15:33.505 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.03969778, 2024-09-18T10:15:33.505 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20149926400000001, 2024-09-18T10:15:33.505 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:33.505 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:33.505 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:33.505 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.505 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:33.505 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:28.389120+0000", 2024-09-18T10:15:33.506 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:33.506 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.506 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.506 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:33.506 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:28.389120+0000", 2024-09-18T10:15:33.506 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:33.506 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.506 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.506 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:33.506 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:28.389112+0000", 2024-09-18T10:15:33.506 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999919 2024-09-18T10:15:33.506 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.506 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.506 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:33.507 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:28.389153+0000", 2024-09-18T10:15:33.507 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0138000000000001e-05 2024-09-18T10:15:33.507 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.507 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.507 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:33.507 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:28.389182+0000", 2024-09-18T10:15:33.507 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9113000000000001e-05 2024-09-18T10:15:33.507 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.507 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.507 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:33.507 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:28.489269+0000", 2024-09-18T10:15:33.507 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100087491 2024-09-18T10:15:33.507 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.507 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.508 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:33.508 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:28.589445+0000", 2024-09-18T10:15:33.508 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100175537 2024-09-18T10:15:33.508 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.508 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.508 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:33.508 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:28.589468+0000", 2024-09-18T10:15:33.508 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2781000000000001e-05 2024-09-18T10:15:33.508 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.508 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.508 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:33.508 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:28.590550+0000", 2024-09-18T10:15:33.508 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001082251 2024-09-18T10:15:33.508 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.509 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.509 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:33.509 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:28.590620+0000", 2024-09-18T10:15:33.509 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9822999999999999e-05 2024-09-18T10:15:33.509 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:33.509 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:33.509 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:33.509 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.509 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.509 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.bs5 74/32 ECSubWrite(tid=157, reqid=osd.4.0:157, at_version=74'25, trim_to=0'0, roll_forward_to=72'23))", 2024-09-18T10:15:33.509 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:29.664321+0000", 2024-09-18T10:15:33.509 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.7644974109999998, 2024-09-18T10:15:33.509 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.104388913, 2024-09-18T10:15:33.509 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:33.510 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:33.510 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:33.510 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.510 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:33.510 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.664321+0000", 2024-09-18T10:15:33.510 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:33.510 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.510 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.511 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:33.511 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.664321+0000", 2024-09-18T10:15:33.511 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:33.511 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.511 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.511 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:33.511 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.664319+0000", 2024-09-18T10:15:33.512 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:33.512 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.512 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.512 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:33.512 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.664380+0000", 2024-09-18T10:15:33.512 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0717999999999997e-05 2024-09-18T10:15:33.512 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.512 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.512 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:33.512 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.664409+0000", 2024-09-18T10:15:33.512 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9047000000000002e-05 2024-09-18T10:15:33.513 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.513 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.513 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:33.513 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.664420+0000", 2024-09-18T10:15:33.513 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.148e-05 2024-09-18T10:15:33.513 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.514 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.514 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:33.514 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.664513+0000", 2024-09-18T10:15:33.514 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2947000000000003e-05 2024-09-18T10:15:33.514 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.514 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.514 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:33.514 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.664537+0000", 2024-09-18T10:15:33.514 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4042000000000001e-05 2024-09-18T10:15:33.514 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.514 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.514 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:33.514 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.768653+0000", 2024-09-18T10:15:33.515 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.104115914 2024-09-18T10:15:33.515 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.515 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.515 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:33.515 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.768710+0000", 2024-09-18T10:15:33.515 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6585000000000001e-05 2024-09-18T10:15:33.515 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:33.515 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:33.515 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:33.515 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.515 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.515 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.8s5 74/40 ECSubWrite(tid=202, reqid=osd.6.0:202, at_version=74'27, trim_to=0'0, roll_forward_to=72'23))", 2024-09-18T10:15:33.515 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:29.719110+0000", 2024-09-18T10:15:33.515 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.7097082239999999, 2024-09-18T10:15:33.516 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.19912816699999999, 2024-09-18T10:15:33.516 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:33.516 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:33.516 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:33.516 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.516 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:33.516 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.719110+0000", 2024-09-18T10:15:33.516 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:33.516 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.516 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.516 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:33.516 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.719110+0000", 2024-09-18T10:15:33.516 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:33.516 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.517 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.517 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:33.517 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.719109+0000", 2024-09-18T10:15:33.517 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:15:33.517 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.517 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.517 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:33.517 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.719133+0000", 2024-09-18T10:15:33.517 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4409999999999998e-05 2024-09-18T10:15:33.520 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.520 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.520 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:33.521 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.719151+0000", 2024-09-18T10:15:33.521 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7441000000000001e-05 2024-09-18T10:15:33.521 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.521 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.521 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:33.521 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.719159+0000", 2024-09-18T10:15:33.521 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1370000000000002e-06 2024-09-18T10:15:33.521 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.521 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.521 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:33.521 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.916605+0000", 2024-09-18T10:15:33.521 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.19744605700000001 2024-09-18T10:15:33.521 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.521 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.522 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:33.522 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.916621+0000", 2024-09-18T10:15:33.522 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6059e-05 2024-09-18T10:15:33.522 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.522 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.522 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:33.522 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.918184+0000", 2024-09-18T10:15:33.522 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001562765 2024-09-18T10:15:33.522 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.522 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.522 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:33.522 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.918238+0000", 2024-09-18T10:15:33.522 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4348999999999998e-05 2024-09-18T10:15:33.522 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:33.523 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:33.523 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:33.523 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.523 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.523 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWriteReply(3.13s0 74/56 ECSubWriteReply(tid=23, last_complete=74'46, committed=1, applied=1))", 2024-09-18T10:15:33.523 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:29.774402+0000", 2024-09-18T10:15:33.523 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.65441634, 2024-09-18T10:15:33.523 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.19560270900000001, 2024-09-18T10:15:33.523 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:33.523 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:33.523 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:33.523 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.523 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:33.523 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.774402+0000", 2024-09-18T10:15:33.524 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:33.524 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.524 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.524 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:33.524 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.774402+0000", 2024-09-18T10:15:33.524 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:33.524 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.524 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.524 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:33.524 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.774401+0000", 2024-09-18T10:15:33.524 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:15:33.524 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.524 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.525 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:33.525 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.774407+0000", 2024-09-18T10:15:33.525 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.173e-06 2024-09-18T10:15:33.525 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.525 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.525 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:33.525 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.774423+0000", 2024-09-18T10:15:33.525 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6568000000000001e-05 2024-09-18T10:15:33.525 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.525 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.525 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:33.525 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.774431+0000", 2024-09-18T10:15:33.525 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4649999999999997e-06 2024-09-18T10:15:33.525 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.526 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.526 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:33.526 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.969971+0000", 2024-09-18T10:15:33.526 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.19554042199999999 2024-09-18T10:15:33.526 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.526 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.526 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:33.526 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.970004+0000", 2024-09-18T10:15:33.526 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3151999999999997e-05 2024-09-18T10:15:33.526 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:33.526 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:33.526 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:33.526 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.526 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.527 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.bs5 74/32 ECSubWrite(tid=163, reqid=client.4632.0:650, at_version=74'27, trim_to=0'0, roll_forward_to=72'23))", 2024-09-18T10:15:33.527 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:30.069481+0000", 2024-09-18T10:15:33.527 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.3593368809999999, 2024-09-18T10:15:33.527 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30291274699999998, 2024-09-18T10:15:33.527 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:33.527 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:33.528 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:33.528 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.528 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:33.528 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:30.069481+0000", 2024-09-18T10:15:33.528 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:33.528 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.529 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.529 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:33.529 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:30.069481+0000", 2024-09-18T10:15:33.529 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:33.529 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.529 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.530 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:33.530 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:30.069479+0000", 2024-09-18T10:15:33.530 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:33.530 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.530 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.530 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:33.530 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:30.069535+0000", 2024-09-18T10:15:33.530 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5674999999999998e-05 2024-09-18T10:15:33.531 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.531 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.531 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:33.531 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:30.069577+0000", 2024-09-18T10:15:33.531 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1588000000000002e-05 2024-09-18T10:15:33.531 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.531 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.531 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:33.531 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:30.069595+0000", 2024-09-18T10:15:33.531 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8128999999999999e-05 2024-09-18T10:15:33.531 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.532 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.532 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:33.532 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:30.269862+0000", 2024-09-18T10:15:33.532 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20026767000000001 2024-09-18T10:15:33.532 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.532 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.532 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:33.532 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:30.269881+0000", 2024-09-18T10:15:33.532 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8402000000000001e-05 2024-09-18T10:15:33.532 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.533 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.533 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:33.533 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:30.372285+0000", 2024-09-18T10:15:33.533 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10240445099999999 2024-09-18T10:15:33.533 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.533 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.533 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:33.533 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:30.372394+0000", 2024-09-18T10:15:33.533 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010880200000000001 2024-09-18T10:15:33.533 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:33.533 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:33.533 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:33.533 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:33.533 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:33.534 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-18T10:15:33.535 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-18T10:15:33.535 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-18T10:15:33.535 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-18T10:15:33.536 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T10:15:33.537 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-18T10:15:33.532+0000 7f00046a8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 22136) UID: 0 2024-09-18T10:15:33.543 INFO:teuthology.orchestra.run.smithi029.stdout:osd.6: {} 2024-09-18T10:15:33.568 INFO:tasks.rados.rados.0.smithi189.stdout:276: done (1 left) 2024-09-18T10:15:33.568 INFO:tasks.rados.rados.0.smithi189.stdout:277: done (0 left) 2024-09-18T10:15:33.568 INFO:tasks.rados.rados.0.smithi189.stdout:append oid 33 current snap is 23 2024-09-18T10:15:33.568 INFO:tasks.rados.rados.0.smithi189.stdout:278: seq_num 112 ranges {262144=720896,983040=802816,1785856=360448} 2024-09-18T10:15:33.572 INFO:teuthology.orchestra.run.smithi029.stdout:osd.7: {} 2024-09-18T10:15:33.582 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-18T10:15:33.583 INFO:tasks.rados.rados.0.smithi189.stdout:278: writing smithi18925896-33 from 262144 to 983040 tid 1 2024-09-18T10:15:33.586 INFO:tasks.rados.rados.0.smithi189.stdout:278: writing smithi18925896-33 from 983040 to 1785856 tid 2 2024-09-18T10:15:33.589 INFO:tasks.rados.rados.0.smithi189.stdout:278: writing smithi18925896-33 from 1785856 to 2146304 tid 3 2024-09-18T10:15:33.589 INFO:tasks.rados.rados.0.smithi189.stdout:279: copy_from oid 18 from oid 13 current snap is 23 2024-09-18T10:15:33.589 INFO:tasks.rados.rados.0.smithi189.stdout:280: snap_create 2024-09-18T10:15:33.599 INFO:teuthology.orchestra.run.smithi029.stdout:osd.8: {} 2024-09-18T10:15:33.623 INFO:teuthology.orchestra.run.smithi029.stdout:osd.9: {} 2024-09-18T10:15:33.636 INFO:tasks.ceph.osd.3.smithi189.stderr:2024-09-18T10:15:33.630+0000 7f2440fc7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22396) UID: 0 2024-09-18T10:15:33.684 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-18T10:15:33.685 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-18T10:15:33.685 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-09-18T10:15:33.685 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-18T10:15:33.695 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T10:15:33.715 INFO:teuthology.orchestra.run.smithi029.stdout:osd.10: {} 2024-09-18T10:15:33.737 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-18T10:15:33.732+0000 7f00046a8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 22136) UID: 0 2024-09-18T10:15:33.743 INFO:teuthology.orchestra.run.smithi029.stdout:osd.11: {} 2024-09-18T10:15:33.762 INFO:teuthology.orchestra.run.smithi029.stdout:osd.12: {} 2024-09-18T10:15:33.789 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-18T10:15:33.789 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-18T10:15:33.789 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-18T10:15:33.789 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-18T10:15:33.789 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-18T10:15:33.799 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T10:15:33.819 INFO:teuthology.orchestra.run.smithi029.stdout:osd.13: {} 2024-09-18T10:15:33.838 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-18T10:15:33.836+0000 7f14c6a1a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 22138) UID: 0 2024-09-18T10:15:33.840 INFO:teuthology.orchestra.run.smithi029.stdout:osd.14: {} 2024-09-18T10:15:33.862 INFO:teuthology.orchestra.run.smithi029.stdout:osd.15: {} 2024-09-18T10:15:33.865 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T10:15:33.865 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T10:15:33.865 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T10:15:33.865 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T10:15:33.865 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T10:15:33.865 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T10:15:33.865 INFO:teuthology.orchestra.run.smithi029.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T10:15:33.865 INFO:teuthology.orchestra.run.smithi029.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T10:15:33.865 INFO:teuthology.orchestra.run.smithi029.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-18T10:15:33.865 INFO:teuthology.orchestra.run.smithi029.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-18T10:15:33.865 INFO:teuthology.orchestra.run.smithi029.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-18T10:15:33.866 INFO:teuthology.orchestra.run.smithi029.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-18T10:15:33.866 INFO:teuthology.orchestra.run.smithi029.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-18T10:15:33.866 INFO:teuthology.orchestra.run.smithi029.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-18T10:15:33.866 INFO:teuthology.orchestra.run.smithi029.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-18T10:15:33.866 INFO:teuthology.orchestra.run.smithi029.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-18T10:15:33.889 INFO:tasks.rados.rados.0.smithi189.stdout:278: finishing write tid 1 to smithi18925896-33 2024-09-18T10:15:33.889 INFO:tasks.rados.rados.0.smithi189.stdout:278: finishing write tid 2 to smithi18925896-33 2024-09-18T10:15:33.889 INFO:tasks.rados.rados.0.smithi189.stdout:278: finishing write tid 3 to smithi18925896-33 2024-09-18T10:15:33.890 INFO:tasks.rados.rados.0.smithi189.stdout:278: finishing write tid 4 to smithi18925896-33 2024-09-18T10:15:33.897 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-18T10:15:33.897 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-18T10:15:33.897 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-18T10:15:33.897 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-18T10:15:33.897 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.898 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:150 3.cs0 3:3972e5c3:::smithi18925896-50:head [append 0~458752,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:33.898 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:51.125225+0000", 2024-09-18T10:15:33.898 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 42.768378181000003, 2024-09-18T10:15:33.898 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.40871674099999999, 2024-09-18T10:15:33.898 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:33.898 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:33.898 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:33.898 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:33.898 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:33.898 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 150 2024-09-18T10:15:33.898 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.898 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:33.898 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.899 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:33.899 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.125225+0000", 2024-09-18T10:15:33.899 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:33.899 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.899 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.899 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:33.899 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.125225+0000", 2024-09-18T10:15:33.899 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:33.899 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.899 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.899 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:33.899 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.125222+0000", 2024-09-18T10:15:33.899 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999967 2024-09-18T10:15:33.900 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.900 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.900 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:33.900 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.126038+0000", 2024-09-18T10:15:33.900 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00081693399999999995 2024-09-18T10:15:33.900 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.900 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.900 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:33.900 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.126072+0000", 2024-09-18T10:15:33.900 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4038999999999997e-05 2024-09-18T10:15:33.900 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.900 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.900 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:33.900 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.126084+0000", 2024-09-18T10:15:33.901 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1908e-05 2024-09-18T10:15:33.901 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.901 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.901 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:33.901 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.126184+0000", 2024-09-18T10:15:33.901 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9868e-05 2024-09-18T10:15:33.901 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.901 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.901 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:33.901 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.126439+0000", 2024-09-18T10:15:33.901 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00025457799999999999 2024-09-18T10:15:33.901 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.901 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.902 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:33.902 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.128215+0000", 2024-09-18T10:15:33.902 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0017764499999999999 2024-09-18T10:15:33.902 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.902 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.902 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:33.902 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.232589+0000", 2024-09-18T10:15:33.902 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.104374015 2024-09-18T10:15:33.902 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.902 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.902 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:15:33.902 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.533834+0000", 2024-09-18T10:15:33.902 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30124498 2024-09-18T10:15:33.903 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.903 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.903 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:33.903 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.533941+0000", 2024-09-18T10:15:33.903 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010718299999999999 2024-09-18T10:15:33.903 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:33.903 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:33.903 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:33.903 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.903 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.903 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:151 3.cs0 3:3972e5c3:::smithi18925896-50:head [append 0~376832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:33.903 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:51.127704+0000", 2024-09-18T10:15:33.903 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 42.765899075999997, 2024-09-18T10:15:33.903 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.40663223799999998, 2024-09-18T10:15:33.904 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:33.904 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:33.904 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:33.904 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:33.904 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:33.904 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 151 2024-09-18T10:15:33.904 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.904 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:33.904 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.905 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:33.905 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.127704+0000", 2024-09-18T10:15:33.905 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:33.905 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.905 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.906 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:33.906 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.127704+0000", 2024-09-18T10:15:33.906 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:33.906 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.906 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.906 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:33.906 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.127702+0000", 2024-09-18T10:15:33.906 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:33.906 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.906 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.906 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:33.906 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.128054+0000", 2024-09-18T10:15:33.907 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00035175199999999999 2024-09-18T10:15:33.907 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.907 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.907 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:33.907 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.128091+0000", 2024-09-18T10:15:33.907 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7035e-05 2024-09-18T10:15:33.907 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.907 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.907 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:33.907 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.128105+0000", 2024-09-18T10:15:33.907 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.428e-05 2024-09-18T10:15:33.907 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.907 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.907 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:33.908 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.129336+0000", 2024-09-18T10:15:33.908 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001231326 2024-09-18T10:15:33.908 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.908 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.908 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:33.908 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.129489+0000", 2024-09-18T10:15:33.908 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015266100000000001 2024-09-18T10:15:33.908 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.908 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.908 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:33.908 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.130757+0000", 2024-09-18T10:15:33.908 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0012684409999999999 2024-09-18T10:15:33.909 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.909 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.909 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:33.909 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.332853+0000", 2024-09-18T10:15:33.909 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20209580599999999 2024-09-18T10:15:33.909 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.909 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.909 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:15:33.909 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.534218+0000", 2024-09-18T10:15:33.909 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.2013653 2024-09-18T10:15:33.909 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.909 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.909 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:33.909 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.534336+0000", 2024-09-18T10:15:33.910 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011760900000000001 2024-09-18T10:15:33.910 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:33.910 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:33.910 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:33.910 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.910 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.910 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:152 3.cs0 3:3972e5c3:::smithi18925896-50:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:33.910 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:51.128123+0000", 2024-09-18T10:15:33.910 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 42.765479509000002, 2024-09-18T10:15:33.910 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.80287000200000003, 2024-09-18T10:15:33.911 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:33.911 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:33.911 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:33.911 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:33.911 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:33.911 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 152 2024-09-18T10:15:33.911 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.911 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:33.911 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.911 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:33.911 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.128123+0000", 2024-09-18T10:15:33.911 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:33.912 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.912 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.912 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:33.912 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.128123+0000", 2024-09-18T10:15:33.912 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:33.912 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.912 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.912 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:33.912 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.128122+0000", 2024-09-18T10:15:33.912 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:33.912 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.912 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.912 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:33.913 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.128129+0000", 2024-09-18T10:15:33.913 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6730000000000003e-06 2024-09-18T10:15:33.913 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.913 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.913 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:33.913 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.128156+0000", 2024-09-18T10:15:33.913 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7489000000000001e-05 2024-09-18T10:15:33.913 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.913 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.913 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:33.913 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.128173+0000", 2024-09-18T10:15:33.913 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7102999999999999e-05 2024-09-18T10:15:33.913 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.914 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.914 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:33.914 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.535139+0000", 2024-09-18T10:15:33.914 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.40696592599999998 2024-09-18T10:15:33.914 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.914 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.914 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:33.914 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.535380+0000", 2024-09-18T10:15:33.914 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00024023600000000001 2024-09-18T10:15:33.914 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.914 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.914 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:33.915 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.536784+0000", 2024-09-18T10:15:33.915 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0014046950000000001 2024-09-18T10:15:33.915 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.915 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.915 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:33.915 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.537964+0000", 2024-09-18T10:15:33.915 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0011793050000000001 2024-09-18T10:15:33.915 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.915 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.915 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:15:33.915 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.930841+0000", 2024-09-18T10:15:33.915 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.39287700599999997 2024-09-18T10:15:33.915 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.916 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.916 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:33.916 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.930993+0000", 2024-09-18T10:15:33.916 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015280799999999999 2024-09-18T10:15:33.916 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:33.916 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:33.916 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:33.916 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.916 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.916 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:153 3.cs0 3:3972e5c3:::smithi18925896-50:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:33.916 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:51.128181+0000", 2024-09-18T10:15:33.916 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 42.765421613999997, 2024-09-18T10:15:33.917 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2060332149999999, 2024-09-18T10:15:33.917 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:33.917 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-18T10:15:33.917 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:33.917 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:33.917 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:33.917 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 153 2024-09-18T10:15:33.917 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.917 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:33.917 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.917 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:33.917 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.128181+0000", 2024-09-18T10:15:33.917 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:33.918 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.918 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.918 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:33.918 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.128181+0000", 2024-09-18T10:15:33.918 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:33.918 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.918 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.918 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:33.918 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.128180+0000", 2024-09-18T10:15:33.918 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:15:33.918 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.918 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.919 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:33.919 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.128186+0000", 2024-09-18T10:15:33.919 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.31e-06 2024-09-18T10:15:33.919 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.919 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.919 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:33.919 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.128195+0000", 2024-09-18T10:15:33.919 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5540000000000003e-06 2024-09-18T10:15:33.919 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.919 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.919 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:33.919 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.128201+0000", 2024-09-18T10:15:33.919 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2129999999999996e-06 2024-09-18T10:15:33.920 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.920 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.920 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:33.920 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.537382+0000", 2024-09-18T10:15:33.920 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.40918055399999997 2024-09-18T10:15:33.920 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.920 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.920 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2024-09-18T10:15:33.920 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.537571+0000", 2024-09-18T10:15:33.920 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00018903299999999999 2024-09-18T10:15:33.920 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.920 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.920 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:33.920 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.931789+0000", 2024-09-18T10:15:33.920 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.39421780499999998 2024-09-18T10:15:33.921 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.921 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.921 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:33.921 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.932023+0000", 2024-09-18T10:15:33.921 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000233868 2024-09-18T10:15:33.921 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.921 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.921 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:33.921 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.334215+0000", 2024-09-18T10:15:33.921 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.40219177499999997 2024-09-18T10:15:33.921 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:33.921 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:33.921 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:33.922 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.922 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.922 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:161 3.cs0 3:34d26c88:::smithi18925896-36:head [read 0~589824,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:33.922 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:51.532533+0000", 2024-09-18T10:15:33.922 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 42.361069956000001, 2024-09-18T10:15:33.922 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30885369800000001, 2024-09-18T10:15:33.922 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:33.922 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-18T10:15:33.922 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:33.922 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:33.922 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:33.922 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 161 2024-09-18T10:15:33.922 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.923 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:33.923 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.923 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:33.923 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.532533+0000", 2024-09-18T10:15:33.923 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:33.923 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.923 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.923 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:33.923 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.532533+0000", 2024-09-18T10:15:33.923 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:33.923 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.923 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.923 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:33.924 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.532530+0000", 2024-09-18T10:15:33.924 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999971 2024-09-18T10:15:33.924 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.924 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.924 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:33.924 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.532544+0000", 2024-09-18T10:15:33.924 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4219999999999999e-05 2024-09-18T10:15:33.924 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.924 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.924 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:33.924 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.532580+0000", 2024-09-18T10:15:33.924 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5315000000000003e-05 2024-09-18T10:15:33.924 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.925 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.925 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:33.925 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.532594+0000", 2024-09-18T10:15:33.925 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4216000000000001e-05 2024-09-18T10:15:33.925 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.925 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.925 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:33.925 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.538016+0000", 2024-09-18T10:15:33.925 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0054220559999999998 2024-09-18T10:15:33.925 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.925 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.925 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:33.925 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.538264+0000", 2024-09-18T10:15:33.925 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000248471 2024-09-18T10:15:33.926 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.926 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.926 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:33.926 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.841387+0000", 2024-09-18T10:15:33.926 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30312219699999998 2024-09-18T10:15:33.926 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:33.926 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:33.926 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:33.926 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.926 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.926 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:162 3.cs0 3:34d26c88:::smithi18925896-36:head [read 0~589824,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:33.926 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:51.532607+0000", 2024-09-18T10:15:33.926 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 42.360996272000001, 2024-09-18T10:15:33.926 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.400964512, 2024-09-18T10:15:33.927 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:33.927 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-18T10:15:33.927 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:33.927 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:33.927 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:33.927 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 162 2024-09-18T10:15:33.927 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.927 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:33.927 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.927 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:33.927 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.532607+0000", 2024-09-18T10:15:33.927 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:33.927 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.927 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.928 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:33.928 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.532607+0000", 2024-09-18T10:15:33.928 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:33.928 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.928 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.928 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:33.928 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.532606+0000", 2024-09-18T10:15:33.928 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:15:33.928 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.928 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.928 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:33.928 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.532612+0000", 2024-09-18T10:15:33.929 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0599999999999996e-06 2024-09-18T10:15:33.929 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.929 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.929 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:33.929 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.532621+0000", 2024-09-18T10:15:33.929 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6190000000000006e-06 2024-09-18T10:15:33.933 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.933 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.933 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:33.933 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.532627+0000", 2024-09-18T10:15:33.933 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3469999999999998e-06 2024-09-18T10:15:33.933 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.933 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.933 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:33.933 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.538755+0000", 2024-09-18T10:15:33.933 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0061281729999999998 2024-09-18T10:15:33.934 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.934 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.934 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:33.934 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.538992+0000", 2024-09-18T10:15:33.934 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00023700500000000001 2024-09-18T10:15:33.934 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.934 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.934 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:33.934 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.933571+0000", 2024-09-18T10:15:33.934 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.39457930800000002 2024-09-18T10:15:33.934 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:33.934 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:33.934 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:33.934 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.935 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.935 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:163 3.cs0 3:34d26c88:::smithi18925896-36:head [read 0~589824,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:33.935 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:51.532633+0000", 2024-09-18T10:15:33.935 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 42.360970027, 2024-09-18T10:15:33.935 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.70024756899999996, 2024-09-18T10:15:33.935 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:33.935 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-18T10:15:33.935 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:33.935 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:33.935 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:33.935 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 163 2024-09-18T10:15:33.935 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.935 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:33.936 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.936 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:33.936 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.532633+0000", 2024-09-18T10:15:33.936 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:33.936 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.936 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.936 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:33.936 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.532633+0000", 2024-09-18T10:15:33.936 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:33.936 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.936 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.936 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:33.937 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.532632+0000", 2024-09-18T10:15:33.937 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:15:33.937 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.937 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.937 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:33.937 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.532636+0000", 2024-09-18T10:15:33.937 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3529999999999997e-06 2024-09-18T10:15:33.937 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.937 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.937 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:33.937 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.532645+0000", 2024-09-18T10:15:33.937 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7099999999999996e-06 2024-09-18T10:15:33.937 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.937 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.938 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:33.938 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.532650+0000", 2024-09-18T10:15:33.938 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3029999999999999e-06 2024-09-18T10:15:33.938 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.938 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.938 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:33.938 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.930003+0000", 2024-09-18T10:15:33.938 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.397352333 2024-09-18T10:15:33.938 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.938 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.938 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:33.938 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.930174+0000", 2024-09-18T10:15:33.938 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017110000000000001 2024-09-18T10:15:33.939 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.939 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.939 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:33.939 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.232880+0000", 2024-09-18T10:15:33.939 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30270652300000001 2024-09-18T10:15:33.939 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:33.939 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:33.939 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:33.939 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.939 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.939 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:181 3.cs0 3:3a4fded2:::smithi18925896-23:head [append 0~524288] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:33.939 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:52.691563+0000", 2024-09-18T10:15:33.939 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 41.202039796000001, 2024-09-18T10:15:33.939 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.418704519, 2024-09-18T10:15:33.940 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:33.940 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:33.940 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:33.940 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:33.940 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:33.940 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 181 2024-09-18T10:15:33.940 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.940 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:33.940 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.940 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:33.940 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.691563+0000", 2024-09-18T10:15:33.940 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:33.940 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.941 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.941 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:33.941 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.691563+0000", 2024-09-18T10:15:33.941 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:33.941 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.941 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.941 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:33.941 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.691560+0000", 2024-09-18T10:15:33.941 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999971 2024-09-18T10:15:33.941 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.941 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.941 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:33.941 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.692740+0000", 2024-09-18T10:15:33.941 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001180195 2024-09-18T10:15:33.942 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.942 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.942 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:33.942 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.692779+0000", 2024-09-18T10:15:33.942 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8535000000000003e-05 2024-09-18T10:15:33.942 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.942 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.942 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:33.942 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.692794+0000", 2024-09-18T10:15:33.942 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4807000000000001e-05 2024-09-18T10:15:33.942 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.942 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.942 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:33.943 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.793038+0000", 2024-09-18T10:15:33.943 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100244729 2024-09-18T10:15:33.943 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.943 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.943 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-09-18T10:15:33.943 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.793050+0000", 2024-09-18T10:15:33.943 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1776e-05 2024-09-18T10:15:33.943 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.943 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.943 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:33.943 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.893883+0000", 2024-09-18T10:15:33.943 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100833383 2024-09-18T10:15:33.943 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.943 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.943 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:33.944 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.894056+0000", 2024-09-18T10:15:33.944 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017296999999999999 2024-09-18T10:15:33.944 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.944 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.944 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:33.944 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.896702+0000", 2024-09-18T10:15:33.944 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002645728 2024-09-18T10:15:33.944 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.944 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.944 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:33.944 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.001443+0000", 2024-09-18T10:15:33.944 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.104740824 2024-09-18T10:15:33.944 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.945 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.945 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:15:33.945 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.110173+0000", 2024-09-18T10:15:33.945 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.108729645 2024-09-18T10:15:33.945 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.945 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.945 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:33.945 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.110268+0000", 2024-09-18T10:15:33.945 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4987000000000004e-05 2024-09-18T10:15:33.945 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:33.945 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:33.945 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:33.945 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.946 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.946 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:182 3.cs0 3:3a4fded2:::smithi18925896-23:head [append 0~376832] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:33.946 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:52.693795+0000", 2024-09-18T10:15:33.946 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 41.199808068000003, 2024-09-18T10:15:33.946 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.51656788200000003, 2024-09-18T10:15:33.946 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:33.946 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:33.946 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:33.946 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:33.946 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:33.946 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 182 2024-09-18T10:15:33.946 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.946 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:33.947 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.947 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:33.947 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.693795+0000", 2024-09-18T10:15:33.947 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:33.947 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.947 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.947 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:33.947 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.693795+0000", 2024-09-18T10:15:33.947 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:33.947 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.947 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.947 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:33.947 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.693787+0000", 2024-09-18T10:15:33.948 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999919 2024-09-18T10:15:33.948 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.948 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.948 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:33.948 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694491+0000", 2024-09-18T10:15:33.948 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00070477199999999999 2024-09-18T10:15:33.948 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.948 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.949 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:33.949 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694526+0000", 2024-09-18T10:15:33.949 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4304999999999998e-05 2024-09-18T10:15:33.949 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.949 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.949 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:33.949 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694539+0000", 2024-09-18T10:15:33.949 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3107e-05 2024-09-18T10:15:33.949 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.949 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.949 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:33.949 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.793145+0000", 2024-09-18T10:15:33.950 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.098605719999999994 2024-09-18T10:15:33.950 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.950 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.950 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:33.950 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.793156+0000", 2024-09-18T10:15:33.950 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1892000000000001e-05 2024-09-18T10:15:33.950 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.950 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.950 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:33.950 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.898472+0000", 2024-09-18T10:15:33.950 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10531558000000001 2024-09-18T10:15:33.950 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.950 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.951 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:33.951 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.898643+0000", 2024-09-18T10:15:33.951 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017116399999999999 2024-09-18T10:15:33.951 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.951 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.951 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:33.951 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.900074+0000", 2024-09-18T10:15:33.951 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0014304299999999999 2024-09-18T10:15:33.951 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.951 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.951 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:33.951 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.001446+0000", 2024-09-18T10:15:33.951 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.101371934 2024-09-18T10:15:33.952 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.952 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.952 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:15:33.952 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.210292+0000", 2024-09-18T10:15:33.952 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.208846693 2024-09-18T10:15:33.952 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.952 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.952 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:33.952 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.210363+0000", 2024-09-18T10:15:33.952 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0470999999999997e-05 2024-09-18T10:15:33.952 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:33.952 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:33.952 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:33.953 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.953 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.953 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:183 3.cs0 3:3a4fded2:::smithi18925896-23:head [setxattr _header (56) in=7b] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:33.953 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:52.694557+0000", 2024-09-18T10:15:33.953 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 41.199045507999998, 2024-09-18T10:15:33.953 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.51641106000000003, 2024-09-18T10:15:33.953 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:33.953 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:33.953 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:33.953 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:33.953 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:33.953 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 183 2024-09-18T10:15:33.953 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.954 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:33.954 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.954 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:33.954 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694557+0000", 2024-09-18T10:15:33.954 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:33.954 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.954 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.954 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:33.954 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694557+0000", 2024-09-18T10:15:33.954 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:33.954 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.954 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.954 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:33.954 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694556+0000", 2024-09-18T10:15:33.955 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:33.955 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.955 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.955 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:33.955 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694570+0000", 2024-09-18T10:15:33.955 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4249e-05 2024-09-18T10:15:33.955 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.955 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.955 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:33.955 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694580+0000", 2024-09-18T10:15:33.955 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7459999999999995e-06 2024-09-18T10:15:33.955 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.955 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.956 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:33.956 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694601+0000", 2024-09-18T10:15:33.956 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1583e-05 2024-09-18T10:15:33.956 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.956 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.956 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:33.956 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.793251+0000", 2024-09-18T10:15:33.956 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.098649465000000006 2024-09-18T10:15:33.956 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.956 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.956 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:33.956 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.793262+0000", 2024-09-18T10:15:33.956 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1113e-05 2024-09-18T10:15:33.956 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.957 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.957 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:33.957 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.001995+0000", 2024-09-18T10:15:33.957 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20873336200000001 2024-09-18T10:15:33.957 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.957 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.957 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:33.957 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.002285+0000", 2024-09-18T10:15:33.957 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00028924300000000001 2024-09-18T10:15:33.957 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.957 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.957 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:33.957 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.003993+0000", 2024-09-18T10:15:33.958 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0017085410000000001 2024-09-18T10:15:33.958 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.958 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.958 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:33.958 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.109904+0000", 2024-09-18T10:15:33.958 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.105911038 2024-09-18T10:15:33.958 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.958 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.958 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:15:33.958 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.210762+0000", 2024-09-18T10:15:33.958 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100858146 2024-09-18T10:15:33.958 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.958 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.958 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:33.958 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.210968+0000", 2024-09-18T10:15:33.959 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00020613000000000001 2024-09-18T10:15:33.959 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:33.959 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:33.959 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:33.959 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.959 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.959 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:184 3.cs0 3:3a4fded2:::smithi18925896-23:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:33.959 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:52.694648+0000", 2024-09-18T10:15:33.959 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 41.198955284, 2024-09-18T10:15:33.959 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.72409976600000003, 2024-09-18T10:15:33.959 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:33.959 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-18T10:15:33.959 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:33.959 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:33.960 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:33.960 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 184 2024-09-18T10:15:33.960 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.960 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:33.960 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.960 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:33.960 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694648+0000", 2024-09-18T10:15:33.960 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:33.960 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.960 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.960 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:33.960 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694648+0000", 2024-09-18T10:15:33.960 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:33.960 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.961 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.961 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:33.961 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694647+0000", 2024-09-18T10:15:33.961 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:15:33.961 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.961 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.961 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:33.961 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694652+0000", 2024-09-18T10:15:33.961 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1170000000000003e-06 2024-09-18T10:15:33.961 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.961 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.961 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:33.961 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694668+0000", 2024-09-18T10:15:33.962 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6019e-05 2024-09-18T10:15:33.962 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.962 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.962 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:33.962 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694674+0000", 2024-09-18T10:15:33.962 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7000000000000002e-06 2024-09-18T10:15:33.962 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.962 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.962 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:33.962 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.893402+0000", 2024-09-18T10:15:33.962 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.19872805700000001 2024-09-18T10:15:33.962 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.962 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.962 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:33.963 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.893413+0000", 2024-09-18T10:15:33.963 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0329999999999999e-05 2024-09-18T10:15:33.963 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.963 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.963 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:33.963 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.004678+0000", 2024-09-18T10:15:33.963 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.111265369 2024-09-18T10:15:33.963 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.963 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.963 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2024-09-18T10:15:33.963 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.004893+0000", 2024-09-18T10:15:33.963 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00021510499999999999 2024-09-18T10:15:33.963 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.964 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.964 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:33.964 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.211096+0000", 2024-09-18T10:15:33.964 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.206202899 2024-09-18T10:15:33.964 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.964 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.964 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:33.964 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.211230+0000", 2024-09-18T10:15:33.964 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000133907 2024-09-18T10:15:33.964 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.964 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.964 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:33.965 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.418747+0000", 2024-09-18T10:15:33.965 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20751733 2024-09-18T10:15:33.965 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:33.965 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:33.965 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:33.965 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.965 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.966 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:196 3.cs0 3:351da1bf:::smithi18925896-6:head [append 0~0] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:33.966 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:52.694697+0000", 2024-09-18T10:15:33.966 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 41.198906186000002, 2024-09-18T10:15:33.966 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.61758450099999995, 2024-09-18T10:15:33.966 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:33.966 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:33.966 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:33.966 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:33.966 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:33.966 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 196 2024-09-18T10:15:33.966 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.966 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:33.967 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.967 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:33.967 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694697+0000", 2024-09-18T10:15:33.967 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:33.967 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.967 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.967 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:33.967 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694697+0000", 2024-09-18T10:15:33.967 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:33.967 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.967 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.967 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:33.967 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694696+0000", 2024-09-18T10:15:33.967 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:15:33.968 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.968 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.968 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:33.968 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694705+0000", 2024-09-18T10:15:33.968 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7799999999999995e-06 2024-09-18T10:15:33.968 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.968 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.968 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:33.968 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694721+0000", 2024-09-18T10:15:33.968 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5102e-05 2024-09-18T10:15:33.968 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.968 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.968 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:33.969 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694727+0000", 2024-09-18T10:15:33.969 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2570000000000004e-06 2024-09-18T10:15:33.969 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.969 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.969 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:33.969 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.893457+0000", 2024-09-18T10:15:33.969 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.198729824 2024-09-18T10:15:33.969 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.969 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.969 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:33.969 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.893464+0000", 2024-09-18T10:15:33.969 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6860000000000006e-06 2024-09-18T10:15:33.969 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.969 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.970 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:33.970 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.105086+0000", 2024-09-18T10:15:33.970 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.21162193000000001 2024-09-18T10:15:33.970 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.970 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.970 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:33.970 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.105357+0000", 2024-09-18T10:15:33.970 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00027109699999999998 2024-09-18T10:15:33.970 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.970 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.970 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:33.970 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.107161+0000", 2024-09-18T10:15:33.970 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001803727 2024-09-18T10:15:33.971 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.971 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.971 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:33.971 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.109961+0000", 2024-09-18T10:15:33.971 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0028003080000000001 2024-09-18T10:15:33.971 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.971 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.971 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:15:33.971 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.312174+0000", 2024-09-18T10:15:33.971 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20221219700000001 2024-09-18T10:15:33.971 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.971 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.971 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:33.971 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.312281+0000", 2024-09-18T10:15:33.972 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000107656 2024-09-18T10:15:33.972 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:33.972 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:33.972 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:33.972 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.972 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.972 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:197 3.cs0 3:351da1bf:::smithi18925896-6:head [rollback 2] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:33.972 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:52.694734+0000", 2024-09-18T10:15:33.972 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 41.198869144, 2024-09-18T10:15:33.972 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.61830483899999999, 2024-09-18T10:15:33.972 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:33.972 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:33.972 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:33.973 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:33.973 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:33.973 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 197 2024-09-18T10:15:33.973 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.973 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:33.973 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.973 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:33.973 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694734+0000", 2024-09-18T10:15:33.973 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:33.973 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.973 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.973 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:33.973 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694734+0000", 2024-09-18T10:15:33.974 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:33.974 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.974 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.974 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:33.974 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694733+0000", 2024-09-18T10:15:33.974 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:15:33.974 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.974 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.974 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:33.974 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694737+0000", 2024-09-18T10:15:33.974 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.636e-06 2024-09-18T10:15:33.974 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.974 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.974 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:33.975 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694752+0000", 2024-09-18T10:15:33.975 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5792999999999999e-05 2024-09-18T10:15:33.975 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.975 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.975 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:33.975 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694758+0000", 2024-09-18T10:15:33.975 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2660000000000001e-06 2024-09-18T10:15:33.975 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.975 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.975 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:33.975 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.108286+0000", 2024-09-18T10:15:33.975 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.41352792700000002 2024-09-18T10:15:33.975 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.975 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.976 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:33.976 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.108540+0000", 2024-09-18T10:15:33.976 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000253928 2024-09-18T10:15:33.976 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.976 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.976 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:33.976 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.109284+0000", 2024-09-18T10:15:33.976 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00074488600000000005 2024-09-18T10:15:33.976 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.976 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.976 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:33.976 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.210174+0000", 2024-09-18T10:15:33.976 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10088973699999999 2024-09-18T10:15:33.977 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.977 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.977 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:15:33.977 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.312857+0000", 2024-09-18T10:15:33.977 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10268295600000001 2024-09-18T10:15:33.977 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.977 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.977 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:33.977 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.313039+0000", 2024-09-18T10:15:33.978 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000181476 2024-09-18T10:15:33.978 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:33.978 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:33.978 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:33.978 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.978 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.978 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:331 3.cs0 3:3a4fded2:::smithi18925896-23:head [create,setxattr qPd\\P_iff[nbYodrbXe[dQ_okmVWqe[RceaqQRNm (630) in=40b,setxattr NWYlZi[dPXnppNo]`fatPqfSaTlsQVetOmrQm^ap (138) in=40b,setxattr iVfcaeRT]TQppeYbq_`lRobpUsaVcWXphplasbgY (1507) in=40b,setxattr YeaeOf[jt`g\\k^]VfXTjOONb`jT[WYUgdiglVjYl (8841) in=40b,setxattr [iqZhSWl_sclXkajVYTXlOX]eeeWkPi\\dOYYt_Wb (1928) in=40b,setxattr \\[gaSPWYO`NQXpTccfbk`SZPaU`lbpTk]a] (1262) in=35b,setxattr ]\\]U^niYqQmjcgO[dsO`YpRZeRV]`X]bU[X^_Qis (1086) in=40b,setxattr VafomYY]aa`a^pbWebSRaithSn_dQjQYq]aTarm\\ (1402) in=40b,setxattr acj^VZhfaaikXrXq]_haYqkl[^_atWUTaloiVj`a (105) in=40b,setxattr VNUlaeZOglfodan^Ycb`dqVeZhSTmO_rabViYVch (70) in=40b,setxattr Qka\\PoUoghRc\\Qcnl`kVbj[rs\\hSdVXmlNqha`ta (594) in=40b,setxattr qsSQhb_apjdTlikdpj\\maaVdS[mVfsh_Vqi[ogWo (771) in=40b,setxattr Tm]oi_Rf_V`pSY[VhYSXSQN^bqjtgmm^fUo^f\\ZV (1087) in=40b,setxattr \\dTq (1262) in=4b,setxattr nettWfThbXj]cUpsYrTjZ^qk^\\[ZoZ[qb]Sf[ZbN (1684) in=40b,setxattr Sa\\hrdN^YombnT\\torZ`N]aRqicS_lP^XgP`e`Oc (1963) in=40b,setxattr dWhoTgYPSTh]jfWhPTs^iVdYbtfhi_WjOdes]NXh (1051) in=40b,setxattr Qbp_QrhXQf`O]blcqeqgRchO]RrhbshoV\\cW[_es (10033) in=40b,setxattr WYbQqgdRjmdbeoaP]U]n]PTWOYkP^lWOanNTnaOr (1998) in=40b,setxattr Rfbdg`[^ZYrN]cl][Ypr^tpXmb_mbh[gf\\idocW] (10804) in=40b] snapc b=[b,4] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-18T10:15:33.978 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:02.781073+0000", 2024-09-18T10:15:33.978 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 31.112529836, 2024-09-18T10:15:33.978 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.32781067699999999, 2024-09-18T10:15:33.978 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:33.978 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:33.979 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:33.979 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:33.979 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:33.979 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 331 2024-09-18T10:15:33.979 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.979 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:33.979 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.979 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:33.979 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.781073+0000", 2024-09-18T10:15:33.979 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:33.979 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.979 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.979 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:33.979 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.781073+0000", 2024-09-18T10:15:33.980 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:33.980 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.980 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.980 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:33.980 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.781070+0000", 2024-09-18T10:15:33.980 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999971 2024-09-18T10:15:33.980 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.980 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.980 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:33.980 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.781166+0000", 2024-09-18T10:15:33.980 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5959999999999996e-05 2024-09-18T10:15:33.980 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.980 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.980 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:33.981 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.781206+0000", 2024-09-18T10:15:33.981 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9307999999999997e-05 2024-09-18T10:15:33.981 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.981 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.981 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:33.981 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.781221+0000", 2024-09-18T10:15:33.981 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5549999999999999e-05 2024-09-18T10:15:33.981 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.981 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.981 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:33.981 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.903358+0000", 2024-09-18T10:15:33.981 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.12213652799999999 2024-09-18T10:15:33.981 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.981 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.981 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-09-18T10:15:33.982 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.903375+0000", 2024-09-18T10:15:33.982 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7856999999999999e-05 2024-09-18T10:15:33.982 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.982 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.982 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:33.982 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.904269+0000", 2024-09-18T10:15:33.982 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00089356799999999997 2024-09-18T10:15:33.982 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.982 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.982 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:33.982 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.904590+0000", 2024-09-18T10:15:33.982 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00032100100000000001 2024-09-18T10:15:33.982 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.982 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.982 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:33.983 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.906644+0000", 2024-09-18T10:15:33.983 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0020539830000000001 2024-09-18T10:15:33.983 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.983 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.983 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:33.983 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.912414+0000", 2024-09-18T10:15:33.983 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0057699070000000003 2024-09-18T10:15:33.983 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.983 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.983 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:15:33.983 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:03.108757+0000", 2024-09-18T10:15:33.983 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.196343085 2024-09-18T10:15:33.983 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.983 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.983 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:33.984 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:03.108884+0000", 2024-09-18T10:15:33.984 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012672000000000001 2024-09-18T10:15:33.984 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:33.984 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:33.984 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:33.984 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.984 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.984 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:340 3.cs0 3:3d6b9cb0:::smithi18925896-31:head [append 0~0] snapc b=[b,4] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-18T10:15:33.984 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:02.803124+0000", 2024-09-18T10:15:33.984 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 31.090479022, 2024-09-18T10:15:33.984 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30615964099999998, 2024-09-18T10:15:33.984 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:33.984 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:33.984 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:33.985 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:33.985 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:33.985 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 340 2024-09-18T10:15:33.985 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.985 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:33.985 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.985 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:33.985 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.803124+0000", 2024-09-18T10:15:33.985 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:33.985 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.985 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.985 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:33.985 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.803124+0000", 2024-09-18T10:15:33.986 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:33.986 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.986 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.986 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:33.986 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.803122+0000", 2024-09-18T10:15:33.986 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999976 2024-09-18T10:15:33.986 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.986 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.986 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:33.986 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.803131+0000", 2024-09-18T10:15:33.986 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1929999999999999e-06 2024-09-18T10:15:33.986 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.986 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.987 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:33.987 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.803151+0000", 2024-09-18T10:15:33.987 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9998000000000001e-05 2024-09-18T10:15:33.987 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.987 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.987 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:33.987 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.803160+0000", 2024-09-18T10:15:33.987 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3729999999999999e-06 2024-09-18T10:15:33.987 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.987 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.987 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:33.987 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.903508+0000", 2024-09-18T10:15:33.987 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100347275 2024-09-18T10:15:33.988 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.988 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.988 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:33.988 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.903516+0000", 2024-09-18T10:15:33.988 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2260000000000005e-06 2024-09-18T10:15:33.988 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.988 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.988 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:33.988 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.908957+0000", 2024-09-18T10:15:33.988 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00544138 2024-09-18T10:15:33.988 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.988 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.988 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:33.989 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.909232+0000", 2024-09-18T10:15:33.989 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00027507699999999998 2024-09-18T10:15:33.989 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.989 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.989 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:33.989 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.910206+0000", 2024-09-18T10:15:33.989 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00097342100000000003 2024-09-18T10:15:33.989 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.989 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.989 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:33.989 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.912421+0000", 2024-09-18T10:15:33.989 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0022157470000000001 2024-09-18T10:15:33.989 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.990 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.990 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:15:33.990 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:03.109124+0000", 2024-09-18T10:15:33.990 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.19670262499999999 2024-09-18T10:15:33.990 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.990 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.990 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:33.990 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:03.109284+0000", 2024-09-18T10:15:33.990 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000159512 2024-09-18T10:15:33.990 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:33.990 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:33.990 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:33.990 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.990 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.991 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:341 3.cs0 3:3d6b9cb0:::smithi18925896-31:head [rollback 4] snapc b=[b,4] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-18T10:15:33.991 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:02.803176+0000", 2024-09-18T10:15:33.991 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 31.090427168000002, 2024-09-18T10:15:33.991 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30684217000000003, 2024-09-18T10:15:33.991 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:33.991 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:33.991 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:33.991 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:33.991 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:33.991 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 341 2024-09-18T10:15:33.991 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.991 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:33.992 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.992 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:33.992 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.803176+0000", 2024-09-18T10:15:33.992 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:33.992 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.992 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.992 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:33.992 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.803176+0000", 2024-09-18T10:15:33.992 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:33.992 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.992 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.992 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:33.992 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.803175+0000", 2024-09-18T10:15:33.993 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:33.993 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.993 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.993 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:33.993 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.803181+0000", 2024-09-18T10:15:33.993 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3579999999999999e-06 2024-09-18T10:15:33.993 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.993 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.993 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:33.993 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.803198+0000", 2024-09-18T10:15:33.993 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6855e-05 2024-09-18T10:15:33.993 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.993 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.993 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:33.994 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.803213+0000", 2024-09-18T10:15:33.994 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4626e-05 2024-09-18T10:15:33.994 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.994 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.994 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:33.994 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.911165+0000", 2024-09-18T10:15:33.994 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.107952006 2024-09-18T10:15:33.994 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.994 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.994 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:33.994 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.911317+0000", 2024-09-18T10:15:33.994 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015228999999999999 2024-09-18T10:15:33.994 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.994 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.995 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:33.995 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.911914+0000", 2024-09-18T10:15:33.995 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000597056 2024-09-18T10:15:33.995 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.995 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.995 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:33.995 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:03.012693+0000", 2024-09-18T10:15:33.995 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100778485 2024-09-18T10:15:33.995 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.995 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.995 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:15:33.995 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:03.109913+0000", 2024-09-18T10:15:33.995 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.097220438000000006 2024-09-18T10:15:33.996 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.996 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.996 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:33.996 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:03.110018+0000", 2024-09-18T10:15:33.996 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000104746 2024-09-18T10:15:33.996 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:33.996 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:33.996 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:33.996 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.996 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.996 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:342 3.cs0 3:3d6b9cb0:::smithi18925896-31:head [append 0~0] snapc b=[b,4] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-18T10:15:33.996 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:02.803223+0000", 2024-09-18T10:15:33.996 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 31.090380280000002, 2024-09-18T10:15:33.997 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30756660499999999, 2024-09-18T10:15:33.997 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:33.997 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:33.997 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:33.997 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:33.997 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:33.997 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 342 2024-09-18T10:15:33.997 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.997 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:33.997 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.997 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:33.997 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.803223+0000", 2024-09-18T10:15:33.997 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:33.998 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.998 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.998 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:33.998 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.803223+0000", 2024-09-18T10:15:33.998 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:33.998 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.998 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.998 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:33.998 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.803222+0000", 2024-09-18T10:15:33.998 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:15:33.998 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.998 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.998 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:33.998 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.803226+0000", 2024-09-18T10:15:33.999 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1289999999999998e-06 2024-09-18T10:15:33.999 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.999 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.999 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:33.999 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.803235+0000", 2024-09-18T10:15:33.999 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1120000000000002e-06 2024-09-18T10:15:33.999 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.999 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.999 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:33.999 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.803239+0000", 2024-09-18T10:15:33.999 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8639999999999998e-06 2024-09-18T10:15:33.999 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:33.999 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:33.999 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:34.000 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.913431+0000", 2024-09-18T10:15:34.000 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.110192522 2024-09-18T10:15:34.000 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.000 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.000 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:34.000 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.913523+0000", 2024-09-18T10:15:34.000 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1885000000000005e-05 2024-09-18T10:15:34.000 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.000 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.000 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:34.000 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.913971+0000", 2024-09-18T10:15:34.000 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00044766599999999999 2024-09-18T10:15:34.000 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.001 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.001 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:34.001 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:03.012696+0000", 2024-09-18T10:15:34.001 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.098725270000000004 2024-09-18T10:15:34.001 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.001 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.001 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:15:34.001 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:03.110683+0000", 2024-09-18T10:15:34.001 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.097986670999999997 2024-09-18T10:15:34.001 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.001 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.001 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:34.001 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:03.110789+0000", 2024-09-18T10:15:34.002 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000106543 2024-09-18T10:15:34.002 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:34.002 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:34.002 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:34.002 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.002 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.002 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:631 3.cs0 3:3972e5c3:::smithi18925896-50:head [read 0~3981312,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-18T10:15:34.002 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:28.715182+0000", 2024-09-18T10:15:34.002 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.1784211500000001, 2024-09-18T10:15:34.002 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.31168768400000002, 2024-09-18T10:15:34.002 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:34.002 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-18T10:15:34.002 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:34.003 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:34.003 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:34.003 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 631 2024-09-18T10:15:34.003 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.003 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:34.003 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.003 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:34.003 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:28.715182+0000", 2024-09-18T10:15:34.003 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:34.003 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.003 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.003 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:34.003 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:28.715182+0000", 2024-09-18T10:15:34.003 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:34.003 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.004 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.004 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:34.004 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:28.715179+0000", 2024-09-18T10:15:34.004 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999967 2024-09-18T10:15:34.004 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.004 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.004 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:34.004 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:28.715194+0000", 2024-09-18T10:15:34.004 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5027e-05 2024-09-18T10:15:34.004 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.004 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.004 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:34.005 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:28.715229+0000", 2024-09-18T10:15:34.005 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5051e-05 2024-09-18T10:15:34.005 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.005 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.005 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:34.005 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:28.815373+0000", 2024-09-18T10:15:34.005 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10014455899999999 2024-09-18T10:15:34.005 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.005 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.005 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:34.005 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:28.915672+0000", 2024-09-18T10:15:34.005 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100298469 2024-09-18T10:15:34.005 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.006 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.006 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:34.006 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:28.915889+0000", 2024-09-18T10:15:34.006 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00021754000000000001 2024-09-18T10:15:34.006 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.006 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.006 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:34.006 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.026869+0000", 2024-09-18T10:15:34.006 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.110980232 2024-09-18T10:15:34.006 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:34.006 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:34.006 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:34.006 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.006 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.007 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:633 3.cs0 3:3972e5c3:::smithi18925896-50:head [read 0~3981312,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-18T10:15:34.007 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:28.815454+0000", 2024-09-18T10:15:34.007 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.0781492100000003, 2024-09-18T10:15:34.007 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.41958616100000001, 2024-09-18T10:15:34.007 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:34.007 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-18T10:15:34.007 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:34.007 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:34.007 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:34.007 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 633 2024-09-18T10:15:34.007 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.007 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:34.007 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.008 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:34.008 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:28.815454+0000", 2024-09-18T10:15:34.008 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:34.008 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.008 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.008 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:34.008 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:28.815454+0000", 2024-09-18T10:15:34.008 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:34.008 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.008 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.008 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:34.008 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:28.815453+0000", 2024-09-18T10:15:34.008 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:15:34.008 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.009 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.009 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:34.009 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:28.815458+0000", 2024-09-18T10:15:34.009 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1760000000000001e-06 2024-09-18T10:15:34.009 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.009 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.009 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:34.009 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:28.815467+0000", 2024-09-18T10:15:34.009 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1300000000000007e-06 2024-09-18T10:15:34.009 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.009 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.009 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:34.009 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:28.815484+0000", 2024-09-18T10:15:34.010 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6722e-05 2024-09-18T10:15:34.010 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.010 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.010 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:34.010 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.021577+0000", 2024-09-18T10:15:34.010 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20609329200000001 2024-09-18T10:15:34.010 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.010 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.010 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:34.010 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.021788+0000", 2024-09-18T10:15:34.010 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000211115 2024-09-18T10:15:34.010 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.010 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.010 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:34.011 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.235040+0000", 2024-09-18T10:15:34.011 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.213251576 2024-09-18T10:15:34.011 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:34.011 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:34.011 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:34.011 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.011 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.011 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:651 3.cs0 3:3a4fded2:::smithi18925896-23:head [rmxattr Qka\\PoUoghRc\\Qcnl`kVbj[rs\\hSdVXmlNqha`ta in=40b,rmxattr nettWfThbXj]cUpsYrTjZ^qk^\\[ZoZ[qb]Sf[ZbN in=40b] snapc 21=[20,1f,1e,1a,16,11,10] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-18T10:15:34.011 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:29.817673+0000", 2024-09-18T10:15:34.012 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.0759299919999998, 2024-09-18T10:15:34.012 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.356122946, 2024-09-18T10:15:34.012 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:34.012 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:34.012 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:34.012 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:34.012 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:34.012 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 651 2024-09-18T10:15:34.012 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.012 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:34.012 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.012 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:34.012 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.817673+0000", 2024-09-18T10:15:34.013 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:34.013 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.013 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.013 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:34.013 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.817673+0000", 2024-09-18T10:15:34.013 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:34.013 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.013 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.013 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:34.013 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.817671+0000", 2024-09-18T10:15:34.013 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999976 2024-09-18T10:15:34.013 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.013 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.014 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:34.014 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.817680+0000", 2024-09-18T10:15:34.014 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9989999999999997e-06 2024-09-18T10:15:34.014 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.014 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.014 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:34.014 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.817700+0000", 2024-09-18T10:15:34.014 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0023000000000001e-05 2024-09-18T10:15:34.014 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.014 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.014 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:34.014 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.817709+0000", 2024-09-18T10:15:34.014 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6059999999999992e-06 2024-09-18T10:15:34.014 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.015 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.015 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:34.015 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.860191+0000", 2024-09-18T10:15:34.015 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.042481679000000001 2024-09-18T10:15:34.015 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.015 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.015 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:34.015 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.860501+0000", 2024-09-18T10:15:34.015 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00031000299999999999 2024-09-18T10:15:34.015 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.015 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.015 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:34.015 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.861896+0000", 2024-09-18T10:15:34.015 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0013950519999999999 2024-09-18T10:15:34.016 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.016 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.016 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:34.016 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.864804+0000", 2024-09-18T10:15:34.016 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0029076010000000001 2024-09-18T10:15:34.016 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.016 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.016 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:15:34.016 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:30.173657+0000", 2024-09-18T10:15:34.016 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.308853082 2024-09-18T10:15:34.016 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.016 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.016 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:34.017 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:30.173796+0000", 2024-09-18T10:15:34.017 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013926699999999999 2024-09-18T10:15:34.017 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:34.017 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:34.017 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:34.017 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:34.017 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:34.017 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-18T10:15:34.018 INFO:tasks.rados.rados.0.smithi189.stdout:279: finishing copy_from to smithi18925896-18 2024-09-18T10:15:34.019 INFO:tasks.rados.rados.0.smithi189.stdout:279: got expected ENOENT (src dne) 2024-09-18T10:15:34.019 INFO:tasks.rados.rados.0.smithi189.stdout:279: finishing copy_from racing read to smithi18925896-18 2024-09-18T10:15:34.019 INFO:tasks.ceph.osd.3.smithi189.stderr:2024-09-18T10:15:34.014+0000 7f2440fc7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22396) UID: 0 2024-09-18T10:15:34.020 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-09-18T10:15:34.120 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-18T10:15:34.116+0000 7f578c52f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22134) UID: 0 2024-09-18T10:15:34.168 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-18T10:15:34.169 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-18T10:15:34.169 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-09-18T10:15:34.169 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-18T10:15:34.171 INFO:tasks.rados.rados.0.smithi189.stdout:278: finishing write tid 5 to smithi18925896-33 2024-09-18T10:15:34.171 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 33 v 55 (ObjNum 112 snap 23 seq_num 112) dirty exists 2024-09-18T10:15:34.171 INFO:tasks.rados.rados.0.smithi189.stdout:278: left oid 33 (ObjNum 112 snap 23 seq_num 112) 2024-09-18T10:15:34.178 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-09-18T10:15:34.221 INFO:tasks.ceph.osd.14.smithi154.stderr:2024-09-18T10:15:34.218+0000 7f75a19e7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 22093) UID: 0 2024-09-18T10:15:34.275 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-18T10:15:34.275 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-18T10:15:34.275 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-18T10:15:34.275 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-18T10:15:34.275 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-18T10:15:34.284 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-09-18T10:15:34.321 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-18T10:15:34.316+0000 7f14c6a1a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 22138) UID: 0 2024-09-18T10:15:34.383 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-18T10:15:34.383 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-18T10:15:34.384 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-18T10:15:34.384 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-18T10:15:34.384 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.384 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.ds4 37/32 ECSubWrite(tid=31, reqid=client.4632.0:176, at_version=37'4, trim_to=0'0, roll_forward_to=35'2))", 2024-09-18T10:15:34.384 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:53.002892+0000", 2024-09-18T10:15:34.384 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 41.377381501000002, 2024-09-18T10:15:34.384 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10294343, 2024-09-18T10:15:34.384 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:34.384 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:34.384 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:34.384 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.384 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:34.385 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.002892+0000", 2024-09-18T10:15:34.385 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:34.385 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.385 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.385 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:34.385 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.002892+0000", 2024-09-18T10:15:34.385 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:34.385 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.385 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.385 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:34.385 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.002891+0000", 2024-09-18T10:15:34.385 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:15:34.385 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.386 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.386 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:34.386 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.003053+0000", 2024-09-18T10:15:34.386 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00016189399999999999 2024-09-18T10:15:34.386 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.386 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.386 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:34.386 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.003089+0000", 2024-09-18T10:15:34.386 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5329e-05 2024-09-18T10:15:34.386 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.386 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.386 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:34.386 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.003108+0000", 2024-09-18T10:15:34.387 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8963000000000001e-05 2024-09-18T10:15:34.387 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.387 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.387 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:34.387 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.103401+0000", 2024-09-18T10:15:34.387 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100293436 2024-09-18T10:15:34.387 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.387 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.387 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:34.387 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.103418+0000", 2024-09-18T10:15:34.387 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6722999999999999e-05 2024-09-18T10:15:34.387 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.387 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.387 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:34.388 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.105765+0000", 2024-09-18T10:15:34.388 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0023477279999999999 2024-09-18T10:15:34.388 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.388 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.388 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:34.388 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.105836+0000", 2024-09-18T10:15:34.388 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0246999999999993e-05 2024-09-18T10:15:34.388 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:34.388 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:34.388 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:34.388 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.388 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.388 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.3s1 37/32 ECSubWrite(tid=11, reqid=client.4632.0:189, at_version=37'8, trim_to=0'0, roll_forward_to=36'6))", 2024-09-18T10:15:34.389 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:53.096486+0000", 2024-09-18T10:15:34.389 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 41.283787560999997, 2024-09-18T10:15:34.389 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10315053, 2024-09-18T10:15:34.389 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:34.389 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:34.389 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:34.389 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.389 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:34.389 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.096486+0000", 2024-09-18T10:15:34.389 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:34.389 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.389 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.390 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:34.390 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.096486+0000", 2024-09-18T10:15:34.390 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:34.390 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.390 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.390 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:34.390 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.096486+0000", 2024-09-18T10:15:34.390 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:34.390 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.390 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.391 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:34.391 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.096802+0000", 2024-09-18T10:15:34.391 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00031615899999999999 2024-09-18T10:15:34.391 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.391 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.391 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:34.391 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.096841+0000", 2024-09-18T10:15:34.391 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9452000000000001e-05 2024-09-18T10:15:34.391 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.391 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.391 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:34.391 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.096866+0000", 2024-09-18T10:15:34.392 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5069e-05 2024-09-18T10:15:34.392 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.392 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.392 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:34.392 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.197104+0000", 2024-09-18T10:15:34.392 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100237886 2024-09-18T10:15:34.392 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.392 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.392 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:34.393 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.197130+0000", 2024-09-18T10:15:34.393 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5666000000000001e-05 2024-09-18T10:15:34.393 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.393 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.393 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:34.393 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.199585+0000", 2024-09-18T10:15:34.393 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002454777 2024-09-18T10:15:34.393 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.393 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.393 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:34.393 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.199637+0000", 2024-09-18T10:15:34.393 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2221e-05 2024-09-18T10:15:34.394 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:34.394 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:34.394 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:34.394 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.394 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.394 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.ds4 41/32 ECSubWrite(tid=41, reqid=osd.13.0:41, at_version=41'6, trim_to=0'0, roll_forward_to=37'4))", 2024-09-18T10:15:34.394 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:57.138491+0000", 2024-09-18T10:15:34.394 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 37.241782987999997, 2024-09-18T10:15:34.394 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.200128009, 2024-09-18T10:15:34.394 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:34.394 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:34.394 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:34.395 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.395 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:34.395 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.138491+0000", 2024-09-18T10:15:34.395 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:34.395 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.395 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.395 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:34.395 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.138491+0000", 2024-09-18T10:15:34.395 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:34.395 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.396 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.396 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:34.396 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.138490+0000", 2024-09-18T10:15:34.396 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:15:34.396 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.396 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.396 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:34.396 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.138555+0000", 2024-09-18T10:15:34.396 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4571000000000002e-05 2024-09-18T10:15:34.396 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.396 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.396 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:34.397 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.138581+0000", 2024-09-18T10:15:34.397 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6069000000000001e-05 2024-09-18T10:15:34.397 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.397 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.397 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:34.397 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.138598+0000", 2024-09-18T10:15:34.397 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7059e-05 2024-09-18T10:15:34.397 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.397 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.397 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:34.397 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.336697+0000", 2024-09-18T10:15:34.397 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.198099517 2024-09-18T10:15:34.397 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.398 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.398 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:34.398 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.336733+0000", 2024-09-18T10:15:34.398 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5369e-05 2024-09-18T10:15:34.398 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.398 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.398 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:34.398 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.338552+0000", 2024-09-18T10:15:34.398 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001819773 2024-09-18T10:15:34.398 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.398 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.398 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:34.399 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.338619+0000", 2024-09-18T10:15:34.399 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6390999999999995e-05 2024-09-18T10:15:34.399 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:34.399 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:34.399 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:34.399 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.399 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.399 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.ds4 41/32 ECSubWrite(tid=43, reqid=osd.13.0:41, at_version=0'0, trim_to=0'0, roll_forward_to=41'6))", 2024-09-18T10:15:34.399 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:57.441404+0000", 2024-09-18T10:15:34.399 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 36.938869379000003, 2024-09-18T10:15:34.399 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30108198600000002, 2024-09-18T10:15:34.399 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:34.400 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:34.400 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:34.400 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.400 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:34.400 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.441404+0000", 2024-09-18T10:15:34.400 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:34.400 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.400 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.400 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:34.400 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.441404+0000", 2024-09-18T10:15:34.400 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:34.400 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.401 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.401 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:34.401 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.441403+0000", 2024-09-18T10:15:34.401 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:34.401 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.401 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.401 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:34.401 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.441435+0000", 2024-09-18T10:15:34.401 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2502000000000002e-05 2024-09-18T10:15:34.401 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.401 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.402 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:34.402 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.441473+0000", 2024-09-18T10:15:34.402 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8302e-05 2024-09-18T10:15:34.402 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.402 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.402 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:34.402 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.441524+0000", 2024-09-18T10:15:34.402 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0679e-05 2024-09-18T10:15:34.402 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.402 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.402 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:34.402 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.641739+0000", 2024-09-18T10:15:34.402 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20021524299999999 2024-09-18T10:15:34.402 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.403 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.403 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:34.403 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.641766+0000", 2024-09-18T10:15:34.403 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7081999999999999e-05 2024-09-18T10:15:34.403 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.403 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.403 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:34.403 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.742432+0000", 2024-09-18T10:15:34.403 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10066531300000001 2024-09-18T10:15:34.403 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.403 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.403 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:34.403 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.742486+0000", 2024-09-18T10:15:34.403 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4691999999999997e-05 2024-09-18T10:15:34.403 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:34.404 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:34.404 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:34.404 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.404 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.404 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.6s5 41/40 ECSubWrite(tid=31, reqid=client.4632.0:249, at_version=41'13, trim_to=0'0, roll_forward_to=0'0))", 2024-09-18T10:15:34.404 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:57.550603+0000", 2024-09-18T10:15:34.404 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 36.829671189000003, 2024-09-18T10:15:34.404 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.103590501, 2024-09-18T10:15:34.404 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:34.404 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:34.404 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:34.404 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.404 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:34.405 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.550603+0000", 2024-09-18T10:15:34.405 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:34.405 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.405 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.405 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:34.405 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.550603+0000", 2024-09-18T10:15:34.405 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:34.405 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.405 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.405 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:34.405 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.550602+0000", 2024-09-18T10:15:34.405 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:15:34.405 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.406 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.406 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:34.406 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.551120+0000", 2024-09-18T10:15:34.406 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000518867 2024-09-18T10:15:34.406 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.406 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.406 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:34.406 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.551141+0000", 2024-09-18T10:15:34.406 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0873000000000002e-05 2024-09-18T10:15:34.406 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.406 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.406 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:34.406 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.551150+0000", 2024-09-18T10:15:34.407 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.884e-06 2024-09-18T10:15:34.407 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.407 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.407 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:34.407 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.651352+0000", 2024-09-18T10:15:34.407 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100201533 2024-09-18T10:15:34.407 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.407 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.407 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:34.407 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.651384+0000", 2024-09-18T10:15:34.407 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2471999999999997e-05 2024-09-18T10:15:34.407 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.407 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.407 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:34.408 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.654157+0000", 2024-09-18T10:15:34.408 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0027727440000000002 2024-09-18T10:15:34.408 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.408 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.408 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:34.408 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.654193+0000", 2024-09-18T10:15:34.408 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6242000000000003e-05 2024-09-18T10:15:34.408 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:34.408 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:34.408 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:34.408 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.408 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.408 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.ds4 47/32 ECSubWrite(tid=55, reqid=client.4632.0:332, at_version=47'8, trim_to=0'0, roll_forward_to=41'6))", 2024-09-18T10:15:34.409 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:02.796736+0000", 2024-09-18T10:15:34.409 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 31.58353761, 2024-09-18T10:15:34.409 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.106482128, 2024-09-18T10:15:34.409 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:34.409 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:34.409 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:34.409 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.409 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:34.409 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.796736+0000", 2024-09-18T10:15:34.409 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:34.409 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.409 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.409 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:34.410 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.796736+0000", 2024-09-18T10:15:34.410 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:34.410 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.410 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.410 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:34.410 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.796735+0000", 2024-09-18T10:15:34.410 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:15:34.410 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.410 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.410 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:34.410 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.796958+0000", 2024-09-18T10:15:34.410 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00022249800000000001 2024-09-18T10:15:34.410 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.411 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.411 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:34.411 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.796983+0000", 2024-09-18T10:15:34.411 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5454000000000001e-05 2024-09-18T10:15:34.411 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.411 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.411 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:34.411 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.897088+0000", 2024-09-18T10:15:34.411 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10010422099999999 2024-09-18T10:15:34.411 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.411 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.411 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:34.411 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.900995+0000", 2024-09-18T10:15:34.411 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0039075589999999997 2024-09-18T10:15:34.412 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.412 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.412 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:34.412 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.901018+0000", 2024-09-18T10:15:34.412 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2592999999999999e-05 2024-09-18T10:15:34.412 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.412 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.412 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:34.412 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.903153+0000", 2024-09-18T10:15:34.412 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0021355300000000001 2024-09-18T10:15:34.412 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.413 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.413 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:34.413 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.903218+0000", 2024-09-18T10:15:34.413 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5135000000000006e-05 2024-09-18T10:15:34.413 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:34.413 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:34.413 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:34.413 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.413 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.413 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:509 3.16s0 3:6c64c4fa:::smithi18925896-41:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e62)", 2024-09-18T10:15:34.413 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:17.914565+0000", 2024-09-18T10:15:34.413 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 16.465708771999999, 2024-09-18T10:15:34.414 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.059666403, 2024-09-18T10:15:34.414 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:34.414 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-18T10:15:34.414 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:34.414 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:34.414 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:34.414 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 509 2024-09-18T10:15:34.414 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.414 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:34.414 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.414 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:34.414 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.914565+0000", 2024-09-18T10:15:34.414 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:34.415 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.415 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.415 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:34.415 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.914565+0000", 2024-09-18T10:15:34.415 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:34.415 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.415 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.415 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:34.415 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.914564+0000", 2024-09-18T10:15:34.415 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:15:34.415 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.415 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.415 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:34.416 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.914570+0000", 2024-09-18T10:15:34.416 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3940000000000001e-06 2024-09-18T10:15:34.416 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.416 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.416 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:34.416 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.914585+0000", 2024-09-18T10:15:34.416 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4348000000000001e-05 2024-09-18T10:15:34.416 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.416 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.416 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:34.416 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.914591+0000", 2024-09-18T10:15:34.417 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4450000000000001e-06 2024-09-18T10:15:34.417 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.417 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.417 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:34.417 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.101342+0000", 2024-09-18T10:15:34.417 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.186750576 2024-09-18T10:15:34.417 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.417 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.417 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-18T10:15:34.417 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.101343+0000", 2024-09-18T10:15:34.417 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4300000000000001e-07 2024-09-18T10:15:34.417 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.417 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.418 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:34.418 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.334525+0000", 2024-09-18T10:15:34.418 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2331822159999999 2024-09-18T10:15:34.418 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.418 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.418 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for missing object", 2024-09-18T10:15:34.418 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.334706+0000", 2024-09-18T10:15:34.418 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00018068500000000001 2024-09-18T10:15:34.418 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.418 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.418 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:34.418 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.838318+0000", 2024-09-18T10:15:34.418 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.50361270300000005 2024-09-18T10:15:34.419 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.419 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.419 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2024-09-18T10:15:34.419 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.838493+0000", 2024-09-18T10:15:34.419 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017465199999999999 2024-09-18T10:15:34.419 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.419 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.419 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:34.419 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.972913+0000", 2024-09-18T10:15:34.419 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.134419977 2024-09-18T10:15:34.419 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.419 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.419 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:34.419 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.973261+0000", 2024-09-18T10:15:34.420 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00034786399999999997 2024-09-18T10:15:34.420 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.420 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.420 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:34.420 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.974231+0000", 2024-09-18T10:15:34.420 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00097053099999999995 2024-09-18T10:15:34.420 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:34.420 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:34.420 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:34.420 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.420 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.420 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.0s2 63/40 ECSubWrite(tid=25, reqid=client.4632.0:511, at_version=63'12, trim_to=0'0, roll_forward_to=60'10))", 2024-09-18T10:15:34.420 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:18.905739+0000", 2024-09-18T10:15:34.420 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 15.474534494, 2024-09-18T10:15:34.421 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.19488354899999999, 2024-09-18T10:15:34.421 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:34.421 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:34.421 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:34.421 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.421 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:34.421 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.905739+0000", 2024-09-18T10:15:34.421 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:34.421 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.421 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.421 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:34.421 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.905739+0000", 2024-09-18T10:15:34.421 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:34.422 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.422 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.422 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:34.422 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.905739+0000", 2024-09-18T10:15:34.422 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:34.422 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.422 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.422 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:34.422 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.905789+0000", 2024-09-18T10:15:34.422 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0754999999999998e-05 2024-09-18T10:15:34.422 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.422 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.423 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:34.423 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.905833+0000", 2024-09-18T10:15:34.423 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3779000000000002e-05 2024-09-18T10:15:34.423 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.423 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.423 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:34.423 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.905858+0000", 2024-09-18T10:15:34.423 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5208999999999999e-05 2024-09-18T10:15:34.423 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.423 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.423 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:34.424 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.006073+0000", 2024-09-18T10:15:34.424 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100214601 2024-09-18T10:15:34.424 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.424 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.424 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:34.424 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.006091+0000", 2024-09-18T10:15:34.424 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8301999999999999e-05 2024-09-18T10:15:34.424 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.424 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.424 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:34.424 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.100567+0000", 2024-09-18T10:15:34.424 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.094475297999999999 2024-09-18T10:15:34.425 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.425 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.425 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:34.425 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.100623+0000", 2024-09-18T10:15:34.425 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6237999999999999e-05 2024-09-18T10:15:34.425 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:34.425 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:34.425 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:34.425 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.425 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.425 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDScrubReserve(3.6s5 REQUEST e63)", 2024-09-18T10:15:34.426 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:19.068302+0000", 2024-09-18T10:15:34.426 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 15.311972244, 2024-09-18T10:15:34.426 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.26520614300000001, 2024-09-18T10:15:34.426 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:34.426 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-18T10:15:34.426 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:34.426 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.426 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:34.426 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.068302+0000", 2024-09-18T10:15:34.426 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:34.426 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.426 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.426 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:34.426 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.068302+0000", 2024-09-18T10:15:34.427 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:34.427 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.427 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.427 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:34.427 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.068300+0000", 2024-09-18T10:15:34.427 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:34.427 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.427 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.427 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:34.427 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.068311+0000", 2024-09-18T10:15:34.427 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1005999999999999e-05 2024-09-18T10:15:34.427 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.427 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.428 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:34.428 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.068345+0000", 2024-09-18T10:15:34.428 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3983000000000003e-05 2024-09-18T10:15:34.428 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.428 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.428 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:34.428 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.068372+0000", 2024-09-18T10:15:34.428 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7385999999999998e-05 2024-09-18T10:15:34.428 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.428 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.428 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:34.428 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.333409+0000", 2024-09-18T10:15:34.428 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.26503692400000001 2024-09-18T10:15:34.428 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.429 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.429 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:34.429 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.333444+0000", 2024-09-18T10:15:34.429 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4916000000000002e-05 2024-09-18T10:15:34.429 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.429 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.429 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:34.429 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.333508+0000", 2024-09-18T10:15:34.429 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3658000000000006e-05 2024-09-18T10:15:34.429 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:34.429 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:34.429 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:34.429 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.430 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.430 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpReadReply(3.16s0 63/62 ECSubReadReply(tid=0, attrs_read=0))", 2024-09-18T10:15:34.430 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:19.230883+0000", 2024-09-18T10:15:34.430 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 15.149391192, 2024-09-18T10:15:34.430 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.103113091, 2024-09-18T10:15:34.430 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:34.430 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:34.430 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:34.430 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.430 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:34.430 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.230883+0000", 2024-09-18T10:15:34.430 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:34.430 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.430 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.431 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:34.431 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.230883+0000", 2024-09-18T10:15:34.431 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:34.431 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.431 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.431 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:34.431 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.230882+0000", 2024-09-18T10:15:34.431 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:15:34.431 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.431 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.431 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:34.431 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.230892+0000", 2024-09-18T10:15:34.431 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0193999999999999e-05 2024-09-18T10:15:34.432 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.432 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.432 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:34.432 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.230908+0000", 2024-09-18T10:15:34.432 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6368e-05 2024-09-18T10:15:34.432 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.432 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.432 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:34.432 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.230918+0000", 2024-09-18T10:15:34.432 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0125e-05 2024-09-18T10:15:34.432 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.432 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.432 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:34.433 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.333866+0000", 2024-09-18T10:15:34.433 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.102947178 2024-09-18T10:15:34.433 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.433 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.433 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:34.433 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.333996+0000", 2024-09-18T10:15:34.433 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013009 2024-09-18T10:15:34.433 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:34.433 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:34.433 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:34.433 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.433 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.433 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDPGPushReply(3.16s0 63/62 [PushReplyOp(3:6c64c4fa:::smithi18925896-41:head)])", 2024-09-18T10:15:34.433 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:19.524127+0000", 2024-09-18T10:15:34.434 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 14.856146809, 2024-09-18T10:15:34.434 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20108187499999999, 2024-09-18T10:15:34.434 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:34.434 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:34.434 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:34.434 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.434 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:34.434 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.524127+0000", 2024-09-18T10:15:34.434 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:34.434 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.434 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.434 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:34.434 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.524127+0000", 2024-09-18T10:15:34.435 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:34.435 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.435 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.435 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:34.435 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.524117+0000", 2024-09-18T10:15:34.435 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999905 2024-09-18T10:15:34.435 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.435 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.435 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:34.435 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.524143+0000", 2024-09-18T10:15:34.435 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5534999999999999e-05 2024-09-18T10:15:34.435 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.435 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.436 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:34.436 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.524209+0000", 2024-09-18T10:15:34.436 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6192000000000006e-05 2024-09-18T10:15:34.436 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.436 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.436 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:34.436 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.524227+0000", 2024-09-18T10:15:34.436 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7986e-05 2024-09-18T10:15:34.436 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.437 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.437 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:34.437 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.724925+0000", 2024-09-18T10:15:34.437 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.200697662 2024-09-18T10:15:34.437 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.437 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.437 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:34.437 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.725209+0000", 2024-09-18T10:15:34.437 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000284102 2024-09-18T10:15:34.437 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:34.437 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:34.437 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:34.438 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.438 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.438 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDPGPushReply(3.16s0 63/62 [PushReplyOp(3:6c64c4fa:::smithi18925896-41:head)])", 2024-09-18T10:15:34.438 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:19.524280+0000", 2024-09-18T10:15:34.438 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 14.855993465999999, 2024-09-18T10:15:34.438 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.201340348, 2024-09-18T10:15:34.438 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:34.438 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:34.438 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:34.438 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.438 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:34.438 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.524280+0000", 2024-09-18T10:15:34.438 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:34.439 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.439 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.439 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:34.439 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.524280+0000", 2024-09-18T10:15:34.439 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:34.439 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.439 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.439 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:34.439 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.524274+0000", 2024-09-18T10:15:34.439 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999933 2024-09-18T10:15:34.439 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.439 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.439 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:34.440 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.524289+0000", 2024-09-18T10:15:34.440 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5722999999999998e-05 2024-09-18T10:15:34.440 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.440 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.440 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:34.440 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.524315+0000", 2024-09-18T10:15:34.440 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5188999999999999e-05 2024-09-18T10:15:34.440 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.440 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.440 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:34.440 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.524325+0000", 2024-09-18T10:15:34.440 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0736e-05 2024-09-18T10:15:34.440 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.441 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.441 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:34.441 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.725489+0000", 2024-09-18T10:15:34.441 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.201163377 2024-09-18T10:15:34.441 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.441 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.441 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:34.441 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.725621+0000", 2024-09-18T10:15:34.441 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013202600000000001 2024-09-18T10:15:34.441 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:34.441 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:34.441 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:34.442 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.442 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.442 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDPGPushReply(3.16s0 63/62 [PushReplyOp(3:6c64c4fa:::smithi18925896-41:head)])", 2024-09-18T10:15:34.442 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:19.524382+0000", 2024-09-18T10:15:34.442 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 14.855891811999999, 2024-09-18T10:15:34.442 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.21126635699999999, 2024-09-18T10:15:34.442 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:34.442 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:34.442 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:34.442 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.442 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:34.442 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.524382+0000", 2024-09-18T10:15:34.442 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:34.443 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.443 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.443 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:34.443 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.524382+0000", 2024-09-18T10:15:34.443 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:34.443 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.443 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.443 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:34.443 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.524381+0000", 2024-09-18T10:15:34.443 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:34.443 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.443 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.443 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:34.444 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.524388+0000", 2024-09-18T10:15:34.444 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4860000000000002e-06 2024-09-18T10:15:34.444 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.444 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.444 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:34.444 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.524420+0000", 2024-09-18T10:15:34.444 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2623000000000003e-05 2024-09-18T10:15:34.444 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.444 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.444 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:34.444 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.624550+0000", 2024-09-18T10:15:34.444 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100129128 2024-09-18T10:15:34.444 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.445 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.445 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:34.445 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.726236+0000", 2024-09-18T10:15:34.445 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10168595699999999 2024-09-18T10:15:34.445 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.445 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.445 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:34.445 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.735648+0000", 2024-09-18T10:15:34.445 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0094127829999999992 2024-09-18T10:15:34.445 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:34.445 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:34.445 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:34.445 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.445 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.446 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.ds4 63/32 ECSubWrite(tid=97, reqid=client.4632.0:526, at_version=63'18, trim_to=0'0, roll_forward_to=60'14))", 2024-09-18T10:15:34.446 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:19.563004+0000", 2024-09-18T10:15:34.446 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 14.817270275, 2024-09-18T10:15:34.446 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.103788874, 2024-09-18T10:15:34.446 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:34.446 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:34.446 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:34.446 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.446 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:34.446 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.563004+0000", 2024-09-18T10:15:34.446 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:34.446 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.447 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.447 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:34.447 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.563004+0000", 2024-09-18T10:15:34.447 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:34.447 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.447 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.447 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:34.447 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.563002+0000", 2024-09-18T10:15:34.447 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:34.447 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.447 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.447 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:34.447 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.563242+0000", 2024-09-18T10:15:34.447 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000240363 2024-09-18T10:15:34.448 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.448 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.448 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:34.448 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.563275+0000", 2024-09-18T10:15:34.448 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2505000000000001e-05 2024-09-18T10:15:34.448 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.448 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.448 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:34.448 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.563293+0000", 2024-09-18T10:15:34.448 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8266000000000001e-05 2024-09-18T10:15:34.448 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.448 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.448 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:34.449 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.663489+0000", 2024-09-18T10:15:34.449 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10019536900000001 2024-09-18T10:15:34.449 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.449 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.449 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:34.449 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.663515+0000", 2024-09-18T10:15:34.449 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6145000000000001e-05 2024-09-18T10:15:34.449 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.449 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.450 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:34.450 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.666737+0000", 2024-09-18T10:15:34.450 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0032219280000000002 2024-09-18T10:15:34.450 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.450 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.450 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:34.450 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.666792+0000", 2024-09-18T10:15:34.450 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5764999999999999e-05 2024-09-18T10:15:34.450 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:34.451 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:34.451 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:34.451 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.451 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.451 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.10s6 73/46 ECSubWrite(tid=194, reqid=client.4632.0:626, at_version=73'16, trim_to=0'0, roll_forward_to=60'14))", 2024-09-18T10:15:34.451 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:29.000022+0000", 2024-09-18T10:15:34.451 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.3802522379999997, 2024-09-18T10:15:34.451 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20370728099999999, 2024-09-18T10:15:34.451 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:34.451 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:34.451 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:34.451 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.451 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:34.452 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.000022+0000", 2024-09-18T10:15:34.452 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:34.452 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.452 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.452 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:34.452 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.000022+0000", 2024-09-18T10:15:34.452 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:34.452 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.452 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.452 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:34.452 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.000020+0000", 2024-09-18T10:15:34.452 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:34.453 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.453 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.453 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:34.453 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.000428+0000", 2024-09-18T10:15:34.453 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000408689 2024-09-18T10:15:34.453 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.453 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.453 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:34.453 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.000464+0000", 2024-09-18T10:15:34.453 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5768e-05 2024-09-18T10:15:34.453 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.453 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.453 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:34.453 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.100621+0000", 2024-09-18T10:15:34.454 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100157223 2024-09-18T10:15:34.454 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.454 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.454 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:34.454 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.200904+0000", 2024-09-18T10:15:34.454 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10028234900000001 2024-09-18T10:15:34.454 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.454 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.454 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:34.454 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.200921+0000", 2024-09-18T10:15:34.454 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7217000000000001e-05 2024-09-18T10:15:34.454 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.454 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.454 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:34.455 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.203672+0000", 2024-09-18T10:15:34.455 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0027511879999999999 2024-09-18T10:15:34.455 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.455 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.455 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:34.455 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.203729+0000", 2024-09-18T10:15:34.455 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6823999999999997e-05 2024-09-18T10:15:34.455 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:34.455 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:34.455 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:34.455 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.455 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.455 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.10s6 73/46 ECSubWrite(tid=195, reqid=client.4632.0:627, at_version=73'17, trim_to=0'0, roll_forward_to=60'14))", 2024-09-18T10:15:34.456 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:29.103724+0000", 2024-09-18T10:15:34.456 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.2765501549999998, 2024-09-18T10:15:34.456 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.202940394, 2024-09-18T10:15:34.456 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:34.456 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:34.456 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:34.456 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.456 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:34.456 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.103724+0000", 2024-09-18T10:15:34.456 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:34.456 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.456 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.456 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:34.457 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.103724+0000", 2024-09-18T10:15:34.457 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:34.457 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.457 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.457 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:34.457 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.103722+0000", 2024-09-18T10:15:34.457 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:34.457 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.457 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.457 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:34.457 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.104024+0000", 2024-09-18T10:15:34.457 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00030194100000000001 2024-09-18T10:15:34.457 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.458 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.458 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:34.458 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.104067+0000", 2024-09-18T10:15:34.458 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2840999999999999e-05 2024-09-18T10:15:34.458 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.458 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.458 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:34.458 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.204203+0000", 2024-09-18T10:15:34.458 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100136243 2024-09-18T10:15:34.458 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.458 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.458 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:34.458 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.304498+0000", 2024-09-18T10:15:34.459 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100294809 2024-09-18T10:15:34.459 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.459 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.459 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:34.459 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.304514+0000", 2024-09-18T10:15:34.459 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6163000000000001e-05 2024-09-18T10:15:34.459 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.459 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.459 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:34.459 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.306599+0000", 2024-09-18T10:15:34.459 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0020848759999999998 2024-09-18T10:15:34.459 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.459 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.460 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:34.460 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.306664+0000", 2024-09-18T10:15:34.460 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5331000000000002e-05 2024-09-18T10:15:34.460 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:34.460 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:34.460 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:34.460 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.460 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.460 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.6s5 74/40 ECSubWrite(tid=64, reqid=osd.0.0:64, at_version=74'40, trim_to=0'0, roll_forward_to=72'38))", 2024-09-18T10:15:34.460 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:29.670320+0000", 2024-09-18T10:15:34.460 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.7099533860000005, 2024-09-18T10:15:34.460 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.199229986, 2024-09-18T10:15:34.460 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:34.460 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:34.461 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:34.461 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.461 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:34.461 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.670320+0000", 2024-09-18T10:15:34.461 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:34.461 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.461 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.461 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:34.461 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.670320+0000", 2024-09-18T10:15:34.461 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:34.461 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.461 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.461 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:34.462 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.670319+0000", 2024-09-18T10:15:34.462 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:34.462 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.462 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.462 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:34.462 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.670369+0000", 2024-09-18T10:15:34.462 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0130000000000003e-05 2024-09-18T10:15:34.462 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.462 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.462 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:34.462 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.670398+0000", 2024-09-18T10:15:34.462 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9122e-05 2024-09-18T10:15:34.462 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.462 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.463 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:34.463 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.670415+0000", 2024-09-18T10:15:34.463 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6351e-05 2024-09-18T10:15:34.463 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.463 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.463 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:34.463 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.769393+0000", 2024-09-18T10:15:34.463 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.098978678 2024-09-18T10:15:34.463 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.463 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.463 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:34.463 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.769420+0000", 2024-09-18T10:15:34.463 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6704e-05 2024-09-18T10:15:34.464 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.464 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.464 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:34.464 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.869470+0000", 2024-09-18T10:15:34.464 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100049836 2024-09-18T10:15:34.464 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.464 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.464 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:34.464 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.869550+0000", 2024-09-18T10:15:34.464 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0487000000000004e-05 2024-09-18T10:15:34.464 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:34.464 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:34.464 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:34.464 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.465 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.465 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWriteReply(3.16s0 74/62 ECSubWriteReply(tid=8, last_complete=74'22, committed=1, applied=1))", 2024-09-18T10:15:34.465 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:29.670821+0000", 2024-09-18T10:15:34.465 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.7094530429999999, 2024-09-18T10:15:34.465 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.198638966, 2024-09-18T10:15:34.465 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:34.465 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:34.465 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:34.465 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.465 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:34.465 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.670821+0000", 2024-09-18T10:15:34.465 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:34.465 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.465 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.466 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:34.466 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.670821+0000", 2024-09-18T10:15:34.466 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:34.466 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.466 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.466 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:34.466 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.670820+0000", 2024-09-18T10:15:34.466 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:15:34.466 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.466 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.466 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:34.466 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.670826+0000", 2024-09-18T10:15:34.466 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7529999999999998e-06 2024-09-18T10:15:34.467 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.467 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.467 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:34.467 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.670840+0000", 2024-09-18T10:15:34.467 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4484000000000001e-05 2024-09-18T10:15:34.467 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.467 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.467 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:34.467 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.670847+0000", 2024-09-18T10:15:34.467 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7880000000000001e-06 2024-09-18T10:15:34.467 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.467 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.467 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:34.467 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.869415+0000", 2024-09-18T10:15:34.468 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.198568033 2024-09-18T10:15:34.468 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.468 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.468 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:34.468 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.869460+0000", 2024-09-18T10:15:34.468 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4765000000000003e-05 2024-09-18T10:15:34.468 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:34.468 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:34.468 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:34.468 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.468 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.468 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.10s6 75/46 ECSubWrite(tid=207, reqid=osd.6.0:207, at_version=75'23, trim_to=0'0, roll_forward_to=74'21))", 2024-09-18T10:15:34.469 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:30.569775+0000", 2024-09-18T10:15:34.469 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.8104988909999999, 2024-09-18T10:15:34.469 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.193288983, 2024-09-18T10:15:34.469 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:34.469 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:34.469 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:34.469 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.469 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:34.469 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:30.569775+0000", 2024-09-18T10:15:34.469 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:34.469 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.469 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.469 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:34.470 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:30.569775+0000", 2024-09-18T10:15:34.470 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:34.470 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.470 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.470 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:34.470 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:30.569775+0000", 2024-09-18T10:15:34.470 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:34.470 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.470 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.470 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:34.470 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:30.569810+0000", 2024-09-18T10:15:34.470 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5540000000000002e-05 2024-09-18T10:15:34.470 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.470 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.471 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:34.471 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:30.569826+0000", 2024-09-18T10:15:34.471 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6276000000000001e-05 2024-09-18T10:15:34.471 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.471 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.471 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:34.471 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:30.569836+0000", 2024-09-18T10:15:34.471 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0158e-05 2024-09-18T10:15:34.471 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.471 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.471 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:34.471 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:30.669996+0000", 2024-09-18T10:15:34.471 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10015942899999999 2024-09-18T10:15:34.472 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.472 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.472 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:34.472 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:30.670007+0000", 2024-09-18T10:15:34.472 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0903e-05 2024-09-18T10:15:34.472 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.472 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.472 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:34.472 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:30.763014+0000", 2024-09-18T10:15:34.472 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.093006722 2024-09-18T10:15:34.472 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.472 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.472 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:34.472 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:30.763064+0000", 2024-09-18T10:15:34.473 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0348e-05 2024-09-18T10:15:34.473 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:34.473 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:34.473 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:34.473 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.473 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.473 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.6s5 75/40 ECSubWrite(tid=66, reqid=osd.0.0:66, at_version=75'42, trim_to=0'0, roll_forward_to=74'40))", 2024-09-18T10:15:34.473 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:30.668776+0000", 2024-09-18T10:15:34.473 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.7114974940000001, 2024-09-18T10:15:34.473 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30218223300000002, 2024-09-18T10:15:34.473 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:34.473 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:34.473 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:34.473 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.474 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:34.474 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:30.668776+0000", 2024-09-18T10:15:34.474 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:34.474 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.474 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.474 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:34.474 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:30.668776+0000", 2024-09-18T10:15:34.474 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:34.474 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.474 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.474 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:34.474 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:30.668775+0000", 2024-09-18T10:15:34.474 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:34.474 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.474 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.475 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:34.475 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:30.668821+0000", 2024-09-18T10:15:34.475 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6147999999999999e-05 2024-09-18T10:15:34.475 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.475 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.475 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:34.475 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:30.668864+0000", 2024-09-18T10:15:34.475 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3004999999999999e-05 2024-09-18T10:15:34.475 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.475 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.475 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:34.475 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:30.768961+0000", 2024-09-18T10:15:34.475 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10009678499999999 2024-09-18T10:15:34.476 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.476 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.476 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:34.476 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:30.969322+0000", 2024-09-18T10:15:34.476 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20036179000000001 2024-09-18T10:15:34.476 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.476 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.476 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:34.476 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:30.969349+0000", 2024-09-18T10:15:34.476 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6795999999999999e-05 2024-09-18T10:15:34.476 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.476 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.476 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:34.476 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:30.970891+0000", 2024-09-18T10:15:34.477 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0015418459999999999 2024-09-18T10:15:34.477 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.477 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.477 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:34.477 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:30.970959+0000", 2024-09-18T10:15:34.477 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7490000000000006e-05 2024-09-18T10:15:34.477 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:34.477 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:34.477 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:34.477 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:34.477 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:34.477 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-18T10:15:34.479 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_ops_in_flight 2024-09-18T10:15:34.479 INFO:tasks.ceph.osd.10.smithi154.stderr:2024-09-18T10:15:34.474+0000 7fb73221a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 22091) UID: 0 2024-09-18T10:15:34.571 INFO:tasks.rados.rados.0.smithi189.stdout:278: done (2 left) 2024-09-18T10:15:34.572 INFO:tasks.rados.rados.0.smithi189.stdout:279: done (1 left) 2024-09-18T10:15:34.572 INFO:tasks.rados.rados.0.smithi189.stdout:280: done (0 left) 2024-09-18T10:15:34.572 INFO:tasks.rados.rados.0.smithi189.stdout:281: snap_create 2024-09-18T10:15:34.577 INFO:teuthology.orchestra.run.smithi029.stderr:nodeep-scrub is set 2024-09-18T10:15:34.580 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-18T10:15:34.573+0000 7f302f2c3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22187) UID: 0 2024-09-18T10:15:34.631 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-18T10:15:34.631 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-18T10:15:34.631 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.631 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDPGPush(3.19s4 78/74 [PushOp(3:9a1a774c:::smithi18925896-38:head, version: 71'94, data_included: [0~262144], data_size: 262144, omap_header_size: 0, omap_entries_size: 0, attrset_size: 24, recovery_info: ObjectRecoveryInfo(3:9a1a774c:::smithi18925896-38:head@71'94, size: 1638400, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: 1), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:1048576, data_complete:false, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:true, error:false))])", 2024-09-18T10:15:34.631 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:34.553262+0000", 2024-09-18T10:15:34.631 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.077367743000000003, 2024-09-18T10:15:34.631 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.077442754000000003, 2024-09-18T10:15:34.632 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:34.632 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-18T10:15:34.632 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:34.632 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.632 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:34.632 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:34.553262+0000", 2024-09-18T10:15:34.632 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:34.632 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.632 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.633 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:34.633 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:34.553262+0000", 2024-09-18T10:15:34.633 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:34.633 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.633 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.633 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:34.633 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:34.553261+0000", 2024-09-18T10:15:34.633 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:34.633 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.633 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.633 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:34.633 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:34.554050+0000", 2024-09-18T10:15:34.633 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000789483 2024-09-18T10:15:34.633 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.634 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.634 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:34.634 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:34.554088+0000", 2024-09-18T10:15:34.634 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7676000000000003e-05 2024-09-18T10:15:34.634 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.634 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.634 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:34.634 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:34.554102+0000", 2024-09-18T10:15:34.634 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4783000000000001e-05 2024-09-18T10:15:34.634 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:34.634 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:34.634 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:34.634 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:34.634 INFO:teuthology.orchestra.run.smithi136.stdout: ], 2024-09-18T10:15:34.635 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 1 2024-09-18T10:15:34.635 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-18T10:15:34.642 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_blocked_ops 2024-09-18T10:15:34.681 INFO:tasks.ceph.osd.11.smithi189.stderr:2024-09-18T10:15:34.678+0000 7f718b60a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 22400) UID: 0 2024-09-18T10:15:34.737 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-18T10:15:34.737 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-18T10:15:34.737 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-18T10:15:34.737 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-18T10:15:34.737 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-18T10:15:34.746 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_historic_ops 2024-09-18T10:15:34.782 INFO:tasks.ceph.osd.6.smithi154.stderr:2024-09-18T10:15:34.778+0000 7f0abc9e1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22089) UID: 0 2024-09-18T10:15:34.853 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-18T10:15:34.853 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-18T10:15:34.853 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-18T10:15:34.853 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-18T10:15:34.853 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.853 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:176 3.ds0 3:b1a14543:::smithi18925896-15:head [copy-from ver 4] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:34.853 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:52.680518+0000", 2024-09-18T10:15:34.853 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 42.168016643999998, 2024-09-18T10:15:34.853 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.42607371300000002, 2024-09-18T10:15:34.853 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:34.853 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:34.854 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:34.854 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:34.854 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:34.854 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 176 2024-09-18T10:15:34.854 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.854 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:34.854 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.854 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:34.854 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.680518+0000", 2024-09-18T10:15:34.854 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:34.854 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.854 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.854 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:34.854 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.680518+0000", 2024-09-18T10:15:34.855 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:34.855 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.855 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.855 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:34.855 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.680515+0000", 2024-09-18T10:15:34.855 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999971 2024-09-18T10:15:34.855 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.855 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.855 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:34.855 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.680533+0000", 2024-09-18T10:15:34.855 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8756000000000001e-05 2024-09-18T10:15:34.855 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.856 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.856 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:34.856 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.680573+0000", 2024-09-18T10:15:34.856 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9431999999999998e-05 2024-09-18T10:15:34.856 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.856 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.856 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:34.856 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.680587+0000", 2024-09-18T10:15:34.856 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.447e-05 2024-09-18T10:15:34.856 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.856 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.856 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:34.856 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.680755+0000", 2024-09-18T10:15:34.857 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000167584 2024-09-18T10:15:34.857 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.857 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.857 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-09-18T10:15:34.857 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.680784+0000", 2024-09-18T10:15:34.857 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9439000000000001e-05 2024-09-18T10:15:34.857 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.857 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.857 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:34.857 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.685398+0000", 2024-09-18T10:15:34.857 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0046140060000000004 2024-09-18T10:15:34.858 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.858 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.858 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:34.858 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.685502+0000", 2024-09-18T10:15:34.858 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000103702 2024-09-18T10:15:34.858 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.858 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.858 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:34.858 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.003083+0000", 2024-09-18T10:15:34.858 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.31758088600000001 2024-09-18T10:15:34.858 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.858 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.859 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:34.859 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.006087+0000", 2024-09-18T10:15:34.859 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0030044820000000002 2024-09-18T10:15:34.859 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.859 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.859 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:15:34.859 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.106397+0000", 2024-09-18T10:15:34.860 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100309469 2024-09-18T10:15:34.860 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.860 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.860 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:34.860 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.106591+0000", 2024-09-18T10:15:34.861 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000194513 2024-09-18T10:15:34.861 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:34.861 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:34.861 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:34.861 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.861 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.861 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:177 3.ds0 3:b1a14543:::smithi18925896-15:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:34.861 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:52.680610+0000", 2024-09-18T10:15:34.861 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 42.167923922999996, 2024-09-18T10:15:34.861 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.32474139800000001, 2024-09-18T10:15:34.861 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:34.861 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-18T10:15:34.861 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:34.862 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:34.862 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:34.862 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 177 2024-09-18T10:15:34.862 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.862 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:34.862 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.862 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:34.862 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.680610+0000", 2024-09-18T10:15:34.862 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:34.862 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.862 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.862 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:34.862 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.680610+0000", 2024-09-18T10:15:34.863 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:34.863 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.863 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.863 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:34.863 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.680609+0000", 2024-09-18T10:15:34.863 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:15:34.863 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.863 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.863 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:34.863 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.680614+0000", 2024-09-18T10:15:34.863 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6840000000000004e-06 2024-09-18T10:15:34.863 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.863 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.864 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:34.864 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.680625+0000", 2024-09-18T10:15:34.864 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0739e-05 2024-09-18T10:15:34.864 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.864 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.864 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:34.864 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.680655+0000", 2024-09-18T10:15:34.864 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0022000000000002e-05 2024-09-18T10:15:34.864 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.864 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.864 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:34.864 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.680965+0000", 2024-09-18T10:15:34.864 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00031078300000000002 2024-09-18T10:15:34.865 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.865 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.865 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:34.865 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.680976+0000", 2024-09-18T10:15:34.865 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0589e-05 2024-09-18T10:15:34.865 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.865 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.865 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:34.865 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.785862+0000", 2024-09-18T10:15:34.865 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10488597500000001 2024-09-18T10:15:34.865 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.865 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.865 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2024-09-18T10:15:34.866 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.785963+0000", 2024-09-18T10:15:34.866 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000100523 2024-09-18T10:15:34.866 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.866 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.866 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:34.866 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.005029+0000", 2024-09-18T10:15:34.866 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.219066329 2024-09-18T10:15:34.866 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.866 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.866 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:34.866 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.005193+0000", 2024-09-18T10:15:34.866 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00016436200000000001 2024-09-18T10:15:34.866 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.867 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.867 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:34.867 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.005352+0000", 2024-09-18T10:15:34.867 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000158486 2024-09-18T10:15:34.867 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:34.867 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:34.867 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:34.867 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.867 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.867 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:200 3.as0 3:5dc9cbed:::smithi18925896-13:2 [read 0~212992,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:34.867 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:52.694751+0000", 2024-09-18T10:15:34.867 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 42.153783492999999, 2024-09-18T10:15:34.867 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.40590051599999999, 2024-09-18T10:15:34.867 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:34.868 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-18T10:15:34.868 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:34.868 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:34.868 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:34.868 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 200 2024-09-18T10:15:34.868 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.868 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:34.868 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.868 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:34.868 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694751+0000", 2024-09-18T10:15:34.868 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:34.868 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.868 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.868 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:34.869 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694751+0000", 2024-09-18T10:15:34.869 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:34.869 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.869 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.869 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:34.869 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694750+0000", 2024-09-18T10:15:34.869 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:34.869 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.869 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.869 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:34.869 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694756+0000", 2024-09-18T10:15:34.869 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8119999999999996e-06 2024-09-18T10:15:34.870 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.870 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.870 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:34.870 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694762+0000", 2024-09-18T10:15:34.870 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8050000000000003e-06 2024-09-18T10:15:34.870 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.870 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.870 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:34.870 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694768+0000", 2024-09-18T10:15:34.870 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5549999999999999e-06 2024-09-18T10:15:34.870 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.870 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.870 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:34.871 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.695694+0000", 2024-09-18T10:15:34.871 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00092562999999999999 2024-09-18T10:15:34.871 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.871 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.871 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:34.871 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.695903+0000", 2024-09-18T10:15:34.871 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00020888699999999999 2024-09-18T10:15:34.871 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.871 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.871 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:34.871 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.100651+0000", 2024-09-18T10:15:34.871 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.40474845599999998 2024-09-18T10:15:34.871 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:34.872 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:34.872 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:34.872 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.872 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.872 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:201 3.as0 3:5dc9cbed:::smithi18925896-13:2 [read 0~212992,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:34.872 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:52.694813+0000", 2024-09-18T10:15:34.872 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 42.153721689000001, 2024-09-18T10:15:34.872 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.50652904899999995, 2024-09-18T10:15:34.872 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:34.872 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-18T10:15:34.872 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:34.872 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:34.873 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:34.873 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 201 2024-09-18T10:15:34.873 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.873 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:34.873 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.873 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:34.873 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694813+0000", 2024-09-18T10:15:34.873 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:34.873 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.873 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.873 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:34.873 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694813+0000", 2024-09-18T10:15:34.874 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:34.874 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.874 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.874 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:34.874 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694812+0000", 2024-09-18T10:15:34.874 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:34.874 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.874 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.874 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:34.874 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694816+0000", 2024-09-18T10:15:34.874 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7720000000000001e-06 2024-09-18T10:15:34.874 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.874 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.875 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:34.875 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694822+0000", 2024-09-18T10:15:34.875 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5529999999999998e-06 2024-09-18T10:15:34.875 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.875 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.875 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:34.875 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694827+0000", 2024-09-18T10:15:34.875 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1500000000000001e-06 2024-09-18T10:15:34.875 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.875 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.875 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:34.875 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.797100+0000", 2024-09-18T10:15:34.876 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.102273424 2024-09-18T10:15:34.876 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.876 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.876 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:34.876 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.797329+0000", 2024-09-18T10:15:34.876 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00022870200000000001 2024-09-18T10:15:34.876 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.876 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.876 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:34.876 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.201342+0000", 2024-09-18T10:15:34.876 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.40401293799999999 2024-09-18T10:15:34.876 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:34.876 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:34.877 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:34.877 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.877 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.877 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:207 3.as0 3:5c566071:::smithi18925896-28:head [copy-from ver 6] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:34.877 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:52.694855+0000", 2024-09-18T10:15:34.877 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 42.153678886999998, 2024-09-18T10:15:34.877 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.81394701899999999, 2024-09-18T10:15:34.877 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:34.877 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:34.877 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:34.877 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:34.877 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:34.878 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 207 2024-09-18T10:15:34.878 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.878 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:34.878 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.878 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:34.878 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694855+0000", 2024-09-18T10:15:34.878 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:34.878 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.878 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.878 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:34.878 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694855+0000", 2024-09-18T10:15:34.878 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:34.879 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.879 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.879 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:34.879 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694852+0000", 2024-09-18T10:15:34.879 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999967 2024-09-18T10:15:34.879 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.879 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.879 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:34.879 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694859+0000", 2024-09-18T10:15:34.879 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.08e-06 2024-09-18T10:15:34.879 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.879 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.879 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:34.880 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694866+0000", 2024-09-18T10:15:34.880 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9249999999999998e-06 2024-09-18T10:15:34.880 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.880 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.880 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:34.880 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694870+0000", 2024-09-18T10:15:34.880 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0330000000000002e-06 2024-09-18T10:15:34.880 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.880 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.880 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:34.880 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.900743+0000", 2024-09-18T10:15:34.881 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20587325400000001 2024-09-18T10:15:34.881 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.881 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.881 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:34.881 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.901026+0000", 2024-09-18T10:15:34.881 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00028313600000000002 2024-09-18T10:15:34.881 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.881 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.881 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:34.881 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.204074+0000", 2024-09-18T10:15:34.881 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30304722299999998 2024-09-18T10:15:34.881 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.881 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.882 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:34.882 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.207280+0000", 2024-09-18T10:15:34.882 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0032067049999999998 2024-09-18T10:15:34.882 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.882 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.882 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:15:34.882 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.508672+0000", 2024-09-18T10:15:34.882 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30139174800000001 2024-09-18T10:15:34.882 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.882 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.882 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:34.882 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.508802+0000", 2024-09-18T10:15:34.883 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013037500000000001 2024-09-18T10:15:34.883 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:34.883 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:34.883 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:34.883 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.883 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.883 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:208 3.as0 3:5c566071:::smithi18925896-28:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:34.883 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:52.694875+0000", 2024-09-18T10:15:34.883 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 42.153659726000001, 2024-09-18T10:15:34.883 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.51199464299999997, 2024-09-18T10:15:34.883 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:34.883 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-18T10:15:34.883 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:34.884 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:34.884 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:34.884 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 208 2024-09-18T10:15:34.884 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.884 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:34.884 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.884 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:34.884 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694875+0000", 2024-09-18T10:15:34.884 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:34.884 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.884 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.884 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:34.884 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694875+0000", 2024-09-18T10:15:34.885 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:34.885 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.885 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.885 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:34.885 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694874+0000", 2024-09-18T10:15:34.885 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:34.885 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.885 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.885 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:34.885 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694876+0000", 2024-09-18T10:15:34.885 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2369999999999999e-06 2024-09-18T10:15:34.885 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.886 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.886 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:34.886 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694881+0000", 2024-09-18T10:15:34.886 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0950000000000003e-06 2024-09-18T10:15:34.886 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.886 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.886 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:34.886 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694887+0000", 2024-09-18T10:15:34.886 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5690000000000004e-06 2024-09-18T10:15:34.886 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.886 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.886 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:34.886 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.901475+0000", 2024-09-18T10:15:34.886 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20658803000000001 2024-09-18T10:15:34.887 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.887 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.887 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2024-09-18T10:15:34.887 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.901550+0000", 2024-09-18T10:15:34.887 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5020999999999999e-05 2024-09-18T10:15:34.887 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.887 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.887 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:34.887 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.206415+0000", 2024-09-18T10:15:34.887 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30486486000000002 2024-09-18T10:15:34.887 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.887 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.887 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:34.888 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.206634+0000", 2024-09-18T10:15:34.888 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00021925499999999999 2024-09-18T10:15:34.888 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.888 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.888 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:34.888 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.206869+0000", 2024-09-18T10:15:34.888 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00023503599999999999 2024-09-18T10:15:34.888 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:34.888 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:34.888 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:34.888 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.888 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.888 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:332 3.ds0 3:b1a14543:::smithi18925896-15:head [append 0~737280 in=737280b] snapc b=[b,4] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-18T10:15:34.889 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:02.789344+0000", 2024-09-18T10:15:34.889 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 32.059190455, 2024-09-18T10:15:34.889 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.31055802500000002, 2024-09-18T10:15:34.889 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:34.889 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:34.889 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:34.889 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:34.889 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:34.889 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 332 2024-09-18T10:15:34.889 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.889 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:34.889 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.889 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:34.890 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.789344+0000", 2024-09-18T10:15:34.890 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:34.890 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.890 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.890 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:34.890 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.789344+0000", 2024-09-18T10:15:34.890 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:34.890 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.890 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.890 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:34.890 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.789342+0000", 2024-09-18T10:15:34.890 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999976 2024-09-18T10:15:34.890 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.891 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.891 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:34.891 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.790370+0000", 2024-09-18T10:15:34.891 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010286150000000001 2024-09-18T10:15:34.891 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.891 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.891 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:34.891 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.790403+0000", 2024-09-18T10:15:34.891 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2549000000000003e-05 2024-09-18T10:15:34.891 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.891 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.891 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:34.891 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.790414+0000", 2024-09-18T10:15:34.892 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1279e-05 2024-09-18T10:15:34.892 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.892 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.892 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:34.892 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.790526+0000", 2024-09-18T10:15:34.892 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000111994 2024-09-18T10:15:34.892 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.892 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.892 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-09-18T10:15:34.892 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.790540+0000", 2024-09-18T10:15:34.892 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4296e-05 2024-09-18T10:15:34.892 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.892 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.893 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:34.893 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.794331+0000", 2024-09-18T10:15:34.893 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0037907029999999999 2024-09-18T10:15:34.893 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.893 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.893 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:34.893 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.794483+0000", 2024-09-18T10:15:34.893 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000151772 2024-09-18T10:15:34.893 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.893 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.893 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:34.893 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.796730+0000", 2024-09-18T10:15:34.893 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0022475009999999998 2024-09-18T10:15:34.894 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.894 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.894 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:34.894 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.804001+0000", 2024-09-18T10:15:34.894 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0072711420000000004 2024-09-18T10:15:34.894 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.894 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.894 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:15:34.894 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:03.099777+0000", 2024-09-18T10:15:34.894 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.295775552 2024-09-18T10:15:34.894 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.894 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.895 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:34.895 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:03.099902+0000", 2024-09-18T10:15:34.895 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012481999999999999 2024-09-18T10:15:34.895 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:34.895 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:34.895 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:34.895 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.895 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.895 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:333 3.ds0 3:b1a14543:::smithi18925896-15:head [append 0~376832 in=376832b] snapc b=[b,4] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-18T10:15:34.895 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:02.792703+0000", 2024-09-18T10:15:34.895 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 32.055831060000003, 2024-09-18T10:15:34.895 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.40779418200000001, 2024-09-18T10:15:34.896 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:34.896 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:34.896 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:34.896 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:34.896 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:34.896 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 333 2024-09-18T10:15:34.896 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.896 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:34.896 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.896 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:34.896 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.792703+0000", 2024-09-18T10:15:34.896 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:34.896 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.896 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.897 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:34.897 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.792703+0000", 2024-09-18T10:15:34.897 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:34.897 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.897 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.897 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:34.897 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.792702+0000", 2024-09-18T10:15:34.897 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:34.897 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.897 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.897 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:34.897 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.793113+0000", 2024-09-18T10:15:34.897 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00041176199999999998 2024-09-18T10:15:34.898 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.898 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.898 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:34.898 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.793131+0000", 2024-09-18T10:15:34.898 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7802e-05 2024-09-18T10:15:34.898 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.898 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.898 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:34.898 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.793140+0000", 2024-09-18T10:15:34.898 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0790000000000007e-06 2024-09-18T10:15:34.898 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.898 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.898 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:34.899 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.793223+0000", 2024-09-18T10:15:34.899 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2414000000000001e-05 2024-09-18T10:15:34.899 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.899 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.899 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:34.899 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.793233+0000", 2024-09-18T10:15:34.899 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.076e-05 2024-09-18T10:15:34.899 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.899 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.899 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:34.899 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.799019+0000", 2024-09-18T10:15:34.899 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0057857109999999998 2024-09-18T10:15:34.899 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.899 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.900 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:34.900 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.799289+0000", 2024-09-18T10:15:34.900 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00026981800000000001 2024-09-18T10:15:34.900 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.900 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.900 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:34.900 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.801174+0000", 2024-09-18T10:15:34.900 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00188511 2024-09-18T10:15:34.900 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.900 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.900 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:34.900 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.804009+0000", 2024-09-18T10:15:34.900 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002835181 2024-09-18T10:15:34.901 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.901 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.901 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:15:34.901 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:03.200390+0000", 2024-09-18T10:15:34.901 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.39638093699999999 2024-09-18T10:15:34.901 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.901 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.901 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:34.901 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:03.200497+0000", 2024-09-18T10:15:34.901 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000107243 2024-09-18T10:15:34.901 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:34.901 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:34.901 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:34.902 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.902 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.902 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:334 3.ds0 3:b1a14543:::smithi18925896-15:head [setxattr _header (56) in=63b] snapc b=[b,4] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-18T10:15:34.902 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:02.793164+0000", 2024-09-18T10:15:34.902 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 32.055369915, 2024-09-18T10:15:34.902 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.408134104, 2024-09-18T10:15:34.902 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:34.902 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:34.902 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:34.902 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:34.902 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:34.902 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 334 2024-09-18T10:15:34.902 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.903 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:34.903 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.903 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:34.903 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.793164+0000", 2024-09-18T10:15:34.903 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:34.903 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.903 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.903 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:34.903 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.793164+0000", 2024-09-18T10:15:34.903 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:34.903 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.903 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.903 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:34.903 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.793163+0000", 2024-09-18T10:15:34.904 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:34.904 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.904 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.904 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:34.904 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.793170+0000", 2024-09-18T10:15:34.904 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3050000000000004e-06 2024-09-18T10:15:34.904 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.904 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.904 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:34.904 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.793180+0000", 2024-09-18T10:15:34.904 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8069999999999994e-06 2024-09-18T10:15:34.904 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.904 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.905 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:34.905 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.793187+0000", 2024-09-18T10:15:34.905 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6879999999999999e-06 2024-09-18T10:15:34.905 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.905 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.905 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:34.905 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.793293+0000", 2024-09-18T10:15:34.905 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000105441 2024-09-18T10:15:34.905 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.905 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.905 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:34.905 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.793302+0000", 2024-09-18T10:15:34.905 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.5769999999999994e-06 2024-09-18T10:15:34.906 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.906 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.906 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:34.906 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.802703+0000", 2024-09-18T10:15:34.906 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0094011400000000005 2024-09-18T10:15:34.906 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.906 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.906 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:34.906 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.802949+0000", 2024-09-18T10:15:34.906 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00024604500000000002 2024-09-18T10:15:34.906 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.906 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.906 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:34.907 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.803628+0000", 2024-09-18T10:15:34.907 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000679391 2024-09-18T10:15:34.907 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.907 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.907 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:34.907 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.804224+0000", 2024-09-18T10:15:34.907 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00059597000000000003 2024-09-18T10:15:34.907 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.907 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.907 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:15:34.907 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:03.201019+0000", 2024-09-18T10:15:34.907 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.39679527799999997 2024-09-18T10:15:34.907 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.907 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.908 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:34.908 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:03.201298+0000", 2024-09-18T10:15:34.908 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00027909199999999999 2024-09-18T10:15:34.908 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:34.908 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:34.908 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:34.908 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.908 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.908 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:335 3.ds0 3:b1a14543:::smithi18925896-15:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e47)", 2024-09-18T10:15:34.908 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:02.793201+0000", 2024-09-18T10:15:34.908 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 32.055333453999999, 2024-09-18T10:15:34.908 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.51366298600000004, 2024-09-18T10:15:34.908 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:34.909 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-18T10:15:34.909 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:34.909 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:34.909 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:34.909 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 335 2024-09-18T10:15:34.909 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.909 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:34.909 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.909 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:34.909 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.793201+0000", 2024-09-18T10:15:34.909 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:34.909 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.909 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.909 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:34.910 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.793201+0000", 2024-09-18T10:15:34.910 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:34.910 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.910 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.910 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:34.910 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.793200+0000", 2024-09-18T10:15:34.910 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:34.910 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.910 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.910 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:34.910 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.793206+0000", 2024-09-18T10:15:34.910 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2199999999999997e-06 2024-09-18T10:15:34.910 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.911 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.911 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:34.911 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.793221+0000", 2024-09-18T10:15:34.911 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5092000000000001e-05 2024-09-18T10:15:34.911 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.911 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.911 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:34.911 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.793229+0000", 2024-09-18T10:15:34.911 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1820000000000006e-06 2024-09-18T10:15:34.911 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.911 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.911 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:34.911 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.793367+0000", 2024-09-18T10:15:34.912 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000138057 2024-09-18T10:15:34.912 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.912 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.912 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:34.912 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.793384+0000", 2024-09-18T10:15:34.912 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6731e-05 2024-09-18T10:15:34.912 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.912 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.912 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:34.912 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.804064+0000", 2024-09-18T10:15:34.912 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010680185 2024-09-18T10:15:34.912 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.912 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.912 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2024-09-18T10:15:34.913 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.804199+0000", 2024-09-18T10:15:34.913 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013532999999999999 2024-09-18T10:15:34.913 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.913 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.913 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:34.913 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:03.202193+0000", 2024-09-18T10:15:34.913 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.39799325099999999 2024-09-18T10:15:34.913 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.913 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.913 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:34.913 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:03.202292+0000", 2024-09-18T10:15:34.913 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9628000000000002e-05 2024-09-18T10:15:34.913 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.914 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.914 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:34.914 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:03.306864+0000", 2024-09-18T10:15:34.914 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.104571572 2024-09-18T10:15:34.914 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:34.914 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:34.914 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:34.914 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.914 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.914 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:503 3.fs0 3:feb35297:::smithi18925896-37:head [append 0~753664] snapc 19=[19,18,16,12,11,10,f,e] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T10:15:34.914 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:17.908895+0000", 2024-09-18T10:15:34.914 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 16.939639473, 2024-09-18T10:15:34.915 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.51525800200000005, 2024-09-18T10:15:34.915 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:34.915 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:34.915 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:34.915 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:34.915 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:34.915 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 503 2024-09-18T10:15:34.915 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.915 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:34.915 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.915 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:34.915 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.908895+0000", 2024-09-18T10:15:34.915 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:34.916 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.916 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.916 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:34.916 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.908895+0000", 2024-09-18T10:15:34.916 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:34.916 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.916 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.916 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:34.916 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.908892+0000", 2024-09-18T10:15:34.916 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999976 2024-09-18T10:15:34.916 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.916 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.916 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:34.917 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.910143+0000", 2024-09-18T10:15:34.917 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0012507040000000001 2024-09-18T10:15:34.917 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.917 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.917 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:34.917 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.910167+0000", 2024-09-18T10:15:34.917 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4176999999999999e-05 2024-09-18T10:15:34.917 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.917 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.917 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:34.917 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.910178+0000", 2024-09-18T10:15:34.917 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0669999999999999e-05 2024-09-18T10:15:34.917 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.918 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.918 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:34.918 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.910329+0000", 2024-09-18T10:15:34.918 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000150746 2024-09-18T10:15:34.918 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.918 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.918 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-09-18T10:15:34.918 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.910346+0000", 2024-09-18T10:15:34.918 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7680999999999999e-05 2024-09-18T10:15:34.918 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.918 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.918 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:34.918 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.015009+0000", 2024-09-18T10:15:34.918 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.104662914 2024-09-18T10:15:34.919 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.919 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.919 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:34.919 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.015304+0000", 2024-09-18T10:15:34.919 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00029439899999999998 2024-09-18T10:15:34.919 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.919 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.919 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:34.919 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.017669+0000", 2024-09-18T10:15:34.919 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0023647910000000002 2024-09-18T10:15:34.919 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.919 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.919 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:34.920 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.126654+0000", 2024-09-18T10:15:34.920 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.108985055 2024-09-18T10:15:34.920 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.920 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.920 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:15:34.920 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.423961+0000", 2024-09-18T10:15:34.920 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.29730786399999998 2024-09-18T10:15:34.920 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.921 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.921 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:34.921 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.424153+0000", 2024-09-18T10:15:34.921 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019128099999999999 2024-09-18T10:15:34.921 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:34.921 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:34.921 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:34.921 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.921 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.922 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:504 3.fs0 3:feb35297:::smithi18925896-37:head [append 0~720896] snapc 19=[19,18,16,12,11,10,f,e] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T10:15:34.922 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:17.912996+0000", 2024-09-18T10:15:34.922 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 16.935538438000002, 2024-09-18T10:15:34.922 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.61195014999999997, 2024-09-18T10:15:34.922 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:34.922 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:34.922 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:34.922 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:34.922 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:34.922 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 504 2024-09-18T10:15:34.922 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.923 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:34.923 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.923 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:34.923 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.912996+0000", 2024-09-18T10:15:34.923 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:34.923 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.923 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.923 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:34.923 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.912996+0000", 2024-09-18T10:15:34.923 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:34.923 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.923 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.923 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:34.924 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.912994+0000", 2024-09-18T10:15:34.924 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:34.924 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.924 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.924 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:34.924 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.913907+0000", 2024-09-18T10:15:34.924 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00091254300000000003 2024-09-18T10:15:34.924 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.924 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.924 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:34.924 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.913950+0000", 2024-09-18T10:15:34.924 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3019000000000003e-05 2024-09-18T10:15:34.924 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.925 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.925 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:34.925 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.913960+0000", 2024-09-18T10:15:34.925 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0305e-05 2024-09-18T10:15:34.925 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.925 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.925 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:34.925 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.914067+0000", 2024-09-18T10:15:34.925 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000106696 2024-09-18T10:15:34.925 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.925 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.926 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:34.926 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.914079+0000", 2024-09-18T10:15:34.926 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2330000000000001e-05 2024-09-18T10:15:34.926 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.926 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.926 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:34.926 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.119322+0000", 2024-09-18T10:15:34.926 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20524277899999999 2024-09-18T10:15:34.926 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.926 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.926 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:34.927 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.119508+0000", 2024-09-18T10:15:34.927 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00018648800000000001 2024-09-18T10:15:34.927 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.927 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.927 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:34.927 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.121315+0000", 2024-09-18T10:15:34.927 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001806483 2024-09-18T10:15:34.927 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.927 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.927 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:34.927 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.127304+0000", 2024-09-18T10:15:34.927 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0059895599999999997 2024-09-18T10:15:34.927 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.928 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.928 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:15:34.928 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.524828+0000", 2024-09-18T10:15:34.928 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.39752343600000001 2024-09-18T10:15:34.928 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.928 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.928 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:34.928 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.524946+0000", 2024-09-18T10:15:34.928 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000118193 2024-09-18T10:15:34.928 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:34.928 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:34.928 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:34.928 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.928 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.929 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:505 3.fs0 3:feb35297:::smithi18925896-37:head [append 0~49152] snapc 19=[19,18,16,12,11,10,f,e] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T10:15:34.929 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:17.913979+0000", 2024-09-18T10:15:34.929 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 16.934554959, 2024-09-18T10:15:34.929 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.71150136900000005, 2024-09-18T10:15:34.929 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:34.929 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:34.929 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:34.929 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:34.929 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:34.929 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 505 2024-09-18T10:15:34.929 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.929 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:34.929 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.930 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:34.930 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.913979+0000", 2024-09-18T10:15:34.930 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:34.930 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.930 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.930 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:34.930 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.913979+0000", 2024-09-18T10:15:34.930 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:34.930 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.930 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.930 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:34.930 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.913978+0000", 2024-09-18T10:15:34.931 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:34.931 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.931 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.931 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:34.931 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.914061+0000", 2024-09-18T10:15:34.931 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2533000000000004e-05 2024-09-18T10:15:34.931 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.931 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.931 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:34.931 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.914077+0000", 2024-09-18T10:15:34.931 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6277e-05 2024-09-18T10:15:34.931 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.931 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.932 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:34.932 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.914087+0000", 2024-09-18T10:15:34.932 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0282000000000001e-05 2024-09-18T10:15:34.932 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.932 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.932 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:34.932 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.914144+0000", 2024-09-18T10:15:34.932 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6784999999999999e-05 2024-09-18T10:15:34.932 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.932 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.932 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:34.932 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.914151+0000", 2024-09-18T10:15:34.932 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.968e-06 2024-09-18T10:15:34.933 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.933 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.933 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:34.933 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.122495+0000", 2024-09-18T10:15:34.933 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.208343742 2024-09-18T10:15:34.933 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.933 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.933 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:34.933 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.122788+0000", 2024-09-18T10:15:34.933 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00029334199999999998 2024-09-18T10:15:34.933 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.933 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.933 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:34.934 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.124367+0000", 2024-09-18T10:15:34.934 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0015794699999999999 2024-09-18T10:15:34.934 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.934 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.934 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:34.934 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.127307+0000", 2024-09-18T10:15:34.934 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0029394780000000001 2024-09-18T10:15:34.934 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.934 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.934 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:15:34.934 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.625281+0000", 2024-09-18T10:15:34.934 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.497974166 2024-09-18T10:15:34.934 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.935 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.935 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:34.935 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.625481+0000", 2024-09-18T10:15:34.935 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019956100000000001 2024-09-18T10:15:34.935 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:34.935 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:34.935 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:34.935 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.935 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.935 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:506 3.fs0 3:feb35297:::smithi18925896-37:head [setxattr _header (57) in=7b] snapc 19=[19,18,16,12,11,10,f,e] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T10:15:34.935 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:17.914106+0000", 2024-09-18T10:15:34.935 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 16.934428692000001, 2024-09-18T10:15:34.935 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.71184716599999998, 2024-09-18T10:15:34.935 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:34.936 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:34.936 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:34.936 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:34.936 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:34.936 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 506 2024-09-18T10:15:34.936 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.936 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:34.936 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.936 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:34.936 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.914106+0000", 2024-09-18T10:15:34.936 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:34.936 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.936 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.937 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:34.937 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.914106+0000", 2024-09-18T10:15:34.937 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:34.937 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.937 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.937 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:34.937 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.914105+0000", 2024-09-18T10:15:34.937 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:15:34.937 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.937 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.937 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:34.937 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.914110+0000", 2024-09-18T10:15:34.937 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.045e-06 2024-09-18T10:15:34.938 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.938 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.938 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:34.938 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.914117+0000", 2024-09-18T10:15:34.938 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6510000000000001e-06 2024-09-18T10:15:34.938 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.938 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.938 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:34.938 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.914122+0000", 2024-09-18T10:15:34.938 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5839999999999996e-06 2024-09-18T10:15:34.938 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.938 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.938 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:34.939 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.014305+0000", 2024-09-18T10:15:34.939 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100183091 2024-09-18T10:15:34.939 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.939 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.939 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:34.939 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.014325+0000", 2024-09-18T10:15:34.939 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0032000000000001e-05 2024-09-18T10:15:34.939 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.939 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.939 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:34.939 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.125511+0000", 2024-09-18T10:15:34.939 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.111185942 2024-09-18T10:15:34.939 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.939 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.940 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:34.940 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.125760+0000", 2024-09-18T10:15:34.940 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00024864100000000002 2024-09-18T10:15:34.940 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.940 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.940 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:34.940 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.126375+0000", 2024-09-18T10:15:34.940 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00061536299999999996 2024-09-18T10:15:34.940 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.940 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.940 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:34.940 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.227656+0000", 2024-09-18T10:15:34.940 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.101281097 2024-09-18T10:15:34.940 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.941 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.941 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:15:34.941 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.625791+0000", 2024-09-18T10:15:34.941 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.39813470299999998 2024-09-18T10:15:34.941 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.941 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.941 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:34.941 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.625953+0000", 2024-09-18T10:15:34.941 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00016192999999999999 2024-09-18T10:15:34.941 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:34.941 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:34.941 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:34.941 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.942 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.942 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:507 3.fs0 3:feb35297:::smithi18925896-37:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e62)", 2024-09-18T10:15:34.942 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:17.914159+0000", 2024-09-18T10:15:34.942 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 16.934375576000001, 2024-09-18T10:15:34.942 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0151776809999999, 2024-09-18T10:15:34.942 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:34.942 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-18T10:15:34.942 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:34.942 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:34.942 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:34.942 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 507 2024-09-18T10:15:34.942 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.942 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:34.942 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.943 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:34.943 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.914159+0000", 2024-09-18T10:15:34.943 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:34.943 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.943 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.943 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:34.943 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.914159+0000", 2024-09-18T10:15:34.943 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:34.943 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.943 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.943 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:34.943 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.914158+0000", 2024-09-18T10:15:34.943 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:15:34.944 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.944 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.944 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:34.944 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.914162+0000", 2024-09-18T10:15:34.944 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0729999999999998e-06 2024-09-18T10:15:34.944 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.944 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.944 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:34.944 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.914176+0000", 2024-09-18T10:15:34.944 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4258e-05 2024-09-18T10:15:34.944 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.944 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.944 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:34.945 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.014273+0000", 2024-09-18T10:15:34.945 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10009702400000001 2024-09-18T10:15:34.945 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.945 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.945 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:34.945 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.126754+0000", 2024-09-18T10:15:34.945 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.112480581 2024-09-18T10:15:34.945 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.945 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.945 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2024-09-18T10:15:34.945 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.127051+0000", 2024-09-18T10:15:34.945 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00029721199999999999 2024-09-18T10:15:34.945 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.946 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.946 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:34.946 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.626655+0000", 2024-09-18T10:15:34.946 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.49960442399999999 2024-09-18T10:15:34.946 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.946 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.946 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:34.946 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.626826+0000", 2024-09-18T10:15:34.946 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000170459 2024-09-18T10:15:34.946 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.946 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.946 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:34.946 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.929336+0000", 2024-09-18T10:15:34.946 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30251042900000003 2024-09-18T10:15:34.947 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:34.947 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:34.947 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:34.947 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.947 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.947 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:523 3.as0 3:5c566071:::smithi18925896-28:head [read 0~557056,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T10:15:34.947 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:19.442344+0000", 2024-09-18T10:15:34.947 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 15.406190370000001, 2024-09-18T10:15:34.947 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.385121027, 2024-09-18T10:15:34.947 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:34.948 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-18T10:15:34.948 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:34.948 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:34.948 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:34.948 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 523 2024-09-18T10:15:34.948 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.948 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:34.948 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.948 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:34.948 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.442344+0000", 2024-09-18T10:15:34.948 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:34.949 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.949 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.949 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:34.949 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.442344+0000", 2024-09-18T10:15:34.949 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:34.949 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.949 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.949 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:34.949 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.442343+0000", 2024-09-18T10:15:34.949 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:15:34.950 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.950 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.950 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:34.950 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.442350+0000", 2024-09-18T10:15:34.950 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7939999999999997e-06 2024-09-18T10:15:34.950 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.950 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.950 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:34.950 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.442364+0000", 2024-09-18T10:15:34.950 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3909e-05 2024-09-18T10:15:34.950 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.950 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.950 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:34.950 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.442373+0000", 2024-09-18T10:15:34.951 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3659999999999998e-06 2024-09-18T10:15:34.951 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.951 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.951 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:34.951 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.444188+0000", 2024-09-18T10:15:34.951 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001814735 2024-09-18T10:15:34.951 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.951 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.951 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:34.951 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.444337+0000", 2024-09-18T10:15:34.951 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014881599999999999 2024-09-18T10:15:34.951 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.951 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.952 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:34.952 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.827465+0000", 2024-09-18T10:15:34.952 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.38312838100000002 2024-09-18T10:15:34.952 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:34.952 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:34.952 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:34.952 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.952 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.952 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:524 3.ds0 3:b1a14543:::smithi18925896-15:head [append 0~442368] snapc 1a=[1a,19,18,16,12,11,10,f,e] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-18T10:15:34.952 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:19.452015+0000", 2024-09-18T10:15:34.952 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 15.396519594000001, 2024-09-18T10:15:34.952 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.55248773200000001, 2024-09-18T10:15:34.952 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:34.953 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:34.953 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:34.953 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:34.953 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:34.953 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 524 2024-09-18T10:15:34.953 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.953 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:34.953 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.953 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:34.953 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.452015+0000", 2024-09-18T10:15:34.953 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:34.953 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.954 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.954 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:34.954 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.452015+0000", 2024-09-18T10:15:34.954 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:34.954 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.954 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.954 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:34.954 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.452012+0000", 2024-09-18T10:15:34.954 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999976 2024-09-18T10:15:34.954 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.954 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.954 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:34.954 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.452793+0000", 2024-09-18T10:15:34.954 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00078048600000000005 2024-09-18T10:15:34.955 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.955 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.955 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:34.955 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.452825+0000", 2024-09-18T10:15:34.955 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2672999999999997e-05 2024-09-18T10:15:34.955 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.955 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.955 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:34.955 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.552970+0000", 2024-09-18T10:15:34.955 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100144728 2024-09-18T10:15:34.955 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.955 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.955 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:34.955 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.553127+0000", 2024-09-18T10:15:34.956 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000157202 2024-09-18T10:15:34.956 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.956 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.956 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:34.956 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.553439+0000", 2024-09-18T10:15:34.956 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00031139500000000001 2024-09-18T10:15:34.956 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.956 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.956 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:34.956 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.556240+0000", 2024-09-18T10:15:34.956 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0028012509999999998 2024-09-18T10:15:34.956 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.956 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.957 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:34.957 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.665529+0000", 2024-09-18T10:15:34.957 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10928916700000001 2024-09-18T10:15:34.957 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.957 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.957 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:15:34.957 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:20.004378+0000", 2024-09-18T10:15:34.957 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.33884916300000001 2024-09-18T10:15:34.957 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.957 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.957 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:34.957 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:20.004502+0000", 2024-09-18T10:15:34.957 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012407999999999999 2024-09-18T10:15:34.957 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:34.958 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:34.958 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:34.958 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.958 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.958 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:525 3.ds0 3:b1a14543:::smithi18925896-15:head [append 0~540672] snapc 1a=[1a,19,18,16,12,11,10,f,e] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-18T10:15:34.958 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:19.553028+0000", 2024-09-18T10:15:34.958 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 15.295505755000001, 2024-09-18T10:15:34.958 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.45756244000000001, 2024-09-18T10:15:34.958 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:34.958 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:34.958 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:34.960 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:34.961 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:34.961 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 525 2024-09-18T10:15:34.961 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.961 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:34.961 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.961 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:34.961 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.553028+0000", 2024-09-18T10:15:34.961 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:34.961 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.961 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.961 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:34.961 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.553028+0000", 2024-09-18T10:15:34.962 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:34.962 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.962 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.962 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:34.962 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.553026+0000", 2024-09-18T10:15:34.962 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:34.962 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.962 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.962 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:34.962 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.553796+0000", 2024-09-18T10:15:34.962 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00076915200000000005 2024-09-18T10:15:34.962 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.962 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.963 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:34.963 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.553819+0000", 2024-09-18T10:15:34.963 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3008999999999999e-05 2024-09-18T10:15:34.963 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.963 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.963 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:34.963 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.553829+0000", 2024-09-18T10:15:34.963 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0543999999999999e-05 2024-09-18T10:15:34.963 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.963 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.963 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:34.963 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.558075+0000", 2024-09-18T10:15:34.963 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.004245932 2024-09-18T10:15:34.963 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.964 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.964 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:34.964 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.558384+0000", 2024-09-18T10:15:34.964 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00030926100000000002 2024-09-18T10:15:34.964 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.964 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.964 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:34.964 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.559950+0000", 2024-09-18T10:15:34.964 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001565615 2024-09-18T10:15:34.964 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.964 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.964 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:34.964 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.665535+0000", 2024-09-18T10:15:34.965 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.105584707 2024-09-18T10:15:34.965 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.965 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.965 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:15:34.965 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:20.010428+0000", 2024-09-18T10:15:34.965 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.34489341800000001 2024-09-18T10:15:34.965 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.965 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.965 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:34.965 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:20.010591+0000", 2024-09-18T10:15:34.965 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000162845 2024-09-18T10:15:34.966 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:34.966 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:34.966 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:34.966 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.966 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.966 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:526 3.ds0 3:b1a14543:::smithi18925896-15:head [append 0~622592] snapc 1a=[1a,19,18,16,12,11,10,f,e] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-18T10:15:34.966 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:19.553845+0000", 2024-09-18T10:15:34.966 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 15.29468904, 2024-09-18T10:15:34.966 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.55723060999999996, 2024-09-18T10:15:34.966 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:34.966 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:34.966 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:34.966 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:34.967 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:34.967 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 526 2024-09-18T10:15:34.967 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.967 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:34.967 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.967 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:34.967 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.553845+0000", 2024-09-18T10:15:34.967 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:34.967 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.967 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.967 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:34.967 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.553845+0000", 2024-09-18T10:15:34.967 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:34.967 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.968 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.968 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:34.968 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.553844+0000", 2024-09-18T10:15:34.968 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:34.968 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.968 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.968 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:34.968 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.554666+0000", 2024-09-18T10:15:34.968 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00082239100000000001 2024-09-18T10:15:34.968 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.968 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.968 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:34.968 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.554679+0000", 2024-09-18T10:15:34.969 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.271e-05 2024-09-18T10:15:34.969 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.969 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.969 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:34.969 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.554685+0000", 2024-09-18T10:15:34.969 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4230000000000002e-06 2024-09-18T10:15:34.969 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.969 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.969 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:34.969 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.561235+0000", 2024-09-18T10:15:34.969 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0065498179999999998 2024-09-18T10:15:34.969 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.969 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.970 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:34.970 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.561523+0000", 2024-09-18T10:15:34.970 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000287862 2024-09-18T10:15:34.970 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.970 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.970 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:34.970 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.562904+0000", 2024-09-18T10:15:34.970 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0013810090000000001 2024-09-18T10:15:34.970 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.970 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.970 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:34.970 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.666083+0000", 2024-09-18T10:15:34.970 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.103178495 2024-09-18T10:15:34.971 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.971 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.971 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:15:34.971 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:20.110956+0000", 2024-09-18T10:15:34.971 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.44487371999999997 2024-09-18T10:15:34.971 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.971 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.971 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:34.971 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:20.111076+0000", 2024-09-18T10:15:34.971 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000119436 2024-09-18T10:15:34.971 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:34.971 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:34.971 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:34.972 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.972 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.972 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:527 3.ds0 3:b1a14543:::smithi18925896-15:head [setxattr _header (57) in=7b] snapc 1a=[1a,19,18,16,12,11,10,f,e] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-18T10:15:34.972 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:19.554694+0000", 2024-09-18T10:15:34.972 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 15.293840673, 2024-09-18T10:15:34.972 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.55695282099999999, 2024-09-18T10:15:34.972 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:34.972 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:34.972 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:34.972 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:34.972 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:34.972 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 527 2024-09-18T10:15:34.972 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.973 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:34.973 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.973 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:34.973 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.554694+0000", 2024-09-18T10:15:34.973 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:34.973 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.973 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.973 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:34.973 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.554694+0000", 2024-09-18T10:15:34.973 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:34.973 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.973 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.973 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:34.974 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.554692+0000", 2024-09-18T10:15:34.974 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:15:34.974 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.974 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.974 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:34.974 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.554698+0000", 2024-09-18T10:15:34.974 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8599999999999998e-06 2024-09-18T10:15:34.974 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.974 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.974 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:34.974 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.554707+0000", 2024-09-18T10:15:34.974 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4090000000000003e-06 2024-09-18T10:15:34.974 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.974 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.975 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:34.975 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.554712+0000", 2024-09-18T10:15:34.975 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4500000000000003e-06 2024-09-18T10:15:34.975 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.975 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.975 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:34.975 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.663747+0000", 2024-09-18T10:15:34.975 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10903526400000001 2024-09-18T10:15:34.975 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.975 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.975 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:34.975 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.664066+0000", 2024-09-18T10:15:34.975 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00031885099999999999 2024-09-18T10:15:34.975 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.976 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.976 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:34.976 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.665080+0000", 2024-09-18T10:15:34.976 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010141200000000001 2024-09-18T10:15:34.976 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.976 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.976 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:34.976 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.666476+0000", 2024-09-18T10:15:34.976 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001395456 2024-09-18T10:15:34.976 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.976 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.976 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:15:34.977 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:20.111512+0000", 2024-09-18T10:15:34.977 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.445035929 2024-09-18T10:15:34.977 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:34.977 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:34.977 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:34.977 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:20.111646+0000", 2024-09-18T10:15:34.977 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000134542 2024-09-18T10:15:34.977 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:34.977 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:34.977 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:34.978 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:34.978 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:34.978 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-18T10:15:34.979 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T10:15:34.980 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-18T10:15:34.973+0000 7f302f2c3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22187) UID: 0 2024-09-18T10:15:35.080 INFO:tasks.ceph.osd.12.smithi029.stderr:2024-09-18T10:15:35.073+0000 7f8e4e627700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 22190) UID: 0 2024-09-18T10:15:35.089 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-09-18T10:15:35.089 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-09-18T10:15:35.089 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-09-18T10:15:35.089 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-09-18T10:15:35.099 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T10:15:35.181 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-18T10:15:35.177+0000 7f302f2c3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22187) UID: 0 2024-09-18T10:15:35.198 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-09-18T10:15:35.199 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-09-18T10:15:35.199 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-09-18T10:15:35.199 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-09-18T10:15:35.199 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-09-18T10:15:35.208 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T10:15:35.283 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-18T10:15:35.280+0000 7f578c52f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22134) UID: 0 2024-09-18T10:15:35.311 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-09-18T10:15:35.312 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-09-18T10:15:35.312 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-09-18T10:15:35.312 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-09-18T10:15:35.312 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.312 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.6s3 36/32 ECSubWrite(tid=15, reqid=client.4632.0:122, at_version=36'10, trim_to=0'0, roll_forward_to=36'7))", 2024-09-18T10:15:35.312 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:50.804653+0000", 2024-09-18T10:15:35.312 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 44.503497828999997, 2024-09-18T10:15:35.312 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.108947892, 2024-09-18T10:15:35.312 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:35.312 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:15:35.312 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:35.313 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.313 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:35.313 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.804653+0000", 2024-09-18T10:15:35.313 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:35.313 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.313 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.313 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:35.313 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.804653+0000", 2024-09-18T10:15:35.313 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:35.313 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.313 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.314 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:35.314 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.804653+0000", 2024-09-18T10:15:35.314 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:15:35.314 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.314 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.314 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:35.314 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.804677+0000", 2024-09-18T10:15:35.314 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4043e-05 2024-09-18T10:15:35.314 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.314 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.314 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:35.315 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.804691+0000", 2024-09-18T10:15:35.315 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.402e-05 2024-09-18T10:15:35.315 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.315 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.315 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:35.315 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.906029+0000", 2024-09-18T10:15:35.315 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.101337911 2024-09-18T10:15:35.315 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.315 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.315 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:35.315 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.906208+0000", 2024-09-18T10:15:35.315 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000179292 2024-09-18T10:15:35.316 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.316 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.316 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:35.316 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.906232+0000", 2024-09-18T10:15:35.316 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.3739000000000001e-05 2024-09-18T10:15:35.316 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.316 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.316 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:35.316 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.913534+0000", 2024-09-18T10:15:35.316 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0073023029999999996 2024-09-18T10:15:35.316 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.316 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.317 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:35.317 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:50.913601+0000", 2024-09-18T10:15:35.317 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.7314e-05 2024-09-18T10:15:35.317 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:35.317 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:35.318 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:35.318 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.318 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.318 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.9s1 36/32 ECSubWrite(tid=16, reqid=client.4632.0:143, at_version=36'12, trim_to=0'0, roll_forward_to=36'9))", 2024-09-18T10:15:35.318 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:51.130949+0000", 2024-09-18T10:15:35.318 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 44.177202065000003, 2024-09-18T10:15:35.318 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.202020333, 2024-09-18T10:15:35.318 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:35.318 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:15:35.318 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:35.318 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.318 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:35.318 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.130949+0000", 2024-09-18T10:15:35.319 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:35.319 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.319 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.319 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:35.319 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.130949+0000", 2024-09-18T10:15:35.319 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:35.319 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.319 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.319 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:35.319 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.130947+0000", 2024-09-18T10:15:35.319 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:35.319 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.319 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.319 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:35.320 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.131149+0000", 2024-09-18T10:15:35.320 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00020114800000000001 2024-09-18T10:15:35.320 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.320 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.320 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:35.320 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.131182+0000", 2024-09-18T10:15:35.320 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.3844000000000003e-05 2024-09-18T10:15:35.321 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.321 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.322 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:35.322 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.131204+0000", 2024-09-18T10:15:35.322 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.1753999999999999e-05 2024-09-18T10:15:35.322 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.322 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.322 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:35.323 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.231367+0000", 2024-09-18T10:15:35.323 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100162472 2024-09-18T10:15:35.323 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.323 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.323 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:35.323 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.231380+0000", 2024-09-18T10:15:35.323 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3624e-05 2024-09-18T10:15:35.324 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.324 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.324 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:35.324 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.332934+0000", 2024-09-18T10:15:35.324 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.101553714 2024-09-18T10:15:35.325 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.326 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.326 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:35.326 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.332969+0000", 2024-09-18T10:15:35.326 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5525999999999998e-05 2024-09-18T10:15:35.326 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:35.326 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:35.326 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:35.326 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.327 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.327 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.9s1 36/32 ECSubWrite(tid=17, reqid=client.4632.0:144, at_version=36'13, trim_to=0'0, roll_forward_to=36'9))", 2024-09-18T10:15:35.327 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:51.131916+0000", 2024-09-18T10:15:35.327 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 44.176235237999997, 2024-09-18T10:15:35.327 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20110955899999999, 2024-09-18T10:15:35.327 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:35.327 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:15:35.327 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:35.327 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.327 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:35.327 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.131916+0000", 2024-09-18T10:15:35.327 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:35.327 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.327 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.328 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:35.328 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.131916+0000", 2024-09-18T10:15:35.328 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:35.328 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.328 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.328 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:35.328 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.131915+0000", 2024-09-18T10:15:35.328 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:35.328 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.328 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.328 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:35.328 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.131949+0000", 2024-09-18T10:15:35.328 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.4576999999999998e-05 2024-09-18T10:15:35.329 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.329 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.329 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:35.329 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.131979+0000", 2024-09-18T10:15:35.329 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9454e-05 2024-09-18T10:15:35.329 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.329 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.329 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:35.329 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.131992+0000", 2024-09-18T10:15:35.329 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3029e-05 2024-09-18T10:15:35.329 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.329 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.329 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:35.329 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.332112+0000", 2024-09-18T10:15:35.330 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.200120295 2024-09-18T10:15:35.330 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.330 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.330 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:35.330 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.332126+0000", 2024-09-18T10:15:35.330 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4494999999999999e-05 2024-09-18T10:15:35.330 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.330 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.330 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:35.330 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.332996+0000", 2024-09-18T10:15:35.330 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00086989000000000001 2024-09-18T10:15:35.330 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.330 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.331 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:35.331 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.333026+0000", 2024-09-18T10:15:35.331 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9159999999999999e-05 2024-09-18T10:15:35.331 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:35.331 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:35.332 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:35.332 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.332 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.332 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpRead(3.4s3 36/32 ECSubRead(tid=11, to_read={3:2650be24:::smithi18925896-45:head=0,4096,0}, subchunks={3:2650be24:::smithi18925896-45:head=[0,1]}, attrs_to_read=))", 2024-09-18T10:15:35.332 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:51.425236+0000", 2024-09-18T10:15:35.332 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 43.882915150999999, 2024-09-18T10:15:35.333 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.200812289, 2024-09-18T10:15:35.333 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:35.333 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:15:35.333 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:35.333 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.333 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:35.334 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.425236+0000", 2024-09-18T10:15:35.334 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:35.334 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.334 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.335 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:35.335 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.425236+0000", 2024-09-18T10:15:35.335 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:35.335 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.335 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.335 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:35.336 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.425234+0000", 2024-09-18T10:15:35.336 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:35.336 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.336 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.336 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:35.336 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.425253+0000", 2024-09-18T10:15:35.336 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8468999999999999e-05 2024-09-18T10:15:35.337 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.337 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.337 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:35.337 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.425289+0000", 2024-09-18T10:15:35.337 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.6514999999999999e-05 2024-09-18T10:15:35.337 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.338 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.338 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:35.338 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.525424+0000", 2024-09-18T10:15:35.338 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100135157 2024-09-18T10:15:35.338 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.338 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.339 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:35.339 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.625672+0000", 2024-09-18T10:15:35.339 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100247555 2024-09-18T10:15:35.339 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.339 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.339 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:35.339 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.626048+0000", 2024-09-18T10:15:35.339 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00037635000000000001 2024-09-18T10:15:35.339 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:35.339 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:35.339 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:35.339 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.339 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.339 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.9s1 37/32 ECSubWrite(tid=30, reqid=client.4632.0:187, at_version=0'0, trim_to=0'0, roll_forward_to=37'17))", 2024-09-18T10:15:35.340 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:53.090512+0000", 2024-09-18T10:15:35.340 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 42.217639296999998, 2024-09-18T10:15:35.340 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.21478031, 2024-09-18T10:15:35.340 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:35.340 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:15:35.340 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:35.340 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.340 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:35.340 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.090512+0000", 2024-09-18T10:15:35.340 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:35.340 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.341 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.341 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:35.341 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.090512+0000", 2024-09-18T10:15:35.341 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:35.342 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.342 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.342 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:35.342 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.090510+0000", 2024-09-18T10:15:35.342 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:35.342 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.342 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.342 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:35.342 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.090535+0000", 2024-09-18T10:15:35.342 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4848999999999999e-05 2024-09-18T10:15:35.342 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.342 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.342 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:35.343 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.090566+0000", 2024-09-18T10:15:35.343 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1096000000000002e-05 2024-09-18T10:15:35.343 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.343 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.343 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:35.343 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.190713+0000", 2024-09-18T10:15:35.343 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10014709300000001 2024-09-18T10:15:35.343 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.343 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.343 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:35.343 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.290966+0000", 2024-09-18T10:15:35.343 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10025297399999999 2024-09-18T10:15:35.344 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.344 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.344 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:35.344 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.290981+0000", 2024-09-18T10:15:35.344 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4960000000000001e-05 2024-09-18T10:15:35.344 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.344 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.344 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:35.344 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.305252+0000", 2024-09-18T10:15:35.344 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.014270738999999999 2024-09-18T10:15:35.344 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.345 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.345 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:35.345 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.305292+0000", 2024-09-18T10:15:35.345 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.0092000000000002e-05 2024-09-18T10:15:35.345 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:35.345 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:35.345 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:35.345 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.346 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.346 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.6s3 41/40 ECSubWrite(tid=32, reqid=client.4632.0:250, at_version=41'14, trim_to=0'0, roll_forward_to=0'0))", 2024-09-18T10:15:35.346 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:57.552410+0000", 2024-09-18T10:15:35.346 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 37.755741192000002, 2024-09-18T10:15:35.346 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20275494299999999, 2024-09-18T10:15:35.346 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:35.346 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:15:35.346 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:35.346 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.346 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:35.346 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.552410+0000", 2024-09-18T10:15:35.346 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:35.346 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.346 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.347 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:35.347 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.552410+0000", 2024-09-18T10:15:35.347 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:35.347 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.347 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.347 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:35.347 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.552409+0000", 2024-09-18T10:15:35.347 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:35.347 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.347 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.347 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:35.348 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.552478+0000", 2024-09-18T10:15:35.348 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.9084999999999993e-05 2024-09-18T10:15:35.348 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.348 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.348 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:35.348 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.552488+0000", 2024-09-18T10:15:35.348 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.7990000000000005e-06 2024-09-18T10:15:35.348 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.348 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.348 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:35.348 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.552496+0000", 2024-09-18T10:15:35.349 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.3200000000000002e-06 2024-09-18T10:15:35.350 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.350 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.350 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:35.350 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.653130+0000", 2024-09-18T10:15:35.350 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100634549 2024-09-18T10:15:35.350 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.350 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.351 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:35.351 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.653144+0000", 2024-09-18T10:15:35.351 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3619e-05 2024-09-18T10:15:35.351 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.351 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.351 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:35.351 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.755128+0000", 2024-09-18T10:15:35.351 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.101983924 2024-09-18T10:15:35.351 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.352 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.352 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:35.352 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.755165+0000", 2024-09-18T10:15:35.352 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.7361999999999999e-05 2024-09-18T10:15:35.352 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:35.352 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:35.352 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:35.352 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.352 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.352 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.6s3 41/40 ECSubWrite(tid=33, reqid=client.4632.0:251, at_version=41'15, trim_to=0'0, roll_forward_to=0'0))", 2024-09-18T10:15:35.352 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:57.553805+0000", 2024-09-18T10:15:35.352 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 37.754346611999999, 2024-09-18T10:15:35.352 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20199056600000001, 2024-09-18T10:15:35.353 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:35.353 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:15:35.353 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:35.353 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.353 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:35.353 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.553805+0000", 2024-09-18T10:15:35.353 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:35.353 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.353 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.353 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:35.353 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.553805+0000", 2024-09-18T10:15:35.353 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:35.353 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.354 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.354 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:35.354 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.553804+0000", 2024-09-18T10:15:35.354 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:15:35.354 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.354 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.354 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:35.354 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.553827+0000", 2024-09-18T10:15:35.354 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.3516e-05 2024-09-18T10:15:35.354 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.354 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.354 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:35.354 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.553845+0000", 2024-09-18T10:15:35.354 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7881999999999999e-05 2024-09-18T10:15:35.355 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.355 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.355 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:35.355 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.553853+0000", 2024-09-18T10:15:35.355 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.7109999999999995e-06 2024-09-18T10:15:35.355 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.355 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.355 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:35.355 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.754527+0000", 2024-09-18T10:15:35.355 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.200674198 2024-09-18T10:15:35.355 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.355 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.355 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:35.356 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.754543+0000", 2024-09-18T10:15:35.356 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.5682e-05 2024-09-18T10:15:35.356 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.356 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.356 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:35.356 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.755758+0000", 2024-09-18T10:15:35.356 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0012155860000000001 2024-09-18T10:15:35.356 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.356 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.356 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:35.356 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.755795+0000", 2024-09-18T10:15:35.356 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.6808999999999999e-05 2024-09-18T10:15:35.356 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:35.357 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:35.357 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:35.357 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.357 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.357 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.19s1 51/40 ECSubWrite(tid=77, reqid=client.4632.0:407, at_version=51'45, trim_to=0'0, roll_forward_to=50'39))", 2024-09-18T10:15:35.357 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:06.877639+0000", 2024-09-18T10:15:35.357 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 28.430511987999999, 2024-09-18T10:15:35.357 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.301778663, 2024-09-18T10:15:35.357 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:35.357 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:15:35.357 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:35.357 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.357 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:35.357 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:06.877639+0000", 2024-09-18T10:15:35.358 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:35.358 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.358 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.358 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:35.358 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:06.877639+0000", 2024-09-18T10:15:35.358 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:35.358 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.358 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.358 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:35.358 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:06.877638+0000", 2024-09-18T10:15:35.358 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:35.358 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.358 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.359 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:35.359 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:06.877679+0000", 2024-09-18T10:15:35.359 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.0908000000000002e-05 2024-09-18T10:15:35.359 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.359 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.359 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:35.359 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:06.877717+0000", 2024-09-18T10:15:35.359 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.7988999999999998e-05 2024-09-18T10:15:35.359 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.359 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.359 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:35.359 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:06.977811+0000", 2024-09-18T10:15:35.359 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100094533 2024-09-18T10:15:35.359 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.360 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.360 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:35.360 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:06.977916+0000", 2024-09-18T10:15:35.360 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000104823 2024-09-18T10:15:35.360 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.360 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.360 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:35.360 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:06.977932+0000", 2024-09-18T10:15:35.360 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.6099999999999998e-05 2024-09-18T10:15:35.360 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.360 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.360 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:35.360 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:07.179377+0000", 2024-09-18T10:15:35.361 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.201444861 2024-09-18T10:15:35.361 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.361 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.361 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:35.361 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:07.179418+0000", 2024-09-18T10:15:35.361 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.0645999999999997e-05 2024-09-18T10:15:35.361 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:35.361 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:35.361 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:35.361 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.361 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.361 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.19s1 51/40 ECSubWrite(tid=78, reqid=client.4632.0:408, at_version=51'46, trim_to=0'0, roll_forward_to=50'39))", 2024-09-18T10:15:35.361 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:06.977840+0000", 2024-09-18T10:15:35.362 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 28.330311283, 2024-09-18T10:15:35.362 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.201619306, 2024-09-18T10:15:35.362 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:35.362 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:15:35.362 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:35.362 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.362 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:35.362 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:06.977840+0000", 2024-09-18T10:15:35.362 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:35.362 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.362 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.362 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:35.362 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:06.977840+0000", 2024-09-18T10:15:35.362 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:35.363 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.363 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.363 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:35.363 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:06.977839+0000", 2024-09-18T10:15:35.363 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:15:35.363 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.363 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.363 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:35.363 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:06.977887+0000", 2024-09-18T10:15:35.363 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.8470999999999997e-05 2024-09-18T10:15:35.363 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.363 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.363 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:35.364 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:06.977924+0000", 2024-09-18T10:15:35.364 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.6729e-05 2024-09-18T10:15:35.364 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.364 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.364 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:35.364 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:06.977937+0000", 2024-09-18T10:15:35.364 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2476e-05 2024-09-18T10:15:35.364 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.364 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.364 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:35.364 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:07.078614+0000", 2024-09-18T10:15:35.364 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100677007 2024-09-18T10:15:35.364 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.364 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.365 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:35.365 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:07.078634+0000", 2024-09-18T10:15:35.365 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.0134000000000001e-05 2024-09-18T10:15:35.369 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.369 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.369 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:35.369 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:07.179429+0000", 2024-09-18T10:15:35.370 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100795736 2024-09-18T10:15:35.370 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.370 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.370 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:35.370 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:07.179459+0000", 2024-09-18T10:15:35.370 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9788999999999999e-05 2024-09-18T10:15:35.370 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:35.370 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:35.370 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:35.370 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.370 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.370 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDScrubReserve(3.11s6 REQUEST e51)", 2024-09-18T10:15:35.370 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:07.146316+0000", 2024-09-18T10:15:35.371 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 28.161834730999999, 2024-09-18T10:15:35.371 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.200408688, 2024-09-18T10:15:35.371 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:35.371 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "started", 2024-09-18T10:15:35.371 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:35.371 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.371 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:35.371 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:07.146316+0000", 2024-09-18T10:15:35.371 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:35.371 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.372 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.372 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:35.372 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:07.146316+0000", 2024-09-18T10:15:35.372 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:35.372 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.372 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.372 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:35.372 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:07.146316+0000", 2024-09-18T10:15:35.372 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:35.372 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.372 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.372 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:35.372 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:07.146320+0000", 2024-09-18T10:15:35.373 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.3340000000000003e-06 2024-09-18T10:15:35.373 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.373 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.373 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:35.373 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:07.146347+0000", 2024-09-18T10:15:35.373 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6899000000000001e-05 2024-09-18T10:15:35.373 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.373 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.373 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:35.373 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:07.246429+0000", 2024-09-18T10:15:35.373 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10008152100000001 2024-09-18T10:15:35.373 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.373 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.374 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:35.374 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:07.346637+0000", 2024-09-18T10:15:35.374 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10020834100000001 2024-09-18T10:15:35.374 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.374 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.374 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:35.374 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:07.346666+0000", 2024-09-18T10:15:35.374 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9269e-05 2024-09-18T10:15:35.374 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.374 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.374 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:35.374 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:07.346725+0000", 2024-09-18T10:15:35.374 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.8748000000000001e-05 2024-09-18T10:15:35.375 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:35.375 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:35.375 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:35.375 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.375 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.375 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.19s1 57/40 ECSubWrite(tid=94, reqid=osd.7.0:92, at_version=0'0, trim_to=0'0, roll_forward_to=57'50))", 2024-09-18T10:15:35.375 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:13.157902+0000", 2024-09-18T10:15:35.375 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 22.150249491, 2024-09-18T10:15:35.375 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.111878063, 2024-09-18T10:15:35.375 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:35.375 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:15:35.375 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:35.375 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.376 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:35.376 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.157902+0000", 2024-09-18T10:15:35.376 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:35.376 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.376 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.376 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:35.376 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.157902+0000", 2024-09-18T10:15:35.376 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:35.376 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.376 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.376 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:35.376 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.157901+0000", 2024-09-18T10:15:35.376 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:15:35.376 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.377 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.377 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:35.377 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.157919+0000", 2024-09-18T10:15:35.377 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8241999999999999e-05 2024-09-18T10:15:35.377 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.377 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.377 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:35.377 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.157946+0000", 2024-09-18T10:15:35.377 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.7025999999999998e-05 2024-09-18T10:15:35.377 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.377 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.377 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:35.377 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.157957+0000", 2024-09-18T10:15:35.378 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1422e-05 2024-09-18T10:15:35.378 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.378 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.378 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:35.378 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.263418+0000", 2024-09-18T10:15:35.378 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.105460525 2024-09-18T10:15:35.378 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.378 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.378 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:35.378 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.263436+0000", 2024-09-18T10:15:35.378 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8451e-05 2024-09-18T10:15:35.378 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.378 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.378 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:35.379 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.269729+0000", 2024-09-18T10:15:35.379 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0062925519999999999 2024-09-18T10:15:35.379 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.379 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.379 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:35.379 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.269780+0000", 2024-09-18T10:15:35.379 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.0926e-05 2024-09-18T10:15:35.379 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:35.379 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:35.379 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:35.379 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.379 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.379 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.1s6 58/40 ECSubWrite(tid=113, reqid=osd.4.0:112, at_version=0'0, trim_to=0'0, roll_forward_to=57'20))", 2024-09-18T10:15:35.380 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:13.649564+0000", 2024-09-18T10:15:35.380 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 21.658586824, 2024-09-18T10:15:35.380 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.204868989, 2024-09-18T10:15:35.380 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:35.380 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:15:35.380 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:35.380 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.380 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:35.380 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.649564+0000", 2024-09-18T10:15:35.380 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:35.380 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.380 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.380 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:35.381 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.649564+0000", 2024-09-18T10:15:35.381 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:35.381 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.381 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.381 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:35.381 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.649563+0000", 2024-09-18T10:15:35.381 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:35.381 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.381 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.381 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:35.381 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.649600+0000", 2024-09-18T10:15:35.381 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.7224999999999997e-05 2024-09-18T10:15:35.381 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.381 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.382 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:35.382 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.649651+0000", 2024-09-18T10:15:35.382 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.0905000000000001e-05 2024-09-18T10:15:35.382 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.382 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.382 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:35.382 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.649669+0000", 2024-09-18T10:15:35.382 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8522999999999999e-05 2024-09-18T10:15:35.382 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.382 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.382 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:35.382 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.750460+0000", 2024-09-18T10:15:35.382 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100791013 2024-09-18T10:15:35.383 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.383 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.383 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:35.383 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.750487+0000", 2024-09-18T10:15:35.383 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6389000000000001e-05 2024-09-18T10:15:35.383 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.383 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.383 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:35.383 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.854315+0000", 2024-09-18T10:15:35.383 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.103828238 2024-09-18T10:15:35.383 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.383 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.383 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:35.383 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.854433+0000", 2024-09-18T10:15:35.384 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000118449 2024-09-18T10:15:35.384 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:35.384 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:35.384 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:35.384 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.384 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.384 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.19s1 58/40 ECSubWrite(tid=97, reqid=osd.7.0:96, at_version=0'0, trim_to=0'0, roll_forward_to=57'54))", 2024-09-18T10:15:35.384 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:13.672411+0000", 2024-09-18T10:15:35.384 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 21.635740394999999, 2024-09-18T10:15:35.384 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.28289547100000001, 2024-09-18T10:15:35.384 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:35.384 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:15:35.384 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:35.385 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.385 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:35.385 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.672411+0000", 2024-09-18T10:15:35.385 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:35.385 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.385 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.385 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:35.385 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.672411+0000", 2024-09-18T10:15:35.385 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:35.385 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.385 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.385 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:35.385 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.672410+0000", 2024-09-18T10:15:35.385 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:35.386 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.386 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.386 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:35.386 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.672428+0000", 2024-09-18T10:15:35.386 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7116e-05 2024-09-18T10:15:35.386 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.386 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.386 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:35.386 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.672451+0000", 2024-09-18T10:15:35.386 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.3611000000000001e-05 2024-09-18T10:15:35.386 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.386 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.386 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:35.386 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.672465+0000", 2024-09-18T10:15:35.387 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3818999999999999e-05 2024-09-18T10:15:35.387 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.387 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.387 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:35.387 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.853423+0000", 2024-09-18T10:15:35.387 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.180957599 2024-09-18T10:15:35.387 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.387 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.387 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:35.387 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.853438+0000", 2024-09-18T10:15:35.387 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.5446000000000001e-05 2024-09-18T10:15:35.387 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.387 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.387 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:35.388 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.955241+0000", 2024-09-18T10:15:35.388 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.101803053 2024-09-18T10:15:35.388 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.388 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.388 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:35.388 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.955306+0000", 2024-09-18T10:15:35.388 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.5215000000000005e-05 2024-09-18T10:15:35.388 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:35.388 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:35.388 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:35.388 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.388 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.388 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.1s6 58/40 ECSubWrite(tid=120, reqid=osd.4.0:117, at_version=0'0, trim_to=0'0, roll_forward_to=58'22))", 2024-09-18T10:15:35.389 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:14.170897+0000", 2024-09-18T10:15:35.389 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 21.137254265999999, 2024-09-18T10:15:35.389 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20147283599999999, 2024-09-18T10:15:35.389 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:35.389 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:15:35.389 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:35.389 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.389 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:35.389 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:14.170897+0000", 2024-09-18T10:15:35.389 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:35.389 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.389 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.389 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:35.390 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:14.170897+0000", 2024-09-18T10:15:35.390 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:35.390 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.390 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.390 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:35.390 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:14.170895+0000", 2024-09-18T10:15:35.390 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:35.390 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.390 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.390 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:35.390 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:14.170921+0000", 2024-09-18T10:15:35.390 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5848999999999999e-05 2024-09-18T10:15:35.390 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.390 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.391 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:35.391 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:14.170998+0000", 2024-09-18T10:15:35.391 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.7341000000000004e-05 2024-09-18T10:15:35.391 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.391 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.391 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:35.391 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:14.171019+0000", 2024-09-18T10:15:35.391 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.0443000000000001e-05 2024-09-18T10:15:35.391 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.391 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.391 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:35.391 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:14.271283+0000", 2024-09-18T10:15:35.391 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100263848 2024-09-18T10:15:35.392 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.392 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.392 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:35.392 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:14.271316+0000", 2024-09-18T10:15:35.392 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.3516000000000002e-05 2024-09-18T10:15:35.392 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.392 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.392 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:35.392 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:14.372304+0000", 2024-09-18T10:15:35.392 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100987524 2024-09-18T10:15:35.392 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.392 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.392 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:35.392 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:14.372370+0000", 2024-09-18T10:15:35.393 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.6107999999999998e-05 2024-09-18T10:15:35.393 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:35.393 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:35.393 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:35.393 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.393 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.393 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.9s1 63/40 ECSubWrite(tid=118, reqid=client.4632.0:531, at_version=63'56, trim_to=0'0, roll_forward_to=60'52))", 2024-09-18T10:15:35.393 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:19.565882+0000", 2024-09-18T10:15:35.393 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 15.742269322, 2024-09-18T10:15:35.393 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.201669185, 2024-09-18T10:15:35.393 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:35.393 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:15:35.393 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:35.394 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.394 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:35.394 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.565882+0000", 2024-09-18T10:15:35.394 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:35.394 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.394 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.394 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:35.394 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.565882+0000", 2024-09-18T10:15:35.394 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:35.394 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.394 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.394 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:35.394 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.565881+0000", 2024-09-18T10:15:35.394 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:35.395 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.395 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.395 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:35.395 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.565919+0000", 2024-09-18T10:15:35.395 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8254999999999999e-05 2024-09-18T10:15:35.395 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.395 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.395 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:35.395 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.565952+0000", 2024-09-18T10:15:35.395 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.3219000000000002e-05 2024-09-18T10:15:35.395 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.395 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.395 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:35.396 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.565965+0000", 2024-09-18T10:15:35.396 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.272e-05 2024-09-18T10:15:35.396 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.396 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.396 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:35.396 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.766254+0000", 2024-09-18T10:15:35.396 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.200289512 2024-09-18T10:15:35.396 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.396 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.396 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:35.396 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.766272+0000", 2024-09-18T10:15:35.396 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7306e-05 2024-09-18T10:15:35.396 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.397 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.397 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:35.397 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.767501+0000", 2024-09-18T10:15:35.397 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001229794 2024-09-18T10:15:35.397 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.397 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.397 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:35.397 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.767551+0000", 2024-09-18T10:15:35.397 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9662e-05 2024-09-18T10:15:35.397 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:35.397 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:35.397 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:35.397 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.397 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.398 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.4s3 66/40 ECSubWrite(tid=57, reqid=osd.8.0:57, at_version=66'40, trim_to=0'0, roll_forward_to=66'38))", 2024-09-18T10:15:35.398 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:22.543395+0000", 2024-09-18T10:15:35.398 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 12.764756547999999, 2024-09-18T10:15:35.398 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20190205, 2024-09-18T10:15:35.398 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:35.398 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:15:35.398 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:35.398 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.398 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:35.398 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:22.543395+0000", 2024-09-18T10:15:35.398 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:35.398 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.398 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.399 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:35.399 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:22.543395+0000", 2024-09-18T10:15:35.399 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:35.399 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.399 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.399 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:35.399 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:22.543393+0000", 2024-09-18T10:15:35.399 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:35.399 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.399 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.399 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:35.399 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:22.543438+0000", 2024-09-18T10:15:35.399 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.5504999999999999e-05 2024-09-18T10:15:35.399 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.400 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.400 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:35.400 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:22.543471+0000", 2024-09-18T10:15:35.400 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2969000000000002e-05 2024-09-18T10:15:35.400 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.400 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.400 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:35.400 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:22.643568+0000", 2024-09-18T10:15:35.400 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100096327 2024-09-18T10:15:35.400 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.400 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.400 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:35.400 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:22.743796+0000", 2024-09-18T10:15:35.400 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100228206 2024-09-18T10:15:35.401 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.401 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.401 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:35.401 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:22.743818+0000", 2024-09-18T10:15:35.401 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.1778999999999999e-05 2024-09-18T10:15:35.401 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.401 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.401 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:35.401 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:22.745263+0000", 2024-09-18T10:15:35.401 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0014448729999999999 2024-09-18T10:15:35.401 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.401 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.401 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:35.402 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:22.745297+0000", 2024-09-18T10:15:35.402 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.4190999999999998e-05 2024-09-18T10:15:35.402 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:35.402 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:35.402 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:35.402 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.402 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.402 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.9s1 71/40 ECSubWrite(tid=171, reqid=osd.7.0:168, at_version=0'0, trim_to=0'0, roll_forward_to=71'74))", 2024-09-18T10:15:35.402 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:27.693404+0000", 2024-09-18T10:15:35.402 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 7.6147470290000001, 2024-09-18T10:15:35.402 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.196191646, 2024-09-18T10:15:35.402 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:35.402 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:15:35.402 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:35.403 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.403 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:35.403 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:27.693404+0000", 2024-09-18T10:15:35.403 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:35.403 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.403 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.403 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:35.403 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:27.693404+0000", 2024-09-18T10:15:35.403 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:35.403 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.403 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.403 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:35.403 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:27.693402+0000", 2024-09-18T10:15:35.404 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:35.404 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.404 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.404 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:35.404 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:27.693429+0000", 2024-09-18T10:15:35.404 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6733e-05 2024-09-18T10:15:35.404 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.404 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.404 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:35.404 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:27.693471+0000", 2024-09-18T10:15:35.404 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.2024999999999998e-05 2024-09-18T10:15:35.404 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.404 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.404 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:35.405 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:27.693538+0000", 2024-09-18T10:15:35.405 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.7231999999999996e-05 2024-09-18T10:15:35.405 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.405 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.405 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:35.405 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:27.888225+0000", 2024-09-18T10:15:35.405 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.19468701499999999 2024-09-18T10:15:35.405 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.405 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.405 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:35.405 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:27.888252+0000", 2024-09-18T10:15:35.405 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6699999999999998e-05 2024-09-18T10:15:35.405 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.405 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.406 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:35.406 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:27.889524+0000", 2024-09-18T10:15:35.406 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001271933 2024-09-18T10:15:35.406 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.406 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.406 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:35.406 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:27.889596+0000", 2024-09-18T10:15:35.406 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.1967999999999999e-05 2024-09-18T10:15:35.406 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:35.406 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:35.406 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:35.406 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.406 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.407 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.6s3 72/40 ECSubWrite(tid=62, reqid=osd.0.0:62, at_version=72'38, trim_to=0'0, roll_forward_to=71'36))", 2024-09-18T10:15:35.407 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:28.189402+0000", 2024-09-18T10:15:35.407 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 7.1187490059999998, 2024-09-18T10:15:35.407 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.201226193, 2024-09-18T10:15:35.407 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:35.407 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:15:35.407 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:35.407 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.407 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:35.407 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.189402+0000", 2024-09-18T10:15:35.407 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:35.407 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.407 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.407 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:35.408 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.189402+0000", 2024-09-18T10:15:35.408 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:35.408 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.408 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.408 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:35.408 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.189402+0000", 2024-09-18T10:15:35.408 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:35.408 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.408 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.408 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:35.408 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.189461+0000", 2024-09-18T10:15:35.408 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.9077999999999999e-05 2024-09-18T10:15:35.408 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.409 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.409 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:35.409 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.189484+0000", 2024-09-18T10:15:35.409 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.3353000000000002e-05 2024-09-18T10:15:35.409 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.409 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.409 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:35.409 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.289578+0000", 2024-09-18T10:15:35.409 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100094161 2024-09-18T10:15:35.409 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.409 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.409 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:35.409 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.389741+0000", 2024-09-18T10:15:35.410 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100162923 2024-09-18T10:15:35.410 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.410 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.410 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:35.410 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.389749+0000", 2024-09-18T10:15:35.410 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.2889999999999998e-06 2024-09-18T10:15:35.410 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.410 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.410 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:35.410 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.390593+0000", 2024-09-18T10:15:35.410 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00084384499999999997 2024-09-18T10:15:35.410 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.410 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.410 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:35.411 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.390628+0000", 2024-09-18T10:15:35.411 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5120999999999998e-05 2024-09-18T10:15:35.411 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:35.411 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:35.411 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:35.411 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.411 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.411 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.6s3 74/40 ECSubWrite(tid=64, reqid=osd.0.0:64, at_version=74'40, trim_to=0'0, roll_forward_to=72'38))", 2024-09-18T10:15:35.411 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:29.762852+0000", 2024-09-18T10:15:35.411 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 5.5452988840000002, 2024-09-18T10:15:35.411 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.202373787, 2024-09-18T10:15:35.411 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:35.411 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:15:35.411 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:35.412 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.412 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:35.412 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.762852+0000", 2024-09-18T10:15:35.412 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:35.412 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.412 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.412 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:35.412 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.762852+0000", 2024-09-18T10:15:35.413 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:35.413 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.413 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.413 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:35.413 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.762852+0000", 2024-09-18T10:15:35.413 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:35.413 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.413 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.413 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:35.413 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.762877+0000", 2024-09-18T10:15:35.413 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4759000000000001e-05 2024-09-18T10:15:35.414 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.414 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.414 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:35.414 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.762885+0000", 2024-09-18T10:15:35.414 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.7549999999999994e-06 2024-09-18T10:15:35.414 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.414 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.414 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:35.414 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.762900+0000", 2024-09-18T10:15:35.414 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.5364999999999999e-05 2024-09-18T10:15:35.414 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.414 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.414 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:35.415 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.963772+0000", 2024-09-18T10:15:35.415 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20087190899999999 2024-09-18T10:15:35.415 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.415 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.415 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:35.415 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.963781+0000", 2024-09-18T10:15:35.415 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.516e-06 2024-09-18T10:15:35.415 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.415 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.415 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:35.415 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.965178+0000", 2024-09-18T10:15:35.415 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0013961959999999999 2024-09-18T10:15:35.415 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.415 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.416 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:35.416 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.965226+0000", 2024-09-18T10:15:35.416 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.8578000000000001e-05 2024-09-18T10:15:35.416 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:35.416 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:35.416 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:35.416 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.416 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.416 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.16s4 74/62 ECSubWrite(tid=8, reqid=osd.9.0:8, at_version=74'22, trim_to=0'0, roll_forward_to=71'20))", 2024-09-18T10:15:35.416 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:29.768136+0000", 2024-09-18T10:15:35.417 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 5.5400148629999997, 2024-09-18T10:15:35.417 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.19712143500000001, 2024-09-18T10:15:35.417 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:35.417 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:15:35.417 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:35.417 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.417 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:35.417 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.768136+0000", 2024-09-18T10:15:35.417 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:35.417 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.417 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.417 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:35.417 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.768136+0000", 2024-09-18T10:15:35.418 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:35.418 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.418 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.418 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:35.418 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.768135+0000", 2024-09-18T10:15:35.418 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:15:35.418 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.418 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.418 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:35.418 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.768183+0000", 2024-09-18T10:15:35.418 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.7342000000000002e-05 2024-09-18T10:15:35.418 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.418 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.418 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:35.419 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.768212+0000", 2024-09-18T10:15:35.419 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8705e-05 2024-09-18T10:15:35.419 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.419 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.419 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:35.419 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.768227+0000", 2024-09-18T10:15:35.419 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.5874000000000001e-05 2024-09-18T10:15:35.419 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.419 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.419 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:35.419 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.963760+0000", 2024-09-18T10:15:35.419 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.195532979 2024-09-18T10:15:35.419 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.420 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.420 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:35.420 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.963776+0000", 2024-09-18T10:15:35.420 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.5285999999999999e-05 2024-09-18T10:15:35.420 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.420 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.420 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:35.420 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.965236+0000", 2024-09-18T10:15:35.420 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001459976 2024-09-18T10:15:35.420 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.420 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.420 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:35.420 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.965258+0000", 2024-09-18T10:15:35.421 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.2133e-05 2024-09-18T10:15:35.421 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:35.421 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:35.421 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:35.421 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:35.421 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:35.421 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-09-18T10:15:35.423 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T10:15:35.423 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-09-18T10:15:35.418+0000 7fdfa8698700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22087) UID: 0 2024-09-18T10:15:35.524 INFO:tasks.ceph.osd.4.smithi029.stderr:2024-09-18T10:15:35.517+0000 7f550cf6e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22189) UID: 0 2024-09-18T10:15:35.569 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-09-18T10:15:35.569 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-09-18T10:15:35.569 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-09-18T10:15:35.569 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-09-18T10:15:35.580 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T10:15:35.581 INFO:tasks.rados.rados.0.smithi189.stdout:281: done (0 left) 2024-09-18T10:15:35.581 INFO:tasks.rados.rados.0.smithi189.stdout:282: read oid 15 snap -1 2024-09-18T10:15:35.581 INFO:tasks.rados.rados.0.smithi189.stdout:282: expect (ObjNum 95 snap 17 seq_num 95) 2024-09-18T10:15:35.581 INFO:tasks.rados.rados.0.smithi189.stdout:283: read oid 1 snap -1 2024-09-18T10:15:35.581 INFO:tasks.rados.rados.0.smithi189.stdout:283: expect (ObjNum 3 snap 0 seq_num 3) 2024-09-18T10:15:35.581 INFO:tasks.rados.rados.0.smithi189.stdout:284: read oid 10 snap -1 2024-09-18T10:15:35.581 INFO:tasks.rados.rados.0.smithi189.stdout:284: expect (ObjNum 68 snap 6 seq_num 68) 2024-09-18T10:15:35.581 INFO:tasks.rados.rados.0.smithi189.stdout:285: delete oid 49 current snap is 25 2024-09-18T10:15:35.625 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-18T10:15:35.620+0000 7f578c52f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22134) UID: 0 2024-09-18T10:15:35.678 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-09-18T10:15:35.678 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-09-18T10:15:35.678 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-09-18T10:15:35.678 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-09-18T10:15:35.678 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-09-18T10:15:35.688 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T10:15:35.726 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-18T10:15:35.724+0000 7f578c52f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22134) UID: 0 2024-09-18T10:15:35.788 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-09-18T10:15:35.788 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-09-18T10:15:35.788 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-09-18T10:15:35.788 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-09-18T10:15:35.789 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.789 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T10:15:35.789 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:36.786936+0000", 2024-09-18T10:15:35.789 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 58.996582852000003, 2024-09-18T10:15:35.789 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.71130157999999999, 2024-09-18T10:15:35.789 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:35.789 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:35.789 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:35.789 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:15:35.789 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:15:35.789 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 72 2024-09-18T10:15:35.789 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.789 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:35.789 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.790 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:35.790 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.786936+0000", 2024-09-18T10:15:35.790 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:35.790 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.790 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.790 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:35.790 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.786936+0000", 2024-09-18T10:15:35.790 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:35.790 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.790 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.790 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:35.790 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.786935+0000", 2024-09-18T10:15:35.790 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:35.790 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.791 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.791 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:35.791 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787018+0000", 2024-09-18T10:15:35.791 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.2511999999999999e-05 2024-09-18T10:15:35.791 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.791 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.791 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:35.791 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787028+0000", 2024-09-18T10:15:35.791 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0709e-05 2024-09-18T10:15:35.791 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.791 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.791 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:35.791 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787034+0000", 2024-09-18T10:15:35.792 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.7769999999999999e-06 2024-09-18T10:15:35.792 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.792 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.792 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:35.792 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.295095+0000", 2024-09-18T10:15:35.792 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.50806109099999996 2024-09-18T10:15:35.792 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.792 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.792 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:35.792 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.295221+0000", 2024-09-18T10:15:35.792 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00012563599999999999 2024-09-18T10:15:35.792 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.792 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.792 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:15:35.793 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.295453+0000", 2024-09-18T10:15:35.793 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000232387 2024-09-18T10:15:35.793 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.793 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.793 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:15:35.793 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.301373+0000", 2024-09-18T10:15:35.794 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0059202279999999996 2024-09-18T10:15:35.794 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.794 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.794 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:15:35.794 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.498044+0000", 2024-09-18T10:15:35.794 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.19667104699999999 2024-09-18T10:15:35.794 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.794 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.794 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:15:35.794 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.498133+0000", 2024-09-18T10:15:35.794 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.9044999999999999e-05 2024-09-18T10:15:35.794 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.794 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.795 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:35.795 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.498237+0000", 2024-09-18T10:15:35.795 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00010385100000000001 2024-09-18T10:15:35.795 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:35.795 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:35.795 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:35.795 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.795 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.795 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T10:15:35.795 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:36.787042+0000", 2024-09-18T10:15:35.795 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 58.996476145000003, 2024-09-18T10:15:35.795 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.71152325100000002, 2024-09-18T10:15:35.795 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:35.795 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:35.796 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:35.796 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:15:35.796 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:15:35.796 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 73 2024-09-18T10:15:35.796 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.796 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:35.796 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.796 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:35.796 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787042+0000", 2024-09-18T10:15:35.796 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:35.796 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.796 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.796 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:35.796 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787042+0000", 2024-09-18T10:15:35.797 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:35.797 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.797 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.797 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:35.797 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787042+0000", 2024-09-18T10:15:35.797 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:15:35.797 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.797 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.797 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:35.797 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787048+0000", 2024-09-18T10:15:35.797 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.8900000000000001e-06 2024-09-18T10:15:35.797 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.797 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.797 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:35.798 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787057+0000", 2024-09-18T10:15:35.798 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.4430000000000003e-06 2024-09-18T10:15:35.798 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.798 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.798 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:35.798 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787063+0000", 2024-09-18T10:15:35.798 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.6459999999999998e-06 2024-09-18T10:15:35.798 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.798 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.798 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:35.798 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.297548+0000", 2024-09-18T10:15:35.798 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.51048589499999997 2024-09-18T10:15:35.798 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.798 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.799 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:35.799 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.297668+0000", 2024-09-18T10:15:35.799 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00011917 2024-09-18T10:15:35.799 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.799 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.799 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:15:35.799 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.297891+0000", 2024-09-18T10:15:35.799 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00022301999999999999 2024-09-18T10:15:35.799 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.799 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.799 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:15:35.799 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.301383+0000", 2024-09-18T10:15:35.799 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.003492762 2024-09-18T10:15:35.799 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.800 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.800 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:15:35.800 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.498374+0000", 2024-09-18T10:15:35.800 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.19699053899999999 2024-09-18T10:15:35.800 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.800 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.800 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:15:35.800 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.498506+0000", 2024-09-18T10:15:35.800 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00013246899999999999 2024-09-18T10:15:35.800 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.800 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.800 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:35.800 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.498566+0000", 2024-09-18T10:15:35.800 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.9274000000000002e-05 2024-09-18T10:15:35.801 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:35.801 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:35.801 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:35.801 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.801 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.801 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T10:15:35.801 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:36.787128+0000", 2024-09-18T10:15:35.801 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 58.996390837, 2024-09-18T10:15:35.801 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.71169541800000002, 2024-09-18T10:15:35.801 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:35.801 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:35.801 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:35.801 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:15:35.801 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:15:35.802 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 74 2024-09-18T10:15:35.802 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.802 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:35.802 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.802 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:35.802 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787128+0000", 2024-09-18T10:15:35.802 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:35.802 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.802 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.802 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:35.802 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787128+0000", 2024-09-18T10:15:35.802 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:35.802 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.802 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.803 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:35.803 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787127+0000", 2024-09-18T10:15:35.803 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:15:35.803 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.803 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.803 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:35.803 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787133+0000", 2024-09-18T10:15:35.803 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.6930000000000004e-06 2024-09-18T10:15:35.803 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.803 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.803 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:35.803 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787142+0000", 2024-09-18T10:15:35.803 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.1959999999999997e-06 2024-09-18T10:15:35.803 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.804 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.804 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:35.804 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787150+0000", 2024-09-18T10:15:35.804 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.6399999999999997e-06 2024-09-18T10:15:35.804 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.804 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.804 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:35.804 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.298925+0000", 2024-09-18T10:15:35.804 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.51177523599999997 2024-09-18T10:15:35.804 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.804 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.804 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:35.804 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.299064+0000", 2024-09-18T10:15:35.805 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000139622 2024-09-18T10:15:35.805 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.805 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.805 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:15:35.805 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.299287+0000", 2024-09-18T10:15:35.805 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00022309700000000001 2024-09-18T10:15:35.805 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.805 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.805 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:15:35.805 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.301389+0000", 2024-09-18T10:15:35.805 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0021018759999999999 2024-09-18T10:15:35.805 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.805 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.805 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:15:35.805 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.498648+0000", 2024-09-18T10:15:35.806 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.19725873999999999 2024-09-18T10:15:35.806 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.806 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.806 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:15:35.806 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.498731+0000", 2024-09-18T10:15:35.806 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.2429000000000006e-05 2024-09-18T10:15:35.806 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.806 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.807 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:35.807 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.498823+0000", 2024-09-18T10:15:35.807 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.2655e-05 2024-09-18T10:15:35.807 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:35.807 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:35.807 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:35.807 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.807 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.807 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T10:15:35.807 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:36.787157+0000", 2024-09-18T10:15:35.808 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 58.996361612000001, 2024-09-18T10:15:35.808 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.81204889999999996, 2024-09-18T10:15:35.808 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:35.808 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:35.808 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:35.808 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:15:35.808 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:15:35.808 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 75 2024-09-18T10:15:35.808 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.808 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:35.808 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.808 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:35.808 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787157+0000", 2024-09-18T10:15:35.809 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:35.809 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.809 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.809 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:35.809 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787157+0000", 2024-09-18T10:15:35.809 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:35.809 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.809 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.809 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:35.809 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787156+0000", 2024-09-18T10:15:35.809 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:15:35.809 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.809 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.809 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:35.810 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787236+0000", 2024-09-18T10:15:35.810 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.0175999999999999e-05 2024-09-18T10:15:35.810 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.810 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.810 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:35.810 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787247+0000", 2024-09-18T10:15:35.810 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0448999999999999e-05 2024-09-18T10:15:35.810 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.810 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.810 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:35.810 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787253+0000", 2024-09-18T10:15:35.810 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.2469999999999997e-06 2024-09-18T10:15:35.810 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.811 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.811 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:35.811 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.299974+0000", 2024-09-18T10:15:35.811 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.51272114499999999 2024-09-18T10:15:35.811 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.811 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.811 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:35.811 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.300115+0000", 2024-09-18T10:15:35.811 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000141452 2024-09-18T10:15:35.811 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.811 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.811 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:15:35.811 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.300349+0000", 2024-09-18T10:15:35.811 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00023363999999999999 2024-09-18T10:15:35.812 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.812 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.812 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:15:35.812 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.301947+0000", 2024-09-18T10:15:35.812 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0015976510000000001 2024-09-18T10:15:35.812 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.812 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.812 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:15:35.812 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.599027+0000", 2024-09-18T10:15:35.812 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.29708051600000002 2024-09-18T10:15:35.812 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.812 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.812 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:15:35.813 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.599106+0000", 2024-09-18T10:15:35.813 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.9033e-05 2024-09-18T10:15:35.813 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.813 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.813 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:35.813 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.599206+0000", 2024-09-18T10:15:35.813 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.9561000000000004e-05 2024-09-18T10:15:35.813 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:35.813 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:35.813 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:35.813 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.813 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.813 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T10:15:35.814 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:36.787261+0000", 2024-09-18T10:15:35.814 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 58.996257960999998, 2024-09-18T10:15:35.814 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.81223649499999995, 2024-09-18T10:15:35.814 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:35.814 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:35.814 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:35.814 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:15:35.814 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:15:35.814 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 76 2024-09-18T10:15:35.814 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.814 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:35.814 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.814 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:35.814 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787261+0000", 2024-09-18T10:15:35.815 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:35.815 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.815 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.815 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:35.815 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787261+0000", 2024-09-18T10:15:35.815 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:35.815 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.815 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.815 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:35.815 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787260+0000", 2024-09-18T10:15:35.815 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:15:35.815 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.815 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.816 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:35.816 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787265+0000", 2024-09-18T10:15:35.816 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1590000000000001e-06 2024-09-18T10:15:35.816 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.816 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.816 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:35.816 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787273+0000", 2024-09-18T10:15:35.816 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.4100000000000008e-06 2024-09-18T10:15:35.816 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.816 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.816 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:35.816 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787279+0000", 2024-09-18T10:15:35.816 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.3630000000000002e-06 2024-09-18T10:15:35.816 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.817 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.817 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:35.817 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.496955+0000", 2024-09-18T10:15:35.817 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.70967585099999997 2024-09-18T10:15:35.817 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.817 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.817 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:35.817 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.497107+0000", 2024-09-18T10:15:35.817 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000152044 2024-09-18T10:15:35.817 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.817 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.817 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:15:35.818 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.497345+0000", 2024-09-18T10:15:35.818 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000238693 2024-09-18T10:15:35.818 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.818 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.818 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:15:35.818 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.599248+0000", 2024-09-18T10:15:35.818 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.101902836 2024-09-18T10:15:35.818 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.818 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.818 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:15:35.818 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.599329+0000", 2024-09-18T10:15:35.819 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.1064999999999997e-05 2024-09-18T10:15:35.819 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.819 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.819 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:15:35.819 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.599441+0000", 2024-09-18T10:15:35.819 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000111797 2024-09-18T10:15:35.819 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.819 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.819 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:35.819 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.599497+0000", 2024-09-18T10:15:35.819 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.6119999999999998e-05 2024-09-18T10:15:35.819 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:35.819 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:35.819 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:35.820 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.820 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.820 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:93 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T10:15:35.820 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:38.616472+0000", 2024-09-18T10:15:35.820 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 57.167046677999998, 2024-09-18T10:15:35.820 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.60940747100000003, 2024-09-18T10:15:35.820 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:35.820 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:35.820 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:35.820 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:15:35.820 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:15:35.820 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 93 2024-09-18T10:15:35.820 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.821 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:35.821 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.821 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:35.821 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.616472+0000", 2024-09-18T10:15:35.821 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:35.821 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.821 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.821 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:35.821 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.616472+0000", 2024-09-18T10:15:35.821 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:35.821 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.821 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.821 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:35.821 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.616470+0000", 2024-09-18T10:15:35.822 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:35.822 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.822 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.822 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:35.822 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.616644+0000", 2024-09-18T10:15:35.822 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00017372000000000001 2024-09-18T10:15:35.822 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.822 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.822 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:35.822 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.616695+0000", 2024-09-18T10:15:35.822 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.0991e-05 2024-09-18T10:15:35.822 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.822 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.822 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:35.823 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.616709+0000", 2024-09-18T10:15:35.823 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4008999999999999e-05 2024-09-18T10:15:35.823 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.823 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.823 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:35.823 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.720960+0000", 2024-09-18T10:15:35.823 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.104250624 2024-09-18T10:15:35.823 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.823 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.823 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:35.823 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.721068+0000", 2024-09-18T10:15:35.823 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00010814700000000001 2024-09-18T10:15:35.823 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.824 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.824 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:15:35.824 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.721270+0000", 2024-09-18T10:15:35.824 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00020184200000000001 2024-09-18T10:15:35.824 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.824 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.824 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:15:35.824 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.024691+0000", 2024-09-18T10:15:35.824 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.30342105600000002 2024-09-18T10:15:35.824 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.824 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.824 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:15:35.824 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.225725+0000", 2024-09-18T10:15:35.824 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20103395099999999 2024-09-18T10:15:35.825 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.825 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.825 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:15:35.825 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.225816+0000", 2024-09-18T10:15:35.825 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.1928999999999993e-05 2024-09-18T10:15:35.825 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.825 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.825 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:35.825 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.225879+0000", 2024-09-18T10:15:35.825 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.2918999999999998e-05 2024-09-18T10:15:35.825 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:35.825 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:35.825 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:35.826 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.826 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.826 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:94 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T10:15:35.826 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:38.616729+0000", 2024-09-18T10:15:35.826 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 57.166789653000002, 2024-09-18T10:15:35.826 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.60943170999999996, 2024-09-18T10:15:35.826 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:35.826 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:35.826 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:35.826 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:15:35.826 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:15:35.826 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 94 2024-09-18T10:15:35.826 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.826 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:35.827 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.827 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:35.827 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.616729+0000", 2024-09-18T10:15:35.827 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:35.827 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.827 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.827 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:35.827 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.616729+0000", 2024-09-18T10:15:35.827 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:35.827 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.827 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.827 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:35.827 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.616727+0000", 2024-09-18T10:15:35.827 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:35.828 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.828 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.828 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:35.828 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.616738+0000", 2024-09-18T10:15:35.828 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0776e-05 2024-09-18T10:15:35.828 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.828 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.828 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:35.828 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.616756+0000", 2024-09-18T10:15:35.828 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8079000000000001e-05 2024-09-18T10:15:35.828 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.828 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.829 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:35.829 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.716882+0000", 2024-09-18T10:15:35.829 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10012612999999999 2024-09-18T10:15:35.829 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.829 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.829 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:35.829 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.822315+0000", 2024-09-18T10:15:35.829 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.105433162 2024-09-18T10:15:35.829 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.829 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.829 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:35.829 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.822453+0000", 2024-09-18T10:15:35.829 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000137396 2024-09-18T10:15:35.829 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.830 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.830 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:15:35.830 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.822674+0000", 2024-09-18T10:15:35.830 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00022154400000000001 2024-09-18T10:15:35.830 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.830 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.830 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:15:35.830 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.024888+0000", 2024-09-18T10:15:35.830 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20221397199999999 2024-09-18T10:15:35.830 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.830 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.830 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:15:35.830 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.226011+0000", 2024-09-18T10:15:35.830 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20112316299999999 2024-09-18T10:15:35.831 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.831 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.831 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:15:35.831 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.226094+0000", 2024-09-18T10:15:35.831 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.2155000000000002e-05 2024-09-18T10:15:35.831 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.831 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.831 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:35.831 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.226161+0000", 2024-09-18T10:15:35.831 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.7138999999999995e-05 2024-09-18T10:15:35.831 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:35.831 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:35.831 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:35.831 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.832 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.832 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:95 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T10:15:35.832 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:38.716909+0000", 2024-09-18T10:15:35.832 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 57.066609939000003, 2024-09-18T10:15:35.832 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.60960763600000001, 2024-09-18T10:15:35.832 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:35.832 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:35.832 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:35.832 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:15:35.832 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:15:35.832 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 95 2024-09-18T10:15:35.832 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.832 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:35.833 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.833 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:35.833 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.716909+0000", 2024-09-18T10:15:35.833 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:35.833 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.833 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.833 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:35.833 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.716909+0000", 2024-09-18T10:15:35.833 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:35.833 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.833 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.833 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:35.833 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.716907+0000", 2024-09-18T10:15:35.833 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:35.834 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.834 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.834 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:35.834 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.716921+0000", 2024-09-18T10:15:35.834 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4572e-05 2024-09-18T10:15:35.834 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.834 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.834 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:35.834 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.716939+0000", 2024-09-18T10:15:35.834 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7149000000000001e-05 2024-09-18T10:15:35.834 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.834 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.834 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:35.834 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.716948+0000", 2024-09-18T10:15:35.835 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.7000000000000003e-06 2024-09-18T10:15:35.835 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.835 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.835 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:35.835 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.923430+0000", 2024-09-18T10:15:35.835 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20648133299999999 2024-09-18T10:15:35.835 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.835 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.835 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:35.835 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.923566+0000", 2024-09-18T10:15:35.835 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00013657599999999999 2024-09-18T10:15:35.835 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.835 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.835 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:15:35.836 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.923782+0000", 2024-09-18T10:15:35.836 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00021599400000000001 2024-09-18T10:15:35.836 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.836 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.836 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:15:35.836 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.024899+0000", 2024-09-18T10:15:35.836 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10111666800000001 2024-09-18T10:15:35.836 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.836 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.836 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:15:35.836 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.326379+0000", 2024-09-18T10:15:35.836 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.30148048799999999 2024-09-18T10:15:35.836 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.836 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.837 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:15:35.837 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.326454+0000", 2024-09-18T10:15:35.837 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.4988999999999996e-05 2024-09-18T10:15:35.837 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.837 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.837 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:35.837 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.326516+0000", 2024-09-18T10:15:35.837 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.1902999999999993e-05 2024-09-18T10:15:35.837 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:35.837 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:35.837 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:35.838 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.838 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.838 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:96 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T10:15:35.838 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:38.716972+0000", 2024-09-18T10:15:35.838 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 57.066546948999999, 2024-09-18T10:15:35.838 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.91539460800000005, 2024-09-18T10:15:35.838 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:35.838 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:35.838 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:35.838 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:15:35.838 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:15:35.838 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 96 2024-09-18T10:15:35.838 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.838 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:35.839 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.839 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:35.839 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.716972+0000", 2024-09-18T10:15:35.839 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:35.839 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.839 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.839 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:35.839 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.716972+0000", 2024-09-18T10:15:35.839 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:35.839 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.839 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.839 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:35.839 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.716971+0000", 2024-09-18T10:15:35.839 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:15:35.840 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.840 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.840 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:35.840 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717061+0000", 2024-09-18T10:15:35.840 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.0744999999999999e-05 2024-09-18T10:15:35.840 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.840 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.840 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:35.840 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717073+0000", 2024-09-18T10:15:35.840 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1943000000000001e-05 2024-09-18T10:15:35.840 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.840 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.840 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:35.841 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717080+0000", 2024-09-18T10:15:35.841 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0060000000000003e-06 2024-09-18T10:15:35.841 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.841 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.841 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:35.841 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.426708+0000", 2024-09-18T10:15:35.841 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.70962730399999996 2024-09-18T10:15:35.841 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.841 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.841 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:35.841 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.426854+0000", 2024-09-18T10:15:35.841 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00014654199999999999 2024-09-18T10:15:35.841 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.841 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.842 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:15:35.842 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.427106+0000", 2024-09-18T10:15:35.842 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00025226600000000001 2024-09-18T10:15:35.842 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.842 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.842 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:15:35.842 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.433087+0000", 2024-09-18T10:15:35.842 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0059804860000000001 2024-09-18T10:15:35.842 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.842 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.842 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:15:35.842 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.632096+0000", 2024-09-18T10:15:35.842 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.19900922099999999 2024-09-18T10:15:35.842 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.843 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.843 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:15:35.843 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.632257+0000", 2024-09-18T10:15:35.843 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00016058399999999999 2024-09-18T10:15:35.843 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.843 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.843 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:35.843 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.632366+0000", 2024-09-18T10:15:35.843 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000109601 2024-09-18T10:15:35.843 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:35.843 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:35.843 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:35.843 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.843 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.843 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:97 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T10:15:35.844 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:38.717093+0000", 2024-09-18T10:15:35.844 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 57.066425139000003, 2024-09-18T10:15:35.844 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.91561460900000002, 2024-09-18T10:15:35.844 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:35.844 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:35.844 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:35.844 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:15:35.844 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:15:35.844 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 97 2024-09-18T10:15:35.844 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.844 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:35.844 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.844 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:35.844 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717093+0000", 2024-09-18T10:15:35.845 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:35.845 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.845 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.845 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:35.845 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717093+0000", 2024-09-18T10:15:35.845 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:35.845 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.845 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.845 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:35.845 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717092+0000", 2024-09-18T10:15:35.845 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:15:35.845 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.845 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.846 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:35.846 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717099+0000", 2024-09-18T10:15:35.846 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.246e-06 2024-09-18T10:15:35.846 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.846 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.846 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:35.846 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717109+0000", 2024-09-18T10:15:35.846 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0499999999999999e-05 2024-09-18T10:15:35.846 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.846 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.846 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:35.846 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717115+0000", 2024-09-18T10:15:35.846 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.7690000000000002e-06 2024-09-18T10:15:35.846 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.847 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.847 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:35.847 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.428509+0000", 2024-09-18T10:15:35.847 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.71139444299999999 2024-09-18T10:15:35.847 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.847 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.847 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:35.847 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.428621+0000", 2024-09-18T10:15:35.847 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00011156 2024-09-18T10:15:35.847 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.848 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.848 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:15:35.848 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.428805+0000", 2024-09-18T10:15:35.848 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000183597 2024-09-18T10:15:35.848 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.848 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.848 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:15:35.848 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.433106+0000", 2024-09-18T10:15:35.848 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0043015019999999996 2024-09-18T10:15:35.848 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.848 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.848 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:15:35.848 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.632474+0000", 2024-09-18T10:15:35.849 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.199368092 2024-09-18T10:15:35.849 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.849 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.849 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:15:35.849 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.632605+0000", 2024-09-18T10:15:35.849 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00013110899999999999 2024-09-18T10:15:35.849 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.849 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.849 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:35.849 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.632708+0000", 2024-09-18T10:15:35.849 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000102748 2024-09-18T10:15:35.849 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:35.849 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:35.850 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:35.850 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.850 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.850 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:98 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T10:15:35.850 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:38.717126+0000", 2024-09-18T10:15:35.850 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 57.066392876999998, 2024-09-18T10:15:35.850 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.015084141, 2024-09-18T10:15:35.850 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:35.850 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:35.850 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:35.850 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:15:35.850 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:15:35.850 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 98 2024-09-18T10:15:35.850 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.851 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:35.851 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.851 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:35.851 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717126+0000", 2024-09-18T10:15:35.851 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:35.851 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.851 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.851 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:35.851 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717126+0000", 2024-09-18T10:15:35.851 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:35.851 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.851 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.851 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:35.852 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717125+0000", 2024-09-18T10:15:35.852 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:15:35.852 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.852 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.852 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:35.852 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717130+0000", 2024-09-18T10:15:35.852 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.4160000000000003e-06 2024-09-18T10:15:35.852 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.852 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.852 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:35.852 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717139+0000", 2024-09-18T10:15:35.852 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.6230000000000003e-06 2024-09-18T10:15:35.852 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.853 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.853 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:35.853 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717144+0000", 2024-09-18T10:15:35.853 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5330000000000002e-06 2024-09-18T10:15:35.853 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.853 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.853 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:35.853 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.429491+0000", 2024-09-18T10:15:35.853 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.71234629500000002 2024-09-18T10:15:35.853 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.853 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.853 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:35.853 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.429741+0000", 2024-09-18T10:15:35.853 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000250516 2024-09-18T10:15:35.854 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.854 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.854 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:15:35.854 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.429989+0000", 2024-09-18T10:15:35.854 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00024811599999999999 2024-09-18T10:15:35.854 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.854 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.854 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:15:35.854 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.435056+0000", 2024-09-18T10:15:35.854 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0050669670000000003 2024-09-18T10:15:35.854 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.854 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.854 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:15:35.854 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.732068+0000", 2024-09-18T10:15:35.855 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.29701180900000002 2024-09-18T10:15:35.855 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.855 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.855 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:15:35.855 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.732145+0000", 2024-09-18T10:15:35.855 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.7152999999999998e-05 2024-09-18T10:15:35.855 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.855 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.855 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:35.855 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.732210+0000", 2024-09-18T10:15:35.855 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.4503000000000002e-05 2024-09-18T10:15:35.855 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:35.855 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:35.855 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:35.856 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.856 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.856 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:99 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T10:15:35.856 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:38.717230+0000", 2024-09-18T10:15:35.856 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 57.066288317000001, 2024-09-18T10:15:35.856 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0157905819999999, 2024-09-18T10:15:35.856 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:35.856 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:35.856 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:35.856 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:15:35.856 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:15:35.856 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 99 2024-09-18T10:15:35.856 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.857 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:35.857 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.857 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:35.857 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717230+0000", 2024-09-18T10:15:35.857 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:35.857 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.857 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.857 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:35.857 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717230+0000", 2024-09-18T10:15:35.857 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:35.857 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.857 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.857 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:35.857 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717229+0000", 2024-09-18T10:15:35.858 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:15:35.858 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.858 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.858 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:35.858 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717318+0000", 2024-09-18T10:15:35.858 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.8197999999999998e-05 2024-09-18T10:15:35.858 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.858 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.858 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:35.858 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717342+0000", 2024-09-18T10:15:35.858 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4445000000000001e-05 2024-09-18T10:15:35.858 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.858 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.858 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:35.858 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717360+0000", 2024-09-18T10:15:35.859 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7782999999999999e-05 2024-09-18T10:15:35.859 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.859 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.859 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:35.859 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.430650+0000", 2024-09-18T10:15:35.859 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.71329020200000004 2024-09-18T10:15:35.859 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.859 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.859 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:35.859 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.430805+0000", 2024-09-18T10:15:35.859 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00015474900000000001 2024-09-18T10:15:35.859 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.859 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.859 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:15:35.860 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.431021+0000", 2024-09-18T10:15:35.860 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000215911 2024-09-18T10:15:35.860 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.860 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.860 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:15:35.860 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.435099+0000", 2024-09-18T10:15:35.860 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0040787439999999996 2024-09-18T10:15:35.860 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.860 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.860 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:15:35.860 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.732758+0000", 2024-09-18T10:15:35.860 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.297658637 2024-09-18T10:15:35.860 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.860 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.861 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:15:35.861 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.732909+0000", 2024-09-18T10:15:35.861 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00015069899999999999 2024-09-18T10:15:35.861 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.861 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.861 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:35.861 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.733021+0000", 2024-09-18T10:15:35.861 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00011207 2024-09-18T10:15:35.861 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:35.861 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:35.861 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:35.861 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.861 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.862 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:100 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T10:15:35.862 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:38.717370+0000", 2024-09-18T10:15:35.862 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 57.066148755, 2024-09-18T10:15:35.862 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1158324850000001, 2024-09-18T10:15:35.862 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:35.862 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:35.862 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:35.862 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:15:35.862 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:15:35.862 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 100 2024-09-18T10:15:35.862 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.862 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:35.862 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.863 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:35.863 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717370+0000", 2024-09-18T10:15:35.863 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:35.863 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.863 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.863 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:35.863 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717370+0000", 2024-09-18T10:15:35.863 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:35.863 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.863 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.863 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:35.863 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717369+0000", 2024-09-18T10:15:35.863 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:15:35.863 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.864 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.864 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:35.864 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717378+0000", 2024-09-18T10:15:35.864 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.3559999999999999e-06 2024-09-18T10:15:35.864 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.864 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.864 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:35.864 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717398+0000", 2024-09-18T10:15:35.864 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.9962e-05 2024-09-18T10:15:35.864 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.864 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.864 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:35.864 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717407+0000", 2024-09-18T10:15:35.865 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.2699999999999993e-06 2024-09-18T10:15:35.865 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.865 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.865 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:35.865 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.432184+0000", 2024-09-18T10:15:35.865 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.71477630299999995 2024-09-18T10:15:35.865 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.865 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.865 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:35.865 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.432316+0000", 2024-09-18T10:15:35.865 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000132082 2024-09-18T10:15:35.865 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.865 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.865 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:15:35.866 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.432526+0000", 2024-09-18T10:15:35.866 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000210659 2024-09-18T10:15:35.866 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.866 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.866 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:15:35.866 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.435651+0000", 2024-09-18T10:15:35.866 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.003124169 2024-09-18T10:15:35.866 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.866 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.866 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:15:35.866 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.832931+0000", 2024-09-18T10:15:35.866 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.39728084899999999 2024-09-18T10:15:35.866 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.866 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.867 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:15:35.867 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.833058+0000", 2024-09-18T10:15:35.867 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000126969 2024-09-18T10:15:35.867 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.867 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.867 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:35.867 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.833202+0000", 2024-09-18T10:15:35.867 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000143922 2024-09-18T10:15:35.867 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:35.867 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:35.867 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:35.867 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.867 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.867 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:141 3.8s0 3:1c02e63c:::smithi18925896-47:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:35.868 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:51.119430+0000", 2024-09-18T10:15:35.868 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 44.664088550999999, 2024-09-18T10:15:35.868 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.60306919000000003, 2024-09-18T10:15:35.868 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:35.868 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "started", 2024-09-18T10:15:35.868 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:35.868 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:15:35.868 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:35.868 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 141 2024-09-18T10:15:35.868 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.868 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:35.868 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.868 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:35.869 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.119430+0000", 2024-09-18T10:15:35.869 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:35.869 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.869 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.869 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:35.869 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.119430+0000", 2024-09-18T10:15:35.869 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:35.869 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.869 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.869 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:35.869 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.119430+0000", 2024-09-18T10:15:35.869 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:35.869 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.870 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.870 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:35.870 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.119432+0000", 2024-09-18T10:15:35.870 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6240000000000002e-06 2024-09-18T10:15:35.870 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.870 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.870 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:35.870 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.119439+0000", 2024-09-18T10:15:35.870 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.8329999999999996e-06 2024-09-18T10:15:35.870 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.870 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.870 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:35.870 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.119444+0000", 2024-09-18T10:15:35.871 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.485e-06 2024-09-18T10:15:35.871 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.871 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.871 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:35.871 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.620626+0000", 2024-09-18T10:15:35.871 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.50118172299999997 2024-09-18T10:15:35.871 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.871 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.871 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:35.871 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.620781+0000", 2024-09-18T10:15:35.871 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00015495 2024-09-18T10:15:35.871 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.871 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.872 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:35.872 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.722499+0000", 2024-09-18T10:15:35.872 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.101718085 2024-09-18T10:15:35.872 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:35.872 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:35.872 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:35.872 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.872 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.872 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:154 3.8s0 3:1d386431:::smithi18925896-2:head [copy-from ver 4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:35.872 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:51.211083+0000", 2024-09-18T10:15:35.872 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 44.572435325999997, 2024-09-18T10:15:35.872 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.73193688499999998, 2024-09-18T10:15:35.873 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:35.873 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:35.873 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:35.873 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:15:35.873 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:35.873 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 154 2024-09-18T10:15:35.873 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.873 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:35.873 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.873 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:35.873 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.211083+0000", 2024-09-18T10:15:35.873 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:35.873 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.873 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.874 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:35.874 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.211083+0000", 2024-09-18T10:15:35.874 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:35.874 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.874 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.874 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:35.874 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.211080+0000", 2024-09-18T10:15:35.874 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999971 2024-09-18T10:15:35.874 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.874 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.874 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:35.874 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.211096+0000", 2024-09-18T10:15:35.874 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.5699999999999999e-05 2024-09-18T10:15:35.874 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.875 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.875 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:35.875 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.211134+0000", 2024-09-18T10:15:35.875 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.7694000000000002e-05 2024-09-18T10:15:35.875 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.875 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.875 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:35.875 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.211149+0000", 2024-09-18T10:15:35.875 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4746000000000001e-05 2024-09-18T10:15:35.875 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.875 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.875 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:35.875 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.621341+0000", 2024-09-18T10:15:35.875 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.41019203799999998 2024-09-18T10:15:35.876 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.876 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.876 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:35.876 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.621505+0000", 2024-09-18T10:15:35.876 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00016443299999999999 2024-09-18T10:15:35.876 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.876 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.876 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:35.876 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.840064+0000", 2024-09-18T10:15:35.876 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.21855904700000001 2024-09-18T10:15:35.876 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.876 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.876 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:35.876 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.842430+0000", 2024-09-18T10:15:35.877 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0023664100000000002 2024-09-18T10:15:35.877 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.877 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.877 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:15:35.877 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.942864+0000", 2024-09-18T10:15:35.877 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10043373 2024-09-18T10:15:35.877 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.877 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.877 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:35.877 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.943020+0000", 2024-09-18T10:15:35.877 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00015598000000000001 2024-09-18T10:15:35.877 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:35.877 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:35.877 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:35.878 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.878 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.878 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:155 3.8s0 3:1d386431:::smithi18925896-2:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:35.878 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:51.211164+0000", 2024-09-18T10:15:35.878 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 44.572354955999998, 2024-09-18T10:15:35.878 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.630957349, 2024-09-18T10:15:35.878 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:35.878 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "started", 2024-09-18T10:15:35.878 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:35.878 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:15:35.878 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:35.878 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 155 2024-09-18T10:15:35.878 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.879 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:35.879 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.879 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:35.879 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.211164+0000", 2024-09-18T10:15:35.879 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:35.879 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.879 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.879 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:35.879 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.211164+0000", 2024-09-18T10:15:35.879 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:35.879 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.879 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.879 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:35.880 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.211163+0000", 2024-09-18T10:15:35.880 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:15:35.880 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.880 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.880 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:35.880 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.211167+0000", 2024-09-18T10:15:35.880 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.6299999999999997e-06 2024-09-18T10:15:35.880 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.880 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.880 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:35.880 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.211178+0000", 2024-09-18T10:15:35.880 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0798999999999999e-05 2024-09-18T10:15:35.881 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.881 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.881 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:35.881 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.211186+0000", 2024-09-18T10:15:35.881 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.5669999999999997e-06 2024-09-18T10:15:35.881 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.881 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.881 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:35.881 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.623024+0000", 2024-09-18T10:15:35.881 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.411838433 2024-09-18T10:15:35.881 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.881 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.881 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for blocked object", 2024-09-18T10:15:35.881 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.623104+0000", 2024-09-18T10:15:35.881 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.0136000000000007e-05 2024-09-18T10:15:35.882 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.882 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.882 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:35.882 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.841600+0000", 2024-09-18T10:15:35.882 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.21849595399999999 2024-09-18T10:15:35.882 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.882 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.882 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:35.882 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.841868+0000", 2024-09-18T10:15:35.882 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00026825800000000001 2024-09-18T10:15:35.882 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.882 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.882 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:35.882 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.842121+0000", 2024-09-18T10:15:35.883 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00025266500000000001 2024-09-18T10:15:35.883 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:35.883 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:35.883 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:35.883 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.883 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.883 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:619 3.8s0 3:1162852a:::smithi18925896-44:head [copy-from ver 14] snapc 21=[20,1f,1e,1a,16,12,11,10] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-18T10:15:35.883 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:27.977392+0000", 2024-09-18T10:15:35.883 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 7.8061267020000003, 2024-09-18T10:15:35.883 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.71139311599999999, 2024-09-18T10:15:35.883 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:35.883 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:35.883 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:35.883 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:15:35.884 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:35.884 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 619 2024-09-18T10:15:35.884 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.884 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:35.884 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.884 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:35.884 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:27.977392+0000", 2024-09-18T10:15:35.884 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:35.884 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.884 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.884 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:35.884 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:27.977392+0000", 2024-09-18T10:15:35.884 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:35.885 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.885 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.885 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:35.885 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:27.977390+0000", 2024-09-18T10:15:35.885 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:35.885 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.885 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.885 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:35.885 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:27.977399+0000", 2024-09-18T10:15:35.885 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.1239999999999994e-06 2024-09-18T10:15:35.885 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.885 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.885 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:35.885 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:27.977422+0000", 2024-09-18T10:15:35.886 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.2734999999999998e-05 2024-09-18T10:15:35.886 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.886 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.886 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:35.886 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.077547+0000", 2024-09-18T10:15:35.886 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100124728 2024-09-18T10:15:35.886 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.886 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.886 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:35.886 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.077713+0000", 2024-09-18T10:15:35.886 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000166515 2024-09-18T10:15:35.886 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.886 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.887 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op must wait for map", 2024-09-18T10:15:35.887 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.077728+0000", 2024-09-18T10:15:35.887 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.5036e-05 2024-09-18T10:15:35.887 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.887 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.887 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:35.887 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.275819+0000", 2024-09-18T10:15:35.887 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.19809117600000001 2024-09-18T10:15:35.887 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.887 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.887 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:35.887 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.276220+0000", 2024-09-18T10:15:35.887 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000400426 2024-09-18T10:15:35.888 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.888 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.888 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:35.888 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.383764+0000", 2024-09-18T10:15:35.888 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.107543814 2024-09-18T10:15:35.888 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.888 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.888 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:35.888 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.486402+0000", 2024-09-18T10:15:35.888 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.102638484 2024-09-18T10:15:35.888 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.888 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.888 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:15:35.889 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.688521+0000", 2024-09-18T10:15:35.889 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.202118873 2024-09-18T10:15:35.889 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.889 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.889 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:35.889 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.688785+0000", 2024-09-18T10:15:35.889 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000264133 2024-09-18T10:15:35.889 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:35.889 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:35.889 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:35.889 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.889 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.890 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:626 3.10s0 3:0a83d6b3:::smithi18925896-20:head [append 0~638976] snapc 21=[20,1f,1e,1a,16,12,11,10] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-18T10:15:35.890 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:28.708261+0000", 2024-09-18T10:15:35.890 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 7.0752571370000004, 2024-09-18T10:15:35.890 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.70096385699999997, 2024-09-18T10:15:35.890 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:35.890 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:35.890 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:35.890 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:15:35.890 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:35.890 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 626 2024-09-18T10:15:35.890 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.891 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:35.891 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.891 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:35.891 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.708261+0000", 2024-09-18T10:15:35.891 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:35.891 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.891 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.891 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:35.891 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.708261+0000", 2024-09-18T10:15:35.891 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:35.891 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.891 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.891 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:35.892 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.708259+0000", 2024-09-18T10:15:35.892 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:35.892 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.892 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.892 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:35.892 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.709175+0000", 2024-09-18T10:15:35.892 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00091571199999999995 2024-09-18T10:15:35.892 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.892 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.892 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:35.892 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.709199+0000", 2024-09-18T10:15:35.893 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.3785e-05 2024-09-18T10:15:35.893 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.893 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.893 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:35.893 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.809328+0000", 2024-09-18T10:15:35.893 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100129494 2024-09-18T10:15:35.893 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.893 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.893 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:35.893 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.996826+0000", 2024-09-18T10:15:35.893 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.18749712700000001 2024-09-18T10:15:35.893 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.893 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.894 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:35.894 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.997086+0000", 2024-09-18T10:15:35.894 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00026047499999999998 2024-09-18T10:15:35.894 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.894 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.894 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:35.894 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.998987+0000", 2024-09-18T10:15:35.894 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019014889999999999 2024-09-18T10:15:35.894 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.894 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.894 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:35.894 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.104118+0000", 2024-09-18T10:15:35.895 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10513012100000001 2024-09-18T10:15:35.895 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.895 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.895 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:15:35.895 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.409085+0000", 2024-09-18T10:15:35.895 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.30496710599999999 2024-09-18T10:15:35.895 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.895 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.895 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:35.895 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.409225+0000", 2024-09-18T10:15:35.895 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00014059899999999999 2024-09-18T10:15:35.895 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:35.896 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:35.896 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:35.896 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.896 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.896 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:629 3.10s0 3:0a83d6b3:::smithi18925896-20:head [setxattr _header (57) in=7b] snapc 21=[20,1f,1e,1a,16,12,11,10] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-18T10:15:35.896 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:28.910973+0000", 2024-09-18T10:15:35.896 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 6.8725460189999996, 2024-09-18T10:15:35.896 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.80485965999999998, 2024-09-18T10:15:35.896 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:35.896 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:35.896 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:35.896 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:15:35.896 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:35.897 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 629 2024-09-18T10:15:35.897 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.897 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:35.897 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.897 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:35.897 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.910973+0000", 2024-09-18T10:15:35.897 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:35.897 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.897 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.897 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:35.897 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.910973+0000", 2024-09-18T10:15:35.897 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:35.897 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.898 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.898 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:35.898 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.910971+0000", 2024-09-18T10:15:35.898 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:15:35.898 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.898 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.898 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:35.898 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.910976+0000", 2024-09-18T10:15:35.898 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.0069999999999997e-06 2024-09-18T10:15:35.898 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.898 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.898 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:35.898 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.910984+0000", 2024-09-18T10:15:35.899 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.7109999999999995e-06 2024-09-18T10:15:35.899 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.899 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.899 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:35.899 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.910989+0000", 2024-09-18T10:15:35.899 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1549999999999998e-06 2024-09-18T10:15:35.899 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.899 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.899 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:35.899 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.411082+0000", 2024-09-18T10:15:35.899 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.50009252199999998 2024-09-18T10:15:35.899 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.900 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.900 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:35.900 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.411264+0000", 2024-09-18T10:15:35.900 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00018190799999999999 2024-09-18T10:15:35.900 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.900 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.900 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:35.900 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.412262+0000", 2024-09-18T10:15:35.900 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00099852199999999995 2024-09-18T10:15:35.900 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.900 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.900 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:35.900 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.513736+0000", 2024-09-18T10:15:35.901 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.101474077 2024-09-18T10:15:35.901 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.901 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.901 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:15:35.901 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.715730+0000", 2024-09-18T10:15:35.901 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20199325600000001 2024-09-18T10:15:35.901 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.901 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.901 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:35.901 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.715832+0000", 2024-09-18T10:15:35.901 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000102633 2024-09-18T10:15:35.901 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:35.902 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:35.902 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:35.902 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.902 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.902 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:630 3.10s0 3:0a83d6b3:::smithi18925896-20:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e72)", 2024-09-18T10:15:35.902 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:28.910995+0000", 2024-09-18T10:15:35.902 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 6.8725239880000002, 2024-09-18T10:15:35.902 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2560086020000001, 2024-09-18T10:15:35.902 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:35.902 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "started", 2024-09-18T10:15:35.902 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:35.902 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:15:35.902 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:35.902 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 630 2024-09-18T10:15:35.903 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.903 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:35.903 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.903 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:35.903 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.910995+0000", 2024-09-18T10:15:35.903 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:35.903 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.903 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.903 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:35.903 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.910995+0000", 2024-09-18T10:15:35.903 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:35.903 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.903 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.903 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:35.904 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.910994+0000", 2024-09-18T10:15:35.904 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:15:35.904 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.904 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.904 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:35.904 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.910997+0000", 2024-09-18T10:15:35.904 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2820000000000001e-06 2024-09-18T10:15:35.904 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.904 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.904 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:35.904 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.911008+0000", 2024-09-18T10:15:35.904 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0617e-05 2024-09-18T10:15:35.904 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.904 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.905 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:35.905 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.911013+0000", 2024-09-18T10:15:35.905 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.8160000000000001e-06 2024-09-18T10:15:35.905 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.905 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.905 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:35.905 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.413034+0000", 2024-09-18T10:15:35.905 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.50202176600000004 2024-09-18T10:15:35.905 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.905 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.905 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for rw locks", 2024-09-18T10:15:35.905 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.413303+0000", 2024-09-18T10:15:35.906 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00026880600000000001 2024-09-18T10:15:35.906 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.906 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.906 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:35.906 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.716533+0000", 2024-09-18T10:15:35.906 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.30323016000000003 2024-09-18T10:15:35.906 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.906 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.906 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:35.906 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.716647+0000", 2024-09-18T10:15:35.906 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00011372899999999999 2024-09-18T10:15:35.906 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:35.906 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:35.906 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:35.907 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:30.167003+0000", 2024-09-18T10:15:35.907 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.450356172 2024-09-18T10:15:35.907 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:35.907 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:35.907 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:35.907 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:35.907 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:35.907 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-09-18T10:15:35.908 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T10:15:35.909 INFO:tasks.rados.rados.0.smithi189.stdout:284: expect (ObjNum 68 snap 6 seq_num 68) 2024-09-18T10:15:35.910 INFO:tasks.ceph.osd.15.smithi189.stderr:2024-09-18T10:15:35.902+0000 7fd0e725e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 22401) UID: 0 2024-09-18T10:15:35.910 INFO:tasks.rados.rados.0.smithi189.stdout:283: expect (ObjNum 3 snap 0 seq_num 3) 2024-09-18T10:15:35.910 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-09-18T10:15:35.986 INFO:tasks.rados.rados.0.smithi189.stdout:283: done (3 left) 2024-09-18T10:15:35.986 INFO:tasks.rados.rados.0.smithi189.stdout:284: done (2 left) 2024-09-18T10:15:35.986 INFO:tasks.rados.rados.0.smithi189.stdout:285: done (1 left) 2024-09-18T10:15:35.986 INFO:tasks.rados.rados.0.smithi189.stdout:286: delete oid 36 current snap is 25 2024-09-18T10:15:36.010 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-18T10:15:36.006+0000 7f14dcc5b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22398) UID: 0 2024-09-18T10:15:36.019 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-09-18T10:15:36.019 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-09-18T10:15:36.019 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-09-18T10:15:36.019 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-09-18T10:15:36.029 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-09-18T10:15:36.056 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-18T10:15:36.081 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-18T10:15:36.099 INFO:tasks.rados.rados.0.smithi189.stdout:282: expect (ObjNum 95 snap 17 seq_num 95) 2024-09-18T10:15:36.105 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-18T10:15:36.111 INFO:tasks.ceph.osd.8.smithi029.stderr:2024-09-18T10:15:36.105+0000 7f2555c27700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 22192) UID: 0 2024-09-18T10:15:36.127 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-09-18T10:15:36.127 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-09-18T10:15:36.127 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-09-18T10:15:36.128 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-09-18T10:15:36.128 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-09-18T10:15:36.130 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-18T10:15:36.136 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-09-18T10:15:36.152 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-18T10:15:36.175 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-18T10:15:36.202 INFO:teuthology.orchestra.run.smithi029.stdout:osd.6: {} 2024-09-18T10:15:36.212 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-09-18T10:15:36.210+0000 7fdfa8698700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22087) UID: 0 2024-09-18T10:15:36.223 INFO:teuthology.orchestra.run.smithi029.stdout:osd.7: {} 2024-09-18T10:15:36.240 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-09-18T10:15:36.240 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-09-18T10:15:36.240 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-09-18T10:15:36.241 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-09-18T10:15:36.241 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.241 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:189 3.3s0 3:ce8164ce:::smithi18925896-7:head [copy-from ver 6] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:36.241 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:52.694279+0000", 2024-09-18T10:15:36.241 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 43.542627088000003, 2024-09-18T10:15:36.241 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.51539283599999997, 2024-09-18T10:15:36.241 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:36.241 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:36.241 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:36.241 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:15:36.241 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:36.241 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 189 2024-09-18T10:15:36.242 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.242 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:36.242 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.242 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:36.242 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694279+0000", 2024-09-18T10:15:36.242 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:36.242 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.242 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.242 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:36.242 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694279+0000", 2024-09-18T10:15:36.242 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:36.242 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.242 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.242 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:36.243 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694277+0000", 2024-09-18T10:15:36.243 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999971 2024-09-18T10:15:36.243 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.243 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.243 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:36.243 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694290+0000", 2024-09-18T10:15:36.243 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3862e-05 2024-09-18T10:15:36.243 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.243 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.243 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:36.243 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694311+0000", 2024-09-18T10:15:36.243 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.0273999999999999e-05 2024-09-18T10:15:36.243 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.244 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.244 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:36.244 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694322+0000", 2024-09-18T10:15:36.244 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1722999999999999e-05 2024-09-18T10:15:36.244 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.244 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.244 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:36.244 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.794494+0000", 2024-09-18T10:15:36.244 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100171653 2024-09-18T10:15:36.244 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.244 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.244 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:36.245 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.794705+0000", 2024-09-18T10:15:36.245 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00021126000000000001 2024-09-18T10:15:36.245 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.245 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.245 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:36.245 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.096113+0000", 2024-09-18T10:15:36.245 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.30140739100000002 2024-09-18T10:15:36.245 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.245 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.245 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:36.245 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.098602+0000", 2024-09-18T10:15:36.245 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.002489593 2024-09-18T10:15:36.245 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.246 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.246 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:15:36.246 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.209538+0000", 2024-09-18T10:15:36.246 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.110935482 2024-09-18T10:15:36.246 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.246 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.246 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:36.246 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.209672+0000", 2024-09-18T10:15:36.246 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00013435399999999999 2024-09-18T10:15:36.246 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:36.246 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:36.246 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:36.246 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.247 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.247 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:194 3.3s0 3:c9ce6440:::smithi18925896-24:head [copy-from ver 6] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:36.247 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:52.694388+0000", 2024-09-18T10:15:36.247 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 43.542518225000002, 2024-09-18T10:15:36.247 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.92118689200000003, 2024-09-18T10:15:36.247 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:36.247 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:36.247 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:36.247 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:15:36.247 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:36.247 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 194 2024-09-18T10:15:36.247 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.247 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:36.248 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.248 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:36.248 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694388+0000", 2024-09-18T10:15:36.248 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:36.248 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.248 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.248 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:36.248 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694388+0000", 2024-09-18T10:15:36.248 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:36.248 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.248 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.248 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:36.248 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694388+0000", 2024-09-18T10:15:36.248 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:15:36.249 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.249 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.249 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:36.249 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694393+0000", 2024-09-18T10:15:36.249 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.2299999999999999e-06 2024-09-18T10:15:36.249 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.249 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.249 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:36.249 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694411+0000", 2024-09-18T10:15:36.249 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7802999999999999e-05 2024-09-18T10:15:36.249 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.250 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.250 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:36.250 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694417+0000", 2024-09-18T10:15:36.250 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.4640000000000003e-06 2024-09-18T10:15:36.250 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.250 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.250 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:36.250 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.795289+0000", 2024-09-18T10:15:36.250 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100872288 2024-09-18T10:15:36.250 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.251 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.251 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:36.251 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.795548+0000", 2024-09-18T10:15:36.251 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00025869199999999998 2024-09-18T10:15:36.251 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.251 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.251 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:36.251 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.412215+0000", 2024-09-18T10:15:36.251 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.61666667100000006 2024-09-18T10:15:36.251 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.251 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.251 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:36.251 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.514838+0000", 2024-09-18T10:15:36.252 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10262323399999999 2024-09-18T10:15:36.252 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.252 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.252 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:15:36.253 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.615386+0000", 2024-09-18T10:15:36.253 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100548106 2024-09-18T10:15:36.253 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.253 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.253 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:36.253 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.615575+0000", 2024-09-18T10:15:36.253 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000189128 2024-09-18T10:15:36.253 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:36.253 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:36.253 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:36.254 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.254 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.254 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:195 3.3s0 3:c9ce6440:::smithi18925896-24:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:36.254 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:52.694424+0000", 2024-09-18T10:15:36.254 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 43.542482489000001, 2024-09-18T10:15:36.254 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.82009685499999996, 2024-09-18T10:15:36.254 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:36.254 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "started", 2024-09-18T10:15:36.254 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:36.254 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:15:36.254 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:36.254 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 195 2024-09-18T10:15:36.255 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.255 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:36.255 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.255 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:36.255 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694424+0000", 2024-09-18T10:15:36.255 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:36.255 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.255 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.255 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:36.255 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694424+0000", 2024-09-18T10:15:36.255 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:36.255 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.256 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.256 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:36.256 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694423+0000", 2024-09-18T10:15:36.256 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:36.256 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.256 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.256 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:36.256 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694427+0000", 2024-09-18T10:15:36.256 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2650000000000001e-06 2024-09-18T10:15:36.256 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.256 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.256 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:36.256 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694434+0000", 2024-09-18T10:15:36.257 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.5410000000000003e-06 2024-09-18T10:15:36.257 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.257 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.257 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:36.257 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694441+0000", 2024-09-18T10:15:36.257 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.187e-06 2024-09-18T10:15:36.257 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.257 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.257 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:36.257 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.795870+0000", 2024-09-18T10:15:36.257 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.101428617 2024-09-18T10:15:36.257 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.257 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.258 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for blocked object", 2024-09-18T10:15:36.258 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.796040+0000", 2024-09-18T10:15:36.258 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00017008 2024-09-18T10:15:36.258 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.258 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.258 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:36.258 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.514130+0000", 2024-09-18T10:15:36.258 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.71809022199999994 2024-09-18T10:15:36.258 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.259 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.259 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:36.259 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.514348+0000", 2024-09-18T10:15:36.259 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000217488 2024-09-18T10:15:36.259 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.259 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.259 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:36.259 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.514521+0000", 2024-09-18T10:15:36.259 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00017300900000000001 2024-09-18T10:15:36.259 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:36.260 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:36.260 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:36.260 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.260 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.260 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.bs2 41/32 ECSubWrite(tid=58, reqid=osd.4.0:58, at_version=41'14, trim_to=0'0, roll_forward_to=39'12))", 2024-09-18T10:15:36.260 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:56.829698+0000", 2024-09-18T10:15:36.260 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 39.407208394999998, 2024-09-18T10:15:36.260 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.71711602600000002, 2024-09-18T10:15:36.260 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:36.260 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:15:36.260 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:36.260 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.260 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:36.260 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.829698+0000", 2024-09-18T10:15:36.260 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:36.261 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.261 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.261 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:36.261 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.829698+0000", 2024-09-18T10:15:36.261 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:36.261 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.261 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.261 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:36.261 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.829697+0000", 2024-09-18T10:15:36.261 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:36.261 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.262 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.262 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:36.262 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.829733+0000", 2024-09-18T10:15:36.262 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.6217999999999998e-05 2024-09-18T10:15:36.262 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.262 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.262 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:36.262 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.829757+0000", 2024-09-18T10:15:36.262 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.3915000000000001e-05 2024-09-18T10:15:36.262 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.262 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.262 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:36.262 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.829769+0000", 2024-09-18T10:15:36.262 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1754e-05 2024-09-18T10:15:36.263 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.263 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.263 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:36.263 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.342965+0000", 2024-09-18T10:15:36.263 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.51319593799999996 2024-09-18T10:15:36.263 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.263 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.263 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:36.263 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.343020+0000", 2024-09-18T10:15:36.263 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5367e-05 2024-09-18T10:15:36.263 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.263 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.263 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:36.263 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.546772+0000", 2024-09-18T10:15:36.264 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.203751933 2024-09-18T10:15:36.264 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.264 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.264 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:36.264 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.546814+0000", 2024-09-18T10:15:36.264 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.2196000000000001e-05 2024-09-18T10:15:36.264 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:36.264 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:36.264 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:36.264 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.264 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.264 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDPGPushReply(3.17s0 67/64 [PushReplyOp(3:ee962131:::smithi18925896-29:10)])", 2024-09-18T10:15:36.264 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:23.671965+0000", 2024-09-18T10:15:36.265 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 12.564941944999999, 2024-09-18T10:15:36.265 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.60673808500000004, 2024-09-18T10:15:36.265 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:36.265 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:15:36.265 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:36.265 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.265 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:36.265 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.671965+0000", 2024-09-18T10:15:36.265 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:36.265 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.265 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.265 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:36.265 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.671965+0000", 2024-09-18T10:15:36.265 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:36.266 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.266 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.266 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:36.266 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.671963+0000", 2024-09-18T10:15:36.266 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:36.266 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.266 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.266 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:36.266 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.671977+0000", 2024-09-18T10:15:36.266 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4093e-05 2024-09-18T10:15:36.266 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.266 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.266 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:36.267 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.672009+0000", 2024-09-18T10:15:36.267 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1939e-05 2024-09-18T10:15:36.267 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.267 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.267 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:36.267 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.772102+0000", 2024-09-18T10:15:36.267 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10009319 2024-09-18T10:15:36.267 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.267 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.267 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:36.267 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.278591+0000", 2024-09-18T10:15:36.267 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.50648915999999999 2024-09-18T10:15:36.267 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.267 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.267 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:36.268 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.278703+0000", 2024-09-18T10:15:36.268 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000111433 2024-09-18T10:15:36.268 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:36.268 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:36.268 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:36.268 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.268 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.268 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDPGPushReply(3.17s0 67/64 [PushReplyOp(3:ee962131:::smithi18925896-29:10)])", 2024-09-18T10:15:36.268 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:23.672359+0000", 2024-09-18T10:15:36.268 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 12.564547489000001, 2024-09-18T10:15:36.268 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.50581707200000003, 2024-09-18T10:15:36.268 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:36.268 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:15:36.269 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:36.269 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.269 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:36.269 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.672359+0000", 2024-09-18T10:15:36.269 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:36.269 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.269 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.269 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:36.269 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.672359+0000", 2024-09-18T10:15:36.269 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:36.269 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.269 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.269 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:36.269 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.672358+0000", 2024-09-18T10:15:36.270 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:36.270 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.270 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.270 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:36.270 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.672367+0000", 2024-09-18T10:15:36.270 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.2529999999999993e-06 2024-09-18T10:15:36.270 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.270 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.271 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:36.271 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.672388+0000", 2024-09-18T10:15:36.271 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.0999e-05 2024-09-18T10:15:36.271 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.271 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.271 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:36.271 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.672397+0000", 2024-09-18T10:15:36.271 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.8300000000000002e-06 2024-09-18T10:15:36.271 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.271 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.271 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:36.271 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.178110+0000", 2024-09-18T10:15:36.271 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.50571351399999998 2024-09-18T10:15:36.272 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.272 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.272 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:36.272 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.178176+0000", 2024-09-18T10:15:36.272 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.5703000000000004e-05 2024-09-18T10:15:36.272 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:36.272 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:36.272 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:36.272 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.272 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.272 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDPGPushReply(3.17s0 67/64 [PushReplyOp(3:ee962131:::smithi18925896-29:10)])", 2024-09-18T10:15:36.272 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:23.770889+0000", 2024-09-18T10:15:36.272 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 12.466017044999999, 2024-09-18T10:15:36.273 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.40748533399999998, 2024-09-18T10:15:36.273 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:36.273 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:15:36.273 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:36.273 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.273 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:36.273 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.770889+0000", 2024-09-18T10:15:36.273 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:36.273 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.273 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.273 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:36.274 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.770889+0000", 2024-09-18T10:15:36.274 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:36.274 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.274 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.274 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:36.274 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.770888+0000", 2024-09-18T10:15:36.274 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:36.274 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.274 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.274 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:36.274 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.770901+0000", 2024-09-18T10:15:36.274 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3216e-05 2024-09-18T10:15:36.274 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.274 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.275 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:36.275 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.770933+0000", 2024-09-18T10:15:36.275 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1955999999999997e-05 2024-09-18T10:15:36.275 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.275 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.275 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:36.275 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.770946+0000", 2024-09-18T10:15:36.275 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.289e-05 2024-09-18T10:15:36.275 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.275 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.275 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:36.275 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.178302+0000", 2024-09-18T10:15:36.276 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.40735659000000002 2024-09-18T10:15:36.276 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.276 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.276 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:36.276 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.178375+0000", 2024-09-18T10:15:36.276 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.2335000000000005e-05 2024-09-18T10:15:36.276 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:36.276 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:36.276 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:36.276 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.276 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.276 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDPGPushReply(3.17s0 67/64 [PushReplyOp(3:ee962131:::smithi18925896-29:10)])", 2024-09-18T10:15:36.276 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:23.772436+0000", 2024-09-18T10:15:36.277 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 12.464470709, 2024-09-18T10:15:36.277 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.50655292500000004, 2024-09-18T10:15:36.277 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:36.277 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:15:36.277 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:36.277 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.277 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:36.277 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.772436+0000", 2024-09-18T10:15:36.277 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:36.277 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.277 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.277 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:36.277 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.772436+0000", 2024-09-18T10:15:36.278 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:36.278 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.278 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.278 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:36.279 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.772434+0000", 2024-09-18T10:15:36.279 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:36.280 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.280 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.280 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:36.280 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.772448+0000", 2024-09-18T10:15:36.280 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3682e-05 2024-09-18T10:15:36.280 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.280 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.280 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:36.280 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.772477+0000", 2024-09-18T10:15:36.280 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9255999999999999e-05 2024-09-18T10:15:36.280 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.280 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.281 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:36.281 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.772490+0000", 2024-09-18T10:15:36.281 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2962999999999999e-05 2024-09-18T10:15:36.281 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.281 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.281 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:36.281 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.278888+0000", 2024-09-18T10:15:36.281 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.50639794699999996 2024-09-18T10:15:36.281 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.281 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.282 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:36.282 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.278989+0000", 2024-09-18T10:15:36.282 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000100777 2024-09-18T10:15:36.282 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:36.282 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:36.282 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:36.282 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.282 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.282 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDPGPushReply(3.17s0 67/64 [PushReplyOp(3:ee962131:::smithi18925896-29:10)])", 2024-09-18T10:15:36.282 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:23.772911+0000", 2024-09-18T10:15:36.282 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 12.463995430000001, 2024-09-18T10:15:36.283 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.50636473400000004, 2024-09-18T10:15:36.283 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:36.283 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:15:36.283 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:36.283 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.283 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:36.283 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.772911+0000", 2024-09-18T10:15:36.283 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:36.284 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.284 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.284 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:36.285 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.772911+0000", 2024-09-18T10:15:36.285 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:36.285 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.285 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.285 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:36.285 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.772909+0000", 2024-09-18T10:15:36.285 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:36.285 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.285 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.285 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:36.286 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.772923+0000", 2024-09-18T10:15:36.286 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3957e-05 2024-09-18T10:15:36.286 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.286 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.286 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:36.286 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.772956+0000", 2024-09-18T10:15:36.286 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2925000000000001e-05 2024-09-18T10:15:36.286 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.286 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.287 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:36.288 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.772969+0000", 2024-09-18T10:15:36.288 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2516e-05 2024-09-18T10:15:36.288 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.288 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.288 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:36.288 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.279192+0000", 2024-09-18T10:15:36.288 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.50622349799999999 2024-09-18T10:15:36.288 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.288 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.288 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:36.288 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.279276+0000", 2024-09-18T10:15:36.288 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.3498000000000006e-05 2024-09-18T10:15:36.288 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:36.289 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:36.289 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:36.289 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.289 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.289 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDPGPush(3.17s0 67/64 [PushOp(3:ead53b0f:::smithi18925896-25:13, version: 60'28, data_included: [262144~53248], data_size: 53248, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:ead53b0f:::smithi18925896-25:13@60'28, size: 1261568, copy_subset: [], clone_subset: {}, snapset: 13=[]:{13=[12,11,10,f,e]}, object_exist: 1), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:1261568, data_complete:true, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to:1048576, data_complete:false, omap_recovered_to:, omap_complete:true, error:false))])", 2024-09-18T10:15:36.289 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:23.871838+0000", 2024-09-18T10:15:36.289 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 12.365068215999999, 2024-09-18T10:15:36.289 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.409266511, 2024-09-18T10:15:36.289 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:36.290 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:15:36.290 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:36.290 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.290 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:36.290 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.871838+0000", 2024-09-18T10:15:36.290 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:36.290 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.290 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.290 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:36.290 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.871838+0000", 2024-09-18T10:15:36.290 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:36.290 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.290 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.291 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:36.291 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.871838+0000", 2024-09-18T10:15:36.291 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:36.291 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.291 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.291 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:36.291 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.871838+0000", 2024-09-18T10:15:36.291 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:36.291 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.291 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.291 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:36.291 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.871884+0000", 2024-09-18T10:15:36.291 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.5512000000000001e-05 2024-09-18T10:15:36.292 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.292 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.292 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:36.292 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.871901+0000", 2024-09-18T10:15:36.292 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.6906e-05 2024-09-18T10:15:36.292 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.292 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.292 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:36.292 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.279416+0000", 2024-09-18T10:15:36.292 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.407515605 2024-09-18T10:15:36.292 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.292 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.292 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:36.293 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.281105+0000", 2024-09-18T10:15:36.293 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0016884879999999999 2024-09-18T10:15:36.293 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:36.293 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:36.293 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:36.293 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.293 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.293 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDPGPushReply(3.17s0 67/64 [PushReplyOp(3:ead53b0f:::smithi18925896-25:13)])", 2024-09-18T10:15:36.293 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:23.874483+0000", 2024-09-18T10:15:36.293 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 12.362423389, 2024-09-18T10:15:36.293 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.40688253899999999, 2024-09-18T10:15:36.293 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:36.293 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:15:36.294 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:36.294 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.294 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:36.294 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.874483+0000", 2024-09-18T10:15:36.294 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:36.294 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.294 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.294 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:36.294 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.874483+0000", 2024-09-18T10:15:36.294 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:36.294 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.294 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.294 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:36.294 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.874483+0000", 2024-09-18T10:15:36.295 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:36.295 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.295 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.295 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:36.295 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.874489+0000", 2024-09-18T10:15:36.295 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.4869999999999999e-06 2024-09-18T10:15:36.295 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.295 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.295 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:36.295 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.874501+0000", 2024-09-18T10:15:36.295 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1803e-05 2024-09-18T10:15:36.295 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.296 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.296 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:36.296 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.874511+0000", 2024-09-18T10:15:36.296 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0363000000000001e-05 2024-09-18T10:15:36.296 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.296 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.296 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:36.296 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.281278+0000", 2024-09-18T10:15:36.296 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.40676716899999998 2024-09-18T10:15:36.296 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.296 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.296 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:36.296 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.281366+0000", 2024-09-18T10:15:36.297 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.7233999999999999e-05 2024-09-18T10:15:36.297 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:36.297 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:36.297 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:36.297 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.297 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.297 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDPGPushReply(3.17s0 67/64 [PushReplyOp(3:ead53b0f:::smithi18925896-25:13)])", 2024-09-18T10:15:36.297 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:23.874889+0000", 2024-09-18T10:15:36.297 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 12.36201709, 2024-09-18T10:15:36.297 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.40661987900000002, 2024-09-18T10:15:36.297 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:36.297 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:15:36.297 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:36.297 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.298 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:36.298 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.874889+0000", 2024-09-18T10:15:36.298 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:36.298 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.298 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.298 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:36.298 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.874889+0000", 2024-09-18T10:15:36.298 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:36.298 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.298 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.298 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:36.298 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.874889+0000", 2024-09-18T10:15:36.299 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:36.299 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.299 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.299 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:36.299 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.874895+0000", 2024-09-18T10:15:36.299 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.6459999999999998e-06 2024-09-18T10:15:36.299 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.299 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.299 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:36.299 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.874905+0000", 2024-09-18T10:15:36.299 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.005e-05 2024-09-18T10:15:36.299 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.299 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.300 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:36.300 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.874917+0000", 2024-09-18T10:15:36.300 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.204e-05 2024-09-18T10:15:36.300 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.300 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.300 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:36.300 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.281419+0000", 2024-09-18T10:15:36.300 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.40650187300000001 2024-09-18T10:15:36.300 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.300 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.300 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:36.300 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.281509+0000", 2024-09-18T10:15:36.300 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.0729999999999994e-05 2024-09-18T10:15:36.300 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:36.301 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:36.301 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:36.301 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.301 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.301 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDPGPushReply(3.17s0 67/64 [PushReplyOp(3:ead53b0f:::smithi18925896-25:13)])", 2024-09-18T10:15:36.301 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:23.875511+0000", 2024-09-18T10:15:36.301 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 12.361395805000001, 2024-09-18T10:15:36.301 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.50638906400000006, 2024-09-18T10:15:36.301 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:36.301 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:15:36.301 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:36.301 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.301 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:36.302 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.875511+0000", 2024-09-18T10:15:36.302 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:36.302 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.302 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.302 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:36.302 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.875511+0000", 2024-09-18T10:15:36.302 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:36.302 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.302 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.302 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:36.302 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.875510+0000", 2024-09-18T10:15:36.302 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:36.302 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.303 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.303 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:36.303 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.875516+0000", 2024-09-18T10:15:36.303 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.8930000000000002e-06 2024-09-18T10:15:36.303 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.303 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.303 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:36.303 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.875526+0000", 2024-09-18T10:15:36.303 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0183000000000001e-05 2024-09-18T10:15:36.303 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.303 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.303 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:36.303 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.975617+0000", 2024-09-18T10:15:36.303 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10009069299999999 2024-09-18T10:15:36.303 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.304 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.304 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:36.304 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.381839+0000", 2024-09-18T10:15:36.304 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.40622223600000001 2024-09-18T10:15:36.304 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.304 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.304 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:36.304 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.381900+0000", 2024-09-18T10:15:36.304 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.0572000000000002e-05 2024-09-18T10:15:36.304 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:36.304 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:36.304 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:36.304 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.305 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.305 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDPGPushReply(3.17s0 67/64 [PushReplyOp(3:ead53b0f:::smithi18925896-25:13)])", 2024-09-18T10:15:36.305 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:23.975748+0000", 2024-09-18T10:15:36.305 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 12.261158505999999, 2024-09-18T10:15:36.305 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.406336629, 2024-09-18T10:15:36.305 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:36.305 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:15:36.305 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:36.305 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.305 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:36.305 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.975748+0000", 2024-09-18T10:15:36.305 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:36.305 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.305 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.306 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:36.306 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.975748+0000", 2024-09-18T10:15:36.306 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:36.306 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.306 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.306 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:36.306 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.975748+0000", 2024-09-18T10:15:36.306 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:36.306 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.306 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.306 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:36.306 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.975757+0000", 2024-09-18T10:15:36.306 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.9369999999999993e-06 2024-09-18T10:15:36.307 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.307 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.307 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:36.307 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.975771+0000", 2024-09-18T10:15:36.307 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4722999999999999e-05 2024-09-18T10:15:36.307 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.307 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.307 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:36.307 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.975779+0000", 2024-09-18T10:15:36.307 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.2029999999999992e-06 2024-09-18T10:15:36.307 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.307 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.307 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:36.308 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.381963+0000", 2024-09-18T10:15:36.308 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.40618386400000001 2024-09-18T10:15:36.308 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.308 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.308 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:36.308 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.382085+0000", 2024-09-18T10:15:36.308 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000121279 2024-09-18T10:15:36.308 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:36.308 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:36.308 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:36.308 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.308 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.308 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:635 3.17s0 3:ead53b0f:::smithi18925896-25:head [append 0~655360] snapc 21=[20,1f,1e,1a,16,11,10] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-18T10:15:36.309 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:29.755594+0000", 2024-09-18T10:15:36.309 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 6.4813128850000004, 2024-09-18T10:15:36.309 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.62129057300000001, 2024-09-18T10:15:36.309 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:36.309 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:36.309 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:36.309 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:15:36.309 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:36.309 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 635 2024-09-18T10:15:36.310 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.310 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:36.310 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.310 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:36.310 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.755594+0000", 2024-09-18T10:15:36.310 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:36.310 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.310 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.310 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:36.310 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.755594+0000", 2024-09-18T10:15:36.310 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:36.310 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.310 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.311 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:36.311 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.755590+0000", 2024-09-18T10:15:36.311 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999967 2024-09-18T10:15:36.311 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.311 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.311 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:36.311 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.756465+0000", 2024-09-18T10:15:36.311 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00087483000000000005 2024-09-18T10:15:36.311 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.311 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.311 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:36.311 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.756502+0000", 2024-09-18T10:15:36.311 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.7067999999999998e-05 2024-09-18T10:15:36.311 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.312 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.312 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:36.312 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.756516+0000", 2024-09-18T10:15:36.312 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4071e-05 2024-09-18T10:15:36.312 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.312 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.312 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:36.312 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.760022+0000", 2024-09-18T10:15:36.312 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0035059570000000001 2024-09-18T10:15:36.312 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.312 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.312 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:36.313 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.760234+0000", 2024-09-18T10:15:36.313 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00021161199999999999 2024-09-18T10:15:36.313 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.313 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.313 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:36.313 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.762520+0000", 2024-09-18T10:15:36.313 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0022855760000000001 2024-09-18T10:15:36.313 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.313 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.313 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:36.313 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.770533+0000", 2024-09-18T10:15:36.313 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0080138269999999994 2024-09-18T10:15:36.313 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.313 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.314 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:15:36.314 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:30.376766+0000", 2024-09-18T10:15:36.314 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.60623257500000005 2024-09-18T10:15:36.314 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.314 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.314 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:36.314 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:30.376884+0000", 2024-09-18T10:15:36.314 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00011819699999999999 2024-09-18T10:15:36.314 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:36.314 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:36.315 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:36.315 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.315 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.315 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:636 3.17s0 3:ead53b0f:::smithi18925896-25:head [append 0~606208] snapc 21=[20,1f,1e,1a,16,11,10] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-18T10:15:36.316 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:29.758851+0000", 2024-09-18T10:15:36.317 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 6.478055232, 2024-09-18T10:15:36.317 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.61852410899999999, 2024-09-18T10:15:36.317 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:36.317 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:36.317 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:36.317 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:15:36.317 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:36.317 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 636 2024-09-18T10:15:36.317 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.317 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:36.317 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.317 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:36.318 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.758851+0000", 2024-09-18T10:15:36.318 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:36.318 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.318 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.318 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:36.318 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.758851+0000", 2024-09-18T10:15:36.318 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:36.318 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.318 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.318 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:36.318 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.758849+0000", 2024-09-18T10:15:36.318 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999976 2024-09-18T10:15:36.318 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.318 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.319 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:36.319 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.759643+0000", 2024-09-18T10:15:36.319 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00079385800000000004 2024-09-18T10:15:36.319 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.319 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.319 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:36.320 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.759677+0000", 2024-09-18T10:15:36.320 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.4504e-05 2024-09-18T10:15:36.320 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.320 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.320 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:36.321 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.759691+0000", 2024-09-18T10:15:36.321 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3586e-05 2024-09-18T10:15:36.321 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.322 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.322 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:36.322 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.765309+0000", 2024-09-18T10:15:36.322 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0056179460000000004 2024-09-18T10:15:36.322 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.322 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.322 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:36.322 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.765656+0000", 2024-09-18T10:15:36.322 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000347783 2024-09-18T10:15:36.322 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.322 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.323 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:36.323 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.766735+0000", 2024-09-18T10:15:36.323 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0010785460000000001 2024-09-18T10:15:36.323 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.323 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.323 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:36.324 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.973532+0000", 2024-09-18T10:15:36.324 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20679750599999999 2024-09-18T10:15:36.324 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.324 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.324 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:15:36.324 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:30.377254+0000", 2024-09-18T10:15:36.324 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.40372181400000001 2024-09-18T10:15:36.324 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.324 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.324 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:36.324 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:30.377375+0000", 2024-09-18T10:15:36.325 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000121037 2024-09-18T10:15:36.325 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:36.325 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:36.325 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:36.325 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.325 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.325 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:637 3.17s0 3:ead53b0f:::smithi18925896-25:head [append 0~475136] snapc 21=[20,1f,1e,1a,16,11,10] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-18T10:15:36.325 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:29.761784+0000", 2024-09-18T10:15:36.326 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 6.4751223790000001, 2024-09-18T10:15:36.326 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.61605535199999995, 2024-09-18T10:15:36.326 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:36.326 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:36.326 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:36.326 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:15:36.326 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:36.326 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 637 2024-09-18T10:15:36.326 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.326 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:36.326 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.326 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:36.327 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.761784+0000", 2024-09-18T10:15:36.327 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:36.327 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.327 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.327 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:36.327 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.761784+0000", 2024-09-18T10:15:36.327 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:36.327 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.328 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.328 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:36.328 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.761776+0000", 2024-09-18T10:15:36.328 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999919 2024-09-18T10:15:36.328 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.328 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.328 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:36.328 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.762424+0000", 2024-09-18T10:15:36.328 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00064803899999999999 2024-09-18T10:15:36.328 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.329 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.329 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:36.329 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.762455+0000", 2024-09-18T10:15:36.329 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1316000000000003e-05 2024-09-18T10:15:36.329 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.329 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.329 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:36.329 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.762471+0000", 2024-09-18T10:15:36.329 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.6174000000000001e-05 2024-09-18T10:15:36.329 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.329 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.329 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:36.329 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.767732+0000", 2024-09-18T10:15:36.329 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0052603190000000003 2024-09-18T10:15:36.330 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.330 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.330 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:36.330 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.767873+0000", 2024-09-18T10:15:36.330 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000140949 2024-09-18T10:15:36.330 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.330 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.330 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:36.330 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.768566+0000", 2024-09-18T10:15:36.330 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00069325799999999998 2024-09-18T10:15:36.330 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.330 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.331 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:36.331 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.973546+0000", 2024-09-18T10:15:36.331 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.204980144 2024-09-18T10:15:36.331 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.331 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.331 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:15:36.331 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:30.377732+0000", 2024-09-18T10:15:36.331 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.404185558 2024-09-18T10:15:36.331 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.331 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.332 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:36.332 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:30.377839+0000", 2024-09-18T10:15:36.332 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000107909 2024-09-18T10:15:36.332 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:36.332 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:36.332 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:36.332 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.332 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.332 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:638 3.17s0 3:ead53b0f:::smithi18925896-25:head [append 0~81920] snapc 21=[20,1f,1e,1a,16,11,10] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-18T10:15:36.332 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:29.762493+0000", 2024-09-18T10:15:36.332 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 6.4744133589999997, 2024-09-18T10:15:36.332 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.61563624900000002, 2024-09-18T10:15:36.332 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:36.333 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:36.333 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:36.333 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:15:36.333 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:36.333 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 638 2024-09-18T10:15:36.333 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.333 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:36.333 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.333 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:36.333 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.762493+0000", 2024-09-18T10:15:36.333 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:36.333 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.333 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.333 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:36.334 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.762493+0000", 2024-09-18T10:15:36.334 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:36.334 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.334 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.334 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:36.334 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.762492+0000", 2024-09-18T10:15:36.334 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:36.335 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.335 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.335 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:36.335 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.762566+0000", 2024-09-18T10:15:36.335 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.3977e-05 2024-09-18T10:15:36.335 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.335 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.335 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:36.336 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.762585+0000", 2024-09-18T10:15:36.336 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.9074000000000001e-05 2024-09-18T10:15:36.336 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.336 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.336 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:36.336 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.762595+0000", 2024-09-18T10:15:36.336 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0913999999999999e-05 2024-09-18T10:15:36.336 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.336 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.336 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:36.336 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.769730+0000", 2024-09-18T10:15:36.336 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0071346680000000003 2024-09-18T10:15:36.336 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.337 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.337 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:36.337 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.769816+0000", 2024-09-18T10:15:36.337 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.5422999999999998e-05 2024-09-18T10:15:36.337 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.337 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.337 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:36.337 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.770214+0000", 2024-09-18T10:15:36.337 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00039818999999999998 2024-09-18T10:15:36.337 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.337 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.338 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:36.338 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.973818+0000", 2024-09-18T10:15:36.338 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20360445499999999 2024-09-18T10:15:36.338 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.338 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.338 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:15:36.338 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:30.378035+0000", 2024-09-18T10:15:36.338 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.40421707600000001 2024-09-18T10:15:36.338 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.338 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.338 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:36.338 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:30.378129+0000", 2024-09-18T10:15:36.338 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.4056999999999997e-05 2024-09-18T10:15:36.338 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:36.339 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:36.339 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:36.339 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.339 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.339 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:639 3.17s0 3:ead53b0f:::smithi18925896-25:head [setxattr _header (57) in=7b] snapc 21=[20,1f,1e,1a,16,11,10] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-18T10:15:36.339 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:29.762609+0000", 2024-09-18T10:15:36.339 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 6.4742972050000001, 2024-09-18T10:15:36.339 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.61577652400000005, 2024-09-18T10:15:36.339 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:36.339 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:36.339 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:36.339 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:15:36.339 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:36.339 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 639 2024-09-18T10:15:36.340 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.340 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:36.340 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.340 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:36.340 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.762609+0000", 2024-09-18T10:15:36.340 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:36.340 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.340 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.340 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:36.340 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.762609+0000", 2024-09-18T10:15:36.340 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:36.340 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.340 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.340 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:36.341 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.762608+0000", 2024-09-18T10:15:36.341 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:15:36.341 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.341 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.341 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:36.341 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.762615+0000", 2024-09-18T10:15:36.341 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.2339999999999997e-06 2024-09-18T10:15:36.341 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.341 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.341 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:36.342 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.762628+0000", 2024-09-18T10:15:36.342 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2848e-05 2024-09-18T10:15:36.342 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.342 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.342 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:36.342 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.762636+0000", 2024-09-18T10:15:36.342 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.0970000000000006e-06 2024-09-18T10:15:36.342 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.342 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.342 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:36.342 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.870703+0000", 2024-09-18T10:15:36.342 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.108067121 2024-09-18T10:15:36.342 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.342 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.343 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:36.343 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.870949+0000", 2024-09-18T10:15:36.343 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00024526700000000002 2024-09-18T10:15:36.343 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.343 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.343 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:36.343 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.871876+0000", 2024-09-18T10:15:36.343 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00092680400000000004 2024-09-18T10:15:36.343 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.343 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.343 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:36.343 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.973821+0000", 2024-09-18T10:15:36.343 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10194552599999999 2024-09-18T10:15:36.343 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.343 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.344 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:15:36.344 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:30.378300+0000", 2024-09-18T10:15:36.344 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.40447920900000001 2024-09-18T10:15:36.344 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.344 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.344 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:36.344 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:30.378386+0000", 2024-09-18T10:15:36.344 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.5531999999999994e-05 2024-09-18T10:15:36.344 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:36.344 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:36.344 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:36.344 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.344 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.345 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.7s5 74/40 ECSubWrite(tid=168, reqid=osd.4.0:166, at_version=0'0, trim_to=0'0, roll_forward_to=74'47))", 2024-09-18T10:15:36.345 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:30.378493+0000", 2024-09-18T10:15:36.345 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 5.8584135609999999, 2024-09-18T10:15:36.345 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.605881475, 2024-09-18T10:15:36.345 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:36.345 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:15:36.345 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:36.345 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.345 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:36.345 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:30.378493+0000", 2024-09-18T10:15:36.345 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:36.345 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.345 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.345 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:36.346 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:30.378493+0000", 2024-09-18T10:15:36.346 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:36.346 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.346 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.346 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:36.346 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:30.378492+0000", 2024-09-18T10:15:36.346 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:36.346 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.346 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.346 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:36.346 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:30.378513+0000", 2024-09-18T10:15:36.346 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.1427e-05 2024-09-18T10:15:36.346 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.346 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.347 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:36.347 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:30.378550+0000", 2024-09-18T10:15:36.347 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.7351999999999997e-05 2024-09-18T10:15:36.347 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.347 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.347 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:36.347 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:30.478641+0000", 2024-09-18T10:15:36.347 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100090965 2024-09-18T10:15:36.347 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.347 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.347 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:36.347 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:30.882297+0000", 2024-09-18T10:15:36.347 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.40365594799999999 2024-09-18T10:15:36.347 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.348 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.348 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:36.348 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:30.882320+0000", 2024-09-18T10:15:36.348 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.3127999999999999e-05 2024-09-18T10:15:36.348 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.348 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.348 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:36.348 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:30.984327+0000", 2024-09-18T10:15:36.348 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10200699100000001 2024-09-18T10:15:36.348 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.348 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.348 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:36.348 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:30.984374+0000", 2024-09-18T10:15:36.348 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.6953000000000003e-05 2024-09-18T10:15:36.349 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:36.349 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:36.349 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:36.349 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:36.349 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:36.349 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-09-18T10:15:36.351 INFO:teuthology.orchestra.run.smithi029.stdout:osd.8: {} 2024-09-18T10:15:36.351 INFO:teuthology.orchestra.run.smithi029.stdout:osd.9: {} 2024-09-18T10:15:36.351 INFO:tasks.ceph.osd.12.smithi029.stderr:2024-09-18T10:15:36.345+0000 7f8e4e627700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 22190) UID: 0 2024-09-18T10:15:36.352 INFO:tasks.rados.rados.0.smithi189.stdout:282: done (1 left) 2024-09-18T10:15:36.352 INFO:tasks.rados.rados.0.smithi189.stdout:286: done (0 left) 2024-09-18T10:15:36.352 INFO:tasks.rados.rados.0.smithi189.stdout:append oid 25 current snap is 25 2024-09-18T10:15:36.352 INFO:tasks.rados.rados.0.smithi189.stdout:287: seq_num 115 ranges {4456448=770048,5226496=704512,5931008=458752} 2024-09-18T10:15:36.352 INFO:tasks.rados.rados.0.smithi189.stdout:287: writing smithi18925896-25 from 4456448 to 5226496 tid 1 2024-09-18T10:15:36.352 INFO:tasks.rados.rados.0.smithi189.stdout:287: writing smithi18925896-25 from 5226496 to 5931008 tid 2 2024-09-18T10:15:36.352 INFO:tasks.rados.rados.0.smithi189.stdout:287: writing smithi18925896-25 from 5931008 to 6389760 tid 3 2024-09-18T10:15:36.352 INFO:tasks.rados.rados.0.smithi189.stdout:288: delete oid 42 current snap is 25 2024-09-18T10:15:36.353 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_ops_in_flight 2024-09-18T10:15:36.447 INFO:tasks.rados.rados.0.smithi189.stdout:287: finishing write tid 1 to smithi18925896-25 2024-09-18T10:15:36.451 INFO:tasks.ceph.osd.8.smithi029.stderr:2024-09-18T10:15:36.445+0000 7f2555c27700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 22192) UID: 0 2024-09-18T10:15:36.457 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-09-18T10:15:36.457 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-09-18T10:15:36.457 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.457 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:701 3.2s0 3:427b2c0b:::smithi18925896-42:head [stat out=16b,delete] snapc 2a=[2a,29,28,26,20,1f,1e,1a,11] ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-09-18T10:15:36.457 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:36.339016+0000", 2024-09-18T10:15:36.457 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 0.117789189, 2024-09-18T10:15:36.457 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.117817265, 2024-09-18T10:15:36.457 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:36.457 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "started", 2024-09-18T10:15:36.458 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:36.458 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:15:36.458 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:36.458 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 701 2024-09-18T10:15:36.458 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.458 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:36.458 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.458 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:36.458 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:36.339016+0000", 2024-09-18T10:15:36.458 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:36.458 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.458 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.458 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:36.459 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:36.339016+0000", 2024-09-18T10:15:36.459 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:36.459 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.459 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.459 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:36.459 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:36.339013+0000", 2024-09-18T10:15:36.459 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999971 2024-09-18T10:15:36.459 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.459 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.459 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:36.459 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:36.339030+0000", 2024-09-18T10:15:36.459 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.6405999999999998e-05 2024-09-18T10:15:36.459 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.459 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.460 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:36.460 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:36.339059+0000", 2024-09-18T10:15:36.460 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9456000000000001e-05 2024-09-18T10:15:36.460 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.460 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.460 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:36.460 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:36.339078+0000", 2024-09-18T10:15:36.460 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.9026e-05 2024-09-18T10:15:36.460 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.460 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.460 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:36.460 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:36.439294+0000", 2024-09-18T10:15:36.460 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100215531 2024-09-18T10:15:36.460 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.461 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.461 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:36.461 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:36.439504+0000", 2024-09-18T10:15:36.461 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00021003499999999999 2024-09-18T10:15:36.461 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.461 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.461 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:36.461 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:36.441657+0000", 2024-09-18T10:15:36.461 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0021535550000000001 2024-09-18T10:15:36.461 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.461 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.461 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:36.461 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:36.445203+0000", 2024-09-18T10:15:36.461 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0035460769999999999 2024-09-18T10:15:36.462 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:36.462 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:36.462 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:36.462 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:36.462 INFO:teuthology.orchestra.run.smithi154.stdout: ], 2024-09-18T10:15:36.462 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 1 2024-09-18T10:15:36.462 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-09-18T10:15:36.466 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_blocked_ops 2024-09-18T10:15:36.545 INFO:tasks.rados.rados.0.smithi189.stdout:288: done (1 left) 2024-09-18T10:15:36.545 INFO:tasks.rados.rados.0.smithi189.stdout:289: snap_remove snap 21 2024-09-18T10:15:36.552 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-18T10:15:36.546+0000 7f14dcc5b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22398) UID: 0 2024-09-18T10:15:36.561 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-09-18T10:15:36.561 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-09-18T10:15:36.561 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-09-18T10:15:36.561 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-09-18T10:15:36.561 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-09-18T10:15:36.570 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_historic_ops 2024-09-18T10:15:36.572 INFO:teuthology.orchestra.run.smithi029.stdout:osd.10: {} 2024-09-18T10:15:36.581 INFO:tasks.rados.rados.0.smithi189.stdout:289: done (1 left) 2024-09-18T10:15:36.581 INFO:tasks.rados.rados.0.smithi189.stdout:290: delete oid 48 current snap is 25 2024-09-18T10:15:36.581 INFO:tasks.rados.rados.0.smithi189.stdout:287: finishing write tid 2 to smithi18925896-25 2024-09-18T10:15:36.582 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15] out_osds: [] dead_osds: [] live_osds: [0, 4, 8, 12, 1, 5, 9, 13, 2, 6, 10, 14, 3, 7, 11, 15] 2024-09-18T10:15:36.582 INFO:tasks.thrashosds.thrasher:choose_action: min_in 8 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T10:15:36.582 INFO:tasks.thrashosds.thrasher:Removing osd 8, in_osds are: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15] 2024-09-18T10:15:36.582 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 8 2024-09-18T10:15:36.592 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-18T10:15:36.600 INFO:teuthology.orchestra.run.smithi029.stdout:osd.11: {} 2024-09-18T10:15:36.622 INFO:teuthology.orchestra.run.smithi029.stdout:osd.12: {} 2024-09-18T10:15:36.650 INFO:teuthology.orchestra.run.smithi029.stdout:osd.13: {} 2024-09-18T10:15:36.653 INFO:tasks.ceph.osd.12.smithi029.stderr:2024-09-18T10:15:36.649+0000 7f8e4e627700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 22190) UID: 0 2024-09-18T10:15:36.662 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-09-18T10:15:36.663 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-09-18T10:15:36.663 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-09-18T10:15:36.663 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-09-18T10:15:36.663 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.663 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4341.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-18T10:15:36.663 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:35.324351+0000", 2024-09-18T10:15:36.663 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 61.334589227999999, 2024-09-18T10:15:36.663 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.36640713699999999, 2024-09-18T10:15:36.663 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:36.663 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:36.664 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:36.664 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4341", 2024-09-18T10:15:36.664 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.29:0/3001591505", 2024-09-18T10:15:36.664 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 1 2024-09-18T10:15:36.664 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.664 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:36.664 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.664 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:36.664 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.324351+0000", 2024-09-18T10:15:36.664 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:36.664 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.664 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.664 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:36.665 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.324351+0000", 2024-09-18T10:15:36.665 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:36.665 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.665 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.665 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:36.665 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.324348+0000", 2024-09-18T10:15:36.665 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999976 2024-09-18T10:15:36.665 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.665 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.665 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:36.665 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.324359+0000", 2024-09-18T10:15:36.665 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1048e-05 2024-09-18T10:15:36.666 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.666 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.666 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:36.666 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.324383+0000", 2024-09-18T10:15:36.666 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.3813000000000001e-05 2024-09-18T10:15:36.666 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.666 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.666 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:36.666 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.324421+0000", 2024-09-18T10:15:36.666 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8012000000000001e-05 2024-09-18T10:15:36.666 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.666 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.666 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:36.666 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.324548+0000", 2024-09-18T10:15:36.667 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00012702199999999999 2024-09-18T10:15:36.667 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.667 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.667 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for peered", 2024-09-18T10:15:36.667 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.324565+0000", 2024-09-18T10:15:36.667 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.6325e-05 2024-09-18T10:15:36.667 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.667 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.668 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:36.668 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.588380+0000", 2024-09-18T10:15:36.668 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.263815458 2024-09-18T10:15:36.668 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.668 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.668 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:36.668 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.588626+0000", 2024-09-18T10:15:36.668 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00024633499999999998 2024-09-18T10:15:36.668 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.668 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.668 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 5", 2024-09-18T10:15:36.668 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.588803+0000", 2024-09-18T10:15:36.668 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00017640599999999999 2024-09-18T10:15:36.669 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.669 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.669 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:15:36.669 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.589943+0000", 2024-09-18T10:15:36.669 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0011400049999999999 2024-09-18T10:15:36.669 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.669 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.669 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:15:36.669 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.690623+0000", 2024-09-18T10:15:36.669 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100680032 2024-09-18T10:15:36.669 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.669 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.670 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:15:36.670 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.690702+0000", 2024-09-18T10:15:36.670 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.9342e-05 2024-09-18T10:15:36.670 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.670 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.670 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:36.670 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.690758+0000", 2024-09-18T10:15:36.670 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5639e-05 2024-09-18T10:15:36.670 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:36.670 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:36.670 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:36.670 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.671 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.671 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(osd.4.21:1 3.2s0 3:46b2871c:::smithi18925896-34:head [assert-version v6,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:36.671 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:51.536666+0000", 2024-09-18T10:15:36.671 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 45.122273849000003, 2024-09-18T10:15:36.671 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.30016440500000002, 2024-09-18T10:15:36.671 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:36.671 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "started", 2024-09-18T10:15:36.671 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:36.671 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.671 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:36.671 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.536666+0000", 2024-09-18T10:15:36.671 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:36.671 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.672 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.672 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:36.672 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.536666+0000", 2024-09-18T10:15:36.672 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:36.672 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.672 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.672 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:36.672 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.536664+0000", 2024-09-18T10:15:36.672 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:36.672 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.672 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.673 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:36.673 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.536673+0000", 2024-09-18T10:15:36.673 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.6319999999999997e-06 2024-09-18T10:15:36.673 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.673 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.673 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:36.673 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.536688+0000", 2024-09-18T10:15:36.674 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.5058999999999999e-05 2024-09-18T10:15:36.674 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.674 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.674 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:36.674 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.536696+0000", 2024-09-18T10:15:36.674 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.8860000000000004e-06 2024-09-18T10:15:36.674 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.674 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.674 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:36.674 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.536868+0000", 2024-09-18T10:15:36.674 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000172475 2024-09-18T10:15:36.674 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.674 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.675 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:36.675 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.537125+0000", 2024-09-18T10:15:36.675 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000256906 2024-09-18T10:15:36.675 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.675 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.675 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:36.675 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.836830+0000", 2024-09-18T10:15:36.675 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.299705414 2024-09-18T10:15:36.675 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:36.675 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:36.675 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:36.675 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.675 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.676 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:264 3.2s0 3:46b2871c:::smithi18925896-34:head [append 0~606208] snapc 5=[5,4,3] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-18T10:15:36.676 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:56.739704+0000", 2024-09-18T10:15:36.676 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 39.919236243, 2024-09-18T10:15:36.676 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.79624036899999995, 2024-09-18T10:15:36.676 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:36.676 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:36.676 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:36.676 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:15:36.676 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:36.676 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 264 2024-09-18T10:15:36.676 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.676 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:36.676 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.677 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:36.677 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.739704+0000", 2024-09-18T10:15:36.677 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:36.677 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.677 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.677 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:36.677 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.739704+0000", 2024-09-18T10:15:36.677 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:36.677 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.677 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.677 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:36.677 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.739702+0000", 2024-09-18T10:15:36.677 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:36.678 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.678 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.678 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:36.678 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.740467+0000", 2024-09-18T10:15:36.678 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00076493100000000005 2024-09-18T10:15:36.678 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.678 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.678 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:36.678 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.740485+0000", 2024-09-18T10:15:36.679 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8516e-05 2024-09-18T10:15:36.679 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.679 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.679 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:36.679 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.740495+0000", 2024-09-18T10:15:36.679 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.2669999999999995e-06 2024-09-18T10:15:36.679 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.679 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.679 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:36.679 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.028948+0000", 2024-09-18T10:15:36.679 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.28845313500000003 2024-09-18T10:15:36.679 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.679 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.679 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for peered", 2024-09-18T10:15:36.680 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.028969+0000", 2024-09-18T10:15:36.680 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.1369000000000002e-05 2024-09-18T10:15:36.680 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.680 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.680 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:36.680 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.228324+0000", 2024-09-18T10:15:36.680 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.19935476699999999 2024-09-18T10:15:36.680 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.680 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.680 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:36.680 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.228558+0000", 2024-09-18T10:15:36.680 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00023393099999999999 2024-09-18T10:15:36.680 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.681 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.681 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:36.681 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.230390+0000", 2024-09-18T10:15:36.681 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018326970000000001 2024-09-18T10:15:36.681 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.681 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.681 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:36.681 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.239491+0000", 2024-09-18T10:15:36.681 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0091007950000000001 2024-09-18T10:15:36.681 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.681 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.681 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:15:36.682 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.535834+0000", 2024-09-18T10:15:36.682 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.29634234100000001 2024-09-18T10:15:36.682 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.682 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.682 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:36.682 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.535944+0000", 2024-09-18T10:15:36.682 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000110401 2024-09-18T10:15:36.682 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:36.682 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:36.682 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:36.682 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.682 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.682 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:265 3.2s0 3:46b2871c:::smithi18925896-34:head [append 0~376832] snapc 5=[5,4,3] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-18T10:15:36.683 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:56.741882+0000", 2024-09-18T10:15:36.683 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 39.917058220999998, 2024-09-18T10:15:36.683 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.80071129500000005, 2024-09-18T10:15:36.683 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:36.683 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:36.683 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:36.683 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:15:36.683 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:36.683 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 265 2024-09-18T10:15:36.683 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.683 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:36.683 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.683 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:36.684 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.741882+0000", 2024-09-18T10:15:36.684 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:36.684 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.684 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.684 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:36.684 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.741882+0000", 2024-09-18T10:15:36.684 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:36.684 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.684 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.684 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:36.684 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.741880+0000", 2024-09-18T10:15:36.684 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:36.684 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.685 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.685 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:36.685 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742251+0000", 2024-09-18T10:15:36.685 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00037050499999999998 2024-09-18T10:15:36.685 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.685 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.685 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:36.685 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742264+0000", 2024-09-18T10:15:36.685 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2787e-05 2024-09-18T10:15:36.685 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.685 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.685 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:36.685 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742269+0000", 2024-09-18T10:15:36.686 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.0509999999999996e-06 2024-09-18T10:15:36.686 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.686 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.686 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:36.686 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.234342+0000", 2024-09-18T10:15:36.686 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.492073712 2024-09-18T10:15:36.686 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.686 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.686 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:36.686 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.234556+0000", 2024-09-18T10:15:36.686 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00021380699999999999 2024-09-18T10:15:36.686 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.686 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.686 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:36.687 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.236316+0000", 2024-09-18T10:15:36.687 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0017597100000000001 2024-09-18T10:15:36.687 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.687 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.687 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:36.687 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.241476+0000", 2024-09-18T10:15:36.687 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0051604449999999996 2024-09-18T10:15:36.687 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.687 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.687 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:15:36.687 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.542502+0000", 2024-09-18T10:15:36.687 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.30102573700000002 2024-09-18T10:15:36.687 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.687 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.688 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:36.688 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.542593+0000", 2024-09-18T10:15:36.688 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.0859000000000005e-05 2024-09-18T10:15:36.688 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:36.688 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:36.688 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:36.688 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.688 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.688 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:266 3.2s0 3:46b2871c:::smithi18925896-34:head [setxattr _header (56) in=7b] snapc 5=[5,4,3] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-18T10:15:36.688 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:56.742274+0000", 2024-09-18T10:15:36.688 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 39.916665590000001, 2024-09-18T10:15:36.688 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.80056005299999999, 2024-09-18T10:15:36.689 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:36.689 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:36.689 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:36.689 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:15:36.689 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:36.689 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 266 2024-09-18T10:15:36.689 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.689 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:36.689 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.689 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:36.689 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742274+0000", 2024-09-18T10:15:36.689 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:36.689 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.689 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.690 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:36.690 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742274+0000", 2024-09-18T10:15:36.690 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:36.690 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.690 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.690 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:36.690 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742274+0000", 2024-09-18T10:15:36.690 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:36.690 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.690 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.690 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:36.690 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742278+0000", 2024-09-18T10:15:36.690 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.0840000000000002e-06 2024-09-18T10:15:36.690 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.691 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.691 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:36.691 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742284+0000", 2024-09-18T10:15:36.691 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.1090000000000003e-06 2024-09-18T10:15:36.691 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.691 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.691 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:36.691 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742289+0000", 2024-09-18T10:15:36.691 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.4890000000000003e-06 2024-09-18T10:15:36.692 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.692 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.692 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:36.692 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.237660+0000", 2024-09-18T10:15:36.692 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.49537068299999998 2024-09-18T10:15:36.692 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.692 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.692 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:36.692 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.237924+0000", 2024-09-18T10:15:36.692 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00026387300000000003 2024-09-18T10:15:36.692 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.692 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.693 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:36.693 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.238979+0000", 2024-09-18T10:15:36.693 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0010549380000000001 2024-09-18T10:15:36.693 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.693 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.693 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:36.693 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.243571+0000", 2024-09-18T10:15:36.693 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0045919749999999999 2024-09-18T10:15:36.693 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.693 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.693 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:15:36.693 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.542742+0000", 2024-09-18T10:15:36.693 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.29917095999999999 2024-09-18T10:15:36.693 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.694 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.694 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:36.694 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.542834+0000", 2024-09-18T10:15:36.694 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.2536999999999998e-05 2024-09-18T10:15:36.694 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:36.694 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:36.694 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:36.694 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.694 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.694 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:267 3.2s0 3:46b2871c:::smithi18925896-34:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e41)", 2024-09-18T10:15:36.694 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:56.742292+0000", 2024-09-18T10:15:36.694 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 39.916647752000003, 2024-09-18T10:15:36.694 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.98973012599999999, 2024-09-18T10:15:36.695 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:36.695 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "started", 2024-09-18T10:15:36.695 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:36.695 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:15:36.695 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:36.695 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 267 2024-09-18T10:15:36.695 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.695 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:36.695 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.695 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:36.695 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742292+0000", 2024-09-18T10:15:36.695 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:36.695 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.696 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.696 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:36.696 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742292+0000", 2024-09-18T10:15:36.696 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:36.696 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.696 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.696 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:36.696 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742292+0000", 2024-09-18T10:15:36.696 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:36.696 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.696 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.696 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:36.696 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742294+0000", 2024-09-18T10:15:36.696 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.103e-06 2024-09-18T10:15:36.697 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.697 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.697 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:36.697 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742298+0000", 2024-09-18T10:15:36.697 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8070000000000001e-06 2024-09-18T10:15:36.697 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.697 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.697 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:36.697 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742300+0000", 2024-09-18T10:15:36.697 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5890000000000001e-06 2024-09-18T10:15:36.697 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.697 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.697 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:36.697 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.239711+0000", 2024-09-18T10:15:36.698 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.497410625 2024-09-18T10:15:36.698 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.698 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.698 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for rw locks", 2024-09-18T10:15:36.698 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.239786+0000", 2024-09-18T10:15:36.698 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.4827999999999996e-05 2024-09-18T10:15:36.698 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.698 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.698 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:36.698 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.543234+0000", 2024-09-18T10:15:36.698 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.30344799099999997 2024-09-18T10:15:36.698 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.698 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.699 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:36.699 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.543335+0000", 2024-09-18T10:15:36.699 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000101625 2024-09-18T10:15:36.699 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.699 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.699 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:36.699 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.732022+0000", 2024-09-18T10:15:36.699 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.18868686700000001 2024-09-18T10:15:36.699 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:36.699 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:36.699 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:36.699 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.699 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.699 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.es1 41/32 ECSubWrite(tid=16, reqid=osd.11.0:15, at_version=0'0, trim_to=0'0, roll_forward_to=41'11))", 2024-09-18T10:15:36.700 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:56.930285+0000", 2024-09-18T10:15:36.700 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 39.728654839000001, 2024-09-18T10:15:36.700 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.211817538, 2024-09-18T10:15:36.700 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:36.700 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:15:36.700 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:36.700 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.700 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:36.700 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.930285+0000", 2024-09-18T10:15:36.700 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:36.700 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.700 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.700 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:36.701 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.930285+0000", 2024-09-18T10:15:36.701 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:36.701 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.701 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.701 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:36.701 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.930284+0000", 2024-09-18T10:15:36.701 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:36.701 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.701 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.701 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:36.701 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.930313+0000", 2024-09-18T10:15:36.701 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9394999999999999e-05 2024-09-18T10:15:36.701 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.702 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.702 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:36.702 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.930337+0000", 2024-09-18T10:15:36.702 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4239e-05 2024-09-18T10:15:36.702 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.702 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.702 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:36.702 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.930352+0000", 2024-09-18T10:15:36.702 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4577e-05 2024-09-18T10:15:36.702 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.702 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.702 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:36.702 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.030572+0000", 2024-09-18T10:15:36.702 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100219771 2024-09-18T10:15:36.703 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.703 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.703 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:36.703 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.030604+0000", 2024-09-18T10:15:36.703 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2654999999999998e-05 2024-09-18T10:15:36.703 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.703 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.703 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:36.703 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.142047+0000", 2024-09-18T10:15:36.703 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.111442992 2024-09-18T10:15:36.703 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.703 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.703 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:36.703 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.142103+0000", 2024-09-18T10:15:36.704 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5291999999999998e-05 2024-09-18T10:15:36.704 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:36.704 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:36.704 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:36.704 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.704 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.704 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(osd.7.22:2 3.2s0 3:427b2c0b:::smithi18925896-42:head [assert-version v8,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-18T10:15:36.704 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:08.756861+0000", 2024-09-18T10:15:36.704 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 27.902079025999999, 2024-09-18T10:15:36.704 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.27903143600000002, 2024-09-18T10:15:36.704 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:36.704 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "started", 2024-09-18T10:15:36.704 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:36.705 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.705 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:36.705 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:08.756861+0000", 2024-09-18T10:15:36.705 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:36.705 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.705 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.705 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:36.705 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:08.756861+0000", 2024-09-18T10:15:36.705 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:36.705 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.705 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.705 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:36.705 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:08.756859+0000", 2024-09-18T10:15:36.706 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999976 2024-09-18T10:15:36.706 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.706 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.706 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:36.706 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:08.756869+0000", 2024-09-18T10:15:36.706 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0389e-05 2024-09-18T10:15:36.706 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.706 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.706 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:36.706 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:08.756887+0000", 2024-09-18T10:15:36.706 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8094e-05 2024-09-18T10:15:36.706 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.706 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.706 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:36.707 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:08.756896+0000", 2024-09-18T10:15:36.707 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.9290000000000004e-06 2024-09-18T10:15:36.707 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.707 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.707 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:36.707 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:08.757008+0000", 2024-09-18T10:15:36.707 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00011184 2024-09-18T10:15:36.707 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.707 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.707 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op must wait for map", 2024-09-18T10:15:36.707 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:08.757025+0000", 2024-09-18T10:15:36.707 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.6912999999999999e-05 2024-09-18T10:15:36.707 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.707 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.708 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:36.708 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:08.931998+0000", 2024-09-18T10:15:36.708 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.174973289 2024-09-18T10:15:36.708 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.708 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.708 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:36.708 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:08.932274+0000", 2024-09-18T10:15:36.708 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00027573799999999998 2024-09-18T10:15:36.708 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.708 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.708 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:36.708 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:09.035892+0000", 2024-09-18T10:15:36.708 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10361846700000001 2024-09-18T10:15:36.709 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:36.709 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:36.709 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:36.709 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.709 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.709 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:452 3.2s0 3:47202842:::smithi18925896-22:head [delete] snapc 13=[12,11,10,f,e,b] ondisk+write+known_if_redirected+full_force+supports_pool_eio e58)", 2024-09-18T10:15:36.709 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:13.551955+0000", 2024-09-18T10:15:36.709 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 23.106984739000001, 2024-09-18T10:15:36.709 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.31127974800000002, 2024-09-18T10:15:36.709 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:36.709 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:36.709 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:36.709 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:15:36.709 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:36.710 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 452 2024-09-18T10:15:36.710 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.710 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:36.710 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.710 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:36.710 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.551955+0000", 2024-09-18T10:15:36.710 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:36.710 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.710 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.710 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:36.710 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.551955+0000", 2024-09-18T10:15:36.710 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:36.711 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.711 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.711 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:36.711 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.551952+0000", 2024-09-18T10:15:36.711 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999967 2024-09-18T10:15:36.711 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.711 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.711 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:36.711 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.551970+0000", 2024-09-18T10:15:36.711 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8366999999999999e-05 2024-09-18T10:15:36.711 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.711 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.712 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:36.712 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.552016+0000", 2024-09-18T10:15:36.712 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.5868000000000002e-05 2024-09-18T10:15:36.712 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.712 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.712 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:36.712 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.552037+0000", 2024-09-18T10:15:36.712 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.1311999999999999e-05 2024-09-18T10:15:36.712 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.712 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.712 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:36.712 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.560400+0000", 2024-09-18T10:15:36.712 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0083625920000000003 2024-09-18T10:15:36.712 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.713 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.713 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op must wait for map", 2024-09-18T10:15:36.713 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.560411+0000", 2024-09-18T10:15:36.713 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1043000000000001e-05 2024-09-18T10:15:36.713 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.713 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.713 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:36.713 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.562108+0000", 2024-09-18T10:15:36.713 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0016968479999999999 2024-09-18T10:15:36.713 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.713 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.713 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:36.713 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.562286+0000", 2024-09-18T10:15:36.713 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00017818599999999999 2024-09-18T10:15:36.714 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.714 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.714 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:36.714 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.563336+0000", 2024-09-18T10:15:36.714 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001050051 2024-09-18T10:15:36.714 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.714 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.714 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:36.714 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.565490+0000", 2024-09-18T10:15:36.714 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0021545119999999999 2024-09-18T10:15:36.714 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.714 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.714 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:15:36.714 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.863047+0000", 2024-09-18T10:15:36.715 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.29755694199999999 2024-09-18T10:15:36.715 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.715 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.715 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:36.715 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.863235+0000", 2024-09-18T10:15:36.715 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000187524 2024-09-18T10:15:36.715 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:36.715 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:36.715 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:36.715 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.715 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.715 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDPGPushReply(3.15s0 61/60 [PushReplyOp(3:acecabac:::smithi18925896-4:head)])", 2024-09-18T10:15:36.715 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:17.756036+0000", 2024-09-18T10:15:36.716 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 18.902904036999999, 2024-09-18T10:15:36.716 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.30209286600000002, 2024-09-18T10:15:36.716 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:36.716 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:15:36.716 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:36.716 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.716 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:36.716 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.756036+0000", 2024-09-18T10:15:36.716 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:36.716 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.716 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.716 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:36.716 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.756036+0000", 2024-09-18T10:15:36.717 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:36.717 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.717 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.717 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:36.717 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.756035+0000", 2024-09-18T10:15:36.717 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:15:36.717 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.717 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.717 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:36.717 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.756042+0000", 2024-09-18T10:15:36.717 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.6630000000000004e-06 2024-09-18T10:15:36.717 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.718 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.718 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:36.718 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.756057+0000", 2024-09-18T10:15:36.718 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.5719e-05 2024-09-18T10:15:36.718 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.718 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.718 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:36.718 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.756068+0000", 2024-09-18T10:15:36.718 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0207000000000001e-05 2024-09-18T10:15:36.718 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.718 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.718 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:36.718 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:18.058067+0000", 2024-09-18T10:15:36.719 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.30199969599999998 2024-09-18T10:15:36.719 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.719 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.719 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:36.719 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:18.058129+0000", 2024-09-18T10:15:36.719 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.1303999999999994e-05 2024-09-18T10:15:36.719 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:36.719 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:36.719 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:36.719 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.719 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.719 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:496 3.2s0 3:44cb9e27:::smithi18925896-5:head [read 0~81920,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T10:15:36.719 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:17.889532+0000", 2024-09-18T10:15:36.720 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 18.769408132999999, 2024-09-18T10:15:36.720 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.32688004999999998, 2024-09-18T10:15:36.720 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:36.720 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "started", 2024-09-18T10:15:36.720 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:36.720 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:15:36.720 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:36.720 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 496 2024-09-18T10:15:36.720 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.720 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:36.720 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.720 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:36.721 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889532+0000", 2024-09-18T10:15:36.721 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:36.721 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.721 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.721 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:36.721 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889532+0000", 2024-09-18T10:15:36.721 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:36.721 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.721 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.721 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:36.721 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889529+0000", 2024-09-18T10:15:36.721 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999971 2024-09-18T10:15:36.721 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.722 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.722 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:36.722 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889546+0000", 2024-09-18T10:15:36.722 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.6690000000000001e-05 2024-09-18T10:15:36.722 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.722 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.722 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:36.722 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889580+0000", 2024-09-18T10:15:36.722 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.4638999999999998e-05 2024-09-18T10:15:36.722 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.722 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.722 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:36.722 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889596+0000", 2024-09-18T10:15:36.723 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.5571999999999999e-05 2024-09-18T10:15:36.723 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.723 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.723 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:36.723 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.989817+0000", 2024-09-18T10:15:36.723 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100221713 2024-09-18T10:15:36.723 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.723 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.723 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op must wait for map", 2024-09-18T10:15:36.723 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.989836+0000", 2024-09-18T10:15:36.723 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8516e-05 2024-09-18T10:15:36.723 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.723 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.723 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:36.724 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.990420+0000", 2024-09-18T10:15:36.724 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000583918 2024-09-18T10:15:36.724 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.724 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.724 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:36.724 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.990693+0000", 2024-09-18T10:15:36.724 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00027301499999999999 2024-09-18T10:15:36.724 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.724 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.724 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:36.724 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:18.216412+0000", 2024-09-18T10:15:36.724 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.22571880699999999 2024-09-18T10:15:36.725 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:36.725 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:36.725 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:36.725 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.725 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.725 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:497 3.2s0 3:44cb9e27:::smithi18925896-5:head [read 0~81920,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T10:15:36.725 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:17.889627+0000", 2024-09-18T10:15:36.725 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 18.769313216, 2024-09-18T10:15:36.725 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.32738385800000003, 2024-09-18T10:15:36.726 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:36.726 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "started", 2024-09-18T10:15:36.726 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:36.726 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:15:36.726 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:36.726 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 497 2024-09-18T10:15:36.726 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.726 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:36.726 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.726 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:36.726 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889627+0000", 2024-09-18T10:15:36.726 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:36.726 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.727 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.727 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:36.727 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889627+0000", 2024-09-18T10:15:36.727 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:36.727 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.727 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.727 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:36.727 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889625+0000", 2024-09-18T10:15:36.727 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:15:36.727 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.727 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.727 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:36.727 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889633+0000", 2024-09-18T10:15:36.727 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.4170000000000003e-06 2024-09-18T10:15:36.728 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.728 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.728 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:36.728 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889664+0000", 2024-09-18T10:15:36.728 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0589000000000001e-05 2024-09-18T10:15:36.728 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.728 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.728 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:36.728 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889672+0000", 2024-09-18T10:15:36.728 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.3729999999999993e-06 2024-09-18T10:15:36.728 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.728 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.728 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:36.729 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.990017+0000", 2024-09-18T10:15:36.729 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100345608 2024-09-18T10:15:36.729 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.729 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.729 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:36.729 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.990046+0000", 2024-09-18T10:15:36.729 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8912999999999999e-05 2024-09-18T10:15:36.729 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.729 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.729 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:36.729 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.991459+0000", 2024-09-18T10:15:36.729 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0014122340000000001 2024-09-18T10:15:36.729 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.729 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.730 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:36.730 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.991631+0000", 2024-09-18T10:15:36.730 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000172029 2024-09-18T10:15:36.730 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.730 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.730 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:36.730 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:18.217010+0000", 2024-09-18T10:15:36.730 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.22537980499999999 2024-09-18T10:15:36.731 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:36.731 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:36.731 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:36.731 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.731 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.731 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:498 3.2s0 3:44cb9e27:::smithi18925896-5:head [read 0~81920,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T10:15:36.731 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:17.889709+0000", 2024-09-18T10:15:36.731 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 18.769230858, 2024-09-18T10:15:36.731 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.32775850600000001, 2024-09-18T10:15:36.731 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:36.731 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "started", 2024-09-18T10:15:36.731 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:36.731 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:15:36.732 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:36.732 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 498 2024-09-18T10:15:36.732 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.732 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:36.732 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.732 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:36.732 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889709+0000", 2024-09-18T10:15:36.732 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:36.732 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.732 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.732 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:36.732 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889709+0000", 2024-09-18T10:15:36.732 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:36.732 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.733 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.733 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:36.733 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889708+0000", 2024-09-18T10:15:36.733 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:15:36.733 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.733 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.733 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:36.733 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889716+0000", 2024-09-18T10:15:36.733 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.0989999999999999e-06 2024-09-18T10:15:36.733 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.733 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.733 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:36.733 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889732+0000", 2024-09-18T10:15:36.734 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.6093e-05 2024-09-18T10:15:36.734 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.734 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.734 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:36.734 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889742+0000", 2024-09-18T10:15:36.734 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.4599999999999992e-06 2024-09-18T10:15:36.734 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.734 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.734 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:36.734 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.992175+0000", 2024-09-18T10:15:36.734 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10243377200000001 2024-09-18T10:15:36.734 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.734 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.734 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:36.735 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.992333+0000", 2024-09-18T10:15:36.735 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00015753499999999999 2024-09-18T10:15:36.735 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.735 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.735 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:36.735 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:18.217467+0000", 2024-09-18T10:15:36.735 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.22513459999999999 2024-09-18T10:15:36.735 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:36.735 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:36.735 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:36.735 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.735 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.735 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:516 3.2s0 3:44cb9e27:::smithi18925896-5:head [append 0~425984] snapc 1a=[1a,19,18,16,12,11,10,f,e] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-18T10:15:36.735 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:19.430973+0000", 2024-09-18T10:15:36.736 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 17.227966468999998, 2024-09-18T10:15:36.736 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.59812562800000002, 2024-09-18T10:15:36.736 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:36.736 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:36.736 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:36.736 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:15:36.736 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:36.736 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 516 2024-09-18T10:15:36.736 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.736 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:36.736 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.736 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:36.736 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.430973+0000", 2024-09-18T10:15:36.737 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:36.737 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.737 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.737 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:36.737 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.430973+0000", 2024-09-18T10:15:36.737 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:36.737 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.737 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.737 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:36.737 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.430970+0000", 2024-09-18T10:15:36.737 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999967 2024-09-18T10:15:36.737 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.737 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.737 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:36.737 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.431995+0000", 2024-09-18T10:15:36.738 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001024896 2024-09-18T10:15:36.738 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.738 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.738 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:36.738 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.432030+0000", 2024-09-18T10:15:36.738 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5241999999999999e-05 2024-09-18T10:15:36.738 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.738 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.738 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:36.738 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.432044+0000", 2024-09-18T10:15:36.738 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.411e-05 2024-09-18T10:15:36.738 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.738 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.739 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:36.739 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.432226+0000", 2024-09-18T10:15:36.739 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000181841 2024-09-18T10:15:36.739 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.739 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.739 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:36.739 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.432421+0000", 2024-09-18T10:15:36.739 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000194527 2024-09-18T10:15:36.739 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.739 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.739 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:36.739 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.434150+0000", 2024-09-18T10:15:36.739 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001729742 2024-09-18T10:15:36.740 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.740 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.740 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:36.740 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.436513+0000", 2024-09-18T10:15:36.740 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0023622299999999999 2024-09-18T10:15:36.740 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.740 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.740 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:15:36.740 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:20.029005+0000", 2024-09-18T10:15:36.740 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.59249230100000005 2024-09-18T10:15:36.740 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.740 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.740 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:36.740 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:20.029099+0000", 2024-09-18T10:15:36.741 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.4042000000000005e-05 2024-09-18T10:15:36.741 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:36.741 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:36.741 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:36.741 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.741 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.741 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:517 3.2s0 3:44cb9e27:::smithi18925896-5:head [append 0~786432] snapc 1a=[1a,19,18,16,12,11,10,f,e] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-18T10:15:36.741 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:19.439831+0000", 2024-09-18T10:15:36.741 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 17.219108930000001, 2024-09-18T10:15:36.741 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.58958403800000003, 2024-09-18T10:15:36.741 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:36.741 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:36.741 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:36.742 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:15:36.742 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:36.742 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 517 2024-09-18T10:15:36.742 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.742 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:36.742 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.742 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:36.742 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.439831+0000", 2024-09-18T10:15:36.742 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:36.742 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.742 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.742 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:36.742 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.439831+0000", 2024-09-18T10:15:36.742 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:36.743 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.743 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.743 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:36.743 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.439828+0000", 2024-09-18T10:15:36.743 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999976 2024-09-18T10:15:36.743 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.743 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.743 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:36.743 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.441109+0000", 2024-09-18T10:15:36.743 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001280988 2024-09-18T10:15:36.743 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.743 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.743 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:36.744 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.441168+0000", 2024-09-18T10:15:36.744 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.8869000000000002e-05 2024-09-18T10:15:36.744 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.744 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.744 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:36.744 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.541302+0000", 2024-09-18T10:15:36.744 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100134055 2024-09-18T10:15:36.744 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.744 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.744 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:36.744 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.637962+0000", 2024-09-18T10:15:36.744 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.096659682999999996 2024-09-18T10:15:36.744 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.744 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.745 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:36.745 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.638164+0000", 2024-09-18T10:15:36.745 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00020226700000000001 2024-09-18T10:15:36.745 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.745 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.745 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:36.745 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.639933+0000", 2024-09-18T10:15:36.745 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001768501 2024-09-18T10:15:36.745 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.745 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.745 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:36.745 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.646572+0000", 2024-09-18T10:15:36.745 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.006639069 2024-09-18T10:15:36.746 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.746 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.746 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:15:36.746 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:20.029341+0000", 2024-09-18T10:15:36.746 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.38276939599999998 2024-09-18T10:15:36.746 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.746 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.746 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:36.746 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:20.029415+0000", 2024-09-18T10:15:36.746 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.3726000000000005e-05 2024-09-18T10:15:36.746 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:36.746 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:36.746 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:36.746 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.747 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.747 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:518 3.2s0 3:44cb9e27:::smithi18925896-5:head [append 0~376832] snapc 1a=[1a,19,18,16,12,11,10,f,e] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-18T10:15:36.747 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:19.541352+0000", 2024-09-18T10:15:36.747 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 17.117587892, 2024-09-18T10:15:36.747 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.48828847600000003, 2024-09-18T10:15:36.747 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:36.747 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:36.747 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:36.747 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:15:36.747 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:36.747 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 518 2024-09-18T10:15:36.747 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.747 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:36.748 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.748 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:36.748 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.541352+0000", 2024-09-18T10:15:36.748 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:36.748 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.748 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.748 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:36.748 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.541352+0000", 2024-09-18T10:15:36.748 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:36.748 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.748 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.748 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:36.749 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.541350+0000", 2024-09-18T10:15:36.749 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:36.749 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.749 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.749 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:36.749 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.541834+0000", 2024-09-18T10:15:36.749 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00048402500000000002 2024-09-18T10:15:36.749 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.749 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.749 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:36.749 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.541861+0000", 2024-09-18T10:15:36.749 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.7333000000000001e-05 2024-09-18T10:15:36.749 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.749 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.750 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:36.750 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.541875+0000", 2024-09-18T10:15:36.750 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4025999999999999e-05 2024-09-18T10:15:36.750 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.750 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.750 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:36.750 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.641684+0000", 2024-09-18T10:15:36.750 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.099808872000000007 2024-09-18T10:15:36.750 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.750 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.750 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:36.750 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.641873+0000", 2024-09-18T10:15:36.750 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00018933200000000001 2024-09-18T10:15:36.751 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.751 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.751 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:36.751 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.643401+0000", 2024-09-18T10:15:36.751 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001527104 2024-09-18T10:15:36.751 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.751 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.751 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:36.751 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.646854+0000", 2024-09-18T10:15:36.751 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0034532069999999998 2024-09-18T10:15:36.751 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.751 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.751 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:15:36.751 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:20.029569+0000", 2024-09-18T10:15:36.752 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.38271509599999998 2024-09-18T10:15:36.752 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.752 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.752 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:36.752 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:20.029640+0000", 2024-09-18T10:15:36.752 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.1594000000000006e-05 2024-09-18T10:15:36.752 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:36.753 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:36.753 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:36.753 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.753 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.753 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:519 3.2s0 3:44cb9e27:::smithi18925896-5:head [setxattr _header (56) in=7b] snapc 1a=[1a,19,18,16,12,11,10,f,e] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-18T10:15:36.753 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:19.541886+0000", 2024-09-18T10:15:36.753 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 17.117054204999999, 2024-09-18T10:15:36.753 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.488165768, 2024-09-18T10:15:36.753 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:36.753 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:36.753 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:36.753 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:15:36.753 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:36.754 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 519 2024-09-18T10:15:36.754 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.754 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:36.754 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.754 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:36.754 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.541886+0000", 2024-09-18T10:15:36.754 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:36.754 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.754 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.754 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:36.755 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.541886+0000", 2024-09-18T10:15:36.755 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:36.755 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.755 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.755 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:36.755 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.541884+0000", 2024-09-18T10:15:36.755 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:36.755 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.755 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.755 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:36.755 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.541890+0000", 2024-09-18T10:15:36.755 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.0589999999999999e-06 2024-09-18T10:15:36.755 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.756 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.756 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:36.756 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.541899+0000", 2024-09-18T10:15:36.756 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.02e-06 2024-09-18T10:15:36.756 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.756 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.756 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:36.756 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.541905+0000", 2024-09-18T10:15:36.756 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.9399999999999999e-06 2024-09-18T10:15:36.756 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.756 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.756 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:36.756 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.644561+0000", 2024-09-18T10:15:36.757 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.102655408 2024-09-18T10:15:36.757 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.757 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.757 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:36.757 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.644869+0000", 2024-09-18T10:15:36.757 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00030766700000000002 2024-09-18T10:15:36.757 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.757 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.757 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:36.757 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.645745+0000", 2024-09-18T10:15:36.757 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000876314 2024-09-18T10:15:36.757 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.757 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.757 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:36.758 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.647092+0000", 2024-09-18T10:15:36.758 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0013469269999999999 2024-09-18T10:15:36.758 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.758 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.758 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:15:36.758 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:20.029901+0000", 2024-09-18T10:15:36.758 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.38280895199999998 2024-09-18T10:15:36.758 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.758 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.758 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:36.758 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:20.030051+0000", 2024-09-18T10:15:36.758 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00015067699999999999 2024-09-18T10:15:36.758 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:36.759 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:36.759 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:36.759 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.759 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.759 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:572 3.2s0 3:46b2871c:::smithi18925896-34:head [read 0~557056,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-18T10:15:36.759 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:24.942246+0000", 2024-09-18T10:15:36.759 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 11.716693448000001, 2024-09-18T10:15:36.759 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.262248706, 2024-09-18T10:15:36.759 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:36.759 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "started", 2024-09-18T10:15:36.759 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:36.759 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:15:36.759 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:36.760 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 572 2024-09-18T10:15:36.760 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.760 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:36.760 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.760 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:36.760 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.942246+0000", 2024-09-18T10:15:36.760 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:36.760 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.760 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.761 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:36.761 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.942246+0000", 2024-09-18T10:15:36.761 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:36.761 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.761 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.761 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:36.761 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.942243+0000", 2024-09-18T10:15:36.761 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999971 2024-09-18T10:15:36.761 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.761 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.761 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:36.761 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.942262+0000", 2024-09-18T10:15:36.761 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8303000000000001e-05 2024-09-18T10:15:36.761 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.762 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.762 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:36.762 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.942298+0000", 2024-09-18T10:15:36.762 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.6396000000000002e-05 2024-09-18T10:15:36.762 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.762 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.762 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:36.762 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.942313+0000", 2024-09-18T10:15:36.762 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.5421999999999999e-05 2024-09-18T10:15:36.762 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.762 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.762 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:36.762 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.942453+0000", 2024-09-18T10:15:36.763 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00014003900000000001 2024-09-18T10:15:36.763 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.763 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.763 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op must wait for map", 2024-09-18T10:15:36.763 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.942471+0000", 2024-09-18T10:15:36.763 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7859999999999998e-05 2024-09-18T10:15:36.763 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.763 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.763 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:36.763 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:25.100684+0000", 2024-09-18T10:15:36.763 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.15821273599999999 2024-09-18T10:15:36.763 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.763 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.763 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:36.764 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:25.100921+0000", 2024-09-18T10:15:36.764 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000236963 2024-09-18T10:15:36.764 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.764 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.764 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:36.764 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:25.204495+0000", 2024-09-18T10:15:36.764 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10357403 2024-09-18T10:15:36.764 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:36.764 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:36.764 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:36.764 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.764 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.764 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:573 3.2s0 3:46b2871c:::smithi18925896-34:head [read 0~557056,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-18T10:15:36.765 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:24.942342+0000", 2024-09-18T10:15:36.765 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 11.716597491, 2024-09-18T10:15:36.765 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.36312128700000001, 2024-09-18T10:15:36.765 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:36.765 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "started", 2024-09-18T10:15:36.765 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:36.765 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:15:36.765 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:36.765 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 573 2024-09-18T10:15:36.765 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.765 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:36.765 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.765 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:36.765 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.942342+0000", 2024-09-18T10:15:36.766 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:36.766 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.766 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.766 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:36.766 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.942342+0000", 2024-09-18T10:15:36.766 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:36.766 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.766 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.766 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:36.766 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.942341+0000", 2024-09-18T10:15:36.766 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:15:36.766 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.766 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.767 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:36.767 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.942363+0000", 2024-09-18T10:15:36.767 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.1406000000000001e-05 2024-09-18T10:15:36.767 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.767 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.767 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:36.767 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.942374+0000", 2024-09-18T10:15:36.767 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1466e-05 2024-09-18T10:15:36.767 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.767 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.767 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:36.767 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.942381+0000", 2024-09-18T10:15:36.767 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.9530000000000002e-06 2024-09-18T10:15:36.767 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.768 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.768 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:36.768 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.942654+0000", 2024-09-18T10:15:36.768 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000272692 2024-09-18T10:15:36.768 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.768 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.768 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:36.768 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.942672+0000", 2024-09-18T10:15:36.768 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7643e-05 2024-09-18T10:15:36.768 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.768 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.768 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:36.768 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:25.101730+0000", 2024-09-18T10:15:36.769 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.15905815800000001 2024-09-18T10:15:36.769 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.769 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.769 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:36.769 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:25.101881+0000", 2024-09-18T10:15:36.769 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000151299 2024-09-18T10:15:36.769 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.769 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.769 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:36.769 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:25.305464+0000", 2024-09-18T10:15:36.769 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.203582656 2024-09-18T10:15:36.769 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:36.769 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:36.770 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:36.770 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.770 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.770 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:574 3.2s0 3:46b2871c:::smithi18925896-34:head [read 0~557056,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-18T10:15:36.770 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:24.942392+0000", 2024-09-18T10:15:36.770 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 11.716547694999999, 2024-09-18T10:15:36.770 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.36415850300000002, 2024-09-18T10:15:36.770 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:36.770 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "started", 2024-09-18T10:15:36.770 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:36.770 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:15:36.770 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:36.770 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 574 2024-09-18T10:15:36.770 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.771 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:36.771 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.771 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:36.771 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.942392+0000", 2024-09-18T10:15:36.771 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:36.771 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.771 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.771 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:36.771 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.942392+0000", 2024-09-18T10:15:36.771 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:36.771 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.771 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.771 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:36.771 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.942391+0000", 2024-09-18T10:15:36.772 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:15:36.772 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.772 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.772 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:36.772 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.942396+0000", 2024-09-18T10:15:36.772 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.6759999999999998e-06 2024-09-18T10:15:36.772 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.772 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.772 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:36.772 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.942407+0000", 2024-09-18T10:15:36.772 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0686999999999999e-05 2024-09-18T10:15:36.772 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.772 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.772 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:36.773 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.942413+0000", 2024-09-18T10:15:36.773 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.1659999999999999e-06 2024-09-18T10:15:36.773 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.773 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.773 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:36.773 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.942750+0000", 2024-09-18T10:15:36.773 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00033731700000000001 2024-09-18T10:15:36.773 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.773 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.773 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:36.773 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.942776+0000", 2024-09-18T10:15:36.773 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6342999999999998e-05 2024-09-18T10:15:36.773 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.773 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.774 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:36.774 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:25.102396+0000", 2024-09-18T10:15:36.774 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.15961924199999999 2024-09-18T10:15:36.774 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.774 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.774 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:36.774 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:25.102659+0000", 2024-09-18T10:15:36.774 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00026355500000000001 2024-09-18T10:15:36.774 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:36.774 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:36.774 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:36.774 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:25.306551+0000", 2024-09-18T10:15:36.774 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20389143700000001 2024-09-18T10:15:36.774 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:36.774 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:36.775 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:36.775 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:36.775 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:36.775 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-09-18T10:15:36.776 INFO:teuthology.orchestra.run.smithi029.stdout:osd.14: {} 2024-09-18T10:15:36.776 INFO:teuthology.orchestra.run.smithi029.stdout:osd.15: {} 2024-09-18T10:15:36.777 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T10:15:36.778 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T10:15:36.778 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T10:15:36.778 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T10:15:36.778 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T10:15:36.778 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T10:15:36.778 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T10:15:36.778 INFO:teuthology.orchestra.run.smithi029.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T10:15:36.779 INFO:teuthology.orchestra.run.smithi029.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T10:15:36.779 INFO:teuthology.orchestra.run.smithi029.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-18T10:15:36.779 INFO:teuthology.orchestra.run.smithi029.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-18T10:15:36.779 INFO:teuthology.orchestra.run.smithi029.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-18T10:15:36.779 INFO:teuthology.orchestra.run.smithi029.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-18T10:15:36.779 INFO:teuthology.orchestra.run.smithi029.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-18T10:15:36.779 INFO:teuthology.orchestra.run.smithi029.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-18T10:15:36.779 INFO:teuthology.orchestra.run.smithi029.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-18T10:15:36.779 INFO:teuthology.orchestra.run.smithi029.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-18T10:15:36.780 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-18T10:15:36.772+0000 7f578c52f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22134) UID: 0 2024-09-18T10:15:36.878 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-18T10:15:36.876+0000 7f00046a8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 22136) UID: 0 2024-09-18T10:15:36.885 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:36.885 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:36.889 INFO:tasks.ceph.mon.a.smithi029.stderr:2024-09-18T10:15:36.885+0000 7fa61494a700 -1 mon.a@0(leader).osd e81 definitely_dead 0 2024-09-18T10:15:36.895 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T10:15:36.948 INFO:tasks.rados.rados.0.smithi189.stdout:287: finishing write tid 3 to smithi18925896-25 2024-09-18T10:15:36.978 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-18T10:15:36.973+0000 7f302f2c3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22187) UID: 0 2024-09-18T10:15:36.994 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:36.994 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:37.003 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T10:15:37.079 INFO:tasks.ceph.osd.15.smithi189.stderr:2024-09-18T10:15:37.074+0000 7fd0e725e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 22401) UID: 0 2024-09-18T10:15:37.093 INFO:tasks.rados.rados.0.smithi189.stdout:287: finishing write tid 4 to smithi18925896-25 2024-09-18T10:15:37.098 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:37.099 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:37.108 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T10:15:37.180 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-18T10:15:37.176+0000 7f1981873700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22132) UID: 0 2024-09-18T10:15:37.208 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:37.208 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:37.218 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T10:15:37.258 INFO:tasks.rados.rados.0.smithi189.stdout:290: done (1 left) 2024-09-18T10:15:37.259 INFO:tasks.rados.rados.0.smithi189.stdout:291: copy_from oid 5 from oid 27 current snap is 25 2024-09-18T10:15:37.259 INFO:tasks.rados.rados.0.smithi189.stdout:292: rollback oid 17 current snap is 25 2024-09-18T10:15:37.259 INFO:tasks.rados.rados.0.smithi189.stdout:rollback oid 17 to 18 2024-09-18T10:15:37.259 INFO:tasks.rados.rados.0.smithi189.stdout:293: setattr oid 3 current snap is 25 2024-09-18T10:15:37.259 INFO:tasks.rados.rados.0.smithi189.stdout:append oid 20 current snap is 25 2024-09-18T10:15:37.259 INFO:tasks.rados.rados.0.smithi189.stdout:294: seq_num 118 ranges {3489792=442368,3932160=753664,4685824=655360,5341184=147456} 2024-09-18T10:15:37.281 INFO:tasks.ceph.osd.8.smithi029.stderr:2024-09-18T10:15:37.273+0000 7f2555c27700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 22192) UID: 0 2024-09-18T10:15:37.282 INFO:tasks.rados.rados.0.smithi189.stdout:294: writing smithi18925896-20 from 3489792 to 3932160 tid 1 2024-09-18T10:15:37.291 INFO:tasks.rados.rados.0.smithi189.stdout:294: writing smithi18925896-20 from 3932160 to 4685824 tid 2 2024-09-18T10:15:37.298 INFO:tasks.rados.rados.0.smithi189.stdout:294: writing smithi18925896-20 from 4685824 to 5341184 tid 3 2024-09-18T10:15:37.300 INFO:tasks.rados.rados.0.smithi189.stdout:294: writing smithi18925896-20 from 5341184 to 5488640 tid 4 2024-09-18T10:15:37.300 INFO:tasks.rados.rados.0.smithi189.stdout:295: read oid 46 snap 16 2024-09-18T10:15:37.300 INFO:tasks.rados.rados.0.smithi189.stdout:295: expect (ObjNum 76 snap 8 seq_num 76) 2024-09-18T10:15:37.300 INFO:tasks.rados.rados.0.smithi189.stdout:append oid 43 current snap is 25 2024-09-18T10:15:37.300 INFO:tasks.rados.rados.0.smithi189.stdout:296: seq_num 119 ranges {720896=458752,1179648=507904,1687552=507904,2195456=458752,2654208=81920} 2024-09-18T10:15:37.307 INFO:tasks.rados.rados.0.smithi189.stdout:296: writing smithi18925896-43 from 720896 to 1179648 tid 1 2024-09-18T10:15:37.310 INFO:tasks.rados.rados.0.smithi189.stdout:296: writing smithi18925896-43 from 1179648 to 1687552 tid 2 2024-09-18T10:15:37.313 INFO:tasks.rados.rados.0.smithi189.stdout:296: writing smithi18925896-43 from 1687552 to 2195456 tid 3 2024-09-18T10:15:37.315 INFO:tasks.rados.rados.0.smithi189.stdout:296: writing smithi18925896-43 from 2195456 to 2654208 tid 4 2024-09-18T10:15:37.316 INFO:tasks.rados.rados.0.smithi189.stdout:296: writing smithi18925896-43 from 2654208 to 2736128 tid 5 2024-09-18T10:15:37.316 INFO:tasks.rados.rados.0.smithi189.stdout:append oid 9 current snap is 25 2024-09-18T10:15:37.316 INFO:tasks.rados.rados.0.smithi189.stdout:297: seq_num 120 ranges {0=475136,475136=655360,1130496=622592,1753088=278528} 2024-09-18T10:15:37.319 INFO:tasks.rados.rados.0.smithi189.stdout:297: writing smithi18925896-9 from 0 to 475136 tid 1 2024-09-18T10:15:37.322 INFO:tasks.rados.rados.0.smithi189.stdout:297: writing smithi18925896-9 from 475136 to 1130496 tid 2 2024-09-18T10:15:37.326 INFO:tasks.rados.rados.0.smithi189.stdout:297: writing smithi18925896-9 from 1130496 to 1753088 tid 3 2024-09-18T10:15:37.328 INFO:tasks.rados.rados.0.smithi189.stdout:297: writing smithi18925896-9 from 1753088 to 2031616 tid 4 2024-09-18T10:15:37.328 INFO:tasks.rados.rados.0.smithi189.stdout:298: rollback oid 33 current snap is 25 2024-09-18T10:15:37.328 INFO:tasks.rados.rados.0.smithi189.stdout:rollback oid 33 to 19 2024-09-18T10:15:37.328 INFO:tasks.rados.rados.0.smithi189.stdout:299: snap_remove snap 24 2024-09-18T10:15:37.328 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:37.328 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:37.338 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T10:15:37.382 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-18T10:15:37.380+0000 7f1981873700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22132) UID: 0 2024-09-18T10:15:37.434 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:37.434 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:37.444 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-09-18T10:15:37.484 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-09-18T10:15:37.482+0000 7fdfa8698700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22087) UID: 0 2024-09-18T10:15:37.542 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:37.542 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:37.552 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-09-18T10:15:37.599 INFO:tasks.rados.rados.0.smithi189.stdout:299: done (9 left) 2024-09-18T10:15:37.599 INFO:tasks.rados.rados.0.smithi189.stdout:300: rollback oid 45 current snap is 25 2024-09-18T10:15:37.599 INFO:tasks.rados.rados.0.smithi189.stdout:rollback oid 45 to 18 2024-09-18T10:15:37.599 INFO:tasks.rados.rados.0.smithi189.stdout:294: finishing write tid 1 to smithi18925896-20 2024-09-18T10:15:37.599 INFO:tasks.rados.rados.0.smithi189.stdout:287: finishing write tid 5 to smithi18925896-25 2024-09-18T10:15:37.599 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 25 v 58 (ObjNum 115 snap 25 seq_num 115) dirty exists 2024-09-18T10:15:37.599 INFO:tasks.rados.rados.0.smithi189.stdout:287: left oid 25 (ObjNum 115 snap 25 seq_num 115) 2024-09-18T10:15:37.599 INFO:tasks.rados.rados.0.smithi189.stdout:297: finishing write tid 1 to smithi18925896-9 2024-09-18T10:15:37.599 INFO:tasks.rados.rados.0.smithi189.stdout:297: finishing write tid 2 to smithi18925896-9 2024-09-18T10:15:37.600 INFO:tasks.rados.rados.0.smithi189.stdout:287: done (9 left) 2024-09-18T10:15:37.600 INFO:tasks.rados.rados.0.smithi189.stdout:301: snap_create 2024-09-18T10:15:37.600 INFO:tasks.ceph.osd.11.smithi189.stderr:2024-09-18T10:15:37.594+0000 7f718b60a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 22400) UID: 0 2024-09-18T10:15:37.603 INFO:teuthology.orchestra.run.smithi029.stderr:marked out osd.8. 2024-09-18T10:15:37.608 INFO:teuthology.orchestra.run.smithi029.stderr:noscrub is unset 2024-09-18T10:15:37.622 INFO:tasks.rados.rados.0.smithi189.stdout:291: finishing copy_from racing read to smithi18925896-5 2024-09-18T10:15:37.649 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:37.649 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:37.658 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-09-18T10:15:37.673 INFO:tasks.rados.rados.0.smithi189.stdout:296: finishing write tid 1 to smithi18925896-43 2024-09-18T10:15:37.673 INFO:tasks.rados.rados.0.smithi189.stdout:296: finishing write tid 2 to smithi18925896-43 2024-09-18T10:15:37.717 INFO:tasks.ceph.osd.15.smithi189.stderr:2024-09-18T10:15:37.714+0000 7fd0e725e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 22401) UID: 0 2024-09-18T10:15:37.765 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:37.765 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:37.774 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_ops_in_flight 2024-09-18T10:15:37.786 INFO:tasks.ceph.osd.6.smithi154.stderr:2024-09-18T10:15:37.782+0000 7f0abc9e1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22089) UID: 0 2024-09-18T10:15:37.793 INFO:tasks.rados.rados.0.smithi189.stdout:294: finishing write tid 2 to smithi18925896-20 2024-09-18T10:15:37.827 INFO:tasks.rados.rados.0.smithi189.stdout:296: finishing write tid 3 to smithi18925896-43 2024-09-18T10:15:37.872 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:37.872 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:37.877 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 3 v 32 (ObjNum 161 snap 0 seq_num 0) dirty exists 2024-09-18T10:15:37.881 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_blocked_ops 2024-09-18T10:15:37.887 INFO:tasks.ceph.osd.12.smithi029.stderr:2024-09-18T10:15:37.881+0000 7f8e4e627700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 22190) UID: 0 2024-09-18T10:15:37.896 INFO:tasks.rados.rados.0.smithi189.stdout:297: finishing write tid 3 to smithi18925896-9 2024-09-18T10:15:37.979 INFO:tasks.rados.rados.0.smithi189.stdout:291: finishing copy_from to smithi18925896-5 2024-09-18T10:15:37.979 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 5 v 28 (ObjNum 26 snap 0 seq_num 26) dirty exists 2024-09-18T10:15:37.980 INFO:tasks.rados.rados.0.smithi189.stdout:298: finishing rollback tid 0 to smithi18925896-33 2024-09-18T10:15:37.980 INFO:tasks.rados.rados.0.smithi189.stdout:298: finishing rollback tid 1 to smithi18925896-33 2024-09-18T10:15:37.980 INFO:tasks.rados.rados.0.smithi189.stdout:298: finishing rollback tid 2 to smithi18925896-33 2024-09-18T10:15:37.980 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 33 v 61 (ObjNum 15 snap 0 seq_num 15) dirty exists 2024-09-18T10:15:37.986 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:37.987 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:37.988 INFO:tasks.ceph.osd.15.smithi189.stderr:2024-09-18T10:15:37.982+0000 7fd0e725e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 22401) UID: 0 2024-09-18T10:15:37.996 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_historic_ops 2024-09-18T10:15:37.997 INFO:tasks.rados.rados.0.smithi189.stdout:297: finishing write tid 4 to smithi18925896-9 2024-09-18T10:15:37.997 INFO:tasks.rados.rados.0.smithi189.stdout:297: finishing write tid 5 to smithi18925896-9 2024-09-18T10:15:38.029 INFO:tasks.rados.rados.0.smithi189.stdout:296: finishing write tid 4 to smithi18925896-43 2024-09-18T10:15:38.029 INFO:tasks.rados.rados.0.smithi189.stdout:296: finishing write tid 5 to smithi18925896-43 2024-09-18T10:15:38.030 INFO:tasks.rados.rados.0.smithi189.stdout:296: finishing write tid 6 to smithi18925896-43 2024-09-18T10:15:38.088 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:38.088 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:38.089 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-18T10:15:38.084+0000 7f14c6a1a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 22138) UID: 0 2024-09-18T10:15:38.096 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T10:15:38.190 INFO:tasks.ceph.osd.14.smithi154.stderr:2024-09-18T10:15:38.186+0000 7f75a19e7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 22093) UID: 0 2024-09-18T10:15:38.206 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:38.207 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:38.217 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T10:15:38.235 INFO:tasks.rados.rados.0.smithi189.stdout:296: finishing write tid 7 to smithi18925896-43 2024-09-18T10:15:38.235 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 43 v 58 (ObjNum 119 snap 25 seq_num 119) dirty exists 2024-09-18T10:15:38.235 INFO:tasks.rados.rados.0.smithi189.stdout:296: left oid 43 (ObjNum 119 snap 25 seq_num 119) 2024-09-18T10:15:38.290 INFO:tasks.ceph.osd.3.smithi189.stderr:2024-09-18T10:15:38.286+0000 7f2440fc7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22396) UID: 0 2024-09-18T10:15:38.301 INFO:tasks.rados.rados.0.smithi189.stdout:297: finishing write tid 6 to smithi18925896-9 2024-09-18T10:15:38.301 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 9 v 47 (ObjNum 120 snap 25 seq_num 120) dirty exists 2024-09-18T10:15:38.301 INFO:tasks.rados.rados.0.smithi189.stdout:297: left oid 9 (ObjNum 120 snap 25 seq_num 120) 2024-09-18T10:15:38.319 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:38.319 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:38.329 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T10:15:38.392 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-18T10:15:38.388+0000 7f1981873700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22132) UID: 0 2024-09-18T10:15:38.430 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:38.430 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:38.439 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T10:15:38.493 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-18T10:15:38.489+0000 7f302f2c3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22187) UID: 0 2024-09-18T10:15:38.536 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:38.536 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:38.546 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T10:15:38.558 INFO:tasks.rados.rados.0.smithi189.stdout:291: done (9 left) 2024-09-18T10:15:38.558 INFO:tasks.rados.rados.0.smithi189.stdout:293: done (8 left) 2024-09-18T10:15:38.558 INFO:tasks.rados.rados.0.smithi189.stdout:296: done (7 left) 2024-09-18T10:15:38.558 INFO:tasks.rados.rados.0.smithi189.stdout:297: done (6 left) 2024-09-18T10:15:38.558 INFO:tasks.rados.rados.0.smithi189.stdout:298: done (5 left) 2024-09-18T10:15:38.558 INFO:tasks.rados.rados.0.smithi189.stdout:301: done (4 left) 2024-09-18T10:15:38.558 INFO:tasks.rados.rados.0.smithi189.stdout:302: read oid 43 snap -1 2024-09-18T10:15:38.558 INFO:tasks.rados.rados.0.smithi189.stdout:302: expect (ObjNum 119 snap 25 seq_num 119) 2024-09-18T10:15:38.558 INFO:tasks.rados.rados.0.smithi189.stdout:303: snap_remove snap 25 2024-09-18T10:15:38.594 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-18T10:15:38.592+0000 7f14c6a1a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 22138) UID: 0 2024-09-18T10:15:38.694 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:38.695 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:38.695 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-18T10:15:38.692+0000 7f00046a8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 22136) UID: 0 2024-09-18T10:15:38.703 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T10:15:38.780 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T10:15:38.795 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-18T10:15:38.792+0000 7f00046a8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 22136) UID: 0 2024-09-18T10:15:38.804 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:38.804 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:38.815 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-09-18T10:15:38.896 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-18T10:15:38.892+0000 7f14c6a1a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 22138) UID: 0 2024-09-18T10:15:38.904 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-18T10:15:38.925 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-18T10:15:38.949 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-18T10:15:38.958 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:38.959 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:38.968 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-09-18T10:15:38.972 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-18T10:15:38.995 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-18T10:15:38.997 INFO:tasks.ceph.osd.6.smithi154.stderr:2024-09-18T10:15:38.994+0000 7f0abc9e1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22089) UID: 0 2024-09-18T10:15:39.019 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-18T10:15:39.046 INFO:teuthology.orchestra.run.smithi029.stdout:osd.6: {} 2024-09-18T10:15:39.066 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:39.066 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:39.070 INFO:teuthology.orchestra.run.smithi029.stdout:osd.7: {} 2024-09-18T10:15:39.076 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-09-18T10:15:39.094 INFO:teuthology.orchestra.run.smithi029.stdout:osd.8: {} 2024-09-18T10:15:39.098 INFO:tasks.ceph.osd.15.smithi189.stderr:2024-09-18T10:15:39.094+0000 7fd0e725e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 22401) UID: 0 2024-09-18T10:15:39.130 INFO:teuthology.orchestra.run.smithi029.stdout:osd.9: {} 2024-09-18T10:15:39.164 INFO:teuthology.orchestra.run.smithi029.stdout:osd.10: {} 2024-09-18T10:15:39.197 INFO:teuthology.orchestra.run.smithi029.stdout:osd.11: {} 2024-09-18T10:15:39.199 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-18T10:15:39.199 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-18T10:15:39.199 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-18T10:15:39.199 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-18T10:15:39.200 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.200 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:134 3.4s0 3:2650be24:::smithi18925896-45:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:39.200 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:50.915670+0000", 2024-09-18T10:15:39.200 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 48.279399542999997, 2024-09-18T10:15:39.200 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.40775830200000002, 2024-09-18T10:15:39.201 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:39.201 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:39.201 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:39.201 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:39.201 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:39.201 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 134 2024-09-18T10:15:39.201 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.201 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:39.201 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.201 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:39.201 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.915670+0000", 2024-09-18T10:15:39.202 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:39.202 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.202 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.202 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:39.202 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.915670+0000", 2024-09-18T10:15:39.202 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:39.202 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.202 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.202 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:39.202 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.915668+0000", 2024-09-18T10:15:39.202 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:39.202 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.202 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.203 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:39.203 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.915676+0000", 2024-09-18T10:15:39.203 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1899999999999995e-06 2024-09-18T10:15:39.203 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.203 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.203 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:39.203 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.915690+0000", 2024-09-18T10:15:39.203 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3827e-05 2024-09-18T10:15:39.203 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.203 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.204 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:39.204 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.915702+0000", 2024-09-18T10:15:39.204 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1477999999999999e-05 2024-09-18T10:15:39.204 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.204 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.204 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:39.204 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.918373+0000", 2024-09-18T10:15:39.204 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0026713000000000002 2024-09-18T10:15:39.204 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.204 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.204 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:39.204 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.918547+0000", 2024-09-18T10:15:39.205 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00017375999999999999 2024-09-18T10:15:39.205 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.205 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.205 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:39.205 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.919485+0000", 2024-09-18T10:15:39.205 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00093826999999999995 2024-09-18T10:15:39.205 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.205 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.205 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:39.205 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.120632+0000", 2024-09-18T10:15:39.205 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.20114729200000001 2024-09-18T10:15:39.205 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.205 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.206 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-18T10:15:39.206 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.323281+0000", 2024-09-18T10:15:39.206 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.20264834500000001 2024-09-18T10:15:39.206 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.206 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.206 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:39.206 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.323428+0000", 2024-09-18T10:15:39.206 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000147334 2024-09-18T10:15:39.206 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:39.206 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:39.206 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:39.206 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.207 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.207 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:135 3.4s0 3:2650be24:::smithi18925896-45:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:39.207 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:50.915712+0000", 2024-09-18T10:15:39.207 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 48.279357683999997, 2024-09-18T10:15:39.207 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.81161388099999998, 2024-09-18T10:15:39.207 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:39.207 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:39.207 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:39.207 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:39.207 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:39.207 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 135 2024-09-18T10:15:39.207 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.207 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:39.208 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.208 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:39.208 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.915712+0000", 2024-09-18T10:15:39.208 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:39.208 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.208 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.208 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:39.208 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.915712+0000", 2024-09-18T10:15:39.208 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:39.209 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.209 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.209 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:39.209 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.915711+0000", 2024-09-18T10:15:39.209 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.999999 2024-09-18T10:15:39.209 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.209 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.209 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:39.209 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.915715+0000", 2024-09-18T10:15:39.209 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.3159999999999998e-06 2024-09-18T10:15:39.209 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.209 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.210 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:39.210 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.915726+0000", 2024-09-18T10:15:39.210 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1375e-05 2024-09-18T10:15:39.210 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.210 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.210 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:39.210 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.915736+0000", 2024-09-18T10:15:39.210 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.4860000000000003e-06 2024-09-18T10:15:39.210 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.211 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.211 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:39.211 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.919925+0000", 2024-09-18T10:15:39.211 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0041886689999999999 2024-09-18T10:15:39.211 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.211 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.211 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for rw locks", 2024-09-18T10:15:39.211 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.920019+0000", 2024-09-18T10:15:39.211 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.4196000000000004e-05 2024-09-18T10:15:39.211 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.211 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.211 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:39.212 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.424171+0000", 2024-09-18T10:15:39.212 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.50415232399999999 2024-09-18T10:15:39.212 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.212 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.212 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:39.212 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.424320+0000", 2024-09-18T10:15:39.212 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00014877299999999999 2024-09-18T10:15:39.212 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.212 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.212 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:39.212 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.727325+0000", 2024-09-18T10:15:39.212 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.30300555400000001 2024-09-18T10:15:39.212 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:39.213 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:39.213 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:39.213 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.213 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.213 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:188 3.4s0 3:213f864e:::smithi18925896-32:head [create,setxattr XNeZS[lOPtVca\\Pa[\\rdfbinb[bcjhtYSpmcafZm (1928) in=40b,setxattr TfflYYqPnps]lo^qSlp___TitNdd`]kaPTVOTcsh (1648) in=40b,setxattr TNb_onpY[j_dkhfmnWtm]Yktg]]RNtbQe[OUTNlX (770) in=40b,setxattr ZcgnkhY\\orTYaZh]PPldj`SNq`p\\cX`k]h\\bZY_Y (700) in=40b,setxattr gPQVnqhpsVhW_kq`QnafXrlRZtZ\\YnRftgQTishR (595) in=40b,setxattr U`XqYi[WhoqnpSPRc\\RhnppZp^qY\\qUfjpqs[r^a (1892) in=40b,setxattr cTmhd`pWgrRf_VfVgOtgeOP^ShP]lsOtpoTj[\\gl (418) in=40b,setxattr aa_taYTfslg`lRRrOkkraN\\XiW^V\\ljl]NqXin`a (314) in=40b,setxattr ibl]`igtgVThY^Qse\\okYYUtTmolX\\W]p]Q[P\\il (1403) in=40b,setxattr \\ZabSO[UUqeZa`hddNh[ZjaRRgUZoQtkn\\QdsX]W (1541) in=40b,setxattr jUkQUPTNbo`rZNmYlUmhb]ojc]aeheoVRfp]bPWf (5052) in=40b,setxattr Ofj`oQ^cR]da^SPsTaXmmPse\\OZ`VSqQ]U`efsZl (419) in=40b,setxattr a[cdkWos\\OrUVPciUaa^_^`lWVTScffiWkd\\[kgf (1261) in=40b,setxattr jXb]\\WhbatdZ[qTTiZO]kPd^`l[kajOYiQ[iqSYc (244) in=40b,setxattr \\im]jQhbanUn\\_^h`jaU]W`ReaZN_je[]kgg (1578) in=36b,setxattr ZbosoRORS^_de]rN^_daOiha\\Q\\eRae^[sQUjXsV (9120) in=40b,setxattr gcn]XpZUkOZr\\pkrVVZpYiR\\alh_PaZa_S[PYcn] (384) in=40b,setxattr \\^qUOaNXSTjtmafhfbo\\gseTTkV`mgnacYq^bkcc (34) in=40b,setxattr iUa^mOQ]TqjVSUU_nXfY]f`NjljiiaYbrm[o]aes (1611) in=40b,setxattr tXaQPQbc^aiV[^reVOPpib^caoWf^XgqilO^_g[k (1016) in=40b] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:39.213 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:52.693780+0000", 2024-09-18T10:15:39.213 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 46.501289106999998, 2024-09-18T10:15:39.213 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.40440725999999999, 2024-09-18T10:15:39.213 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:39.213 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:39.213 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:39.214 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:39.214 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:39.214 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 188 2024-09-18T10:15:39.214 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.214 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:39.214 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.214 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:39.214 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.693780+0000", 2024-09-18T10:15:39.214 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:39.214 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.214 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.215 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:39.215 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.693780+0000", 2024-09-18T10:15:39.215 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:39.215 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.215 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.215 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:39.216 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.693777+0000", 2024-09-18T10:15:39.216 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999967 2024-09-18T10:15:39.216 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.216 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.216 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:39.216 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.693873+0000", 2024-09-18T10:15:39.217 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.6496000000000006e-05 2024-09-18T10:15:39.217 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.217 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.218 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:39.218 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.693926+0000", 2024-09-18T10:15:39.218 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2148999999999998e-05 2024-09-18T10:15:39.218 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.218 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.218 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:39.219 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.693945+0000", 2024-09-18T10:15:39.219 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9296999999999998e-05 2024-09-18T10:15:39.219 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.219 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.219 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:39.219 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.694035+0000", 2024-09-18T10:15:39.219 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.0439999999999995e-05 2024-09-18T10:15:39.220 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.220 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.220 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op must wait for map", 2024-09-18T10:15:39.220 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.694044+0000", 2024-09-18T10:15:39.220 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.1479999999999995e-06 2024-09-18T10:15:39.220 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.220 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.220 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:39.221 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.797388+0000", 2024-09-18T10:15:39.221 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10334315199999999 2024-09-18T10:15:39.221 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.221 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.221 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:39.221 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.797847+0000", 2024-09-18T10:15:39.221 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00045971299999999999 2024-09-18T10:15:39.221 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.222 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.222 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:39.222 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.800981+0000", 2024-09-18T10:15:39.222 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0031333189999999999 2024-09-18T10:15:39.222 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.222 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.222 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:39.222 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.803005+0000", 2024-09-18T10:15:39.223 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0020248359999999999 2024-09-18T10:15:39.223 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.223 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.223 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-18T10:15:39.223 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:53.097948+0000", 2024-09-18T10:15:39.223 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.29494210599999998 2024-09-18T10:15:39.223 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.224 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.224 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:39.224 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:53.098187+0000", 2024-09-18T10:15:39.224 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000239769 2024-09-18T10:15:39.224 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:39.224 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:39.224 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:39.225 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.225 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.225 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWrite(3.cs1 41/32 ECSubWrite(tid=47, reqid=osd.5.0:46, at_version=0'0, trim_to=0'0, roll_forward_to=41'27))", 2024-09-18T10:15:39.225 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:56.935584+0000", 2024-09-18T10:15:39.225 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 42.259485411, 2024-09-18T10:15:39.225 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.60626800999999997, 2024-09-18T10:15:39.225 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:39.225 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:39.225 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:39.225 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.225 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:39.225 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.935584+0000", 2024-09-18T10:15:39.225 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:39.226 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.226 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.226 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:39.226 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.935584+0000", 2024-09-18T10:15:39.226 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:39.226 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.226 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.226 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:39.226 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.935582+0000", 2024-09-18T10:15:39.226 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:39.226 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.226 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.226 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:39.226 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.935616+0000", 2024-09-18T10:15:39.227 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.4588999999999997e-05 2024-09-18T10:15:39.227 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.227 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.227 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:39.227 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.935661+0000", 2024-09-18T10:15:39.227 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4705e-05 2024-09-18T10:15:39.227 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.227 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.227 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:39.227 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.935681+0000", 2024-09-18T10:15:39.227 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0290000000000001e-05 2024-09-18T10:15:39.227 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.227 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.227 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:39.228 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.339911+0000", 2024-09-18T10:15:39.228 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.40422952000000001 2024-09-18T10:15:39.228 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.228 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.228 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:39.228 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.339925+0000", 2024-09-18T10:15:39.228 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4523e-05 2024-09-18T10:15:39.228 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.229 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.229 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:39.229 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.541788+0000", 2024-09-18T10:15:39.229 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.20186222000000001 2024-09-18T10:15:39.229 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.229 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.229 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:39.229 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.541852+0000", 2024-09-18T10:15:39.229 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.4171999999999995e-05 2024-09-18T10:15:39.229 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:39.229 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:39.229 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:39.229 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.229 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.230 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWrite(3.cs1 41/32 ECSubWrite(tid=48, reqid=osd.5.0:48, at_version=41'29, trim_to=0'0, roll_forward_to=41'27))", 2024-09-18T10:15:39.230 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:56.937458+0000", 2024-09-18T10:15:39.230 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 42.257611621999999, 2024-09-18T10:15:39.230 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.60464621900000004, 2024-09-18T10:15:39.230 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:39.230 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:39.230 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:39.230 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.230 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:39.230 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.937458+0000", 2024-09-18T10:15:39.230 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:39.230 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.230 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.231 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:39.231 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.937458+0000", 2024-09-18T10:15:39.231 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:39.231 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.231 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.231 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:39.231 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.937456+0000", 2024-09-18T10:15:39.231 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.999999 2024-09-18T10:15:39.231 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.231 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.232 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:39.232 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.937505+0000", 2024-09-18T10:15:39.232 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8809e-05 2024-09-18T10:15:39.232 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.232 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.232 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:39.232 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.937540+0000", 2024-09-18T10:15:39.232 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.4662000000000001e-05 2024-09-18T10:15:39.232 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.232 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.232 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:39.232 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.937554+0000", 2024-09-18T10:15:39.233 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4137e-05 2024-09-18T10:15:39.233 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.233 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.233 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:39.233 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.340361+0000", 2024-09-18T10:15:39.233 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.40280652500000003 2024-09-18T10:15:39.233 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.233 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.233 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:39.233 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.340382+0000", 2024-09-18T10:15:39.233 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.16e-05 2024-09-18T10:15:39.233 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.234 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.234 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:39.234 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.542055+0000", 2024-09-18T10:15:39.234 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.20167278999999999 2024-09-18T10:15:39.234 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.234 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.234 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:39.234 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.542104+0000", 2024-09-18T10:15:39.235 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8866000000000003e-05 2024-09-18T10:15:39.235 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:39.235 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:39.235 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:39.235 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.235 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.235 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:413 3.4s0 3:2650be24:::smithi18925896-45:head [read 0~753664,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-18T10:15:39.235 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:07.553960+0000", 2024-09-18T10:15:39.236 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 31.641109440000001, 2024-09-18T10:15:39.236 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.30658152999999999, 2024-09-18T10:15:39.236 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:39.236 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:39.236 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:39.236 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:39.236 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:39.236 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 413 2024-09-18T10:15:39.236 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.236 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:39.236 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.236 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:39.236 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.553960+0000", 2024-09-18T10:15:39.237 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:39.237 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.237 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.237 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:39.237 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.553960+0000", 2024-09-18T10:15:39.237 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:39.237 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.237 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.237 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:39.237 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.553956+0000", 2024-09-18T10:15:39.237 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999967 2024-09-18T10:15:39.237 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.237 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.238 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:39.238 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.553973+0000", 2024-09-18T10:15:39.238 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.6602000000000001e-05 2024-09-18T10:15:39.238 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.238 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.238 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:39.238 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.554010+0000", 2024-09-18T10:15:39.238 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7129999999999999e-05 2024-09-18T10:15:39.238 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.238 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.238 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:39.238 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.554032+0000", 2024-09-18T10:15:39.238 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2316000000000001e-05 2024-09-18T10:15:39.238 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.239 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.239 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:39.239 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.554187+0000", 2024-09-18T10:15:39.239 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00015407499999999999 2024-09-18T10:15:39.239 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.239 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.239 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op must wait for map", 2024-09-18T10:15:39.239 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.554232+0000", 2024-09-18T10:15:39.239 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.5173999999999998e-05 2024-09-18T10:15:39.239 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.239 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.239 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:39.240 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.655961+0000", 2024-09-18T10:15:39.240 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10172934 2024-09-18T10:15:39.240 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.240 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.240 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:39.240 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.656152+0000", 2024-09-18T10:15:39.240 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00019075599999999999 2024-09-18T10:15:39.240 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.240 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.240 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:39.240 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.860541+0000", 2024-09-18T10:15:39.240 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.20438944000000001 2024-09-18T10:15:39.240 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:39.241 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:39.241 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:39.241 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.241 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.241 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:414 3.4s0 3:2650be24:::smithi18925896-45:head [read 0~753664,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-18T10:15:39.241 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:07.554078+0000", 2024-09-18T10:15:39.241 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 31.640991136, 2024-09-18T10:15:39.241 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.30769232899999999, 2024-09-18T10:15:39.241 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:39.241 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:39.241 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:39.241 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:39.241 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:39.242 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 414 2024-09-18T10:15:39.242 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.242 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:39.242 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.242 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:39.242 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.554078+0000", 2024-09-18T10:15:39.242 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:39.242 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.242 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.242 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:39.243 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.554078+0000", 2024-09-18T10:15:39.243 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:39.243 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.243 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.243 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:39.243 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.554077+0000", 2024-09-18T10:15:39.243 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:39.243 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.243 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.243 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:39.243 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.554086+0000", 2024-09-18T10:15:39.243 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.6569999999999992e-06 2024-09-18T10:15:39.243 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.244 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.244 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:39.244 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.554106+0000", 2024-09-18T10:15:39.244 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9969000000000001e-05 2024-09-18T10:15:39.244 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.245 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.245 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:39.245 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.554118+0000", 2024-09-18T10:15:39.245 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.217e-05 2024-09-18T10:15:39.245 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.245 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.246 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:39.246 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.554389+0000", 2024-09-18T10:15:39.246 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00027051000000000001 2024-09-18T10:15:39.246 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.246 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.246 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:39.246 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.554407+0000", 2024-09-18T10:15:39.246 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7677e-05 2024-09-18T10:15:39.247 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.247 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.247 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:39.247 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.656732+0000", 2024-09-18T10:15:39.247 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.102325597 2024-09-18T10:15:39.247 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.247 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.247 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:39.247 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.656894+0000", 2024-09-18T10:15:39.247 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000161383 2024-09-18T10:15:39.247 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.247 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.247 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:39.247 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.861770+0000", 2024-09-18T10:15:39.248 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.20487676599999999 2024-09-18T10:15:39.248 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:39.248 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:39.248 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:39.248 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.248 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.248 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:415 3.4s0 3:2650be24:::smithi18925896-45:head [read 0~753664,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-18T10:15:39.248 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:07.554169+0000", 2024-09-18T10:15:39.248 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 31.640900675000001, 2024-09-18T10:15:39.248 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.308598176, 2024-09-18T10:15:39.248 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:39.248 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:39.248 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:39.249 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:39.249 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:39.249 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 415 2024-09-18T10:15:39.249 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.249 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:39.249 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.249 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:39.249 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.554169+0000", 2024-09-18T10:15:39.249 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:39.249 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.249 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.249 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:39.249 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.554169+0000", 2024-09-18T10:15:39.250 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:39.250 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.250 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.250 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:39.250 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.554167+0000", 2024-09-18T10:15:39.250 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:39.250 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.250 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.250 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:39.250 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.554176+0000", 2024-09-18T10:15:39.250 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6430000000000001e-06 2024-09-18T10:15:39.250 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.251 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.251 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:39.251 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.554197+0000", 2024-09-18T10:15:39.251 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1209e-05 2024-09-18T10:15:39.251 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.251 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.251 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:39.252 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.554206+0000", 2024-09-18T10:15:39.252 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.3570000000000004e-06 2024-09-18T10:15:39.252 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.252 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.252 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:39.252 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.554515+0000", 2024-09-18T10:15:39.253 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00030821000000000001 2024-09-18T10:15:39.253 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.253 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.253 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:39.253 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.554535+0000", 2024-09-18T10:15:39.253 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0092999999999999e-05 2024-09-18T10:15:39.254 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.254 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.254 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:39.254 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.756075+0000", 2024-09-18T10:15:39.254 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.201540831 2024-09-18T10:15:39.254 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.255 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.255 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:39.255 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.756260+0000", 2024-09-18T10:15:39.255 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00018466000000000001 2024-09-18T10:15:39.255 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.256 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.256 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:39.256 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.862767+0000", 2024-09-18T10:15:39.256 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.106506579 2024-09-18T10:15:39.256 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:39.256 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:39.257 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:39.257 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.257 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.257 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(osd.4.21:3 3.4s0 3:2619bf65:::smithi18925896-16:head [assert-version v2,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-18T10:15:39.257 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:11.936400+0000", 2024-09-18T10:15:39.257 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 27.258669520000002, 2024-09-18T10:15:39.258 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.312109203, 2024-09-18T10:15:39.258 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:39.258 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:39.258 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:39.258 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.259 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:39.259 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:11.936400+0000", 2024-09-18T10:15:39.259 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:39.259 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.260 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.260 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:39.260 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:11.936400+0000", 2024-09-18T10:15:39.260 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:39.260 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.261 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.261 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:39.261 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:11.936396+0000", 2024-09-18T10:15:39.261 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999962 2024-09-18T10:15:39.261 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.262 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.262 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:39.262 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:11.936434+0000", 2024-09-18T10:15:39.262 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7580000000000003e-05 2024-09-18T10:15:39.262 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.262 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.263 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:39.263 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:11.936473+0000", 2024-09-18T10:15:39.263 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9876000000000002e-05 2024-09-18T10:15:39.263 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.263 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.263 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:39.264 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:11.936499+0000", 2024-09-18T10:15:39.264 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6012000000000001e-05 2024-09-18T10:15:39.264 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.264 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.264 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:39.264 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.039131+0000", 2024-09-18T10:15:39.265 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10263111799999999 2024-09-18T10:15:39.265 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.265 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.265 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:39.265 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.039402+0000", 2024-09-18T10:15:39.266 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00027099399999999999 2024-09-18T10:15:39.266 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.266 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.266 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:39.266 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.248509+0000", 2024-09-18T10:15:39.266 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.20910732700000001 2024-09-18T10:15:39.267 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:39.267 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:39.267 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:39.267 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.267 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.267 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:446 3.4s0 3:2650be24:::smithi18925896-45:head [append 0~622592 in=622592b] snapc 13=[13,12,11,10,f,e,b] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-18T10:15:39.268 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:12.892106+0000", 2024-09-18T10:15:39.268 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 26.302962891, 2024-09-18T10:15:39.268 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.77541641100000003, 2024-09-18T10:15:39.268 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:39.268 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:39.269 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:39.269 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:39.269 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:39.269 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 446 2024-09-18T10:15:39.269 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.269 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:39.270 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.270 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:39.270 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.892106+0000", 2024-09-18T10:15:39.270 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:39.270 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.271 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.271 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:39.271 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.892106+0000", 2024-09-18T10:15:39.271 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:39.271 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.271 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.272 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:39.272 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.892103+0000", 2024-09-18T10:15:39.272 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999971 2024-09-18T10:15:39.272 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.272 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.273 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:39.273 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.893050+0000", 2024-09-18T10:15:39.273 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00094607000000000003 2024-09-18T10:15:39.273 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.273 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.273 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:39.274 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.893084+0000", 2024-09-18T10:15:39.274 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.4759999999999999e-05 2024-09-18T10:15:39.274 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.274 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.274 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:39.275 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.893098+0000", 2024-09-18T10:15:39.275 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3859000000000001e-05 2024-09-18T10:15:39.275 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.275 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.275 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:39.276 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.145987+0000", 2024-09-18T10:15:39.276 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.252889115 2024-09-18T10:15:39.276 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.276 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.276 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:39.276 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.146112+0000", 2024-09-18T10:15:39.277 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00012469299999999999 2024-09-18T10:15:39.277 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.277 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.277 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:39.277 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.146923+0000", 2024-09-18T10:15:39.277 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00081091500000000005 2024-09-18T10:15:39.278 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.278 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.278 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:39.278 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.152437+0000", 2024-09-18T10:15:39.278 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0055141169999999998 2024-09-18T10:15:39.279 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.279 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.279 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-18T10:15:39.279 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.667347+0000", 2024-09-18T10:15:39.279 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.51491041199999998 2024-09-18T10:15:39.279 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.280 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.280 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:39.280 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.667523+0000", 2024-09-18T10:15:39.280 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00017533199999999999 2024-09-18T10:15:39.280 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:39.281 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:39.281 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:39.281 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.281 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.281 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:447 3.4s0 3:2650be24:::smithi18925896-45:head [append 0~475136 in=475136b] snapc 13=[13,12,11,10,f,e,b] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-18T10:15:39.281 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:12.896134+0000", 2024-09-18T10:15:39.282 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 26.298935528000001, 2024-09-18T10:15:39.282 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.87223328099999997, 2024-09-18T10:15:39.282 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:39.282 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:39.282 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:39.283 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:39.283 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:39.283 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 447 2024-09-18T10:15:39.283 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.283 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:39.283 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.284 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:39.284 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.896134+0000", 2024-09-18T10:15:39.284 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:39.284 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.284 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.284 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:39.285 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.896134+0000", 2024-09-18T10:15:39.285 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:39.285 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.285 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.285 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:39.286 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.896131+0000", 2024-09-18T10:15:39.286 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999976 2024-09-18T10:15:39.286 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.286 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.286 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:39.286 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.896661+0000", 2024-09-18T10:15:39.287 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00052970299999999997 2024-09-18T10:15:39.287 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.287 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.287 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:39.287 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.896693+0000", 2024-09-18T10:15:39.287 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1903000000000003e-05 2024-09-18T10:15:39.288 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.288 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.288 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:39.288 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.896705+0000", 2024-09-18T10:15:39.289 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2288e-05 2024-09-18T10:15:39.289 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.289 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.289 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:39.289 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.147534+0000", 2024-09-18T10:15:39.289 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.25082866300000001 2024-09-18T10:15:39.290 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.290 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.290 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:39.290 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.147612+0000", 2024-09-18T10:15:39.290 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7789999999999999e-05 2024-09-18T10:15:39.291 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.291 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.291 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:39.291 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.148220+0000", 2024-09-18T10:15:39.291 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00060859599999999999 2024-09-18T10:15:39.291 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.292 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.292 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:39.292 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.152441+0000", 2024-09-18T10:15:39.292 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0042208250000000001 2024-09-18T10:15:39.292 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.293 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.293 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-18T10:15:39.293 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.768118+0000", 2024-09-18T10:15:39.293 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.615676471 2024-09-18T10:15:39.293 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.294 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.294 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:39.294 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.768367+0000", 2024-09-18T10:15:39.294 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00024930800000000001 2024-09-18T10:15:39.294 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:39.294 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:39.294 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:39.295 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.295 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.295 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:448 3.4s0 3:2650be24:::smithi18925896-45:head [append 0~294912 in=294912b] snapc 13=[13,12,11,10,f,e,b] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-18T10:15:39.295 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:12.897981+0000", 2024-09-18T10:15:39.295 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 26.297088349999999, 2024-09-18T10:15:39.295 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.87089765500000005, 2024-09-18T10:15:39.295 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:39.295 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:39.295 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:39.295 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:39.295 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:39.295 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 448 2024-09-18T10:15:39.295 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.296 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:39.296 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.296 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:39.296 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.897981+0000", 2024-09-18T10:15:39.296 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:39.296 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.296 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.296 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:39.296 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.897981+0000", 2024-09-18T10:15:39.296 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:39.296 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.296 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.297 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:39.297 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.897979+0000", 2024-09-18T10:15:39.297 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:39.297 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.297 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.297 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:39.298 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.898390+0000", 2024-09-18T10:15:39.298 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00041124999999999999 2024-09-18T10:15:39.298 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.298 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.298 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:39.298 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.898416+0000", 2024-09-18T10:15:39.298 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.525e-05 2024-09-18T10:15:39.298 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.298 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.298 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:39.299 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.898434+0000", 2024-09-18T10:15:39.299 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8615999999999999e-05 2024-09-18T10:15:39.299 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.299 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.299 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:39.299 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.148720+0000", 2024-09-18T10:15:39.299 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.25028612900000002 2024-09-18T10:15:39.299 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.299 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.299 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:39.299 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.148799+0000", 2024-09-18T10:15:39.299 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.8481999999999998e-05 2024-09-18T10:15:39.299 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.300 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.300 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:39.300 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.149383+0000", 2024-09-18T10:15:39.300 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000584255 2024-09-18T10:15:39.300 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.300 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.300 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:39.301 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.152442+0000", 2024-09-18T10:15:39.301 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0030591049999999999 2024-09-18T10:15:39.301 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.301 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.301 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-18T10:15:39.301 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.768702+0000", 2024-09-18T10:15:39.302 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.61626031599999997 2024-09-18T10:15:39.302 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.302 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.302 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:39.302 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.768878+0000", 2024-09-18T10:15:39.302 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00017600600000000001 2024-09-18T10:15:39.302 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:39.302 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:39.302 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:39.302 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.302 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.303 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:449 3.4s0 3:2650be24:::smithi18925896-45:head [setxattr _header (57) in=64b] snapc 13=[13,12,11,10,f,e,b] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-18T10:15:39.303 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:12.898446+0000", 2024-09-18T10:15:39.303 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 26.296622832000001, 2024-09-18T10:15:39.303 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.87093100999999995, 2024-09-18T10:15:39.303 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:39.303 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:39.303 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:39.303 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:39.303 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:39.303 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 449 2024-09-18T10:15:39.303 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.303 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:39.303 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.304 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:39.304 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.898446+0000", 2024-09-18T10:15:39.304 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:39.304 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.304 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.304 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:39.304 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.898446+0000", 2024-09-18T10:15:39.304 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:39.304 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.304 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.304 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:39.304 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.898445+0000", 2024-09-18T10:15:39.304 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.999999 2024-09-18T10:15:39.304 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.305 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.305 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:39.305 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.898451+0000", 2024-09-18T10:15:39.305 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.054e-06 2024-09-18T10:15:39.305 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.305 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.305 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:39.305 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.898462+0000", 2024-09-18T10:15:39.305 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.081e-05 2024-09-18T10:15:39.305 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.305 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.305 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:39.305 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.898467+0000", 2024-09-18T10:15:39.306 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4220000000000002e-06 2024-09-18T10:15:39.306 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.306 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.306 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:39.306 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.149769+0000", 2024-09-18T10:15:39.306 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.25130268 2024-09-18T10:15:39.306 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.306 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.306 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:39.306 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.149964+0000", 2024-09-18T10:15:39.306 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00019506699999999999 2024-09-18T10:15:39.306 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.307 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.307 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:39.307 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.151037+0000", 2024-09-18T10:15:39.307 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0010723200000000001 2024-09-18T10:15:39.307 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.307 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.307 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:39.307 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.152623+0000", 2024-09-18T10:15:39.307 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0015865829999999999 2024-09-18T10:15:39.307 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.307 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.307 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-18T10:15:39.307 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.769215+0000", 2024-09-18T10:15:39.307 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.61659198800000004 2024-09-18T10:15:39.308 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.308 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.308 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:39.308 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.769377+0000", 2024-09-18T10:15:39.308 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000162134 2024-09-18T10:15:39.308 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:39.308 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:39.308 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:39.308 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.308 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.308 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:450 3.4s0 3:2650be24:::smithi18925896-45:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e57)", 2024-09-18T10:15:39.309 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:12.898472+0000", 2024-09-18T10:15:39.309 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 26.296597247000001, 2024-09-18T10:15:39.309 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.172788277, 2024-09-18T10:15:39.309 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:39.309 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:39.309 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:39.309 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:39.309 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:39.309 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 450 2024-09-18T10:15:39.309 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.309 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:39.309 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.309 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:39.310 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.898472+0000", 2024-09-18T10:15:39.310 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:39.310 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.310 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.310 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:39.310 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.898472+0000", 2024-09-18T10:15:39.310 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:39.310 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.310 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.310 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:39.310 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.898471+0000", 2024-09-18T10:15:39.311 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:39.311 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.311 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.311 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:39.311 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.898475+0000", 2024-09-18T10:15:39.311 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1480000000000002e-06 2024-09-18T10:15:39.311 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.312 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.312 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:39.312 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.898485+0000", 2024-09-18T10:15:39.312 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0342e-05 2024-09-18T10:15:39.312 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.313 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.313 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:39.313 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.898490+0000", 2024-09-18T10:15:39.313 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7060000000000003e-06 2024-09-18T10:15:39.313 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.313 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.314 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:39.314 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.151519+0000", 2024-09-18T10:15:39.314 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.25302985300000003 2024-09-18T10:15:39.314 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.314 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.314 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for rw locks", 2024-09-18T10:15:39.314 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.151633+0000", 2024-09-18T10:15:39.315 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000113121 2024-09-18T10:15:39.315 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.315 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.315 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:39.315 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.770549+0000", 2024-09-18T10:15:39.315 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.61891632200000002 2024-09-18T10:15:39.316 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.316 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.316 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:39.316 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.770780+0000", 2024-09-18T10:15:39.316 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00023144999999999999 2024-09-18T10:15:39.316 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.316 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.317 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:39.317 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:14.071260+0000", 2024-09-18T10:15:39.317 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.30047984999999999 2024-09-18T10:15:39.317 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:39.317 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:39.317 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:39.318 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.318 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.318 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:622 3.4s0 3:213f864e:::smithi18925896-32:head [read 0~606208,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-18T10:15:39.318 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:28.103279+0000", 2024-09-18T10:15:39.318 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 11.091789963, 2024-09-18T10:15:39.318 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.33052193600000002, 2024-09-18T10:15:39.319 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:39.319 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:39.319 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:39.319 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:39.319 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:39.319 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 622 2024-09-18T10:15:39.320 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.320 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:39.320 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.320 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:39.320 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:28.103279+0000", 2024-09-18T10:15:39.320 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:39.321 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.321 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.321 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:39.321 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:28.103279+0000", 2024-09-18T10:15:39.321 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:39.321 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.322 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.322 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:39.322 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:28.103279+0000", 2024-09-18T10:15:39.322 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:39.322 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.322 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.322 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:39.322 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:28.103284+0000", 2024-09-18T10:15:39.322 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0250000000000002e-06 2024-09-18T10:15:39.322 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.322 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.322 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:39.322 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:28.103300+0000", 2024-09-18T10:15:39.323 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.6850999999999998e-05 2024-09-18T10:15:39.323 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.323 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.323 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:39.323 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:28.103308+0000", 2024-09-18T10:15:39.323 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1450000000000002e-06 2024-09-18T10:15:39.323 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.323 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.323 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:39.323 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:28.103428+0000", 2024-09-18T10:15:39.324 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00012029200000000001 2024-09-18T10:15:39.324 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.324 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.324 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:39.324 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:28.103435+0000", 2024-09-18T10:15:39.324 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.6930000000000001e-06 2024-09-18T10:15:39.324 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.324 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.324 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:39.324 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:28.187445+0000", 2024-09-18T10:15:39.325 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.084010325999999996 2024-09-18T10:15:39.325 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.325 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.325 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:39.325 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:28.187954+0000", 2024-09-18T10:15:39.325 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00050906599999999997 2024-09-18T10:15:39.325 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.325 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.325 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:39.325 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:28.433801+0000", 2024-09-18T10:15:39.325 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.24584714399999999 2024-09-18T10:15:39.325 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:39.325 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:39.326 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:39.326 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.326 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.326 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:623 3.4s0 3:213f864e:::smithi18925896-32:head [read 0~606208,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-18T10:15:39.326 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:28.103315+0000", 2024-09-18T10:15:39.326 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 11.091754547000001, 2024-09-18T10:15:39.326 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.33131258499999999, 2024-09-18T10:15:39.326 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:39.326 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:39.326 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:39.326 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:39.326 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:39.326 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 623 2024-09-18T10:15:39.327 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.327 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:39.327 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.327 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:39.327 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:28.103315+0000", 2024-09-18T10:15:39.327 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:39.327 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.327 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.327 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:39.327 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:28.103315+0000", 2024-09-18T10:15:39.327 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:39.327 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.327 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.327 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:39.328 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:28.103314+0000", 2024-09-18T10:15:39.328 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:39.328 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.328 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.328 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:39.328 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:28.103319+0000", 2024-09-18T10:15:39.328 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6829999999999999e-06 2024-09-18T10:15:39.328 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.328 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.328 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:39.329 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:28.103329+0000", 2024-09-18T10:15:39.329 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0511e-05 2024-09-18T10:15:39.329 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.329 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.329 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:39.329 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:28.203443+0000", 2024-09-18T10:15:39.329 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10011328799999999 2024-09-18T10:15:39.329 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.329 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.329 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:39.329 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:28.292935+0000", 2024-09-18T10:15:39.329 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.089492877999999998 2024-09-18T10:15:39.329 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.330 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.330 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:39.330 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:28.293133+0000", 2024-09-18T10:15:39.330 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00019793 2024-09-18T10:15:39.330 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.330 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.330 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:39.330 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:28.434627+0000", 2024-09-18T10:15:39.330 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.14149382099999999 2024-09-18T10:15:39.330 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:39.331 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:39.331 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:39.331 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.331 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.331 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWrite(3.9s5 74/40 ECSubWrite(tid=178, reqid=osd.7.0:178, at_version=74'78, trim_to=0'0, roll_forward_to=72'76))", 2024-09-18T10:15:39.331 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:29.615287+0000", 2024-09-18T10:15:39.331 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 9.5797819559999997, 2024-09-18T10:15:39.331 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.35347188099999999, 2024-09-18T10:15:39.331 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:39.331 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:39.332 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:39.332 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.332 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:39.332 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:29.615287+0000", 2024-09-18T10:15:39.332 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:39.332 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.332 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.332 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:39.332 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:29.615287+0000", 2024-09-18T10:15:39.332 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:39.332 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.332 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.333 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:39.333 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:29.615286+0000", 2024-09-18T10:15:39.333 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:39.333 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.333 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.333 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:39.333 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:29.615331+0000", 2024-09-18T10:15:39.333 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4873000000000002e-05 2024-09-18T10:15:39.333 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.333 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.334 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:39.334 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:29.615363+0000", 2024-09-18T10:15:39.334 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1953999999999999e-05 2024-09-18T10:15:39.334 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.334 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.334 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:39.334 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:29.615376+0000", 2024-09-18T10:15:39.334 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3071000000000001e-05 2024-09-18T10:15:39.334 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.334 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.334 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:39.334 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:29.767783+0000", 2024-09-18T10:15:39.334 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.15240696000000001 2024-09-18T10:15:39.335 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.335 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.335 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:39.335 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:29.767797+0000", 2024-09-18T10:15:39.335 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4688000000000001e-05 2024-09-18T10:15:39.335 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.335 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.335 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:39.335 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:29.968659+0000", 2024-09-18T10:15:39.335 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.20086167699999999 2024-09-18T10:15:39.335 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.335 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.335 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:39.336 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:29.968759+0000", 2024-09-18T10:15:39.336 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000100175 2024-09-18T10:15:39.336 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:39.336 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:39.336 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:39.336 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.336 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.336 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWrite(3.9s5 74/40 ECSubWrite(tid=179, reqid=osd.7.0:179, at_version=74'80, trim_to=0'0, roll_forward_to=72'76))", 2024-09-18T10:15:39.336 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:29.616697+0000", 2024-09-18T10:15:39.336 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 9.5783716939999994, 2024-09-18T10:15:39.336 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.35257461800000001, 2024-09-18T10:15:39.336 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:39.336 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:39.337 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:39.337 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.337 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:39.337 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:29.616697+0000", 2024-09-18T10:15:39.337 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:39.337 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.337 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.337 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:39.337 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:29.616697+0000", 2024-09-18T10:15:39.337 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:39.337 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.337 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.337 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:39.337 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:29.616689+0000", 2024-09-18T10:15:39.338 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999914 2024-09-18T10:15:39.338 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.338 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.338 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:39.338 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:29.616734+0000", 2024-09-18T10:15:39.338 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.5073e-05 2024-09-18T10:15:39.338 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.338 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.338 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:39.338 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:29.616764+0000", 2024-09-18T10:15:39.338 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9536e-05 2024-09-18T10:15:39.338 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.338 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.339 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:39.339 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:29.616812+0000", 2024-09-18T10:15:39.339 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8127999999999997e-05 2024-09-18T10:15:39.339 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.339 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.339 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:39.339 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:29.867763+0000", 2024-09-18T10:15:39.339 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.25095157400000001 2024-09-18T10:15:39.339 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.339 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.339 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:39.339 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:29.867775+0000", 2024-09-18T10:15:39.339 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1501000000000001e-05 2024-09-18T10:15:39.339 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.340 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.340 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:39.340 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:29.969220+0000", 2024-09-18T10:15:39.340 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.101445459 2024-09-18T10:15:39.340 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.340 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.340 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:39.340 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:29.969272+0000", 2024-09-18T10:15:39.340 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1746000000000002e-05 2024-09-18T10:15:39.340 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:39.340 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:39.340 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:39.340 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.341 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.341 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDPGPushReply(3.19s0 79/74 [PushReplyOp(3:9a1a774c:::smithi18925896-38:19)])", 2024-09-18T10:15:39.341 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:35.704115+0000", 2024-09-18T10:15:39.341 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 3.4909537230000001, 2024-09-18T10:15:39.341 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.30460024099999999, 2024-09-18T10:15:39.341 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:39.341 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:39.341 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:39.341 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.341 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:39.341 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:35.704115+0000", 2024-09-18T10:15:39.341 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:39.341 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.341 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.342 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:39.342 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:35.704115+0000", 2024-09-18T10:15:39.342 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:39.342 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.342 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.342 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:39.342 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:35.704115+0000", 2024-09-18T10:15:39.342 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:39.342 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.342 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.342 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:39.342 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:35.704121+0000", 2024-09-18T10:15:39.342 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.434e-06 2024-09-18T10:15:39.342 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.343 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.343 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:39.343 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:35.704141+0000", 2024-09-18T10:15:39.343 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0299e-05 2024-09-18T10:15:39.343 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.343 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.343 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:39.343 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:35.704149+0000", 2024-09-18T10:15:39.343 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.3499999999999997e-06 2024-09-18T10:15:39.343 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.343 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.343 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:39.343 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:36.008607+0000", 2024-09-18T10:15:39.343 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.30445813700000002 2024-09-18T10:15:39.344 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.344 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.344 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:39.344 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:36.008716+0000", 2024-09-18T10:15:39.344 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000108385 2024-09-18T10:15:39.344 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:39.344 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:39.344 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:39.344 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.344 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.344 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDPGPushReply(3.19s0 80/74 [PushReplyOp(3:9ab7c4c0:::smithi18925896-46:head)])", 2024-09-18T10:15:39.344 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:35.810513+0000", 2024-09-18T10:15:39.344 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 3.3845560479999999, 2024-09-18T10:15:39.344 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.30170829700000001, 2024-09-18T10:15:39.345 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:39.345 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:39.345 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:39.345 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.345 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:39.345 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:35.810513+0000", 2024-09-18T10:15:39.345 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:39.345 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.345 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.345 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:39.345 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:35.810513+0000", 2024-09-18T10:15:39.345 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:39.345 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.346 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.346 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:39.346 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:35.810512+0000", 2024-09-18T10:15:39.346 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.999999 2024-09-18T10:15:39.346 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.346 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.346 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:39.346 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:35.810523+0000", 2024-09-18T10:15:39.346 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1109e-05 2024-09-18T10:15:39.346 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.346 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.346 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:39.346 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:35.810553+0000", 2024-09-18T10:15:39.346 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.955e-05 2024-09-18T10:15:39.347 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.347 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.347 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:39.347 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:35.810571+0000", 2024-09-18T10:15:39.347 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8076000000000001e-05 2024-09-18T10:15:39.347 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.347 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.347 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:39.347 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:36.112117+0000", 2024-09-18T10:15:39.347 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.30154614000000002 2024-09-18T10:15:39.347 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.347 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.347 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:39.347 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:36.112221+0000", 2024-09-18T10:15:39.348 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000104512 2024-09-18T10:15:39.348 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:39.348 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:39.348 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:39.348 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:39.348 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:39.348 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-18T10:15:39.348 INFO:tasks.ceph.osd.4.smithi029.stderr:2024-09-18T10:15:39.193+0000 7f550cf6e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22189) UID: 0 2024-09-18T10:15:39.349 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_ops_in_flight 2024-09-18T10:15:39.350 INFO:teuthology.orchestra.run.smithi029.stdout:osd.12: {} 2024-09-18T10:15:39.350 INFO:teuthology.orchestra.run.smithi029.stdout:osd.13: {} 2024-09-18T10:15:39.350 INFO:teuthology.orchestra.run.smithi029.stdout:osd.14: {} 2024-09-18T10:15:39.350 INFO:teuthology.orchestra.run.smithi029.stdout:osd.15: {} 2024-09-18T10:15:39.351 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T10:15:39.351 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T10:15:39.351 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T10:15:39.351 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T10:15:39.351 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T10:15:39.352 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T10:15:39.352 INFO:teuthology.orchestra.run.smithi029.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T10:15:39.352 INFO:teuthology.orchestra.run.smithi029.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T10:15:39.352 INFO:teuthology.orchestra.run.smithi029.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-18T10:15:39.352 INFO:teuthology.orchestra.run.smithi029.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-18T10:15:39.352 INFO:teuthology.orchestra.run.smithi029.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-18T10:15:39.352 INFO:teuthology.orchestra.run.smithi029.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-18T10:15:39.352 INFO:teuthology.orchestra.run.smithi029.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-18T10:15:39.352 INFO:teuthology.orchestra.run.smithi029.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-18T10:15:39.352 INFO:teuthology.orchestra.run.smithi029.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-18T10:15:39.352 INFO:teuthology.orchestra.run.smithi029.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-18T10:15:39.394 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-18T10:15:39.389+0000 7f302f2c3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22187) UID: 0 2024-09-18T10:15:39.454 INFO:tasks.ceph.osd.12.smithi029.stderr:2024-09-18T10:15:39.449+0000 7f8e4e627700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 22190) UID: 0 2024-09-18T10:15:39.506 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-18T10:15:39.506 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-18T10:15:39.507 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.507 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpReadReply(3.4s0 83/82 ECSubReadReply(tid=3, attrs_read=1))", 2024-09-18T10:15:39.507 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:39.489000+0000", 2024-09-18T10:15:39.507 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.016491781, 2024-09-18T10:15:39.507 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.016522524, 2024-09-18T10:15:39.507 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:39.507 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-18T10:15:39.507 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:39.507 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.507 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:39.507 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:39.489000+0000", 2024-09-18T10:15:39.507 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:39.507 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.508 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.508 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:39.508 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:39.489000+0000", 2024-09-18T10:15:39.508 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:39.508 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.508 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.508 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:39.508 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:39.488998+0000", 2024-09-18T10:15:39.508 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:39.508 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.508 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.508 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:39.508 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:39.489398+0000", 2024-09-18T10:15:39.509 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00040001799999999998 2024-09-18T10:15:39.509 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.509 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.509 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:39.509 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:39.489419+0000", 2024-09-18T10:15:39.509 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0733e-05 2024-09-18T10:15:39.509 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.509 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.509 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:39.509 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:39.489430+0000", 2024-09-18T10:15:39.509 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.065e-05 2024-09-18T10:15:39.509 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:39.509 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:39.510 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:39.510 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:39.510 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-18T10:15:39.510 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 1 2024-09-18T10:15:39.510 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-18T10:15:39.516 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_blocked_ops 2024-09-18T10:15:39.560 INFO:tasks.rados.rados.0.smithi189.stdout:303: done (5 left) 2024-09-18T10:15:39.560 INFO:tasks.rados.rados.0.smithi189.stdout:append oid 16 current snap is 26 2024-09-18T10:15:39.561 INFO:tasks.rados.rados.0.smithi189.stdout:304: seq_num 121 ranges {1703936=491520,2195456=802816,2998272=720896,3719168=32768} 2024-09-18T10:15:39.561 INFO:tasks.ceph.osd.4.smithi029.stderr:2024-09-18T10:15:39.557+0000 7f550cf6e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22189) UID: 0 2024-09-18T10:15:39.581 INFO:tasks.rados.rados.0.smithi189.stdout:304: writing smithi18925896-16 from 1703936 to 2195456 tid 1 2024-09-18T10:15:39.586 INFO:tasks.rados.rados.0.smithi189.stdout:304: writing smithi18925896-16 from 2195456 to 2998272 tid 2 2024-09-18T10:15:39.590 INFO:tasks.rados.rados.0.smithi189.stdout:304: writing smithi18925896-16 from 2998272 to 3719168 tid 3 2024-09-18T10:15:39.591 INFO:tasks.rados.rados.0.smithi189.stdout:304: writing smithi18925896-16 from 3719168 to 3751936 tid 4 2024-09-18T10:15:39.591 INFO:tasks.rados.rados.0.smithi189.stdout:append oid 23 current snap is 26 2024-09-18T10:15:39.591 INFO:tasks.rados.rados.0.smithi189.stdout:302: expect (ObjNum 119 snap 25 seq_num 119) 2024-09-18T10:15:39.624 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-18T10:15:39.625 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-18T10:15:39.625 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-18T10:15:39.625 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-18T10:15:39.626 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-18T10:15:39.626 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-18T10:15:39.635 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_historic_ops 2024-09-18T10:15:39.656 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-18T10:15:39.652+0000 7f00046a8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 22136) UID: 0 2024-09-18T10:15:39.681 INFO:tasks.rados.rados.0.smithi189.stdout:305: seq_num 122 ranges {1277952=507904,1785856=425984,2211840=442368,2654208=540672,3194880=147456} 2024-09-18T10:15:39.699 INFO:tasks.rados.rados.0.smithi189.stdout:305: writing smithi18925896-23 from 1277952 to 1785856 tid 1 2024-09-18T10:15:39.701 INFO:tasks.rados.rados.0.smithi189.stdout:305: writing smithi18925896-23 from 1785856 to 2211840 tid 2 2024-09-18T10:15:39.703 INFO:tasks.rados.rados.0.smithi189.stdout:305: writing smithi18925896-23 from 2211840 to 2654208 tid 3 2024-09-18T10:15:39.707 INFO:tasks.rados.rados.0.smithi189.stdout:305: writing smithi18925896-23 from 2654208 to 3194880 tid 4 2024-09-18T10:15:39.708 INFO:tasks.rados.rados.0.smithi189.stdout:305: writing smithi18925896-23 from 3194880 to 3342336 tid 5 2024-09-18T10:15:39.708 INFO:tasks.rados.rados.0.smithi189.stdout:302: done (6 left) 2024-09-18T10:15:39.708 INFO:tasks.rados.rados.0.smithi189.stdout:306: snap_create 2024-09-18T10:15:39.741 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-18T10:15:39.742 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-18T10:15:39.742 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-18T10:15:39.742 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-18T10:15:39.742 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.742 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWrite(3.bs3 36/32 ECSubWrite(tid=29, reqid=client.4632.0:119, at_version=0'0, trim_to=0'0, roll_forward_to=36'4))", 2024-09-18T10:15:39.742 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:51.111475+0000", 2024-09-18T10:15:39.742 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 48.626484327999997, 2024-09-18T10:15:39.742 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.101555193, 2024-09-18T10:15:39.742 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:39.742 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:39.742 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:39.742 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.742 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:39.742 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.111475+0000", 2024-09-18T10:15:39.743 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:39.743 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.743 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.743 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:39.743 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.111475+0000", 2024-09-18T10:15:39.743 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:39.743 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.743 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.743 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:39.743 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.111474+0000", 2024-09-18T10:15:39.743 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:39.743 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.744 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.744 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:39.744 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.111492+0000", 2024-09-18T10:15:39.744 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7985000000000001e-05 2024-09-18T10:15:39.744 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.744 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.745 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:39.745 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.111511+0000", 2024-09-18T10:15:39.745 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8584e-05 2024-09-18T10:15:39.745 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.745 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.745 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:39.745 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.111521+0000", 2024-09-18T10:15:39.745 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0513000000000001e-05 2024-09-18T10:15:39.745 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.745 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.745 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:39.745 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.211779+0000", 2024-09-18T10:15:39.746 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10025748800000001 2024-09-18T10:15:39.746 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.746 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.746 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:39.746 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.211796+0000", 2024-09-18T10:15:39.746 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7655999999999998e-05 2024-09-18T10:15:39.746 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.746 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.746 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:39.746 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.212965+0000", 2024-09-18T10:15:39.747 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0011683679999999999 2024-09-18T10:15:39.747 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.747 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.747 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:39.747 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.213031+0000", 2024-09-18T10:15:39.747 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.5962000000000003e-05 2024-09-18T10:15:39.747 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:39.748 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:39.748 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:39.748 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.748 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.748 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWrite(3.cs5 36/32 ECSubWrite(tid=20, reqid=client.4632.0:150, at_version=36'9, trim_to=0'0, roll_forward_to=36'8))", 2024-09-18T10:15:39.748 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:51.127656+0000", 2024-09-18T10:15:39.748 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 48.610303723000001, 2024-09-18T10:15:39.748 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10369326700000001, 2024-09-18T10:15:39.748 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:39.748 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:39.748 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:39.748 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.749 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:39.749 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.127656+0000", 2024-09-18T10:15:39.749 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:39.749 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.749 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.749 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:39.749 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.127656+0000", 2024-09-18T10:15:39.749 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:39.749 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.749 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.749 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:39.749 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.127655+0000", 2024-09-18T10:15:39.749 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.999999 2024-09-18T10:15:39.750 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.750 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.750 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:39.750 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.127882+0000", 2024-09-18T10:15:39.750 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00022688799999999999 2024-09-18T10:15:39.750 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.750 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.750 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:39.750 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.127906+0000", 2024-09-18T10:15:39.751 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.4749e-05 2024-09-18T10:15:39.751 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.751 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.751 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:39.751 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.127917+0000", 2024-09-18T10:15:39.751 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0344e-05 2024-09-18T10:15:39.751 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.751 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.751 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:39.752 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.228119+0000", 2024-09-18T10:15:39.752 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100201944 2024-09-18T10:15:39.752 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.752 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.752 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:39.752 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.228135+0000", 2024-09-18T10:15:39.752 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.6354e-05 2024-09-18T10:15:39.752 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.752 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.752 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:39.752 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.231252+0000", 2024-09-18T10:15:39.752 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0031171200000000001 2024-09-18T10:15:39.752 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.753 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.753 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:39.753 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.231349+0000", 2024-09-18T10:15:39.753 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.7021999999999994e-05 2024-09-18T10:15:39.753 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:39.753 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:39.753 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:39.753 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.753 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.753 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWrite(3.cs5 36/32 ECSubWrite(tid=21, reqid=client.4632.0:151, at_version=36'10, trim_to=0'0, roll_forward_to=36'8))", 2024-09-18T10:15:39.753 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:51.130378+0000", 2024-09-18T10:15:39.753 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 48.607581664000001, 2024-09-18T10:15:39.753 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10150170999999999, 2024-09-18T10:15:39.753 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:39.754 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:39.754 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:39.754 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.754 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:39.754 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.130378+0000", 2024-09-18T10:15:39.754 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:39.754 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.754 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.754 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:39.754 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.130378+0000", 2024-09-18T10:15:39.754 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:39.754 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.754 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.755 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:39.755 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.130375+0000", 2024-09-18T10:15:39.755 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999967 2024-09-18T10:15:39.755 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.755 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.755 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:39.755 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.130518+0000", 2024-09-18T10:15:39.755 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00014364299999999999 2024-09-18T10:15:39.755 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.755 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.755 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:39.755 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.130540+0000", 2024-09-18T10:15:39.755 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2072000000000001e-05 2024-09-18T10:15:39.755 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.755 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.756 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:39.756 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.130553+0000", 2024-09-18T10:15:39.756 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2432999999999999e-05 2024-09-18T10:15:39.756 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.756 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.756 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:39.756 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.230723+0000", 2024-09-18T10:15:39.756 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10016994899999999 2024-09-18T10:15:39.756 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.756 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.756 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:39.756 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.230734+0000", 2024-09-18T10:15:39.756 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0874e-05 2024-09-18T10:15:39.757 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.757 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.757 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:39.757 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.231854+0000", 2024-09-18T10:15:39.757 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0011205399999999999 2024-09-18T10:15:39.757 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.757 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.757 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:39.757 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.231880+0000", 2024-09-18T10:15:39.757 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.5571e-05 2024-09-18T10:15:39.757 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:39.758 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:39.758 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:39.758 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.758 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.758 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWrite(3.es6 36/32 ECSubWrite(tid=4, reqid=client.4632.0:146, at_version=36'3, trim_to=0'0, roll_forward_to=36'2))", 2024-09-18T10:15:39.758 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:51.226533+0000", 2024-09-18T10:15:39.758 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 48.511426427000004, 2024-09-18T10:15:39.758 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.102071464, 2024-09-18T10:15:39.758 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:39.758 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:39.758 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:39.758 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.758 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:39.759 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.226533+0000", 2024-09-18T10:15:39.759 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:39.759 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.759 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.759 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:39.759 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.226533+0000", 2024-09-18T10:15:39.759 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:39.759 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.759 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.759 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:39.759 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.226531+0000", 2024-09-18T10:15:39.759 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:39.759 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.759 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.760 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:39.760 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.226807+0000", 2024-09-18T10:15:39.760 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00027599800000000001 2024-09-18T10:15:39.760 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.760 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.760 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:39.760 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.226849+0000", 2024-09-18T10:15:39.760 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1439000000000001e-05 2024-09-18T10:15:39.760 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.760 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.760 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:39.760 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.326947+0000", 2024-09-18T10:15:39.761 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100098859 2024-09-18T10:15:39.761 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.761 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.761 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:39.761 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.327061+0000", 2024-09-18T10:15:39.761 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00011343199999999999 2024-09-18T10:15:39.761 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.761 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.761 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:39.761 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.327083+0000", 2024-09-18T10:15:39.761 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2509000000000001e-05 2024-09-18T10:15:39.761 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.761 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.762 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:39.762 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.328552+0000", 2024-09-18T10:15:39.762 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0014686339999999999 2024-09-18T10:15:39.762 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.762 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.762 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:39.762 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.328605+0000", 2024-09-18T10:15:39.762 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2639000000000002e-05 2024-09-18T10:15:39.762 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:39.762 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:39.762 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:39.762 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.763 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.763 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWrite(3.bs3 36/32 ECSubWrite(tid=37, reqid=client.4632.0:160, at_version=0'0, trim_to=0'0, roll_forward_to=36'5))", 2024-09-18T10:15:39.763 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:51.531559+0000", 2024-09-18T10:15:39.763 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 48.206400238000001, 2024-09-18T10:15:39.763 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10164537799999999, 2024-09-18T10:15:39.763 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:39.763 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:39.763 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:39.763 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.763 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:39.764 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.531559+0000", 2024-09-18T10:15:39.764 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:39.764 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.764 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.764 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:39.764 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.531559+0000", 2024-09-18T10:15:39.764 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:39.764 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.764 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.764 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:39.764 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.531558+0000", 2024-09-18T10:15:39.764 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:39.765 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.765 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.765 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:39.765 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.531584+0000", 2024-09-18T10:15:39.765 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6066000000000001e-05 2024-09-18T10:15:39.765 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.765 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.765 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:39.765 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.531620+0000", 2024-09-18T10:15:39.765 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.6625999999999998e-05 2024-09-18T10:15:39.765 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.765 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.765 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:39.766 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.531635+0000", 2024-09-18T10:15:39.766 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4175999999999999e-05 2024-09-18T10:15:39.766 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.766 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.766 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:39.766 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.631831+0000", 2024-09-18T10:15:39.766 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10019641 2024-09-18T10:15:39.766 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.766 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.766 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:39.766 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.631859+0000", 2024-09-18T10:15:39.766 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.8289e-05 2024-09-18T10:15:39.766 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.766 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.767 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:39.767 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.633127+0000", 2024-09-18T10:15:39.767 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001267287 2024-09-18T10:15:39.767 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.767 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.767 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:39.767 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.633205+0000", 2024-09-18T10:15:39.767 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.8244000000000005e-05 2024-09-18T10:15:39.767 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:39.767 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:39.767 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:39.767 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.767 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.767 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWrite(3.cs5 37/32 ECSubWrite(tid=29, reqid=client.4632.0:181, at_version=37'13, trim_to=0'0, roll_forward_to=36'11))", 2024-09-18T10:15:39.768 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:52.895578+0000", 2024-09-18T10:15:39.768 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 46.842381168999999, 2024-09-18T10:15:39.768 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.102246569, 2024-09-18T10:15:39.768 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:39.768 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:39.768 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:39.768 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.768 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:39.768 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.895578+0000", 2024-09-18T10:15:39.768 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:39.768 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.768 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.768 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:39.768 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.895578+0000", 2024-09-18T10:15:39.769 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:39.769 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.769 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.769 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:39.769 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.895578+0000", 2024-09-18T10:15:39.769 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:39.769 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.769 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.769 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:39.769 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.895958+0000", 2024-09-18T10:15:39.769 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00038039299999999997 2024-09-18T10:15:39.769 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.769 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.769 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:39.770 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.895978+0000", 2024-09-18T10:15:39.770 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9896999999999999e-05 2024-09-18T10:15:39.770 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.770 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.770 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:39.770 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.895989+0000", 2024-09-18T10:15:39.770 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0438000000000001e-05 2024-09-18T10:15:39.770 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.770 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.770 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:39.770 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.996171+0000", 2024-09-18T10:15:39.770 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10018293 2024-09-18T10:15:39.770 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.770 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.771 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:39.771 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.996182+0000", 2024-09-18T10:15:39.771 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.006e-05 2024-09-18T10:15:39.771 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.771 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.771 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:39.771 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.997758+0000", 2024-09-18T10:15:39.771 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0015764520000000001 2024-09-18T10:15:39.771 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.771 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.771 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:39.771 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.997825+0000", 2024-09-18T10:15:39.771 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7061e-05 2024-09-18T10:15:39.771 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:39.772 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:39.772 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:39.772 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.772 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.772 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWrite(3.cs5 37/32 ECSubWrite(tid=33, reqid=client.4632.0:197, at_version=37'18, trim_to=0'0, roll_forward_to=36'11))", 2024-09-18T10:15:39.772 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:53.108605+0000", 2024-09-18T10:15:39.772 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 46.629354401999997, 2024-09-18T10:15:39.772 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.20153053300000001, 2024-09-18T10:15:39.772 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:39.772 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:39.772 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:39.772 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.772 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:39.773 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:53.108605+0000", 2024-09-18T10:15:39.773 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:39.773 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.773 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.773 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:39.773 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:53.108605+0000", 2024-09-18T10:15:39.773 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:39.773 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.773 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.773 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:39.773 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:53.108604+0000", 2024-09-18T10:15:39.773 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.999999 2024-09-18T10:15:39.773 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.774 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.774 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:39.774 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:53.108627+0000", 2024-09-18T10:15:39.774 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2954000000000001e-05 2024-09-18T10:15:39.774 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.774 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.774 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:39.774 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:53.108644+0000", 2024-09-18T10:15:39.775 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.6867e-05 2024-09-18T10:15:39.775 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.775 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.775 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:39.775 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:53.208724+0000", 2024-09-18T10:15:39.775 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10007999099999999 2024-09-18T10:15:39.776 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.776 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.776 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:39.776 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:53.308920+0000", 2024-09-18T10:15:39.776 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100196046 2024-09-18T10:15:39.776 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.776 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.776 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:39.776 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:53.308936+0000", 2024-09-18T10:15:39.776 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5537e-05 2024-09-18T10:15:39.776 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.777 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.777 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:39.777 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:53.310086+0000", 2024-09-18T10:15:39.777 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0011503850000000001 2024-09-18T10:15:39.777 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.777 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.777 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:39.777 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:53.310136+0000", 2024-09-18T10:15:39.777 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9514000000000001e-05 2024-09-18T10:15:39.777 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:39.777 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:39.777 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:39.777 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.778 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.778 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWrite(3.bs3 41/32 ECSubWrite(tid=58, reqid=osd.4.0:58, at_version=41'14, trim_to=0'0, roll_forward_to=39'12))", 2024-09-18T10:15:39.778 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:56.830265+0000", 2024-09-18T10:15:39.778 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 42.907694556999999, 2024-09-18T10:15:39.778 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10214519699999999, 2024-09-18T10:15:39.778 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:39.778 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:39.778 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:39.778 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.778 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:39.778 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.830265+0000", 2024-09-18T10:15:39.778 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:39.778 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.778 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.779 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:39.779 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.830265+0000", 2024-09-18T10:15:39.779 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:39.779 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.779 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.779 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:39.779 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.830263+0000", 2024-09-18T10:15:39.779 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:39.779 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.779 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.779 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:39.779 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.830323+0000", 2024-09-18T10:15:39.779 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9846000000000003e-05 2024-09-18T10:15:39.779 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.780 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.780 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:39.780 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.830370+0000", 2024-09-18T10:15:39.780 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6798999999999997e-05 2024-09-18T10:15:39.780 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.780 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.780 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:39.780 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.830389+0000", 2024-09-18T10:15:39.780 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9530000000000001e-05 2024-09-18T10:15:39.780 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.780 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.780 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:39.780 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.930612+0000", 2024-09-18T10:15:39.781 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100222795 2024-09-18T10:15:39.781 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.781 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.781 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:39.781 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.930648+0000", 2024-09-18T10:15:39.781 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.6115999999999998e-05 2024-09-18T10:15:39.781 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.781 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.781 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:39.781 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.932338+0000", 2024-09-18T10:15:39.781 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0016898130000000001 2024-09-18T10:15:39.781 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.781 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.782 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:39.782 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.932410+0000", 2024-09-18T10:15:39.782 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.2214999999999999e-05 2024-09-18T10:15:39.782 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:39.782 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:39.782 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:39.782 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.782 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.782 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWrite(3.bs3 41/32 ECSubWrite(tid=62, reqid=osd.4.0:58, at_version=0'0, trim_to=0'0, roll_forward_to=41'14))", 2024-09-18T10:15:39.782 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:57.647903+0000", 2024-09-18T10:15:39.782 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 42.090056586000003, 2024-09-18T10:15:39.782 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.20108230399999999, 2024-09-18T10:15:39.782 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:39.782 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:39.783 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:39.783 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.783 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:39.783 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.647903+0000", 2024-09-18T10:15:39.783 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:39.783 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.783 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.783 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:39.783 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.647903+0000", 2024-09-18T10:15:39.783 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:39.783 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.783 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.783 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:39.783 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.647901+0000", 2024-09-18T10:15:39.784 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999976 2024-09-18T10:15:39.784 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.784 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.784 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:39.784 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.647936+0000", 2024-09-18T10:15:39.784 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.5329e-05 2024-09-18T10:15:39.784 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.784 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.784 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:39.784 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.647978+0000", 2024-09-18T10:15:39.784 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2145999999999999e-05 2024-09-18T10:15:39.784 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.784 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.784 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:39.785 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.647996+0000", 2024-09-18T10:15:39.785 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7331999999999999e-05 2024-09-18T10:15:39.785 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.785 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.785 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:39.785 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.748248+0000", 2024-09-18T10:15:39.785 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100252307 2024-09-18T10:15:39.785 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.785 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.785 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:39.785 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.748280+0000", 2024-09-18T10:15:39.785 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2316e-05 2024-09-18T10:15:39.785 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.786 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.786 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:39.786 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.848945+0000", 2024-09-18T10:15:39.786 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100664434 2024-09-18T10:15:39.786 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.786 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.786 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:39.786 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.848985+0000", 2024-09-18T10:15:39.786 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0605999999999997e-05 2024-09-18T10:15:39.786 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:39.786 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:39.786 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:39.786 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.786 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.787 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWrite(3.bs3 46/32 ECSubWrite(tid=75, reqid=client.4632.0:327, at_version=0'0, trim_to=0'0, roll_forward_to=46'17))", 2024-09-18T10:15:39.787 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:02.228270+0000", 2024-09-18T10:15:39.787 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 37.509689899999998, 2024-09-18T10:15:39.787 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.101562179, 2024-09-18T10:15:39.787 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:39.787 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:39.787 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:39.787 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.787 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:39.787 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.228270+0000", 2024-09-18T10:15:39.787 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:39.787 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.788 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.788 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:39.788 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.228270+0000", 2024-09-18T10:15:39.788 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:39.788 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.788 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.788 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:39.788 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.228268+0000", 2024-09-18T10:15:39.788 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:39.788 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.788 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.788 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:39.788 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.228295+0000", 2024-09-18T10:15:39.789 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6809e-05 2024-09-18T10:15:39.789 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.789 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.789 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:39.789 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.228369+0000", 2024-09-18T10:15:39.789 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.3645000000000004e-05 2024-09-18T10:15:39.789 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.789 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.789 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:39.789 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.228388+0000", 2024-09-18T10:15:39.789 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9403e-05 2024-09-18T10:15:39.789 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.789 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.789 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:39.790 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.328674+0000", 2024-09-18T10:15:39.790 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100285687 2024-09-18T10:15:39.790 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.790 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.790 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:39.790 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.328690+0000", 2024-09-18T10:15:39.790 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5956000000000001e-05 2024-09-18T10:15:39.790 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.790 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.790 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:39.790 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.329701+0000", 2024-09-18T10:15:39.790 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0010117609999999999 2024-09-18T10:15:39.791 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.791 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.791 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:39.791 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.329832+0000", 2024-09-18T10:15:39.791 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000130541 2024-09-18T10:15:39.791 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:39.791 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:39.791 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:39.791 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.792 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.792 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWrite(3.es6 47/32 ECSubWrite(tid=21, reqid=client.4632.0:346, at_version=47'17, trim_to=0'0, roll_forward_to=44'15))", 2024-09-18T10:15:39.792 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:02.905682+0000", 2024-09-18T10:15:39.792 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 36.832277677, 2024-09-18T10:15:39.792 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.104106903, 2024-09-18T10:15:39.792 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:39.792 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:39.792 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:39.792 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.792 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:39.792 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.905682+0000", 2024-09-18T10:15:39.792 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:39.792 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.793 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.793 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:39.793 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.905682+0000", 2024-09-18T10:15:39.793 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:39.793 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.793 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.793 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:39.793 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.905682+0000", 2024-09-18T10:15:39.793 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:39.793 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.793 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.793 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:39.793 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.906384+0000", 2024-09-18T10:15:39.794 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000702362 2024-09-18T10:15:39.794 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.794 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.794 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:39.794 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.906428+0000", 2024-09-18T10:15:39.794 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4199000000000002e-05 2024-09-18T10:15:39.794 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.794 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.794 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:39.794 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:03.006578+0000", 2024-09-18T10:15:39.794 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100149618 2024-09-18T10:15:39.794 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.794 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.794 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:39.795 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:03.006758+0000", 2024-09-18T10:15:39.795 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00018075599999999999 2024-09-18T10:15:39.795 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.795 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.795 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:39.795 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:03.006775+0000", 2024-09-18T10:15:39.795 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.6673000000000001e-05 2024-09-18T10:15:39.795 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.795 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.795 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:39.795 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:03.009712+0000", 2024-09-18T10:15:39.795 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.002936569 2024-09-18T10:15:39.795 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.796 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.796 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:39.796 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:03.009789+0000", 2024-09-18T10:15:39.796 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7169000000000006e-05 2024-09-18T10:15:39.796 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:39.796 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:39.796 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:39.796 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.796 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.796 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWrite(3.cs5 47/32 ECSubWrite(tid=60, reqid=client.4632.0:341, at_version=47'38, trim_to=0'0, roll_forward_to=45'33))", 2024-09-18T10:15:39.796 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:02.911299+0000", 2024-09-18T10:15:39.796 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 36.826661051000002, 2024-09-18T10:15:39.796 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.101776582, 2024-09-18T10:15:39.796 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:39.797 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:39.797 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:39.797 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.797 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:39.797 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.911299+0000", 2024-09-18T10:15:39.797 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:39.797 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.797 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.797 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:39.797 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.911299+0000", 2024-09-18T10:15:39.797 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:39.797 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.797 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.797 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:39.798 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.911298+0000", 2024-09-18T10:15:39.798 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.999999 2024-09-18T10:15:39.798 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.798 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.798 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:39.798 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.911361+0000", 2024-09-18T10:15:39.798 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3900999999999997e-05 2024-09-18T10:15:39.798 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.798 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.798 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:39.798 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.911392+0000", 2024-09-18T10:15:39.798 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0275000000000001e-05 2024-09-18T10:15:39.798 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.799 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.799 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:39.799 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.911404+0000", 2024-09-18T10:15:39.799 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2682e-05 2024-09-18T10:15:39.799 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.799 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.799 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:39.799 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:03.011587+0000", 2024-09-18T10:15:39.799 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10018239 2024-09-18T10:15:39.799 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.799 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.799 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:39.799 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:03.011607+0000", 2024-09-18T10:15:39.799 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0329000000000001e-05 2024-09-18T10:15:39.800 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.800 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.800 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:39.800 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:03.013018+0000", 2024-09-18T10:15:39.800 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001410863 2024-09-18T10:15:39.800 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.800 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.800 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:39.800 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:03.013075+0000", 2024-09-18T10:15:39.800 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7182000000000002e-05 2024-09-18T10:15:39.800 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:39.800 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:39.800 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:39.801 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.801 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.801 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDScrubReserve(3.15s4 RELEASE e52)", 2024-09-18T10:15:39.801 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:07.816881+0000", 2024-09-18T10:15:39.801 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 31.921078219000002, 2024-09-18T10:15:39.801 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.103296636, 2024-09-18T10:15:39.801 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:39.801 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:39.801 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:39.801 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.801 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:39.801 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.816881+0000", 2024-09-18T10:15:39.801 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:39.801 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.802 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.802 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:39.802 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.816881+0000", 2024-09-18T10:15:39.802 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:39.802 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.802 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.802 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:39.802 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.816881+0000", 2024-09-18T10:15:39.802 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:39.803 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.803 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.803 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:39.803 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.816886+0000", 2024-09-18T10:15:39.803 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0769999999999998e-06 2024-09-18T10:15:39.803 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.803 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.803 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:39.803 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.816911+0000", 2024-09-18T10:15:39.803 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.495e-05 2024-09-18T10:15:39.803 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.803 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.803 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:39.803 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.816926+0000", 2024-09-18T10:15:39.804 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5148e-05 2024-09-18T10:15:39.804 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.804 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.804 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:39.804 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.817027+0000", 2024-09-18T10:15:39.804 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00010067499999999999 2024-09-18T10:15:39.804 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.804 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.804 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op must wait for map", 2024-09-18T10:15:39.804 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.817046+0000", 2024-09-18T10:15:39.804 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8836999999999999e-05 2024-09-18T10:15:39.804 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.804 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.805 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:39.805 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.920112+0000", 2024-09-18T10:15:39.805 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.103066402 2024-09-18T10:15:39.805 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.805 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.805 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:39.805 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.920137+0000", 2024-09-18T10:15:39.805 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.4454e-05 2024-09-18T10:15:39.805 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.805 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.805 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:39.805 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.920178+0000", 2024-09-18T10:15:39.805 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1570999999999999e-05 2024-09-18T10:15:39.806 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:39.806 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:39.806 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:39.806 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.806 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.806 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWrite(3.cs5 58/32 ECSubWrite(tid=78, reqid=osd.5.0:77, at_version=0'0, trim_to=0'0, roll_forward_to=57'56))", 2024-09-18T10:15:39.806 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:13.858719+0000", 2024-09-18T10:15:39.806 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 25.879240554999999, 2024-09-18T10:15:39.806 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.105635402, 2024-09-18T10:15:39.806 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:39.806 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:39.806 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:39.806 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.807 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:39.807 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.858719+0000", 2024-09-18T10:15:39.807 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:39.807 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.807 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.807 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:39.807 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.858719+0000", 2024-09-18T10:15:39.807 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:39.807 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.807 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.807 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:39.807 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.858719+0000", 2024-09-18T10:15:39.807 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:39.807 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.808 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.808 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:39.808 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.858731+0000", 2024-09-18T10:15:39.808 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2007999999999999e-05 2024-09-18T10:15:39.808 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.808 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.808 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:39.808 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.858755+0000", 2024-09-18T10:15:39.808 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.4681e-05 2024-09-18T10:15:39.808 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.808 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.808 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:39.808 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.958911+0000", 2024-09-18T10:15:39.808 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10015579300000001 2024-09-18T10:15:39.809 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.809 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.809 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:39.809 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.962752+0000", 2024-09-18T10:15:39.809 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0038403259999999998 2024-09-18T10:15:39.809 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.809 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.809 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:39.809 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.962767+0000", 2024-09-18T10:15:39.809 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5494000000000001e-05 2024-09-18T10:15:39.809 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.809 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.809 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:39.810 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.964314+0000", 2024-09-18T10:15:39.810 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0015469189999999999 2024-09-18T10:15:39.810 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.810 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.810 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:39.810 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.964355+0000", 2024-09-18T10:15:39.810 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0491000000000002e-05 2024-09-18T10:15:39.810 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:39.810 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:39.810 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:39.811 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.811 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.811 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWrite(3.cs5 74/32 ECSubWrite(tid=119, reqid=client.4632.0:651, at_version=74'93, trim_to=0'0, roll_forward_to=74'91))", 2024-09-18T10:15:39.811 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:29.861325+0000", 2024-09-18T10:15:39.811 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 9.8766344030000006, 2024-09-18T10:15:39.811 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.103081672, 2024-09-18T10:15:39.811 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:39.811 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:39.811 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:39.811 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.811 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:39.811 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:29.861325+0000", 2024-09-18T10:15:39.811 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:39.812 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.812 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.812 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:39.812 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:29.861325+0000", 2024-09-18T10:15:39.812 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:39.812 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.812 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.812 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:39.812 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:29.861324+0000", 2024-09-18T10:15:39.812 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.999999 2024-09-18T10:15:39.812 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.812 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.812 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:39.813 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:29.861391+0000", 2024-09-18T10:15:39.813 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.6503999999999999e-05 2024-09-18T10:15:39.813 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.813 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.813 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:39.813 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:29.861417+0000", 2024-09-18T10:15:39.813 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6219000000000001e-05 2024-09-18T10:15:39.813 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.813 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.813 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:39.813 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:29.961533+0000", 2024-09-18T10:15:39.813 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100116302 2024-09-18T10:15:39.813 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.814 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.814 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:39.814 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:29.961634+0000", 2024-09-18T10:15:39.814 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000101307 2024-09-18T10:15:39.814 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.814 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.814 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:39.814 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:29.961649+0000", 2024-09-18T10:15:39.814 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4963999999999999e-05 2024-09-18T10:15:39.814 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.814 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.814 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:39.815 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:29.964376+0000", 2024-09-18T10:15:39.815 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0027269299999999998 2024-09-18T10:15:39.815 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.815 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.815 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:39.815 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:29.964407+0000", 2024-09-18T10:15:39.815 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0608999999999998e-05 2024-09-18T10:15:39.815 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:39.815 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:39.815 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:39.815 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.815 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.815 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWrite(3.cs5 75/32 ECSubWrite(tid=121, reqid=osd.5.0:121, at_version=75'95, trim_to=0'0, roll_forward_to=74'93))", 2024-09-18T10:15:39.816 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:30.665025+0000", 2024-09-18T10:15:39.816 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 9.0729344039999997, 2024-09-18T10:15:39.816 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10202127499999999, 2024-09-18T10:15:39.816 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:39.816 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:39.816 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:39.816 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.816 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:39.816 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:30.665025+0000", 2024-09-18T10:15:39.816 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:39.816 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.816 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.816 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:39.816 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:30.665025+0000", 2024-09-18T10:15:39.817 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:39.817 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.817 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.817 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:39.817 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:30.665025+0000", 2024-09-18T10:15:39.817 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:39.817 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.817 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.817 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:39.817 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:30.665084+0000", 2024-09-18T10:15:39.817 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.9861000000000002e-05 2024-09-18T10:15:39.817 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.817 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.817 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:39.818 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:30.665122+0000", 2024-09-18T10:15:39.818 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7863e-05 2024-09-18T10:15:39.818 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.818 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.818 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:39.818 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:30.765239+0000", 2024-09-18T10:15:39.818 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100116569 2024-09-18T10:15:39.818 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.818 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.818 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:39.818 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:30.765331+0000", 2024-09-18T10:15:39.818 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.2e-05 2024-09-18T10:15:39.818 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.818 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.819 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:39.819 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:30.765346+0000", 2024-09-18T10:15:39.819 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5126000000000001e-05 2024-09-18T10:15:39.819 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.819 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.819 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:39.819 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:30.767018+0000", 2024-09-18T10:15:39.819 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0016717209999999999 2024-09-18T10:15:39.819 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.819 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.819 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:39.819 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:30.767047+0000", 2024-09-18T10:15:39.819 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.8815000000000001e-05 2024-09-18T10:15:39.819 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:39.820 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:39.820 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:39.820 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.820 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.820 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWrite(3.bs3 75/32 ECSubWrite(tid=172, reqid=osd.4.0:172, at_version=75'29, trim_to=0'0, roll_forward_to=74'27))", 2024-09-18T10:15:39.820 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:30.860042+0000", 2024-09-18T10:15:39.820 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 8.8779179050000003, 2024-09-18T10:15:39.820 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.101783085, 2024-09-18T10:15:39.820 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:39.820 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:39.820 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:39.820 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.820 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:39.821 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:30.860042+0000", 2024-09-18T10:15:39.821 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:39.821 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.821 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.821 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:39.821 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:30.860042+0000", 2024-09-18T10:15:39.821 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:39.821 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.821 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.821 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:39.821 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:30.860040+0000", 2024-09-18T10:15:39.821 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:39.821 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.821 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.822 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:39.822 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:30.860081+0000", 2024-09-18T10:15:39.822 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1424000000000002e-05 2024-09-18T10:15:39.822 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.822 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.822 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:39.822 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:30.860106+0000", 2024-09-18T10:15:39.822 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.5284999999999999e-05 2024-09-18T10:15:39.822 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.822 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.822 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:39.822 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:30.860117+0000", 2024-09-18T10:15:39.822 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0942e-05 2024-09-18T10:15:39.823 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.823 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.823 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:39.823 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:30.960368+0000", 2024-09-18T10:15:39.823 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10025012799999999 2024-09-18T10:15:39.823 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.823 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.823 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:39.823 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:30.960384+0000", 2024-09-18T10:15:39.823 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.6697e-05 2024-09-18T10:15:39.823 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.823 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.823 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:39.823 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:30.961755+0000", 2024-09-18T10:15:39.823 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001370867 2024-09-18T10:15:39.824 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.824 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.824 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:39.824 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:30.961825+0000", 2024-09-18T10:15:39.824 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9777999999999995e-05 2024-09-18T10:15:39.824 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:39.824 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:39.824 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:39.824 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.824 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.824 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWrite(3.es6 80/32 ECSubWrite(tid=68, reqid=client.4632.0:694, at_version=0'0, trim_to=0'0, roll_forward_to=80'58))", 2024-09-18T10:15:39.824 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:35.985863+0000", 2024-09-18T10:15:39.824 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 3.7520962839999998, 2024-09-18T10:15:39.825 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10176993600000001, 2024-09-18T10:15:39.825 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:39.825 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:39.825 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:39.825 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.825 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:39.825 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:35.985863+0000", 2024-09-18T10:15:39.825 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:39.825 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.825 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.825 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:39.825 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:35.985863+0000", 2024-09-18T10:15:39.825 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:39.826 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.826 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.826 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:39.826 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:35.985862+0000", 2024-09-18T10:15:39.826 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:39.826 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.826 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.826 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:39.826 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:35.985887+0000", 2024-09-18T10:15:39.826 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.499e-05 2024-09-18T10:15:39.826 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.826 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.826 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:39.827 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:35.985923+0000", 2024-09-18T10:15:39.827 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.6736999999999997e-05 2024-09-18T10:15:39.827 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.827 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.827 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:39.827 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:35.985937+0000", 2024-09-18T10:15:39.827 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3619e-05 2024-09-18T10:15:39.827 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.827 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.827 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:39.827 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:36.086140+0000", 2024-09-18T10:15:39.827 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10020322600000001 2024-09-18T10:15:39.827 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.828 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.828 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:39.828 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:36.086158+0000", 2024-09-18T10:15:39.828 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7835000000000001e-05 2024-09-18T10:15:39.828 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.828 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.828 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:39.828 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:36.087556+0000", 2024-09-18T10:15:39.828 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0013977880000000001 2024-09-18T10:15:39.828 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.828 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.828 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:39.828 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:36.087633+0000", 2024-09-18T10:15:39.829 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7460999999999996e-05 2024-09-18T10:15:39.829 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:39.829 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:39.829 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:39.829 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.829 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.829 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpReadReply(3.4s0 83/82 ECSubReadReply(tid=3, attrs_read=1))", 2024-09-18T10:15:39.829 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:39.489000+0000", 2024-09-18T10:15:39.829 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.248959553, 2024-09-18T10:15:39.829 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.103026321, 2024-09-18T10:15:39.829 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:39.829 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:39.830 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:39.830 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.830 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:39.830 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:39.489000+0000", 2024-09-18T10:15:39.830 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:39.830 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.830 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.830 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:39.830 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:39.489000+0000", 2024-09-18T10:15:39.830 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:39.830 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.830 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.830 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:39.830 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:39.488998+0000", 2024-09-18T10:15:39.831 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:39.831 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.831 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.831 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:39.831 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:39.489398+0000", 2024-09-18T10:15:39.831 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00040001799999999998 2024-09-18T10:15:39.831 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.831 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.831 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:39.831 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:39.489419+0000", 2024-09-18T10:15:39.831 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0733e-05 2024-09-18T10:15:39.831 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.831 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.832 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:39.832 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:39.489430+0000", 2024-09-18T10:15:39.832 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.065e-05 2024-09-18T10:15:39.832 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.832 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.832 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:39.832 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:39.591044+0000", 2024-09-18T10:15:39.832 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.101613748 2024-09-18T10:15:39.832 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.832 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.832 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:39.832 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:39.592026+0000", 2024-09-18T10:15:39.832 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00098283799999999998 2024-09-18T10:15:39.832 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:39.833 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:39.833 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:39.833 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.833 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.833 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDPGPush(3.4s0 84/82 [PushOp(3:2650be24:::smithi18925896-45:head, version: 77'72, data_included: [786432~184320], data_size: 184320, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:2650be24:::smithi18925896-45:head@77'72, size: 3883008, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: 1), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:3883008, data_complete:true, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to:3145728, data_complete:false, omap_recovered_to:, omap_complete:true, error:false))])", 2024-09-18T10:15:39.833 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:39.597297+0000", 2024-09-18T10:15:39.833 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.140662289, 2024-09-18T10:15:39.833 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10230498, 2024-09-18T10:15:39.833 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:39.833 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:39.833 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:39.833 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.833 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:39.834 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:39.597297+0000", 2024-09-18T10:15:39.834 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:39.834 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.834 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.834 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:39.834 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:39.597297+0000", 2024-09-18T10:15:39.834 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:39.834 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.834 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.834 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:39.834 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:39.597297+0000", 2024-09-18T10:15:39.834 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:39.834 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.834 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.835 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:39.835 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:39.597297+0000", 2024-09-18T10:15:39.835 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:39.835 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.835 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.835 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:39.835 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:39.597320+0000", 2024-09-18T10:15:39.835 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2568000000000001e-05 2024-09-18T10:15:39.835 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.835 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.835 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:39.835 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:39.597328+0000", 2024-09-18T10:15:39.835 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.4740000000000005e-06 2024-09-18T10:15:39.835 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.836 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.836 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:39.836 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:39.697725+0000", 2024-09-18T10:15:39.836 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100396099 2024-09-18T10:15:39.836 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:39.836 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:39.836 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:39.836 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:39.699602+0000", 2024-09-18T10:15:39.836 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0018778390000000001 2024-09-18T10:15:39.836 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:39.836 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:39.836 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:39.836 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:39.837 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:39.837 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-18T10:15:39.838 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T10:15:39.839 INFO:tasks.ceph.osd.11.smithi189.stderr:2024-09-18T10:15:39.834+0000 7f718b60a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 22400) UID: 0 2024-09-18T10:15:39.939 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-18T10:15:39.936+0000 7f00046a8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 22136) UID: 0 2024-09-18T10:15:39.950 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-18T10:15:39.951 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-18T10:15:39.951 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-09-18T10:15:39.951 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-18T10:15:39.960 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T10:15:40.021 INFO:tasks.rados.rados.0.smithi189.stdout:292: finishing rollback tid 0 to smithi18925896-17 2024-09-18T10:15:40.022 INFO:tasks.rados.rados.0.smithi189.stdout:292: finishing rollback tid 1 to smithi18925896-17 2024-09-18T10:15:40.022 INFO:tasks.rados.rados.0.smithi189.stdout:292: finishing rollback tid 2 to smithi18925896-17 2024-09-18T10:15:40.022 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 17 v 109 (ObjNum 12 snap 0 seq_num 12) dirty exists 2024-09-18T10:15:40.041 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-18T10:15:40.036+0000 7f14c6a1a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 22138) UID: 0 2024-09-18T10:15:40.060 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-18T10:15:40.060 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-18T10:15:40.060 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-18T10:15:40.060 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-18T10:15:40.060 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-18T10:15:40.073 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T10:15:40.083 INFO:tasks.rados.rados.0.smithi189.stdout:300: finishing rollback tid 0 to smithi18925896-45 2024-09-18T10:15:40.114 INFO:tasks.rados.rados.0.smithi189.stdout:294: finishing write tid 3 to smithi18925896-20 2024-09-18T10:15:40.142 INFO:tasks.ceph.osd.3.smithi189.stderr:2024-09-18T10:15:40.138+0000 7f2440fc7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22396) UID: 0 2024-09-18T10:15:40.214 INFO:tasks.rados.rados.0.smithi189.stdout:294: finishing write tid 4 to smithi18925896-20 2024-09-18T10:15:40.215 INFO:tasks.rados.rados.0.smithi189.stdout:294: finishing write tid 5 to smithi18925896-20 2024-09-18T10:15:40.217 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-18T10:15:40.217 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-18T10:15:40.217 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-18T10:15:40.218 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-18T10:15:40.218 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.218 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.8s5 36/32 ECSubWrite(tid=119, reqid=client.4632.0:130, at_version=36'5, trim_to=0'0, roll_forward_to=35'4))", 2024-09-18T10:15:40.218 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:50.722395+0000", 2024-09-18T10:15:40.218 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 49.493376611000002, 2024-09-18T10:15:40.218 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10448325999999999, 2024-09-18T10:15:40.218 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:40.218 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:40.218 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:40.218 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.218 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:40.218 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.722395+0000", 2024-09-18T10:15:40.218 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:40.219 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.219 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.219 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:40.219 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.722395+0000", 2024-09-18T10:15:40.219 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:40.219 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.219 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.219 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:40.219 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.722393+0000", 2024-09-18T10:15:40.219 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:40.219 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.219 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.219 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:40.219 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.722993+0000", 2024-09-18T10:15:40.220 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00059969600000000004 2024-09-18T10:15:40.220 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.220 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.220 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:40.220 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.723025+0000", 2024-09-18T10:15:40.220 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2364999999999999e-05 2024-09-18T10:15:40.220 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.220 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.220 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:40.220 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.723039+0000", 2024-09-18T10:15:40.220 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3709999999999999e-05 2024-09-18T10:15:40.220 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.220 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.221 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:40.221 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.823280+0000", 2024-09-18T10:15:40.221 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100241126 2024-09-18T10:15:40.221 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.221 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.221 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:40.221 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.823298+0000", 2024-09-18T10:15:40.221 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7745999999999999e-05 2024-09-18T10:15:40.221 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.221 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.221 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:40.221 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.826820+0000", 2024-09-18T10:15:40.221 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0035221470000000002 2024-09-18T10:15:40.221 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.222 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.222 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:40.222 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.826878+0000", 2024-09-18T10:15:40.222 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8344000000000002e-05 2024-09-18T10:15:40.222 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:40.222 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:40.222 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:40.222 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.222 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.222 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.bs5 36/32 ECSubWrite(tid=26, reqid=client.4632.0:119, at_version=36'4, trim_to=0'0, roll_forward_to=35'2))", 2024-09-18T10:15:40.222 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:50.812131+0000", 2024-09-18T10:15:40.222 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 49.403640805000002, 2024-09-18T10:15:40.222 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.201739738, 2024-09-18T10:15:40.222 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:40.223 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:40.223 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:40.223 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.223 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:40.223 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.812131+0000", 2024-09-18T10:15:40.223 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:40.223 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.223 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.223 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:40.223 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.812131+0000", 2024-09-18T10:15:40.223 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:40.223 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.223 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.223 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:40.224 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.812129+0000", 2024-09-18T10:15:40.224 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:40.224 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.224 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.224 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:40.224 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.812166+0000", 2024-09-18T10:15:40.224 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6309000000000001e-05 2024-09-18T10:15:40.224 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.224 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.224 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:40.224 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.812199+0000", 2024-09-18T10:15:40.224 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3219000000000002e-05 2024-09-18T10:15:40.224 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.225 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.225 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:40.225 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.812225+0000", 2024-09-18T10:15:40.225 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6302999999999999e-05 2024-09-18T10:15:40.225 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.225 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.225 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:40.225 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.012537+0000", 2024-09-18T10:15:40.225 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20031156899999999 2024-09-18T10:15:40.225 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.225 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.225 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:40.225 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.012564+0000", 2024-09-18T10:15:40.225 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7165000000000002e-05 2024-09-18T10:15:40.226 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.226 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.226 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:40.226 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.013814+0000", 2024-09-18T10:15:40.226 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001249836 2024-09-18T10:15:40.226 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.226 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.226 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:40.226 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.013871+0000", 2024-09-18T10:15:40.226 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6894000000000001e-05 2024-09-18T10:15:40.226 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:40.226 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:40.226 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:40.226 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.227 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.227 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.2s3 41/40 ECSubWrite(tid=23, reqid=client.4632.0:265, at_version=41'11, trim_to=0'0, roll_forward_to=0'0))", 2024-09-18T10:15:40.227 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:57.236544+0000", 2024-09-18T10:15:40.227 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 42.979227553000001, 2024-09-18T10:15:40.227 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20209979, 2024-09-18T10:15:40.227 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:40.227 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:40.227 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:40.227 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.227 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:40.227 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.236544+0000", 2024-09-18T10:15:40.227 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:40.227 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.228 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.228 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:40.228 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.236544+0000", 2024-09-18T10:15:40.228 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:40.228 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.228 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.228 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:40.228 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.236543+0000", 2024-09-18T10:15:40.228 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:40.228 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.228 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.228 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:40.228 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.236820+0000", 2024-09-18T10:15:40.229 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000277068 2024-09-18T10:15:40.229 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.229 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.229 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:40.229 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.236876+0000", 2024-09-18T10:15:40.229 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6555000000000003e-05 2024-09-18T10:15:40.229 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.229 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.229 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:40.229 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.336982+0000", 2024-09-18T10:15:40.229 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100105679 2024-09-18T10:15:40.229 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.229 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.230 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:40.230 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.337122+0000", 2024-09-18T10:15:40.230 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000139777 2024-09-18T10:15:40.230 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.230 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.230 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:40.230 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.337135+0000", 2024-09-18T10:15:40.230 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3559e-05 2024-09-18T10:15:40.230 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.230 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.230 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:40.230 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.438600+0000", 2024-09-18T10:15:40.230 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.101464572 2024-09-18T10:15:40.230 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.230 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.231 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:40.231 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.438644+0000", 2024-09-18T10:15:40.231 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4085000000000002e-05 2024-09-18T10:15:40.231 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:40.231 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:40.231 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:40.231 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.231 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.231 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.2s3 41/40 ECSubWrite(tid=25, reqid=client.4632.0:266, at_version=0'0, trim_to=0'0, roll_forward_to=41'12))", 2024-09-18T10:15:40.231 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:57.543345+0000", 2024-09-18T10:15:40.231 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 42.672427143, 2024-09-18T10:15:40.231 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.18858536100000001, 2024-09-18T10:15:40.231 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:40.231 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:40.232 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:40.232 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.232 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:40.232 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.543345+0000", 2024-09-18T10:15:40.232 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:40.232 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.232 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.232 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:40.232 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.543345+0000", 2024-09-18T10:15:40.232 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:40.232 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.232 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.232 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:40.232 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.543343+0000", 2024-09-18T10:15:40.233 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:40.233 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.233 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.233 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:40.233 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.543385+0000", 2024-09-18T10:15:40.233 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2487999999999997e-05 2024-09-18T10:15:40.233 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.233 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.233 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:40.233 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.543428+0000", 2024-09-18T10:15:40.233 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2315999999999999e-05 2024-09-18T10:15:40.233 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.233 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.234 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:40.234 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.543446+0000", 2024-09-18T10:15:40.234 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8189000000000002e-05 2024-09-18T10:15:40.234 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.234 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.234 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:40.234 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.730560+0000", 2024-09-18T10:15:40.234 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.187114473 2024-09-18T10:15:40.234 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.234 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.234 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:40.234 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.730585+0000", 2024-09-18T10:15:40.234 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4652e-05 2024-09-18T10:15:40.235 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.235 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.235 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:40.235 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.731859+0000", 2024-09-18T10:15:40.235 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001274425 2024-09-18T10:15:40.235 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.235 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.235 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:40.235 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.731930+0000", 2024-09-18T10:15:40.235 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0474000000000003e-05 2024-09-18T10:15:40.235 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:40.235 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:40.235 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:40.236 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.236 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.236 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpRead(3.2s3 41/40 ECSubRead(tid=26, to_read={3:46b2871c:::smithi18925896-34:head=0,4096,0}, subchunks={3:46b2871c:::smithi18925896-34:head=[0,1]}, attrs_to_read=))", 2024-09-18T10:15:40.236 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:57.543972+0000", 2024-09-18T10:15:40.236 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 42.671799516999997, 2024-09-18T10:15:40.236 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.18721849300000001, 2024-09-18T10:15:40.236 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:40.236 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:40.236 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:40.236 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.236 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:40.236 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.543972+0000", 2024-09-18T10:15:40.236 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:40.236 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.237 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.237 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:40.237 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.543972+0000", 2024-09-18T10:15:40.237 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:40.237 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.237 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.237 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:40.237 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.543970+0000", 2024-09-18T10:15:40.237 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:40.237 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.237 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.237 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:40.237 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.543996+0000", 2024-09-18T10:15:40.238 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5069e-05 2024-09-18T10:15:40.238 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.238 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.238 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:40.238 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.544056+0000", 2024-09-18T10:15:40.238 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0355000000000001e-05 2024-09-18T10:15:40.238 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.238 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.238 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:40.238 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.544079+0000", 2024-09-18T10:15:40.238 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3396000000000001e-05 2024-09-18T10:15:40.238 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.238 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.238 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:40.238 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.730946+0000", 2024-09-18T10:15:40.239 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.186866644 2024-09-18T10:15:40.239 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.239 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.239 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:40.239 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.731191+0000", 2024-09-18T10:15:40.239 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00024467899999999998 2024-09-18T10:15:40.239 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:40.239 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:40.239 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:40.239 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.239 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.239 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.ds1 47/32 ECSubWrite(tid=55, reqid=client.4632.0:332, at_version=47'8, trim_to=0'0, roll_forward_to=41'6))", 2024-09-18T10:15:40.239 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:02.796404+0000", 2024-09-18T10:15:40.239 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 37.419367921000003, 2024-09-18T10:15:40.239 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.103838836, 2024-09-18T10:15:40.240 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:40.240 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:40.240 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:40.240 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.240 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:40.240 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.796404+0000", 2024-09-18T10:15:40.240 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:40.240 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.240 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.240 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:40.240 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.796404+0000", 2024-09-18T10:15:40.240 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:40.240 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.241 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.241 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:40.241 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.796403+0000", 2024-09-18T10:15:40.241 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:15:40.241 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.241 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.241 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:40.241 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.796757+0000", 2024-09-18T10:15:40.241 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00035467799999999998 2024-09-18T10:15:40.241 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.241 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.241 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:40.242 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.796832+0000", 2024-09-18T10:15:40.242 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4661999999999998e-05 2024-09-18T10:15:40.242 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.242 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.242 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:40.242 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.896954+0000", 2024-09-18T10:15:40.242 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10012193799999999 2024-09-18T10:15:40.242 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.242 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.242 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:40.242 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.897127+0000", 2024-09-18T10:15:40.242 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017293500000000001 2024-09-18T10:15:40.242 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.243 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.243 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:40.243 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.897149+0000", 2024-09-18T10:15:40.243 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2262000000000001e-05 2024-09-18T10:15:40.243 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.243 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.243 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:40.243 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.900201+0000", 2024-09-18T10:15:40.243 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0030515379999999999 2024-09-18T10:15:40.243 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.243 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.243 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:40.243 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.900243+0000", 2024-09-18T10:15:40.244 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1758999999999998e-05 2024-09-18T10:15:40.244 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:40.244 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:40.244 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:40.244 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.244 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.244 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.as1 57/32 ECSubWrite(tid=68, reqid=osd.13.0:68, at_version=57'16, trim_to=0'0, roll_forward_to=56'14))", 2024-09-18T10:15:40.244 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:12.949505+0000", 2024-09-18T10:15:40.244 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 27.266266955999999, 2024-09-18T10:15:40.244 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.29432398300000001, 2024-09-18T10:15:40.244 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:40.244 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:40.244 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:40.244 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.245 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:40.245 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:12.949505+0000", 2024-09-18T10:15:40.245 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:40.245 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.245 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.245 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:40.245 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:12.949505+0000", 2024-09-18T10:15:40.245 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:40.245 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.245 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.245 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:40.245 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:12.949503+0000", 2024-09-18T10:15:40.245 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:40.246 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.246 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.246 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:40.246 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:12.949684+0000", 2024-09-18T10:15:40.246 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00018156800000000001 2024-09-18T10:15:40.246 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.246 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.246 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:40.246 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:12.950018+0000", 2024-09-18T10:15:40.246 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00033348699999999999 2024-09-18T10:15:40.246 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.246 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.246 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:40.246 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:12.950036+0000", 2024-09-18T10:15:40.247 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7669e-05 2024-09-18T10:15:40.247 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.247 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.247 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:40.247 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:13.143164+0000", 2024-09-18T10:15:40.247 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.19312819000000001 2024-09-18T10:15:40.247 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.247 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.247 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:40.247 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:13.143178+0000", 2024-09-18T10:15:40.247 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3974999999999999e-05 2024-09-18T10:15:40.247 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.247 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.247 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:40.248 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:13.243785+0000", 2024-09-18T10:15:40.248 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10060730499999999 2024-09-18T10:15:40.248 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.248 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.248 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:40.248 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:13.243829+0000", 2024-09-18T10:15:40.248 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3634999999999998e-05 2024-09-18T10:15:40.248 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:40.248 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:40.248 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:40.248 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.248 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.248 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.2s3 57/40 ECSubWrite(tid=43, reqid=osd.14.0:42, at_version=0'0, trim_to=0'0, roll_forward_to=57'26))", 2024-09-18T10:15:40.249 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:13.057362+0000", 2024-09-18T10:15:40.249 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 27.158410074999999, 2024-09-18T10:15:40.249 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.286917694, 2024-09-18T10:15:40.249 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:40.249 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:40.249 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:40.249 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.249 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:40.249 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:13.057362+0000", 2024-09-18T10:15:40.249 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:40.249 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.249 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.249 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:40.249 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:13.057362+0000", 2024-09-18T10:15:40.250 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:40.250 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.250 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.250 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:40.250 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:13.057359+0000", 2024-09-18T10:15:40.250 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999976 2024-09-18T10:15:40.250 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.250 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.250 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:40.250 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:13.057392+0000", 2024-09-18T10:15:40.250 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2836999999999997e-05 2024-09-18T10:15:40.250 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.250 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.250 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:40.251 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:13.057432+0000", 2024-09-18T10:15:40.251 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0195999999999999e-05 2024-09-18T10:15:40.251 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.251 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.251 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:40.251 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:13.057450+0000", 2024-09-18T10:15:40.251 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8156e-05 2024-09-18T10:15:40.251 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.251 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.251 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:40.251 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:13.243225+0000", 2024-09-18T10:15:40.251 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.18577438499999999 2024-09-18T10:15:40.251 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.252 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.252 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:40.252 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:13.243241+0000", 2024-09-18T10:15:40.252 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.643e-05 2024-09-18T10:15:40.252 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.252 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.252 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:40.252 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:13.344203+0000", 2024-09-18T10:15:40.252 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100961541 2024-09-18T10:15:40.252 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.252 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.252 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:40.252 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:13.344279+0000", 2024-09-18T10:15:40.252 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6487000000000001e-05 2024-09-18T10:15:40.253 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:40.253 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:40.253 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:40.253 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.253 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.253 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.2s3 58/40 ECSubWrite(tid=44, reqid=client.4632.0:452, at_version=58'28, trim_to=0'0, roll_forward_to=57'26))", 2024-09-18T10:15:40.253 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:13.563698+0000", 2024-09-18T10:15:40.253 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 26.652073533999999, 2024-09-18T10:15:40.253 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20210718, 2024-09-18T10:15:40.253 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:40.254 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:40.254 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:40.254 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.254 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:40.254 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:13.563698+0000", 2024-09-18T10:15:40.254 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:40.254 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.254 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.254 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:40.254 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:13.563698+0000", 2024-09-18T10:15:40.254 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:40.254 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.254 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.255 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:40.255 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:13.563697+0000", 2024-09-18T10:15:40.255 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:40.255 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.255 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.255 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:40.255 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:13.563729+0000", 2024-09-18T10:15:40.255 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1358e-05 2024-09-18T10:15:40.255 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.255 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.255 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:40.255 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:13.563740+0000", 2024-09-18T10:15:40.255 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.083e-05 2024-09-18T10:15:40.256 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.256 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.256 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:40.256 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:13.563747+0000", 2024-09-18T10:15:40.256 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6520000000000006e-06 2024-09-18T10:15:40.256 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.256 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.256 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:40.256 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:13.762333+0000", 2024-09-18T10:15:40.256 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.19858611700000001 2024-09-18T10:15:40.256 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.256 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.256 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:40.257 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:13.762350+0000", 2024-09-18T10:15:40.257 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6796e-05 2024-09-18T10:15:40.257 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.257 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.257 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:40.257 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:13.765695+0000", 2024-09-18T10:15:40.257 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0033452540000000002 2024-09-18T10:15:40.257 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.257 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.257 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:40.257 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:13.765805+0000", 2024-09-18T10:15:40.257 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000109843 2024-09-18T10:15:40.257 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:40.258 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:40.258 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:40.258 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.258 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.258 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDPGRecoveryDeleteReply(3.13s0 e58,56 3:c9ce6440:::smithi18925896-24:2,41'17)", 2024-09-18T10:15:40.258 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:13.987390+0000", 2024-09-18T10:15:40.258 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 26.228381252999998, 2024-09-18T10:15:40.258 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10444218, 2024-09-18T10:15:40.258 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:40.258 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:40.258 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:40.258 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.258 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:40.258 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:13.987390+0000", 2024-09-18T10:15:40.259 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:40.259 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.259 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.259 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:40.259 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:13.987390+0000", 2024-09-18T10:15:40.259 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:40.259 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.259 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.259 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:40.259 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:13.987388+0000", 2024-09-18T10:15:40.259 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:40.259 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.259 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.260 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:40.260 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:13.987402+0000", 2024-09-18T10:15:40.260 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4083000000000001e-05 2024-09-18T10:15:40.260 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.260 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.260 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:40.260 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:13.987449+0000", 2024-09-18T10:15:40.260 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6514999999999998e-05 2024-09-18T10:15:40.260 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.260 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.260 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:40.260 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:13.987463+0000", 2024-09-18T10:15:40.260 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4443e-05 2024-09-18T10:15:40.261 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.261 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.261 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:40.261 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:14.089024+0000", 2024-09-18T10:15:40.261 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.101560597 2024-09-18T10:15:40.261 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.261 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.261 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:40.261 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:14.091833+0000", 2024-09-18T10:15:40.261 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0028085720000000001 2024-09-18T10:15:40.261 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:40.261 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:40.261 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:40.262 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.262 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.262 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDPGRecoveryDeleteReply(3.13s0 e58,56 3:c9ce6440:::smithi18925896-24:2,41'17)", 2024-09-18T10:15:40.262 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:14.163960+0000", 2024-09-18T10:15:40.262 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 26.051811552, 2024-09-18T10:15:40.262 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.105232166, 2024-09-18T10:15:40.262 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:40.262 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:40.262 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:40.262 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.262 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:40.262 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:14.163960+0000", 2024-09-18T10:15:40.262 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:40.262 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.263 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.263 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:40.263 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:14.163960+0000", 2024-09-18T10:15:40.263 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:40.263 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.263 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.263 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:40.263 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:14.163958+0000", 2024-09-18T10:15:40.263 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:40.263 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.263 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.263 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:40.263 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:14.163975+0000", 2024-09-18T10:15:40.263 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6552999999999999e-05 2024-09-18T10:15:40.264 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.264 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.264 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:40.264 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:14.164009+0000", 2024-09-18T10:15:40.264 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4476e-05 2024-09-18T10:15:40.264 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.264 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.264 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:40.264 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:14.264163+0000", 2024-09-18T10:15:40.264 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100153834 2024-09-18T10:15:40.264 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.264 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.264 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:40.265 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:14.264384+0000", 2024-09-18T10:15:40.265 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000221301 2024-09-18T10:15:40.265 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.265 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.265 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:40.265 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:14.269192+0000", 2024-09-18T10:15:40.265 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0048078749999999996 2024-09-18T10:15:40.265 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:40.265 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:40.265 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:40.265 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.265 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.265 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDScrubReserve(3.18s5 REQUEST e61)", 2024-09-18T10:15:40.265 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:17.424550+0000", 2024-09-18T10:15:40.266 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 22.791221997000001, 2024-09-18T10:15:40.266 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.200564715, 2024-09-18T10:15:40.266 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:40.266 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-18T10:15:40.266 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:40.266 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.266 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:40.266 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.424550+0000", 2024-09-18T10:15:40.266 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:40.266 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.266 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.266 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:40.266 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.424550+0000", 2024-09-18T10:15:40.267 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:40.267 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.267 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.267 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:40.267 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.424548+0000", 2024-09-18T10:15:40.267 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:40.267 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.267 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.267 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:40.267 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.424560+0000", 2024-09-18T10:15:40.267 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2859e-05 2024-09-18T10:15:40.267 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.267 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.267 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:40.268 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.424595+0000", 2024-09-18T10:15:40.268 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4576000000000002e-05 2024-09-18T10:15:40.268 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.268 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.268 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:40.268 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.524738+0000", 2024-09-18T10:15:40.268 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100143278 2024-09-18T10:15:40.268 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.268 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.268 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:40.268 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.625032+0000", 2024-09-18T10:15:40.268 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10029397399999999 2024-09-18T10:15:40.268 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.268 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.269 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:40.269 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.625064+0000", 2024-09-18T10:15:40.269 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1848999999999999e-05 2024-09-18T10:15:40.269 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.269 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.269 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:40.269 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.625114+0000", 2024-09-18T10:15:40.269 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0305e-05 2024-09-18T10:15:40.269 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:40.269 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:40.269 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:40.269 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.269 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.270 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:515 3.13s0 3:c9ce6440:::smithi18925896-24:head [delete] snapc 1a=[1a,19,18,16,12,11,10,f,e] ondisk+write+known_if_redirected+full_force+supports_pool_eio e63)", 2024-09-18T10:15:40.270 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:19.217452+0000", 2024-09-18T10:15:40.270 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 20.998320066000002, 2024-09-18T10:15:40.270 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20492691099999999, 2024-09-18T10:15:40.270 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:40.270 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:40.270 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:40.270 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:40.270 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:40.270 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 515 2024-09-18T10:15:40.270 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.270 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:40.270 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.271 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:40.271 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.217452+0000", 2024-09-18T10:15:40.271 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:40.271 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.271 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.271 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:40.271 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.217452+0000", 2024-09-18T10:15:40.271 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:40.271 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.271 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.271 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:40.271 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.217448+0000", 2024-09-18T10:15:40.271 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999967 2024-09-18T10:15:40.271 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.272 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.272 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:40.272 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.217464+0000", 2024-09-18T10:15:40.272 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5763000000000001e-05 2024-09-18T10:15:40.272 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.272 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.272 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:40.272 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.217500+0000", 2024-09-18T10:15:40.272 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5536e-05 2024-09-18T10:15:40.272 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.272 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.272 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:40.272 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.217528+0000", 2024-09-18T10:15:40.272 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8609e-05 2024-09-18T10:15:40.273 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.273 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.273 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:40.273 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.217647+0000", 2024-09-18T10:15:40.273 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000118917 2024-09-18T10:15:40.273 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.273 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.273 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:40.273 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.217866+0000", 2024-09-18T10:15:40.273 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00021885399999999999 2024-09-18T10:15:40.273 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.273 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.273 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:40.274 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.219038+0000", 2024-09-18T10:15:40.274 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001172125 2024-09-18T10:15:40.274 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.274 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.274 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:40.274 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.220845+0000", 2024-09-18T10:15:40.274 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001806385 2024-09-18T10:15:40.274 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.274 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.274 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:15:40.274 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.422256+0000", 2024-09-18T10:15:40.274 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20141129799999999 2024-09-18T10:15:40.274 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.274 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.275 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:40.275 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.422379+0000", 2024-09-18T10:15:40.275 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012262699999999999 2024-09-18T10:15:40.275 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:40.275 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:40.275 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:40.275 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.275 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.275 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.bs5 72/32 ECSubWrite(tid=153, reqid=client.4632.0:624, at_version=72'23, trim_to=0'0, roll_forward_to=60'21))", 2024-09-18T10:15:40.275 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:28.105186+0000", 2024-09-18T10:15:40.275 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 12.110585553, 2024-09-18T10:15:40.275 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.38192188100000002, 2024-09-18T10:15:40.275 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:40.275 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:40.276 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:40.276 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.276 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:40.276 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:28.105186+0000", 2024-09-18T10:15:40.276 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:40.276 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.276 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.276 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:40.276 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:28.105186+0000", 2024-09-18T10:15:40.276 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:40.276 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.276 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.276 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:40.277 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:28.105186+0000", 2024-09-18T10:15:40.277 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:40.277 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.277 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.277 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:40.277 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:28.105214+0000", 2024-09-18T10:15:40.277 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7985000000000001e-05 2024-09-18T10:15:40.277 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.277 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.277 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:40.277 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:28.105228+0000", 2024-09-18T10:15:40.277 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3930999999999999e-05 2024-09-18T10:15:40.277 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.278 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.278 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:40.278 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:28.205370+0000", 2024-09-18T10:15:40.278 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10014197800000001 2024-09-18T10:15:40.278 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.278 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.278 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:40.278 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:28.305658+0000", 2024-09-18T10:15:40.278 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.1002884 2024-09-18T10:15:40.278 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.278 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.278 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:40.278 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:28.305679+0000", 2024-09-18T10:15:40.278 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0812e-05 2024-09-18T10:15:40.279 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.279 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.279 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:40.279 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:28.487015+0000", 2024-09-18T10:15:40.279 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.18133600499999999 2024-09-18T10:15:40.279 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.279 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.279 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:40.279 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:28.487108+0000", 2024-09-18T10:15:40.279 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3011999999999996e-05 2024-09-18T10:15:40.279 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:40.279 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:40.280 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:40.280 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.280 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.280 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.2s3 72/40 ECSubWrite(tid=93, reqid=osd.14.0:92, at_version=0'0, trim_to=0'0, roll_forward_to=72'55))", 2024-09-18T10:15:40.280 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:28.389120+0000", 2024-09-18T10:15:40.280 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 11.826651318, 2024-09-18T10:15:40.280 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20149926400000001, 2024-09-18T10:15:40.280 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:40.280 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:40.280 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:40.280 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.280 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:40.280 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:28.389120+0000", 2024-09-18T10:15:40.280 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:40.281 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.281 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.281 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:40.281 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:28.389120+0000", 2024-09-18T10:15:40.281 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:40.281 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.281 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.281 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:40.281 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:28.389112+0000", 2024-09-18T10:15:40.281 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999919 2024-09-18T10:15:40.281 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.281 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.281 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:40.282 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:28.389153+0000", 2024-09-18T10:15:40.282 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0138000000000001e-05 2024-09-18T10:15:40.282 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.282 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.282 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:40.282 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:28.389182+0000", 2024-09-18T10:15:40.282 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9113000000000001e-05 2024-09-18T10:15:40.282 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.282 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.282 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:40.282 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:28.489269+0000", 2024-09-18T10:15:40.282 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100087491 2024-09-18T10:15:40.282 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.282 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.283 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:40.283 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:28.589445+0000", 2024-09-18T10:15:40.283 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100175537 2024-09-18T10:15:40.283 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.283 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.283 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:40.283 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:28.589468+0000", 2024-09-18T10:15:40.283 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2781000000000001e-05 2024-09-18T10:15:40.283 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.283 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.283 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:40.283 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:28.590550+0000", 2024-09-18T10:15:40.283 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001082251 2024-09-18T10:15:40.283 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.284 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.284 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:40.284 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:28.590620+0000", 2024-09-18T10:15:40.284 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9822999999999999e-05 2024-09-18T10:15:40.284 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:40.284 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:40.284 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:40.284 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.284 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.284 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.bs5 74/32 ECSubWrite(tid=157, reqid=osd.4.0:157, at_version=74'25, trim_to=0'0, roll_forward_to=72'23))", 2024-09-18T10:15:40.284 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:29.664321+0000", 2024-09-18T10:15:40.284 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.551450948999999, 2024-09-18T10:15:40.284 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.104388913, 2024-09-18T10:15:40.285 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:40.285 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:40.285 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:40.285 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.285 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:40.285 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.664321+0000", 2024-09-18T10:15:40.285 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:40.285 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.285 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.285 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:40.285 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.664321+0000", 2024-09-18T10:15:40.285 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:40.285 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.285 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.286 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:40.286 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.664319+0000", 2024-09-18T10:15:40.286 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:40.286 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.286 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.286 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:40.286 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.664380+0000", 2024-09-18T10:15:40.286 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0717999999999997e-05 2024-09-18T10:15:40.286 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.286 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.286 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:40.286 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.664409+0000", 2024-09-18T10:15:40.286 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9047000000000002e-05 2024-09-18T10:15:40.287 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.287 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.287 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:40.287 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.664420+0000", 2024-09-18T10:15:40.287 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.148e-05 2024-09-18T10:15:40.287 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.287 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.287 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:40.287 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.664513+0000", 2024-09-18T10:15:40.287 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2947000000000003e-05 2024-09-18T10:15:40.287 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.287 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.287 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:40.287 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.664537+0000", 2024-09-18T10:15:40.288 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4042000000000001e-05 2024-09-18T10:15:40.288 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.288 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.288 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:40.288 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.768653+0000", 2024-09-18T10:15:40.288 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.104115914 2024-09-18T10:15:40.288 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.288 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.288 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:40.288 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.768710+0000", 2024-09-18T10:15:40.288 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6585000000000001e-05 2024-09-18T10:15:40.288 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:40.288 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:40.288 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:40.288 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.289 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.289 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.8s5 74/40 ECSubWrite(tid=202, reqid=osd.6.0:202, at_version=74'27, trim_to=0'0, roll_forward_to=72'23))", 2024-09-18T10:15:40.289 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:29.719110+0000", 2024-09-18T10:15:40.289 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.496661762, 2024-09-18T10:15:40.289 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.19912816699999999, 2024-09-18T10:15:40.289 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:40.289 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:40.289 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:40.289 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.289 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:40.289 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.719110+0000", 2024-09-18T10:15:40.289 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:40.289 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.290 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.290 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:40.290 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.719110+0000", 2024-09-18T10:15:40.290 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:40.290 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.290 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.290 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:40.290 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.719109+0000", 2024-09-18T10:15:40.290 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:15:40.290 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.290 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.290 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:40.290 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.719133+0000", 2024-09-18T10:15:40.290 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4409999999999998e-05 2024-09-18T10:15:40.291 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.291 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.291 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:40.291 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.719151+0000", 2024-09-18T10:15:40.291 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7441000000000001e-05 2024-09-18T10:15:40.291 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.291 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.291 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:40.291 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.719159+0000", 2024-09-18T10:15:40.291 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1370000000000002e-06 2024-09-18T10:15:40.291 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.291 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.291 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:40.291 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.916605+0000", 2024-09-18T10:15:40.291 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.19744605700000001 2024-09-18T10:15:40.291 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.292 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.292 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:40.292 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.916621+0000", 2024-09-18T10:15:40.292 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6059e-05 2024-09-18T10:15:40.292 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.292 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.292 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:40.292 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.918184+0000", 2024-09-18T10:15:40.292 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001562765 2024-09-18T10:15:40.292 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.292 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.292 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:40.292 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.918238+0000", 2024-09-18T10:15:40.292 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4348999999999998e-05 2024-09-18T10:15:40.293 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:40.293 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:40.293 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:40.293 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.293 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.293 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWriteReply(3.13s0 74/56 ECSubWriteReply(tid=23, last_complete=74'46, committed=1, applied=1))", 2024-09-18T10:15:40.293 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:29.774402+0000", 2024-09-18T10:15:40.293 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.441369878, 2024-09-18T10:15:40.293 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.19560270900000001, 2024-09-18T10:15:40.293 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:40.293 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:40.293 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:40.293 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.293 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:40.293 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.774402+0000", 2024-09-18T10:15:40.294 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:40.294 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.294 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.294 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:40.294 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.774402+0000", 2024-09-18T10:15:40.294 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:40.294 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.294 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.294 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:40.294 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.774401+0000", 2024-09-18T10:15:40.294 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:15:40.294 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.294 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.294 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:40.295 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.774407+0000", 2024-09-18T10:15:40.295 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.173e-06 2024-09-18T10:15:40.295 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.295 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.295 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:40.295 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.774423+0000", 2024-09-18T10:15:40.295 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6568000000000001e-05 2024-09-18T10:15:40.295 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.295 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.295 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:40.295 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.774431+0000", 2024-09-18T10:15:40.295 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4649999999999997e-06 2024-09-18T10:15:40.295 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.295 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.295 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:40.296 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.969971+0000", 2024-09-18T10:15:40.296 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.19554042199999999 2024-09-18T10:15:40.296 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.296 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.296 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:40.296 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.970004+0000", 2024-09-18T10:15:40.296 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3151999999999997e-05 2024-09-18T10:15:40.296 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:40.296 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:40.296 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:40.296 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.296 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.296 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.bs5 74/32 ECSubWrite(tid=163, reqid=client.4632.0:650, at_version=74'27, trim_to=0'0, roll_forward_to=72'23))", 2024-09-18T10:15:40.296 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:30.069481+0000", 2024-09-18T10:15:40.297 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.146290419, 2024-09-18T10:15:40.297 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30291274699999998, 2024-09-18T10:15:40.297 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:40.297 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:40.297 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:40.297 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.297 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:40.297 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:30.069481+0000", 2024-09-18T10:15:40.297 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:40.297 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.297 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.297 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:40.297 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:30.069481+0000", 2024-09-18T10:15:40.298 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:40.298 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.298 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.298 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:40.298 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:30.069479+0000", 2024-09-18T10:15:40.298 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:40.298 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.298 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.298 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:40.298 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:30.069535+0000", 2024-09-18T10:15:40.298 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5674999999999998e-05 2024-09-18T10:15:40.298 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.298 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.298 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:40.298 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:30.069577+0000", 2024-09-18T10:15:40.298 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1588000000000002e-05 2024-09-18T10:15:40.299 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.299 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.299 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:40.299 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:30.069595+0000", 2024-09-18T10:15:40.299 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8128999999999999e-05 2024-09-18T10:15:40.299 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.299 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.299 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:40.299 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:30.269862+0000", 2024-09-18T10:15:40.299 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20026767000000001 2024-09-18T10:15:40.299 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.299 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.299 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:40.299 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:30.269881+0000", 2024-09-18T10:15:40.299 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8402000000000001e-05 2024-09-18T10:15:40.300 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.300 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.300 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:40.300 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:30.372285+0000", 2024-09-18T10:15:40.300 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10240445099999999 2024-09-18T10:15:40.300 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.300 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.300 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:40.300 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:30.372394+0000", 2024-09-18T10:15:40.300 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010880200000000001 2024-09-18T10:15:40.300 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:40.300 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:40.300 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:40.300 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.301 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.301 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDPGPush(3.19s2 79/74 [PushOp(3:9a1a774c:::smithi18925896-38:19, version: 71'93, data_included: [0~262144], data_size: 262144, omap_header_size: 0, omap_entries_size: 0, attrset_size: 3, recovery_info: ObjectRecoveryInfo(3:9a1a774c:::smithi18925896-38:19@71'93, size: 1638400, copy_subset: [], clone_subset: {}, snapset: 19=[]:{19=[16,12,11,10]}, object_exist: 1), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:1048576, data_complete:false, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:true, error:false))])", 2024-09-18T10:15:40.301 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:35.501041+0000", 2024-09-18T10:15:40.301 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.7147306819999999, 2024-09-18T10:15:40.301 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30191898, 2024-09-18T10:15:40.301 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:40.301 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:40.301 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:40.301 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.301 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:40.301 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:35.501041+0000", 2024-09-18T10:15:40.301 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:40.301 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.301 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.302 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:40.302 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:35.501041+0000", 2024-09-18T10:15:40.302 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:40.302 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.302 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.302 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:40.302 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:35.501039+0000", 2024-09-18T10:15:40.302 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:40.302 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.302 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.302 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:40.302 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:35.501446+0000", 2024-09-18T10:15:40.302 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00040681000000000002 2024-09-18T10:15:40.302 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.302 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.303 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:40.303 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:35.501490+0000", 2024-09-18T10:15:40.303 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4162000000000002e-05 2024-09-18T10:15:40.303 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.303 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.303 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:40.303 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:35.601651+0000", 2024-09-18T10:15:40.303 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100160545 2024-09-18T10:15:40.303 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.303 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.303 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:40.303 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:35.801995+0000", 2024-09-18T10:15:40.304 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20034463699999999 2024-09-18T10:15:40.304 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.304 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.304 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:40.304 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:35.802960+0000", 2024-09-18T10:15:40.304 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00096465600000000004 2024-09-18T10:15:40.304 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:40.305 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:40.305 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:40.305 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:40.305 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:40.305 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-18T10:15:40.306 INFO:tasks.rados.rados.0.smithi189.stdout:294: finishing write tid 6 to smithi18925896-20 2024-09-18T10:15:40.306 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 20 v 31 (ObjNum 118 snap 25 seq_num 118) dirty exists 2024-09-18T10:15:40.306 INFO:tasks.rados.rados.0.smithi189.stdout:294: left oid 20 (ObjNum 118 snap 25 seq_num 118) 2024-09-18T10:15:40.307 INFO:tasks.rados.rados.0.smithi189.stdout:304: finishing write tid 1 to smithi18925896-16 2024-09-18T10:15:40.307 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T10:15:40.308 INFO:tasks.ceph.osd.8.smithi029.stderr:2024-09-18T10:15:40.301+0000 7f2555c27700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 22192) UID: 0 2024-09-18T10:15:40.407 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-18T10:15:40.404+0000 7f00046a8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 22136) UID: 0 2024-09-18T10:15:40.409 INFO:tasks.rados.rados.0.smithi189.stdout:305: finishing write tid 1 to smithi18925896-23 2024-09-18T10:15:40.410 INFO:tasks.rados.rados.0.smithi189.stdout:305: finishing write tid 2 to smithi18925896-23 2024-09-18T10:15:40.412 INFO:tasks.rados.rados.0.smithi189.stdout:305: finishing write tid 3 to smithi18925896-23 2024-09-18T10:15:40.421 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-18T10:15:40.421 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-18T10:15:40.421 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.421 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWriteReply(3.cs0 84/83 ECSubWriteReply(tid=140, last_complete=84'116, committed=1, applied=1))", 2024-09-18T10:15:40.421 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:40.207380+0000", 2024-09-18T10:15:40.421 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.21214208700000001, 2024-09-18T10:15:40.421 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.21217912799999999, 2024-09-18T10:15:40.421 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:40.421 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-18T10:15:40.421 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:40.422 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.422 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:40.422 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.207380+0000", 2024-09-18T10:15:40.422 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:40.422 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.422 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.422 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:40.422 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.207380+0000", 2024-09-18T10:15:40.422 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:40.422 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.422 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.422 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:40.422 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.207372+0000", 2024-09-18T10:15:40.422 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999928 2024-09-18T10:15:40.423 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.423 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.423 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:40.423 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.207389+0000", 2024-09-18T10:15:40.423 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6886000000000001e-05 2024-09-18T10:15:40.423 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.423 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.423 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:40.423 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.207414+0000", 2024-09-18T10:15:40.423 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4666000000000001e-05 2024-09-18T10:15:40.423 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.423 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.423 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:40.424 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.207424+0000", 2024-09-18T10:15:40.424 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0192e-05 2024-09-18T10:15:40.424 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:40.424 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:40.424 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:40.424 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.424 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.424 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:753 3.cs0 3:3a4fded2:::smithi18925896-23:head [append 0~540672 in=540672b] snapc 2d=[29,28,20,1f,1e,1a,11] ondisk+write+known_if_redirected+supports_pool_eio e84)", 2024-09-18T10:15:40.424 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:39.707399+0000", 2024-09-18T10:15:40.424 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.71212289699999998, 2024-09-18T10:15:40.424 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.71225483199999995, 2024-09-18T10:15:40.424 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:40.424 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-18T10:15:40.425 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:40.425 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:40.425 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:40.425 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 753 2024-09-18T10:15:40.425 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.425 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:40.425 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.425 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:40.425 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.707399+0000", 2024-09-18T10:15:40.425 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:40.425 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.425 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.425 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:40.426 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.707399+0000", 2024-09-18T10:15:40.426 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:40.426 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.426 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.426 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:40.426 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.707391+0000", 2024-09-18T10:15:40.426 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999928 2024-09-18T10:15:40.426 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.426 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.426 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:40.426 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.708412+0000", 2024-09-18T10:15:40.426 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010203759999999999 2024-09-18T10:15:40.426 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.426 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.427 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:40.427 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.708445+0000", 2024-09-18T10:15:40.427 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3674999999999999e-05 2024-09-18T10:15:40.427 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.427 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.427 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:40.427 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.708458+0000", 2024-09-18T10:15:40.427 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2943e-05 2024-09-18T10:15:40.427 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.427 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.427 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:40.427 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.994678+0000", 2024-09-18T10:15:40.427 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.28621953700000002 2024-09-18T10:15:40.428 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.428 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.428 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:40.428 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.995068+0000", 2024-09-18T10:15:40.428 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000390338 2024-09-18T10:15:40.428 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.428 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.428 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:40.428 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.997188+0000", 2024-09-18T10:15:40.428 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0021194159999999998 2024-09-18T10:15:40.428 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.428 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.428 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:40.429 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.206267+0000", 2024-09-18T10:15:40.429 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20907888599999999 2024-09-18T10:15:40.429 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:40.429 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:40.429 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:40.429 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.429 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.429 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWriteReply(3.cs0 84/83 ECSubWriteReply(tid=140, last_complete=84'116, committed=1, applied=1))", 2024-09-18T10:15:40.429 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:40.207481+0000", 2024-09-18T10:15:40.429 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.212040914, 2024-09-18T10:15:40.429 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.21229305700000001, 2024-09-18T10:15:40.429 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:40.429 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-18T10:15:40.430 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:40.430 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.430 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:40.430 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.207481+0000", 2024-09-18T10:15:40.430 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:40.430 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.430 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.430 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:40.430 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.207481+0000", 2024-09-18T10:15:40.430 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:40.431 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.431 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.431 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:40.431 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.207480+0000", 2024-09-18T10:15:40.431 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:15:40.431 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.431 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.431 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:40.431 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.207485+0000", 2024-09-18T10:15:40.431 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6470000000000003e-06 2024-09-18T10:15:40.431 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.431 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.431 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:40.432 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.207502+0000", 2024-09-18T10:15:40.432 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6209e-05 2024-09-18T10:15:40.432 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.432 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.432 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:40.432 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.207510+0000", 2024-09-18T10:15:40.432 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9670000000000001e-06 2024-09-18T10:15:40.432 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:40.433 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:40.433 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:40.433 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.433 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.433 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:754 3.cs0 3:3a4fded2:::smithi18925896-23:head [append 0~147456 in=147456b] snapc 2d=[29,28,20,1f,1e,1a,11] ondisk+write+known_if_redirected+supports_pool_eio e84)", 2024-09-18T10:15:40.433 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:39.708472+0000", 2024-09-18T10:15:40.433 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.71104993900000002, 2024-09-18T10:15:40.433 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.711381759, 2024-09-18T10:15:40.433 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:40.433 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-18T10:15:40.433 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:40.433 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:40.434 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:40.434 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 754 2024-09-18T10:15:40.434 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.434 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:40.434 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.434 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:40.434 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.708472+0000", 2024-09-18T10:15:40.434 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:40.434 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.434 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.434 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:40.434 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.708472+0000", 2024-09-18T10:15:40.434 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:40.435 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.435 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.435 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:40.435 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.708470+0000", 2024-09-18T10:15:40.435 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:40.435 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.435 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.435 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:40.435 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.708676+0000", 2024-09-18T10:15:40.435 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00020609599999999999 2024-09-18T10:15:40.435 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.435 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.435 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:40.435 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.708704+0000", 2024-09-18T10:15:40.436 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7138999999999999e-05 2024-09-18T10:15:40.436 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.436 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.436 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:40.436 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.708713+0000", 2024-09-18T10:15:40.436 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1500000000000005e-06 2024-09-18T10:15:40.436 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.436 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.436 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:40.436 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.099648+0000", 2024-09-18T10:15:40.436 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.39093543800000002 2024-09-18T10:15:40.436 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.436 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.437 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:40.437 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.100061+0000", 2024-09-18T10:15:40.437 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000412958 2024-09-18T10:15:40.437 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.437 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.437 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:40.437 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.101562+0000", 2024-09-18T10:15:40.437 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001500758 2024-09-18T10:15:40.437 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.437 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.437 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:40.437 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.206942+0000", 2024-09-18T10:15:40.437 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10538027 2024-09-18T10:15:40.437 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:40.438 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:40.438 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:40.438 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.438 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.438 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:755 3.cs0 3:3a4fded2:::smithi18925896-23:head [setxattr _header (57) in=64b] snapc 2d=[29,28,20,1f,1e,1a,11] ondisk+write+known_if_redirected+supports_pool_eio e84)", 2024-09-18T10:15:40.438 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:39.708729+0000", 2024-09-18T10:15:40.438 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.71079272599999999, 2024-09-18T10:15:40.438 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.71124788500000002, 2024-09-18T10:15:40.438 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:40.438 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-18T10:15:40.438 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:40.438 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:40.438 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:40.438 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 755 2024-09-18T10:15:40.439 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.439 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:40.439 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.439 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:40.439 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.708729+0000", 2024-09-18T10:15:40.439 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:40.439 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.439 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.439 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:40.439 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.708729+0000", 2024-09-18T10:15:40.439 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:40.439 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.439 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.439 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:40.439 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.708728+0000", 2024-09-18T10:15:40.440 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:15:40.440 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.440 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.440 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:40.440 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.708738+0000", 2024-09-18T10:15:40.440 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.01e-05 2024-09-18T10:15:40.440 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.440 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.440 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:40.440 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.708752+0000", 2024-09-18T10:15:40.440 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3713000000000001e-05 2024-09-18T10:15:40.440 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.440 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.440 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:40.440 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.708762+0000", 2024-09-18T10:15:40.441 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0631999999999999e-05 2024-09-18T10:15:40.441 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.441 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.441 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:40.441 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.203162+0000", 2024-09-18T10:15:40.441 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.49439976200000002 2024-09-18T10:15:40.441 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.441 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.441 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:40.441 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.203686+0000", 2024-09-18T10:15:40.441 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00052412100000000001 2024-09-18T10:15:40.441 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.441 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.441 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:40.441 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.205437+0000", 2024-09-18T10:15:40.442 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001750759 2024-09-18T10:15:40.442 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.442 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.442 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:40.442 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.307503+0000", 2024-09-18T10:15:40.442 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10206599199999999 2024-09-18T10:15:40.442 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:40.442 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:40.442 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:40.442 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.442 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.442 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:756 3.cs0 3:3a4fded2:::smithi18925896-23:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e84)", 2024-09-18T10:15:40.443 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:39.708769+0000", 2024-09-18T10:15:40.443 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.71075217400000001, 2024-09-18T10:15:40.443 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.71132429900000005, 2024-09-18T10:15:40.443 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:40.443 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "waiting for rw locks", 2024-09-18T10:15:40.443 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:40.443 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:40.443 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:40.443 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 756 2024-09-18T10:15:40.443 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.443 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:40.443 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.443 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:40.444 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.708769+0000", 2024-09-18T10:15:40.444 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:40.444 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.444 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.444 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:40.444 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.708769+0000", 2024-09-18T10:15:40.444 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:40.444 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.444 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.444 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:40.444 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.708769+0000", 2024-09-18T10:15:40.444 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:15:40.444 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.444 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.445 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:40.445 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.708773+0000", 2024-09-18T10:15:40.445 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9929999999999997e-06 2024-09-18T10:15:40.445 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.445 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.445 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:40.445 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.708786+0000", 2024-09-18T10:15:40.445 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3606e-05 2024-09-18T10:15:40.445 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.445 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.445 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:40.445 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.708792+0000", 2024-09-18T10:15:40.446 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7559999999999996e-06 2024-09-18T10:15:40.446 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.446 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.446 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:40.446 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.206394+0000", 2024-09-18T10:15:40.446 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.49760259400000001 2024-09-18T10:15:40.446 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.446 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.446 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2024-09-18T10:15:40.446 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.206929+0000", 2024-09-18T10:15:40.446 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00053430700000000003 2024-09-18T10:15:40.446 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:40.446 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:40.446 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:40.446 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.447 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.447 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWriteReply(3.cs0 84/83 ECSubWriteReply(tid=139, last_complete=84'115, committed=1, applied=1))", 2024-09-18T10:15:40.447 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:40.295443+0000", 2024-09-18T10:15:40.447 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.124078808, 2024-09-18T10:15:40.447 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.12474629600000001, 2024-09-18T10:15:40.447 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:40.447 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-18T10:15:40.447 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:40.447 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.447 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:40.447 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.295443+0000", 2024-09-18T10:15:40.447 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:40.447 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.447 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.448 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:40.448 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.295443+0000", 2024-09-18T10:15:40.448 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:40.448 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.448 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.448 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:40.448 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.295442+0000", 2024-09-18T10:15:40.448 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:40.448 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.448 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.448 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:40.448 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.295448+0000", 2024-09-18T10:15:40.448 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7060000000000001e-06 2024-09-18T10:15:40.448 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.448 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.449 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:40.449 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.295458+0000", 2024-09-18T10:15:40.449 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0263e-05 2024-09-18T10:15:40.449 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.449 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.449 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:40.449 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.295468+0000", 2024-09-18T10:15:40.449 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3570000000000004e-06 2024-09-18T10:15:40.449 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:40.449 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:40.449 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:40.449 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.449 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.450 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWriteReply(3.cs0 84/83 ECSubWriteReply(tid=140, last_complete=84'116, committed=1, applied=1))", 2024-09-18T10:15:40.450 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:40.296368+0000", 2024-09-18T10:15:40.450 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.12315330200000001, 2024-09-18T10:15:40.450 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.123889817, 2024-09-18T10:15:40.450 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:40.450 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-18T10:15:40.450 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:40.450 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.450 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:40.450 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.296368+0000", 2024-09-18T10:15:40.450 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:40.450 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.450 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.451 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:40.451 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.296368+0000", 2024-09-18T10:15:40.451 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:40.451 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.451 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.451 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:40.451 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.296359+0000", 2024-09-18T10:15:40.451 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999905 2024-09-18T10:15:40.451 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.451 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.451 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:40.451 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.296377+0000", 2024-09-18T10:15:40.451 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8658999999999999e-05 2024-09-18T10:15:40.452 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.452 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.452 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:40.452 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.296408+0000", 2024-09-18T10:15:40.453 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0688999999999997e-05 2024-09-18T10:15:40.453 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.453 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.453 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:40.453 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.296420+0000", 2024-09-18T10:15:40.453 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1477e-05 2024-09-18T10:15:40.453 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:40.453 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:40.453 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:40.453 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.453 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.453 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWriteReply(3.cs0 84/83 ECSubWriteReply(tid=138, last_complete=84'114, committed=1, applied=1))", 2024-09-18T10:15:40.453 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:40.306113+0000", 2024-09-18T10:15:40.454 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.113408409, 2024-09-18T10:15:40.454 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.114215545, 2024-09-18T10:15:40.454 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:40.454 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-18T10:15:40.454 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:40.454 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.454 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:40.454 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.306113+0000", 2024-09-18T10:15:40.454 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:40.455 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.455 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.455 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:40.455 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.306113+0000", 2024-09-18T10:15:40.455 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:40.455 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.455 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.455 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:40.455 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.306112+0000", 2024-09-18T10:15:40.455 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:15:40.455 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.455 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.456 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:40.456 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.306119+0000", 2024-09-18T10:15:40.456 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.99e-06 2024-09-18T10:15:40.456 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.456 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.456 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:40.456 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.306140+0000", 2024-09-18T10:15:40.456 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1032000000000001e-05 2024-09-18T10:15:40.456 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.456 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.456 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:40.456 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.306151+0000", 2024-09-18T10:15:40.456 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0593e-05 2024-09-18T10:15:40.457 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:40.457 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:40.457 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:40.457 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.457 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.457 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWriteReply(3.cs0 84/83 ECSubWriteReply(tid=139, last_complete=84'115, committed=1, applied=1))", 2024-09-18T10:15:40.457 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:40.306164+0000", 2024-09-18T10:15:40.457 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.11335801, 2024-09-18T10:15:40.457 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.11423298699999999, 2024-09-18T10:15:40.457 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:40.457 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-18T10:15:40.457 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:40.457 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.457 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:40.457 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.306164+0000", 2024-09-18T10:15:40.458 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:40.458 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.458 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.458 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:40.458 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.306164+0000", 2024-09-18T10:15:40.458 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:40.458 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.458 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.458 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:40.458 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.306163+0000", 2024-09-18T10:15:40.459 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:15:40.459 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.459 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.459 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:40.459 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.306169+0000", 2024-09-18T10:15:40.459 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4529999999999999e-06 2024-09-18T10:15:40.459 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.459 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.459 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:40.459 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.306184+0000", 2024-09-18T10:15:40.459 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5213e-05 2024-09-18T10:15:40.459 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.459 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.460 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:40.460 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.306193+0000", 2024-09-18T10:15:40.460 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9900000000000003e-06 2024-09-18T10:15:40.460 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:40.460 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:40.460 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:40.461 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:40.461 INFO:teuthology.orchestra.run.smithi136.stdout: ], 2024-09-18T10:15:40.461 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 10 2024-09-18T10:15:40.461 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-18T10:15:40.463 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T10:15:40.508 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-18T10:15:40.505+0000 7f302f2c3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22187) UID: 0 2024-09-18T10:15:40.511 INFO:tasks.rados.rados.0.smithi189.stdout:305: finishing write tid 4 to smithi18925896-23 2024-09-18T10:15:40.605 INFO:tasks.rados.rados.0.smithi189.stdout:292: done (6 left) 2024-09-18T10:15:40.606 INFO:tasks.rados.rados.0.smithi189.stdout:294: done (5 left) 2024-09-18T10:15:40.606 INFO:tasks.rados.rados.0.smithi189.stdout:306: done (4 left) 2024-09-18T10:15:40.606 INFO:tasks.rados.rados.0.smithi189.stdout:append oid 29 current snap is 27 2024-09-18T10:15:40.606 INFO:tasks.rados.rados.0.smithi189.stdout:307: seq_num 123 ranges {0=524288,524288=573440,1097728=688128,1785856=294912} 2024-09-18T10:15:40.608 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-09-18T10:15:40.606+0000 7fdfa8698700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22087) UID: 0 2024-09-18T10:15:40.610 INFO:teuthology.orchestra.run.smithi029.stderr:nodeep-scrub is unset 2024-09-18T10:15:40.612 INFO:tasks.rados.rados.0.smithi189.stdout:307: writing smithi18925896-29 from 0 to 524288 tid 1 2024-09-18T10:15:40.620 INFO:tasks.rados.rados.0.smithi189.stdout:307: writing smithi18925896-29 from 524288 to 1097728 tid 2 2024-09-18T10:15:40.620 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-18T10:15:40.620 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-18T10:15:40.620 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-18T10:15:40.621 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-18T10:15:40.621 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-18T10:15:40.624 INFO:tasks.rados.rados.0.smithi189.stdout:307: writing smithi18925896-29 from 1097728 to 1785856 tid 3 2024-09-18T10:15:40.626 INFO:tasks.rados.rados.0.smithi189.stdout:307: writing smithi18925896-29 from 1785856 to 2080768 tid 4 2024-09-18T10:15:40.626 INFO:tasks.rados.rados.0.smithi189.stdout:308: read oid 7 snap -1 2024-09-18T10:15:40.627 INFO:tasks.rados.rados.0.smithi189.stdout:308: expect deleted 2024-09-18T10:15:40.627 INFO:tasks.rados.rados.0.smithi189.stdout:309: read oid 28 snap -1 2024-09-18T10:15:40.627 INFO:tasks.rados.rados.0.smithi189.stdout:309: expect (ObjNum 33 snap 0 seq_num 33) 2024-09-18T10:15:40.627 INFO:tasks.rados.rados.0.smithi189.stdout:append oid 6 current snap is 27 2024-09-18T10:15:40.627 INFO:tasks.rados.rados.0.smithi189.stdout:310: seq_num 124 ranges {1261568=540672,1802240=720896,2523136=786432,3309568=49152} 2024-09-18T10:15:40.631 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T10:15:40.637 INFO:tasks.rados.rados.0.smithi189.stdout:310: writing smithi18925896-6 from 1261568 to 1802240 tid 1 2024-09-18T10:15:40.641 INFO:tasks.rados.rados.0.smithi189.stdout:310: writing smithi18925896-6 from 1802240 to 2523136 tid 2 2024-09-18T10:15:40.647 INFO:tasks.rados.rados.0.smithi189.stdout:310: writing smithi18925896-6 from 2523136 to 3309568 tid 3 2024-09-18T10:15:40.647 INFO:tasks.rados.rados.0.smithi189.stdout:310: writing smithi18925896-6 from 3309568 to 3358720 tid 4 2024-09-18T10:15:40.647 INFO:tasks.rados.rados.0.smithi189.stdout:311: read oid 42 snap -1 2024-09-18T10:15:40.647 INFO:tasks.rados.rados.0.smithi189.stdout:311: expect deleted 2024-09-18T10:15:40.647 INFO:tasks.rados.rados.0.smithi189.stdout:312: copy_from oid 13 from oid 34 current snap is 27 2024-09-18T10:15:40.647 INFO:tasks.rados.rados.0.smithi189.stdout:305: finishing write tid 5 to smithi18925896-23 2024-09-18T10:15:40.647 INFO:tasks.rados.rados.0.smithi189.stdout:313: rmattr oid 12 current snap is 27 2024-09-18T10:15:40.647 INFO:tasks.rados.rados.0.smithi189.stdout:313: done (10 left) 2024-09-18T10:15:40.648 INFO:tasks.rados.rados.0.smithi189.stdout:314: read oid 12 snap -1 2024-09-18T10:15:40.648 INFO:tasks.rados.rados.0.smithi189.stdout:314: expect (ObjNum 11 snap 0 seq_num 11) 2024-09-18T10:15:40.648 INFO:tasks.rados.rados.0.smithi189.stdout:append oid 41 current snap is 27 2024-09-18T10:15:40.648 INFO:tasks.rados.rados.0.smithi189.stdout:315: seq_num 125 ranges {1933312=557056,2490368=475136,2965504=507904,3473408=475136,3948544=98304} 2024-09-18T10:15:40.661 INFO:tasks.rados.rados.0.smithi189.stdout:315: writing smithi18925896-41 from 1933312 to 2490368 tid 1 2024-09-18T10:15:40.663 INFO:tasks.rados.rados.0.smithi189.stdout:315: writing smithi18925896-41 from 2490368 to 2965504 tid 2 2024-09-18T10:15:40.666 INFO:tasks.rados.rados.0.smithi189.stdout:315: writing smithi18925896-41 from 2965504 to 3473408 tid 3 2024-09-18T10:15:40.669 INFO:tasks.rados.rados.0.smithi189.stdout:315: writing smithi18925896-41 from 3473408 to 3948544 tid 4 2024-09-18T10:15:40.670 INFO:tasks.rados.rados.0.smithi189.stdout:315: writing smithi18925896-41 from 3948544 to 4046848 tid 5 2024-09-18T10:15:40.670 INFO:tasks.rados.rados.0.smithi189.stdout:316: delete oid 14 current snap is 27 2024-09-18T10:15:40.709 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-18T10:15:40.704+0000 7f1981873700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22132) UID: 0 2024-09-18T10:15:40.729 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-18T10:15:40.729 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-18T10:15:40.729 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-18T10:15:40.730 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-18T10:15:40.730 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.730 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:150 3.cs0 3:3972e5c3:::smithi18925896-50:head [append 0~458752,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:40.730 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:51.125225+0000", 2024-09-18T10:15:40.730 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 49.601395130999997, 2024-09-18T10:15:40.730 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.40871674099999999, 2024-09-18T10:15:40.731 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:40.731 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:40.731 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:40.731 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:40.731 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:40.732 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 150 2024-09-18T10:15:40.732 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.732 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:40.732 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.732 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:40.732 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.125225+0000", 2024-09-18T10:15:40.733 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:40.733 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.733 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.733 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:40.733 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.125225+0000", 2024-09-18T10:15:40.733 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:40.733 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.733 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.733 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:40.734 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.125222+0000", 2024-09-18T10:15:40.734 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999967 2024-09-18T10:15:40.734 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.734 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.734 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:40.734 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.126038+0000", 2024-09-18T10:15:40.734 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00081693399999999995 2024-09-18T10:15:40.735 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.735 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.735 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:40.735 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.126072+0000", 2024-09-18T10:15:40.735 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4038999999999997e-05 2024-09-18T10:15:40.735 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.736 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.736 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:40.736 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.126084+0000", 2024-09-18T10:15:40.736 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1908e-05 2024-09-18T10:15:40.736 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.736 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.737 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:40.737 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.126184+0000", 2024-09-18T10:15:40.737 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9868e-05 2024-09-18T10:15:40.738 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.738 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.738 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:40.738 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.126439+0000", 2024-09-18T10:15:40.738 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00025457799999999999 2024-09-18T10:15:40.739 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.739 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.739 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:40.739 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.128215+0000", 2024-09-18T10:15:40.740 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0017764499999999999 2024-09-18T10:15:40.740 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.740 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.740 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:40.740 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.232589+0000", 2024-09-18T10:15:40.740 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.104374015 2024-09-18T10:15:40.741 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.741 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.741 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:15:40.741 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.533834+0000", 2024-09-18T10:15:40.741 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30124498 2024-09-18T10:15:40.742 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.742 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.742 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:40.742 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.533941+0000", 2024-09-18T10:15:40.742 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010718299999999999 2024-09-18T10:15:40.742 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:40.743 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:40.743 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:40.743 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.743 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.743 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:151 3.cs0 3:3972e5c3:::smithi18925896-50:head [append 0~376832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:40.743 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:51.127704+0000", 2024-09-18T10:15:40.744 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 49.598916025999998, 2024-09-18T10:15:40.744 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.40663223799999998, 2024-09-18T10:15:40.744 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:40.744 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:40.744 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:40.745 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:40.745 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:40.745 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 151 2024-09-18T10:15:40.745 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.745 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:40.746 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.746 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:40.746 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.127704+0000", 2024-09-18T10:15:40.746 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:40.746 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.746 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.747 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:40.747 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.127704+0000", 2024-09-18T10:15:40.747 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:40.747 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.747 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.748 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:40.748 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.127702+0000", 2024-09-18T10:15:40.748 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:40.748 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.748 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.749 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:40.749 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.128054+0000", 2024-09-18T10:15:40.749 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00035175199999999999 2024-09-18T10:15:40.749 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.749 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.749 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:40.750 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.128091+0000", 2024-09-18T10:15:40.750 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7035e-05 2024-09-18T10:15:40.750 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.750 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.750 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:40.750 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.128105+0000", 2024-09-18T10:15:40.751 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.428e-05 2024-09-18T10:15:40.751 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.751 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.751 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:40.751 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.129336+0000", 2024-09-18T10:15:40.751 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001231326 2024-09-18T10:15:40.752 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.752 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.752 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:40.752 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.129489+0000", 2024-09-18T10:15:40.752 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015266100000000001 2024-09-18T10:15:40.753 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.753 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.753 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:40.753 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.130757+0000", 2024-09-18T10:15:40.753 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0012684409999999999 2024-09-18T10:15:40.753 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.754 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.754 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:40.754 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.332853+0000", 2024-09-18T10:15:40.754 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20209580599999999 2024-09-18T10:15:40.754 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.755 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.755 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:15:40.755 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.534218+0000", 2024-09-18T10:15:40.755 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.2013653 2024-09-18T10:15:40.755 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.755 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.756 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:40.756 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.534336+0000", 2024-09-18T10:15:40.756 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011760900000000001 2024-09-18T10:15:40.756 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:40.756 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:40.757 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:40.757 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.757 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.757 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:152 3.cs0 3:3972e5c3:::smithi18925896-50:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:40.757 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:51.128123+0000", 2024-09-18T10:15:40.757 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 49.598496459000003, 2024-09-18T10:15:40.758 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.80287000200000003, 2024-09-18T10:15:40.758 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:40.758 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:40.758 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:40.758 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:40.759 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:40.759 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 152 2024-09-18T10:15:40.759 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.759 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:40.759 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.760 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:40.760 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.128123+0000", 2024-09-18T10:15:40.760 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:40.760 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.761 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.761 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:40.761 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.128123+0000", 2024-09-18T10:15:40.761 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:40.761 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.761 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.762 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:40.762 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.128122+0000", 2024-09-18T10:15:40.762 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:40.762 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.763 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.764 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:40.764 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.128129+0000", 2024-09-18T10:15:40.764 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6730000000000003e-06 2024-09-18T10:15:40.765 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.765 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.765 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:40.765 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.128156+0000", 2024-09-18T10:15:40.765 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7489000000000001e-05 2024-09-18T10:15:40.765 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.765 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.765 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:40.765 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.128173+0000", 2024-09-18T10:15:40.765 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7102999999999999e-05 2024-09-18T10:15:40.765 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.765 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.766 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:40.766 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.535139+0000", 2024-09-18T10:15:40.766 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.40696592599999998 2024-09-18T10:15:40.766 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.766 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.766 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:40.766 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.535380+0000", 2024-09-18T10:15:40.766 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00024023600000000001 2024-09-18T10:15:40.766 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.766 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.766 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:40.766 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.536784+0000", 2024-09-18T10:15:40.766 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0014046950000000001 2024-09-18T10:15:40.766 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.767 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.767 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:40.767 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.537964+0000", 2024-09-18T10:15:40.767 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0011793050000000001 2024-09-18T10:15:40.767 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.767 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.767 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:15:40.767 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.930841+0000", 2024-09-18T10:15:40.767 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.39287700599999997 2024-09-18T10:15:40.767 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.767 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.767 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:40.767 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.930993+0000", 2024-09-18T10:15:40.767 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015280799999999999 2024-09-18T10:15:40.768 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:40.768 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:40.768 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:40.768 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.768 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.768 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:153 3.cs0 3:3972e5c3:::smithi18925896-50:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:40.768 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:51.128181+0000", 2024-09-18T10:15:40.768 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 49.598438563999999, 2024-09-18T10:15:40.768 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2060332149999999, 2024-09-18T10:15:40.768 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:40.768 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-18T10:15:40.768 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:40.768 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:40.769 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:40.769 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 153 2024-09-18T10:15:40.769 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.769 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:40.769 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.769 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:40.769 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.128181+0000", 2024-09-18T10:15:40.769 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:40.769 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.769 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.769 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:40.769 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.128181+0000", 2024-09-18T10:15:40.769 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:40.769 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.770 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.770 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:40.770 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.128180+0000", 2024-09-18T10:15:40.770 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:15:40.770 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.770 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.770 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:40.770 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.128186+0000", 2024-09-18T10:15:40.770 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.31e-06 2024-09-18T10:15:40.770 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.770 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.770 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:40.770 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.128195+0000", 2024-09-18T10:15:40.770 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5540000000000003e-06 2024-09-18T10:15:40.771 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.771 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.771 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:40.771 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.128201+0000", 2024-09-18T10:15:40.771 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2129999999999996e-06 2024-09-18T10:15:40.771 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.771 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.771 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:40.771 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.537382+0000", 2024-09-18T10:15:40.771 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.40918055399999997 2024-09-18T10:15:40.771 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.771 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.771 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2024-09-18T10:15:40.771 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.537571+0000", 2024-09-18T10:15:40.772 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00018903299999999999 2024-09-18T10:15:40.772 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.772 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.772 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:40.772 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.931789+0000", 2024-09-18T10:15:40.772 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.39421780499999998 2024-09-18T10:15:40.772 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.772 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.772 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:40.772 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.932023+0000", 2024-09-18T10:15:40.772 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000233868 2024-09-18T10:15:40.772 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.772 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.773 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:40.773 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.334215+0000", 2024-09-18T10:15:40.773 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.40219177499999997 2024-09-18T10:15:40.773 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:40.773 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:40.773 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:40.773 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.773 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.773 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:162 3.cs0 3:34d26c88:::smithi18925896-36:head [read 0~589824,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:40.773 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:51.532607+0000", 2024-09-18T10:15:40.773 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 49.194013222000002, 2024-09-18T10:15:40.773 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.400964512, 2024-09-18T10:15:40.773 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:40.773 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-18T10:15:40.774 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:40.774 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:40.774 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:40.774 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 162 2024-09-18T10:15:40.774 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.774 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:40.774 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.774 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:40.774 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.532607+0000", 2024-09-18T10:15:40.774 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:40.774 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.774 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.774 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:40.774 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.532607+0000", 2024-09-18T10:15:40.775 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:40.775 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.775 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.775 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:40.775 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.532606+0000", 2024-09-18T10:15:40.775 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:15:40.775 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.775 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.775 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:40.775 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.532612+0000", 2024-09-18T10:15:40.775 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0599999999999996e-06 2024-09-18T10:15:40.775 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.775 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.775 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:40.776 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.532621+0000", 2024-09-18T10:15:40.776 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6190000000000006e-06 2024-09-18T10:15:40.776 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.776 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.776 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:40.776 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.532627+0000", 2024-09-18T10:15:40.776 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3469999999999998e-06 2024-09-18T10:15:40.776 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.776 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.776 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:40.776 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.538755+0000", 2024-09-18T10:15:40.776 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0061281729999999998 2024-09-18T10:15:40.776 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.776 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.777 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:40.777 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.538992+0000", 2024-09-18T10:15:40.777 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00023700500000000001 2024-09-18T10:15:40.777 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.777 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.777 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:40.777 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.933571+0000", 2024-09-18T10:15:40.777 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.39457930800000002 2024-09-18T10:15:40.777 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:40.777 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:40.777 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:40.777 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.777 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.777 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:163 3.cs0 3:34d26c88:::smithi18925896-36:head [read 0~589824,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:40.778 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:51.532633+0000", 2024-09-18T10:15:40.778 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 49.193986977000002, 2024-09-18T10:15:40.778 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.70024756899999996, 2024-09-18T10:15:40.778 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:40.778 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-18T10:15:40.778 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:40.778 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:40.778 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:40.778 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 163 2024-09-18T10:15:40.778 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.778 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:40.778 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.778 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:40.778 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.532633+0000", 2024-09-18T10:15:40.779 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:40.779 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.779 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.779 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:40.779 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.532633+0000", 2024-09-18T10:15:40.779 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:40.779 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.779 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.779 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:40.779 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.532632+0000", 2024-09-18T10:15:40.779 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:15:40.779 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.779 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.779 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:40.780 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.532636+0000", 2024-09-18T10:15:40.780 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3529999999999997e-06 2024-09-18T10:15:40.780 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.780 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.780 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:40.780 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.532645+0000", 2024-09-18T10:15:40.780 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7099999999999996e-06 2024-09-18T10:15:40.780 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.780 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.780 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:40.780 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.532650+0000", 2024-09-18T10:15:40.780 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3029999999999999e-06 2024-09-18T10:15:40.780 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.781 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.781 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:40.781 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.930003+0000", 2024-09-18T10:15:40.781 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.397352333 2024-09-18T10:15:40.781 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.781 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.781 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:40.781 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.930174+0000", 2024-09-18T10:15:40.781 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017110000000000001 2024-09-18T10:15:40.781 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.781 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.781 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:40.781 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.232880+0000", 2024-09-18T10:15:40.782 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30270652300000001 2024-09-18T10:15:40.782 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:40.782 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:40.782 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:40.782 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.782 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.782 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:181 3.cs0 3:3a4fded2:::smithi18925896-23:head [append 0~524288] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:40.782 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:52.691563+0000", 2024-09-18T10:15:40.782 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 48.035056746000002, 2024-09-18T10:15:40.782 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.418704519, 2024-09-18T10:15:40.782 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:40.782 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:40.782 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:40.782 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:40.783 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:40.783 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 181 2024-09-18T10:15:40.783 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.783 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:40.783 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.783 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:40.783 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.691563+0000", 2024-09-18T10:15:40.783 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:40.783 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.783 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.783 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:40.783 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.691563+0000", 2024-09-18T10:15:40.783 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:40.783 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.784 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.784 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:40.784 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.691560+0000", 2024-09-18T10:15:40.784 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999971 2024-09-18T10:15:40.784 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.784 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.784 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:40.784 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.692740+0000", 2024-09-18T10:15:40.784 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001180195 2024-09-18T10:15:40.784 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.784 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.784 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:40.784 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.692779+0000", 2024-09-18T10:15:40.784 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8535000000000003e-05 2024-09-18T10:15:40.785 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.785 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.785 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:40.785 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.692794+0000", 2024-09-18T10:15:40.785 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4807000000000001e-05 2024-09-18T10:15:40.785 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.785 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.785 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:40.785 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.793038+0000", 2024-09-18T10:15:40.785 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100244729 2024-09-18T10:15:40.785 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.785 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.785 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-09-18T10:15:40.785 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.793050+0000", 2024-09-18T10:15:40.786 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1776e-05 2024-09-18T10:15:40.786 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.786 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.786 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:40.786 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.893883+0000", 2024-09-18T10:15:40.786 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100833383 2024-09-18T10:15:40.786 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.786 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.786 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:40.786 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.894056+0000", 2024-09-18T10:15:40.786 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017296999999999999 2024-09-18T10:15:40.786 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.786 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.786 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:40.787 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.896702+0000", 2024-09-18T10:15:40.787 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002645728 2024-09-18T10:15:40.787 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.787 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.787 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:40.787 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.001443+0000", 2024-09-18T10:15:40.787 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.104740824 2024-09-18T10:15:40.787 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.787 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.787 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:15:40.787 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.110173+0000", 2024-09-18T10:15:40.787 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.108729645 2024-09-18T10:15:40.787 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.787 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.788 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:40.788 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.110268+0000", 2024-09-18T10:15:40.788 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4987000000000004e-05 2024-09-18T10:15:40.788 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:40.788 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:40.788 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:40.788 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.788 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.788 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:182 3.cs0 3:3a4fded2:::smithi18925896-23:head [append 0~376832] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:40.788 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:52.693795+0000", 2024-09-18T10:15:40.788 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 48.032825017999997, 2024-09-18T10:15:40.788 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.51656788200000003, 2024-09-18T10:15:40.788 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:40.788 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:40.789 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:40.789 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:40.789 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:40.789 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 182 2024-09-18T10:15:40.789 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.789 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:40.789 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.789 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:40.789 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.693795+0000", 2024-09-18T10:15:40.789 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:40.789 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.789 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.789 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:40.789 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.693795+0000", 2024-09-18T10:15:40.790 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:40.790 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.790 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.790 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:40.790 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.693787+0000", 2024-09-18T10:15:40.790 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999919 2024-09-18T10:15:40.790 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.790 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.790 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:40.790 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694491+0000", 2024-09-18T10:15:40.790 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00070477199999999999 2024-09-18T10:15:40.790 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.790 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.790 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:40.791 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694526+0000", 2024-09-18T10:15:40.791 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4304999999999998e-05 2024-09-18T10:15:40.791 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.791 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.791 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:40.791 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694539+0000", 2024-09-18T10:15:40.791 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3107e-05 2024-09-18T10:15:40.791 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.791 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.791 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:40.791 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.793145+0000", 2024-09-18T10:15:40.791 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.098605719999999994 2024-09-18T10:15:40.791 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.791 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.792 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:40.792 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.793156+0000", 2024-09-18T10:15:40.792 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1892000000000001e-05 2024-09-18T10:15:40.792 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.792 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.792 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:40.792 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.898472+0000", 2024-09-18T10:15:40.792 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10531558000000001 2024-09-18T10:15:40.792 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.792 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.792 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:40.792 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.898643+0000", 2024-09-18T10:15:40.792 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017116399999999999 2024-09-18T10:15:40.792 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.793 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.793 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:40.793 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.900074+0000", 2024-09-18T10:15:40.793 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0014304299999999999 2024-09-18T10:15:40.793 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.793 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.793 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:40.793 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.001446+0000", 2024-09-18T10:15:40.793 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.101371934 2024-09-18T10:15:40.793 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.793 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.793 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:15:40.793 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.210292+0000", 2024-09-18T10:15:40.793 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.208846693 2024-09-18T10:15:40.794 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.794 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.794 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:40.794 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.210363+0000", 2024-09-18T10:15:40.794 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0470999999999997e-05 2024-09-18T10:15:40.794 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:40.794 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:40.794 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:40.794 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.794 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.794 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:183 3.cs0 3:3a4fded2:::smithi18925896-23:head [setxattr _header (56) in=7b] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:40.794 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:52.694557+0000", 2024-09-18T10:15:40.794 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 48.032062457999999, 2024-09-18T10:15:40.794 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.51641106000000003, 2024-09-18T10:15:40.795 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:40.795 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:40.795 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:40.795 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:40.795 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:40.795 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 183 2024-09-18T10:15:40.795 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.795 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:40.795 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.795 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:40.795 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694557+0000", 2024-09-18T10:15:40.795 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:40.795 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.795 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.796 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:40.796 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694557+0000", 2024-09-18T10:15:40.796 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:40.796 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.796 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.796 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:40.796 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694556+0000", 2024-09-18T10:15:40.796 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:40.796 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.797 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.797 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:40.797 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694570+0000", 2024-09-18T10:15:40.797 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4249e-05 2024-09-18T10:15:40.797 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.797 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.797 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:40.797 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694580+0000", 2024-09-18T10:15:40.797 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7459999999999995e-06 2024-09-18T10:15:40.797 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.797 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.797 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:40.797 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694601+0000", 2024-09-18T10:15:40.797 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1583e-05 2024-09-18T10:15:40.798 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.798 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.798 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:40.798 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.793251+0000", 2024-09-18T10:15:40.798 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.098649465000000006 2024-09-18T10:15:40.798 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.798 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.798 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:40.798 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.793262+0000", 2024-09-18T10:15:40.798 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1113e-05 2024-09-18T10:15:40.798 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.798 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.798 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:40.798 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.001995+0000", 2024-09-18T10:15:40.799 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20873336200000001 2024-09-18T10:15:40.799 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.799 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.799 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:40.799 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.002285+0000", 2024-09-18T10:15:40.799 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00028924300000000001 2024-09-18T10:15:40.799 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.799 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.799 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:40.799 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.003993+0000", 2024-09-18T10:15:40.799 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0017085410000000001 2024-09-18T10:15:40.799 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.799 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.800 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:40.800 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.109904+0000", 2024-09-18T10:15:40.800 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.105911038 2024-09-18T10:15:40.800 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.800 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.800 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:15:40.800 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.210762+0000", 2024-09-18T10:15:40.800 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100858146 2024-09-18T10:15:40.800 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.800 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.801 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:40.801 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.210968+0000", 2024-09-18T10:15:40.801 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00020613000000000001 2024-09-18T10:15:40.801 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:40.801 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:40.801 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:40.801 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.801 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.801 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:184 3.cs0 3:3a4fded2:::smithi18925896-23:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:40.801 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:52.694648+0000", 2024-09-18T10:15:40.801 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 48.031972234000001, 2024-09-18T10:15:40.801 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.72409976600000003, 2024-09-18T10:15:40.801 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:40.801 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-18T10:15:40.802 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:40.802 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:40.802 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:40.802 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 184 2024-09-18T10:15:40.802 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.802 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:40.802 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.802 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:40.802 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694648+0000", 2024-09-18T10:15:40.802 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:40.802 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.802 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.802 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:40.803 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694648+0000", 2024-09-18T10:15:40.803 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:40.803 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.803 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.803 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:40.803 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694647+0000", 2024-09-18T10:15:40.803 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:15:40.803 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.803 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.803 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:40.803 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694652+0000", 2024-09-18T10:15:40.803 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1170000000000003e-06 2024-09-18T10:15:40.803 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.803 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.804 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:40.804 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694668+0000", 2024-09-18T10:15:40.804 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6019e-05 2024-09-18T10:15:40.804 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.804 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.804 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:40.804 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694674+0000", 2024-09-18T10:15:40.804 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7000000000000002e-06 2024-09-18T10:15:40.804 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.804 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.804 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:40.804 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.893402+0000", 2024-09-18T10:15:40.804 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.19872805700000001 2024-09-18T10:15:40.804 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.805 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.805 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:40.805 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.893413+0000", 2024-09-18T10:15:40.805 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0329999999999999e-05 2024-09-18T10:15:40.805 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.805 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.805 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:40.805 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.004678+0000", 2024-09-18T10:15:40.805 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.111265369 2024-09-18T10:15:40.805 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.805 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.805 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2024-09-18T10:15:40.805 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.004893+0000", 2024-09-18T10:15:40.806 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00021510499999999999 2024-09-18T10:15:40.806 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.806 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.806 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:40.806 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.211096+0000", 2024-09-18T10:15:40.806 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.206202899 2024-09-18T10:15:40.806 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.806 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.806 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:40.806 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.211230+0000", 2024-09-18T10:15:40.806 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000133907 2024-09-18T10:15:40.807 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.807 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.807 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:40.807 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.418747+0000", 2024-09-18T10:15:40.807 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20751733 2024-09-18T10:15:40.807 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:40.807 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:40.807 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:40.807 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.807 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.807 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:196 3.cs0 3:351da1bf:::smithi18925896-6:head [append 0~0] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:40.807 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:52.694697+0000", 2024-09-18T10:15:40.807 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 48.031923136000003, 2024-09-18T10:15:40.807 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.61758450099999995, 2024-09-18T10:15:40.808 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:40.808 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:40.808 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:40.808 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:40.808 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:40.808 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 196 2024-09-18T10:15:40.808 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.808 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:40.808 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.808 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:40.808 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694697+0000", 2024-09-18T10:15:40.808 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:40.808 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.809 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.809 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:40.809 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694697+0000", 2024-09-18T10:15:40.809 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:40.809 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.809 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.809 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:40.809 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694696+0000", 2024-09-18T10:15:40.809 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:15:40.809 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.809 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.809 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:40.809 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694705+0000", 2024-09-18T10:15:40.810 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7799999999999995e-06 2024-09-18T10:15:40.810 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.810 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.810 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:40.810 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694721+0000", 2024-09-18T10:15:40.810 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5102e-05 2024-09-18T10:15:40.810 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.810 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.810 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:40.810 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694727+0000", 2024-09-18T10:15:40.810 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2570000000000004e-06 2024-09-18T10:15:40.810 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.810 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.810 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:40.811 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.893457+0000", 2024-09-18T10:15:40.811 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.198729824 2024-09-18T10:15:40.811 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.811 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.811 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:40.811 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.893464+0000", 2024-09-18T10:15:40.811 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6860000000000006e-06 2024-09-18T10:15:40.811 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.811 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.811 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:40.811 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.105086+0000", 2024-09-18T10:15:40.811 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.21162193000000001 2024-09-18T10:15:40.811 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.811 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.812 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:40.812 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.105357+0000", 2024-09-18T10:15:40.812 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00027109699999999998 2024-09-18T10:15:40.812 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.812 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.812 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:40.812 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.107161+0000", 2024-09-18T10:15:40.812 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001803727 2024-09-18T10:15:40.812 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.812 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.812 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:40.812 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.109961+0000", 2024-09-18T10:15:40.812 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0028003080000000001 2024-09-18T10:15:40.813 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.813 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.813 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:15:40.813 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.312174+0000", 2024-09-18T10:15:40.813 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20221219700000001 2024-09-18T10:15:40.813 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.813 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.813 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:40.813 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.312281+0000", 2024-09-18T10:15:40.813 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000107656 2024-09-18T10:15:40.813 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:40.813 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:40.813 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:40.814 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.814 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.814 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:197 3.cs0 3:351da1bf:::smithi18925896-6:head [rollback 2] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:40.814 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:52.694734+0000", 2024-09-18T10:15:40.814 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 48.031886094000001, 2024-09-18T10:15:40.814 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.61830483899999999, 2024-09-18T10:15:40.814 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:40.814 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:40.814 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:40.814 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:40.814 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:40.814 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 197 2024-09-18T10:15:40.814 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.814 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:40.815 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.815 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:40.815 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694734+0000", 2024-09-18T10:15:40.815 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:40.815 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.815 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.815 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:40.815 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694734+0000", 2024-09-18T10:15:40.815 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:40.815 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.815 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.815 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:40.815 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694733+0000", 2024-09-18T10:15:40.815 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:15:40.816 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.816 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.816 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:40.816 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694737+0000", 2024-09-18T10:15:40.816 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.636e-06 2024-09-18T10:15:40.816 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.816 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.816 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:40.816 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694752+0000", 2024-09-18T10:15:40.816 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5792999999999999e-05 2024-09-18T10:15:40.816 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.816 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.816 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:40.817 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694758+0000", 2024-09-18T10:15:40.817 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2660000000000001e-06 2024-09-18T10:15:40.817 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.817 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.817 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:40.817 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.108286+0000", 2024-09-18T10:15:40.817 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.41352792700000002 2024-09-18T10:15:40.817 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.817 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.817 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:40.817 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.108540+0000", 2024-09-18T10:15:40.817 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000253928 2024-09-18T10:15:40.817 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.817 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.818 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:40.818 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.109284+0000", 2024-09-18T10:15:40.818 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00074488600000000005 2024-09-18T10:15:40.818 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.818 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.818 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:40.818 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.210174+0000", 2024-09-18T10:15:40.818 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10088973699999999 2024-09-18T10:15:40.818 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.818 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.818 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:15:40.818 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.312857+0000", 2024-09-18T10:15:40.818 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10268295600000001 2024-09-18T10:15:40.818 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.819 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.819 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:40.819 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.313039+0000", 2024-09-18T10:15:40.819 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000181476 2024-09-18T10:15:40.819 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:40.819 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:40.819 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:40.819 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.819 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.819 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:633 3.cs0 3:3972e5c3:::smithi18925896-50:head [read 0~3981312,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-18T10:15:40.819 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:28.815454+0000", 2024-09-18T10:15:40.819 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 11.911166160000001, 2024-09-18T10:15:40.819 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.41958616100000001, 2024-09-18T10:15:40.819 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:40.820 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-18T10:15:40.820 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:40.820 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:40.820 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:40.820 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 633 2024-09-18T10:15:40.820 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.820 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:40.820 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.820 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:40.820 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:28.815454+0000", 2024-09-18T10:15:40.820 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:40.820 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.820 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.820 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:40.821 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:28.815454+0000", 2024-09-18T10:15:40.821 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:40.821 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.821 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.821 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:40.821 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:28.815453+0000", 2024-09-18T10:15:40.821 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:15:40.821 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.821 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.821 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:40.821 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:28.815458+0000", 2024-09-18T10:15:40.821 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1760000000000001e-06 2024-09-18T10:15:40.821 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.821 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.821 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:40.822 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:28.815467+0000", 2024-09-18T10:15:40.822 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1300000000000007e-06 2024-09-18T10:15:40.822 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.822 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.822 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:40.822 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:28.815484+0000", 2024-09-18T10:15:40.822 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6722e-05 2024-09-18T10:15:40.822 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.822 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.822 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:40.822 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.021577+0000", 2024-09-18T10:15:40.822 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20609329200000001 2024-09-18T10:15:40.822 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.823 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.823 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:40.823 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.021788+0000", 2024-09-18T10:15:40.823 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000211115 2024-09-18T10:15:40.823 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.823 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.823 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:40.823 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.235040+0000", 2024-09-18T10:15:40.823 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.213251576 2024-09-18T10:15:40.823 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:40.823 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:40.823 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:40.823 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.823 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.824 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:750 3.cs0 3:3a4fded2:::smithi18925896-23:head [append 0~507904] snapc 2d=[29,28,20,1f,1e,1a,11] ondisk+write+known_if_redirected+supports_pool_eio e84)", 2024-09-18T10:15:40.824 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:39.699368+0000", 2024-09-18T10:15:40.824 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.0272523200000001, 2024-09-18T10:15:40.824 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.71043271500000005, 2024-09-18T10:15:40.824 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:40.824 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:40.824 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:40.824 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:40.824 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:40.824 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 750 2024-09-18T10:15:40.824 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.824 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:40.824 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.824 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:40.825 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.699368+0000", 2024-09-18T10:15:40.825 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:40.825 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.825 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.825 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:40.825 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.699368+0000", 2024-09-18T10:15:40.825 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:40.825 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.825 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.825 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:40.825 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.699365+0000", 2024-09-18T10:15:40.825 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999971 2024-09-18T10:15:40.825 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.825 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.826 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:40.826 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.700330+0000", 2024-09-18T10:15:40.826 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00096560400000000001 2024-09-18T10:15:40.826 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.826 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.826 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:40.826 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.700363+0000", 2024-09-18T10:15:40.826 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2335999999999997e-05 2024-09-18T10:15:40.826 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.826 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.826 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:40.826 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.700375+0000", 2024-09-18T10:15:40.826 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2319e-05 2024-09-18T10:15:40.826 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.827 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.827 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:40.827 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.880022+0000", 2024-09-18T10:15:40.827 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.17964736000000001 2024-09-18T10:15:40.827 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.827 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.827 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:40.827 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.880560+0000", 2024-09-18T10:15:40.827 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00053748100000000003 2024-09-18T10:15:40.827 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.827 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.827 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:40.827 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.883059+0000", 2024-09-18T10:15:40.827 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002499631 2024-09-18T10:15:40.828 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.828 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.828 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:40.828 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.990736+0000", 2024-09-18T10:15:40.828 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10767652 2024-09-18T10:15:40.828 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.828 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.828 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:15:40.828 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.408641+0000", 2024-09-18T10:15:40.828 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.41790554299999999 2024-09-18T10:15:40.828 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.828 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.828 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:40.828 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.409800+0000", 2024-09-18T10:15:40.829 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001158854 2024-09-18T10:15:40.829 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:40.829 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:40.829 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:40.829 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.829 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.829 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:751 3.cs0 3:3a4fded2:::smithi18925896-23:head [append 0~425984] snapc 2d=[29,28,20,1f,1e,1a,11] ondisk+write+known_if_redirected+supports_pool_eio e84)", 2024-09-18T10:15:40.829 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:39.701793+0000", 2024-09-18T10:15:40.829 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.024827229, 2024-09-18T10:15:40.829 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.70943514100000005, 2024-09-18T10:15:40.829 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:40.829 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:40.829 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:40.829 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:40.830 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:40.830 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 751 2024-09-18T10:15:40.830 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.830 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:40.830 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.830 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:40.830 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.701793+0000", 2024-09-18T10:15:40.830 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:40.830 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.830 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.830 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:40.831 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.701793+0000", 2024-09-18T10:15:40.831 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:40.831 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.831 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.831 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:40.831 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.701791+0000", 2024-09-18T10:15:40.831 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:40.831 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.831 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.831 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:40.831 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.702524+0000", 2024-09-18T10:15:40.831 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00073345599999999995 2024-09-18T10:15:40.832 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.832 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.832 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:40.832 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.702550+0000", 2024-09-18T10:15:40.832 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5989000000000001e-05 2024-09-18T10:15:40.832 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.832 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.832 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:40.832 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.702560+0000", 2024-09-18T10:15:40.832 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7270000000000002e-06 2024-09-18T10:15:40.832 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.832 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.832 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:40.832 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.986833+0000", 2024-09-18T10:15:40.833 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.28427270399999999 2024-09-18T10:15:40.833 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.833 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.833 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:40.833 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.987210+0000", 2024-09-18T10:15:40.833 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00037690899999999998 2024-09-18T10:15:40.833 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.833 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.833 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:40.833 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.989054+0000", 2024-09-18T10:15:40.833 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001844856 2024-09-18T10:15:40.833 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.833 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.833 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:40.834 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.102805+0000", 2024-09-18T10:15:40.834 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.113750928 2024-09-18T10:15:40.834 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.834 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.834 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:15:40.834 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.410623+0000", 2024-09-18T10:15:40.834 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30781788199999999 2024-09-18T10:15:40.834 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.834 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.834 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:40.834 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.411228+0000", 2024-09-18T10:15:40.834 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00060455000000000003 2024-09-18T10:15:40.834 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:40.834 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:40.835 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:40.835 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.835 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.835 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:752 3.cs0 3:3a4fded2:::smithi18925896-23:head [append 0~442368] snapc 2d=[29,28,20,1f,1e,1a,11] ondisk+write+known_if_redirected+supports_pool_eio e84)", 2024-09-18T10:15:40.835 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:39.704347+0000", 2024-09-18T10:15:40.835 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.0222731030000001, 2024-09-18T10:15:40.835 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.70797012699999995, 2024-09-18T10:15:40.835 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:40.835 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:40.835 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:40.835 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:40.835 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:40.835 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 752 2024-09-18T10:15:40.835 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.836 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:40.836 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.836 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:40.836 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.704347+0000", 2024-09-18T10:15:40.836 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:40.836 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.836 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.836 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:40.836 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.704347+0000", 2024-09-18T10:15:40.836 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:40.836 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.836 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.836 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:40.837 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.704339+0000", 2024-09-18T10:15:40.837 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999928 2024-09-18T10:15:40.837 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.837 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.837 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:40.837 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.705024+0000", 2024-09-18T10:15:40.837 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00068492100000000001 2024-09-18T10:15:40.837 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.837 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.837 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:40.837 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.705059+0000", 2024-09-18T10:15:40.837 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4739000000000001e-05 2024-09-18T10:15:40.837 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.837 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.838 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:40.838 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.705072+0000", 2024-09-18T10:15:40.838 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3125999999999999e-05 2024-09-18T10:15:40.838 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.838 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.838 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:40.838 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.991070+0000", 2024-09-18T10:15:40.838 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.285997483 2024-09-18T10:15:40.838 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.838 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.838 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:40.838 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.991414+0000", 2024-09-18T10:15:40.838 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00034466500000000002 2024-09-18T10:15:40.838 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.839 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.839 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:40.839 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.993220+0000", 2024-09-18T10:15:40.839 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001805891 2024-09-18T10:15:40.839 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.839 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.839 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:40.839 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.206261+0000", 2024-09-18T10:15:40.839 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.21304089300000001 2024-09-18T10:15:40.839 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.839 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.839 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:15:40.839 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.411747+0000", 2024-09-18T10:15:40.840 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.205486321 2024-09-18T10:15:40.840 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.840 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.840 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:40.840 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.412317+0000", 2024-09-18T10:15:40.840 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00056940799999999998 2024-09-18T10:15:40.840 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:40.840 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:40.840 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:40.840 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.840 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.840 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:753 3.cs0 3:3a4fded2:::smithi18925896-23:head [append 0~540672 in=540672b] snapc 2d=[29,28,20,1f,1e,1a,11] ondisk+write+known_if_redirected+supports_pool_eio e84)", 2024-09-18T10:15:40.840 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:39.707399+0000", 2024-09-18T10:15:40.841 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.0192211630000001, 2024-09-18T10:15:40.841 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.80469576600000003, 2024-09-18T10:15:40.841 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:40.841 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:40.841 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:40.841 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:40.841 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:40.841 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 753 2024-09-18T10:15:40.841 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.841 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:40.841 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.841 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:40.841 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.707399+0000", 2024-09-18T10:15:40.841 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:40.842 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.842 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.842 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:40.842 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.707399+0000", 2024-09-18T10:15:40.842 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:40.842 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.842 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.842 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:40.842 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.707391+0000", 2024-09-18T10:15:40.842 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999928 2024-09-18T10:15:40.842 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.842 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.842 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:40.842 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.708412+0000", 2024-09-18T10:15:40.843 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010203759999999999 2024-09-18T10:15:40.843 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.843 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.843 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:40.843 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.708445+0000", 2024-09-18T10:15:40.843 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3674999999999999e-05 2024-09-18T10:15:40.843 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.843 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.843 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:40.843 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.708458+0000", 2024-09-18T10:15:40.844 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2943e-05 2024-09-18T10:15:40.844 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.844 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.844 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:40.844 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.994678+0000", 2024-09-18T10:15:40.844 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.28621953700000002 2024-09-18T10:15:40.844 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.844 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.844 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:40.844 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.995068+0000", 2024-09-18T10:15:40.844 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000390338 2024-09-18T10:15:40.844 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.844 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.845 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:40.845 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.997188+0000", 2024-09-18T10:15:40.845 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0021194159999999998 2024-09-18T10:15:40.845 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.845 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.845 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:40.845 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.206267+0000", 2024-09-18T10:15:40.845 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20907888599999999 2024-09-18T10:15:40.845 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.845 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.845 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:15:40.845 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.510874+0000", 2024-09-18T10:15:40.845 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30460767599999999 2024-09-18T10:15:40.845 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.846 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.846 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:40.846 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.512094+0000", 2024-09-18T10:15:40.846 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001220215 2024-09-18T10:15:40.846 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:40.846 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:40.846 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:40.846 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.846 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.846 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:754 3.cs0 3:3a4fded2:::smithi18925896-23:head [append 0~147456 in=147456b] snapc 2d=[29,28,20,1f,1e,1a,11] ondisk+write+known_if_redirected+supports_pool_eio e84)", 2024-09-18T10:15:40.846 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:39.708472+0000", 2024-09-18T10:15:40.846 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.0181482049999999, 2024-09-18T10:15:40.846 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.90559658799999998, 2024-09-18T10:15:40.846 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:40.847 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:40.847 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:40.847 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:40.847 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:40.847 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 754 2024-09-18T10:15:40.847 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.847 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:40.847 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.847 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:40.847 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.708472+0000", 2024-09-18T10:15:40.847 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:40.847 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.847 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.847 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:40.847 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.708472+0000", 2024-09-18T10:15:40.848 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:40.848 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.848 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.848 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:40.848 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.708470+0000", 2024-09-18T10:15:40.848 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:40.848 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.848 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.848 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:40.848 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.708676+0000", 2024-09-18T10:15:40.848 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00020609599999999999 2024-09-18T10:15:40.848 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.848 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.848 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:40.849 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.708704+0000", 2024-09-18T10:15:40.849 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7138999999999999e-05 2024-09-18T10:15:40.849 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.849 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.849 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:40.849 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.708713+0000", 2024-09-18T10:15:40.849 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1500000000000005e-06 2024-09-18T10:15:40.849 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.849 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.849 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:40.849 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.099648+0000", 2024-09-18T10:15:40.849 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.39093543800000002 2024-09-18T10:15:40.849 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.850 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.850 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:40.850 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.100061+0000", 2024-09-18T10:15:40.850 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000412958 2024-09-18T10:15:40.850 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.850 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.850 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:40.850 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.101562+0000", 2024-09-18T10:15:40.850 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001500758 2024-09-18T10:15:40.850 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.850 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.850 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:40.850 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.206942+0000", 2024-09-18T10:15:40.850 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10538027 2024-09-18T10:15:40.851 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.851 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.851 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:15:40.851 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.612979+0000", 2024-09-18T10:15:40.851 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.40603637599999998 2024-09-18T10:15:40.851 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.851 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.851 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:40.851 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.614068+0000", 2024-09-18T10:15:40.851 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001089717 2024-09-18T10:15:40.851 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:40.851 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:40.851 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:40.851 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.852 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.852 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWriteReply(3.cs0 84/83 ECSubWriteReply(tid=140, last_complete=84'116, committed=1, applied=1))", 2024-09-18T10:15:40.852 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:40.207380+0000", 2024-09-18T10:15:40.852 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.51924035300000004, 2024-09-18T10:15:40.852 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.405148905, 2024-09-18T10:15:40.852 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:40.852 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:40.852 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:40.852 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.852 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:40.852 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.207380+0000", 2024-09-18T10:15:40.852 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:40.852 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.852 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.853 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:40.853 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.207380+0000", 2024-09-18T10:15:40.853 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:40.853 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.853 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.853 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:40.853 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.207372+0000", 2024-09-18T10:15:40.853 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999928 2024-09-18T10:15:40.853 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.853 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.853 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:40.853 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.207389+0000", 2024-09-18T10:15:40.853 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6886000000000001e-05 2024-09-18T10:15:40.853 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.854 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.854 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:40.854 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.207414+0000", 2024-09-18T10:15:40.854 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4666000000000001e-05 2024-09-18T10:15:40.854 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.854 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.854 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:40.854 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.207424+0000", 2024-09-18T10:15:40.854 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0192e-05 2024-09-18T10:15:40.854 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.854 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.854 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:40.854 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.612475+0000", 2024-09-18T10:15:40.854 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.40505118299999998 2024-09-18T10:15:40.855 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.855 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.855 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:40.855 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.612528+0000", 2024-09-18T10:15:40.855 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3223999999999997e-05 2024-09-18T10:15:40.855 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:40.855 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:40.855 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:40.855 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.855 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.855 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWriteReply(3.cs0 84/83 ECSubWriteReply(tid=140, last_complete=84'116, committed=1, applied=1))", 2024-09-18T10:15:40.855 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:40.207481+0000", 2024-09-18T10:15:40.855 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.51913918000000003, 2024-09-18T10:15:40.856 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.40487226399999998, 2024-09-18T10:15:40.856 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:40.856 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:40.856 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:40.856 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.856 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:40.856 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.207481+0000", 2024-09-18T10:15:40.856 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:40.856 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.856 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.856 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:40.856 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.207481+0000", 2024-09-18T10:15:40.856 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:40.856 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.857 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.857 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:40.857 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.207480+0000", 2024-09-18T10:15:40.857 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:15:40.857 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.857 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.857 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:40.857 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.207485+0000", 2024-09-18T10:15:40.857 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6470000000000003e-06 2024-09-18T10:15:40.857 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.857 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.857 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:40.857 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.207502+0000", 2024-09-18T10:15:40.858 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6209e-05 2024-09-18T10:15:40.858 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.858 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.858 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:40.858 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.207510+0000", 2024-09-18T10:15:40.858 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9670000000000001e-06 2024-09-18T10:15:40.858 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.858 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.858 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:40.858 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.612319+0000", 2024-09-18T10:15:40.858 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.40480951300000001 2024-09-18T10:15:40.858 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.858 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.858 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:40.858 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.612353+0000", 2024-09-18T10:15:40.859 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3895e-05 2024-09-18T10:15:40.859 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:40.859 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:40.859 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:40.859 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:40.859 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:40.859 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-18T10:15:40.860 INFO:tasks.rados.rados.0.smithi189.stdout:305: finishing write tid 6 to smithi18925896-23 2024-09-18T10:15:40.861 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-09-18T10:15:40.862 INFO:tasks.ceph.osd.10.smithi154.stderr:2024-09-18T10:15:40.858+0000 7fb73221a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 22091) UID: 0 2024-09-18T10:15:40.961 INFO:tasks.ceph.osd.4.smithi029.stderr:2024-09-18T10:15:40.953+0000 7f550cf6e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22189) UID: 0 2024-09-18T10:15:40.972 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-18T10:15:40.972 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-18T10:15:40.972 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.972 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:784 3.16s0 3:6c64c4fa:::smithi18925896-41:head [append 0~557056 in=557056b] snapc 2f=[2f,29,28,20,1f,1e,1a,11] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-18T10:15:40.972 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:40.660850+0000", 2024-09-18T10:15:40.972 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.31025934300000002, 2024-09-18T10:15:40.972 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.310281955, 2024-09-18T10:15:40.973 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:40.973 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "waiting for degraded object", 2024-09-18T10:15:40.973 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:40.973 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:40.973 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:40.973 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 784 2024-09-18T10:15:40.973 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.973 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:40.973 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.973 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:40.973 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.660850+0000", 2024-09-18T10:15:40.973 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:40.973 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.974 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.974 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:40.974 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.660850+0000", 2024-09-18T10:15:40.974 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:40.974 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.974 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.974 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:40.974 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.660847+0000", 2024-09-18T10:15:40.974 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999976 2024-09-18T10:15:40.974 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.974 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.974 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:40.974 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.661846+0000", 2024-09-18T10:15:40.974 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00099824000000000006 2024-09-18T10:15:40.975 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.975 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.975 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:40.975 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.661878+0000", 2024-09-18T10:15:40.975 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2224000000000002e-05 2024-09-18T10:15:40.975 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.975 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.975 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:40.975 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.761995+0000", 2024-09-18T10:15:40.975 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100117568 2024-09-18T10:15:40.975 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.975 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.975 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:40.975 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.762894+0000", 2024-09-18T10:15:40.976 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00089831700000000004 2024-09-18T10:15:40.976 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.976 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.976 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for degraded object", 2024-09-18T10:15:40.976 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.764145+0000", 2024-09-18T10:15:40.976 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001251203 2024-09-18T10:15:40.976 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:40.976 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:40.976 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:40.976 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.976 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.976 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:785 3.16s0 3:6c64c4fa:::smithi18925896-41:head [append 0~475136 in=475136b] snapc 2f=[2f,29,28,20,1f,1e,1a,11] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-18T10:15:40.976 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:40.762044+0000", 2024-09-18T10:15:40.977 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.209065746, 2024-09-18T10:15:40.977 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20913469500000001, 2024-09-18T10:15:40.977 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:40.977 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "waiting for degraded object", 2024-09-18T10:15:40.977 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:40.977 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:40.977 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:40.977 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 785 2024-09-18T10:15:40.977 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.977 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:40.977 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.977 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:40.977 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.762044+0000", 2024-09-18T10:15:40.977 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:40.978 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.978 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.978 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:40.978 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.762044+0000", 2024-09-18T10:15:40.978 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:40.978 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.978 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.978 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:40.978 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.762041+0000", 2024-09-18T10:15:40.978 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999971 2024-09-18T10:15:40.978 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.978 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.978 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:40.978 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.762719+0000", 2024-09-18T10:15:40.979 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00067863199999999998 2024-09-18T10:15:40.979 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.979 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.979 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:40.979 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.762746+0000", 2024-09-18T10:15:40.979 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6729000000000001e-05 2024-09-18T10:15:40.979 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.979 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.979 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:40.980 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.762757+0000", 2024-09-18T10:15:40.980 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.065e-05 2024-09-18T10:15:40.980 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.980 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.980 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:40.980 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.862224+0000", 2024-09-18T10:15:40.980 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.099467441000000004 2024-09-18T10:15:40.980 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.980 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.980 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for degraded object", 2024-09-18T10:15:40.980 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.862415+0000", 2024-09-18T10:15:40.980 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000191323 2024-09-18T10:15:40.980 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:40.981 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:40.981 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:40.981 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.981 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.981 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:786 3.16s0 3:6c64c4fa:::smithi18925896-41:head [append 0~507904 in=507904b] snapc 2f=[2f,29,28,20,1f,1e,1a,11] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-18T10:15:40.981 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:40.762770+0000", 2024-09-18T10:15:40.981 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.20833927299999999, 2024-09-18T10:15:40.981 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.208447138, 2024-09-18T10:15:40.981 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:40.981 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "waiting for degraded object", 2024-09-18T10:15:40.981 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:40.981 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:40.981 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:40.981 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 786 2024-09-18T10:15:40.982 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.982 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:40.982 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.982 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:40.982 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.762770+0000", 2024-09-18T10:15:40.982 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:40.982 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.982 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.982 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:40.982 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.762770+0000", 2024-09-18T10:15:40.982 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:40.982 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.982 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.982 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:40.983 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.762769+0000", 2024-09-18T10:15:40.983 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:15:40.983 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.983 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.983 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:40.983 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.763385+0000", 2024-09-18T10:15:40.983 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00061559099999999997 2024-09-18T10:15:40.983 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.983 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.983 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:40.983 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.763396+0000", 2024-09-18T10:15:40.983 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1590000000000001e-05 2024-09-18T10:15:40.983 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.983 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.984 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:40.984 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.763403+0000", 2024-09-18T10:15:40.984 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5860000000000001e-06 2024-09-18T10:15:40.984 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.984 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.984 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:40.984 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.862478+0000", 2024-09-18T10:15:40.984 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.099075239999999995 2024-09-18T10:15:40.984 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.984 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.984 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for degraded object", 2024-09-18T10:15:40.984 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.862630+0000", 2024-09-18T10:15:40.984 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000152361 2024-09-18T10:15:40.985 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:40.985 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:40.985 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:40.985 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.985 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.985 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:787 3.16s0 3:6c64c4fa:::smithi18925896-41:head [append 0~475136 in=475136b] snapc 2f=[2f,29,28,20,1f,1e,1a,11] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-18T10:15:40.985 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:40.763410+0000", 2024-09-18T10:15:40.985 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.20769916999999999, 2024-09-18T10:15:40.985 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20784301099999999, 2024-09-18T10:15:40.985 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:40.985 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "waiting for degraded object", 2024-09-18T10:15:40.985 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:40.985 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:40.985 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:40.986 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 787 2024-09-18T10:15:40.986 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.986 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:40.986 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.986 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:40.986 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.763410+0000", 2024-09-18T10:15:40.986 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:40.986 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.986 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.986 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:40.986 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.763410+0000", 2024-09-18T10:15:40.986 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:40.986 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.986 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.987 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:40.987 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.763409+0000", 2024-09-18T10:15:40.987 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:15:40.987 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.987 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.987 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:40.987 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.764104+0000", 2024-09-18T10:15:40.987 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00069473400000000002 2024-09-18T10:15:40.987 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.987 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.987 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:40.987 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.764161+0000", 2024-09-18T10:15:40.987 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6535e-05 2024-09-18T10:15:40.987 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.988 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.988 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:40.988 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.764186+0000", 2024-09-18T10:15:40.988 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5219e-05 2024-09-18T10:15:40.988 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.988 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.988 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:40.988 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.962897+0000", 2024-09-18T10:15:40.988 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.19871138699999999 2024-09-18T10:15:40.988 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.988 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.988 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for degraded object", 2024-09-18T10:15:40.988 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.963052+0000", 2024-09-18T10:15:40.989 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015532400000000001 2024-09-18T10:15:40.989 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:40.989 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:40.989 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:40.989 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.989 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.989 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:788 3.16s0 3:6c64c4fa:::smithi18925896-41:head [append 0~98304 in=98304b] snapc 2f=[2f,29,28,20,1f,1e,1a,11] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-18T10:15:40.989 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:40.764216+0000", 2024-09-18T10:15:40.989 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.206893416, 2024-09-18T10:15:40.989 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.207073699, 2024-09-18T10:15:40.989 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:40.989 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "waiting for degraded object", 2024-09-18T10:15:40.989 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:40.989 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:40.990 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:40.990 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 788 2024-09-18T10:15:40.990 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.990 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:40.990 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.990 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:40.990 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.764216+0000", 2024-09-18T10:15:40.990 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:40.990 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.990 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.990 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:40.990 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.764216+0000", 2024-09-18T10:15:40.990 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:40.990 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.991 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.991 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:40.991 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.764215+0000", 2024-09-18T10:15:40.991 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:15:40.991 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.991 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.991 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:40.991 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.764363+0000", 2024-09-18T10:15:40.991 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000147675 2024-09-18T10:15:40.991 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.991 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.991 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:40.991 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.764381+0000", 2024-09-18T10:15:40.991 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8022000000000001e-05 2024-09-18T10:15:40.992 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.992 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.992 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:40.992 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.764388+0000", 2024-09-18T10:15:40.992 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.323e-06 2024-09-18T10:15:40.992 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.992 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.992 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:40.992 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.963158+0000", 2024-09-18T10:15:40.992 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.198770002 2024-09-18T10:15:40.992 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.992 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.992 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for degraded object", 2024-09-18T10:15:40.992 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.963350+0000", 2024-09-18T10:15:40.993 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019216599999999999 2024-09-18T10:15:40.993 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:40.993 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:40.993 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:40.993 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.993 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.993 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:789 3.16s0 3:6c64c4fa:::smithi18925896-41:head [setxattr _header (57) in=64b] snapc 2f=[2f,29,28,20,1f,1e,1a,11] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-18T10:15:40.993 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:40.764395+0000", 2024-09-18T10:15:40.993 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.20671383300000001, 2024-09-18T10:15:40.993 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20693097199999999, 2024-09-18T10:15:40.993 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:40.993 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "waiting for degraded object", 2024-09-18T10:15:40.993 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:40.993 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:40.994 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:40.994 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 789 2024-09-18T10:15:40.994 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.994 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:40.994 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.994 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:40.994 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.764395+0000", 2024-09-18T10:15:40.994 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:40.994 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.994 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.994 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:40.994 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.764395+0000", 2024-09-18T10:15:40.994 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:40.995 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.995 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.995 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:40.995 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.764395+0000", 2024-09-18T10:15:40.995 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:15:40.995 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.995 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.995 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:40.995 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.764400+0000", 2024-09-18T10:15:40.995 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0069999999999997e-06 2024-09-18T10:15:40.995 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.995 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.995 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:40.995 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.764412+0000", 2024-09-18T10:15:40.996 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2706e-05 2024-09-18T10:15:40.996 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.996 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.996 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:40.996 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.764422+0000", 2024-09-18T10:15:40.996 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.346e-06 2024-09-18T10:15:40.996 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.996 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.996 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:40.996 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.964208+0000", 2024-09-18T10:15:40.996 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.19978633500000001 2024-09-18T10:15:40.996 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.996 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.996 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for degraded object", 2024-09-18T10:15:40.997 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.964450+0000", 2024-09-18T10:15:40.997 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000241759 2024-09-18T10:15:40.997 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:40.997 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:40.997 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:40.997 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.997 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.997 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:790 3.16s0 3:6c64c4fa:::smithi18925896-41:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e85)", 2024-09-18T10:15:40.997 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:40.764432+0000", 2024-09-18T10:15:40.997 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.20667701799999999, 2024-09-18T10:15:40.998 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20693012799999999, 2024-09-18T10:15:40.998 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:40.998 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "waiting for degraded object", 2024-09-18T10:15:40.998 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:40.998 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:40.998 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:40.998 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 790 2024-09-18T10:15:40.998 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.998 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:40.998 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.998 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:40.998 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.764432+0000", 2024-09-18T10:15:40.998 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:40.998 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.999 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.999 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:40.999 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.764432+0000", 2024-09-18T10:15:40.999 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:40.999 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.999 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.999 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:40.999 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.764431+0000", 2024-09-18T10:15:40.999 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:15:40.999 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:40.999 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:40.999 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:40.999 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.764437+0000", 2024-09-18T10:15:40.999 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.587e-06 2024-09-18T10:15:41.000 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.000 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.000 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:41.000 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.764460+0000", 2024-09-18T10:15:41.000 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3232000000000001e-05 2024-09-18T10:15:41.000 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.000 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.000 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:41.000 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.764467+0000", 2024-09-18T10:15:41.000 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5370000000000003e-06 2024-09-18T10:15:41.000 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.001 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.001 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:41.001 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.964540+0000", 2024-09-18T10:15:41.001 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20007328099999999 2024-09-18T10:15:41.001 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.001 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.001 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for degraded object", 2024-09-18T10:15:41.001 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.964721+0000", 2024-09-18T10:15:41.001 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00018072699999999999 2024-09-18T10:15:41.001 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:41.001 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:41.001 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:41.001 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:41.002 INFO:teuthology.orchestra.run.smithi136.stdout: ], 2024-09-18T10:15:41.002 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 7 2024-09-18T10:15:41.002 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-18T10:15:41.003 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-09-18T10:15:41.062 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-18T10:15:41.060+0000 7f14c6a1a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 22138) UID: 0 2024-09-18T10:15:41.068 INFO:tasks.rados.rados.0.smithi189.stdout:312: finishing copy_from racing read to smithi18925896-13 2024-09-18T10:15:41.103 INFO:tasks.rados.rados.0.smithi189.stdout:304: finishing write tid 2 to smithi18925896-16 2024-09-18T10:15:41.106 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-18T10:15:41.106 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-18T10:15:41.106 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-18T10:15:41.106 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-18T10:15:41.106 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-18T10:15:41.109 INFO:tasks.rados.rados.0.smithi189.stdout:304: finishing write tid 3 to smithi18925896-16 2024-09-18T10:15:41.117 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-09-18T10:15:41.158 INFO:tasks.rados.rados.0.smithi189.stdout:314: expect (ObjNum 11 snap 0 seq_num 11) 2024-09-18T10:15:41.163 INFO:tasks.ceph.osd.4.smithi029.stderr:2024-09-18T10:15:41.157+0000 7f550cf6e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22189) UID: 0 2024-09-18T10:15:41.173 INFO:tasks.rados.rados.0.smithi189.stdout:309: expect (ObjNum 33 snap 0 seq_num 33) 2024-09-18T10:15:41.211 INFO:tasks.rados.rados.0.smithi189.stdout:304: finishing write tid 4 to smithi18925896-16 2024-09-18T10:15:41.214 INFO:tasks.rados.rados.0.smithi189.stdout:304: finishing write tid 5 to smithi18925896-16 2024-09-18T10:15:41.216 INFO:tasks.rados.rados.0.smithi189.stdout:307: finishing write tid 1 to smithi18925896-29 2024-09-18T10:15:41.216 INFO:tasks.rados.rados.0.smithi189.stdout:307: finishing write tid 2 to smithi18925896-29 2024-09-18T10:15:41.218 INFO:tasks.rados.rados.0.smithi189.stdout:308: done (12 left) 2024-09-18T10:15:41.218 INFO:tasks.rados.rados.0.smithi189.stdout:309: done (11 left) 2024-09-18T10:15:41.218 INFO:tasks.rados.rados.0.smithi189.stdout:311: done (10 left) 2024-09-18T10:15:41.218 INFO:tasks.rados.rados.0.smithi189.stdout:314: done (9 left) 2024-09-18T10:15:41.218 INFO:tasks.rados.rados.0.smithi189.stdout:316: done (8 left) 2024-09-18T10:15:41.218 INFO:tasks.rados.rados.0.smithi189.stdout:317: delete oid 38 current snap is 27 2024-09-18T10:15:41.218 INFO:tasks.rados.rados.0.smithi189.stdout:307: finishing write tid 3 to smithi18925896-29 2024-09-18T10:15:41.218 INFO:tasks.rados.rados.0.smithi189.stdout:307: finishing write tid 4 to smithi18925896-29 2024-09-18T10:15:41.221 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-18T10:15:41.221 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-18T10:15:41.221 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-18T10:15:41.221 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-18T10:15:41.221 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.221 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.3s1 37/32 ECSubWrite(tid=11, reqid=client.4632.0:189, at_version=37'8, trim_to=0'0, roll_forward_to=36'6))", 2024-09-18T10:15:41.221 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:53.096486+0000", 2024-09-18T10:15:41.221 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 48.120968732999998, 2024-09-18T10:15:41.221 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10315053, 2024-09-18T10:15:41.221 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:41.221 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:41.222 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:41.222 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.222 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:41.222 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.096486+0000", 2024-09-18T10:15:41.222 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:41.222 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.222 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.222 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:41.222 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.096486+0000", 2024-09-18T10:15:41.222 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:41.222 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.222 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.222 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:41.223 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.096486+0000", 2024-09-18T10:15:41.223 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:41.223 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.223 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.223 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:41.223 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.096802+0000", 2024-09-18T10:15:41.223 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00031615899999999999 2024-09-18T10:15:41.223 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.223 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.223 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:41.223 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.096841+0000", 2024-09-18T10:15:41.223 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9452000000000001e-05 2024-09-18T10:15:41.224 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.224 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.224 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:41.224 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.096866+0000", 2024-09-18T10:15:41.224 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5069e-05 2024-09-18T10:15:41.224 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.224 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.224 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:41.224 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.197104+0000", 2024-09-18T10:15:41.224 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100237886 2024-09-18T10:15:41.224 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.224 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.224 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:41.225 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.197130+0000", 2024-09-18T10:15:41.225 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5666000000000001e-05 2024-09-18T10:15:41.225 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.225 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.225 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:41.225 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.199585+0000", 2024-09-18T10:15:41.225 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002454777 2024-09-18T10:15:41.225 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.225 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.225 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:41.225 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.199637+0000", 2024-09-18T10:15:41.225 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2221e-05 2024-09-18T10:15:41.226 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:41.226 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:41.226 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:41.226 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.226 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.226 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.ds4 41/32 ECSubWrite(tid=41, reqid=osd.13.0:41, at_version=41'6, trim_to=0'0, roll_forward_to=37'4))", 2024-09-18T10:15:41.226 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:57.138491+0000", 2024-09-18T10:15:41.226 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 44.078964159999998, 2024-09-18T10:15:41.226 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.200128009, 2024-09-18T10:15:41.226 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:41.226 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:41.226 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:41.226 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.226 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:41.227 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.138491+0000", 2024-09-18T10:15:41.227 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:41.227 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.227 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.227 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:41.227 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.138491+0000", 2024-09-18T10:15:41.227 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:41.227 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.227 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.227 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:41.227 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.138490+0000", 2024-09-18T10:15:41.227 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:15:41.227 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.228 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.228 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:41.228 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.138555+0000", 2024-09-18T10:15:41.228 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4571000000000002e-05 2024-09-18T10:15:41.228 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.228 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.228 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:41.228 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.138581+0000", 2024-09-18T10:15:41.228 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6069000000000001e-05 2024-09-18T10:15:41.228 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.228 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.228 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:41.228 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.138598+0000", 2024-09-18T10:15:41.229 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7059e-05 2024-09-18T10:15:41.229 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.229 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.229 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:41.229 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.336697+0000", 2024-09-18T10:15:41.229 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.198099517 2024-09-18T10:15:41.229 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.229 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.229 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:41.229 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.336733+0000", 2024-09-18T10:15:41.229 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5369e-05 2024-09-18T10:15:41.229 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.230 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.230 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:41.230 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.338552+0000", 2024-09-18T10:15:41.230 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001819773 2024-09-18T10:15:41.230 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.230 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.230 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:41.230 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.338619+0000", 2024-09-18T10:15:41.230 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6390999999999995e-05 2024-09-18T10:15:41.230 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:41.230 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:41.230 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:41.231 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.231 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.231 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.ds4 41/32 ECSubWrite(tid=43, reqid=osd.13.0:41, at_version=0'0, trim_to=0'0, roll_forward_to=41'6))", 2024-09-18T10:15:41.231 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:57.441404+0000", 2024-09-18T10:15:41.231 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 43.776050550999997, 2024-09-18T10:15:41.231 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30108198600000002, 2024-09-18T10:15:41.231 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:41.231 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:41.231 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:41.231 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.231 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:41.231 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.441404+0000", 2024-09-18T10:15:41.232 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:41.232 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.232 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.232 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:41.232 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.441404+0000", 2024-09-18T10:15:41.232 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:41.232 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.232 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.232 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:41.232 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.441403+0000", 2024-09-18T10:15:41.232 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:41.232 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.233 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.233 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:41.233 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.441435+0000", 2024-09-18T10:15:41.233 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2502000000000002e-05 2024-09-18T10:15:41.233 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.233 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.233 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:41.233 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.441473+0000", 2024-09-18T10:15:41.233 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8302e-05 2024-09-18T10:15:41.233 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.233 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.233 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:41.234 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.441524+0000", 2024-09-18T10:15:41.234 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0679e-05 2024-09-18T10:15:41.234 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.234 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.234 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:41.234 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.641739+0000", 2024-09-18T10:15:41.234 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20021524299999999 2024-09-18T10:15:41.234 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.235 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.235 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:41.235 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.641766+0000", 2024-09-18T10:15:41.235 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7081999999999999e-05 2024-09-18T10:15:41.235 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.235 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.235 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:41.235 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.742432+0000", 2024-09-18T10:15:41.235 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10066531300000001 2024-09-18T10:15:41.235 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.235 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.235 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:41.236 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.742486+0000", 2024-09-18T10:15:41.236 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4691999999999997e-05 2024-09-18T10:15:41.236 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:41.236 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:41.236 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:41.236 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.236 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.236 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.6s5 41/40 ECSubWrite(tid=31, reqid=client.4632.0:249, at_version=41'13, trim_to=0'0, roll_forward_to=0'0))", 2024-09-18T10:15:41.236 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:57.550603+0000", 2024-09-18T10:15:41.236 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 43.666852360999997, 2024-09-18T10:15:41.236 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.103590501, 2024-09-18T10:15:41.236 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:41.236 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:41.236 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:41.237 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.237 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:41.237 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.550603+0000", 2024-09-18T10:15:41.237 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:41.237 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.237 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.237 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:41.237 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.550603+0000", 2024-09-18T10:15:41.237 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:41.237 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.237 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.237 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:41.238 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.550602+0000", 2024-09-18T10:15:41.238 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:15:41.238 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.238 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.238 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:41.238 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.551120+0000", 2024-09-18T10:15:41.238 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000518867 2024-09-18T10:15:41.238 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.238 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.238 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:41.238 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.551141+0000", 2024-09-18T10:15:41.238 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0873000000000002e-05 2024-09-18T10:15:41.238 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.239 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.239 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:41.239 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.551150+0000", 2024-09-18T10:15:41.239 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.884e-06 2024-09-18T10:15:41.239 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.239 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.239 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:41.239 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.651352+0000", 2024-09-18T10:15:41.239 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100201533 2024-09-18T10:15:41.239 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.239 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.239 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:41.239 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.651384+0000", 2024-09-18T10:15:41.240 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2471999999999997e-05 2024-09-18T10:15:41.240 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.240 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.240 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:41.240 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.654157+0000", 2024-09-18T10:15:41.240 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0027727440000000002 2024-09-18T10:15:41.240 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.240 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.240 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:41.240 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.654193+0000", 2024-09-18T10:15:41.240 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6242000000000003e-05 2024-09-18T10:15:41.240 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:41.240 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:41.241 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:41.241 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.241 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.241 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.ds4 47/32 ECSubWrite(tid=55, reqid=client.4632.0:332, at_version=47'8, trim_to=0'0, roll_forward_to=41'6))", 2024-09-18T10:15:41.241 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:02.796736+0000", 2024-09-18T10:15:41.241 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 38.420718782000002, 2024-09-18T10:15:41.241 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.106482128, 2024-09-18T10:15:41.241 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:41.241 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:41.241 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:41.241 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.241 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:41.241 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.796736+0000", 2024-09-18T10:15:41.242 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:41.242 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.242 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.242 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:41.242 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.796736+0000", 2024-09-18T10:15:41.242 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:41.242 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.242 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.242 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:41.242 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.796735+0000", 2024-09-18T10:15:41.242 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:15:41.242 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.242 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.243 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:41.243 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.796958+0000", 2024-09-18T10:15:41.243 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00022249800000000001 2024-09-18T10:15:41.243 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.243 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.243 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:41.243 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.796983+0000", 2024-09-18T10:15:41.243 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5454000000000001e-05 2024-09-18T10:15:41.243 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.243 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.243 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:41.243 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.897088+0000", 2024-09-18T10:15:41.243 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10010422099999999 2024-09-18T10:15:41.243 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.244 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.244 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:41.244 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.900995+0000", 2024-09-18T10:15:41.244 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0039075589999999997 2024-09-18T10:15:41.244 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.244 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.244 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:41.244 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.901018+0000", 2024-09-18T10:15:41.244 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2592999999999999e-05 2024-09-18T10:15:41.244 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.244 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.245 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:41.245 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.903153+0000", 2024-09-18T10:15:41.245 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0021355300000000001 2024-09-18T10:15:41.245 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.245 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.245 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:41.245 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.903218+0000", 2024-09-18T10:15:41.245 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5135000000000006e-05 2024-09-18T10:15:41.245 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:41.245 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:41.245 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:41.246 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.246 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.246 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:509 3.16s0 3:6c64c4fa:::smithi18925896-41:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e62)", 2024-09-18T10:15:41.246 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:17.914565+0000", 2024-09-18T10:15:41.246 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 23.302889944, 2024-09-18T10:15:41.246 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.059666403, 2024-09-18T10:15:41.246 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:41.246 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-18T10:15:41.246 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:41.246 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:41.246 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:41.246 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 509 2024-09-18T10:15:41.246 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.247 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:41.247 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.247 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:41.247 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.914565+0000", 2024-09-18T10:15:41.247 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:41.247 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.247 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.247 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:41.247 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.914565+0000", 2024-09-18T10:15:41.247 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:41.247 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.247 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.247 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:41.248 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.914564+0000", 2024-09-18T10:15:41.248 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:15:41.248 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.248 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.248 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:41.248 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.914570+0000", 2024-09-18T10:15:41.248 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3940000000000001e-06 2024-09-18T10:15:41.248 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.248 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.248 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:41.248 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.914585+0000", 2024-09-18T10:15:41.248 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4348000000000001e-05 2024-09-18T10:15:41.248 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.248 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.249 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:41.249 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.914591+0000", 2024-09-18T10:15:41.249 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4450000000000001e-06 2024-09-18T10:15:41.249 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.249 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.249 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:41.249 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.101342+0000", 2024-09-18T10:15:41.249 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.186750576 2024-09-18T10:15:41.249 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.249 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.249 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-18T10:15:41.249 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.101343+0000", 2024-09-18T10:15:41.249 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4300000000000001e-07 2024-09-18T10:15:41.250 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.250 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.250 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:41.250 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.334525+0000", 2024-09-18T10:15:41.250 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2331822159999999 2024-09-18T10:15:41.250 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.250 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.250 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for missing object", 2024-09-18T10:15:41.250 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.334706+0000", 2024-09-18T10:15:41.250 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00018068500000000001 2024-09-18T10:15:41.250 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.250 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.250 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:41.251 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.838318+0000", 2024-09-18T10:15:41.251 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.50361270300000005 2024-09-18T10:15:41.251 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.251 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.251 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2024-09-18T10:15:41.251 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.838493+0000", 2024-09-18T10:15:41.251 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017465199999999999 2024-09-18T10:15:41.251 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.251 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.251 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:41.251 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.972913+0000", 2024-09-18T10:15:41.251 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.134419977 2024-09-18T10:15:41.251 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.251 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.251 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:41.252 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.973261+0000", 2024-09-18T10:15:41.252 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00034786399999999997 2024-09-18T10:15:41.252 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.252 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.252 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:41.252 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.974231+0000", 2024-09-18T10:15:41.252 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00097053099999999995 2024-09-18T10:15:41.252 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:41.252 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:41.252 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:41.252 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.253 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.253 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.0s2 63/40 ECSubWrite(tid=25, reqid=client.4632.0:511, at_version=63'12, trim_to=0'0, roll_forward_to=60'10))", 2024-09-18T10:15:41.253 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:18.905739+0000", 2024-09-18T10:15:41.253 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 22.311715666000001, 2024-09-18T10:15:41.253 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.19488354899999999, 2024-09-18T10:15:41.253 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:41.253 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:41.253 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:41.253 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.253 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:41.253 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.905739+0000", 2024-09-18T10:15:41.253 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:41.254 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.254 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.254 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:41.254 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.905739+0000", 2024-09-18T10:15:41.254 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:41.254 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.254 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.254 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:41.254 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.905739+0000", 2024-09-18T10:15:41.254 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:41.254 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.254 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.255 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:41.255 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.905789+0000", 2024-09-18T10:15:41.255 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0754999999999998e-05 2024-09-18T10:15:41.255 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.255 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.255 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:41.255 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.905833+0000", 2024-09-18T10:15:41.255 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3779000000000002e-05 2024-09-18T10:15:41.255 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.255 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.255 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:41.255 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.905858+0000", 2024-09-18T10:15:41.255 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5208999999999999e-05 2024-09-18T10:15:41.256 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.256 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.256 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:41.256 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.006073+0000", 2024-09-18T10:15:41.256 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100214601 2024-09-18T10:15:41.256 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.256 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.256 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:41.256 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.006091+0000", 2024-09-18T10:15:41.256 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8301999999999999e-05 2024-09-18T10:15:41.256 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.256 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.257 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:41.257 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.100567+0000", 2024-09-18T10:15:41.257 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.094475297999999999 2024-09-18T10:15:41.257 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.257 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.257 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:41.257 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.100623+0000", 2024-09-18T10:15:41.257 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6237999999999999e-05 2024-09-18T10:15:41.258 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:41.258 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:41.258 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:41.258 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.258 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.258 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDScrubReserve(3.6s5 REQUEST e63)", 2024-09-18T10:15:41.258 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:19.068302+0000", 2024-09-18T10:15:41.258 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 22.149153416000001, 2024-09-18T10:15:41.258 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.26520614300000001, 2024-09-18T10:15:41.259 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:41.259 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-18T10:15:41.259 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:41.259 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.259 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:41.259 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.068302+0000", 2024-09-18T10:15:41.259 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:41.259 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.259 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.259 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:41.259 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.068302+0000", 2024-09-18T10:15:41.259 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:41.259 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.260 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.260 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:41.260 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.068300+0000", 2024-09-18T10:15:41.260 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:41.260 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.260 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.260 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:41.260 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.068311+0000", 2024-09-18T10:15:41.260 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1005999999999999e-05 2024-09-18T10:15:41.260 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.260 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.260 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:41.261 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.068345+0000", 2024-09-18T10:15:41.261 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3983000000000003e-05 2024-09-18T10:15:41.261 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.261 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.261 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:41.261 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.068372+0000", 2024-09-18T10:15:41.261 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7385999999999998e-05 2024-09-18T10:15:41.261 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.262 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.262 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:41.262 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.333409+0000", 2024-09-18T10:15:41.262 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.26503692400000001 2024-09-18T10:15:41.262 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.262 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.262 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:41.262 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.333444+0000", 2024-09-18T10:15:41.262 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4916000000000002e-05 2024-09-18T10:15:41.262 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.263 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.263 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:41.263 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.333508+0000", 2024-09-18T10:15:41.263 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3658000000000006e-05 2024-09-18T10:15:41.263 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:41.263 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:41.263 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:41.263 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.263 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.263 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpReadReply(3.16s0 63/62 ECSubReadReply(tid=0, attrs_read=0))", 2024-09-18T10:15:41.263 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:19.230883+0000", 2024-09-18T10:15:41.264 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 21.986572364000001, 2024-09-18T10:15:41.264 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.103113091, 2024-09-18T10:15:41.264 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:41.264 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:41.264 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:41.264 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.264 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:41.264 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.230883+0000", 2024-09-18T10:15:41.264 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:41.264 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.264 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.264 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:41.265 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.230883+0000", 2024-09-18T10:15:41.265 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:41.265 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.265 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.265 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:41.265 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.230882+0000", 2024-09-18T10:15:41.265 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:15:41.265 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.265 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.265 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:41.265 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.230892+0000", 2024-09-18T10:15:41.265 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0193999999999999e-05 2024-09-18T10:15:41.265 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.266 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.266 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:41.266 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.230908+0000", 2024-09-18T10:15:41.266 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6368e-05 2024-09-18T10:15:41.266 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.266 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.266 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:41.266 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.230918+0000", 2024-09-18T10:15:41.266 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0125e-05 2024-09-18T10:15:41.266 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.266 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.266 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:41.266 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.333866+0000", 2024-09-18T10:15:41.267 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.102947178 2024-09-18T10:15:41.267 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.267 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.267 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:41.267 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.333996+0000", 2024-09-18T10:15:41.267 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013009 2024-09-18T10:15:41.267 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:41.267 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:41.267 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:41.267 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.267 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.267 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDPGPushReply(3.16s0 63/62 [PushReplyOp(3:6c64c4fa:::smithi18925896-41:head)])", 2024-09-18T10:15:41.267 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:19.524127+0000", 2024-09-18T10:15:41.267 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 21.693327980999999, 2024-09-18T10:15:41.268 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20108187499999999, 2024-09-18T10:15:41.268 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:41.268 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:41.268 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:41.268 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.268 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:41.268 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.524127+0000", 2024-09-18T10:15:41.268 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:41.268 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.268 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.268 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:41.268 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.524127+0000", 2024-09-18T10:15:41.268 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:41.269 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.269 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.269 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:41.269 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.524117+0000", 2024-09-18T10:15:41.269 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999905 2024-09-18T10:15:41.269 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.269 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.269 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:41.269 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.524143+0000", 2024-09-18T10:15:41.269 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5534999999999999e-05 2024-09-18T10:15:41.269 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.269 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.269 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:41.270 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.524209+0000", 2024-09-18T10:15:41.270 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6192000000000006e-05 2024-09-18T10:15:41.270 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.270 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.270 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:41.270 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.524227+0000", 2024-09-18T10:15:41.270 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7986e-05 2024-09-18T10:15:41.270 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.270 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.270 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:41.270 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.724925+0000", 2024-09-18T10:15:41.270 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.200697662 2024-09-18T10:15:41.271 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.271 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.271 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:41.271 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.725209+0000", 2024-09-18T10:15:41.271 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000284102 2024-09-18T10:15:41.271 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:41.271 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:41.271 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:41.271 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.271 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.271 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDPGPushReply(3.16s0 63/62 [PushReplyOp(3:6c64c4fa:::smithi18925896-41:head)])", 2024-09-18T10:15:41.271 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:19.524280+0000", 2024-09-18T10:15:41.271 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 21.693174637999999, 2024-09-18T10:15:41.272 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.201340348, 2024-09-18T10:15:41.272 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:41.272 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:41.272 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:41.272 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.272 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:41.272 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.524280+0000", 2024-09-18T10:15:41.272 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:41.272 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.272 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.272 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:41.272 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.524280+0000", 2024-09-18T10:15:41.272 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:41.273 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.273 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.273 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:41.273 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.524274+0000", 2024-09-18T10:15:41.273 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999933 2024-09-18T10:15:41.273 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.273 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.273 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:41.273 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.524289+0000", 2024-09-18T10:15:41.273 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5722999999999998e-05 2024-09-18T10:15:41.273 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.273 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.273 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:41.274 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.524315+0000", 2024-09-18T10:15:41.274 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5188999999999999e-05 2024-09-18T10:15:41.274 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.274 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.274 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:41.274 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.524325+0000", 2024-09-18T10:15:41.274 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0736e-05 2024-09-18T10:15:41.274 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.274 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.274 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:41.274 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.725489+0000", 2024-09-18T10:15:41.274 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.201163377 2024-09-18T10:15:41.274 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.275 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.275 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:41.275 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.725621+0000", 2024-09-18T10:15:41.275 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013202600000000001 2024-09-18T10:15:41.275 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:41.275 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:41.275 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:41.275 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.275 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.275 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDPGPushReply(3.16s0 63/62 [PushReplyOp(3:6c64c4fa:::smithi18925896-41:head)])", 2024-09-18T10:15:41.275 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:19.524382+0000", 2024-09-18T10:15:41.275 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 21.693072984000001, 2024-09-18T10:15:41.275 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.21126635699999999, 2024-09-18T10:15:41.276 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:41.276 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:41.276 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:41.276 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.276 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:41.276 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.524382+0000", 2024-09-18T10:15:41.276 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:41.276 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.276 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.276 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:41.276 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.524382+0000", 2024-09-18T10:15:41.276 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:41.276 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.277 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.277 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:41.277 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.524381+0000", 2024-09-18T10:15:41.277 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:41.277 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.277 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.277 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:41.277 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.524388+0000", 2024-09-18T10:15:41.277 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4860000000000002e-06 2024-09-18T10:15:41.277 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.277 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.277 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:41.277 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.524420+0000", 2024-09-18T10:15:41.278 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2623000000000003e-05 2024-09-18T10:15:41.278 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.278 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.278 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:41.278 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.624550+0000", 2024-09-18T10:15:41.278 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100129128 2024-09-18T10:15:41.278 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.278 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.278 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:41.278 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.726236+0000", 2024-09-18T10:15:41.278 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10168595699999999 2024-09-18T10:15:41.278 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.278 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.278 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:41.279 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.735648+0000", 2024-09-18T10:15:41.279 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0094127829999999992 2024-09-18T10:15:41.279 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:41.279 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:41.279 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:41.279 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.279 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.279 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.ds4 63/32 ECSubWrite(tid=97, reqid=client.4632.0:526, at_version=63'18, trim_to=0'0, roll_forward_to=60'14))", 2024-09-18T10:15:41.279 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:19.563004+0000", 2024-09-18T10:15:41.279 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 21.654451447, 2024-09-18T10:15:41.279 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.103788874, 2024-09-18T10:15:41.279 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:41.279 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:41.280 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:41.280 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.280 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:41.280 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.563004+0000", 2024-09-18T10:15:41.280 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:41.280 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.280 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.280 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:41.280 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.563004+0000", 2024-09-18T10:15:41.280 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:41.280 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.280 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.280 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:41.281 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.563002+0000", 2024-09-18T10:15:41.281 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:41.281 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.281 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.281 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:41.281 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.563242+0000", 2024-09-18T10:15:41.281 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000240363 2024-09-18T10:15:41.281 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.281 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.281 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:41.281 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.563275+0000", 2024-09-18T10:15:41.281 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2505000000000001e-05 2024-09-18T10:15:41.281 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.281 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.282 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:41.282 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.563293+0000", 2024-09-18T10:15:41.282 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8266000000000001e-05 2024-09-18T10:15:41.282 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.282 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.282 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:41.282 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.663489+0000", 2024-09-18T10:15:41.282 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10019536900000001 2024-09-18T10:15:41.282 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.282 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.282 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:41.282 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.663515+0000", 2024-09-18T10:15:41.282 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6145000000000001e-05 2024-09-18T10:15:41.282 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.283 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.283 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:41.283 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.666737+0000", 2024-09-18T10:15:41.283 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0032219280000000002 2024-09-18T10:15:41.283 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.283 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.283 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:41.283 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.666792+0000", 2024-09-18T10:15:41.283 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5764999999999999e-05 2024-09-18T10:15:41.283 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:41.283 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:41.283 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:41.284 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.284 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.284 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.10s6 73/46 ECSubWrite(tid=194, reqid=client.4632.0:626, at_version=73'16, trim_to=0'0, roll_forward_to=60'14))", 2024-09-18T10:15:41.284 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:29.000022+0000", 2024-09-18T10:15:41.284 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 12.21743341, 2024-09-18T10:15:41.284 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20370728099999999, 2024-09-18T10:15:41.284 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:41.284 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:41.284 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:41.285 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.285 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:41.285 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.000022+0000", 2024-09-18T10:15:41.285 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:41.285 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.285 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.285 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:41.285 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.000022+0000", 2024-09-18T10:15:41.285 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:41.285 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.285 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.285 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:41.285 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.000020+0000", 2024-09-18T10:15:41.286 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:41.286 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.286 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.286 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:41.286 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.000428+0000", 2024-09-18T10:15:41.286 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000408689 2024-09-18T10:15:41.286 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.286 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.286 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:41.286 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.000464+0000", 2024-09-18T10:15:41.286 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5768e-05 2024-09-18T10:15:41.286 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.286 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.286 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:41.287 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.100621+0000", 2024-09-18T10:15:41.287 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100157223 2024-09-18T10:15:41.287 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.287 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.287 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:41.287 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.200904+0000", 2024-09-18T10:15:41.287 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10028234900000001 2024-09-18T10:15:41.287 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.287 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.287 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:41.287 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.200921+0000", 2024-09-18T10:15:41.287 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7217000000000001e-05 2024-09-18T10:15:41.287 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.288 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.288 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:41.288 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.203672+0000", 2024-09-18T10:15:41.288 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0027511879999999999 2024-09-18T10:15:41.288 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.288 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.288 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:41.288 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.203729+0000", 2024-09-18T10:15:41.288 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6823999999999997e-05 2024-09-18T10:15:41.288 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:41.288 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:41.288 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:41.288 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.289 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.289 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.10s6 73/46 ECSubWrite(tid=195, reqid=client.4632.0:627, at_version=73'17, trim_to=0'0, roll_forward_to=60'14))", 2024-09-18T10:15:41.289 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:29.103724+0000", 2024-09-18T10:15:41.289 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 12.113731327, 2024-09-18T10:15:41.289 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.202940394, 2024-09-18T10:15:41.289 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:41.289 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:41.289 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:41.289 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.289 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:41.289 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.103724+0000", 2024-09-18T10:15:41.289 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:41.289 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.289 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.290 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:41.290 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.103724+0000", 2024-09-18T10:15:41.290 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:41.290 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.290 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.290 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:41.290 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.103722+0000", 2024-09-18T10:15:41.290 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:41.290 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.290 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.290 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:41.290 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.104024+0000", 2024-09-18T10:15:41.290 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00030194100000000001 2024-09-18T10:15:41.290 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.291 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.291 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:41.291 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.104067+0000", 2024-09-18T10:15:41.291 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2840999999999999e-05 2024-09-18T10:15:41.291 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.291 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.291 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:41.291 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.204203+0000", 2024-09-18T10:15:41.291 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100136243 2024-09-18T10:15:41.291 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.291 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.291 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:41.291 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.304498+0000", 2024-09-18T10:15:41.292 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100294809 2024-09-18T10:15:41.292 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.292 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.292 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:41.292 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.304514+0000", 2024-09-18T10:15:41.292 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6163000000000001e-05 2024-09-18T10:15:41.292 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.292 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.292 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:41.292 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.306599+0000", 2024-09-18T10:15:41.292 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0020848759999999998 2024-09-18T10:15:41.292 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.292 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.292 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:41.293 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.306664+0000", 2024-09-18T10:15:41.293 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5331000000000002e-05 2024-09-18T10:15:41.293 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:41.293 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:41.293 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:41.293 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.293 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.293 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.6s5 74/40 ECSubWrite(tid=64, reqid=osd.0.0:64, at_version=74'40, trim_to=0'0, roll_forward_to=72'38))", 2024-09-18T10:15:41.293 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:29.670320+0000", 2024-09-18T10:15:41.293 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 11.547134558, 2024-09-18T10:15:41.293 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.199229986, 2024-09-18T10:15:41.294 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:41.294 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:41.294 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:41.294 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.294 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:41.294 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.670320+0000", 2024-09-18T10:15:41.294 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:41.294 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.294 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.294 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:41.294 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.670320+0000", 2024-09-18T10:15:41.294 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:41.294 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.294 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.295 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:41.295 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.670319+0000", 2024-09-18T10:15:41.295 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:41.295 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.295 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.295 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:41.295 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.670369+0000", 2024-09-18T10:15:41.295 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0130000000000003e-05 2024-09-18T10:15:41.295 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.295 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.295 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:41.295 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.670398+0000", 2024-09-18T10:15:41.295 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9122e-05 2024-09-18T10:15:41.295 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.296 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.296 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:41.296 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.670415+0000", 2024-09-18T10:15:41.296 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6351e-05 2024-09-18T10:15:41.296 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.296 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.296 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:41.296 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.769393+0000", 2024-09-18T10:15:41.296 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.098978678 2024-09-18T10:15:41.296 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.296 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.296 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:41.296 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.769420+0000", 2024-09-18T10:15:41.297 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6704e-05 2024-09-18T10:15:41.297 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.297 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.297 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:41.297 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.869470+0000", 2024-09-18T10:15:41.297 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100049836 2024-09-18T10:15:41.297 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.297 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.297 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:41.297 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.869550+0000", 2024-09-18T10:15:41.297 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0487000000000004e-05 2024-09-18T10:15:41.297 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:41.297 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:41.297 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:41.298 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.298 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.298 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWriteReply(3.16s0 74/62 ECSubWriteReply(tid=8, last_complete=74'22, committed=1, applied=1))", 2024-09-18T10:15:41.298 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:29.670821+0000", 2024-09-18T10:15:41.298 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 11.546634214999999, 2024-09-18T10:15:41.298 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.198638966, 2024-09-18T10:15:41.298 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:41.298 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:41.298 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:41.298 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.298 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:41.298 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.670821+0000", 2024-09-18T10:15:41.298 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:41.298 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.299 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.299 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:41.299 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.670821+0000", 2024-09-18T10:15:41.299 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:41.299 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.299 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.299 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:41.299 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.670820+0000", 2024-09-18T10:15:41.299 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:15:41.299 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.299 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.299 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:41.299 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.670826+0000", 2024-09-18T10:15:41.300 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7529999999999998e-06 2024-09-18T10:15:41.300 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.300 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.300 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:41.300 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.670840+0000", 2024-09-18T10:15:41.300 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4484000000000001e-05 2024-09-18T10:15:41.300 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.300 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.300 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:41.300 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.670847+0000", 2024-09-18T10:15:41.300 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7880000000000001e-06 2024-09-18T10:15:41.300 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.300 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.300 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:41.301 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.869415+0000", 2024-09-18T10:15:41.301 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.198568033 2024-09-18T10:15:41.301 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.301 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.301 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:41.301 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.869460+0000", 2024-09-18T10:15:41.301 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4765000000000003e-05 2024-09-18T10:15:41.301 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:41.301 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:41.301 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:41.301 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.301 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.301 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.10s6 75/46 ECSubWrite(tid=207, reqid=osd.6.0:207, at_version=75'23, trim_to=0'0, roll_forward_to=74'21))", 2024-09-18T10:15:41.302 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:30.569775+0000", 2024-09-18T10:15:41.302 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.647680062999999, 2024-09-18T10:15:41.302 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.193288983, 2024-09-18T10:15:41.302 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:41.302 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:41.302 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:41.302 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.302 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:41.302 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:30.569775+0000", 2024-09-18T10:15:41.302 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:41.302 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.302 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.302 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:41.303 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:30.569775+0000", 2024-09-18T10:15:41.303 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:41.303 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.303 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.303 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:41.303 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:30.569775+0000", 2024-09-18T10:15:41.303 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:41.303 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.303 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.303 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:41.303 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:30.569810+0000", 2024-09-18T10:15:41.303 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5540000000000002e-05 2024-09-18T10:15:41.303 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.304 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.304 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:41.304 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:30.569826+0000", 2024-09-18T10:15:41.304 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6276000000000001e-05 2024-09-18T10:15:41.304 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.304 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.304 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:41.304 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:30.569836+0000", 2024-09-18T10:15:41.304 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0158e-05 2024-09-18T10:15:41.304 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.304 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.304 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:41.305 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:30.669996+0000", 2024-09-18T10:15:41.305 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10015942899999999 2024-09-18T10:15:41.305 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.305 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.305 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:41.305 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:30.670007+0000", 2024-09-18T10:15:41.305 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0903e-05 2024-09-18T10:15:41.305 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.305 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.305 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:41.305 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:30.763014+0000", 2024-09-18T10:15:41.305 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.093006722 2024-09-18T10:15:41.305 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.305 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.306 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:41.306 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:30.763064+0000", 2024-09-18T10:15:41.306 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0348e-05 2024-09-18T10:15:41.306 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:41.306 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:41.306 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:41.306 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.306 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.306 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.6s5 75/40 ECSubWrite(tid=66, reqid=osd.0.0:66, at_version=75'42, trim_to=0'0, roll_forward_to=74'40))", 2024-09-18T10:15:41.306 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:30.668776+0000", 2024-09-18T10:15:41.306 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.548678666000001, 2024-09-18T10:15:41.306 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30218223300000002, 2024-09-18T10:15:41.306 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:41.307 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:41.307 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:41.307 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.307 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:41.307 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:30.668776+0000", 2024-09-18T10:15:41.307 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:41.307 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.307 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.307 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:41.307 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:30.668776+0000", 2024-09-18T10:15:41.307 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:41.307 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.307 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.307 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:41.308 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:30.668775+0000", 2024-09-18T10:15:41.308 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:41.308 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.308 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.308 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:41.308 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:30.668821+0000", 2024-09-18T10:15:41.308 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6147999999999999e-05 2024-09-18T10:15:41.308 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.308 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.308 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:41.308 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:30.668864+0000", 2024-09-18T10:15:41.308 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3004999999999999e-05 2024-09-18T10:15:41.308 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.309 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.309 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:41.309 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:30.768961+0000", 2024-09-18T10:15:41.309 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10009678499999999 2024-09-18T10:15:41.309 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.309 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.309 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:41.309 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:30.969322+0000", 2024-09-18T10:15:41.309 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20036179000000001 2024-09-18T10:15:41.310 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.310 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.310 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:41.310 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:30.969349+0000", 2024-09-18T10:15:41.310 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6795999999999999e-05 2024-09-18T10:15:41.310 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.310 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.310 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:41.310 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:30.970891+0000", 2024-09-18T10:15:41.310 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0015418459999999999 2024-09-18T10:15:41.310 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.310 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.310 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:41.310 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:30.970959+0000", 2024-09-18T10:15:41.311 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7490000000000006e-05 2024-09-18T10:15:41.311 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:41.311 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:41.311 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:41.311 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.311 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.311 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpReadReply(3.16s0 85/82 ECSubReadReply(tid=14, attrs_read=0))", 2024-09-18T10:15:41.311 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:40.765788+0000", 2024-09-18T10:15:41.311 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.45166682600000002, 2024-09-18T10:15:41.311 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.19785048, 2024-09-18T10:15:41.311 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:41.311 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:15:41.311 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:41.312 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.312 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:41.312 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.765788+0000", 2024-09-18T10:15:41.312 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:41.312 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.312 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.312 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:41.312 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.765788+0000", 2024-09-18T10:15:41.312 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:41.312 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.312 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.312 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:41.312 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.765786+0000", 2024-09-18T10:15:41.312 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:41.313 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.313 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.313 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:41.313 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.766201+0000", 2024-09-18T10:15:41.313 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00041439500000000002 2024-09-18T10:15:41.313 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.313 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.313 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:41.313 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.766231+0000", 2024-09-18T10:15:41.313 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0059000000000001e-05 2024-09-18T10:15:41.313 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.313 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.313 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:41.313 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.866344+0000", 2024-09-18T10:15:41.314 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100112995 2024-09-18T10:15:41.314 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.314 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.314 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:41.314 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.963540+0000", 2024-09-18T10:15:41.314 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.097195631000000005 2024-09-18T10:15:41.314 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.314 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.314 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:41.314 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.963639+0000", 2024-09-18T10:15:41.314 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9086999999999995e-05 2024-09-18T10:15:41.314 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:41.314 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:41.314 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:41.315 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:41.315 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:41.315 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-18T10:15:41.316 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_ops_in_flight 2024-09-18T10:15:41.317 INFO:tasks.rados.rados.0.smithi189.stdout:307: finishing write tid 5 to smithi18925896-29 2024-09-18T10:15:41.317 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-18T10:15:41.310+0000 7f14dcc5b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22398) UID: 0 2024-09-18T10:15:41.353 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T10:15:41.419 INFO:tasks.ceph.osd.15.smithi189.stderr:2024-09-18T10:15:41.414+0000 7fd0e725e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 22401) UID: 0 2024-09-18T10:15:41.431 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-18T10:15:41.432 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-18T10:15:41.432 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.432 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:779 3.as0 3:5dc9cbed:::smithi18925896-13:head [copy-from ver 47 in=55b] snapc 2f=[2f,29,28,20,1f,1e,1a,11] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-18T10:15:41.432 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:40.727404+0000", 2024-09-18T10:15:41.432 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.694064232, 2024-09-18T10:15:41.432 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.69411217300000005, 2024-09-18T10:15:41.432 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:41.432 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-18T10:15:41.432 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:41.432 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:41.432 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:41.432 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 779 2024-09-18T10:15:41.433 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.433 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:41.433 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.433 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:41.433 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.727404+0000", 2024-09-18T10:15:41.433 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:41.433 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.433 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.433 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:41.433 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.727404+0000", 2024-09-18T10:15:41.433 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:41.433 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.433 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.433 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:41.434 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.727403+0000", 2024-09-18T10:15:41.434 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:41.434 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.434 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.434 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:41.434 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.727413+0000", 2024-09-18T10:15:41.434 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6940000000000007e-06 2024-09-18T10:15:41.434 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.434 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.434 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:41.434 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.727426+0000", 2024-09-18T10:15:41.435 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2896e-05 2024-09-18T10:15:41.435 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.435 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.435 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:41.435 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.727440+0000", 2024-09-18T10:15:41.435 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4285e-05 2024-09-18T10:15:41.435 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.435 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.435 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:41.436 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.727569+0000", 2024-09-18T10:15:41.436 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000128906 2024-09-18T10:15:41.436 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.436 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.436 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:41.436 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.727580+0000", 2024-09-18T10:15:41.436 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0720000000000001e-05 2024-09-18T10:15:41.436 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.436 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.436 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:41.436 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.936548+0000", 2024-09-18T10:15:41.436 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20896883299999999 2024-09-18T10:15:41.436 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.437 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.437 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:41.437 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.936872+0000", 2024-09-18T10:15:41.437 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00032318499999999998 2024-09-18T10:15:41.437 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.437 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.437 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:41.437 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:41.066546+0000", 2024-09-18T10:15:41.437 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.12967457600000001 2024-09-18T10:15:41.437 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.437 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.437 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:41.437 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:41.068954+0000", 2024-09-18T10:15:41.438 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002408154 2024-09-18T10:15:41.438 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:41.438 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:41.438 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:41.438 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.438 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.438 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWriteReply(3.as0 85/32 ECSubWriteReply(tid=169, last_complete=85'48, committed=1, applied=1))", 2024-09-18T10:15:41.438 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:41.358319+0000", 2024-09-18T10:15:41.438 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.063149801000000005, 2024-09-18T10:15:41.438 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.063458645999999994, 2024-09-18T10:15:41.438 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:41.438 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-09-18T10:15:41.438 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:41.438 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.439 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:41.439 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:41.358319+0000", 2024-09-18T10:15:41.439 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:41.439 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.439 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.439 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:41.439 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:41.358319+0000", 2024-09-18T10:15:41.439 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:41.439 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.439 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.439 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:41.439 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:41.358317+0000", 2024-09-18T10:15:41.439 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:41.440 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.440 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.440 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:41.440 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:41.358329+0000", 2024-09-18T10:15:41.440 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1402e-05 2024-09-18T10:15:41.440 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.440 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.440 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:41.440 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:41.358348+0000", 2024-09-18T10:15:41.440 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9074000000000001e-05 2024-09-18T10:15:41.440 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.440 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.440 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:41.440 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:41.358358+0000", 2024-09-18T10:15:41.441 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0662999999999999e-05 2024-09-18T10:15:41.441 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:41.441 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:41.441 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:41.441 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:41.441 INFO:teuthology.orchestra.run.smithi136.stdout: ], 2024-09-18T10:15:41.441 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 2 2024-09-18T10:15:41.441 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-18T10:15:41.442 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_blocked_ops 2024-09-18T10:15:41.459 INFO:tasks.rados.rados.0.smithi189.stdout:312: finishing copy_from to smithi18925896-13 2024-09-18T10:15:41.459 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 13 v 47 (ObjNum 33 snap 0 seq_num 33) dirty exists 2024-09-18T10:15:41.502 INFO:tasks.rados.rados.0.smithi189.stdout:295: expect (ObjNum 76 snap 8 seq_num 76) 2024-09-18T10:15:41.509 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-18T10:15:41.519 INFO:tasks.ceph.osd.4.smithi029.stderr:2024-09-18T10:15:41.513+0000 7f550cf6e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22189) UID: 0 2024-09-18T10:15:41.539 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-18T10:15:41.539 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-18T10:15:41.539 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-18T10:15:41.539 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-18T10:15:41.539 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-18T10:15:41.548 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_historic_ops 2024-09-18T10:15:41.578 INFO:tasks.rados.rados.0.smithi189.stdout:307: finishing write tid 6 to smithi18925896-29 2024-09-18T10:15:41.578 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 29 v 65 (ObjNum 123 snap 27 seq_num 123) dirty exists 2024-09-18T10:15:41.578 INFO:tasks.rados.rados.0.smithi189.stdout:307: left oid 29 (ObjNum 123 snap 27 seq_num 123) 2024-09-18T10:15:41.587 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-18T10:15:41.611 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-18T10:15:41.621 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-18T10:15:41.616+0000 7f14c6a1a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 22138) UID: 0 2024-09-18T10:15:41.626 INFO:tasks.rados.rados.0.smithi189.stdout:305: finishing write tid 7 to smithi18925896-23 2024-09-18T10:15:41.626 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 23 v 116 (ObjNum 122 snap 26 seq_num 122) dirty exists 2024-09-18T10:15:41.626 INFO:tasks.rados.rados.0.smithi189.stdout:305: left oid 23 (ObjNum 122 snap 26 seq_num 122) 2024-09-18T10:15:41.635 INFO:tasks.rados.rados.0.smithi189.stdout:300: finishing rollback tid 1 to smithi18925896-45 2024-09-18T10:15:41.639 INFO:tasks.rados.rados.0.smithi189.stdout:315: finishing write tid 1 to smithi18925896-41 2024-09-18T10:15:41.640 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-18T10:15:41.651 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-18T10:15:41.651 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-18T10:15:41.651 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-18T10:15:41.651 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-18T10:15:41.651 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.651 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:176 3.ds0 3:b1a14543:::smithi18925896-15:head [copy-from ver 4] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:41.652 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:52.680518+0000", 2024-09-18T10:15:41.652 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 48.968520431999998, 2024-09-18T10:15:41.652 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.42607371300000002, 2024-09-18T10:15:41.652 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:41.652 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:41.652 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:41.652 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:41.652 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:41.652 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 176 2024-09-18T10:15:41.652 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.652 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:41.652 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.652 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:41.653 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.680518+0000", 2024-09-18T10:15:41.653 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:41.653 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.653 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.653 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:41.653 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.680518+0000", 2024-09-18T10:15:41.653 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:41.653 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.653 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.653 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:41.653 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.680515+0000", 2024-09-18T10:15:41.653 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999971 2024-09-18T10:15:41.653 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.654 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.654 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:41.654 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.680533+0000", 2024-09-18T10:15:41.654 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8756000000000001e-05 2024-09-18T10:15:41.654 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.654 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.654 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:41.654 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.680573+0000", 2024-09-18T10:15:41.654 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9431999999999998e-05 2024-09-18T10:15:41.654 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.654 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.654 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:41.654 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.680587+0000", 2024-09-18T10:15:41.654 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.447e-05 2024-09-18T10:15:41.655 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.655 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.655 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:41.655 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.680755+0000", 2024-09-18T10:15:41.655 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000167584 2024-09-18T10:15:41.655 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.655 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.655 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-09-18T10:15:41.655 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.680784+0000", 2024-09-18T10:15:41.655 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9439000000000001e-05 2024-09-18T10:15:41.655 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.655 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.655 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:41.656 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.685398+0000", 2024-09-18T10:15:41.656 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0046140060000000004 2024-09-18T10:15:41.656 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.656 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.656 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:41.656 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.685502+0000", 2024-09-18T10:15:41.656 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000103702 2024-09-18T10:15:41.656 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.656 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.656 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:41.656 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.003083+0000", 2024-09-18T10:15:41.656 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.31758088600000001 2024-09-18T10:15:41.657 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.657 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.657 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:41.657 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.006087+0000", 2024-09-18T10:15:41.657 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0030044820000000002 2024-09-18T10:15:41.657 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.657 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.657 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:15:41.657 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.106397+0000", 2024-09-18T10:15:41.657 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100309469 2024-09-18T10:15:41.657 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.657 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.657 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:41.657 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.106591+0000", 2024-09-18T10:15:41.658 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000194513 2024-09-18T10:15:41.658 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:41.658 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:41.658 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:41.658 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.658 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.658 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:201 3.as0 3:5dc9cbed:::smithi18925896-13:2 [read 0~212992,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:41.658 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:52.694813+0000", 2024-09-18T10:15:41.658 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 48.954225477000001, 2024-09-18T10:15:41.658 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.50652904899999995, 2024-09-18T10:15:41.658 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:41.658 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-18T10:15:41.658 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:41.659 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:41.659 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:41.659 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 201 2024-09-18T10:15:41.659 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.659 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:41.659 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.659 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:41.659 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694813+0000", 2024-09-18T10:15:41.659 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:41.659 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.659 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.659 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:41.659 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694813+0000", 2024-09-18T10:15:41.659 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:41.660 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.660 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.660 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:41.660 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694812+0000", 2024-09-18T10:15:41.660 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:41.660 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.660 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.660 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:41.660 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694816+0000", 2024-09-18T10:15:41.660 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7720000000000001e-06 2024-09-18T10:15:41.660 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.660 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.661 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:41.661 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694822+0000", 2024-09-18T10:15:41.661 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5529999999999998e-06 2024-09-18T10:15:41.661 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.661 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.661 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:41.661 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694827+0000", 2024-09-18T10:15:41.661 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1500000000000001e-06 2024-09-18T10:15:41.661 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.661 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.661 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:41.661 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.797100+0000", 2024-09-18T10:15:41.661 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.102273424 2024-09-18T10:15:41.661 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.662 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.662 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:41.662 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.797329+0000", 2024-09-18T10:15:41.662 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00022870200000000001 2024-09-18T10:15:41.662 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.662 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.662 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:41.662 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.201342+0000", 2024-09-18T10:15:41.662 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.40401293799999999 2024-09-18T10:15:41.662 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:41.662 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:41.663 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:41.663 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.663 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.663 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:207 3.as0 3:5c566071:::smithi18925896-28:head [copy-from ver 6] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:41.663 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:52.694855+0000", 2024-09-18T10:15:41.663 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 48.954182674999998, 2024-09-18T10:15:41.663 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.81394701899999999, 2024-09-18T10:15:41.663 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:41.663 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:41.663 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:41.663 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:41.664 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:41.664 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 207 2024-09-18T10:15:41.664 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.664 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:41.664 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.664 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:41.664 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694855+0000", 2024-09-18T10:15:41.664 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:41.664 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.664 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.664 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:41.664 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694855+0000", 2024-09-18T10:15:41.664 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:41.665 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.665 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.665 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:41.665 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694852+0000", 2024-09-18T10:15:41.665 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999967 2024-09-18T10:15:41.665 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.665 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.665 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:41.665 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694859+0000", 2024-09-18T10:15:41.665 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.08e-06 2024-09-18T10:15:41.665 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.665 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.665 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:41.666 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694866+0000", 2024-09-18T10:15:41.666 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9249999999999998e-06 2024-09-18T10:15:41.666 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.666 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.666 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:41.666 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694870+0000", 2024-09-18T10:15:41.666 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0330000000000002e-06 2024-09-18T10:15:41.666 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.666 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.666 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:41.666 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.900743+0000", 2024-09-18T10:15:41.666 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20587325400000001 2024-09-18T10:15:41.666 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.666 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.667 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:41.667 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.901026+0000", 2024-09-18T10:15:41.667 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00028313600000000002 2024-09-18T10:15:41.667 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.667 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.667 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:41.667 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.204074+0000", 2024-09-18T10:15:41.667 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30304722299999998 2024-09-18T10:15:41.667 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.667 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.667 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:41.667 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.207280+0000", 2024-09-18T10:15:41.667 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0032067049999999998 2024-09-18T10:15:41.667 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.668 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.668 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:15:41.668 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.508672+0000", 2024-09-18T10:15:41.668 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30139174800000001 2024-09-18T10:15:41.668 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.668 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.668 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:41.668 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.508802+0000", 2024-09-18T10:15:41.668 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013037500000000001 2024-09-18T10:15:41.669 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:41.669 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:41.669 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:41.669 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.669 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.669 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:208 3.as0 3:5c566071:::smithi18925896-28:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:41.669 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:52.694875+0000", 2024-09-18T10:15:41.669 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 48.954163514000001, 2024-09-18T10:15:41.669 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.51199464299999997, 2024-09-18T10:15:41.669 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:41.669 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-18T10:15:41.669 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:41.669 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:41.670 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:41.670 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 208 2024-09-18T10:15:41.670 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.670 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:41.670 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.670 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:41.670 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694875+0000", 2024-09-18T10:15:41.670 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:41.670 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.670 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.670 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:41.670 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694875+0000", 2024-09-18T10:15:41.670 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:41.670 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.671 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.671 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:41.671 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694874+0000", 2024-09-18T10:15:41.671 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:41.671 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.671 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.671 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:41.671 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694876+0000", 2024-09-18T10:15:41.671 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2369999999999999e-06 2024-09-18T10:15:41.671 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.671 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.671 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:41.671 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694881+0000", 2024-09-18T10:15:41.671 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0950000000000003e-06 2024-09-18T10:15:41.672 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.672 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.672 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:41.672 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694887+0000", 2024-09-18T10:15:41.672 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5690000000000004e-06 2024-09-18T10:15:41.672 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.672 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.672 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:41.672 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.901475+0000", 2024-09-18T10:15:41.672 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20658803000000001 2024-09-18T10:15:41.672 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.672 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.672 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2024-09-18T10:15:41.673 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.901550+0000", 2024-09-18T10:15:41.673 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5020999999999999e-05 2024-09-18T10:15:41.673 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.673 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.673 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:41.673 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.206415+0000", 2024-09-18T10:15:41.673 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30486486000000002 2024-09-18T10:15:41.673 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.673 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.673 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:41.673 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.206634+0000", 2024-09-18T10:15:41.673 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00021925499999999999 2024-09-18T10:15:41.673 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.674 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.674 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:41.674 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.206869+0000", 2024-09-18T10:15:41.674 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00023503599999999999 2024-09-18T10:15:41.674 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:41.674 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:41.674 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:41.674 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.674 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.674 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:334 3.ds0 3:b1a14543:::smithi18925896-15:head [setxattr _header (56) in=63b] snapc b=[b,4] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-18T10:15:41.674 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:02.793164+0000", 2024-09-18T10:15:41.674 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 38.855873703, 2024-09-18T10:15:41.674 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.408134104, 2024-09-18T10:15:41.674 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:41.675 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:41.675 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:41.675 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:41.675 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:41.675 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 334 2024-09-18T10:15:41.675 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.675 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:41.675 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.675 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:41.675 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.793164+0000", 2024-09-18T10:15:41.675 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:41.675 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.675 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.675 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:41.676 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.793164+0000", 2024-09-18T10:15:41.676 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:41.676 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.676 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.676 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:41.676 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.793163+0000", 2024-09-18T10:15:41.676 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:41.676 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.676 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.676 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:41.676 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.793170+0000", 2024-09-18T10:15:41.676 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3050000000000004e-06 2024-09-18T10:15:41.676 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.677 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.677 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:41.677 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.793180+0000", 2024-09-18T10:15:41.677 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8069999999999994e-06 2024-09-18T10:15:41.677 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.677 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.677 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:41.677 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.793187+0000", 2024-09-18T10:15:41.677 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6879999999999999e-06 2024-09-18T10:15:41.677 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.677 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.677 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:41.677 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.793293+0000", 2024-09-18T10:15:41.677 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000105441 2024-09-18T10:15:41.677 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.678 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.678 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:41.678 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.793302+0000", 2024-09-18T10:15:41.678 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.5769999999999994e-06 2024-09-18T10:15:41.678 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.678 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.678 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:41.678 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.802703+0000", 2024-09-18T10:15:41.678 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0094011400000000005 2024-09-18T10:15:41.678 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.678 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.678 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:41.678 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.802949+0000", 2024-09-18T10:15:41.679 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00024604500000000002 2024-09-18T10:15:41.679 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.679 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.679 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:41.679 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.803628+0000", 2024-09-18T10:15:41.679 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000679391 2024-09-18T10:15:41.679 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.679 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.679 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:41.679 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.804224+0000", 2024-09-18T10:15:41.679 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00059597000000000003 2024-09-18T10:15:41.679 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.679 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.679 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:15:41.679 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:03.201019+0000", 2024-09-18T10:15:41.680 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.39679527799999997 2024-09-18T10:15:41.680 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.680 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.680 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:41.680 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:03.201298+0000", 2024-09-18T10:15:41.680 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00027909199999999999 2024-09-18T10:15:41.680 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:41.680 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:41.680 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:41.680 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.680 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.680 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:335 3.ds0 3:b1a14543:::smithi18925896-15:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e47)", 2024-09-18T10:15:41.680 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:02.793201+0000", 2024-09-18T10:15:41.681 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 38.855837242, 2024-09-18T10:15:41.681 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.51366298600000004, 2024-09-18T10:15:41.681 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:41.681 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-18T10:15:41.681 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:41.681 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:41.681 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:41.681 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 335 2024-09-18T10:15:41.681 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.681 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:41.681 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.681 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:41.681 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.793201+0000", 2024-09-18T10:15:41.682 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:41.682 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.682 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.682 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:41.682 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.793201+0000", 2024-09-18T10:15:41.682 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:41.682 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.682 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.682 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:41.682 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.793200+0000", 2024-09-18T10:15:41.682 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:41.682 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.682 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.682 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:41.683 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.793206+0000", 2024-09-18T10:15:41.683 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2199999999999997e-06 2024-09-18T10:15:41.683 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.683 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.683 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:41.683 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.793221+0000", 2024-09-18T10:15:41.683 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5092000000000001e-05 2024-09-18T10:15:41.683 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.683 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.683 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:41.683 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.793229+0000", 2024-09-18T10:15:41.683 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1820000000000006e-06 2024-09-18T10:15:41.683 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.684 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.684 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:41.684 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.793367+0000", 2024-09-18T10:15:41.684 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000138057 2024-09-18T10:15:41.684 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.684 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.684 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:41.684 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.793384+0000", 2024-09-18T10:15:41.684 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6731e-05 2024-09-18T10:15:41.684 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.684 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.684 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:41.684 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.804064+0000", 2024-09-18T10:15:41.684 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010680185 2024-09-18T10:15:41.685 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.685 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.685 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2024-09-18T10:15:41.685 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.804199+0000", 2024-09-18T10:15:41.685 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013532999999999999 2024-09-18T10:15:41.685 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.685 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.685 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:41.685 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:03.202193+0000", 2024-09-18T10:15:41.685 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.39799325099999999 2024-09-18T10:15:41.685 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.685 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.685 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:41.686 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:03.202292+0000", 2024-09-18T10:15:41.686 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9628000000000002e-05 2024-09-18T10:15:41.686 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.686 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.686 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:41.686 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:03.306864+0000", 2024-09-18T10:15:41.686 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.104571572 2024-09-18T10:15:41.686 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:41.686 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:41.686 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:41.686 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.686 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.686 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:503 3.fs0 3:feb35297:::smithi18925896-37:head [append 0~753664] snapc 19=[19,18,16,12,11,10,f,e] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T10:15:41.686 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:17.908895+0000", 2024-09-18T10:15:41.687 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 23.740143261, 2024-09-18T10:15:41.687 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.51525800200000005, 2024-09-18T10:15:41.687 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:41.687 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:41.687 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:41.687 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:41.687 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:41.687 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 503 2024-09-18T10:15:41.687 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.687 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:41.687 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.687 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:41.687 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.908895+0000", 2024-09-18T10:15:41.688 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:41.688 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.688 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.688 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:41.688 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.908895+0000", 2024-09-18T10:15:41.688 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:41.688 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.688 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.688 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:41.688 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.908892+0000", 2024-09-18T10:15:41.688 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999976 2024-09-18T10:15:41.688 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.688 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.688 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:41.689 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.910143+0000", 2024-09-18T10:15:41.689 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0012507040000000001 2024-09-18T10:15:41.689 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.689 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.689 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:41.689 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.910167+0000", 2024-09-18T10:15:41.689 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4176999999999999e-05 2024-09-18T10:15:41.689 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.689 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.689 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:41.689 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.910178+0000", 2024-09-18T10:15:41.689 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0669999999999999e-05 2024-09-18T10:15:41.689 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.690 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.690 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:41.690 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.910329+0000", 2024-09-18T10:15:41.690 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000150746 2024-09-18T10:15:41.690 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.690 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.690 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-09-18T10:15:41.690 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.910346+0000", 2024-09-18T10:15:41.690 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7680999999999999e-05 2024-09-18T10:15:41.690 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.690 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.690 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:41.690 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.015009+0000", 2024-09-18T10:15:41.690 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.104662914 2024-09-18T10:15:41.691 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.691 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.691 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:41.691 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.015304+0000", 2024-09-18T10:15:41.691 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00029439899999999998 2024-09-18T10:15:41.691 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.691 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.691 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:41.691 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.017669+0000", 2024-09-18T10:15:41.691 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0023647910000000002 2024-09-18T10:15:41.691 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.691 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.691 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:41.692 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.126654+0000", 2024-09-18T10:15:41.692 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.108985055 2024-09-18T10:15:41.692 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.692 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.692 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:15:41.692 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.423961+0000", 2024-09-18T10:15:41.692 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.29730786399999998 2024-09-18T10:15:41.692 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.692 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.692 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:41.692 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.424153+0000", 2024-09-18T10:15:41.692 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019128099999999999 2024-09-18T10:15:41.692 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:41.692 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:41.693 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:41.693 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.693 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.693 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:504 3.fs0 3:feb35297:::smithi18925896-37:head [append 0~720896] snapc 19=[19,18,16,12,11,10,f,e] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T10:15:41.693 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:17.912996+0000", 2024-09-18T10:15:41.693 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 23.736042225999999, 2024-09-18T10:15:41.693 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.61195014999999997, 2024-09-18T10:15:41.693 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:41.693 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:41.693 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:41.693 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:41.693 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:41.693 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 504 2024-09-18T10:15:41.694 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.694 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:41.694 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.694 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:41.694 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.912996+0000", 2024-09-18T10:15:41.694 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:41.694 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.694 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.694 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:41.694 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.912996+0000", 2024-09-18T10:15:41.694 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:41.694 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.694 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.694 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:41.695 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.912994+0000", 2024-09-18T10:15:41.695 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:41.695 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.695 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.695 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:41.695 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.913907+0000", 2024-09-18T10:15:41.695 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00091254300000000003 2024-09-18T10:15:41.695 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.695 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.695 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:41.695 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.913950+0000", 2024-09-18T10:15:41.695 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3019000000000003e-05 2024-09-18T10:15:41.695 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.696 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.696 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:41.696 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.913960+0000", 2024-09-18T10:15:41.696 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0305e-05 2024-09-18T10:15:41.696 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.696 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.696 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:41.696 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.914067+0000", 2024-09-18T10:15:41.696 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000106696 2024-09-18T10:15:41.696 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.696 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.696 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:41.696 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.914079+0000", 2024-09-18T10:15:41.697 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2330000000000001e-05 2024-09-18T10:15:41.697 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.697 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.697 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:41.697 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.119322+0000", 2024-09-18T10:15:41.697 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20524277899999999 2024-09-18T10:15:41.697 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.697 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.697 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:41.697 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.119508+0000", 2024-09-18T10:15:41.697 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00018648800000000001 2024-09-18T10:15:41.697 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.697 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.698 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:41.698 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.121315+0000", 2024-09-18T10:15:41.698 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001806483 2024-09-18T10:15:41.698 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.698 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.698 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:41.698 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.127304+0000", 2024-09-18T10:15:41.698 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0059895599999999997 2024-09-18T10:15:41.698 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.698 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.698 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:15:41.698 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.524828+0000", 2024-09-18T10:15:41.698 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.39752343600000001 2024-09-18T10:15:41.698 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.699 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.699 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:41.699 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.524946+0000", 2024-09-18T10:15:41.699 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000118193 2024-09-18T10:15:41.699 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:41.699 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:41.699 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:41.699 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.699 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.699 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:505 3.fs0 3:feb35297:::smithi18925896-37:head [append 0~49152] snapc 19=[19,18,16,12,11,10,f,e] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T10:15:41.699 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:17.913979+0000", 2024-09-18T10:15:41.699 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 23.735058747, 2024-09-18T10:15:41.699 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.71150136900000005, 2024-09-18T10:15:41.699 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:41.700 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:41.700 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:41.700 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:41.700 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:41.700 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 505 2024-09-18T10:15:41.700 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.700 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:41.700 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.700 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:41.700 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.913979+0000", 2024-09-18T10:15:41.700 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:41.700 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.700 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.701 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:41.701 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.913979+0000", 2024-09-18T10:15:41.701 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:41.701 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.701 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.701 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:41.701 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.913978+0000", 2024-09-18T10:15:41.701 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:41.701 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.701 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.701 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:41.701 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.914061+0000", 2024-09-18T10:15:41.701 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2533000000000004e-05 2024-09-18T10:15:41.702 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.702 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.702 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:41.702 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.914077+0000", 2024-09-18T10:15:41.702 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6277e-05 2024-09-18T10:15:41.702 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.702 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.702 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:41.702 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.914087+0000", 2024-09-18T10:15:41.702 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0282000000000001e-05 2024-09-18T10:15:41.702 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.702 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.702 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:41.702 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.914144+0000", 2024-09-18T10:15:41.703 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6784999999999999e-05 2024-09-18T10:15:41.703 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.703 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.703 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:41.703 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.914151+0000", 2024-09-18T10:15:41.703 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.968e-06 2024-09-18T10:15:41.703 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.703 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.703 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:41.703 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.122495+0000", 2024-09-18T10:15:41.703 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.208343742 2024-09-18T10:15:41.703 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.703 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.703 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:41.704 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.122788+0000", 2024-09-18T10:15:41.704 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00029334199999999998 2024-09-18T10:15:41.704 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.704 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.704 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:41.704 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.124367+0000", 2024-09-18T10:15:41.704 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0015794699999999999 2024-09-18T10:15:41.704 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.704 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.704 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:41.704 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.127307+0000", 2024-09-18T10:15:41.704 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0029394780000000001 2024-09-18T10:15:41.704 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.704 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.705 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:15:41.705 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.625281+0000", 2024-09-18T10:15:41.705 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.497974166 2024-09-18T10:15:41.705 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.705 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.705 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:41.705 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.625481+0000", 2024-09-18T10:15:41.705 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019956100000000001 2024-09-18T10:15:41.705 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:41.705 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:41.705 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:41.705 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.705 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.705 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:506 3.fs0 3:feb35297:::smithi18925896-37:head [setxattr _header (57) in=7b] snapc 19=[19,18,16,12,11,10,f,e] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T10:15:41.706 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:17.914106+0000", 2024-09-18T10:15:41.706 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 23.734932480000001, 2024-09-18T10:15:41.706 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.71184716599999998, 2024-09-18T10:15:41.706 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:41.706 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:41.706 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:41.706 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:41.706 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:41.706 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 506 2024-09-18T10:15:41.706 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.706 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:41.706 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.706 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:41.706 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.914106+0000", 2024-09-18T10:15:41.707 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:41.707 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.707 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.707 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:41.707 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.914106+0000", 2024-09-18T10:15:41.707 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:41.707 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.707 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.707 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:41.707 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.914105+0000", 2024-09-18T10:15:41.707 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:15:41.707 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.707 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.707 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:41.708 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.914110+0000", 2024-09-18T10:15:41.708 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.045e-06 2024-09-18T10:15:41.708 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.708 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.708 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:41.708 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.914117+0000", 2024-09-18T10:15:41.708 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6510000000000001e-06 2024-09-18T10:15:41.708 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.708 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.708 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:41.708 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.914122+0000", 2024-09-18T10:15:41.708 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5839999999999996e-06 2024-09-18T10:15:41.708 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.709 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.709 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:41.709 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.014305+0000", 2024-09-18T10:15:41.709 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100183091 2024-09-18T10:15:41.709 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.709 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.709 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:41.709 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.014325+0000", 2024-09-18T10:15:41.709 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0032000000000001e-05 2024-09-18T10:15:41.709 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.709 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.709 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:41.709 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.125511+0000", 2024-09-18T10:15:41.710 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.111185942 2024-09-18T10:15:41.710 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.710 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.710 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:41.710 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.125760+0000", 2024-09-18T10:15:41.710 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00024864100000000002 2024-09-18T10:15:41.710 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.710 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.710 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:41.711 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.126375+0000", 2024-09-18T10:15:41.711 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00061536299999999996 2024-09-18T10:15:41.711 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.711 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.711 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:41.711 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.227656+0000", 2024-09-18T10:15:41.711 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.101281097 2024-09-18T10:15:41.711 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.711 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.711 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:15:41.711 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.625791+0000", 2024-09-18T10:15:41.711 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.39813470299999998 2024-09-18T10:15:41.711 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.712 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.712 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:41.712 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.625953+0000", 2024-09-18T10:15:41.712 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00016192999999999999 2024-09-18T10:15:41.712 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:41.712 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:41.712 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:41.712 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.712 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.712 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:507 3.fs0 3:feb35297:::smithi18925896-37:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e62)", 2024-09-18T10:15:41.712 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:17.914159+0000", 2024-09-18T10:15:41.712 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 23.734879364000001, 2024-09-18T10:15:41.712 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0151776809999999, 2024-09-18T10:15:41.713 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:41.713 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-18T10:15:41.713 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:41.713 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:41.713 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:41.713 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 507 2024-09-18T10:15:41.713 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.713 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:41.713 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.713 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:41.713 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.914159+0000", 2024-09-18T10:15:41.713 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:41.714 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.714 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.714 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:41.714 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.914159+0000", 2024-09-18T10:15:41.714 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:41.714 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.714 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.714 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:41.714 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.914158+0000", 2024-09-18T10:15:41.714 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:15:41.714 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.714 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.714 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:41.714 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.914162+0000", 2024-09-18T10:15:41.715 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0729999999999998e-06 2024-09-18T10:15:41.715 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.715 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.715 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:41.715 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.914176+0000", 2024-09-18T10:15:41.715 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4258e-05 2024-09-18T10:15:41.715 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.715 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.715 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:41.715 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.014273+0000", 2024-09-18T10:15:41.715 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10009702400000001 2024-09-18T10:15:41.715 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.715 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.715 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:41.716 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.126754+0000", 2024-09-18T10:15:41.716 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.112480581 2024-09-18T10:15:41.716 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.716 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.716 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2024-09-18T10:15:41.716 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.127051+0000", 2024-09-18T10:15:41.716 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00029721199999999999 2024-09-18T10:15:41.716 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.716 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.716 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:41.716 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.626655+0000", 2024-09-18T10:15:41.716 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.49960442399999999 2024-09-18T10:15:41.716 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.717 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.717 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:41.717 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.626826+0000", 2024-09-18T10:15:41.717 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000170459 2024-09-18T10:15:41.717 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.717 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.717 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:41.717 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.929336+0000", 2024-09-18T10:15:41.717 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30251042900000003 2024-09-18T10:15:41.717 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:41.717 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:41.717 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:41.717 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.717 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.718 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:524 3.ds0 3:b1a14543:::smithi18925896-15:head [append 0~442368] snapc 1a=[1a,19,18,16,12,11,10,f,e] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-18T10:15:41.718 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:19.452015+0000", 2024-09-18T10:15:41.718 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 22.197023382000001, 2024-09-18T10:15:41.718 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.55248773200000001, 2024-09-18T10:15:41.718 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:41.718 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:41.718 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:41.719 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:41.720 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:41.720 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 524 2024-09-18T10:15:41.720 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.720 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:41.720 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.720 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:41.720 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.452015+0000", 2024-09-18T10:15:41.720 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:41.720 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.720 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.720 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:41.720 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.452015+0000", 2024-09-18T10:15:41.720 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:41.721 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.721 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.721 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:41.721 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.452012+0000", 2024-09-18T10:15:41.721 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999976 2024-09-18T10:15:41.721 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.721 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.721 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:41.721 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.452793+0000", 2024-09-18T10:15:41.721 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00078048600000000005 2024-09-18T10:15:41.721 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.722 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.722 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:41.722 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.452825+0000", 2024-09-18T10:15:41.722 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2672999999999997e-05 2024-09-18T10:15:41.722 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.722 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.722 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:41.722 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.552970+0000", 2024-09-18T10:15:41.722 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100144728 2024-09-18T10:15:41.722 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.722 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.722 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:41.722 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.553127+0000", 2024-09-18T10:15:41.722 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000157202 2024-09-18T10:15:41.722 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.723 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.723 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:41.723 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.553439+0000", 2024-09-18T10:15:41.723 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00031139500000000001 2024-09-18T10:15:41.723 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.723 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.723 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:41.723 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.556240+0000", 2024-09-18T10:15:41.723 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0028012509999999998 2024-09-18T10:15:41.723 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.723 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.723 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:41.724 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.665529+0000", 2024-09-18T10:15:41.724 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10928916700000001 2024-09-18T10:15:41.724 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.724 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.724 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:15:41.724 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:20.004378+0000", 2024-09-18T10:15:41.724 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.33884916300000001 2024-09-18T10:15:41.724 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.724 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.724 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:41.724 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:20.004502+0000", 2024-09-18T10:15:41.724 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012407999999999999 2024-09-18T10:15:41.724 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:41.725 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:41.725 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:41.725 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.725 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.725 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:525 3.ds0 3:b1a14543:::smithi18925896-15:head [append 0~540672] snapc 1a=[1a,19,18,16,12,11,10,f,e] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-18T10:15:41.725 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:19.553028+0000", 2024-09-18T10:15:41.725 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 22.096009543000001, 2024-09-18T10:15:41.725 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.45756244000000001, 2024-09-18T10:15:41.725 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:41.725 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:41.725 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:41.726 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:41.726 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:41.726 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 525 2024-09-18T10:15:41.726 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.726 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:41.726 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.726 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:41.726 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.553028+0000", 2024-09-18T10:15:41.726 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:41.726 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.726 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.726 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:41.727 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.553028+0000", 2024-09-18T10:15:41.727 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:41.727 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.727 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.727 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:41.727 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.553026+0000", 2024-09-18T10:15:41.727 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:41.727 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.727 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.727 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:41.727 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.553796+0000", 2024-09-18T10:15:41.727 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00076915200000000005 2024-09-18T10:15:41.727 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.727 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.728 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:41.728 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.553819+0000", 2024-09-18T10:15:41.728 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3008999999999999e-05 2024-09-18T10:15:41.728 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.728 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.728 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:41.728 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.553829+0000", 2024-09-18T10:15:41.728 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0543999999999999e-05 2024-09-18T10:15:41.728 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.728 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.728 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:41.728 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.558075+0000", 2024-09-18T10:15:41.728 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.004245932 2024-09-18T10:15:41.728 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.729 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.729 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:41.729 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.558384+0000", 2024-09-18T10:15:41.729 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00030926100000000002 2024-09-18T10:15:41.729 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.729 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.729 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:41.729 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.559950+0000", 2024-09-18T10:15:41.729 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001565615 2024-09-18T10:15:41.729 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.729 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.729 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:41.730 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.665535+0000", 2024-09-18T10:15:41.730 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.105584707 2024-09-18T10:15:41.730 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.730 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.730 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:15:41.730 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:20.010428+0000", 2024-09-18T10:15:41.730 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.34489341800000001 2024-09-18T10:15:41.730 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.730 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.730 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:41.730 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:20.010591+0000", 2024-09-18T10:15:41.730 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000162845 2024-09-18T10:15:41.730 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:41.730 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:41.731 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:41.731 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.731 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.731 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:526 3.ds0 3:b1a14543:::smithi18925896-15:head [append 0~622592] snapc 1a=[1a,19,18,16,12,11,10,f,e] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-18T10:15:41.731 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:19.553845+0000", 2024-09-18T10:15:41.731 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 22.095192827999998, 2024-09-18T10:15:41.731 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.55723060999999996, 2024-09-18T10:15:41.731 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:41.731 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:41.731 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:41.731 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:41.731 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:41.731 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 526 2024-09-18T10:15:41.731 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.732 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:41.732 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.732 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:41.732 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.553845+0000", 2024-09-18T10:15:41.732 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:41.732 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.732 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.732 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:41.732 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.553845+0000", 2024-09-18T10:15:41.732 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:41.732 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.732 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.732 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:41.733 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.553844+0000", 2024-09-18T10:15:41.733 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:41.733 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.733 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.733 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:41.733 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.554666+0000", 2024-09-18T10:15:41.733 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00082239100000000001 2024-09-18T10:15:41.733 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.733 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.733 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:41.733 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.554679+0000", 2024-09-18T10:15:41.733 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.271e-05 2024-09-18T10:15:41.733 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.733 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.734 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:41.734 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.554685+0000", 2024-09-18T10:15:41.734 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4230000000000002e-06 2024-09-18T10:15:41.734 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.734 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.734 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:41.734 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.561235+0000", 2024-09-18T10:15:41.734 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0065498179999999998 2024-09-18T10:15:41.734 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.734 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.734 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:41.734 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.561523+0000", 2024-09-18T10:15:41.734 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000287862 2024-09-18T10:15:41.734 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.735 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.735 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:41.735 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.562904+0000", 2024-09-18T10:15:41.735 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0013810090000000001 2024-09-18T10:15:41.735 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.735 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.735 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:41.735 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.666083+0000", 2024-09-18T10:15:41.735 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.103178495 2024-09-18T10:15:41.735 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.735 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.735 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:15:41.735 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:20.110956+0000", 2024-09-18T10:15:41.735 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.44487371999999997 2024-09-18T10:15:41.736 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.736 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.736 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:41.736 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:20.111076+0000", 2024-09-18T10:15:41.736 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000119436 2024-09-18T10:15:41.736 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:41.736 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:41.736 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:41.736 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.736 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.736 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:527 3.ds0 3:b1a14543:::smithi18925896-15:head [setxattr _header (57) in=7b] snapc 1a=[1a,19,18,16,12,11,10,f,e] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-18T10:15:41.736 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:19.554694+0000", 2024-09-18T10:15:41.736 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 22.094344460999999, 2024-09-18T10:15:41.737 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.55695282099999999, 2024-09-18T10:15:41.737 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:41.737 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:41.737 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:41.737 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:41.737 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:41.737 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 527 2024-09-18T10:15:41.737 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.737 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:41.737 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.738 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:41.738 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.554694+0000", 2024-09-18T10:15:41.738 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:41.738 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.738 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.738 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:41.738 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.554694+0000", 2024-09-18T10:15:41.738 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:41.738 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.738 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.738 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:41.738 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.554692+0000", 2024-09-18T10:15:41.738 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:15:41.739 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.739 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.739 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:41.739 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.554698+0000", 2024-09-18T10:15:41.739 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8599999999999998e-06 2024-09-18T10:15:41.739 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.739 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.739 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:41.739 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.554707+0000", 2024-09-18T10:15:41.739 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4090000000000003e-06 2024-09-18T10:15:41.739 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.739 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.739 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:41.739 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.554712+0000", 2024-09-18T10:15:41.740 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4500000000000003e-06 2024-09-18T10:15:41.740 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.740 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.740 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:41.740 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.663747+0000", 2024-09-18T10:15:41.740 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10903526400000001 2024-09-18T10:15:41.740 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.740 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.740 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:41.740 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.664066+0000", 2024-09-18T10:15:41.740 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00031885099999999999 2024-09-18T10:15:41.740 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.741 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.741 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:41.741 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.665080+0000", 2024-09-18T10:15:41.741 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010141200000000001 2024-09-18T10:15:41.741 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.741 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.741 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:41.741 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.666476+0000", 2024-09-18T10:15:41.741 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001395456 2024-09-18T10:15:41.741 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.741 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.741 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:15:41.741 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:20.111512+0000", 2024-09-18T10:15:41.741 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.445035929 2024-09-18T10:15:41.742 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.742 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.742 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:41.742 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:20.111646+0000", 2024-09-18T10:15:41.742 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000134542 2024-09-18T10:15:41.742 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:41.742 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:41.742 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:41.742 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.742 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.742 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:687 3.ds0 3:b1a14543:::smithi18925896-15:head [read 0~2899968,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-18T10:15:41.742 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:35.580026+0000", 2024-09-18T10:15:41.743 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 6.0690123490000003, 2024-09-18T10:15:41.743 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.51409843, 2024-09-18T10:15:41.743 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:41.743 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-18T10:15:41.743 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:41.743 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:41.743 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:41.743 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 687 2024-09-18T10:15:41.743 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.743 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:41.743 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.743 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:41.743 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:35.580026+0000", 2024-09-18T10:15:41.743 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:41.744 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.744 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.744 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:41.744 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:35.580026+0000", 2024-09-18T10:15:41.744 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:41.744 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.744 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.744 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:41.744 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:35.580025+0000", 2024-09-18T10:15:41.744 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:15:41.744 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.745 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.745 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:41.745 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:35.580032+0000", 2024-09-18T10:15:41.745 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1500000000000002e-06 2024-09-18T10:15:41.745 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.745 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.745 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:41.745 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:35.580045+0000", 2024-09-18T10:15:41.745 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3086e-05 2024-09-18T10:15:41.745 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.745 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.745 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:41.745 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:35.580054+0000", 2024-09-18T10:15:41.746 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9770000000000006e-06 2024-09-18T10:15:41.746 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.746 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.746 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:41.746 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:35.580361+0000", 2024-09-18T10:15:41.746 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00030747400000000002 2024-09-18T10:15:41.746 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.746 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.746 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:41.746 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:35.580372+0000", 2024-09-18T10:15:41.746 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0550000000000001e-05 2024-09-18T10:15:41.746 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.746 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.746 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:41.747 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:35.686406+0000", 2024-09-18T10:15:41.747 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10603446 2024-09-18T10:15:41.747 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.747 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.747 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:41.747 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:35.686592+0000", 2024-09-18T10:15:41.747 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00018559300000000001 2024-09-18T10:15:41.747 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.747 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.747 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:41.747 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:36.094124+0000", 2024-09-18T10:15:41.747 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.40753219699999998 2024-09-18T10:15:41.747 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:41.748 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:41.748 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:41.748 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.748 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.748 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:767 3.as0 3:5c566071:::smithi18925896-28:head [read 0~557056,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T10:15:41.748 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:40.627001+0000", 2024-09-18T10:15:41.748 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.022036671, 2024-09-18T10:15:41.748 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.53039000400000003, 2024-09-18T10:15:41.748 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:41.748 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-18T10:15:41.748 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:41.748 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:41.749 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:41.749 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 767 2024-09-18T10:15:41.749 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.749 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:41.749 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.749 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:41.749 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.627001+0000", 2024-09-18T10:15:41.749 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:41.749 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.749 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.749 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:41.749 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.627001+0000", 2024-09-18T10:15:41.749 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:41.749 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.750 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.750 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:41.750 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.626999+0000", 2024-09-18T10:15:41.750 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999971 2024-09-18T10:15:41.750 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.750 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.750 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:41.750 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.627016+0000", 2024-09-18T10:15:41.750 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7051999999999999e-05 2024-09-18T10:15:41.750 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.750 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.750 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:41.750 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.627055+0000", 2024-09-18T10:15:41.750 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9099e-05 2024-09-18T10:15:41.751 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.751 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.751 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:41.751 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.727187+0000", 2024-09-18T10:15:41.751 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100131951 2024-09-18T10:15:41.751 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.751 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.751 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:41.751 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.727299+0000", 2024-09-18T10:15:41.751 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011210899999999999 2024-09-18T10:15:41.751 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.751 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.751 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-09-18T10:15:41.751 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.727308+0000", 2024-09-18T10:15:41.752 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7539999999999995e-06 2024-09-18T10:15:41.752 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.752 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.752 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:41.752 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.832882+0000", 2024-09-18T10:15:41.752 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.105574662 2024-09-18T10:15:41.752 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.752 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.752 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:41.752 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.833170+0000", 2024-09-18T10:15:41.752 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00028771299999999999 2024-09-18T10:15:41.752 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.752 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.752 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:41.753 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:41.157391+0000", 2024-09-18T10:15:41.753 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.324221431 2024-09-18T10:15:41.753 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:41.753 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:41.753 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:41.753 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.753 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.753 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:768 3.as0 3:5c566071:::smithi18925896-28:head [read 0~557056,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T10:15:41.753 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:40.727231+0000", 2024-09-18T10:15:41.753 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.92180655499999997, 2024-09-18T10:15:41.753 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.43091931700000002, 2024-09-18T10:15:41.753 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:41.753 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-18T10:15:41.753 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:41.754 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:41.754 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:41.754 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 768 2024-09-18T10:15:41.754 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.754 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:41.754 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.754 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:41.754 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.727231+0000", 2024-09-18T10:15:41.754 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:41.754 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.754 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.754 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:41.754 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.727231+0000", 2024-09-18T10:15:41.755 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:41.755 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.755 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.755 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:41.755 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.727229+0000", 2024-09-18T10:15:41.755 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999971 2024-09-18T10:15:41.755 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.755 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.755 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:41.755 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.727244+0000", 2024-09-18T10:15:41.755 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4956e-05 2024-09-18T10:15:41.755 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.755 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.756 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:41.756 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.727271+0000", 2024-09-18T10:15:41.756 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6959000000000001e-05 2024-09-18T10:15:41.756 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.756 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.756 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:41.756 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.727285+0000", 2024-09-18T10:15:41.756 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4192999999999999e-05 2024-09-18T10:15:41.756 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.756 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.756 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:41.756 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.727413+0000", 2024-09-18T10:15:41.756 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012852100000000001 2024-09-18T10:15:41.757 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.757 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.757 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:41.757 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.727436+0000", 2024-09-18T10:15:41.757 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3065999999999999e-05 2024-09-18T10:15:41.757 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.757 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.757 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:41.757 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.834297+0000", 2024-09-18T10:15:41.757 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.106860119 2024-09-18T10:15:41.757 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.757 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.757 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:41.757 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.834552+0000", 2024-09-18T10:15:41.758 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000255336 2024-09-18T10:15:41.758 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.758 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.758 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:41.758 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:41.158151+0000", 2024-09-18T10:15:41.758 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.32359887999999998 2024-09-18T10:15:41.758 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:41.758 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:41.758 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:41.758 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.758 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.758 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:769 3.as0 3:5c566071:::smithi18925896-28:head [read 0~557056,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T10:15:41.758 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:40.727311+0000", 2024-09-18T10:15:41.758 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.92172731799999996, 2024-09-18T10:15:41.759 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.43185177800000002, 2024-09-18T10:15:41.759 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:41.759 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-18T10:15:41.759 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:41.759 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:41.759 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:41.759 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 769 2024-09-18T10:15:41.759 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.759 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:41.759 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.759 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:41.759 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.727311+0000", 2024-09-18T10:15:41.759 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:41.760 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.760 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.760 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:41.760 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.727311+0000", 2024-09-18T10:15:41.760 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:41.760 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.760 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.760 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:41.760 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.727309+0000", 2024-09-18T10:15:41.760 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:41.760 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.760 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.760 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:41.761 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.727318+0000", 2024-09-18T10:15:41.761 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8060000000000001e-06 2024-09-18T10:15:41.761 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.761 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.761 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:41.761 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.727336+0000", 2024-09-18T10:15:41.761 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8073000000000001e-05 2024-09-18T10:15:41.761 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.761 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.761 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:41.761 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.727347+0000", 2024-09-18T10:15:41.761 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1165999999999999e-05 2024-09-18T10:15:41.761 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.762 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.762 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:41.762 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.727497+0000", 2024-09-18T10:15:41.762 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014967600000000001 2024-09-18T10:15:41.762 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.762 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.762 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:41.762 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.727520+0000", 2024-09-18T10:15:41.762 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2759e-05 2024-09-18T10:15:41.762 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.762 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.762 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:41.762 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.835360+0000", 2024-09-18T10:15:41.763 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.107840695 2024-09-18T10:15:41.763 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.763 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.763 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:41.763 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.835611+0000", 2024-09-18T10:15:41.763 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00025039000000000002 2024-09-18T10:15:41.763 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.763 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.763 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:41.763 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:41.159162+0000", 2024-09-18T10:15:41.763 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.32355174599999997 2024-09-18T10:15:41.763 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:41.763 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:41.764 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:41.764 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.764 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.764 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:779 3.as0 3:5dc9cbed:::smithi18925896-13:head [copy-from ver 47 in=55b] snapc 2f=[2f,29,28,20,1f,1e,1a,11] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-18T10:15:41.764 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:40.727404+0000", 2024-09-18T10:15:41.764 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.92163369799999995, 2024-09-18T10:15:41.764 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.73144490900000003, 2024-09-18T10:15:41.764 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:15:41.764 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:41.764 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:15:41.764 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:15:41.764 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:41.764 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 779 2024-09-18T10:15:41.764 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.765 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:15:41.765 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.765 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:15:41.765 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.727404+0000", 2024-09-18T10:15:41.765 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:41.765 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.765 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.765 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:15:41.765 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.727404+0000", 2024-09-18T10:15:41.765 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:15:41.765 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.765 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.765 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:15:41.765 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.727403+0000", 2024-09-18T10:15:41.766 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:41.766 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.766 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.766 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:15:41.766 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.727413+0000", 2024-09-18T10:15:41.766 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6940000000000007e-06 2024-09-18T10:15:41.766 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.766 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.766 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:15:41.766 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.727426+0000", 2024-09-18T10:15:41.766 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2896e-05 2024-09-18T10:15:41.766 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.766 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.766 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:15:41.767 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.727440+0000", 2024-09-18T10:15:41.767 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4285e-05 2024-09-18T10:15:41.767 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.767 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.767 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:41.767 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.727569+0000", 2024-09-18T10:15:41.767 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000128906 2024-09-18T10:15:41.767 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.767 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.767 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:41.767 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.727580+0000", 2024-09-18T10:15:41.767 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0720000000000001e-05 2024-09-18T10:15:41.767 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.767 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.767 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:15:41.767 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.936548+0000", 2024-09-18T10:15:41.768 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20896883299999999 2024-09-18T10:15:41.768 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.768 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.768 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:15:41.768 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.936872+0000", 2024-09-18T10:15:41.768 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00032318499999999998 2024-09-18T10:15:41.768 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.768 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.768 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:15:41.768 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:41.066546+0000", 2024-09-18T10:15:41.768 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.12967457600000001 2024-09-18T10:15:41.768 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.768 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.768 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:15:41.769 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:41.068954+0000", 2024-09-18T10:15:41.769 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002408154 2024-09-18T10:15:41.769 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.769 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.769 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:15:41.769 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:41.458701+0000", 2024-09-18T10:15:41.769 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.38974719899999999 2024-09-18T10:15:41.769 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:15:41.769 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:15:41.769 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:15:41.769 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:41.458849+0000", 2024-09-18T10:15:41.769 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014777799999999999 2024-09-18T10:15:41.770 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:41.770 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:41.770 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:41.770 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:15:41.770 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:15:41.770 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-18T10:15:41.772 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-18T10:15:41.772 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-18T10:15:41.773 INFO:teuthology.orchestra.run.smithi029.stdout:osd.6: {} 2024-09-18T10:15:41.773 INFO:teuthology.orchestra.run.smithi029.stdout:osd.7: {} 2024-09-18T10:15:41.773 INFO:teuthology.orchestra.run.smithi029.stdout:osd.8: {} 2024-09-18T10:15:41.773 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T10:15:41.774 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-18T10:15:41.768+0000 7f1981873700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22132) UID: 0 2024-09-18T10:15:41.774 INFO:tasks.rados.rados.0.smithi189.stdout:295: done (8 left) 2024-09-18T10:15:41.774 INFO:tasks.rados.rados.0.smithi189.stdout:305: done (7 left) 2024-09-18T10:15:41.774 INFO:tasks.rados.rados.0.smithi189.stdout:307: done (6 left) 2024-09-18T10:15:41.774 INFO:tasks.rados.rados.0.smithi189.stdout:312: done (5 left) 2024-09-18T10:15:41.775 INFO:tasks.rados.rados.0.smithi189.stdout:317: done (4 left) 2024-09-18T10:15:41.775 INFO:tasks.rados.rados.0.smithi189.stdout:318: copy_from oid 44 from oid 37 current snap is 27 2024-09-18T10:15:41.775 INFO:tasks.rados.rados.0.smithi189.stdout:319: read oid 31 snap -1 2024-09-18T10:15:41.775 INFO:tasks.rados.rados.0.smithi189.stdout:319: expect (ObjNum 84 snap 14 seq_num 84) 2024-09-18T10:15:41.775 INFO:tasks.rados.rados.0.smithi189.stdout:320: rollback oid 33 current snap is 27 2024-09-18T10:15:41.775 INFO:tasks.rados.rados.0.smithi189.stdout:rollback oid 33 to 16 2024-09-18T10:15:41.775 INFO:tasks.rados.rados.0.smithi189.stdout:321: read oid 39 snap -1 2024-09-18T10:15:41.775 INFO:tasks.rados.rados.0.smithi189.stdout:321: expect deleted 2024-09-18T10:15:41.775 INFO:tasks.rados.rados.0.smithi189.stdout:322: delete oid 35 current snap is 27 2024-09-18T10:15:41.775 INFO:tasks.rados.rados.0.smithi189.stdout:304: finishing write tid 6 to smithi18925896-16 2024-09-18T10:15:41.775 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 16 v 84 (ObjNum 121 snap 26 seq_num 121) dirty exists 2024-09-18T10:15:41.775 INFO:tasks.rados.rados.0.smithi189.stdout:304: left oid 16 (ObjNum 121 snap 26 seq_num 121) 2024-09-18T10:15:41.776 INFO:tasks.rados.rados.0.smithi189.stdout:310: finishing write tid 1 to smithi18925896-6 2024-09-18T10:15:41.777 INFO:teuthology.orchestra.run.smithi029.stdout:osd.9: {} 2024-09-18T10:15:41.829 INFO:tasks.rados.rados.0.smithi189.stdout:304: done (8 left) 2024-09-18T10:15:41.830 INFO:tasks.rados.rados.0.smithi189.stdout:321: done (7 left) 2024-09-18T10:15:41.830 INFO:tasks.rados.rados.0.smithi189.stdout:322: done (6 left) 2024-09-18T10:15:41.830 INFO:tasks.rados.rados.0.smithi189.stdout:323: rollback oid 25 current snap is 27 2024-09-18T10:15:41.830 INFO:tasks.rados.rados.0.smithi189.stdout:rollback oid 25 to 18 2024-09-18T10:15:41.830 INFO:tasks.rados.rados.0.smithi189.stdout:324: rollback oid 9 current snap is 27 2024-09-18T10:15:41.830 INFO:tasks.rados.rados.0.smithi189.stdout:rollback oid 9 to 17 2024-09-18T10:15:41.830 INFO:tasks.rados.rados.0.smithi189.stdout:append oid 42 current snap is 27 2024-09-18T10:15:41.830 INFO:tasks.rados.rados.0.smithi189.stdout:325: seq_num 129 ranges {0=622592,622592=655360,1277952=688128,1966080=212992} 2024-09-18T10:15:41.839 INFO:tasks.rados.rados.0.smithi189.stdout:325: writing smithi18925896-42 from 0 to 622592 tid 1 2024-09-18T10:15:41.843 INFO:tasks.rados.rados.0.smithi189.stdout:325: writing smithi18925896-42 from 622592 to 1277952 tid 2 2024-09-18T10:15:41.847 INFO:tasks.rados.rados.0.smithi189.stdout:325: writing smithi18925896-42 from 1277952 to 1966080 tid 3 2024-09-18T10:15:41.848 INFO:tasks.rados.rados.0.smithi189.stdout:325: writing smithi18925896-42 from 1966080 to 2179072 tid 4 2024-09-18T10:15:41.849 INFO:tasks.rados.rados.0.smithi189.stdout:326: read oid 39 snap -1 2024-09-18T10:15:41.849 INFO:tasks.rados.rados.0.smithi189.stdout:326: expect deleted 2024-09-18T10:15:41.849 INFO:tasks.rados.rados.0.smithi189.stdout:327: rollback oid 18 current snap is 27 2024-09-18T10:15:41.849 INFO:tasks.rados.rados.0.smithi189.stdout:rollback oid 18 to 18 2024-09-18T10:15:41.849 INFO:tasks.rados.rados.0.smithi189.stdout:300: finishing rollback tid 2 to smithi18925896-45 2024-09-18T10:15:41.849 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 45 v 86 (ObjNum 82 snap 13 seq_num 82) dirty exists 2024-09-18T10:15:41.849 INFO:tasks.rados.rados.0.smithi189.stdout:300: done (10 left) 2024-09-18T10:15:41.849 INFO:tasks.rados.rados.0.smithi189.stdout:328: snap_create 2024-09-18T10:15:41.862 INFO:teuthology.orchestra.run.smithi029.stdout:osd.10: {} 2024-09-18T10:15:41.872 INFO:tasks.ceph.osd.10.smithi154.stderr:2024-09-18T10:15:41.870+0000 7fb73221a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 22091) UID: 0 2024-09-18T10:15:41.883 INFO:teuthology.orchestra.run.smithi029.stdout:osd.11: {} 2024-09-18T10:15:41.885 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:41.885 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:41.895 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T10:15:41.904 INFO:teuthology.orchestra.run.smithi029.stdout:osd.12: {} 2024-09-18T10:15:41.927 INFO:teuthology.orchestra.run.smithi029.stdout:osd.13: {} 2024-09-18T10:15:41.933 INFO:tasks.rados.rados.0.smithi189.stdout:324: finishing rollback tid 0 to smithi18925896-9 2024-09-18T10:15:41.944 INFO:tasks.rados.rados.0.smithi189.stdout:310: finishing write tid 2 to smithi18925896-6 2024-09-18T10:15:41.946 INFO:tasks.rados.rados.0.smithi189.stdout:310: finishing write tid 3 to smithi18925896-6 2024-09-18T10:15:41.947 INFO:tasks.rados.rados.0.smithi189.stdout:310: finishing write tid 4 to smithi18925896-6 2024-09-18T10:15:41.948 INFO:tasks.rados.rados.0.smithi189.stdout:310: finishing write tid 5 to smithi18925896-6 2024-09-18T10:15:41.973 INFO:tasks.ceph.osd.15.smithi189.stderr:2024-09-18T10:15:41.970+0000 7fd0e725e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 22401) UID: 0 2024-09-18T10:15:42.000 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:42.000 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:42.009 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T10:15:42.036 INFO:tasks.rados.rados.0.smithi189.stdout:324: finishing rollback tid 1 to smithi18925896-9 2024-09-18T10:15:42.037 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 9 v 51 (ObjNum 65 snap 0 seq_num 1834248542) dirty dne 2024-09-18T10:15:42.043 INFO:tasks.rados.rados.0.smithi189.stdout:319: expect (ObjNum 84 snap 14 seq_num 84) 2024-09-18T10:15:42.061 INFO:teuthology.orchestra.run.smithi029.stdout:osd.14: {} 2024-09-18T10:15:42.074 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-18T10:15:42.072+0000 7f14c6a1a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 22138) UID: 0 2024-09-18T10:15:42.083 INFO:teuthology.orchestra.run.smithi029.stdout:osd.15: {} 2024-09-18T10:15:42.086 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T10:15:42.086 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T10:15:42.086 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T10:15:42.086 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T10:15:42.086 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T10:15:42.086 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T10:15:42.086 INFO:teuthology.orchestra.run.smithi029.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T10:15:42.087 INFO:teuthology.orchestra.run.smithi029.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T10:15:42.087 INFO:teuthology.orchestra.run.smithi029.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-18T10:15:42.087 INFO:teuthology.orchestra.run.smithi029.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-18T10:15:42.087 INFO:teuthology.orchestra.run.smithi029.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-18T10:15:42.087 INFO:teuthology.orchestra.run.smithi029.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-18T10:15:42.087 INFO:teuthology.orchestra.run.smithi029.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-18T10:15:42.087 INFO:teuthology.orchestra.run.smithi029.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-18T10:15:42.087 INFO:teuthology.orchestra.run.smithi029.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-18T10:15:42.087 INFO:teuthology.orchestra.run.smithi029.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-18T10:15:42.107 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:42.107 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:42.117 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T10:15:42.133 INFO:tasks.rados.rados.0.smithi189.stdout:320: finishing rollback tid 0 to smithi18925896-33 2024-09-18T10:15:42.141 INFO:tasks.rados.rados.0.smithi189.stdout:315: finishing write tid 2 to smithi18925896-41 2024-09-18T10:15:42.175 INFO:tasks.ceph.osd.14.smithi154.stderr:2024-09-18T10:15:42.170+0000 7f75a19e7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 22093) UID: 0 2024-09-18T10:15:42.215 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:42.215 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:42.224 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T10:15:42.244 INFO:tasks.rados.rados.0.smithi189.stdout:315: finishing write tid 3 to smithi18925896-41 2024-09-18T10:15:42.253 INFO:tasks.rados.rados.0.smithi189.stdout:310: finishing write tid 6 to smithi18925896-6 2024-09-18T10:15:42.254 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 6 v 122 (ObjNum 124 snap 27 seq_num 124) dirty exists 2024-09-18T10:15:42.254 INFO:tasks.rados.rados.0.smithi189.stdout:310: left oid 6 (ObjNum 124 snap 27 seq_num 124) 2024-09-18T10:15:42.260 INFO:tasks.rados.rados.0.smithi189.stdout:318: finishing copy_from racing read to smithi18925896-44 2024-09-18T10:15:42.276 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-18T10:15:42.272+0000 7f00046a8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 22136) UID: 0 2024-09-18T10:15:42.334 INFO:tasks.rados.rados.0.smithi189.stdout:320: finishing rollback tid 1 to smithi18925896-33 2024-09-18T10:15:42.364 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:42.364 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:42.373 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T10:15:42.376 INFO:tasks.ceph.osd.3.smithi189.stderr:2024-09-18T10:15:42.370+0000 7f2440fc7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22396) UID: 0 2024-09-18T10:15:42.434 INFO:tasks.rados.rados.0.smithi189.stdout:320: finishing rollback tid 2 to smithi18925896-33 2024-09-18T10:15:42.435 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 33 v 67 (ObjNum 15 snap 0 seq_num 15) dirty exists 2024-09-18T10:15:42.455 INFO:tasks.rados.rados.0.smithi189.stdout:315: finishing write tid 4 to smithi18925896-41 2024-09-18T10:15:42.455 INFO:tasks.rados.rados.0.smithi189.stdout:327: finishing rollback tid 1 to smithi18925896-18 2024-09-18T10:15:42.455 INFO:tasks.rados.rados.0.smithi189.stdout:327: finishing rollback tid 2 to smithi18925896-18 2024-09-18T10:15:42.456 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 18 v 46 (ObjNum 17 snap 0 seq_num 17) dirty exists 2024-09-18T10:15:42.456 INFO:tasks.rados.rados.0.smithi189.stdout:315: finishing write tid 5 to smithi18925896-41 2024-09-18T10:15:42.463 INFO:tasks.rados.rados.0.smithi189.stdout:318: finishing copy_from to smithi18925896-44 2024-09-18T10:15:42.464 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 44 v 33 (ObjNum 90 snap 16 seq_num 90) dirty exists 2024-09-18T10:15:42.476 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:42.477 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:42.477 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-18T10:15:42.472+0000 7f00046a8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 22136) UID: 0 2024-09-18T10:15:42.486 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-09-18T10:15:42.538 INFO:tasks.rados.rados.0.smithi189.stdout:323: finishing rollback tid 0 to smithi18925896-25 2024-09-18T10:15:42.538 INFO:tasks.rados.rados.0.smithi189.stdout:323: finishing rollback tid 1 to smithi18925896-25 2024-09-18T10:15:42.538 INFO:tasks.rados.rados.0.smithi189.stdout:323: finishing rollback tid 2 to smithi18925896-25 2024-09-18T10:15:42.538 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 25 v 69 (ObjNum 81 snap 13 seq_num 81) dirty exists 2024-09-18T10:15:42.548 INFO:tasks.rados.rados.0.smithi189.stdout:325: finishing write tid 1 to smithi18925896-42 2024-09-18T10:15:42.548 INFO:tasks.rados.rados.0.smithi189.stdout:325: finishing write tid 2 to smithi18925896-42 2024-09-18T10:15:42.548 INFO:tasks.rados.rados.0.smithi189.stdout:325: finishing write tid 3 to smithi18925896-42 2024-09-18T10:15:42.556 INFO:tasks.rados.rados.0.smithi189.stdout:325: finishing write tid 4 to smithi18925896-42 2024-09-18T10:15:42.578 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-18T10:15:42.576+0000 7f578c52f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22134) UID: 0 2024-09-18T10:15:42.588 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:42.588 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:42.597 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-09-18T10:15:42.620 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7, 9, 10, 11, 12, 13, 14, 15] out_osds: [8] dead_osds: [] live_osds: [0, 4, 8, 12, 1, 5, 9, 13, 2, 6, 10, 14, 3, 7, 11, 15] 2024-09-18T10:15:42.620 INFO:tasks.thrashosds.thrasher:choose_action: min_in 8 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T10:15:42.620 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-18T10:15:42.621 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-18T10:15:42.623 INFO:tasks.rados.rados.0.smithi189.stdout:325: finishing write tid 5 to smithi18925896-42 2024-09-18T10:15:42.624 INFO:tasks.rados.rados.0.smithi189.stdout:310: done (10 left) 2024-09-18T10:15:42.624 INFO:tasks.rados.rados.0.smithi189.stdout:318: done (9 left) 2024-09-18T10:15:42.624 INFO:tasks.rados.rados.0.smithi189.stdout:319: done (8 left) 2024-09-18T10:15:42.624 INFO:tasks.rados.rados.0.smithi189.stdout:320: done (7 left) 2024-09-18T10:15:42.624 INFO:tasks.rados.rados.0.smithi189.stdout:323: done (6 left) 2024-09-18T10:15:42.625 INFO:tasks.rados.rados.0.smithi189.stdout:324: done (5 left) 2024-09-18T10:15:42.625 INFO:tasks.rados.rados.0.smithi189.stdout:326: done (4 left) 2024-09-18T10:15:42.625 INFO:tasks.rados.rados.0.smithi189.stdout:327: done (3 left) 2024-09-18T10:15:42.625 INFO:tasks.rados.rados.0.smithi189.stdout:328: done (2 left) 2024-09-18T10:15:42.625 INFO:tasks.rados.rados.0.smithi189.stdout:329: rollback oid 21 current snap is 28 2024-09-18T10:15:42.625 INFO:tasks.rados.rados.0.smithi189.stdout:rollback oid 21 to 22 2024-09-18T10:15:42.625 INFO:tasks.rados.rados.0.smithi189.stdout:330: rmattr oid 49 current snap is 28 2024-09-18T10:15:42.625 INFO:tasks.rados.rados.0.smithi189.stdout:330: done (3 left) 2024-09-18T10:15:42.625 INFO:tasks.rados.rados.0.smithi189.stdout:append oid 38 current snap is 28 2024-09-18T10:15:42.625 INFO:tasks.rados.rados.0.smithi189.stdout:331: seq_num 130 ranges {0=638976,638976=671744,1310720=802816,2113536=81920} 2024-09-18T10:15:42.627 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-18T10:15:42.634 INFO:tasks.rados.rados.0.smithi189.stdout:331: writing smithi18925896-38 from 0 to 638976 tid 1 2024-09-18T10:15:42.638 INFO:tasks.rados.rados.0.smithi189.stdout:331: writing smithi18925896-38 from 638976 to 1310720 tid 2 2024-09-18T10:15:42.643 INFO:tasks.rados.rados.0.smithi189.stdout:331: writing smithi18925896-38 from 1310720 to 2113536 tid 3 2024-09-18T10:15:42.644 INFO:tasks.rados.rados.0.smithi189.stdout:331: writing smithi18925896-38 from 2113536 to 2195456 tid 4 2024-09-18T10:15:42.644 INFO:tasks.rados.rados.0.smithi189.stdout:332: copy_from oid 49 from oid 35 current snap is 28 2024-09-18T10:15:42.644 INFO:tasks.rados.rados.0.smithi189.stdout:333: rmattr oid 46 current snap is 28 2024-09-18T10:15:42.644 INFO:tasks.rados.rados.0.smithi189.stdout:333: done (5 left) 2024-09-18T10:15:42.644 INFO:tasks.rados.rados.0.smithi189.stdout:append oid 28 current snap is 28 2024-09-18T10:15:42.644 INFO:tasks.rados.rados.0.smithi189.stdout:334: seq_num 131 ranges {557056=655360,1212416=425984,1638400=638976,2277376=491520} 2024-09-18T10:15:42.650 INFO:tasks.rados.rados.0.smithi189.stdout:334: writing smithi18925896-28 from 557056 to 1212416 tid 1 2024-09-18T10:15:42.653 INFO:tasks.rados.rados.0.smithi189.stdout:334: writing smithi18925896-28 from 1212416 to 1638400 tid 2 2024-09-18T10:15:42.657 INFO:tasks.rados.rados.0.smithi189.stdout:334: writing smithi18925896-28 from 1638400 to 2277376 tid 3 2024-09-18T10:15:42.660 INFO:tasks.rados.rados.0.smithi189.stdout:334: writing smithi18925896-28 from 2277376 to 2768896 tid 4 2024-09-18T10:15:42.660 INFO:tasks.rados.rados.0.smithi189.stdout:append oid 23 current snap is 28 2024-09-18T10:15:42.660 INFO:tasks.rados.rados.0.smithi189.stdout:335: seq_num 132 ranges {3342336=671744,4014080=573440,4587520=753664,5341184=229376} 2024-09-18T10:15:42.679 INFO:tasks.ceph.osd.8.smithi029.stderr:2024-09-18T10:15:42.673+0000 7f2555c27700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 22192) UID: 0 2024-09-18T10:15:42.682 INFO:tasks.rados.rados.0.smithi189.stdout:335: writing smithi18925896-23 from 3342336 to 4014080 tid 1 2024-09-18T10:15:42.685 INFO:tasks.rados.rados.0.smithi189.stdout:335: writing smithi18925896-23 from 4014080 to 4587520 tid 2 2024-09-18T10:15:42.689 INFO:tasks.rados.rados.0.smithi189.stdout:335: writing smithi18925896-23 from 4587520 to 5341184 tid 3 2024-09-18T10:15:42.691 INFO:tasks.rados.rados.0.smithi189.stdout:335: writing smithi18925896-23 from 5341184 to 5570560 tid 4 2024-09-18T10:15:42.691 INFO:tasks.rados.rados.0.smithi189.stdout:336: copy_from oid 40 from oid 37 current snap is 28 2024-09-18T10:15:42.691 INFO:tasks.rados.rados.0.smithi189.stdout:append oid 31 current snap is 28 2024-09-18T10:15:42.691 INFO:tasks.rados.rados.0.smithi189.stdout:337: seq_num 133 ranges {1933312=688128,2621440=720896,3342336=475136,3817472=360448} 2024-09-18T10:15:42.696 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:42.696 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:42.706 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-09-18T10:15:42.706 INFO:tasks.rados.rados.0.smithi189.stdout:337: writing smithi18925896-31 from 1933312 to 2621440 tid 1 2024-09-18T10:15:42.710 INFO:tasks.rados.rados.0.smithi189.stdout:337: writing smithi18925896-31 from 2621440 to 3342336 tid 2 2024-09-18T10:15:42.712 INFO:tasks.rados.rados.0.smithi189.stdout:337: writing smithi18925896-31 from 3342336 to 3817472 tid 3 2024-09-18T10:15:42.714 INFO:tasks.rados.rados.0.smithi189.stdout:337: writing smithi18925896-31 from 3817472 to 4177920 tid 4 2024-09-18T10:15:42.714 INFO:tasks.rados.rados.0.smithi189.stdout:338: snap_remove snap 23 2024-09-18T10:15:42.780 INFO:tasks.ceph.osd.14.smithi154.stderr:2024-09-18T10:15:42.778+0000 7f75a19e7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 22093) UID: 0 2024-09-18T10:15:42.812 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:42.813 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:42.822 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_ops_in_flight 2024-09-18T10:15:42.881 INFO:tasks.ceph.osd.11.smithi189.stderr:2024-09-18T10:15:42.878+0000 7f718b60a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 22400) UID: 0 2024-09-18T10:15:42.922 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-18T10:15:42.922 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-18T10:15:42.923 INFO:teuthology.orchestra.run.smithi029.stdout: "epoch": 87, 2024-09-18T10:15:42.923 INFO:teuthology.orchestra.run.smithi029.stdout: "fsid": "ab6d6beb-0627-4b7f-8cdb-4e0a44970f10", 2024-09-18T10:15:42.923 INFO:teuthology.orchestra.run.smithi029.stdout: "created": "2024-09-18T10:14:18.827644+0000", 2024-09-18T10:15:42.923 INFO:teuthology.orchestra.run.smithi029.stdout: "modified": "2024-09-18T10:15:42.616205+0000", 2024-09-18T10:15:42.923 INFO:teuthology.orchestra.run.smithi029.stdout: "last_up_change": "2024-09-18T10:14:30.228591+0000", 2024-09-18T10:15:42.923 INFO:teuthology.orchestra.run.smithi029.stdout: "last_in_change": "2024-09-18T10:15:37.582801+0000", 2024-09-18T10:15:42.923 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-18T10:15:42.923 INFO:teuthology.orchestra.run.smithi029.stdout: "flags_num": 5799936, 2024-09-18T10:15:42.923 INFO:teuthology.orchestra.run.smithi029.stdout: "flags_set": [ 2024-09-18T10:15:42.923 INFO:teuthology.orchestra.run.smithi029.stdout: "pglog_hardlimit", 2024-09-18T10:15:42.923 INFO:teuthology.orchestra.run.smithi029.stdout: "purged_snapdirs", 2024-09-18T10:15:42.923 INFO:teuthology.orchestra.run.smithi029.stdout: "recovery_deletes", 2024-09-18T10:15:42.924 INFO:teuthology.orchestra.run.smithi029.stdout: "sortbitwise" 2024-09-18T10:15:42.924 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-18T10:15:42.924 INFO:teuthology.orchestra.run.smithi029.stdout: "crush_version": 6, 2024-09-18T10:15:42.924 INFO:teuthology.orchestra.run.smithi029.stdout: "full_ratio": 0.94999998807907104, 2024-09-18T10:15:42.924 INFO:teuthology.orchestra.run.smithi029.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-18T10:15:42.924 INFO:teuthology.orchestra.run.smithi029.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-18T10:15:42.924 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_snapshot": "", 2024-09-18T10:15:42.924 INFO:teuthology.orchestra.run.smithi029.stdout: "pool_max": 3, 2024-09-18T10:15:42.924 INFO:teuthology.orchestra.run.smithi029.stdout: "max_osd": 16, 2024-09-18T10:15:42.924 INFO:teuthology.orchestra.run.smithi029.stdout: "require_min_compat_client": "luminous", 2024-09-18T10:15:42.924 INFO:teuthology.orchestra.run.smithi029.stdout: "min_compat_client": "luminous", 2024-09-18T10:15:42.924 INFO:teuthology.orchestra.run.smithi029.stdout: "require_osd_release": "quincy", 2024-09-18T10:15:42.925 INFO:teuthology.orchestra.run.smithi029.stdout: "pools": [ 2024-09-18T10:15:42.925 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:42.925 INFO:teuthology.orchestra.run.smithi029.stdout: "pool": 1, 2024-09-18T10:15:42.925 INFO:teuthology.orchestra.run.smithi029.stdout: "pool_name": ".mgr", 2024-09-18T10:15:42.925 INFO:teuthology.orchestra.run.smithi029.stdout: "create_time": "2024-09-18T10:14:31.266113+0000", 2024-09-18T10:15:42.925 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 1, 2024-09-18T10:15:42.925 INFO:teuthology.orchestra.run.smithi029.stdout: "flags_names": "hashpspool", 2024-09-18T10:15:42.925 INFO:teuthology.orchestra.run.smithi029.stdout: "type": 1, 2024-09-18T10:15:42.925 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 2, 2024-09-18T10:15:42.925 INFO:teuthology.orchestra.run.smithi029.stdout: "min_size": 1, 2024-09-18T10:15:42.925 INFO:teuthology.orchestra.run.smithi029.stdout: "crush_rule": 0, 2024-09-18T10:15:42.925 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_count": 0, 2024-09-18T10:15:42.925 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_target": 0, 2024-09-18T10:15:42.926 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_barrier": 0, 2024-09-18T10:15:42.926 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-18T10:15:42.926 INFO:teuthology.orchestra.run.smithi029.stdout: "object_hash": 2, 2024-09-18T10:15:42.926 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_autoscale_mode": "off", 2024-09-18T10:15:42.926 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num": 1, 2024-09-18T10:15:42.926 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_placement_num": 1, 2024-09-18T10:15:42.926 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_placement_num_target": 1, 2024-09-18T10:15:42.926 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num_target": 1, 2024-09-18T10:15:42.926 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num_pending": 1, 2024-09-18T10:15:42.926 INFO:teuthology.orchestra.run.smithi029.stdout: "last_pg_merge_meta": { 2024-09-18T10:15:42.926 INFO:teuthology.orchestra.run.smithi029.stdout: "source_pgid": "0.0", 2024-09-18T10:15:42.926 INFO:teuthology.orchestra.run.smithi029.stdout: "ready_epoch": 0, 2024-09-18T10:15:42.927 INFO:teuthology.orchestra.run.smithi029.stdout: "last_epoch_started": 0, 2024-09-18T10:15:42.927 INFO:teuthology.orchestra.run.smithi029.stdout: "last_epoch_clean": 0, 2024-09-18T10:15:42.927 INFO:teuthology.orchestra.run.smithi029.stdout: "source_version": "0'0", 2024-09-18T10:15:42.927 INFO:teuthology.orchestra.run.smithi029.stdout: "target_version": "0'0" 2024-09-18T10:15:42.927 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:42.927 INFO:teuthology.orchestra.run.smithi029.stdout: "last_change": "25", 2024-09-18T10:15:42.927 INFO:teuthology.orchestra.run.smithi029.stdout: "last_force_op_resend": "0", 2024-09-18T10:15:42.927 INFO:teuthology.orchestra.run.smithi029.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-18T10:15:42.927 INFO:teuthology.orchestra.run.smithi029.stdout: "last_force_op_resend_preluminous": "0", 2024-09-18T10:15:42.927 INFO:teuthology.orchestra.run.smithi029.stdout: "auid": 0, 2024-09-18T10:15:42.927 INFO:teuthology.orchestra.run.smithi029.stdout: "snap_mode": "selfmanaged", 2024-09-18T10:15:42.927 INFO:teuthology.orchestra.run.smithi029.stdout: "snap_seq": 0, 2024-09-18T10:15:42.928 INFO:teuthology.orchestra.run.smithi029.stdout: "snap_epoch": 0, 2024-09-18T10:15:42.928 INFO:teuthology.orchestra.run.smithi029.stdout: "pool_snaps": [], 2024-09-18T10:15:42.928 INFO:teuthology.orchestra.run.smithi029.stdout: "removed_snaps": "[]", 2024-09-18T10:15:42.928 INFO:teuthology.orchestra.run.smithi029.stdout: "quota_max_bytes": 0, 2024-09-18T10:15:42.928 INFO:teuthology.orchestra.run.smithi029.stdout: "quota_max_objects": 0, 2024-09-18T10:15:42.928 INFO:teuthology.orchestra.run.smithi029.stdout: "tiers": [], 2024-09-18T10:15:42.928 INFO:teuthology.orchestra.run.smithi029.stdout: "tier_of": -1, 2024-09-18T10:15:42.928 INFO:teuthology.orchestra.run.smithi029.stdout: "read_tier": -1, 2024-09-18T10:15:42.928 INFO:teuthology.orchestra.run.smithi029.stdout: "write_tier": -1, 2024-09-18T10:15:42.928 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_mode": "none", 2024-09-18T10:15:42.928 INFO:teuthology.orchestra.run.smithi029.stdout: "target_max_bytes": 0, 2024-09-18T10:15:42.928 INFO:teuthology.orchestra.run.smithi029.stdout: "target_max_objects": 0, 2024-09-18T10:15:42.928 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-18T10:15:42.929 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-18T10:15:42.929 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-18T10:15:42.929 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_min_flush_age": 0, 2024-09-18T10:15:42.929 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_min_evict_age": 0, 2024-09-18T10:15:42.929 INFO:teuthology.orchestra.run.smithi029.stdout: "erasure_code_profile": "", 2024-09-18T10:15:42.929 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_params": { 2024-09-18T10:15:42.929 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "none" 2024-09-18T10:15:42.929 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:42.929 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_period": 0, 2024-09-18T10:15:42.929 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_count": 0, 2024-09-18T10:15:42.929 INFO:teuthology.orchestra.run.smithi029.stdout: "use_gmt_hitset": true, 2024-09-18T10:15:42.929 INFO:teuthology.orchestra.run.smithi029.stdout: "min_read_recency_for_promote": 0, 2024-09-18T10:15:42.930 INFO:teuthology.orchestra.run.smithi029.stdout: "min_write_recency_for_promote": 0, 2024-09-18T10:15:42.930 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_grade_decay_rate": 0, 2024-09-18T10:15:42.930 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_search_last_n": 0, 2024-09-18T10:15:42.930 INFO:teuthology.orchestra.run.smithi029.stdout: "grade_table": [], 2024-09-18T10:15:42.930 INFO:teuthology.orchestra.run.smithi029.stdout: "stripe_width": 0, 2024-09-18T10:15:42.930 INFO:teuthology.orchestra.run.smithi029.stdout: "expected_num_objects": 0, 2024-09-18T10:15:42.930 INFO:teuthology.orchestra.run.smithi029.stdout: "fast_read": false, 2024-09-18T10:15:42.930 INFO:teuthology.orchestra.run.smithi029.stdout: "options": { 2024-09-18T10:15:42.930 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num_max": 32, 2024-09-18T10:15:42.930 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num_min": 1 2024-09-18T10:15:42.930 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:42.930 INFO:teuthology.orchestra.run.smithi029.stdout: "application_metadata": { 2024-09-18T10:15:42.930 INFO:teuthology.orchestra.run.smithi029.stdout: "mgr": {} 2024-09-18T10:15:42.931 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:42.931 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:42.931 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:42.931 INFO:teuthology.orchestra.run.smithi029.stdout: "pool": 2, 2024-09-18T10:15:42.931 INFO:teuthology.orchestra.run.smithi029.stdout: "pool_name": "rbd", 2024-09-18T10:15:42.931 INFO:teuthology.orchestra.run.smithi029.stdout: "create_time": "2024-09-18T10:14:33.358485+0000", 2024-09-18T10:15:42.931 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 8193, 2024-09-18T10:15:42.931 INFO:teuthology.orchestra.run.smithi029.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-18T10:15:42.931 INFO:teuthology.orchestra.run.smithi029.stdout: "type": 1, 2024-09-18T10:15:42.931 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 2, 2024-09-18T10:15:42.931 INFO:teuthology.orchestra.run.smithi029.stdout: "min_size": 1, 2024-09-18T10:15:42.931 INFO:teuthology.orchestra.run.smithi029.stdout: "crush_rule": 0, 2024-09-18T10:15:42.931 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_count": 0, 2024-09-18T10:15:42.932 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_target": 0, 2024-09-18T10:15:42.932 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_barrier": 0, 2024-09-18T10:15:42.932 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-18T10:15:42.932 INFO:teuthology.orchestra.run.smithi029.stdout: "object_hash": 2, 2024-09-18T10:15:42.932 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_autoscale_mode": "off", 2024-09-18T10:15:42.932 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num": 8, 2024-09-18T10:15:42.932 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_placement_num": 8, 2024-09-18T10:15:42.932 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_placement_num_target": 8, 2024-09-18T10:15:42.932 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num_target": 8, 2024-09-18T10:15:42.932 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num_pending": 8, 2024-09-18T10:15:42.932 INFO:teuthology.orchestra.run.smithi029.stdout: "last_pg_merge_meta": { 2024-09-18T10:15:42.932 INFO:teuthology.orchestra.run.smithi029.stdout: "source_pgid": "0.0", 2024-09-18T10:15:42.932 INFO:teuthology.orchestra.run.smithi029.stdout: "ready_epoch": 0, 2024-09-18T10:15:42.933 INFO:teuthology.orchestra.run.smithi029.stdout: "last_epoch_started": 0, 2024-09-18T10:15:42.933 INFO:teuthology.orchestra.run.smithi029.stdout: "last_epoch_clean": 0, 2024-09-18T10:15:42.933 INFO:teuthology.orchestra.run.smithi029.stdout: "source_version": "0'0", 2024-09-18T10:15:42.933 INFO:teuthology.orchestra.run.smithi029.stdout: "target_version": "0'0" 2024-09-18T10:15:42.933 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:42.933 INFO:teuthology.orchestra.run.smithi029.stdout: "last_change": "29", 2024-09-18T10:15:42.933 INFO:teuthology.orchestra.run.smithi029.stdout: "last_force_op_resend": "0", 2024-09-18T10:15:42.933 INFO:teuthology.orchestra.run.smithi029.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-18T10:15:42.933 INFO:teuthology.orchestra.run.smithi029.stdout: "last_force_op_resend_preluminous": "0", 2024-09-18T10:15:42.933 INFO:teuthology.orchestra.run.smithi029.stdout: "auid": 0, 2024-09-18T10:15:42.933 INFO:teuthology.orchestra.run.smithi029.stdout: "snap_mode": "selfmanaged", 2024-09-18T10:15:42.933 INFO:teuthology.orchestra.run.smithi029.stdout: "snap_seq": 3, 2024-09-18T10:15:42.933 INFO:teuthology.orchestra.run.smithi029.stdout: "snap_epoch": 29, 2024-09-18T10:15:42.934 INFO:teuthology.orchestra.run.smithi029.stdout: "pool_snaps": [], 2024-09-18T10:15:42.934 INFO:teuthology.orchestra.run.smithi029.stdout: "removed_snaps": "[]", 2024-09-18T10:15:42.934 INFO:teuthology.orchestra.run.smithi029.stdout: "quota_max_bytes": 0, 2024-09-18T10:15:42.934 INFO:teuthology.orchestra.run.smithi029.stdout: "quota_max_objects": 0, 2024-09-18T10:15:42.934 INFO:teuthology.orchestra.run.smithi029.stdout: "tiers": [], 2024-09-18T10:15:42.934 INFO:teuthology.orchestra.run.smithi029.stdout: "tier_of": -1, 2024-09-18T10:15:42.934 INFO:teuthology.orchestra.run.smithi029.stdout: "read_tier": -1, 2024-09-18T10:15:42.934 INFO:teuthology.orchestra.run.smithi029.stdout: "write_tier": -1, 2024-09-18T10:15:42.934 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_mode": "none", 2024-09-18T10:15:42.934 INFO:teuthology.orchestra.run.smithi029.stdout: "target_max_bytes": 0, 2024-09-18T10:15:42.934 INFO:teuthology.orchestra.run.smithi029.stdout: "target_max_objects": 0, 2024-09-18T10:15:42.934 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-18T10:15:42.935 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-18T10:15:42.935 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-18T10:15:42.935 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_min_flush_age": 0, 2024-09-18T10:15:42.935 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_min_evict_age": 0, 2024-09-18T10:15:42.935 INFO:teuthology.orchestra.run.smithi029.stdout: "erasure_code_profile": "", 2024-09-18T10:15:42.935 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_params": { 2024-09-18T10:15:42.935 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "none" 2024-09-18T10:15:42.935 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:42.935 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_period": 0, 2024-09-18T10:15:42.935 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_count": 0, 2024-09-18T10:15:42.935 INFO:teuthology.orchestra.run.smithi029.stdout: "use_gmt_hitset": true, 2024-09-18T10:15:42.936 INFO:teuthology.orchestra.run.smithi029.stdout: "min_read_recency_for_promote": 0, 2024-09-18T10:15:42.936 INFO:teuthology.orchestra.run.smithi029.stdout: "min_write_recency_for_promote": 0, 2024-09-18T10:15:42.936 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_grade_decay_rate": 0, 2024-09-18T10:15:42.936 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_search_last_n": 0, 2024-09-18T10:15:42.936 INFO:teuthology.orchestra.run.smithi029.stdout: "grade_table": [], 2024-09-18T10:15:42.936 INFO:teuthology.orchestra.run.smithi029.stdout: "stripe_width": 0, 2024-09-18T10:15:42.936 INFO:teuthology.orchestra.run.smithi029.stdout: "expected_num_objects": 0, 2024-09-18T10:15:42.936 INFO:teuthology.orchestra.run.smithi029.stdout: "fast_read": false, 2024-09-18T10:15:42.936 INFO:teuthology.orchestra.run.smithi029.stdout: "options": {}, 2024-09-18T10:15:42.936 INFO:teuthology.orchestra.run.smithi029.stdout: "application_metadata": { 2024-09-18T10:15:42.936 INFO:teuthology.orchestra.run.smithi029.stdout: "rbd": {} 2024-09-18T10:15:42.936 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:42.937 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:42.937 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:42.937 INFO:teuthology.orchestra.run.smithi029.stdout: "pool": 3, 2024-09-18T10:15:42.937 INFO:teuthology.orchestra.run.smithi029.stdout: "pool_name": "unique_pool_0", 2024-09-18T10:15:42.937 INFO:teuthology.orchestra.run.smithi029.stdout: "create_time": "2024-09-18T10:14:44.624320+0000", 2024-09-18T10:15:42.937 INFO:teuthology.orchestra.run.smithi029.stdout: "flags": 8193, 2024-09-18T10:15:42.938 INFO:teuthology.orchestra.run.smithi029.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-18T10:15:42.938 INFO:teuthology.orchestra.run.smithi029.stdout: "type": 3, 2024-09-18T10:15:42.938 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 7, 2024-09-18T10:15:42.938 INFO:teuthology.orchestra.run.smithi029.stdout: "min_size": 5, 2024-09-18T10:15:42.938 INFO:teuthology.orchestra.run.smithi029.stdout: "crush_rule": 1, 2024-09-18T10:15:42.938 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_count": 0, 2024-09-18T10:15:42.938 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_target": 0, 2024-09-18T10:15:42.938 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_barrier": 0, 2024-09-18T10:15:42.938 INFO:teuthology.orchestra.run.smithi029.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-18T10:15:42.939 INFO:teuthology.orchestra.run.smithi029.stdout: "object_hash": 2, 2024-09-18T10:15:42.939 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_autoscale_mode": "off", 2024-09-18T10:15:42.939 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num": 26, 2024-09-18T10:15:42.939 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_placement_num": 26, 2024-09-18T10:15:42.939 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_placement_num_target": 26, 2024-09-18T10:15:42.939 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num_target": 26, 2024-09-18T10:15:42.939 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_num_pending": 26, 2024-09-18T10:15:42.939 INFO:teuthology.orchestra.run.smithi029.stdout: "last_pg_merge_meta": { 2024-09-18T10:15:42.939 INFO:teuthology.orchestra.run.smithi029.stdout: "source_pgid": "0.0", 2024-09-18T10:15:42.939 INFO:teuthology.orchestra.run.smithi029.stdout: "ready_epoch": 0, 2024-09-18T10:15:42.939 INFO:teuthology.orchestra.run.smithi029.stdout: "last_epoch_started": 0, 2024-09-18T10:15:42.940 INFO:teuthology.orchestra.run.smithi029.stdout: "last_epoch_clean": 0, 2024-09-18T10:15:42.940 INFO:teuthology.orchestra.run.smithi029.stdout: "source_version": "0'0", 2024-09-18T10:15:42.940 INFO:teuthology.orchestra.run.smithi029.stdout: "target_version": "0'0" 2024-09-18T10:15:42.940 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:42.940 INFO:teuthology.orchestra.run.smithi029.stdout: "last_change": "87", 2024-09-18T10:15:42.940 INFO:teuthology.orchestra.run.smithi029.stdout: "last_force_op_resend": "0", 2024-09-18T10:15:42.941 INFO:teuthology.orchestra.run.smithi029.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-18T10:15:42.941 INFO:teuthology.orchestra.run.smithi029.stdout: "last_force_op_resend_preluminous": "40", 2024-09-18T10:15:42.941 INFO:teuthology.orchestra.run.smithi029.stdout: "auid": 0, 2024-09-18T10:15:42.941 INFO:teuthology.orchestra.run.smithi029.stdout: "snap_mode": "selfmanaged", 2024-09-18T10:15:42.941 INFO:teuthology.orchestra.run.smithi029.stdout: "snap_seq": 48, 2024-09-18T10:15:42.941 INFO:teuthology.orchestra.run.smithi029.stdout: "snap_epoch": 87, 2024-09-18T10:15:42.941 INFO:teuthology.orchestra.run.smithi029.stdout: "pool_snaps": [], 2024-09-18T10:15:42.941 INFO:teuthology.orchestra.run.smithi029.stdout: "removed_snaps": "[]", 2024-09-18T10:15:42.941 INFO:teuthology.orchestra.run.smithi029.stdout: "quota_max_bytes": 0, 2024-09-18T10:15:42.941 INFO:teuthology.orchestra.run.smithi029.stdout: "quota_max_objects": 0, 2024-09-18T10:15:42.941 INFO:teuthology.orchestra.run.smithi029.stdout: "tiers": [], 2024-09-18T10:15:42.941 INFO:teuthology.orchestra.run.smithi029.stdout: "tier_of": -1, 2024-09-18T10:15:42.941 INFO:teuthology.orchestra.run.smithi029.stdout: "read_tier": -1, 2024-09-18T10:15:42.942 INFO:teuthology.orchestra.run.smithi029.stdout: "write_tier": -1, 2024-09-18T10:15:42.942 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_mode": "none", 2024-09-18T10:15:42.942 INFO:teuthology.orchestra.run.smithi029.stdout: "target_max_bytes": 0, 2024-09-18T10:15:42.942 INFO:teuthology.orchestra.run.smithi029.stdout: "target_max_objects": 0, 2024-09-18T10:15:42.942 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-18T10:15:42.942 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-18T10:15:42.942 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-18T10:15:42.942 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_min_flush_age": 0, 2024-09-18T10:15:42.942 INFO:teuthology.orchestra.run.smithi029.stdout: "cache_min_evict_age": 0, 2024-09-18T10:15:42.942 INFO:teuthology.orchestra.run.smithi029.stdout: "erasure_code_profile": "shecprofile", 2024-09-18T10:15:42.942 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_params": { 2024-09-18T10:15:42.942 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "none" 2024-09-18T10:15:42.943 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:42.943 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_period": 0, 2024-09-18T10:15:42.943 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_count": 0, 2024-09-18T10:15:42.943 INFO:teuthology.orchestra.run.smithi029.stdout: "use_gmt_hitset": true, 2024-09-18T10:15:42.943 INFO:teuthology.orchestra.run.smithi029.stdout: "min_read_recency_for_promote": 0, 2024-09-18T10:15:42.943 INFO:teuthology.orchestra.run.smithi029.stdout: "min_write_recency_for_promote": 0, 2024-09-18T10:15:42.943 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_grade_decay_rate": 0, 2024-09-18T10:15:42.943 INFO:teuthology.orchestra.run.smithi029.stdout: "hit_set_search_last_n": 0, 2024-09-18T10:15:42.943 INFO:teuthology.orchestra.run.smithi029.stdout: "grade_table": [], 2024-09-18T10:15:42.943 INFO:teuthology.orchestra.run.smithi029.stdout: "stripe_width": 16384, 2024-09-18T10:15:42.943 INFO:teuthology.orchestra.run.smithi029.stdout: "expected_num_objects": 0, 2024-09-18T10:15:42.943 INFO:teuthology.orchestra.run.smithi029.stdout: "fast_read": false, 2024-09-18T10:15:42.943 INFO:teuthology.orchestra.run.smithi029.stdout: "options": {}, 2024-09-18T10:15:42.944 INFO:teuthology.orchestra.run.smithi029.stdout: "application_metadata": { 2024-09-18T10:15:42.944 INFO:teuthology.orchestra.run.smithi029.stdout: "rados": {} 2024-09-18T10:15:42.944 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:42.944 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:42.944 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-18T10:15:42.944 INFO:teuthology.orchestra.run.smithi029.stdout: "osds": [ 2024-09-18T10:15:42.944 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:42.944 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 0, 2024-09-18T10:15:42.944 INFO:teuthology.orchestra.run.smithi029.stdout: "uuid": "39cf0511-0009-433f-92f2-e5686f33d713", 2024-09-18T10:15:42.944 INFO:teuthology.orchestra.run.smithi029.stdout: "up": 1, 2024-09-18T10:15:42.944 INFO:teuthology.orchestra.run.smithi029.stdout: "in": 1, 2024-09-18T10:15:42.944 INFO:teuthology.orchestra.run.smithi029.stdout: "weight": 1, 2024-09-18T10:15:42.944 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_affinity": 1, 2024-09-18T10:15:42.945 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_begin": 0, 2024-09-18T10:15:42.945 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_end": 0, 2024-09-18T10:15:42.945 INFO:teuthology.orchestra.run.smithi029.stdout: "up_from": 21, 2024-09-18T10:15:42.945 INFO:teuthology.orchestra.run.smithi029.stdout: "up_thru": 40, 2024-09-18T10:15:42.945 INFO:teuthology.orchestra.run.smithi029.stdout: "down_at": 0, 2024-09-18T10:15:42.945 INFO:teuthology.orchestra.run.smithi029.stdout: "lost_at": 0, 2024-09-18T10:15:42.945 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addrs": { 2024-09-18T10:15:42.945 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:42.945 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:42.945 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:42.946 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6808", 2024-09-18T10:15:42.946 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22197 2024-09-18T10:15:42.946 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:42.946 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:42.946 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:42.946 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6809", 2024-09-18T10:15:42.946 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22197 2024-09-18T10:15:42.946 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:42.946 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:42.946 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:42.946 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addrs": { 2024-09-18T10:15:42.946 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:42.946 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:42.947 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:42.947 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6810", 2024-09-18T10:15:42.947 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22197 2024-09-18T10:15:42.947 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:42.947 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:42.947 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:42.947 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6811", 2024-09-18T10:15:42.947 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22197 2024-09-18T10:15:42.947 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:42.947 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:42.948 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:42.948 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addrs": { 2024-09-18T10:15:42.948 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:42.948 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:42.948 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:42.948 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6814", 2024-09-18T10:15:42.949 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22197 2024-09-18T10:15:42.949 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:42.949 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:42.949 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:42.949 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6815", 2024-09-18T10:15:42.949 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22197 2024-09-18T10:15:42.949 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:42.949 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:42.949 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:42.949 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addrs": { 2024-09-18T10:15:42.949 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:42.950 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:42.950 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:42.950 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6812", 2024-09-18T10:15:42.950 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22197 2024-09-18T10:15:42.950 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:42.950 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:42.951 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:42.951 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6813", 2024-09-18T10:15:42.951 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22197 2024-09-18T10:15:42.951 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:42.951 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:42.951 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:42.951 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addr": "172.21.15.29:6809/22197", 2024-09-18T10:15:42.951 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addr": "172.21.15.29:6811/22197", 2024-09-18T10:15:42.952 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addr": "172.21.15.29:6815/22197", 2024-09-18T10:15:42.952 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addr": "172.21.15.29:6813/22197", 2024-09-18T10:15:42.952 INFO:teuthology.orchestra.run.smithi029.stdout: "state": [ 2024-09-18T10:15:42.952 INFO:teuthology.orchestra.run.smithi029.stdout: "exists", 2024-09-18T10:15:42.952 INFO:teuthology.orchestra.run.smithi029.stdout: "up" 2024-09-18T10:15:42.952 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:42.952 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:42.952 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:42.952 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 1, 2024-09-18T10:15:42.952 INFO:teuthology.orchestra.run.smithi029.stdout: "uuid": "a0bf06ab-b98e-4130-91d2-7d4283540c08", 2024-09-18T10:15:42.952 INFO:teuthology.orchestra.run.smithi029.stdout: "up": 1, 2024-09-18T10:15:42.953 INFO:teuthology.orchestra.run.smithi029.stdout: "in": 1, 2024-09-18T10:15:42.953 INFO:teuthology.orchestra.run.smithi029.stdout: "weight": 1, 2024-09-18T10:15:42.953 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_affinity": 1, 2024-09-18T10:15:42.953 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_begin": 0, 2024-09-18T10:15:42.953 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_end": 0, 2024-09-18T10:15:42.953 INFO:teuthology.orchestra.run.smithi029.stdout: "up_from": 22, 2024-09-18T10:15:42.953 INFO:teuthology.orchestra.run.smithi029.stdout: "up_thru": 58, 2024-09-18T10:15:42.953 INFO:teuthology.orchestra.run.smithi029.stdout: "down_at": 0, 2024-09-18T10:15:42.953 INFO:teuthology.orchestra.run.smithi029.stdout: "lost_at": 0, 2024-09-18T10:15:42.953 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addrs": { 2024-09-18T10:15:42.953 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:42.953 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:42.954 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:42.954 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.136:6808", 2024-09-18T10:15:42.954 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22140 2024-09-18T10:15:42.954 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:42.954 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:42.954 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:42.954 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.136:6809", 2024-09-18T10:15:42.954 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22140 2024-09-18T10:15:42.954 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:42.954 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:42.954 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:42.955 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addrs": { 2024-09-18T10:15:42.955 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:42.955 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:42.955 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:42.955 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.136:6810", 2024-09-18T10:15:42.955 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22140 2024-09-18T10:15:42.955 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:42.955 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:42.955 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:42.955 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.136:6811", 2024-09-18T10:15:42.955 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22140 2024-09-18T10:15:42.955 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:42.955 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:42.956 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:42.956 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addrs": { 2024-09-18T10:15:42.956 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:42.956 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:42.956 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:42.956 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.136:6814", 2024-09-18T10:15:42.956 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22140 2024-09-18T10:15:42.956 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:42.956 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:42.956 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:42.956 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.136:6815", 2024-09-18T10:15:42.956 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22140 2024-09-18T10:15:42.956 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:42.957 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:42.957 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:42.957 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addrs": { 2024-09-18T10:15:42.957 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:42.957 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:42.957 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:42.957 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.136:6812", 2024-09-18T10:15:42.957 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22140 2024-09-18T10:15:42.957 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:42.957 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:42.957 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:42.957 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.136:6813", 2024-09-18T10:15:42.958 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22140 2024-09-18T10:15:42.958 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:42.958 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:42.958 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:42.958 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addr": "172.21.15.136:6809/22140", 2024-09-18T10:15:42.958 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addr": "172.21.15.136:6811/22140", 2024-09-18T10:15:42.958 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addr": "172.21.15.136:6815/22140", 2024-09-18T10:15:42.958 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addr": "172.21.15.136:6813/22140", 2024-09-18T10:15:42.958 INFO:teuthology.orchestra.run.smithi029.stdout: "state": [ 2024-09-18T10:15:42.958 INFO:teuthology.orchestra.run.smithi029.stdout: "exists", 2024-09-18T10:15:42.958 INFO:teuthology.orchestra.run.smithi029.stdout: "up" 2024-09-18T10:15:42.958 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:42.959 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:42.959 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:42.959 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 2, 2024-09-18T10:15:42.959 INFO:teuthology.orchestra.run.smithi029.stdout: "uuid": "742a786e-6129-4fbc-bb5b-e2e3c03c2d77", 2024-09-18T10:15:42.959 INFO:teuthology.orchestra.run.smithi029.stdout: "up": 1, 2024-09-18T10:15:42.959 INFO:teuthology.orchestra.run.smithi029.stdout: "in": 1, 2024-09-18T10:15:42.959 INFO:teuthology.orchestra.run.smithi029.stdout: "weight": 1, 2024-09-18T10:15:42.959 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_affinity": 1, 2024-09-18T10:15:42.959 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_begin": 0, 2024-09-18T10:15:42.959 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_end": 0, 2024-09-18T10:15:42.959 INFO:teuthology.orchestra.run.smithi029.stdout: "up_from": 22, 2024-09-18T10:15:42.959 INFO:teuthology.orchestra.run.smithi029.stdout: "up_thru": 0, 2024-09-18T10:15:42.959 INFO:teuthology.orchestra.run.smithi029.stdout: "down_at": 0, 2024-09-18T10:15:42.960 INFO:teuthology.orchestra.run.smithi029.stdout: "lost_at": 0, 2024-09-18T10:15:42.960 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addrs": { 2024-09-18T10:15:42.960 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:42.960 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:42.960 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:42.960 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.154:6824", 2024-09-18T10:15:42.960 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22095 2024-09-18T10:15:42.960 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:42.960 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:42.960 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:42.960 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.154:6825", 2024-09-18T10:15:42.961 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22095 2024-09-18T10:15:42.961 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:42.961 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:42.961 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:42.961 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addrs": { 2024-09-18T10:15:42.961 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:42.961 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:42.961 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:42.961 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.154:6826", 2024-09-18T10:15:42.961 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22095 2024-09-18T10:15:42.961 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:42.961 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:42.961 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:42.961 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.154:6827", 2024-09-18T10:15:42.962 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22095 2024-09-18T10:15:42.962 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:42.962 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:42.962 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:42.962 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addrs": { 2024-09-18T10:15:42.962 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:42.962 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:42.962 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:42.962 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.154:6830", 2024-09-18T10:15:42.962 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22095 2024-09-18T10:15:42.962 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:42.962 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:42.962 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:42.962 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.154:6831", 2024-09-18T10:15:42.962 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22095 2024-09-18T10:15:42.962 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:42.963 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:42.963 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:42.963 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addrs": { 2024-09-18T10:15:42.963 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:42.963 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:42.963 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:42.963 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.154:6828", 2024-09-18T10:15:42.963 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22095 2024-09-18T10:15:42.963 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:42.963 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:42.963 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:42.963 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.154:6829", 2024-09-18T10:15:42.963 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22095 2024-09-18T10:15:42.963 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:42.963 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:42.963 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:42.964 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addr": "172.21.15.154:6825/22095", 2024-09-18T10:15:42.964 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addr": "172.21.15.154:6827/22095", 2024-09-18T10:15:42.964 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addr": "172.21.15.154:6831/22095", 2024-09-18T10:15:42.964 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addr": "172.21.15.154:6829/22095", 2024-09-18T10:15:42.964 INFO:teuthology.orchestra.run.smithi029.stdout: "state": [ 2024-09-18T10:15:42.964 INFO:teuthology.orchestra.run.smithi029.stdout: "exists", 2024-09-18T10:15:42.964 INFO:teuthology.orchestra.run.smithi029.stdout: "up" 2024-09-18T10:15:42.964 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:42.964 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:42.964 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:42.964 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 3, 2024-09-18T10:15:42.964 INFO:teuthology.orchestra.run.smithi029.stdout: "uuid": "3aa77114-24ed-49ab-86ff-a0600794cbf8", 2024-09-18T10:15:42.964 INFO:teuthology.orchestra.run.smithi029.stdout: "up": 1, 2024-09-18T10:15:42.965 INFO:teuthology.orchestra.run.smithi029.stdout: "in": 1, 2024-09-18T10:15:42.965 INFO:teuthology.orchestra.run.smithi029.stdout: "weight": 1, 2024-09-18T10:15:42.965 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_affinity": 1, 2024-09-18T10:15:42.965 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_begin": 0, 2024-09-18T10:15:42.965 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_end": 0, 2024-09-18T10:15:42.965 INFO:teuthology.orchestra.run.smithi029.stdout: "up_from": 22, 2024-09-18T10:15:42.965 INFO:teuthology.orchestra.run.smithi029.stdout: "up_thru": 48, 2024-09-18T10:15:42.965 INFO:teuthology.orchestra.run.smithi029.stdout: "down_at": 0, 2024-09-18T10:15:42.965 INFO:teuthology.orchestra.run.smithi029.stdout: "lost_at": 0, 2024-09-18T10:15:42.965 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addrs": { 2024-09-18T10:15:42.965 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:42.965 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:42.966 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:42.966 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.189:6802", 2024-09-18T10:15:42.966 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22404 2024-09-18T10:15:42.966 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:42.966 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:42.966 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:42.966 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.189:6803", 2024-09-18T10:15:42.966 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22404 2024-09-18T10:15:42.966 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:42.966 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:42.966 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:42.966 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addrs": { 2024-09-18T10:15:42.966 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:42.966 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:42.967 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:42.967 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.189:6804", 2024-09-18T10:15:42.967 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22404 2024-09-18T10:15:42.967 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:42.967 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:42.967 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:42.967 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.189:6805", 2024-09-18T10:15:42.967 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22404 2024-09-18T10:15:42.967 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:42.967 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:42.967 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:42.967 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addrs": { 2024-09-18T10:15:42.968 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:42.968 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:42.968 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:42.968 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.189:6808", 2024-09-18T10:15:42.968 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22404 2024-09-18T10:15:42.968 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:42.968 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:42.968 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:42.968 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.189:6809", 2024-09-18T10:15:42.968 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22404 2024-09-18T10:15:42.968 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:42.969 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:42.969 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:42.969 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addrs": { 2024-09-18T10:15:42.969 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:42.969 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:42.969 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:42.969 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.189:6806", 2024-09-18T10:15:42.969 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22404 2024-09-18T10:15:42.969 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:42.969 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:42.969 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:42.969 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.189:6807", 2024-09-18T10:15:42.969 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22404 2024-09-18T10:15:42.970 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:42.970 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:42.970 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:42.970 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addr": "172.21.15.189:6803/22404", 2024-09-18T10:15:42.970 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addr": "172.21.15.189:6805/22404", 2024-09-18T10:15:42.970 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addr": "172.21.15.189:6809/22404", 2024-09-18T10:15:42.970 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addr": "172.21.15.189:6807/22404", 2024-09-18T10:15:42.970 INFO:teuthology.orchestra.run.smithi029.stdout: "state": [ 2024-09-18T10:15:42.970 INFO:teuthology.orchestra.run.smithi029.stdout: "exists", 2024-09-18T10:15:42.970 INFO:teuthology.orchestra.run.smithi029.stdout: "up" 2024-09-18T10:15:42.970 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:42.970 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:42.971 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:42.971 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 4, 2024-09-18T10:15:42.971 INFO:teuthology.orchestra.run.smithi029.stdout: "uuid": "426567ad-b0be-41d4-afcf-dc56ec3974cd", 2024-09-18T10:15:42.971 INFO:teuthology.orchestra.run.smithi029.stdout: "up": 1, 2024-09-18T10:15:42.971 INFO:teuthology.orchestra.run.smithi029.stdout: "in": 1, 2024-09-18T10:15:42.971 INFO:teuthology.orchestra.run.smithi029.stdout: "weight": 1, 2024-09-18T10:15:42.971 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_affinity": 1, 2024-09-18T10:15:42.971 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_begin": 0, 2024-09-18T10:15:42.971 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_end": 0, 2024-09-18T10:15:42.971 INFO:teuthology.orchestra.run.smithi029.stdout: "up_from": 21, 2024-09-18T10:15:42.971 INFO:teuthology.orchestra.run.smithi029.stdout: "up_thru": 83, 2024-09-18T10:15:42.971 INFO:teuthology.orchestra.run.smithi029.stdout: "down_at": 0, 2024-09-18T10:15:42.972 INFO:teuthology.orchestra.run.smithi029.stdout: "lost_at": 0, 2024-09-18T10:15:42.972 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addrs": { 2024-09-18T10:15:42.972 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:42.972 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:42.972 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:42.972 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6800", 2024-09-18T10:15:42.972 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22195 2024-09-18T10:15:42.972 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:42.972 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:42.972 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:42.972 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6801", 2024-09-18T10:15:42.972 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22195 2024-09-18T10:15:42.973 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:42.973 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:42.973 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:42.973 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addrs": { 2024-09-18T10:15:42.973 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:42.973 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:42.973 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:42.973 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6802", 2024-09-18T10:15:42.973 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22195 2024-09-18T10:15:42.973 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:42.973 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:42.974 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:42.974 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6803", 2024-09-18T10:15:42.974 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22195 2024-09-18T10:15:42.974 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:42.974 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:42.974 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:42.974 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addrs": { 2024-09-18T10:15:42.974 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:42.974 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:42.974 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:42.974 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6806", 2024-09-18T10:15:42.974 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22195 2024-09-18T10:15:42.974 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:42.975 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:42.975 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:42.975 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6807", 2024-09-18T10:15:42.975 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22195 2024-09-18T10:15:42.975 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:42.975 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:42.975 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:42.975 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addrs": { 2024-09-18T10:15:42.975 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:42.975 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:42.975 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:42.975 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6804", 2024-09-18T10:15:42.976 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22195 2024-09-18T10:15:42.976 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:42.976 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:42.976 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:42.976 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6805", 2024-09-18T10:15:42.976 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22195 2024-09-18T10:15:42.976 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:42.976 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:42.976 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:42.976 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addr": "172.21.15.29:6801/22195", 2024-09-18T10:15:42.976 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addr": "172.21.15.29:6803/22195", 2024-09-18T10:15:42.976 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addr": "172.21.15.29:6807/22195", 2024-09-18T10:15:42.977 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addr": "172.21.15.29:6805/22195", 2024-09-18T10:15:42.977 INFO:teuthology.orchestra.run.smithi029.stdout: "state": [ 2024-09-18T10:15:42.977 INFO:teuthology.orchestra.run.smithi029.stdout: "exists", 2024-09-18T10:15:42.977 INFO:teuthology.orchestra.run.smithi029.stdout: "up" 2024-09-18T10:15:42.977 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:42.977 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:42.977 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:42.977 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 5, 2024-09-18T10:15:42.977 INFO:teuthology.orchestra.run.smithi029.stdout: "uuid": "d9559e42-1f4e-4373-8392-a536a0afd871", 2024-09-18T10:15:42.977 INFO:teuthology.orchestra.run.smithi029.stdout: "up": 1, 2024-09-18T10:15:42.978 INFO:teuthology.orchestra.run.smithi029.stdout: "in": 1, 2024-09-18T10:15:42.978 INFO:teuthology.orchestra.run.smithi029.stdout: "weight": 1, 2024-09-18T10:15:42.978 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_affinity": 1, 2024-09-18T10:15:42.978 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_begin": 0, 2024-09-18T10:15:42.978 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_end": 0, 2024-09-18T10:15:42.978 INFO:teuthology.orchestra.run.smithi029.stdout: "up_from": 22, 2024-09-18T10:15:42.978 INFO:teuthology.orchestra.run.smithi029.stdout: "up_thru": 83, 2024-09-18T10:15:42.978 INFO:teuthology.orchestra.run.smithi029.stdout: "down_at": 0, 2024-09-18T10:15:42.978 INFO:teuthology.orchestra.run.smithi029.stdout: "lost_at": 0, 2024-09-18T10:15:42.978 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addrs": { 2024-09-18T10:15:42.979 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:42.979 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:42.979 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:42.979 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.136:6817", 2024-09-18T10:15:42.979 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22141 2024-09-18T10:15:42.979 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:42.979 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:42.979 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:42.979 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.136:6819", 2024-09-18T10:15:42.979 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22141 2024-09-18T10:15:42.979 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:42.979 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:42.979 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:42.980 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addrs": { 2024-09-18T10:15:42.980 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:42.980 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:42.980 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:42.980 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.136:6821", 2024-09-18T10:15:42.980 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22141 2024-09-18T10:15:42.980 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:42.980 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:42.980 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:42.981 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.136:6823", 2024-09-18T10:15:42.981 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22141 2024-09-18T10:15:42.981 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:42.981 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:42.981 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:42.981 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addrs": { 2024-09-18T10:15:42.981 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:42.981 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:42.981 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:42.981 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.136:6829", 2024-09-18T10:15:42.981 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22141 2024-09-18T10:15:42.981 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:42.982 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:42.982 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:42.982 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.136:6831", 2024-09-18T10:15:42.982 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22141 2024-09-18T10:15:42.982 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:42.982 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:42.982 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:42.982 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addrs": { 2024-09-18T10:15:42.982 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:42.982 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:42.982 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:42.982 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.136:6825", 2024-09-18T10:15:42.983 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22141 2024-09-18T10:15:42.983 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:42.983 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:42.983 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:42.983 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.136:6827", 2024-09-18T10:15:42.983 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22141 2024-09-18T10:15:42.983 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:42.983 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:42.983 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:42.984 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addr": "172.21.15.136:6819/22141", 2024-09-18T10:15:42.984 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addr": "172.21.15.136:6823/22141", 2024-09-18T10:15:42.984 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addr": "172.21.15.136:6831/22141", 2024-09-18T10:15:42.984 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addr": "172.21.15.136:6827/22141", 2024-09-18T10:15:42.984 INFO:teuthology.orchestra.run.smithi029.stdout: "state": [ 2024-09-18T10:15:42.984 INFO:teuthology.orchestra.run.smithi029.stdout: "exists", 2024-09-18T10:15:42.984 INFO:teuthology.orchestra.run.smithi029.stdout: "up" 2024-09-18T10:15:42.984 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:42.984 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:42.984 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:42.984 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 6, 2024-09-18T10:15:42.984 INFO:teuthology.orchestra.run.smithi029.stdout: "uuid": "d50ef1e2-8bc5-4a90-9be3-365a0db2be6f", 2024-09-18T10:15:42.984 INFO:teuthology.orchestra.run.smithi029.stdout: "up": 1, 2024-09-18T10:15:42.985 INFO:teuthology.orchestra.run.smithi029.stdout: "in": 1, 2024-09-18T10:15:42.985 INFO:teuthology.orchestra.run.smithi029.stdout: "weight": 1, 2024-09-18T10:15:42.985 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_affinity": 1, 2024-09-18T10:15:42.985 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_begin": 0, 2024-09-18T10:15:42.985 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_end": 0, 2024-09-18T10:15:42.985 INFO:teuthology.orchestra.run.smithi029.stdout: "up_from": 22, 2024-09-18T10:15:42.985 INFO:teuthology.orchestra.run.smithi029.stdout: "up_thru": 82, 2024-09-18T10:15:42.985 INFO:teuthology.orchestra.run.smithi029.stdout: "down_at": 0, 2024-09-18T10:15:42.985 INFO:teuthology.orchestra.run.smithi029.stdout: "lost_at": 0, 2024-09-18T10:15:42.985 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addrs": { 2024-09-18T10:15:42.985 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:42.985 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:42.985 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:42.985 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.154:6800", 2024-09-18T10:15:42.986 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22096 2024-09-18T10:15:42.986 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:42.986 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:42.986 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:42.986 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.154:6801", 2024-09-18T10:15:42.986 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22096 2024-09-18T10:15:42.986 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:42.986 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:42.986 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:42.986 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addrs": { 2024-09-18T10:15:42.986 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:42.986 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:42.986 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:42.987 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.154:6802", 2024-09-18T10:15:42.987 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22096 2024-09-18T10:15:42.987 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:42.987 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:42.987 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:42.987 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.154:6803", 2024-09-18T10:15:42.987 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22096 2024-09-18T10:15:42.987 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:42.987 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:42.987 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:42.987 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addrs": { 2024-09-18T10:15:42.987 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:42.987 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:42.988 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:42.988 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.154:6806", 2024-09-18T10:15:42.988 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22096 2024-09-18T10:15:42.988 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:42.988 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:42.988 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:42.988 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.154:6807", 2024-09-18T10:15:42.988 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22096 2024-09-18T10:15:42.988 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:42.988 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:42.988 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:42.988 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addrs": { 2024-09-18T10:15:42.989 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:42.989 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:42.989 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:42.989 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.154:6804", 2024-09-18T10:15:42.989 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22096 2024-09-18T10:15:42.989 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:42.989 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:42.989 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:42.989 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.154:6805", 2024-09-18T10:15:42.989 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22096 2024-09-18T10:15:42.989 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:42.989 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:42.989 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:42.989 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addr": "172.21.15.154:6801/22096", 2024-09-18T10:15:42.990 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addr": "172.21.15.154:6803/22096", 2024-09-18T10:15:42.990 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addr": "172.21.15.154:6807/22096", 2024-09-18T10:15:42.990 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addr": "172.21.15.154:6805/22096", 2024-09-18T10:15:42.990 INFO:teuthology.orchestra.run.smithi029.stdout: "state": [ 2024-09-18T10:15:42.990 INFO:teuthology.orchestra.run.smithi029.stdout: "exists", 2024-09-18T10:15:42.990 INFO:teuthology.orchestra.run.smithi029.stdout: "up" 2024-09-18T10:15:42.990 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:42.990 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:42.990 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:42.990 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 7, 2024-09-18T10:15:42.990 INFO:teuthology.orchestra.run.smithi029.stdout: "uuid": "bd0b85e4-cfee-4d19-90a5-79854794b48b", 2024-09-18T10:15:42.990 INFO:teuthology.orchestra.run.smithi029.stdout: "up": 1, 2024-09-18T10:15:42.990 INFO:teuthology.orchestra.run.smithi029.stdout: "in": 1, 2024-09-18T10:15:42.990 INFO:teuthology.orchestra.run.smithi029.stdout: "weight": 1, 2024-09-18T10:15:42.991 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_affinity": 1, 2024-09-18T10:15:42.991 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_begin": 0, 2024-09-18T10:15:42.991 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_end": 0, 2024-09-18T10:15:42.991 INFO:teuthology.orchestra.run.smithi029.stdout: "up_from": 22, 2024-09-18T10:15:42.991 INFO:teuthology.orchestra.run.smithi029.stdout: "up_thru": 82, 2024-09-18T10:15:42.991 INFO:teuthology.orchestra.run.smithi029.stdout: "down_at": 0, 2024-09-18T10:15:42.991 INFO:teuthology.orchestra.run.smithi029.stdout: "lost_at": 0, 2024-09-18T10:15:42.991 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addrs": { 2024-09-18T10:15:42.991 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:42.991 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:42.991 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:42.991 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.189:6818", 2024-09-18T10:15:42.992 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22405 2024-09-18T10:15:42.992 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:42.992 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:42.992 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:42.992 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.189:6819", 2024-09-18T10:15:42.992 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22405 2024-09-18T10:15:42.992 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:42.992 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:42.992 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:42.993 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addrs": { 2024-09-18T10:15:42.993 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:42.993 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:42.993 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:42.993 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.189:6820", 2024-09-18T10:15:42.993 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22405 2024-09-18T10:15:42.993 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:42.993 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:42.993 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:42.993 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.189:6821", 2024-09-18T10:15:42.993 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22405 2024-09-18T10:15:42.993 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:42.994 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:42.994 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:42.994 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addrs": { 2024-09-18T10:15:42.994 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:42.994 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:42.994 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:42.994 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.189:6824", 2024-09-18T10:15:42.994 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22405 2024-09-18T10:15:42.994 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:42.994 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:42.994 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:42.994 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.189:6825", 2024-09-18T10:15:42.994 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22405 2024-09-18T10:15:42.995 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:42.995 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:42.995 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:42.995 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addrs": { 2024-09-18T10:15:42.995 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:42.995 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:42.995 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:42.995 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.189:6822", 2024-09-18T10:15:42.995 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22405 2024-09-18T10:15:42.995 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:42.995 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:42.996 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:42.996 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.189:6823", 2024-09-18T10:15:42.996 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22405 2024-09-18T10:15:42.996 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:42.996 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:42.996 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:42.996 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addr": "172.21.15.189:6819/22405", 2024-09-18T10:15:42.996 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addr": "172.21.15.189:6821/22405", 2024-09-18T10:15:42.996 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addr": "172.21.15.189:6825/22405", 2024-09-18T10:15:42.996 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addr": "172.21.15.189:6823/22405", 2024-09-18T10:15:42.996 INFO:teuthology.orchestra.run.smithi029.stdout: "state": [ 2024-09-18T10:15:42.996 INFO:teuthology.orchestra.run.smithi029.stdout: "exists", 2024-09-18T10:15:42.997 INFO:teuthology.orchestra.run.smithi029.stdout: "up" 2024-09-18T10:15:42.997 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:42.997 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:42.997 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:42.997 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 8, 2024-09-18T10:15:42.997 INFO:teuthology.orchestra.run.smithi029.stdout: "uuid": "cd333f56-c165-45d7-8c8b-48d34bc4ddc5", 2024-09-18T10:15:42.997 INFO:teuthology.orchestra.run.smithi029.stdout: "up": 1, 2024-09-18T10:15:42.997 INFO:teuthology.orchestra.run.smithi029.stdout: "in": 0, 2024-09-18T10:15:42.997 INFO:teuthology.orchestra.run.smithi029.stdout: "weight": 0, 2024-09-18T10:15:42.997 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_affinity": 1, 2024-09-18T10:15:42.997 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_begin": 0, 2024-09-18T10:15:42.998 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_end": 0, 2024-09-18T10:15:42.998 INFO:teuthology.orchestra.run.smithi029.stdout: "up_from": 22, 2024-09-18T10:15:42.998 INFO:teuthology.orchestra.run.smithi029.stdout: "up_thru": 74, 2024-09-18T10:15:42.998 INFO:teuthology.orchestra.run.smithi029.stdout: "down_at": 0, 2024-09-18T10:15:42.998 INFO:teuthology.orchestra.run.smithi029.stdout: "lost_at": 0, 2024-09-18T10:15:42.998 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addrs": { 2024-09-18T10:15:42.998 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:42.998 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:42.998 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:42.998 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6824", 2024-09-18T10:15:42.998 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22200 2024-09-18T10:15:42.998 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:42.998 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:42.999 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:42.999 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6825", 2024-09-18T10:15:42.999 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22200 2024-09-18T10:15:42.999 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:42.999 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:42.999 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:42.999 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addrs": { 2024-09-18T10:15:42.999 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:42.999 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:42.999 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:42.999 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6826", 2024-09-18T10:15:42.999 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22200 2024-09-18T10:15:43.000 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:43.000 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:43.000 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:43.000 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6827", 2024-09-18T10:15:43.000 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22200 2024-09-18T10:15:43.000 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:43.000 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:43.000 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:43.000 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addrs": { 2024-09-18T10:15:43.000 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:43.000 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:43.000 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:43.001 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6830", 2024-09-18T10:15:43.001 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22200 2024-09-18T10:15:43.001 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:43.001 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:43.001 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:43.001 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6831", 2024-09-18T10:15:43.001 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22200 2024-09-18T10:15:43.001 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:43.001 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:43.001 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:43.002 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addrs": { 2024-09-18T10:15:43.002 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:43.002 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:43.002 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:43.002 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6828", 2024-09-18T10:15:43.002 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22200 2024-09-18T10:15:43.002 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:43.002 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:43.002 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:43.002 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6829", 2024-09-18T10:15:43.002 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22200 2024-09-18T10:15:43.002 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:43.002 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:43.003 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:43.003 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addr": "172.21.15.29:6825/22200", 2024-09-18T10:15:43.003 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addr": "172.21.15.29:6827/22200", 2024-09-18T10:15:43.003 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addr": "172.21.15.29:6831/22200", 2024-09-18T10:15:43.003 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addr": "172.21.15.29:6829/22200", 2024-09-18T10:15:43.003 INFO:teuthology.orchestra.run.smithi029.stdout: "state": [ 2024-09-18T10:15:43.003 INFO:teuthology.orchestra.run.smithi029.stdout: "exists", 2024-09-18T10:15:43.003 INFO:teuthology.orchestra.run.smithi029.stdout: "up" 2024-09-18T10:15:43.003 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:43.003 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:43.003 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:43.003 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 9, 2024-09-18T10:15:43.004 INFO:teuthology.orchestra.run.smithi029.stdout: "uuid": "1e079e4c-8801-4754-9d0d-d0b68695d32e", 2024-09-18T10:15:43.004 INFO:teuthology.orchestra.run.smithi029.stdout: "up": 1, 2024-09-18T10:15:43.004 INFO:teuthology.orchestra.run.smithi029.stdout: "in": 1, 2024-09-18T10:15:43.004 INFO:teuthology.orchestra.run.smithi029.stdout: "weight": 1, 2024-09-18T10:15:43.004 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_affinity": 1, 2024-09-18T10:15:43.004 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_begin": 0, 2024-09-18T10:15:43.004 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_end": 0, 2024-09-18T10:15:43.004 INFO:teuthology.orchestra.run.smithi029.stdout: "up_from": 21, 2024-09-18T10:15:43.004 INFO:teuthology.orchestra.run.smithi029.stdout: "up_thru": 82, 2024-09-18T10:15:43.004 INFO:teuthology.orchestra.run.smithi029.stdout: "down_at": 0, 2024-09-18T10:15:43.004 INFO:teuthology.orchestra.run.smithi029.stdout: "lost_at": 0, 2024-09-18T10:15:43.005 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addrs": { 2024-09-18T10:15:43.005 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:43.005 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:43.005 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:43.005 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.136:6800", 2024-09-18T10:15:43.005 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22142 2024-09-18T10:15:43.006 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:43.006 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:43.006 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:43.006 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.136:6801", 2024-09-18T10:15:43.006 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22142 2024-09-18T10:15:43.006 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:43.007 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:43.007 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:43.007 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addrs": { 2024-09-18T10:15:43.007 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:43.007 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:43.007 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:43.007 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.136:6802", 2024-09-18T10:15:43.008 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22142 2024-09-18T10:15:43.008 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:43.008 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:43.008 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:43.008 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.136:6803", 2024-09-18T10:15:43.008 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22142 2024-09-18T10:15:43.009 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:43.009 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:43.009 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:43.009 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addrs": { 2024-09-18T10:15:43.009 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:43.009 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:43.009 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:43.010 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.136:6806", 2024-09-18T10:15:43.010 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22142 2024-09-18T10:15:43.010 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:43.010 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:43.010 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:43.010 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.136:6807", 2024-09-18T10:15:43.011 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22142 2024-09-18T10:15:43.011 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:43.011 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:43.011 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:43.011 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addrs": { 2024-09-18T10:15:43.011 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:43.012 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:43.012 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:43.012 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.136:6804", 2024-09-18T10:15:43.012 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22142 2024-09-18T10:15:43.012 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:43.013 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:43.013 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:43.013 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.136:6805", 2024-09-18T10:15:43.013 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22142 2024-09-18T10:15:43.013 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:43.013 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:43.014 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:43.014 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addr": "172.21.15.136:6801/22142", 2024-09-18T10:15:43.014 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addr": "172.21.15.136:6803/22142", 2024-09-18T10:15:43.014 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addr": "172.21.15.136:6807/22142", 2024-09-18T10:15:43.014 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addr": "172.21.15.136:6805/22142", 2024-09-18T10:15:43.014 INFO:teuthology.orchestra.run.smithi029.stdout: "state": [ 2024-09-18T10:15:43.015 INFO:teuthology.orchestra.run.smithi029.stdout: "exists", 2024-09-18T10:15:43.015 INFO:teuthology.orchestra.run.smithi029.stdout: "up" 2024-09-18T10:15:43.015 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:43.015 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:43.015 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:43.016 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 10, 2024-09-18T10:15:43.016 INFO:teuthology.orchestra.run.smithi029.stdout: "uuid": "92ee68d4-371c-4849-930f-a8fa18d64b99", 2024-09-18T10:15:43.016 INFO:teuthology.orchestra.run.smithi029.stdout: "up": 1, 2024-09-18T10:15:43.016 INFO:teuthology.orchestra.run.smithi029.stdout: "in": 1, 2024-09-18T10:15:43.016 INFO:teuthology.orchestra.run.smithi029.stdout: "weight": 1, 2024-09-18T10:15:43.017 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_affinity": 1, 2024-09-18T10:15:43.017 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_begin": 0, 2024-09-18T10:15:43.017 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_end": 0, 2024-09-18T10:15:43.017 INFO:teuthology.orchestra.run.smithi029.stdout: "up_from": 22, 2024-09-18T10:15:43.017 INFO:teuthology.orchestra.run.smithi029.stdout: "up_thru": 72, 2024-09-18T10:15:43.017 INFO:teuthology.orchestra.run.smithi029.stdout: "down_at": 0, 2024-09-18T10:15:43.018 INFO:teuthology.orchestra.run.smithi029.stdout: "lost_at": 0, 2024-09-18T10:15:43.018 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addrs": { 2024-09-18T10:15:43.018 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:43.018 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:43.018 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:43.018 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.154:6808", 2024-09-18T10:15:43.019 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22097 2024-09-18T10:15:43.019 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:43.019 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:43.019 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:43.019 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.154:6809", 2024-09-18T10:15:43.019 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22097 2024-09-18T10:15:43.019 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:43.020 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:43.020 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:43.020 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addrs": { 2024-09-18T10:15:43.020 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:43.020 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:43.021 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:43.021 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.154:6810", 2024-09-18T10:15:43.021 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22097 2024-09-18T10:15:43.021 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:43.021 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:43.021 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:43.022 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.154:6811", 2024-09-18T10:15:43.022 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22097 2024-09-18T10:15:43.022 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:43.022 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:43.022 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:43.022 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addrs": { 2024-09-18T10:15:43.023 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:43.023 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:43.023 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:43.023 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.154:6814", 2024-09-18T10:15:43.023 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22097 2024-09-18T10:15:43.023 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:43.024 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:43.024 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:43.024 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.154:6815", 2024-09-18T10:15:43.024 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22097 2024-09-18T10:15:43.024 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:43.024 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:43.025 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:43.025 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addrs": { 2024-09-18T10:15:43.025 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:43.025 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:43.025 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:43.026 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.154:6812", 2024-09-18T10:15:43.026 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22097 2024-09-18T10:15:43.026 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:43.026 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:43.026 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:43.026 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.154:6813", 2024-09-18T10:15:43.027 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22097 2024-09-18T10:15:43.027 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:43.027 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:43.027 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:43.027 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addr": "172.21.15.154:6809/22097", 2024-09-18T10:15:43.028 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addr": "172.21.15.154:6811/22097", 2024-09-18T10:15:43.028 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addr": "172.21.15.154:6815/22097", 2024-09-18T10:15:43.028 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addr": "172.21.15.154:6813/22097", 2024-09-18T10:15:43.028 INFO:teuthology.orchestra.run.smithi029.stdout: "state": [ 2024-09-18T10:15:43.028 INFO:teuthology.orchestra.run.smithi029.stdout: "exists", 2024-09-18T10:15:43.028 INFO:teuthology.orchestra.run.smithi029.stdout: "up" 2024-09-18T10:15:43.029 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:43.029 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:43.029 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:43.029 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 11, 2024-09-18T10:15:43.029 INFO:teuthology.orchestra.run.smithi029.stdout: "uuid": "634ddaf5-f7b8-4b27-a46a-04e21b73fcf0", 2024-09-18T10:15:43.029 INFO:teuthology.orchestra.run.smithi029.stdout: "up": 1, 2024-09-18T10:15:43.030 INFO:teuthology.orchestra.run.smithi029.stdout: "in": 1, 2024-09-18T10:15:43.030 INFO:teuthology.orchestra.run.smithi029.stdout: "weight": 1, 2024-09-18T10:15:43.030 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_affinity": 1, 2024-09-18T10:15:43.030 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_begin": 0, 2024-09-18T10:15:43.030 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_end": 0, 2024-09-18T10:15:43.030 INFO:teuthology.orchestra.run.smithi029.stdout: "up_from": 21, 2024-09-18T10:15:43.031 INFO:teuthology.orchestra.run.smithi029.stdout: "up_thru": 82, 2024-09-18T10:15:43.031 INFO:teuthology.orchestra.run.smithi029.stdout: "down_at": 0, 2024-09-18T10:15:43.031 INFO:teuthology.orchestra.run.smithi029.stdout: "lost_at": 0, 2024-09-18T10:15:43.031 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addrs": { 2024-09-18T10:15:43.031 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:43.031 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:43.031 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:43.032 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.189:6826", 2024-09-18T10:15:43.032 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22416 2024-09-18T10:15:43.033 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:43.033 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:43.033 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:43.033 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.189:6827", 2024-09-18T10:15:43.033 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22416 2024-09-18T10:15:43.033 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:43.033 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:43.033 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:43.033 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addrs": { 2024-09-18T10:15:43.033 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:43.034 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:43.034 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:43.034 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.189:6828", 2024-09-18T10:15:43.034 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22416 2024-09-18T10:15:43.034 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:43.034 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:43.035 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:43.035 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.189:6829", 2024-09-18T10:15:43.035 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22416 2024-09-18T10:15:43.035 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:43.035 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:43.035 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:43.035 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addrs": { 2024-09-18T10:15:43.035 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:43.035 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:43.035 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:43.035 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.189:6832", 2024-09-18T10:15:43.035 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22416 2024-09-18T10:15:43.035 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:43.036 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:43.036 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:43.036 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.189:6833", 2024-09-18T10:15:43.036 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22416 2024-09-18T10:15:43.036 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:43.036 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:43.036 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:43.036 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addrs": { 2024-09-18T10:15:43.036 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:43.036 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:43.036 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:43.037 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.189:6830", 2024-09-18T10:15:43.037 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22416 2024-09-18T10:15:43.037 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:43.037 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:43.037 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:43.037 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.189:6831", 2024-09-18T10:15:43.037 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22416 2024-09-18T10:15:43.037 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:43.037 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:43.037 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:43.037 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addr": "172.21.15.189:6827/22416", 2024-09-18T10:15:43.037 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addr": "172.21.15.189:6829/22416", 2024-09-18T10:15:43.037 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addr": "172.21.15.189:6833/22416", 2024-09-18T10:15:43.037 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addr": "172.21.15.189:6831/22416", 2024-09-18T10:15:43.038 INFO:teuthology.orchestra.run.smithi029.stdout: "state": [ 2024-09-18T10:15:43.038 INFO:teuthology.orchestra.run.smithi029.stdout: "exists", 2024-09-18T10:15:43.038 INFO:teuthology.orchestra.run.smithi029.stdout: "up" 2024-09-18T10:15:43.038 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:43.038 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:43.038 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:43.038 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 12, 2024-09-18T10:15:43.038 INFO:teuthology.orchestra.run.smithi029.stdout: "uuid": "bc58137b-5948-400d-b377-193b15824d8d", 2024-09-18T10:15:43.038 INFO:teuthology.orchestra.run.smithi029.stdout: "up": 1, 2024-09-18T10:15:43.038 INFO:teuthology.orchestra.run.smithi029.stdout: "in": 1, 2024-09-18T10:15:43.038 INFO:teuthology.orchestra.run.smithi029.stdout: "weight": 1, 2024-09-18T10:15:43.038 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_affinity": 1, 2024-09-18T10:15:43.039 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_begin": 0, 2024-09-18T10:15:43.039 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_end": 0, 2024-09-18T10:15:43.039 INFO:teuthology.orchestra.run.smithi029.stdout: "up_from": 21, 2024-09-18T10:15:43.039 INFO:teuthology.orchestra.run.smithi029.stdout: "up_thru": 82, 2024-09-18T10:15:43.039 INFO:teuthology.orchestra.run.smithi029.stdout: "down_at": 0, 2024-09-18T10:15:43.039 INFO:teuthology.orchestra.run.smithi029.stdout: "lost_at": 0, 2024-09-18T10:15:43.039 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addrs": { 2024-09-18T10:15:43.039 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:43.039 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:43.039 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:43.039 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6816", 2024-09-18T10:15:43.039 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22210 2024-09-18T10:15:43.040 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:43.040 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:43.040 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:43.040 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6817", 2024-09-18T10:15:43.040 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22210 2024-09-18T10:15:43.040 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:43.040 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:43.040 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:43.040 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addrs": { 2024-09-18T10:15:43.040 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:43.040 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:43.041 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:43.041 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6818", 2024-09-18T10:15:43.041 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22210 2024-09-18T10:15:43.041 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:43.041 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:43.041 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:43.041 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6819", 2024-09-18T10:15:43.041 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22210 2024-09-18T10:15:43.041 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:43.041 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:43.041 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:43.041 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addrs": { 2024-09-18T10:15:43.041 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:43.041 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:43.042 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:43.042 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6822", 2024-09-18T10:15:43.042 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22210 2024-09-18T10:15:43.042 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:43.042 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:43.042 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:43.042 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6823", 2024-09-18T10:15:43.042 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22210 2024-09-18T10:15:43.042 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:43.042 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:43.042 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:43.042 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addrs": { 2024-09-18T10:15:43.042 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:43.043 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:43.043 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:43.043 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6820", 2024-09-18T10:15:43.043 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22210 2024-09-18T10:15:43.043 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:43.043 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:43.043 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:43.043 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.29:6821", 2024-09-18T10:15:43.043 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22210 2024-09-18T10:15:43.043 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:43.043 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:43.043 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:43.043 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addr": "172.21.15.29:6817/22210", 2024-09-18T10:15:43.044 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addr": "172.21.15.29:6819/22210", 2024-09-18T10:15:43.044 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addr": "172.21.15.29:6823/22210", 2024-09-18T10:15:43.044 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addr": "172.21.15.29:6821/22210", 2024-09-18T10:15:43.044 INFO:teuthology.orchestra.run.smithi029.stdout: "state": [ 2024-09-18T10:15:43.044 INFO:teuthology.orchestra.run.smithi029.stdout: "exists", 2024-09-18T10:15:43.044 INFO:teuthology.orchestra.run.smithi029.stdout: "up" 2024-09-18T10:15:43.044 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:43.044 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:43.044 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:43.044 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 13, 2024-09-18T10:15:43.044 INFO:teuthology.orchestra.run.smithi029.stdout: "uuid": "a8b753a7-ab69-4853-ac36-66d9b85c2e20", 2024-09-18T10:15:43.045 INFO:teuthology.orchestra.run.smithi029.stdout: "up": 1, 2024-09-18T10:15:43.045 INFO:teuthology.orchestra.run.smithi029.stdout: "in": 1, 2024-09-18T10:15:43.045 INFO:teuthology.orchestra.run.smithi029.stdout: "weight": 1, 2024-09-18T10:15:43.045 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_affinity": 1, 2024-09-18T10:15:43.045 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_begin": 0, 2024-09-18T10:15:43.045 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_end": 0, 2024-09-18T10:15:43.045 INFO:teuthology.orchestra.run.smithi029.stdout: "up_from": 22, 2024-09-18T10:15:43.045 INFO:teuthology.orchestra.run.smithi029.stdout: "up_thru": 82, 2024-09-18T10:15:43.045 INFO:teuthology.orchestra.run.smithi029.stdout: "down_at": 0, 2024-09-18T10:15:43.045 INFO:teuthology.orchestra.run.smithi029.stdout: "lost_at": 0, 2024-09-18T10:15:43.045 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addrs": { 2024-09-18T10:15:43.046 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:43.046 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:43.046 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:43.046 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.136:6816", 2024-09-18T10:15:43.046 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22143 2024-09-18T10:15:43.046 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:43.046 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:43.046 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:43.046 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.136:6818", 2024-09-18T10:15:43.046 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22143 2024-09-18T10:15:43.046 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:43.046 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:43.047 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:43.047 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addrs": { 2024-09-18T10:15:43.047 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:43.047 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:43.047 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:43.047 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.136:6820", 2024-09-18T10:15:43.047 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22143 2024-09-18T10:15:43.047 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:43.047 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:43.047 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:43.047 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.136:6822", 2024-09-18T10:15:43.047 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22143 2024-09-18T10:15:43.047 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:43.048 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:43.048 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:43.048 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addrs": { 2024-09-18T10:15:43.048 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:43.048 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:43.048 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:43.048 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.136:6828", 2024-09-18T10:15:43.048 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22143 2024-09-18T10:15:43.048 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:43.048 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:43.048 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:43.048 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.136:6830", 2024-09-18T10:15:43.049 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22143 2024-09-18T10:15:43.049 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:43.049 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:43.049 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:43.049 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addrs": { 2024-09-18T10:15:43.049 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:43.049 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:43.049 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:43.049 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.136:6824", 2024-09-18T10:15:43.049 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22143 2024-09-18T10:15:43.050 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:43.050 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:43.050 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:43.050 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.136:6826", 2024-09-18T10:15:43.050 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22143 2024-09-18T10:15:43.050 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:43.051 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:43.051 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:43.051 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addr": "172.21.15.136:6818/22143", 2024-09-18T10:15:43.051 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addr": "172.21.15.136:6822/22143", 2024-09-18T10:15:43.051 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addr": "172.21.15.136:6830/22143", 2024-09-18T10:15:43.051 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addr": "172.21.15.136:6826/22143", 2024-09-18T10:15:43.051 INFO:teuthology.orchestra.run.smithi029.stdout: "state": [ 2024-09-18T10:15:43.051 INFO:teuthology.orchestra.run.smithi029.stdout: "exists", 2024-09-18T10:15:43.051 INFO:teuthology.orchestra.run.smithi029.stdout: "up" 2024-09-18T10:15:43.051 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:43.052 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:43.052 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:43.052 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 14, 2024-09-18T10:15:43.052 INFO:teuthology.orchestra.run.smithi029.stdout: "uuid": "8ad0848f-0de4-4a75-9c63-8c4a4a7b8524", 2024-09-18T10:15:43.052 INFO:teuthology.orchestra.run.smithi029.stdout: "up": 1, 2024-09-18T10:15:43.052 INFO:teuthology.orchestra.run.smithi029.stdout: "in": 1, 2024-09-18T10:15:43.052 INFO:teuthology.orchestra.run.smithi029.stdout: "weight": 1, 2024-09-18T10:15:43.052 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_affinity": 1, 2024-09-18T10:15:43.052 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_begin": 0, 2024-09-18T10:15:43.052 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_end": 0, 2024-09-18T10:15:43.053 INFO:teuthology.orchestra.run.smithi029.stdout: "up_from": 22, 2024-09-18T10:15:43.053 INFO:teuthology.orchestra.run.smithi029.stdout: "up_thru": 82, 2024-09-18T10:15:43.053 INFO:teuthology.orchestra.run.smithi029.stdout: "down_at": 0, 2024-09-18T10:15:43.053 INFO:teuthology.orchestra.run.smithi029.stdout: "lost_at": 0, 2024-09-18T10:15:43.054 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addrs": { 2024-09-18T10:15:43.054 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:43.054 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:43.054 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:43.054 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.154:6816", 2024-09-18T10:15:43.054 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22098 2024-09-18T10:15:43.055 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:43.055 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:43.055 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:43.055 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.154:6817", 2024-09-18T10:15:43.055 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22098 2024-09-18T10:15:43.055 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:43.055 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:43.056 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:43.056 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addrs": { 2024-09-18T10:15:43.056 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:43.056 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:43.056 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:43.056 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.154:6818", 2024-09-18T10:15:43.056 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22098 2024-09-18T10:15:43.056 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:43.056 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:43.056 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:43.056 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.154:6819", 2024-09-18T10:15:43.057 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22098 2024-09-18T10:15:43.057 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:43.057 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:43.057 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:43.057 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addrs": { 2024-09-18T10:15:43.057 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:43.057 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:43.057 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:43.057 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.154:6822", 2024-09-18T10:15:43.057 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22098 2024-09-18T10:15:43.058 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:43.058 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:43.058 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:43.058 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.154:6823", 2024-09-18T10:15:43.058 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22098 2024-09-18T10:15:43.058 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:43.058 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:43.058 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:43.058 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addrs": { 2024-09-18T10:15:43.058 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:43.058 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:43.058 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:43.058 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.154:6820", 2024-09-18T10:15:43.059 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22098 2024-09-18T10:15:43.059 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:43.059 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:43.059 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:43.059 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.154:6821", 2024-09-18T10:15:43.059 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22098 2024-09-18T10:15:43.059 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:43.059 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:43.059 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:43.059 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addr": "172.21.15.154:6817/22098", 2024-09-18T10:15:43.059 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addr": "172.21.15.154:6819/22098", 2024-09-18T10:15:43.059 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addr": "172.21.15.154:6823/22098", 2024-09-18T10:15:43.060 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addr": "172.21.15.154:6821/22098", 2024-09-18T10:15:43.060 INFO:teuthology.orchestra.run.smithi029.stdout: "state": [ 2024-09-18T10:15:43.060 INFO:teuthology.orchestra.run.smithi029.stdout: "exists", 2024-09-18T10:15:43.060 INFO:teuthology.orchestra.run.smithi029.stdout: "up" 2024-09-18T10:15:43.060 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:43.061 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:43.061 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:43.061 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 15, 2024-09-18T10:15:43.061 INFO:teuthology.orchestra.run.smithi029.stdout: "uuid": "069099b3-ec1a-483b-b967-98cddbf2f678", 2024-09-18T10:15:43.061 INFO:teuthology.orchestra.run.smithi029.stdout: "up": 1, 2024-09-18T10:15:43.061 INFO:teuthology.orchestra.run.smithi029.stdout: "in": 1, 2024-09-18T10:15:43.062 INFO:teuthology.orchestra.run.smithi029.stdout: "weight": 1, 2024-09-18T10:15:43.062 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_affinity": 1, 2024-09-18T10:15:43.062 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_begin": 0, 2024-09-18T10:15:43.062 INFO:teuthology.orchestra.run.smithi029.stdout: "last_clean_end": 0, 2024-09-18T10:15:43.062 INFO:teuthology.orchestra.run.smithi029.stdout: "up_from": 22, 2024-09-18T10:15:43.062 INFO:teuthology.orchestra.run.smithi029.stdout: "up_thru": 0, 2024-09-18T10:15:43.062 INFO:teuthology.orchestra.run.smithi029.stdout: "down_at": 0, 2024-09-18T10:15:43.062 INFO:teuthology.orchestra.run.smithi029.stdout: "lost_at": 0, 2024-09-18T10:15:43.062 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addrs": { 2024-09-18T10:15:43.062 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:43.062 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:43.062 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:43.062 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.189:6810", 2024-09-18T10:15:43.063 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22415 2024-09-18T10:15:43.063 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:43.063 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:43.063 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:43.063 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.189:6811", 2024-09-18T10:15:43.063 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22415 2024-09-18T10:15:43.063 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:43.063 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:43.063 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:43.063 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addrs": { 2024-09-18T10:15:43.063 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:43.063 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:43.063 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:43.064 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.189:6812", 2024-09-18T10:15:43.064 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22415 2024-09-18T10:15:43.064 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:43.064 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:43.064 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:43.064 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.189:6813", 2024-09-18T10:15:43.064 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22415 2024-09-18T10:15:43.064 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:43.064 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:43.064 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:43.065 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addrs": { 2024-09-18T10:15:43.065 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:43.065 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:43.065 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:43.065 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.189:6816", 2024-09-18T10:15:43.066 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22415 2024-09-18T10:15:43.066 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:43.066 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:43.066 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:43.066 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.189:6817", 2024-09-18T10:15:43.066 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22415 2024-09-18T10:15:43.067 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:43.067 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:43.067 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:43.067 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addrs": { 2024-09-18T10:15:43.067 INFO:teuthology.orchestra.run.smithi029.stdout: "addrvec": [ 2024-09-18T10:15:43.067 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:43.068 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v2", 2024-09-18T10:15:43.068 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.189:6814", 2024-09-18T10:15:43.068 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22415 2024-09-18T10:15:43.068 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:43.068 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:43.068 INFO:teuthology.orchestra.run.smithi029.stdout: "type": "v1", 2024-09-18T10:15:43.068 INFO:teuthology.orchestra.run.smithi029.stdout: "addr": "172.21.15.189:6815", 2024-09-18T10:15:43.068 INFO:teuthology.orchestra.run.smithi029.stdout: "nonce": 22415 2024-09-18T10:15:43.068 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:43.068 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:43.068 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:43.068 INFO:teuthology.orchestra.run.smithi029.stdout: "public_addr": "172.21.15.189:6811/22415", 2024-09-18T10:15:43.069 INFO:teuthology.orchestra.run.smithi029.stdout: "cluster_addr": "172.21.15.189:6813/22415", 2024-09-18T10:15:43.069 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_back_addr": "172.21.15.189:6817/22415", 2024-09-18T10:15:43.069 INFO:teuthology.orchestra.run.smithi029.stdout: "heartbeat_front_addr": "172.21.15.189:6815/22415", 2024-09-18T10:15:43.069 INFO:teuthology.orchestra.run.smithi029.stdout: "state": [ 2024-09-18T10:15:43.069 INFO:teuthology.orchestra.run.smithi029.stdout: "exists", 2024-09-18T10:15:43.069 INFO:teuthology.orchestra.run.smithi029.stdout: "up" 2024-09-18T10:15:43.069 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:43.069 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:43.069 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-18T10:15:43.069 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_xinfo": [ 2024-09-18T10:15:43.069 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:43.069 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 0, 2024-09-18T10:15:43.069 INFO:teuthology.orchestra.run.smithi029.stdout: "down_stamp": "0.000000", 2024-09-18T10:15:43.070 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_probability": 0, 2024-09-18T10:15:43.070 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_interval": 0, 2024-09-18T10:15:43.070 INFO:teuthology.orchestra.run.smithi029.stdout: "features": 4540138320759226367, 2024-09-18T10:15:43.070 INFO:teuthology.orchestra.run.smithi029.stdout: "old_weight": 0, 2024-09-18T10:15:43.070 INFO:teuthology.orchestra.run.smithi029.stdout: "last_purged_snaps_scrub": "2024-09-18T10:14:28.132367+0000", 2024-09-18T10:15:43.070 INFO:teuthology.orchestra.run.smithi029.stdout: "dead_epoch": 0 2024-09-18T10:15:43.070 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:43.070 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:43.070 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 1, 2024-09-18T10:15:43.070 INFO:teuthology.orchestra.run.smithi029.stdout: "down_stamp": "0.000000", 2024-09-18T10:15:43.070 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_probability": 0, 2024-09-18T10:15:43.070 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_interval": 0, 2024-09-18T10:15:43.071 INFO:teuthology.orchestra.run.smithi029.stdout: "features": 4540138320759226367, 2024-09-18T10:15:43.071 INFO:teuthology.orchestra.run.smithi029.stdout: "old_weight": 0, 2024-09-18T10:15:43.071 INFO:teuthology.orchestra.run.smithi029.stdout: "last_purged_snaps_scrub": "2024-09-18T10:15:32.497046+0000", 2024-09-18T10:15:43.071 INFO:teuthology.orchestra.run.smithi029.stdout: "dead_epoch": 0 2024-09-18T10:15:43.071 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:43.071 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:43.071 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 2, 2024-09-18T10:15:43.071 INFO:teuthology.orchestra.run.smithi029.stdout: "down_stamp": "0.000000", 2024-09-18T10:15:43.071 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_probability": 0, 2024-09-18T10:15:43.071 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_interval": 0, 2024-09-18T10:15:43.071 INFO:teuthology.orchestra.run.smithi029.stdout: "features": 4540138320759226367, 2024-09-18T10:15:43.072 INFO:teuthology.orchestra.run.smithi029.stdout: "old_weight": 0, 2024-09-18T10:15:43.072 INFO:teuthology.orchestra.run.smithi029.stdout: "last_purged_snaps_scrub": "2024-09-18T10:14:28.486853+0000", 2024-09-18T10:15:43.072 INFO:teuthology.orchestra.run.smithi029.stdout: "dead_epoch": 0 2024-09-18T10:15:43.072 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:43.072 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:43.072 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 3, 2024-09-18T10:15:43.072 INFO:teuthology.orchestra.run.smithi029.stdout: "down_stamp": "0.000000", 2024-09-18T10:15:43.073 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_probability": 0, 2024-09-18T10:15:43.073 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_interval": 0, 2024-09-18T10:15:43.073 INFO:teuthology.orchestra.run.smithi029.stdout: "features": 4540138320759226367, 2024-09-18T10:15:43.073 INFO:teuthology.orchestra.run.smithi029.stdout: "old_weight": 0, 2024-09-18T10:15:43.073 INFO:teuthology.orchestra.run.smithi029.stdout: "last_purged_snaps_scrub": "2024-09-18T10:14:28.299130+0000", 2024-09-18T10:15:43.073 INFO:teuthology.orchestra.run.smithi029.stdout: "dead_epoch": 0 2024-09-18T10:15:43.073 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:43.074 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:43.074 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 4, 2024-09-18T10:15:43.074 INFO:teuthology.orchestra.run.smithi029.stdout: "down_stamp": "0.000000", 2024-09-18T10:15:43.074 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_probability": 0, 2024-09-18T10:15:43.074 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_interval": 0, 2024-09-18T10:15:43.074 INFO:teuthology.orchestra.run.smithi029.stdout: "features": 4540138320759226367, 2024-09-18T10:15:43.074 INFO:teuthology.orchestra.run.smithi029.stdout: "old_weight": 0, 2024-09-18T10:15:43.074 INFO:teuthology.orchestra.run.smithi029.stdout: "last_purged_snaps_scrub": "2024-09-18T10:14:28.027740+0000", 2024-09-18T10:15:43.075 INFO:teuthology.orchestra.run.smithi029.stdout: "dead_epoch": 0 2024-09-18T10:15:43.075 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:43.075 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:43.075 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 5, 2024-09-18T10:15:43.075 INFO:teuthology.orchestra.run.smithi029.stdout: "down_stamp": "0.000000", 2024-09-18T10:15:43.075 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_probability": 0, 2024-09-18T10:15:43.075 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_interval": 0, 2024-09-18T10:15:43.075 INFO:teuthology.orchestra.run.smithi029.stdout: "features": 4540138320759226367, 2024-09-18T10:15:43.075 INFO:teuthology.orchestra.run.smithi029.stdout: "old_weight": 0, 2024-09-18T10:15:43.076 INFO:teuthology.orchestra.run.smithi029.stdout: "last_purged_snaps_scrub": "2024-09-18T10:14:28.450120+0000", 2024-09-18T10:15:43.076 INFO:teuthology.orchestra.run.smithi029.stdout: "dead_epoch": 0 2024-09-18T10:15:43.076 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:43.076 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:43.076 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 6, 2024-09-18T10:15:43.076 INFO:teuthology.orchestra.run.smithi029.stdout: "down_stamp": "0.000000", 2024-09-18T10:15:43.076 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_probability": 0, 2024-09-18T10:15:43.076 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_interval": 0, 2024-09-18T10:15:43.076 INFO:teuthology.orchestra.run.smithi029.stdout: "features": 4540138320759226367, 2024-09-18T10:15:43.076 INFO:teuthology.orchestra.run.smithi029.stdout: "old_weight": 0, 2024-09-18T10:15:43.076 INFO:teuthology.orchestra.run.smithi029.stdout: "last_purged_snaps_scrub": "2024-09-18T10:15:41.309784+0000", 2024-09-18T10:15:43.077 INFO:teuthology.orchestra.run.smithi029.stdout: "dead_epoch": 0 2024-09-18T10:15:43.077 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:43.077 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:43.077 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 7, 2024-09-18T10:15:43.077 INFO:teuthology.orchestra.run.smithi029.stdout: "down_stamp": "0.000000", 2024-09-18T10:15:43.077 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_probability": 0, 2024-09-18T10:15:43.077 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_interval": 0, 2024-09-18T10:15:43.077 INFO:teuthology.orchestra.run.smithi029.stdout: "features": 4540138320759226367, 2024-09-18T10:15:43.077 INFO:teuthology.orchestra.run.smithi029.stdout: "old_weight": 0, 2024-09-18T10:15:43.077 INFO:teuthology.orchestra.run.smithi029.stdout: "last_purged_snaps_scrub": "2024-09-18T10:15:36.256262+0000", 2024-09-18T10:15:43.077 INFO:teuthology.orchestra.run.smithi029.stdout: "dead_epoch": 0 2024-09-18T10:15:43.077 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:43.078 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:43.078 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 8, 2024-09-18T10:15:43.078 INFO:teuthology.orchestra.run.smithi029.stdout: "down_stamp": "0.000000", 2024-09-18T10:15:43.078 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_probability": 0, 2024-09-18T10:15:43.078 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_interval": 0, 2024-09-18T10:15:43.078 INFO:teuthology.orchestra.run.smithi029.stdout: "features": 4540138320759226367, 2024-09-18T10:15:43.078 INFO:teuthology.orchestra.run.smithi029.stdout: "old_weight": 65536, 2024-09-18T10:15:43.078 INFO:teuthology.orchestra.run.smithi029.stdout: "last_purged_snaps_scrub": "2024-09-18T10:14:28.238445+0000", 2024-09-18T10:15:43.078 INFO:teuthology.orchestra.run.smithi029.stdout: "dead_epoch": 0 2024-09-18T10:15:43.078 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:43.078 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:43.079 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 9, 2024-09-18T10:15:43.079 INFO:teuthology.orchestra.run.smithi029.stdout: "down_stamp": "0.000000", 2024-09-18T10:15:43.079 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_probability": 0, 2024-09-18T10:15:43.079 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_interval": 0, 2024-09-18T10:15:43.079 INFO:teuthology.orchestra.run.smithi029.stdout: "features": 4540138320759226367, 2024-09-18T10:15:43.079 INFO:teuthology.orchestra.run.smithi029.stdout: "old_weight": 0, 2024-09-18T10:15:43.079 INFO:teuthology.orchestra.run.smithi029.stdout: "last_purged_snaps_scrub": "2024-09-18T10:15:36.896626+0000", 2024-09-18T10:15:43.079 INFO:teuthology.orchestra.run.smithi029.stdout: "dead_epoch": 0 2024-09-18T10:15:43.079 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:43.079 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:43.079 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 10, 2024-09-18T10:15:43.080 INFO:teuthology.orchestra.run.smithi029.stdout: "down_stamp": "0.000000", 2024-09-18T10:15:43.080 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_probability": 0, 2024-09-18T10:15:43.080 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_interval": 0, 2024-09-18T10:15:43.080 INFO:teuthology.orchestra.run.smithi029.stdout: "features": 4540138320759226367, 2024-09-18T10:15:43.080 INFO:teuthology.orchestra.run.smithi029.stdout: "old_weight": 0, 2024-09-18T10:15:43.080 INFO:teuthology.orchestra.run.smithi029.stdout: "last_purged_snaps_scrub": "2024-09-18T10:14:28.345864+0000", 2024-09-18T10:15:43.080 INFO:teuthology.orchestra.run.smithi029.stdout: "dead_epoch": 0 2024-09-18T10:15:43.080 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:43.080 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:43.080 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 11, 2024-09-18T10:15:43.081 INFO:teuthology.orchestra.run.smithi029.stdout: "down_stamp": "0.000000", 2024-09-18T10:15:43.081 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_probability": 0, 2024-09-18T10:15:43.081 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_interval": 0, 2024-09-18T10:15:43.081 INFO:teuthology.orchestra.run.smithi029.stdout: "features": 4540138320759226367, 2024-09-18T10:15:43.081 INFO:teuthology.orchestra.run.smithi029.stdout: "old_weight": 0, 2024-09-18T10:15:43.081 INFO:teuthology.orchestra.run.smithi029.stdout: "last_purged_snaps_scrub": "2024-09-18T10:14:28.131654+0000", 2024-09-18T10:15:43.082 INFO:teuthology.orchestra.run.smithi029.stdout: "dead_epoch": 0 2024-09-18T10:15:43.082 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:43.082 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:43.082 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 12, 2024-09-18T10:15:43.082 INFO:teuthology.orchestra.run.smithi029.stdout: "down_stamp": "0.000000", 2024-09-18T10:15:43.082 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_probability": 0, 2024-09-18T10:15:43.083 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_interval": 0, 2024-09-18T10:15:43.083 INFO:teuthology.orchestra.run.smithi029.stdout: "features": 4540138320759226367, 2024-09-18T10:15:43.083 INFO:teuthology.orchestra.run.smithi029.stdout: "old_weight": 0, 2024-09-18T10:15:43.083 INFO:teuthology.orchestra.run.smithi029.stdout: "last_purged_snaps_scrub": "2024-09-18T10:15:31.985850+0000", 2024-09-18T10:15:43.083 INFO:teuthology.orchestra.run.smithi029.stdout: "dead_epoch": 0 2024-09-18T10:15:43.084 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:43.084 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:43.084 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 13, 2024-09-18T10:15:43.084 INFO:teuthology.orchestra.run.smithi029.stdout: "down_stamp": "0.000000", 2024-09-18T10:15:43.084 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_probability": 0, 2024-09-18T10:15:43.084 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_interval": 0, 2024-09-18T10:15:43.085 INFO:teuthology.orchestra.run.smithi029.stdout: "features": 4540138320759226367, 2024-09-18T10:15:43.085 INFO:teuthology.orchestra.run.smithi029.stdout: "old_weight": 0, 2024-09-18T10:15:43.085 INFO:teuthology.orchestra.run.smithi029.stdout: "last_purged_snaps_scrub": "2024-09-18T10:15:36.748402+0000", 2024-09-18T10:15:43.085 INFO:teuthology.orchestra.run.smithi029.stdout: "dead_epoch": 0 2024-09-18T10:15:43.085 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:43.085 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:43.086 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 14, 2024-09-18T10:15:43.086 INFO:teuthology.orchestra.run.smithi029.stdout: "down_stamp": "0.000000", 2024-09-18T10:15:43.086 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_probability": 0, 2024-09-18T10:15:43.086 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_interval": 0, 2024-09-18T10:15:43.086 INFO:teuthology.orchestra.run.smithi029.stdout: "features": 4540138320759226367, 2024-09-18T10:15:43.086 INFO:teuthology.orchestra.run.smithi029.stdout: "old_weight": 0, 2024-09-18T10:15:43.086 INFO:teuthology.orchestra.run.smithi029.stdout: "last_purged_snaps_scrub": "2024-09-18T10:14:28.510528+0000", 2024-09-18T10:15:43.087 INFO:teuthology.orchestra.run.smithi029.stdout: "dead_epoch": 0 2024-09-18T10:15:43.087 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:43.087 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:43.087 INFO:teuthology.orchestra.run.smithi029.stdout: "osd": 15, 2024-09-18T10:15:43.087 INFO:teuthology.orchestra.run.smithi029.stdout: "down_stamp": "0.000000", 2024-09-18T10:15:43.087 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_probability": 0, 2024-09-18T10:15:43.088 INFO:teuthology.orchestra.run.smithi029.stdout: "laggy_interval": 0, 2024-09-18T10:15:43.088 INFO:teuthology.orchestra.run.smithi029.stdout: "features": 4540138320759226367, 2024-09-18T10:15:43.088 INFO:teuthology.orchestra.run.smithi029.stdout: "old_weight": 0, 2024-09-18T10:15:43.088 INFO:teuthology.orchestra.run.smithi029.stdout: "last_purged_snaps_scrub": "2024-09-18T10:14:28.398477+0000", 2024-09-18T10:15:43.088 INFO:teuthology.orchestra.run.smithi029.stdout: "dead_epoch": 0 2024-09-18T10:15:43.088 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:43.089 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-18T10:15:43.089 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_upmap": [], 2024-09-18T10:15:43.089 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_upmap_items": [ 2024-09-18T10:15:43.089 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:43.089 INFO:teuthology.orchestra.run.smithi029.stdout: "pgid": "3.15", 2024-09-18T10:15:43.089 INFO:teuthology.orchestra.run.smithi029.stdout: "mappings": [ 2024-09-18T10:15:43.090 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:43.090 INFO:teuthology.orchestra.run.smithi029.stdout: "from": 7, 2024-09-18T10:15:43.090 INFO:teuthology.orchestra.run.smithi029.stdout: "to": 14 2024-09-18T10:15:43.090 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:43.090 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:43.090 INFO:teuthology.orchestra.run.smithi029.stdout: "from": 10, 2024-09-18T10:15:43.091 INFO:teuthology.orchestra.run.smithi029.stdout: "to": 15 2024-09-18T10:15:43.091 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:43.091 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:43.091 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:43.091 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-18T10:15:43.091 INFO:teuthology.orchestra.run.smithi029.stdout: "pg_temp": [ 2024-09-18T10:15:43.092 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:43.092 INFO:teuthology.orchestra.run.smithi029.stdout: "pgid": "3.c", 2024-09-18T10:15:43.092 INFO:teuthology.orchestra.run.smithi029.stdout: "osds": [ 2024-09-18T10:15:43.092 INFO:teuthology.orchestra.run.smithi029.stdout: 5, 2024-09-18T10:15:43.092 INFO:teuthology.orchestra.run.smithi029.stdout: 2147483647, 2024-09-18T10:15:43.093 INFO:teuthology.orchestra.run.smithi029.stdout: 13, 2024-09-18T10:15:43.093 INFO:teuthology.orchestra.run.smithi029.stdout: 3, 2024-09-18T10:15:43.093 INFO:teuthology.orchestra.run.smithi029.stdout: 7, 2024-09-18T10:15:43.093 INFO:teuthology.orchestra.run.smithi029.stdout: 12, 2024-09-18T10:15:43.093 INFO:teuthology.orchestra.run.smithi029.stdout: 14 2024-09-18T10:15:43.093 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:43.094 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:43.094 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:43.094 INFO:teuthology.orchestra.run.smithi029.stdout: "pgid": "3.19", 2024-09-18T10:15:43.094 INFO:teuthology.orchestra.run.smithi029.stdout: "osds": [ 2024-09-18T10:15:43.094 INFO:teuthology.orchestra.run.smithi029.stdout: 2147483647, 2024-09-18T10:15:43.094 INFO:teuthology.orchestra.run.smithi029.stdout: 4, 2024-09-18T10:15:43.095 INFO:teuthology.orchestra.run.smithi029.stdout: 1, 2024-09-18T10:15:43.095 INFO:teuthology.orchestra.run.smithi029.stdout: 2, 2024-09-18T10:15:43.095 INFO:teuthology.orchestra.run.smithi029.stdout: 13, 2024-09-18T10:15:43.095 INFO:teuthology.orchestra.run.smithi029.stdout: 9, 2024-09-18T10:15:43.095 INFO:teuthology.orchestra.run.smithi029.stdout: 14 2024-09-18T10:15:43.095 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:43.096 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:43.096 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-18T10:15:43.096 INFO:teuthology.orchestra.run.smithi029.stdout: "primary_temp": [], 2024-09-18T10:15:43.096 INFO:teuthology.orchestra.run.smithi029.stdout: "blocklist": {}, 2024-09-18T10:15:43.096 INFO:teuthology.orchestra.run.smithi029.stdout: "range_blocklist": {}, 2024-09-18T10:15:43.096 INFO:teuthology.orchestra.run.smithi029.stdout: "erasure_code_profiles": { 2024-09-18T10:15:43.097 INFO:teuthology.orchestra.run.smithi029.stdout: "default": { 2024-09-18T10:15:43.097 INFO:teuthology.orchestra.run.smithi029.stdout: "crush-failure-domain": "osd", 2024-09-18T10:15:43.097 INFO:teuthology.orchestra.run.smithi029.stdout: "k": "2", 2024-09-18T10:15:43.097 INFO:teuthology.orchestra.run.smithi029.stdout: "m": "1", 2024-09-18T10:15:43.097 INFO:teuthology.orchestra.run.smithi029.stdout: "plugin": "jerasure", 2024-09-18T10:15:43.097 INFO:teuthology.orchestra.run.smithi029.stdout: "technique": "reed_sol_van" 2024-09-18T10:15:43.098 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:43.098 INFO:teuthology.orchestra.run.smithi029.stdout: "shecprofile": { 2024-09-18T10:15:43.098 INFO:teuthology.orchestra.run.smithi029.stdout: "c": "2", 2024-09-18T10:15:43.098 INFO:teuthology.orchestra.run.smithi029.stdout: "crush-device-class": "", 2024-09-18T10:15:43.098 INFO:teuthology.orchestra.run.smithi029.stdout: "crush-failure-domain": "osd", 2024-09-18T10:15:43.098 INFO:teuthology.orchestra.run.smithi029.stdout: "crush-root": "default", 2024-09-18T10:15:43.099 INFO:teuthology.orchestra.run.smithi029.stdout: "k": "4", 2024-09-18T10:15:43.099 INFO:teuthology.orchestra.run.smithi029.stdout: "m": "3", 2024-09-18T10:15:43.099 INFO:teuthology.orchestra.run.smithi029.stdout: "name": "shecprofile", 2024-09-18T10:15:43.099 INFO:teuthology.orchestra.run.smithi029.stdout: "plugin": "shec", 2024-09-18T10:15:43.099 INFO:teuthology.orchestra.run.smithi029.stdout: "technique": "multiple" 2024-09-18T10:15:43.100 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:43.100 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:43.100 INFO:teuthology.orchestra.run.smithi029.stdout: "removed_snaps_queue": [ 2024-09-18T10:15:43.100 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:43.100 INFO:teuthology.orchestra.run.smithi029.stdout: "pool": 3, 2024-09-18T10:15:43.100 INFO:teuthology.orchestra.run.smithi029.stdout: "snaps": [ 2024-09-18T10:15:43.101 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:43.101 INFO:teuthology.orchestra.run.smithi029.stdout: "begin": 16, 2024-09-18T10:15:43.101 INFO:teuthology.orchestra.run.smithi029.stdout: "length": 1 2024-09-18T10:15:43.101 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:43.101 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:43.101 INFO:teuthology.orchestra.run.smithi029.stdout: "begin": 18, 2024-09-18T10:15:43.102 INFO:teuthology.orchestra.run.smithi029.stdout: "length": 1 2024-09-18T10:15:43.102 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:43.102 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:43.102 INFO:teuthology.orchestra.run.smithi029.stdout: "begin": 22, 2024-09-18T10:15:43.102 INFO:teuthology.orchestra.run.smithi029.stdout: "length": 1 2024-09-18T10:15:43.102 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:43.103 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:43.103 INFO:teuthology.orchestra.run.smithi029.stdout: "begin": 38, 2024-09-18T10:15:43.103 INFO:teuthology.orchestra.run.smithi029.stdout: "length": 1 2024-09-18T10:15:43.103 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:43.103 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:43.103 INFO:teuthology.orchestra.run.smithi029.stdout: "begin": 42, 2024-09-18T10:15:43.104 INFO:teuthology.orchestra.run.smithi029.stdout: "length": 1 2024-09-18T10:15:43.104 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:43.104 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:43.104 INFO:teuthology.orchestra.run.smithi029.stdout: "begin": 45, 2024-09-18T10:15:43.104 INFO:teuthology.orchestra.run.smithi029.stdout: "length": 1 2024-09-18T10:15:43.104 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:43.105 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:43.105 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:43.105 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-18T10:15:43.105 INFO:teuthology.orchestra.run.smithi029.stdout: "new_removed_snaps": [], 2024-09-18T10:15:43.105 INFO:teuthology.orchestra.run.smithi029.stdout: "new_purged_snaps": [], 2024-09-18T10:15:43.106 INFO:teuthology.orchestra.run.smithi029.stdout: "crush_node_flags": {}, 2024-09-18T10:15:43.106 INFO:teuthology.orchestra.run.smithi029.stdout: "device_class_flags": {}, 2024-09-18T10:15:43.106 INFO:teuthology.orchestra.run.smithi029.stdout: "stretch_mode": { 2024-09-18T10:15:43.106 INFO:teuthology.orchestra.run.smithi029.stdout: "stretch_mode_enabled": false, 2024-09-18T10:15:43.106 INFO:teuthology.orchestra.run.smithi029.stdout: "stretch_bucket_count": 0, 2024-09-18T10:15:43.107 INFO:teuthology.orchestra.run.smithi029.stdout: "degraded_stretch_mode": 0, 2024-09-18T10:15:43.107 INFO:teuthology.orchestra.run.smithi029.stdout: "recovering_stretch_mode": 0, 2024-09-18T10:15:43.107 INFO:teuthology.orchestra.run.smithi029.stdout: "stretch_mode_bucket": 0 2024-09-18T10:15:43.107 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:43.107 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-18T10:15:43.110 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-18T10:15:43.104+0000 7f00046a8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 22136) UID: 0 2024-09-18T10:15:43.111 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:43.112 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:43.114 INFO:tasks.thrashosds.thrasher:j is {'epoch': 87, 'fsid': 'ab6d6beb-0627-4b7f-8cdb-4e0a44970f10', 'created': '2024-09-18T10:14:18.827644+0000', 'modified': '2024-09-18T10:15:42.616205+0000', 'last_up_change': '2024-09-18T10:14:30.228591+0000', 'last_in_change': '2024-09-18T10:15:37.582801+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 6, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 16, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'quincy', 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-18T10:14:31.266113+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-18T10:14:33.358485+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '29', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 29, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'unique_pool_0', 'create_time': '2024-09-18T10:14:44.624320+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 3, 'size': 7, 'min_size': 5, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 26, 'pg_placement_num': 26, 'pg_placement_num_target': 26, 'pg_num_target': 26, 'pg_num_pending': 26, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '40', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 48, 'snap_epoch': 87, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'shecprofile', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 16384, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rados': {}}}], 'osds': [{'osd': 0, 'uuid': '39cf0511-0009-433f-92f2-e5686f33d713', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 40, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6808', 'nonce': 22197}, {'type': 'v1', 'addr': '172.21.15.29:6809', 'nonce': 22197}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6810', 'nonce': 22197}, {'type': 'v1', 'addr': '172.21.15.29:6811', 'nonce': 22197}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6814', 'nonce': 22197}, {'type': 'v1', 'addr': '172.21.15.29:6815', 'nonce': 22197}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6812', 'nonce': 22197}, {'type': 'v1', 'addr': '172.21.15.29:6813', 'nonce': 22197}]}, 'public_addr': '172.21.15.29:6809/22197', 'cluster_addr': '172.21.15.29:6811/22197', 'heartbeat_back_addr': '172.21.15.29:6815/22197', 'heartbeat_front_addr': '172.21.15.29:6813/22197', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'a0bf06ab-b98e-4130-91d2-7d4283540c08', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 58, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6808', 'nonce': 22140}, {'type': 'v1', 'addr': '172.21.15.136:6809', 'nonce': 22140}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6810', 'nonce': 22140}, {'type': 'v1', 'addr': '172.21.15.136:6811', 'nonce': 22140}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6814', 'nonce': 22140}, {'type': 'v1', 'addr': '172.21.15.136:6815', 'nonce': 22140}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6812', 'nonce': 22140}, {'type': 'v1', 'addr': '172.21.15.136:6813', 'nonce': 22140}]}, 'public_addr': '172.21.15.136:6809/22140', 'cluster_addr': '172.21.15.136:6811/22140', 'heartbeat_back_addr': '172.21.15.136:6815/22140', 'heartbeat_front_addr': '172.21.15.136:6813/22140', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '742a786e-6129-4fbc-bb5b-e2e3c03c2d77', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6824', 'nonce': 22095}, {'type': 'v1', 'addr': '172.21.15.154:6825', 'nonce': 22095}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6826', 'nonce': 22095}, {'type': 'v1', 'addr': '172.21.15.154:6827', 'nonce': 22095}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6830', 'nonce': 22095}, {'type': 'v1', 'addr': '172.21.15.154:6831', 'nonce': 22095}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6828', 'nonce': 22095}, {'type': 'v1', 'addr': '172.21.15.154:6829', 'nonce': 22095}]}, 'public_addr': '172.21.15.154:6825/22095', 'cluster_addr': '172.21.15.154:6827/22095', 'heartbeat_back_addr': '172.21.15.154:6831/22095', 'heartbeat_front_addr': '172.21.15.154:6829/22095', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '3aa77114-24ed-49ab-86ff-a0600794cbf8', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 48, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6802', 'nonce': 22404}, {'type': 'v1', 'addr': '172.21.15.189:6803', 'nonce': 22404}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6804', 'nonce': 22404}, {'type': 'v1', 'addr': '172.21.15.189:6805', 'nonce': 22404}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6808', 'nonce': 22404}, {'type': 'v1', 'addr': '172.21.15.189:6809', 'nonce': 22404}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6806', 'nonce': 22404}, {'type': 'v1', 'addr': '172.21.15.189:6807', 'nonce': 22404}]}, 'public_addr': '172.21.15.189:6803/22404', 'cluster_addr': '172.21.15.189:6805/22404', 'heartbeat_back_addr': '172.21.15.189:6809/22404', 'heartbeat_front_addr': '172.21.15.189:6807/22404', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '426567ad-b0be-41d4-afcf-dc56ec3974cd', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 83, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6800', 'nonce': 22195}, {'type': 'v1', 'addr': '172.21.15.29:6801', 'nonce': 22195}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6802', 'nonce': 22195}, {'type': 'v1', 'addr': '172.21.15.29:6803', 'nonce': 22195}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6806', 'nonce': 22195}, {'type': 'v1', 'addr': '172.21.15.29:6807', 'nonce': 22195}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6804', 'nonce': 22195}, {'type': 'v1', 'addr': '172.21.15.29:6805', 'nonce': 22195}]}, 'public_addr': '172.21.15.29:6801/22195', 'cluster_addr': '172.21.15.29:6803/22195', 'heartbeat_back_addr': '172.21.15.29:6807/22195', 'heartbeat_front_addr': '172.21.15.29:6805/22195', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'd9559e42-1f4e-4373-8392-a536a0afd871', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 83, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6817', 'nonce': 22141}, {'type': 'v1', 'addr': '172.21.15.136:6819', 'nonce': 22141}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6821', 'nonce': 22141}, {'type': 'v1', 'addr': '172.21.15.136:6823', 'nonce': 22141}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6829', 'nonce': 22141}, {'type': 'v1', 'addr': '172.21.15.136:6831', 'nonce': 22141}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6825', 'nonce': 22141}, {'type': 'v1', 'addr': '172.21.15.136:6827', 'nonce': 22141}]}, 'public_addr': '172.21.15.136:6819/22141', 'cluster_addr': '172.21.15.136:6823/22141', 'heartbeat_back_addr': '172.21.15.136:6831/22141', 'heartbeat_front_addr': '172.21.15.136:6827/22141', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'd50ef1e2-8bc5-4a90-9be3-365a0db2be6f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 82, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6800', 'nonce': 22096}, {'type': 'v1', 'addr': '172.21.15.154:6801', 'nonce': 22096}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6802', 'nonce': 22096}, {'type': 'v1', 'addr': '172.21.15.154:6803', 'nonce': 22096}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6806', 'nonce': 22096}, {'type': 'v1', 'addr': '172.21.15.154:6807', 'nonce': 22096}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6804', 'nonce': 22096}, {'type': 'v1', 'addr': '172.21.15.154:6805', 'nonce': 22096}]}, 'public_addr': '172.21.15.154:6801/22096', 'cluster_addr': '172.21.15.154:6803/22096', 'heartbeat_back_addr': '172.21.15.154:6807/22096', 'heartbeat_front_addr': '172.21.15.154:6805/22096', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'bd0b85e4-cfee-4d19-90a5-79854794b48b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 82, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6818', 'nonce': 22405}, {'type': 'v1', 'addr': '172.21.15.189:6819', 'nonce': 22405}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6820', 'nonce': 22405}, {'type': 'v1', 'addr': '172.21.15.189:6821', 'nonce': 22405}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6824', 'nonce': 22405}, {'type': 'v1', 'addr': '172.21.15.189:6825', 'nonce': 22405}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6822', 'nonce': 22405}, {'type': 'v1', 'addr': '172.21.15.189:6823', 'nonce': 22405}]}, 'public_addr': '172.21.15.189:6819/22405', 'cluster_addr': '172.21.15.189:6821/22405', 'heartbeat_back_addr': '172.21.15.189:6825/22405', 'heartbeat_front_addr': '172.21.15.189:6823/22405', 'state': ['exists', 'up']}, {'osd': 8, 'uuid': 'cd333f56-c165-45d7-8c8b-48d34bc4ddc5', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 74, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6824', 'nonce': 22200}, {'type': 'v1', 'addr': '172.21.15.29:6825', 'nonce': 22200}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6826', 'nonce': 22200}, {'type': 'v1', 'addr': '172.21.15.29:6827', 'nonce': 22200}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6830', 'nonce': 22200}, {'type': 'v1', 'addr': '172.21.15.29:6831', 'nonce': 22200}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6828', 'nonce': 22200}, {'type': 'v1', 'addr': '172.21.15.29:6829', 'nonce': 22200}]}, 'public_addr': '172.21.15.29:6825/22200', 'cluster_addr': '172.21.15.29:6827/22200', 'heartbeat_back_addr': '172.21.15.29:6831/22200', 'heartbeat_front_addr': '172.21.15.29:6829/22200', 'state': ['exists', 'up']}, {'osd': 9, 'uuid': '1e079e4c-8801-4754-9d0d-d0b68695d32e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 82, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6800', 'nonce': 22142}, {'type': 'v1', 'addr': '172.21.15.136:6801', 'nonce': 22142}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6802', 'nonce': 22142}, {'type': 'v1', 'addr': '172.21.15.136:6803', 'nonce': 22142}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6806', 'nonce': 22142}, {'type': 'v1', 'addr': '172.21.15.136:6807', 'nonce': 22142}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6804', 'nonce': 22142}, {'type': 'v1', 'addr': '172.21.15.136:6805', 'nonce': 22142}]}, 'public_addr': '172.21.15.136:6801/22142', 'cluster_addr': '172.21.15.136:6803/22142', 'heartbeat_back_addr': '172.21.15.136:6807/22142', 'heartbeat_front_addr': '172.21.15.136:6805/22142', 'state': ['exists', 'up']}, {'osd': 10, 'uuid': '92ee68d4-371c-4849-930f-a8fa18d64b99', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 72, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6808', 'nonce': 22097}, {'type': 'v1', 'addr': '172.21.15.154:6809', 'nonce': 22097}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6810', 'nonce': 22097}, {'type': 'v1', 'addr': '172.21.15.154:6811', 'nonce': 22097}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6814', 'nonce': 22097}, {'type': 'v1', 'addr': '172.21.15.154:6815', 'nonce': 22097}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6812', 'nonce': 22097}, {'type': 'v1', 'addr': '172.21.15.154:6813', 'nonce': 22097}]}, 'public_addr': '172.21.15.154:6809/22097', 'cluster_addr': '172.21.15.154:6811/22097', 'heartbeat_back_addr': '172.21.15.154:6815/22097', 'heartbeat_front_addr': '172.21.15.154:6813/22097', 'state': ['exists', 'up']}, {'osd': 11, 'uuid': '634ddaf5-f7b8-4b27-a46a-04e21b73fcf0', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 82, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6826', 'nonce': 22416}, {'type': 'v1', 'addr': '172.21.15.189:6827', 'nonce': 22416}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6828', 'nonce': 22416}, {'type': 'v1', 'addr': '172.21.15.189:6829', 'nonce': 22416}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6832', 'nonce': 22416}, {'type': 'v1', 'addr': '172.21.15.189:6833', 'nonce': 22416}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6830', 'nonce': 22416}, {'type': 'v1', 'addr': '172.21.15.189:6831', 'nonce': 22416}]}, 'public_addr': '172.21.15.189:6827/22416', 'cluster_addr': '172.21.15.189:6829/22416', 'heartbeat_back_addr': '172.21.15.189:6833/22416', 'heartbeat_front_addr': '172.21.15.189:6831/22416', 'state': ['exists', 'up']}, {'osd': 12, 'uuid': 'bc58137b-5948-400d-b377-193b15824d8d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 82, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6816', 'nonce': 22210}, {'type': 'v1', 'addr': '172.21.15.29:6817', 'nonce': 22210}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6818', 'nonce': 22210}, {'type': 'v1', 'addr': '172.21.15.29:6819', 'nonce': 22210}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6822', 'nonce': 22210}, {'type': 'v1', 'addr': '172.21.15.29:6823', 'nonce': 22210}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6820', 'nonce': 22210}, {'type': 'v1', 'addr': '172.21.15.29:6821', 'nonce': 22210}]}, 'public_addr': '172.21.15.29:6817/22210', 'cluster_addr': '172.21.15.29:6819/22210', 'heartbeat_back_addr': '172.21.15.29:6823/22210', 'heartbeat_front_addr': '172.21.15.29:6821/22210', 'state': ['exists', 'up']}, {'osd': 13, 'uuid': 'a8b753a7-ab69-4853-ac36-66d9b85c2e20', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 82, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6816', 'nonce': 22143}, {'type': 'v1', 'addr': '172.21.15.136:6818', 'nonce': 22143}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6820', 'nonce': 22143}, {'type': 'v1', 'addr': '172.21.15.136:6822', 'nonce': 22143}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6828', 'nonce': 22143}, {'type': 'v1', 'addr': '172.21.15.136:6830', 'nonce': 22143}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6824', 'nonce': 22143}, {'type': 'v1', 'addr': '172.21.15.136:6826', 'nonce': 22143}]}, 'public_addr': '172.21.15.136:6818/22143', 'cluster_addr': '172.21.15.136:6822/22143', 'heartbeat_back_addr': '172.21.15.136:6830/22143', 'heartbeat_front_addr': '172.21.15.136:6826/22143', 'state': ['exists', 'up']}, {'osd': 14, 'uuid': '8ad0848f-0de4-4a75-9c63-8c4a4a7b8524', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 82, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6816', 'nonce': 22098}, {'type': 'v1', 'addr': '172.21.15.154:6817', 'nonce': 22098}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6818', 'nonce': 22098}, {'type': 'v1', 'addr': '172.21.15.154:6819', 'nonce': 22098}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6822', 'nonce': 22098}, {'type': 'v1', 'addr': '172.21.15.154:6823', 'nonce': 22098}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6820', 'nonce': 22098}, {'type': 'v1', 'addr': '172.21.15.154:6821', 'nonce': 22098}]}, 'public_addr': '172.21.15.154:6817/22098', 'cluster_addr': '172.21.15.154:6819/22098', 'heartbeat_back_addr': '172.21.15.154:6823/22098', 'heartbeat_front_addr': '172.21.15.154:6821/22098', 'state': ['exists', 'up']}, {'osd': 15, 'uuid': '069099b3-ec1a-483b-b967-98cddbf2f678', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6810', 'nonce': 22415}, {'type': 'v1', 'addr': '172.21.15.189:6811', 'nonce': 22415}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6812', 'nonce': 22415}, {'type': 'v1', 'addr': '172.21.15.189:6813', 'nonce': 22415}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6816', 'nonce': 22415}, {'type': 'v1', 'addr': '172.21.15.189:6817', 'nonce': 22415}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6814', 'nonce': 22415}, {'type': 'v1', 'addr': '172.21.15.189:6815', 'nonce': 22415}]}, 'public_addr': '172.21.15.189:6811/22415', 'cluster_addr': '172.21.15.189:6813/22415', 'heartbeat_back_addr': '172.21.15.189:6817/22415', 'heartbeat_front_addr': '172.21.15.189:6815/22415', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T10:14:28.132367+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T10:15:32.497046+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T10:14:28.486853+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T10:14:28.299130+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T10:14:28.027740+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T10:14:28.450120+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T10:15:41.309784+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T10:15:36.256262+0000', 'dead_epoch': 0}, {'osd': 8, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-18T10:14:28.238445+0000', 'dead_epoch': 0}, {'osd': 9, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T10:15:36.896626+0000', 'dead_epoch': 0}, {'osd': 10, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T10:14:28.345864+0000', 'dead_epoch': 0}, {'osd': 11, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T10:14:28.131654+0000', 'dead_epoch': 0}, {'osd': 12, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T10:15:31.985850+0000', 'dead_epoch': 0}, {'osd': 13, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T10:15:36.748402+0000', 'dead_epoch': 0}, {'osd': 14, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T10:14:28.510528+0000', 'dead_epoch': 0}, {'osd': 15, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T10:14:28.398477+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [{'pgid': '3.15', 'mappings': [{'from': 7, 'to': 14}, {'from': 10, 'to': 15}]}], 'pg_temp': [{'pgid': '3.c', 'osds': [5, 2147483647, 13, 3, 7, 12, 14]}, {'pgid': '3.19', 'osds': [2147483647, 4, 1, 2, 13, 9, 14]}], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}, 'shecprofile': {'c': '2', 'crush-device-class': '', 'crush-failure-domain': 'osd', 'crush-root': 'default', 'k': '4', 'm': '3', 'name': 'shecprofile', 'plugin': 'shec', 'technique': 'multiple'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 16, 'length': 1}, {'begin': 18, 'length': 1}, {'begin': 22, 'length': 1}, {'begin': 38, 'length': 1}, {'begin': 42, 'length': 1}, {'begin': 45, 'length': 1}]}], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-18T10:15:43.114 INFO:tasks.thrashosds.thrasher:Clearing pg_upmap on 3.15 2024-09-18T10:15:43.114 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd rm-pg-upmap-items 3.15 2024-09-18T10:15:43.115 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_blocked_ops 2024-09-18T10:15:43.210 INFO:tasks.ceph.osd.14.smithi154.stderr:2024-09-18T10:15:43.206+0000 7f75a19e7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 22093) UID: 0 2024-09-18T10:15:43.224 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:43.224 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:43.234 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_historic_ops 2024-09-18T10:15:43.311 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-18T10:15:43.308+0000 7f578c52f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22134) UID: 0 2024-09-18T10:15:43.378 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:43.378 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:43.387 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T10:15:43.411 INFO:tasks.ceph.osd.4.smithi029.stderr:2024-09-18T10:15:43.405+0000 7f550cf6e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22189) UID: 0 2024-09-18T10:15:43.495 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:43.496 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:43.505 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T10:15:43.512 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-18T10:15:43.505+0000 7f302f2c3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22187) UID: 0 2024-09-18T10:15:43.601 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:43.601 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:43.610 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T10:15:43.613 INFO:tasks.ceph.osd.3.smithi189.stderr:2024-09-18T10:15:43.610+0000 7f2440fc7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22396) UID: 0 2024-09-18T10:15:43.635 INFO:tasks.rados.rados.0.smithi189.stdout:338: done (9 left) 2024-09-18T10:15:43.636 INFO:tasks.rados.rados.0.smithi189.stdout:339: copy_from oid 30 from oid 9 current snap is 28 2024-09-18T10:15:43.636 INFO:tasks.rados.rados.0.smithi189.stdout:340: snap_remove snap 17 2024-09-18T10:15:43.640 INFO:teuthology.orchestra.run.smithi029.stderr:clear 3.15 pg_upmap_items mapping 2024-09-18T10:15:43.643 INFO:teuthology.orchestra.run.smithi029.stderr:noscrub is set 2024-09-18T10:15:43.710 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:43.710 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:43.714 INFO:tasks.ceph.osd.15.smithi189.stderr:2024-09-18T10:15:43.710+0000 7fd0e725e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 22401) UID: 0 2024-09-18T10:15:43.719 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T10:15:43.816 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-18T10:15:43.809+0000 7f302f2c3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22187) UID: 0 2024-09-18T10:15:43.816 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:43.816 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:43.826 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T10:15:43.917 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-09-18T10:15:43.914+0000 7fdfa8698700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22087) UID: 0 2024-09-18T10:15:43.927 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:43.927 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:43.936 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T10:15:44.018 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-18T10:15:44.016+0000 7f578c52f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22134) UID: 0 2024-09-18T10:15:44.031 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:44.031 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:44.041 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-09-18T10:15:44.099 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T10:15:44.119 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-18T10:15:44.116+0000 7f14c6a1a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 22138) UID: 0 2024-09-18T10:15:44.143 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:44.143 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:44.152 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-09-18T10:15:44.220 INFO:tasks.ceph.osd.3.smithi189.stderr:2024-09-18T10:15:44.214+0000 7f2440fc7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22396) UID: 0 2024-09-18T10:15:44.241 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-18T10:15:44.250 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:44.250 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:44.259 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-09-18T10:15:44.264 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-18T10:15:44.287 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-18T10:15:44.308 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-18T10:15:44.321 INFO:tasks.ceph.osd.15.smithi189.stderr:2024-09-18T10:15:44.318+0000 7fd0e725e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 22401) UID: 0 2024-09-18T10:15:44.356 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:44.356 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:44.366 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_ops_in_flight 2024-09-18T10:15:44.379 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-18T10:15:44.400 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-18T10:15:44.423 INFO:teuthology.orchestra.run.smithi029.stdout:osd.6: {} 2024-09-18T10:15:44.423 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-18T10:15:44.420+0000 7f578c52f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22134) UID: 0 2024-09-18T10:15:44.446 INFO:teuthology.orchestra.run.smithi029.stdout:osd.7: {} 2024-09-18T10:15:44.471 INFO:teuthology.orchestra.run.smithi029.stdout:osd.8: {} 2024-09-18T10:15:44.500 INFO:teuthology.orchestra.run.smithi029.stdout:osd.9: {} 2024-09-18T10:15:44.517 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:44.517 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:44.522 INFO:teuthology.orchestra.run.smithi029.stdout:osd.10: {} 2024-09-18T10:15:44.523 INFO:tasks.ceph.osd.3.smithi189.stderr:2024-09-18T10:15:44.518+0000 7f2440fc7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22396) UID: 0 2024-09-18T10:15:44.526 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_blocked_ops 2024-09-18T10:15:44.545 INFO:teuthology.orchestra.run.smithi029.stdout:osd.11: {} 2024-09-18T10:15:44.571 INFO:teuthology.orchestra.run.smithi029.stdout:osd.12: {} 2024-09-18T10:15:44.592 INFO:teuthology.orchestra.run.smithi029.stdout:osd.13: {} 2024-09-18T10:15:44.615 INFO:teuthology.orchestra.run.smithi029.stdout:osd.14: {} 2024-09-18T10:15:44.621 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:44.622 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:44.625 INFO:tasks.ceph.osd.11.smithi189.stderr:2024-09-18T10:15:44.618+0000 7f718b60a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 22400) UID: 0 2024-09-18T10:15:44.631 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_historic_ops 2024-09-18T10:15:44.638 INFO:teuthology.orchestra.run.smithi029.stdout:osd.15: {} 2024-09-18T10:15:44.642 INFO:tasks.rados.rados.0.smithi189.stdout:340: done (10 left) 2024-09-18T10:15:44.642 INFO:tasks.rados.rados.0.smithi189.stdout:341: snap_create 2024-09-18T10:15:44.642 INFO:tasks.rados.rados.0.smithi189.stdout:329: finishing rollback tid 1 to smithi18925896-21 2024-09-18T10:15:44.642 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 21 v 0 (ObjNum 33 snap 0 seq_num 3858798608) dirty dne 2024-09-18T10:15:44.642 INFO:tasks.rados.rados.0.smithi189.stdout:332: finishing copy_from to smithi18925896-49 2024-09-18T10:15:44.642 INFO:tasks.rados.rados.0.smithi189.stdout:332: got expected ENOENT (src dne) 2024-09-18T10:15:44.643 INFO:tasks.rados.rados.0.smithi189.stdout:332: finishing copy_from racing read to smithi18925896-49 2024-09-18T10:15:44.643 INFO:tasks.rados.rados.0.smithi189.stdout:315: finishing write tid 6 to smithi18925896-41 2024-09-18T10:15:44.643 INFO:tasks.rados.rados.0.smithi189.stdout:315: finishing write tid 7 to smithi18925896-41 2024-09-18T10:15:44.643 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 41 v 67 (ObjNum 125 snap 27 seq_num 125) dirty exists 2024-09-18T10:15:44.643 INFO:tasks.rados.rados.0.smithi189.stdout:315: left oid 41 (ObjNum 125 snap 27 seq_num 125) 2024-09-18T10:15:44.643 INFO:tasks.rados.rados.0.smithi189.stdout:325: finishing write tid 6 to smithi18925896-42 2024-09-18T10:15:44.643 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 42 v 86 (ObjNum 129 snap 27 seq_num 129) dirty exists 2024-09-18T10:15:44.643 INFO:tasks.rados.rados.0.smithi189.stdout:325: left oid 42 (ObjNum 129 snap 27 seq_num 129) 2024-09-18T10:15:44.643 INFO:tasks.rados.rados.0.smithi189.stdout:334: finishing write tid 1 to smithi18925896-28 2024-09-18T10:15:44.643 INFO:tasks.rados.rados.0.smithi189.stdout:334: finishing write tid 2 to smithi18925896-28 2024-09-18T10:15:44.643 INFO:tasks.rados.rados.0.smithi189.stdout:334: finishing write tid 3 to smithi18925896-28 2024-09-18T10:15:44.643 INFO:tasks.rados.rados.0.smithi189.stdout:334: finishing write tid 4 to smithi18925896-28 2024-09-18T10:15:44.643 INFO:tasks.rados.rados.0.smithi189.stdout:331: finishing write tid 1 to smithi18925896-38 2024-09-18T10:15:44.644 INFO:tasks.rados.rados.0.smithi189.stdout:331: finishing write tid 2 to smithi18925896-38 2024-09-18T10:15:44.644 INFO:tasks.rados.rados.0.smithi189.stdout:331: finishing write tid 3 to smithi18925896-38 2024-09-18T10:15:44.644 INFO:tasks.rados.rados.0.smithi189.stdout:331: finishing write tid 4 to smithi18925896-38 2024-09-18T10:15:44.644 INFO:tasks.rados.rados.0.smithi189.stdout:336: finishing copy_from racing read to smithi18925896-40 2024-09-18T10:15:44.644 INFO:tasks.rados.rados.0.smithi189.stdout:334: finishing write tid 5 to smithi18925896-28 2024-09-18T10:15:44.644 INFO:tasks.rados.rados.0.smithi189.stdout:331: finishing write tid 5 to smithi18925896-38 2024-09-18T10:15:44.644 INFO:tasks.rados.rados.0.smithi189.stdout:335: finishing write tid 1 to smithi18925896-23 2024-09-18T10:15:44.644 INFO:tasks.rados.rados.0.smithi189.stdout:336: finishing copy_from to smithi18925896-40 2024-09-18T10:15:44.644 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 40 v 33 (ObjNum 90 snap 16 seq_num 90) dirty exists 2024-09-18T10:15:44.644 INFO:tasks.rados.rados.0.smithi189.stdout:334: finishing write tid 6 to smithi18925896-28 2024-09-18T10:15:44.644 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 28 v 57 (ObjNum 131 snap 28 seq_num 131) dirty exists 2024-09-18T10:15:44.644 INFO:tasks.rados.rados.0.smithi189.stdout:334: left oid 28 (ObjNum 131 snap 28 seq_num 131) 2024-09-18T10:15:44.644 INFO:tasks.rados.rados.0.smithi189.stdout:335: finishing write tid 2 to smithi18925896-23 2024-09-18T10:15:44.645 INFO:tasks.rados.rados.0.smithi189.stdout:335: finishing write tid 3 to smithi18925896-23 2024-09-18T10:15:44.645 INFO:tasks.rados.rados.0.smithi189.stdout:331: finishing write tid 6 to smithi18925896-38 2024-09-18T10:15:44.645 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 38 v 116 (ObjNum 130 snap 28 seq_num 130) dirty exists 2024-09-18T10:15:44.645 INFO:tasks.rados.rados.0.smithi189.stdout:331: left oid 38 (ObjNum 130 snap 28 seq_num 130) 2024-09-18T10:15:44.645 INFO:tasks.rados.rados.0.smithi189.stdout:339: finishing copy_from to smithi18925896-30 2024-09-18T10:15:44.645 INFO:tasks.rados.rados.0.smithi189.stdout:339: got expected ENOENT (src dne) 2024-09-18T10:15:44.645 INFO:tasks.rados.rados.0.smithi189.stdout:339: finishing copy_from racing read to smithi18925896-30 2024-09-18T10:15:44.645 INFO:tasks.rados.rados.0.smithi189.stdout:335: finishing write tid 4 to smithi18925896-23 2024-09-18T10:15:44.645 INFO:tasks.rados.rados.0.smithi189.stdout:335: finishing write tid 5 to smithi18925896-23 2024-09-18T10:15:44.645 INFO:tasks.rados.rados.0.smithi189.stdout:337: finishing write tid 1 to smithi18925896-31 2024-09-18T10:15:44.645 INFO:tasks.rados.rados.0.smithi189.stdout:337: finishing write tid 2 to smithi18925896-31 2024-09-18T10:15:44.645 INFO:tasks.rados.rados.0.smithi189.stdout:337: finishing write tid 3 to smithi18925896-31 2024-09-18T10:15:44.645 INFO:tasks.rados.rados.0.smithi189.stdout:337: finishing write tid 4 to smithi18925896-31 2024-09-18T10:15:44.646 INFO:tasks.rados.rados.0.smithi189.stdout:337: finishing write tid 5 to smithi18925896-31 2024-09-18T10:15:44.646 INFO:tasks.rados.rados.0.smithi189.stdout:335: finishing write tid 6 to smithi18925896-23 2024-09-18T10:15:44.646 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 23 v 128 (ObjNum 132 snap 28 seq_num 132) dirty exists 2024-09-18T10:15:44.646 INFO:tasks.rados.rados.0.smithi189.stdout:335: left oid 23 (ObjNum 132 snap 28 seq_num 132) 2024-09-18T10:15:44.646 INFO:tasks.rados.rados.0.smithi189.stdout:337: finishing write tid 6 to smithi18925896-31 2024-09-18T10:15:44.646 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 31 v 134 (ObjNum 133 snap 28 seq_num 133) dirty exists 2024-09-18T10:15:44.646 INFO:tasks.rados.rados.0.smithi189.stdout:337: left oid 31 (ObjNum 133 snap 28 seq_num 133) 2024-09-18T10:15:44.646 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T10:15:44.646 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T10:15:44.647 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T10:15:44.647 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T10:15:44.647 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T10:15:44.647 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T10:15:44.647 INFO:teuthology.orchestra.run.smithi029.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T10:15:44.647 INFO:teuthology.orchestra.run.smithi029.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T10:15:44.647 INFO:teuthology.orchestra.run.smithi029.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-18T10:15:44.647 INFO:teuthology.orchestra.run.smithi029.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-18T10:15:44.647 INFO:teuthology.orchestra.run.smithi029.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-18T10:15:44.647 INFO:teuthology.orchestra.run.smithi029.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-18T10:15:44.647 INFO:teuthology.orchestra.run.smithi029.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-18T10:15:44.647 INFO:teuthology.orchestra.run.smithi029.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-18T10:15:44.647 INFO:teuthology.orchestra.run.smithi029.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-18T10:15:44.648 INFO:teuthology.orchestra.run.smithi029.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-18T10:15:44.725 INFO:tasks.ceph.osd.8.smithi029.stderr:2024-09-18T10:15:44.717+0000 7f2555c27700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 22192) UID: 0 2024-09-18T10:15:44.728 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-18T10:15:44.728 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 20, 2024-09-18T10:15:44.728 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 600, 2024-09-18T10:15:44.729 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [ 2024-09-18T10:15:44.729 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.729 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4257.0:19 1.0 e24/23 1:30306672:devicehealth::main.db.0000000000000000:head v 24'9, mlcod=24'7)", 2024-09-18T10:15:44.729 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:33.346192+0000", 2024-09-18T10:15:44.729 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 71.379243750000001, 2024-09-18T10:15:44.729 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20178458699999999, 2024-09-18T10:15:44.729 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:44.729 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:44.729 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:44.729 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.729 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:44.729 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:33.346192+0000", 2024-09-18T10:15:44.729 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:44.730 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.730 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.730 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:44.730 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:33.346192+0000", 2024-09-18T10:15:44.730 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:44.730 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.730 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.730 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:44.730 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:33.346190+0000", 2024-09-18T10:15:44.730 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:44.730 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.730 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.730 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:44.731 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:33.346206+0000", 2024-09-18T10:15:44.731 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6257e-05 2024-09-18T10:15:44.731 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.731 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.731 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:44.731 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:33.346237+0000", 2024-09-18T10:15:44.731 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0471999999999999e-05 2024-09-18T10:15:44.731 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.731 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.731 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:44.731 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:33.446377+0000", 2024-09-18T10:15:44.731 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10014044499999999 2024-09-18T10:15:44.731 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.731 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.732 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:44.732 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:33.546722+0000", 2024-09-18T10:15:44.732 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100344593 2024-09-18T10:15:44.732 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.732 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.732 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:44.732 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:33.546787+0000", 2024-09-18T10:15:44.732 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.5347999999999998e-05 2024-09-18T10:15:44.732 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.732 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.732 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:44.732 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:33.547891+0000", 2024-09-18T10:15:44.732 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001104379 2024-09-18T10:15:44.733 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.733 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.733 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:44.733 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:33.547976+0000", 2024-09-18T10:15:44.733 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.5023000000000002e-05 2024-09-18T10:15:44.733 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:44.733 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:44.733 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:44.733 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.733 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.733 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4257.0:48 1.0 e26/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 26'30, mlcod=26'22)", 2024-09-18T10:15:44.733 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:35.294144+0000", 2024-09-18T10:15:44.733 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 69.431291607000006, 2024-09-18T10:15:44.734 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20346468400000001, 2024-09-18T10:15:44.734 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:44.734 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:44.734 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:44.734 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.734 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:44.734 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:35.294144+0000", 2024-09-18T10:15:44.734 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:44.734 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.734 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.734 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:44.734 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:35.294144+0000", 2024-09-18T10:15:44.734 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:44.735 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.735 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.735 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:44.735 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:35.294142+0000", 2024-09-18T10:15:44.735 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:44.735 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.735 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.735 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:44.735 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:35.294156+0000", 2024-09-18T10:15:44.735 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4059e-05 2024-09-18T10:15:44.735 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.735 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.735 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:44.735 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:35.294209+0000", 2024-09-18T10:15:44.736 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.2164999999999999e-05 2024-09-18T10:15:44.736 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.736 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.736 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:44.736 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:35.394305+0000", 2024-09-18T10:15:44.736 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10009605000000001 2024-09-18T10:15:44.736 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.736 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.736 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:44.736 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:35.394422+0000", 2024-09-18T10:15:44.736 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000117093 2024-09-18T10:15:44.736 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.737 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.737 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:44.737 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:35.394460+0000", 2024-09-18T10:15:44.737 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8037999999999997e-05 2024-09-18T10:15:44.737 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.737 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.737 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:44.737 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:35.497437+0000", 2024-09-18T10:15:44.737 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.102977206 2024-09-18T10:15:44.737 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.737 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.737 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:44.737 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:35.497609+0000", 2024-09-18T10:15:44.737 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00017165299999999999 2024-09-18T10:15:44.738 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:44.738 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:44.738 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:44.738 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.738 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.738 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4257.0:96 1.0 e29/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 29'70, mlcod=29'68)", 2024-09-18T10:15:44.738 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:39.427232+0000", 2024-09-18T10:15:44.738 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 65.298203482999995, 2024-09-18T10:15:44.738 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20408627300000001, 2024-09-18T10:15:44.738 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:44.738 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:44.738 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:44.738 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.739 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:44.739 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.427232+0000", 2024-09-18T10:15:44.739 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:44.739 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.739 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.739 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:44.739 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.427232+0000", 2024-09-18T10:15:44.739 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:44.739 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.739 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.739 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:44.739 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.427230+0000", 2024-09-18T10:15:44.739 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:44.739 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.740 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.740 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:44.740 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.427399+0000", 2024-09-18T10:15:44.740 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000168787 2024-09-18T10:15:44.740 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.740 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.740 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:44.740 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.427431+0000", 2024-09-18T10:15:44.740 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2372000000000001e-05 2024-09-18T10:15:44.740 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.740 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.740 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:44.741 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.427444+0000", 2024-09-18T10:15:44.741 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2303000000000001e-05 2024-09-18T10:15:44.741 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.741 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.741 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:44.741 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.427580+0000", 2024-09-18T10:15:44.741 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000136019 2024-09-18T10:15:44.741 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.741 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.741 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:44.741 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.427622+0000", 2024-09-18T10:15:44.741 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.2551e-05 2024-09-18T10:15:44.742 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.742 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.742 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:44.742 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.631238+0000", 2024-09-18T10:15:44.742 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20361580100000001 2024-09-18T10:15:44.742 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.742 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.742 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:44.742 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.631318+0000", 2024-09-18T10:15:44.742 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.0370000000000005e-05 2024-09-18T10:15:44.742 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:44.742 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:44.742 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:44.742 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.742 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.743 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4257.0:97 1.0 e29/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 29'71, mlcod=29'68)", 2024-09-18T10:15:44.743 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:39.428800+0000", 2024-09-18T10:15:44.743 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 65.296635541000001, 2024-09-18T10:15:44.743 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.202566529, 2024-09-18T10:15:44.743 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:44.743 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:44.743 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:44.743 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.743 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:44.743 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.428800+0000", 2024-09-18T10:15:44.743 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:44.743 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.743 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.744 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:44.744 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.428800+0000", 2024-09-18T10:15:44.744 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:44.744 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.744 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.744 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:44.744 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.428798+0000", 2024-09-18T10:15:44.744 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:44.744 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.744 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.744 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:44.744 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.428814+0000", 2024-09-18T10:15:44.744 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5262999999999999e-05 2024-09-18T10:15:44.745 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.745 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.745 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:44.745 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.428836+0000", 2024-09-18T10:15:44.745 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.2096e-05 2024-09-18T10:15:44.745 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.745 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.745 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:44.745 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.428847+0000", 2024-09-18T10:15:44.745 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1272e-05 2024-09-18T10:15:44.745 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.745 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.745 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:44.747 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.529081+0000", 2024-09-18T10:15:44.747 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100233853 2024-09-18T10:15:44.747 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.747 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.747 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:44.747 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.529115+0000", 2024-09-18T10:15:44.748 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3862000000000002e-05 2024-09-18T10:15:44.748 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.748 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.748 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:44.748 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.631329+0000", 2024-09-18T10:15:44.748 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.102214465 2024-09-18T10:15:44.748 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.748 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.748 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:44.748 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.631367+0000", 2024-09-18T10:15:44.748 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7450999999999998e-05 2024-09-18T10:15:44.748 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:44.749 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:44.749 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:44.749 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.749 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.749 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4257.0:98 1.0 e29/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 29'72, mlcod=29'68)", 2024-09-18T10:15:44.749 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:39.429995+0000", 2024-09-18T10:15:44.749 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 65.295440318000004, 2024-09-18T10:15:44.749 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20141884500000001, 2024-09-18T10:15:44.749 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:44.749 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:44.749 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:44.749 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.749 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:44.750 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.429995+0000", 2024-09-18T10:15:44.750 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:44.750 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.750 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.750 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:44.750 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.429995+0000", 2024-09-18T10:15:44.750 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:44.750 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.750 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.750 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:44.750 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.429994+0000", 2024-09-18T10:15:44.750 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:44.750 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.751 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.751 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:44.751 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.430008+0000", 2024-09-18T10:15:44.751 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4185999999999999e-05 2024-09-18T10:15:44.751 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.751 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.751 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:44.751 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.430036+0000", 2024-09-18T10:15:44.751 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7878999999999999e-05 2024-09-18T10:15:44.751 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.751 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.751 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:44.751 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.430048+0000", 2024-09-18T10:15:44.751 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1725999999999999e-05 2024-09-18T10:15:44.752 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.752 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.752 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:44.752 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.629731+0000", 2024-09-18T10:15:44.752 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.19968341100000001 2024-09-18T10:15:44.752 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.752 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.752 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:44.752 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.629764+0000", 2024-09-18T10:15:44.752 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3572999999999999e-05 2024-09-18T10:15:44.752 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.752 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.752 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:44.752 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.631372+0000", 2024-09-18T10:15:44.753 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0016074279999999999 2024-09-18T10:15:44.753 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.753 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.753 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:44.753 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.631414+0000", 2024-09-18T10:15:44.753 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.2191999999999999e-05 2024-09-18T10:15:44.753 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:44.753 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:44.753 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:44.753 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.753 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.753 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4257.0:99 1.0 e29/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 29'73, mlcod=29'68)", 2024-09-18T10:15:44.753 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:39.431146+0000", 2024-09-18T10:15:44.754 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 65.294289594000006, 2024-09-18T10:15:44.754 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.200928938, 2024-09-18T10:15:44.754 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:44.754 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:44.754 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:44.754 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.754 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:44.754 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.431146+0000", 2024-09-18T10:15:44.754 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:44.754 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.754 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.754 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:44.754 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.431146+0000", 2024-09-18T10:15:44.755 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:44.755 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.755 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.755 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:44.755 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.431144+0000", 2024-09-18T10:15:44.755 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:44.755 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.755 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.755 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:44.755 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.431218+0000", 2024-09-18T10:15:44.755 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.3577000000000004e-05 2024-09-18T10:15:44.755 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.756 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.756 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:44.756 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.431250+0000", 2024-09-18T10:15:44.756 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2268999999999999e-05 2024-09-18T10:15:44.756 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.756 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.756 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:44.756 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.431262+0000", 2024-09-18T10:15:44.756 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2053e-05 2024-09-18T10:15:44.756 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.756 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.757 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:44.757 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.630328+0000", 2024-09-18T10:15:44.757 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.19906532900000001 2024-09-18T10:15:44.757 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.757 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.757 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:44.757 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.630359+0000", 2024-09-18T10:15:44.757 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1606000000000002e-05 2024-09-18T10:15:44.757 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.757 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.757 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:44.757 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.632015+0000", 2024-09-18T10:15:44.757 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0016557900000000001 2024-09-18T10:15:44.757 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.758 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.758 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:44.758 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.632075+0000", 2024-09-18T10:15:44.758 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9904000000000001e-05 2024-09-18T10:15:44.758 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:44.758 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:44.758 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:44.758 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.758 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.758 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4257.0:103 1.0 e29/23 1:30306672:devicehealth::main.db.0000000000000000:head v 29'77, mlcod=29'73)", 2024-09-18T10:15:44.758 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:40.036738+0000", 2024-09-18T10:15:44.758 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 64.688697535000003, 2024-09-18T10:15:44.758 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20132596899999999, 2024-09-18T10:15:44.758 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:44.759 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:44.759 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:44.759 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.759 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:44.759 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.036738+0000", 2024-09-18T10:15:44.759 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:44.759 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.759 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.759 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:44.759 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.036738+0000", 2024-09-18T10:15:44.759 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:44.759 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.759 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.759 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:44.759 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.036737+0000", 2024-09-18T10:15:44.760 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:44.760 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.760 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.760 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:44.760 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.036804+0000", 2024-09-18T10:15:44.760 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.7178e-05 2024-09-18T10:15:44.760 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.760 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.760 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:44.760 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.036821+0000", 2024-09-18T10:15:44.760 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7153e-05 2024-09-18T10:15:44.760 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.760 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.761 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:44.761 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.136954+0000", 2024-09-18T10:15:44.761 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100132572 2024-09-18T10:15:44.761 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.761 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.761 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:44.761 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.137090+0000", 2024-09-18T10:15:44.761 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000136332 2024-09-18T10:15:44.761 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.761 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.761 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:44.761 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.137124+0000", 2024-09-18T10:15:44.761 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3458999999999999e-05 2024-09-18T10:15:44.761 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.762 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.762 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:44.762 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.237999+0000", 2024-09-18T10:15:44.762 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100875385 2024-09-18T10:15:44.762 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.762 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.762 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:44.762 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.238064+0000", 2024-09-18T10:15:44.762 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.5086999999999995e-05 2024-09-18T10:15:44.762 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:44.762 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:44.762 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:44.762 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.762 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.762 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4257.0:113 1.0 e29/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 29'83, mlcod=29'80)", 2024-09-18T10:15:44.763 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:40.801786+0000", 2024-09-18T10:15:44.763 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 63.923650031000001, 2024-09-18T10:15:44.763 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.203408961, 2024-09-18T10:15:44.763 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:44.763 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:44.763 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:44.763 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.763 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:44.763 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.801786+0000", 2024-09-18T10:15:44.763 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:44.763 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.763 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.764 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:44.764 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.801786+0000", 2024-09-18T10:15:44.764 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:44.764 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.764 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.764 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:44.764 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.801784+0000", 2024-09-18T10:15:44.764 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:44.764 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.764 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.764 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:44.765 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.801801+0000", 2024-09-18T10:15:44.765 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7223000000000001e-05 2024-09-18T10:15:44.765 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.765 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.765 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:44.765 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.801823+0000", 2024-09-18T10:15:44.765 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1469000000000001e-05 2024-09-18T10:15:44.765 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.765 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.765 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:44.765 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.801835+0000", 2024-09-18T10:15:44.765 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2033e-05 2024-09-18T10:15:44.765 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.765 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.766 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:44.766 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.902049+0000", 2024-09-18T10:15:44.766 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100213659 2024-09-18T10:15:44.766 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.766 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.766 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:44.766 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.902082+0000", 2024-09-18T10:15:44.766 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3668999999999999e-05 2024-09-18T10:15:44.766 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.766 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.766 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:44.766 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.005131+0000", 2024-09-18T10:15:44.767 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.103048379 2024-09-18T10:15:44.767 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.767 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.767 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:44.767 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.005195+0000", 2024-09-18T10:15:44.767 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.3805000000000003e-05 2024-09-18T10:15:44.767 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:44.767 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:44.767 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:44.767 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.767 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.767 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4257.0:114 1.0 e29/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 29'84, mlcod=29'80)", 2024-09-18T10:15:44.767 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:40.802657+0000", 2024-09-18T10:15:44.767 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 63.922778897000001, 2024-09-18T10:15:44.768 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20256802600000001, 2024-09-18T10:15:44.768 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:44.768 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:44.768 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:44.768 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.768 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:44.768 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.802657+0000", 2024-09-18T10:15:44.768 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:44.768 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.768 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.768 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:44.768 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.802657+0000", 2024-09-18T10:15:44.768 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:44.769 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.769 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.769 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:44.769 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.802656+0000", 2024-09-18T10:15:44.769 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:15:44.769 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.769 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.769 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:44.769 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.802824+0000", 2024-09-18T10:15:44.769 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000168784 2024-09-18T10:15:44.769 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.769 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.769 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:44.769 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.802844+0000", 2024-09-18T10:15:44.770 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0058999999999999e-05 2024-09-18T10:15:44.770 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.770 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.770 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:44.770 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.802857+0000", 2024-09-18T10:15:44.770 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2163e-05 2024-09-18T10:15:44.770 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.770 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.770 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:44.770 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.902531+0000", 2024-09-18T10:15:44.770 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.099674591000000007 2024-09-18T10:15:44.770 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.770 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.770 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:44.771 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.902559+0000", 2024-09-18T10:15:44.771 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7483000000000001e-05 2024-09-18T10:15:44.771 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.771 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.771 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:44.771 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.005198+0000", 2024-09-18T10:15:44.771 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.102638853 2024-09-18T10:15:44.771 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.771 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.771 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:44.771 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.005225+0000", 2024-09-18T10:15:44.771 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7206e-05 2024-09-18T10:15:44.772 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:44.772 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:44.772 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:44.772 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.772 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.772 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4257.0:115 1.0 e29/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 29'85, mlcod=29'80)", 2024-09-18T10:15:44.772 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:40.803806+0000", 2024-09-18T10:15:44.772 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 63.921629510000002, 2024-09-18T10:15:44.772 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.201443711, 2024-09-18T10:15:44.772 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:44.772 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:44.772 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:44.772 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.772 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:44.773 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.803806+0000", 2024-09-18T10:15:44.773 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:44.773 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.773 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.773 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:44.773 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.803806+0000", 2024-09-18T10:15:44.773 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:44.773 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.773 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.773 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:44.773 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.803804+0000", 2024-09-18T10:15:44.773 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:44.773 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.773 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.774 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:44.774 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.803818+0000", 2024-09-18T10:15:44.774 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3477e-05 2024-09-18T10:15:44.774 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.774 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.774 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:44.774 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.803851+0000", 2024-09-18T10:15:44.774 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3086000000000001e-05 2024-09-18T10:15:44.774 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.774 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.774 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:44.774 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.803867+0000", 2024-09-18T10:15:44.774 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5885000000000001e-05 2024-09-18T10:15:44.775 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.775 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.775 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:44.775 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.903206+0000", 2024-09-18T10:15:44.775 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.099338990000000002 2024-09-18T10:15:44.775 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.775 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.775 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:44.775 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.903253+0000", 2024-09-18T10:15:44.775 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.7419000000000001e-05 2024-09-18T10:15:44.775 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.775 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.775 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:44.775 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.005227+0000", 2024-09-18T10:15:44.776 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.101973199 2024-09-18T10:15:44.776 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.776 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.776 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:44.776 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.005250+0000", 2024-09-18T10:15:44.776 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3261999999999999e-05 2024-09-18T10:15:44.776 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:44.776 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:44.776 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:44.776 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.776 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.776 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4257.0:117 1.0 e29/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 29'87, mlcod=29'80)", 2024-09-18T10:15:44.776 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:40.805727+0000", 2024-09-18T10:15:44.777 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 63.919708272000001, 2024-09-18T10:15:44.777 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.200946191, 2024-09-18T10:15:44.777 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:44.777 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:44.777 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:44.777 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.777 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:44.777 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.805727+0000", 2024-09-18T10:15:44.777 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:44.777 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.777 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.777 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:44.778 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.805727+0000", 2024-09-18T10:15:44.778 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:44.778 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.778 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.778 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:44.778 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.805726+0000", 2024-09-18T10:15:44.778 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:44.778 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.778 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.778 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:44.778 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.805783+0000", 2024-09-18T10:15:44.778 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.7068000000000003e-05 2024-09-18T10:15:44.778 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.779 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.779 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:44.779 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.805817+0000", 2024-09-18T10:15:44.779 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4471999999999998e-05 2024-09-18T10:15:44.779 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.779 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.779 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:44.779 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.805833+0000", 2024-09-18T10:15:44.779 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6313000000000001e-05 2024-09-18T10:15:44.779 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.779 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.779 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:44.779 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.004481+0000", 2024-09-18T10:15:44.779 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.198647823 2024-09-18T10:15:44.779 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.780 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.780 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:44.780 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.004520+0000", 2024-09-18T10:15:44.780 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8992000000000002e-05 2024-09-18T10:15:44.780 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.780 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.780 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:44.780 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.006585+0000", 2024-09-18T10:15:44.780 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0020647389999999999 2024-09-18T10:15:44.780 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.780 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.780 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:44.780 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.006673+0000", 2024-09-18T10:15:44.780 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.8443999999999995e-05 2024-09-18T10:15:44.781 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:44.781 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:44.781 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:44.781 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.781 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.781 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4257.0:128 1.0 e29/23 1:30306672:devicehealth::main.db.0000000000000000:head v 29'98, mlcod=29'95)", 2024-09-18T10:15:44.781 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:41.222806+0000", 2024-09-18T10:15:44.781 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 63.502629630999998, 2024-09-18T10:15:44.781 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20200015099999999, 2024-09-18T10:15:44.781 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:44.781 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:44.781 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:44.782 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.782 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:44.782 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.222806+0000", 2024-09-18T10:15:44.782 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:44.782 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.782 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.782 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:44.782 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.222806+0000", 2024-09-18T10:15:44.782 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:44.782 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.782 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.782 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:44.782 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.222804+0000", 2024-09-18T10:15:44.782 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:44.783 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.783 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.783 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:44.783 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.222885+0000", 2024-09-18T10:15:44.783 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.0140000000000002e-05 2024-09-18T10:15:44.783 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.783 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.783 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:44.783 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.222915+0000", 2024-09-18T10:15:44.783 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0168999999999999e-05 2024-09-18T10:15:44.783 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.783 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.783 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:44.783 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.323060+0000", 2024-09-18T10:15:44.784 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100145475 2024-09-18T10:15:44.784 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.784 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.784 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:44.784 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.423304+0000", 2024-09-18T10:15:44.784 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100243924 2024-09-18T10:15:44.784 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.784 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.784 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:44.784 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.423338+0000", 2024-09-18T10:15:44.784 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3819000000000003e-05 2024-09-18T10:15:44.784 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.784 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.784 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:44.784 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.424724+0000", 2024-09-18T10:15:44.785 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0013862690000000001 2024-09-18T10:15:44.785 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.785 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.785 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:44.785 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.424806+0000", 2024-09-18T10:15:44.785 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.1901000000000001e-05 2024-09-18T10:15:44.785 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:44.785 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:44.785 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:44.785 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.785 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.785 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.4s1 41/40 ECSubWrite(tid=17, reqid=osd.8.0:17, at_version=41'10, trim_to=0'0, roll_forward_to=0'0))", 2024-09-18T10:15:44.785 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:57.445149+0000", 2024-09-18T10:15:44.785 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 47.280287069000003, 2024-09-18T10:15:44.786 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20394425099999999, 2024-09-18T10:15:44.786 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:44.786 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:44.786 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:44.786 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.786 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:44.786 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.445149+0000", 2024-09-18T10:15:44.786 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:44.786 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.786 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.786 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:44.786 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.445149+0000", 2024-09-18T10:15:44.786 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:44.787 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.787 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.787 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:44.787 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.445147+0000", 2024-09-18T10:15:44.787 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:44.787 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.787 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.787 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:44.787 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.445221+0000", 2024-09-18T10:15:44.787 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.4140999999999994e-05 2024-09-18T10:15:44.787 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.787 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.787 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:44.787 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.445260+0000", 2024-09-18T10:15:44.787 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8998999999999997e-05 2024-09-18T10:15:44.788 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.788 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.788 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:44.788 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.545386+0000", 2024-09-18T10:15:44.788 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10012621300000001 2024-09-18T10:15:44.788 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.788 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.788 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:44.788 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.645963+0000", 2024-09-18T10:15:44.788 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10057662000000001 2024-09-18T10:15:44.788 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.788 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.788 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:44.789 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.645988+0000", 2024-09-18T10:15:44.789 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5449e-05 2024-09-18T10:15:44.789 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.789 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.789 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:44.789 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.649028+0000", 2024-09-18T10:15:44.789 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0030390840000000001 2024-09-18T10:15:44.789 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.789 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.789 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:44.789 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.649093+0000", 2024-09-18T10:15:44.789 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.5258000000000004e-05 2024-09-18T10:15:44.789 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:44.790 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:44.790 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:44.791 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.792 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.792 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.6s6 41/40 ECSubWrite(tid=31, reqid=client.4632.0:249, at_version=41'13, trim_to=0'0, roll_forward_to=0'0))", 2024-09-18T10:15:44.792 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:57.550336+0000", 2024-09-18T10:15:44.792 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 47.175100030000003, 2024-09-18T10:15:44.792 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.203590574, 2024-09-18T10:15:44.792 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:44.792 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:44.792 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:44.792 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.792 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:44.792 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.550336+0000", 2024-09-18T10:15:44.792 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:44.793 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.793 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.793 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:44.793 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.550336+0000", 2024-09-18T10:15:44.793 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:44.793 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.793 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.793 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:44.793 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.550329+0000", 2024-09-18T10:15:44.793 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999933 2024-09-18T10:15:44.793 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.793 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.793 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:44.793 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.550932+0000", 2024-09-18T10:15:44.793 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00060276999999999998 2024-09-18T10:15:44.794 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.794 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.794 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:44.794 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.550955+0000", 2024-09-18T10:15:44.794 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.2884999999999999e-05 2024-09-18T10:15:44.794 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.794 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.794 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:44.794 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.651052+0000", 2024-09-18T10:15:44.794 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100097174 2024-09-18T10:15:44.794 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.794 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.794 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:44.795 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.651160+0000", 2024-09-18T10:15:44.795 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000108483 2024-09-18T10:15:44.795 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.795 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.795 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:44.795 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.651172+0000", 2024-09-18T10:15:44.795 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1790999999999999e-05 2024-09-18T10:15:44.795 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.795 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.795 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:44.795 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.753875+0000", 2024-09-18T10:15:44.795 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10270337 2024-09-18T10:15:44.795 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.795 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.795 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:44.796 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.753926+0000", 2024-09-18T10:15:44.796 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0767999999999999e-05 2024-09-18T10:15:44.796 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:44.796 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:44.796 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:44.796 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.796 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.796 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.8s4 42/40 ECSubWrite(tid=133, reqid=osd.6.0:132, at_version=0'0, trim_to=0'0, roll_forward_to=41'13))", 2024-09-18T10:15:44.796 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:58.049763+0000", 2024-09-18T10:15:44.796 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 46.675672294000002, 2024-09-18T10:15:44.796 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.21446268399999999, 2024-09-18T10:15:44.796 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:44.797 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:44.797 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:44.797 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.797 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:44.797 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:58.049763+0000", 2024-09-18T10:15:44.797 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:44.797 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.797 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.797 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:44.797 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:58.049763+0000", 2024-09-18T10:15:44.797 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:44.797 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.797 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.797 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:44.798 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:58.049763+0000", 2024-09-18T10:15:44.798 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:44.798 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.798 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.798 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:44.798 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:58.049787+0000", 2024-09-18T10:15:44.798 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3819e-05 2024-09-18T10:15:44.798 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.798 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.798 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:44.798 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:58.049817+0000", 2024-09-18T10:15:44.798 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0749e-05 2024-09-18T10:15:44.798 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.799 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.799 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:44.799 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:58.049836+0000", 2024-09-18T10:15:44.799 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9182999999999999e-05 2024-09-18T10:15:44.799 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.799 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.799 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:44.799 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:58.163409+0000", 2024-09-18T10:15:44.799 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.11357204899999999 2024-09-18T10:15:44.799 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.799 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.799 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:44.799 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:58.163424+0000", 2024-09-18T10:15:44.799 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5403000000000001e-05 2024-09-18T10:15:44.800 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.800 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.800 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:44.800 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:58.264161+0000", 2024-09-18T10:15:44.800 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100736942 2024-09-18T10:15:44.800 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.800 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.800 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:44.800 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:58.264226+0000", 2024-09-18T10:15:44.800 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.5085000000000005e-05 2024-09-18T10:15:44.800 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:44.800 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:44.800 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:44.801 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.801 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.801 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.ds3 47/32 ECSubWrite(tid=56, reqid=client.4632.0:333, at_version=47'9, trim_to=0'0, roll_forward_to=41'6))", 2024-09-18T10:15:44.801 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:02.801124+0000", 2024-09-18T10:15:44.801 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 41.924311136, 2024-09-18T10:15:44.801 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20185083000000001, 2024-09-18T10:15:44.801 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:44.801 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:44.801 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:44.801 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.801 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:44.801 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.801124+0000", 2024-09-18T10:15:44.801 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:44.802 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.802 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.802 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:44.802 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.801124+0000", 2024-09-18T10:15:44.802 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:44.802 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.802 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.802 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:44.802 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.801103+0000", 2024-09-18T10:15:44.802 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999785 2024-09-18T10:15:44.802 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.802 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.802 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:44.802 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.801366+0000", 2024-09-18T10:15:44.803 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00026269299999999999 2024-09-18T10:15:44.803 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.803 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.803 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:44.803 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.801395+0000", 2024-09-18T10:15:44.803 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.9572000000000001e-05 2024-09-18T10:15:44.803 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.803 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.803 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:44.803 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.801412+0000", 2024-09-18T10:15:44.803 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6872999999999999e-05 2024-09-18T10:15:44.803 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.803 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.803 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:44.803 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.000543+0000", 2024-09-18T10:15:44.804 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.19913046700000001 2024-09-18T10:15:44.804 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.804 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.804 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:44.804 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.000579+0000", 2024-09-18T10:15:44.804 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6649000000000001e-05 2024-09-18T10:15:44.804 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.804 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.804 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:44.804 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.002922+0000", 2024-09-18T10:15:44.805 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0023426139999999998 2024-09-18T10:15:44.805 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.805 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.805 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:44.805 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.002975+0000", 2024-09-18T10:15:44.805 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3232000000000001e-05 2024-09-18T10:15:44.805 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:44.805 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:44.805 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:44.805 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.805 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.806 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.15s1 63/60 ECSubWrite(tid=54, reqid=client.4632.0:512, at_version=0'0, trim_to=0'0, roll_forward_to=63'4))", 2024-09-18T10:15:44.806 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:19.215356+0000", 2024-09-18T10:15:44.806 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 25.510079273999999, 2024-09-18T10:15:44.806 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20187551500000001, 2024-09-18T10:15:44.806 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:44.806 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:44.806 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:44.806 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.806 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:44.806 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.215356+0000", 2024-09-18T10:15:44.806 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:44.806 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.806 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.807 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:44.807 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.215356+0000", 2024-09-18T10:15:44.807 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:44.807 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.807 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.807 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:44.807 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.215355+0000", 2024-09-18T10:15:44.807 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:44.807 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.807 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.807 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:44.807 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.215386+0000", 2024-09-18T10:15:44.807 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0809000000000003e-05 2024-09-18T10:15:44.808 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.808 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.808 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:44.808 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.215434+0000", 2024-09-18T10:15:44.808 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.8751999999999997e-05 2024-09-18T10:15:44.808 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.808 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.808 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:44.808 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.315532+0000", 2024-09-18T10:15:44.808 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100097281 2024-09-18T10:15:44.808 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.808 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.809 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:44.809 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.415795+0000", 2024-09-18T10:15:44.809 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10026387 2024-09-18T10:15:44.809 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.809 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.809 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:44.809 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.415823+0000", 2024-09-18T10:15:44.809 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7469000000000001e-05 2024-09-18T10:15:44.809 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.809 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.809 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:44.809 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.417173+0000", 2024-09-18T10:15:44.809 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0013505030000000001 2024-09-18T10:15:44.810 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.810 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.810 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:44.810 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.417232+0000", 2024-09-18T10:15:44.810 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8421000000000002e-05 2024-09-18T10:15:44.810 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:44.810 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:44.810 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:44.810 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.810 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.810 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.4s1 72/40 ECSubWrite(tid=72, reqid=osd.8.0:72, at_version=72'53, trim_to=0'0, roll_forward_to=71'51))", 2024-09-18T10:15:44.810 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:28.190950+0000", 2024-09-18T10:15:44.810 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 16.534485449999998, 2024-09-18T10:15:44.811 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20228795599999999, 2024-09-18T10:15:44.811 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:44.811 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:44.811 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:44.811 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.811 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:44.811 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.190950+0000", 2024-09-18T10:15:44.811 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:44.811 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.811 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.811 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:44.811 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.190950+0000", 2024-09-18T10:15:44.811 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:44.811 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.812 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.812 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:44.812 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.190949+0000", 2024-09-18T10:15:44.812 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:44.812 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.812 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.812 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:44.812 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.191001+0000", 2024-09-18T10:15:44.812 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.2581999999999999e-05 2024-09-18T10:15:44.812 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.812 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.812 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:44.813 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.191022+0000", 2024-09-18T10:15:44.813 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0792999999999999e-05 2024-09-18T10:15:44.813 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.813 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.813 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:44.813 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.191034+0000", 2024-09-18T10:15:44.813 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1603000000000001e-05 2024-09-18T10:15:44.813 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.813 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.813 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:44.813 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.391648+0000", 2024-09-18T10:15:44.813 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20061451299999999 2024-09-18T10:15:44.814 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.814 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.814 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:44.814 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.391678+0000", 2024-09-18T10:15:44.814 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0199e-05 2024-09-18T10:15:44.814 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.814 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.814 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:44.814 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.393192+0000", 2024-09-18T10:15:44.814 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0015135599999999999 2024-09-18T10:15:44.814 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.815 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.815 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:44.815 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.393238+0000", 2024-09-18T10:15:44.815 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.6128999999999998e-05 2024-09-18T10:15:44.815 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:44.815 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:44.815 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:44.815 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.815 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.815 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.4s1 72/40 ECSubWrite(tid=74, reqid=osd.8.0:72, at_version=0'0, trim_to=0'0, roll_forward_to=72'53))", 2024-09-18T10:15:44.815 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:28.496056+0000", 2024-09-18T10:15:44.815 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 16.229379367, 2024-09-18T10:15:44.816 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20153745300000001, 2024-09-18T10:15:44.816 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:44.816 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:44.816 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:44.816 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.816 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:44.816 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.496056+0000", 2024-09-18T10:15:44.816 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:44.816 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.816 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.816 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:44.816 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.496056+0000", 2024-09-18T10:15:44.816 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:44.816 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.817 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.817 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:44.817 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.496055+0000", 2024-09-18T10:15:44.817 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:44.817 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.817 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.817 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:44.817 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.496088+0000", 2024-09-18T10:15:44.817 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3411999999999998e-05 2024-09-18T10:15:44.817 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.817 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.817 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:44.817 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.496120+0000", 2024-09-18T10:15:44.817 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1745999999999997e-05 2024-09-18T10:15:44.818 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.818 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.818 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:44.818 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.596220+0000", 2024-09-18T10:15:44.818 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100100381 2024-09-18T10:15:44.818 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.818 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.818 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:44.818 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.596320+0000", 2024-09-18T10:15:44.818 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.9745000000000001e-05 2024-09-18T10:15:44.818 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.818 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.818 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:44.819 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.596344+0000", 2024-09-18T10:15:44.819 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4598999999999999e-05 2024-09-18T10:15:44.819 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.819 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.819 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:44.819 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.697512+0000", 2024-09-18T10:15:44.819 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10116728699999999 2024-09-18T10:15:44.819 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.819 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.819 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:44.819 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.697594+0000", 2024-09-18T10:15:44.819 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.1972999999999996e-05 2024-09-18T10:15:44.819 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:44.819 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:44.820 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:44.820 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.820 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.820 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.4s1 75/40 ECSubWrite(tid=80, reqid=osd.8.0:80, at_version=75'61, trim_to=0'0, roll_forward_to=74'59))", 2024-09-18T10:15:44.820 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:30.566285+0000", 2024-09-18T10:15:44.820 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 14.159150207, 2024-09-18T10:15:44.820 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.201584069, 2024-09-18T10:15:44.820 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:44.821 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:44.821 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:44.821 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.821 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:44.821 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.566285+0000", 2024-09-18T10:15:44.821 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:44.821 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.821 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.821 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:44.821 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.566285+0000", 2024-09-18T10:15:44.822 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:44.822 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.822 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.822 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:44.822 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.566285+0000", 2024-09-18T10:15:44.822 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:44.822 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.822 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.822 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:44.822 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.566310+0000", 2024-09-18T10:15:44.823 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5250999999999999e-05 2024-09-18T10:15:44.823 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.823 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.823 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:44.823 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.566319+0000", 2024-09-18T10:15:44.823 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.5159999999999994e-06 2024-09-18T10:15:44.823 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.823 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.823 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:44.823 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.666393+0000", 2024-09-18T10:15:44.823 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10007392399999999 2024-09-18T10:15:44.823 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.823 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.823 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:44.823 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.666455+0000", 2024-09-18T10:15:44.824 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.2372999999999994e-05 2024-09-18T10:15:44.824 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.824 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.824 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:44.824 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.666465+0000", 2024-09-18T10:15:44.824 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.7820000000000005e-06 2024-09-18T10:15:44.824 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.824 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.824 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:44.824 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.767822+0000", 2024-09-18T10:15:44.824 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10135730900000001 2024-09-18T10:15:44.824 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:44.824 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:44.825 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:44.825 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.767869+0000", 2024-09-18T10:15:44.825 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.7181000000000001e-05 2024-09-18T10:15:44.825 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:44.825 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:44.825 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:44.825 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:44.825 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:44.825 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-18T10:15:44.827 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T10:15:44.828 INFO:tasks.ceph.osd.4.smithi029.stderr:2024-09-18T10:15:44.821+0000 7f550cf6e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22189) UID: 0 2024-09-18T10:15:44.928 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-09-18T10:15:44.926+0000 7fdfa8698700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22087) UID: 0 2024-09-18T10:15:44.937 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-18T10:15:44.937 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-18T10:15:44.937 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:44.937 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWriteReply(3.6s0 89/40 ECSubWriteReply(tid=77, last_complete=89'48, committed=1, applied=1))", 2024-09-18T10:15:44.937 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:44.896182+0000", 2024-09-18T10:15:44.937 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.039842711000000003, 2024-09-18T10:15:44.937 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.039876635000000001, 2024-09-18T10:15:44.937 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:44.937 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-18T10:15:44.937 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:44.937 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:44.937 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:44.938 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:44.896182+0000", 2024-09-18T10:15:44.938 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:44.938 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:44.938 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:44.938 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:44.938 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:44.896182+0000", 2024-09-18T10:15:44.938 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:44.938 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:44.938 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:44.938 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:44.938 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:44.896179+0000", 2024-09-18T10:15:44.938 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999976 2024-09-18T10:15:44.938 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:44.938 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:44.939 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:44.939 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:44.896195+0000", 2024-09-18T10:15:44.939 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5455999999999999e-05 2024-09-18T10:15:44.939 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:44.939 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:44.939 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:44.939 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:44.896228+0000", 2024-09-18T10:15:44.939 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2857000000000001e-05 2024-09-18T10:15:44.939 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:44.939 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:44.939 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:44.939 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:44.896246+0000", 2024-09-18T10:15:44.940 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8342000000000002e-05 2024-09-18T10:15:44.940 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:44.940 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:44.940 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:44.940 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:44.940 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-18T10:15:44.940 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 1 2024-09-18T10:15:44.940 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-18T10:15:44.947 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T10:15:45.028 INFO:tasks.ceph.osd.10.smithi154.stderr:2024-09-18T10:15:45.026+0000 7fb73221a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 22091) UID: 0 2024-09-18T10:15:45.046 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-18T10:15:45.046 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-18T10:15:45.046 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-18T10:15:45.046 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-18T10:15:45.046 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-18T10:15:45.055 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T10:15:45.130 INFO:tasks.ceph.osd.15.smithi189.stderr:2024-09-18T10:15:45.126+0000 7fd0e725e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 22401) UID: 0 2024-09-18T10:15:45.158 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-18T10:15:45.158 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-18T10:15:45.159 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-18T10:15:45.159 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-18T10:15:45.159 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.159 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWriteReply(3.6s0 36/32 ECSubWriteReply(tid=11, last_complete=36'7, committed=1, applied=1))", 2024-09-18T10:15:45.159 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:50.290945+0000", 2024-09-18T10:15:45.159 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 54.863201599999996, 2024-09-18T10:15:45.159 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.208748928, 2024-09-18T10:15:45.159 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:45.159 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:45.159 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:45.159 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.159 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:45.159 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.290945+0000", 2024-09-18T10:15:45.160 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:45.160 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.160 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.160 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:45.160 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.290945+0000", 2024-09-18T10:15:45.160 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:45.160 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.160 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.160 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:45.160 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.290945+0000", 2024-09-18T10:15:45.160 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:45.160 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.160 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.161 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:45.161 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.290951+0000", 2024-09-18T10:15:45.161 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.5729999999999996e-06 2024-09-18T10:15:45.161 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.161 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.161 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:45.161 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.290964+0000", 2024-09-18T10:15:45.161 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3012e-05 2024-09-18T10:15:45.161 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.161 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.161 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:45.161 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.290972+0000", 2024-09-18T10:15:45.162 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4699999999999996e-06 2024-09-18T10:15:45.162 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.162 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.162 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:45.162 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.499661+0000", 2024-09-18T10:15:45.162 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.20868911500000001 2024-09-18T10:15:45.162 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.162 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.162 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:45.162 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.499694+0000", 2024-09-18T10:15:45.162 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.3470000000000003e-05 2024-09-18T10:15:45.162 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:45.162 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:45.162 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:45.163 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.163 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.163 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWriteReply(3.6s0 36/32 ECSubWriteReply(tid=11, last_complete=36'7, committed=1, applied=1))", 2024-09-18T10:15:45.163 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:50.291030+0000", 2024-09-18T10:15:45.163 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 54.863117101, 2024-09-18T10:15:45.163 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.208840265, 2024-09-18T10:15:45.163 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:45.163 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:45.163 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:45.163 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.163 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:45.163 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.291030+0000", 2024-09-18T10:15:45.163 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:45.164 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.164 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.164 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:45.164 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.291030+0000", 2024-09-18T10:15:45.164 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:45.164 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.164 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.164 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:45.164 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.291030+0000", 2024-09-18T10:15:45.164 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:45.164 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.164 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.164 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:45.164 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.291032+0000", 2024-09-18T10:15:45.165 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.604e-06 2024-09-18T10:15:45.165 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.165 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.165 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:45.165 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.291038+0000", 2024-09-18T10:15:45.165 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8170000000000004e-06 2024-09-18T10:15:45.165 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.166 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.166 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:45.166 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.291042+0000", 2024-09-18T10:15:45.166 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.839e-06 2024-09-18T10:15:45.166 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.166 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.166 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:45.166 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.499812+0000", 2024-09-18T10:15:45.166 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.208769805 2024-09-18T10:15:45.166 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.166 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.166 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:45.166 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.499870+0000", 2024-09-18T10:15:45.166 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8449000000000002e-05 2024-09-18T10:15:45.167 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:45.167 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:45.167 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:45.167 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.167 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.167 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWriteReply(3.6s0 36/32 ECSubWriteReply(tid=11, last_complete=36'7, committed=1, applied=1))", 2024-09-18T10:15:45.167 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:50.302109+0000", 2024-09-18T10:15:45.167 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 54.852037942000003, 2024-09-18T10:15:45.167 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.29912391399999999, 2024-09-18T10:15:45.167 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:45.167 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:45.167 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:45.167 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.167 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:45.168 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.302109+0000", 2024-09-18T10:15:45.168 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:45.168 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.168 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.168 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:45.168 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.302109+0000", 2024-09-18T10:15:45.168 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:45.168 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.168 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.168 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:45.168 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.302107+0000", 2024-09-18T10:15:45.168 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:45.168 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.168 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.169 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:45.169 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.302119+0000", 2024-09-18T10:15:45.169 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1207e-05 2024-09-18T10:15:45.169 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.169 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.169 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:45.169 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.302150+0000", 2024-09-18T10:15:45.169 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.133e-05 2024-09-18T10:15:45.169 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.169 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.169 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:45.169 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.302164+0000", 2024-09-18T10:15:45.169 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3942e-05 2024-09-18T10:15:45.169 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.170 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.170 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:45.170 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.600015+0000", 2024-09-18T10:15:45.170 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.29785072000000001 2024-09-18T10:15:45.170 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.170 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.170 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:45.170 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.601233+0000", 2024-09-18T10:15:45.170 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0012183599999999999 2024-09-18T10:15:45.170 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:45.170 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:45.170 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:45.170 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.171 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.171 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpReadReply(3.6s0 36/32 ECSubReadReply(tid=10, attrs_read=0))", 2024-09-18T10:15:45.171 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:50.302179+0000", 2024-09-18T10:15:45.171 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 54.851967876000003, 2024-09-18T10:15:45.171 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.30043391800000002, 2024-09-18T10:15:45.171 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:45.171 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:45.171 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:45.171 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.171 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:45.171 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.302179+0000", 2024-09-18T10:15:45.171 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:45.171 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.171 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.172 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:45.172 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.302179+0000", 2024-09-18T10:15:45.172 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:45.172 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.172 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.172 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:45.172 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.302178+0000", 2024-09-18T10:15:45.172 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:45.172 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.172 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.172 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:45.172 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.302201+0000", 2024-09-18T10:15:45.172 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2824e-05 2024-09-18T10:15:45.172 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.172 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.173 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:45.173 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.302217+0000", 2024-09-18T10:15:45.173 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5993000000000001e-05 2024-09-18T10:15:45.173 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.173 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.173 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:45.173 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.302223+0000", 2024-09-18T10:15:45.173 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0820000000000004e-06 2024-09-18T10:15:45.173 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.173 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.173 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:45.173 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.602454+0000", 2024-09-18T10:15:45.173 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.30023072499999998 2024-09-18T10:15:45.173 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.174 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.174 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:45.174 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.602613+0000", 2024-09-18T10:15:45.174 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00015896399999999999 2024-09-18T10:15:45.174 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:45.174 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:45.174 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:45.174 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.174 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.174 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:121 3.6s0 3:6c64c4fa:::smithi18925896-41:head [append 0~688128,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:45.174 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:50.627356+0000", 2024-09-18T10:15:45.174 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 54.526790644999998, 2024-09-18T10:15:45.174 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.38269042599999997, 2024-09-18T10:15:45.175 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:45.175 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:45.175 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:45.175 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:45.175 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:45.175 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 121 2024-09-18T10:15:45.175 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.175 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:45.175 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.175 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:45.175 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.627356+0000", 2024-09-18T10:15:45.175 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:45.175 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.175 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.176 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:45.176 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.627356+0000", 2024-09-18T10:15:45.176 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:45.176 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.176 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.176 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:45.176 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.627353+0000", 2024-09-18T10:15:45.176 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999971 2024-09-18T10:15:45.176 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.176 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.176 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:45.177 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.629112+0000", 2024-09-18T10:15:45.177 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0017585980000000001 2024-09-18T10:15:45.177 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.177 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.177 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:45.177 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.629149+0000", 2024-09-18T10:15:45.177 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.6532999999999997e-05 2024-09-18T10:15:45.177 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.177 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.177 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:45.177 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.629169+0000", 2024-09-18T10:15:45.177 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0699e-05 2024-09-18T10:15:45.177 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.177 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.178 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:45.178 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.706567+0000", 2024-09-18T10:15:45.178 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.077397621999999999 2024-09-18T10:15:45.178 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.178 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.178 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:45.178 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.706944+0000", 2024-09-18T10:15:45.178 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00037710599999999999 2024-09-18T10:15:45.178 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.178 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.178 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:45.178 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.709283+0000", 2024-09-18T10:15:45.178 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0023391689999999999 2024-09-18T10:15:45.178 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.179 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.179 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:45.179 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.711791+0000", 2024-09-18T10:15:45.179 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0025078689999999998 2024-09-18T10:15:45.179 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.179 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.179 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-18T10:15:45.179 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.009925+0000", 2024-09-18T10:15:45.179 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.298134493 2024-09-18T10:15:45.179 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.180 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.180 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:45.180 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.010047+0000", 2024-09-18T10:15:45.180 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000121264 2024-09-18T10:15:45.180 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:45.180 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:45.180 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:45.180 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.180 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.180 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:122 3.6s0 3:6c64c4fa:::smithi18925896-41:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:45.180 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:50.629189+0000", 2024-09-18T10:15:45.180 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 54.524957919000002, 2024-09-18T10:15:45.180 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.47849547399999998, 2024-09-18T10:15:45.180 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:45.181 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:45.181 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:45.181 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:45.181 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:45.181 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 122 2024-09-18T10:15:45.181 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.181 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:45.181 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.181 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:45.181 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.629189+0000", 2024-09-18T10:15:45.181 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:45.181 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.181 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.182 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:45.182 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.629189+0000", 2024-09-18T10:15:45.182 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:45.182 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.182 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.182 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:45.182 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.629187+0000", 2024-09-18T10:15:45.182 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:45.182 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.183 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.183 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:45.183 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.629198+0000", 2024-09-18T10:15:45.183 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0623e-05 2024-09-18T10:15:45.183 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.183 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.183 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:45.183 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.629219+0000", 2024-09-18T10:15:45.183 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0665999999999998e-05 2024-09-18T10:15:45.183 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.183 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.184 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:45.184 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.629231+0000", 2024-09-18T10:15:45.184 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.291e-05 2024-09-18T10:15:45.184 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.184 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.184 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:45.184 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.710475+0000", 2024-09-18T10:15:45.184 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.081244021 2024-09-18T10:15:45.184 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.184 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.184 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:45.184 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.710639+0000", 2024-09-18T10:15:45.184 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000163325 2024-09-18T10:15:45.185 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.185 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.185 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:45.185 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.711435+0000", 2024-09-18T10:15:45.185 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00079575999999999996 2024-09-18T10:15:45.185 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.185 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.185 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:45.185 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.812067+0000", 2024-09-18T10:15:45.185 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10063237799999999 2024-09-18T10:15:45.185 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.185 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.185 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-18T10:15:45.186 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.107562+0000", 2024-09-18T10:15:45.186 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.295494906 2024-09-18T10:15:45.186 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.186 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.186 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:45.186 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.107685+0000", 2024-09-18T10:15:45.186 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00012271100000000001 2024-09-18T10:15:45.186 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:45.186 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:45.186 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:45.186 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.186 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.186 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:123 3.6s0 3:6c64c4fa:::smithi18925896-41:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:45.186 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:50.629246+0000", 2024-09-18T10:15:45.186 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 54.524901092999997, 2024-09-18T10:15:45.187 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.58067324300000001, 2024-09-18T10:15:45.187 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:45.187 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:45.187 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:45.187 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:45.187 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:45.187 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 123 2024-09-18T10:15:45.187 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.187 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:45.187 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.187 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:45.187 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.629246+0000", 2024-09-18T10:15:45.187 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:45.187 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.188 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.188 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:45.188 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.629246+0000", 2024-09-18T10:15:45.188 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:45.188 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.188 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.188 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:45.188 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.629244+0000", 2024-09-18T10:15:45.188 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:45.188 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.188 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.188 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:45.188 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.629254+0000", 2024-09-18T10:15:45.188 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.0759999999999992e-06 2024-09-18T10:15:45.189 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.189 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.189 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:45.189 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.629279+0000", 2024-09-18T10:15:45.189 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.5905999999999999e-05 2024-09-18T10:15:45.189 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.189 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.189 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:45.189 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.629293+0000", 2024-09-18T10:15:45.189 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3073e-05 2024-09-18T10:15:45.189 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.189 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.189 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:45.190 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.806055+0000", 2024-09-18T10:15:45.190 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.17676249999999999 2024-09-18T10:15:45.190 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.190 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.190 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for rw locks", 2024-09-18T10:15:45.190 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.806211+0000", 2024-09-18T10:15:45.190 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000156384 2024-09-18T10:15:45.190 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.190 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.190 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:45.190 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.108025+0000", 2024-09-18T10:15:45.190 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.30181311500000002 2024-09-18T10:15:45.190 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.190 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.191 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:45.191 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.108107+0000", 2024-09-18T10:15:45.191 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.2330000000000006e-05 2024-09-18T10:15:45.192 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.192 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.192 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:45.192 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.209919+0000", 2024-09-18T10:15:45.192 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.101812262 2024-09-18T10:15:45.192 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:45.192 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:45.192 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:45.192 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.192 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.192 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:167 3.6s0 3:6619023d:::smithi18925896-19:head [copy-from ver 4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:45.193 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:51.531881+0000", 2024-09-18T10:15:45.193 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 53.622265874999997, 2024-09-18T10:15:45.193 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.41157491600000001, 2024-09-18T10:15:45.193 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:45.193 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:45.193 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:45.193 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:45.193 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:45.193 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 167 2024-09-18T10:15:45.193 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.193 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:45.194 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.194 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:45.194 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.531881+0000", 2024-09-18T10:15:45.194 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:45.194 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.194 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.194 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:45.194 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.531881+0000", 2024-09-18T10:15:45.194 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:45.194 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.194 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.194 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:45.195 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.531877+0000", 2024-09-18T10:15:45.195 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999962 2024-09-18T10:15:45.195 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.195 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.195 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:45.195 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.531894+0000", 2024-09-18T10:15:45.195 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7269999999999999e-05 2024-09-18T10:15:45.195 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.195 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.195 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:45.195 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.531939+0000", 2024-09-18T10:15:45.195 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4815999999999999e-05 2024-09-18T10:15:45.195 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.195 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.196 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:45.196 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.531963+0000", 2024-09-18T10:15:45.196 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3312e-05 2024-09-18T10:15:45.196 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.196 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.196 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:45.196 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.532107+0000", 2024-09-18T10:15:45.196 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000144547 2024-09-18T10:15:45.196 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.196 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.196 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:45.196 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.532348+0000", 2024-09-18T10:15:45.196 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000240955 2024-09-18T10:15:45.197 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.197 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.197 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:45.197 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.839634+0000", 2024-09-18T10:15:45.197 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.30728566400000001 2024-09-18T10:15:45.197 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.197 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.197 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:45.197 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.841560+0000", 2024-09-18T10:15:45.197 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001926449 2024-09-18T10:15:45.197 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.197 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.197 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-18T10:15:45.197 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.943372+0000", 2024-09-18T10:15:45.198 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.101812176 2024-09-18T10:15:45.198 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.198 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.198 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:45.198 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.943456+0000", 2024-09-18T10:15:45.198 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.3614000000000003e-05 2024-09-18T10:15:45.198 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:45.198 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:45.198 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:45.198 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.198 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.198 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:168 3.6s0 3:6619023d:::smithi18925896-19:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:45.198 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:51.532000+0000", 2024-09-18T10:15:45.198 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 53.622146888000003, 2024-09-18T10:15:45.199 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.30898515999999998, 2024-09-18T10:15:45.199 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:45.199 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:45.199 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:45.199 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:45.199 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:45.199 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 168 2024-09-18T10:15:45.199 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.199 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:45.199 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.199 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:45.199 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.532000+0000", 2024-09-18T10:15:45.200 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:45.200 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.200 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.200 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:45.200 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.532000+0000", 2024-09-18T10:15:45.200 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:45.200 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.200 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.200 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:45.200 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.531998+0000", 2024-09-18T10:15:45.200 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:45.200 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.200 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.200 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:45.201 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.532009+0000", 2024-09-18T10:15:45.201 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0988999999999999e-05 2024-09-18T10:15:45.201 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.201 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.201 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:45.201 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.532032+0000", 2024-09-18T10:15:45.201 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2203000000000001e-05 2024-09-18T10:15:45.201 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.201 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.201 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:45.201 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.532043+0000", 2024-09-18T10:15:45.201 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1707e-05 2024-09-18T10:15:45.202 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.202 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.202 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:45.202 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.532663+0000", 2024-09-18T10:15:45.202 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000619993 2024-09-18T10:15:45.202 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.202 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.202 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for blocked object", 2024-09-18T10:15:45.202 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.532735+0000", 2024-09-18T10:15:45.202 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.2025000000000002e-05 2024-09-18T10:15:45.202 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.202 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.202 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:45.203 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.840712+0000", 2024-09-18T10:15:45.204 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.307976582 2024-09-18T10:15:45.204 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.204 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.204 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:45.204 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.840866+0000", 2024-09-18T10:15:45.204 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000154203 2024-09-18T10:15:45.204 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.204 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.204 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:45.204 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.840985+0000", 2024-09-18T10:15:45.204 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000119084 2024-09-18T10:15:45.204 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:45.204 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:45.204 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:45.205 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.205 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.205 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:206 3.6s0 3:61491e68:::smithi18925896-27:head [read 0~442368,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:45.205 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:52.694386+0000", 2024-09-18T10:15:45.205 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 52.459760551999999, 2024-09-18T10:15:45.205 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.40544065200000001, 2024-09-18T10:15:45.205 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:45.205 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:45.205 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:45.205 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:45.205 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:45.205 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 206 2024-09-18T10:15:45.205 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.206 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:45.206 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.206 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:45.206 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.694386+0000", 2024-09-18T10:15:45.206 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:45.206 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.206 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.206 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:45.206 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.694386+0000", 2024-09-18T10:15:45.206 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:45.206 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.206 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.206 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:45.207 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.694385+0000", 2024-09-18T10:15:45.207 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.999999 2024-09-18T10:15:45.207 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.207 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.207 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:45.207 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.694392+0000", 2024-09-18T10:15:45.207 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7970000000000003e-06 2024-09-18T10:15:45.207 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.207 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.207 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:45.207 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.694405+0000", 2024-09-18T10:15:45.207 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3406e-05 2024-09-18T10:15:45.207 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.208 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.208 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:45.208 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.694413+0000", 2024-09-18T10:15:45.208 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.9070000000000007e-06 2024-09-18T10:15:45.208 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.208 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.208 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:45.208 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.794592+0000", 2024-09-18T10:15:45.208 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100178879 2024-09-18T10:15:45.208 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.208 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.208 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:45.208 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.794598+0000", 2024-09-18T10:15:45.208 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6289999999999998e-06 2024-09-18T10:15:45.209 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.209 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.209 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:45.209 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.796805+0000", 2024-09-18T10:15:45.209 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0022073930000000002 2024-09-18T10:15:45.209 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.209 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.209 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:45.209 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.796886+0000", 2024-09-18T10:15:45.209 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0641000000000003e-05 2024-09-18T10:15:45.209 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.209 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.210 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:45.210 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:53.099827+0000", 2024-09-18T10:15:45.210 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.302941191 2024-09-18T10:15:45.210 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:45.210 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:45.210 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:45.210 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.210 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.210 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:323 3.6s0 3:61491e68:::smithi18925896-27:head [append 0~0] snapc 7=[4] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-18T10:15:45.210 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:01.795354+0000", 2024-09-18T10:15:45.210 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 43.358793038000002, 2024-09-18T10:15:45.210 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.30418701500000001, 2024-09-18T10:15:45.211 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:45.211 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:45.211 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:45.211 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:45.211 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:45.211 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 323 2024-09-18T10:15:45.211 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.211 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:45.211 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.211 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:45.211 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.795354+0000", 2024-09-18T10:15:45.211 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:45.211 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.212 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.212 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:45.212 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.795354+0000", 2024-09-18T10:15:45.212 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:45.212 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.212 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.212 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:45.212 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.795351+0000", 2024-09-18T10:15:45.212 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999971 2024-09-18T10:15:45.212 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.212 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.212 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:45.213 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.795365+0000", 2024-09-18T10:15:45.213 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3713999999999999e-05 2024-09-18T10:15:45.213 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.213 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.213 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:45.213 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.795406+0000", 2024-09-18T10:15:45.213 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0846999999999997e-05 2024-09-18T10:15:45.213 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.213 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.213 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:45.213 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.795425+0000", 2024-09-18T10:15:45.213 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9936999999999999e-05 2024-09-18T10:15:45.214 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.214 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.214 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:45.214 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.795648+0000", 2024-09-18T10:15:45.214 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000222547 2024-09-18T10:15:45.214 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.214 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.214 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:45.214 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.795872+0000", 2024-09-18T10:15:45.214 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00022384799999999999 2024-09-18T10:15:45.214 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.214 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.214 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:45.215 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.797018+0000", 2024-09-18T10:15:45.215 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001146055 2024-09-18T10:15:45.215 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.215 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.215 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:45.215 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.798609+0000", 2024-09-18T10:15:45.215 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0015905959999999999 2024-09-18T10:15:45.215 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.215 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.215 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-18T10:15:45.215 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.099372+0000", 2024-09-18T10:15:45.215 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.30076301500000002 2024-09-18T10:15:45.215 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.216 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.216 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:45.216 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.099541+0000", 2024-09-18T10:15:45.216 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000169407 2024-09-18T10:15:45.216 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:45.216 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:45.216 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:45.216 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.216 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.216 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:324 3.6s0 3:61491e68:::smithi18925896-27:head [rollback 4] snapc 7=[4] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-18T10:15:45.216 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:01.795482+0000", 2024-09-18T10:15:45.216 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 43.358665424000002, 2024-09-18T10:15:45.216 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.30451560100000002, 2024-09-18T10:15:45.217 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:45.217 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:45.217 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:45.217 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:45.217 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:45.217 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 324 2024-09-18T10:15:45.217 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.217 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:45.217 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.217 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:45.217 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.795482+0000", 2024-09-18T10:15:45.217 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:45.218 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.218 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.218 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:45.218 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.795482+0000", 2024-09-18T10:15:45.218 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:45.218 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.218 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.218 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:45.218 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.795480+0000", 2024-09-18T10:15:45.218 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:45.218 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.218 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.218 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:45.218 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.795487+0000", 2024-09-18T10:15:45.219 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7170000000000002e-06 2024-09-18T10:15:45.219 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.219 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.219 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:45.219 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.795548+0000", 2024-09-18T10:15:45.219 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1647000000000001e-05 2024-09-18T10:15:45.219 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.219 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.219 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:45.219 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.795566+0000", 2024-09-18T10:15:45.219 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7560000000000001e-05 2024-09-18T10:15:45.219 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.220 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.220 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:45.220 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.797676+0000", 2024-09-18T10:15:45.220 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0021101919999999999 2024-09-18T10:15:45.220 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.220 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.220 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:45.220 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.797742+0000", 2024-09-18T10:15:45.220 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.6351000000000002e-05 2024-09-18T10:15:45.220 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.220 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.220 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:45.220 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.798290+0000", 2024-09-18T10:15:45.221 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000548063 2024-09-18T10:15:45.221 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.221 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.221 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:45.221 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.800131+0000", 2024-09-18T10:15:45.221 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001840759 2024-09-18T10:15:45.221 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.221 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.221 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-18T10:15:45.221 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.099840+0000", 2024-09-18T10:15:45.221 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.29970881300000002 2024-09-18T10:15:45.221 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.222 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.222 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:45.222 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.099997+0000", 2024-09-18T10:15:45.222 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000157124 2024-09-18T10:15:45.222 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:45.222 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:45.222 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:45.222 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.222 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.222 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:325 3.6s0 3:61491e68:::smithi18925896-27:head [append 0~0] snapc 7=[4] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-18T10:15:45.222 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:01.795591+0000", 2024-09-18T10:15:45.222 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 43.358556368999999, 2024-09-18T10:15:45.222 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.30495726400000001, 2024-09-18T10:15:45.223 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:45.223 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:45.223 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:45.223 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:45.223 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:45.223 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 325 2024-09-18T10:15:45.223 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.223 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:45.223 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.223 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:45.224 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.795591+0000", 2024-09-18T10:15:45.224 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:45.224 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.224 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.224 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:45.224 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.795591+0000", 2024-09-18T10:15:45.224 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:45.224 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.224 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.224 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:45.224 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.795589+0000", 2024-09-18T10:15:45.224 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:45.224 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.224 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.225 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:45.225 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.795598+0000", 2024-09-18T10:15:45.225 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.7940000000000008e-06 2024-09-18T10:15:45.225 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.225 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.225 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:45.225 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.795617+0000", 2024-09-18T10:15:45.225 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8485e-05 2024-09-18T10:15:45.225 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.225 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.225 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:45.225 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.795627+0000", 2024-09-18T10:15:45.225 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0730999999999999e-05 2024-09-18T10:15:45.226 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.226 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.226 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:45.226 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.798672+0000", 2024-09-18T10:15:45.226 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00304484 2024-09-18T10:15:45.226 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.226 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.226 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:45.226 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.798918+0000", 2024-09-18T10:15:45.226 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00024598000000000003 2024-09-18T10:15:45.226 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.226 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.226 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:45.226 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.799703+0000", 2024-09-18T10:15:45.227 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000785262 2024-09-18T10:15:45.227 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.227 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.227 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:45.227 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.900558+0000", 2024-09-18T10:15:45.227 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100854108 2024-09-18T10:15:45.227 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.227 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.227 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-18T10:15:45.227 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.100434+0000", 2024-09-18T10:15:45.227 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.19987638699999999 2024-09-18T10:15:45.227 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.227 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.228 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:45.228 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.100548+0000", 2024-09-18T10:15:45.228 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00011390799999999999 2024-09-18T10:15:45.228 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:45.228 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:45.228 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:45.228 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.228 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.228 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWriteReply(3.6s0 57/40 ECSubWriteReply(tid=50, last_complete=57'30, committed=1, applied=1))", 2024-09-18T10:15:45.228 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:13.143948+0000", 2024-09-18T10:15:45.228 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 32.010199327000002, 2024-09-18T10:15:45.228 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.21095676099999999, 2024-09-18T10:15:45.228 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:45.229 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:45.229 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:45.229 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.229 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:45.229 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.143948+0000", 2024-09-18T10:15:45.229 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:45.229 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.229 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.229 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:45.229 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.143948+0000", 2024-09-18T10:15:45.229 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:45.229 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.230 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.230 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:45.230 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.143947+0000", 2024-09-18T10:15:45.230 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:45.230 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.230 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.230 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:45.230 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.143950+0000", 2024-09-18T10:15:45.230 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.8959999999999999e-06 2024-09-18T10:15:45.230 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.230 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.230 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:45.230 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.143958+0000", 2024-09-18T10:15:45.230 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7230000000000004e-06 2024-09-18T10:15:45.230 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.231 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.231 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:45.231 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.143961+0000", 2024-09-18T10:15:45.231 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0010000000000002e-06 2024-09-18T10:15:45.231 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.231 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.231 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:45.231 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.354877+0000", 2024-09-18T10:15:45.231 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.21091578599999999 2024-09-18T10:15:45.231 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.231 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.231 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:45.231 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.354904+0000", 2024-09-18T10:15:45.231 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.779e-05 2024-09-18T10:15:45.231 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:45.232 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:45.232 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:45.232 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.232 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.232 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWriteReply(3.6s0 57/40 ECSubWriteReply(tid=51, last_complete=57'32, committed=1, applied=1))", 2024-09-18T10:15:45.232 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:13.144581+0000", 2024-09-18T10:15:45.232 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 32.009565631999997, 2024-09-18T10:15:45.232 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.210571697, 2024-09-18T10:15:45.232 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:45.232 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:45.232 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:45.232 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.232 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:45.233 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.144581+0000", 2024-09-18T10:15:45.233 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:45.233 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.233 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.233 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:45.233 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.144581+0000", 2024-09-18T10:15:45.233 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:45.233 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.233 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.233 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:45.233 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.144581+0000", 2024-09-18T10:15:45.233 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:45.233 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.233 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.234 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:45.234 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.144584+0000", 2024-09-18T10:15:45.234 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.4760000000000001e-06 2024-09-18T10:15:45.234 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.234 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.234 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:45.234 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.144591+0000", 2024-09-18T10:15:45.234 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1969999999999999e-06 2024-09-18T10:15:45.234 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.234 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.234 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:45.234 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.144595+0000", 2024-09-18T10:15:45.234 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9899999999999999e-06 2024-09-18T10:15:45.234 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.234 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.235 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:45.235 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.355136+0000", 2024-09-18T10:15:45.235 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.21054089200000001 2024-09-18T10:15:45.235 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.235 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.235 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:45.235 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.355153+0000", 2024-09-18T10:15:45.235 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7442999999999999e-05 2024-09-18T10:15:45.235 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:45.235 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:45.235 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:45.235 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.235 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.236 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWriteReply(3.6s0 57/40 ECSubWriteReply(tid=51, last_complete=57'32, committed=1, applied=1))", 2024-09-18T10:15:45.236 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:13.144719+0000", 2024-09-18T10:15:45.236 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 32.009427688999999, 2024-09-18T10:15:45.236 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.210498871, 2024-09-18T10:15:45.236 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:45.236 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:45.236 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:45.236 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.236 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:45.236 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.144719+0000", 2024-09-18T10:15:45.236 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:45.236 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.236 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.237 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:45.237 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.144719+0000", 2024-09-18T10:15:45.237 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:45.237 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.237 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.237 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:45.237 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.144719+0000", 2024-09-18T10:15:45.237 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:45.237 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.237 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.237 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:45.237 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.144721+0000", 2024-09-18T10:15:45.237 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0169999999999999e-06 2024-09-18T10:15:45.238 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.238 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.238 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:45.238 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.144728+0000", 2024-09-18T10:15:45.238 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9600000000000003e-06 2024-09-18T10:15:45.238 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.238 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.238 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:45.238 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.144731+0000", 2024-09-18T10:15:45.238 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.7760000000000002e-06 2024-09-18T10:15:45.238 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.238 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.238 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:45.238 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.355198+0000", 2024-09-18T10:15:45.239 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.210467401 2024-09-18T10:15:45.239 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.239 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.239 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:45.239 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.355218+0000", 2024-09-18T10:15:45.239 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9979999999999998e-05 2024-09-18T10:15:45.239 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:45.239 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:45.239 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:45.239 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.239 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.239 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDScrubReserve(3.6s0 REJECT e62)", 2024-09-18T10:15:45.239 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:18.118418+0000", 2024-09-18T10:15:45.240 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 27.035729331999999, 2024-09-18T10:15:45.240 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.379472953, 2024-09-18T10:15:45.240 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:45.240 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:45.240 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:45.240 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.240 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:45.240 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:18.118418+0000", 2024-09-18T10:15:45.240 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:45.240 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.240 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.240 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:45.240 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:18.118418+0000", 2024-09-18T10:15:45.241 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:45.241 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.241 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.241 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:45.241 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:18.118416+0000", 2024-09-18T10:15:45.241 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:45.241 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.241 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.241 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:45.241 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:18.118424+0000", 2024-09-18T10:15:45.241 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.576e-06 2024-09-18T10:15:45.241 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.241 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.241 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:45.242 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:18.118450+0000", 2024-09-18T10:15:45.242 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6673e-05 2024-09-18T10:15:45.242 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.242 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.242 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:45.242 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:18.218541+0000", 2024-09-18T10:15:45.242 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100090107 2024-09-18T10:15:45.242 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.242 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.242 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:45.242 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:18.497802+0000", 2024-09-18T10:15:45.242 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.279261392 2024-09-18T10:15:45.243 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.243 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.243 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:45.243 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:18.497866+0000", 2024-09-18T10:15:45.243 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3989e-05 2024-09-18T10:15:45.243 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.243 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.243 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:45.243 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:18.497873+0000", 2024-09-18T10:15:45.243 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1500000000000002e-06 2024-09-18T10:15:45.243 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.243 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.244 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:45.244 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:18.497891+0000", 2024-09-18T10:15:45.244 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7462999999999999e-05 2024-09-18T10:15:45.244 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:45.244 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:45.244 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:45.244 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.244 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.244 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(osd.7.22:5 3.6s0 3:6619023d:::smithi18925896-19:head [assert-version v4,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T10:15:45.244 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:19.216031+0000", 2024-09-18T10:15:45.244 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 25.938115894999999, 2024-09-18T10:15:45.244 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.40429900499999999, 2024-09-18T10:15:45.244 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:45.245 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:45.245 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:45.245 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.245 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:45.245 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:19.216031+0000", 2024-09-18T10:15:45.245 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:45.245 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.245 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.245 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:45.245 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:19.216031+0000", 2024-09-18T10:15:45.245 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:45.245 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.245 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.246 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:45.246 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:19.216028+0000", 2024-09-18T10:15:45.246 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999971 2024-09-18T10:15:45.246 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.246 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.246 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:45.246 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:19.216046+0000", 2024-09-18T10:15:45.246 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7166000000000001e-05 2024-09-18T10:15:45.246 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.246 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.246 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:45.246 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:19.216075+0000", 2024-09-18T10:15:45.246 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9862999999999999e-05 2024-09-18T10:15:45.247 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.247 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.247 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:45.247 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:19.316231+0000", 2024-09-18T10:15:45.247 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10015547599999999 2024-09-18T10:15:45.247 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.247 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.247 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:45.247 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:19.316393+0000", 2024-09-18T10:15:45.247 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00016166999999999999 2024-09-18T10:15:45.247 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.247 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.247 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:45.247 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:19.316691+0000", 2024-09-18T10:15:45.248 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00029805700000000001 2024-09-18T10:15:45.248 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.248 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.248 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:45.248 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:19.620330+0000", 2024-09-18T10:15:45.248 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.30363942100000002 2024-09-18T10:15:45.248 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:45.248 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:45.248 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:45.248 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.248 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.248 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:681 3.6s0 3:646542dd:::smithi18925896-18:head [copy-from ver 0] snapc 28=[28,26,20,1f,1e,1a,11] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-09-18T10:15:45.248 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:33.588980+0000", 2024-09-18T10:15:45.249 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 11.565166672, 2024-09-18T10:15:45.249 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.30778096900000002, 2024-09-18T10:15:45.249 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:45.249 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:45.249 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:45.249 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:45.249 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:45.249 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 681 2024-09-18T10:15:45.249 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.249 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:45.249 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.249 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:45.249 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:33.588980+0000", 2024-09-18T10:15:45.250 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:45.250 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.250 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.250 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:45.250 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:33.588980+0000", 2024-09-18T10:15:45.250 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:45.250 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.250 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.250 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:45.250 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:33.588977+0000", 2024-09-18T10:15:45.250 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999967 2024-09-18T10:15:45.250 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.250 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.250 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:45.251 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:33.588995+0000", 2024-09-18T10:15:45.251 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8309e-05 2024-09-18T10:15:45.251 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.251 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.251 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:45.251 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:33.589037+0000", 2024-09-18T10:15:45.251 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1412999999999998e-05 2024-09-18T10:15:45.251 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.251 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.251 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:45.251 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:33.689144+0000", 2024-09-18T10:15:45.251 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100106977 2024-09-18T10:15:45.251 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.252 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.252 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:45.252 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:33.689280+0000", 2024-09-18T10:15:45.252 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000136174 2024-09-18T10:15:45.252 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.252 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.252 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op must wait for map", 2024-09-18T10:15:45.252 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:33.689292+0000", 2024-09-18T10:15:45.252 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1528999999999999e-05 2024-09-18T10:15:45.252 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.252 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.253 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:45.253 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:33.792469+0000", 2024-09-18T10:15:45.253 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.103177796 2024-09-18T10:15:45.253 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.253 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.253 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:45.253 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:33.792711+0000", 2024-09-18T10:15:45.253 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00024172000000000001 2024-09-18T10:15:45.253 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.253 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.253 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:45.253 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:33.896761+0000", 2024-09-18T10:15:45.254 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.104050246 2024-09-18T10:15:45.254 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:45.254 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:45.254 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:45.254 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.254 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.254 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:690 3.6s0 3:647d434f:::smithi18925896-1:head [read 0~65536,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-18T10:15:45.254 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:35.579593+0000", 2024-09-18T10:15:45.254 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 9.5745541050000007, 2024-09-18T10:15:45.254 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.30995702400000003, 2024-09-18T10:15:45.254 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:45.254 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:45.254 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:45.255 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:45.255 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:45.255 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 690 2024-09-18T10:15:45.255 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.255 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:45.255 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.255 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:45.255 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:35.579593+0000", 2024-09-18T10:15:45.255 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:45.255 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.255 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.255 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:45.255 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:35.579593+0000", 2024-09-18T10:15:45.256 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:45.256 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.256 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.256 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:45.256 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:35.579592+0000", 2024-09-18T10:15:45.256 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.999999 2024-09-18T10:15:45.256 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.256 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.256 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:45.256 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:35.579618+0000", 2024-09-18T10:15:45.256 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6512999999999998e-05 2024-09-18T10:15:45.256 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.256 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.257 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:45.257 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:35.579648+0000", 2024-09-18T10:15:45.257 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9679000000000002e-05 2024-09-18T10:15:45.257 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.257 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.257 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:45.257 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:35.579657+0000", 2024-09-18T10:15:45.257 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.3899999999999999e-06 2024-09-18T10:15:45.257 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.257 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.257 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:45.257 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:35.579818+0000", 2024-09-18T10:15:45.257 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000161027 2024-09-18T10:15:45.258 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.258 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.258 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:45.258 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:35.579828+0000", 2024-09-18T10:15:45.258 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.8330000000000006e-06 2024-09-18T10:15:45.258 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.258 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.258 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:45.258 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:35.787489+0000", 2024-09-18T10:15:45.258 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.20766110700000001 2024-09-18T10:15:45.258 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.258 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.258 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:45.258 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:35.787666+0000", 2024-09-18T10:15:45.259 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00017699299999999999 2024-09-18T10:15:45.259 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.259 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.259 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:45.259 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:35.889550+0000", 2024-09-18T10:15:45.259 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.101883602 2024-09-18T10:15:45.259 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:45.259 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:45.259 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:45.259 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:45.259 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:45.259 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-18T10:15:45.261 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T10:15:45.262 INFO:tasks.ceph.osd.14.smithi154.stderr:2024-09-18T10:15:45.258+0000 7f75a19e7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 22093) UID: 0 2024-09-18T10:15:45.362 INFO:tasks.ceph.osd.14.smithi154.stderr:2024-09-18T10:15:45.358+0000 7f75a19e7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 22093) UID: 0 2024-09-18T10:15:45.371 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-18T10:15:45.371 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-18T10:15:45.371 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.371 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDPGPushReply(3.19s1 89/83 [PushReplyOp(3:9bcb6eef:::smithi18925896-11:1a)])", 2024-09-18T10:15:45.372 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:45.207286+0000", 2024-09-18T10:15:45.372 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.16308477900000001, 2024-09-18T10:15:45.372 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.163123879, 2024-09-18T10:15:45.372 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:45.372 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-18T10:15:45.372 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:45.372 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.372 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:45.372 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:45.207286+0000", 2024-09-18T10:15:45.372 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:45.372 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.372 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.372 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:45.373 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:45.207286+0000", 2024-09-18T10:15:45.373 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:45.373 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.373 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.373 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:45.373 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:45.207286+0000", 2024-09-18T10:15:45.373 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:45.373 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.373 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.373 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:45.373 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:45.207295+0000", 2024-09-18T10:15:45.373 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.5070000000000006e-06 2024-09-18T10:15:45.373 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.374 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.374 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:45.374 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:45.207311+0000", 2024-09-18T10:15:45.374 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.6239000000000002e-05 2024-09-18T10:15:45.374 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.374 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.374 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:45.374 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:45.207319+0000", 2024-09-18T10:15:45.374 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7030000000000006e-06 2024-09-18T10:15:45.374 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:45.374 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:45.374 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:45.374 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.375 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.375 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWriteReply(3.1s0 89/40 ECSubWriteReply(tid=252, last_complete=89'73, committed=1, applied=1))", 2024-09-18T10:15:45.375 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:45.299851+0000", 2024-09-18T10:15:45.375 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.070519888000000003, 2024-09-18T10:15:45.375 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.070642702000000002, 2024-09-18T10:15:45.375 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:45.375 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-18T10:15:45.375 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:45.375 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.375 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:45.375 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:45.299851+0000", 2024-09-18T10:15:45.375 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:45.375 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.375 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.376 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:45.376 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:45.299851+0000", 2024-09-18T10:15:45.376 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:45.376 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.376 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.376 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:45.376 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:45.299850+0000", 2024-09-18T10:15:45.376 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:45.376 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.376 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.376 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:45.376 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:45.299862+0000", 2024-09-18T10:15:45.377 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2434e-05 2024-09-18T10:15:45.377 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.377 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.377 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:45.377 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:45.299890+0000", 2024-09-18T10:15:45.377 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.8432000000000002e-05 2024-09-18T10:15:45.377 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.377 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.377 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:45.377 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:45.299901+0000", 2024-09-18T10:15:45.377 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0356999999999999e-05 2024-09-18T10:15:45.377 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:45.377 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:45.377 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:45.378 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:45.378 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-18T10:15:45.378 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 2 2024-09-18T10:15:45.378 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-18T10:15:45.381 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T10:15:45.463 INFO:tasks.ceph.osd.14.smithi154.stderr:2024-09-18T10:15:45.458+0000 7f75a19e7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 22093) UID: 0 2024-09-18T10:15:45.479 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-18T10:15:45.480 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-18T10:15:45.480 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-18T10:15:45.480 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-18T10:15:45.480 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-18T10:15:45.490 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T10:15:45.564 INFO:tasks.ceph.osd.15.smithi189.stderr:2024-09-18T10:15:45.558+0000 7fd0e725e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 22401) UID: 0 2024-09-18T10:15:45.592 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-18T10:15:45.592 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-18T10:15:45.592 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-18T10:15:45.592 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-18T10:15:45.592 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.593 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:129 3.7s0 3:e4da5b5e:::smithi18925896-43:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:45.593 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:50.713961+0000", 2024-09-18T10:15:45.593 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 54.873852935000002, 2024-09-18T10:15:45.593 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.819334071, 2024-09-18T10:15:45.593 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:45.593 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:45.593 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:45.593 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:45.593 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:45.593 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 129 2024-09-18T10:15:45.593 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.593 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:45.593 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.594 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:45.594 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.713961+0000", 2024-09-18T10:15:45.594 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:45.594 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.594 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.594 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:45.594 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.713961+0000", 2024-09-18T10:15:45.594 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:45.594 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.594 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.594 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:45.594 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.713960+0000", 2024-09-18T10:15:45.594 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.999999 2024-09-18T10:15:45.595 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.595 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.595 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:45.595 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.713964+0000", 2024-09-18T10:15:45.595 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.3059999999999999e-06 2024-09-18T10:15:45.595 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.595 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.595 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:45.595 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.713974+0000", 2024-09-18T10:15:45.595 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.9969999999999993e-06 2024-09-18T10:15:45.595 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.595 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.595 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:45.596 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.713981+0000", 2024-09-18T10:15:45.596 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1229999999999997e-06 2024-09-18T10:15:45.596 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.596 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.596 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:45.596 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.000661+0000", 2024-09-18T10:15:45.596 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.28668068699999999 2024-09-18T10:15:45.596 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.596 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.596 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for rw locks", 2024-09-18T10:15:45.596 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.000844+0000", 2024-09-18T10:15:45.596 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00018289599999999999 2024-09-18T10:15:45.596 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.597 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.597 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:45.597 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.219209+0000", 2024-09-18T10:15:45.597 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.21836466900000001 2024-09-18T10:15:45.597 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.597 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.597 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:45.597 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.219351+0000", 2024-09-18T10:15:45.597 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00014262300000000001 2024-09-18T10:15:45.597 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.597 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.597 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:45.597 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.533295+0000", 2024-09-18T10:15:45.598 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.31394377299999998 2024-09-18T10:15:45.598 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:45.598 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:45.598 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:45.598 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.598 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.598 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:159 3.7s0 3:ead53b0f:::smithi18925896-25:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:45.598 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:51.226988+0000", 2024-09-18T10:15:45.598 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 54.360825875000003, 2024-09-18T10:15:45.598 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.81465928499999996, 2024-09-18T10:15:45.598 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:45.598 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:45.599 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:45.599 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:45.599 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:45.599 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 159 2024-09-18T10:15:45.599 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.599 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:45.599 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.599 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:45.599 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.226988+0000", 2024-09-18T10:15:45.599 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:45.599 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.599 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.599 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:45.600 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.226988+0000", 2024-09-18T10:15:45.600 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:45.600 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.600 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.600 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:45.600 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.226987+0000", 2024-09-18T10:15:45.600 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.999999 2024-09-18T10:15:45.600 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.600 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.600 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:45.600 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.226992+0000", 2024-09-18T10:15:45.600 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.5900000000000001e-06 2024-09-18T10:15:45.600 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.601 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.601 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:45.601 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.227003+0000", 2024-09-18T10:15:45.601 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1222e-05 2024-09-18T10:15:45.601 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.601 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.601 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:45.601 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.227011+0000", 2024-09-18T10:15:45.601 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.6279999999999996e-06 2024-09-18T10:15:45.601 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.601 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.601 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:45.602 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.533792+0000", 2024-09-18T10:15:45.602 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.30678158100000003 2024-09-18T10:15:45.602 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.602 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.602 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for rw locks", 2024-09-18T10:15:45.602 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.533941+0000", 2024-09-18T10:15:45.602 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00014846 2024-09-18T10:15:45.602 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.602 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.602 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:45.602 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.633576+0000", 2024-09-18T10:15:45.602 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099635153000000004 2024-09-18T10:15:45.602 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.602 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.603 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:45.603 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.633832+0000", 2024-09-18T10:15:45.603 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00025610699999999999 2024-09-18T10:15:45.603 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.603 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.603 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:45.603 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.041647+0000", 2024-09-18T10:15:45.603 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.407815388 2024-09-18T10:15:45.603 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:45.603 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:45.603 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:45.603 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.603 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.603 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:169 3.7s0 3:e1df0280:::smithi18925896-26:head [copy-from ver 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:45.604 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:51.532055+0000", 2024-09-18T10:15:45.604 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 54.055758687000001, 2024-09-18T10:15:45.604 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.60955003100000005, 2024-09-18T10:15:45.604 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:45.604 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:45.604 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:45.604 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:45.604 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:45.604 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 169 2024-09-18T10:15:45.604 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.604 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:45.604 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.604 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:45.605 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.532055+0000", 2024-09-18T10:15:45.605 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:45.605 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.605 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.605 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:45.605 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.532055+0000", 2024-09-18T10:15:45.605 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:45.605 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.605 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.605 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:45.605 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.532054+0000", 2024-09-18T10:15:45.605 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:45.605 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.606 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.606 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:45.606 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.532062+0000", 2024-09-18T10:15:45.606 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0430000000000008e-06 2024-09-18T10:15:45.606 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.606 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.606 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:45.606 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.532080+0000", 2024-09-18T10:15:45.606 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8283000000000001e-05 2024-09-18T10:15:45.606 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.606 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.606 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:45.606 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.532092+0000", 2024-09-18T10:15:45.606 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1742999999999999e-05 2024-09-18T10:15:45.607 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.607 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.607 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:45.607 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.534038+0000", 2024-09-18T10:15:45.607 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0019461610000000001 2024-09-18T10:15:45.607 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.607 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.607 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:45.607 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.534285+0000", 2024-09-18T10:15:45.607 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00024667800000000001 2024-09-18T10:15:45.607 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.607 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.607 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:45.608 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.938506+0000", 2024-09-18T10:15:45.608 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.40422132199999999 2024-09-18T10:15:45.608 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.608 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.608 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:45.608 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.041810+0000", 2024-09-18T10:15:45.608 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10330328900000001 2024-09-18T10:15:45.608 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.608 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.608 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-18T10:15:45.608 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.141492+0000", 2024-09-18T10:15:45.608 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099682251999999999 2024-09-18T10:15:45.608 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.609 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.609 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:45.609 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.141605+0000", 2024-09-18T10:15:45.609 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00011349300000000001 2024-09-18T10:15:45.609 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:45.609 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:45.609 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:45.609 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.609 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.609 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:170 3.7s0 3:e1df0280:::smithi18925896-26:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:45.609 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:51.532106+0000", 2024-09-18T10:15:45.609 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 54.055708408000001, 2024-09-18T10:15:45.609 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.50908735800000005, 2024-09-18T10:15:45.609 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:45.610 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:45.610 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:45.610 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:45.610 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:45.610 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 170 2024-09-18T10:15:45.610 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.610 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:45.610 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.610 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:45.610 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.532106+0000", 2024-09-18T10:15:45.610 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:45.610 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.610 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.611 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:45.611 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.532106+0000", 2024-09-18T10:15:45.611 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:45.611 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.611 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.611 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:45.611 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.532104+0000", 2024-09-18T10:15:45.611 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:45.611 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.611 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.611 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:45.611 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.532113+0000", 2024-09-18T10:15:45.611 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8659999999999995e-06 2024-09-18T10:15:45.611 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.612 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.612 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:45.612 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.532131+0000", 2024-09-18T10:15:45.612 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7723999999999998e-05 2024-09-18T10:15:45.612 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.612 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.612 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:45.612 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.532142+0000", 2024-09-18T10:15:45.612 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0669000000000001e-05 2024-09-18T10:15:45.612 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.612 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.612 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:45.612 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.534705+0000", 2024-09-18T10:15:45.613 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0025635380000000002 2024-09-18T10:15:45.613 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.613 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.613 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for blocked object", 2024-09-18T10:15:45.613 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.534831+0000", 2024-09-18T10:15:45.613 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00012540399999999999 2024-09-18T10:15:45.613 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.613 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.613 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:45.613 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.040545+0000", 2024-09-18T10:15:45.613 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.50571483699999997 2024-09-18T10:15:45.613 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.613 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.614 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:45.614 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.040869+0000", 2024-09-18T10:15:45.614 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00032386999999999997 2024-09-18T10:15:45.614 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.614 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.614 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:45.615 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.041193+0000", 2024-09-18T10:15:45.615 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000323803 2024-09-18T10:15:45.616 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:45.616 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:45.616 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:45.616 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.616 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.616 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:211 3.7s0 3:e4da5b5e:::smithi18925896-43:head [read 0~720896,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:45.616 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:52.694469+0000", 2024-09-18T10:15:45.616 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 52.893344589999998, 2024-09-18T10:15:45.616 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.51035930399999996, 2024-09-18T10:15:45.616 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:45.616 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:45.616 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:45.616 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:45.617 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:45.617 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 211 2024-09-18T10:15:45.617 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.617 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:45.617 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.617 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:45.617 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.694469+0000", 2024-09-18T10:15:45.617 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:45.617 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.617 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.617 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:45.617 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.694469+0000", 2024-09-18T10:15:45.617 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:45.618 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.618 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.618 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:45.618 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.694469+0000", 2024-09-18T10:15:45.618 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.999999 2024-09-18T10:15:45.618 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.618 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.618 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:45.618 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.694475+0000", 2024-09-18T10:15:45.618 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7349999999999999e-06 2024-09-18T10:15:45.618 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.618 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.618 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:45.618 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.694489+0000", 2024-09-18T10:15:45.619 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4209999999999999e-05 2024-09-18T10:15:45.619 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.619 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.619 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:45.619 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.694498+0000", 2024-09-18T10:15:45.619 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.4339999999999992e-06 2024-09-18T10:15:45.619 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.619 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.619 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:45.619 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.698348+0000", 2024-09-18T10:15:45.619 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0038499670000000001 2024-09-18T10:15:45.619 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.619 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.620 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:45.620 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.698366+0000", 2024-09-18T10:15:45.620 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7999000000000002e-05 2024-09-18T10:15:45.620 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.620 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.620 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:45.620 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:53.000010+0000", 2024-09-18T10:15:45.620 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.30164412699999998 2024-09-18T10:15:45.620 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.620 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.620 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:45.620 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:53.000223+0000", 2024-09-18T10:15:45.620 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000213169 2024-09-18T10:15:45.620 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.621 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.621 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:45.621 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:53.204829+0000", 2024-09-18T10:15:45.621 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.20460561299999999 2024-09-18T10:15:45.621 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:45.621 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:45.621 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:45.621 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.621 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.621 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(osd.10.22:2 3.7s0 3:ee962131:::smithi18925896-29:head [assert-version v6,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:45.621 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:52.798083+0000", 2024-09-18T10:15:45.621 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 52.789730698, 2024-09-18T10:15:45.622 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.60989146800000005, 2024-09-18T10:15:45.622 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:45.622 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:45.622 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:45.622 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.622 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:45.622 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.798083+0000", 2024-09-18T10:15:45.622 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:45.622 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.622 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.622 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:45.622 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.798083+0000", 2024-09-18T10:15:45.622 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:45.623 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.623 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.623 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:45.623 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.798080+0000", 2024-09-18T10:15:45.623 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999962 2024-09-18T10:15:45.623 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.623 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.623 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:45.623 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.798097+0000", 2024-09-18T10:15:45.623 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7657000000000001e-05 2024-09-18T10:15:45.623 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.624 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.624 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:45.624 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.798127+0000", 2024-09-18T10:15:45.624 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9532000000000001e-05 2024-09-18T10:15:45.624 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.624 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.624 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:45.624 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.898205+0000", 2024-09-18T10:15:45.624 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100078525 2024-09-18T10:15:45.624 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.624 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.624 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:45.625 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:53.204957+0000", 2024-09-18T10:15:45.625 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.30675138899999999 2024-09-18T10:15:45.625 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.625 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.625 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:45.625 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:53.205092+0000", 2024-09-18T10:15:45.625 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00013556700000000001 2024-09-18T10:15:45.625 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.625 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.625 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:45.625 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:53.407975+0000", 2024-09-18T10:15:45.625 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.20288240799999999 2024-09-18T10:15:45.625 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:45.625 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:45.626 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:45.626 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.626 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.626 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:437 3.1s0 3:82c9401b:::smithi18925896-33:head [copy-from ver 2] snapc 13=[13,12,11,10,f,e,b,4] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-18T10:15:45.626 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:11.831109+0000", 2024-09-18T10:15:45.626 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 33.756704919999997, 2024-09-18T10:15:45.626 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.709244349, 2024-09-18T10:15:45.626 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:45.626 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:45.626 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:45.626 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:45.626 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:45.626 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 437 2024-09-18T10:15:45.627 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.627 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:45.627 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.627 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:45.627 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:11.831109+0000", 2024-09-18T10:15:45.627 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:45.627 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.627 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.627 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:45.627 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:11.831109+0000", 2024-09-18T10:15:45.627 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:45.627 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.627 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.627 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:45.628 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:11.831106+0000", 2024-09-18T10:15:45.628 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999971 2024-09-18T10:15:45.628 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.628 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.628 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:45.628 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:11.831122+0000", 2024-09-18T10:15:45.628 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5971e-05 2024-09-18T10:15:45.628 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.628 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.628 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:45.628 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:11.831155+0000", 2024-09-18T10:15:45.628 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.3259999999999997e-05 2024-09-18T10:15:45.628 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.629 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.629 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:45.629 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:11.831172+0000", 2024-09-18T10:15:45.629 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.6433999999999999e-05 2024-09-18T10:15:45.629 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.629 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.629 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:45.629 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:11.931398+0000", 2024-09-18T10:15:45.629 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10022644 2024-09-18T10:15:45.629 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.629 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.629 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op must wait for map", 2024-09-18T10:15:45.629 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:11.931412+0000", 2024-09-18T10:15:45.630 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3586e-05 2024-09-18T10:15:45.630 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.630 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.630 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:45.630 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:11.934231+0000", 2024-09-18T10:15:45.630 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0028191819999999999 2024-09-18T10:15:45.630 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.630 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.630 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:45.630 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:11.934501+0000", 2024-09-18T10:15:45.630 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00026996099999999999 2024-09-18T10:15:45.630 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.631 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.631 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:45.631 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.352429+0000", 2024-09-18T10:15:45.631 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.41792792699999998 2024-09-18T10:15:45.631 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.631 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.631 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:45.631 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.356112+0000", 2024-09-18T10:15:45.631 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0036833199999999999 2024-09-18T10:15:45.631 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.632 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.632 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-18T10:15:45.632 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.540229+0000", 2024-09-18T10:15:45.632 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.18411702599999999 2024-09-18T10:15:45.632 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.632 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.632 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:45.632 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.540354+0000", 2024-09-18T10:15:45.632 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00012415399999999999 2024-09-18T10:15:45.632 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:45.632 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:45.632 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:45.632 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.633 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.633 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:438 3.1s0 3:82c9401b:::smithi18925896-33:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e56)", 2024-09-18T10:15:45.633 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:11.831206+0000", 2024-09-18T10:15:45.633 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 33.756608337999999, 2024-09-18T10:15:45.633 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.52445609999999998, 2024-09-18T10:15:45.633 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:45.633 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:45.633 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:45.633 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:45.633 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:45.633 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 438 2024-09-18T10:15:45.633 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.633 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:45.634 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.634 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:45.634 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:11.831206+0000", 2024-09-18T10:15:45.634 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:45.634 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.634 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.634 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:45.634 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:11.831206+0000", 2024-09-18T10:15:45.634 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:45.634 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.634 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.634 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:45.635 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:11.831205+0000", 2024-09-18T10:15:45.635 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.999999 2024-09-18T10:15:45.635 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.635 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.635 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:45.635 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:11.831210+0000", 2024-09-18T10:15:45.635 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8539999999999996e-06 2024-09-18T10:15:45.635 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.635 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.635 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:45.635 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:11.831230+0000", 2024-09-18T10:15:45.635 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9899999999999999e-05 2024-09-18T10:15:45.635 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.636 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.636 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:45.636 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:11.831238+0000", 2024-09-18T10:15:45.636 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.7639999999999994e-06 2024-09-18T10:15:45.636 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.636 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.636 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:45.636 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:11.931577+0000", 2024-09-18T10:15:45.636 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10033874299999999 2024-09-18T10:15:45.636 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.636 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.636 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:45.636 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:11.931604+0000", 2024-09-18T10:15:45.636 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.7239000000000001e-05 2024-09-18T10:15:45.637 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.637 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.637 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:45.637 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:11.935139+0000", 2024-09-18T10:15:45.637 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00353484 2024-09-18T10:15:45.637 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.637 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.637 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for blocked object", 2024-09-18T10:15:45.637 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:11.935380+0000", 2024-09-18T10:15:45.637 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00024033400000000001 2024-09-18T10:15:45.637 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.637 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.637 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:45.638 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.355102+0000", 2024-09-18T10:15:45.638 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.41972236200000002 2024-09-18T10:15:45.638 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.638 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.638 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:45.638 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.355403+0000", 2024-09-18T10:15:45.638 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000301537 2024-09-18T10:15:45.638 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.638 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.638 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:45.638 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.355662+0000", 2024-09-18T10:15:45.638 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000258421 2024-09-18T10:15:45.638 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:45.639 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:45.639 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:45.639 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.639 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.639 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:441 3.17s0 3:ead53b0f:::smithi18925896-25:head [append 0~606208 in=606208b] snapc 13=[13,12,11,10,f,e,b] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-18T10:15:45.639 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:12.856902+0000", 2024-09-18T10:15:45.639 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 32.730911818000003, 2024-09-18T10:15:45.639 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.80182116999999997, 2024-09-18T10:15:45.639 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:45.639 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:45.639 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:45.639 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:45.639 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:45.639 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 441 2024-09-18T10:15:45.640 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.640 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:45.640 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.640 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:45.640 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.856902+0000", 2024-09-18T10:15:45.640 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:45.640 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.640 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.640 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:45.640 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.856902+0000", 2024-09-18T10:15:45.640 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:45.640 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.640 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.641 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:45.641 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.856900+0000", 2024-09-18T10:15:45.641 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999976 2024-09-18T10:15:45.641 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.641 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.641 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:45.641 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.857946+0000", 2024-09-18T10:15:45.641 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0010459620000000001 2024-09-18T10:15:45.641 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.641 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.641 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:45.641 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.857973+0000", 2024-09-18T10:15:45.641 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6758999999999999e-05 2024-09-18T10:15:45.642 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.642 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.642 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:45.642 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.857983+0000", 2024-09-18T10:15:45.642 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0105000000000001e-05 2024-09-18T10:15:45.642 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.642 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.642 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:45.642 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.946351+0000", 2024-09-18T10:15:45.642 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.088368302999999995 2024-09-18T10:15:45.642 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.642 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.642 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:45.642 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.946465+0000", 2024-09-18T10:15:45.643 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00011425999999999999 2024-09-18T10:15:45.643 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.643 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.643 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:45.643 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.947656+0000", 2024-09-18T10:15:45.643 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0011901710000000001 2024-09-18T10:15:45.643 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.643 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.643 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:45.643 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.040655+0000", 2024-09-18T10:15:45.643 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.092999126000000001 2024-09-18T10:15:45.643 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.643 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.643 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-18T10:15:45.644 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.658646+0000", 2024-09-18T10:15:45.644 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.61799096399999998 2024-09-18T10:15:45.644 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.644 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.644 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:45.644 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.658723+0000", 2024-09-18T10:15:45.644 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7805000000000005e-05 2024-09-18T10:15:45.644 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:45.644 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:45.644 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:45.644 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.644 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.644 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:442 3.17s0 3:ead53b0f:::smithi18925896-25:head [append 0~720896 in=720896b] snapc 13=[13,12,11,10,f,e,b] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-18T10:15:45.645 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:12.861303+0000", 2024-09-18T10:15:45.645 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 32.726511424000002, 2024-09-18T10:15:45.645 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.89783094100000005, 2024-09-18T10:15:45.645 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:45.645 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:45.645 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:45.645 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:45.645 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:45.645 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 442 2024-09-18T10:15:45.645 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.645 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:45.645 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.645 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:45.646 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.861303+0000", 2024-09-18T10:15:45.646 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:45.646 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.646 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.646 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:45.646 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.861303+0000", 2024-09-18T10:15:45.646 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:45.646 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.646 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.646 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:45.646 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.861301+0000", 2024-09-18T10:15:45.646 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:45.646 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.647 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.647 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:45.647 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.862343+0000", 2024-09-18T10:15:45.647 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0010422820000000001 2024-09-18T10:15:45.647 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.647 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.647 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:45.647 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.862381+0000", 2024-09-18T10:15:45.647 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7407000000000003e-05 2024-09-18T10:15:45.647 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.647 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.647 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:45.647 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.862391+0000", 2024-09-18T10:15:45.647 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0572e-05 2024-09-18T10:15:45.648 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.648 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.648 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:45.648 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.948744+0000", 2024-09-18T10:15:45.648 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.086352683999999999 2024-09-18T10:15:45.648 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.648 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.648 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:45.648 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.948864+0000", 2024-09-18T10:15:45.648 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000119855 2024-09-18T10:15:45.648 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.648 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.648 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:45.649 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.949834+0000", 2024-09-18T10:15:45.649 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00097020199999999998 2024-09-18T10:15:45.649 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.649 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.649 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:45.649 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.040659+0000", 2024-09-18T10:15:45.649 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.090824659000000002 2024-09-18T10:15:45.649 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.649 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.649 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-18T10:15:45.649 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.759042+0000", 2024-09-18T10:15:45.649 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.71838323100000001 2024-09-18T10:15:45.649 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.650 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.650 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:45.650 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.759134+0000", 2024-09-18T10:15:45.650 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.1760000000000002e-05 2024-09-18T10:15:45.650 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:45.650 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:45.650 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:45.650 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.650 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.650 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:443 3.17s0 3:ead53b0f:::smithi18925896-25:head [append 0~49152 in=49152b] snapc 13=[13,12,11,10,f,e,b] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-18T10:15:45.650 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:12.862402+0000", 2024-09-18T10:15:45.650 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 32.725411645999998, 2024-09-18T10:15:45.650 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.99734852100000004, 2024-09-18T10:15:45.650 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:45.651 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:45.651 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:45.651 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:45.651 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:45.651 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 443 2024-09-18T10:15:45.651 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.651 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:45.651 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.651 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:45.651 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.862402+0000", 2024-09-18T10:15:45.651 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:45.651 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.651 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.651 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:45.651 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.862402+0000", 2024-09-18T10:15:45.652 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:45.652 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.652 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.652 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:45.652 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.862402+0000", 2024-09-18T10:15:45.652 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.999999 2024-09-18T10:15:45.652 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.652 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.652 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:45.652 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.862441+0000", 2024-09-18T10:15:45.652 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9116000000000003e-05 2024-09-18T10:15:45.652 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.653 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.653 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:45.653 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.862455+0000", 2024-09-18T10:15:45.653 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4501999999999999e-05 2024-09-18T10:15:45.653 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.653 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.653 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:45.653 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.862462+0000", 2024-09-18T10:15:45.653 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.4659999999999996e-06 2024-09-18T10:15:45.653 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.653 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.653 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:45.653 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.950570+0000", 2024-09-18T10:15:45.654 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.088107850000000001 2024-09-18T10:15:45.654 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.654 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.654 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:45.654 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.950675+0000", 2024-09-18T10:15:45.654 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000105445 2024-09-18T10:15:45.654 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.654 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.654 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:45.654 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.951279+0000", 2024-09-18T10:15:45.654 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00060383599999999996 2024-09-18T10:15:45.654 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.654 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.655 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:45.655 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.040661+0000", 2024-09-18T10:15:45.655 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.089382448000000003 2024-09-18T10:15:45.655 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.655 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.655 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-18T10:15:45.655 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.859668+0000", 2024-09-18T10:15:45.655 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.81900669800000003 2024-09-18T10:15:45.655 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.655 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.655 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:45.655 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.859751+0000", 2024-09-18T10:15:45.655 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.2943999999999997e-05 2024-09-18T10:15:45.656 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:45.656 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:45.656 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:45.656 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.656 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.656 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:444 3.17s0 3:ead53b0f:::smithi18925896-25:head [setxattr _header (57) in=64b] snapc 13=[13,12,11,10,f,e,b] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-18T10:15:45.656 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:12.862469+0000", 2024-09-18T10:15:45.656 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 32.725345435999998, 2024-09-18T10:15:45.656 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.99750343200000002, 2024-09-18T10:15:45.656 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:45.656 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:45.656 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:45.657 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:45.657 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:45.657 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 444 2024-09-18T10:15:45.657 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.657 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:45.657 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.657 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:45.657 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.862469+0000", 2024-09-18T10:15:45.657 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:45.657 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.657 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.657 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:45.658 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.862469+0000", 2024-09-18T10:15:45.658 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:45.658 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.658 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.658 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:45.658 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.862468+0000", 2024-09-18T10:15:45.658 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:45.658 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.658 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.658 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:45.658 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.862473+0000", 2024-09-18T10:15:45.659 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.5839999999999996e-06 2024-09-18T10:15:45.659 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.659 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.659 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:45.659 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.862483+0000", 2024-09-18T10:15:45.659 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0188e-05 2024-09-18T10:15:45.659 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.659 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.659 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:45.659 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.862487+0000", 2024-09-18T10:15:45.659 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6580000000000001e-06 2024-09-18T10:15:45.659 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.659 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.659 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:45.660 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.952057+0000", 2024-09-18T10:15:45.660 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.089569930000000006 2024-09-18T10:15:45.660 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.660 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.660 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:45.660 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.952227+0000", 2024-09-18T10:15:45.660 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000170106 2024-09-18T10:15:45.660 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.660 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.660 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:45.660 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.953127+0000", 2024-09-18T10:15:45.660 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00089958300000000002 2024-09-18T10:15:45.660 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.660 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.661 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:45.661 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.040664+0000", 2024-09-18T10:15:45.661 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.087536931999999998 2024-09-18T10:15:45.661 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.661 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.661 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-18T10:15:45.661 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.859892+0000", 2024-09-18T10:15:45.661 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.81922844699999997 2024-09-18T10:15:45.661 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.661 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.661 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:45.661 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.859972+0000", 2024-09-18T10:15:45.661 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.9662999999999999e-05 2024-09-18T10:15:45.662 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:45.662 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:45.662 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:45.662 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.662 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.662 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:445 3.17s0 3:ead53b0f:::smithi18925896-25:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e57)", 2024-09-18T10:15:45.662 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:12.862493+0000", 2024-09-18T10:15:45.662 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 32.725321567999998, 2024-09-18T10:15:45.662 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1938188730000001, 2024-09-18T10:15:45.662 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:45.662 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:45.662 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:45.662 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:45.662 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:45.663 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 445 2024-09-18T10:15:45.663 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.663 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:45.663 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.663 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:45.663 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.862493+0000", 2024-09-18T10:15:45.663 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:45.663 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.663 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.663 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:45.663 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.862493+0000", 2024-09-18T10:15:45.663 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:45.663 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.663 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.664 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:45.664 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.862492+0000", 2024-09-18T10:15:45.664 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:45.664 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.664 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.664 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:45.664 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.862495+0000", 2024-09-18T10:15:45.664 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1829999999999998e-06 2024-09-18T10:15:45.664 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.664 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.664 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:45.664 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.862506+0000", 2024-09-18T10:15:45.664 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0386e-05 2024-09-18T10:15:45.665 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.665 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.665 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:45.665 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.962582+0000", 2024-09-18T10:15:45.665 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100076741 2024-09-18T10:15:45.665 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.665 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.665 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:45.665 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.962632+0000", 2024-09-18T10:15:45.665 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9364999999999999e-05 2024-09-18T10:15:45.665 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.665 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.665 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for rw locks", 2024-09-18T10:15:45.666 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.962715+0000", 2024-09-18T10:15:45.666 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.3617999999999998e-05 2024-09-18T10:15:45.666 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.666 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.666 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:45.666 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.860479+0000", 2024-09-18T10:15:45.666 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.89776372699999996 2024-09-18T10:15:45.666 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.666 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.666 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:45.666 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.860908+0000", 2024-09-18T10:15:45.666 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00042866099999999999 2024-09-18T10:15:45.666 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.667 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.667 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:45.667 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:14.056311+0000", 2024-09-18T10:15:45.667 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.19540364199999999 2024-09-18T10:15:45.667 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:45.667 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:45.667 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:45.667 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.667 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.667 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:624 3.bs0 3:d62b3dfa:::smithi18925896-40:head [delete] snapc 21=[20,1f,1e,1a,16,12,11,10] ondisk+write+known_if_redirected+full_force+supports_pool_eio e72)", 2024-09-18T10:15:45.667 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:28.103242+0000", 2024-09-18T10:15:45.667 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 17.484571867, 2024-09-18T10:15:45.667 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.58417659399999999, 2024-09-18T10:15:45.667 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:45.668 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:45.668 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:45.668 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:45.668 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:45.668 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 624 2024-09-18T10:15:45.668 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.668 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:45.668 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.668 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:45.669 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:28.103242+0000", 2024-09-18T10:15:45.669 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:45.669 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.669 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.669 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:45.669 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:28.103242+0000", 2024-09-18T10:15:45.669 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:45.669 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.669 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.669 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:45.669 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:28.103240+0000", 2024-09-18T10:15:45.669 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999976 2024-09-18T10:15:45.669 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.669 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.670 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:45.670 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:28.103251+0000", 2024-09-18T10:15:45.670 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1311999999999999e-05 2024-09-18T10:15:45.670 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.670 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.670 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:45.670 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:28.103283+0000", 2024-09-18T10:15:45.670 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1634000000000002e-05 2024-09-18T10:15:45.670 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.670 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.670 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:45.670 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:28.103294+0000", 2024-09-18T10:15:45.670 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1586000000000001e-05 2024-09-18T10:15:45.670 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.671 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.671 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:45.671 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:28.103385+0000", 2024-09-18T10:15:45.671 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.1249999999999995e-05 2024-09-18T10:15:45.671 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.671 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.671 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:45.671 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:28.103530+0000", 2024-09-18T10:15:45.671 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000144556 2024-09-18T10:15:45.671 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.671 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.671 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:45.671 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:28.104587+0000", 2024-09-18T10:15:45.671 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0010573010000000001 2024-09-18T10:15:45.672 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.672 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.672 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:45.672 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:28.106223+0000", 2024-09-18T10:15:45.672 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001635613 2024-09-18T10:15:45.672 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.672 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.672 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-18T10:15:45.672 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:28.687252+0000", 2024-09-18T10:15:45.672 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.58102913300000003 2024-09-18T10:15:45.672 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.672 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.672 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:45.673 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:28.687419+0000", 2024-09-18T10:15:45.673 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00016671599999999999 2024-09-18T10:15:45.673 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:45.673 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:45.673 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:45.673 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.673 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.673 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:650 3.bs0 3:d62b3dfa:::smithi18925896-40:head [append 0~0] snapc 21=[20,1f,1e,1a,16,11,10] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-18T10:15:45.673 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:29.784110+0000", 2024-09-18T10:15:45.673 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 15.803703898, 2024-09-18T10:15:45.673 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.68818560399999995, 2024-09-18T10:15:45.673 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:45.673 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:45.674 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:45.674 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:45.674 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:45.674 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 650 2024-09-18T10:15:45.674 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.674 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:45.674 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.674 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:45.674 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:29.784110+0000", 2024-09-18T10:15:45.674 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:45.674 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.674 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.674 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:45.674 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:29.784110+0000", 2024-09-18T10:15:45.675 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:45.675 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.675 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.675 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:45.675 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:29.784109+0000", 2024-09-18T10:15:45.675 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.999999 2024-09-18T10:15:45.675 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.675 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.675 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:45.675 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:29.784114+0000", 2024-09-18T10:15:45.675 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1719999999999998e-06 2024-09-18T10:15:45.675 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.675 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.676 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:45.676 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:29.784135+0000", 2024-09-18T10:15:45.676 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0693999999999999e-05 2024-09-18T10:15:45.676 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.676 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.676 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:45.676 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:29.784154+0000", 2024-09-18T10:15:45.676 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9218000000000001e-05 2024-09-18T10:15:45.676 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.676 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.676 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:45.676 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:29.884343+0000", 2024-09-18T10:15:45.676 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100188467 2024-09-18T10:15:45.677 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.677 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.677 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:45.677 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:29.884485+0000", 2024-09-18T10:15:45.677 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00014232599999999999 2024-09-18T10:15:45.677 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.677 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.677 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:45.677 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:29.885103+0000", 2024-09-18T10:15:45.677 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00061802700000000005 2024-09-18T10:15:45.677 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.677 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.677 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:45.677 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:29.885722+0000", 2024-09-18T10:15:45.678 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00061855099999999995 2024-09-18T10:15:45.678 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.678 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.678 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-18T10:15:45.678 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:30.472145+0000", 2024-09-18T10:15:45.678 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.58642300599999997 2024-09-18T10:15:45.678 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.678 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.678 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:45.678 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:30.472296+0000", 2024-09-18T10:15:45.678 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00015104300000000001 2024-09-18T10:15:45.678 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:45.678 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:45.678 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:45.679 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.679 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.679 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:659 3.7s0 3:e4da5b5e:::smithi18925896-43:head [append 0~0] snapc 21=[20,1f,1e,1a,16,11,10] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-18T10:15:45.679 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:29.784410+0000", 2024-09-18T10:15:45.679 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 15.803403755, 2024-09-18T10:15:45.679 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.58207420399999998, 2024-09-18T10:15:45.679 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:45.679 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:45.679 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:45.679 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:45.679 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:45.679 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 659 2024-09-18T10:15:45.679 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.679 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:45.680 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.680 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:45.680 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:29.784410+0000", 2024-09-18T10:15:45.680 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:45.680 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.680 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.680 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:45.680 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:29.784410+0000", 2024-09-18T10:15:45.680 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:45.680 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.680 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.680 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:45.680 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:29.784409+0000", 2024-09-18T10:15:45.680 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.999999 2024-09-18T10:15:45.681 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.681 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.681 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:45.681 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:29.784415+0000", 2024-09-18T10:15:45.681 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1529999999999996e-06 2024-09-18T10:15:45.681 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.681 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.681 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:45.681 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:29.784427+0000", 2024-09-18T10:15:45.681 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2249000000000001e-05 2024-09-18T10:15:45.681 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.681 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.681 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:45.681 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:29.784435+0000", 2024-09-18T10:15:45.682 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.8439999999999998e-06 2024-09-18T10:15:45.682 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.682 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.682 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:45.682 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:29.887958+0000", 2024-09-18T10:15:45.682 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.103523322 2024-09-18T10:15:45.682 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.682 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.682 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:45.682 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:29.888077+0000", 2024-09-18T10:15:45.682 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00011933400000000001 2024-09-18T10:15:45.682 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.682 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.683 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:45.683 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:29.888440+0000", 2024-09-18T10:15:45.683 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00036258699999999999 2024-09-18T10:15:45.683 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.683 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.683 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:45.683 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:29.888965+0000", 2024-09-18T10:15:45.683 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00052479299999999996 2024-09-18T10:15:45.683 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.683 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.683 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-18T10:15:45.683 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:30.366426+0000", 2024-09-18T10:15:45.683 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.47746138900000001 2024-09-18T10:15:45.684 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.684 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.684 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:45.684 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:30.366485+0000", 2024-09-18T10:15:45.684 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8415000000000002e-05 2024-09-18T10:15:45.684 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:45.684 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:45.684 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:45.684 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.684 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.684 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:680 3.1s0 3:82c9401b:::smithi18925896-33:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e78)", 2024-09-18T10:15:45.684 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:33.589596+0000", 2024-09-18T10:15:45.684 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 11.998218565, 2024-09-18T10:15:45.684 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.58108971899999995, 2024-09-18T10:15:45.685 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:45.685 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:45.685 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:45.685 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:45.685 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:45.685 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 680 2024-09-18T10:15:45.685 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.685 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:45.685 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.685 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:45.685 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:33.589596+0000", 2024-09-18T10:15:45.685 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:45.685 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.685 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.686 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:45.686 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:33.589596+0000", 2024-09-18T10:15:45.686 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:45.686 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.686 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.686 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:45.686 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:33.589594+0000", 2024-09-18T10:15:45.686 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:45.686 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.686 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.686 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:45.686 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:33.589601+0000", 2024-09-18T10:15:45.686 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1890000000000001e-06 2024-09-18T10:15:45.686 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.687 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.687 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:45.687 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:33.589619+0000", 2024-09-18T10:15:45.687 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7139e-05 2024-09-18T10:15:45.687 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.687 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.687 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:45.687 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:33.589629+0000", 2024-09-18T10:15:45.687 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0906e-05 2024-09-18T10:15:45.687 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.687 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.687 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:45.687 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:33.695533+0000", 2024-09-18T10:15:45.687 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10590384 2024-09-18T10:15:45.688 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.688 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.688 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for rw locks", 2024-09-18T10:15:45.688 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:33.695726+0000", 2024-09-18T10:15:45.688 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00019242400000000001 2024-09-18T10:15:45.688 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.688 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.688 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:45.688 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:33.889917+0000", 2024-09-18T10:15:45.688 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.194191329 2024-09-18T10:15:45.688 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.688 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.688 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:45.688 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:33.890024+0000", 2024-09-18T10:15:45.689 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000107445 2024-09-18T10:15:45.689 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.689 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.689 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:45.689 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:34.170685+0000", 2024-09-18T10:15:45.689 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.28066073899999999 2024-09-18T10:15:45.689 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:45.689 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:45.689 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:45.689 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.689 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.689 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:781 3.1s0 3:84357f33:::smithi18925896-12:head [read 0~196608,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T10:15:45.689 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:40.646300+0000", 2024-09-18T10:15:45.690 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 4.9415144619999998, 2024-09-18T10:15:45.690 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.50956152099999996, 2024-09-18T10:15:45.690 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:45.690 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:45.690 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:45.690 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:45.690 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:45.690 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 781 2024-09-18T10:15:45.690 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.690 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:45.690 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.690 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:45.690 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:40.646300+0000", 2024-09-18T10:15:45.691 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:45.691 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.691 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.691 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:45.691 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:40.646300+0000", 2024-09-18T10:15:45.691 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:45.691 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.691 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.691 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:45.691 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:40.646296+0000", 2024-09-18T10:15:45.691 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999967 2024-09-18T10:15:45.692 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.692 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.692 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:45.692 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:40.646315+0000", 2024-09-18T10:15:45.692 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8085999999999999e-05 2024-09-18T10:15:45.692 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.692 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.692 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:45.692 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:40.646346+0000", 2024-09-18T10:15:45.692 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1853000000000001e-05 2024-09-18T10:15:45.692 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.692 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.693 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:45.693 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:40.646361+0000", 2024-09-18T10:15:45.693 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4843e-05 2024-09-18T10:15:45.693 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.693 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.693 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:45.693 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:40.646503+0000", 2024-09-18T10:15:45.693 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00014215399999999999 2024-09-18T10:15:45.693 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.693 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.693 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op must wait for map", 2024-09-18T10:15:45.693 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:40.646527+0000", 2024-09-18T10:15:45.693 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3272e-05 2024-09-18T10:15:45.693 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.694 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.694 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:45.694 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:40.752054+0000", 2024-09-18T10:15:45.694 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10552738 2024-09-18T10:15:45.694 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.694 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.694 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:45.694 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:40.752175+0000", 2024-09-18T10:15:45.694 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00012100299999999999 2024-09-18T10:15:45.694 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.694 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.694 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:45.694 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:41.155861+0000", 2024-09-18T10:15:45.695 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.40368607299999998 2024-09-18T10:15:45.695 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:45.695 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:45.695 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:45.695 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.695 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.695 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:782 3.1s0 3:84357f33:::smithi18925896-12:head [read 0~196608,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T10:15:45.695 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:40.646399+0000", 2024-09-18T10:15:45.695 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 4.9414154010000004, 2024-09-18T10:15:45.695 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.51008173099999998, 2024-09-18T10:15:45.695 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:45.695 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:45.695 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:45.695 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:45.696 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:45.696 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 782 2024-09-18T10:15:45.696 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.696 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:45.696 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.696 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:45.696 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:40.646399+0000", 2024-09-18T10:15:45.696 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:45.696 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.696 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.696 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:45.696 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:40.646399+0000", 2024-09-18T10:15:45.696 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:45.697 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.697 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.697 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:45.697 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:40.646397+0000", 2024-09-18T10:15:45.697 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:45.697 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.697 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.697 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:45.697 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:40.646413+0000", 2024-09-18T10:15:45.697 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5336999999999999e-05 2024-09-18T10:15:45.697 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.697 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.697 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:45.697 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:40.646434+0000", 2024-09-18T10:15:45.698 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1406000000000001e-05 2024-09-18T10:15:45.698 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.698 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.698 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:45.698 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:40.646447+0000", 2024-09-18T10:15:45.698 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2935999999999999e-05 2024-09-18T10:15:45.698 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.698 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.698 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:45.698 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:40.646629+0000", 2024-09-18T10:15:45.698 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000181776 2024-09-18T10:15:45.698 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.698 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.699 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:45.699 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:40.646646+0000", 2024-09-18T10:15:45.699 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7404000000000001e-05 2024-09-18T10:15:45.699 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.699 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.699 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:45.699 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:40.754087+0000", 2024-09-18T10:15:45.699 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10744084700000001 2024-09-18T10:15:45.699 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.699 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.699 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:45.699 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:40.754211+0000", 2024-09-18T10:15:45.699 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00012409300000000001 2024-09-18T10:15:45.700 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.700 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.700 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:45.700 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:41.156480+0000", 2024-09-18T10:15:45.700 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.40226916200000001 2024-09-18T10:15:45.700 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:45.700 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:45.700 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:45.700 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.700 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.700 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:783 3.1s0 3:84357f33:::smithi18925896-12:head [read 0~196608,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T10:15:45.700 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:40.646467+0000", 2024-09-18T10:15:45.700 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 4.9413472890000003, 2024-09-18T10:15:45.700 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.51080194700000003, 2024-09-18T10:15:45.701 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:45.701 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:45.701 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:45.701 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:45.701 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:45.701 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 783 2024-09-18T10:15:45.701 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.701 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:45.701 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.701 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:45.701 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:40.646467+0000", 2024-09-18T10:15:45.701 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:45.701 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.702 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.702 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:45.702 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:40.646467+0000", 2024-09-18T10:15:45.702 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:45.702 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.702 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.702 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:45.702 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:40.646465+0000", 2024-09-18T10:15:45.702 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:45.702 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.702 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.702 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:45.702 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:40.646474+0000", 2024-09-18T10:15:45.703 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0330000000000008e-06 2024-09-18T10:15:45.703 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.703 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.703 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:45.703 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:40.646491+0000", 2024-09-18T10:15:45.703 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7269999999999999e-05 2024-09-18T10:15:45.703 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.703 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.703 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:45.703 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:40.646501+0000", 2024-09-18T10:15:45.703 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0423e-05 2024-09-18T10:15:45.703 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.703 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.704 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:45.704 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:40.646703+0000", 2024-09-18T10:15:45.704 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000201549 2024-09-18T10:15:45.704 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.704 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.704 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:45.704 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:40.646720+0000", 2024-09-18T10:15:45.704 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7343e-05 2024-09-18T10:15:45.704 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.704 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.704 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:45.704 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:40.754556+0000", 2024-09-18T10:15:45.704 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.107835921 2024-09-18T10:15:45.705 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.705 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.705 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:45.705 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:40.754667+0000", 2024-09-18T10:15:45.705 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000111042 2024-09-18T10:15:45.705 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:45.705 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:45.705 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:45.705 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:41.157269+0000", 2024-09-18T10:15:45.705 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.40260167899999999 2024-09-18T10:15:45.705 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:45.705 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:45.705 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:45.706 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:45.706 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:45.706 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-18T10:15:45.706 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-18T10:15:45.709 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-09-18T10:15:45.710 INFO:tasks.rados.rados.0.smithi189.stdout:315: done (10 left) 2024-09-18T10:15:45.710 INFO:tasks.rados.rados.0.smithi189.stdout:325: done (9 left) 2024-09-18T10:15:45.710 INFO:tasks.rados.rados.0.smithi189.stdout:329: done (8 left) 2024-09-18T10:15:45.711 INFO:tasks.rados.rados.0.smithi189.stdout:331: done (7 left) 2024-09-18T10:15:45.711 INFO:tasks.rados.rados.0.smithi189.stdout:332: done (6 left) 2024-09-18T10:15:45.711 INFO:tasks.rados.rados.0.smithi189.stdout:334: done (5 left) 2024-09-18T10:15:45.711 INFO:tasks.rados.rados.0.smithi189.stdout:335: done (4 left) 2024-09-18T10:15:45.711 INFO:tasks.rados.rados.0.smithi189.stdout:336: done (3 left) 2024-09-18T10:15:45.711 INFO:tasks.rados.rados.0.smithi189.stdout:337: done (2 left) 2024-09-18T10:15:45.711 INFO:tasks.rados.rados.0.smithi189.stdout:339: done (1 left) 2024-09-18T10:15:45.711 INFO:tasks.rados.rados.0.smithi189.stdout:341: done (0 left) 2024-09-18T10:15:45.711 INFO:tasks.rados.rados.0.smithi189.stdout:342: read oid 15 snap -1 2024-09-18T10:15:45.711 INFO:tasks.rados.rados.0.smithi189.stdout:342: expect (ObjNum 95 snap 17 seq_num 95) 2024-09-18T10:15:45.711 INFO:tasks.rados.rados.0.smithi189.stdout:343: copy_from oid 3 from oid 47 current snap is 29 2024-09-18T10:15:45.711 INFO:tasks.rados.rados.0.smithi189.stdout:append oid 38 current snap is 29 2024-09-18T10:15:45.711 INFO:tasks.rados.rados.0.smithi189.stdout:344: seq_num 134 ranges {2195456=704512,2899968=475136,3375104=573440,3948544=507904} 2024-09-18T10:15:45.712 INFO:tasks.rados.rados.0.smithi189.stdout:344: writing smithi18925896-38 from 2195456 to 2899968 tid 1 2024-09-18T10:15:45.712 INFO:tasks.rados.rados.0.smithi189.stdout:344: writing smithi18925896-38 from 2899968 to 3375104 tid 2 2024-09-18T10:15:45.712 INFO:tasks.rados.rados.0.smithi189.stdout:344: writing smithi18925896-38 from 3375104 to 3948544 tid 3 2024-09-18T10:15:45.712 INFO:tasks.rados.rados.0.smithi189.stdout:344: writing smithi18925896-38 from 3948544 to 4456448 tid 4 2024-09-18T10:15:45.712 INFO:tasks.rados.rados.0.smithi189.stdout:345: read oid 39 snap -1 2024-09-18T10:15:45.712 INFO:tasks.rados.rados.0.smithi189.stdout:345: expect deleted 2024-09-18T10:15:45.712 INFO:tasks.rados.rados.0.smithi189.stdout:346: rmattr oid 46 current snap is 29 2024-09-18T10:15:45.712 INFO:tasks.rados.rados.0.smithi189.stdout:346: done (4 left) 2024-09-18T10:15:45.712 INFO:tasks.rados.rados.0.smithi189.stdout:append oid 35 current snap is 29 2024-09-18T10:15:45.712 INFO:tasks.rados.rados.0.smithi189.stdout:347: seq_num 135 ranges {0=720896,720896=622592,1343488=688128,2031616=245760} 2024-09-18T10:15:45.712 INFO:tasks.rados.rados.0.smithi189.stdout:347: writing smithi18925896-35 from 0 to 720896 tid 1 2024-09-18T10:15:45.712 INFO:tasks.rados.rados.0.smithi189.stdout:347: writing smithi18925896-35 from 720896 to 1343488 tid 2 2024-09-18T10:15:45.712 INFO:tasks.rados.rados.0.smithi189.stdout:347: writing smithi18925896-35 from 1343488 to 2031616 tid 3 2024-09-18T10:15:45.712 INFO:tasks.rados.rados.0.smithi189.stdout:347: writing smithi18925896-35 from 2031616 to 2277376 tid 4 2024-09-18T10:15:45.713 INFO:tasks.rados.rados.0.smithi189.stdout:348: read oid 10 snap -1 2024-09-18T10:15:45.713 INFO:tasks.rados.rados.0.smithi189.stdout:348: expect (ObjNum 68 snap 6 seq_num 68) 2024-09-18T10:15:45.713 INFO:tasks.rados.rados.0.smithi189.stdout:349: copy_from oid 41 from oid 16 current snap is 29 2024-09-18T10:15:45.713 INFO:tasks.rados.rados.0.smithi189.stdout:350: read oid 48 snap -1 2024-09-18T10:15:45.713 INFO:tasks.rados.rados.0.smithi189.stdout:350: expect deleted 2024-09-18T10:15:45.713 INFO:tasks.rados.rados.0.smithi189.stdout:351: read oid 45 snap 22 2024-09-18T10:15:45.713 INFO:tasks.rados.rados.0.smithi189.stdout:351: expect (ObjNum 103 snap 21 seq_num 103) 2024-09-18T10:15:45.713 INFO:tasks.rados.rados.0.smithi189.stdout:352: read oid 13 snap -1 2024-09-18T10:15:45.713 INFO:tasks.rados.rados.0.smithi189.stdout:352: expect (ObjNum 33 snap 0 seq_num 33) 2024-09-18T10:15:45.713 INFO:tasks.rados.rados.0.smithi189.stdout:353: snap_create 2024-09-18T10:15:45.714 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-18T10:15:45.704+0000 7f14c6a1a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 22138) UID: 0 2024-09-18T10:15:45.808 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-18T10:15:45.804+0000 7f14c6a1a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 22138) UID: 0 2024-09-18T10:15:45.862 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-18T10:15:45.862 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-18T10:15:45.862 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 0 2024-09-18T10:15:45.862 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-18T10:15:45.872 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-09-18T10:15:45.897 INFO:tasks.rados.rados.0.smithi189.stdout:347: finishing write tid 1 to smithi18925896-35 2024-09-18T10:15:45.909 INFO:tasks.ceph.osd.6.smithi154.stderr:2024-09-18T10:15:45.906+0000 7f0abc9e1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22089) UID: 0 2024-09-18T10:15:45.914 INFO:tasks.rados.rados.0.smithi189.stdout:348: expect (ObjNum 68 snap 6 seq_num 68) 2024-09-18T10:15:45.973 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-18T10:15:45.974 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-18T10:15:45.974 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-18T10:15:45.974 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-18T10:15:45.974 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-18T10:15:45.983 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-09-18T10:15:46.010 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-18T10:15:46.006+0000 7f14dcc5b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22398) UID: 0 2024-09-18T10:15:46.027 INFO:tasks.rados.rados.0.smithi189.stdout:342: expect (ObjNum 95 snap 17 seq_num 95) 2024-09-18T10:15:46.086 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-18T10:15:46.086 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-18T10:15:46.086 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-18T10:15:46.086 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-18T10:15:46.087 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.087 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:134 3.4s0 3:2650be24:::smithi18925896-45:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:46.087 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:50.915670+0000", 2024-09-18T10:15:46.087 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 55.167125296000002, 2024-09-18T10:15:46.087 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.40775830200000002, 2024-09-18T10:15:46.087 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:46.087 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:46.087 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:46.087 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:46.087 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:46.087 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 134 2024-09-18T10:15:46.087 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.088 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:46.088 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.088 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:46.088 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.915670+0000", 2024-09-18T10:15:46.088 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:46.088 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.088 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.088 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:46.088 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.915670+0000", 2024-09-18T10:15:46.088 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:46.088 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.088 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.088 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:46.089 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.915668+0000", 2024-09-18T10:15:46.089 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:46.089 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.089 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.089 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:46.089 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.915676+0000", 2024-09-18T10:15:46.089 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1899999999999995e-06 2024-09-18T10:15:46.089 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.089 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.089 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:46.089 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.915690+0000", 2024-09-18T10:15:46.089 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3827e-05 2024-09-18T10:15:46.089 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.090 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.090 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:46.090 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.915702+0000", 2024-09-18T10:15:46.090 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1477999999999999e-05 2024-09-18T10:15:46.090 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.090 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.090 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:46.090 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.918373+0000", 2024-09-18T10:15:46.090 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0026713000000000002 2024-09-18T10:15:46.090 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.090 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.090 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:46.090 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.918547+0000", 2024-09-18T10:15:46.091 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00017375999999999999 2024-09-18T10:15:46.091 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.091 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.091 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:46.091 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.919485+0000", 2024-09-18T10:15:46.091 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00093826999999999995 2024-09-18T10:15:46.091 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.091 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.091 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:46.091 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.120632+0000", 2024-09-18T10:15:46.091 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.20114729200000001 2024-09-18T10:15:46.091 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.091 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.091 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-18T10:15:46.092 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.323281+0000", 2024-09-18T10:15:46.092 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.20264834500000001 2024-09-18T10:15:46.092 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.092 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.092 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:46.092 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.323428+0000", 2024-09-18T10:15:46.092 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000147334 2024-09-18T10:15:46.092 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:46.092 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:46.092 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:46.092 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.092 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.092 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:135 3.4s0 3:2650be24:::smithi18925896-45:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:46.093 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:50.915712+0000", 2024-09-18T10:15:46.093 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 55.167083437000002, 2024-09-18T10:15:46.093 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.81161388099999998, 2024-09-18T10:15:46.093 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:46.093 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:46.093 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:46.093 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:46.093 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:46.093 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 135 2024-09-18T10:15:46.093 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.093 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:46.094 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.094 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:46.094 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.915712+0000", 2024-09-18T10:15:46.094 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:46.094 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.094 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.094 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:46.095 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.915712+0000", 2024-09-18T10:15:46.095 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:46.095 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.095 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.095 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:46.095 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.915711+0000", 2024-09-18T10:15:46.095 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.999999 2024-09-18T10:15:46.095 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.095 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.095 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:46.095 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.915715+0000", 2024-09-18T10:15:46.095 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.3159999999999998e-06 2024-09-18T10:15:46.096 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.096 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.096 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:46.096 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.915726+0000", 2024-09-18T10:15:46.096 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1375e-05 2024-09-18T10:15:46.096 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.096 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.096 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:46.096 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.915736+0000", 2024-09-18T10:15:46.096 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.4860000000000003e-06 2024-09-18T10:15:46.096 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.096 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.097 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:46.097 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.919925+0000", 2024-09-18T10:15:46.097 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0041886689999999999 2024-09-18T10:15:46.097 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.097 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.097 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for rw locks", 2024-09-18T10:15:46.097 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.920019+0000", 2024-09-18T10:15:46.097 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.4196000000000004e-05 2024-09-18T10:15:46.097 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.097 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.097 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:46.098 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.424171+0000", 2024-09-18T10:15:46.098 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.50415232399999999 2024-09-18T10:15:46.098 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.098 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.098 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:46.098 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.424320+0000", 2024-09-18T10:15:46.098 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00014877299999999999 2024-09-18T10:15:46.098 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.098 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.098 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:46.098 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.727325+0000", 2024-09-18T10:15:46.098 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.30300555400000001 2024-09-18T10:15:46.098 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:46.098 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:46.099 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:46.099 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.099 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.099 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:188 3.4s0 3:213f864e:::smithi18925896-32:head [create,setxattr XNeZS[lOPtVca\\Pa[\\rdfbinb[bcjhtYSpmcafZm (1928) in=40b,setxattr TfflYYqPnps]lo^qSlp___TitNdd`]kaPTVOTcsh (1648) in=40b,setxattr TNb_onpY[j_dkhfmnWtm]Yktg]]RNtbQe[OUTNlX (770) in=40b,setxattr ZcgnkhY\\orTYaZh]PPldj`SNq`p\\cX`k]h\\bZY_Y (700) in=40b,setxattr gPQVnqhpsVhW_kq`QnafXrlRZtZ\\YnRftgQTishR (595) in=40b,setxattr U`XqYi[WhoqnpSPRc\\RhnppZp^qY\\qUfjpqs[r^a (1892) in=40b,setxattr cTmhd`pWgrRf_VfVgOtgeOP^ShP]lsOtpoTj[\\gl (418) in=40b,setxattr aa_taYTfslg`lRRrOkkraN\\XiW^V\\ljl]NqXin`a (314) in=40b,setxattr ibl]`igtgVThY^Qse\\okYYUtTmolX\\W]p]Q[P\\il (1403) in=40b,setxattr \\ZabSO[UUqeZa`hddNh[ZjaRRgUZoQtkn\\QdsX]W (1541) in=40b,setxattr jUkQUPTNbo`rZNmYlUmhb]ojc]aeheoVRfp]bPWf (5052) in=40b,setxattr Ofj`oQ^cR]da^SPsTaXmmPse\\OZ`VSqQ]U`efsZl (419) in=40b,setxattr a[cdkWos\\OrUVPciUaa^_^`lWVTScffiWkd\\[kgf (1261) in=40b,setxattr jXb]\\WhbatdZ[qTTiZO]kPd^`l[kajOYiQ[iqSYc (244) in=40b,setxattr \\im]jQhbanUn\\_^h`jaU]W`ReaZN_je[]kgg (1578) in=36b,setxattr ZbosoRORS^_de]rN^_daOiha\\Q\\eRae^[sQUjXsV (9120) in=40b,setxattr gcn]XpZUkOZr\\pkrVVZpYiR\\alh_PaZa_S[PYcn] (384) in=40b,setxattr \\^qUOaNXSTjtmafhfbo\\gseTTkV`mgnacYq^bkcc (34) in=40b,setxattr iUa^mOQ]TqjVSUU_nXfY]f`NjljiiaYbrm[o]aes (1611) in=40b,setxattr tXaQPQbc^aiV[^reVOPpib^caoWf^XgqilO^_g[k (1016) in=40b] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:46.099 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:52.693780+0000", 2024-09-18T10:15:46.099 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 53.389014860000003, 2024-09-18T10:15:46.099 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.40440725999999999, 2024-09-18T10:15:46.099 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:46.099 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:46.099 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:46.099 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:46.099 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:46.099 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 188 2024-09-18T10:15:46.100 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.100 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:46.100 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.100 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:46.100 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.693780+0000", 2024-09-18T10:15:46.100 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:46.100 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.100 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.100 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:46.100 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.693780+0000", 2024-09-18T10:15:46.100 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:46.100 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.100 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.100 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:46.101 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.693777+0000", 2024-09-18T10:15:46.101 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999967 2024-09-18T10:15:46.101 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.101 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.101 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:46.101 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.693873+0000", 2024-09-18T10:15:46.101 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.6496000000000006e-05 2024-09-18T10:15:46.101 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.101 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.101 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:46.101 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.693926+0000", 2024-09-18T10:15:46.101 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2148999999999998e-05 2024-09-18T10:15:46.101 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.102 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.102 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:46.102 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.693945+0000", 2024-09-18T10:15:46.102 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9296999999999998e-05 2024-09-18T10:15:46.102 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.102 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.102 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:46.102 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.694035+0000", 2024-09-18T10:15:46.102 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.0439999999999995e-05 2024-09-18T10:15:46.102 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.102 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.102 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op must wait for map", 2024-09-18T10:15:46.102 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.694044+0000", 2024-09-18T10:15:46.102 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.1479999999999995e-06 2024-09-18T10:15:46.102 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.103 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.103 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:46.103 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.797388+0000", 2024-09-18T10:15:46.103 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10334315199999999 2024-09-18T10:15:46.103 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.103 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.103 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:46.103 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.797847+0000", 2024-09-18T10:15:46.103 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00045971299999999999 2024-09-18T10:15:46.103 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.103 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.103 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:46.103 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.800981+0000", 2024-09-18T10:15:46.103 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0031333189999999999 2024-09-18T10:15:46.104 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.104 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.104 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:46.104 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.803005+0000", 2024-09-18T10:15:46.104 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0020248359999999999 2024-09-18T10:15:46.104 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.104 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.104 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-18T10:15:46.104 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:53.097948+0000", 2024-09-18T10:15:46.104 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.29494210599999998 2024-09-18T10:15:46.105 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.105 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.105 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:46.105 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:53.098187+0000", 2024-09-18T10:15:46.105 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000239769 2024-09-18T10:15:46.105 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:46.105 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:46.105 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:46.105 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.105 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.105 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWrite(3.cs1 41/32 ECSubWrite(tid=47, reqid=osd.5.0:46, at_version=0'0, trim_to=0'0, roll_forward_to=41'27))", 2024-09-18T10:15:46.105 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:56.935584+0000", 2024-09-18T10:15:46.105 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 49.147211163999998, 2024-09-18T10:15:46.105 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.60626800999999997, 2024-09-18T10:15:46.106 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:46.106 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:46.106 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:46.106 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.106 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:46.106 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.935584+0000", 2024-09-18T10:15:46.106 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:46.106 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.106 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.106 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:46.106 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.935584+0000", 2024-09-18T10:15:46.106 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:46.106 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.107 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.107 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:46.107 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.935582+0000", 2024-09-18T10:15:46.107 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:46.107 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.107 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.107 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:46.107 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.935616+0000", 2024-09-18T10:15:46.107 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.4588999999999997e-05 2024-09-18T10:15:46.107 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.107 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.107 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:46.107 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.935661+0000", 2024-09-18T10:15:46.107 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4705e-05 2024-09-18T10:15:46.108 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.108 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.108 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:46.108 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.935681+0000", 2024-09-18T10:15:46.108 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0290000000000001e-05 2024-09-18T10:15:46.108 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.108 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.108 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:46.108 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.339911+0000", 2024-09-18T10:15:46.108 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.40422952000000001 2024-09-18T10:15:46.108 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.108 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.109 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:46.109 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.339925+0000", 2024-09-18T10:15:46.109 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4523e-05 2024-09-18T10:15:46.109 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.109 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.109 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:46.109 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.541788+0000", 2024-09-18T10:15:46.109 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.20186222000000001 2024-09-18T10:15:46.109 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.109 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.109 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:46.109 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.541852+0000", 2024-09-18T10:15:46.109 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.4171999999999995e-05 2024-09-18T10:15:46.109 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:46.110 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:46.110 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:46.110 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.110 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.110 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWrite(3.cs1 41/32 ECSubWrite(tid=48, reqid=osd.5.0:48, at_version=41'29, trim_to=0'0, roll_forward_to=41'27))", 2024-09-18T10:15:46.110 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:56.937458+0000", 2024-09-18T10:15:46.110 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 49.145337374999997, 2024-09-18T10:15:46.110 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.60464621900000004, 2024-09-18T10:15:46.110 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:46.110 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:46.110 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:46.110 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.110 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:46.111 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.937458+0000", 2024-09-18T10:15:46.111 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:46.111 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.111 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.111 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:46.111 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.937458+0000", 2024-09-18T10:15:46.111 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:46.111 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.111 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.111 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:46.111 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.937456+0000", 2024-09-18T10:15:46.111 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.999999 2024-09-18T10:15:46.111 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.112 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.112 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:46.112 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.937505+0000", 2024-09-18T10:15:46.112 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8809e-05 2024-09-18T10:15:46.112 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.112 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.112 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:46.112 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.937540+0000", 2024-09-18T10:15:46.112 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.4662000000000001e-05 2024-09-18T10:15:46.112 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.112 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.112 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:46.113 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.937554+0000", 2024-09-18T10:15:46.113 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4137e-05 2024-09-18T10:15:46.113 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.113 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.113 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:46.113 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.340361+0000", 2024-09-18T10:15:46.113 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.40280652500000003 2024-09-18T10:15:46.113 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.113 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.113 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:46.113 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.340382+0000", 2024-09-18T10:15:46.113 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.16e-05 2024-09-18T10:15:46.113 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.114 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.114 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:46.114 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.542055+0000", 2024-09-18T10:15:46.114 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.20167278999999999 2024-09-18T10:15:46.114 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.114 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.114 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:46.114 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.542104+0000", 2024-09-18T10:15:46.114 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8866000000000003e-05 2024-09-18T10:15:46.114 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:46.115 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:46.115 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:46.115 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.115 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.115 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:413 3.4s0 3:2650be24:::smithi18925896-45:head [read 0~753664,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-18T10:15:46.115 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:07.553960+0000", 2024-09-18T10:15:46.115 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 38.528835192999999, 2024-09-18T10:15:46.115 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.30658152999999999, 2024-09-18T10:15:46.115 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:46.115 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:46.115 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:46.115 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:46.116 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:46.116 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 413 2024-09-18T10:15:46.116 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.116 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:46.116 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.116 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:46.116 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.553960+0000", 2024-09-18T10:15:46.116 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:46.116 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.116 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.116 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:46.116 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.553960+0000", 2024-09-18T10:15:46.116 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:46.116 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.117 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.117 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:46.117 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.553956+0000", 2024-09-18T10:15:46.117 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999967 2024-09-18T10:15:46.117 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.117 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.117 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:46.117 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.553973+0000", 2024-09-18T10:15:46.117 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.6602000000000001e-05 2024-09-18T10:15:46.117 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.117 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.117 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:46.118 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.554010+0000", 2024-09-18T10:15:46.118 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7129999999999999e-05 2024-09-18T10:15:46.118 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.118 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.118 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:46.118 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.554032+0000", 2024-09-18T10:15:46.118 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2316000000000001e-05 2024-09-18T10:15:46.118 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.118 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.118 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:46.118 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.554187+0000", 2024-09-18T10:15:46.118 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00015407499999999999 2024-09-18T10:15:46.118 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.119 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.119 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op must wait for map", 2024-09-18T10:15:46.119 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.554232+0000", 2024-09-18T10:15:46.119 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.5173999999999998e-05 2024-09-18T10:15:46.119 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.119 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.119 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:46.119 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.655961+0000", 2024-09-18T10:15:46.119 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10172934 2024-09-18T10:15:46.119 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.119 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.119 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:46.119 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.656152+0000", 2024-09-18T10:15:46.120 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00019075599999999999 2024-09-18T10:15:46.120 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.120 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.120 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:46.120 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.860541+0000", 2024-09-18T10:15:46.120 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.20438944000000001 2024-09-18T10:15:46.120 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:46.120 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:46.120 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:46.120 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.120 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.120 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:414 3.4s0 3:2650be24:::smithi18925896-45:head [read 0~753664,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-18T10:15:46.120 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:07.554078+0000", 2024-09-18T10:15:46.121 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 38.528716889000002, 2024-09-18T10:15:46.121 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.30769232899999999, 2024-09-18T10:15:46.121 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:46.121 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:46.121 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:46.121 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:46.121 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:46.121 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 414 2024-09-18T10:15:46.121 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.121 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:46.121 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.121 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:46.121 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.554078+0000", 2024-09-18T10:15:46.121 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:46.122 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.122 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.122 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:46.122 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.554078+0000", 2024-09-18T10:15:46.122 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:46.122 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.122 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.122 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:46.122 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.554077+0000", 2024-09-18T10:15:46.122 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:46.122 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.122 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.122 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:46.123 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.554086+0000", 2024-09-18T10:15:46.123 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.6569999999999992e-06 2024-09-18T10:15:46.123 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.123 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.123 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:46.123 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.554106+0000", 2024-09-18T10:15:46.123 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9969000000000001e-05 2024-09-18T10:15:46.123 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.123 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.123 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:46.123 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.554118+0000", 2024-09-18T10:15:46.123 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.217e-05 2024-09-18T10:15:46.123 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.124 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.124 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:46.124 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.554389+0000", 2024-09-18T10:15:46.124 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00027051000000000001 2024-09-18T10:15:46.124 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.124 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.124 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:46.124 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.554407+0000", 2024-09-18T10:15:46.124 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7677e-05 2024-09-18T10:15:46.124 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.124 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.125 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:46.125 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.656732+0000", 2024-09-18T10:15:46.125 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.102325597 2024-09-18T10:15:46.125 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.125 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.125 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:46.125 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.656894+0000", 2024-09-18T10:15:46.125 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000161383 2024-09-18T10:15:46.125 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.125 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.126 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:46.126 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.861770+0000", 2024-09-18T10:15:46.126 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.20487676599999999 2024-09-18T10:15:46.126 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:46.126 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:46.126 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:46.126 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.126 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.126 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:415 3.4s0 3:2650be24:::smithi18925896-45:head [read 0~753664,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-18T10:15:46.126 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:07.554169+0000", 2024-09-18T10:15:46.126 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 38.528626428000003, 2024-09-18T10:15:46.126 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.308598176, 2024-09-18T10:15:46.127 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:46.127 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:46.127 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:46.127 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:46.127 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:46.127 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 415 2024-09-18T10:15:46.127 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.127 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:46.127 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.127 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:46.127 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.554169+0000", 2024-09-18T10:15:46.127 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:46.127 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.128 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.128 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:46.128 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.554169+0000", 2024-09-18T10:15:46.128 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:46.128 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.128 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.128 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:46.128 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.554167+0000", 2024-09-18T10:15:46.128 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:46.128 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.128 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.128 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:46.129 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.554176+0000", 2024-09-18T10:15:46.129 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6430000000000001e-06 2024-09-18T10:15:46.129 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.129 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.129 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:46.129 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.554197+0000", 2024-09-18T10:15:46.129 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1209e-05 2024-09-18T10:15:46.129 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.129 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.129 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:46.129 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.554206+0000", 2024-09-18T10:15:46.129 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.3570000000000004e-06 2024-09-18T10:15:46.129 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.129 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.130 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:46.130 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.554515+0000", 2024-09-18T10:15:46.130 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00030821000000000001 2024-09-18T10:15:46.130 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.130 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.130 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:46.130 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.554535+0000", 2024-09-18T10:15:46.130 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0092999999999999e-05 2024-09-18T10:15:46.130 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.130 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.130 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:46.131 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.756075+0000", 2024-09-18T10:15:46.131 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.201540831 2024-09-18T10:15:46.131 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.131 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.131 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:46.131 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.756260+0000", 2024-09-18T10:15:46.131 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00018466000000000001 2024-09-18T10:15:46.131 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.131 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.131 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:46.131 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.862767+0000", 2024-09-18T10:15:46.131 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.106506579 2024-09-18T10:15:46.131 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:46.131 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:46.132 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:46.132 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.132 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.132 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(osd.4.21:3 3.4s0 3:2619bf65:::smithi18925896-16:head [assert-version v2,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-18T10:15:46.132 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:11.936400+0000", 2024-09-18T10:15:46.132 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 34.146395273000003, 2024-09-18T10:15:46.132 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.312109203, 2024-09-18T10:15:46.132 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:46.132 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:46.132 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:46.132 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.132 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:46.133 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:11.936400+0000", 2024-09-18T10:15:46.133 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:46.133 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.133 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.133 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:46.133 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:11.936400+0000", 2024-09-18T10:15:46.133 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:46.133 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.133 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.133 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:46.133 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:11.936396+0000", 2024-09-18T10:15:46.133 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999962 2024-09-18T10:15:46.133 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.134 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.134 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:46.134 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:11.936434+0000", 2024-09-18T10:15:46.134 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7580000000000003e-05 2024-09-18T10:15:46.134 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.134 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.134 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:46.134 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:11.936473+0000", 2024-09-18T10:15:46.134 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9876000000000002e-05 2024-09-18T10:15:46.134 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.134 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.134 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:46.134 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:11.936499+0000", 2024-09-18T10:15:46.135 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6012000000000001e-05 2024-09-18T10:15:46.135 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.135 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.135 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:46.135 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.039131+0000", 2024-09-18T10:15:46.135 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10263111799999999 2024-09-18T10:15:46.135 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.135 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.135 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:46.135 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.039402+0000", 2024-09-18T10:15:46.135 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00027099399999999999 2024-09-18T10:15:46.135 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.135 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.136 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:46.136 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.248509+0000", 2024-09-18T10:15:46.136 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.20910732700000001 2024-09-18T10:15:46.136 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:46.136 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:46.136 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:46.136 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.136 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.136 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:446 3.4s0 3:2650be24:::smithi18925896-45:head [append 0~622592 in=622592b] snapc 13=[13,12,11,10,f,e,b] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-18T10:15:46.136 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:12.892106+0000", 2024-09-18T10:15:46.136 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 33.190688643999998, 2024-09-18T10:15:46.136 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.77541641100000003, 2024-09-18T10:15:46.137 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:46.137 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:46.137 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:46.137 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:46.137 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:46.137 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 446 2024-09-18T10:15:46.137 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.137 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:46.137 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.137 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:46.137 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.892106+0000", 2024-09-18T10:15:46.137 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:46.137 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.138 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.138 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:46.138 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.892106+0000", 2024-09-18T10:15:46.138 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:46.138 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.138 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.138 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:46.138 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.892103+0000", 2024-09-18T10:15:46.138 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999971 2024-09-18T10:15:46.138 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.138 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.138 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:46.138 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.893050+0000", 2024-09-18T10:15:46.138 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00094607000000000003 2024-09-18T10:15:46.139 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.139 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.139 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:46.139 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.893084+0000", 2024-09-18T10:15:46.139 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.4759999999999999e-05 2024-09-18T10:15:46.139 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.139 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.139 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:46.139 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.893098+0000", 2024-09-18T10:15:46.139 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3859000000000001e-05 2024-09-18T10:15:46.139 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.139 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.139 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:46.140 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.145987+0000", 2024-09-18T10:15:46.140 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.252889115 2024-09-18T10:15:46.140 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.140 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.140 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:46.140 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.146112+0000", 2024-09-18T10:15:46.140 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00012469299999999999 2024-09-18T10:15:46.140 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.140 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.140 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:46.140 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.146923+0000", 2024-09-18T10:15:46.140 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00081091500000000005 2024-09-18T10:15:46.140 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.141 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.141 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:46.141 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.152437+0000", 2024-09-18T10:15:46.141 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0055141169999999998 2024-09-18T10:15:46.141 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.141 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.141 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-18T10:15:46.141 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.667347+0000", 2024-09-18T10:15:46.141 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.51491041199999998 2024-09-18T10:15:46.141 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.141 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.141 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:46.141 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.667523+0000", 2024-09-18T10:15:46.141 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00017533199999999999 2024-09-18T10:15:46.142 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:46.142 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:46.142 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:46.142 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.142 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.142 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:447 3.4s0 3:2650be24:::smithi18925896-45:head [append 0~475136 in=475136b] snapc 13=[13,12,11,10,f,e,b] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-18T10:15:46.142 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:12.896134+0000", 2024-09-18T10:15:46.142 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 33.186661280999999, 2024-09-18T10:15:46.142 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.87223328099999997, 2024-09-18T10:15:46.142 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:46.142 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:46.142 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:46.142 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:46.142 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:46.143 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 447 2024-09-18T10:15:46.143 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.143 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:46.143 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.143 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:46.143 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.896134+0000", 2024-09-18T10:15:46.143 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:46.143 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.143 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.143 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:46.143 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.896134+0000", 2024-09-18T10:15:46.143 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:46.143 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.143 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.144 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:46.144 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.896131+0000", 2024-09-18T10:15:46.144 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999976 2024-09-18T10:15:46.144 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.144 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.144 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:46.144 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.896661+0000", 2024-09-18T10:15:46.144 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00052970299999999997 2024-09-18T10:15:46.144 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.144 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.144 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:46.144 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.896693+0000", 2024-09-18T10:15:46.144 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1903000000000003e-05 2024-09-18T10:15:46.145 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.145 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.145 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:46.145 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.896705+0000", 2024-09-18T10:15:46.145 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2288e-05 2024-09-18T10:15:46.145 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.145 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.145 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:46.145 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.147534+0000", 2024-09-18T10:15:46.145 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.25082866300000001 2024-09-18T10:15:46.145 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.145 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.145 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:46.145 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.147612+0000", 2024-09-18T10:15:46.146 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7789999999999999e-05 2024-09-18T10:15:46.146 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.146 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.146 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:46.146 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.148220+0000", 2024-09-18T10:15:46.146 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00060859599999999999 2024-09-18T10:15:46.146 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.147 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.147 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:46.147 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.152441+0000", 2024-09-18T10:15:46.147 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0042208250000000001 2024-09-18T10:15:46.147 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.147 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.147 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-18T10:15:46.147 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.768118+0000", 2024-09-18T10:15:46.147 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.615676471 2024-09-18T10:15:46.147 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.147 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.147 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:46.147 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.768367+0000", 2024-09-18T10:15:46.148 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00024930800000000001 2024-09-18T10:15:46.148 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:46.148 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:46.148 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:46.148 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.148 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.148 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:448 3.4s0 3:2650be24:::smithi18925896-45:head [append 0~294912 in=294912b] snapc 13=[13,12,11,10,f,e,b] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-18T10:15:46.148 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:12.897981+0000", 2024-09-18T10:15:46.148 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 33.184814103000001, 2024-09-18T10:15:46.148 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.87089765500000005, 2024-09-18T10:15:46.148 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:46.148 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:46.148 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:46.149 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:46.149 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:46.149 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 448 2024-09-18T10:15:46.149 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.149 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:46.149 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.149 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:46.149 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.897981+0000", 2024-09-18T10:15:46.149 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:46.149 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.149 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.149 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:46.150 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.897981+0000", 2024-09-18T10:15:46.150 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:46.150 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.150 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.150 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:46.150 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.897979+0000", 2024-09-18T10:15:46.150 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:46.150 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.150 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.150 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:46.150 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.898390+0000", 2024-09-18T10:15:46.150 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00041124999999999999 2024-09-18T10:15:46.150 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.150 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.151 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:46.151 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.898416+0000", 2024-09-18T10:15:46.151 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.525e-05 2024-09-18T10:15:46.151 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.151 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.151 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:46.151 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.898434+0000", 2024-09-18T10:15:46.151 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8615999999999999e-05 2024-09-18T10:15:46.151 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.151 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.151 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:46.151 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.148720+0000", 2024-09-18T10:15:46.151 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.25028612900000002 2024-09-18T10:15:46.152 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.152 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.152 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:46.152 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.148799+0000", 2024-09-18T10:15:46.152 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.8481999999999998e-05 2024-09-18T10:15:46.152 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.152 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.152 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:46.152 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.149383+0000", 2024-09-18T10:15:46.152 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000584255 2024-09-18T10:15:46.152 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.152 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.153 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:46.153 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.152442+0000", 2024-09-18T10:15:46.153 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0030591049999999999 2024-09-18T10:15:46.153 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.153 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.153 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-18T10:15:46.153 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.768702+0000", 2024-09-18T10:15:46.153 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.61626031599999997 2024-09-18T10:15:46.153 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.153 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.153 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:46.154 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.768878+0000", 2024-09-18T10:15:46.154 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00017600600000000001 2024-09-18T10:15:46.154 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:46.154 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:46.154 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:46.154 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.154 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.154 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:449 3.4s0 3:2650be24:::smithi18925896-45:head [setxattr _header (57) in=64b] snapc 13=[13,12,11,10,f,e,b] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-18T10:15:46.154 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:12.898446+0000", 2024-09-18T10:15:46.154 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 33.184348585000002, 2024-09-18T10:15:46.154 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.87093100999999995, 2024-09-18T10:15:46.154 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:46.154 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:46.155 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:46.155 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:46.155 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:46.155 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 449 2024-09-18T10:15:46.155 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.155 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:46.155 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.155 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:46.155 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.898446+0000", 2024-09-18T10:15:46.155 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:46.155 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.155 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.155 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:46.155 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.898446+0000", 2024-09-18T10:15:46.156 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:46.156 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.156 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.156 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:46.156 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.898445+0000", 2024-09-18T10:15:46.156 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.999999 2024-09-18T10:15:46.156 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.156 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.156 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:46.156 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.898451+0000", 2024-09-18T10:15:46.156 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.054e-06 2024-09-18T10:15:46.156 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.156 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.156 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:46.157 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.898462+0000", 2024-09-18T10:15:46.157 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.081e-05 2024-09-18T10:15:46.157 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.157 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.157 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:46.157 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.898467+0000", 2024-09-18T10:15:46.157 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4220000000000002e-06 2024-09-18T10:15:46.157 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.157 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.157 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:46.157 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.149769+0000", 2024-09-18T10:15:46.157 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.25130268 2024-09-18T10:15:46.157 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.157 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.158 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:46.158 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.149964+0000", 2024-09-18T10:15:46.158 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00019506699999999999 2024-09-18T10:15:46.158 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.158 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.158 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:46.158 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.151037+0000", 2024-09-18T10:15:46.158 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0010723200000000001 2024-09-18T10:15:46.158 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.158 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.158 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:46.158 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.152623+0000", 2024-09-18T10:15:46.158 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0015865829999999999 2024-09-18T10:15:46.158 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.159 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.159 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-18T10:15:46.159 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.769215+0000", 2024-09-18T10:15:46.159 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.61659198800000004 2024-09-18T10:15:46.159 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.159 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.159 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:46.159 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.769377+0000", 2024-09-18T10:15:46.159 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000162134 2024-09-18T10:15:46.159 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:46.159 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:46.159 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:46.159 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.159 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.160 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:450 3.4s0 3:2650be24:::smithi18925896-45:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e57)", 2024-09-18T10:15:46.160 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:12.898472+0000", 2024-09-18T10:15:46.160 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 33.184322999999999, 2024-09-18T10:15:46.160 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.172788277, 2024-09-18T10:15:46.160 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:46.160 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:46.160 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:46.160 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:46.160 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:46.160 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 450 2024-09-18T10:15:46.160 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.160 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:46.161 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.161 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:46.161 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.898472+0000", 2024-09-18T10:15:46.161 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:46.161 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.161 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.161 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:46.161 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.898472+0000", 2024-09-18T10:15:46.161 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:46.161 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.161 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.161 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:46.161 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.898471+0000", 2024-09-18T10:15:46.161 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:46.162 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.162 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.162 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:46.162 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.898475+0000", 2024-09-18T10:15:46.162 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1480000000000002e-06 2024-09-18T10:15:46.162 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.162 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.162 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:46.162 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.898485+0000", 2024-09-18T10:15:46.162 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0342e-05 2024-09-18T10:15:46.162 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.162 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.162 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:46.162 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.898490+0000", 2024-09-18T10:15:46.163 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7060000000000003e-06 2024-09-18T10:15:46.163 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.163 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.163 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:46.163 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.151519+0000", 2024-09-18T10:15:46.163 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.25302985300000003 2024-09-18T10:15:46.163 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.163 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.163 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for rw locks", 2024-09-18T10:15:46.163 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.151633+0000", 2024-09-18T10:15:46.163 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000113121 2024-09-18T10:15:46.163 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.163 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.164 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:46.164 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.770549+0000", 2024-09-18T10:15:46.164 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.61891632200000002 2024-09-18T10:15:46.164 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.164 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.164 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:46.164 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.770780+0000", 2024-09-18T10:15:46.164 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00023144999999999999 2024-09-18T10:15:46.164 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.164 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.164 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:46.164 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:14.071260+0000", 2024-09-18T10:15:46.164 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.30047984999999999 2024-09-18T10:15:46.164 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:46.165 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:46.165 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:46.165 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.165 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.165 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:622 3.4s0 3:213f864e:::smithi18925896-32:head [read 0~606208,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-18T10:15:46.165 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:28.103279+0000", 2024-09-18T10:15:46.165 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 17.979515716000002, 2024-09-18T10:15:46.165 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.33052193600000002, 2024-09-18T10:15:46.165 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:46.165 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:46.165 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:46.165 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:46.165 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:46.165 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 622 2024-09-18T10:15:46.166 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.166 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:46.166 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.166 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:46.166 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:28.103279+0000", 2024-09-18T10:15:46.166 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:46.166 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.166 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.166 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:46.166 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:28.103279+0000", 2024-09-18T10:15:46.166 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:46.166 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.166 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.166 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:46.167 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:28.103279+0000", 2024-09-18T10:15:46.167 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:46.167 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.167 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.167 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:46.167 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:28.103284+0000", 2024-09-18T10:15:46.167 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0250000000000002e-06 2024-09-18T10:15:46.167 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.167 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.167 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:46.167 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:28.103300+0000", 2024-09-18T10:15:46.167 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.6850999999999998e-05 2024-09-18T10:15:46.167 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.167 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.168 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:46.168 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:28.103308+0000", 2024-09-18T10:15:46.168 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1450000000000002e-06 2024-09-18T10:15:46.168 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.168 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.168 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:46.168 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:28.103428+0000", 2024-09-18T10:15:46.168 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00012029200000000001 2024-09-18T10:15:46.168 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.168 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.168 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:46.168 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:28.103435+0000", 2024-09-18T10:15:46.168 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.6930000000000001e-06 2024-09-18T10:15:46.169 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.169 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.169 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:46.169 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:28.187445+0000", 2024-09-18T10:15:46.169 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.084010325999999996 2024-09-18T10:15:46.169 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.169 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.169 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:46.169 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:28.187954+0000", 2024-09-18T10:15:46.169 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00050906599999999997 2024-09-18T10:15:46.169 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.169 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.169 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:46.169 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:28.433801+0000", 2024-09-18T10:15:46.170 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.24584714399999999 2024-09-18T10:15:46.170 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:46.170 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:46.170 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:46.170 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.170 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.170 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:623 3.4s0 3:213f864e:::smithi18925896-32:head [read 0~606208,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-18T10:15:46.170 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:28.103315+0000", 2024-09-18T10:15:46.170 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 17.979480299999999, 2024-09-18T10:15:46.170 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.33131258499999999, 2024-09-18T10:15:46.170 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:46.170 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:46.170 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:46.170 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:46.170 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:46.171 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 623 2024-09-18T10:15:46.171 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.171 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:46.171 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.171 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:46.171 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:28.103315+0000", 2024-09-18T10:15:46.171 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:46.171 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.171 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.171 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:46.171 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:28.103315+0000", 2024-09-18T10:15:46.171 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:46.171 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.171 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.172 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:46.172 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:28.103314+0000", 2024-09-18T10:15:46.172 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:46.172 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.172 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.172 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:46.172 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:28.103319+0000", 2024-09-18T10:15:46.172 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6829999999999999e-06 2024-09-18T10:15:46.172 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.172 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.172 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:46.172 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:28.103329+0000", 2024-09-18T10:15:46.172 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0511e-05 2024-09-18T10:15:46.172 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.172 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.173 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:46.173 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:28.203443+0000", 2024-09-18T10:15:46.173 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10011328799999999 2024-09-18T10:15:46.173 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.173 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.173 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:46.173 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:28.292935+0000", 2024-09-18T10:15:46.173 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.089492877999999998 2024-09-18T10:15:46.173 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.173 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.173 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:46.173 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:28.293133+0000", 2024-09-18T10:15:46.173 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00019793 2024-09-18T10:15:46.173 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.174 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.174 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:46.174 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:28.434627+0000", 2024-09-18T10:15:46.174 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.14149382099999999 2024-09-18T10:15:46.174 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:46.174 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:46.174 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:46.174 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.174 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.174 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWrite(3.9s5 74/40 ECSubWrite(tid=178, reqid=osd.7.0:178, at_version=74'78, trim_to=0'0, roll_forward_to=72'76))", 2024-09-18T10:15:46.174 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:29.615287+0000", 2024-09-18T10:15:46.174 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 16.467507708999999, 2024-09-18T10:15:46.174 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.35347188099999999, 2024-09-18T10:15:46.174 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:46.175 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:46.175 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:46.175 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.175 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:46.175 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:29.615287+0000", 2024-09-18T10:15:46.175 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:46.175 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.175 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.175 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:46.175 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:29.615287+0000", 2024-09-18T10:15:46.175 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:46.175 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.175 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.175 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:46.176 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:29.615286+0000", 2024-09-18T10:15:46.176 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:46.176 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.176 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.176 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:46.176 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:29.615331+0000", 2024-09-18T10:15:46.176 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4873000000000002e-05 2024-09-18T10:15:46.176 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.176 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.176 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:46.176 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:29.615363+0000", 2024-09-18T10:15:46.176 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1953999999999999e-05 2024-09-18T10:15:46.176 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.177 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.177 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:46.177 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:29.615376+0000", 2024-09-18T10:15:46.177 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3071000000000001e-05 2024-09-18T10:15:46.177 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.177 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.177 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:46.177 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:29.767783+0000", 2024-09-18T10:15:46.177 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.15240696000000001 2024-09-18T10:15:46.177 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.177 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.177 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:46.177 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:29.767797+0000", 2024-09-18T10:15:46.177 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4688000000000001e-05 2024-09-18T10:15:46.178 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.178 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.178 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:46.178 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:29.968659+0000", 2024-09-18T10:15:46.178 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.20086167699999999 2024-09-18T10:15:46.178 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.178 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.178 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:46.178 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:29.968759+0000", 2024-09-18T10:15:46.178 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000100175 2024-09-18T10:15:46.178 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:46.178 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:46.178 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:46.178 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.179 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.179 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWrite(3.9s5 74/40 ECSubWrite(tid=179, reqid=osd.7.0:179, at_version=74'80, trim_to=0'0, roll_forward_to=72'76))", 2024-09-18T10:15:46.179 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:29.616697+0000", 2024-09-18T10:15:46.179 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 16.466097446999999, 2024-09-18T10:15:46.179 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.35257461800000001, 2024-09-18T10:15:46.179 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:46.179 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:46.179 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:46.179 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.179 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:46.179 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:29.616697+0000", 2024-09-18T10:15:46.179 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:46.179 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.180 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.180 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:46.180 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:29.616697+0000", 2024-09-18T10:15:46.180 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:46.180 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.180 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.180 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:46.180 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:29.616689+0000", 2024-09-18T10:15:46.180 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999914 2024-09-18T10:15:46.180 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.180 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.180 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:46.180 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:29.616734+0000", 2024-09-18T10:15:46.180 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.5073e-05 2024-09-18T10:15:46.181 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.181 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.181 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:46.181 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:29.616764+0000", 2024-09-18T10:15:46.181 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9536e-05 2024-09-18T10:15:46.181 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.181 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.181 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:46.181 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:29.616812+0000", 2024-09-18T10:15:46.181 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8127999999999997e-05 2024-09-18T10:15:46.181 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.181 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.181 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:46.181 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:29.867763+0000", 2024-09-18T10:15:46.182 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.25095157400000001 2024-09-18T10:15:46.182 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.182 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.182 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:46.182 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:29.867775+0000", 2024-09-18T10:15:46.182 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1501000000000001e-05 2024-09-18T10:15:46.182 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.182 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.182 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:46.182 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:29.969220+0000", 2024-09-18T10:15:46.182 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.101445459 2024-09-18T10:15:46.182 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.182 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.183 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:46.183 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:29.969272+0000", 2024-09-18T10:15:46.183 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1746000000000002e-05 2024-09-18T10:15:46.183 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:46.183 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:46.183 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:46.183 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.183 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.183 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDPGPushReply(3.19s0 79/74 [PushReplyOp(3:9a1a774c:::smithi18925896-38:19)])", 2024-09-18T10:15:46.183 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:35.704115+0000", 2024-09-18T10:15:46.183 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 10.378679476, 2024-09-18T10:15:46.183 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.30460024099999999, 2024-09-18T10:15:46.183 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:46.183 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:46.184 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:46.184 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.184 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:46.184 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:35.704115+0000", 2024-09-18T10:15:46.184 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:46.184 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.184 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.184 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:46.184 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:35.704115+0000", 2024-09-18T10:15:46.184 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:46.184 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.184 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.184 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:46.184 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:35.704115+0000", 2024-09-18T10:15:46.185 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:46.185 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.185 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.185 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:46.185 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:35.704121+0000", 2024-09-18T10:15:46.185 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.434e-06 2024-09-18T10:15:46.185 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.185 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.185 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:46.185 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:35.704141+0000", 2024-09-18T10:15:46.185 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0299e-05 2024-09-18T10:15:46.185 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.185 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.185 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:46.186 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:35.704149+0000", 2024-09-18T10:15:46.186 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.3499999999999997e-06 2024-09-18T10:15:46.186 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.186 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.186 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:46.186 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:36.008607+0000", 2024-09-18T10:15:46.186 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.30445813700000002 2024-09-18T10:15:46.186 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.186 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.186 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:46.186 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:36.008716+0000", 2024-09-18T10:15:46.186 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000108385 2024-09-18T10:15:46.186 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:46.187 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:46.187 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:46.187 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.187 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.187 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDPGPushReply(3.19s0 80/74 [PushReplyOp(3:9ab7c4c0:::smithi18925896-46:head)])", 2024-09-18T10:15:46.187 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:35.810513+0000", 2024-09-18T10:15:46.187 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 10.272281801, 2024-09-18T10:15:46.187 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.30170829700000001, 2024-09-18T10:15:46.187 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:46.187 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:46.187 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:46.187 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.187 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:46.187 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:35.810513+0000", 2024-09-18T10:15:46.188 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:46.188 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.188 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.188 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:46.188 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:35.810513+0000", 2024-09-18T10:15:46.188 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:46.188 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.188 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.188 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:46.188 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:35.810512+0000", 2024-09-18T10:15:46.188 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.999999 2024-09-18T10:15:46.188 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.188 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.189 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:46.189 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:35.810523+0000", 2024-09-18T10:15:46.189 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1109e-05 2024-09-18T10:15:46.189 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.189 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.189 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:46.189 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:35.810553+0000", 2024-09-18T10:15:46.189 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.955e-05 2024-09-18T10:15:46.189 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.189 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.189 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:46.189 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:35.810571+0000", 2024-09-18T10:15:46.189 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8076000000000001e-05 2024-09-18T10:15:46.189 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.189 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.190 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:46.190 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:36.112117+0000", 2024-09-18T10:15:46.190 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.30154614000000002 2024-09-18T10:15:46.190 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.190 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.190 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:46.190 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:36.112221+0000", 2024-09-18T10:15:46.190 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000104512 2024-09-18T10:15:46.190 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:46.190 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:46.190 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:46.190 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:46.190 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:46.190 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-18T10:15:46.192 INFO:tasks.rados.rados.0.smithi189.stdout:347: finishing write tid 2 to smithi18925896-35 2024-09-18T10:15:46.192 INFO:tasks.rados.rados.0.smithi189.stdout:344: finishing write tid 1 to smithi18925896-38 2024-09-18T10:15:46.192 INFO:tasks.rados.rados.0.smithi189.stdout:352: expect (ObjNum 33 snap 0 seq_num 33) 2024-09-18T10:15:46.192 INFO:tasks.rados.rados.0.smithi189.stdout:347: finishing write tid 3 to smithi18925896-35 2024-09-18T10:15:46.192 INFO:tasks.rados.rados.0.smithi189.stdout:347: finishing write tid 4 to smithi18925896-35 2024-09-18T10:15:46.192 INFO:tasks.rados.rados.0.smithi189.stdout:343: finishing copy_from racing read to smithi18925896-3 2024-09-18T10:15:46.193 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_ops_in_flight 2024-09-18T10:15:46.193 INFO:tasks.ceph.osd.12.smithi029.stderr:2024-09-18T10:15:46.185+0000 7f8e4e627700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 22190) UID: 0 2024-09-18T10:15:46.293 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-18T10:15:46.285+0000 7f302f2c3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22187) UID: 0 2024-09-18T10:15:46.308 INFO:tasks.rados.rados.0.smithi189.stdout:343: finishing copy_from to smithi18925896-3 2024-09-18T10:15:46.308 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 3 v 42 (ObjNum 41 snap 0 seq_num 41) dirty exists 2024-09-18T10:15:46.308 INFO:tasks.rados.rados.0.smithi189.stdout:344: finishing write tid 2 to smithi18925896-38 2024-09-18T10:15:46.308 INFO:tasks.rados.rados.0.smithi189.stdout:344: finishing write tid 3 to smithi18925896-38 2024-09-18T10:15:46.346 INFO:tasks.rados.rados.0.smithi189.stdout:347: finishing write tid 5 to smithi18925896-35 2024-09-18T10:15:46.347 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-18T10:15:46.347 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-18T10:15:46.347 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.347 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpRead(3.cs5 90/83 ECSubRead(tid=193, to_read={3:3d6b9cb0:::smithi18925896-31:head=0,262144,0}, subchunks={3:3d6b9cb0:::smithi18925896-31:head=[0,1]}, attrs_to_read=))", 2024-09-18T10:15:46.347 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:45.755762+0000", 2024-09-18T10:15:46.348 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.58970978200000002, 2024-09-18T10:15:46.348 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.58975633400000005, 2024-09-18T10:15:46.348 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:46.348 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-18T10:15:46.348 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:46.348 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.349 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:46.349 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:45.755762+0000", 2024-09-18T10:15:46.349 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:46.349 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.349 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.349 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:46.349 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:45.755762+0000", 2024-09-18T10:15:46.349 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:46.349 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.349 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.349 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:46.350 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:45.755761+0000", 2024-09-18T10:15:46.350 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.999999 2024-09-18T10:15:46.350 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.350 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.350 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:46.350 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:45.755773+0000", 2024-09-18T10:15:46.350 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1620999999999999e-05 2024-09-18T10:15:46.350 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.350 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.350 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:46.351 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:45.755806+0000", 2024-09-18T10:15:46.351 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.3584999999999998e-05 2024-09-18T10:15:46.351 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.351 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.351 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:46.351 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:45.755817+0000", 2024-09-18T10:15:46.351 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0859999999999999e-05 2024-09-18T10:15:46.351 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:46.351 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:46.351 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:46.351 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.352 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.352 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(osd.9.21:2 3.4s0 3.a6fd9864 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T10:15:46.352 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:45.796940+0000", 2024-09-18T10:15:46.352 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.54853236900000002, 2024-09-18T10:15:46.352 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.54866792200000003, 2024-09-18T10:15:46.352 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:46.352 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-18T10:15:46.352 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:46.352 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.353 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:46.353 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:45.796940+0000", 2024-09-18T10:15:46.353 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:46.353 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.353 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.353 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:46.354 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:45.796940+0000", 2024-09-18T10:15:46.354 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:46.354 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.354 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.354 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:46.354 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:45.796937+0000", 2024-09-18T10:15:46.354 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999976 2024-09-18T10:15:46.354 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.354 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.354 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:46.354 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:45.796948+0000", 2024-09-18T10:15:46.355 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0711e-05 2024-09-18T10:15:46.355 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.355 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.355 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:46.355 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:45.796967+0000", 2024-09-18T10:15:46.355 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9009e-05 2024-09-18T10:15:46.355 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.355 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.355 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:46.355 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:45.796977+0000", 2024-09-18T10:15:46.356 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.781e-06 2024-09-18T10:15:46.356 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:46.356 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:46.356 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:46.356 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.356 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.356 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpRead(3.cs5 90/83 ECSubRead(tid=195, to_read={3:3a4fded2:::smithi18925896-23:head=1310720,262144,0}, subchunks={3:3a4fded2:::smithi18925896-23:head=[0,1]}, attrs_to_read=))", 2024-09-18T10:15:46.356 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:46.163022+0000", 2024-09-18T10:15:46.356 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.18244940300000001, 2024-09-18T10:15:46.356 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.18266766800000001, 2024-09-18T10:15:46.357 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:46.357 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-18T10:15:46.357 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:46.357 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.357 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:46.357 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:46.163022+0000", 2024-09-18T10:15:46.357 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:46.357 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.357 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.357 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:46.358 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:46.163022+0000", 2024-09-18T10:15:46.358 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:46.358 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.358 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.358 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:46.358 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:46.163021+0000", 2024-09-18T10:15:46.358 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:46.358 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.358 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.358 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:46.358 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:46.163042+0000", 2024-09-18T10:15:46.359 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0839000000000002e-05 2024-09-18T10:15:46.359 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.359 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.359 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:46.359 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:46.163072+0000", 2024-09-18T10:15:46.359 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0202999999999999e-05 2024-09-18T10:15:46.359 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.359 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.359 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:46.359 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:46.163086+0000", 2024-09-18T10:15:46.359 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3889e-05 2024-09-18T10:15:46.360 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:46.360 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:46.360 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:46.360 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.360 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.360 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpRead(3.4s0 90/82 ECSubRead(tid=39, to_read={3:2650be24:::smithi18925896-45:2a=0,970752,0}, subchunks={3:2650be24:::smithi18925896-45:2a=[0,1]}, attrs_to_read=))", 2024-09-18T10:15:46.360 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:46.261472+0000", 2024-09-18T10:15:46.360 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.083999613000000001, 2024-09-18T10:15:46.360 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.084297887000000002, 2024-09-18T10:15:46.360 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:46.360 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-18T10:15:46.361 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:46.361 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.361 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:46.361 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:46.261472+0000", 2024-09-18T10:15:46.361 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:46.361 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.362 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.362 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:46.362 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:46.261472+0000", 2024-09-18T10:15:46.362 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:46.362 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.362 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.362 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:46.362 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:46.261472+0000", 2024-09-18T10:15:46.362 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:46.363 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.363 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.363 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:46.363 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:46.261472+0000", 2024-09-18T10:15:46.363 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:46.363 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.363 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.363 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:46.363 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:46.261556+0000", 2024-09-18T10:15:46.363 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.3537999999999999e-05 2024-09-18T10:15:46.364 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.364 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.364 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:46.364 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:46.261603+0000", 2024-09-18T10:15:46.364 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7175000000000001e-05 2024-09-18T10:15:46.364 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:46.364 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:46.364 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:46.364 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.364 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.364 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpReadReply(3.4s0 90/82 ECSubReadReply(tid=39, attrs_read=0))", 2024-09-18T10:15:46.365 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:46.262895+0000", 2024-09-18T10:15:46.365 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.082576687999999995, 2024-09-18T10:15:46.365 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.082942482999999997, 2024-09-18T10:15:46.365 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:46.365 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-18T10:15:46.365 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:46.365 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.365 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:46.365 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:46.262895+0000", 2024-09-18T10:15:46.365 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:46.365 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.365 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.366 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:46.366 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:46.262895+0000", 2024-09-18T10:15:46.366 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:46.366 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.366 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.366 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:46.366 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:46.262892+0000", 2024-09-18T10:15:46.366 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999971 2024-09-18T10:15:46.366 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.366 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.366 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:46.366 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:46.264340+0000", 2024-09-18T10:15:46.367 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001447282 2024-09-18T10:15:46.367 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.367 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.367 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:46.367 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:46.264388+0000", 2024-09-18T10:15:46.367 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8729000000000001e-05 2024-09-18T10:15:46.367 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.367 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.367 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:46.367 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:46.264407+0000", 2024-09-18T10:15:46.367 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9052999999999999e-05 2024-09-18T10:15:46.368 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:46.368 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:46.368 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:46.368 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.368 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.368 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpReadReply(3.4s0 90/82 ECSubReadReply(tid=39, attrs_read=0))", 2024-09-18T10:15:46.368 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:46.262495+0000", 2024-09-18T10:15:46.368 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.082977091000000003, 2024-09-18T10:15:46.369 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.083476833, 2024-09-18T10:15:46.369 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:46.369 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-18T10:15:46.369 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:46.369 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.369 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:46.369 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:46.262495+0000", 2024-09-18T10:15:46.369 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:46.369 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.369 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.369 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:46.370 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:46.262495+0000", 2024-09-18T10:15:46.370 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:46.370 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.370 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.370 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:46.370 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:46.262476+0000", 2024-09-18T10:15:46.370 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999814 2024-09-18T10:15:46.370 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.370 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.370 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:46.370 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:46.264809+0000", 2024-09-18T10:15:46.370 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.002333229 2024-09-18T10:15:46.371 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.371 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.371 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:46.371 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:46.264834+0000", 2024-09-18T10:15:46.371 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.4737999999999999e-05 2024-09-18T10:15:46.371 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.371 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.371 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:46.371 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:46.264843+0000", 2024-09-18T10:15:46.371 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.3819999999999993e-06 2024-09-18T10:15:46.371 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:46.371 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:46.372 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:46.372 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.372 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.372 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpReadReply(3.4s0 90/82 ECSubReadReply(tid=39, attrs_read=0))", 2024-09-18T10:15:46.372 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:46.264478+0000", 2024-09-18T10:15:46.372 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.080993642000000005, 2024-09-18T10:15:46.372 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.081563086000000007, 2024-09-18T10:15:46.372 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:46.372 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-18T10:15:46.372 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:46.372 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.373 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:46.373 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:46.264478+0000", 2024-09-18T10:15:46.373 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:46.373 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.373 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.373 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:46.373 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:46.264478+0000", 2024-09-18T10:15:46.373 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:46.373 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.373 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.373 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:46.373 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:46.264476+0000", 2024-09-18T10:15:46.374 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:46.374 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.374 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.374 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:46.374 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:46.265041+0000", 2024-09-18T10:15:46.374 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00056412499999999996 2024-09-18T10:15:46.374 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.374 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.374 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:46.374 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:46.265080+0000", 2024-09-18T10:15:46.374 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9311999999999999e-05 2024-09-18T10:15:46.375 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.375 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.375 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:46.375 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:46.265100+0000", 2024-09-18T10:15:46.375 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0394000000000002e-05 2024-09-18T10:15:46.375 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:46.375 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:46.375 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:46.375 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.375 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.375 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:886 3.4s0 3:2650be24:::smithi18925896-45:28 [read 0~3883008,checksum in=4b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T10:15:46.376 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:45.695753+0000", 2024-09-18T10:15:46.376 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.64971870099999995, 2024-09-18T10:15:46.376 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.65037088700000001, 2024-09-18T10:15:46.376 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:46.376 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:46.376 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:46.376 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:46.376 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:46.376 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 886 2024-09-18T10:15:46.376 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.376 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:46.377 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.377 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:46.377 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:45.695753+0000", 2024-09-18T10:15:46.377 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:46.377 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.377 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.377 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:46.377 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:45.695753+0000", 2024-09-18T10:15:46.377 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:46.377 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.377 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.378 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:46.378 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:45.695752+0000", 2024-09-18T10:15:46.378 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.999999 2024-09-18T10:15:46.378 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.378 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.378 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:46.378 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:45.695758+0000", 2024-09-18T10:15:46.378 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.3229999999999997e-06 2024-09-18T10:15:46.378 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.378 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.378 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:46.379 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:45.695773+0000", 2024-09-18T10:15:46.379 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.524e-05 2024-09-18T10:15:46.379 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.379 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.379 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:46.379 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:45.795871+0000", 2024-09-18T10:15:46.379 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100098198 2024-09-18T10:15:46.379 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.379 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.379 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:46.379 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:46.260893+0000", 2024-09-18T10:15:46.380 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.465021619 2024-09-18T10:15:46.380 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.380 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.380 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:46.380 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:46.261099+0000", 2024-09-18T10:15:46.380 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00020610999999999999 2024-09-18T10:15:46.380 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:46.380 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:46.380 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:46.380 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:46.380 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-18T10:15:46.381 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 8 2024-09-18T10:15:46.381 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-18T10:15:46.381 INFO:tasks.rados.rados.0.smithi189.stdout:351: expect (ObjNum 103 snap 21 seq_num 103) 2024-09-18T10:15:46.382 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_blocked_ops 2024-09-18T10:15:46.394 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-09-18T10:15:46.390+0000 7fdfa8698700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22087) UID: 0 2024-09-18T10:15:46.487 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-18T10:15:46.487 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-18T10:15:46.487 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-18T10:15:46.488 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-18T10:15:46.488 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-18T10:15:46.495 INFO:tasks.ceph.osd.4.smithi029.stderr:2024-09-18T10:15:46.489+0000 7f550cf6e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22189) UID: 0 2024-09-18T10:15:46.497 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_historic_ops 2024-09-18T10:15:46.534 INFO:tasks.rados.rados.0.smithi189.stdout:344: finishing write tid 4 to smithi18925896-38 2024-09-18T10:15:46.534 INFO:tasks.rados.rados.0.smithi189.stdout:344: finishing write tid 5 to smithi18925896-38 2024-09-18T10:15:46.590 INFO:tasks.rados.rados.0.smithi189.stdout:349: finishing copy_from racing read to smithi18925896-41 2024-09-18T10:15:46.595 INFO:tasks.ceph.osd.10.smithi154.stderr:2024-09-18T10:15:46.594+0000 7fb73221a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 22091) UID: 0 2024-09-18T10:15:46.613 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-18T10:15:46.613 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-18T10:15:46.614 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-18T10:15:46.614 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-18T10:15:46.614 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.614 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:744 3.4s0 3:2619bf65:::smithi18925896-16:head [append 0~491520] snapc 2d=[29,28,20,1f,1e,1a,11] ondisk+write+known_if_redirected+supports_pool_eio e84)", 2024-09-18T10:15:46.614 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:39.580920+0000", 2024-09-18T10:15:46.614 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 7.0279618189999997, 2024-09-18T10:15:46.614 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.70503880200000002, 2024-09-18T10:15:46.614 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:46.614 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:46.614 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:46.614 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:46.614 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:46.614 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 744 2024-09-18T10:15:46.615 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.615 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:46.615 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.615 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:46.615 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:39.580920+0000", 2024-09-18T10:15:46.615 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:46.615 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.615 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.615 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:46.615 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:39.580920+0000", 2024-09-18T10:15:46.615 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:46.615 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.615 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.616 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:46.616 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:39.580918+0000", 2024-09-18T10:15:46.616 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:46.616 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.616 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.616 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:46.616 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:39.581759+0000", 2024-09-18T10:15:46.616 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00084170600000000001 2024-09-18T10:15:46.616 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.616 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.616 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:46.616 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:39.581786+0000", 2024-09-18T10:15:46.617 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6893000000000002e-05 2024-09-18T10:15:46.617 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.617 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.617 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:46.617 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:39.581797+0000", 2024-09-18T10:15:46.617 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0718e-05 2024-09-18T10:15:46.617 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.617 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.617 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:46.617 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:39.595632+0000", 2024-09-18T10:15:46.618 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.013835136 2024-09-18T10:15:46.618 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.618 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.618 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for missing object", 2024-09-18T10:15:46.618 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:39.595946+0000", 2024-09-18T10:15:46.618 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000313708 2024-09-18T10:15:46.618 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.618 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.618 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:46.618 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:39.874314+0000", 2024-09-18T10:15:46.618 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.278368114 2024-09-18T10:15:46.618 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.618 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.619 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for rw locks", 2024-09-18T10:15:46.619 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:39.875170+0000", 2024-09-18T10:15:46.619 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00085585800000000003 2024-09-18T10:15:46.619 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.619 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.619 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:46.619 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:40.073179+0000", 2024-09-18T10:15:46.619 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.19800965100000001 2024-09-18T10:15:46.619 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.619 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.619 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:46.619 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:40.073524+0000", 2024-09-18T10:15:46.620 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000344202 2024-09-18T10:15:46.620 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.620 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.620 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:46.620 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:40.076496+0000", 2024-09-18T10:15:46.620 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.002972429 2024-09-18T10:15:46.620 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.620 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.620 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:46.620 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:40.083201+0000", 2024-09-18T10:15:46.620 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0067049780000000003 2024-09-18T10:15:46.620 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.621 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.621 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-18T10:15:46.621 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:40.284627+0000", 2024-09-18T10:15:46.621 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.20142610599999999 2024-09-18T10:15:46.621 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.621 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.621 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:46.621 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:40.285958+0000", 2024-09-18T10:15:46.621 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0013313579999999999 2024-09-18T10:15:46.621 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:46.621 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:46.621 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:46.621 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.622 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.622 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:745 3.4s0 3:2619bf65:::smithi18925896-16:head [append 0~802816] snapc 2d=[29,28,20,1f,1e,1a,11] ondisk+write+known_if_redirected+supports_pool_eio e84)", 2024-09-18T10:15:46.622 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:39.585725+0000", 2024-09-18T10:15:46.622 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 7.0231559380000004, 2024-09-18T10:15:46.622 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4205835529999999, 2024-09-18T10:15:46.622 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:46.622 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:46.622 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:46.622 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:46.622 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:46.622 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 745 2024-09-18T10:15:46.622 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.623 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:46.623 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.623 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:46.623 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:39.585725+0000", 2024-09-18T10:15:46.623 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:46.623 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.623 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.623 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:46.623 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:39.585725+0000", 2024-09-18T10:15:46.623 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:46.624 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.624 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.624 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:46.624 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:39.585720+0000", 2024-09-18T10:15:46.624 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999948 2024-09-18T10:15:46.624 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.624 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.624 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:46.624 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:39.587168+0000", 2024-09-18T10:15:46.624 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001447982 2024-09-18T10:15:46.624 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.624 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.625 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:46.625 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:39.587194+0000", 2024-09-18T10:15:46.625 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6406000000000001e-05 2024-09-18T10:15:46.625 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.625 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.625 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:46.625 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:39.587211+0000", 2024-09-18T10:15:46.625 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.6310000000000001e-05 2024-09-18T10:15:46.625 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.625 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.625 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:46.625 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:39.597345+0000", 2024-09-18T10:15:46.625 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.010133837999999999 2024-09-18T10:15:46.625 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.626 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.626 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for missing object", 2024-09-18T10:15:46.626 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:39.597482+0000", 2024-09-18T10:15:46.626 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000137665 2024-09-18T10:15:46.626 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.626 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.626 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:46.626 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:40.692366+0000", 2024-09-18T10:15:46.626 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.094883829 2024-09-18T10:15:46.626 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.626 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.626 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:46.626 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:40.692834+0000", 2024-09-18T10:15:46.626 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00046827300000000001 2024-09-18T10:15:46.627 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.627 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.627 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:46.627 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:40.696372+0000", 2024-09-18T10:15:46.627 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0035373129999999998 2024-09-18T10:15:46.627 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.627 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.627 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:46.627 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:40.806691+0000", 2024-09-18T10:15:46.627 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.110319442 2024-09-18T10:15:46.627 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.627 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.627 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-18T10:15:46.627 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:41.005002+0000", 2024-09-18T10:15:46.628 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.198310717 2024-09-18T10:15:46.628 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.628 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.628 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:46.628 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:41.006309+0000", 2024-09-18T10:15:46.628 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001307168 2024-09-18T10:15:46.628 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:46.628 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:46.628 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:46.628 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.628 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.628 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:746 3.4s0 3:2619bf65:::smithi18925896-16:head [append 0~720896] snapc 2d=[29,28,20,1f,1e,1a,11] ondisk+write+known_if_redirected+supports_pool_eio e84)", 2024-09-18T10:15:46.628 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:39.590157+0000", 2024-09-18T10:15:46.629 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 7.018724261, 2024-09-18T10:15:46.629 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5189854789999999, 2024-09-18T10:15:46.629 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:46.629 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:46.629 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:46.629 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:46.629 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:46.629 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 746 2024-09-18T10:15:46.629 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.629 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:46.629 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.629 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:46.629 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:39.590157+0000", 2024-09-18T10:15:46.630 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:46.630 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.630 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.630 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:46.630 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:39.590157+0000", 2024-09-18T10:15:46.630 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:46.630 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.630 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.630 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:46.630 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:39.590155+0000", 2024-09-18T10:15:46.630 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999976 2024-09-18T10:15:46.630 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.630 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.631 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:46.631 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:39.591256+0000", 2024-09-18T10:15:46.631 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001101084 2024-09-18T10:15:46.631 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.631 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.631 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:46.631 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:39.591289+0000", 2024-09-18T10:15:46.631 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.2879000000000001e-05 2024-09-18T10:15:46.631 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.631 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.631 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:46.631 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:39.591316+0000", 2024-09-18T10:15:46.631 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6849999999999999e-05 2024-09-18T10:15:46.632 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.632 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.632 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:46.632 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:39.769524+0000", 2024-09-18T10:15:46.632 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.17820868400000001 2024-09-18T10:15:46.632 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.632 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.632 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for missing object", 2024-09-18T10:15:46.632 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:39.769787+0000", 2024-09-18T10:15:46.632 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00026244499999999999 2024-09-18T10:15:46.632 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.632 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.632 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:46.632 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:40.699212+0000", 2024-09-18T10:15:46.633 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.92942537700000005 2024-09-18T10:15:46.633 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.633 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.633 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:46.633 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:40.699674+0000", 2024-09-18T10:15:46.633 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00046235699999999999 2024-09-18T10:15:46.633 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.633 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.633 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:46.633 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:40.702073+0000", 2024-09-18T10:15:46.633 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0023981250000000001 2024-09-18T10:15:46.633 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.634 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.634 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:46.634 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:40.806695+0000", 2024-09-18T10:15:46.634 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10462266200000001 2024-09-18T10:15:46.634 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.634 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.634 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-18T10:15:46.634 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:41.108058+0000", 2024-09-18T10:15:46.634 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.30136298 2024-09-18T10:15:46.634 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.634 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.634 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:46.634 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:41.109143+0000", 2024-09-18T10:15:46.635 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001084411 2024-09-18T10:15:46.635 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:46.635 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:46.635 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:46.635 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.635 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.635 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:747 3.4s0 3:2619bf65:::smithi18925896-16:head [append 0~32768] snapc 2d=[29,28,20,1f,1e,1a,11] ondisk+write+known_if_redirected+supports_pool_eio e84)", 2024-09-18T10:15:46.635 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:39.591337+0000", 2024-09-18T10:15:46.635 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 7.0175443890000002, 2024-09-18T10:15:46.635 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.6215779050000001, 2024-09-18T10:15:46.635 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:46.635 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:46.636 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:46.636 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:46.636 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:46.636 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 747 2024-09-18T10:15:46.636 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.636 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:46.636 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.636 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:46.636 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:39.591337+0000", 2024-09-18T10:15:46.636 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:46.637 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.637 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.637 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:46.637 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:39.591337+0000", 2024-09-18T10:15:46.637 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:46.637 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.637 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.637 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:46.637 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:39.591335+0000", 2024-09-18T10:15:46.637 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:46.637 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.637 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.638 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:46.638 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:39.591410+0000", 2024-09-18T10:15:46.638 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4201000000000003e-05 2024-09-18T10:15:46.638 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.638 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.638 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:46.638 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:39.591427+0000", 2024-09-18T10:15:46.638 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.6937000000000001e-05 2024-09-18T10:15:46.638 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.638 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.638 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:46.638 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:39.591452+0000", 2024-09-18T10:15:46.639 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.5429999999999999e-05 2024-09-18T10:15:46.639 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.639 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.639 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:46.639 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:39.769866+0000", 2024-09-18T10:15:46.639 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.178413928 2024-09-18T10:15:46.639 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.639 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.639 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for missing object", 2024-09-18T10:15:46.639 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:39.770123+0000", 2024-09-18T10:15:46.639 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00025703100000000001 2024-09-18T10:15:46.639 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.640 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.640 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:46.640 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:40.702804+0000", 2024-09-18T10:15:46.640 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.93268105700000004 2024-09-18T10:15:46.640 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.640 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.640 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:46.640 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:40.702977+0000", 2024-09-18T10:15:46.640 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00017276499999999999 2024-09-18T10:15:46.640 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.640 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.640 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:46.641 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:40.703855+0000", 2024-09-18T10:15:46.641 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00087768700000000004 2024-09-18T10:15:46.641 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.641 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.641 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:46.641 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:40.806698+0000", 2024-09-18T10:15:46.641 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10284320499999999 2024-09-18T10:15:46.641 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.641 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.641 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-18T10:15:46.641 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:41.210364+0000", 2024-09-18T10:15:46.641 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.403666421 2024-09-18T10:15:46.642 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.642 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.642 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:46.642 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:41.212915+0000", 2024-09-18T10:15:46.642 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0025508079999999999 2024-09-18T10:15:46.642 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:46.642 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:46.642 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:46.642 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.642 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.642 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:748 3.4s0 3:2619bf65:::smithi18925896-16:head [setxattr _header (57) in=7b] snapc 2d=[29,28,20,1f,1e,1a,11] ondisk+write+known_if_redirected+supports_pool_eio e84)", 2024-09-18T10:15:46.642 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:39.591464+0000", 2024-09-18T10:15:46.642 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 7.0174175989999998, 2024-09-18T10:15:46.642 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.62443162, 2024-09-18T10:15:46.643 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:46.643 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:46.643 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:46.643 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:46.643 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:46.643 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 748 2024-09-18T10:15:46.643 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.643 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:46.643 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.643 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:46.643 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:39.591464+0000", 2024-09-18T10:15:46.643 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:46.643 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.643 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.643 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:46.644 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:39.591464+0000", 2024-09-18T10:15:46.644 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:46.644 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.644 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.644 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:46.644 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:39.591463+0000", 2024-09-18T10:15:46.644 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.999999 2024-09-18T10:15:46.644 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.644 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.644 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:46.644 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:39.591469+0000", 2024-09-18T10:15:46.644 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.652e-06 2024-09-18T10:15:46.644 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.644 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.645 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:46.645 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:39.591488+0000", 2024-09-18T10:15:46.645 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8113000000000001e-05 2024-09-18T10:15:46.645 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.645 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.645 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:46.645 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:39.591503+0000", 2024-09-18T10:15:46.645 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5461e-05 2024-09-18T10:15:46.645 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.645 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.645 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:46.645 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:39.870372+0000", 2024-09-18T10:15:46.645 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.27886917100000003 2024-09-18T10:15:46.646 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.646 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.646 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for missing object", 2024-09-18T10:15:46.646 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:39.870667+0000", 2024-09-18T10:15:46.646 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00029504999999999999 2024-09-18T10:15:46.646 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.646 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.646 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:46.646 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:40.809100+0000", 2024-09-18T10:15:46.646 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.93843269900000004 2024-09-18T10:15:46.646 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.646 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.646 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:46.646 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:40.809727+0000", 2024-09-18T10:15:46.647 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00062731600000000005 2024-09-18T10:15:46.647 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.647 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.647 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:46.647 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:40.812639+0000", 2024-09-18T10:15:46.647 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.002911711 2024-09-18T10:15:46.647 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.647 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.647 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:46.647 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:40.815754+0000", 2024-09-18T10:15:46.647 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0031149989999999998 2024-09-18T10:15:46.647 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.647 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.647 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-18T10:15:46.647 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:41.213833+0000", 2024-09-18T10:15:46.648 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.39807867800000002 2024-09-18T10:15:46.648 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.648 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.648 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:46.648 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:41.215895+0000", 2024-09-18T10:15:46.648 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0020627950000000001 2024-09-18T10:15:46.648 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:46.648 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:46.648 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:46.648 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.648 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.648 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:749 3.4s0 3:2619bf65:::smithi18925896-16:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e84)", 2024-09-18T10:15:46.648 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:39.591512+0000", 2024-09-18T10:15:46.649 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 7.017368973, 2024-09-18T10:15:46.649 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1456294050000002, 2024-09-18T10:15:46.649 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:46.649 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:46.649 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:46.649 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:46.649 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:46.649 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 749 2024-09-18T10:15:46.649 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.649 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:46.649 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.649 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:46.650 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:39.591512+0000", 2024-09-18T10:15:46.650 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:46.650 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.650 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.650 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:46.650 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:39.591512+0000", 2024-09-18T10:15:46.650 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:46.650 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.650 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.650 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:46.650 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:39.591512+0000", 2024-09-18T10:15:46.651 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.999999 2024-09-18T10:15:46.651 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.651 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.651 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:46.651 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:39.591526+0000", 2024-09-18T10:15:46.651 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3947e-05 2024-09-18T10:15:46.651 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.651 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.651 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:46.651 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:39.591541+0000", 2024-09-18T10:15:46.651 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5625e-05 2024-09-18T10:15:46.651 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.652 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.652 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:46.652 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:39.591552+0000", 2024-09-18T10:15:46.652 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0705e-05 2024-09-18T10:15:46.652 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.652 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.652 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:46.652 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:39.870881+0000", 2024-09-18T10:15:46.652 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.27932869399999999 2024-09-18T10:15:46.652 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.652 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.653 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for missing object", 2024-09-18T10:15:46.653 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:39.871201+0000", 2024-09-18T10:15:46.653 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00032053300000000002 2024-09-18T10:15:46.653 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.653 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.653 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:46.653 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:40.813899+0000", 2024-09-18T10:15:46.653 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.94269780800000003 2024-09-18T10:15:46.653 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.653 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.653 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for rw locks", 2024-09-18T10:15:46.653 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:40.814222+0000", 2024-09-18T10:15:46.654 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00032354999999999998 2024-09-18T10:15:46.654 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.654 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.654 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:46.654 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:41.217063+0000", 2024-09-18T10:15:46.654 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.402840594 2024-09-18T10:15:46.654 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.654 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.654 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:46.654 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:41.217965+0000", 2024-09-18T10:15:46.654 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00090176099999999995 2024-09-18T10:15:46.654 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.654 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.654 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:46.655 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:41.737142+0000", 2024-09-18T10:15:46.655 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.51917703800000004 2024-09-18T10:15:46.655 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:46.655 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:46.655 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:46.655 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.655 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.655 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWrite(3.cs5 84/83 ECSubWrite(tid=137, reqid=client.4632.0:752, at_version=84'113, trim_to=0'0, roll_forward_to=0'0))", 2024-09-18T10:15:46.655 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:39.992883+0000", 2024-09-18T10:15:46.655 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 6.6159987180000002, 2024-09-18T10:15:46.655 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.29445843399999999, 2024-09-18T10:15:46.656 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:46.656 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:46.656 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:46.656 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.656 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:46.656 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:39.992883+0000", 2024-09-18T10:15:46.656 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:46.656 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.656 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.656 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:46.656 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:39.992883+0000", 2024-09-18T10:15:46.656 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:46.656 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.657 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.657 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:46.657 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:39.992876+0000", 2024-09-18T10:15:46.657 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999928 2024-09-18T10:15:46.657 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.657 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.657 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:46.657 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:39.992990+0000", 2024-09-18T10:15:46.657 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00011399400000000001 2024-09-18T10:15:46.657 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.657 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.657 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:46.657 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:39.993021+0000", 2024-09-18T10:15:46.658 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0998999999999999e-05 2024-09-18T10:15:46.658 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.658 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.658 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:46.658 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:39.993033+0000", 2024-09-18T10:15:46.658 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2736e-05 2024-09-18T10:15:46.658 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.658 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.658 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:46.658 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:40.084593+0000", 2024-09-18T10:15:46.658 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.091559711000000002 2024-09-18T10:15:46.658 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.658 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.659 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:46.659 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:40.084625+0000", 2024-09-18T10:15:46.659 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1578999999999997e-05 2024-09-18T10:15:46.659 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.659 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.659 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:46.659 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:40.287311+0000", 2024-09-18T10:15:46.659 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.20268594300000001 2024-09-18T10:15:46.659 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.659 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.659 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:46.659 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:40.287341+0000", 2024-09-18T10:15:46.659 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0614999999999997e-05 2024-09-18T10:15:46.660 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:46.660 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:46.660 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:46.660 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.660 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.660 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWrite(3.cs5 84/83 ECSubWrite(tid=138, reqid=client.4632.0:753, at_version=84'114, trim_to=0'0, roll_forward_to=0'0))", 2024-09-18T10:15:46.660 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:39.996915+0000", 2024-09-18T10:15:46.660 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 6.6119667159999995, 2024-09-18T10:15:46.660 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.29244602200000003, 2024-09-18T10:15:46.660 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:46.660 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:46.660 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:46.660 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.661 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:46.661 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:39.996915+0000", 2024-09-18T10:15:46.661 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:46.661 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.661 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.661 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:46.661 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:39.996915+0000", 2024-09-18T10:15:46.661 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:46.661 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.661 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.661 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:46.661 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:39.996907+0000", 2024-09-18T10:15:46.661 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999924 2024-09-18T10:15:46.662 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.662 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.662 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:46.662 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:39.997047+0000", 2024-09-18T10:15:46.662 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000139431 2024-09-18T10:15:46.662 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.662 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.662 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:46.662 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:39.997078+0000", 2024-09-18T10:15:46.662 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1109000000000003e-05 2024-09-18T10:15:46.662 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.662 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.662 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:46.662 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:39.997091+0000", 2024-09-18T10:15:46.663 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2903e-05 2024-09-18T10:15:46.663 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.663 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.663 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:46.663 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:40.186118+0000", 2024-09-18T10:15:46.663 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.18902743799999999 2024-09-18T10:15:46.663 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.663 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.663 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:46.663 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:40.186136+0000", 2024-09-18T10:15:46.663 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7909000000000001e-05 2024-09-18T10:15:46.663 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.664 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.664 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:46.664 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:40.289324+0000", 2024-09-18T10:15:46.664 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10318746600000001 2024-09-18T10:15:46.664 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.664 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.664 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:46.664 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:40.289361+0000", 2024-09-18T10:15:46.664 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7206000000000003e-05 2024-09-18T10:15:46.664 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:46.665 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:46.665 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:46.665 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.665 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.665 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWrite(3.cs5 84/83 ECSubWrite(tid=140, reqid=client.4632.0:755, at_version=84'116, trim_to=0'0, roll_forward_to=0'0))", 2024-09-18T10:15:46.665 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:40.204993+0000", 2024-09-18T10:15:46.665 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 6.4038883999999996, 2024-09-18T10:15:46.665 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.499374031, 2024-09-18T10:15:46.665 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:46.665 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:46.665 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:46.666 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.666 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:46.666 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:40.204993+0000", 2024-09-18T10:15:46.666 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:46.666 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.666 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.666 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:46.666 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:40.204993+0000", 2024-09-18T10:15:46.666 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:46.666 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.666 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.666 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:46.667 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:40.204991+0000", 2024-09-18T10:15:46.667 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:46.667 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.667 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.667 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:46.667 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:40.205039+0000", 2024-09-18T10:15:46.667 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7552000000000001e-05 2024-09-18T10:15:46.667 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.667 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.667 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:46.667 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:40.205074+0000", 2024-09-18T10:15:46.667 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.5102999999999999e-05 2024-09-18T10:15:46.668 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.668 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.668 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:46.668 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:40.305193+0000", 2024-09-18T10:15:46.668 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10011943600000001 2024-09-18T10:15:46.668 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.668 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.668 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:46.668 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:40.690990+0000", 2024-09-18T10:15:46.668 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.38579699699999997 2024-09-18T10:15:46.668 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.668 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.669 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:46.669 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:40.691020+0000", 2024-09-18T10:15:46.669 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9173e-05 2024-09-18T10:15:46.669 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.669 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.669 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:46.669 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:40.704331+0000", 2024-09-18T10:15:46.669 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.013311388 2024-09-18T10:15:46.669 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.669 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.669 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:46.669 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:40.704367+0000", 2024-09-18T10:15:46.670 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.6019000000000002e-05 2024-09-18T10:15:46.670 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:46.670 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:46.670 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:46.670 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.670 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.670 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDPGPush(3.4s0 84/82 [PushOp(3:2650be24:::smithi18925896-45:21, version: 75'62, data_included: [0~262144], data_size: 262144, omap_header_size: 0, omap_entries_size: 0, attrset_size: 3, recovery_info: ObjectRecoveryInfo(3:2650be24:::smithi18925896-45:21@75'62, size: 2146304, copy_subset: [], clone_subset: {}, snapset: 2a=[]:{13=[11],21=[20,1f,1e,1a],2a=[29,28]}, object_exist: 1), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:1048576, data_complete:false, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:true, error:false))])", 2024-09-18T10:15:46.670 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:40.288831+0000", 2024-09-18T10:15:46.670 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 6.3200500689999997, 2024-09-18T10:15:46.670 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.40329756, 2024-09-18T10:15:46.670 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:46.670 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:46.670 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:46.671 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.671 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:46.671 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:40.288831+0000", 2024-09-18T10:15:46.671 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:46.671 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.671 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.671 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:46.671 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:40.288831+0000", 2024-09-18T10:15:46.671 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:46.671 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.671 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.671 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:46.671 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:40.288831+0000", 2024-09-18T10:15:46.672 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:46.672 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.672 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.672 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:46.672 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:40.288831+0000", 2024-09-18T10:15:46.672 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:46.672 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.672 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.672 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:46.672 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:40.288900+0000", 2024-09-18T10:15:46.672 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8571000000000005e-05 2024-09-18T10:15:46.672 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.672 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.673 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:46.673 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:40.288919+0000", 2024-09-18T10:15:46.673 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8896999999999999e-05 2024-09-18T10:15:46.673 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.673 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.673 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:46.673 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:40.690942+0000", 2024-09-18T10:15:46.673 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.402023293 2024-09-18T10:15:46.673 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.673 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.673 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:46.674 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:40.692129+0000", 2024-09-18T10:15:46.674 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0011867990000000001 2024-09-18T10:15:46.674 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:46.674 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:46.674 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:46.674 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.674 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.674 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWrite(3.cs5 85/83 ECSubWrite(tid=142, reqid=client.4632.0:755, at_version=0'0, trim_to=0'0, roll_forward_to=84'116))", 2024-09-18T10:15:46.674 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:40.903236+0000", 2024-09-18T10:15:46.674 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 5.7056451299999997, 2024-09-18T10:15:46.674 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.31677315499999997, 2024-09-18T10:15:46.674 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:46.674 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:46.675 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:46.675 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.675 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:46.675 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:40.903236+0000", 2024-09-18T10:15:46.675 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:46.675 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.675 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.675 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:46.675 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:40.903236+0000", 2024-09-18T10:15:46.675 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:46.675 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.675 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.675 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:46.676 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:40.903235+0000", 2024-09-18T10:15:46.676 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.999999 2024-09-18T10:15:46.676 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.676 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.676 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:46.676 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:40.903274+0000", 2024-09-18T10:15:46.676 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9252999999999998e-05 2024-09-18T10:15:46.676 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.676 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.676 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:46.676 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:40.903307+0000", 2024-09-18T10:15:46.676 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.3065000000000003e-05 2024-09-18T10:15:46.676 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.677 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.677 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:46.677 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:40.903322+0000", 2024-09-18T10:15:46.677 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.473e-05 2024-09-18T10:15:46.677 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.677 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.677 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:46.677 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:41.213131+0000", 2024-09-18T10:15:46.677 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.30980836299999998 2024-09-18T10:15:46.677 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.677 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.677 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:46.678 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:41.213138+0000", 2024-09-18T10:15:46.678 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4989999999999997e-06 2024-09-18T10:15:46.678 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.678 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.678 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:46.678 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:41.219979+0000", 2024-09-18T10:15:46.678 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0068409509999999996 2024-09-18T10:15:46.678 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.678 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.678 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:46.678 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:41.220009+0000", 2024-09-18T10:15:46.679 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0431000000000001e-05 2024-09-18T10:15:46.679 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:46.679 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:46.679 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:46.679 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.679 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.679 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpReadReply(3.4s0 85/82 ECSubReadReply(tid=16, attrs_read=0))", 2024-09-18T10:15:46.679 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:40.903248+0000", 2024-09-18T10:15:46.679 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 5.7056330739999996, 2024-09-18T10:15:46.679 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.31775736799999998, 2024-09-18T10:15:46.679 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:46.679 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:46.680 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:46.680 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.680 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:46.680 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:40.903248+0000", 2024-09-18T10:15:46.680 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:46.680 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.680 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.680 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:46.680 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:40.903248+0000", 2024-09-18T10:15:46.680 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:46.680 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.680 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.681 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:46.681 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:40.903247+0000", 2024-09-18T10:15:46.681 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:46.681 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.681 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.681 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:46.681 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:40.903298+0000", 2024-09-18T10:15:46.681 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1022e-05 2024-09-18T10:15:46.681 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.681 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.681 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:46.682 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:40.903319+0000", 2024-09-18T10:15:46.682 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0857e-05 2024-09-18T10:15:46.682 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.682 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.682 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:46.682 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:41.003424+0000", 2024-09-18T10:15:46.682 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100105339 2024-09-18T10:15:46.682 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.682 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.683 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:46.683 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:41.220496+0000", 2024-09-18T10:15:46.683 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.21707167799999999 2024-09-18T10:15:46.683 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.683 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.683 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:46.683 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:41.221006+0000", 2024-09-18T10:15:46.683 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00051009199999999997 2024-09-18T10:15:46.683 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:46.683 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:46.683 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:46.683 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.683 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.683 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDPGPush(3.4s0 85/82 [PushOp(3:2650be24:::smithi18925896-45:13, version: 77'71, data_included: [0~188416], data_size: 188416, omap_header_size: 0, omap_entries_size: 0, attrset_size: 3, recovery_info: ObjectRecoveryInfo(3:2650be24:::smithi18925896-45:13@77'71, size: 753664, copy_subset: [], clone_subset: {}, snapset: 2a=[]:{13=[11],21=[20,1f,1e,1a],2a=[29,28]}, object_exist: 1), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:753664, data_complete:true, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:true, error:false))])", 2024-09-18T10:15:46.683 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:41.227308+0000", 2024-09-18T10:15:46.684 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 5.3815735240000002, 2024-09-18T10:15:46.684 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.41242936299999999, 2024-09-18T10:15:46.684 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:46.684 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:46.684 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:46.684 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.684 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:46.684 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:41.227308+0000", 2024-09-18T10:15:46.684 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:46.684 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.684 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.684 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:46.684 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:41.227308+0000", 2024-09-18T10:15:46.684 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:46.684 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.685 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.685 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:46.685 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:41.227308+0000", 2024-09-18T10:15:46.685 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:46.685 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.685 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.685 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:46.685 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:41.227308+0000", 2024-09-18T10:15:46.685 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:46.685 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.685 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.685 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:46.685 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:41.227336+0000", 2024-09-18T10:15:46.686 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.8011999999999999e-05 2024-09-18T10:15:46.686 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.686 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.686 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:46.686 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:41.227350+0000", 2024-09-18T10:15:46.686 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4124000000000001e-05 2024-09-18T10:15:46.686 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.686 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.686 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:46.686 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:41.637342+0000", 2024-09-18T10:15:46.686 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.40999157200000003 2024-09-18T10:15:46.686 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.686 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.687 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:46.687 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:41.639737+0000", 2024-09-18T10:15:46.687 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.002395655 2024-09-18T10:15:46.687 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:46.687 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:46.687 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:46.687 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.687 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.687 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpReadReply(3.4s0 85/82 ECSubReadReply(tid=18, attrs_read=0))", 2024-09-18T10:15:46.687 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:41.319730+0000", 2024-09-18T10:15:46.687 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 5.2891510019999997, 2024-09-18T10:15:46.687 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.320311293, 2024-09-18T10:15:46.687 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:46.687 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:46.687 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:46.688 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.688 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:46.688 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:41.319730+0000", 2024-09-18T10:15:46.688 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:46.688 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.688 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.688 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:46.688 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:41.319730+0000", 2024-09-18T10:15:46.689 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:46.689 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.689 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.689 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:46.689 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:41.319729+0000", 2024-09-18T10:15:46.689 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:46.689 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.689 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.689 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:46.689 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:41.319747+0000", 2024-09-18T10:15:46.689 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7665000000000001e-05 2024-09-18T10:15:46.689 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.690 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.690 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:46.690 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:41.319773+0000", 2024-09-18T10:15:46.690 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6404e-05 2024-09-18T10:15:46.690 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.690 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.690 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:46.690 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:41.319787+0000", 2024-09-18T10:15:46.690 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4095e-05 2024-09-18T10:15:46.690 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.690 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.690 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:46.691 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:41.639831+0000", 2024-09-18T10:15:46.691 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.32004390999999999 2024-09-18T10:15:46.691 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.691 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.691 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:46.691 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:41.640042+0000", 2024-09-18T10:15:46.691 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00021074200000000001 2024-09-18T10:15:46.691 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:46.691 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:46.691 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:46.691 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.691 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.691 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWrite(3.cs5 85/83 ECSubWrite(tid=145, reqid=client.4632.0:770, at_version=85'118, trim_to=0'0, roll_forward_to=84'116))", 2024-09-18T10:15:46.692 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:41.320620+0000", 2024-09-18T10:15:46.692 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 5.2882610259999998, 2024-09-18T10:15:46.692 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.31910936200000001, 2024-09-18T10:15:46.692 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:46.692 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:46.692 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:46.692 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.692 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:46.692 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:41.320620+0000", 2024-09-18T10:15:46.692 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:46.692 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.692 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.692 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:46.693 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:41.320620+0000", 2024-09-18T10:15:46.693 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:46.693 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.693 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.693 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:46.693 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:41.320619+0000", 2024-09-18T10:15:46.693 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:46.693 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.693 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.693 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:46.693 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:41.320822+0000", 2024-09-18T10:15:46.693 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00020313500000000001 2024-09-18T10:15:46.693 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.693 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.694 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:46.694 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:41.320849+0000", 2024-09-18T10:15:46.694 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.7132e-05 2024-09-18T10:15:46.694 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.694 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.694 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:46.694 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:41.320859+0000", 2024-09-18T10:15:46.694 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0103e-05 2024-09-18T10:15:46.694 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.694 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.694 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:46.694 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:41.632525+0000", 2024-09-18T10:15:46.694 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.31166550700000001 2024-09-18T10:15:46.694 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.694 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.695 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:46.695 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:41.632543+0000", 2024-09-18T10:15:46.695 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8182e-05 2024-09-18T10:15:46.695 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.695 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.695 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:46.695 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:41.639523+0000", 2024-09-18T10:15:46.695 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0069796670000000002 2024-09-18T10:15:46.695 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.695 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.695 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:46.695 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:41.639730+0000", 2024-09-18T10:15:46.695 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000207111 2024-09-18T10:15:46.695 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:46.695 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:46.696 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:46.696 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.696 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.696 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 85/82 ECSubWriteReply(tid=20, last_complete=85'85, committed=1, applied=1))", 2024-09-18T10:15:46.696 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:41.332386+0000", 2024-09-18T10:15:46.696 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 5.2764951399999998, 2024-09-18T10:15:46.696 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.29990105, 2024-09-18T10:15:46.696 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:46.696 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:46.696 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:46.696 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.696 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:46.696 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:41.332386+0000", 2024-09-18T10:15:46.697 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:46.697 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.697 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.697 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:46.697 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:41.332386+0000", 2024-09-18T10:15:46.697 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:46.697 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.697 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.697 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:46.697 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:41.332385+0000", 2024-09-18T10:15:46.697 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:46.697 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.697 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.697 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:46.698 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:41.332395+0000", 2024-09-18T10:15:46.698 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0233e-05 2024-09-18T10:15:46.698 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.698 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.698 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:46.698 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:41.332424+0000", 2024-09-18T10:15:46.698 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.83e-05 2024-09-18T10:15:46.698 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.698 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.698 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:46.698 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:41.332434+0000", 2024-09-18T10:15:46.698 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0803e-05 2024-09-18T10:15:46.699 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.699 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.699 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:46.699 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:41.632242+0000", 2024-09-18T10:15:46.699 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.29980755399999998 2024-09-18T10:15:46.699 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.699 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.699 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:46.699 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:41.632287+0000", 2024-09-18T10:15:46.699 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.5426999999999997e-05 2024-09-18T10:15:46.699 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:46.699 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:46.700 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:46.700 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.700 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.700 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWrite(3.cs5 85/83 ECSubWrite(tid=146, reqid=client.4632.0:771, at_version=85'119, trim_to=0'0, roll_forward_to=84'116))", 2024-09-18T10:15:46.700 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:41.421154+0000", 2024-09-18T10:15:46.700 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 5.1877272440000004, 2024-09-18T10:15:46.700 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.42165365799999999, 2024-09-18T10:15:46.700 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:46.700 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:46.700 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:46.700 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.700 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:46.700 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:41.421154+0000", 2024-09-18T10:15:46.700 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:46.701 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.701 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.701 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:46.701 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:41.421154+0000", 2024-09-18T10:15:46.701 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:46.701 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.701 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.701 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:46.701 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:41.421152+0000", 2024-09-18T10:15:46.701 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:46.701 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.701 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.701 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:46.701 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:41.421498+0000", 2024-09-18T10:15:46.701 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00034613599999999998 2024-09-18T10:15:46.702 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.702 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.702 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:46.702 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:41.421533+0000", 2024-09-18T10:15:46.702 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.4947999999999998e-05 2024-09-18T10:15:46.702 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.702 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.702 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:46.702 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:41.421548+0000", 2024-09-18T10:15:46.702 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4698999999999999e-05 2024-09-18T10:15:46.702 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.702 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.702 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:46.703 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:41.732907+0000", 2024-09-18T10:15:46.703 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.31135915800000002 2024-09-18T10:15:46.703 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.703 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.703 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:46.703 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:41.732913+0000", 2024-09-18T10:15:46.703 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0020000000000003e-06 2024-09-18T10:15:46.703 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.703 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.703 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:46.703 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:41.842769+0000", 2024-09-18T10:15:46.703 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.109855936 2024-09-18T10:15:46.703 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.703 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.703 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:46.704 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:41.842808+0000", 2024-09-18T10:15:46.704 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8877000000000001e-05 2024-09-18T10:15:46.704 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:46.704 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:46.704 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:46.704 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.704 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.704 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpReadReply(3.4s0 85/82 ECSubReadReply(tid=18, attrs_read=0))", 2024-09-18T10:15:46.704 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:41.430069+0000", 2024-09-18T10:15:46.704 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 5.1788125320000002, 2024-09-18T10:15:46.704 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.30709219399999999, 2024-09-18T10:15:46.704 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:46.704 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:46.705 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:46.705 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.705 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:46.705 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:41.430069+0000", 2024-09-18T10:15:46.705 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:46.705 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.705 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.705 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:46.705 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:41.430069+0000", 2024-09-18T10:15:46.705 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:46.705 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.705 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.705 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:46.705 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:41.430068+0000", 2024-09-18T10:15:46.706 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.999999 2024-09-18T10:15:46.706 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.706 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.706 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:46.706 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:41.430106+0000", 2024-09-18T10:15:46.706 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7982000000000003e-05 2024-09-18T10:15:46.706 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.706 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.706 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:46.706 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:41.430124+0000", 2024-09-18T10:15:46.706 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7719000000000001e-05 2024-09-18T10:15:46.706 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.707 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.707 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:46.707 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:41.430141+0000", 2024-09-18T10:15:46.707 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7543000000000001e-05 2024-09-18T10:15:46.707 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.707 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.707 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:46.707 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:41.735970+0000", 2024-09-18T10:15:46.707 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.30582816600000001 2024-09-18T10:15:46.707 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.707 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.707 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:46.707 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:41.737161+0000", 2024-09-18T10:15:46.708 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0011915719999999999 2024-09-18T10:15:46.708 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:46.708 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:46.708 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:46.708 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.708 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.708 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:884 3.4s0 3:2650be24:::smithi18925896-45:28 [read 0~3883008,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T10:15:46.708 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:45.695624+0000", 2024-09-18T10:15:46.708 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.91325737900000004, 2024-09-18T10:15:46.708 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.35940762500000001, 2024-09-18T10:15:46.708 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:46.708 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:46.708 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:46.709 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:46.709 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:46.709 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 884 2024-09-18T10:15:46.709 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.709 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:46.709 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.709 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:46.709 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:45.695624+0000", 2024-09-18T10:15:46.709 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:46.709 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.709 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.709 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:46.710 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:45.695624+0000", 2024-09-18T10:15:46.710 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:46.710 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.710 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.710 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:46.710 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:45.695620+0000", 2024-09-18T10:15:46.710 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999962 2024-09-18T10:15:46.710 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.710 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.710 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:46.710 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:45.695642+0000", 2024-09-18T10:15:46.710 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1522000000000002e-05 2024-09-18T10:15:46.710 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.711 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.711 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:46.711 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:45.695681+0000", 2024-09-18T10:15:46.711 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8776000000000003e-05 2024-09-18T10:15:46.711 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.711 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.711 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:46.711 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:45.695695+0000", 2024-09-18T10:15:46.711 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4776e-05 2024-09-18T10:15:46.711 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.711 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.711 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:46.711 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:45.735605+0000", 2024-09-18T10:15:46.712 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.039909294999999997 2024-09-18T10:15:46.712 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.712 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.712 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op must wait for map", 2024-09-18T10:15:46.712 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:45.735635+0000", 2024-09-18T10:15:46.712 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0709e-05 2024-09-18T10:15:46.712 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.712 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.712 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:46.712 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:45.748912+0000", 2024-09-18T10:15:46.712 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.013276872 2024-09-18T10:15:46.712 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.712 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.713 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:46.713 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:45.749138+0000", 2024-09-18T10:15:46.713 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00022611500000000001 2024-09-18T10:15:46.713 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.713 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.713 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:46.713 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:46.055032+0000", 2024-09-18T10:15:46.713 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.30589326 2024-09-18T10:15:46.713 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:46.713 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:46.713 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:46.713 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.713 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.713 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:885 3.4s0 3:2650be24:::smithi18925896-45:28 [read 0~3883008,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-18T10:15:46.714 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:45.695710+0000", 2024-09-18T10:15:46.714 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.91317156499999996, 2024-09-18T10:15:46.714 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.46487797199999997, 2024-09-18T10:15:46.714 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:46.714 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:46.714 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:46.714 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:46.714 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:46.714 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 885 2024-09-18T10:15:46.714 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.714 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:46.714 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.714 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:46.714 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:45.695710+0000", 2024-09-18T10:15:46.714 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:46.715 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.715 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.715 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:46.715 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:45.695710+0000", 2024-09-18T10:15:46.715 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:46.715 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.715 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.715 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:46.715 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:45.695709+0000", 2024-09-18T10:15:46.715 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.999999 2024-09-18T10:15:46.715 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.715 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.715 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:46.716 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:45.695716+0000", 2024-09-18T10:15:46.716 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.8340000000000001e-06 2024-09-18T10:15:46.716 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.716 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.716 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:46.716 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:45.695732+0000", 2024-09-18T10:15:46.716 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.6716000000000001e-05 2024-09-18T10:15:46.716 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.716 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.716 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:46.716 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:45.695739+0000", 2024-09-18T10:15:46.716 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.6959999999999999e-06 2024-09-18T10:15:46.716 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.716 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.717 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:46.717 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:45.736001+0000", 2024-09-18T10:15:46.717 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.040262118999999999 2024-09-18T10:15:46.717 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.717 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.717 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:46.717 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:45.736009+0000", 2024-09-18T10:15:46.717 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4699999999999996e-06 2024-09-18T10:15:46.717 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.717 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.717 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:46.717 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:45.749984+0000", 2024-09-18T10:15:46.717 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.013975411 2024-09-18T10:15:46.718 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.718 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.718 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:46.718 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:45.750350+0000", 2024-09-18T10:15:46.718 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00036625900000000002 2024-09-18T10:15:46.718 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:46.718 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:46.718 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:46.718 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:46.160588+0000", 2024-09-18T10:15:46.718 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.41023764699999998 2024-09-18T10:15:46.718 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:46.718 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:46.718 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:46.718 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:46.719 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:46.719 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-18T10:15:46.719 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T10:15:46.722 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T10:15:46.723 INFO:teuthology.orchestra.run.smithi029.stderr:nodeep-scrub is set 2024-09-18T10:15:46.725 INFO:tasks.rados.rados.0.smithi189.stdout:347: finishing write tid 6 to smithi18925896-35 2024-09-18T10:15:46.725 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 35 v 45 (ObjNum 135 snap 29 seq_num 135) dirty exists 2024-09-18T10:15:46.725 INFO:tasks.rados.rados.0.smithi189.stdout:347: left oid 35 (ObjNum 135 snap 29 seq_num 135) 2024-09-18T10:15:46.725 INFO:tasks.rados.rados.0.smithi189.stdout:342: done (10 left) 2024-09-18T10:15:46.725 INFO:tasks.rados.rados.0.smithi189.stdout:343: done (9 left) 2024-09-18T10:15:46.725 INFO:tasks.rados.rados.0.smithi189.stdout:345: done (8 left) 2024-09-18T10:15:46.725 INFO:tasks.rados.rados.0.smithi189.stdout:347: done (7 left) 2024-09-18T10:15:46.725 INFO:tasks.rados.rados.0.smithi189.stdout:348: done (6 left) 2024-09-18T10:15:46.725 INFO:tasks.rados.rados.0.smithi189.stdout:350: done (5 left) 2024-09-18T10:15:46.725 INFO:tasks.rados.rados.0.smithi189.stdout:351: done (4 left) 2024-09-18T10:15:46.726 INFO:tasks.rados.rados.0.smithi189.stdout:352: done (3 left) 2024-09-18T10:15:46.726 INFO:tasks.rados.rados.0.smithi189.stdout:353: done (2 left) 2024-09-18T10:15:46.726 INFO:tasks.rados.rados.0.smithi189.stdout:354: rollback oid 42 current snap is 30 2024-09-18T10:15:46.726 INFO:tasks.rados.rados.0.smithi189.stdout:rollback oid 42 to 22 2024-09-18T10:15:46.726 INFO:tasks.rados.rados.0.smithi189.stdout:355: rollback oid 50 current snap is 30 2024-09-18T10:15:46.726 INFO:tasks.rados.rados.0.smithi189.stdout:rollback oid 50 to 26 2024-09-18T10:15:46.726 INFO:tasks.rados.rados.0.smithi189.stdout:356: rmattr oid 48 current snap is 30 2024-09-18T10:15:46.726 INFO:tasks.rados.rados.0.smithi189.stdout:356: done (4 left) 2024-09-18T10:15:46.726 INFO:tasks.rados.rados.0.smithi189.stdout:357: copy_from oid 14 from oid 22 current snap is 30 2024-09-18T10:15:46.726 INFO:tasks.rados.rados.0.smithi189.stdout:append oid 2 current snap is 30 2024-09-18T10:15:46.726 INFO:tasks.rados.rados.0.smithi189.stdout:358: seq_num 136 ranges {0=737280,737280=770048,1507328=540672,2048000=245760} 2024-09-18T10:15:46.726 INFO:tasks.rados.rados.0.smithi189.stdout:358: writing smithi18925896-2 from 0 to 737280 tid 1 2024-09-18T10:15:46.726 INFO:tasks.rados.rados.0.smithi189.stdout:358: writing smithi18925896-2 from 737280 to 1507328 tid 2 2024-09-18T10:15:46.726 INFO:tasks.rados.rados.0.smithi189.stdout:358: writing smithi18925896-2 from 1507328 to 2048000 tid 3 2024-09-18T10:15:46.727 INFO:tasks.rados.rados.0.smithi189.stdout:358: writing smithi18925896-2 from 2048000 to 2293760 tid 4 2024-09-18T10:15:46.727 INFO:tasks.rados.rados.0.smithi189.stdout:append oid 12 current snap is 30 2024-09-18T10:15:46.727 INFO:tasks.rados.rados.0.smithi189.stdout:359: seq_num 137 ranges {196608=753664,950272=786432,1736704=655360,2392064=114688} 2024-09-18T10:15:46.727 INFO:tasks.rados.rados.0.smithi189.stdout:359: writing smithi18925896-12 from 196608 to 950272 tid 1 2024-09-18T10:15:46.727 INFO:tasks.rados.rados.0.smithi189.stdout:359: writing smithi18925896-12 from 950272 to 1736704 tid 2 2024-09-18T10:15:46.727 INFO:tasks.rados.rados.0.smithi189.stdout:359: writing smithi18925896-12 from 1736704 to 2392064 tid 3 2024-09-18T10:15:46.727 INFO:tasks.rados.rados.0.smithi189.stdout:359: writing smithi18925896-12 from 2392064 to 2506752 tid 4 2024-09-18T10:15:46.727 INFO:tasks.rados.rados.0.smithi189.stdout:360: setattr oid 4 current snap is 30 2024-09-18T10:15:46.727 INFO:tasks.rados.rados.0.smithi189.stdout:344: finishing write tid 6 to smithi18925896-38 2024-09-18T10:15:46.727 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 38 v 122 (ObjNum 134 snap 29 seq_num 134) dirty exists 2024-09-18T10:15:46.727 INFO:tasks.rados.rados.0.smithi189.stdout:344: left oid 38 (ObjNum 134 snap 29 seq_num 134) 2024-09-18T10:15:46.727 INFO:tasks.rados.rados.0.smithi189.stdout:344: done (7 left) 2024-09-18T10:15:46.727 INFO:tasks.rados.rados.0.smithi189.stdout:361: snap_create 2024-09-18T10:15:46.728 INFO:tasks.rados.rados.0.smithi189.stdout:349: finishing copy_from to smithi18925896-41 2024-09-18T10:15:46.728 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 41 v 84 (ObjNum 121 snap 26 seq_num 121) dirty exists 2024-09-18T10:15:46.728 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-18T10:15:46.716+0000 7f14c6a1a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 22138) UID: 0 2024-09-18T10:15:46.822 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-18T10:15:46.820+0000 7f578c52f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22134) UID: 0 2024-09-18T10:15:46.840 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-18T10:15:46.840 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-18T10:15:46.840 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-09-18T10:15:46.840 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-18T10:15:46.849 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T10:15:46.914 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-18T10:15:46.922 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-18T10:15:46.920+0000 7f578c52f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22134) UID: 0 2024-09-18T10:15:46.935 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-18T10:15:46.950 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:46.950 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:46.959 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T10:15:46.960 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-18T10:15:46.982 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-18T10:15:47.012 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 4 v 19 (ObjNum -894497824 snap 32534 seq_num 304) dirty exists 2024-09-18T10:15:47.024 INFO:tasks.ceph.osd.4.smithi029.stderr:2024-09-18T10:15:47.017+0000 7f550cf6e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22189) UID: 0 2024-09-18T10:15:47.071 INFO:tasks.rados.rados.0.smithi189.stdout:355: finishing rollback tid 0 to smithi18925896-50 2024-09-18T10:15:47.106 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:47.106 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:47.110 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-18T10:15:47.113 INFO:tasks.rados.rados.0.smithi189.stdout:357: finishing copy_from to smithi18925896-14 2024-09-18T10:15:47.113 INFO:tasks.rados.rados.0.smithi189.stdout:357: got expected ENOENT (src dne) 2024-09-18T10:15:47.113 INFO:tasks.rados.rados.0.smithi189.stdout:354: finishing rollback tid 0 to smithi18925896-42 2024-09-18T10:15:47.113 INFO:tasks.rados.rados.0.smithi189.stdout:354: finishing rollback tid 1 to smithi18925896-42 2024-09-18T10:15:47.115 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T10:15:47.125 INFO:tasks.ceph.osd.14.smithi154.stderr:2024-09-18T10:15:47.122+0000 7f75a19e7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 22093) UID: 0 2024-09-18T10:15:47.133 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-18T10:15:47.155 INFO:teuthology.orchestra.run.smithi029.stdout:osd.6: {} 2024-09-18T10:15:47.172 INFO:tasks.rados.rados.0.smithi189.stdout:355: finishing rollback tid 1 to smithi18925896-50 2024-09-18T10:15:47.185 INFO:tasks.rados.rados.0.smithi189.stdout:359: finishing write tid 1 to smithi18925896-12 2024-09-18T10:15:47.185 INFO:tasks.rados.rados.0.smithi189.stdout:359: finishing write tid 2 to smithi18925896-12 2024-09-18T10:15:47.185 INFO:tasks.rados.rados.0.smithi189.stdout:359: finishing write tid 3 to smithi18925896-12 2024-09-18T10:15:47.185 INFO:tasks.rados.rados.0.smithi189.stdout:359: finishing write tid 4 to smithi18925896-12 2024-09-18T10:15:47.210 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:47.211 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:47.213 INFO:tasks.rados.rados.0.smithi189.stdout:357: finishing copy_from racing read to smithi18925896-14 2024-09-18T10:15:47.220 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T10:15:47.226 INFO:tasks.ceph.osd.14.smithi154.stderr:2024-09-18T10:15:47.222+0000 7f75a19e7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 22093) UID: 0 2024-09-18T10:15:47.241 INFO:teuthology.orchestra.run.smithi029.stdout:osd.7: {} 2024-09-18T10:15:47.264 INFO:teuthology.orchestra.run.smithi029.stdout:osd.8: {} 2024-09-18T10:15:47.284 INFO:teuthology.orchestra.run.smithi029.stdout:osd.9: {} 2024-09-18T10:15:47.317 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:47.317 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:47.326 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T10:15:47.327 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-18T10:15:47.322+0000 7f14dcc5b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22398) UID: 0 2024-09-18T10:15:47.332 INFO:teuthology.orchestra.run.smithi029.stdout:osd.10: {} 2024-09-18T10:15:47.354 INFO:teuthology.orchestra.run.smithi029.stdout:osd.11: {} 2024-09-18T10:15:47.377 INFO:teuthology.orchestra.run.smithi029.stdout:osd.12: {} 2024-09-18T10:15:47.381 INFO:tasks.rados.rados.0.smithi189.stdout:359: finishing write tid 5 to smithi18925896-12 2024-09-18T10:15:47.399 INFO:teuthology.orchestra.run.smithi029.stdout:osd.13: {} 2024-09-18T10:15:47.419 INFO:tasks.rados.rados.0.smithi189.stdout:354: finishing rollback tid 2 to smithi18925896-42 2024-09-18T10:15:47.419 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 42 v 100 (ObjNum 41 snap 0 seq_num 41) dirty exists 2024-09-18T10:15:47.426 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:47.426 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:47.429 INFO:tasks.ceph.osd.3.smithi189.stderr:2024-09-18T10:15:47.426+0000 7f2440fc7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22396) UID: 0 2024-09-18T10:15:47.435 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-09-18T10:15:47.442 INFO:teuthology.orchestra.run.smithi029.stdout:osd.14: {} 2024-09-18T10:15:47.466 INFO:teuthology.orchestra.run.smithi029.stdout:osd.15: {} 2024-09-18T10:15:47.468 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T10:15:47.469 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T10:15:47.469 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T10:15:47.469 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T10:15:47.469 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T10:15:47.469 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T10:15:47.469 INFO:teuthology.orchestra.run.smithi029.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T10:15:47.469 INFO:teuthology.orchestra.run.smithi029.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T10:15:47.469 INFO:teuthology.orchestra.run.smithi029.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-18T10:15:47.469 INFO:teuthology.orchestra.run.smithi029.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-18T10:15:47.469 INFO:teuthology.orchestra.run.smithi029.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-18T10:15:47.469 INFO:teuthology.orchestra.run.smithi029.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-18T10:15:47.469 INFO:teuthology.orchestra.run.smithi029.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-18T10:15:47.469 INFO:teuthology.orchestra.run.smithi029.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-18T10:15:47.469 INFO:teuthology.orchestra.run.smithi029.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-18T10:15:47.470 INFO:teuthology.orchestra.run.smithi029.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-18T10:15:47.479 INFO:tasks.rados.rados.0.smithi189.stdout:355: finishing rollback tid 2 to smithi18925896-50 2024-09-18T10:15:47.479 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 50 v 138 (ObjNum 100 snap 17 seq_num 100) dirty exists 2024-09-18T10:15:47.484 INFO:tasks.rados.rados.0.smithi189.stdout:359: finishing write tid 6 to smithi18925896-12 2024-09-18T10:15:47.484 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 12 v 79 (ObjNum 137 snap 30 seq_num 137) dirty exists 2024-09-18T10:15:47.484 INFO:tasks.rados.rados.0.smithi189.stdout:359: left oid 12 (ObjNum 137 snap 30 seq_num 137) 2024-09-18T10:15:47.530 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-09-18T10:15:47.526+0000 7fdfa8698700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22087) UID: 0 2024-09-18T10:15:47.533 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:47.533 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:47.542 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-09-18T10:15:47.610 INFO:tasks.rados.rados.0.smithi189.stdout:358: finishing write tid 1 to smithi18925896-2 2024-09-18T10:15:47.611 INFO:tasks.rados.rados.0.smithi189.stdout:358: finishing write tid 2 to smithi18925896-2 2024-09-18T10:15:47.611 INFO:tasks.rados.rados.0.smithi189.stdout:358: finishing write tid 3 to smithi18925896-2 2024-09-18T10:15:47.630 INFO:tasks.ceph.osd.4.smithi029.stderr:2024-09-18T10:15:47.625+0000 7f550cf6e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22189) UID: 0 2024-09-18T10:15:47.636 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:47.636 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:47.645 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-09-18T10:15:47.672 INFO:tasks.rados.rados.0.smithi189.stdout:349: done (7 left) 2024-09-18T10:15:47.673 INFO:tasks.rados.rados.0.smithi189.stdout:354: done (6 left) 2024-09-18T10:15:47.673 INFO:tasks.rados.rados.0.smithi189.stdout:355: done (5 left) 2024-09-18T10:15:47.673 INFO:tasks.rados.rados.0.smithi189.stdout:357: done (4 left) 2024-09-18T10:15:47.673 INFO:tasks.rados.rados.0.smithi189.stdout:359: done (3 left) 2024-09-18T10:15:47.673 INFO:tasks.rados.rados.0.smithi189.stdout:360: done (2 left) 2024-09-18T10:15:47.673 INFO:tasks.rados.rados.0.smithi189.stdout:361: done (1 left) 2024-09-18T10:15:47.673 INFO:tasks.rados.rados.0.smithi189.stdout:362: copy_from oid 16 from oid 20 current snap is 31 2024-09-18T10:15:47.673 INFO:tasks.rados.rados.0.smithi189.stdout:363: snap_remove snap 16 2024-09-18T10:15:47.732 INFO:tasks.ceph.osd.12.smithi029.stderr:2024-09-18T10:15:47.725+0000 7f8e4e627700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 22190) UID: 0 2024-09-18T10:15:47.748 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:47.748 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:47.757 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_ops_in_flight 2024-09-18T10:15:47.833 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-09-18T10:15:47.830+0000 7fdfa8698700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22087) UID: 0 2024-09-18T10:15:47.855 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:47.855 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:47.866 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_blocked_ops 2024-09-18T10:15:47.934 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-18T10:15:47.932+0000 7f14c6a1a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 22138) UID: 0 2024-09-18T10:15:47.962 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:47.962 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:47.971 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_historic_ops 2024-09-18T10:15:48.035 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-18T10:15:48.032+0000 7f1981873700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22132) UID: 0 2024-09-18T10:15:48.116 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:48.116 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:48.125 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T10:15:48.172 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-09-18T10:15:48.170+0000 7fdfa8698700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22087) UID: 0 2024-09-18T10:15:48.233 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:48.233 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:48.237 INFO:tasks.ceph.osd.6.smithi154.stderr:2024-09-18T10:15:48.234+0000 7f0abc9e1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22089) UID: 0 2024-09-18T10:15:48.243 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T10:15:48.338 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-18T10:15:48.336+0000 7f1981873700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22132) UID: 0 2024-09-18T10:15:48.341 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:48.341 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:48.350 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T10:15:48.438 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-09-18T10:15:48.434+0000 7fdfa8698700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22087) UID: 0 2024-09-18T10:15:48.451 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:48.451 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:48.461 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T10:15:48.539 INFO:tasks.ceph.osd.15.smithi189.stderr:2024-09-18T10:15:48.534+0000 7fd0e725e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 22401) UID: 0 2024-09-18T10:15:48.556 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:48.556 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:48.562 INFO:tasks.rados.rados.0.smithi189.stdout:363: done (2 left) 2024-09-18T10:15:48.562 INFO:tasks.rados.rados.0.smithi189.stdout:364: rollback oid 32 current snap is 31 2024-09-18T10:15:48.562 INFO:tasks.rados.rados.0.smithi189.stdout:rollback oid 32 to 18 2024-09-18T10:15:48.562 INFO:tasks.rados.rados.0.smithi189.stdout:358: finishing write tid 4 to smithi18925896-2 2024-09-18T10:15:48.562 INFO:tasks.rados.rados.0.smithi189.stdout:358: finishing write tid 5 to smithi18925896-2 2024-09-18T10:15:48.562 INFO:tasks.rados.rados.0.smithi189.stdout:358: finishing write tid 6 to smithi18925896-2 2024-09-18T10:15:48.562 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 2 v 57 (ObjNum 136 snap 30 seq_num 136) dirty exists 2024-09-18T10:15:48.563 INFO:tasks.rados.rados.0.smithi189.stdout:358: left oid 2 (ObjNum 136 snap 30 seq_num 136) 2024-09-18T10:15:48.563 INFO:tasks.rados.rados.0.smithi189.stdout:362: finishing copy_from racing read to smithi18925896-16 2024-09-18T10:15:48.563 INFO:tasks.rados.rados.0.smithi189.stdout:362: finishing copy_from to smithi18925896-16 2024-09-18T10:15:48.563 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 16 v 31 (ObjNum 118 snap 25 seq_num 118) dirty exists 2024-09-18T10:15:48.563 INFO:tasks.rados.rados.0.smithi189.stdout:358: done (2 left) 2024-09-18T10:15:48.563 INFO:tasks.rados.rados.0.smithi189.stdout:362: done (1 left) 2024-09-18T10:15:48.563 INFO:tasks.rados.rados.0.smithi189.stdout:365: read oid 28 snap -1 2024-09-18T10:15:48.563 INFO:tasks.rados.rados.0.smithi189.stdout:365: expect (ObjNum 131 snap 28 seq_num 131) 2024-09-18T10:15:48.563 INFO:tasks.rados.rados.0.smithi189.stdout:append oid 21 current snap is 31 2024-09-18T10:15:48.563 INFO:tasks.rados.rados.0.smithi189.stdout:366: seq_num 138 ranges {0=770048,770048=540672,1310720=753664,2064384=262144} 2024-09-18T10:15:48.566 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T10:15:48.576 INFO:tasks.rados.rados.0.smithi189.stdout:366: writing smithi18925896-21 from 0 to 770048 tid 1 2024-09-18T10:15:48.579 INFO:tasks.rados.rados.0.smithi189.stdout:366: writing smithi18925896-21 from 770048 to 1310720 tid 2 2024-09-18T10:15:48.583 INFO:tasks.rados.rados.0.smithi189.stdout:366: writing smithi18925896-21 from 1310720 to 2064384 tid 3 2024-09-18T10:15:48.585 INFO:tasks.rados.rados.0.smithi189.stdout:366: writing smithi18925896-21 from 2064384 to 2326528 tid 4 2024-09-18T10:15:48.585 INFO:tasks.rados.rados.0.smithi189.stdout:append oid 4 current snap is 31 2024-09-18T10:15:48.585 INFO:tasks.rados.rados.0.smithi189.stdout:367: seq_num 139 ranges {0=786432,786432=688128,1474560=475136,1949696=393216} 2024-09-18T10:15:48.589 INFO:tasks.rados.rados.0.smithi189.stdout:367: writing smithi18925896-4 from 0 to 786432 tid 1 2024-09-18T10:15:48.593 INFO:tasks.rados.rados.0.smithi189.stdout:367: writing smithi18925896-4 from 786432 to 1474560 tid 2 2024-09-18T10:15:48.596 INFO:tasks.rados.rados.0.smithi189.stdout:367: writing smithi18925896-4 from 1474560 to 1949696 tid 3 2024-09-18T10:15:48.598 INFO:tasks.rados.rados.0.smithi189.stdout:367: writing smithi18925896-4 from 1949696 to 2342912 tid 4 2024-09-18T10:15:48.599 INFO:tasks.rados.rados.0.smithi189.stdout:368: delete oid 9 current snap is 31 2024-09-18T10:15:48.640 INFO:tasks.ceph.osd.8.smithi029.stderr:2024-09-18T10:15:48.633+0000 7f2555c27700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 22192) UID: 0 2024-09-18T10:15:48.657 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:48.657 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:48.659 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7, 9, 10, 11, 12, 13, 14, 15] out_osds: [8] dead_osds: [] live_osds: [0, 4, 8, 12, 1, 5, 9, 13, 2, 6, 10, 14, 3, 7, 11, 15] 2024-09-18T10:15:48.659 INFO:tasks.thrashosds.thrasher:choose_action: min_in 8 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T10:15:48.659 INFO:tasks.thrashosds.thrasher:Adding osd 8 2024-09-18T10:15:48.659 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 8 2024-09-18T10:15:48.666 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T10:15:48.725 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-18T10:15:48.741 INFO:tasks.ceph.osd.14.smithi154.stderr:2024-09-18T10:15:48.738+0000 7f75a19e7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 22093) UID: 0 2024-09-18T10:15:48.760 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:48.760 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:48.770 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-09-18T10:15:48.777 INFO:tasks.rados.rados.0.smithi189.stdout:364: finishing rollback tid 0 to smithi18925896-32 2024-09-18T10:15:48.842 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-18T10:15:48.838+0000 7f14dcc5b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22398) UID: 0 2024-09-18T10:15:48.859 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:48.859 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:48.868 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-09-18T10:15:48.876 INFO:tasks.rados.rados.0.smithi189.stdout:364: finishing rollback tid 1 to smithi18925896-32 2024-09-18T10:15:48.876 INFO:tasks.rados.rados.0.smithi189.stdout:364: finishing rollback tid 2 to smithi18925896-32 2024-09-18T10:15:48.876 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 32 v 102 (ObjNum 36 snap 0 seq_num 36) dirty exists 2024-09-18T10:15:48.943 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-18T10:15:48.940+0000 7f00046a8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 22136) UID: 0 2024-09-18T10:15:48.959 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:48.959 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:48.968 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-09-18T10:15:48.992 INFO:tasks.ceph.mon.a.smithi029.stderr:2024-09-18T10:15:48.985+0000 7fa61494a700 -1 mon.a@0(leader).osd e93 definitely_dead 0 2024-09-18T10:15:49.044 INFO:tasks.ceph.osd.6.smithi154.stderr:2024-09-18T10:15:49.042+0000 7f0abc9e1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22089) UID: 0 2024-09-18T10:15:49.062 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:49.062 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:49.071 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_ops_in_flight 2024-09-18T10:15:49.080 INFO:tasks.rados.rados.0.smithi189.stdout:367: finishing write tid 1 to smithi18925896-4 2024-09-18T10:15:49.145 INFO:tasks.ceph.osd.6.smithi154.stderr:2024-09-18T10:15:49.142+0000 7f0abc9e1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22089) UID: 0 2024-09-18T10:15:49.172 INFO:tasks.rados.rados.0.smithi189.stdout:366: finishing write tid 1 to smithi18925896-21 2024-09-18T10:15:49.173 INFO:tasks.rados.rados.0.smithi189.stdout:366: finishing write tid 2 to smithi18925896-21 2024-09-18T10:15:49.180 INFO:tasks.rados.rados.0.smithi189.stdout:367: finishing write tid 2 to smithi18925896-4 2024-09-18T10:15:49.224 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:49.224 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:49.233 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_blocked_ops 2024-09-18T10:15:49.246 INFO:tasks.ceph.osd.14.smithi154.stderr:2024-09-18T10:15:49.242+0000 7f75a19e7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 22093) UID: 0 2024-09-18T10:15:49.274 INFO:tasks.rados.rados.0.smithi189.stdout:366: finishing write tid 3 to smithi18925896-21 2024-09-18T10:15:49.274 INFO:tasks.rados.rados.0.smithi189.stdout:366: finishing write tid 4 to smithi18925896-21 2024-09-18T10:15:49.283 INFO:tasks.rados.rados.0.smithi189.stdout:365: expect (ObjNum 131 snap 28 seq_num 131) 2024-09-18T10:15:49.334 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:49.334 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:49.343 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_historic_ops 2024-09-18T10:15:49.347 INFO:tasks.ceph.osd.12.smithi029.stderr:2024-09-18T10:15:49.345+0000 7f8e4e627700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 22190) UID: 0 2024-09-18T10:15:49.372 INFO:tasks.rados.rados.0.smithi189.stdout:364: done (4 left) 2024-09-18T10:15:49.372 INFO:tasks.rados.rados.0.smithi189.stdout:365: done (3 left) 2024-09-18T10:15:49.372 INFO:tasks.rados.rados.0.smithi189.stdout:368: done (2 left) 2024-09-18T10:15:49.372 INFO:tasks.rados.rados.0.smithi189.stdout:append oid 5 current snap is 31 2024-09-18T10:15:49.372 INFO:tasks.rados.rados.0.smithi189.stdout:369: seq_num 141 ranges {442368=425984,868352=573440,1441792=425984,1867776=524288,2392064=425984} 2024-09-18T10:15:49.383 INFO:tasks.rados.rados.0.smithi189.stdout:369: writing smithi18925896-5 from 442368 to 868352 tid 1 2024-09-18T10:15:49.387 INFO:tasks.rados.rados.0.smithi189.stdout:369: writing smithi18925896-5 from 868352 to 1441792 tid 2 2024-09-18T10:15:49.390 INFO:tasks.rados.rados.0.smithi189.stdout:369: writing smithi18925896-5 from 1441792 to 1867776 tid 3 2024-09-18T10:15:49.393 INFO:tasks.rados.rados.0.smithi189.stdout:369: writing smithi18925896-5 from 1867776 to 2392064 tid 4 2024-09-18T10:15:49.396 INFO:tasks.rados.rados.0.smithi189.stdout:369: writing smithi18925896-5 from 2392064 to 2818048 tid 5 2024-09-18T10:15:49.396 INFO:tasks.rados.rados.0.smithi189.stdout:370: delete oid 16 current snap is 31 2024-09-18T10:15:49.396 INFO:tasks.rados.rados.0.smithi189.stdout:366: finishing write tid 5 to smithi18925896-21 2024-09-18T10:15:49.396 INFO:tasks.rados.rados.0.smithi189.stdout:367: finishing write tid 3 to smithi18925896-4 2024-09-18T10:15:49.396 INFO:tasks.rados.rados.0.smithi189.stdout:367: finishing write tid 4 to smithi18925896-4 2024-09-18T10:15:49.448 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-18T10:15:49.442+0000 7f14dcc5b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22398) UID: 0 2024-09-18T10:15:49.481 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T10:15:49.486 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:49.486 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:49.501 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T10:15:49.548 INFO:tasks.ceph.osd.15.smithi189.stderr:2024-09-18T10:15:49.542+0000 7fd0e725e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 22401) UID: 0 2024-09-18T10:15:49.574 INFO:teuthology.orchestra.run.smithi029.stderr:marked in osd.8. 2024-09-18T10:15:49.575 INFO:teuthology.orchestra.run.smithi029.stderr:noscrub is unset 2024-09-18T10:15:49.589 INFO:tasks.thrashosds.thrasher:Added osd 8 2024-09-18T10:15:49.613 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:49.613 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:49.623 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T10:15:49.650 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-09-18T10:15:49.646+0000 7fdfa8698700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22087) UID: 0 2024-09-18T10:15:49.679 INFO:tasks.rados.rados.0.smithi189.stdout:366: finishing write tid 6 to smithi18925896-21 2024-09-18T10:15:49.679 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 21 v 28 (ObjNum 138 snap 31 seq_num 138) dirty exists 2024-09-18T10:15:49.679 INFO:tasks.rados.rados.0.smithi189.stdout:366: left oid 21 (ObjNum 138 snap 31 seq_num 138) 2024-09-18T10:15:49.697 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-18T10:15:49.719 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-18T10:15:49.721 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:49.722 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:49.731 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T10:15:49.742 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-18T10:15:49.751 INFO:tasks.ceph.osd.12.smithi029.stderr:2024-09-18T10:15:49.745+0000 7f8e4e627700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 22190) UID: 0 2024-09-18T10:15:49.765 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-18T10:15:49.790 INFO:tasks.rados.rados.0.smithi189.stdout:367: finishing write tid 5 to smithi18925896-4 2024-09-18T10:15:49.791 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-18T10:15:49.794 INFO:tasks.rados.rados.0.smithi189.stdout:369: finishing write tid 1 to smithi18925896-5 2024-09-18T10:15:49.816 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-18T10:15:49.831 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-18T10:15:49.831 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 20, 2024-09-18T10:15:49.831 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 600, 2024-09-18T10:15:49.831 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [ 2024-09-18T10:15:49.831 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.831 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 36/32 ECSubWriteReply(tid=5, last_complete=36'4, committed=1, applied=1))", 2024-09-18T10:15:49.831 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:50.303102+0000", 2024-09-18T10:15:49.831 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 59.524209673000001, 2024-09-18T10:15:49.832 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20304619600000001, 2024-09-18T10:15:49.832 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:49.832 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:49.832 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:49.832 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.832 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:49.832 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.303102+0000", 2024-09-18T10:15:49.832 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:49.832 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.832 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.832 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:49.832 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.303102+0000", 2024-09-18T10:15:49.833 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:49.833 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.833 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.833 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:49.833 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.303100+0000", 2024-09-18T10:15:49.833 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:49.833 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.833 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.833 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:49.833 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.303114+0000", 2024-09-18T10:15:49.833 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3866000000000001e-05 2024-09-18T10:15:49.833 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.833 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.834 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:49.834 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.303149+0000", 2024-09-18T10:15:49.834 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.5249000000000001e-05 2024-09-18T10:15:49.834 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.834 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.834 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:49.834 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.303174+0000", 2024-09-18T10:15:49.834 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4785000000000001e-05 2024-09-18T10:15:49.834 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.834 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.834 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:49.834 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.506129+0000", 2024-09-18T10:15:49.834 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20295469199999999 2024-09-18T10:15:49.835 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.835 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.835 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:49.835 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.506148+0000", 2024-09-18T10:15:49.835 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9720000000000001e-05 2024-09-18T10:15:49.835 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:49.835 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:49.835 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:49.835 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.835 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.835 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 36/32 ECSubWriteReply(tid=5, last_complete=36'4, committed=1, applied=1))", 2024-09-18T10:15:49.835 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:50.305623+0000", 2024-09-18T10:15:49.835 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 59.521688701000002, 2024-09-18T10:15:49.835 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40085840299999997, 2024-09-18T10:15:49.836 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:49.836 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:49.836 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:49.836 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.836 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:49.836 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.305623+0000", 2024-09-18T10:15:49.836 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:49.836 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.836 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.836 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:49.836 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.305623+0000", 2024-09-18T10:15:49.836 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:49.837 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.837 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.837 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:49.837 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.305621+0000", 2024-09-18T10:15:49.837 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999976 2024-09-18T10:15:49.837 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.837 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.837 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:49.838 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.305635+0000", 2024-09-18T10:15:49.838 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4283e-05 2024-09-18T10:15:49.838 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.838 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.838 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:49.838 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.305663+0000", 2024-09-18T10:15:49.838 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7475999999999999e-05 2024-09-18T10:15:49.838 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.839 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.839 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:49.839 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.305680+0000", 2024-09-18T10:15:49.839 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7609e-05 2024-09-18T10:15:49.839 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.839 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.839 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:49.839 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.706451+0000", 2024-09-18T10:15:49.839 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40077027399999998 2024-09-18T10:15:49.839 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.839 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.839 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:49.839 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.706481+0000", 2024-09-18T10:15:49.840 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0935000000000001e-05 2024-09-18T10:15:49.840 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:49.840 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:49.840 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:49.840 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.840 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.840 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 36/32 ECSubWriteReply(tid=5, last_complete=36'4, committed=1, applied=1))", 2024-09-18T10:15:49.840 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:50.305688+0000", 2024-09-18T10:15:49.841 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 59.521623337000001, 2024-09-18T10:15:49.841 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40093562900000002, 2024-09-18T10:15:49.841 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:49.841 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:49.841 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:49.841 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.841 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:49.841 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.305688+0000", 2024-09-18T10:15:49.841 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:49.841 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.842 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.842 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:49.842 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.305688+0000", 2024-09-18T10:15:49.842 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:49.842 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.842 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.842 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:49.842 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.305687+0000", 2024-09-18T10:15:49.842 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:49.842 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.842 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.842 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:49.843 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.305698+0000", 2024-09-18T10:15:49.843 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1763000000000001e-05 2024-09-18T10:15:49.843 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.843 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.843 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:49.843 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.305729+0000", 2024-09-18T10:15:49.843 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0502e-05 2024-09-18T10:15:49.843 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.843 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.843 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:49.843 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.305741+0000", 2024-09-18T10:15:49.843 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2363e-05 2024-09-18T10:15:49.843 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.843 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.844 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:49.844 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.706585+0000", 2024-09-18T10:15:49.844 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.400843265 2024-09-18T10:15:49.844 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.844 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.844 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:49.844 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.706624+0000", 2024-09-18T10:15:49.844 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9459000000000003e-05 2024-09-18T10:15:49.844 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:49.844 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:49.844 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:49.844 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.844 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.845 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 36/32 ECSubWriteReply(tid=4, last_complete=36'3, committed=1, applied=1))", 2024-09-18T10:15:49.845 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:50.404544+0000", 2024-09-18T10:15:49.845 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 59.422767821999997, 2024-09-18T10:15:49.845 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.30239659400000002, 2024-09-18T10:15:49.845 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:49.845 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:49.845 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:49.845 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.845 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:49.845 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.404544+0000", 2024-09-18T10:15:49.845 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:49.845 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.845 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.845 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:49.846 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.404544+0000", 2024-09-18T10:15:49.846 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:49.846 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.846 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.846 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:49.846 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.404542+0000", 2024-09-18T10:15:49.846 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:49.846 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.846 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.846 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:49.846 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.404555+0000", 2024-09-18T10:15:49.846 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3096e-05 2024-09-18T10:15:49.846 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.846 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.847 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:49.847 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.404576+0000", 2024-09-18T10:15:49.847 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1165000000000001e-05 2024-09-18T10:15:49.847 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.847 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.847 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:49.847 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.404587+0000", 2024-09-18T10:15:49.847 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0549e-05 2024-09-18T10:15:49.847 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.847 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.847 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:49.848 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.706657+0000", 2024-09-18T10:15:49.848 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.30206971999999999 2024-09-18T10:15:49.848 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.848 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.848 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:49.848 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.706941+0000", 2024-09-18T10:15:49.848 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00028382100000000002 2024-09-18T10:15:49.849 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:49.849 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:49.849 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:49.849 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.849 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.849 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 36/32 ECSubWriteReply(tid=5, last_complete=36'4, committed=1, applied=1))", 2024-09-18T10:15:49.849 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:50.406036+0000", 2024-09-18T10:15:49.850 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 59.421275604999998, 2024-09-18T10:15:49.850 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.30215917399999997, 2024-09-18T10:15:49.850 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:49.850 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:49.850 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:49.850 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.851 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:49.851 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.406036+0000", 2024-09-18T10:15:49.851 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:49.851 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.851 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.851 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:49.851 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.406036+0000", 2024-09-18T10:15:49.851 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:49.851 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.851 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.852 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:49.852 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.406035+0000", 2024-09-18T10:15:49.852 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:49.852 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.852 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.852 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:49.852 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.406046+0000", 2024-09-18T10:15:49.852 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1704e-05 2024-09-18T10:15:49.852 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.852 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.852 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:49.852 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.406065+0000", 2024-09-18T10:15:49.852 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.8448999999999999e-05 2024-09-18T10:15:49.853 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.853 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.853 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:49.853 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.406076+0000", 2024-09-18T10:15:49.853 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1286e-05 2024-09-18T10:15:49.853 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.853 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.853 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:49.853 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.707005+0000", 2024-09-18T10:15:49.853 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.30092866899999998 2024-09-18T10:15:49.853 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.853 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.854 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:49.854 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.708195+0000", 2024-09-18T10:15:49.854 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001190546 2024-09-18T10:15:49.854 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:49.854 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:49.854 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:49.856 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.856 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.856 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.cs3 36/32 ECSubWrite(tid=14, reqid=client.4632.0:92, at_version=0'0, trim_to=0'0, roll_forward_to=36'6))", 2024-09-18T10:15:49.856 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:50.504842+0000", 2024-09-18T10:15:49.856 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 59.322470203000002, 2024-09-18T10:15:49.856 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.202186116, 2024-09-18T10:15:49.856 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:49.856 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:49.856 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:49.856 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.856 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:49.857 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.504842+0000", 2024-09-18T10:15:49.857 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:49.857 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.857 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.857 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:49.857 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.504842+0000", 2024-09-18T10:15:49.857 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:49.857 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.857 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.857 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:49.857 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.504840+0000", 2024-09-18T10:15:49.857 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:49.858 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.858 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.858 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:49.858 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.504868+0000", 2024-09-18T10:15:49.858 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7809000000000001e-05 2024-09-18T10:15:49.858 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.858 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.858 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:49.858 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.504900+0000", 2024-09-18T10:15:49.858 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1921999999999997e-05 2024-09-18T10:15:49.858 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.858 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.859 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:49.859 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.605012+0000", 2024-09-18T10:15:49.859 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10011272 2024-09-18T10:15:49.859 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.859 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.859 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:49.859 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.605115+0000", 2024-09-18T10:15:49.859 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000103003 2024-09-18T10:15:49.859 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.859 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.859 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:49.859 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.605132+0000", 2024-09-18T10:15:49.859 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6359999999999999e-05 2024-09-18T10:15:49.860 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.860 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.860 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:49.860 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.706978+0000", 2024-09-18T10:15:49.860 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.101846672 2024-09-18T10:15:49.860 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.860 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.860 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:49.860 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.707028+0000", 2024-09-18T10:15:49.860 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9443000000000001e-05 2024-09-18T10:15:49.860 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:49.860 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:49.860 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:49.861 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.861 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.861 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:179 3.0s0 3:02f544f0:::smithi18925896-30:head [read 0~491520,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:49.861 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:52.680315+0000", 2024-09-18T10:15:49.861 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 57.146997073999998, 2024-09-18T10:15:49.861 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20435319499999999, 2024-09-18T10:15:49.861 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:49.861 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:15:49.861 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:49.861 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:49.861 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:49.862 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 179 2024-09-18T10:15:49.862 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.862 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:49.862 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.862 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:49.862 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.680315+0000", 2024-09-18T10:15:49.862 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:49.862 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.862 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.862 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:49.862 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.680315+0000", 2024-09-18T10:15:49.862 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:49.862 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.863 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.863 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:49.863 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.680313+0000", 2024-09-18T10:15:49.863 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:49.863 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.863 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.863 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:49.863 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.680322+0000", 2024-09-18T10:15:49.863 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.9299999999999992e-06 2024-09-18T10:15:49.863 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.863 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.863 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:49.864 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.680343+0000", 2024-09-18T10:15:49.864 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0556000000000001e-05 2024-09-18T10:15:49.864 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.864 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.864 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:49.864 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.680373+0000", 2024-09-18T10:15:49.864 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0592000000000001e-05 2024-09-18T10:15:49.864 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.864 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.864 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:49.864 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.681360+0000", 2024-09-18T10:15:49.864 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00098610400000000002 2024-09-18T10:15:49.864 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.864 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.865 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:49.865 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.681624+0000", 2024-09-18T10:15:49.865 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000263992 2024-09-18T10:15:49.865 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.865 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.865 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:49.865 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.884668+0000", 2024-09-18T10:15:49.865 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.203044321 2024-09-18T10:15:49.865 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:49.865 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:49.865 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:49.865 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.865 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.866 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:180 3.0s0 3:02f544f0:::smithi18925896-30:head [read 0~491520,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:49.866 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:52.680394+0000", 2024-09-18T10:15:49.866 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 57.146917995999999, 2024-09-18T10:15:49.866 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20540486599999999, 2024-09-18T10:15:49.866 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:49.866 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:15:49.866 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:49.866 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:49.866 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:49.867 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 180 2024-09-18T10:15:49.867 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.867 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:49.867 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.867 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:49.867 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.680394+0000", 2024-09-18T10:15:49.867 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:49.867 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.867 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.867 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:49.867 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.680394+0000", 2024-09-18T10:15:49.867 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:49.867 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.868 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.868 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:49.868 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.680393+0000", 2024-09-18T10:15:49.868 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:49.868 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.868 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.868 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:49.868 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.680401+0000", 2024-09-18T10:15:49.868 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.4169999999999992e-06 2024-09-18T10:15:49.868 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.868 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.868 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:49.868 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.680422+0000", 2024-09-18T10:15:49.869 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1186e-05 2024-09-18T10:15:49.869 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.869 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.869 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:49.869 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.680433+0000", 2024-09-18T10:15:49.869 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1103e-05 2024-09-18T10:15:49.869 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.869 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.869 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:49.869 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.783888+0000", 2024-09-18T10:15:49.869 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.103454746 2024-09-18T10:15:49.869 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.869 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.870 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:49.870 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.784067+0000", 2024-09-18T10:15:49.870 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00017865399999999999 2024-09-18T10:15:49.870 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.870 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.870 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:49.870 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.885799+0000", 2024-09-18T10:15:49.870 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10173201699999999 2024-09-18T10:15:49.870 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:49.870 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:49.870 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:49.870 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.870 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.870 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:191 3.0s0 3:0a83d6b3:::smithi18925896-20:2 [read 0~327680,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:49.871 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:52.694194+0000", 2024-09-18T10:15:49.871 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 57.133117556000002, 2024-09-18T10:15:49.871 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.29753027399999998, 2024-09-18T10:15:49.871 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:49.871 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:15:49.871 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:49.871 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:49.871 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:49.871 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 191 2024-09-18T10:15:49.871 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.871 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:49.871 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.871 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:49.872 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.694194+0000", 2024-09-18T10:15:49.872 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:49.872 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.872 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.872 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:49.872 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.694194+0000", 2024-09-18T10:15:49.872 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:49.872 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.872 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.872 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:49.872 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.694192+0000", 2024-09-18T10:15:49.872 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:49.872 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.873 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.873 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:49.873 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.694204+0000", 2024-09-18T10:15:49.873 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2233e-05 2024-09-18T10:15:49.873 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.873 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.873 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:49.873 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.694280+0000", 2024-09-18T10:15:49.873 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.5584e-05 2024-09-18T10:15:49.873 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.873 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.873 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:49.873 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.694292+0000", 2024-09-18T10:15:49.874 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2478e-05 2024-09-18T10:15:49.874 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.874 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.874 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:49.874 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.885869+0000", 2024-09-18T10:15:49.874 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.191576304 2024-09-18T10:15:49.874 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.874 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.874 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:49.874 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.886008+0000", 2024-09-18T10:15:49.874 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00013880300000000001 2024-09-18T10:15:49.874 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.874 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.875 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:49.875 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.991725+0000", 2024-09-18T10:15:49.875 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10571691 2024-09-18T10:15:49.875 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:49.875 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:49.875 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:49.875 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.875 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.875 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:192 3.0s0 3:0a83d6b3:::smithi18925896-20:2 [read 0~327680,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:49.875 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:52.694306+0000", 2024-09-18T10:15:49.875 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 57.133005605999998, 2024-09-18T10:15:49.875 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.41051474799999998, 2024-09-18T10:15:49.875 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:49.876 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:15:49.876 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:49.876 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:49.876 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:49.876 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 192 2024-09-18T10:15:49.876 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.876 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:49.876 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.876 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:49.876 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.694306+0000", 2024-09-18T10:15:49.876 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:49.876 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.877 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.877 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:49.877 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.694306+0000", 2024-09-18T10:15:49.877 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:49.877 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.877 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.877 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:49.877 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.694305+0000", 2024-09-18T10:15:49.877 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:49.877 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.877 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.877 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:49.877 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.694312+0000", 2024-09-18T10:15:49.878 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.5129999999999999e-06 2024-09-18T10:15:49.878 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.878 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.878 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:49.878 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.694328+0000", 2024-09-18T10:15:49.878 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5766000000000001e-05 2024-09-18T10:15:49.878 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.878 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.878 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:49.878 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.694349+0000", 2024-09-18T10:15:49.878 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0990000000000001e-05 2024-09-18T10:15:49.878 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.878 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.879 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:49.879 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.887405+0000", 2024-09-18T10:15:49.879 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.193055791 2024-09-18T10:15:49.879 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.879 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.879 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:49.879 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.887535+0000", 2024-09-18T10:15:49.879 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00013043000000000001 2024-09-18T10:15:49.879 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.879 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.879 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:49.879 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:53.104821+0000", 2024-09-18T10:15:49.879 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.217285486 2024-09-18T10:15:49.880 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:49.880 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:49.880 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:49.880 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.880 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.880 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:193 3.0s0 3:0a83d6b3:::smithi18925896-20:2 [read 0~327680,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:49.880 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:52.694359+0000", 2024-09-18T10:15:49.880 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 57.132952424999999, 2024-09-18T10:15:49.880 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.411282918, 2024-09-18T10:15:49.880 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:49.880 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:15:49.880 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:49.881 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:49.881 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:49.881 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 193 2024-09-18T10:15:49.881 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.881 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:49.881 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.881 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:49.881 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.694359+0000", 2024-09-18T10:15:49.881 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:49.881 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.881 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.881 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:49.881 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.694359+0000", 2024-09-18T10:15:49.882 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:49.882 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.882 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.882 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:49.882 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.694358+0000", 2024-09-18T10:15:49.882 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:15:49.882 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.882 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.882 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:49.882 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.694364+0000", 2024-09-18T10:15:49.882 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8089999999999998e-06 2024-09-18T10:15:49.883 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.883 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.883 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:49.883 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.694381+0000", 2024-09-18T10:15:49.883 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6949e-05 2024-09-18T10:15:49.883 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.883 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.883 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:49.883 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.694388+0000", 2024-09-18T10:15:49.883 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.8079999999999999e-06 2024-09-18T10:15:49.883 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.883 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.883 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:49.884 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.988184+0000", 2024-09-18T10:15:49.884 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.29379642900000003 2024-09-18T10:15:49.884 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.884 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.884 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:49.884 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.988399+0000", 2024-09-18T10:15:49.884 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000214537 2024-09-18T10:15:49.884 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.884 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.884 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:49.884 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:53.105642+0000", 2024-09-18T10:15:49.884 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.11724332699999999 2024-09-18T10:15:49.884 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:49.885 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:49.885 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:49.885 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.885 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.885 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.10s0 45/40 ECSubWriteReply(tid=18, last_complete=45'6, committed=1, applied=1))", 2024-09-18T10:15:49.885 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:01.742819+0000", 2024-09-18T10:15:49.885 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 48.084492822000001, 2024-09-18T10:15:49.885 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.30317059800000001, 2024-09-18T10:15:49.885 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:49.885 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:49.885 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:49.886 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.886 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:49.886 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.742819+0000", 2024-09-18T10:15:49.886 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:49.886 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.886 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.886 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:49.886 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.742819+0000", 2024-09-18T10:15:49.886 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:49.886 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.886 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.886 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:49.886 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.742817+0000", 2024-09-18T10:15:49.886 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:49.887 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.887 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.887 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:49.887 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.742832+0000", 2024-09-18T10:15:49.887 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.466e-05 2024-09-18T10:15:49.887 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.887 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.887 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:49.887 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.742866+0000", 2024-09-18T10:15:49.887 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4242000000000002e-05 2024-09-18T10:15:49.887 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.887 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.888 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:49.888 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.843033+0000", 2024-09-18T10:15:49.888 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100166479 2024-09-18T10:15:49.888 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.888 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.888 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:49.888 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.045959+0000", 2024-09-18T10:15:49.888 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.202926 2024-09-18T10:15:49.888 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.888 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.888 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:49.888 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.045990+0000", 2024-09-18T10:15:49.888 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0889999999999997e-05 2024-09-18T10:15:49.888 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:49.889 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:49.889 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:49.889 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.889 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.889 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.10s0 45/40 ECSubWriteReply(tid=18, last_complete=45'6, committed=1, applied=1))", 2024-09-18T10:15:49.889 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:01.843113+0000", 2024-09-18T10:15:49.889 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 47.984198976000002, 2024-09-18T10:15:49.889 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.203080129, 2024-09-18T10:15:49.889 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:49.889 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:49.889 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:49.889 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.889 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:49.889 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.843113+0000", 2024-09-18T10:15:49.890 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:49.890 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.890 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.890 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:49.890 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.843113+0000", 2024-09-18T10:15:49.890 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:49.890 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.890 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.890 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:49.890 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.843111+0000", 2024-09-18T10:15:49.890 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:49.890 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.890 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.890 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:49.891 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.843122+0000", 2024-09-18T10:15:49.891 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0199e-05 2024-09-18T10:15:49.891 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.891 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.891 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:49.891 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.843147+0000", 2024-09-18T10:15:49.891 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5113000000000002e-05 2024-09-18T10:15:49.891 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.891 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.891 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:49.891 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.943289+0000", 2024-09-18T10:15:49.891 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100141813 2024-09-18T10:15:49.891 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.891 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.892 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:49.892 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.046146+0000", 2024-09-18T10:15:49.892 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.102857768 2024-09-18T10:15:49.892 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.892 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.892 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:49.892 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.046193+0000", 2024-09-18T10:15:49.892 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.6566000000000001e-05 2024-09-18T10:15:49.892 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:49.892 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:49.892 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:49.892 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.892 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.893 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.1s5 56/40 ECSubWrite(tid=102, reqid=client.4632.0:437, at_version=0'0, trim_to=0'0, roll_forward_to=56'16))", 2024-09-18T10:15:49.893 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:12.541097+0000", 2024-09-18T10:15:49.893 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 37.286214614000002, 2024-09-18T10:15:49.893 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20186336399999999, 2024-09-18T10:15:49.893 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:49.893 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:49.893 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:49.893 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.893 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:49.893 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:12.541097+0000", 2024-09-18T10:15:49.893 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:49.893 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.893 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.894 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:49.894 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:12.541097+0000", 2024-09-18T10:15:49.894 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:49.894 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.894 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.894 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:49.894 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:12.541095+0000", 2024-09-18T10:15:49.894 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:49.894 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.894 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.894 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:49.894 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:12.541147+0000", 2024-09-18T10:15:49.894 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.2203000000000001e-05 2024-09-18T10:15:49.894 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.895 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.895 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:49.895 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:12.541184+0000", 2024-09-18T10:15:49.895 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6495000000000002e-05 2024-09-18T10:15:49.895 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.895 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.895 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:49.895 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:12.641359+0000", 2024-09-18T10:15:49.895 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100175135 2024-09-18T10:15:49.895 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.895 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.895 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:49.895 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:12.741675+0000", 2024-09-18T10:15:49.895 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100316457 2024-09-18T10:15:49.896 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.896 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.896 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:49.896 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:12.741693+0000", 2024-09-18T10:15:49.896 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7269e-05 2024-09-18T10:15:49.896 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.896 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.896 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:49.896 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:12.742894+0000", 2024-09-18T10:15:49.896 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0012013639999999999 2024-09-18T10:15:49.896 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.896 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.896 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:49.897 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:12.742961+0000", 2024-09-18T10:15:49.897 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.6575000000000005e-05 2024-09-18T10:15:49.897 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:49.897 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:49.897 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:49.897 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.897 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.897 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.1s5 57/40 ECSubWrite(tid=111, reqid=osd.4.0:111, at_version=57'18, trim_to=0'0, roll_forward_to=56'16))", 2024-09-18T10:15:49.897 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:13.054749+0000", 2024-09-18T10:15:49.897 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 36.772562553, 2024-09-18T10:15:49.897 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.201940535, 2024-09-18T10:15:49.897 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:49.897 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:49.898 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:49.898 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.898 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:49.898 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.054749+0000", 2024-09-18T10:15:49.898 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:49.898 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.898 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.898 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:49.898 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.054749+0000", 2024-09-18T10:15:49.898 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:49.898 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.898 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.898 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:49.898 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.054749+0000", 2024-09-18T10:15:49.899 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:49.899 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.899 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.899 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:49.899 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.054785+0000", 2024-09-18T10:15:49.899 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6480999999999999e-05 2024-09-18T10:15:49.899 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.899 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.899 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:49.899 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.054808+0000", 2024-09-18T10:15:49.899 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.2351e-05 2024-09-18T10:15:49.899 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.899 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.900 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:49.900 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.154914+0000", 2024-09-18T10:15:49.900 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100106862 2024-09-18T10:15:49.900 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.900 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.900 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:49.900 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.155065+0000", 2024-09-18T10:15:49.900 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000150762 2024-09-18T10:15:49.900 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.901 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.901 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:49.901 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.155083+0000", 2024-09-18T10:15:49.901 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.8207e-05 2024-09-18T10:15:49.901 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.901 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.901 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:49.901 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.256244+0000", 2024-09-18T10:15:49.901 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10116046200000001 2024-09-18T10:15:49.901 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.901 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.901 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:49.901 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.256690+0000", 2024-09-18T10:15:49.902 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00044597000000000001 2024-09-18T10:15:49.902 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:49.902 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:49.902 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:49.902 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.902 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.902 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 58/40 ECSubWriteReply(tid=21, last_complete=57'8, committed=1, applied=1))", 2024-09-18T10:15:49.902 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:13.649729+0000", 2024-09-18T10:15:49.902 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 36.177582946000001, 2024-09-18T10:15:49.902 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20172436599999999, 2024-09-18T10:15:49.902 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:49.902 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:49.903 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:49.903 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.903 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:49.903 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.649729+0000", 2024-09-18T10:15:49.903 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:49.903 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.903 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.903 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:49.903 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.649729+0000", 2024-09-18T10:15:49.903 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:49.903 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.903 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.903 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:49.903 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.649728+0000", 2024-09-18T10:15:49.904 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:49.904 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.904 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.904 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:49.904 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.649736+0000", 2024-09-18T10:15:49.904 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.0530000000000007e-06 2024-09-18T10:15:49.904 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.904 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.904 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:49.904 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.649748+0000", 2024-09-18T10:15:49.904 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1173e-05 2024-09-18T10:15:49.904 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.904 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.905 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:49.905 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.749876+0000", 2024-09-18T10:15:49.905 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10012805 2024-09-18T10:15:49.905 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.905 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.905 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:49.905 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.850098+0000", 2024-09-18T10:15:49.905 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100222371 2024-09-18T10:15:49.905 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.905 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.905 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:49.905 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.851453+0000", 2024-09-18T10:15:49.905 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0013552219999999999 2024-09-18T10:15:49.906 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:49.906 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:49.906 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:49.906 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.906 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.906 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:511 3.0s0 3:02f544f0:::smithi18925896-30:head [stat,delete] snapc 1a=[1a,19,18,16,12,11,10,f,e] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-18T10:15:49.906 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:18.899317+0000", 2024-09-18T10:15:49.906 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 30.927995176, 2024-09-18T10:15:49.906 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.21051439599999999, 2024-09-18T10:15:49.906 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:49.906 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:49.906 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:49.906 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:49.907 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:49.907 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 511 2024-09-18T10:15:49.907 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.907 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:49.907 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.907 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:49.907 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.899317+0000", 2024-09-18T10:15:49.907 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:49.907 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.907 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.907 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:49.907 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.899317+0000", 2024-09-18T10:15:49.907 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:49.907 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.908 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.908 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:49.908 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.899314+0000", 2024-09-18T10:15:49.908 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999971 2024-09-18T10:15:49.908 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.908 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.908 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:49.908 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.899329+0000", 2024-09-18T10:15:49.908 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5160000000000001e-05 2024-09-18T10:15:49.908 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.908 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.908 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:49.908 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.899364+0000", 2024-09-18T10:15:49.909 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.5456000000000001e-05 2024-09-18T10:15:49.909 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.909 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.909 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:49.909 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.899385+0000", 2024-09-18T10:15:49.909 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0279e-05 2024-09-18T10:15:49.909 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.909 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.909 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:49.909 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.899497+0000", 2024-09-18T10:15:49.909 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000111773 2024-09-18T10:15:49.909 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.909 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.909 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "op must wait for map", 2024-09-18T10:15:49.910 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.899508+0000", 2024-09-18T10:15:49.910 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1177e-05 2024-09-18T10:15:49.910 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.910 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.910 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:49.910 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.903698+0000", 2024-09-18T10:15:49.910 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0041902850000000002 2024-09-18T10:15:49.910 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.910 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.910 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:49.910 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.903878+0000", 2024-09-18T10:15:49.910 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00018009499999999999 2024-09-18T10:15:49.910 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.911 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.911 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:49.911 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.905286+0000", 2024-09-18T10:15:49.911 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0014077720000000001 2024-09-18T10:15:49.911 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.911 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.911 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:49.911 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.003449+0000", 2024-09-18T10:15:49.911 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.098163081999999999 2024-09-18T10:15:49.911 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.911 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.911 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:49.911 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.109709+0000", 2024-09-18T10:15:49.912 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10625963400000001 2024-09-18T10:15:49.912 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.912 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.912 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:49.912 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.109831+0000", 2024-09-18T10:15:49.912 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00012246699999999999 2024-09-18T10:15:49.912 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:49.912 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:49.912 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:49.912 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.912 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.912 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.19s6 72/40 ECSubWrite(tid=176, reqid=osd.7.0:174, at_version=0'0, trim_to=0'0, roll_forward_to=71'100))", 2024-09-18T10:15:49.912 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:28.330736+0000", 2024-09-18T10:15:49.913 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 21.496575709999998, 2024-09-18T10:15:49.913 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20302320200000001, 2024-09-18T10:15:49.913 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:49.913 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:49.913 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:49.913 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.913 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:49.913 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.330736+0000", 2024-09-18T10:15:49.913 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:49.913 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.913 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.913 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:49.913 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.330736+0000", 2024-09-18T10:15:49.914 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:49.914 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.914 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.914 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:49.914 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.330735+0000", 2024-09-18T10:15:49.914 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:49.914 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.914 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.914 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:49.914 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.330759+0000", 2024-09-18T10:15:49.914 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4549000000000002e-05 2024-09-18T10:15:49.914 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.914 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.915 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:49.915 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.330816+0000", 2024-09-18T10:15:49.915 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.6693000000000001e-05 2024-09-18T10:15:49.915 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.915 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.915 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:49.915 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.430957+0000", 2024-09-18T10:15:49.915 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10014150199999999 2024-09-18T10:15:49.915 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.915 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.915 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:49.915 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.531236+0000", 2024-09-18T10:15:49.915 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100278201 2024-09-18T10:15:49.916 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.916 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.916 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:49.916 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.531263+0000", 2024-09-18T10:15:49.916 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7172999999999999e-05 2024-09-18T10:15:49.916 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.916 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.916 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:49.916 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.533701+0000", 2024-09-18T10:15:49.916 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0024382760000000001 2024-09-18T10:15:49.916 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.916 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.916 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:49.917 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.533759+0000", 2024-09-18T10:15:49.917 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8267999999999998e-05 2024-09-18T10:15:49.917 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:49.917 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:49.917 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:49.917 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.917 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.917 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.2s1 89/40 ECSubWrite(tid=132, reqid=osd.14.0:132, at_version=89'92, trim_to=0'0, roll_forward_to=88'90))", 2024-09-18T10:15:49.917 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:44.791495+0000", 2024-09-18T10:15:49.917 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 5.0358166009999996, 2024-09-18T10:15:49.917 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20177673600000001, 2024-09-18T10:15:49.917 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:49.918 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:49.918 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:49.918 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.918 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:49.918 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:44.791495+0000", 2024-09-18T10:15:49.918 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:49.918 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.918 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.918 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:49.918 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:44.791495+0000", 2024-09-18T10:15:49.918 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:49.918 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.918 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.919 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:49.919 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:44.791495+0000", 2024-09-18T10:15:49.919 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:49.919 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.919 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.919 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:49.919 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:44.791561+0000", 2024-09-18T10:15:49.919 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.6339000000000003e-05 2024-09-18T10:15:49.919 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.919 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.919 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:49.919 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:44.791584+0000", 2024-09-18T10:15:49.920 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3405999999999999e-05 2024-09-18T10:15:49.920 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.920 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.920 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:49.920 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:44.791599+0000", 2024-09-18T10:15:49.920 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.447e-05 2024-09-18T10:15:49.920 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.920 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.920 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:49.920 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:44.891786+0000", 2024-09-18T10:15:49.920 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10018745799999999 2024-09-18T10:15:49.920 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.920 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.921 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:49.921 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:44.891805+0000", 2024-09-18T10:15:49.921 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.8522999999999999e-05 2024-09-18T10:15:49.921 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.921 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.921 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:49.921 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:44.993190+0000", 2024-09-18T10:15:49.921 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.101385559 2024-09-18T10:15:49.921 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.921 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.921 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:49.921 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:44.993272+0000", 2024-09-18T10:15:49.921 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.1524000000000001e-05 2024-09-18T10:15:49.922 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:49.922 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:49.922 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:49.922 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.922 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.922 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.2s1 89/40 ECSubWrite(tid=133, reqid=osd.14.0:133, at_version=89'94, trim_to=0'0, roll_forward_to=88'90))", 2024-09-18T10:15:49.922 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:44.791629+0000", 2024-09-18T10:15:49.922 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 5.0356827500000003, 2024-09-18T10:15:49.922 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20167474799999999, 2024-09-18T10:15:49.922 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:49.922 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:49.922 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:49.922 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.923 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:49.923 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:44.791629+0000", 2024-09-18T10:15:49.923 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:49.923 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.923 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.923 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:49.923 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:44.791629+0000", 2024-09-18T10:15:49.923 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:49.923 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.923 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.923 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:49.923 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:44.791628+0000", 2024-09-18T10:15:49.923 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:15:49.923 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.924 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.924 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:49.924 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:44.791701+0000", 2024-09-18T10:15:49.924 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.3239999999999997e-05 2024-09-18T10:15:49.924 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.924 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.924 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:49.924 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:44.791720+0000", 2024-09-18T10:15:49.924 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.8731000000000001e-05 2024-09-18T10:15:49.924 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.924 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.924 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:49.924 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:44.891823+0000", 2024-09-18T10:15:49.925 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100102674 2024-09-18T10:15:49.925 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.925 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.925 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:49.925 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:44.892843+0000", 2024-09-18T10:15:49.925 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0010199409999999999 2024-09-18T10:15:49.925 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.925 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.925 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:49.925 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:44.892865+0000", 2024-09-18T10:15:49.925 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1722e-05 2024-09-18T10:15:49.925 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.925 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.926 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:49.926 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:44.993282+0000", 2024-09-18T10:15:49.926 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10041747400000001 2024-09-18T10:15:49.926 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:49.926 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:49.926 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:49.926 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:44.993304+0000", 2024-09-18T10:15:49.926 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1812999999999999e-05 2024-09-18T10:15:49.926 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:49.926 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:49.926 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:49.926 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:49.926 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:49.927 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-18T10:15:49.929 INFO:teuthology.orchestra.run.smithi029.stdout:osd.6: {} 2024-09-18T10:15:49.929 INFO:teuthology.orchestra.run.smithi029.stdout:osd.7: {} 2024-09-18T10:15:49.929 INFO:teuthology.orchestra.run.smithi029.stdout:osd.8: {} 2024-09-18T10:15:49.929 INFO:teuthology.orchestra.run.smithi029.stdout:osd.9: {} 2024-09-18T10:15:49.929 INFO:teuthology.orchestra.run.smithi029.stdout:osd.10: {} 2024-09-18T10:15:49.929 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T10:15:49.930 INFO:tasks.rados.rados.0.smithi189.stdout:369: finishing write tid 2 to smithi18925896-5 2024-09-18T10:15:49.931 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-18T10:15:49.924+0000 7f14c6a1a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 22138) UID: 0 2024-09-18T10:15:49.950 INFO:teuthology.orchestra.run.smithi029.stdout:osd.11: {} 2024-09-18T10:15:49.970 INFO:teuthology.orchestra.run.smithi029.stdout:osd.12: {} 2024-09-18T10:15:49.992 INFO:teuthology.orchestra.run.smithi029.stdout:osd.13: {} 2024-09-18T10:15:50.029 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-18T10:15:50.024+0000 7f14c6a1a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 22138) UID: 0 2024-09-18T10:15:50.036 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-18T10:15:50.037 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [ 2024-09-18T10:15:50.037 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.037 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.2s6 94/40 ECSubWrite(tid=162, reqid=client.4632.0:938, at_version=94'111, trim_to=0'0, roll_forward_to=93'109))", 2024-09-18T10:15:50.037 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:49.901476+0000", 2024-09-18T10:15:50.037 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.13469646599999999, 2024-09-18T10:15:50.037 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.13474093500000001, 2024-09-18T10:15:50.037 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:50.037 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "queued for pg", 2024-09-18T10:15:50.037 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:50.037 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.037 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:50.037 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:49.901476+0000", 2024-09-18T10:15:50.038 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:50.038 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.038 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.038 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:50.038 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:49.901476+0000", 2024-09-18T10:15:50.038 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:50.038 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.038 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.038 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:50.038 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:49.901474+0000", 2024-09-18T10:15:50.038 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:50.038 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.039 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.039 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:50.039 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:49.901752+0000", 2024-09-18T10:15:50.039 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00027746299999999998 2024-09-18T10:15:50.039 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.039 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.039 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:50.039 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:49.901778+0000", 2024-09-18T10:15:50.039 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.6703000000000002e-05 2024-09-18T10:15:50.040 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.040 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.040 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:50.040 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:50.001875+0000", 2024-09-18T10:15:50.040 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10009709999999999 2024-09-18T10:15:50.040 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:50.040 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:50.040 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:50.040 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:50.040 INFO:teuthology.orchestra.run.smithi189.stdout: ], 2024-09-18T10:15:50.040 INFO:teuthology.orchestra.run.smithi189.stdout: "num_ops": 1 2024-09-18T10:15:50.040 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-18T10:15:50.041 INFO:teuthology.orchestra.run.smithi029.stdout:osd.14: {} 2024-09-18T10:15:50.046 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T10:15:50.073 INFO:teuthology.orchestra.run.smithi029.stdout:osd.15: {} 2024-09-18T10:15:50.076 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T10:15:50.076 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T10:15:50.076 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T10:15:50.076 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T10:15:50.076 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T10:15:50.076 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T10:15:50.076 INFO:teuthology.orchestra.run.smithi029.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T10:15:50.076 INFO:teuthology.orchestra.run.smithi029.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T10:15:50.077 INFO:teuthology.orchestra.run.smithi029.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-18T10:15:50.077 INFO:teuthology.orchestra.run.smithi029.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-18T10:15:50.077 INFO:teuthology.orchestra.run.smithi029.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-18T10:15:50.077 INFO:teuthology.orchestra.run.smithi029.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-18T10:15:50.077 INFO:teuthology.orchestra.run.smithi029.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-18T10:15:50.077 INFO:teuthology.orchestra.run.smithi029.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-18T10:15:50.077 INFO:teuthology.orchestra.run.smithi029.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-18T10:15:50.077 INFO:teuthology.orchestra.run.smithi029.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-18T10:15:50.095 INFO:tasks.rados.rados.0.smithi189.stdout:367: finishing write tid 6 to smithi18925896-4 2024-09-18T10:15:50.095 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 4 v 27 (ObjNum 139 snap 31 seq_num 139) dirty exists 2024-09-18T10:15:50.095 INFO:tasks.rados.rados.0.smithi189.stdout:367: left oid 4 (ObjNum 139 snap 31 seq_num 139) 2024-09-18T10:15:50.105 INFO:tasks.rados.rados.0.smithi189.stdout:369: finishing write tid 3 to smithi18925896-5 2024-09-18T10:15:50.105 INFO:tasks.rados.rados.0.smithi189.stdout:369: finishing write tid 4 to smithi18925896-5 2024-09-18T10:15:50.130 INFO:tasks.ceph.osd.10.smithi154.stderr:2024-09-18T10:15:50.126+0000 7fb73221a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 22091) UID: 0 2024-09-18T10:15:50.144 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-18T10:15:50.144 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-18T10:15:50.144 INFO:teuthology.orchestra.run.smithi189.stdout: "complaint_time": 30, 2024-09-18T10:15:50.144 INFO:teuthology.orchestra.run.smithi189.stdout: "num_blocked_ops": 0 2024-09-18T10:15:50.144 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-18T10:15:50.153 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T10:15:50.231 INFO:tasks.ceph.osd.4.smithi029.stderr:2024-09-18T10:15:50.229+0000 7f550cf6e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22189) UID: 0 2024-09-18T10:15:50.252 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-18T10:15:50.253 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 20, 2024-09-18T10:15:50.253 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 600, 2024-09-18T10:15:50.253 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [ 2024-09-18T10:15:50.253 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.253 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:114 3.9s0 3:9a1a774c:::smithi18925896-38:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:50.253 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:50.816650+0000", 2024-09-18T10:15:50.253 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 59.432117785999999, 2024-09-18T10:15:50.253 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.41853457900000002, 2024-09-18T10:15:50.253 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:50.253 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:15:50.253 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:50.253 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:50.254 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:50.254 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 114 2024-09-18T10:15:50.254 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.254 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:50.254 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.254 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:50.254 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.816650+0000", 2024-09-18T10:15:50.254 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:50.254 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.254 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.254 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:50.254 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.816650+0000", 2024-09-18T10:15:50.254 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:50.255 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.255 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.255 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:50.255 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.816648+0000", 2024-09-18T10:15:50.255 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:50.255 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.255 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.255 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:50.255 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.816661+0000", 2024-09-18T10:15:50.255 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2619000000000001e-05 2024-09-18T10:15:50.255 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.255 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.256 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:50.256 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.816696+0000", 2024-09-18T10:15:50.256 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4885999999999998e-05 2024-09-18T10:15:50.256 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.256 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.256 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:50.256 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.816708+0000", 2024-09-18T10:15:50.256 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2843000000000001e-05 2024-09-18T10:15:50.256 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.256 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.256 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:50.256 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.818228+0000", 2024-09-18T10:15:50.256 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001519882 2024-09-18T10:15:50.257 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.257 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.257 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for rw locks", 2024-09-18T10:15:50.257 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.818557+0000", 2024-09-18T10:15:50.257 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00032873700000000001 2024-09-18T10:15:50.257 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.257 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.257 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:50.257 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.925488+0000", 2024-09-18T10:15:50.257 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.106930502 2024-09-18T10:15:50.257 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.257 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.257 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:50.258 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.925563+0000", 2024-09-18T10:15:50.258 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.5386e-05 2024-09-18T10:15:50.258 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.258 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.258 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:50.258 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.235185+0000", 2024-09-18T10:15:50.258 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.30962182399999999 2024-09-18T10:15:50.258 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:50.258 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:50.258 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:50.258 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.258 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.258 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:138 3.9s0 3:9ab7c4c0:::smithi18925896-46:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:50.258 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:50.921054+0000", 2024-09-18T10:15:50.259 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 59.327713488000001, 2024-09-18T10:15:50.259 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.61288337900000001, 2024-09-18T10:15:50.259 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:50.259 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:15:50.259 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:50.259 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:50.259 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:50.259 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 138 2024-09-18T10:15:50.259 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.259 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:50.259 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.259 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:50.259 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.921054+0000", 2024-09-18T10:15:50.259 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:50.260 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.260 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.260 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:50.260 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.921054+0000", 2024-09-18T10:15:50.260 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:50.260 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.260 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.260 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:50.260 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.921054+0000", 2024-09-18T10:15:50.260 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:50.260 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.260 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.260 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:50.261 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.921057+0000", 2024-09-18T10:15:50.261 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1829999999999998e-06 2024-09-18T10:15:50.261 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.261 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.261 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:50.261 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.921090+0000", 2024-09-18T10:15:50.261 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3281999999999997e-05 2024-09-18T10:15:50.261 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.261 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.261 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:50.261 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.921096+0000", 2024-09-18T10:15:50.261 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.0519999999999998e-06 2024-09-18T10:15:50.261 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.262 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.262 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:50.262 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.127704+0000", 2024-09-18T10:15:50.262 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20660720499999999 2024-09-18T10:15:50.262 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.262 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.262 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for rw locks", 2024-09-18T10:15:50.262 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.127851+0000", 2024-09-18T10:15:50.262 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00014710999999999999 2024-09-18T10:15:50.262 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.262 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.262 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:50.262 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.234300+0000", 2024-09-18T10:15:50.262 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10644893900000001 2024-09-18T10:15:50.263 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.263 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.263 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:50.263 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.234462+0000", 2024-09-18T10:15:50.263 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00016218999999999999 2024-09-18T10:15:50.263 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.263 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.263 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:50.263 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.533938+0000", 2024-09-18T10:15:50.263 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.29947596999999998 2024-09-18T10:15:50.263 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:50.263 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:50.263 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:50.263 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.263 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.264 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:143 3.9s0 3:9221fff5:::smithi18925896-48:head [append 0~376832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:50.264 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:50.932038+0000", 2024-09-18T10:15:50.264 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 59.316729887000001, 2024-09-18T10:15:50.264 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.60109878000000005, 2024-09-18T10:15:50.264 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:50.264 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:50.264 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:50.264 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:50.264 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:50.264 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 143 2024-09-18T10:15:50.264 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.264 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:50.264 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.265 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:50.265 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932038+0000", 2024-09-18T10:15:50.265 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:50.265 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.265 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.265 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:50.265 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932038+0000", 2024-09-18T10:15:50.265 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:50.265 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.265 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.265 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:50.266 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932037+0000", 2024-09-18T10:15:50.266 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:15:50.266 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.266 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.266 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:50.266 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932309+0000", 2024-09-18T10:15:50.266 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000272086 2024-09-18T10:15:50.266 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.266 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.266 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:50.266 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932323+0000", 2024-09-18T10:15:50.266 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4354e-05 2024-09-18T10:15:50.266 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.266 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.267 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:50.267 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932329+0000", 2024-09-18T10:15:50.267 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.6840000000000001e-06 2024-09-18T10:15:50.267 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.267 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.267 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:50.267 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.130012+0000", 2024-09-18T10:15:50.267 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.19768277100000001 2024-09-18T10:15:50.267 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.267 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.267 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:50.267 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.130068+0000", 2024-09-18T10:15:50.267 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.6158e-05 2024-09-18T10:15:50.268 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.268 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.268 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:50.268 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.130809+0000", 2024-09-18T10:15:50.268 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00074078700000000002 2024-09-18T10:15:50.268 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.268 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.268 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:50.268 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.132840+0000", 2024-09-18T10:15:50.268 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0020308879999999998 2024-09-18T10:15:50.268 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.268 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.268 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:50.269 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.533025+0000", 2024-09-18T10:15:50.269 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.400185491 2024-09-18T10:15:50.269 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.269 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.269 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:50.269 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.533137+0000", 2024-09-18T10:15:50.269 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000111727 2024-09-18T10:15:50.269 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:50.269 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:50.269 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:50.269 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.269 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.269 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:144 3.9s0 3:9221fff5:::smithi18925896-48:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:50.269 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:50.932336+0000", 2024-09-18T10:15:50.270 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 59.316432251000002, 2024-09-18T10:15:50.270 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.60198766699999995, 2024-09-18T10:15:50.270 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:50.270 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:50.270 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:50.270 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:50.270 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:50.270 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 144 2024-09-18T10:15:50.270 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.270 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:50.270 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.270 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:50.270 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932336+0000", 2024-09-18T10:15:50.271 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:50.271 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.271 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.271 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:50.271 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932336+0000", 2024-09-18T10:15:50.271 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:50.271 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.271 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.271 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:50.271 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932335+0000", 2024-09-18T10:15:50.271 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:15:50.271 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.271 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.272 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:50.272 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932339+0000", 2024-09-18T10:15:50.272 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8539999999999999e-06 2024-09-18T10:15:50.272 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.272 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.272 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:50.272 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932347+0000", 2024-09-18T10:15:50.272 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.2910000000000007e-06 2024-09-18T10:15:50.272 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.272 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.272 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:50.272 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932351+0000", 2024-09-18T10:15:50.272 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.5420000000000001e-06 2024-09-18T10:15:50.272 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.273 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.273 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:50.273 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.131380+0000", 2024-09-18T10:15:50.273 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.199029236 2024-09-18T10:15:50.273 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.273 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.273 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:50.273 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.131436+0000", 2024-09-18T10:15:50.273 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.6098999999999999e-05 2024-09-18T10:15:50.273 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.273 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.273 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:50.273 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.131777+0000", 2024-09-18T10:15:50.274 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00034058800000000001 2024-09-18T10:15:50.274 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.274 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.274 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:50.274 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.133245+0000", 2024-09-18T10:15:50.274 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0014682300000000001 2024-09-18T10:15:50.274 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.274 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.274 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:50.274 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.534187+0000", 2024-09-18T10:15:50.274 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40094253299999999 2024-09-18T10:15:50.274 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.274 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.274 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:50.275 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.534323+0000", 2024-09-18T10:15:50.275 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00013606299999999999 2024-09-18T10:15:50.275 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:50.275 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:50.275 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:50.275 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.275 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.275 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:145 3.9s0 3:9221fff5:::smithi18925896-48:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:50.275 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:50.932355+0000", 2024-09-18T10:15:50.275 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 59.316412767000003, 2024-09-18T10:15:50.275 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.80502307200000001, 2024-09-18T10:15:50.275 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:50.275 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:15:50.276 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:50.276 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:50.276 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:50.276 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 145 2024-09-18T10:15:50.276 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.276 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:50.276 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.276 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:50.276 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932355+0000", 2024-09-18T10:15:50.276 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:50.276 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.276 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.276 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:50.277 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932355+0000", 2024-09-18T10:15:50.277 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:50.277 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.277 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.277 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:50.277 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932355+0000", 2024-09-18T10:15:50.277 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:50.277 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.277 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.277 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:50.277 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932358+0000", 2024-09-18T10:15:50.277 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4369999999999999e-06 2024-09-18T10:15:50.277 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.278 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.278 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:50.278 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932366+0000", 2024-09-18T10:15:50.278 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.3010000000000007e-06 2024-09-18T10:15:50.278 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.278 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.278 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:50.278 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932370+0000", 2024-09-18T10:15:50.278 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.5619999999999999e-06 2024-09-18T10:15:50.278 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.278 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.278 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:50.278 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.132210+0000", 2024-09-18T10:15:50.278 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.199840145 2024-09-18T10:15:50.279 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.279 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.279 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for rw locks", 2024-09-18T10:15:50.279 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.132312+0000", 2024-09-18T10:15:50.279 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000102335 2024-09-18T10:15:50.279 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.279 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.279 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:50.279 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.535310+0000", 2024-09-18T10:15:50.279 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40299779000000002 2024-09-18T10:15:50.279 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.279 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.279 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:50.280 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.535541+0000", 2024-09-18T10:15:50.280 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00023038299999999999 2024-09-18T10:15:50.280 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.280 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.280 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:50.280 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.737378+0000", 2024-09-18T10:15:50.280 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.201837604 2024-09-18T10:15:50.280 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:50.280 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:50.280 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:50.280 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.280 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.280 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:174 3.9s0 3:9fc2eacb:::smithi18925896-17:head [read 0~278528,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:50.280 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:51.836672+0000", 2024-09-18T10:15:50.281 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 58.412095512, 2024-09-18T10:15:50.281 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.407996215, 2024-09-18T10:15:50.281 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:50.281 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:15:50.281 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:50.281 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:50.281 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:50.281 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 174 2024-09-18T10:15:50.281 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.281 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:50.281 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.281 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:50.281 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.836672+0000", 2024-09-18T10:15:50.281 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:50.282 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.282 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.282 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:50.282 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.836672+0000", 2024-09-18T10:15:50.282 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:50.282 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.282 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.282 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:50.282 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.836671+0000", 2024-09-18T10:15:50.282 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:50.282 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.282 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.282 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:50.283 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.836680+0000", 2024-09-18T10:15:50.283 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.5830000000000007e-06 2024-09-18T10:15:50.283 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.283 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.283 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:50.283 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.836696+0000", 2024-09-18T10:15:50.283 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6823000000000001e-05 2024-09-18T10:15:50.283 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.283 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.283 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:50.283 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.836710+0000", 2024-09-18T10:15:50.283 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.325e-05 2024-09-18T10:15:50.283 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.283 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.284 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:50.284 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.142312+0000", 2024-09-18T10:15:50.284 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.30560266400000002 2024-09-18T10:15:50.284 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.284 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.284 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:50.284 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.142549+0000", 2024-09-18T10:15:50.284 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00023630299999999999 2024-09-18T10:15:50.284 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.284 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.284 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:50.284 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.244669+0000", 2024-09-18T10:15:50.284 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10211993599999999 2024-09-18T10:15:50.285 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:50.285 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:50.285 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:50.285 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.285 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.285 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:202 3.9s0 3:9221fff5:::smithi18925896-48:head [copy-from ver 2] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:50.285 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:52.793605+0000", 2024-09-18T10:15:50.285 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 57.455162917000003, 2024-09-18T10:15:50.285 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.61381690899999997, 2024-09-18T10:15:50.285 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:50.285 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:50.285 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:50.285 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:50.285 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:50.286 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 202 2024-09-18T10:15:50.286 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.286 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:50.286 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.286 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:50.286 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.793605+0000", 2024-09-18T10:15:50.286 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:50.286 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.286 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.286 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:50.286 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.793605+0000", 2024-09-18T10:15:50.286 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:50.286 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.287 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.287 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:50.287 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.793604+0000", 2024-09-18T10:15:50.287 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:15:50.287 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.287 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.287 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:50.287 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.793613+0000", 2024-09-18T10:15:50.287 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.4360000000000008e-06 2024-09-18T10:15:50.287 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.287 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.287 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:50.287 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.793628+0000", 2024-09-18T10:15:50.287 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4953000000000001e-05 2024-09-18T10:15:50.288 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.288 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.288 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:50.288 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.793634+0000", 2024-09-18T10:15:50.288 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.0170000000000002e-06 2024-09-18T10:15:50.288 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.288 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.288 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:50.288 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.905359+0000", 2024-09-18T10:15:50.288 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.111724328 2024-09-18T10:15:50.288 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.288 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.288 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:50.288 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.905540+0000", 2024-09-18T10:15:50.289 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000181552 2024-09-18T10:15:50.289 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.289 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.289 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:50.289 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:53.203724+0000", 2024-09-18T10:15:50.289 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.29818339500000002 2024-09-18T10:15:50.289 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.289 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.289 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:50.289 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:53.206308+0000", 2024-09-18T10:15:50.289 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0025841620000000001 2024-09-18T10:15:50.289 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.289 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.290 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:50.290 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:53.407237+0000", 2024-09-18T10:15:50.290 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.200928739 2024-09-18T10:15:50.290 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.290 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.290 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:50.290 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:53.407422+0000", 2024-09-18T10:15:50.290 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000185384 2024-09-18T10:15:50.290 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:50.290 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:50.290 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:50.290 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.290 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.291 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:203 3.9s0 3:9221fff5:::smithi18925896-48:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:50.291 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:52.793641+0000", 2024-09-18T10:15:50.291 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 57.455126532000001, 2024-09-18T10:15:50.291 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.41220821099999999, 2024-09-18T10:15:50.291 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:50.291 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:15:50.291 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:50.291 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:50.291 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:50.291 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 203 2024-09-18T10:15:50.291 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.291 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:50.291 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.291 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:50.292 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.793641+0000", 2024-09-18T10:15:50.292 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:50.292 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.292 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.292 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:50.292 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.793641+0000", 2024-09-18T10:15:50.292 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:50.292 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.292 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.292 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:50.292 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.793641+0000", 2024-09-18T10:15:50.292 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:15:50.292 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.293 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.293 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:50.293 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.793644+0000", 2024-09-18T10:15:50.293 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7170000000000002e-06 2024-09-18T10:15:50.293 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.293 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.293 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:50.293 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.793658+0000", 2024-09-18T10:15:50.293 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3353000000000001e-05 2024-09-18T10:15:50.293 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.293 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.293 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:50.293 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.793663+0000", 2024-09-18T10:15:50.293 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.7729999999999996e-06 2024-09-18T10:15:50.294 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.294 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.294 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:50.294 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.905813+0000", 2024-09-18T10:15:50.294 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.112150033 2024-09-18T10:15:50.294 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.294 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.294 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for blocked object", 2024-09-18T10:15:50.294 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.905901+0000", 2024-09-18T10:15:50.294 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.7319999999999997e-05 2024-09-18T10:15:50.294 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.294 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.294 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:50.294 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:53.205317+0000", 2024-09-18T10:15:50.295 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.299415826 2024-09-18T10:15:50.295 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.295 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.295 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:50.295 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:53.205528+0000", 2024-09-18T10:15:50.295 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000211485 2024-09-18T10:15:50.295 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.295 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.295 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:50.295 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:53.205850+0000", 2024-09-18T10:15:50.295 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000321544 2024-09-18T10:15:50.295 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:50.295 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:50.295 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:50.296 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.296 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.296 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:351 3.19s0 3:9a1a774c:::smithi18925896-38:b [read 0~1638400,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-18T10:15:50.296 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:03.544302+0000", 2024-09-18T10:15:50.296 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 46.704465528, 2024-09-18T10:15:50.296 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.41512137599999999, 2024-09-18T10:15:50.296 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:50.296 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:15:50.296 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:50.296 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:50.296 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:50.296 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 351 2024-09-18T10:15:50.296 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.296 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:50.297 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.297 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:50.297 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.544302+0000", 2024-09-18T10:15:50.297 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:50.297 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.297 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.297 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:50.297 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.544302+0000", 2024-09-18T10:15:50.297 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:50.297 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.297 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.297 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:50.297 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.544298+0000", 2024-09-18T10:15:50.298 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999957 2024-09-18T10:15:50.298 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.298 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.298 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:50.298 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.544320+0000", 2024-09-18T10:15:50.298 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1549e-05 2024-09-18T10:15:50.298 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.298 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.298 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:50.298 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.544366+0000", 2024-09-18T10:15:50.298 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.6752999999999998e-05 2024-09-18T10:15:50.298 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.298 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.298 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:50.299 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.544387+0000", 2024-09-18T10:15:50.299 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1052000000000001e-05 2024-09-18T10:15:50.299 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.299 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.299 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:50.299 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.544758+0000", 2024-09-18T10:15:50.299 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00037069700000000002 2024-09-18T10:15:50.299 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.299 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.299 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "op must wait for map", 2024-09-18T10:15:50.299 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.544786+0000", 2024-09-18T10:15:50.299 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7472000000000001e-05 2024-09-18T10:15:50.299 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.299 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.299 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:50.300 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.649236+0000", 2024-09-18T10:15:50.300 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.104450378 2024-09-18T10:15:50.300 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.300 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.300 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:50.300 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.649492+0000", 2024-09-18T10:15:50.300 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00025605300000000001 2024-09-18T10:15:50.300 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.300 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.300 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:50.300 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.959424+0000", 2024-09-18T10:15:50.300 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.30993174899999998 2024-09-18T10:15:50.300 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:50.301 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:50.301 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:50.301 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.301 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.301 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:417 3.9s0 3:9221fff5:::smithi18925896-48:head [copy-from ver 8] snapc 10=[10,f,e,b,4] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-18T10:15:50.301 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:08.547348+0000", 2024-09-18T10:15:50.301 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 41.701420302000002, 2024-09-18T10:15:50.301 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.69832097199999998, 2024-09-18T10:15:50.301 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:50.301 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:50.301 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:50.301 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:50.301 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:50.301 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 417 2024-09-18T10:15:50.302 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.302 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:50.302 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.302 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:50.302 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:08.547348+0000", 2024-09-18T10:15:50.302 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:50.302 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.302 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.302 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:50.302 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:08.547348+0000", 2024-09-18T10:15:50.302 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:50.302 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.302 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.303 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:50.303 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:08.547344+0000", 2024-09-18T10:15:50.303 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999967 2024-09-18T10:15:50.303 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.303 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.303 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:50.303 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:08.547367+0000", 2024-09-18T10:15:50.303 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.2316000000000001e-05 2024-09-18T10:15:50.303 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.303 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.303 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:50.303 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:08.547415+0000", 2024-09-18T10:15:50.303 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.8541999999999997e-05 2024-09-18T10:15:50.303 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.304 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.304 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:50.304 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:08.647531+0000", 2024-09-18T10:15:50.304 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10011588 2024-09-18T10:15:50.304 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.304 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.304 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:50.304 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:08.647713+0000", 2024-09-18T10:15:50.304 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000181965 2024-09-18T10:15:50.304 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.304 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.304 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "op must wait for map", 2024-09-18T10:15:50.304 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:08.647726+0000", 2024-09-18T10:15:50.304 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3576e-05 2024-09-18T10:15:50.305 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.305 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.305 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:50.305 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:08.754339+0000", 2024-09-18T10:15:50.305 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10661229699999999 2024-09-18T10:15:50.305 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.305 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.305 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:50.305 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:08.754855+0000", 2024-09-18T10:15:50.305 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00051627900000000002 2024-09-18T10:15:50.305 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.305 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.305 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:50.305 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:09.040700+0000", 2024-09-18T10:15:50.306 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.285844922 2024-09-18T10:15:50.306 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.306 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.306 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:50.306 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:09.045961+0000", 2024-09-18T10:15:50.306 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0052605769999999998 2024-09-18T10:15:50.306 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.306 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.306 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:50.306 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:09.245481+0000", 2024-09-18T10:15:50.306 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.19952081599999999 2024-09-18T10:15:50.306 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.306 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.306 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:50.306 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:09.245669+0000", 2024-09-18T10:15:50.307 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000187245 2024-09-18T10:15:50.307 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:50.307 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:50.307 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:50.307 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.307 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.307 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.1s4 57/40 ECSubWrite(tid=112, reqid=osd.4.0:112, at_version=57'20, trim_to=0'0, roll_forward_to=56'16))", 2024-09-18T10:15:50.307 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:13.062699+0000", 2024-09-18T10:15:50.307 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 37.186068613000003, 2024-09-18T10:15:50.307 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40470361999999999, 2024-09-18T10:15:50.307 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:50.307 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:50.307 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:50.308 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.308 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:50.308 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.062699+0000", 2024-09-18T10:15:50.308 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:50.308 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.308 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.308 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:50.308 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.062699+0000", 2024-09-18T10:15:50.308 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:50.308 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.308 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.308 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:50.308 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.062699+0000", 2024-09-18T10:15:50.308 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:50.309 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.309 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.309 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:50.309 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.062734+0000", 2024-09-18T10:15:50.309 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.5108999999999999e-05 2024-09-18T10:15:50.309 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.309 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.309 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:50.309 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.062746+0000", 2024-09-18T10:15:50.309 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2062000000000001e-05 2024-09-18T10:15:50.309 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.309 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.309 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:50.309 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.062754+0000", 2024-09-18T10:15:50.310 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.5800000000000003e-06 2024-09-18T10:15:50.310 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.310 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.310 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:50.310 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.254656+0000", 2024-09-18T10:15:50.310 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.19190244400000001 2024-09-18T10:15:50.310 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.310 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.310 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:50.310 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.254673+0000", 2024-09-18T10:15:50.310 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7146000000000002e-05 2024-09-18T10:15:50.310 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.310 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.311 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:50.311 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.467347+0000", 2024-09-18T10:15:50.311 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.21267335500000001 2024-09-18T10:15:50.311 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.311 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.311 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:50.311 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.467403+0000", 2024-09-18T10:15:50.311 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.6443999999999997e-05 2024-09-18T10:15:50.311 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:50.311 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:50.311 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:50.311 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.311 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.311 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:499 3.9s0 3:9221fff5:::smithi18925896-48:head [read 0~704512,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T10:15:50.312 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:17.889501+0000", 2024-09-18T10:15:50.312 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 32.359266576000003, 2024-09-18T10:15:50.312 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.41188610399999998, 2024-09-18T10:15:50.312 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:50.312 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:15:50.312 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:50.312 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:50.312 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:50.312 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 499 2024-09-18T10:15:50.312 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.312 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:50.312 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.312 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:50.312 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889501+0000", 2024-09-18T10:15:50.313 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:50.313 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.313 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.313 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:50.313 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889501+0000", 2024-09-18T10:15:50.313 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:50.313 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.313 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.313 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:50.313 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889499+0000", 2024-09-18T10:15:50.313 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999976 2024-09-18T10:15:50.313 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.313 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.314 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:50.314 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889514+0000", 2024-09-18T10:15:50.314 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4793999999999999e-05 2024-09-18T10:15:50.314 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.314 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.314 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:50.314 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889538+0000", 2024-09-18T10:15:50.314 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4218000000000001e-05 2024-09-18T10:15:50.314 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.314 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.314 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:50.314 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889552+0000", 2024-09-18T10:15:50.314 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4021e-05 2024-09-18T10:15:50.314 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.314 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.315 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:50.315 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889705+0000", 2024-09-18T10:15:50.315 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00015229499999999999 2024-09-18T10:15:50.315 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.315 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.315 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "op must wait for map", 2024-09-18T10:15:50.315 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889725+0000", 2024-09-18T10:15:50.315 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0251999999999999e-05 2024-09-18T10:15:50.315 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.315 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.315 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:50.315 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.894166+0000", 2024-09-18T10:15:50.315 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0044407969999999998 2024-09-18T10:15:50.316 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.316 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.316 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:50.316 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.894321+0000", 2024-09-18T10:15:50.316 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000155403 2024-09-18T10:15:50.316 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.316 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.316 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:50.316 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.301387+0000", 2024-09-18T10:15:50.316 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40706650900000002 2024-09-18T10:15:50.316 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:50.316 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:50.316 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:50.316 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.317 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.317 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:500 3.9s0 3:9221fff5:::smithi18925896-48:head [read 0~704512,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T10:15:50.317 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:17.889581+0000", 2024-09-18T10:15:50.317 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 32.359187034999998, 2024-09-18T10:15:50.317 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.41275783799999999, 2024-09-18T10:15:50.317 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:50.317 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:15:50.317 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:50.317 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:50.317 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:50.317 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 500 2024-09-18T10:15:50.317 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.317 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:50.318 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.318 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:50.318 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889581+0000", 2024-09-18T10:15:50.318 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:50.318 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.318 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.318 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:50.318 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889581+0000", 2024-09-18T10:15:50.318 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:50.318 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.318 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.318 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:50.318 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889580+0000", 2024-09-18T10:15:50.318 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:50.319 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.319 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.319 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:50.319 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889586+0000", 2024-09-18T10:15:50.319 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8340000000000004e-06 2024-09-18T10:15:50.319 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.319 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.319 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:50.319 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889594+0000", 2024-09-18T10:15:50.319 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.8320000000000006e-06 2024-09-18T10:15:50.319 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.319 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.319 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:50.319 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889599+0000", 2024-09-18T10:15:50.320 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.7890000000000002e-06 2024-09-18T10:15:50.320 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.320 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.320 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:50.320 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889906+0000", 2024-09-18T10:15:50.320 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00030686400000000001 2024-09-18T10:15:50.320 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.320 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.320 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:50.320 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889930+0000", 2024-09-18T10:15:50.320 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4318999999999999e-05 2024-09-18T10:15:50.320 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.320 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.320 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:50.321 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.895541+0000", 2024-09-18T10:15:50.321 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0056113029999999998 2024-09-18T10:15:50.321 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.321 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.321 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:50.321 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.895817+0000", 2024-09-18T10:15:50.321 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000275612 2024-09-18T10:15:50.321 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.321 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.321 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:50.321 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.302339+0000", 2024-09-18T10:15:50.321 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40652197699999998 2024-09-18T10:15:50.321 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:50.321 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:50.322 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:50.322 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.322 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.322 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:501 3.9s0 3:9221fff5:::smithi18925896-48:head [read 0~704512,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T10:15:50.322 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:17.889668+0000", 2024-09-18T10:15:50.322 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 32.359099925000002, 2024-09-18T10:15:50.322 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.51380426099999998, 2024-09-18T10:15:50.322 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:50.322 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:15:50.322 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:50.322 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:50.322 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:50.322 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 501 2024-09-18T10:15:50.322 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.323 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:50.323 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.323 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:50.323 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889668+0000", 2024-09-18T10:15:50.323 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:50.323 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.323 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.323 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:50.323 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889668+0000", 2024-09-18T10:15:50.323 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:50.323 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.323 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.323 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:50.323 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889667+0000", 2024-09-18T10:15:50.324 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:50.324 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.324 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.324 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:50.324 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889671+0000", 2024-09-18T10:15:50.324 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7570000000000002e-06 2024-09-18T10:15:50.324 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.324 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.324 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:50.324 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889682+0000", 2024-09-18T10:15:50.324 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0709e-05 2024-09-18T10:15:50.324 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.324 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.325 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:50.325 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889686+0000", 2024-09-18T10:15:50.325 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.437e-06 2024-09-18T10:15:50.325 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.325 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.325 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:50.325 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889996+0000", 2024-09-18T10:15:50.325 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00030937899999999998 2024-09-18T10:15:50.325 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.325 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.325 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:50.325 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.890018+0000", 2024-09-18T10:15:50.325 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.2592999999999999e-05 2024-09-18T10:15:50.326 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.326 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.326 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:50.326 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.996777+0000", 2024-09-18T10:15:50.326 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.106758192 2024-09-18T10:15:50.326 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.326 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.326 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:50.326 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.997045+0000", 2024-09-18T10:15:50.326 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00026851300000000001 2024-09-18T10:15:50.326 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.326 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.326 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:50.326 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.403472+0000", 2024-09-18T10:15:50.326 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40642724899999999 2024-09-18T10:15:50.327 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:50.327 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:50.327 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:50.327 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.327 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.327 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:502 3.19s0 3:9a1a774c:::smithi18925896-38:head [create,setxattr Rj\\mZOfQXo^b_ZWWfqN]m[iPt^aRVgsYNetVd`VO (19508) in=40b,setxattr [adnUNmYmWdN__]tR]msPnVSRiY^psh`dY\\Z (630) in=36b,setxattr cPsWXtVeXTPagdn_WVZbVcV\\UjSZUqsVlPZbY` (630) in=38b,setxattr pqbrReOSsgZlmftc]`Z`]USaXjsacf`fao_VP`eU (1190) in=40b,setxattr aYfgr^q[\\aPg_aZsakfUfsdhYbra^QXjlkrpdY\\\\ (138) in=40b,setxattr e\\XXWiVknihNQQ[OsThQUbRmtprrfYPeRX[[mRcQ (1719) in=40b,setxattr TNQYR^jgR`pPaYitmrcliV[dttobncTntcfTlQkt (8420) in=40b,setxattr W_aq[RVsaUpQtX_RpV^Vpqrg`QgQViPeVaRqZSmf (525) in=40b,setxattr RWdkeaaUP\\PZteTNscS^UVRtjo`oqqdNqe]cdZUa (245) in=40b,setxattr R^`al]TqifatYOiOU]p`ZegfXmNsXVaZcdd]b]YX (1435) in=40b,setxattr p]OS_Znl[dNa^qTn`WW[j\\dtgbNac_bijYmg (630) in=36b,setxattr n_jr[tV[VhYPqU]UenciobhtYfY^a^c\\hd^YrcQ[ (13472) in=40b,setxattr gReQSUg`cdl_ilU_lo[ggg\\fZsVQY[PgY]jPsZ_U (14664) in=40b,setxattr cNWaQrsdt^dic_qN_^mTXmc`_X]YOTXNk[WldRe` (523) in=40b,setxattr tZ]grhlaq]kUO[k^d^dr``XNio^^Oaebpj\\[Ytdq (33) in=40b,setxattr jqS_j^cefaXab`pQoPXfRkSmdako`fZ_heb^_fma (1191) in=40b,setxattr dgXPi (630) in=5b,setxattr Or_kkpbnNkTrQ\\apUaUsZR\\XW^nabhaiQ\\]gOjR\\ (525) in=40b,setxattr Rf`aR[fdtjQ\\NaqZke]l_rUssrVpOPXYWpXkl_kd (2876) in=40b,setxattr _sUSYWYUmOrlUVhZajZeYbbnsdra[]`Q`^Ogea[m (631) in=40b] snapc 19=[19,18,16,12,11,10,f,e] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T10:15:50.327 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:17.894821+0000", 2024-09-18T10:15:50.327 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 32.353946897, 2024-09-18T10:15:50.327 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.61297755899999995, 2024-09-18T10:15:50.327 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:50.327 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:50.327 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:50.327 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:50.327 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:50.327 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 502 2024-09-18T10:15:50.328 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.328 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:50.328 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.328 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:50.328 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.894821+0000", 2024-09-18T10:15:50.328 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:50.328 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.328 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.328 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:50.328 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.894821+0000", 2024-09-18T10:15:50.328 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:50.328 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.328 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.328 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:50.329 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.894819+0000", 2024-09-18T10:15:50.329 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:50.329 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.329 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.329 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:50.329 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.894942+0000", 2024-09-18T10:15:50.329 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00012299699999999999 2024-09-18T10:15:50.329 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.329 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.329 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:50.329 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.894987+0000", 2024-09-18T10:15:50.329 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.515e-05 2024-09-18T10:15:50.329 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.329 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.330 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:50.330 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.895021+0000", 2024-09-18T10:15:50.330 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3905000000000001e-05 2024-09-18T10:15:50.330 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.330 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.331 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:50.331 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.302420+0000", 2024-09-18T10:15:50.331 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40739912099999998 2024-09-18T10:15:50.331 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.332 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.332 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:50.332 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.302573+0000", 2024-09-18T10:15:50.332 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00015244400000000001 2024-09-18T10:15:50.332 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.332 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.332 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:50.332 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.304119+0000", 2024-09-18T10:15:50.332 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001546504 2024-09-18T10:15:50.332 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.332 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.332 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:50.332 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.306309+0000", 2024-09-18T10:15:50.333 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.002189992 2024-09-18T10:15:50.333 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.333 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.333 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:50.333 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.507653+0000", 2024-09-18T10:15:50.333 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20134429600000001 2024-09-18T10:15:50.333 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.333 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.333 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:50.333 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.507799+0000", 2024-09-18T10:15:50.333 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000145153 2024-09-18T10:15:50.333 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:50.333 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:50.334 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:50.334 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.334 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.334 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:514 3.19s0 3:9fc2eacb:::smithi18925896-17:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e63)", 2024-09-18T10:15:50.334 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:19.215577+0000", 2024-09-18T10:15:50.334 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 31.033190655999999, 2024-09-18T10:15:50.334 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.55261972699999995, 2024-09-18T10:15:50.334 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:50.334 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:15:50.334 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:50.334 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:50.334 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:50.334 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 514 2024-09-18T10:15:50.335 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.335 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:50.335 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.335 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:50.335 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.215577+0000", 2024-09-18T10:15:50.335 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:50.335 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.335 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.335 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:50.335 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.215577+0000", 2024-09-18T10:15:50.335 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:50.335 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.335 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.335 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:50.336 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.215575+0000", 2024-09-18T10:15:50.336 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:50.336 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.336 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.336 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:50.336 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.215590+0000", 2024-09-18T10:15:50.336 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4153e-05 2024-09-18T10:15:50.336 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.336 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.336 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:50.336 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.215610+0000", 2024-09-18T10:15:50.336 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9962e-05 2024-09-18T10:15:50.336 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.336 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.337 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:50.337 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.215622+0000", 2024-09-18T10:15:50.337 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2714000000000001e-05 2024-09-18T10:15:50.337 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.337 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.337 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:50.337 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.216288+0000", 2024-09-18T10:15:50.337 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00066566900000000003 2024-09-18T10:15:50.337 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.337 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.337 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for blocked object", 2024-09-18T10:15:50.337 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.216400+0000", 2024-09-18T10:15:50.337 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000112101 2024-09-18T10:15:50.338 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.338 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.338 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:50.338 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.767868+0000", 2024-09-18T10:15:50.338 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.55146753900000001 2024-09-18T10:15:50.338 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.338 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.338 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:50.338 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.768017+0000", 2024-09-18T10:15:50.338 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000149567 2024-09-18T10:15:50.338 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.338 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.338 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:50.339 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.768197+0000", 2024-09-18T10:15:50.339 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00017981200000000001 2024-09-18T10:15:50.339 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:50.339 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:50.339 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:50.339 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.339 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.339 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:561 3.19s0 3:9fc2eacb:::smithi18925896-17:head [append 0~622592] snapc 1f=[1f,1e,1a,16,12,11,10,f] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-18T10:15:50.339 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:23.955639+0000", 2024-09-18T10:15:50.339 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 26.293129310000001, 2024-09-18T10:15:50.339 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.51252563100000004, 2024-09-18T10:15:50.339 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:50.339 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:50.340 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:50.340 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:50.340 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:50.340 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 561 2024-09-18T10:15:50.340 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.340 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:50.340 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.340 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:50.340 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.955639+0000", 2024-09-18T10:15:50.340 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:50.341 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.341 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.341 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:50.341 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.955639+0000", 2024-09-18T10:15:50.341 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:50.341 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.341 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.341 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:50.341 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.955637+0000", 2024-09-18T10:15:50.341 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:50.341 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.341 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.341 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:50.342 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956211+0000", 2024-09-18T10:15:50.342 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00057413099999999997 2024-09-18T10:15:50.342 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.342 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.342 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:50.342 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956241+0000", 2024-09-18T10:15:50.342 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0244999999999999e-05 2024-09-18T10:15:50.342 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.342 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.342 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:50.342 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956253+0000", 2024-09-18T10:15:50.342 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1824000000000001e-05 2024-09-18T10:15:50.342 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.342 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.343 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:50.343 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956338+0000", 2024-09-18T10:15:50.343 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.5544999999999995e-05 2024-09-18T10:15:50.343 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.343 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.343 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:50.343 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956349+0000", 2024-09-18T10:15:50.343 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0203e-05 2024-09-18T10:15:50.343 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.343 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.343 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:50.343 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:24.162636+0000", 2024-09-18T10:15:50.343 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.206287267 2024-09-18T10:15:50.344 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.344 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.344 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:50.344 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:24.162935+0000", 2024-09-18T10:15:50.344 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00029932800000000002 2024-09-18T10:15:50.344 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.344 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.344 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:50.344 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:24.164750+0000", 2024-09-18T10:15:50.344 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0018148400000000001 2024-09-18T10:15:50.344 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.344 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.344 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:50.344 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:24.168548+0000", 2024-09-18T10:15:50.345 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0037977089999999998 2024-09-18T10:15:50.345 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.345 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.345 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:50.345 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:24.467969+0000", 2024-09-18T10:15:50.345 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.29942109700000002 2024-09-18T10:15:50.345 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.345 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.345 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:50.345 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:24.468164+0000", 2024-09-18T10:15:50.345 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00019534000000000001 2024-09-18T10:15:50.345 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:50.345 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:50.345 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:50.346 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.346 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.346 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:562 3.19s0 3:9fc2eacb:::smithi18925896-17:head [append 0~32768] snapc 1f=[1f,1e,1a,16,12,11,10,f] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-18T10:15:50.346 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:23.956275+0000", 2024-09-18T10:15:50.346 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 26.292493322999999, 2024-09-18T10:15:50.346 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.61260356900000001, 2024-09-18T10:15:50.346 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:50.346 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:50.346 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:50.346 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:50.346 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:50.346 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 562 2024-09-18T10:15:50.346 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.347 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:50.347 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.347 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:50.347 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956275+0000", 2024-09-18T10:15:50.347 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:50.347 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.347 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.347 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:50.347 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956275+0000", 2024-09-18T10:15:50.347 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:50.347 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.347 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.347 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:50.348 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956273+0000", 2024-09-18T10:15:50.348 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:50.348 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.348 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.348 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:50.348 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956355+0000", 2024-09-18T10:15:50.348 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.1661000000000003e-05 2024-09-18T10:15:50.348 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.348 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.348 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:50.348 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956366+0000", 2024-09-18T10:15:50.348 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1192000000000001e-05 2024-09-18T10:15:50.348 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.349 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.349 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:50.349 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956373+0000", 2024-09-18T10:15:50.349 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.5599999999999999e-06 2024-09-18T10:15:50.349 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.349 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.349 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:50.349 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956434+0000", 2024-09-18T10:15:50.349 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.0761999999999999e-05 2024-09-18T10:15:50.349 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.349 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.349 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:50.349 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956441+0000", 2024-09-18T10:15:50.349 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.1430000000000001e-06 2024-09-18T10:15:50.350 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.350 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.350 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:50.350 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:24.165954+0000", 2024-09-18T10:15:50.350 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20951336100000001 2024-09-18T10:15:50.350 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.350 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.350 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:50.350 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:24.166077+0000", 2024-09-18T10:15:50.350 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00012297299999999999 2024-09-18T10:15:50.350 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.350 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.350 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:50.350 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:24.166875+0000", 2024-09-18T10:15:50.351 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00079825999999999996 2024-09-18T10:15:50.351 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.351 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.351 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:50.351 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:24.168650+0000", 2024-09-18T10:15:50.351 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0017746839999999999 2024-09-18T10:15:50.351 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.351 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.351 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:50.351 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:24.568726+0000", 2024-09-18T10:15:50.351 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40007642799999998 2024-09-18T10:15:50.351 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.351 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.351 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:50.352 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:24.568878+0000", 2024-09-18T10:15:50.352 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000151803 2024-09-18T10:15:50.352 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:50.352 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:50.352 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:50.352 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.352 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.352 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:563 3.19s0 3:9fc2eacb:::smithi18925896-17:head [setxattr _header (57) in=7b] snapc 1f=[1f,1e,1a,16,12,11,10,f] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-18T10:15:50.352 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:23.956389+0000", 2024-09-18T10:15:50.352 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 26.292378976999998, 2024-09-18T10:15:50.352 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.61276698100000004, 2024-09-18T10:15:50.352 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:50.352 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:50.352 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:50.353 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:50.353 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:50.353 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 563 2024-09-18T10:15:50.353 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.353 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:50.353 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.353 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:50.353 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956389+0000", 2024-09-18T10:15:50.353 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:50.353 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.353 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.353 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:50.353 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956389+0000", 2024-09-18T10:15:50.354 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:50.354 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.354 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.354 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:50.354 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956387+0000", 2024-09-18T10:15:50.354 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:50.354 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.354 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.354 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:50.354 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956395+0000", 2024-09-18T10:15:50.354 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.537e-06 2024-09-18T10:15:50.354 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.354 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.355 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:50.355 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956405+0000", 2024-09-18T10:15:50.355 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0057999999999999e-05 2024-09-18T10:15:50.355 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.355 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.355 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:50.355 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956416+0000", 2024-09-18T10:15:50.355 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1311999999999999e-05 2024-09-18T10:15:50.355 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.355 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.355 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:50.355 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956502+0000", 2024-09-18T10:15:50.355 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.5413000000000004e-05 2024-09-18T10:15:50.355 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.356 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.356 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:50.356 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956512+0000", 2024-09-18T10:15:50.356 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0299000000000001e-05 2024-09-18T10:15:50.356 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.356 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.356 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:50.356 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:24.167493+0000", 2024-09-18T10:15:50.356 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.21098060699999999 2024-09-18T10:15:50.356 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.356 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.356 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:50.356 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:24.167618+0000", 2024-09-18T10:15:50.357 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00012526599999999999 2024-09-18T10:15:50.357 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.357 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.357 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:50.357 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:24.168157+0000", 2024-09-18T10:15:50.357 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00053891400000000004 2024-09-18T10:15:50.357 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.357 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.357 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:50.357 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:24.168870+0000", 2024-09-18T10:15:50.357 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00071273300000000001 2024-09-18T10:15:50.357 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.357 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.358 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:50.358 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:24.569055+0000", 2024-09-18T10:15:50.358 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40018569599999998 2024-09-18T10:15:50.358 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.358 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.358 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:50.358 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:24.569156+0000", 2024-09-18T10:15:50.358 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000100738 2024-09-18T10:15:50.358 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:50.358 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:50.358 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:50.358 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.358 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.358 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:564 3.19s0 3:9fc2eacb:::smithi18925896-17:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e68)", 2024-09-18T10:15:50.359 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:23.956428+0000", 2024-09-18T10:15:50.359 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 26.292339877, 2024-09-18T10:15:50.359 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.91697059800000003, 2024-09-18T10:15:50.359 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:50.359 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:15:50.359 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:50.359 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:50.359 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:50.359 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 564 2024-09-18T10:15:50.359 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.359 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:50.359 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.359 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:50.359 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956428+0000", 2024-09-18T10:15:50.359 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:50.360 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.360 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.360 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:50.360 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956428+0000", 2024-09-18T10:15:50.360 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:50.360 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.360 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.360 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:50.360 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956427+0000", 2024-09-18T10:15:50.360 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:15:50.360 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.360 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.360 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:50.360 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956432+0000", 2024-09-18T10:15:50.360 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9980000000000003e-06 2024-09-18T10:15:50.361 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.361 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.361 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:50.361 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956446+0000", 2024-09-18T10:15:50.361 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3602e-05 2024-09-18T10:15:50.361 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.361 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.361 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:50.361 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956454+0000", 2024-09-18T10:15:50.361 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.2369999999999992e-06 2024-09-18T10:15:50.361 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.361 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.361 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:50.361 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956551+0000", 2024-09-18T10:15:50.362 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.7424999999999996e-05 2024-09-18T10:15:50.362 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.362 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.362 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:50.362 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956558+0000", 2024-09-18T10:15:50.362 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.8050000000000001e-06 2024-09-18T10:15:50.362 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.362 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.362 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:50.362 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:24.168428+0000", 2024-09-18T10:15:50.362 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.21186939299999999 2024-09-18T10:15:50.362 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.362 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.362 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for rw locks", 2024-09-18T10:15:50.363 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:24.168531+0000", 2024-09-18T10:15:50.363 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000103707 2024-09-18T10:15:50.363 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.363 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.363 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:50.363 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:24.669928+0000", 2024-09-18T10:15:50.363 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.50139628999999997 2024-09-18T10:15:50.363 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.363 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.363 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:50.363 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:24.670206+0000", 2024-09-18T10:15:50.363 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00027815000000000001 2024-09-18T10:15:50.363 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.363 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.363 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:50.364 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:24.873399+0000", 2024-09-18T10:15:50.364 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20319285200000001 2024-09-18T10:15:50.364 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:50.364 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:50.364 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:50.364 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:50.364 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:50.364 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-18T10:15:50.366 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-09-18T10:15:50.366 INFO:tasks.rados.rados.0.smithi189.stdout:369: finishing write tid 5 to smithi18925896-5 2024-09-18T10:15:50.366 INFO:tasks.rados.rados.0.smithi189.stdout:369: finishing write tid 6 to smithi18925896-5 2024-09-18T10:15:50.367 INFO:tasks.ceph.osd.3.smithi189.stderr:2024-09-18T10:15:50.362+0000 7f2440fc7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22396) UID: 0 2024-09-18T10:15:50.466 INFO:tasks.ceph.osd.14.smithi154.stderr:2024-09-18T10:15:50.462+0000 7f75a19e7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 22093) UID: 0 2024-09-18T10:15:50.474 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-18T10:15:50.474 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-18T10:15:50.474 INFO:teuthology.orchestra.run.smithi189.stdout: "num_ops": 0 2024-09-18T10:15:50.474 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-18T10:15:50.484 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-09-18T10:15:50.515 INFO:tasks.rados.rados.0.smithi189.stdout:369: finishing write tid 7 to smithi18925896-5 2024-09-18T10:15:50.516 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 5 v 113 (ObjNum 141 snap 31 seq_num 141) dirty exists 2024-09-18T10:15:50.516 INFO:tasks.rados.rados.0.smithi189.stdout:369: left oid 5 (ObjNum 141 snap 31 seq_num 141) 2024-09-18T10:15:50.567 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-18T10:15:50.562+0000 7f14dcc5b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22398) UID: 0 2024-09-18T10:15:50.588 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-18T10:15:50.588 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-18T10:15:50.588 INFO:teuthology.orchestra.run.smithi189.stdout: "complaint_time": 30, 2024-09-18T10:15:50.588 INFO:teuthology.orchestra.run.smithi189.stdout: "num_blocked_ops": 0 2024-09-18T10:15:50.589 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-18T10:15:50.597 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-09-18T10:15:50.668 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-09-18T10:15:50.666+0000 7fdfa8698700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22087) UID: 0 2024-09-18T10:15:50.695 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-18T10:15:50.696 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 20, 2024-09-18T10:15:50.696 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 600, 2024-09-18T10:15:50.696 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [ 2024-09-18T10:15:50.696 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.696 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:146 3.es0 3:78aad6f5:::smithi18925896-49:head [append 0~442368,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:50.696 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:51.117496+0000", 2024-09-18T10:15:50.696 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 59.574977318999998, 2024-09-18T10:15:50.696 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.61437150200000001, 2024-09-18T10:15:50.696 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:50.697 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:50.697 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:50.697 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:50.697 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:50.697 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 146 2024-09-18T10:15:50.697 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.697 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:50.697 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.697 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:50.697 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.117496+0000", 2024-09-18T10:15:50.697 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:50.697 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.697 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.697 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:50.698 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.117496+0000", 2024-09-18T10:15:50.698 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:50.698 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.698 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.698 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:50.698 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.117493+0000", 2024-09-18T10:15:50.698 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999976 2024-09-18T10:15:50.698 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.698 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.698 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:50.698 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.117787+0000", 2024-09-18T10:15:50.698 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00029378100000000001 2024-09-18T10:15:50.698 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.699 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.699 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:50.699 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.117820+0000", 2024-09-18T10:15:50.699 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3073e-05 2024-09-18T10:15:50.699 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.699 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.699 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:50.699 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.117834+0000", 2024-09-18T10:15:50.699 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3369e-05 2024-09-18T10:15:50.699 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.699 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.699 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:50.699 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.127090+0000", 2024-09-18T10:15:50.700 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0092563769999999997 2024-09-18T10:15:50.700 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.700 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.700 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:50.700 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.127435+0000", 2024-09-18T10:15:50.700 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00034522300000000003 2024-09-18T10:15:50.700 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.700 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.700 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:50.700 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.129115+0000", 2024-09-18T10:15:50.700 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0016792980000000001 2024-09-18T10:15:50.700 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.700 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.700 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:50.701 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.430505+0000", 2024-09-18T10:15:50.701 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.30139002300000001 2024-09-18T10:15:50.701 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.701 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.701 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:50.701 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.731753+0000", 2024-09-18T10:15:50.701 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.30124798800000002 2024-09-18T10:15:50.701 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.701 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.701 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:50.701 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.731867+0000", 2024-09-18T10:15:50.701 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000114764 2024-09-18T10:15:50.701 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:50.702 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:50.702 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:50.702 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.702 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.702 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:147 3.es0 3:78aad6f5:::smithi18925896-49:head [append 0~376832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:50.702 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:51.122092+0000", 2024-09-18T10:15:50.702 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 59.570380901, 2024-09-18T10:15:50.702 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.61047882200000003, 2024-09-18T10:15:50.702 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:50.702 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:50.702 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:50.702 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:50.702 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:50.702 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 147 2024-09-18T10:15:50.703 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.703 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:50.703 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.703 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:50.703 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122092+0000", 2024-09-18T10:15:50.703 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:50.703 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.703 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.703 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:50.703 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122092+0000", 2024-09-18T10:15:50.703 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:50.703 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.703 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.703 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:50.704 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122091+0000", 2024-09-18T10:15:50.704 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:50.704 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.704 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.704 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:50.704 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122504+0000", 2024-09-18T10:15:50.704 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00041322100000000002 2024-09-18T10:15:50.704 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.704 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.704 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:50.704 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122527+0000", 2024-09-18T10:15:50.704 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3504999999999999e-05 2024-09-18T10:15:50.704 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.705 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.705 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:50.705 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122537+0000", 2024-09-18T10:15:50.705 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.8460000000000003e-06 2024-09-18T10:15:50.705 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.705 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.705 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:50.705 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.327415+0000", 2024-09-18T10:15:50.705 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20487799700000001 2024-09-18T10:15:50.705 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.705 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.705 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:50.705 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.327614+0000", 2024-09-18T10:15:50.705 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00019855400000000001 2024-09-18T10:15:50.706 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.706 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.706 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:50.706 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.328920+0000", 2024-09-18T10:15:50.706 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001306263 2024-09-18T10:15:50.706 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.706 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.706 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:50.706 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.430570+0000", 2024-09-18T10:15:50.706 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.101650508 2024-09-18T10:15:50.706 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.706 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.706 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:50.706 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.732477+0000", 2024-09-18T10:15:50.707 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.30190700100000001 2024-09-18T10:15:50.707 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.707 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.707 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:50.707 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.732571+0000", 2024-09-18T10:15:50.707 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.3580999999999997e-05 2024-09-18T10:15:50.707 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:50.707 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:50.707 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:50.707 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.707 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.707 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:148 3.es0 3:78aad6f5:::smithi18925896-49:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:50.707 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:51.122547+0000", 2024-09-18T10:15:50.708 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 59.569925808999997, 2024-09-18T10:15:50.708 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.91462763400000002, 2024-09-18T10:15:50.708 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:50.708 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:50.708 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:50.708 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:50.708 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:50.708 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 148 2024-09-18T10:15:50.708 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.708 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:50.708 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.708 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:50.708 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122547+0000", 2024-09-18T10:15:50.708 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:50.709 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.709 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.709 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:50.709 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122547+0000", 2024-09-18T10:15:50.709 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:50.709 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.709 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.709 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:50.709 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122546+0000", 2024-09-18T10:15:50.709 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:15:50.709 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.709 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.709 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:50.709 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122552+0000", 2024-09-18T10:15:50.710 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3569999999999997e-06 2024-09-18T10:15:50.710 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.710 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.710 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:50.710 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122560+0000", 2024-09-18T10:15:50.710 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.191e-06 2024-09-18T10:15:50.710 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.710 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.710 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:50.710 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122569+0000", 2024-09-18T10:15:50.710 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.8230000000000001e-06 2024-09-18T10:15:50.710 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.710 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.710 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:50.710 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.834492+0000", 2024-09-18T10:15:50.711 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.71192345599999995 2024-09-18T10:15:50.711 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.711 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.711 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:50.711 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.834669+0000", 2024-09-18T10:15:50.711 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000177035 2024-09-18T10:15:50.711 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.711 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.711 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:50.711 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.835442+0000", 2024-09-18T10:15:50.711 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00077327799999999996 2024-09-18T10:15:50.711 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.711 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.712 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:50.712 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.936574+0000", 2024-09-18T10:15:50.712 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.101131163 2024-09-18T10:15:50.712 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.712 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.712 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:50.712 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.037028+0000", 2024-09-18T10:15:50.712 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100454026 2024-09-18T10:15:50.712 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.712 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.712 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:50.712 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.037175+0000", 2024-09-18T10:15:50.712 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000147305 2024-09-18T10:15:50.712 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:50.713 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:50.713 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:50.713 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.713 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.713 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:149 3.es0 3:78aad6f5:::smithi18925896-49:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:50.713 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:51.122575+0000", 2024-09-18T10:15:50.713 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 59.569898600999998, 2024-09-18T10:15:50.713 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.016813872, 2024-09-18T10:15:50.713 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:50.713 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:15:50.713 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:50.713 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:50.713 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:50.714 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 149 2024-09-18T10:15:50.714 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.714 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:50.714 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.714 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:50.714 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122575+0000", 2024-09-18T10:15:50.714 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:50.714 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.714 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.714 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:50.714 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122575+0000", 2024-09-18T10:15:50.714 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:50.714 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.714 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.715 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:50.715 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122574+0000", 2024-09-18T10:15:50.715 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:15:50.715 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.715 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.715 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:50.715 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122579+0000", 2024-09-18T10:15:50.715 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3650000000000003e-06 2024-09-18T10:15:50.715 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.715 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.715 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:50.715 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122587+0000", 2024-09-18T10:15:50.715 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.3000000000000002e-06 2024-09-18T10:15:50.716 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.716 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.716 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:50.716 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122592+0000", 2024-09-18T10:15:50.716 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9080000000000003e-06 2024-09-18T10:15:50.716 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.716 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.716 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:50.716 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.935931+0000", 2024-09-18T10:15:50.716 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.81333878199999998 2024-09-18T10:15:50.716 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.716 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.716 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for rw locks", 2024-09-18T10:15:50.716 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.936216+0000", 2024-09-18T10:15:50.717 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00028463400000000003 2024-09-18T10:15:50.717 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.717 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.717 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:50.717 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.038007+0000", 2024-09-18T10:15:50.717 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.101791539 2024-09-18T10:15:50.717 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.717 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.717 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:50.717 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.038254+0000", 2024-09-18T10:15:50.717 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000246932 2024-09-18T10:15:50.717 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.717 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.717 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:50.718 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.139388+0000", 2024-09-18T10:15:50.718 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10113417199999999 2024-09-18T10:15:50.718 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:50.718 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:50.718 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:50.718 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.718 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.718 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpRead(3.es0 36/32 ECSubRead(tid=3, to_read={3:7255ddf4:::smithi18925896-39:head=0,4096,0}, subchunks={3:7255ddf4:::smithi18925896-39:head=[0,1]}, attrs_to_read=))", 2024-09-18T10:15:50.718 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:51.126838+0000", 2024-09-18T10:15:50.718 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 59.565635178000001, 2024-09-18T10:15:50.718 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.707585825, 2024-09-18T10:15:50.718 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:50.718 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:50.719 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:50.719 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.719 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:50.719 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.126838+0000", 2024-09-18T10:15:50.719 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:50.719 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.719 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.719 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:50.719 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.126838+0000", 2024-09-18T10:15:50.719 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:50.719 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.719 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.719 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:50.719 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.126838+0000", 2024-09-18T10:15:50.720 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:50.720 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.720 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.720 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:50.720 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.126838+0000", 2024-09-18T10:15:50.720 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:50.720 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.720 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.720 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:50.720 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.127004+0000", 2024-09-18T10:15:50.720 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00016626499999999999 2024-09-18T10:15:50.720 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.720 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.721 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:50.721 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.227100+0000", 2024-09-18T10:15:50.721 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.1000955 2024-09-18T10:15:50.721 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.721 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.721 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:50.721 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.834083+0000", 2024-09-18T10:15:50.721 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.60698300100000002 2024-09-18T10:15:50.721 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.721 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.721 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:50.721 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.834424+0000", 2024-09-18T10:15:50.721 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000341059 2024-09-18T10:15:50.721 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:50.722 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:50.722 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:50.722 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.722 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.722 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpReadReply(3.es0 36/32 ECSubReadReply(tid=3, attrs_read=0))", 2024-09-18T10:15:50.722 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:51.214895+0000", 2024-09-18T10:15:50.722 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 59.477578500999996, 2024-09-18T10:15:50.722 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.61857800100000004, 2024-09-18T10:15:50.722 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:50.722 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:50.722 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:50.722 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.722 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:50.722 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.214895+0000", 2024-09-18T10:15:50.722 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:50.723 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.723 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.723 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:50.723 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.214895+0000", 2024-09-18T10:15:50.723 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:50.723 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.723 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.723 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:50.723 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.214893+0000", 2024-09-18T10:15:50.723 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:50.723 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.723 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.723 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:50.723 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.214927+0000", 2024-09-18T10:15:50.724 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3841999999999999e-05 2024-09-18T10:15:50.724 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.724 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.724 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:50.724 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.214953+0000", 2024-09-18T10:15:50.724 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.6519000000000001e-05 2024-09-18T10:15:50.724 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.724 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.724 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:50.724 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.214971+0000", 2024-09-18T10:15:50.724 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7535999999999999e-05 2024-09-18T10:15:50.724 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.724 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.725 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:50.725 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.833397+0000", 2024-09-18T10:15:50.725 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.61842642599999997 2024-09-18T10:15:50.725 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.725 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.725 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:50.725 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.833473+0000", 2024-09-18T10:15:50.725 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.5621e-05 2024-09-18T10:15:50.725 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:50.725 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:50.725 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:50.725 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.725 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.725 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpReadReply(3.es0 36/32 ECSubReadReply(tid=3, attrs_read=0))", 2024-09-18T10:15:50.726 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:51.227596+0000", 2024-09-18T10:15:50.726 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 59.464876949999997, 2024-09-18T10:15:50.726 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.60597977400000003, 2024-09-18T10:15:50.726 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:50.726 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:50.726 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:50.726 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.726 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:50.726 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.227596+0000", 2024-09-18T10:15:50.726 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:50.726 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.726 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.726 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:50.726 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.227596+0000", 2024-09-18T10:15:50.727 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:50.727 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.727 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.727 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:50.727 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.227595+0000", 2024-09-18T10:15:50.727 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:15:50.727 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.727 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.727 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:50.727 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.227605+0000", 2024-09-18T10:15:50.727 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.7040000000000006e-06 2024-09-18T10:15:50.727 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.727 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.727 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:50.727 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.227621+0000", 2024-09-18T10:15:50.728 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6107999999999999e-05 2024-09-18T10:15:50.728 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.728 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.728 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:50.728 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.227637+0000", 2024-09-18T10:15:50.728 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5282999999999999e-05 2024-09-18T10:15:50.728 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.728 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.728 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:50.728 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.833511+0000", 2024-09-18T10:15:50.728 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.60587403699999998 2024-09-18T10:15:50.728 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.728 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.729 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:50.729 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.833576+0000", 2024-09-18T10:15:50.729 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.5392e-05 2024-09-18T10:15:50.729 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:50.729 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:50.729 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:50.729 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.729 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.729 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.es0 36/32 ECSubWriteReply(tid=4, last_complete=36'3, committed=1, applied=1))", 2024-09-18T10:15:50.729 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:51.233227+0000", 2024-09-18T10:15:50.729 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 59.459245936000002, 2024-09-18T10:15:50.729 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.39820466900000001, 2024-09-18T10:15:50.729 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:50.729 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:50.730 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:50.730 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.730 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:50.730 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.233227+0000", 2024-09-18T10:15:50.730 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:50.730 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.730 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.730 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:50.730 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.233227+0000", 2024-09-18T10:15:50.730 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:50.730 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.730 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.730 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:50.730 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.233226+0000", 2024-09-18T10:15:50.731 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:15:50.731 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.731 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.731 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:50.731 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.233232+0000", 2024-09-18T10:15:50.731 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9370000000000001e-06 2024-09-18T10:15:50.731 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.731 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.731 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:50.731 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.233248+0000", 2024-09-18T10:15:50.731 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5574999999999999e-05 2024-09-18T10:15:50.731 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.731 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.731 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:50.732 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.233254+0000", 2024-09-18T10:15:50.732 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.3659999999999997e-06 2024-09-18T10:15:50.732 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.732 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.732 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:50.732 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.631420+0000", 2024-09-18T10:15:50.732 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.39816591699999998 2024-09-18T10:15:50.732 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.732 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.732 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:50.732 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.631432+0000", 2024-09-18T10:15:50.732 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1737e-05 2024-09-18T10:15:50.732 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:50.732 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:50.733 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:50.733 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.733 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.733 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.es0 36/32 ECSubWriteReply(tid=5, last_complete=36'4, committed=1, applied=1))", 2024-09-18T10:15:50.733 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:51.331048+0000", 2024-09-18T10:15:50.733 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 59.361425509, 2024-09-18T10:15:50.733 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40123123900000002, 2024-09-18T10:15:50.733 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:50.733 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:50.733 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:50.733 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.733 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:50.733 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.331048+0000", 2024-09-18T10:15:50.734 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:50.734 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.734 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.734 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:50.734 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.331048+0000", 2024-09-18T10:15:50.734 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:50.734 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.734 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.734 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:50.734 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.331046+0000", 2024-09-18T10:15:50.734 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:50.734 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.734 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.734 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:50.735 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.331057+0000", 2024-09-18T10:15:50.735 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1683e-05 2024-09-18T10:15:50.735 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.735 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.735 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:50.735 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.331090+0000", 2024-09-18T10:15:50.735 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2576000000000001e-05 2024-09-18T10:15:50.735 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.735 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.735 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:50.735 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.331107+0000", 2024-09-18T10:15:50.735 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7382999999999999e-05 2024-09-18T10:15:50.735 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.735 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.736 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:50.736 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.732241+0000", 2024-09-18T10:15:50.736 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40113375200000001 2024-09-18T10:15:50.736 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.736 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.736 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:50.736 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.732279+0000", 2024-09-18T10:15:50.736 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7719000000000003e-05 2024-09-18T10:15:50.736 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:50.736 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:50.736 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:50.736 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.736 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.736 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.es0 36/32 ECSubWriteReply(tid=5, last_complete=36'4, committed=1, applied=1))", 2024-09-18T10:15:50.737 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:51.331214+0000", 2024-09-18T10:15:50.737 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 59.361259115000003, 2024-09-18T10:15:50.737 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40084109800000001, 2024-09-18T10:15:50.737 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:50.737 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:50.737 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:50.737 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.737 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:50.737 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.331214+0000", 2024-09-18T10:15:50.737 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:50.737 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.737 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.737 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:50.737 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.331214+0000", 2024-09-18T10:15:50.738 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:50.738 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.738 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.738 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:50.738 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.331213+0000", 2024-09-18T10:15:50.738 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:15:50.738 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.738 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.738 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:50.738 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.331220+0000", 2024-09-18T10:15:50.738 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.1369999999999996e-06 2024-09-18T10:15:50.738 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.738 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.738 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:50.739 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.331235+0000", 2024-09-18T10:15:50.740 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4849e-05 2024-09-18T10:15:50.741 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.741 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.741 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:50.741 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.331243+0000", 2024-09-18T10:15:50.741 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.157e-06 2024-09-18T10:15:50.741 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.741 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.741 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:50.741 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.732010+0000", 2024-09-18T10:15:50.741 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40076711300000001 2024-09-18T10:15:50.741 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.741 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.741 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:50.742 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.732055+0000", 2024-09-18T10:15:50.742 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4821999999999999e-05 2024-09-18T10:15:50.742 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:50.742 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:50.742 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:50.742 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.742 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.742 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:347 3.es0 3:7255ddf4:::smithi18925896-39:head [append 0~376832] snapc b=[b,4] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-18T10:15:50.742 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:02.898980+0000", 2024-09-18T10:15:50.742 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 47.793493091000002, 2024-09-18T10:15:50.742 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.50845237799999998, 2024-09-18T10:15:50.742 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:50.742 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:50.742 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:50.743 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:50.743 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:50.743 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 347 2024-09-18T10:15:50.743 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.743 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:50.743 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.743 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:50.743 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.898980+0000", 2024-09-18T10:15:50.743 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:50.743 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.743 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.743 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:50.743 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.898980+0000", 2024-09-18T10:15:50.743 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:50.744 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.744 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.744 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:50.744 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.898978+0000", 2024-09-18T10:15:50.744 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:50.744 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.744 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.744 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:50.744 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.899378+0000", 2024-09-18T10:15:50.744 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000399465 2024-09-18T10:15:50.744 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.744 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.744 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:50.745 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.899397+0000", 2024-09-18T10:15:50.745 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9219999999999999e-05 2024-09-18T10:15:50.745 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.745 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.745 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:50.745 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.899410+0000", 2024-09-18T10:15:50.745 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2486e-05 2024-09-18T10:15:50.745 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.745 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.745 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:50.745 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.103248+0000", 2024-09-18T10:15:50.745 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20383807200000001 2024-09-18T10:15:50.745 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.746 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.746 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:50.746 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.103447+0000", 2024-09-18T10:15:50.746 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00019970400000000001 2024-09-18T10:15:50.746 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.746 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.746 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:50.746 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.104894+0000", 2024-09-18T10:15:50.746 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001447083 2024-09-18T10:15:50.746 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.746 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.746 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:50.746 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.209378+0000", 2024-09-18T10:15:50.746 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.104483942 2024-09-18T10:15:50.746 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.747 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.747 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:50.747 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.407310+0000", 2024-09-18T10:15:50.747 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.197931406 2024-09-18T10:15:50.747 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.747 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.747 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:50.747 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.407432+0000", 2024-09-18T10:15:50.747 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00012266999999999999 2024-09-18T10:15:50.747 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:50.747 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:50.747 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:50.747 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.748 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.748 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:348 3.es0 3:7255ddf4:::smithi18925896-39:head [setxattr _header (56) in=7b] snapc b=[b,4] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-18T10:15:50.748 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:02.899423+0000", 2024-09-18T10:15:50.748 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 47.793049662999998, 2024-09-18T10:15:50.748 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.50859120400000002, 2024-09-18T10:15:50.748 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:50.748 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:50.748 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:50.748 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:50.748 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:50.748 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 348 2024-09-18T10:15:50.748 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.748 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:50.748 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.749 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:50.749 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.899423+0000", 2024-09-18T10:15:50.749 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:50.749 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.749 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.749 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:50.749 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.899423+0000", 2024-09-18T10:15:50.749 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:50.749 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.750 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.750 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:50.750 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.899422+0000", 2024-09-18T10:15:50.750 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:50.750 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.750 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.750 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:50.750 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.899431+0000", 2024-09-18T10:15:50.750 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.5960000000000004e-06 2024-09-18T10:15:50.750 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.750 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.750 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:50.750 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.899451+0000", 2024-09-18T10:15:50.750 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0089999999999999e-05 2024-09-18T10:15:50.751 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.751 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.751 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:50.751 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.899461+0000", 2024-09-18T10:15:50.751 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0373e-05 2024-09-18T10:15:50.751 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.751 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.751 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:50.751 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.106191+0000", 2024-09-18T10:15:50.751 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20672991700000001 2024-09-18T10:15:50.751 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.751 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.751 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:50.752 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.106370+0000", 2024-09-18T10:15:50.752 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00017880500000000001 2024-09-18T10:15:50.752 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.752 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.752 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:50.752 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.107175+0000", 2024-09-18T10:15:50.752 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00080489699999999995 2024-09-18T10:15:50.752 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.752 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.752 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:50.752 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.209383+0000", 2024-09-18T10:15:50.752 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10220804999999999 2024-09-18T10:15:50.752 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.752 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.753 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:50.753 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.407801+0000", 2024-09-18T10:15:50.753 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.19841824199999999 2024-09-18T10:15:50.753 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.753 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.753 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:50.753 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.408015+0000", 2024-09-18T10:15:50.753 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00021364100000000001 2024-09-18T10:15:50.753 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:50.753 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:50.753 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:50.753 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.753 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.753 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:349 3.es0 3:7255ddf4:::smithi18925896-39:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e47)", 2024-09-18T10:15:50.754 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:02.899474+0000", 2024-09-18T10:15:50.754 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 47.792999451999997, 2024-09-18T10:15:50.754 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.81294787599999996, 2024-09-18T10:15:50.754 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:50.754 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:15:50.754 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:50.754 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:50.754 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:50.754 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 349 2024-09-18T10:15:50.754 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.754 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:50.754 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.754 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:50.755 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.899474+0000", 2024-09-18T10:15:50.755 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:50.755 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.755 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.755 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:50.755 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.899474+0000", 2024-09-18T10:15:50.755 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:50.755 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.755 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.755 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:50.755 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.899472+0000", 2024-09-18T10:15:50.755 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:50.755 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.755 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.756 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:50.756 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.899480+0000", 2024-09-18T10:15:50.756 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.0129999999999993e-06 2024-09-18T10:15:50.756 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.756 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.756 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:50.756 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.899501+0000", 2024-09-18T10:15:50.756 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0959999999999999e-05 2024-09-18T10:15:50.756 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.756 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.756 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:50.756 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.899513+0000", 2024-09-18T10:15:50.756 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1736000000000001e-05 2024-09-18T10:15:50.757 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.757 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.757 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:50.757 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.208436+0000", 2024-09-18T10:15:50.757 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.30892317800000002 2024-09-18T10:15:50.757 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.757 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.757 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for rw locks", 2024-09-18T10:15:50.757 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.208762+0000", 2024-09-18T10:15:50.757 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00032566099999999998 2024-09-18T10:15:50.757 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.757 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.757 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:50.757 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.408786+0000", 2024-09-18T10:15:50.758 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.200024389 2024-09-18T10:15:50.758 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.758 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.758 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:50.758 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.409088+0000", 2024-09-18T10:15:50.758 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000301272 2024-09-18T10:15:50.758 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.758 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.758 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:50.758 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.712422+0000", 2024-09-18T10:15:50.758 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.303333937 2024-09-18T10:15:50.758 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:50.758 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:50.759 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:50.759 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.759 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.759 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.17s1 57/40 ECSubWrite(tid=106, reqid=client.4632.0:441, at_version=57'22, trim_to=0'0, roll_forward_to=53'18))", 2024-09-18T10:15:50.759 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:12.963189+0000", 2024-09-18T10:15:50.759 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 37.729283735999999, 2024-09-18T10:15:50.759 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.404840753, 2024-09-18T10:15:50.759 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:50.759 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:50.759 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:50.759 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.759 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:50.759 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:12.963189+0000", 2024-09-18T10:15:50.759 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:50.760 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.760 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.760 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:50.760 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:12.963189+0000", 2024-09-18T10:15:50.760 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:50.760 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.760 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.760 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:50.760 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:12.963189+0000", 2024-09-18T10:15:50.760 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:15:50.760 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.760 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.760 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:50.760 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:12.963511+0000", 2024-09-18T10:15:50.761 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00032195800000000001 2024-09-18T10:15:50.761 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.761 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.761 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:50.761 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:12.963536+0000", 2024-09-18T10:15:50.761 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5795e-05 2024-09-18T10:15:50.761 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.761 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.761 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:50.761 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:12.963548+0000", 2024-09-18T10:15:50.761 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1235e-05 2024-09-18T10:15:50.761 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.761 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.761 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:50.762 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.164194+0000", 2024-09-18T10:15:50.762 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.200646044 2024-09-18T10:15:50.762 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.762 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.762 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:50.762 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.164208+0000", 2024-09-18T10:15:50.762 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4107000000000001e-05 2024-09-18T10:15:50.762 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.762 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.762 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:50.762 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.367962+0000", 2024-09-18T10:15:50.762 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20375375800000001 2024-09-18T10:15:50.762 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.762 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.763 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:50.763 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.368030+0000", 2024-09-18T10:15:50.763 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.8664999999999994e-05 2024-09-18T10:15:50.763 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:50.763 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:50.763 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:50.763 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.763 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.763 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.17s1 57/40 ECSubWrite(tid=107, reqid=client.4632.0:442, at_version=57'23, trim_to=0'0, roll_forward_to=53'18))", 2024-09-18T10:15:50.763 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:12.963562+0000", 2024-09-18T10:15:50.763 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 37.728911017000001, 2024-09-18T10:15:50.763 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40594254800000001, 2024-09-18T10:15:50.763 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:50.763 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:50.764 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:50.764 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.764 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:50.764 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:12.963562+0000", 2024-09-18T10:15:50.764 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:50.764 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.764 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.764 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:50.764 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:12.963562+0000", 2024-09-18T10:15:50.764 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:50.764 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.764 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.764 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:50.764 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:12.963561+0000", 2024-09-18T10:15:50.765 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:50.765 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.765 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.765 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:50.765 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:12.963869+0000", 2024-09-18T10:15:50.765 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00030785799999999999 2024-09-18T10:15:50.765 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.765 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.765 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:50.765 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:12.963886+0000", 2024-09-18T10:15:50.765 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6582000000000001e-05 2024-09-18T10:15:50.765 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.765 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.765 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:50.766 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.063959+0000", 2024-09-18T10:15:50.766 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10007302699999999 2024-09-18T10:15:50.766 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.766 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.766 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:50.766 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.166049+0000", 2024-09-18T10:15:50.766 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.102089845 2024-09-18T10:15:50.766 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.766 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.766 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:50.766 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.166064+0000", 2024-09-18T10:15:50.766 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5287000000000001e-05 2024-09-18T10:15:50.766 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.766 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.767 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:50.767 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.369452+0000", 2024-09-18T10:15:50.767 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20338777 2024-09-18T10:15:50.767 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.767 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.767 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:50.767 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.369505+0000", 2024-09-18T10:15:50.767 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.2868000000000003e-05 2024-09-18T10:15:50.767 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:50.767 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:50.767 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:50.767 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.767 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.768 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:495 3.es0 3:7255ddf4:::smithi18925896-39:head [rmxattr R^`al]TqifatYOiOU]p`ZegfXmNsXVaZcdd]b]YX in=40b,rmxattr TNQYR^jgR`pPaYitmrcliV[dttobncTntcfTlQkt in=40b,rmxattr W_aq[RVsaUpQtX_RpV^Vpqrg`QgQViPeVaRqZSmf in=40b,rmxattr aYfgr^q[\\aPg_aZsakfUfsdhYbra^QXjlkrpdY\\\\ in=40b,rmxattr dgXPi in=5b,rmxattr p]OS_Znl[dNa^qTn`WW[j\\dtgbNac_bijYmg in=36b,rmxattr tZ]grhlaq]kUO[k^d^dr``XNio^^Oaebpj\\[Ytdq in=40b] snapc 19=[19,18,16,12,11,10,f,e] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T10:15:50.768 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:17.889335+0000", 2024-09-18T10:15:50.768 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 32.803137784, 2024-09-18T10:15:50.768 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.42846536899999998, 2024-09-18T10:15:50.768 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:50.768 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:50.768 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:50.768 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:50.768 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:50.768 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 495 2024-09-18T10:15:50.768 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.768 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:50.768 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.768 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:50.769 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889335+0000", 2024-09-18T10:15:50.769 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:50.769 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.769 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.769 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:50.769 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889335+0000", 2024-09-18T10:15:50.769 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:50.769 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.769 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.769 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:50.769 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889333+0000", 2024-09-18T10:15:50.769 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999976 2024-09-18T10:15:50.770 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.770 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.770 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:50.770 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889351+0000", 2024-09-18T10:15:50.770 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7795000000000002e-05 2024-09-18T10:15:50.770 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.770 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.770 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:50.770 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889383+0000", 2024-09-18T10:15:50.770 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2740999999999997e-05 2024-09-18T10:15:50.770 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.770 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.770 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:50.770 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889396+0000", 2024-09-18T10:15:50.771 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2926e-05 2024-09-18T10:15:50.771 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.771 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.771 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:50.771 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.989725+0000", 2024-09-18T10:15:50.771 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10032845999999999 2024-09-18T10:15:50.771 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.771 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.771 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "op must wait for map", 2024-09-18T10:15:50.771 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.989751+0000", 2024-09-18T10:15:50.771 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5819000000000001e-05 2024-09-18T10:15:50.771 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.771 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.771 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:50.772 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.999619+0000", 2024-09-18T10:15:50.772 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0098683920000000001 2024-09-18T10:15:50.772 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.772 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.772 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:50.772 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.999776+0000", 2024-09-18T10:15:50.772 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000157336 2024-09-18T10:15:50.772 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.772 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.772 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:50.772 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.001374+0000", 2024-09-18T10:15:50.772 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0015982419999999999 2024-09-18T10:15:50.772 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.773 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.773 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:50.773 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.003908+0000", 2024-09-18T10:15:50.773 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0025331870000000001 2024-09-18T10:15:50.773 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.773 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.773 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:50.773 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.317708+0000", 2024-09-18T10:15:50.773 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.31380064800000002 2024-09-18T10:15:50.773 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.773 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.773 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:50.773 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.317801+0000", 2024-09-18T10:15:50.773 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.2427000000000001e-05 2024-09-18T10:15:50.774 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:50.774 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:50.774 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:50.774 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.774 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.774 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:646 3.es0 3:78aad6f5:::smithi18925896-49:head [append 0~0] snapc 21=[20,1f,1e,1a,16,11,10] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-18T10:15:50.774 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:29.784316+0000", 2024-09-18T10:15:50.774 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 20.908156701999999, 2024-09-18T10:15:50.774 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.38893830699999998, 2024-09-18T10:15:50.774 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:50.774 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:50.774 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:50.774 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:50.775 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:50.775 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 646 2024-09-18T10:15:50.775 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.775 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:50.775 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.775 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:50.775 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:29.784316+0000", 2024-09-18T10:15:50.775 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:50.775 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.775 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.775 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:50.775 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:29.784316+0000", 2024-09-18T10:15:50.775 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:50.775 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.776 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.776 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:50.776 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:29.784314+0000", 2024-09-18T10:15:50.776 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999976 2024-09-18T10:15:50.776 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.776 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.776 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:50.776 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:29.784325+0000", 2024-09-18T10:15:50.776 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0869e-05 2024-09-18T10:15:50.776 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.776 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.776 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:50.776 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:29.784359+0000", 2024-09-18T10:15:50.777 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4496000000000003e-05 2024-09-18T10:15:50.777 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.777 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.777 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:50.777 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:29.784374+0000", 2024-09-18T10:15:50.777 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4945e-05 2024-09-18T10:15:50.777 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.777 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.777 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:50.777 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:29.870174+0000", 2024-09-18T10:15:50.777 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.085799399999999998 2024-09-18T10:15:50.777 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.777 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.777 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:50.778 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:29.870387+0000", 2024-09-18T10:15:50.778 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00021327799999999999 2024-09-18T10:15:50.778 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.778 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.778 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:50.778 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:29.871229+0000", 2024-09-18T10:15:50.778 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00084239500000000001 2024-09-18T10:15:50.778 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.778 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.778 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:50.779 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.172680+0000", 2024-09-18T10:15:50.779 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.301450944 2024-09-18T10:15:50.779 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.779 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.779 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:50.779 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.173021+0000", 2024-09-18T10:15:50.779 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00034057199999999998 2024-09-18T10:15:50.779 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.779 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.779 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:50.779 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.173255+0000", 2024-09-18T10:15:50.779 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000233945 2024-09-18T10:15:50.779 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:50.780 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:50.780 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:50.780 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.780 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.780 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:647 3.es0 3:78aad6f5:::smithi18925896-49:head [rollback 10] snapc 21=[20,1f,1e,1a,16,11,10] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-18T10:15:50.780 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:29.784393+0000", 2024-09-18T10:15:50.780 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 20.908079788999999, 2024-09-18T10:15:50.780 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.38955653200000001, 2024-09-18T10:15:50.780 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:50.780 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:50.780 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:50.780 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:50.780 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:50.781 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 647 2024-09-18T10:15:50.781 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.781 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:50.781 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.781 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:50.781 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:29.784393+0000", 2024-09-18T10:15:50.781 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:50.781 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.781 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.781 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:50.781 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:29.784393+0000", 2024-09-18T10:15:50.781 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:50.781 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.782 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.782 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:50.782 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:29.784393+0000", 2024-09-18T10:15:50.782 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:15:50.782 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.782 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.782 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:50.782 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:29.784397+0000", 2024-09-18T10:15:50.782 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4290000000000003e-06 2024-09-18T10:15:50.782 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.782 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.782 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:50.782 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:29.784409+0000", 2024-09-18T10:15:50.782 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2208000000000001e-05 2024-09-18T10:15:50.782 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.783 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.783 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:50.783 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:29.784418+0000", 2024-09-18T10:15:50.783 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.1060000000000006e-06 2024-09-18T10:15:50.783 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.783 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.783 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:50.783 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:29.871905+0000", 2024-09-18T10:15:50.783 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.087486918999999996 2024-09-18T10:15:50.783 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.783 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.783 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:50.783 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:29.872024+0000", 2024-09-18T10:15:50.784 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000119152 2024-09-18T10:15:50.784 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.784 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.784 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:50.784 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:29.872743+0000", 2024-09-18T10:15:50.784 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00071839500000000004 2024-09-18T10:15:50.784 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.784 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.784 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:50.784 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.172683+0000", 2024-09-18T10:15:50.784 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.29994042399999998 2024-09-18T10:15:50.784 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.784 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.784 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:50.785 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.173719+0000", 2024-09-18T10:15:50.785 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0010353199999999999 2024-09-18T10:15:50.785 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.785 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.785 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:50.785 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.173950+0000", 2024-09-18T10:15:50.785 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00023138499999999999 2024-09-18T10:15:50.785 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:50.785 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:50.785 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:50.785 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.785 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.785 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:648 3.es0 3:78aad6f5:::smithi18925896-49:head [append 0~0] snapc 21=[20,1f,1e,1a,16,11,10] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-18T10:15:50.786 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:29.784506+0000", 2024-09-18T10:15:50.786 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 20.907967124999999, 2024-09-18T10:15:50.786 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.79270480700000001, 2024-09-18T10:15:50.786 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:50.786 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:50.786 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:50.786 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:50.786 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:50.786 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 648 2024-09-18T10:15:50.786 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.786 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:50.786 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.786 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:50.787 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:29.784506+0000", 2024-09-18T10:15:50.787 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:50.787 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.787 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.787 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:50.787 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:29.784506+0000", 2024-09-18T10:15:50.787 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:50.787 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.787 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.787 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:50.787 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:29.784505+0000", 2024-09-18T10:15:50.787 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:15:50.787 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.787 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.788 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:50.788 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:29.784510+0000", 2024-09-18T10:15:50.788 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9540000000000003e-06 2024-09-18T10:15:50.788 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.788 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.788 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:50.788 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:29.784523+0000", 2024-09-18T10:15:50.788 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2791e-05 2024-09-18T10:15:50.788 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.788 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.788 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:50.788 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:29.884663+0000", 2024-09-18T10:15:50.789 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10014081499999999 2024-09-18T10:15:50.789 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.789 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.789 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:50.789 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.175196+0000", 2024-09-18T10:15:50.789 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.29053214300000002 2024-09-18T10:15:50.789 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.789 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.789 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:50.789 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.175436+0000", 2024-09-18T10:15:50.789 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000240084 2024-09-18T10:15:50.789 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.789 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.790 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:50.790 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.176117+0000", 2024-09-18T10:15:50.790 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00068175400000000002 2024-09-18T10:15:50.790 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.790 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.790 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:50.790 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.277609+0000", 2024-09-18T10:15:50.790 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10149114100000001 2024-09-18T10:15:50.790 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.790 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.790 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:50.790 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.577146+0000", 2024-09-18T10:15:50.790 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.29953759600000002 2024-09-18T10:15:50.790 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.791 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.791 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:50.791 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.577211+0000", 2024-09-18T10:15:50.791 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.4630999999999998e-05 2024-09-18T10:15:50.791 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:50.791 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:50.791 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:50.791 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.791 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.791 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:694 3.es0 3:78aad6f5:::smithi18925896-49:head [stat,delete] snapc 2a=[2a,29,28,26,20,1f,1e,1a,11] ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-09-18T10:15:50.791 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:35.579838+0000", 2024-09-18T10:15:50.791 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 15.11263527, 2024-09-18T10:15:50.791 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40589915300000001, 2024-09-18T10:15:50.791 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:50.792 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:50.792 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:50.792 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:50.792 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:50.792 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 694 2024-09-18T10:15:50.792 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.792 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:50.792 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.792 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:50.792 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:35.579838+0000", 2024-09-18T10:15:50.792 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:50.792 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.792 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.792 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:50.793 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:35.579838+0000", 2024-09-18T10:15:50.793 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:50.793 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.793 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.793 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:50.793 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:35.579835+0000", 2024-09-18T10:15:50.793 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999971 2024-09-18T10:15:50.793 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.793 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.793 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:50.793 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:35.579849+0000", 2024-09-18T10:15:50.793 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3786e-05 2024-09-18T10:15:50.793 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.793 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.794 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:50.794 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:35.579886+0000", 2024-09-18T10:15:50.794 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7183e-05 2024-09-18T10:15:50.794 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.794 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.794 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:50.794 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:35.680021+0000", 2024-09-18T10:15:50.794 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10013464599999999 2024-09-18T10:15:50.794 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.794 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.794 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:50.794 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:35.686371+0000", 2024-09-18T10:15:50.794 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0063500789999999998 2024-09-18T10:15:50.794 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.795 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.795 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "op must wait for map", 2024-09-18T10:15:50.795 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:35.686388+0000", 2024-09-18T10:15:50.795 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7662000000000001e-05 2024-09-18T10:15:50.795 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.795 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.795 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:50.795 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:35.880602+0000", 2024-09-18T10:15:50.795 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.19421374999999999 2024-09-18T10:15:50.795 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.795 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.795 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:50.795 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:35.880897+0000", 2024-09-18T10:15:50.795 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00029453999999999998 2024-09-18T10:15:50.796 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.796 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.796 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:50.796 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:35.882269+0000", 2024-09-18T10:15:50.796 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00137261 2024-09-18T10:15:50.796 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.796 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.796 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:50.796 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:35.884030+0000", 2024-09-18T10:15:50.796 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0017602449999999999 2024-09-18T10:15:50.796 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.796 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.796 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:50.796 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:35.985612+0000", 2024-09-18T10:15:50.797 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.101582165 2024-09-18T10:15:50.797 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.797 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.797 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:50.797 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:35.985737+0000", 2024-09-18T10:15:50.797 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00012524699999999999 2024-09-18T10:15:50.797 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:50.797 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:50.797 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:50.797 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:50.797 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:50.797 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-18T10:15:50.799 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_ops_in_flight 2024-09-18T10:15:50.799 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-18T10:15:50.794+0000 7f14dcc5b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22398) UID: 0 2024-09-18T10:15:50.899 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-18T10:15:50.896+0000 7f00046a8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 22136) UID: 0 2024-09-18T10:15:50.952 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-18T10:15:50.952 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [ 2024-09-18T10:15:50.953 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.953 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:942 3.4s1 3.a6fd9864 (undecoded) ondisk+retry+write+known_if_redirected+full_force+supports_pool_eio e95)", 2024-09-18T10:15:50.953 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:50.784970+0000", 2024-09-18T10:15:50.953 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.16677254599999999, 2024-09-18T10:15:50.953 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.16682519400000001, 2024-09-18T10:15:50.953 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:50.953 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "queued for pg", 2024-09-18T10:15:50.953 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:50.953 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:50.953 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:50.953 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 942 2024-09-18T10:15:50.953 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.953 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:50.954 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.954 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:50.954 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:50.784970+0000", 2024-09-18T10:15:50.954 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:50.954 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.954 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.954 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:50.954 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:50.784970+0000", 2024-09-18T10:15:50.954 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:50.954 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.954 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.954 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:50.954 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:50.784967+0000", 2024-09-18T10:15:50.955 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999967 2024-09-18T10:15:50.955 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.955 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.955 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:50.955 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:50.784983+0000", 2024-09-18T10:15:50.955 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5849999999999999e-05 2024-09-18T10:15:50.955 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.955 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.955 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:50.955 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:50.785021+0000", 2024-09-18T10:15:50.955 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8039e-05 2024-09-18T10:15:50.955 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:50.955 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:50.955 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:50.956 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:50.785036+0000", 2024-09-18T10:15:50.956 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4790000000000001e-05 2024-09-18T10:15:50.956 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:50.956 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:50.956 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:50.956 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:50.956 INFO:teuthology.orchestra.run.smithi189.stdout: ], 2024-09-18T10:15:50.956 INFO:teuthology.orchestra.run.smithi189.stdout: "num_ops": 1 2024-09-18T10:15:50.956 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-18T10:15:50.962 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_blocked_ops 2024-09-18T10:15:51.000 INFO:tasks.ceph.osd.4.smithi029.stderr:2024-09-18T10:15:50.993+0000 7f550cf6e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22189) UID: 0 2024-09-18T10:15:51.059 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-18T10:15:51.060 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-18T10:15:51.060 INFO:teuthology.orchestra.run.smithi189.stdout: "complaint_time": 30, 2024-09-18T10:15:51.060 INFO:teuthology.orchestra.run.smithi189.stdout: "num_blocked_ops": 0 2024-09-18T10:15:51.060 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-18T10:15:51.069 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_historic_ops 2024-09-18T10:15:51.100 INFO:tasks.ceph.osd.12.smithi029.stderr:2024-09-18T10:15:51.093+0000 7f8e4e627700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 22190) UID: 0 2024-09-18T10:15:51.164 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-18T10:15:51.165 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 20, 2024-09-18T10:15:51.165 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 600, 2024-09-18T10:15:51.165 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [ 2024-09-18T10:15:51.165 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.165 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4257.0:19 1.0 e24/23 1:30306672:devicehealth::main.db.0000000000000000:head v 24'9, mlcod=24'7)", 2024-09-18T10:15:51.165 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:33.346192+0000", 2024-09-18T10:15:51.165 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 77.815107953999998, 2024-09-18T10:15:51.165 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20178458699999999, 2024-09-18T10:15:51.165 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:51.165 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:51.165 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:51.165 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.166 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:51.166 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:33.346192+0000", 2024-09-18T10:15:51.166 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:51.166 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.166 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.166 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:51.166 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:33.346192+0000", 2024-09-18T10:15:51.166 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:51.166 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.166 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.166 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:51.166 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:33.346190+0000", 2024-09-18T10:15:51.166 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:51.166 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.167 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.167 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:51.167 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:33.346206+0000", 2024-09-18T10:15:51.167 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6257e-05 2024-09-18T10:15:51.167 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.167 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.167 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:51.167 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:33.346237+0000", 2024-09-18T10:15:51.167 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0471999999999999e-05 2024-09-18T10:15:51.167 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.167 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.167 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:51.167 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:33.446377+0000", 2024-09-18T10:15:51.168 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10014044499999999 2024-09-18T10:15:51.168 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.168 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.168 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:51.168 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:33.546722+0000", 2024-09-18T10:15:51.168 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100344593 2024-09-18T10:15:51.168 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.168 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.168 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:51.168 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:33.546787+0000", 2024-09-18T10:15:51.168 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.5347999999999998e-05 2024-09-18T10:15:51.168 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.168 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.168 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:51.169 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:33.547891+0000", 2024-09-18T10:15:51.169 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001104379 2024-09-18T10:15:51.169 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.169 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.169 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:51.169 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:33.547976+0000", 2024-09-18T10:15:51.169 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.5023000000000002e-05 2024-09-18T10:15:51.169 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:51.169 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:51.169 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:51.169 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.169 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.169 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4257.0:48 1.0 e26/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 26'30, mlcod=26'22)", 2024-09-18T10:15:51.170 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:35.294144+0000", 2024-09-18T10:15:51.170 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 75.867155811000003, 2024-09-18T10:15:51.170 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20346468400000001, 2024-09-18T10:15:51.170 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:51.170 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:51.170 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:51.170 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.170 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:51.170 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:35.294144+0000", 2024-09-18T10:15:51.170 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:51.170 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.170 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.170 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:51.171 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:35.294144+0000", 2024-09-18T10:15:51.171 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:51.171 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.171 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.171 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:51.171 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:35.294142+0000", 2024-09-18T10:15:51.171 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:51.171 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.171 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.171 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:51.171 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:35.294156+0000", 2024-09-18T10:15:51.171 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4059e-05 2024-09-18T10:15:51.171 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.171 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.172 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:51.172 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:35.294209+0000", 2024-09-18T10:15:51.172 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.2164999999999999e-05 2024-09-18T10:15:51.172 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.172 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.172 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:51.172 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:35.394305+0000", 2024-09-18T10:15:51.172 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10009605000000001 2024-09-18T10:15:51.172 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.172 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.172 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:51.172 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:35.394422+0000", 2024-09-18T10:15:51.172 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000117093 2024-09-18T10:15:51.172 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.173 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.173 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:51.173 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:35.394460+0000", 2024-09-18T10:15:51.173 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8037999999999997e-05 2024-09-18T10:15:51.173 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.173 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.173 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:51.173 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:35.497437+0000", 2024-09-18T10:15:51.173 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.102977206 2024-09-18T10:15:51.173 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.173 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.173 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:51.173 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:35.497609+0000", 2024-09-18T10:15:51.173 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00017165299999999999 2024-09-18T10:15:51.174 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:51.174 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:51.174 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:51.174 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.174 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.174 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4257.0:96 1.0 e29/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 29'70, mlcod=29'68)", 2024-09-18T10:15:51.174 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:39.427232+0000", 2024-09-18T10:15:51.174 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 71.734067687000007, 2024-09-18T10:15:51.174 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20408627300000001, 2024-09-18T10:15:51.174 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:51.174 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:51.174 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:51.174 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.174 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:51.175 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.427232+0000", 2024-09-18T10:15:51.175 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:51.175 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.175 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.175 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:51.175 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.427232+0000", 2024-09-18T10:15:51.175 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:51.175 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.175 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.175 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:51.175 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.427230+0000", 2024-09-18T10:15:51.175 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:51.175 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.175 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.176 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:51.176 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.427399+0000", 2024-09-18T10:15:51.176 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000168787 2024-09-18T10:15:51.176 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.176 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.176 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:51.176 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.427431+0000", 2024-09-18T10:15:51.176 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2372000000000001e-05 2024-09-18T10:15:51.176 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.176 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.176 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:51.176 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.427444+0000", 2024-09-18T10:15:51.176 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2303000000000001e-05 2024-09-18T10:15:51.176 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.177 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.177 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:51.177 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.427580+0000", 2024-09-18T10:15:51.177 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000136019 2024-09-18T10:15:51.177 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.177 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.177 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:51.177 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.427622+0000", 2024-09-18T10:15:51.177 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.2551e-05 2024-09-18T10:15:51.177 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.177 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.177 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:51.177 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.631238+0000", 2024-09-18T10:15:51.177 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20361580100000001 2024-09-18T10:15:51.178 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.178 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.178 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:51.178 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.631318+0000", 2024-09-18T10:15:51.178 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.0370000000000005e-05 2024-09-18T10:15:51.178 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:51.178 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:51.178 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:51.178 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.178 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.178 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4257.0:97 1.0 e29/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 29'71, mlcod=29'68)", 2024-09-18T10:15:51.178 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:39.428800+0000", 2024-09-18T10:15:51.178 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 71.732499744999998, 2024-09-18T10:15:51.178 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.202566529, 2024-09-18T10:15:51.179 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:51.179 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:51.179 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:51.179 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.179 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:51.179 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.428800+0000", 2024-09-18T10:15:51.179 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:51.179 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.179 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.179 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:51.179 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.428800+0000", 2024-09-18T10:15:51.179 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:51.179 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.179 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.180 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:51.180 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.428798+0000", 2024-09-18T10:15:51.180 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:51.180 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.180 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.180 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:51.180 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.428814+0000", 2024-09-18T10:15:51.180 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5262999999999999e-05 2024-09-18T10:15:51.180 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.180 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.181 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:51.181 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.428836+0000", 2024-09-18T10:15:51.181 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.2096e-05 2024-09-18T10:15:51.181 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.181 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.181 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:51.181 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.428847+0000", 2024-09-18T10:15:51.181 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1272e-05 2024-09-18T10:15:51.181 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.181 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.181 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:51.181 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.529081+0000", 2024-09-18T10:15:51.181 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100233853 2024-09-18T10:15:51.182 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.182 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.182 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:51.182 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.529115+0000", 2024-09-18T10:15:51.182 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3862000000000002e-05 2024-09-18T10:15:51.182 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.182 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.182 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:51.182 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.631329+0000", 2024-09-18T10:15:51.182 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.102214465 2024-09-18T10:15:51.182 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.182 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.182 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:51.182 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.631367+0000", 2024-09-18T10:15:51.183 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7450999999999998e-05 2024-09-18T10:15:51.183 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:51.183 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:51.183 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:51.183 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.183 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.183 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4257.0:98 1.0 e29/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 29'72, mlcod=29'68)", 2024-09-18T10:15:51.183 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:39.429995+0000", 2024-09-18T10:15:51.183 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 71.731304522000002, 2024-09-18T10:15:51.183 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20141884500000001, 2024-09-18T10:15:51.183 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:51.183 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:51.183 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:51.183 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.184 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:51.184 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.429995+0000", 2024-09-18T10:15:51.184 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:51.184 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.184 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.184 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:51.184 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.429995+0000", 2024-09-18T10:15:51.184 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:51.184 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.184 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.184 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:51.184 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.429994+0000", 2024-09-18T10:15:51.184 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:51.185 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.185 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.185 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:51.185 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.430008+0000", 2024-09-18T10:15:51.185 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4185999999999999e-05 2024-09-18T10:15:51.185 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.185 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.185 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:51.185 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.430036+0000", 2024-09-18T10:15:51.185 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7878999999999999e-05 2024-09-18T10:15:51.185 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.185 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.185 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:51.185 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.430048+0000", 2024-09-18T10:15:51.185 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1725999999999999e-05 2024-09-18T10:15:51.186 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.186 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.186 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:51.186 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.629731+0000", 2024-09-18T10:15:51.186 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.19968341100000001 2024-09-18T10:15:51.186 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.186 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.186 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:51.186 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.629764+0000", 2024-09-18T10:15:51.186 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3572999999999999e-05 2024-09-18T10:15:51.186 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.186 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.186 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:51.187 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.631372+0000", 2024-09-18T10:15:51.187 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0016074279999999999 2024-09-18T10:15:51.187 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.187 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.187 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:51.187 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.631414+0000", 2024-09-18T10:15:51.187 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.2191999999999999e-05 2024-09-18T10:15:51.187 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:51.187 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:51.187 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:51.187 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.187 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.187 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4257.0:103 1.0 e29/23 1:30306672:devicehealth::main.db.0000000000000000:head v 29'77, mlcod=29'73)", 2024-09-18T10:15:51.187 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:40.036738+0000", 2024-09-18T10:15:51.188 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 71.124561739000001, 2024-09-18T10:15:51.188 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20132596899999999, 2024-09-18T10:15:51.188 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:51.188 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:51.188 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:51.188 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.188 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:51.188 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.036738+0000", 2024-09-18T10:15:51.188 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:51.188 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.188 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.188 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:51.188 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.036738+0000", 2024-09-18T10:15:51.188 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:51.189 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.189 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.189 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:51.189 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.036737+0000", 2024-09-18T10:15:51.189 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:51.189 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.189 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.189 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:51.189 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.036804+0000", 2024-09-18T10:15:51.189 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.7178e-05 2024-09-18T10:15:51.189 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.189 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.189 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:51.189 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.036821+0000", 2024-09-18T10:15:51.189 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7153e-05 2024-09-18T10:15:51.190 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.190 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.190 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:51.190 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.136954+0000", 2024-09-18T10:15:51.190 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100132572 2024-09-18T10:15:51.190 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.190 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.190 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:51.190 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.137090+0000", 2024-09-18T10:15:51.190 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000136332 2024-09-18T10:15:51.190 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.190 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.190 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:51.191 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.137124+0000", 2024-09-18T10:15:51.191 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3458999999999999e-05 2024-09-18T10:15:51.191 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.191 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.191 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:51.191 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.237999+0000", 2024-09-18T10:15:51.191 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100875385 2024-09-18T10:15:51.191 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.191 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.191 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:51.191 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.238064+0000", 2024-09-18T10:15:51.191 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.5086999999999995e-05 2024-09-18T10:15:51.191 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:51.191 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:51.191 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:51.192 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.192 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.192 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4257.0:113 1.0 e29/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 29'83, mlcod=29'80)", 2024-09-18T10:15:51.192 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:40.801786+0000", 2024-09-18T10:15:51.192 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 70.359514235000006, 2024-09-18T10:15:51.192 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.203408961, 2024-09-18T10:15:51.192 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:51.192 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:51.192 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:51.192 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.192 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:51.192 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.801786+0000", 2024-09-18T10:15:51.192 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:51.193 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.193 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.193 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:51.193 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.801786+0000", 2024-09-18T10:15:51.193 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:51.193 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.193 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.193 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:51.193 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.801784+0000", 2024-09-18T10:15:51.193 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:51.193 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.193 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.193 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:51.193 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.801801+0000", 2024-09-18T10:15:51.194 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7223000000000001e-05 2024-09-18T10:15:51.194 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.194 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.194 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:51.194 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.801823+0000", 2024-09-18T10:15:51.194 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1469000000000001e-05 2024-09-18T10:15:51.194 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.194 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.194 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:51.194 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.801835+0000", 2024-09-18T10:15:51.194 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2033e-05 2024-09-18T10:15:51.194 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.194 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.195 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:51.195 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.902049+0000", 2024-09-18T10:15:51.195 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100213659 2024-09-18T10:15:51.195 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.195 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.195 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:51.195 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.902082+0000", 2024-09-18T10:15:51.195 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3668999999999999e-05 2024-09-18T10:15:51.195 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.195 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.195 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:51.196 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.005131+0000", 2024-09-18T10:15:51.196 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.103048379 2024-09-18T10:15:51.196 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.196 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.196 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:51.196 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.005195+0000", 2024-09-18T10:15:51.196 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.3805000000000003e-05 2024-09-18T10:15:51.196 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:51.196 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:51.196 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:51.196 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.196 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.196 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4257.0:114 1.0 e29/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 29'84, mlcod=29'80)", 2024-09-18T10:15:51.197 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:40.802657+0000", 2024-09-18T10:15:51.197 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 70.358643100999998, 2024-09-18T10:15:51.197 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20256802600000001, 2024-09-18T10:15:51.197 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:51.197 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:51.197 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:51.197 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.197 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:51.197 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.802657+0000", 2024-09-18T10:15:51.197 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:51.197 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.197 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.197 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:51.198 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.802657+0000", 2024-09-18T10:15:51.198 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:51.198 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.198 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.198 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:51.198 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.802656+0000", 2024-09-18T10:15:51.198 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:15:51.198 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.198 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.198 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:51.198 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.802824+0000", 2024-09-18T10:15:51.198 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000168784 2024-09-18T10:15:51.198 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.198 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.199 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:51.199 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.802844+0000", 2024-09-18T10:15:51.199 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0058999999999999e-05 2024-09-18T10:15:51.199 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.199 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.199 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:51.199 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.802857+0000", 2024-09-18T10:15:51.199 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2163e-05 2024-09-18T10:15:51.199 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.199 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.199 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:51.199 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.902531+0000", 2024-09-18T10:15:51.199 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.099674591000000007 2024-09-18T10:15:51.200 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.200 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.200 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:51.200 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.902559+0000", 2024-09-18T10:15:51.200 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7483000000000001e-05 2024-09-18T10:15:51.200 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.200 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.200 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:51.200 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.005198+0000", 2024-09-18T10:15:51.200 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.102638853 2024-09-18T10:15:51.200 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.200 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.200 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:51.201 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.005225+0000", 2024-09-18T10:15:51.201 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7206e-05 2024-09-18T10:15:51.201 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:51.201 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:51.201 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:51.201 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.201 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.201 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4257.0:115 1.0 e29/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 29'85, mlcod=29'80)", 2024-09-18T10:15:51.201 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:40.803806+0000", 2024-09-18T10:15:51.201 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 70.357493714, 2024-09-18T10:15:51.201 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.201443711, 2024-09-18T10:15:51.201 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:51.201 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:51.202 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:51.202 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.202 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:51.202 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.803806+0000", 2024-09-18T10:15:51.202 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:51.202 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.202 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.202 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:51.202 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.803806+0000", 2024-09-18T10:15:51.202 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:51.202 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.202 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.202 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:51.202 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.803804+0000", 2024-09-18T10:15:51.203 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:51.203 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.203 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.203 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:51.203 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.803818+0000", 2024-09-18T10:15:51.203 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3477e-05 2024-09-18T10:15:51.203 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.203 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.203 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:51.203 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.803851+0000", 2024-09-18T10:15:51.203 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3086000000000001e-05 2024-09-18T10:15:51.203 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.203 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.203 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:51.204 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.803867+0000", 2024-09-18T10:15:51.204 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5885000000000001e-05 2024-09-18T10:15:51.204 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.204 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.204 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:51.204 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.903206+0000", 2024-09-18T10:15:51.204 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.099338990000000002 2024-09-18T10:15:51.204 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.204 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.204 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:51.204 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.903253+0000", 2024-09-18T10:15:51.204 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.7419000000000001e-05 2024-09-18T10:15:51.204 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.204 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.205 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:51.205 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.005227+0000", 2024-09-18T10:15:51.205 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.101973199 2024-09-18T10:15:51.205 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.205 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.205 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:51.205 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.005250+0000", 2024-09-18T10:15:51.205 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3261999999999999e-05 2024-09-18T10:15:51.205 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:51.206 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:51.206 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:51.206 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.206 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.206 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4257.0:117 1.0 e29/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 29'87, mlcod=29'80)", 2024-09-18T10:15:51.206 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:40.805727+0000", 2024-09-18T10:15:51.206 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 70.355572476000006, 2024-09-18T10:15:51.206 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.200946191, 2024-09-18T10:15:51.206 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:51.206 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:51.206 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:51.206 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.206 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:51.206 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.805727+0000", 2024-09-18T10:15:51.206 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:51.207 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.207 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.207 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:51.207 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.805727+0000", 2024-09-18T10:15:51.207 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:51.207 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.207 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.207 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:51.207 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.805726+0000", 2024-09-18T10:15:51.207 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:51.207 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.207 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.207 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:51.207 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.805783+0000", 2024-09-18T10:15:51.208 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.7068000000000003e-05 2024-09-18T10:15:51.208 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.208 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.208 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:51.208 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.805817+0000", 2024-09-18T10:15:51.208 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4471999999999998e-05 2024-09-18T10:15:51.208 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.208 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.208 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:51.208 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.805833+0000", 2024-09-18T10:15:51.208 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6313000000000001e-05 2024-09-18T10:15:51.208 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.208 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.209 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:51.209 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.004481+0000", 2024-09-18T10:15:51.209 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.198647823 2024-09-18T10:15:51.209 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.209 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.209 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:51.209 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.004520+0000", 2024-09-18T10:15:51.209 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8992000000000002e-05 2024-09-18T10:15:51.209 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.209 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.209 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:51.209 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.006585+0000", 2024-09-18T10:15:51.209 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0020647389999999999 2024-09-18T10:15:51.209 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.210 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.210 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:51.210 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.006673+0000", 2024-09-18T10:15:51.210 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.8443999999999995e-05 2024-09-18T10:15:51.210 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:51.210 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:51.210 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:51.210 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.210 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.210 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4257.0:128 1.0 e29/23 1:30306672:devicehealth::main.db.0000000000000000:head v 29'98, mlcod=29'95)", 2024-09-18T10:15:51.210 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:41.222806+0000", 2024-09-18T10:15:51.210 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 69.938493835000003, 2024-09-18T10:15:51.210 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20200015099999999, 2024-09-18T10:15:51.211 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:51.211 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:51.211 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:51.211 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.211 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:51.211 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.222806+0000", 2024-09-18T10:15:51.211 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:51.211 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.211 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.211 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:51.211 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.222806+0000", 2024-09-18T10:15:51.211 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:51.211 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.211 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.212 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:51.212 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.222804+0000", 2024-09-18T10:15:51.212 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:51.212 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.212 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.212 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:51.212 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.222885+0000", 2024-09-18T10:15:51.212 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.0140000000000002e-05 2024-09-18T10:15:51.212 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.212 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.212 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:51.212 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.222915+0000", 2024-09-18T10:15:51.212 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0168999999999999e-05 2024-09-18T10:15:51.213 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.213 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.213 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:51.213 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.323060+0000", 2024-09-18T10:15:51.213 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100145475 2024-09-18T10:15:51.213 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.213 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.213 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:51.213 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.423304+0000", 2024-09-18T10:15:51.213 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100243924 2024-09-18T10:15:51.213 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.213 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.213 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:51.213 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.423338+0000", 2024-09-18T10:15:51.214 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3819000000000003e-05 2024-09-18T10:15:51.214 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.214 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.214 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:51.214 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.424724+0000", 2024-09-18T10:15:51.214 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0013862690000000001 2024-09-18T10:15:51.214 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.214 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.214 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:51.214 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.424806+0000", 2024-09-18T10:15:51.214 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.1901000000000001e-05 2024-09-18T10:15:51.214 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:51.214 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:51.214 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:51.215 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.215 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.215 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.4s1 41/40 ECSubWrite(tid=17, reqid=osd.8.0:17, at_version=41'10, trim_to=0'0, roll_forward_to=0'0))", 2024-09-18T10:15:51.215 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:57.445149+0000", 2024-09-18T10:15:51.215 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 53.716151273000001, 2024-09-18T10:15:51.215 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20394425099999999, 2024-09-18T10:15:51.215 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:51.215 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:51.215 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:51.215 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.215 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:51.215 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.445149+0000", 2024-09-18T10:15:51.215 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:51.216 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.216 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.216 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:51.216 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.445149+0000", 2024-09-18T10:15:51.216 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:51.216 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.216 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.216 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:51.216 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.445147+0000", 2024-09-18T10:15:51.216 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:51.216 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.216 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.216 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:51.216 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.445221+0000", 2024-09-18T10:15:51.217 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.4140999999999994e-05 2024-09-18T10:15:51.217 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.217 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.217 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:51.217 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.445260+0000", 2024-09-18T10:15:51.217 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8998999999999997e-05 2024-09-18T10:15:51.217 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.217 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.217 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:51.217 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.545386+0000", 2024-09-18T10:15:51.217 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10012621300000001 2024-09-18T10:15:51.217 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.217 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.217 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:51.218 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.645963+0000", 2024-09-18T10:15:51.218 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10057662000000001 2024-09-18T10:15:51.218 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.218 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.218 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:51.218 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.645988+0000", 2024-09-18T10:15:51.218 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5449e-05 2024-09-18T10:15:51.218 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.218 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.218 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:51.218 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.649028+0000", 2024-09-18T10:15:51.218 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0030390840000000001 2024-09-18T10:15:51.218 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.218 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.219 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:51.219 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.649093+0000", 2024-09-18T10:15:51.219 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.5258000000000004e-05 2024-09-18T10:15:51.219 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:51.219 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:51.219 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:51.219 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.219 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.219 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.6s6 41/40 ECSubWrite(tid=31, reqid=client.4632.0:249, at_version=41'13, trim_to=0'0, roll_forward_to=0'0))", 2024-09-18T10:15:51.219 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:57.550336+0000", 2024-09-18T10:15:51.219 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 53.610964234000001, 2024-09-18T10:15:51.219 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.203590574, 2024-09-18T10:15:51.219 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:51.219 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:51.220 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:51.220 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.220 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:51.220 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.550336+0000", 2024-09-18T10:15:51.220 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:51.220 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.220 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.220 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:51.220 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.550336+0000", 2024-09-18T10:15:51.220 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:51.220 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.220 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.220 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:51.220 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.550329+0000", 2024-09-18T10:15:51.221 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999933 2024-09-18T10:15:51.221 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.221 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.221 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:51.221 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.550932+0000", 2024-09-18T10:15:51.221 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00060276999999999998 2024-09-18T10:15:51.221 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.221 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.221 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:51.221 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.550955+0000", 2024-09-18T10:15:51.221 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.2884999999999999e-05 2024-09-18T10:15:51.221 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.221 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.221 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:51.222 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.651052+0000", 2024-09-18T10:15:51.222 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100097174 2024-09-18T10:15:51.222 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.222 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.222 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:51.222 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.651160+0000", 2024-09-18T10:15:51.222 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000108483 2024-09-18T10:15:51.222 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.222 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.222 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:51.222 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.651172+0000", 2024-09-18T10:15:51.222 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1790999999999999e-05 2024-09-18T10:15:51.222 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.222 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.223 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:51.223 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.753875+0000", 2024-09-18T10:15:51.223 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10270337 2024-09-18T10:15:51.223 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.223 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.223 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:51.223 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.753926+0000", 2024-09-18T10:15:51.223 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0767999999999999e-05 2024-09-18T10:15:51.223 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:51.223 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:51.223 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:51.223 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.223 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.223 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.8s4 42/40 ECSubWrite(tid=133, reqid=osd.6.0:132, at_version=0'0, trim_to=0'0, roll_forward_to=41'13))", 2024-09-18T10:15:51.224 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:58.049763+0000", 2024-09-18T10:15:51.224 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 53.111536498, 2024-09-18T10:15:51.224 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.21446268399999999, 2024-09-18T10:15:51.224 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:51.224 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:51.224 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:51.224 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.224 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:51.224 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:58.049763+0000", 2024-09-18T10:15:51.224 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:51.224 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.224 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.224 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:51.224 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:58.049763+0000", 2024-09-18T10:15:51.225 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:51.225 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.225 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.225 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:51.225 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:58.049763+0000", 2024-09-18T10:15:51.225 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:51.225 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.225 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.225 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:51.225 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:58.049787+0000", 2024-09-18T10:15:51.225 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3819e-05 2024-09-18T10:15:51.225 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.225 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.226 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:51.226 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:58.049817+0000", 2024-09-18T10:15:51.226 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0749e-05 2024-09-18T10:15:51.226 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.226 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.226 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:51.226 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:58.049836+0000", 2024-09-18T10:15:51.226 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9182999999999999e-05 2024-09-18T10:15:51.226 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.226 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.226 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:51.226 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:58.163409+0000", 2024-09-18T10:15:51.226 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.11357204899999999 2024-09-18T10:15:51.226 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.227 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.227 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:51.227 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:58.163424+0000", 2024-09-18T10:15:51.227 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5403000000000001e-05 2024-09-18T10:15:51.227 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.227 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.227 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:51.227 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:58.264161+0000", 2024-09-18T10:15:51.227 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100736942 2024-09-18T10:15:51.227 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.227 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.227 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:51.227 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:58.264226+0000", 2024-09-18T10:15:51.227 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.5085000000000005e-05 2024-09-18T10:15:51.228 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:51.228 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:51.228 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:51.228 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.228 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.228 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.ds3 47/32 ECSubWrite(tid=56, reqid=client.4632.0:333, at_version=47'9, trim_to=0'0, roll_forward_to=41'6))", 2024-09-18T10:15:51.228 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:02.801124+0000", 2024-09-18T10:15:51.228 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 48.360175339999998, 2024-09-18T10:15:51.228 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20185083000000001, 2024-09-18T10:15:51.228 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:51.228 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:51.228 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:51.228 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.229 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:51.229 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.801124+0000", 2024-09-18T10:15:51.229 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:51.229 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.229 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.229 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:51.229 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.801124+0000", 2024-09-18T10:15:51.229 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:51.229 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.229 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.229 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:51.229 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.801103+0000", 2024-09-18T10:15:51.229 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999785 2024-09-18T10:15:51.229 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.230 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.230 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:51.230 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.801366+0000", 2024-09-18T10:15:51.230 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00026269299999999999 2024-09-18T10:15:51.230 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.230 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.230 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:51.230 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.801395+0000", 2024-09-18T10:15:51.230 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.9572000000000001e-05 2024-09-18T10:15:51.230 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.230 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.230 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:51.230 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.801412+0000", 2024-09-18T10:15:51.230 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6872999999999999e-05 2024-09-18T10:15:51.231 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.231 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.231 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:51.231 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.000543+0000", 2024-09-18T10:15:51.231 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.19913046700000001 2024-09-18T10:15:51.231 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.231 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.231 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:51.231 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.000579+0000", 2024-09-18T10:15:51.231 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6649000000000001e-05 2024-09-18T10:15:51.231 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.231 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.231 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:51.231 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.002922+0000", 2024-09-18T10:15:51.232 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0023426139999999998 2024-09-18T10:15:51.232 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.232 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.232 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:51.232 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.002975+0000", 2024-09-18T10:15:51.232 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3232000000000001e-05 2024-09-18T10:15:51.232 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:51.232 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:51.232 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:51.232 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.232 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.232 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.15s1 63/60 ECSubWrite(tid=54, reqid=client.4632.0:512, at_version=0'0, trim_to=0'0, roll_forward_to=63'4))", 2024-09-18T10:15:51.232 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:19.215356+0000", 2024-09-18T10:15:51.232 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 31.945943478, 2024-09-18T10:15:51.233 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20187551500000001, 2024-09-18T10:15:51.233 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:51.233 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:51.233 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:51.233 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.233 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:51.233 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.215356+0000", 2024-09-18T10:15:51.233 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:51.233 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.233 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.233 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:51.233 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.215356+0000", 2024-09-18T10:15:51.233 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:51.233 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.234 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.234 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:51.234 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.215355+0000", 2024-09-18T10:15:51.234 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:51.234 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.234 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.234 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:51.234 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.215386+0000", 2024-09-18T10:15:51.234 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0809000000000003e-05 2024-09-18T10:15:51.234 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.234 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.234 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:51.234 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.215434+0000", 2024-09-18T10:15:51.235 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.8751999999999997e-05 2024-09-18T10:15:51.235 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.235 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.235 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:51.235 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.315532+0000", 2024-09-18T10:15:51.235 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100097281 2024-09-18T10:15:51.235 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.235 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.235 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:51.235 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.415795+0000", 2024-09-18T10:15:51.235 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10026387 2024-09-18T10:15:51.235 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.235 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.235 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:51.236 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.415823+0000", 2024-09-18T10:15:51.236 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7469000000000001e-05 2024-09-18T10:15:51.236 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.236 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.236 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:51.236 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.417173+0000", 2024-09-18T10:15:51.236 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0013505030000000001 2024-09-18T10:15:51.236 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.236 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.236 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:51.236 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.417232+0000", 2024-09-18T10:15:51.236 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8421000000000002e-05 2024-09-18T10:15:51.236 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:51.237 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:51.237 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:51.237 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.237 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.237 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.4s1 72/40 ECSubWrite(tid=72, reqid=osd.8.0:72, at_version=72'53, trim_to=0'0, roll_forward_to=71'51))", 2024-09-18T10:15:51.237 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:28.190950+0000", 2024-09-18T10:15:51.237 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 22.970349654, 2024-09-18T10:15:51.237 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20228795599999999, 2024-09-18T10:15:51.237 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:51.237 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:51.237 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:51.237 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.237 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:51.238 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.190950+0000", 2024-09-18T10:15:51.238 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:51.238 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.238 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.238 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:51.238 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.190950+0000", 2024-09-18T10:15:51.238 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:51.238 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.238 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.238 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:51.238 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.190949+0000", 2024-09-18T10:15:51.238 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:51.238 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.238 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.239 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:51.239 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.191001+0000", 2024-09-18T10:15:51.239 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.2581999999999999e-05 2024-09-18T10:15:51.239 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.239 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.239 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:51.239 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.191022+0000", 2024-09-18T10:15:51.239 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0792999999999999e-05 2024-09-18T10:15:51.239 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.239 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.239 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:51.239 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.191034+0000", 2024-09-18T10:15:51.239 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1603000000000001e-05 2024-09-18T10:15:51.240 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.240 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.240 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:51.240 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.391648+0000", 2024-09-18T10:15:51.240 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20061451299999999 2024-09-18T10:15:51.240 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.240 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.240 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:51.240 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.391678+0000", 2024-09-18T10:15:51.240 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0199e-05 2024-09-18T10:15:51.240 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.240 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.240 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:51.240 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.393192+0000", 2024-09-18T10:15:51.241 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0015135599999999999 2024-09-18T10:15:51.241 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.241 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.241 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:51.241 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.393238+0000", 2024-09-18T10:15:51.241 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.6128999999999998e-05 2024-09-18T10:15:51.241 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:51.241 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:51.241 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:51.241 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.241 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.241 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.4s1 72/40 ECSubWrite(tid=74, reqid=osd.8.0:72, at_version=0'0, trim_to=0'0, roll_forward_to=72'53))", 2024-09-18T10:15:51.241 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:28.496056+0000", 2024-09-18T10:15:51.241 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 22.665243571000001, 2024-09-18T10:15:51.242 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20153745300000001, 2024-09-18T10:15:51.242 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:51.242 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:51.242 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:51.242 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.242 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:51.242 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.496056+0000", 2024-09-18T10:15:51.242 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:51.242 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.242 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.242 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:51.242 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.496056+0000", 2024-09-18T10:15:51.242 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:51.243 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.243 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.243 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:51.243 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.496055+0000", 2024-09-18T10:15:51.243 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:51.243 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.243 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.243 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:51.243 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.496088+0000", 2024-09-18T10:15:51.243 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3411999999999998e-05 2024-09-18T10:15:51.243 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.243 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.243 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:51.243 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.496120+0000", 2024-09-18T10:15:51.244 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1745999999999997e-05 2024-09-18T10:15:51.244 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.244 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.244 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:51.244 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.596220+0000", 2024-09-18T10:15:51.244 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100100381 2024-09-18T10:15:51.244 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.244 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.244 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:51.244 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.596320+0000", 2024-09-18T10:15:51.244 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.9745000000000001e-05 2024-09-18T10:15:51.244 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.244 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.245 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:51.245 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.596344+0000", 2024-09-18T10:15:51.245 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4598999999999999e-05 2024-09-18T10:15:51.245 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.245 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.245 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:51.245 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.697512+0000", 2024-09-18T10:15:51.245 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10116728699999999 2024-09-18T10:15:51.245 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.245 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.245 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:51.245 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.697594+0000", 2024-09-18T10:15:51.245 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.1972999999999996e-05 2024-09-18T10:15:51.245 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:51.246 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:51.246 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:51.246 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.246 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.246 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.4s1 75/40 ECSubWrite(tid=80, reqid=osd.8.0:80, at_version=75'61, trim_to=0'0, roll_forward_to=74'59))", 2024-09-18T10:15:51.246 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:30.566285+0000", 2024-09-18T10:15:51.246 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 20.595014411000001, 2024-09-18T10:15:51.246 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.201584069, 2024-09-18T10:15:51.246 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:51.246 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:51.246 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:51.246 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.246 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:51.246 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.566285+0000", 2024-09-18T10:15:51.246 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:51.247 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.247 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.247 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:51.247 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.566285+0000", 2024-09-18T10:15:51.247 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:51.247 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.247 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.247 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:51.247 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.566285+0000", 2024-09-18T10:15:51.247 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:51.247 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.247 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.247 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:51.247 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.566310+0000", 2024-09-18T10:15:51.248 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5250999999999999e-05 2024-09-18T10:15:51.248 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.248 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.248 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:51.248 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.566319+0000", 2024-09-18T10:15:51.248 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.5159999999999994e-06 2024-09-18T10:15:51.248 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.248 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.248 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:51.248 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.666393+0000", 2024-09-18T10:15:51.248 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10007392399999999 2024-09-18T10:15:51.248 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.248 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.248 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:51.249 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.666455+0000", 2024-09-18T10:15:51.249 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.2372999999999994e-05 2024-09-18T10:15:51.249 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.249 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.249 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:51.249 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.666465+0000", 2024-09-18T10:15:51.249 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.7820000000000005e-06 2024-09-18T10:15:51.249 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.249 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.249 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:51.249 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.767822+0000", 2024-09-18T10:15:51.249 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10135730900000001 2024-09-18T10:15:51.249 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.249 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.250 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:51.250 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.767869+0000", 2024-09-18T10:15:51.250 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.7181000000000001e-05 2024-09-18T10:15:51.250 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:51.250 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:51.250 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:51.250 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.250 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.250 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.fs2 89/32 ECSubWrite(tid=187, reqid=osd.13.0:187, at_version=89'40, trim_to=0'0, roll_forward_to=81'37))", 2024-09-18T10:15:51.250 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:44.898707+0000", 2024-09-18T10:15:51.250 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 6.2625929530000004, 2024-09-18T10:15:51.250 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.203487005, 2024-09-18T10:15:51.250 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:51.250 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:51.251 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:51.251 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.251 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:51.251 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:44.898707+0000", 2024-09-18T10:15:51.251 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:51.251 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.251 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.251 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:51.251 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:44.898707+0000", 2024-09-18T10:15:51.251 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:51.251 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.251 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.251 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:51.251 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:44.898706+0000", 2024-09-18T10:15:51.251 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:15:51.252 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.252 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.252 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:51.252 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:44.898768+0000", 2024-09-18T10:15:51.252 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.2237999999999996e-05 2024-09-18T10:15:51.252 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.252 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.252 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:51.252 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:44.898791+0000", 2024-09-18T10:15:51.252 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.2713000000000001e-05 2024-09-18T10:15:51.252 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.252 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.252 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:51.253 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:44.898808+0000", 2024-09-18T10:15:51.253 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7356000000000001e-05 2024-09-18T10:15:51.253 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.253 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.253 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:51.253 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:45.099963+0000", 2024-09-18T10:15:51.253 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20115494 2024-09-18T10:15:51.253 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.253 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.253 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:51.253 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:45.100014+0000", 2024-09-18T10:15:51.253 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.1298999999999998e-05 2024-09-18T10:15:51.253 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.253 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.253 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:51.254 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:45.102121+0000", 2024-09-18T10:15:51.254 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0021067479999999999 2024-09-18T10:15:51.254 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:51.254 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:51.254 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:51.254 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:45.102194+0000", 2024-09-18T10:15:51.254 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.2700999999999994e-05 2024-09-18T10:15:51.254 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:51.254 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:51.254 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:51.254 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:51.254 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:51.254 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-18T10:15:51.256 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T10:15:51.256 INFO:tasks.ceph.osd.8.smithi029.stderr:2024-09-18T10:15:51.249+0000 7f2555c27700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 22192) UID: 0 2024-09-18T10:15:51.357 INFO:tasks.ceph.osd.4.smithi029.stderr:2024-09-18T10:15:51.349+0000 7f550cf6e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22189) UID: 0 2024-09-18T10:15:51.377 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-18T10:15:51.377 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-18T10:15:51.378 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 0 2024-09-18T10:15:51.378 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-18T10:15:51.389 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T10:15:51.458 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-18T10:15:51.456+0000 7f00046a8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 22136) UID: 0 2024-09-18T10:15:51.491 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-18T10:15:51.491 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-18T10:15:51.491 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-18T10:15:51.492 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-18T10:15:51.492 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-18T10:15:51.504 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T10:15:51.559 INFO:tasks.ceph.osd.4.smithi029.stderr:2024-09-18T10:15:51.553+0000 7f550cf6e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22189) UID: 0 2024-09-18T10:15:51.594 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-18T10:15:51.613 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-18T10:15:51.613 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-18T10:15:51.613 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-18T10:15:51.613 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-18T10:15:51.613 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.613 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWriteReply(3.6s0 36/32 ECSubWriteReply(tid=11, last_complete=36'7, committed=1, applied=1))", 2024-09-18T10:15:51.613 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:50.290945+0000", 2024-09-18T10:15:51.613 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 61.316995914000003, 2024-09-18T10:15:51.614 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.208748928, 2024-09-18T10:15:51.614 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:51.614 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:51.614 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:51.614 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.614 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:51.615 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.290945+0000", 2024-09-18T10:15:51.615 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:51.615 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.615 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.615 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:51.615 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.290945+0000", 2024-09-18T10:15:51.615 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:51.615 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.615 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.615 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:51.615 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.290945+0000", 2024-09-18T10:15:51.615 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:51.615 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.615 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.616 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:51.616 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.290951+0000", 2024-09-18T10:15:51.616 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.5729999999999996e-06 2024-09-18T10:15:51.616 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.616 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.616 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:51.616 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.290964+0000", 2024-09-18T10:15:51.616 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3012e-05 2024-09-18T10:15:51.616 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.616 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.616 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:51.616 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.290972+0000", 2024-09-18T10:15:51.616 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.4699999999999996e-06 2024-09-18T10:15:51.617 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.617 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.617 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:51.617 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.499661+0000", 2024-09-18T10:15:51.617 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.20868911500000001 2024-09-18T10:15:51.617 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.617 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.617 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:51.617 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.499694+0000", 2024-09-18T10:15:51.617 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.3470000000000003e-05 2024-09-18T10:15:51.617 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:51.617 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:51.617 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:51.618 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.618 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.618 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWriteReply(3.6s0 36/32 ECSubWriteReply(tid=11, last_complete=36'7, committed=1, applied=1))", 2024-09-18T10:15:51.618 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:50.291030+0000", 2024-09-18T10:15:51.618 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 61.316911415, 2024-09-18T10:15:51.618 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.208840265, 2024-09-18T10:15:51.618 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:51.618 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:51.618 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:51.618 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.618 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:51.618 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.291030+0000", 2024-09-18T10:15:51.618 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:51.618 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.619 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.619 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:51.619 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.291030+0000", 2024-09-18T10:15:51.619 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:51.619 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.619 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.619 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:51.619 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.291030+0000", 2024-09-18T10:15:51.619 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:51.619 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.619 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.619 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:51.619 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.291032+0000", 2024-09-18T10:15:51.619 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.604e-06 2024-09-18T10:15:51.620 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.620 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.620 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:51.620 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.291038+0000", 2024-09-18T10:15:51.620 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8170000000000004e-06 2024-09-18T10:15:51.620 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.620 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.620 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:51.620 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.291042+0000", 2024-09-18T10:15:51.621 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.839e-06 2024-09-18T10:15:51.621 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.621 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.621 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:51.621 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.499812+0000", 2024-09-18T10:15:51.621 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.208769805 2024-09-18T10:15:51.621 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.621 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.621 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:51.621 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.499870+0000", 2024-09-18T10:15:51.621 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8449000000000002e-05 2024-09-18T10:15:51.621 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:51.621 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:51.621 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:51.622 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.622 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.622 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWriteReply(3.6s0 36/32 ECSubWriteReply(tid=11, last_complete=36'7, committed=1, applied=1))", 2024-09-18T10:15:51.622 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:50.302109+0000", 2024-09-18T10:15:51.622 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 61.305832256000002, 2024-09-18T10:15:51.622 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.29912391399999999, 2024-09-18T10:15:51.622 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:51.622 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:51.622 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:51.622 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.622 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:51.622 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.302109+0000", 2024-09-18T10:15:51.622 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:51.622 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.623 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.623 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:51.623 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.302109+0000", 2024-09-18T10:15:51.623 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:51.623 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.623 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.623 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:51.623 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.302107+0000", 2024-09-18T10:15:51.623 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:51.623 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.623 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.624 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:51.624 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.302119+0000", 2024-09-18T10:15:51.624 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1207e-05 2024-09-18T10:15:51.624 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.624 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.625 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:51.625 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.302150+0000", 2024-09-18T10:15:51.625 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.133e-05 2024-09-18T10:15:51.625 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.625 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.625 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:51.625 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.302164+0000", 2024-09-18T10:15:51.625 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3942e-05 2024-09-18T10:15:51.625 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.625 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.625 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:51.625 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.600015+0000", 2024-09-18T10:15:51.625 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.29785072000000001 2024-09-18T10:15:51.626 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.626 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.626 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:51.626 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.601233+0000", 2024-09-18T10:15:51.626 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0012183599999999999 2024-09-18T10:15:51.626 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:51.626 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:51.626 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:51.626 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.626 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.626 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpReadReply(3.6s0 36/32 ECSubReadReply(tid=10, attrs_read=0))", 2024-09-18T10:15:51.626 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:50.302179+0000", 2024-09-18T10:15:51.626 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 61.305762190000003, 2024-09-18T10:15:51.626 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.30043391800000002, 2024-09-18T10:15:51.627 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:51.627 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:51.627 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:51.627 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.627 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:51.627 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.302179+0000", 2024-09-18T10:15:51.627 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:51.627 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.627 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.627 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:51.627 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.302179+0000", 2024-09-18T10:15:51.627 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:51.627 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.627 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.627 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:51.628 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.302178+0000", 2024-09-18T10:15:51.628 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:51.628 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.628 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.628 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:51.628 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.302201+0000", 2024-09-18T10:15:51.628 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2824e-05 2024-09-18T10:15:51.628 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.628 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.628 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:51.628 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.302217+0000", 2024-09-18T10:15:51.628 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5993000000000001e-05 2024-09-18T10:15:51.628 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.628 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.629 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:51.629 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.302223+0000", 2024-09-18T10:15:51.629 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0820000000000004e-06 2024-09-18T10:15:51.629 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.629 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.629 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:51.629 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.602454+0000", 2024-09-18T10:15:51.629 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.30023072499999998 2024-09-18T10:15:51.629 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.629 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.629 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:51.629 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.602613+0000", 2024-09-18T10:15:51.629 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00015896399999999999 2024-09-18T10:15:51.630 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:51.630 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:51.630 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:51.630 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.630 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.630 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:121 3.6s0 3:6c64c4fa:::smithi18925896-41:head [append 0~688128,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:51.630 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:50.627356+0000", 2024-09-18T10:15:51.630 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 60.980584958999998, 2024-09-18T10:15:51.630 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.38269042599999997, 2024-09-18T10:15:51.630 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:51.630 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:51.631 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:51.631 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:51.631 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:51.631 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 121 2024-09-18T10:15:51.631 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.631 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:51.631 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.631 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:51.631 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.627356+0000", 2024-09-18T10:15:51.631 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:51.631 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.631 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.631 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:51.631 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.627356+0000", 2024-09-18T10:15:51.632 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:51.632 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.632 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.632 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:51.632 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.627353+0000", 2024-09-18T10:15:51.632 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999971 2024-09-18T10:15:51.632 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.632 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.632 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:51.632 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.629112+0000", 2024-09-18T10:15:51.632 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0017585980000000001 2024-09-18T10:15:51.632 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.633 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.633 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:51.633 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.629149+0000", 2024-09-18T10:15:51.633 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.6532999999999997e-05 2024-09-18T10:15:51.633 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.633 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.633 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:51.633 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.629169+0000", 2024-09-18T10:15:51.633 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0699e-05 2024-09-18T10:15:51.633 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.633 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.633 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:51.633 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.706567+0000", 2024-09-18T10:15:51.633 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.077397621999999999 2024-09-18T10:15:51.634 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.634 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.634 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:51.634 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.706944+0000", 2024-09-18T10:15:51.634 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00037710599999999999 2024-09-18T10:15:51.634 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.634 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.634 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:51.634 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.709283+0000", 2024-09-18T10:15:51.634 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0023391689999999999 2024-09-18T10:15:51.635 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.635 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.636 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:51.636 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.711791+0000", 2024-09-18T10:15:51.636 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0025078689999999998 2024-09-18T10:15:51.636 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.636 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.636 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-18T10:15:51.636 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.009925+0000", 2024-09-18T10:15:51.636 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.298134493 2024-09-18T10:15:51.636 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.636 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.637 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:51.637 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.010047+0000", 2024-09-18T10:15:51.637 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000121264 2024-09-18T10:15:51.637 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:51.637 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:51.638 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:51.638 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.638 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.638 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:122 3.6s0 3:6c64c4fa:::smithi18925896-41:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:51.638 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:50.629189+0000", 2024-09-18T10:15:51.638 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 60.978752233000002, 2024-09-18T10:15:51.638 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.47849547399999998, 2024-09-18T10:15:51.638 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:51.638 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:51.639 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:51.639 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:51.639 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:51.639 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 122 2024-09-18T10:15:51.639 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.639 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:51.639 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.639 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:51.639 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.629189+0000", 2024-09-18T10:15:51.639 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:51.639 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.639 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.640 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:51.640 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.629189+0000", 2024-09-18T10:15:51.640 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:51.640 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.640 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.640 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:51.640 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.629187+0000", 2024-09-18T10:15:51.640 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:51.640 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.640 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.640 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:51.640 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.629198+0000", 2024-09-18T10:15:51.641 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0623e-05 2024-09-18T10:15:51.641 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.641 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.641 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:51.641 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.629219+0000", 2024-09-18T10:15:51.641 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0665999999999998e-05 2024-09-18T10:15:51.641 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.641 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.641 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:51.641 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.629231+0000", 2024-09-18T10:15:51.641 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.291e-05 2024-09-18T10:15:51.641 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.642 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.642 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:51.642 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.710475+0000", 2024-09-18T10:15:51.642 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.081244021 2024-09-18T10:15:51.642 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.642 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.642 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:51.642 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.710639+0000", 2024-09-18T10:15:51.642 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000163325 2024-09-18T10:15:51.642 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.642 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.642 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:51.642 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.711435+0000", 2024-09-18T10:15:51.642 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00079575999999999996 2024-09-18T10:15:51.642 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.643 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.643 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:51.643 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.812067+0000", 2024-09-18T10:15:51.643 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10063237799999999 2024-09-18T10:15:51.643 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.643 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.643 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-18T10:15:51.643 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.107562+0000", 2024-09-18T10:15:51.643 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.295494906 2024-09-18T10:15:51.643 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.643 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.643 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:51.643 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.107685+0000", 2024-09-18T10:15:51.643 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00012271100000000001 2024-09-18T10:15:51.643 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:51.644 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:51.644 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:51.644 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.644 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.644 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:123 3.6s0 3:6c64c4fa:::smithi18925896-41:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:51.644 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:50.629246+0000", 2024-09-18T10:15:51.644 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 60.978695407000004, 2024-09-18T10:15:51.644 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.58067324300000001, 2024-09-18T10:15:51.644 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:51.644 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:51.644 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:51.644 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:51.645 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:51.645 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 123 2024-09-18T10:15:51.645 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.645 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:51.645 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.645 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:51.645 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.629246+0000", 2024-09-18T10:15:51.645 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:51.645 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.646 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.646 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:51.646 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.629246+0000", 2024-09-18T10:15:51.646 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:51.646 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.646 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.646 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:51.646 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.629244+0000", 2024-09-18T10:15:51.646 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:51.646 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.646 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.646 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:51.646 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.629254+0000", 2024-09-18T10:15:51.646 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.0759999999999992e-06 2024-09-18T10:15:51.647 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.647 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.647 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:51.647 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.629279+0000", 2024-09-18T10:15:51.647 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.5905999999999999e-05 2024-09-18T10:15:51.647 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.647 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.647 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:51.647 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.629293+0000", 2024-09-18T10:15:51.647 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3073e-05 2024-09-18T10:15:51.647 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.647 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.647 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:51.648 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.806055+0000", 2024-09-18T10:15:51.648 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.17676249999999999 2024-09-18T10:15:51.648 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.648 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.648 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for rw locks", 2024-09-18T10:15:51.648 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.806211+0000", 2024-09-18T10:15:51.648 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000156384 2024-09-18T10:15:51.648 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.648 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.648 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:51.648 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.108025+0000", 2024-09-18T10:15:51.648 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.30181311500000002 2024-09-18T10:15:51.648 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.649 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.649 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:51.649 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.108107+0000", 2024-09-18T10:15:51.649 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.2330000000000006e-05 2024-09-18T10:15:51.649 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.649 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.649 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:51.649 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.209919+0000", 2024-09-18T10:15:51.649 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.101812262 2024-09-18T10:15:51.649 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:51.649 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:51.649 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:51.649 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.650 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.650 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:167 3.6s0 3:6619023d:::smithi18925896-19:head [copy-from ver 4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:51.650 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:51.531881+0000", 2024-09-18T10:15:51.650 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 60.076060189000003, 2024-09-18T10:15:51.650 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.41157491600000001, 2024-09-18T10:15:51.650 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:51.650 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:51.650 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:51.650 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:51.650 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:51.650 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 167 2024-09-18T10:15:51.650 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.650 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:51.650 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.651 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:51.651 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.531881+0000", 2024-09-18T10:15:51.651 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:51.651 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.651 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.651 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:51.651 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.531881+0000", 2024-09-18T10:15:51.651 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:51.651 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.651 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.651 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:51.651 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.531877+0000", 2024-09-18T10:15:51.651 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999962 2024-09-18T10:15:51.652 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.652 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.652 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:51.652 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.531894+0000", 2024-09-18T10:15:51.652 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7269999999999999e-05 2024-09-18T10:15:51.652 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.652 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.652 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:51.652 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.531939+0000", 2024-09-18T10:15:51.652 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4815999999999999e-05 2024-09-18T10:15:51.652 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.652 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.653 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:51.653 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.531963+0000", 2024-09-18T10:15:51.653 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.3312e-05 2024-09-18T10:15:51.653 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.653 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.653 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:51.653 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.532107+0000", 2024-09-18T10:15:51.653 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000144547 2024-09-18T10:15:51.653 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.653 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.653 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:51.654 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.532348+0000", 2024-09-18T10:15:51.654 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000240955 2024-09-18T10:15:51.654 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.654 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.654 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:51.654 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.839634+0000", 2024-09-18T10:15:51.654 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.30728566400000001 2024-09-18T10:15:51.654 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.654 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.654 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:51.654 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.841560+0000", 2024-09-18T10:15:51.654 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001926449 2024-09-18T10:15:51.654 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.655 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.655 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-18T10:15:51.655 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.943372+0000", 2024-09-18T10:15:51.655 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.101812176 2024-09-18T10:15:51.655 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.655 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.655 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:51.655 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.943456+0000", 2024-09-18T10:15:51.655 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.3614000000000003e-05 2024-09-18T10:15:51.655 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:51.655 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:51.655 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:51.655 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.656 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.656 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:168 3.6s0 3:6619023d:::smithi18925896-19:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:51.656 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:51.532000+0000", 2024-09-18T10:15:51.656 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 60.075941202000003, 2024-09-18T10:15:51.656 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.30898515999999998, 2024-09-18T10:15:51.656 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:51.656 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:51.656 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:51.656 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:51.656 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:51.656 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 168 2024-09-18T10:15:51.656 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.656 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:51.656 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.657 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:51.657 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.532000+0000", 2024-09-18T10:15:51.657 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:51.657 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.657 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.657 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:51.657 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.532000+0000", 2024-09-18T10:15:51.657 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:51.657 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.657 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.657 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:51.657 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.531998+0000", 2024-09-18T10:15:51.657 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:51.658 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.658 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.658 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:51.658 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.532009+0000", 2024-09-18T10:15:51.658 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0988999999999999e-05 2024-09-18T10:15:51.658 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.658 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.658 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:51.658 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.532032+0000", 2024-09-18T10:15:51.658 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2203000000000001e-05 2024-09-18T10:15:51.658 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.658 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.659 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:51.659 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.532043+0000", 2024-09-18T10:15:51.659 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1707e-05 2024-09-18T10:15:51.659 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.659 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.659 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:51.659 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.532663+0000", 2024-09-18T10:15:51.659 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000619993 2024-09-18T10:15:51.659 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.659 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.659 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for blocked object", 2024-09-18T10:15:51.659 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.532735+0000", 2024-09-18T10:15:51.659 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.2025000000000002e-05 2024-09-18T10:15:51.660 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.660 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.660 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:51.660 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.840712+0000", 2024-09-18T10:15:51.660 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.307976582 2024-09-18T10:15:51.660 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.660 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.660 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:51.660 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.840866+0000", 2024-09-18T10:15:51.660 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000154203 2024-09-18T10:15:51.660 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.660 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.660 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:51.661 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.840985+0000", 2024-09-18T10:15:51.661 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000119084 2024-09-18T10:15:51.661 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:51.661 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:51.661 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:51.661 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.661 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.661 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:206 3.6s0 3:61491e68:::smithi18925896-27:head [read 0~442368,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:51.661 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:52.694386+0000", 2024-09-18T10:15:51.661 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 58.913554865999998, 2024-09-18T10:15:51.661 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.40544065200000001, 2024-09-18T10:15:51.661 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:51.661 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:51.661 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:51.662 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:51.662 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:51.662 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 206 2024-09-18T10:15:51.662 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.662 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:51.662 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.662 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:51.662 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.694386+0000", 2024-09-18T10:15:51.662 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:51.662 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.662 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.662 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:51.662 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.694386+0000", 2024-09-18T10:15:51.663 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:51.663 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.663 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.663 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:51.663 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.694385+0000", 2024-09-18T10:15:51.663 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.999999 2024-09-18T10:15:51.663 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.663 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.663 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:51.663 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.694392+0000", 2024-09-18T10:15:51.663 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7970000000000003e-06 2024-09-18T10:15:51.663 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.663 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.664 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:51.664 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.694405+0000", 2024-09-18T10:15:51.664 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3406e-05 2024-09-18T10:15:51.664 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.664 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.664 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:51.664 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.694413+0000", 2024-09-18T10:15:51.664 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.9070000000000007e-06 2024-09-18T10:15:51.664 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.664 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.664 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:51.664 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.794592+0000", 2024-09-18T10:15:51.664 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100178879 2024-09-18T10:15:51.664 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.664 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.665 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:51.665 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.794598+0000", 2024-09-18T10:15:51.665 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.6289999999999998e-06 2024-09-18T10:15:51.665 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.665 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.665 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:51.665 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.796805+0000", 2024-09-18T10:15:51.665 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0022073930000000002 2024-09-18T10:15:51.665 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.665 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.665 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:51.665 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.796886+0000", 2024-09-18T10:15:51.666 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.0641000000000003e-05 2024-09-18T10:15:51.666 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.666 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.666 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:51.666 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:53.099827+0000", 2024-09-18T10:15:51.666 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.302941191 2024-09-18T10:15:51.666 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:51.666 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:51.666 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:51.666 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.666 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.666 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:323 3.6s0 3:61491e68:::smithi18925896-27:head [append 0~0] snapc 7=[4] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-18T10:15:51.666 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:01.795354+0000", 2024-09-18T10:15:51.667 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 49.812587352000001, 2024-09-18T10:15:51.667 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.30418701500000001, 2024-09-18T10:15:51.667 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:51.667 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:51.667 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:51.667 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:51.667 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:51.667 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 323 2024-09-18T10:15:51.667 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.667 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:51.667 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.667 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:51.667 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.795354+0000", 2024-09-18T10:15:51.668 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:51.668 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.668 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.668 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:51.668 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.795354+0000", 2024-09-18T10:15:51.668 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:51.668 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.668 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.668 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:51.668 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.795351+0000", 2024-09-18T10:15:51.668 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999971 2024-09-18T10:15:51.668 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.669 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.669 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:51.669 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.795365+0000", 2024-09-18T10:15:51.669 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3713999999999999e-05 2024-09-18T10:15:51.669 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.669 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.669 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:51.669 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.795406+0000", 2024-09-18T10:15:51.669 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.0846999999999997e-05 2024-09-18T10:15:51.669 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.669 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.669 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:51.669 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.795425+0000", 2024-09-18T10:15:51.670 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9936999999999999e-05 2024-09-18T10:15:51.670 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.670 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.670 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:51.670 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.795648+0000", 2024-09-18T10:15:51.670 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000222547 2024-09-18T10:15:51.670 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.670 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.670 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:51.670 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.795872+0000", 2024-09-18T10:15:51.670 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00022384799999999999 2024-09-18T10:15:51.670 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.670 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.671 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:51.671 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.797018+0000", 2024-09-18T10:15:51.671 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001146055 2024-09-18T10:15:51.671 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.671 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.671 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:51.671 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.798609+0000", 2024-09-18T10:15:51.671 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0015905959999999999 2024-09-18T10:15:51.671 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.671 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.671 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-18T10:15:51.671 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.099372+0000", 2024-09-18T10:15:51.671 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.30076301500000002 2024-09-18T10:15:51.672 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.672 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.672 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:51.672 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.099541+0000", 2024-09-18T10:15:51.672 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000169407 2024-09-18T10:15:51.672 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:51.672 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:51.672 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:51.672 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.672 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.672 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:324 3.6s0 3:61491e68:::smithi18925896-27:head [rollback 4] snapc 7=[4] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-18T10:15:51.673 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:01.795482+0000", 2024-09-18T10:15:51.673 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 49.812459738000001, 2024-09-18T10:15:51.673 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.30451560100000002, 2024-09-18T10:15:51.673 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:51.673 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:51.673 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:51.673 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:51.673 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:51.673 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 324 2024-09-18T10:15:51.673 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.673 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:51.673 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.673 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:51.673 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.795482+0000", 2024-09-18T10:15:51.674 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:51.674 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.674 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.674 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:51.674 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.795482+0000", 2024-09-18T10:15:51.674 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:51.674 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.674 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.674 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:51.674 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.795480+0000", 2024-09-18T10:15:51.674 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:51.674 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.674 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.675 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:51.675 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.795487+0000", 2024-09-18T10:15:51.675 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.7170000000000002e-06 2024-09-18T10:15:51.675 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.675 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.675 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:51.675 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.795548+0000", 2024-09-18T10:15:51.675 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1647000000000001e-05 2024-09-18T10:15:51.675 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.675 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.675 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:51.675 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.795566+0000", 2024-09-18T10:15:51.675 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7560000000000001e-05 2024-09-18T10:15:51.676 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.676 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.676 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:51.676 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.797676+0000", 2024-09-18T10:15:51.676 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0021101919999999999 2024-09-18T10:15:51.676 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.676 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.676 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:51.676 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.797742+0000", 2024-09-18T10:15:51.676 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.6351000000000002e-05 2024-09-18T10:15:51.676 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.677 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.677 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:51.677 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.798290+0000", 2024-09-18T10:15:51.677 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000548063 2024-09-18T10:15:51.677 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.677 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.677 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:51.677 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.800131+0000", 2024-09-18T10:15:51.678 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001840759 2024-09-18T10:15:51.678 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.678 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.678 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-18T10:15:51.678 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.099840+0000", 2024-09-18T10:15:51.678 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.29970881300000002 2024-09-18T10:15:51.678 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.678 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.678 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:51.678 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.099997+0000", 2024-09-18T10:15:51.678 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000157124 2024-09-18T10:15:51.679 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:51.679 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:51.679 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:51.679 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.679 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.679 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:325 3.6s0 3:61491e68:::smithi18925896-27:head [append 0~0] snapc 7=[4] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-18T10:15:51.679 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:01.795591+0000", 2024-09-18T10:15:51.679 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 49.812350682999998, 2024-09-18T10:15:51.679 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.30495726400000001, 2024-09-18T10:15:51.679 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:51.679 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:51.679 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:51.679 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:51.680 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:51.680 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 325 2024-09-18T10:15:51.680 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.680 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:51.680 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.680 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:51.680 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.795591+0000", 2024-09-18T10:15:51.680 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:51.680 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.680 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.680 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:51.680 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.795591+0000", 2024-09-18T10:15:51.680 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:51.681 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.681 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.681 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:51.681 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.795589+0000", 2024-09-18T10:15:51.681 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:51.681 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.681 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.681 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:51.681 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.795598+0000", 2024-09-18T10:15:51.681 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.7940000000000008e-06 2024-09-18T10:15:51.681 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.681 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.681 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:51.681 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.795617+0000", 2024-09-18T10:15:51.682 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8485e-05 2024-09-18T10:15:51.682 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.682 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.682 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:51.682 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.795627+0000", 2024-09-18T10:15:51.682 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0730999999999999e-05 2024-09-18T10:15:51.682 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.682 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.682 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:51.682 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.798672+0000", 2024-09-18T10:15:51.682 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00304484 2024-09-18T10:15:51.682 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.682 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.682 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:51.683 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.798918+0000", 2024-09-18T10:15:51.683 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00024598000000000003 2024-09-18T10:15:51.683 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.683 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.683 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:51.683 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.799703+0000", 2024-09-18T10:15:51.683 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000785262 2024-09-18T10:15:51.683 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.683 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.683 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:51.683 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:01.900558+0000", 2024-09-18T10:15:51.683 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100854108 2024-09-18T10:15:51.683 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.684 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.684 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-18T10:15:51.684 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.100434+0000", 2024-09-18T10:15:51.684 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.19987638699999999 2024-09-18T10:15:51.684 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.684 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.684 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:51.684 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:02.100548+0000", 2024-09-18T10:15:51.684 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00011390799999999999 2024-09-18T10:15:51.684 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:51.684 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:51.684 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:51.684 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.684 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.685 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWriteReply(3.6s0 57/40 ECSubWriteReply(tid=50, last_complete=57'30, committed=1, applied=1))", 2024-09-18T10:15:51.685 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:13.143948+0000", 2024-09-18T10:15:51.685 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 38.463993641000002, 2024-09-18T10:15:51.685 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.21095676099999999, 2024-09-18T10:15:51.685 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:51.685 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:51.685 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:51.685 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.685 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:51.685 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.143948+0000", 2024-09-18T10:15:51.685 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:51.685 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.685 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.685 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:51.686 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.143948+0000", 2024-09-18T10:15:51.686 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:51.686 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.686 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.686 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:51.686 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.143947+0000", 2024-09-18T10:15:51.686 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:51.686 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.686 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.686 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:51.686 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.143950+0000", 2024-09-18T10:15:51.686 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.8959999999999999e-06 2024-09-18T10:15:51.687 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.687 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.687 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:51.687 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.143958+0000", 2024-09-18T10:15:51.687 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7230000000000004e-06 2024-09-18T10:15:51.687 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.687 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.687 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:51.687 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.143961+0000", 2024-09-18T10:15:51.687 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0010000000000002e-06 2024-09-18T10:15:51.687 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.687 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.687 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:51.688 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.354877+0000", 2024-09-18T10:15:51.688 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.21091578599999999 2024-09-18T10:15:51.688 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.688 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.688 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:51.688 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.354904+0000", 2024-09-18T10:15:51.688 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.779e-05 2024-09-18T10:15:51.688 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:51.688 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:51.688 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:51.688 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.688 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.688 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWriteReply(3.6s0 57/40 ECSubWriteReply(tid=51, last_complete=57'32, committed=1, applied=1))", 2024-09-18T10:15:51.689 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:13.144581+0000", 2024-09-18T10:15:51.689 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 38.463359945999997, 2024-09-18T10:15:51.689 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.210571697, 2024-09-18T10:15:51.689 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:51.689 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:51.689 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:51.689 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.689 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:51.689 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.144581+0000", 2024-09-18T10:15:51.689 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:51.689 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.689 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.689 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:51.689 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.144581+0000", 2024-09-18T10:15:51.689 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:51.690 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.690 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.690 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:51.690 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.144581+0000", 2024-09-18T10:15:51.690 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:51.690 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.690 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.690 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:51.690 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.144584+0000", 2024-09-18T10:15:51.690 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.4760000000000001e-06 2024-09-18T10:15:51.690 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.690 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.690 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:51.691 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.144591+0000", 2024-09-18T10:15:51.691 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1969999999999999e-06 2024-09-18T10:15:51.691 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.691 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.691 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:51.691 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.144595+0000", 2024-09-18T10:15:51.691 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9899999999999999e-06 2024-09-18T10:15:51.691 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.691 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.691 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:51.691 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.355136+0000", 2024-09-18T10:15:51.691 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.21054089200000001 2024-09-18T10:15:51.691 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.691 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.692 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:51.692 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.355153+0000", 2024-09-18T10:15:51.692 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7442999999999999e-05 2024-09-18T10:15:51.692 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:51.692 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:51.692 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:51.692 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.692 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.692 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWriteReply(3.6s0 57/40 ECSubWriteReply(tid=51, last_complete=57'32, committed=1, applied=1))", 2024-09-18T10:15:51.692 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:13.144719+0000", 2024-09-18T10:15:51.692 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 38.463222002999998, 2024-09-18T10:15:51.692 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.210498871, 2024-09-18T10:15:51.693 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:51.693 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:51.693 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:51.693 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.693 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:51.693 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.144719+0000", 2024-09-18T10:15:51.693 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:51.693 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.693 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.694 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:51.694 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.144719+0000", 2024-09-18T10:15:51.694 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:51.694 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.694 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.694 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:51.694 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.144719+0000", 2024-09-18T10:15:51.694 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:51.694 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.694 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.694 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:51.694 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.144721+0000", 2024-09-18T10:15:51.695 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0169999999999999e-06 2024-09-18T10:15:51.695 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.695 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.695 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:51.695 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.144728+0000", 2024-09-18T10:15:51.695 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.9600000000000003e-06 2024-09-18T10:15:51.695 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.695 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.695 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:51.695 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.144731+0000", 2024-09-18T10:15:51.695 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.7760000000000002e-06 2024-09-18T10:15:51.695 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.695 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.696 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:51.696 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.355198+0000", 2024-09-18T10:15:51.696 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.210467401 2024-09-18T10:15:51.696 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.696 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.696 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:51.696 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.355218+0000", 2024-09-18T10:15:51.696 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9979999999999998e-05 2024-09-18T10:15:51.696 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:51.696 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:51.696 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:51.696 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.696 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.697 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDScrubReserve(3.6s0 REJECT e62)", 2024-09-18T10:15:51.697 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:18.118418+0000", 2024-09-18T10:15:51.697 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 33.489523646000002, 2024-09-18T10:15:51.697 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.379472953, 2024-09-18T10:15:51.697 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:51.697 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:51.697 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:51.697 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.697 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:51.697 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:18.118418+0000", 2024-09-18T10:15:51.697 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:51.697 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.697 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.698 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:51.698 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:18.118418+0000", 2024-09-18T10:15:51.698 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:51.698 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.698 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.698 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:51.698 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:18.118416+0000", 2024-09-18T10:15:51.698 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:51.698 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.698 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.699 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:51.699 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:18.118424+0000", 2024-09-18T10:15:51.699 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.576e-06 2024-09-18T10:15:51.699 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.699 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.699 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:51.699 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:18.118450+0000", 2024-09-18T10:15:51.699 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6673e-05 2024-09-18T10:15:51.699 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.699 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.699 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:51.699 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:18.218541+0000", 2024-09-18T10:15:51.699 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100090107 2024-09-18T10:15:51.700 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.700 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.700 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:51.700 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:18.497802+0000", 2024-09-18T10:15:51.700 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.279261392 2024-09-18T10:15:51.700 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.700 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.700 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:51.700 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:18.497866+0000", 2024-09-18T10:15:51.700 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.3989e-05 2024-09-18T10:15:51.700 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.700 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.701 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:51.701 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:18.497873+0000", 2024-09-18T10:15:51.701 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1500000000000002e-06 2024-09-18T10:15:51.701 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.701 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.701 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:51.701 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:18.497891+0000", 2024-09-18T10:15:51.701 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7462999999999999e-05 2024-09-18T10:15:51.701 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:51.701 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:51.701 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:51.701 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.701 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.701 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(osd.7.22:5 3.6s0 3:6619023d:::smithi18925896-19:head [assert-version v4,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-18T10:15:51.702 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:19.216031+0000", 2024-09-18T10:15:51.702 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 32.391910209000002, 2024-09-18T10:15:51.702 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.40429900499999999, 2024-09-18T10:15:51.702 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:51.702 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:51.702 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:51.702 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.702 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:51.702 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:19.216031+0000", 2024-09-18T10:15:51.702 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:51.702 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.702 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.702 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:51.703 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:19.216031+0000", 2024-09-18T10:15:51.703 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:51.703 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.703 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.703 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:51.703 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:19.216028+0000", 2024-09-18T10:15:51.703 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999971 2024-09-18T10:15:51.703 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.703 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.703 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:51.703 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:19.216046+0000", 2024-09-18T10:15:51.703 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7166000000000001e-05 2024-09-18T10:15:51.703 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.704 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.704 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:51.704 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:19.216075+0000", 2024-09-18T10:15:51.704 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9862999999999999e-05 2024-09-18T10:15:51.704 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.704 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.704 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:51.704 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:19.316231+0000", 2024-09-18T10:15:51.704 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10015547599999999 2024-09-18T10:15:51.704 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.704 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.704 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:51.704 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:19.316393+0000", 2024-09-18T10:15:51.705 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00016166999999999999 2024-09-18T10:15:51.705 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.705 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.705 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:51.705 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:19.316691+0000", 2024-09-18T10:15:51.705 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00029805700000000001 2024-09-18T10:15:51.705 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.705 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.705 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:51.705 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:19.620330+0000", 2024-09-18T10:15:51.705 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.30363942100000002 2024-09-18T10:15:51.705 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:51.705 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:51.706 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:51.706 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.706 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.706 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:681 3.6s0 3:646542dd:::smithi18925896-18:head [copy-from ver 0] snapc 28=[28,26,20,1f,1e,1a,11] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-09-18T10:15:51.706 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:33.588980+0000", 2024-09-18T10:15:51.706 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 18.018960986, 2024-09-18T10:15:51.706 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.30778096900000002, 2024-09-18T10:15:51.706 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:51.706 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:51.707 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:51.707 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:51.707 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:51.707 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 681 2024-09-18T10:15:51.707 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.707 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:51.707 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.707 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:51.707 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:33.588980+0000", 2024-09-18T10:15:51.707 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:51.707 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.707 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.707 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:51.707 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:33.588980+0000", 2024-09-18T10:15:51.708 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:51.708 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.708 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.708 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:51.708 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:33.588977+0000", 2024-09-18T10:15:51.708 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999967 2024-09-18T10:15:51.708 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.708 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.708 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:51.708 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:33.588995+0000", 2024-09-18T10:15:51.708 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8309e-05 2024-09-18T10:15:51.708 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.708 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.708 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:51.709 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:33.589037+0000", 2024-09-18T10:15:51.709 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.1412999999999998e-05 2024-09-18T10:15:51.709 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.709 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.709 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:51.709 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:33.689144+0000", 2024-09-18T10:15:51.709 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100106977 2024-09-18T10:15:51.709 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.710 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.710 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:51.710 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:33.689280+0000", 2024-09-18T10:15:51.710 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000136174 2024-09-18T10:15:51.710 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.710 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.710 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op must wait for map", 2024-09-18T10:15:51.710 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:33.689292+0000", 2024-09-18T10:15:51.710 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1528999999999999e-05 2024-09-18T10:15:51.710 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.710 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.710 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:51.710 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:33.792469+0000", 2024-09-18T10:15:51.711 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.103177796 2024-09-18T10:15:51.711 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.711 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.711 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:51.711 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:33.792711+0000", 2024-09-18T10:15:51.711 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00024172000000000001 2024-09-18T10:15:51.711 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.711 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.711 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:51.711 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:33.896761+0000", 2024-09-18T10:15:51.711 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.104050246 2024-09-18T10:15:51.711 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:51.711 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:51.711 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:51.712 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.712 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.712 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:690 3.6s0 3:647d434f:::smithi18925896-1:head [read 0~65536,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-18T10:15:51.712 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:35.579593+0000", 2024-09-18T10:15:51.712 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 16.028348419, 2024-09-18T10:15:51.712 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.30995702400000003, 2024-09-18T10:15:51.712 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:51.712 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:51.712 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:51.712 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:51.712 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:51.712 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 690 2024-09-18T10:15:51.713 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.713 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:51.713 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.713 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:51.713 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:35.579593+0000", 2024-09-18T10:15:51.713 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:51.713 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.713 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.713 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:51.713 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:35.579593+0000", 2024-09-18T10:15:51.713 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:51.713 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.713 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.714 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:51.714 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:35.579592+0000", 2024-09-18T10:15:51.714 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.999999 2024-09-18T10:15:51.714 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.714 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.714 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:51.714 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:35.579618+0000", 2024-09-18T10:15:51.714 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6512999999999998e-05 2024-09-18T10:15:51.714 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.714 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.714 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:51.715 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:35.579648+0000", 2024-09-18T10:15:51.715 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9679000000000002e-05 2024-09-18T10:15:51.715 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.715 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.715 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:51.715 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:35.579657+0000", 2024-09-18T10:15:51.715 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.3899999999999999e-06 2024-09-18T10:15:51.715 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.715 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.715 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:51.715 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:35.579818+0000", 2024-09-18T10:15:51.715 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000161027 2024-09-18T10:15:51.715 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.716 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.716 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:51.716 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:35.579828+0000", 2024-09-18T10:15:51.716 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.8330000000000006e-06 2024-09-18T10:15:51.716 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.716 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.716 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:51.716 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:35.787489+0000", 2024-09-18T10:15:51.716 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.20766110700000001 2024-09-18T10:15:51.716 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.716 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.716 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:51.716 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:35.787666+0000", 2024-09-18T10:15:51.717 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00017699299999999999 2024-09-18T10:15:51.717 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:51.717 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:51.717 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:51.717 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:35.889550+0000", 2024-09-18T10:15:51.717 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.101883602 2024-09-18T10:15:51.717 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:51.717 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:51.717 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:51.717 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:51.717 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:51.717 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-18T10:15:51.719 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T10:15:51.719 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-09-18T10:15:51.714+0000 7fdfa8698700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22087) UID: 0 2024-09-18T10:15:51.820 INFO:tasks.ceph.osd.11.smithi189.stderr:2024-09-18T10:15:51.814+0000 7f718b60a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 22400) UID: 0 2024-09-18T10:15:51.821 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-18T10:15:51.821 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-18T10:15:51.822 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 0 2024-09-18T10:15:51.822 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-18T10:15:51.831 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T10:15:51.921 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-18T10:15:51.916+0000 7f578c52f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22134) UID: 0 2024-09-18T10:15:51.926 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-18T10:15:51.926 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-18T10:15:51.926 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-18T10:15:51.926 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-18T10:15:51.926 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-18T10:15:51.935 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T10:15:52.022 INFO:tasks.ceph.osd.15.smithi189.stderr:2024-09-18T10:15:52.018+0000 7fd0e725e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 22401) UID: 0 2024-09-18T10:15:52.041 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-18T10:15:52.041 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-18T10:15:52.041 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-18T10:15:52.041 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-18T10:15:52.042 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.042 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:129 3.7s0 3:e4da5b5e:::smithi18925896-43:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:52.042 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:50.713961+0000", 2024-09-18T10:15:52.042 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 61.323259063999998, 2024-09-18T10:15:52.042 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.819334071, 2024-09-18T10:15:52.042 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:52.042 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:52.042 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:52.042 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:52.042 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:52.042 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 129 2024-09-18T10:15:52.042 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.042 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:52.042 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.043 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:52.043 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.713961+0000", 2024-09-18T10:15:52.043 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:52.043 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.043 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.043 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:52.043 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.713961+0000", 2024-09-18T10:15:52.043 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:52.043 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.043 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.043 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:52.043 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.713960+0000", 2024-09-18T10:15:52.043 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.999999 2024-09-18T10:15:52.044 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.044 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.044 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:52.044 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.713964+0000", 2024-09-18T10:15:52.044 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.3059999999999999e-06 2024-09-18T10:15:52.044 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.044 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.044 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:52.044 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.713974+0000", 2024-09-18T10:15:52.044 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.9969999999999993e-06 2024-09-18T10:15:52.044 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.044 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.044 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:52.045 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.713981+0000", 2024-09-18T10:15:52.045 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.1229999999999997e-06 2024-09-18T10:15:52.045 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.045 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.045 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:52.045 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.000661+0000", 2024-09-18T10:15:52.045 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.28668068699999999 2024-09-18T10:15:52.045 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.045 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.045 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for rw locks", 2024-09-18T10:15:52.045 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.000844+0000", 2024-09-18T10:15:52.045 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00018289599999999999 2024-09-18T10:15:52.045 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.045 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.046 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:52.046 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.219209+0000", 2024-09-18T10:15:52.046 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.21836466900000001 2024-09-18T10:15:52.046 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.046 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.046 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:52.046 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.219351+0000", 2024-09-18T10:15:52.046 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00014262300000000001 2024-09-18T10:15:52.046 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.046 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.046 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:52.046 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.533295+0000", 2024-09-18T10:15:52.046 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.31394377299999998 2024-09-18T10:15:52.047 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:52.047 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:52.047 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:52.047 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.047 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.047 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:159 3.7s0 3:ead53b0f:::smithi18925896-25:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:52.047 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:51.226988+0000", 2024-09-18T10:15:52.047 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 60.810232004, 2024-09-18T10:15:52.047 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.81465928499999996, 2024-09-18T10:15:52.047 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:52.048 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:52.048 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:52.048 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:52.048 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:52.048 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 159 2024-09-18T10:15:52.048 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.048 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:52.048 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.048 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:52.048 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.226988+0000", 2024-09-18T10:15:52.048 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:52.048 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.048 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.049 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:52.049 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.226988+0000", 2024-09-18T10:15:52.049 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:52.049 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.049 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.049 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:52.049 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.226987+0000", 2024-09-18T10:15:52.049 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.999999 2024-09-18T10:15:52.049 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.049 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.049 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:52.049 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.226992+0000", 2024-09-18T10:15:52.049 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.5900000000000001e-06 2024-09-18T10:15:52.049 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.050 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.050 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:52.050 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.227003+0000", 2024-09-18T10:15:52.050 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1222e-05 2024-09-18T10:15:52.050 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.050 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.050 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:52.050 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.227011+0000", 2024-09-18T10:15:52.050 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.6279999999999996e-06 2024-09-18T10:15:52.050 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.050 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.050 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:52.050 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.533792+0000", 2024-09-18T10:15:52.051 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.30678158100000003 2024-09-18T10:15:52.051 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.051 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.051 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for rw locks", 2024-09-18T10:15:52.051 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.533941+0000", 2024-09-18T10:15:52.051 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00014846 2024-09-18T10:15:52.051 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.051 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.051 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:52.051 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.633576+0000", 2024-09-18T10:15:52.051 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.099635153000000004 2024-09-18T10:15:52.051 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.051 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.052 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:52.052 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.633832+0000", 2024-09-18T10:15:52.052 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00025610699999999999 2024-09-18T10:15:52.052 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.052 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.052 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:52.052 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.041647+0000", 2024-09-18T10:15:52.052 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.407815388 2024-09-18T10:15:52.052 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:52.052 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:52.052 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:52.052 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.052 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.052 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(osd.10.22:2 3.7s0 3:ee962131:::smithi18925896-29:head [assert-version v6,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:52.053 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:52.798083+0000", 2024-09-18T10:15:52.053 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 59.239136827000003, 2024-09-18T10:15:52.053 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.60989146800000005, 2024-09-18T10:15:52.053 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:52.053 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:52.053 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:52.053 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.053 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:52.053 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.798083+0000", 2024-09-18T10:15:52.053 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:52.053 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.053 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.053 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:52.054 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.798083+0000", 2024-09-18T10:15:52.054 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:52.054 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.054 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.054 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:52.054 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.798080+0000", 2024-09-18T10:15:52.054 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999962 2024-09-18T10:15:52.054 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.054 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.054 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:52.054 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.798097+0000", 2024-09-18T10:15:52.054 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7657000000000001e-05 2024-09-18T10:15:52.054 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.054 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.055 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:52.055 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.798127+0000", 2024-09-18T10:15:52.055 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9532000000000001e-05 2024-09-18T10:15:52.055 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.055 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.055 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:52.055 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.898205+0000", 2024-09-18T10:15:52.055 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100078525 2024-09-18T10:15:52.055 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.055 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.055 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:52.055 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:53.204957+0000", 2024-09-18T10:15:52.055 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.30675138899999999 2024-09-18T10:15:52.055 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.055 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.056 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:52.056 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:53.205092+0000", 2024-09-18T10:15:52.056 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00013556700000000001 2024-09-18T10:15:52.056 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.056 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.056 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:52.056 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:53.407975+0000", 2024-09-18T10:15:52.056 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.20288240799999999 2024-09-18T10:15:52.056 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:52.056 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:52.056 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:52.056 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.056 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.056 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:437 3.1s0 3:82c9401b:::smithi18925896-33:head [copy-from ver 2] snapc 13=[13,12,11,10,f,e,b,4] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-18T10:15:52.057 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:11.831109+0000", 2024-09-18T10:15:52.057 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 40.206111049, 2024-09-18T10:15:52.057 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.709244349, 2024-09-18T10:15:52.057 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:52.057 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:52.057 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:52.057 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:52.057 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:52.057 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 437 2024-09-18T10:15:52.057 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.057 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:52.057 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.057 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:52.058 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:11.831109+0000", 2024-09-18T10:15:52.058 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:52.058 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.058 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.058 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:52.058 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:11.831109+0000", 2024-09-18T10:15:52.058 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:52.058 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.058 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.058 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:52.058 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:11.831106+0000", 2024-09-18T10:15:52.058 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999971 2024-09-18T10:15:52.058 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.059 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.059 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:52.059 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:11.831122+0000", 2024-09-18T10:15:52.059 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.5971e-05 2024-09-18T10:15:52.059 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.059 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.059 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:52.059 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:11.831155+0000", 2024-09-18T10:15:52.059 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.3259999999999997e-05 2024-09-18T10:15:52.059 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.059 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.059 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:52.059 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:11.831172+0000", 2024-09-18T10:15:52.060 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.6433999999999999e-05 2024-09-18T10:15:52.060 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.060 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.060 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:52.060 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:11.931398+0000", 2024-09-18T10:15:52.060 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10022644 2024-09-18T10:15:52.060 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.060 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.060 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op must wait for map", 2024-09-18T10:15:52.060 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:11.931412+0000", 2024-09-18T10:15:52.060 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3586e-05 2024-09-18T10:15:52.060 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.060 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.060 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:52.061 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:11.934231+0000", 2024-09-18T10:15:52.061 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0028191819999999999 2024-09-18T10:15:52.061 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.061 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.061 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:52.061 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:11.934501+0000", 2024-09-18T10:15:52.061 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00026996099999999999 2024-09-18T10:15:52.061 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.061 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.061 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:52.061 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.352429+0000", 2024-09-18T10:15:52.061 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.41792792699999998 2024-09-18T10:15:52.061 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.062 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.062 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:52.062 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.356112+0000", 2024-09-18T10:15:52.062 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0036833199999999999 2024-09-18T10:15:52.062 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.062 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.062 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-18T10:15:52.062 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.540229+0000", 2024-09-18T10:15:52.062 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.18411702599999999 2024-09-18T10:15:52.062 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.062 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.062 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:52.062 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.540354+0000", 2024-09-18T10:15:52.062 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00012415399999999999 2024-09-18T10:15:52.062 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:52.063 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:52.063 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:52.063 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.063 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.063 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:441 3.17s0 3:ead53b0f:::smithi18925896-25:head [append 0~606208 in=606208b] snapc 13=[13,12,11,10,f,e,b] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-18T10:15:52.063 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:12.856902+0000", 2024-09-18T10:15:52.063 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 39.180317946999999, 2024-09-18T10:15:52.063 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.80182116999999997, 2024-09-18T10:15:52.063 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:52.063 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:52.063 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:52.063 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:52.063 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:52.063 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 441 2024-09-18T10:15:52.064 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.064 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:52.064 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.064 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:52.064 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.856902+0000", 2024-09-18T10:15:52.064 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:52.064 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.064 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.064 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:52.064 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.856902+0000", 2024-09-18T10:15:52.064 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:52.064 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.064 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.064 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:52.065 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.856900+0000", 2024-09-18T10:15:52.065 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999976 2024-09-18T10:15:52.065 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.065 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.065 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:52.065 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.857946+0000", 2024-09-18T10:15:52.065 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0010459620000000001 2024-09-18T10:15:52.065 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.065 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.065 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:52.065 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.857973+0000", 2024-09-18T10:15:52.065 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6758999999999999e-05 2024-09-18T10:15:52.065 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.066 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.066 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:52.066 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.857983+0000", 2024-09-18T10:15:52.066 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0105000000000001e-05 2024-09-18T10:15:52.066 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.066 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.066 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:52.066 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.946351+0000", 2024-09-18T10:15:52.066 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.088368302999999995 2024-09-18T10:15:52.066 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.066 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.066 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:52.066 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.946465+0000", 2024-09-18T10:15:52.066 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00011425999999999999 2024-09-18T10:15:52.067 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.067 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.067 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:52.067 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.947656+0000", 2024-09-18T10:15:52.067 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0011901710000000001 2024-09-18T10:15:52.067 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.067 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.067 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:52.067 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.040655+0000", 2024-09-18T10:15:52.067 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.092999126000000001 2024-09-18T10:15:52.067 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.067 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.067 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-18T10:15:52.067 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.658646+0000", 2024-09-18T10:15:52.068 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.61799096399999998 2024-09-18T10:15:52.068 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.068 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.068 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:52.068 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.658723+0000", 2024-09-18T10:15:52.068 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7805000000000005e-05 2024-09-18T10:15:52.068 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:52.068 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:52.068 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:52.068 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.068 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.068 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:442 3.17s0 3:ead53b0f:::smithi18925896-25:head [append 0~720896 in=720896b] snapc 13=[13,12,11,10,f,e,b] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-18T10:15:52.068 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:12.861303+0000", 2024-09-18T10:15:52.069 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 39.175917552999998, 2024-09-18T10:15:52.069 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.89783094100000005, 2024-09-18T10:15:52.069 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:52.069 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:52.069 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:52.069 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:52.069 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:52.069 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 442 2024-09-18T10:15:52.069 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.069 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:52.069 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.069 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:52.069 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.861303+0000", 2024-09-18T10:15:52.069 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:52.070 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.070 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.070 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:52.070 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.861303+0000", 2024-09-18T10:15:52.070 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:52.070 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.070 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.070 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:52.070 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.861301+0000", 2024-09-18T10:15:52.070 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:52.070 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.070 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.070 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:52.071 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.862343+0000", 2024-09-18T10:15:52.071 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0010422820000000001 2024-09-18T10:15:52.071 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.071 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.071 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:52.071 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.862381+0000", 2024-09-18T10:15:52.071 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7407000000000003e-05 2024-09-18T10:15:52.071 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.071 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.071 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:52.071 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.862391+0000", 2024-09-18T10:15:52.071 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0572e-05 2024-09-18T10:15:52.071 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.072 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.072 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:52.072 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.948744+0000", 2024-09-18T10:15:52.072 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.086352683999999999 2024-09-18T10:15:52.072 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.072 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.072 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:52.072 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.948864+0000", 2024-09-18T10:15:52.072 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000119855 2024-09-18T10:15:52.072 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.072 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.072 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:52.072 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.949834+0000", 2024-09-18T10:15:52.073 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00097020199999999998 2024-09-18T10:15:52.073 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.073 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.073 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:52.073 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.040659+0000", 2024-09-18T10:15:52.073 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.090824659000000002 2024-09-18T10:15:52.073 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.073 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.073 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-18T10:15:52.073 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.759042+0000", 2024-09-18T10:15:52.073 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.71838323100000001 2024-09-18T10:15:52.073 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.073 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.074 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:52.074 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.759134+0000", 2024-09-18T10:15:52.074 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.1760000000000002e-05 2024-09-18T10:15:52.074 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:52.074 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:52.074 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:52.074 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.074 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.074 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:443 3.17s0 3:ead53b0f:::smithi18925896-25:head [append 0~49152 in=49152b] snapc 13=[13,12,11,10,f,e,b] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-18T10:15:52.075 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:12.862402+0000", 2024-09-18T10:15:52.075 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 39.174817775000001, 2024-09-18T10:15:52.075 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.99734852100000004, 2024-09-18T10:15:52.075 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:52.075 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:52.075 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:52.075 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:52.075 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:52.075 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 443 2024-09-18T10:15:52.075 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.075 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:52.075 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.075 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:52.075 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.862402+0000", 2024-09-18T10:15:52.076 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:52.076 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.076 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.076 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:52.076 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.862402+0000", 2024-09-18T10:15:52.076 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:52.076 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.076 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.076 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:52.076 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.862402+0000", 2024-09-18T10:15:52.076 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.999999 2024-09-18T10:15:52.076 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.076 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.077 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:52.077 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.862441+0000", 2024-09-18T10:15:52.077 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9116000000000003e-05 2024-09-18T10:15:52.077 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.077 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.077 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:52.077 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.862455+0000", 2024-09-18T10:15:52.077 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4501999999999999e-05 2024-09-18T10:15:52.077 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.077 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.077 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:52.077 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.862462+0000", 2024-09-18T10:15:52.077 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.4659999999999996e-06 2024-09-18T10:15:52.078 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.078 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.078 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:52.078 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.950570+0000", 2024-09-18T10:15:52.078 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.088107850000000001 2024-09-18T10:15:52.078 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.078 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.078 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:52.078 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.950675+0000", 2024-09-18T10:15:52.078 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000105445 2024-09-18T10:15:52.078 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.078 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.078 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:52.079 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.951279+0000", 2024-09-18T10:15:52.079 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00060383599999999996 2024-09-18T10:15:52.079 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.079 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.079 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:52.079 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.040661+0000", 2024-09-18T10:15:52.079 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.089382448000000003 2024-09-18T10:15:52.079 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.079 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.079 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-18T10:15:52.079 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.859668+0000", 2024-09-18T10:15:52.079 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.81900669800000003 2024-09-18T10:15:52.079 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.080 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.080 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:52.080 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.859751+0000", 2024-09-18T10:15:52.080 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.2943999999999997e-05 2024-09-18T10:15:52.080 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:52.080 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:52.080 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:52.080 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.080 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.080 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:444 3.17s0 3:ead53b0f:::smithi18925896-25:head [setxattr _header (57) in=64b] snapc 13=[13,12,11,10,f,e,b] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-18T10:15:52.080 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:12.862469+0000", 2024-09-18T10:15:52.080 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 39.174751565000001, 2024-09-18T10:15:52.080 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.99750343200000002, 2024-09-18T10:15:52.081 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:52.081 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:52.081 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:52.081 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:52.081 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:52.081 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 444 2024-09-18T10:15:52.081 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.081 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:52.081 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.081 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:52.081 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.862469+0000", 2024-09-18T10:15:52.081 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:52.081 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.081 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.082 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:52.082 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.862469+0000", 2024-09-18T10:15:52.082 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:52.082 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.082 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.082 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:52.082 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.862468+0000", 2024-09-18T10:15:52.082 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:52.082 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.082 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.082 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:52.082 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.862473+0000", 2024-09-18T10:15:52.082 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.5839999999999996e-06 2024-09-18T10:15:52.083 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.083 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.083 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:52.083 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.862483+0000", 2024-09-18T10:15:52.083 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0188e-05 2024-09-18T10:15:52.083 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.083 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.083 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:52.083 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.862487+0000", 2024-09-18T10:15:52.083 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6580000000000001e-06 2024-09-18T10:15:52.083 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.083 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.083 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:52.083 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.952057+0000", 2024-09-18T10:15:52.084 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.089569930000000006 2024-09-18T10:15:52.084 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.084 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.084 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:52.084 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.952227+0000", 2024-09-18T10:15:52.084 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000170106 2024-09-18T10:15:52.084 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.084 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.084 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:52.084 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.953127+0000", 2024-09-18T10:15:52.084 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00089958300000000002 2024-09-18T10:15:52.084 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.084 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.084 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:52.085 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.040664+0000", 2024-09-18T10:15:52.085 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.087536931999999998 2024-09-18T10:15:52.085 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.085 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.085 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-18T10:15:52.085 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.859892+0000", 2024-09-18T10:15:52.085 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.81922844699999997 2024-09-18T10:15:52.085 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.085 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.085 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:52.085 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.859972+0000", 2024-09-18T10:15:52.085 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.9662999999999999e-05 2024-09-18T10:15:52.085 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:52.086 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:52.086 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:52.086 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.086 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.086 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:445 3.17s0 3:ead53b0f:::smithi18925896-25:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e57)", 2024-09-18T10:15:52.086 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:12.862493+0000", 2024-09-18T10:15:52.086 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 39.174727697000002, 2024-09-18T10:15:52.086 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1938188730000001, 2024-09-18T10:15:52.086 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:52.086 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:52.086 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:52.086 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:52.086 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:52.086 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 445 2024-09-18T10:15:52.087 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.087 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:52.087 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.087 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:52.087 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.862493+0000", 2024-09-18T10:15:52.087 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:52.087 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.087 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.087 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:52.087 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.862493+0000", 2024-09-18T10:15:52.087 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:52.087 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.087 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.087 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:52.088 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.862492+0000", 2024-09-18T10:15:52.088 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:52.088 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.088 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.088 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:52.088 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.862495+0000", 2024-09-18T10:15:52.088 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1829999999999998e-06 2024-09-18T10:15:52.088 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.088 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.088 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:52.088 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.862506+0000", 2024-09-18T10:15:52.088 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0386e-05 2024-09-18T10:15:52.088 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.088 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.089 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:52.089 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.962582+0000", 2024-09-18T10:15:52.089 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100076741 2024-09-18T10:15:52.089 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.089 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.089 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:52.089 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.962632+0000", 2024-09-18T10:15:52.089 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.9364999999999999e-05 2024-09-18T10:15:52.089 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.089 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.089 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for rw locks", 2024-09-18T10:15:52.089 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.962715+0000", 2024-09-18T10:15:52.089 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.3617999999999998e-05 2024-09-18T10:15:52.089 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.090 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.090 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:52.090 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.860479+0000", 2024-09-18T10:15:52.090 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.89776372699999996 2024-09-18T10:15:52.090 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.090 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.090 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:52.090 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.860908+0000", 2024-09-18T10:15:52.090 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00042866099999999999 2024-09-18T10:15:52.090 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.090 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.090 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:52.090 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:14.056311+0000", 2024-09-18T10:15:52.090 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.19540364199999999 2024-09-18T10:15:52.091 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:52.091 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:52.091 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:52.091 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.091 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.091 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:650 3.bs0 3:d62b3dfa:::smithi18925896-40:head [append 0~0] snapc 21=[20,1f,1e,1a,16,11,10] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-18T10:15:52.091 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:29.784110+0000", 2024-09-18T10:15:52.091 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 22.253110027000002, 2024-09-18T10:15:52.091 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.68818560399999995, 2024-09-18T10:15:52.091 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:52.091 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:52.091 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:52.091 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:52.091 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:52.092 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 650 2024-09-18T10:15:52.092 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.092 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:52.092 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.092 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:52.092 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:29.784110+0000", 2024-09-18T10:15:52.092 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:52.092 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.092 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.092 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:52.092 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:29.784110+0000", 2024-09-18T10:15:52.092 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:52.092 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.092 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.093 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:52.093 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:29.784109+0000", 2024-09-18T10:15:52.093 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.999999 2024-09-18T10:15:52.093 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.093 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.093 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:52.093 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:29.784114+0000", 2024-09-18T10:15:52.093 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1719999999999998e-06 2024-09-18T10:15:52.093 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.093 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.093 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:52.093 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:29.784135+0000", 2024-09-18T10:15:52.093 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0693999999999999e-05 2024-09-18T10:15:52.093 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.094 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.094 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:52.094 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:29.784154+0000", 2024-09-18T10:15:52.094 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9218000000000001e-05 2024-09-18T10:15:52.094 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.094 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.094 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:52.094 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:29.884343+0000", 2024-09-18T10:15:52.094 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100188467 2024-09-18T10:15:52.094 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.094 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.094 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:52.094 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:29.884485+0000", 2024-09-18T10:15:52.094 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00014232599999999999 2024-09-18T10:15:52.095 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.095 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.095 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:52.095 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:29.885103+0000", 2024-09-18T10:15:52.095 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00061802700000000005 2024-09-18T10:15:52.095 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.095 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.095 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:52.095 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:29.885722+0000", 2024-09-18T10:15:52.095 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00061855099999999995 2024-09-18T10:15:52.095 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.095 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.095 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-18T10:15:52.095 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:30.472145+0000", 2024-09-18T10:15:52.096 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.58642300599999997 2024-09-18T10:15:52.096 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.096 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.096 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:52.096 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:30.472296+0000", 2024-09-18T10:15:52.096 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00015104300000000001 2024-09-18T10:15:52.096 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:52.096 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:52.096 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:52.096 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.096 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.096 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDPGPush(3.19s1 89/83 [PushOp(3:9bcb6eef:::smithi18925896-11:1a, version: 71'97, data_included: [262144~32768], data_size: 32768, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:9bcb6eef:::smithi18925896-11:1a@71'97, size: 1179648, copy_subset: [], clone_subset: {}, snapset: 1a=[]:{1a=[1a,16,12,11,10]}, object_exist: 1), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:1179648, data_complete:true, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to:1048576, data_complete:false, omap_recovered_to:, omap_complete:true, error:false))])", 2024-09-18T10:15:52.096 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:45.610598+0000", 2024-09-18T10:15:52.096 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 6.4266220430000001, 2024-09-18T10:15:52.097 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.65560690099999996, 2024-09-18T10:15:52.097 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:52.097 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:52.097 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:52.097 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.097 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:52.097 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:45.610598+0000", 2024-09-18T10:15:52.097 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:52.097 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.097 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.097 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:52.097 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:45.610598+0000", 2024-09-18T10:15:52.097 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:52.098 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.098 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.098 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:52.098 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:45.610598+0000", 2024-09-18T10:15:52.098 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:52.098 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.098 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.098 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:52.098 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:45.610598+0000", 2024-09-18T10:15:52.098 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:52.098 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.098 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.098 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:52.099 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:45.610683+0000", 2024-09-18T10:15:52.099 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.5241999999999995e-05 2024-09-18T10:15:52.099 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.099 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.099 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:52.099 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:45.710843+0000", 2024-09-18T10:15:52.099 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100159308 2024-09-18T10:15:52.099 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.099 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.099 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:52.099 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:46.262062+0000", 2024-09-18T10:15:52.099 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.55121898999999996 2024-09-18T10:15:52.099 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.099 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.099 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:52.100 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:46.266205+0000", 2024-09-18T10:15:52.100 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0041433609999999999 2024-09-18T10:15:52.100 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:52.100 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:52.100 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:52.100 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.100 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.100 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDPGPushReply(3.19s1 89/83 [PushReplyOp(3:9bcb6eef:::smithi18925896-11:1a)])", 2024-09-18T10:15:52.100 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:45.613489+0000", 2024-09-18T10:15:52.100 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 6.4237310040000004, 2024-09-18T10:15:52.100 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.85936725999999997, 2024-09-18T10:15:52.100 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:52.101 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:52.101 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:52.101 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.101 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:52.101 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:45.613489+0000", 2024-09-18T10:15:52.101 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:52.101 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.101 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.101 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:52.101 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:45.613489+0000", 2024-09-18T10:15:52.101 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:52.101 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.101 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.101 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:52.102 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:45.613489+0000", 2024-09-18T10:15:52.102 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:52.102 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.102 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.102 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:52.102 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:45.613497+0000", 2024-09-18T10:15:52.102 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.909e-06 2024-09-18T10:15:52.102 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.102 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.102 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:52.103 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:45.613527+0000", 2024-09-18T10:15:52.103 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.0565999999999998e-05 2024-09-18T10:15:52.103 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.103 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.103 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:52.103 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:45.713661+0000", 2024-09-18T10:15:52.103 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10013393800000001 2024-09-18T10:15:52.103 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.103 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.103 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:52.103 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:46.472828+0000", 2024-09-18T10:15:52.103 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.75916687299999996 2024-09-18T10:15:52.103 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.103 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.104 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:52.104 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:46.472856+0000", 2024-09-18T10:15:52.104 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.8557000000000001e-05 2024-09-18T10:15:52.104 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:52.104 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:52.104 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:52.104 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.104 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.104 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDPGPushReply(3.19s1 89/83 [PushReplyOp(3:9bcb6eef:::smithi18925896-11:1a)])", 2024-09-18T10:15:52.104 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:45.613632+0000", 2024-09-18T10:15:52.104 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 6.4235884990000001, 2024-09-18T10:15:52.104 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.85928448899999998, 2024-09-18T10:15:52.104 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:52.105 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:52.105 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:52.105 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.105 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:52.105 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:45.613632+0000", 2024-09-18T10:15:52.105 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:52.105 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.105 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.105 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:52.105 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:45.613632+0000", 2024-09-18T10:15:52.105 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:52.105 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.105 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.105 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:52.106 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:45.613631+0000", 2024-09-18T10:15:52.106 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:52.106 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.106 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.106 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:52.106 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:45.613635+0000", 2024-09-18T10:15:52.106 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.8099999999999999e-06 2024-09-18T10:15:52.106 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.106 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.106 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:52.106 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:45.613641+0000", 2024-09-18T10:15:52.106 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8919999999999997e-06 2024-09-18T10:15:52.106 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.106 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.107 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:52.107 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:45.713730+0000", 2024-09-18T10:15:52.107 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100088599 2024-09-18T10:15:52.107 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.107 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.107 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:52.107 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:46.472881+0000", 2024-09-18T10:15:52.107 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.75915170399999998 2024-09-18T10:15:52.107 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.107 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.107 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:52.107 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:46.472916+0000", 2024-09-18T10:15:52.107 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.4826000000000002e-05 2024-09-18T10:15:52.108 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:52.108 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:52.108 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:52.108 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.108 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.108 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:859 3.bs0 3:d6558826:::smithi18925896-3:head [copy-from ver 42] snapc 33=[33,30,2f,28,20,1f,1a,11] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-18T10:15:52.108 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:45.650066+0000", 2024-09-18T10:15:52.108 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 6.3871543380000002, 2024-09-18T10:15:52.108 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.61378902400000002, 2024-09-18T10:15:52.108 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:52.108 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:52.108 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:52.108 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:52.109 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:52.109 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 859 2024-09-18T10:15:52.109 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.109 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:52.109 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.109 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:52.109 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:45.650066+0000", 2024-09-18T10:15:52.109 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:52.109 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.109 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.109 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:52.109 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:45.650066+0000", 2024-09-18T10:15:52.109 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:52.109 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.110 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.110 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:52.110 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:45.650064+0000", 2024-09-18T10:15:52.110 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:52.110 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.110 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.110 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:52.110 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:45.650074+0000", 2024-09-18T10:15:52.110 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.9199999999999999e-06 2024-09-18T10:15:52.110 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.110 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.110 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:52.110 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:45.650097+0000", 2024-09-18T10:15:52.111 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2843000000000001e-05 2024-09-18T10:15:52.111 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.111 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.111 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:52.111 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:45.650106+0000", 2024-09-18T10:15:52.111 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.8200000000000003e-06 2024-09-18T10:15:52.111 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.111 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.111 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:52.111 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:45.650202+0000", 2024-09-18T10:15:52.111 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.6741000000000001e-05 2024-09-18T10:15:52.111 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.111 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.111 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op must wait for map", 2024-09-18T10:15:52.111 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:45.650212+0000", 2024-09-18T10:15:52.112 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.5889999999999992e-06 2024-09-18T10:15:52.112 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.112 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.112 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:52.112 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:46.009258+0000", 2024-09-18T10:15:52.112 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.359045685 2024-09-18T10:15:52.112 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.112 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.112 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:52.112 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:46.009452+0000", 2024-09-18T10:15:52.112 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000194981 2024-09-18T10:15:52.112 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.112 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.113 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:52.113 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:46.157911+0000", 2024-09-18T10:15:52.113 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.14845867300000001 2024-09-18T10:15:52.113 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.113 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.113 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:52.113 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:46.162138+0000", 2024-09-18T10:15:52.113 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0042265890000000002 2024-09-18T10:15:52.113 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.113 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.113 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-18T10:15:52.113 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:46.263653+0000", 2024-09-18T10:15:52.113 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10151505800000001 2024-09-18T10:15:52.114 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.114 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.114 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:52.114 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:46.263855+0000", 2024-09-18T10:15:52.114 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000202072 2024-09-18T10:15:52.114 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:52.114 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:52.114 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:52.114 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.114 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.114 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:864 3.19s1 3:9a1a774c:::smithi18925896-38:head [append 0~507904] snapc 33=[33,30,2f,28,20,1f,1a,11] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-18T10:15:52.114 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:45.681535+0000", 2024-09-18T10:15:52.114 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 6.3556850960000002, 2024-09-18T10:15:52.114 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.687005957, 2024-09-18T10:15:52.115 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:52.115 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:52.115 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:52.115 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:52.115 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:52.115 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 864 2024-09-18T10:15:52.115 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.115 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:52.115 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.115 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:52.115 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:45.681535+0000", 2024-09-18T10:15:52.115 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:52.115 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.116 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.116 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:52.116 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:45.681535+0000", 2024-09-18T10:15:52.116 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:52.116 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.116 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.116 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:52.116 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:45.681533+0000", 2024-09-18T10:15:52.116 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999976 2024-09-18T10:15:52.116 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.116 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.116 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:52.116 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:45.682152+0000", 2024-09-18T10:15:52.117 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00061891300000000002 2024-09-18T10:15:52.117 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.117 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.117 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:52.117 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:45.682188+0000", 2024-09-18T10:15:52.117 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.6053000000000002e-05 2024-09-18T10:15:52.117 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.117 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.117 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:52.117 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:45.682201+0000", 2024-09-18T10:15:52.117 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3129000000000001e-05 2024-09-18T10:15:52.117 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.117 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.117 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:52.118 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:46.044487+0000", 2024-09-18T10:15:52.118 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.36228644500000001 2024-09-18T10:15:52.118 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.118 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.118 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:52.118 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:46.045499+0000", 2024-09-18T10:15:52.118 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0010121150000000001 2024-09-18T10:15:52.118 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.118 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.118 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:52.118 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:46.049305+0000", 2024-09-18T10:15:52.118 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0038053060000000001 2024-09-18T10:15:52.119 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.119 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.119 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:52.119 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:46.266193+0000", 2024-09-18T10:15:52.119 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.21688789999999999 2024-09-18T10:15:52.119 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.119 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.119 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-18T10:15:52.119 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:46.367081+0000", 2024-09-18T10:15:52.119 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100888414 2024-09-18T10:15:52.119 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.119 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.119 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:52.119 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:46.368541+0000", 2024-09-18T10:15:52.120 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.001459952 2024-09-18T10:15:52.120 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:52.120 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:52.120 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:52.120 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.120 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.120 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:865 3.19s1 3:9a1a774c:::smithi18925896-38:head [setxattr _header (57) in=7b] snapc 33=[33,30,2f,28,20,1f,1a,11] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-18T10:15:52.120 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:45.682215+0000", 2024-09-18T10:15:52.120 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 6.3550051910000001, 2024-09-18T10:15:52.120 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.788677301, 2024-09-18T10:15:52.120 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:52.120 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:52.120 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:52.120 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:52.121 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:52.121 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 865 2024-09-18T10:15:52.121 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.121 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:52.121 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.121 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:52.121 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:45.682215+0000", 2024-09-18T10:15:52.121 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:52.121 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.121 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.121 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:52.121 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:45.682215+0000", 2024-09-18T10:15:52.121 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:52.122 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.122 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.122 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:52.122 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:45.682214+0000", 2024-09-18T10:15:52.122 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.999999 2024-09-18T10:15:52.122 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.122 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.122 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:52.122 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:45.682225+0000", 2024-09-18T10:15:52.122 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1440000000000001e-05 2024-09-18T10:15:52.122 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.122 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.122 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:52.122 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:45.682236+0000", 2024-09-18T10:15:52.123 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0607e-05 2024-09-18T10:15:52.123 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.123 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.123 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:52.123 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:45.682242+0000", 2024-09-18T10:15:52.123 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.8629999999999996e-06 2024-09-18T10:15:52.123 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.123 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.123 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:52.123 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:46.056507+0000", 2024-09-18T10:15:52.123 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.37426557100000002 2024-09-18T10:15:52.123 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.124 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.124 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:52.124 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:46.057213+0000", 2024-09-18T10:15:52.124 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00070576899999999997 2024-09-18T10:15:52.124 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.124 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.124 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:52.124 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:46.058807+0000", 2024-09-18T10:15:52.124 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0015942020000000001 2024-09-18T10:15:52.124 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.124 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.124 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:52.124 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:46.267918+0000", 2024-09-18T10:15:52.124 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.209110922 2024-09-18T10:15:52.125 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.125 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.125 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-18T10:15:52.125 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:46.469410+0000", 2024-09-18T10:15:52.125 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.20149214600000001 2024-09-18T10:15:52.125 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.125 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.125 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:52.125 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:46.470892+0000", 2024-09-18T10:15:52.125 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0014818679999999999 2024-09-18T10:15:52.125 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:52.125 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:52.125 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:52.126 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.126 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.126 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:866 3.19s1 3:9a1a774c:::smithi18925896-38:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e90)", 2024-09-18T10:15:52.126 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:45.682249+0000", 2024-09-18T10:15:52.126 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 6.3549708320000002, 2024-09-18T10:15:52.126 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.99524631799999996, 2024-09-18T10:15:52.126 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:52.126 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:52.126 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:52.126 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:52.126 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:52.126 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 866 2024-09-18T10:15:52.126 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.127 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:52.127 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.127 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:52.127 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:45.682249+0000", 2024-09-18T10:15:52.127 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:52.127 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.127 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.127 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:52.127 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:45.682249+0000", 2024-09-18T10:15:52.127 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:52.127 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.127 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.127 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:52.128 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:45.682249+0000", 2024-09-18T10:15:52.128 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.999999 2024-09-18T10:15:52.128 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.128 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.128 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:52.128 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:45.682253+0000", 2024-09-18T10:15:52.128 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.3159999999999998e-06 2024-09-18T10:15:52.128 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.128 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.128 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:52.128 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:45.682263+0000", 2024-09-18T10:15:52.128 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0013999999999999e-05 2024-09-18T10:15:52.128 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.129 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.129 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:52.129 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:45.682269+0000", 2024-09-18T10:15:52.129 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.0900000000000001e-06 2024-09-18T10:15:52.129 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.129 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.129 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:52.129 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:46.159872+0000", 2024-09-18T10:15:52.129 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.47760337899999999 2024-09-18T10:15:52.129 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.129 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.129 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for rw locks", 2024-09-18T10:15:52.129 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:46.161490+0000", 2024-09-18T10:15:52.129 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0016171390000000001 2024-09-18T10:15:52.130 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.130 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.130 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:52.130 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:46.472073+0000", 2024-09-18T10:15:52.130 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.31058312399999999 2024-09-18T10:15:52.130 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.130 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.130 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:52.130 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:46.472383+0000", 2024-09-18T10:15:52.130 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00031024500000000001 2024-09-18T10:15:52.130 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.130 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.130 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:52.130 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:46.677496+0000", 2024-09-18T10:15:52.131 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.205112814 2024-09-18T10:15:52.131 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:52.131 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:52.131 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:52.131 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.131 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.131 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDPGPushReply(3.19s1 89/83 [PushReplyOp(3:9bcb6eef:::smithi18925896-11:1a)])", 2024-09-18T10:15:52.131 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:45.713141+0000", 2024-09-18T10:15:52.131 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 6.3240788500000003, 2024-09-18T10:15:52.131 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.75595816900000001, 2024-09-18T10:15:52.131 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:52.131 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:52.131 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:52.131 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.132 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:52.132 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:45.713141+0000", 2024-09-18T10:15:52.132 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:52.132 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.132 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.132 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:52.132 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:45.713141+0000", 2024-09-18T10:15:52.132 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:52.132 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.132 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.132 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:52.132 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:45.713141+0000", 2024-09-18T10:15:52.132 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:52.133 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.133 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.133 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:52.133 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:45.713149+0000", 2024-09-18T10:15:52.133 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.4829999999999999e-06 2024-09-18T10:15:52.133 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.133 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.133 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:52.133 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:45.713192+0000", 2024-09-18T10:15:52.133 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.2685999999999998e-05 2024-09-18T10:15:52.133 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.133 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.133 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:52.133 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:45.713207+0000", 2024-09-18T10:15:52.134 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4780000000000001e-05 2024-09-18T10:15:52.134 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.134 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.134 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:52.134 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:46.468995+0000", 2024-09-18T10:15:52.134 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.75578806799999998 2024-09-18T10:15:52.134 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.134 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.134 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:52.134 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:46.469100+0000", 2024-09-18T10:15:52.134 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000104595 2024-09-18T10:15:52.134 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:52.134 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:52.134 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:52.135 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.135 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.135 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDPGPushReply(3.19s1 89/83 [PushReplyOp(3:9bcb6eef:::smithi18925896-11:1a)])", 2024-09-18T10:15:52.135 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:45.713934+0000", 2024-09-18T10:15:52.135 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 6.3232858499999995, 2024-09-18T10:15:52.135 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.85936337799999996, 2024-09-18T10:15:52.135 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:52.135 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:52.135 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:52.135 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.135 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:52.135 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:45.713934+0000", 2024-09-18T10:15:52.135 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:52.135 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.135 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.136 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:52.136 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:45.713934+0000", 2024-09-18T10:15:52.136 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:52.136 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.136 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.136 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:52.136 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:45.713934+0000", 2024-09-18T10:15:52.136 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.999999 2024-09-18T10:15:52.136 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.136 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.136 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:52.136 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:45.713942+0000", 2024-09-18T10:15:52.136 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.2770000000000005e-06 2024-09-18T10:15:52.137 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.137 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.137 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:52.137 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:45.713954+0000", 2024-09-18T10:15:52.137 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2329e-05 2024-09-18T10:15:52.137 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.137 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.137 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:52.137 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:45.713964+0000", 2024-09-18T10:15:52.137 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.8770000000000005e-06 2024-09-18T10:15:52.137 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.137 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.137 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:52.137 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:46.573203+0000", 2024-09-18T10:15:52.138 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.859239211 2024-09-18T10:15:52.138 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.138 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.138 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:52.138 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:46.573298+0000", 2024-09-18T10:15:52.138 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.4457999999999996e-05 2024-09-18T10:15:52.138 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:52.138 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:52.138 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:52.138 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.138 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.138 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDPGPushReply(3.19s1 89/83 [PushReplyOp(3:9bcb6eef:::smithi18925896-11:1a)])", 2024-09-18T10:15:52.138 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:45.714046+0000", 2024-09-18T10:15:52.138 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 6.323174506, 2024-09-18T10:15:52.139 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0604996339999999, 2024-09-18T10:15:52.139 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:52.139 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:52.139 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:52.139 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.139 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:52.139 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:45.714046+0000", 2024-09-18T10:15:52.139 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:52.139 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.139 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.139 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:52.139 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:45.714046+0000", 2024-09-18T10:15:52.139 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:52.140 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.140 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.140 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:52.140 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:45.714045+0000", 2024-09-18T10:15:52.140 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:52.140 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.140 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.140 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:52.140 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:45.714051+0000", 2024-09-18T10:15:52.141 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.7799999999999997e-06 2024-09-18T10:15:52.141 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.141 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.141 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:52.141 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:45.714062+0000", 2024-09-18T10:15:52.141 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1126e-05 2024-09-18T10:15:52.141 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.141 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.141 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:52.141 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:45.714070+0000", 2024-09-18T10:15:52.141 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7670000000000003e-06 2024-09-18T10:15:52.141 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.141 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.142 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:52.142 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:46.774452+0000", 2024-09-18T10:15:52.142 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0603819459999999 2024-09-18T10:15:52.142 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.142 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.142 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:52.142 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:46.774545+0000", 2024-09-18T10:15:52.142 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.3572000000000005e-05 2024-09-18T10:15:52.142 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:52.142 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:52.142 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:52.142 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:52.142 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:52.142 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-18T10:15:52.143 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T10:15:52.145 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-09-18T10:15:52.146 INFO:tasks.rados.rados.0.smithi189.stdout:366: done (3 left) 2024-09-18T10:15:52.147 INFO:tasks.rados.rados.0.smithi189.stdout:367: done (2 left) 2024-09-18T10:15:52.147 INFO:tasks.rados.rados.0.smithi189.stdout:369: done (1 left) 2024-09-18T10:15:52.147 INFO:tasks.rados.rados.0.smithi189.stdout:370: done (0 left) 2024-09-18T10:15:52.147 INFO:tasks.rados.rados.0.smithi189.stdout:append oid 37 current snap is 31 2024-09-18T10:15:52.147 INFO:tasks.rados.rados.0.smithi189.stdout:371: seq_num 143 ranges {2129920=458752,2588672=475136,3063808=655360,3719168=671744,4390912=147456} 2024-09-18T10:15:52.147 INFO:tasks.rados.rados.0.smithi189.stdout:371: writing smithi18925896-37 from 2129920 to 2588672 tid 1 2024-09-18T10:15:52.147 INFO:tasks.rados.rados.0.smithi189.stdout:371: writing smithi18925896-37 from 2588672 to 3063808 tid 2 2024-09-18T10:15:52.147 INFO:tasks.rados.rados.0.smithi189.stdout:371: writing smithi18925896-37 from 3063808 to 3719168 tid 3 2024-09-18T10:15:52.147 INFO:tasks.rados.rados.0.smithi189.stdout:371: writing smithi18925896-37 from 3719168 to 4390912 tid 4 2024-09-18T10:15:52.147 INFO:tasks.rados.rados.0.smithi189.stdout:371: writing smithi18925896-37 from 4390912 to 4538368 tid 5 2024-09-18T10:15:52.147 INFO:tasks.rados.rados.0.smithi189.stdout:372: copy_from oid 21 from oid 32 current snap is 31 2024-09-18T10:15:52.147 INFO:tasks.rados.rados.0.smithi189.stdout:373: setattr oid 43 current snap is 31 2024-09-18T10:15:52.147 INFO:tasks.rados.rados.0.smithi189.stdout:374: read oid 34 snap -1 2024-09-18T10:15:52.148 INFO:tasks.rados.rados.0.smithi189.stdout:374: expect (ObjNum 33 snap 0 seq_num 33) 2024-09-18T10:15:52.148 INFO:tasks.rados.rados.0.smithi189.stdout:375: read oid 12 snap -1 2024-09-18T10:15:52.148 INFO:tasks.rados.rados.0.smithi189.stdout:375: expect (ObjNum 137 snap 30 seq_num 137) 2024-09-18T10:15:52.148 INFO:tasks.rados.rados.0.smithi189.stdout:376: copy_from oid 15 from oid 3 current snap is 31 2024-09-18T10:15:52.148 INFO:tasks.rados.rados.0.smithi189.stdout:append oid 14 current snap is 31 2024-09-18T10:15:52.148 INFO:tasks.rados.rados.0.smithi189.stdout:377: seq_num 144 ranges {0=16384} 2024-09-18T10:15:52.148 INFO:tasks.rados.rados.0.smithi189.stdout:377: writing smithi18925896-14 from 0 to 16384 tid 1 2024-09-18T10:15:52.148 INFO:tasks.rados.rados.0.smithi189.stdout:378: copy_from oid 6 from oid 48 current snap is 31 2024-09-18T10:15:52.148 INFO:tasks.rados.rados.0.smithi189.stdout:379: snap_create 2024-09-18T10:15:52.149 INFO:tasks.ceph.osd.14.smithi154.stderr:2024-09-18T10:15:52.142+0000 7f75a19e7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 22093) UID: 0 2024-09-18T10:15:52.245 INFO:tasks.ceph.osd.14.smithi154.stderr:2024-09-18T10:15:52.242+0000 7f75a19e7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 22093) UID: 0 2024-09-18T10:15:52.245 INFO:tasks.rados.rados.0.smithi189.stdout:372: finishing copy_from racing read to smithi18925896-21 2024-09-18T10:15:52.255 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-18T10:15:52.256 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-18T10:15:52.256 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.256 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDPGPush(3.cs1 96/94 [PushOp(3:351da1bf:::smithi18925896-6:head, version: 85'122, data_included: [0~262144], data_size: 262144, omap_header_size: 0, omap_entries_size: 0, attrset_size: 4, recovery_info: ObjectRecoveryInfo(3:351da1bf:::smithi18925896-6:head@85'122, size: 3358720, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: 1), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:1048576, data_complete:false, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:true, error:false))])", 2024-09-18T10:15:52.256 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:52.092366+0000", 2024-09-18T10:15:52.256 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.162486608, 2024-09-18T10:15:52.256 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.162531918, 2024-09-18T10:15:52.256 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:52.256 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-18T10:15:52.256 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:52.256 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.256 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:52.257 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:52.092366+0000", 2024-09-18T10:15:52.257 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:52.257 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.257 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.257 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:52.257 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:52.092366+0000", 2024-09-18T10:15:52.257 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:52.257 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.257 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.257 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:52.257 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:52.092365+0000", 2024-09-18T10:15:52.257 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.999999 2024-09-18T10:15:52.257 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.258 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.258 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:52.258 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:52.092826+0000", 2024-09-18T10:15:52.258 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00046091699999999998 2024-09-18T10:15:52.258 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.258 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.258 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:52.258 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:52.092878+0000", 2024-09-18T10:15:52.258 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2658999999999998e-05 2024-09-18T10:15:52.258 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.258 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.258 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:52.258 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:52.193012+0000", 2024-09-18T10:15:52.259 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.100134272 2024-09-18T10:15:52.259 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:52.259 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:52.259 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:52.259 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.259 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.259 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDPGPush(3.16s3 96/94 [PushOp(3:6c64c4fa:::smithi18925896-41:33, version: 90'38, data_included: [524288~262144], data_size: 262144, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:6c64c4fa:::smithi18925896-41:33@90'38, size: 4046848, copy_subset: [], clone_subset: {}, snapset: 33=[]:{16=[11],2f=[2f,28,20,1f],33=[33,30]}, object_exist: 1), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:3145728, data_complete:false, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to:2097152, data_complete:false, omap_recovered_to:, omap_complete:true, error:false))])", 2024-09-18T10:15:52.259 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:52.230881+0000", 2024-09-18T10:15:52.259 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 0.023971681000000002, 2024-09-18T10:15:52.259 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.024083429999999999, 2024-09-18T10:15:52.259 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:52.259 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "queued for pg", 2024-09-18T10:15:52.259 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:52.260 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.260 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:52.260 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:52.230881+0000", 2024-09-18T10:15:52.260 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:52.260 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.260 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.260 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:52.260 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:52.230881+0000", 2024-09-18T10:15:52.260 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:52.260 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.260 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.260 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:52.260 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:52.230879+0000", 2024-09-18T10:15:52.261 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:52.261 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.261 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.261 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:52.261 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:52.231249+0000", 2024-09-18T10:15:52.261 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00036973900000000001 2024-09-18T10:15:52.261 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.261 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.261 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:52.261 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:52.231284+0000", 2024-09-18T10:15:52.261 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.5744999999999997e-05 2024-09-18T10:15:52.261 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.261 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.261 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:52.262 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:52.231306+0000", 2024-09-18T10:15:52.262 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1296000000000001e-05 2024-09-18T10:15:52.262 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:52.262 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:52.262 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:52.262 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:52.262 INFO:teuthology.orchestra.run.smithi029.stdout: ], 2024-09-18T10:15:52.262 INFO:teuthology.orchestra.run.smithi029.stdout: "num_ops": 2 2024-09-18T10:15:52.262 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-18T10:15:52.266 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-09-18T10:15:52.287 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-18T10:15:52.308 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-18T10:15:52.335 INFO:tasks.rados.rados.0.smithi189.stdout:377: finishing write tid 1 to smithi18925896-14 2024-09-18T10:15:52.335 INFO:tasks.rados.rados.0.smithi189.stdout:377: finishing write tid 2 to smithi18925896-14 2024-09-18T10:15:52.336 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-18T10:15:52.346 INFO:tasks.ceph.osd.10.smithi154.stderr:2024-09-18T10:15:52.342+0000 7fb73221a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 22091) UID: 0 2024-09-18T10:15:52.359 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-18T10:15:52.359 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-18T10:15:52.360 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [], 2024-09-18T10:15:52.360 INFO:teuthology.orchestra.run.smithi029.stdout: "complaint_time": 30, 2024-09-18T10:15:52.360 INFO:teuthology.orchestra.run.smithi029.stdout: "num_blocked_ops": 0 2024-09-18T10:15:52.360 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-18T10:15:52.369 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-09-18T10:15:52.380 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-18T10:15:52.418 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-18T10:15:52.439 INFO:tasks.rados.rados.0.smithi189.stdout:377: finishing write tid 3 to smithi18925896-14 2024-09-18T10:15:52.439 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 14 v 53 (ObjNum 144 snap 31 seq_num 144) dirty exists 2024-09-18T10:15:52.439 INFO:tasks.rados.rados.0.smithi189.stdout:377: left oid 14 (ObjNum 144 snap 31 seq_num 144) 2024-09-18T10:15:52.444 INFO:teuthology.orchestra.run.smithi029.stdout:osd.6: {} 2024-09-18T10:15:52.446 INFO:tasks.ceph.osd.10.smithi154.stderr:2024-09-18T10:15:52.442+0000 7fb73221a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 22091) UID: 0 2024-09-18T10:15:52.447 INFO:tasks.rados.rados.0.smithi189.stdout:372: finishing copy_from to smithi18925896-21 2024-09-18T10:15:52.447 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 21 v 102 (ObjNum 36 snap 0 seq_num 36) dirty exists 2024-09-18T10:15:52.478 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-18T10:15:52.479 INFO:teuthology.orchestra.run.smithi029.stdout: "size": 20, 2024-09-18T10:15:52.479 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 600, 2024-09-18T10:15:52.479 INFO:teuthology.orchestra.run.smithi029.stdout: "ops": [ 2024-09-18T10:15:52.479 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.479 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:134 3.4s0 3:2650be24:::smithi18925896-45:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:52.479 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:50.915670+0000", 2024-09-18T10:15:52.479 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 61.558344064000003, 2024-09-18T10:15:52.480 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.40775830200000002, 2024-09-18T10:15:52.480 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:52.480 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:52.480 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:52.480 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:52.480 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:52.481 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 134 2024-09-18T10:15:52.481 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.481 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:52.481 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.481 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:52.481 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.915670+0000", 2024-09-18T10:15:52.481 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:52.481 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.481 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.481 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:52.481 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.915670+0000", 2024-09-18T10:15:52.481 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:52.482 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.482 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.482 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:52.482 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.915668+0000", 2024-09-18T10:15:52.482 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:52.482 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.482 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.482 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:52.482 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.915676+0000", 2024-09-18T10:15:52.482 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.1899999999999995e-06 2024-09-18T10:15:52.482 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.482 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.482 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:52.483 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.915690+0000", 2024-09-18T10:15:52.483 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3827e-05 2024-09-18T10:15:52.483 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.483 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.483 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:52.483 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.915702+0000", 2024-09-18T10:15:52.483 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1477999999999999e-05 2024-09-18T10:15:52.483 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.483 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.483 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:52.483 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.918373+0000", 2024-09-18T10:15:52.483 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0026713000000000002 2024-09-18T10:15:52.483 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.484 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.484 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:52.484 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.918547+0000", 2024-09-18T10:15:52.484 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00017375999999999999 2024-09-18T10:15:52.484 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.484 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.484 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:52.484 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.919485+0000", 2024-09-18T10:15:52.484 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00093826999999999995 2024-09-18T10:15:52.484 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.484 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.484 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:52.484 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.120632+0000", 2024-09-18T10:15:52.484 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.20114729200000001 2024-09-18T10:15:52.485 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.485 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.485 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-18T10:15:52.485 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.323281+0000", 2024-09-18T10:15:52.485 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.20264834500000001 2024-09-18T10:15:52.485 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.485 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.485 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:52.485 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.323428+0000", 2024-09-18T10:15:52.485 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000147334 2024-09-18T10:15:52.485 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:52.485 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:52.485 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:52.485 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.486 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.486 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:135 3.4s0 3:2650be24:::smithi18925896-45:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:52.486 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:50.915712+0000", 2024-09-18T10:15:52.486 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 61.558302204999997, 2024-09-18T10:15:52.486 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.81161388099999998, 2024-09-18T10:15:52.486 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:52.486 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:52.486 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:52.486 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:52.486 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:52.486 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 135 2024-09-18T10:15:52.486 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.486 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:52.486 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.487 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:52.487 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.915712+0000", 2024-09-18T10:15:52.487 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:52.487 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.487 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.487 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:52.487 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.915712+0000", 2024-09-18T10:15:52.487 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:52.487 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.487 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.487 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:52.488 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.915711+0000", 2024-09-18T10:15:52.488 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.999999 2024-09-18T10:15:52.488 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.488 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.488 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:52.488 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.915715+0000", 2024-09-18T10:15:52.488 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.3159999999999998e-06 2024-09-18T10:15:52.488 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.488 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.488 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:52.488 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.915726+0000", 2024-09-18T10:15:52.488 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1375e-05 2024-09-18T10:15:52.489 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.489 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.489 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:52.489 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.915736+0000", 2024-09-18T10:15:52.489 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.4860000000000003e-06 2024-09-18T10:15:52.489 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.489 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.489 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:52.490 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.919925+0000", 2024-09-18T10:15:52.490 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0041886689999999999 2024-09-18T10:15:52.490 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.490 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.490 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for rw locks", 2024-09-18T10:15:52.490 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:50.920019+0000", 2024-09-18T10:15:52.490 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.4196000000000004e-05 2024-09-18T10:15:52.490 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.490 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.490 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:52.490 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.424171+0000", 2024-09-18T10:15:52.490 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.50415232399999999 2024-09-18T10:15:52.490 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.490 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.490 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:52.491 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.424320+0000", 2024-09-18T10:15:52.491 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00014877299999999999 2024-09-18T10:15:52.491 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.491 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.491 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:52.491 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:51.727325+0000", 2024-09-18T10:15:52.491 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.30300555400000001 2024-09-18T10:15:52.491 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:52.491 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:52.491 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:52.491 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.491 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.491 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:188 3.4s0 3:213f864e:::smithi18925896-32:head [create,setxattr XNeZS[lOPtVca\\Pa[\\rdfbinb[bcjhtYSpmcafZm (1928) in=40b,setxattr TfflYYqPnps]lo^qSlp___TitNdd`]kaPTVOTcsh (1648) in=40b,setxattr TNb_onpY[j_dkhfmnWtm]Yktg]]RNtbQe[OUTNlX (770) in=40b,setxattr ZcgnkhY\\orTYaZh]PPldj`SNq`p\\cX`k]h\\bZY_Y (700) in=40b,setxattr gPQVnqhpsVhW_kq`QnafXrlRZtZ\\YnRftgQTishR (595) in=40b,setxattr U`XqYi[WhoqnpSPRc\\RhnppZp^qY\\qUfjpqs[r^a (1892) in=40b,setxattr cTmhd`pWgrRf_VfVgOtgeOP^ShP]lsOtpoTj[\\gl (418) in=40b,setxattr aa_taYTfslg`lRRrOkkraN\\XiW^V\\ljl]NqXin`a (314) in=40b,setxattr ibl]`igtgVThY^Qse\\okYYUtTmolX\\W]p]Q[P\\il (1403) in=40b,setxattr \\ZabSO[UUqeZa`hddNh[ZjaRRgUZoQtkn\\QdsX]W (1541) in=40b,setxattr jUkQUPTNbo`rZNmYlUmhb]ojc]aeheoVRfp]bPWf (5052) in=40b,setxattr Ofj`oQ^cR]da^SPsTaXmmPse\\OZ`VSqQ]U`efsZl (419) in=40b,setxattr a[cdkWos\\OrUVPciUaa^_^`lWVTScffiWkd\\[kgf (1261) in=40b,setxattr jXb]\\WhbatdZ[qTTiZO]kPd^`l[kajOYiQ[iqSYc (244) in=40b,setxattr \\im]jQhbanUn\\_^h`jaU]W`ReaZN_je[]kgg (1578) in=36b,setxattr ZbosoRORS^_de]rN^_daOiha\\Q\\eRae^[sQUjXsV (9120) in=40b,setxattr gcn]XpZUkOZr\\pkrVVZpYiR\\alh_PaZa_S[PYcn] (384) in=40b,setxattr \\^qUOaNXSTjtmafhfbo\\gseTTkV`mgnacYq^bkcc (34) in=40b,setxattr iUa^mOQ]TqjVSUU_nXfY]f`NjljiiaYbrm[o]aes (1611) in=40b,setxattr tXaQPQbc^aiV[^reVOPpib^caoWf^XgqilO^_g[k (1016) in=40b] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:52.491 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:52.693780+0000", 2024-09-18T10:15:52.491 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 59.780233627999998, 2024-09-18T10:15:52.492 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.40440725999999999, 2024-09-18T10:15:52.492 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:52.492 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:52.492 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:52.492 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:52.492 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:52.492 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 188 2024-09-18T10:15:52.492 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.492 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:52.492 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.492 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:52.493 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.693780+0000", 2024-09-18T10:15:52.493 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:52.493 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.493 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.493 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:52.493 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.693780+0000", 2024-09-18T10:15:52.493 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:52.493 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.493 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.493 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:52.493 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.693777+0000", 2024-09-18T10:15:52.493 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999967 2024-09-18T10:15:52.493 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.494 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.494 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:52.494 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.693873+0000", 2024-09-18T10:15:52.494 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.6496000000000006e-05 2024-09-18T10:15:52.494 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.494 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.494 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:52.494 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.693926+0000", 2024-09-18T10:15:52.494 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.2148999999999998e-05 2024-09-18T10:15:52.494 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.494 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.494 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:52.494 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.693945+0000", 2024-09-18T10:15:52.494 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9296999999999998e-05 2024-09-18T10:15:52.494 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.495 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.495 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:52.495 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.694035+0000", 2024-09-18T10:15:52.495 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.0439999999999995e-05 2024-09-18T10:15:52.495 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.495 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.495 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op must wait for map", 2024-09-18T10:15:52.495 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.694044+0000", 2024-09-18T10:15:52.495 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.1479999999999995e-06 2024-09-18T10:15:52.495 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.495 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.495 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:52.495 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.797388+0000", 2024-09-18T10:15:52.496 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10334315199999999 2024-09-18T10:15:52.496 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.496 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.496 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:52.496 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.797847+0000", 2024-09-18T10:15:52.496 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00045971299999999999 2024-09-18T10:15:52.496 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.496 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.496 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:52.496 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.800981+0000", 2024-09-18T10:15:52.496 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0031333189999999999 2024-09-18T10:15:52.496 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.496 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.496 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:52.496 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:52.803005+0000", 2024-09-18T10:15:52.497 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0020248359999999999 2024-09-18T10:15:52.497 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.497 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.497 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-18T10:15:52.497 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:53.097948+0000", 2024-09-18T10:15:52.497 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.29494210599999998 2024-09-18T10:15:52.497 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.497 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.497 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:52.497 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:53.098187+0000", 2024-09-18T10:15:52.497 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000239769 2024-09-18T10:15:52.497 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:52.497 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:52.498 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:52.498 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.498 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.498 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWrite(3.cs1 41/32 ECSubWrite(tid=47, reqid=osd.5.0:46, at_version=0'0, trim_to=0'0, roll_forward_to=41'27))", 2024-09-18T10:15:52.498 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:56.935584+0000", 2024-09-18T10:15:52.498 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 55.538429932, 2024-09-18T10:15:52.498 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.60626800999999997, 2024-09-18T10:15:52.498 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:52.498 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:52.498 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:52.498 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.498 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:52.498 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.935584+0000", 2024-09-18T10:15:52.498 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:52.499 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.499 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.499 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:52.499 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.935584+0000", 2024-09-18T10:15:52.499 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:52.499 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.499 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.499 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:52.499 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.935582+0000", 2024-09-18T10:15:52.499 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:52.499 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.499 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.500 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:52.500 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.935616+0000", 2024-09-18T10:15:52.500 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.4588999999999997e-05 2024-09-18T10:15:52.500 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.500 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.500 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:52.500 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.935661+0000", 2024-09-18T10:15:52.500 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4705e-05 2024-09-18T10:15:52.500 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.500 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.501 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:52.501 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.935681+0000", 2024-09-18T10:15:52.501 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0290000000000001e-05 2024-09-18T10:15:52.501 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.501 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.501 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:52.501 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.339911+0000", 2024-09-18T10:15:52.501 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.40422952000000001 2024-09-18T10:15:52.501 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.501 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.501 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:52.501 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.339925+0000", 2024-09-18T10:15:52.501 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4523e-05 2024-09-18T10:15:52.502 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.502 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.502 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:52.502 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.541788+0000", 2024-09-18T10:15:52.502 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.20186222000000001 2024-09-18T10:15:52.502 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.502 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.502 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:52.502 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.541852+0000", 2024-09-18T10:15:52.502 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.4171999999999995e-05 2024-09-18T10:15:52.502 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:52.502 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:52.502 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:52.503 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.503 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.503 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWrite(3.cs1 41/32 ECSubWrite(tid=48, reqid=osd.5.0:48, at_version=41'29, trim_to=0'0, roll_forward_to=41'27))", 2024-09-18T10:15:52.503 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:14:56.937458+0000", 2024-09-18T10:15:52.503 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 55.536556142999999, 2024-09-18T10:15:52.503 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.60464621900000004, 2024-09-18T10:15:52.503 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:52.503 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:52.503 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:52.503 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.503 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:52.503 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.937458+0000", 2024-09-18T10:15:52.503 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:52.504 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.504 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.504 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:52.504 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.937458+0000", 2024-09-18T10:15:52.504 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:52.504 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.504 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.504 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:52.504 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.937456+0000", 2024-09-18T10:15:52.505 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.999999 2024-09-18T10:15:52.505 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.505 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.505 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:52.505 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.937505+0000", 2024-09-18T10:15:52.505 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8809e-05 2024-09-18T10:15:52.505 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.505 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.505 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:52.505 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.937540+0000", 2024-09-18T10:15:52.505 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.4662000000000001e-05 2024-09-18T10:15:52.505 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.505 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.505 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:52.506 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:56.937554+0000", 2024-09-18T10:15:52.506 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4137e-05 2024-09-18T10:15:52.506 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.506 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.506 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:52.506 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.340361+0000", 2024-09-18T10:15:52.506 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.40280652500000003 2024-09-18T10:15:52.506 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.506 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.506 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:52.506 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.340382+0000", 2024-09-18T10:15:52.507 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.16e-05 2024-09-18T10:15:52.507 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.507 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.507 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:52.507 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.542055+0000", 2024-09-18T10:15:52.507 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.20167278999999999 2024-09-18T10:15:52.507 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.507 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.507 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:52.507 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:14:57.542104+0000", 2024-09-18T10:15:52.507 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8866000000000003e-05 2024-09-18T10:15:52.507 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:52.508 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:52.508 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:52.508 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.508 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.508 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:413 3.4s0 3:2650be24:::smithi18925896-45:head [read 0~753664,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-18T10:15:52.508 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:07.553960+0000", 2024-09-18T10:15:52.508 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 44.920053961000001, 2024-09-18T10:15:52.508 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.30658152999999999, 2024-09-18T10:15:52.508 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:52.508 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:52.508 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:52.509 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:52.509 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:52.509 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 413 2024-09-18T10:15:52.509 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.509 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:52.509 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.509 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:52.509 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.553960+0000", 2024-09-18T10:15:52.509 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:52.509 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.509 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.509 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:52.510 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.553960+0000", 2024-09-18T10:15:52.510 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:52.510 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.510 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.510 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:52.510 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.553956+0000", 2024-09-18T10:15:52.510 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999967 2024-09-18T10:15:52.511 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.511 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.511 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:52.511 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.553973+0000", 2024-09-18T10:15:52.511 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.6602000000000001e-05 2024-09-18T10:15:52.511 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.511 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.512 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:52.512 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.554010+0000", 2024-09-18T10:15:52.512 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7129999999999999e-05 2024-09-18T10:15:52.512 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.512 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.512 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:52.513 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.554032+0000", 2024-09-18T10:15:52.513 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.2316000000000001e-05 2024-09-18T10:15:52.513 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.513 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.513 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:52.513 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.554187+0000", 2024-09-18T10:15:52.513 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00015407499999999999 2024-09-18T10:15:52.514 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.514 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.514 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "op must wait for map", 2024-09-18T10:15:52.514 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.554232+0000", 2024-09-18T10:15:52.514 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.5173999999999998e-05 2024-09-18T10:15:52.514 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.514 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.514 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:52.514 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.655961+0000", 2024-09-18T10:15:52.514 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10172934 2024-09-18T10:15:52.514 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.514 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.514 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:52.515 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.656152+0000", 2024-09-18T10:15:52.515 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00019075599999999999 2024-09-18T10:15:52.515 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.515 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.515 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:52.515 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.860541+0000", 2024-09-18T10:15:52.515 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.20438944000000001 2024-09-18T10:15:52.515 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:52.515 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:52.515 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:52.515 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.515 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.515 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:414 3.4s0 3:2650be24:::smithi18925896-45:head [read 0~753664,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-18T10:15:52.516 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:07.554078+0000", 2024-09-18T10:15:52.516 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 44.919935657000003, 2024-09-18T10:15:52.516 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.30769232899999999, 2024-09-18T10:15:52.516 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:52.516 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:52.516 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:52.516 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:52.516 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:52.516 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 414 2024-09-18T10:15:52.516 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.516 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:52.516 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.517 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:52.517 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.554078+0000", 2024-09-18T10:15:52.517 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:52.517 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.517 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.517 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:52.517 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.554078+0000", 2024-09-18T10:15:52.517 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:52.517 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.517 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.517 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:52.517 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.554077+0000", 2024-09-18T10:15:52.517 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:52.518 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.518 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.518 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:52.518 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.554086+0000", 2024-09-18T10:15:52.518 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.6569999999999992e-06 2024-09-18T10:15:52.518 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.518 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.518 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:52.518 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.554106+0000", 2024-09-18T10:15:52.518 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.9969000000000001e-05 2024-09-18T10:15:52.518 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.518 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.518 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:52.518 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.554118+0000", 2024-09-18T10:15:52.519 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.217e-05 2024-09-18T10:15:52.519 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.519 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.519 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:52.519 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.554389+0000", 2024-09-18T10:15:52.519 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00027051000000000001 2024-09-18T10:15:52.519 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.519 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.519 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:52.519 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.554407+0000", 2024-09-18T10:15:52.519 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.7677e-05 2024-09-18T10:15:52.519 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.520 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.520 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:52.520 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.656732+0000", 2024-09-18T10:15:52.520 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.102325597 2024-09-18T10:15:52.520 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.520 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.520 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:52.520 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.656894+0000", 2024-09-18T10:15:52.520 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000161383 2024-09-18T10:15:52.520 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.520 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.520 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:52.520 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.861770+0000", 2024-09-18T10:15:52.521 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.20487676599999999 2024-09-18T10:15:52.521 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:52.521 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:52.521 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:52.521 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.521 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.521 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:415 3.4s0 3:2650be24:::smithi18925896-45:head [read 0~753664,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-18T10:15:52.521 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:07.554169+0000", 2024-09-18T10:15:52.521 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 44.919845195999997, 2024-09-18T10:15:52.521 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.308598176, 2024-09-18T10:15:52.522 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:52.522 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:52.522 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:52.522 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:52.522 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:52.522 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 415 2024-09-18T10:15:52.522 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.522 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:52.522 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.522 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:52.522 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.554169+0000", 2024-09-18T10:15:52.522 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:52.522 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.523 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.523 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:52.523 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.554169+0000", 2024-09-18T10:15:52.523 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:52.523 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.523 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.523 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:52.523 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.554167+0000", 2024-09-18T10:15:52.523 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:52.523 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.523 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.523 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:52.524 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.554176+0000", 2024-09-18T10:15:52.524 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.6430000000000001e-06 2024-09-18T10:15:52.524 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.524 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.524 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:52.524 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.554197+0000", 2024-09-18T10:15:52.524 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.1209e-05 2024-09-18T10:15:52.524 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.524 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.524 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:52.524 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.554206+0000", 2024-09-18T10:15:52.524 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 9.3570000000000004e-06 2024-09-18T10:15:52.524 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.525 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.525 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:52.525 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.554515+0000", 2024-09-18T10:15:52.525 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00030821000000000001 2024-09-18T10:15:52.525 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.525 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.525 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:52.525 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.554535+0000", 2024-09-18T10:15:52.525 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0092999999999999e-05 2024-09-18T10:15:52.525 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.525 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.525 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:52.525 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.756075+0000", 2024-09-18T10:15:52.526 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.201540831 2024-09-18T10:15:52.526 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.526 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.526 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:52.526 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.756260+0000", 2024-09-18T10:15:52.526 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00018466000000000001 2024-09-18T10:15:52.526 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.526 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.526 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:52.526 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:07.862767+0000", 2024-09-18T10:15:52.526 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.106506579 2024-09-18T10:15:52.526 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:52.526 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:52.527 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:52.527 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.527 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.527 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(osd.4.21:3 3.4s0 3:2619bf65:::smithi18925896-16:head [assert-version v2,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-18T10:15:52.527 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:11.936400+0000", 2024-09-18T10:15:52.527 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 40.537614040999998, 2024-09-18T10:15:52.527 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.312109203, 2024-09-18T10:15:52.527 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:52.527 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:52.527 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:52.527 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.527 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:52.527 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:11.936400+0000", 2024-09-18T10:15:52.527 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:52.528 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.528 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.528 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:52.528 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:11.936400+0000", 2024-09-18T10:15:52.528 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:52.528 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.528 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.528 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:52.528 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:11.936396+0000", 2024-09-18T10:15:52.528 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999962 2024-09-18T10:15:52.528 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.528 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.528 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:52.528 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:11.936434+0000", 2024-09-18T10:15:52.529 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.7580000000000003e-05 2024-09-18T10:15:52.529 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.529 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.529 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:52.529 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:11.936473+0000", 2024-09-18T10:15:52.529 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.9876000000000002e-05 2024-09-18T10:15:52.529 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.529 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.529 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:52.529 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:11.936499+0000", 2024-09-18T10:15:52.529 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.6012000000000001e-05 2024-09-18T10:15:52.529 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.529 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.529 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:52.530 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.039131+0000", 2024-09-18T10:15:52.530 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10263111799999999 2024-09-18T10:15:52.530 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.530 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.530 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:52.530 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.039402+0000", 2024-09-18T10:15:52.530 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00027099399999999999 2024-09-18T10:15:52.530 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.530 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.530 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:52.530 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.248509+0000", 2024-09-18T10:15:52.530 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.20910732700000001 2024-09-18T10:15:52.530 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:52.530 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:52.530 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:52.531 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.531 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.531 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:446 3.4s0 3:2650be24:::smithi18925896-45:head [append 0~622592 in=622592b] snapc 13=[13,12,11,10,f,e,b] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-18T10:15:52.531 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:12.892106+0000", 2024-09-18T10:15:52.531 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 39.581907412, 2024-09-18T10:15:52.531 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.77541641100000003, 2024-09-18T10:15:52.531 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:52.531 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:52.531 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:52.532 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:52.532 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:52.532 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 446 2024-09-18T10:15:52.532 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.532 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:52.532 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.532 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:52.532 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.892106+0000", 2024-09-18T10:15:52.532 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:52.532 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.532 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.532 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:52.533 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.892106+0000", 2024-09-18T10:15:52.533 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:52.533 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.533 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.533 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:52.533 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.892103+0000", 2024-09-18T10:15:52.533 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999971 2024-09-18T10:15:52.533 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.533 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.533 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:52.533 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.893050+0000", 2024-09-18T10:15:52.534 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00094607000000000003 2024-09-18T10:15:52.534 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.534 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.534 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:52.534 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.893084+0000", 2024-09-18T10:15:52.534 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.4759999999999999e-05 2024-09-18T10:15:52.534 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.534 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.534 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:52.534 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.893098+0000", 2024-09-18T10:15:52.534 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3859000000000001e-05 2024-09-18T10:15:52.534 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.535 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.535 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:52.535 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.145987+0000", 2024-09-18T10:15:52.535 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.252889115 2024-09-18T10:15:52.535 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.535 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.535 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:52.535 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.146112+0000", 2024-09-18T10:15:52.535 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00012469299999999999 2024-09-18T10:15:52.535 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.535 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.535 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:52.535 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.146923+0000", 2024-09-18T10:15:52.535 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00081091500000000005 2024-09-18T10:15:52.535 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.536 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.536 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:52.536 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.152437+0000", 2024-09-18T10:15:52.536 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0055141169999999998 2024-09-18T10:15:52.536 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.536 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.536 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-18T10:15:52.536 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.667347+0000", 2024-09-18T10:15:52.536 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.51491041199999998 2024-09-18T10:15:52.536 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.536 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.537 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:52.537 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.667523+0000", 2024-09-18T10:15:52.537 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00017533199999999999 2024-09-18T10:15:52.537 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:52.537 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:52.537 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:52.537 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.537 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.537 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:447 3.4s0 3:2650be24:::smithi18925896-45:head [append 0~475136 in=475136b] snapc 13=[13,12,11,10,f,e,b] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-18T10:15:52.537 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:12.896134+0000", 2024-09-18T10:15:52.537 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 39.577880049000001, 2024-09-18T10:15:52.537 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.87223328099999997, 2024-09-18T10:15:52.538 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:52.538 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:52.538 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:52.538 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:52.538 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:52.538 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 447 2024-09-18T10:15:52.538 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.538 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:52.538 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.538 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:52.538 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.896134+0000", 2024-09-18T10:15:52.538 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:52.538 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.539 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.539 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:52.539 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.896134+0000", 2024-09-18T10:15:52.539 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:52.539 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.539 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.539 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:52.539 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.896131+0000", 2024-09-18T10:15:52.539 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999976 2024-09-18T10:15:52.539 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.539 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.539 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:52.539 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.896661+0000", 2024-09-18T10:15:52.539 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00052970299999999997 2024-09-18T10:15:52.540 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.540 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.540 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:52.540 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.896693+0000", 2024-09-18T10:15:52.540 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1903000000000003e-05 2024-09-18T10:15:52.540 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.540 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.540 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:52.540 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.896705+0000", 2024-09-18T10:15:52.540 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.2288e-05 2024-09-18T10:15:52.540 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.540 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.540 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:52.541 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.147534+0000", 2024-09-18T10:15:52.541 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.25082866300000001 2024-09-18T10:15:52.541 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.541 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.541 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:52.541 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.147612+0000", 2024-09-18T10:15:52.541 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.7789999999999999e-05 2024-09-18T10:15:52.541 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.541 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.541 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:52.541 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.148220+0000", 2024-09-18T10:15:52.541 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00060859599999999999 2024-09-18T10:15:52.542 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.542 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.542 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:52.542 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.152441+0000", 2024-09-18T10:15:52.542 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0042208250000000001 2024-09-18T10:15:52.542 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.542 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.542 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-18T10:15:52.542 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.768118+0000", 2024-09-18T10:15:52.543 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.615676471 2024-09-18T10:15:52.543 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.543 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.543 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:52.543 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.768367+0000", 2024-09-18T10:15:52.543 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00024930800000000001 2024-09-18T10:15:52.543 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:52.543 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:52.543 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:52.543 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.543 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.543 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:448 3.4s0 3:2650be24:::smithi18925896-45:head [append 0~294912 in=294912b] snapc 13=[13,12,11,10,f,e,b] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-18T10:15:52.543 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:12.897981+0000", 2024-09-18T10:15:52.543 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 39.576032871000002, 2024-09-18T10:15:52.544 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.87089765500000005, 2024-09-18T10:15:52.544 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:52.544 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:52.544 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:52.544 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:52.544 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:52.544 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 448 2024-09-18T10:15:52.544 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.544 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:52.544 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.544 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:52.544 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.897981+0000", 2024-09-18T10:15:52.544 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:52.545 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.545 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.545 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:52.545 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.897981+0000", 2024-09-18T10:15:52.545 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:52.545 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.545 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.545 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:52.545 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.897979+0000", 2024-09-18T10:15:52.545 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:52.545 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.545 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.545 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:52.546 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.898390+0000", 2024-09-18T10:15:52.546 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00041124999999999999 2024-09-18T10:15:52.546 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.546 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.546 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:52.546 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.898416+0000", 2024-09-18T10:15:52.546 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.525e-05 2024-09-18T10:15:52.546 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.546 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.546 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:52.546 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.898434+0000", 2024-09-18T10:15:52.546 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8615999999999999e-05 2024-09-18T10:15:52.546 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.546 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.547 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:52.547 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.148720+0000", 2024-09-18T10:15:52.547 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.25028612900000002 2024-09-18T10:15:52.547 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.547 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.547 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:52.547 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.148799+0000", 2024-09-18T10:15:52.547 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.8481999999999998e-05 2024-09-18T10:15:52.547 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.547 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.547 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:52.548 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.149383+0000", 2024-09-18T10:15:52.548 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000584255 2024-09-18T10:15:52.548 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.548 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.548 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:52.548 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.152442+0000", 2024-09-18T10:15:52.548 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0030591049999999999 2024-09-18T10:15:52.548 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.548 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.548 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-18T10:15:52.548 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.768702+0000", 2024-09-18T10:15:52.548 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.61626031599999997 2024-09-18T10:15:52.549 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.549 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.549 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:52.549 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.768878+0000", 2024-09-18T10:15:52.549 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00017600600000000001 2024-09-18T10:15:52.549 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:52.549 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:52.549 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:52.549 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.549 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.549 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:449 3.4s0 3:2650be24:::smithi18925896-45:head [setxattr _header (57) in=64b] snapc 13=[13,12,11,10,f,e,b] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-18T10:15:52.549 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:12.898446+0000", 2024-09-18T10:15:52.549 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 39.575567352999997, 2024-09-18T10:15:52.549 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.87093100999999995, 2024-09-18T10:15:52.550 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:52.550 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:52.550 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:52.550 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:52.550 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:52.550 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 449 2024-09-18T10:15:52.550 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.550 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:52.550 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.550 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:52.550 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.898446+0000", 2024-09-18T10:15:52.551 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:52.551 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.551 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.551 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:52.551 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.898446+0000", 2024-09-18T10:15:52.551 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:52.551 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.551 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.551 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:52.551 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.898445+0000", 2024-09-18T10:15:52.551 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.999999 2024-09-18T10:15:52.551 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.551 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.552 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:52.552 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.898451+0000", 2024-09-18T10:15:52.552 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.054e-06 2024-09-18T10:15:52.552 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.552 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.552 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:52.552 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.898462+0000", 2024-09-18T10:15:52.552 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.081e-05 2024-09-18T10:15:52.552 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.552 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.553 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:52.553 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.898467+0000", 2024-09-18T10:15:52.553 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4220000000000002e-06 2024-09-18T10:15:52.553 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.553 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.553 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:52.553 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.149769+0000", 2024-09-18T10:15:52.553 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.25130268 2024-09-18T10:15:52.553 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.553 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.553 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:52.553 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.149964+0000", 2024-09-18T10:15:52.554 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00019506699999999999 2024-09-18T10:15:52.554 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.554 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.554 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:52.554 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.151037+0000", 2024-09-18T10:15:52.554 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0010723200000000001 2024-09-18T10:15:52.554 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.554 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.554 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:52.554 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.152623+0000", 2024-09-18T10:15:52.554 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.0015865829999999999 2024-09-18T10:15:52.554 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.554 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.554 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "commit_sent", 2024-09-18T10:15:52.555 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.769215+0000", 2024-09-18T10:15:52.555 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.61659198800000004 2024-09-18T10:15:52.555 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.555 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.555 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:52.555 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.769377+0000", 2024-09-18T10:15:52.555 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000162134 2024-09-18T10:15:52.555 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:52.555 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:52.555 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:52.555 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.555 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.556 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:450 3.4s0 3:2650be24:::smithi18925896-45:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e57)", 2024-09-18T10:15:52.556 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:12.898472+0000", 2024-09-18T10:15:52.556 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 39.575541768000001, 2024-09-18T10:15:52.556 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.172788277, 2024-09-18T10:15:52.556 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:52.556 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:52.556 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:52.557 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:52.557 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:52.557 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 450 2024-09-18T10:15:52.557 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.557 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:52.557 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.558 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:52.558 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.898472+0000", 2024-09-18T10:15:52.558 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:52.558 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.558 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.558 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:52.558 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.898472+0000", 2024-09-18T10:15:52.558 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:52.558 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.558 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.558 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:52.558 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.898471+0000", 2024-09-18T10:15:52.558 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:52.559 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.559 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.559 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:52.559 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.898475+0000", 2024-09-18T10:15:52.559 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1480000000000002e-06 2024-09-18T10:15:52.559 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.559 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.559 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:52.559 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.898485+0000", 2024-09-18T10:15:52.560 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0342e-05 2024-09-18T10:15:52.560 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.560 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.560 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:52.560 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:12.898490+0000", 2024-09-18T10:15:52.560 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.7060000000000003e-06 2024-09-18T10:15:52.561 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.561 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.561 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:52.561 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.151519+0000", 2024-09-18T10:15:52.561 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.25302985300000003 2024-09-18T10:15:52.561 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.562 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.562 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting for rw locks", 2024-09-18T10:15:52.562 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.151633+0000", 2024-09-18T10:15:52.562 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000113121 2024-09-18T10:15:52.562 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.562 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.562 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:52.563 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.770549+0000", 2024-09-18T10:15:52.563 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.61891632200000002 2024-09-18T10:15:52.563 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.563 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.563 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:52.564 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:13.770780+0000", 2024-09-18T10:15:52.564 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00023144999999999999 2024-09-18T10:15:52.564 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.564 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.564 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:52.564 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:14.071260+0000", 2024-09-18T10:15:52.565 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.30047984999999999 2024-09-18T10:15:52.565 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:52.565 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:52.565 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:52.565 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.566 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.566 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:622 3.4s0 3:213f864e:::smithi18925896-32:head [read 0~606208,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-18T10:15:52.566 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:28.103279+0000", 2024-09-18T10:15:52.566 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 24.370734484, 2024-09-18T10:15:52.566 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.33052193600000002, 2024-09-18T10:15:52.566 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:52.567 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:52.567 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:52.567 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:52.567 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:52.567 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 622 2024-09-18T10:15:52.567 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.568 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:52.568 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.568 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:52.568 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:28.103279+0000", 2024-09-18T10:15:52.568 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:52.569 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.569 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.569 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:52.569 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:28.103279+0000", 2024-09-18T10:15:52.569 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:52.570 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.570 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.570 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:52.570 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:28.103279+0000", 2024-09-18T10:15:52.570 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:52.570 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.571 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.571 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:52.571 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:28.103284+0000", 2024-09-18T10:15:52.571 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.0250000000000002e-06 2024-09-18T10:15:52.571 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.571 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.572 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:52.572 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:28.103300+0000", 2024-09-18T10:15:52.572 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.6850999999999998e-05 2024-09-18T10:15:52.572 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.572 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.572 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:52.573 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:28.103308+0000", 2024-09-18T10:15:52.573 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 7.1450000000000002e-06 2024-09-18T10:15:52.573 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.573 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.573 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:52.574 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:28.103428+0000", 2024-09-18T10:15:52.574 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00012029200000000001 2024-09-18T10:15:52.575 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.576 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.576 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:52.576 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:28.103435+0000", 2024-09-18T10:15:52.576 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 6.6930000000000001e-06 2024-09-18T10:15:52.576 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.576 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.577 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:52.577 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:28.187445+0000", 2024-09-18T10:15:52.577 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.084010325999999996 2024-09-18T10:15:52.577 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.577 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.577 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:52.578 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:28.187954+0000", 2024-09-18T10:15:52.578 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00050906599999999997 2024-09-18T10:15:52.578 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.578 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.578 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:52.578 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:28.433801+0000", 2024-09-18T10:15:52.579 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.24584714399999999 2024-09-18T10:15:52.579 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:52.579 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:52.579 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:52.579 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.579 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.579 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "osd_op(client.4632.0:623 3.4s0 3:213f864e:::smithi18925896-32:head [read 0~606208,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-18T10:15:52.579 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:28.103315+0000", 2024-09-18T10:15:52.580 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 24.370699068, 2024-09-18T10:15:52.580 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.33131258499999999, 2024-09-18T10:15:52.580 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:52.580 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "started", 2024-09-18T10:15:52.580 INFO:teuthology.orchestra.run.smithi029.stdout: "client_info": { 2024-09-18T10:15:52.580 INFO:teuthology.orchestra.run.smithi029.stdout: "client": "client.4632", 2024-09-18T10:15:52.580 INFO:teuthology.orchestra.run.smithi029.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:52.580 INFO:teuthology.orchestra.run.smithi029.stdout: "tid": 623 2024-09-18T10:15:52.580 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.580 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:52.580 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.581 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:52.581 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:28.103315+0000", 2024-09-18T10:15:52.581 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:52.581 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.581 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.581 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:52.581 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:28.103315+0000", 2024-09-18T10:15:52.581 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:52.581 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.581 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.581 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:52.581 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:28.103314+0000", 2024-09-18T10:15:52.582 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:52.582 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.582 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.582 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:52.582 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:28.103319+0000", 2024-09-18T10:15:52.582 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.6829999999999999e-06 2024-09-18T10:15:52.582 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.582 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.582 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:52.582 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:28.103329+0000", 2024-09-18T10:15:52.582 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.0511e-05 2024-09-18T10:15:52.582 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.582 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.582 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:52.583 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:28.203443+0000", 2024-09-18T10:15:52.583 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.10011328799999999 2024-09-18T10:15:52.583 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.583 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.583 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:52.583 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:28.292935+0000", 2024-09-18T10:15:52.583 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.089492877999999998 2024-09-18T10:15:52.583 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.583 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.583 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "started", 2024-09-18T10:15:52.583 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:28.293133+0000", 2024-09-18T10:15:52.583 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.00019793 2024-09-18T10:15:52.583 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.583 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.584 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:52.584 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:28.434627+0000", 2024-09-18T10:15:52.584 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.14149382099999999 2024-09-18T10:15:52.584 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:52.584 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:52.584 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:52.584 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.584 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.584 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWrite(3.9s5 74/40 ECSubWrite(tid=178, reqid=osd.7.0:178, at_version=74'78, trim_to=0'0, roll_forward_to=72'76))", 2024-09-18T10:15:52.584 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:29.615287+0000", 2024-09-18T10:15:52.584 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 22.858726477000001, 2024-09-18T10:15:52.584 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.35347188099999999, 2024-09-18T10:15:52.584 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:52.585 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:52.585 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:52.585 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.585 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:52.585 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:29.615287+0000", 2024-09-18T10:15:52.585 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:52.585 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.585 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.585 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:52.585 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:29.615287+0000", 2024-09-18T10:15:52.585 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:52.585 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.586 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.586 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:52.586 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:29.615286+0000", 2024-09-18T10:15:52.586 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:52.586 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.586 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.586 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:52.586 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:29.615331+0000", 2024-09-18T10:15:52.586 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.4873000000000002e-05 2024-09-18T10:15:52.586 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.586 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.586 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:52.586 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:29.615363+0000", 2024-09-18T10:15:52.587 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 3.1953999999999999e-05 2024-09-18T10:15:52.587 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.587 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.587 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:52.587 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:29.615376+0000", 2024-09-18T10:15:52.587 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.3071000000000001e-05 2024-09-18T10:15:52.587 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.587 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.587 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:52.587 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:29.767783+0000", 2024-09-18T10:15:52.587 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.15240696000000001 2024-09-18T10:15:52.587 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.588 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.588 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:52.588 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:29.767797+0000", 2024-09-18T10:15:52.588 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.4688000000000001e-05 2024-09-18T10:15:52.588 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.588 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.588 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:52.588 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:29.968659+0000", 2024-09-18T10:15:52.588 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.20086167699999999 2024-09-18T10:15:52.588 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.588 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.588 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:52.589 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:29.968759+0000", 2024-09-18T10:15:52.589 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000100175 2024-09-18T10:15:52.589 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:52.589 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:52.589 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:52.589 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.589 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.589 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDECSubOpWrite(3.9s5 74/40 ECSubWrite(tid=179, reqid=osd.7.0:179, at_version=74'80, trim_to=0'0, roll_forward_to=72'76))", 2024-09-18T10:15:52.589 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:29.616697+0000", 2024-09-18T10:15:52.589 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 22.857316215000001, 2024-09-18T10:15:52.589 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.35257461800000001, 2024-09-18T10:15:52.589 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:52.589 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:52.590 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:52.590 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.590 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:52.590 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:29.616697+0000", 2024-09-18T10:15:52.590 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:52.590 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.590 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.590 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:52.590 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:29.616697+0000", 2024-09-18T10:15:52.590 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:52.590 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.590 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.590 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:52.591 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:29.616689+0000", 2024-09-18T10:15:52.591 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999914 2024-09-18T10:15:52.591 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.591 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.591 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:52.591 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:29.616734+0000", 2024-09-18T10:15:52.591 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.5073e-05 2024-09-18T10:15:52.591 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.591 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.591 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:52.591 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:29.616764+0000", 2024-09-18T10:15:52.591 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.9536e-05 2024-09-18T10:15:52.591 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.592 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.592 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:52.592 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:29.616812+0000", 2024-09-18T10:15:52.592 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4.8127999999999997e-05 2024-09-18T10:15:52.592 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.592 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.592 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:52.592 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:29.867763+0000", 2024-09-18T10:15:52.592 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.25095157400000001 2024-09-18T10:15:52.592 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.592 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.592 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_started", 2024-09-18T10:15:52.592 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:29.867775+0000", 2024-09-18T10:15:52.593 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1501000000000001e-05 2024-09-18T10:15:52.593 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.593 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.593 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "sub_op_committed", 2024-09-18T10:15:52.593 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:29.969220+0000", 2024-09-18T10:15:52.593 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.101445459 2024-09-18T10:15:52.593 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.593 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.593 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:52.593 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:29.969272+0000", 2024-09-18T10:15:52.593 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.1746000000000002e-05 2024-09-18T10:15:52.593 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:52.593 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:52.594 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:52.594 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.594 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.594 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDPGPushReply(3.19s0 79/74 [PushReplyOp(3:9a1a774c:::smithi18925896-38:19)])", 2024-09-18T10:15:52.594 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:35.704115+0000", 2024-09-18T10:15:52.594 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 16.769898244, 2024-09-18T10:15:52.594 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.30460024099999999, 2024-09-18T10:15:52.594 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:52.594 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:52.594 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:52.594 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.594 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:52.594 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:35.704115+0000", 2024-09-18T10:15:52.595 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:52.595 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.595 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.595 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:52.595 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:35.704115+0000", 2024-09-18T10:15:52.595 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:52.595 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.595 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.595 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:52.595 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:35.704115+0000", 2024-09-18T10:15:52.595 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:52.595 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.595 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.596 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:52.596 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:35.704121+0000", 2024-09-18T10:15:52.596 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 5.434e-06 2024-09-18T10:15:52.596 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.596 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.596 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:52.596 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:35.704141+0000", 2024-09-18T10:15:52.596 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.0299e-05 2024-09-18T10:15:52.596 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.596 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.596 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:52.596 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:35.704149+0000", 2024-09-18T10:15:52.596 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 8.3499999999999997e-06 2024-09-18T10:15:52.597 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.597 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.597 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:52.597 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:36.008607+0000", 2024-09-18T10:15:52.597 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.30445813700000002 2024-09-18T10:15:52.597 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.597 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.597 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:52.597 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:36.008716+0000", 2024-09-18T10:15:52.597 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000108385 2024-09-18T10:15:52.597 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:52.597 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:52.597 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:52.598 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.598 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.598 INFO:teuthology.orchestra.run.smithi029.stdout: "description": "MOSDPGPushReply(3.19s0 80/74 [PushReplyOp(3:9ab7c4c0:::smithi18925896-46:head)])", 2024-09-18T10:15:52.598 INFO:teuthology.orchestra.run.smithi029.stdout: "initiated_at": "2024-09-18T10:15:35.810513+0000", 2024-09-18T10:15:52.598 INFO:teuthology.orchestra.run.smithi029.stdout: "age": 16.663500569, 2024-09-18T10:15:52.598 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.30170829700000001, 2024-09-18T10:15:52.598 INFO:teuthology.orchestra.run.smithi029.stdout: "type_data": { 2024-09-18T10:15:52.598 INFO:teuthology.orchestra.run.smithi029.stdout: "flag_point": "reached pg", 2024-09-18T10:15:52.598 INFO:teuthology.orchestra.run.smithi029.stdout: "events": [ 2024-09-18T10:15:52.598 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.598 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "initiated", 2024-09-18T10:15:52.598 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:35.810513+0000", 2024-09-18T10:15:52.598 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:52.599 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.599 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.599 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "throttled", 2024-09-18T10:15:52.599 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:35.810513+0000", 2024-09-18T10:15:52.599 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0 2024-09-18T10:15:52.599 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.599 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.599 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "header_read", 2024-09-18T10:15:52.599 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:35.810512+0000", 2024-09-18T10:15:52.599 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 4294967295.999999 2024-09-18T10:15:52.599 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.599 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.599 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "all_read", 2024-09-18T10:15:52.599 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:35.810523+0000", 2024-09-18T10:15:52.600 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.1109e-05 2024-09-18T10:15:52.600 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.600 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.600 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "dispatched", 2024-09-18T10:15:52.600 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:35.810553+0000", 2024-09-18T10:15:52.600 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 2.955e-05 2024-09-18T10:15:52.600 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.600 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.600 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "queued_for_pg", 2024-09-18T10:15:52.600 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:35.810571+0000", 2024-09-18T10:15:52.600 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 1.8076000000000001e-05 2024-09-18T10:15:52.600 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.600 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.601 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "reached_pg", 2024-09-18T10:15:52.601 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:36.112117+0000", 2024-09-18T10:15:52.601 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.30154614000000002 2024-09-18T10:15:52.601 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2024-09-18T10:15:52.601 INFO:teuthology.orchestra.run.smithi029.stdout: { 2024-09-18T10:15:52.601 INFO:teuthology.orchestra.run.smithi029.stdout: "event": "done", 2024-09-18T10:15:52.601 INFO:teuthology.orchestra.run.smithi029.stdout: "time": "2024-09-18T10:15:36.112221+0000", 2024-09-18T10:15:52.601 INFO:teuthology.orchestra.run.smithi029.stdout: "duration": 0.000104512 2024-09-18T10:15:52.601 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:52.601 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:52.601 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:52.601 INFO:teuthology.orchestra.run.smithi029.stdout: } 2024-09-18T10:15:52.602 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2024-09-18T10:15:52.602 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-18T10:15:52.604 INFO:tasks.rados.rados.0.smithi189.stdout:371: finishing write tid 1 to smithi18925896-37 2024-09-18T10:15:52.604 INFO:tasks.rados.rados.0.smithi189.stdout:371: finishing write tid 2 to smithi18925896-37 2024-09-18T10:15:52.604 INFO:tasks.rados.rados.0.smithi189.stdout:371: finishing write tid 3 to smithi18925896-37 2024-09-18T10:15:52.604 INFO:tasks.rados.rados.0.smithi189.stdout:371: finishing write tid 4 to smithi18925896-37 2024-09-18T10:15:52.604 INFO:tasks.rados.rados.0.smithi189.stdout:371: finishing write tid 5 to smithi18925896-37 2024-09-18T10:15:52.604 INFO:tasks.rados.rados.0.smithi189.stdout:374: expect (ObjNum 33 snap 0 seq_num 33) 2024-09-18T10:15:52.605 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_ops_in_flight 2024-09-18T10:15:52.605 INFO:tasks.ceph.osd.11.smithi189.stderr:2024-09-18T10:15:52.598+0000 7f718b60a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 22400) UID: 0 2024-09-18T10:15:52.606 INFO:teuthology.orchestra.run.smithi029.stdout:osd.7: {} 2024-09-18T10:15:52.606 INFO:teuthology.orchestra.run.smithi029.stdout:osd.8: {} 2024-09-18T10:15:52.606 INFO:teuthology.orchestra.run.smithi029.stdout:osd.9: {} 2024-09-18T10:15:52.612 INFO:teuthology.orchestra.run.smithi029.stdout:osd.10: {} 2024-09-18T10:15:52.635 INFO:teuthology.orchestra.run.smithi029.stdout:osd.11: {} 2024-09-18T10:15:52.638 INFO:tasks.rados.rados.0.smithi189.stdout:371: finishing write tid 6 to smithi18925896-37 2024-09-18T10:15:52.638 INFO:tasks.rados.rados.0.smithi189.stdout:375: expect (ObjNum 137 snap 30 seq_num 137) 2024-09-18T10:15:52.657 INFO:teuthology.orchestra.run.smithi029.stdout:osd.12: {} 2024-09-18T10:15:52.679 INFO:teuthology.orchestra.run.smithi029.stdout:osd.13: {} 2024-09-18T10:15:52.702 INFO:teuthology.orchestra.run.smithi029.stderr:nodeep-scrub is unset 2024-09-18T10:15:52.704 INFO:tasks.ceph.osd.10.smithi154.stderr:2024-09-18T10:15:52.702+0000 7fb73221a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 22091) UID: 0 2024-09-18T10:15:52.715 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:52.716 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:52.717 INFO:tasks.rados.rados.0.smithi189.stdout:372: done (8 left) 2024-09-18T10:15:52.717 INFO:tasks.rados.rados.0.smithi189.stdout:374: done (7 left) 2024-09-18T10:15:52.717 INFO:tasks.rados.rados.0.smithi189.stdout:375: done (6 left) 2024-09-18T10:15:52.717 INFO:tasks.rados.rados.0.smithi189.stdout:377: done (5 left) 2024-09-18T10:15:52.718 INFO:tasks.rados.rados.0.smithi189.stdout:379: done (4 left) 2024-09-18T10:15:52.718 INFO:tasks.rados.rados.0.smithi189.stdout:380: read oid 12 snap -1 2024-09-18T10:15:52.718 INFO:tasks.rados.rados.0.smithi189.stdout:380: expect (ObjNum 137 snap 30 seq_num 137) 2024-09-18T10:15:52.718 INFO:tasks.rados.rados.0.smithi189.stdout:381: delete oid 33 current snap is 32 2024-09-18T10:15:52.725 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_blocked_ops 2024-09-18T10:15:52.734 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 43 v 68 (ObjNum 119 snap 25 seq_num 119) dirty exists 2024-09-18T10:15:52.805 INFO:teuthology.orchestra.run.smithi029.stdout:osd.14: {} 2024-09-18T10:15:52.806 INFO:tasks.ceph.osd.8.smithi029.stderr:2024-09-18T10:15:52.797+0000 7f2555c27700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 22192) UID: 0 2024-09-18T10:15:52.824 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:52.824 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:52.834 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_historic_ops 2024-09-18T10:15:52.834 INFO:teuthology.orchestra.run.smithi029.stdout:osd.15: {} 2024-09-18T10:15:52.837 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T10:15:52.837 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T10:15:52.837 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T10:15:52.837 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T10:15:52.837 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T10:15:52.837 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T10:15:52.837 INFO:teuthology.orchestra.run.smithi029.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T10:15:52.838 INFO:teuthology.orchestra.run.smithi029.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T10:15:52.838 INFO:teuthology.orchestra.run.smithi029.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-18T10:15:52.838 INFO:teuthology.orchestra.run.smithi029.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-18T10:15:52.838 INFO:teuthology.orchestra.run.smithi029.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-18T10:15:52.838 INFO:teuthology.orchestra.run.smithi029.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-18T10:15:52.838 INFO:teuthology.orchestra.run.smithi029.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-18T10:15:52.839 INFO:teuthology.orchestra.run.smithi029.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-18T10:15:52.839 INFO:teuthology.orchestra.run.smithi029.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-18T10:15:52.839 INFO:teuthology.orchestra.run.smithi029.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-18T10:15:52.861 INFO:tasks.rados.rados.0.smithi189.stdout:378: finishing copy_from to smithi18925896-6 2024-09-18T10:15:52.861 INFO:tasks.rados.rados.0.smithi189.stdout:378: got expected ENOENT (src dne) 2024-09-18T10:15:52.906 INFO:tasks.ceph.osd.12.smithi029.stderr:2024-09-18T10:15:52.901+0000 7f8e4e627700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 22190) UID: 0 2024-09-18T10:15:52.928 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:52.928 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:52.938 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T10:15:52.943 INFO:tasks.rados.rados.0.smithi189.stdout:371: finishing write tid 7 to smithi18925896-37 2024-09-18T10:15:52.943 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 37 v 54 (ObjNum 143 snap 31 seq_num 143) dirty exists 2024-09-18T10:15:52.943 INFO:tasks.rados.rados.0.smithi189.stdout:371: left oid 37 (ObjNum 143 snap 31 seq_num 143) 2024-09-18T10:15:52.955 INFO:tasks.rados.rados.0.smithi189.stdout:378: finishing copy_from racing read to smithi18925896-6 2024-09-18T10:15:53.007 INFO:tasks.ceph.osd.8.smithi029.stderr:2024-09-18T10:15:53.001+0000 7f2555c27700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 22192) UID: 0 2024-09-18T10:15:53.051 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:53.051 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:53.060 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T10:15:53.108 INFO:tasks.ceph.osd.14.smithi154.stderr:2024-09-18T10:15:53.106+0000 7f75a19e7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 22093) UID: 0 2024-09-18T10:15:53.137 INFO:tasks.rados.rados.0.smithi189.stdout:380: expect (ObjNum 137 snap 30 seq_num 137) 2024-09-18T10:15:53.160 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:53.160 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:53.169 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T10:15:53.208 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-18T10:15:53.203+0000 7f14c6a1a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 22138) UID: 0 2024-09-18T10:15:53.246 INFO:tasks.rados.rados.0.smithi189.stdout:376: finishing copy_from racing read to smithi18925896-15 2024-09-18T10:15:53.273 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:53.273 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:53.282 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T10:15:53.310 INFO:tasks.ceph.osd.14.smithi154.stderr:2024-09-18T10:15:53.306+0000 7f75a19e7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 22093) UID: 0 2024-09-18T10:15:53.410 INFO:tasks.ceph.osd.15.smithi189.stderr:2024-09-18T10:15:53.406+0000 7fd0e725e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 22401) UID: 0 2024-09-18T10:15:53.426 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:53.426 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:53.435 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T10:15:53.447 INFO:tasks.rados.rados.0.smithi189.stdout:376: finishing copy_from to smithi18925896-15 2024-09-18T10:15:53.447 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 15 v 42 (ObjNum 41 snap 0 seq_num 41) dirty exists 2024-09-18T10:15:53.506 INFO:tasks.rados.rados.0.smithi189.stdout:371: done (5 left) 2024-09-18T10:15:53.506 INFO:tasks.rados.rados.0.smithi189.stdout:373: done (4 left) 2024-09-18T10:15:53.506 INFO:tasks.rados.rados.0.smithi189.stdout:376: done (3 left) 2024-09-18T10:15:53.506 INFO:tasks.rados.rados.0.smithi189.stdout:378: done (2 left) 2024-09-18T10:15:53.506 INFO:tasks.rados.rados.0.smithi189.stdout:380: done (1 left) 2024-09-18T10:15:53.506 INFO:tasks.rados.rados.0.smithi189.stdout:381: done (0 left) 2024-09-18T10:15:53.506 INFO:tasks.rados.rados.0.smithi189.stdout:382: rmattr oid 22 current snap is 32 2024-09-18T10:15:53.506 INFO:tasks.rados.rados.0.smithi189.stdout:382: done (0 left) 2024-09-18T10:15:53.506 INFO:tasks.rados.rados.0.smithi189.stdout:383: rmattr oid 42 current snap is 32 2024-09-18T10:15:53.506 INFO:tasks.rados.rados.0.smithi189.stdout:383: done (0 left) 2024-09-18T10:15:53.507 INFO:tasks.rados.rados.0.smithi189.stdout:384: rmattr oid 23 current snap is 32 2024-09-18T10:15:53.507 INFO:tasks.rados.rados.0.smithi189.stdout:385: copy_from oid 15 from oid 19 current snap is 32 2024-09-18T10:15:53.507 INFO:tasks.rados.rados.0.smithi189.stdout:386: delete oid 30 current snap is 32 2024-09-18T10:15:53.511 INFO:tasks.ceph.osd.15.smithi189.stderr:2024-09-18T10:15:53.506+0000 7fd0e725e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 22401) UID: 0 2024-09-18T10:15:53.531 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:53.531 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:53.540 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T10:15:53.611 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-18T10:15:53.607+0000 7f14c6a1a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 22138) UID: 0 2024-09-18T10:15:53.641 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:53.642 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:53.650 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-09-18T10:15:53.705 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 23 v 140 (ObjNum 132 snap 28 seq_num 132) dirty exists 2024-09-18T10:15:53.713 INFO:tasks.ceph.osd.12.smithi029.stderr:2024-09-18T10:15:53.705+0000 7f8e4e627700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 22190) UID: 0 2024-09-18T10:15:53.753 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:53.753 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:53.763 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-09-18T10:15:53.814 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-18T10:15:53.811+0000 7f578c52f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22134) UID: 0 2024-09-18T10:15:53.819 INFO:tasks.rados.rados.0.smithi189.stdout:384: done (2 left) 2024-09-18T10:15:53.819 INFO:tasks.rados.rados.0.smithi189.stdout:386: done (1 left) 2024-09-18T10:15:53.819 INFO:tasks.rados.rados.0.smithi189.stdout:387: rollback oid 14 current snap is 32 2024-09-18T10:15:53.819 INFO:tasks.rados.rados.0.smithi189.stdout:rollback oid 14 to 22 2024-09-18T10:15:53.819 INFO:tasks.rados.rados.0.smithi189.stdout:388: read oid 20 snap -1 2024-09-18T10:15:53.819 INFO:tasks.rados.rados.0.smithi189.stdout:388: expect (ObjNum 118 snap 25 seq_num 118) 2024-09-18T10:15:53.819 INFO:tasks.rados.rados.0.smithi189.stdout:389: snap_remove snap 26 2024-09-18T10:15:53.859 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:53.860 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:53.870 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-09-18T10:15:53.915 INFO:tasks.ceph.osd.8.smithi029.stderr:2024-09-18T10:15:53.909+0000 7f2555c27700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 22192) UID: 0 2024-09-18T10:15:54.015 INFO:tasks.ceph.osd.15.smithi189.stderr:2024-09-18T10:15:54.010+0000 7fd0e725e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 22401) UID: 0 2024-09-18T10:15:54.018 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:54.018 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:54.027 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_ops_in_flight 2024-09-18T10:15:54.117 INFO:tasks.ceph.osd.15.smithi189.stderr:2024-09-18T10:15:54.114+0000 7fd0e725e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 22401) UID: 0 2024-09-18T10:15:54.123 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:54.123 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:54.133 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_blocked_ops 2024-09-18T10:15:54.218 INFO:tasks.ceph.osd.11.smithi189.stderr:2024-09-18T10:15:54.214+0000 7f718b60a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 22400) UID: 0 2024-09-18T10:15:54.229 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:54.229 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:54.239 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_historic_ops 2024-09-18T10:15:54.318 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-18T10:15:54.315+0000 7f578c52f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22134) UID: 0 2024-09-18T10:15:54.336 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:54.336 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:54.345 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T10:15:54.419 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-18T10:15:54.415+0000 7f1981873700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22132) UID: 0 2024-09-18T10:15:54.452 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:54.453 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:54.462 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T10:15:54.520 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-09-18T10:15:54.518+0000 7fdfa8698700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22087) UID: 0 2024-09-18T10:15:54.562 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:54.562 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:54.571 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T10:15:54.590 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7, 9, 10, 11, 12, 13, 14, 15, 8] out_osds: [] dead_osds: [] live_osds: [0, 4, 8, 12, 1, 5, 9, 13, 2, 6, 10, 14, 3, 7, 11, 15] 2024-09-18T10:15:54.590 INFO:tasks.thrashosds.thrasher:choose_action: min_in 8 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T10:15:54.590 INFO:tasks.thrashosds.thrasher:inject_pause on osd.13 2024-09-18T10:15:54.590 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-18T10:15:54.590 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-18T10:15:54.591 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok config set filestore_inject_stall 3 2024-09-18T10:15:54.622 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-09-18T10:15:54.618+0000 7fdfa8698700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22087) UID: 0 2024-09-18T10:15:54.668 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:54.669 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:54.677 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T10:15:54.700 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-18T10:15:54.700 INFO:teuthology.orchestra.run.smithi136.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-18T10:15:54.700 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-18T10:15:54.711 INFO:tasks.rados.rados.0.smithi189.stdout:389: done (3 left) 2024-09-18T10:15:54.711 INFO:tasks.rados.rados.0.smithi189.stdout:390: delete oid 4 current snap is 32 2024-09-18T10:15:54.711 INFO:tasks.rados.rados.0.smithi189.stdout:385: finishing copy_from racing read to smithi18925896-15 2024-09-18T10:15:54.711 INFO:tasks.rados.rados.0.smithi189.stdout:387: finishing rollback tid 0 to smithi18925896-14 2024-09-18T10:15:54.711 INFO:tasks.rados.rados.0.smithi189.stdout:387: finishing rollback tid 1 to smithi18925896-14 2024-09-18T10:15:54.711 INFO:tasks.rados.rados.0.smithi189.stdout:385: finishing copy_from to smithi18925896-15 2024-09-18T10:15:54.711 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 15 v 4 (ObjNum 12 snap 0 seq_num 12) dirty exists 2024-09-18T10:15:54.712 INFO:tasks.rados.rados.0.smithi189.stdout:387: finishing rollback tid 2 to smithi18925896-14 2024-09-18T10:15:54.712 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 14 v 57 (ObjNum 13 snap 0 seq_num 13) dirty exists 2024-09-18T10:15:54.712 INFO:tasks.rados.rados.0.smithi189.stdout:388: expect (ObjNum 118 snap 25 seq_num 118) 2024-09-18T10:15:54.715 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-18T10:15:54.723 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-18T10:15:54.719+0000 7f578c52f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22134) UID: 0 2024-09-18T10:15:54.773 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:54.774 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:54.783 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T10:15:54.823 INFO:tasks.ceph.osd.10.smithi154.stderr:2024-09-18T10:15:54.822+0000 7fb73221a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 22091) UID: 0 2024-09-18T10:15:54.849 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T10:15:54.880 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:54.881 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:54.889 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T10:15:54.924 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-18T10:15:54.919+0000 7f14c6a1a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 22138) UID: 0 2024-09-18T10:15:54.978 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-18T10:15:54.999 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-18T10:15:55.020 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-18T10:15:55.025 INFO:tasks.ceph.osd.12.smithi029.stderr:2024-09-18T10:15:55.021+0000 7f8e4e627700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 22190) UID: 0 2024-09-18T10:15:55.030 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:55.030 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:55.040 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-09-18T10:15:55.053 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-18T10:15:55.071 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-18T10:15:55.124 INFO:tasks.rados.rados.0.smithi189.stdout:385: done (3 left) 2024-09-18T10:15:55.124 INFO:tasks.rados.rados.0.smithi189.stdout:387: done (2 left) 2024-09-18T10:15:55.124 INFO:tasks.rados.rados.0.smithi189.stdout:388: done (1 left) 2024-09-18T10:15:55.124 INFO:tasks.rados.rados.0.smithi189.stdout:390: done (0 left) 2024-09-18T10:15:55.125 INFO:tasks.rados.rados.0.smithi189.stdout:append oid 12 current snap is 32 2024-09-18T10:15:55.125 INFO:tasks.rados.rados.0.smithi189.stdout:391: seq_num 148 ranges {2506752=81920} 2024-09-18T10:15:55.126 INFO:tasks.ceph.osd.4.smithi029.stderr:2024-09-18T10:15:55.121+0000 7f550cf6e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22189) UID: 0 2024-09-18T10:15:55.135 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:55.135 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:55.144 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-09-18T10:15:55.147 INFO:tasks.rados.rados.0.smithi189.stdout:391: writing smithi18925896-12 from 2506752 to 2588672 tid 1 2024-09-18T10:15:55.147 INFO:tasks.rados.rados.0.smithi189.stdout:392: snap_remove snap 30 2024-09-18T10:15:55.175 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-18T10:15:55.193 INFO:teuthology.orchestra.run.smithi029.stdout:osd.6: {} 2024-09-18T10:15:55.217 INFO:teuthology.orchestra.run.smithi029.stdout:osd.7: {} 2024-09-18T10:15:55.226 INFO:tasks.ceph.osd.10.smithi154.stderr:2024-09-18T10:15:55.222+0000 7fb73221a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 22091) UID: 0 2024-09-18T10:15:55.237 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:55.237 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:55.246 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-09-18T10:15:55.280 INFO:teuthology.orchestra.run.smithi029.stdout:osd.8: {} 2024-09-18T10:15:55.302 INFO:teuthology.orchestra.run.smithi029.stdout:osd.9: {} 2024-09-18T10:15:55.324 INFO:teuthology.orchestra.run.smithi029.stdout:osd.10: {} 2024-09-18T10:15:55.327 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-18T10:15:55.323+0000 7f1981873700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22132) UID: 0 2024-09-18T10:15:55.345 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-09-18T10:15:55.345 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-09-18T10:15:55.345 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-09-18T10:15:55.345 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-09-18T10:15:55.345 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.345 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:189 3.3s0 3:ce8164ce:::smithi18925896-7:head [copy-from ver 6] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:55.346 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:52.694279+0000", 2024-09-18T10:15:55.346 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 62.647037310000002, 2024-09-18T10:15:55.346 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.51539283599999997, 2024-09-18T10:15:55.346 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:55.346 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:55.346 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:55.346 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:15:55.346 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:55.346 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 189 2024-09-18T10:15:55.346 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.346 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:55.346 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.346 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:55.347 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694279+0000", 2024-09-18T10:15:55.347 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:55.347 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.347 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.347 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:55.347 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694279+0000", 2024-09-18T10:15:55.347 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:55.347 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.347 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.347 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:55.347 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694277+0000", 2024-09-18T10:15:55.347 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999971 2024-09-18T10:15:55.347 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.348 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.348 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:55.348 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694290+0000", 2024-09-18T10:15:55.348 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3862e-05 2024-09-18T10:15:55.348 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.348 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.348 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:55.348 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694311+0000", 2024-09-18T10:15:55.348 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.0273999999999999e-05 2024-09-18T10:15:55.348 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.348 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.348 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:55.348 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694322+0000", 2024-09-18T10:15:55.349 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1722999999999999e-05 2024-09-18T10:15:55.349 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.349 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.349 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:55.349 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.794494+0000", 2024-09-18T10:15:55.349 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100171653 2024-09-18T10:15:55.349 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.349 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.349 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:55.349 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.794705+0000", 2024-09-18T10:15:55.349 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00021126000000000001 2024-09-18T10:15:55.349 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.349 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.350 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:55.350 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.096113+0000", 2024-09-18T10:15:55.350 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.30140739100000002 2024-09-18T10:15:55.350 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.350 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.350 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:55.350 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.098602+0000", 2024-09-18T10:15:55.350 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.002489593 2024-09-18T10:15:55.350 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.350 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.350 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:15:55.350 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.209538+0000", 2024-09-18T10:15:55.350 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.110935482 2024-09-18T10:15:55.351 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.351 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.351 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:55.351 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.209672+0000", 2024-09-18T10:15:55.351 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00013435399999999999 2024-09-18T10:15:55.351 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:55.351 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:55.351 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:55.351 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.351 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.351 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:194 3.3s0 3:c9ce6440:::smithi18925896-24:head [copy-from ver 6] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:55.351 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:52.694388+0000", 2024-09-18T10:15:55.351 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 62.646928447000001, 2024-09-18T10:15:55.352 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.92118689200000003, 2024-09-18T10:15:55.352 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:55.352 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:55.352 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:55.352 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:15:55.352 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:55.352 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 194 2024-09-18T10:15:55.352 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.352 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:55.352 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.352 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:55.352 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694388+0000", 2024-09-18T10:15:55.352 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:55.353 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.353 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.353 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:55.353 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694388+0000", 2024-09-18T10:15:55.353 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:55.353 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.353 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.353 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:55.353 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694388+0000", 2024-09-18T10:15:55.353 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:15:55.353 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.353 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.353 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:55.353 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694393+0000", 2024-09-18T10:15:55.354 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.2299999999999999e-06 2024-09-18T10:15:55.354 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.354 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.354 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:55.354 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694411+0000", 2024-09-18T10:15:55.354 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7802999999999999e-05 2024-09-18T10:15:55.354 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.354 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.354 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:55.354 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694417+0000", 2024-09-18T10:15:55.354 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.4640000000000003e-06 2024-09-18T10:15:55.354 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.354 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.355 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:55.355 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.795289+0000", 2024-09-18T10:15:55.355 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100872288 2024-09-18T10:15:55.355 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.355 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.355 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:55.355 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.795548+0000", 2024-09-18T10:15:55.355 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00025869199999999998 2024-09-18T10:15:55.355 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.355 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.355 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:55.355 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.412215+0000", 2024-09-18T10:15:55.355 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.61666667100000006 2024-09-18T10:15:55.355 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.355 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.356 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:55.356 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.514838+0000", 2024-09-18T10:15:55.356 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10262323399999999 2024-09-18T10:15:55.356 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.356 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.356 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:15:55.356 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.615386+0000", 2024-09-18T10:15:55.356 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100548106 2024-09-18T10:15:55.356 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.356 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.356 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:55.356 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.615575+0000", 2024-09-18T10:15:55.356 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000189128 2024-09-18T10:15:55.356 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:55.357 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:55.357 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:55.357 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.357 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.357 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:195 3.3s0 3:c9ce6440:::smithi18925896-24:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:55.357 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:52.694424+0000", 2024-09-18T10:15:55.357 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 62.646892711, 2024-09-18T10:15:55.357 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.82009685499999996, 2024-09-18T10:15:55.357 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:55.357 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "started", 2024-09-18T10:15:55.357 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:55.357 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:15:55.357 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:55.357 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 195 2024-09-18T10:15:55.358 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.358 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:55.358 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.358 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:55.358 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694424+0000", 2024-09-18T10:15:55.358 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:55.358 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.358 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.358 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:55.358 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694424+0000", 2024-09-18T10:15:55.358 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:55.358 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.358 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.359 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:55.359 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694423+0000", 2024-09-18T10:15:55.359 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:55.359 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.359 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.359 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:55.359 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694427+0000", 2024-09-18T10:15:55.359 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2650000000000001e-06 2024-09-18T10:15:55.359 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.359 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.359 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:55.359 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694434+0000", 2024-09-18T10:15:55.359 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.5410000000000003e-06 2024-09-18T10:15:55.359 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.360 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.360 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:55.360 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694441+0000", 2024-09-18T10:15:55.360 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.187e-06 2024-09-18T10:15:55.360 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.360 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.360 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:55.360 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.795870+0000", 2024-09-18T10:15:55.360 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.101428617 2024-09-18T10:15:55.360 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.360 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.361 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for blocked object", 2024-09-18T10:15:55.361 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.796040+0000", 2024-09-18T10:15:55.361 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00017008 2024-09-18T10:15:55.361 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.361 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.361 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:55.361 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.514130+0000", 2024-09-18T10:15:55.361 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.71809022199999994 2024-09-18T10:15:55.361 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.361 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.361 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:55.361 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.514348+0000", 2024-09-18T10:15:55.361 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000217488 2024-09-18T10:15:55.362 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.362 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.362 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:55.362 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.514521+0000", 2024-09-18T10:15:55.362 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00017300900000000001 2024-09-18T10:15:55.362 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:55.362 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:55.362 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:55.362 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.362 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.362 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.bs2 41/32 ECSubWrite(tid=58, reqid=osd.4.0:58, at_version=41'14, trim_to=0'0, roll_forward_to=39'12))", 2024-09-18T10:15:55.362 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:56.829698+0000", 2024-09-18T10:15:55.363 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 58.511618617000003, 2024-09-18T10:15:55.363 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.71711602600000002, 2024-09-18T10:15:55.363 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:55.363 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:15:55.363 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:55.363 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.363 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:55.363 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.829698+0000", 2024-09-18T10:15:55.363 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:55.363 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.363 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.363 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:55.363 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.829698+0000", 2024-09-18T10:15:55.364 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:55.364 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.364 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.364 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:55.364 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.829697+0000", 2024-09-18T10:15:55.364 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:55.364 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.364 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.364 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:55.364 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.829733+0000", 2024-09-18T10:15:55.364 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.6217999999999998e-05 2024-09-18T10:15:55.364 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.364 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.365 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:55.365 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.829757+0000", 2024-09-18T10:15:55.365 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.3915000000000001e-05 2024-09-18T10:15:55.365 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.365 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.365 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:55.365 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.829769+0000", 2024-09-18T10:15:55.365 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1754e-05 2024-09-18T10:15:55.365 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.365 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.365 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:55.365 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.342965+0000", 2024-09-18T10:15:55.365 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.51319593799999996 2024-09-18T10:15:55.365 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.366 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.366 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:55.366 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.343020+0000", 2024-09-18T10:15:55.366 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5367e-05 2024-09-18T10:15:55.366 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.366 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.366 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:55.366 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.546772+0000", 2024-09-18T10:15:55.366 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.203751933 2024-09-18T10:15:55.366 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.366 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.366 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:55.366 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.546814+0000", 2024-09-18T10:15:55.366 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.2196000000000001e-05 2024-09-18T10:15:55.367 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:55.367 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:55.367 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:55.367 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.367 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.367 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDPGPushReply(3.17s0 67/64 [PushReplyOp(3:ee962131:::smithi18925896-29:10)])", 2024-09-18T10:15:55.367 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:23.671965+0000", 2024-09-18T10:15:55.367 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 31.669352167, 2024-09-18T10:15:55.367 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.60673808500000004, 2024-09-18T10:15:55.367 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:55.367 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:15:55.367 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:55.367 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.367 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:55.367 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.671965+0000", 2024-09-18T10:15:55.368 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:55.368 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.368 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.368 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:55.368 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.671965+0000", 2024-09-18T10:15:55.368 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:55.368 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.368 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.368 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:55.368 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.671963+0000", 2024-09-18T10:15:55.368 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:55.368 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.368 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.368 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:55.369 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.671977+0000", 2024-09-18T10:15:55.369 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4093e-05 2024-09-18T10:15:55.369 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.369 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.369 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:55.369 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.672009+0000", 2024-09-18T10:15:55.369 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1939e-05 2024-09-18T10:15:55.369 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.369 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.369 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:55.369 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.772102+0000", 2024-09-18T10:15:55.369 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10009319 2024-09-18T10:15:55.369 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.369 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.370 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:55.370 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.278591+0000", 2024-09-18T10:15:55.370 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.50648915999999999 2024-09-18T10:15:55.370 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.370 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.370 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:55.370 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.278703+0000", 2024-09-18T10:15:55.370 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000111433 2024-09-18T10:15:55.370 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:55.370 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:55.370 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:55.370 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.370 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.371 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDPGPushReply(3.17s0 67/64 [PushReplyOp(3:ee962131:::smithi18925896-29:10)])", 2024-09-18T10:15:55.371 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:23.772436+0000", 2024-09-18T10:15:55.371 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 31.568880930999999, 2024-09-18T10:15:55.371 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.50655292500000004, 2024-09-18T10:15:55.371 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:55.371 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:15:55.371 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:55.371 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.371 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:55.371 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.772436+0000", 2024-09-18T10:15:55.371 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:55.371 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.371 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.371 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:55.371 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.772436+0000", 2024-09-18T10:15:55.372 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:55.372 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.372 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.372 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:55.372 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.772434+0000", 2024-09-18T10:15:55.372 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:55.372 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.372 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.372 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:55.372 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.772448+0000", 2024-09-18T10:15:55.372 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3682e-05 2024-09-18T10:15:55.372 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.372 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.372 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:55.372 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.772477+0000", 2024-09-18T10:15:55.373 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9255999999999999e-05 2024-09-18T10:15:55.373 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.373 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.373 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:55.373 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.772490+0000", 2024-09-18T10:15:55.373 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2962999999999999e-05 2024-09-18T10:15:55.373 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.373 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.373 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:55.373 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.278888+0000", 2024-09-18T10:15:55.373 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.50639794699999996 2024-09-18T10:15:55.373 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.373 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.373 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:55.374 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.278989+0000", 2024-09-18T10:15:55.374 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000100777 2024-09-18T10:15:55.374 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:55.374 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:55.374 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:55.374 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.374 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.374 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDPGPushReply(3.17s0 67/64 [PushReplyOp(3:ee962131:::smithi18925896-29:10)])", 2024-09-18T10:15:55.374 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:23.772911+0000", 2024-09-18T10:15:55.374 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 31.568405651999999, 2024-09-18T10:15:55.374 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.50636473400000004, 2024-09-18T10:15:55.374 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:55.374 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:15:55.375 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:55.375 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.375 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:55.375 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.772911+0000", 2024-09-18T10:15:55.375 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:55.375 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.375 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.375 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:55.375 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.772911+0000", 2024-09-18T10:15:55.375 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:55.375 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.375 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.375 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:55.375 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.772909+0000", 2024-09-18T10:15:55.376 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:55.376 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.376 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.376 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:55.376 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.772923+0000", 2024-09-18T10:15:55.376 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3957e-05 2024-09-18T10:15:55.376 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.376 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.376 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:55.376 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.772956+0000", 2024-09-18T10:15:55.376 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2925000000000001e-05 2024-09-18T10:15:55.376 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.376 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.377 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:55.377 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.772969+0000", 2024-09-18T10:15:55.377 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2516e-05 2024-09-18T10:15:55.377 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.377 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.377 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:55.377 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.279192+0000", 2024-09-18T10:15:55.377 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.50622349799999999 2024-09-18T10:15:55.377 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.377 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.377 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:55.377 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.279276+0000", 2024-09-18T10:15:55.377 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.3498000000000006e-05 2024-09-18T10:15:55.378 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:55.378 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:55.378 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:55.378 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.378 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.378 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDPGPushReply(3.17s0 67/64 [PushReplyOp(3:ead53b0f:::smithi18925896-25:13)])", 2024-09-18T10:15:55.378 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:23.875511+0000", 2024-09-18T10:15:55.378 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 31.465806026999999, 2024-09-18T10:15:55.378 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.50638906400000006, 2024-09-18T10:15:55.378 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:55.378 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:15:55.378 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:55.378 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.378 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:55.378 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.875511+0000", 2024-09-18T10:15:55.379 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:55.379 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.379 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.379 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:55.379 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.875511+0000", 2024-09-18T10:15:55.379 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:55.379 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.379 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.379 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:55.379 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.875510+0000", 2024-09-18T10:15:55.379 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:55.379 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.379 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.380 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:55.380 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.875516+0000", 2024-09-18T10:15:55.380 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.8930000000000002e-06 2024-09-18T10:15:55.380 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.380 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.380 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:55.380 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.875526+0000", 2024-09-18T10:15:55.380 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0183000000000001e-05 2024-09-18T10:15:55.380 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.380 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.380 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:55.380 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.975617+0000", 2024-09-18T10:15:55.380 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10009069299999999 2024-09-18T10:15:55.381 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.381 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.381 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:55.381 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.381839+0000", 2024-09-18T10:15:55.381 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.40622223600000001 2024-09-18T10:15:55.381 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.381 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.381 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:55.381 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.381900+0000", 2024-09-18T10:15:55.381 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.0572000000000002e-05 2024-09-18T10:15:55.381 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:55.381 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:55.381 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:55.381 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.382 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.382 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:635 3.17s0 3:ead53b0f:::smithi18925896-25:head [append 0~655360] snapc 21=[20,1f,1e,1a,16,11,10] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-18T10:15:55.382 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:29.755594+0000", 2024-09-18T10:15:55.382 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 25.585723107, 2024-09-18T10:15:55.382 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.62129057300000001, 2024-09-18T10:15:55.382 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:55.382 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:55.382 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:55.382 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:15:55.382 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:55.382 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 635 2024-09-18T10:15:55.382 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.382 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:55.383 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.383 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:55.383 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.755594+0000", 2024-09-18T10:15:55.383 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:55.383 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.383 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.383 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:55.383 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.755594+0000", 2024-09-18T10:15:55.383 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:55.383 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.383 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.383 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:55.383 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.755590+0000", 2024-09-18T10:15:55.384 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999967 2024-09-18T10:15:55.384 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.384 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.384 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:55.384 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.756465+0000", 2024-09-18T10:15:55.384 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00087483000000000005 2024-09-18T10:15:55.384 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.384 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.384 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:55.384 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.756502+0000", 2024-09-18T10:15:55.384 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.7067999999999998e-05 2024-09-18T10:15:55.384 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.384 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.385 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:55.385 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.756516+0000", 2024-09-18T10:15:55.385 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4071e-05 2024-09-18T10:15:55.385 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.385 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.385 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:55.385 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.760022+0000", 2024-09-18T10:15:55.385 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0035059570000000001 2024-09-18T10:15:55.385 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.385 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.385 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:55.385 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.760234+0000", 2024-09-18T10:15:55.385 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00021161199999999999 2024-09-18T10:15:55.386 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.386 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.386 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:55.386 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.762520+0000", 2024-09-18T10:15:55.386 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0022855760000000001 2024-09-18T10:15:55.386 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.386 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.386 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:55.386 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.770533+0000", 2024-09-18T10:15:55.386 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0080138269999999994 2024-09-18T10:15:55.387 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.387 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.387 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:15:55.387 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:30.376766+0000", 2024-09-18T10:15:55.387 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.60623257500000005 2024-09-18T10:15:55.387 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.387 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.387 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:55.387 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:30.376884+0000", 2024-09-18T10:15:55.387 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00011819699999999999 2024-09-18T10:15:55.387 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:55.387 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:55.387 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:55.387 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.388 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.388 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:636 3.17s0 3:ead53b0f:::smithi18925896-25:head [append 0~606208] snapc 21=[20,1f,1e,1a,16,11,10] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-18T10:15:55.388 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:29.758851+0000", 2024-09-18T10:15:55.388 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 25.582465454000001, 2024-09-18T10:15:55.388 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.61852410899999999, 2024-09-18T10:15:55.388 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:55.388 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:55.388 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:55.388 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:15:55.388 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:55.388 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 636 2024-09-18T10:15:55.388 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.388 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:55.389 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.389 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:55.389 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.758851+0000", 2024-09-18T10:15:55.389 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:55.389 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.389 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.389 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:55.389 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.758851+0000", 2024-09-18T10:15:55.389 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:55.389 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.389 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.389 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:55.389 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.758849+0000", 2024-09-18T10:15:55.390 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999976 2024-09-18T10:15:55.390 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.390 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.390 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:55.390 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.759643+0000", 2024-09-18T10:15:55.390 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00079385800000000004 2024-09-18T10:15:55.390 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.390 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.390 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:55.390 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.759677+0000", 2024-09-18T10:15:55.390 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.4504e-05 2024-09-18T10:15:55.390 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.390 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.390 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:55.391 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.759691+0000", 2024-09-18T10:15:55.391 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3586e-05 2024-09-18T10:15:55.391 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.391 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.391 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:55.391 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.765309+0000", 2024-09-18T10:15:55.391 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0056179460000000004 2024-09-18T10:15:55.391 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.391 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.391 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:55.391 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.765656+0000", 2024-09-18T10:15:55.391 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000347783 2024-09-18T10:15:55.391 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.391 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.391 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:55.392 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.766735+0000", 2024-09-18T10:15:55.392 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0010785460000000001 2024-09-18T10:15:55.392 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.392 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.392 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:55.392 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.973532+0000", 2024-09-18T10:15:55.392 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20679750599999999 2024-09-18T10:15:55.392 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.392 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.392 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:15:55.392 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:30.377254+0000", 2024-09-18T10:15:55.392 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.40372181400000001 2024-09-18T10:15:55.392 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.392 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.393 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:55.393 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:30.377375+0000", 2024-09-18T10:15:55.393 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000121037 2024-09-18T10:15:55.393 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:55.393 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:55.393 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:55.393 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.393 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.393 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:637 3.17s0 3:ead53b0f:::smithi18925896-25:head [append 0~475136] snapc 21=[20,1f,1e,1a,16,11,10] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-18T10:15:55.393 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:29.761784+0000", 2024-09-18T10:15:55.393 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 25.579532601, 2024-09-18T10:15:55.393 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.61605535199999995, 2024-09-18T10:15:55.393 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:55.393 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:55.394 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:55.394 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:15:55.394 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:55.394 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 637 2024-09-18T10:15:55.394 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.394 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:55.394 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.394 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:55.394 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.761784+0000", 2024-09-18T10:15:55.394 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:55.394 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.394 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.394 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:55.395 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.761784+0000", 2024-09-18T10:15:55.395 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:55.395 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.395 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.395 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:55.395 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.761776+0000", 2024-09-18T10:15:55.395 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999919 2024-09-18T10:15:55.395 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.395 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.395 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:55.395 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.762424+0000", 2024-09-18T10:15:55.395 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00064803899999999999 2024-09-18T10:15:55.395 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.395 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.396 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:55.396 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.762455+0000", 2024-09-18T10:15:55.396 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1316000000000003e-05 2024-09-18T10:15:55.396 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.396 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.396 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:55.396 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.762471+0000", 2024-09-18T10:15:55.396 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.6174000000000001e-05 2024-09-18T10:15:55.396 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.396 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.396 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:55.396 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.767732+0000", 2024-09-18T10:15:55.396 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0052603190000000003 2024-09-18T10:15:55.397 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.397 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.397 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:55.397 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.767873+0000", 2024-09-18T10:15:55.397 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000140949 2024-09-18T10:15:55.397 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.397 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.397 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:55.397 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.768566+0000", 2024-09-18T10:15:55.397 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00069325799999999998 2024-09-18T10:15:55.397 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.397 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.397 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:55.397 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.973546+0000", 2024-09-18T10:15:55.398 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.204980144 2024-09-18T10:15:55.398 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.398 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.398 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:15:55.398 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:30.377732+0000", 2024-09-18T10:15:55.398 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.404185558 2024-09-18T10:15:55.398 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.398 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.398 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:55.398 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:30.377839+0000", 2024-09-18T10:15:55.398 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000107909 2024-09-18T10:15:55.398 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:55.398 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:55.398 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:55.399 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.399 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.399 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:638 3.17s0 3:ead53b0f:::smithi18925896-25:head [append 0~81920] snapc 21=[20,1f,1e,1a,16,11,10] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-18T10:15:55.399 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:29.762493+0000", 2024-09-18T10:15:55.399 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 25.578823581000002, 2024-09-18T10:15:55.399 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.61563624900000002, 2024-09-18T10:15:55.399 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:55.399 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:55.399 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:55.399 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:15:55.399 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:55.399 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 638 2024-09-18T10:15:55.399 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.399 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:55.400 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.400 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:55.400 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.762493+0000", 2024-09-18T10:15:55.400 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:55.400 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.400 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.400 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:55.400 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.762493+0000", 2024-09-18T10:15:55.400 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:55.400 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.400 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.400 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:55.400 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.762492+0000", 2024-09-18T10:15:55.400 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:55.401 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.401 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.401 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:55.401 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.762566+0000", 2024-09-18T10:15:55.401 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.3977e-05 2024-09-18T10:15:55.401 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.401 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.401 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:55.401 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.762585+0000", 2024-09-18T10:15:55.401 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.9074000000000001e-05 2024-09-18T10:15:55.401 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.401 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.401 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:55.401 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.762595+0000", 2024-09-18T10:15:55.402 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0913999999999999e-05 2024-09-18T10:15:55.402 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.402 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.402 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:55.402 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.769730+0000", 2024-09-18T10:15:55.402 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0071346680000000003 2024-09-18T10:15:55.402 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.402 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.402 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:55.402 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.769816+0000", 2024-09-18T10:15:55.402 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.5422999999999998e-05 2024-09-18T10:15:55.402 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.402 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.402 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:55.402 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.770214+0000", 2024-09-18T10:15:55.403 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00039818999999999998 2024-09-18T10:15:55.403 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.403 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.403 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:55.403 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.973818+0000", 2024-09-18T10:15:55.403 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20360445499999999 2024-09-18T10:15:55.403 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.403 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.403 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:15:55.403 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:30.378035+0000", 2024-09-18T10:15:55.403 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.40421707600000001 2024-09-18T10:15:55.403 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.403 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.404 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:55.404 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:30.378129+0000", 2024-09-18T10:15:55.404 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.4056999999999997e-05 2024-09-18T10:15:55.404 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:55.404 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:55.404 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:55.404 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.404 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.404 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:639 3.17s0 3:ead53b0f:::smithi18925896-25:head [setxattr _header (57) in=7b] snapc 21=[20,1f,1e,1a,16,11,10] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-18T10:15:55.404 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:29.762609+0000", 2024-09-18T10:15:55.404 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 25.578707427000001, 2024-09-18T10:15:55.404 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.61577652400000005, 2024-09-18T10:15:55.404 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:55.405 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:55.405 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:55.405 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:15:55.405 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:55.405 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 639 2024-09-18T10:15:55.405 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.405 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:55.405 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.405 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:55.405 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.762609+0000", 2024-09-18T10:15:55.405 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:55.406 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.406 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.406 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:55.406 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.762609+0000", 2024-09-18T10:15:55.406 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:55.406 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.406 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.406 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:55.406 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.762608+0000", 2024-09-18T10:15:55.406 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:15:55.406 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.406 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.406 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:55.406 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.762615+0000", 2024-09-18T10:15:55.407 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.2339999999999997e-06 2024-09-18T10:15:55.407 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.407 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.407 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:55.407 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.762628+0000", 2024-09-18T10:15:55.407 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2848e-05 2024-09-18T10:15:55.407 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.407 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.407 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:55.407 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.762636+0000", 2024-09-18T10:15:55.407 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.0970000000000006e-06 2024-09-18T10:15:55.407 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.407 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.407 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:55.408 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.870703+0000", 2024-09-18T10:15:55.408 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.108067121 2024-09-18T10:15:55.408 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.408 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.408 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:55.408 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.870949+0000", 2024-09-18T10:15:55.408 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00024526700000000002 2024-09-18T10:15:55.408 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.408 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.408 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:55.408 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.871876+0000", 2024-09-18T10:15:55.408 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00092680400000000004 2024-09-18T10:15:55.409 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.409 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.409 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:55.409 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.973821+0000", 2024-09-18T10:15:55.409 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10194552599999999 2024-09-18T10:15:55.409 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.409 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.409 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:15:55.409 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:30.378300+0000", 2024-09-18T10:15:55.409 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.40447920900000001 2024-09-18T10:15:55.409 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.409 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.409 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:55.410 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:30.378386+0000", 2024-09-18T10:15:55.410 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.5531999999999994e-05 2024-09-18T10:15:55.410 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:55.410 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:55.410 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:55.410 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.410 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.410 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.7s5 74/40 ECSubWrite(tid=168, reqid=osd.4.0:166, at_version=0'0, trim_to=0'0, roll_forward_to=74'47))", 2024-09-18T10:15:55.410 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:30.378493+0000", 2024-09-18T10:15:55.410 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 24.962823783000001, 2024-09-18T10:15:55.410 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.605881475, 2024-09-18T10:15:55.410 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:55.410 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:15:55.411 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:55.411 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.411 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:55.411 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:30.378493+0000", 2024-09-18T10:15:55.411 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:55.411 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.411 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.411 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:55.411 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:30.378493+0000", 2024-09-18T10:15:55.411 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:55.411 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.411 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.411 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:55.412 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:30.378492+0000", 2024-09-18T10:15:55.412 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:55.412 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.412 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.412 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:55.412 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:30.378513+0000", 2024-09-18T10:15:55.412 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.1427e-05 2024-09-18T10:15:55.412 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.412 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.412 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:55.412 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:30.378550+0000", 2024-09-18T10:15:55.412 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.7351999999999997e-05 2024-09-18T10:15:55.412 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.413 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.413 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:55.413 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:30.478641+0000", 2024-09-18T10:15:55.413 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100090965 2024-09-18T10:15:55.413 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.413 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.413 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:55.413 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:30.882297+0000", 2024-09-18T10:15:55.413 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.40365594799999999 2024-09-18T10:15:55.413 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.413 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.413 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:55.413 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:30.882320+0000", 2024-09-18T10:15:55.413 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.3127999999999999e-05 2024-09-18T10:15:55.414 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.414 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.414 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:55.414 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:30.984327+0000", 2024-09-18T10:15:55.414 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10200699100000001 2024-09-18T10:15:55.414 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.414 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.414 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:55.414 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:30.984374+0000", 2024-09-18T10:15:55.414 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.6953000000000003e-05 2024-09-18T10:15:55.414 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:55.414 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:55.414 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:55.414 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.415 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.415 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:758 3.17s0 3:ee962131:::smithi18925896-29:head [append 0~524288] snapc 2f=[2f,29,28,20,1f,1e,1a,11] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-18T10:15:55.415 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:40.612706+0000", 2024-09-18T10:15:55.415 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 14.728611031, 2024-09-18T10:15:55.415 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.60348796500000002, 2024-09-18T10:15:55.415 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:55.415 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:55.415 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:55.415 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:15:55.415 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:55.415 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 758 2024-09-18T10:15:55.415 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.415 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:55.415 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.416 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:55.416 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.612706+0000", 2024-09-18T10:15:55.416 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:55.416 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.416 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.416 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:55.416 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.612706+0000", 2024-09-18T10:15:55.416 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:55.416 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.416 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.416 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:55.416 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.612702+0000", 2024-09-18T10:15:55.416 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999967 2024-09-18T10:15:55.416 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.417 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.417 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:55.417 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.613684+0000", 2024-09-18T10:15:55.417 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00098185200000000007 2024-09-18T10:15:55.417 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.417 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.417 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:55.417 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.613722+0000", 2024-09-18T10:15:55.417 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.7490000000000002e-05 2024-09-18T10:15:55.417 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.417 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.417 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:55.417 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.613741+0000", 2024-09-18T10:15:55.418 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.9707999999999998e-05 2024-09-18T10:15:55.418 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.418 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.418 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:55.418 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.613931+0000", 2024-09-18T10:15:55.418 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00018992799999999999 2024-09-18T10:15:55.418 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.418 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.418 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op must wait for map", 2024-09-18T10:15:55.418 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.613950+0000", 2024-09-18T10:15:55.418 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8909000000000001e-05 2024-09-18T10:15:55.418 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.418 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.418 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:55.419 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.726459+0000", 2024-09-18T10:15:55.419 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.112508555 2024-09-18T10:15:55.419 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.419 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.419 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:55.419 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.726722+0000", 2024-09-18T10:15:55.419 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00026331999999999999 2024-09-18T10:15:55.419 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.419 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.419 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:55.419 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.728892+0000", 2024-09-18T10:15:55.419 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0021697040000000002 2024-09-18T10:15:55.419 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.419 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.420 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:55.420 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.741279+0000", 2024-09-18T10:15:55.420 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.012386856 2024-09-18T10:15:55.420 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.420 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.420 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:15:55.420 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:41.215936+0000", 2024-09-18T10:15:55.420 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.474657153 2024-09-18T10:15:55.420 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.420 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.420 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:55.420 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:41.216194+0000", 2024-09-18T10:15:55.420 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00025781799999999999 2024-09-18T10:15:55.421 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:55.421 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:55.421 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:55.421 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.421 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.421 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:759 3.17s0 3:ee962131:::smithi18925896-29:head [append 0~573440] snapc 2f=[2f,29,28,20,1f,1e,1a,11] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-18T10:15:55.421 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:40.620305+0000", 2024-09-18T10:15:55.421 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 14.721011369999999, 2024-09-18T10:15:55.421 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.59650130599999995, 2024-09-18T10:15:55.421 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:55.421 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:55.421 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:55.421 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:15:55.421 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:55.422 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 759 2024-09-18T10:15:55.422 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.422 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:55.422 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.422 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:55.422 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.620305+0000", 2024-09-18T10:15:55.422 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:55.422 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.422 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.422 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:55.422 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.620305+0000", 2024-09-18T10:15:55.422 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:55.422 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.423 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.423 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:55.423 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.620302+0000", 2024-09-18T10:15:55.423 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999971 2024-09-18T10:15:55.423 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.423 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.423 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:55.423 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.620967+0000", 2024-09-18T10:15:55.423 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00066427800000000003 2024-09-18T10:15:55.423 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.423 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.423 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:55.423 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.620994+0000", 2024-09-18T10:15:55.424 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.76e-05 2024-09-18T10:15:55.424 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.424 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.424 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:55.424 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.621004+0000", 2024-09-18T10:15:55.424 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0098000000000001e-05 2024-09-18T10:15:55.424 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.424 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.424 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:55.424 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.621152+0000", 2024-09-18T10:15:55.424 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00014735799999999999 2024-09-18T10:15:55.424 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.424 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.424 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:55.425 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.621173+0000", 2024-09-18T10:15:55.425 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.1223e-05 2024-09-18T10:15:55.425 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.425 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.425 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:55.425 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.730351+0000", 2024-09-18T10:15:55.425 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.109177487 2024-09-18T10:15:55.425 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.425 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.425 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:55.425 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.730553+0000", 2024-09-18T10:15:55.425 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00020222600000000001 2024-09-18T10:15:55.426 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.426 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.426 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:55.426 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.732189+0000", 2024-09-18T10:15:55.426 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001636153 2024-09-18T10:15:55.426 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.426 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.426 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:55.426 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.741283+0000", 2024-09-18T10:15:55.426 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0090935889999999991 2024-09-18T10:15:55.426 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.426 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.426 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:15:55.426 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:41.216570+0000", 2024-09-18T10:15:55.427 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.475287444 2024-09-18T10:15:55.427 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.427 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.427 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:55.427 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:41.216807+0000", 2024-09-18T10:15:55.427 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00023667999999999999 2024-09-18T10:15:55.427 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:55.427 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:55.427 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:55.427 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.427 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.427 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:760 3.17s0 3:ee962131:::smithi18925896-29:head [append 0~688128] snapc 2f=[2f,29,28,20,1f,1e,1a,11] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-18T10:15:55.427 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:40.624963+0000", 2024-09-18T10:15:55.427 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 14.716353865, 2024-09-18T10:15:55.427 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.592485437, 2024-09-18T10:15:55.428 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:55.428 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:55.428 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:55.428 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:15:55.428 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:55.428 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 760 2024-09-18T10:15:55.428 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.428 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:55.428 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.428 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:55.428 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.624963+0000", 2024-09-18T10:15:55.428 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:55.428 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.428 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.428 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:55.429 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.624963+0000", 2024-09-18T10:15:55.429 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:55.429 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.429 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.429 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:55.429 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.624960+0000", 2024-09-18T10:15:55.429 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999971 2024-09-18T10:15:55.429 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.429 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.429 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:55.429 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.625766+0000", 2024-09-18T10:15:55.429 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00080539100000000003 2024-09-18T10:15:55.429 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.430 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.430 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:55.430 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.625792+0000", 2024-09-18T10:15:55.430 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5989000000000001e-05 2024-09-18T10:15:55.430 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.430 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.430 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:55.430 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.625801+0000", 2024-09-18T10:15:55.430 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.4080000000000004e-06 2024-09-18T10:15:55.430 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.430 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.430 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:55.430 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.627383+0000", 2024-09-18T10:15:55.430 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0015816440000000001 2024-09-18T10:15:55.430 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.431 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.431 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:55.431 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.627397+0000", 2024-09-18T10:15:55.431 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3973e-05 2024-09-18T10:15:55.431 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.431 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.431 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:55.431 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.733560+0000", 2024-09-18T10:15:55.431 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.106163805 2024-09-18T10:15:55.431 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.431 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.431 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:55.431 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.733770+0000", 2024-09-18T10:15:55.432 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00020980900000000001 2024-09-18T10:15:55.432 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.432 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.432 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:55.432 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.735644+0000", 2024-09-18T10:15:55.432 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001874164 2024-09-18T10:15:55.432 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.432 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.432 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:55.432 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.741750+0000", 2024-09-18T10:15:55.432 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0061058379999999997 2024-09-18T10:15:55.432 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.432 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.433 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:15:55.433 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:41.217140+0000", 2024-09-18T10:15:55.433 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.47538995699999997 2024-09-18T10:15:55.433 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.433 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.433 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:55.433 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:41.217448+0000", 2024-09-18T10:15:55.433 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000308089 2024-09-18T10:15:55.433 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:55.433 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:55.433 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:55.433 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.433 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.433 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:761 3.17s0 3:ee962131:::smithi18925896-29:head [append 0~294912] snapc 2f=[2f,29,28,20,1f,1e,1a,11] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-18T10:15:55.434 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:40.626751+0000", 2024-09-18T10:15:55.434 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 14.714565598, 2024-09-18T10:15:55.434 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.59137154800000002, 2024-09-18T10:15:55.434 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:55.434 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:55.434 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:55.434 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:15:55.434 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:55.434 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 761 2024-09-18T10:15:55.434 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.434 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:55.434 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.434 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:55.435 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.626751+0000", 2024-09-18T10:15:55.435 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:55.435 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.435 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.435 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:55.435 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.626751+0000", 2024-09-18T10:15:55.435 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:55.435 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.435 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.435 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:55.435 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.626749+0000", 2024-09-18T10:15:55.435 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:55.435 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.435 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.436 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:55.436 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.627187+0000", 2024-09-18T10:15:55.436 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000437682 2024-09-18T10:15:55.436 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.436 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.436 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:55.436 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.627209+0000", 2024-09-18T10:15:55.436 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.2195999999999999e-05 2024-09-18T10:15:55.436 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.436 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.436 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:55.436 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.627218+0000", 2024-09-18T10:15:55.437 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.4829999999999999e-06 2024-09-18T10:15:55.437 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.437 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.437 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:55.437 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.627438+0000", 2024-09-18T10:15:55.437 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00022063399999999999 2024-09-18T10:15:55.437 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.437 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.437 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:55.437 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.627447+0000", 2024-09-18T10:15:55.437 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.687e-06 2024-09-18T10:15:55.437 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.438 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.438 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:55.438 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.737040+0000", 2024-09-18T10:15:55.438 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10959300800000001 2024-09-18T10:15:55.438 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.438 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.438 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:55.438 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.737281+0000", 2024-09-18T10:15:55.438 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00024113100000000001 2024-09-18T10:15:55.438 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.438 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.438 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:55.438 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.738816+0000", 2024-09-18T10:15:55.439 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0015353649999999999 2024-09-18T10:15:55.439 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.439 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.439 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:55.439 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.741752+0000", 2024-09-18T10:15:55.439 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0029357239999999998 2024-09-18T10:15:55.439 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.439 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.439 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:15:55.439 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:41.217885+0000", 2024-09-18T10:15:55.439 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.476132522 2024-09-18T10:15:55.439 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.440 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.440 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:55.440 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:41.218123+0000", 2024-09-18T10:15:55.440 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000237968 2024-09-18T10:15:55.440 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:55.440 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:55.440 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:55.440 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.440 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.440 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:762 3.17s0 3:ee962131:::smithi18925896-29:head [setxattr _header (57) in=7b] snapc 2f=[2f,29,28,20,1f,1e,1a,11] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-18T10:15:55.440 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:40.627233+0000", 2024-09-18T10:15:55.440 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 14.714083766, 2024-09-18T10:15:55.441 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.68868450599999997, 2024-09-18T10:15:55.441 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:55.441 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:55.441 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:55.441 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:15:55.441 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:55.441 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 762 2024-09-18T10:15:55.441 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.441 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:55.441 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.441 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:55.441 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.627233+0000", 2024-09-18T10:15:55.441 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:55.442 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.442 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.442 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:55.442 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.627233+0000", 2024-09-18T10:15:55.442 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:55.442 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.442 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.442 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:55.443 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.627232+0000", 2024-09-18T10:15:55.444 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:15:55.444 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.444 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.444 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:55.444 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.627238+0000", 2024-09-18T10:15:55.444 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.7570000000000001e-06 2024-09-18T10:15:55.444 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.444 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.444 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:55.444 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.627246+0000", 2024-09-18T10:15:55.444 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.0199999999999994e-06 2024-09-18T10:15:55.444 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.444 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.444 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:55.445 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.627251+0000", 2024-09-18T10:15:55.445 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9599999999999999e-06 2024-09-18T10:15:55.445 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.445 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.445 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:55.445 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.627481+0000", 2024-09-18T10:15:55.445 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00023046199999999999 2024-09-18T10:15:55.445 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.445 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.445 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:55.445 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.627493+0000", 2024-09-18T10:15:55.446 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2173e-05 2024-09-18T10:15:55.446 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.446 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.446 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:55.446 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.740060+0000", 2024-09-18T10:15:55.446 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.112566686 2024-09-18T10:15:55.446 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.446 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.446 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:55.446 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.740191+0000", 2024-09-18T10:15:55.446 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00013140999999999999 2024-09-18T10:15:55.446 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.446 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.447 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:55.447 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.740901+0000", 2024-09-18T10:15:55.447 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00070978199999999995 2024-09-18T10:15:55.447 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.447 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.447 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:55.447 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.741853+0000", 2024-09-18T10:15:55.447 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00095228399999999996 2024-09-18T10:15:55.447 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.447 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.447 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:15:55.447 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:41.315753+0000", 2024-09-18T10:15:55.447 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.57389904999999997 2024-09-18T10:15:55.447 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.448 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.448 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:55.448 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:41.315917+0000", 2024-09-18T10:15:55.448 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00016491699999999999 2024-09-18T10:15:55.448 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:55.448 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:55.448 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:55.448 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.448 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.448 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:763 3.17s0 3:ee962131:::smithi18925896-29:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e85)", 2024-09-18T10:15:55.448 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:40.627257+0000", 2024-09-18T10:15:55.448 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 14.714059481, 2024-09-18T10:15:55.449 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.89335734899999997, 2024-09-18T10:15:55.449 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:55.449 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "started", 2024-09-18T10:15:55.449 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:55.449 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:15:55.449 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:55.449 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 763 2024-09-18T10:15:55.449 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.449 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:55.449 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.449 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:55.449 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.627257+0000", 2024-09-18T10:15:55.449 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:55.449 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.450 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.450 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:55.450 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.627257+0000", 2024-09-18T10:15:55.450 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:55.450 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.450 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.450 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:55.450 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.627256+0000", 2024-09-18T10:15:55.450 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:15:55.450 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.450 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.450 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:55.450 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.627260+0000", 2024-09-18T10:15:55.451 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.7450000000000001e-06 2024-09-18T10:15:55.451 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.451 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.451 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:55.451 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.627269+0000", 2024-09-18T10:15:55.451 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.4400000000000005e-06 2024-09-18T10:15:55.451 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.451 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.451 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:55.451 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.627273+0000", 2024-09-18T10:15:55.451 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.8600000000000001e-06 2024-09-18T10:15:55.451 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.451 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.452 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:55.452 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.717888+0000", 2024-09-18T10:15:55.452 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.090614586999999996 2024-09-18T10:15:55.452 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.452 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.452 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:55.452 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.717909+0000", 2024-09-18T10:15:55.452 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.0942999999999999e-05 2024-09-18T10:15:55.452 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.452 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.452 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:55.452 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.741375+0000", 2024-09-18T10:15:55.452 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.023465604000000001 2024-09-18T10:15:55.452 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.453 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.453 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for rw locks", 2024-09-18T10:15:55.453 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.741656+0000", 2024-09-18T10:15:55.453 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00028103600000000002 2024-09-18T10:15:55.453 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.453 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.453 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:55.453 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:41.317563+0000", 2024-09-18T10:15:55.453 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.57590774499999997 2024-09-18T10:15:55.453 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.453 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.453 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:55.453 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:41.317798+0000", 2024-09-18T10:15:55.453 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00023456099999999999 2024-09-18T10:15:55.454 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.454 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.454 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:55.454 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:41.520615+0000", 2024-09-18T10:15:55.454 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20281658499999999 2024-09-18T10:15:55.454 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:55.454 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:55.454 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:55.454 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:55.454 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:55.454 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-09-18T10:15:55.456 INFO:teuthology.orchestra.run.smithi029.stdout:osd.11: {} 2024-09-18T10:15:55.456 INFO:teuthology.orchestra.run.smithi029.stdout:osd.12: {} 2024-09-18T10:15:55.456 INFO:teuthology.orchestra.run.smithi029.stdout:osd.13: {} 2024-09-18T10:15:55.457 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_ops_in_flight 2024-09-18T10:15:55.457 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-18T10:15:55.451+0000 7f00046a8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 22136) UID: 0 2024-09-18T10:15:55.489 INFO:teuthology.orchestra.run.smithi029.stdout:osd.14: {} 2024-09-18T10:15:55.513 INFO:teuthology.orchestra.run.smithi029.stdout:osd.15: {} 2024-09-18T10:15:55.516 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T10:15:55.516 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T10:15:55.516 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T10:15:55.516 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T10:15:55.516 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T10:15:55.516 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T10:15:55.516 INFO:teuthology.orchestra.run.smithi029.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T10:15:55.516 INFO:teuthology.orchestra.run.smithi029.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T10:15:55.517 INFO:teuthology.orchestra.run.smithi029.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-18T10:15:55.517 INFO:teuthology.orchestra.run.smithi029.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-18T10:15:55.517 INFO:teuthology.orchestra.run.smithi029.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-18T10:15:55.517 INFO:teuthology.orchestra.run.smithi029.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-18T10:15:55.517 INFO:teuthology.orchestra.run.smithi029.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-18T10:15:55.517 INFO:teuthology.orchestra.run.smithi029.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-18T10:15:55.517 INFO:teuthology.orchestra.run.smithi029.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-18T10:15:55.517 INFO:teuthology.orchestra.run.smithi029.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-18T10:15:55.556 INFO:tasks.ceph.osd.10.smithi154.stderr:2024-09-18T10:15:55.554+0000 7fb73221a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 22091) UID: 0 2024-09-18T10:15:55.565 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-09-18T10:15:55.565 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-09-18T10:15:55.565 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.565 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.cs6 99/94 ECSubWrite(tid=246, reqid=osd.5.0:246, at_version=99'152, trim_to=0'0, roll_forward_to=99'150))", 2024-09-18T10:15:55.566 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:55.560951+0000", 2024-09-18T10:15:55.566 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 0.0041910519999999998, 2024-09-18T10:15:55.566 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0042290009999999996, 2024-09-18T10:15:55.566 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:55.566 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "queued for pg", 2024-09-18T10:15:55.566 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:55.566 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.566 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:55.566 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:55.560951+0000", 2024-09-18T10:15:55.566 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:55.566 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.566 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.566 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:55.566 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:55.560951+0000", 2024-09-18T10:15:55.567 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:55.567 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.567 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.567 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:55.567 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:55.560949+0000", 2024-09-18T10:15:55.567 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:55.567 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.567 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.567 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:55.567 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:55.561000+0000", 2024-09-18T10:15:55.567 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.0092000000000001e-05 2024-09-18T10:15:55.567 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.567 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.568 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:55.568 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:55.561027+0000", 2024-09-18T10:15:55.568 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.7446000000000001e-05 2024-09-18T10:15:55.568 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.568 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.568 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:55.568 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:55.561044+0000", 2024-09-18T10:15:55.568 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7297000000000001e-05 2024-09-18T10:15:55.568 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:55.568 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:55.568 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:55.569 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:55.569 INFO:teuthology.orchestra.run.smithi154.stdout: ], 2024-09-18T10:15:55.569 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 1 2024-09-18T10:15:55.569 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-09-18T10:15:55.575 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_blocked_ops 2024-09-18T10:15:55.657 INFO:tasks.ceph.osd.8.smithi029.stderr:2024-09-18T10:15:55.649+0000 7f2555c27700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 22192) UID: 0 2024-09-18T10:15:55.671 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-09-18T10:15:55.671 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-09-18T10:15:55.671 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-09-18T10:15:55.671 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-09-18T10:15:55.671 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-09-18T10:15:55.681 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_historic_ops 2024-09-18T10:15:55.720 INFO:tasks.rados.rados.0.smithi189.stdout:392: done (1 left) 2024-09-18T10:15:55.720 INFO:tasks.rados.rados.0.smithi189.stdout:393: snap_remove snap 27 2024-09-18T10:15:55.725 INFO:teuthology.orchestra.run.smithi029.stderr:noscrub is set 2024-09-18T10:15:55.757 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-09-18T10:15:55.754+0000 7fdfa8698700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22087) UID: 0 2024-09-18T10:15:55.781 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-09-18T10:15:55.782 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-09-18T10:15:55.782 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-09-18T10:15:55.782 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-09-18T10:15:55.782 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.782 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4341.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-18T10:15:55.782 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:35.324351+0000", 2024-09-18T10:15:55.782 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 80.453584272000001, 2024-09-18T10:15:55.782 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.36640713699999999, 2024-09-18T10:15:55.782 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:55.782 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:55.782 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:55.783 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4341", 2024-09-18T10:15:55.783 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.29:0/3001591505", 2024-09-18T10:15:55.783 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 1 2024-09-18T10:15:55.783 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.783 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:55.783 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.783 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:55.783 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.324351+0000", 2024-09-18T10:15:55.783 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:55.783 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.783 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.783 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:55.783 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.324351+0000", 2024-09-18T10:15:55.784 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:55.784 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.784 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.784 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:55.784 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.324348+0000", 2024-09-18T10:15:55.784 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999976 2024-09-18T10:15:55.784 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.784 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.784 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:55.784 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.324359+0000", 2024-09-18T10:15:55.784 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1048e-05 2024-09-18T10:15:55.784 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.784 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.785 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:55.785 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.324383+0000", 2024-09-18T10:15:55.785 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.3813000000000001e-05 2024-09-18T10:15:55.785 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.785 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.785 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:55.785 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.324421+0000", 2024-09-18T10:15:55.785 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8012000000000001e-05 2024-09-18T10:15:55.785 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.785 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.785 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:55.785 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.324548+0000", 2024-09-18T10:15:55.785 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00012702199999999999 2024-09-18T10:15:55.785 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.786 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.786 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for peered", 2024-09-18T10:15:55.786 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.324565+0000", 2024-09-18T10:15:55.786 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.6325e-05 2024-09-18T10:15:55.786 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.786 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.786 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:55.786 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.588380+0000", 2024-09-18T10:15:55.786 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.263815458 2024-09-18T10:15:55.786 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.786 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.786 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:55.786 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.588626+0000", 2024-09-18T10:15:55.787 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00024633499999999998 2024-09-18T10:15:55.787 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.787 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.787 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 5", 2024-09-18T10:15:55.787 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.588803+0000", 2024-09-18T10:15:55.787 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00017640599999999999 2024-09-18T10:15:55.787 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.787 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.787 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:15:55.787 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.589943+0000", 2024-09-18T10:15:55.788 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0011400049999999999 2024-09-18T10:15:55.788 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.788 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.788 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:15:55.788 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.690623+0000", 2024-09-18T10:15:55.788 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100680032 2024-09-18T10:15:55.788 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.788 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.788 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:15:55.788 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.690702+0000", 2024-09-18T10:15:55.788 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.9342e-05 2024-09-18T10:15:55.788 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.789 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.789 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:55.789 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.690758+0000", 2024-09-18T10:15:55.789 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5639e-05 2024-09-18T10:15:55.789 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:55.789 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:55.789 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:55.789 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.789 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.789 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:264 3.2s0 3:46b2871c:::smithi18925896-34:head [append 0~606208] snapc 5=[5,4,3] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-18T10:15:55.789 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:56.739704+0000", 2024-09-18T10:15:55.789 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 59.038231287000002, 2024-09-18T10:15:55.790 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.79624036899999995, 2024-09-18T10:15:55.790 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:55.790 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:55.790 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:55.790 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:15:55.790 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:55.790 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 264 2024-09-18T10:15:55.790 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.790 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:55.790 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.790 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:55.790 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.739704+0000", 2024-09-18T10:15:55.790 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:55.790 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.791 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.791 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:55.791 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.739704+0000", 2024-09-18T10:15:55.791 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:55.791 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.791 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.791 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:55.791 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.739702+0000", 2024-09-18T10:15:55.791 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:55.791 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.791 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.791 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:55.791 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.740467+0000", 2024-09-18T10:15:55.791 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00076493100000000005 2024-09-18T10:15:55.792 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.792 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.792 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:55.792 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.740485+0000", 2024-09-18T10:15:55.792 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8516e-05 2024-09-18T10:15:55.792 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.792 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.792 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:55.792 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.740495+0000", 2024-09-18T10:15:55.792 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.2669999999999995e-06 2024-09-18T10:15:55.792 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.792 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.792 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:55.792 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.028948+0000", 2024-09-18T10:15:55.793 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.28845313500000003 2024-09-18T10:15:55.793 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.793 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.793 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for peered", 2024-09-18T10:15:55.793 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.028969+0000", 2024-09-18T10:15:55.793 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.1369000000000002e-05 2024-09-18T10:15:55.793 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.793 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.793 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:55.793 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.228324+0000", 2024-09-18T10:15:55.793 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.19935476699999999 2024-09-18T10:15:55.793 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.793 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.793 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:55.794 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.228558+0000", 2024-09-18T10:15:55.794 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00023393099999999999 2024-09-18T10:15:55.794 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.794 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.794 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:55.794 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.230390+0000", 2024-09-18T10:15:55.794 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018326970000000001 2024-09-18T10:15:55.794 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.794 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.794 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:55.794 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.239491+0000", 2024-09-18T10:15:55.794 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0091007950000000001 2024-09-18T10:15:55.794 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.795 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.795 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:15:55.795 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.535834+0000", 2024-09-18T10:15:55.795 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.29634234100000001 2024-09-18T10:15:55.795 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.795 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.795 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:55.795 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.535944+0000", 2024-09-18T10:15:55.795 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000110401 2024-09-18T10:15:55.795 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:55.795 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:55.795 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:55.795 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.796 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.796 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:265 3.2s0 3:46b2871c:::smithi18925896-34:head [append 0~376832] snapc 5=[5,4,3] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-18T10:15:55.796 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:56.741882+0000", 2024-09-18T10:15:55.796 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 59.036053265, 2024-09-18T10:15:55.796 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.80071129500000005, 2024-09-18T10:15:55.796 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:55.796 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:55.796 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:55.796 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:15:55.796 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:55.796 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 265 2024-09-18T10:15:55.796 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.796 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:55.797 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.797 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:55.797 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.741882+0000", 2024-09-18T10:15:55.797 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:55.797 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.797 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.797 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:55.797 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.741882+0000", 2024-09-18T10:15:55.797 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:55.797 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.797 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.797 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:55.797 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.741880+0000", 2024-09-18T10:15:55.798 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:55.798 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.798 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.798 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:55.798 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742251+0000", 2024-09-18T10:15:55.798 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00037050499999999998 2024-09-18T10:15:55.798 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.798 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.798 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:55.798 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742264+0000", 2024-09-18T10:15:55.798 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2787e-05 2024-09-18T10:15:55.798 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.799 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.799 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:55.799 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742269+0000", 2024-09-18T10:15:55.799 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.0509999999999996e-06 2024-09-18T10:15:55.799 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.799 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.799 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:55.799 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.234342+0000", 2024-09-18T10:15:55.799 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.492073712 2024-09-18T10:15:55.799 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.799 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.799 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:55.799 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.234556+0000", 2024-09-18T10:15:55.799 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00021380699999999999 2024-09-18T10:15:55.800 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.800 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.800 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:55.800 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.236316+0000", 2024-09-18T10:15:55.800 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0017597100000000001 2024-09-18T10:15:55.800 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.800 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.800 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:55.800 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.241476+0000", 2024-09-18T10:15:55.800 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0051604449999999996 2024-09-18T10:15:55.800 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.800 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.800 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:15:55.801 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.542502+0000", 2024-09-18T10:15:55.801 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.30102573700000002 2024-09-18T10:15:55.801 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.801 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.801 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:55.801 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.542593+0000", 2024-09-18T10:15:55.801 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.0859000000000005e-05 2024-09-18T10:15:55.801 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:55.801 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:55.801 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:55.801 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.801 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.801 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:266 3.2s0 3:46b2871c:::smithi18925896-34:head [setxattr _header (56) in=7b] snapc 5=[5,4,3] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-18T10:15:55.802 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:56.742274+0000", 2024-09-18T10:15:55.802 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 59.035660634000003, 2024-09-18T10:15:55.802 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.80056005299999999, 2024-09-18T10:15:55.802 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:55.802 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:55.802 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:55.802 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:15:55.802 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:55.802 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 266 2024-09-18T10:15:55.802 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.802 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:55.802 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.803 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:55.803 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742274+0000", 2024-09-18T10:15:55.803 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:55.803 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.803 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.803 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:55.803 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742274+0000", 2024-09-18T10:15:55.803 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:55.803 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.803 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.803 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:55.803 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742274+0000", 2024-09-18T10:15:55.803 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:55.803 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.804 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.804 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:55.804 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742278+0000", 2024-09-18T10:15:55.804 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.0840000000000002e-06 2024-09-18T10:15:55.804 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.804 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.804 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:55.804 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742284+0000", 2024-09-18T10:15:55.804 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.1090000000000003e-06 2024-09-18T10:15:55.804 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.804 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.804 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:55.804 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742289+0000", 2024-09-18T10:15:55.805 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.4890000000000003e-06 2024-09-18T10:15:55.805 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.805 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.805 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:55.805 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.237660+0000", 2024-09-18T10:15:55.805 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.49537068299999998 2024-09-18T10:15:55.805 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.805 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.805 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:55.805 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.237924+0000", 2024-09-18T10:15:55.805 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00026387300000000003 2024-09-18T10:15:55.805 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.806 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.806 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:55.806 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.238979+0000", 2024-09-18T10:15:55.806 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0010549380000000001 2024-09-18T10:15:55.806 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.806 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.806 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:55.806 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.243571+0000", 2024-09-18T10:15:55.806 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0045919749999999999 2024-09-18T10:15:55.806 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.806 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.806 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:15:55.806 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.542742+0000", 2024-09-18T10:15:55.806 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.29917095999999999 2024-09-18T10:15:55.807 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.807 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.807 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:55.807 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.542834+0000", 2024-09-18T10:15:55.807 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.2536999999999998e-05 2024-09-18T10:15:55.807 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:55.807 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:55.807 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:55.807 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.807 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.807 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:267 3.2s0 3:46b2871c:::smithi18925896-34:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e41)", 2024-09-18T10:15:55.807 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:56.742292+0000", 2024-09-18T10:15:55.807 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 59.035642795999998, 2024-09-18T10:15:55.807 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.98973012599999999, 2024-09-18T10:15:55.808 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:55.808 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "started", 2024-09-18T10:15:55.808 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:55.808 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:15:55.808 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:55.808 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 267 2024-09-18T10:15:55.808 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.808 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:55.808 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.808 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:55.808 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742292+0000", 2024-09-18T10:15:55.808 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:55.808 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.808 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.809 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:55.809 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742292+0000", 2024-09-18T10:15:55.809 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:55.809 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.809 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.809 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:55.809 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742292+0000", 2024-09-18T10:15:55.809 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:55.809 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.809 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.809 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:55.809 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742294+0000", 2024-09-18T10:15:55.809 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.103e-06 2024-09-18T10:15:55.809 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.810 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.810 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:55.810 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742298+0000", 2024-09-18T10:15:55.810 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8070000000000001e-06 2024-09-18T10:15:55.810 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.810 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.810 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:55.810 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742300+0000", 2024-09-18T10:15:55.810 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5890000000000001e-06 2024-09-18T10:15:55.810 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.810 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.810 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:55.810 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.239711+0000", 2024-09-18T10:15:55.811 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.497410625 2024-09-18T10:15:55.811 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.811 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.811 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for rw locks", 2024-09-18T10:15:55.811 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.239786+0000", 2024-09-18T10:15:55.811 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.4827999999999996e-05 2024-09-18T10:15:55.811 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.811 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.811 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:55.811 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.543234+0000", 2024-09-18T10:15:55.811 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.30344799099999997 2024-09-18T10:15:55.811 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.811 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.811 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:55.812 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.543335+0000", 2024-09-18T10:15:55.812 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000101625 2024-09-18T10:15:55.812 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.812 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.812 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:55.812 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.732022+0000", 2024-09-18T10:15:55.812 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.18868686700000001 2024-09-18T10:15:55.812 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:55.812 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:55.812 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:55.812 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.812 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.812 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:496 3.2s0 3:44cb9e27:::smithi18925896-5:head [read 0~81920,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T10:15:55.812 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:17.889532+0000", 2024-09-18T10:15:55.813 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 37.888403177000001, 2024-09-18T10:15:55.813 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.32688004999999998, 2024-09-18T10:15:55.813 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:55.813 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "started", 2024-09-18T10:15:55.813 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:55.813 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:15:55.813 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:55.813 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 496 2024-09-18T10:15:55.813 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.813 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:55.813 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.813 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:55.814 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889532+0000", 2024-09-18T10:15:55.814 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:55.814 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.814 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.814 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:55.814 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889532+0000", 2024-09-18T10:15:55.814 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:55.814 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.814 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.814 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:55.814 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889529+0000", 2024-09-18T10:15:55.814 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999971 2024-09-18T10:15:55.814 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.814 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.814 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:55.814 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889546+0000", 2024-09-18T10:15:55.815 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.6690000000000001e-05 2024-09-18T10:15:55.815 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.815 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.815 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:55.815 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889580+0000", 2024-09-18T10:15:55.815 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.4638999999999998e-05 2024-09-18T10:15:55.815 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.815 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.815 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:55.815 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889596+0000", 2024-09-18T10:15:55.815 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.5571999999999999e-05 2024-09-18T10:15:55.815 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.815 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.815 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:55.816 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.989817+0000", 2024-09-18T10:15:55.816 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100221713 2024-09-18T10:15:55.816 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.816 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.816 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op must wait for map", 2024-09-18T10:15:55.816 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.989836+0000", 2024-09-18T10:15:55.816 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8516e-05 2024-09-18T10:15:55.816 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.816 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.816 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:55.816 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.990420+0000", 2024-09-18T10:15:55.816 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000583918 2024-09-18T10:15:55.817 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.817 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.817 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:55.817 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.990693+0000", 2024-09-18T10:15:55.817 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00027301499999999999 2024-09-18T10:15:55.817 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.817 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.817 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:55.817 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:18.216412+0000", 2024-09-18T10:15:55.817 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.22571880699999999 2024-09-18T10:15:55.817 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:55.817 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:55.817 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:55.818 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.818 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.818 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:497 3.2s0 3:44cb9e27:::smithi18925896-5:head [read 0~81920,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T10:15:55.818 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:17.889627+0000", 2024-09-18T10:15:55.818 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 37.888308260000002, 2024-09-18T10:15:55.818 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.32738385800000003, 2024-09-18T10:15:55.818 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:55.818 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "started", 2024-09-18T10:15:55.818 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:55.818 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:15:55.818 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:55.818 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 497 2024-09-18T10:15:55.818 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.819 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:55.819 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.819 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:55.819 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889627+0000", 2024-09-18T10:15:55.819 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:55.819 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.819 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.819 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:55.819 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889627+0000", 2024-09-18T10:15:55.819 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:55.819 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.819 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.819 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:55.820 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889625+0000", 2024-09-18T10:15:55.820 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:15:55.820 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.820 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.820 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:55.820 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889633+0000", 2024-09-18T10:15:55.820 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.4170000000000003e-06 2024-09-18T10:15:55.820 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.820 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.820 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:55.820 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889664+0000", 2024-09-18T10:15:55.820 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0589000000000001e-05 2024-09-18T10:15:55.820 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.820 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.821 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:55.821 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889672+0000", 2024-09-18T10:15:55.821 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.3729999999999993e-06 2024-09-18T10:15:55.821 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.821 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.821 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:55.821 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.990017+0000", 2024-09-18T10:15:55.821 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100345608 2024-09-18T10:15:55.821 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.821 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.821 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:55.821 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.990046+0000", 2024-09-18T10:15:55.821 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8912999999999999e-05 2024-09-18T10:15:55.822 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.822 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.822 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:55.822 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.991459+0000", 2024-09-18T10:15:55.822 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0014122340000000001 2024-09-18T10:15:55.822 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.822 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.822 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:55.822 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.991631+0000", 2024-09-18T10:15:55.822 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000172029 2024-09-18T10:15:55.822 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.823 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.823 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:55.823 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:18.217010+0000", 2024-09-18T10:15:55.823 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.22537980499999999 2024-09-18T10:15:55.823 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:55.823 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:55.823 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:55.823 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.823 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.823 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:498 3.2s0 3:44cb9e27:::smithi18925896-5:head [read 0~81920,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T10:15:55.823 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:17.889709+0000", 2024-09-18T10:15:55.823 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 37.888225902000002, 2024-09-18T10:15:55.823 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.32775850600000001, 2024-09-18T10:15:55.823 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:55.824 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "started", 2024-09-18T10:15:55.824 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:55.824 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:15:55.824 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:55.824 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 498 2024-09-18T10:15:55.824 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.824 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:55.824 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.824 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:55.824 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889709+0000", 2024-09-18T10:15:55.824 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:55.824 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.824 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.825 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:55.825 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889709+0000", 2024-09-18T10:15:55.825 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:55.825 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.825 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.825 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:55.825 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889708+0000", 2024-09-18T10:15:55.825 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:15:55.825 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.825 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.825 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:55.825 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889716+0000", 2024-09-18T10:15:55.825 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.0989999999999999e-06 2024-09-18T10:15:55.826 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.826 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.826 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:55.826 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889732+0000", 2024-09-18T10:15:55.826 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.6093e-05 2024-09-18T10:15:55.826 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.826 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.826 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:55.826 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889742+0000", 2024-09-18T10:15:55.826 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.4599999999999992e-06 2024-09-18T10:15:55.826 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.826 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.826 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:55.827 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.992175+0000", 2024-09-18T10:15:55.827 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10243377200000001 2024-09-18T10:15:55.827 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.827 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.827 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:55.827 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.992333+0000", 2024-09-18T10:15:55.827 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00015753499999999999 2024-09-18T10:15:55.827 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.827 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.827 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:55.827 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:18.217467+0000", 2024-09-18T10:15:55.827 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.22513459999999999 2024-09-18T10:15:55.827 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:55.827 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:55.828 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:55.828 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.828 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.828 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:516 3.2s0 3:44cb9e27:::smithi18925896-5:head [append 0~425984] snapc 1a=[1a,19,18,16,12,11,10,f,e] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-18T10:15:55.828 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:19.430973+0000", 2024-09-18T10:15:55.828 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 36.346961512999997, 2024-09-18T10:15:55.828 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.59812562800000002, 2024-09-18T10:15:55.828 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:55.828 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:55.828 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:55.828 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:15:55.828 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:55.828 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 516 2024-09-18T10:15:55.829 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.829 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:55.829 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.829 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:55.829 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.430973+0000", 2024-09-18T10:15:55.829 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:55.829 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.829 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.829 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:55.829 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.430973+0000", 2024-09-18T10:15:55.829 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:55.829 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.829 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.829 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:55.830 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.430970+0000", 2024-09-18T10:15:55.830 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999967 2024-09-18T10:15:55.830 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.830 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.830 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:55.830 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.431995+0000", 2024-09-18T10:15:55.830 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001024896 2024-09-18T10:15:55.830 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.830 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.830 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:55.830 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.432030+0000", 2024-09-18T10:15:55.830 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5241999999999999e-05 2024-09-18T10:15:55.830 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.830 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.831 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:55.831 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.432044+0000", 2024-09-18T10:15:55.831 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.411e-05 2024-09-18T10:15:55.831 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.831 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.831 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:55.831 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.432226+0000", 2024-09-18T10:15:55.831 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000181841 2024-09-18T10:15:55.831 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.831 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.831 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:55.831 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.432421+0000", 2024-09-18T10:15:55.832 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000194527 2024-09-18T10:15:55.832 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.832 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.832 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:55.832 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.434150+0000", 2024-09-18T10:15:55.832 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001729742 2024-09-18T10:15:55.832 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.832 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.832 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:55.832 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.436513+0000", 2024-09-18T10:15:55.832 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0023622299999999999 2024-09-18T10:15:55.832 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.832 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.832 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:15:55.833 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:20.029005+0000", 2024-09-18T10:15:55.833 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.59249230100000005 2024-09-18T10:15:55.833 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.833 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.833 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:55.833 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:20.029099+0000", 2024-09-18T10:15:55.833 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.4042000000000005e-05 2024-09-18T10:15:55.833 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:55.833 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:55.833 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:55.833 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.833 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.833 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:517 3.2s0 3:44cb9e27:::smithi18925896-5:head [append 0~786432] snapc 1a=[1a,19,18,16,12,11,10,f,e] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-18T10:15:55.833 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:19.439831+0000", 2024-09-18T10:15:55.834 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 36.338103973999999, 2024-09-18T10:15:55.834 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.58958403800000003, 2024-09-18T10:15:55.834 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:55.834 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:55.834 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:55.834 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:15:55.834 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:55.834 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 517 2024-09-18T10:15:55.834 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.834 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:55.834 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.834 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:55.835 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.439831+0000", 2024-09-18T10:15:55.835 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:55.835 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.835 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.835 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:55.835 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.439831+0000", 2024-09-18T10:15:55.835 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:55.835 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.835 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.835 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:55.835 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.439828+0000", 2024-09-18T10:15:55.835 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999976 2024-09-18T10:15:55.835 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.836 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.836 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:55.836 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.441109+0000", 2024-09-18T10:15:55.836 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001280988 2024-09-18T10:15:55.836 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.836 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.836 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:55.836 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.441168+0000", 2024-09-18T10:15:55.836 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.8869000000000002e-05 2024-09-18T10:15:55.836 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.836 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.836 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:55.837 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.541302+0000", 2024-09-18T10:15:55.837 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100134055 2024-09-18T10:15:55.837 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.837 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.837 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:55.837 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.637962+0000", 2024-09-18T10:15:55.837 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.096659682999999996 2024-09-18T10:15:55.837 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.837 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.837 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:55.837 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.638164+0000", 2024-09-18T10:15:55.837 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00020226700000000001 2024-09-18T10:15:55.837 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.838 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.838 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:55.838 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.639933+0000", 2024-09-18T10:15:55.838 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001768501 2024-09-18T10:15:55.838 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.838 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.838 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:55.838 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.646572+0000", 2024-09-18T10:15:55.838 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.006639069 2024-09-18T10:15:55.838 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.838 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.838 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:15:55.838 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:20.029341+0000", 2024-09-18T10:15:55.838 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.38276939599999998 2024-09-18T10:15:55.839 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.839 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.839 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:55.839 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:20.029415+0000", 2024-09-18T10:15:55.839 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.3726000000000005e-05 2024-09-18T10:15:55.839 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:55.839 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:55.839 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:55.839 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.839 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.839 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:518 3.2s0 3:44cb9e27:::smithi18925896-5:head [append 0~376832] snapc 1a=[1a,19,18,16,12,11,10,f,e] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-18T10:15:55.839 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:19.541352+0000", 2024-09-18T10:15:55.839 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 36.236582935999998, 2024-09-18T10:15:55.839 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.48828847600000003, 2024-09-18T10:15:55.840 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:55.840 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:55.840 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:55.840 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:15:55.840 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:55.840 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 518 2024-09-18T10:15:55.840 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.840 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:55.840 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.840 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:55.840 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.541352+0000", 2024-09-18T10:15:55.840 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:55.840 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.841 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.841 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:55.841 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.541352+0000", 2024-09-18T10:15:55.841 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:55.841 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.841 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.841 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:55.841 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.541350+0000", 2024-09-18T10:15:55.841 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:55.841 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.841 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.841 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:55.841 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.541834+0000", 2024-09-18T10:15:55.842 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00048402500000000002 2024-09-18T10:15:55.842 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.842 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.842 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:55.842 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.541861+0000", 2024-09-18T10:15:55.842 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.7333000000000001e-05 2024-09-18T10:15:55.842 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.842 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.842 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:55.842 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.541875+0000", 2024-09-18T10:15:55.842 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4025999999999999e-05 2024-09-18T10:15:55.842 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.842 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.843 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:55.843 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.641684+0000", 2024-09-18T10:15:55.843 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.099808872000000007 2024-09-18T10:15:55.843 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.843 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.843 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:55.843 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.641873+0000", 2024-09-18T10:15:55.843 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00018933200000000001 2024-09-18T10:15:55.843 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.843 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.843 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:55.843 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.643401+0000", 2024-09-18T10:15:55.843 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001527104 2024-09-18T10:15:55.843 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.844 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.844 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:55.844 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.646854+0000", 2024-09-18T10:15:55.844 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0034532069999999998 2024-09-18T10:15:55.844 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.844 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.844 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:15:55.844 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:20.029569+0000", 2024-09-18T10:15:55.844 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.38271509599999998 2024-09-18T10:15:55.844 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.844 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.844 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:55.844 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:20.029640+0000", 2024-09-18T10:15:55.845 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.1594000000000006e-05 2024-09-18T10:15:55.845 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:55.845 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:55.845 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:55.845 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.845 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.845 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:519 3.2s0 3:44cb9e27:::smithi18925896-5:head [setxattr _header (56) in=7b] snapc 1a=[1a,19,18,16,12,11,10,f,e] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-18T10:15:55.845 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:19.541886+0000", 2024-09-18T10:15:55.845 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 36.236049248999997, 2024-09-18T10:15:55.845 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.488165768, 2024-09-18T10:15:55.845 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:55.845 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:55.845 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:55.846 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:15:55.846 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:55.846 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 519 2024-09-18T10:15:55.846 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.846 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:55.846 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.846 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:55.846 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.541886+0000", 2024-09-18T10:15:55.846 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:55.846 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.846 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.846 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:55.846 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.541886+0000", 2024-09-18T10:15:55.846 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:55.847 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.847 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.847 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:55.847 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.541884+0000", 2024-09-18T10:15:55.847 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:55.847 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.847 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.847 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:55.847 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.541890+0000", 2024-09-18T10:15:55.847 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.0589999999999999e-06 2024-09-18T10:15:55.847 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.847 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.847 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:55.847 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.541899+0000", 2024-09-18T10:15:55.848 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.02e-06 2024-09-18T10:15:55.848 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.848 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.848 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:55.848 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.541905+0000", 2024-09-18T10:15:55.848 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.9399999999999999e-06 2024-09-18T10:15:55.848 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.848 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.848 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:55.848 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.644561+0000", 2024-09-18T10:15:55.848 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.102655408 2024-09-18T10:15:55.848 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.848 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.849 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:55.849 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.644869+0000", 2024-09-18T10:15:55.849 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00030766700000000002 2024-09-18T10:15:55.849 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.849 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.849 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:55.849 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.645745+0000", 2024-09-18T10:15:55.849 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000876314 2024-09-18T10:15:55.849 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.849 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.849 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:55.849 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.647092+0000", 2024-09-18T10:15:55.850 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0013469269999999999 2024-09-18T10:15:55.850 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.850 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.850 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:15:55.850 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:20.029901+0000", 2024-09-18T10:15:55.850 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.38280895199999998 2024-09-18T10:15:55.850 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.850 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.850 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:55.850 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:20.030051+0000", 2024-09-18T10:15:55.850 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00015067699999999999 2024-09-18T10:15:55.850 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:55.850 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:55.852 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:55.852 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.852 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.852 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:573 3.2s0 3:46b2871c:::smithi18925896-34:head [read 0~557056,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-18T10:15:55.852 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:24.942342+0000", 2024-09-18T10:15:55.852 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 30.835592535, 2024-09-18T10:15:55.852 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.36312128700000001, 2024-09-18T10:15:55.852 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:55.852 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "started", 2024-09-18T10:15:55.853 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:55.853 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:15:55.853 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:55.853 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 573 2024-09-18T10:15:55.853 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.853 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:55.853 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.853 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:55.853 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.942342+0000", 2024-09-18T10:15:55.853 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:55.853 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.853 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.853 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:55.853 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.942342+0000", 2024-09-18T10:15:55.854 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:55.854 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.854 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.854 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:55.854 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.942341+0000", 2024-09-18T10:15:55.854 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:15:55.854 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.854 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.854 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:55.854 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.942363+0000", 2024-09-18T10:15:55.854 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.1406000000000001e-05 2024-09-18T10:15:55.854 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.854 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.854 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:55.855 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.942374+0000", 2024-09-18T10:15:55.855 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1466e-05 2024-09-18T10:15:55.855 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.855 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.855 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:55.855 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.942381+0000", 2024-09-18T10:15:55.855 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.9530000000000002e-06 2024-09-18T10:15:55.855 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.855 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.855 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:55.855 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.942654+0000", 2024-09-18T10:15:55.855 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000272692 2024-09-18T10:15:55.855 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.855 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.856 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:55.856 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.942672+0000", 2024-09-18T10:15:55.856 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7643e-05 2024-09-18T10:15:55.856 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.856 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.856 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:55.856 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:25.101730+0000", 2024-09-18T10:15:55.856 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.15905815800000001 2024-09-18T10:15:55.856 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.856 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.856 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:55.856 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:25.101881+0000", 2024-09-18T10:15:55.856 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000151299 2024-09-18T10:15:55.857 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.857 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.857 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:55.857 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:25.305464+0000", 2024-09-18T10:15:55.857 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.203582656 2024-09-18T10:15:55.857 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:55.857 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:55.857 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:55.857 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.857 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.857 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:574 3.2s0 3:46b2871c:::smithi18925896-34:head [read 0~557056,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-18T10:15:55.857 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:24.942392+0000", 2024-09-18T10:15:55.857 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 30.835542739000001, 2024-09-18T10:15:55.857 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.36415850300000002, 2024-09-18T10:15:55.858 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:55.858 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "started", 2024-09-18T10:15:55.858 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:55.858 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:15:55.858 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:55.858 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 574 2024-09-18T10:15:55.858 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.858 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:55.858 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.858 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:55.858 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.942392+0000", 2024-09-18T10:15:55.858 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:55.858 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.859 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.859 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:55.859 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.942392+0000", 2024-09-18T10:15:55.859 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:55.859 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.859 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.859 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:55.859 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.942391+0000", 2024-09-18T10:15:55.859 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:15:55.859 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.859 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.859 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:55.859 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.942396+0000", 2024-09-18T10:15:55.859 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.6759999999999998e-06 2024-09-18T10:15:55.860 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.860 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.860 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:55.860 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.942407+0000", 2024-09-18T10:15:55.860 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0686999999999999e-05 2024-09-18T10:15:55.860 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.860 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.860 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:55.860 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.942413+0000", 2024-09-18T10:15:55.860 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.1659999999999999e-06 2024-09-18T10:15:55.860 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.860 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.860 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:55.860 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.942750+0000", 2024-09-18T10:15:55.861 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00033731700000000001 2024-09-18T10:15:55.861 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.861 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.861 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:55.861 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.942776+0000", 2024-09-18T10:15:55.861 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6342999999999998e-05 2024-09-18T10:15:55.861 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.861 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.861 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:55.861 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:25.102396+0000", 2024-09-18T10:15:55.861 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.15961924199999999 2024-09-18T10:15:55.861 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.861 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.862 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:55.862 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:25.102659+0000", 2024-09-18T10:15:55.862 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00026355500000000001 2024-09-18T10:15:55.862 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.862 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.862 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:55.862 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:25.306551+0000", 2024-09-18T10:15:55.862 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20389143700000001 2024-09-18T10:15:55.862 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:55.862 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:55.862 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:55.862 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.862 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.863 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWriteReply(3.2s0 89/40 ECSubWriteReply(tid=135, last_complete=89'96, committed=1, applied=1))", 2024-09-18T10:15:55.863 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:45.106385+0000", 2024-09-18T10:15:55.863 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 10.671549965000001, 2024-09-18T10:15:55.863 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.35616392200000002, 2024-09-18T10:15:55.863 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:55.863 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:15:55.863 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:55.863 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.863 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:55.863 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:45.106385+0000", 2024-09-18T10:15:55.863 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:55.863 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.863 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.863 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:55.864 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:45.106385+0000", 2024-09-18T10:15:55.864 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:55.864 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.864 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.864 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:55.864 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:45.106384+0000", 2024-09-18T10:15:55.864 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:55.864 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.864 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.864 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:55.864 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:45.106388+0000", 2024-09-18T10:15:55.864 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.6100000000000002e-06 2024-09-18T10:15:55.864 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.865 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.865 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:55.865 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:45.106395+0000", 2024-09-18T10:15:55.865 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.8399999999999997e-06 2024-09-18T10:15:55.865 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.865 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.865 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:55.865 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:45.106399+0000", 2024-09-18T10:15:55.865 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.1899999999999997e-06 2024-09-18T10:15:55.865 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.865 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.865 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:55.865 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:45.461488+0000", 2024-09-18T10:15:55.865 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.35508847199999999 2024-09-18T10:15:55.866 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.866 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.866 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:55.866 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:45.462549+0000", 2024-09-18T10:15:55.866 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0010612760000000001 2024-09-18T10:15:55.866 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:55.866 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:55.866 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:55.866 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.866 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.866 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:891 3.2s0 3:427b2c0b:::smithi18925896-42:head [append 0~0] snapc 34=[34,33,30,2f,28,20,1f,1a,11] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-18T10:15:55.866 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:46.655855+0000", 2024-09-18T10:15:55.866 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 9.1220802370000005, 2024-09-18T10:15:55.867 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.45749477399999999, 2024-09-18T10:15:55.867 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:55.867 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:55.867 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:55.867 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:15:55.867 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:55.867 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 891 2024-09-18T10:15:55.867 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.867 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:55.867 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.867 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:55.867 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.655855+0000", 2024-09-18T10:15:55.867 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:55.867 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.868 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.868 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:55.868 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.655855+0000", 2024-09-18T10:15:55.868 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:55.868 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.868 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.868 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:55.868 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.655852+0000", 2024-09-18T10:15:55.868 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999971 2024-09-18T10:15:55.868 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.868 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.868 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:55.868 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.655866+0000", 2024-09-18T10:15:55.868 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4842e-05 2024-09-18T10:15:55.869 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.869 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.869 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:55.869 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.655906+0000", 2024-09-18T10:15:55.869 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.9465000000000002e-05 2024-09-18T10:15:55.869 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.869 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.869 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:55.869 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.756059+0000", 2024-09-18T10:15:55.869 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100153544 2024-09-18T10:15:55.869 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.869 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.869 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:55.870 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.804067+0000", 2024-09-18T10:15:55.870 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.048007288000000002 2024-09-18T10:15:55.870 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.870 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.870 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:55.870 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.804374+0000", 2024-09-18T10:15:55.870 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00030716700000000001 2024-09-18T10:15:55.870 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.870 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.870 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:55.870 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.807008+0000", 2024-09-18T10:15:55.870 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.002633799 2024-09-18T10:15:55.870 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.870 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.871 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:55.871 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.911478+0000", 2024-09-18T10:15:55.871 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.104469986 2024-09-18T10:15:55.871 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.871 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.871 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:15:55.871 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:47.113216+0000", 2024-09-18T10:15:55.871 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.201737789 2024-09-18T10:15:55.871 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.871 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.871 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:55.871 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:47.113349+0000", 2024-09-18T10:15:55.871 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00013389700000000001 2024-09-18T10:15:55.872 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:55.872 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:55.872 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:55.872 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.872 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.872 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:892 3.2s0 3:427b2c0b:::smithi18925896-42:head [rollback 28] snapc 34=[34,33,30,2f,28,20,1f,1a,11] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-18T10:15:55.872 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:46.756090+0000", 2024-09-18T10:15:55.872 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 9.0218451569999996, 2024-09-18T10:15:55.872 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.35768046799999997, 2024-09-18T10:15:55.872 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:55.872 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:55.872 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:55.872 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:15:55.873 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:55.873 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 892 2024-09-18T10:15:55.873 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.873 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:55.873 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.873 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:55.873 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.756090+0000", 2024-09-18T10:15:55.873 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:55.873 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.873 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.873 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:55.873 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.756090+0000", 2024-09-18T10:15:55.873 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:55.873 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.874 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.874 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:55.874 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.756088+0000", 2024-09-18T10:15:55.874 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:55.874 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.874 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.874 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:55.874 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.756099+0000", 2024-09-18T10:15:55.874 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0879999999999999e-05 2024-09-18T10:15:55.874 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.874 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.874 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:55.874 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.756116+0000", 2024-09-18T10:15:55.875 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7849000000000001e-05 2024-09-18T10:15:55.875 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.875 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.875 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:55.875 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.756136+0000", 2024-09-18T10:15:55.875 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.9720000000000001e-05 2024-09-18T10:15:55.875 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.875 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.875 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:55.875 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.808713+0000", 2024-09-18T10:15:55.875 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.052577089 2024-09-18T10:15:55.875 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.875 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.875 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:55.876 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.808963+0000", 2024-09-18T10:15:55.876 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000249824 2024-09-18T10:15:55.876 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.876 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.876 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:55.876 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.810124+0000", 2024-09-18T10:15:55.876 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0011610100000000001 2024-09-18T10:15:55.876 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.876 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.876 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:55.876 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:47.011739+0000", 2024-09-18T10:15:55.876 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20161468900000001 2024-09-18T10:15:55.876 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.877 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.877 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:15:55.877 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:47.113617+0000", 2024-09-18T10:15:55.877 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10187808700000001 2024-09-18T10:15:55.877 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.877 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.877 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:55.877 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:47.113770+0000", 2024-09-18T10:15:55.877 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00015334300000000001 2024-09-18T10:15:55.877 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:55.877 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:55.877 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:55.878 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.878 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.878 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:893 3.2s0 3:427b2c0b:::smithi18925896-42:head [append 0~0] snapc 34=[34,33,30,2f,28,20,1f,1a,11] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-18T10:15:55.878 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:46.756179+0000", 2024-09-18T10:15:55.878 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 9.0217562489999992, 2024-09-18T10:15:55.878 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.66260898499999998, 2024-09-18T10:15:55.878 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:55.878 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:55.878 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:55.878 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:15:55.878 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:55.878 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 893 2024-09-18T10:15:55.878 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.879 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:55.879 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.879 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:55.879 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.756179+0000", 2024-09-18T10:15:55.879 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:55.879 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.879 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.879 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:55.879 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.756179+0000", 2024-09-18T10:15:55.879 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:55.879 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.879 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.879 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:55.880 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.756177+0000", 2024-09-18T10:15:55.880 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:55.880 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.880 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.880 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:55.880 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.756185+0000", 2024-09-18T10:15:55.880 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.32e-06 2024-09-18T10:15:55.880 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.880 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.880 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:55.881 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.756214+0000", 2024-09-18T10:15:55.881 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8992999999999998e-05 2024-09-18T10:15:55.881 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.881 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.881 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:55.881 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.856357+0000", 2024-09-18T10:15:55.881 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100142493 2024-09-18T10:15:55.881 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.881 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.881 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:55.881 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:47.114791+0000", 2024-09-18T10:15:55.881 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.25843391799999998 2024-09-18T10:15:55.881 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.882 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.882 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:55.882 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:47.114948+0000", 2024-09-18T10:15:55.882 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00015682700000000001 2024-09-18T10:15:55.882 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.882 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.882 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:15:55.882 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:47.116043+0000", 2024-09-18T10:15:55.882 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0010952309999999999 2024-09-18T10:15:55.882 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.882 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.882 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:15:55.882 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:47.117006+0000", 2024-09-18T10:15:55.882 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00096323899999999996 2024-09-18T10:15:55.883 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.883 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.883 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:15:55.883 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:47.418568+0000", 2024-09-18T10:15:55.883 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.30156222300000002 2024-09-18T10:15:55.883 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.883 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.883 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:55.883 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:47.418788+0000", 2024-09-18T10:15:55.883 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00021923799999999999 2024-09-18T10:15:55.883 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:55.883 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:55.883 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:55.883 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.884 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.884 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(osd.6.22:5 3.2s0 3:47202842:::smithi18925896-22:head [copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-18T10:15:55.884 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:46.756797+0000", 2024-09-18T10:15:55.884 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 9.0211377349999999, 2024-09-18T10:15:55.884 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.35575880999999998, 2024-09-18T10:15:55.884 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:55.884 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:15:55.884 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:55.884 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.884 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:55.884 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.756797+0000", 2024-09-18T10:15:55.884 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:55.884 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.884 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.885 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:55.885 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.756797+0000", 2024-09-18T10:15:55.885 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:55.885 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.885 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.885 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:55.885 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.756795+0000", 2024-09-18T10:15:55.885 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:55.885 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.885 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.885 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:55.885 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.756807+0000", 2024-09-18T10:15:55.885 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1970000000000001e-05 2024-09-18T10:15:55.886 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.886 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.886 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:55.886 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.756837+0000", 2024-09-18T10:15:55.886 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9706999999999999e-05 2024-09-18T10:15:55.886 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.886 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.886 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:55.886 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.756849+0000", 2024-09-18T10:15:55.886 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1904e-05 2024-09-18T10:15:55.886 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.886 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.886 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:55.887 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:47.112101+0000", 2024-09-18T10:15:55.887 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.35525225900000001 2024-09-18T10:15:55.887 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.887 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.887 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:55.887 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:47.112556+0000", 2024-09-18T10:15:55.887 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00045505300000000002 2024-09-18T10:15:55.887 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:55.887 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:55.887 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:55.887 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.887 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.887 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:953 3.2s0 3:46b2871c:::smithi18925896-34:head [read 0~557056,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-18T10:15:55.887 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:52.128760+0000", 2024-09-18T10:15:55.888 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 3.6491745880000002, 2024-09-18T10:15:55.888 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.40564792700000002, 2024-09-18T10:15:55.888 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:15:55.888 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "started", 2024-09-18T10:15:55.888 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:15:55.888 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:15:55.888 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:55.888 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 953 2024-09-18T10:15:55.888 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.889 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:15:55.889 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.889 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:15:55.889 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:52.128760+0000", 2024-09-18T10:15:55.889 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:55.889 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.889 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.889 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:15:55.889 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:52.128760+0000", 2024-09-18T10:15:55.889 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:15:55.889 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.889 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.889 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:15:55.890 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:52.128757+0000", 2024-09-18T10:15:55.890 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999967 2024-09-18T10:15:55.890 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.890 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.890 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:15:55.890 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:52.128774+0000", 2024-09-18T10:15:55.890 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.6912e-05 2024-09-18T10:15:55.890 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.890 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.890 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:15:55.890 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:52.128809+0000", 2024-09-18T10:15:55.890 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.4913000000000003e-05 2024-09-18T10:15:55.890 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.890 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.891 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:15:55.891 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:52.228946+0000", 2024-09-18T10:15:55.891 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100137698 2024-09-18T10:15:55.891 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.891 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.891 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:15:55.891 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:52.229076+0000", 2024-09-18T10:15:55.891 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00012982 2024-09-18T10:15:55.891 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.891 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.891 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:15:55.891 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:52.229281+0000", 2024-09-18T10:15:55.891 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000204592 2024-09-18T10:15:55.892 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:15:55.892 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:15:55.892 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:15:55.892 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:52.534408+0000", 2024-09-18T10:15:55.892 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.30512741500000001 2024-09-18T10:15:55.892 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:55.892 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:55.892 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:55.892 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:15:55.892 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:15:55.892 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-09-18T10:15:55.894 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T10:15:55.894 INFO:tasks.ceph.osd.8.smithi029.stderr:2024-09-18T10:15:55.889+0000 7f2555c27700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 22192) UID: 0 2024-09-18T10:15:55.995 INFO:tasks.ceph.osd.3.smithi189.stderr:2024-09-18T10:15:55.990+0000 7f2440fc7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22396) UID: 0 2024-09-18T10:15:56.012 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-18T10:15:56.012 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [ 2024-09-18T10:15:56.012 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.012 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 100/40 ECSubWriteReply(tid=53, last_complete=100'38, committed=1, applied=1))", 2024-09-18T10:15:56.012 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:55.831180+0000", 2024-09-18T10:15:56.012 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.179906546, 2024-09-18T10:15:56.012 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.17994431899999999, 2024-09-18T10:15:56.012 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:56.012 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "queued for pg", 2024-09-18T10:15:56.012 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:56.013 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.013 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:56.013 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:55.831180+0000", 2024-09-18T10:15:56.013 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:56.013 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.013 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.013 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:56.013 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:55.831180+0000", 2024-09-18T10:15:56.013 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:56.013 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.014 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.014 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:56.014 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:55.831177+0000", 2024-09-18T10:15:56.014 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999971 2024-09-18T10:15:56.014 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.014 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.014 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:56.014 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:55.831194+0000", 2024-09-18T10:15:56.014 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6552999999999999e-05 2024-09-18T10:15:56.014 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.014 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.015 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:56.015 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:55.831225+0000", 2024-09-18T10:15:56.015 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1473000000000002e-05 2024-09-18T10:15:56.015 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.015 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.015 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:56.015 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:55.931322+0000", 2024-09-18T10:15:56.015 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100096437 2024-09-18T10:15:56.015 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:56.015 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:56.015 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:56.015 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.016 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.016 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 99/40 ECSubWriteReply(tid=53, last_complete=100'38, committed=1, applied=1))", 2024-09-18T10:15:56.016 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:55.930971+0000", 2024-09-18T10:15:56.016 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.080115965999999997, 2024-09-18T10:15:56.016 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.080235307000000006, 2024-09-18T10:15:56.016 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:56.016 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "queued for pg", 2024-09-18T10:15:56.016 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:56.016 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.016 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:56.016 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:55.930971+0000", 2024-09-18T10:15:56.017 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:56.017 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.017 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.017 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:56.017 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:55.930971+0000", 2024-09-18T10:15:56.017 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:56.017 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.017 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.017 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:56.017 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:55.930969+0000", 2024-09-18T10:15:56.017 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:56.018 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.018 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.018 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:56.018 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:55.930980+0000", 2024-09-18T10:15:56.018 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0937e-05 2024-09-18T10:15:56.018 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.018 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.018 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:56.018 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:55.931002+0000", 2024-09-18T10:15:56.018 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1929e-05 2024-09-18T10:15:56.018 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.019 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.019 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:56.019 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:55.931018+0000", 2024-09-18T10:15:56.019 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5486e-05 2024-09-18T10:15:56.019 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:56.019 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:56.019 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:56.019 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.019 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.019 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.cs3 100/94 ECSubWrite(tid=247, reqid=osd.5.0:246, at_version=0'0, trim_to=0'0, roll_forward_to=99'152))", 2024-09-18T10:15:56.019 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:55.931480+0000", 2024-09-18T10:15:56.020 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.079606271000000006, 2024-09-18T10:15:56.020 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.079800731, 2024-09-18T10:15:56.020 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:56.020 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:56.020 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:56.020 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.020 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:56.020 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:55.931480+0000", 2024-09-18T10:15:56.020 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:56.020 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.020 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.020 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:56.021 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:55.931480+0000", 2024-09-18T10:15:56.021 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:56.021 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.021 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.021 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:56.021 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:55.931480+0000", 2024-09-18T10:15:56.021 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:56.021 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.021 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.021 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:56.021 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:55.931501+0000", 2024-09-18T10:15:56.021 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1089000000000001e-05 2024-09-18T10:15:56.021 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.022 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.022 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:56.022 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:55.931524+0000", 2024-09-18T10:15:56.022 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3096999999999999e-05 2024-09-18T10:15:56.022 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.022 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.022 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:56.022 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:55.931537+0000", 2024-09-18T10:15:56.022 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2835e-05 2024-09-18T10:15:56.022 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.022 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.023 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:56.023 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:55.931849+0000", 2024-09-18T10:15:56.023 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00031180200000000002 2024-09-18T10:15:56.023 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.023 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.023 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:56.023 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:55.931888+0000", 2024-09-18T10:15:56.023 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9286000000000003e-05 2024-09-18T10:15:56.023 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:56.023 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:56.023 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:56.023 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:56.023 INFO:teuthology.orchestra.run.smithi189.stdout: ], 2024-09-18T10:15:56.024 INFO:teuthology.orchestra.run.smithi189.stdout: "num_ops": 3 2024-09-18T10:15:56.024 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-18T10:15:56.025 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T10:15:56.096 INFO:tasks.ceph.osd.8.smithi029.stderr:2024-09-18T10:15:56.089+0000 7f2555c27700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 22192) UID: 0 2024-09-18T10:15:56.128 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-18T10:15:56.128 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-18T10:15:56.128 INFO:teuthology.orchestra.run.smithi189.stdout: "complaint_time": 30, 2024-09-18T10:15:56.128 INFO:teuthology.orchestra.run.smithi189.stdout: "num_blocked_ops": 0 2024-09-18T10:15:56.129 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-18T10:15:56.138 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T10:15:56.197 INFO:tasks.ceph.osd.8.smithi029.stderr:2024-09-18T10:15:56.189+0000 7f2555c27700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 22192) UID: 0 2024-09-18T10:15:56.232 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-18T10:15:56.232 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 20, 2024-09-18T10:15:56.232 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 600, 2024-09-18T10:15:56.232 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [ 2024-09-18T10:15:56.233 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.233 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 36/32 ECSubWriteReply(tid=5, last_complete=36'4, committed=1, applied=1))", 2024-09-18T10:15:56.233 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:50.303102+0000", 2024-09-18T10:15:56.233 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 65.926113662999995, 2024-09-18T10:15:56.233 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20304619600000001, 2024-09-18T10:15:56.233 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:56.233 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:56.233 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:56.233 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.233 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:56.233 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.303102+0000", 2024-09-18T10:15:56.233 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:56.233 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.234 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.234 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:56.234 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.303102+0000", 2024-09-18T10:15:56.234 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:56.234 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.234 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.234 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:56.234 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.303100+0000", 2024-09-18T10:15:56.234 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:56.234 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.234 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.234 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:56.234 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.303114+0000", 2024-09-18T10:15:56.234 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3866000000000001e-05 2024-09-18T10:15:56.235 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.235 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.235 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:56.235 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.303149+0000", 2024-09-18T10:15:56.235 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.5249000000000001e-05 2024-09-18T10:15:56.235 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.235 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.235 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:56.235 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.303174+0000", 2024-09-18T10:15:56.235 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4785000000000001e-05 2024-09-18T10:15:56.235 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.235 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.235 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:56.235 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.506129+0000", 2024-09-18T10:15:56.236 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20295469199999999 2024-09-18T10:15:56.236 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.236 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.236 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:56.236 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.506148+0000", 2024-09-18T10:15:56.236 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9720000000000001e-05 2024-09-18T10:15:56.236 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:56.236 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:56.236 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:56.236 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.236 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.236 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 36/32 ECSubWriteReply(tid=5, last_complete=36'4, committed=1, applied=1))", 2024-09-18T10:15:56.236 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:50.305623+0000", 2024-09-18T10:15:56.237 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 65.923592690999996, 2024-09-18T10:15:56.237 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40085840299999997, 2024-09-18T10:15:56.237 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:56.237 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:56.237 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:56.237 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.237 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:56.237 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.305623+0000", 2024-09-18T10:15:56.237 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:56.237 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.237 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.237 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:56.237 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.305623+0000", 2024-09-18T10:15:56.238 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:56.238 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.238 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.238 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:56.238 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.305621+0000", 2024-09-18T10:15:56.238 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999976 2024-09-18T10:15:56.238 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.238 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.238 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:56.238 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.305635+0000", 2024-09-18T10:15:56.238 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4283e-05 2024-09-18T10:15:56.238 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.238 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.238 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:56.239 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.305663+0000", 2024-09-18T10:15:56.239 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7475999999999999e-05 2024-09-18T10:15:56.239 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.239 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.239 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:56.239 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.305680+0000", 2024-09-18T10:15:56.239 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7609e-05 2024-09-18T10:15:56.239 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.239 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.239 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:56.239 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.706451+0000", 2024-09-18T10:15:56.239 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40077027399999998 2024-09-18T10:15:56.239 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.239 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.239 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:56.240 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.706481+0000", 2024-09-18T10:15:56.240 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0935000000000001e-05 2024-09-18T10:15:56.240 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:56.240 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:56.240 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:56.240 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.240 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.240 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 36/32 ECSubWriteReply(tid=5, last_complete=36'4, committed=1, applied=1))", 2024-09-18T10:15:56.240 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:50.305688+0000", 2024-09-18T10:15:56.240 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 65.923527327000002, 2024-09-18T10:15:56.240 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40093562900000002, 2024-09-18T10:15:56.240 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:56.240 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:56.240 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:56.241 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.241 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:56.241 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.305688+0000", 2024-09-18T10:15:56.241 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:56.241 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.241 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.241 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:56.241 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.305688+0000", 2024-09-18T10:15:56.241 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:56.241 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.241 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.241 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:56.241 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.305687+0000", 2024-09-18T10:15:56.241 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:56.242 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.242 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.242 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:56.242 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.305698+0000", 2024-09-18T10:15:56.242 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1763000000000001e-05 2024-09-18T10:15:56.242 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.242 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.242 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:56.242 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.305729+0000", 2024-09-18T10:15:56.242 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0502e-05 2024-09-18T10:15:56.242 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.242 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.242 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:56.243 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.305741+0000", 2024-09-18T10:15:56.243 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2363e-05 2024-09-18T10:15:56.243 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.243 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.243 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:56.243 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.706585+0000", 2024-09-18T10:15:56.243 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.400843265 2024-09-18T10:15:56.243 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.243 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.243 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:56.243 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.706624+0000", 2024-09-18T10:15:56.243 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9459000000000003e-05 2024-09-18T10:15:56.243 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:56.244 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:56.244 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:56.244 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.244 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.244 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 36/32 ECSubWriteReply(tid=4, last_complete=36'3, committed=1, applied=1))", 2024-09-18T10:15:56.244 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:50.404544+0000", 2024-09-18T10:15:56.244 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 65.824671812000005, 2024-09-18T10:15:56.244 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.30239659400000002, 2024-09-18T10:15:56.244 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:56.244 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:56.244 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:56.244 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.244 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:56.244 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.404544+0000", 2024-09-18T10:15:56.245 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:56.245 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.245 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.245 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:56.245 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.404544+0000", 2024-09-18T10:15:56.245 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:56.245 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.245 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.245 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:56.245 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.404542+0000", 2024-09-18T10:15:56.245 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:56.245 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.245 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.245 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:56.246 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.404555+0000", 2024-09-18T10:15:56.246 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3096e-05 2024-09-18T10:15:56.246 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.246 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.246 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:56.246 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.404576+0000", 2024-09-18T10:15:56.246 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1165000000000001e-05 2024-09-18T10:15:56.246 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.246 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.246 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:56.246 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.404587+0000", 2024-09-18T10:15:56.246 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0549e-05 2024-09-18T10:15:56.246 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.246 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.247 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:56.247 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.706657+0000", 2024-09-18T10:15:56.247 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.30206971999999999 2024-09-18T10:15:56.247 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.247 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.247 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:56.247 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.706941+0000", 2024-09-18T10:15:56.247 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00028382100000000002 2024-09-18T10:15:56.247 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:56.247 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:56.247 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:56.247 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.247 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.247 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 36/32 ECSubWriteReply(tid=5, last_complete=36'4, committed=1, applied=1))", 2024-09-18T10:15:56.248 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:50.406036+0000", 2024-09-18T10:15:56.248 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 65.823179594999999, 2024-09-18T10:15:56.248 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.30215917399999997, 2024-09-18T10:15:56.248 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:56.248 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:56.248 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:56.248 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.248 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:56.248 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.406036+0000", 2024-09-18T10:15:56.248 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:56.248 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.248 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.248 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:56.248 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.406036+0000", 2024-09-18T10:15:56.249 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:56.249 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.249 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.249 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:56.249 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.406035+0000", 2024-09-18T10:15:56.249 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:56.249 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.249 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.249 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:56.249 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.406046+0000", 2024-09-18T10:15:56.249 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1704e-05 2024-09-18T10:15:56.249 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.249 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.249 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:56.250 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.406065+0000", 2024-09-18T10:15:56.250 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.8448999999999999e-05 2024-09-18T10:15:56.250 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.250 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.250 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:56.250 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.406076+0000", 2024-09-18T10:15:56.250 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1286e-05 2024-09-18T10:15:56.250 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.250 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.250 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:56.250 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.707005+0000", 2024-09-18T10:15:56.250 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.30092866899999998 2024-09-18T10:15:56.250 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.250 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.251 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:56.251 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.708195+0000", 2024-09-18T10:15:56.251 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001190546 2024-09-18T10:15:56.251 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:56.251 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:56.251 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:56.251 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.251 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.251 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.cs3 36/32 ECSubWrite(tid=14, reqid=client.4632.0:92, at_version=0'0, trim_to=0'0, roll_forward_to=36'6))", 2024-09-18T10:15:56.251 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:50.504842+0000", 2024-09-18T10:15:56.251 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 65.724374193000003, 2024-09-18T10:15:56.251 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.202186116, 2024-09-18T10:15:56.251 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:56.251 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:56.252 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:56.252 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.252 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:56.252 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.504842+0000", 2024-09-18T10:15:56.252 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:56.252 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.252 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.252 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:56.252 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.504842+0000", 2024-09-18T10:15:56.252 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:56.252 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.252 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.252 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:56.253 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.504840+0000", 2024-09-18T10:15:56.253 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:56.253 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.253 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.253 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:56.253 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.504868+0000", 2024-09-18T10:15:56.253 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7809000000000001e-05 2024-09-18T10:15:56.253 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.253 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.253 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:56.253 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.504900+0000", 2024-09-18T10:15:56.253 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1921999999999997e-05 2024-09-18T10:15:56.253 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.254 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.254 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:56.254 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.605012+0000", 2024-09-18T10:15:56.254 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10011272 2024-09-18T10:15:56.254 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.254 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.254 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:56.254 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.605115+0000", 2024-09-18T10:15:56.254 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000103003 2024-09-18T10:15:56.254 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.254 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.254 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:56.254 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.605132+0000", 2024-09-18T10:15:56.254 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6359999999999999e-05 2024-09-18T10:15:56.255 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.255 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.255 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:56.255 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.706978+0000", 2024-09-18T10:15:56.255 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.101846672 2024-09-18T10:15:56.255 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.255 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.255 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:56.255 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.707028+0000", 2024-09-18T10:15:56.255 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9443000000000001e-05 2024-09-18T10:15:56.255 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:56.255 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:56.255 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:56.256 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.256 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.256 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:179 3.0s0 3:02f544f0:::smithi18925896-30:head [read 0~491520,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:56.256 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:52.680315+0000", 2024-09-18T10:15:56.256 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 63.548901063999999, 2024-09-18T10:15:56.256 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20435319499999999, 2024-09-18T10:15:56.256 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:56.256 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:15:56.256 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:56.256 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:56.256 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:56.256 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 179 2024-09-18T10:15:56.256 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.256 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:56.257 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.257 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:56.257 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.680315+0000", 2024-09-18T10:15:56.257 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:56.257 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.257 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.257 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:56.257 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.680315+0000", 2024-09-18T10:15:56.257 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:56.257 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.257 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.257 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:56.257 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.680313+0000", 2024-09-18T10:15:56.258 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:56.258 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.258 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.258 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:56.258 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.680322+0000", 2024-09-18T10:15:56.258 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.9299999999999992e-06 2024-09-18T10:15:56.258 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.258 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.258 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:56.258 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.680343+0000", 2024-09-18T10:15:56.258 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0556000000000001e-05 2024-09-18T10:15:56.260 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.260 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.260 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:56.260 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.680373+0000", 2024-09-18T10:15:56.260 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0592000000000001e-05 2024-09-18T10:15:56.260 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.260 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.260 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:56.260 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.681360+0000", 2024-09-18T10:15:56.260 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00098610400000000002 2024-09-18T10:15:56.260 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.261 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.261 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:56.261 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.681624+0000", 2024-09-18T10:15:56.261 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000263992 2024-09-18T10:15:56.261 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.261 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.261 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:56.261 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.884668+0000", 2024-09-18T10:15:56.261 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.203044321 2024-09-18T10:15:56.261 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:56.261 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:56.261 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:56.261 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.261 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.262 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:180 3.0s0 3:02f544f0:::smithi18925896-30:head [read 0~491520,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:56.262 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:52.680394+0000", 2024-09-18T10:15:56.262 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 63.548821986, 2024-09-18T10:15:56.262 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20540486599999999, 2024-09-18T10:15:56.262 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:56.262 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:15:56.262 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:56.262 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:56.262 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:56.262 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 180 2024-09-18T10:15:56.262 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.262 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:56.262 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.263 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:56.263 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.680394+0000", 2024-09-18T10:15:56.263 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:56.263 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.263 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.263 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:56.263 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.680394+0000", 2024-09-18T10:15:56.263 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:56.263 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.263 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.263 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:56.263 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.680393+0000", 2024-09-18T10:15:56.263 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:56.264 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.264 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.264 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:56.264 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.680401+0000", 2024-09-18T10:15:56.264 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.4169999999999992e-06 2024-09-18T10:15:56.264 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.264 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.264 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:56.264 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.680422+0000", 2024-09-18T10:15:56.264 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1186e-05 2024-09-18T10:15:56.264 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.264 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.264 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:56.264 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.680433+0000", 2024-09-18T10:15:56.265 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1103e-05 2024-09-18T10:15:56.265 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.265 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.265 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:56.265 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.783888+0000", 2024-09-18T10:15:56.265 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.103454746 2024-09-18T10:15:56.265 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.265 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.265 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:56.265 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.784067+0000", 2024-09-18T10:15:56.265 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00017865399999999999 2024-09-18T10:15:56.265 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.265 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.265 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:56.266 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.885799+0000", 2024-09-18T10:15:56.266 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10173201699999999 2024-09-18T10:15:56.266 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:56.266 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:56.266 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:56.266 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.266 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.266 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:191 3.0s0 3:0a83d6b3:::smithi18925896-20:2 [read 0~327680,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:56.266 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:52.694194+0000", 2024-09-18T10:15:56.266 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 63.535021546000003, 2024-09-18T10:15:56.266 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.29753027399999998, 2024-09-18T10:15:56.266 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:56.266 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:15:56.267 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:56.267 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:56.267 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:56.267 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 191 2024-09-18T10:15:56.267 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.267 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:56.267 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.267 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:56.267 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.694194+0000", 2024-09-18T10:15:56.267 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:56.267 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.267 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.267 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:56.267 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.694194+0000", 2024-09-18T10:15:56.268 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:56.268 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.268 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.268 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:56.268 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.694192+0000", 2024-09-18T10:15:56.268 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:56.268 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.268 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.268 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:56.269 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.694204+0000", 2024-09-18T10:15:56.269 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2233e-05 2024-09-18T10:15:56.269 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.269 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.269 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:56.269 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.694280+0000", 2024-09-18T10:15:56.269 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.5584e-05 2024-09-18T10:15:56.269 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.269 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.269 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:56.269 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.694292+0000", 2024-09-18T10:15:56.269 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2478e-05 2024-09-18T10:15:56.269 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.269 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.270 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:56.270 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.885869+0000", 2024-09-18T10:15:56.270 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.191576304 2024-09-18T10:15:56.270 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.270 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.270 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:56.270 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.886008+0000", 2024-09-18T10:15:56.270 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00013880300000000001 2024-09-18T10:15:56.270 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.270 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.270 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:56.270 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.991725+0000", 2024-09-18T10:15:56.270 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10571691 2024-09-18T10:15:56.271 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:56.271 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:56.271 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:56.271 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.271 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.271 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:192 3.0s0 3:0a83d6b3:::smithi18925896-20:2 [read 0~327680,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:56.271 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:52.694306+0000", 2024-09-18T10:15:56.271 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 63.534909595999999, 2024-09-18T10:15:56.271 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.41051474799999998, 2024-09-18T10:15:56.271 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:56.271 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:15:56.271 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:56.271 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:56.271 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:56.272 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 192 2024-09-18T10:15:56.272 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.272 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:56.272 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.272 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:56.272 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.694306+0000", 2024-09-18T10:15:56.272 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:56.272 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.272 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.272 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:56.272 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.694306+0000", 2024-09-18T10:15:56.272 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:56.272 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.273 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.273 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:56.273 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.694305+0000", 2024-09-18T10:15:56.273 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:56.273 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.273 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.273 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:56.273 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.694312+0000", 2024-09-18T10:15:56.273 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.5129999999999999e-06 2024-09-18T10:15:56.273 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.273 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.273 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:56.273 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.694328+0000", 2024-09-18T10:15:56.274 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5766000000000001e-05 2024-09-18T10:15:56.274 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.274 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.274 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:56.274 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.694349+0000", 2024-09-18T10:15:56.274 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0990000000000001e-05 2024-09-18T10:15:56.274 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.274 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.274 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:56.274 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.887405+0000", 2024-09-18T10:15:56.274 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.193055791 2024-09-18T10:15:56.274 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.274 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.274 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:56.275 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.887535+0000", 2024-09-18T10:15:56.275 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00013043000000000001 2024-09-18T10:15:56.275 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.275 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.275 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:56.275 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:53.104821+0000", 2024-09-18T10:15:56.275 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.217285486 2024-09-18T10:15:56.275 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:56.275 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:56.275 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:56.275 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.275 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.275 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:193 3.0s0 3:0a83d6b3:::smithi18925896-20:2 [read 0~327680,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:56.275 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:52.694359+0000", 2024-09-18T10:15:56.276 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 63.534856415, 2024-09-18T10:15:56.276 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.411282918, 2024-09-18T10:15:56.276 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:56.276 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:15:56.276 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:56.276 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:56.276 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:56.276 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 193 2024-09-18T10:15:56.276 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.276 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:56.276 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.276 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:56.276 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.694359+0000", 2024-09-18T10:15:56.276 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:56.277 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.277 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.277 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:56.277 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.694359+0000", 2024-09-18T10:15:56.277 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:56.277 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.277 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.277 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:56.277 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.694358+0000", 2024-09-18T10:15:56.277 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:15:56.277 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.277 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.277 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:56.278 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.694364+0000", 2024-09-18T10:15:56.278 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8089999999999998e-06 2024-09-18T10:15:56.278 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.278 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.278 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:56.278 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.694381+0000", 2024-09-18T10:15:56.278 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6949e-05 2024-09-18T10:15:56.278 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.278 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.278 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:56.278 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.694388+0000", 2024-09-18T10:15:56.278 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.8079999999999999e-06 2024-09-18T10:15:56.278 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.278 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.279 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:56.279 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.988184+0000", 2024-09-18T10:15:56.279 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.29379642900000003 2024-09-18T10:15:56.279 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.279 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.279 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:56.279 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.988399+0000", 2024-09-18T10:15:56.279 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000214537 2024-09-18T10:15:56.279 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.279 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.279 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:56.279 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:53.105642+0000", 2024-09-18T10:15:56.279 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.11724332699999999 2024-09-18T10:15:56.279 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:56.279 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:56.280 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:56.280 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.280 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.280 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.10s0 45/40 ECSubWriteReply(tid=18, last_complete=45'6, committed=1, applied=1))", 2024-09-18T10:15:56.280 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:01.742819+0000", 2024-09-18T10:15:56.280 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 54.486396812000002, 2024-09-18T10:15:56.280 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.30317059800000001, 2024-09-18T10:15:56.280 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:56.280 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:56.280 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:56.280 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.280 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:56.280 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.742819+0000", 2024-09-18T10:15:56.281 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:56.281 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.281 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.281 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:56.281 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.742819+0000", 2024-09-18T10:15:56.281 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:56.281 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.281 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.281 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:56.281 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.742817+0000", 2024-09-18T10:15:56.281 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:56.281 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.281 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.282 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:56.282 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.742832+0000", 2024-09-18T10:15:56.282 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.466e-05 2024-09-18T10:15:56.282 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.282 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.282 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:56.282 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.742866+0000", 2024-09-18T10:15:56.282 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4242000000000002e-05 2024-09-18T10:15:56.282 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.282 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.282 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:56.282 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.843033+0000", 2024-09-18T10:15:56.282 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100166479 2024-09-18T10:15:56.282 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.283 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.283 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:56.283 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.045959+0000", 2024-09-18T10:15:56.283 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.202926 2024-09-18T10:15:56.283 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.283 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.283 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:56.283 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.045990+0000", 2024-09-18T10:15:56.283 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0889999999999997e-05 2024-09-18T10:15:56.283 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:56.283 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:56.283 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:56.283 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.283 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.284 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.10s0 45/40 ECSubWriteReply(tid=18, last_complete=45'6, committed=1, applied=1))", 2024-09-18T10:15:56.284 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:01.843113+0000", 2024-09-18T10:15:56.284 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 54.386102966000003, 2024-09-18T10:15:56.284 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.203080129, 2024-09-18T10:15:56.284 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:56.284 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:56.284 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:56.284 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.284 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:56.284 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.843113+0000", 2024-09-18T10:15:56.284 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:56.284 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.284 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.285 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:56.285 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.843113+0000", 2024-09-18T10:15:56.285 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:56.285 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.285 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.285 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:56.285 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.843111+0000", 2024-09-18T10:15:56.285 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:56.285 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.285 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.285 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:56.285 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.843122+0000", 2024-09-18T10:15:56.285 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0199e-05 2024-09-18T10:15:56.285 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.286 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.286 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:56.286 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.843147+0000", 2024-09-18T10:15:56.286 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5113000000000002e-05 2024-09-18T10:15:56.286 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.286 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.286 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:56.286 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.943289+0000", 2024-09-18T10:15:56.286 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100141813 2024-09-18T10:15:56.286 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.286 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.286 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:56.286 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.046146+0000", 2024-09-18T10:15:56.286 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.102857768 2024-09-18T10:15:56.287 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.287 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.287 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:56.287 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.046193+0000", 2024-09-18T10:15:56.287 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.6566000000000001e-05 2024-09-18T10:15:56.287 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:56.287 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:56.287 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:56.287 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.287 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.287 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.1s5 56/40 ECSubWrite(tid=102, reqid=client.4632.0:437, at_version=0'0, trim_to=0'0, roll_forward_to=56'16))", 2024-09-18T10:15:56.287 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:12.541097+0000", 2024-09-18T10:15:56.287 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 43.688118604000003, 2024-09-18T10:15:56.287 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20186336399999999, 2024-09-18T10:15:56.288 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:56.288 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:56.288 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:56.288 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.288 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:56.288 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:12.541097+0000", 2024-09-18T10:15:56.288 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:56.288 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.288 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.288 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:56.288 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:12.541097+0000", 2024-09-18T10:15:56.288 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:56.288 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.288 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.289 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:56.289 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:12.541095+0000", 2024-09-18T10:15:56.289 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:56.289 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.289 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.289 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:56.289 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:12.541147+0000", 2024-09-18T10:15:56.289 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.2203000000000001e-05 2024-09-18T10:15:56.289 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.289 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.290 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:56.290 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:12.541184+0000", 2024-09-18T10:15:56.290 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6495000000000002e-05 2024-09-18T10:15:56.290 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.290 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.290 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:56.290 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:12.641359+0000", 2024-09-18T10:15:56.290 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100175135 2024-09-18T10:15:56.290 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.290 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.290 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:56.290 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:12.741675+0000", 2024-09-18T10:15:56.290 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100316457 2024-09-18T10:15:56.290 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.291 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.291 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:56.291 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:12.741693+0000", 2024-09-18T10:15:56.291 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7269e-05 2024-09-18T10:15:56.291 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.291 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.291 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:56.291 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:12.742894+0000", 2024-09-18T10:15:56.291 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0012013639999999999 2024-09-18T10:15:56.291 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.291 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.291 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:56.291 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:12.742961+0000", 2024-09-18T10:15:56.291 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.6575000000000005e-05 2024-09-18T10:15:56.292 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:56.292 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:56.292 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:56.292 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.292 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.292 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.1s5 57/40 ECSubWrite(tid=111, reqid=osd.4.0:111, at_version=57'18, trim_to=0'0, roll_forward_to=56'16))", 2024-09-18T10:15:56.292 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:13.054749+0000", 2024-09-18T10:15:56.292 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 43.174466543000001, 2024-09-18T10:15:56.292 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.201940535, 2024-09-18T10:15:56.292 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:56.292 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:56.292 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:56.292 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.293 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:56.293 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.054749+0000", 2024-09-18T10:15:56.293 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:56.293 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.293 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.293 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:56.293 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.054749+0000", 2024-09-18T10:15:56.293 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:56.293 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.293 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.293 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:56.293 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.054749+0000", 2024-09-18T10:15:56.293 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:56.294 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.294 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.294 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:56.294 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.054785+0000", 2024-09-18T10:15:56.294 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6480999999999999e-05 2024-09-18T10:15:56.294 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.294 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.294 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:56.294 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.054808+0000", 2024-09-18T10:15:56.294 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.2351e-05 2024-09-18T10:15:56.294 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.294 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.294 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:56.294 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.154914+0000", 2024-09-18T10:15:56.295 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100106862 2024-09-18T10:15:56.295 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.295 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.295 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:56.295 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.155065+0000", 2024-09-18T10:15:56.295 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000150762 2024-09-18T10:15:56.295 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.295 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.295 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:56.295 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.155083+0000", 2024-09-18T10:15:56.296 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.8207e-05 2024-09-18T10:15:56.296 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.296 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.296 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:56.296 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.256244+0000", 2024-09-18T10:15:56.296 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10116046200000001 2024-09-18T10:15:56.296 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.296 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.296 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:56.296 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.256690+0000", 2024-09-18T10:15:56.296 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00044597000000000001 2024-09-18T10:15:56.296 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:56.296 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:56.297 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:56.297 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.297 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.297 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:511 3.0s0 3:02f544f0:::smithi18925896-30:head [stat,delete] snapc 1a=[1a,19,18,16,12,11,10,f,e] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-18T10:15:56.297 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:18.899317+0000", 2024-09-18T10:15:56.297 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 37.329899165999997, 2024-09-18T10:15:56.297 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.21051439599999999, 2024-09-18T10:15:56.297 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:56.297 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:56.297 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:56.297 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:56.297 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:56.297 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 511 2024-09-18T10:15:56.297 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.298 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:56.298 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.298 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:56.298 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.899317+0000", 2024-09-18T10:15:56.298 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:56.298 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.298 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.298 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:56.298 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.899317+0000", 2024-09-18T10:15:56.298 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:56.298 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.298 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.298 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:56.298 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.899314+0000", 2024-09-18T10:15:56.299 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999971 2024-09-18T10:15:56.299 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.299 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.299 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:56.299 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.899329+0000", 2024-09-18T10:15:56.299 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5160000000000001e-05 2024-09-18T10:15:56.299 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.299 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.299 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:56.299 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.899364+0000", 2024-09-18T10:15:56.299 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.5456000000000001e-05 2024-09-18T10:15:56.299 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.299 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.300 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:56.300 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.899385+0000", 2024-09-18T10:15:56.300 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0279e-05 2024-09-18T10:15:56.300 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.300 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.300 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:56.300 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.899497+0000", 2024-09-18T10:15:56.300 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000111773 2024-09-18T10:15:56.300 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.300 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.300 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "op must wait for map", 2024-09-18T10:15:56.300 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.899508+0000", 2024-09-18T10:15:56.300 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1177e-05 2024-09-18T10:15:56.300 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.301 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.301 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:56.301 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.903698+0000", 2024-09-18T10:15:56.301 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0041902850000000002 2024-09-18T10:15:56.301 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.301 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.301 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:56.301 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.903878+0000", 2024-09-18T10:15:56.301 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00018009499999999999 2024-09-18T10:15:56.301 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.301 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.301 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:56.301 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.905286+0000", 2024-09-18T10:15:56.302 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0014077720000000001 2024-09-18T10:15:56.302 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.302 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.302 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:56.302 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.003449+0000", 2024-09-18T10:15:56.302 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.098163081999999999 2024-09-18T10:15:56.302 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.302 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.302 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:56.302 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.109709+0000", 2024-09-18T10:15:56.302 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10625963400000001 2024-09-18T10:15:56.302 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.302 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.303 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:56.303 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.109831+0000", 2024-09-18T10:15:56.303 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00012246699999999999 2024-09-18T10:15:56.303 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:56.303 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:56.303 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:56.303 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.303 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.303 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.19s6 72/40 ECSubWrite(tid=176, reqid=osd.7.0:174, at_version=0'0, trim_to=0'0, roll_forward_to=71'100))", 2024-09-18T10:15:56.303 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:28.330736+0000", 2024-09-18T10:15:56.303 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 27.898479699999999, 2024-09-18T10:15:56.303 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20302320200000001, 2024-09-18T10:15:56.303 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:56.304 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:56.304 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:56.304 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.304 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:56.304 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.330736+0000", 2024-09-18T10:15:56.304 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:56.304 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.304 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.304 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:56.304 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.330736+0000", 2024-09-18T10:15:56.304 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:56.304 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.304 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.304 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:56.305 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.330735+0000", 2024-09-18T10:15:56.305 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:56.305 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.305 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.305 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:56.305 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.330759+0000", 2024-09-18T10:15:56.305 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4549000000000002e-05 2024-09-18T10:15:56.305 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.305 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.305 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:56.305 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.330816+0000", 2024-09-18T10:15:56.305 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.6693000000000001e-05 2024-09-18T10:15:56.305 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.306 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.306 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:56.306 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.430957+0000", 2024-09-18T10:15:56.306 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10014150199999999 2024-09-18T10:15:56.306 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.306 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.306 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:56.306 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.531236+0000", 2024-09-18T10:15:56.306 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100278201 2024-09-18T10:15:56.306 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.306 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.306 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:56.306 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.531263+0000", 2024-09-18T10:15:56.306 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7172999999999999e-05 2024-09-18T10:15:56.307 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.307 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.307 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:56.307 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.533701+0000", 2024-09-18T10:15:56.307 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0024382760000000001 2024-09-18T10:15:56.307 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.307 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.307 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:56.307 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.533759+0000", 2024-09-18T10:15:56.307 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8267999999999998e-05 2024-09-18T10:15:56.307 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:56.307 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:56.307 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:56.308 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.308 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.308 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.2s1 89/40 ECSubWrite(tid=132, reqid=osd.14.0:132, at_version=89'92, trim_to=0'0, roll_forward_to=88'90))", 2024-09-18T10:15:56.308 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:44.791495+0000", 2024-09-18T10:15:56.308 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 11.437720591, 2024-09-18T10:15:56.308 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20177673600000001, 2024-09-18T10:15:56.308 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:56.308 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:56.308 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:56.308 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.308 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:56.308 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:44.791495+0000", 2024-09-18T10:15:56.308 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:56.308 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.309 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.309 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:56.309 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:44.791495+0000", 2024-09-18T10:15:56.309 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:56.309 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.309 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.309 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:56.309 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:44.791495+0000", 2024-09-18T10:15:56.309 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:56.309 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.309 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.309 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:56.309 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:44.791561+0000", 2024-09-18T10:15:56.309 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.6339000000000003e-05 2024-09-18T10:15:56.310 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.310 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.310 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:56.310 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:44.791584+0000", 2024-09-18T10:15:56.310 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3405999999999999e-05 2024-09-18T10:15:56.310 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.310 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.310 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:56.310 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:44.791599+0000", 2024-09-18T10:15:56.310 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.447e-05 2024-09-18T10:15:56.310 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.310 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.310 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:56.310 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:44.891786+0000", 2024-09-18T10:15:56.311 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10018745799999999 2024-09-18T10:15:56.311 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.311 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.311 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:56.311 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:44.891805+0000", 2024-09-18T10:15:56.311 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.8522999999999999e-05 2024-09-18T10:15:56.311 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.311 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.311 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:56.311 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:44.993190+0000", 2024-09-18T10:15:56.311 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.101385559 2024-09-18T10:15:56.311 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.311 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.311 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:56.312 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:44.993272+0000", 2024-09-18T10:15:56.312 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.1524000000000001e-05 2024-09-18T10:15:56.312 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:56.312 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:56.312 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:56.312 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.312 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.312 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.2s1 100/40 ECSubWrite(tid=175, reqid=osd.14.0:175, at_version=100'117, trim_to=0'0, roll_forward_to=99'115))", 2024-09-18T10:15:56.312 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:55.723156+0000", 2024-09-18T10:15:56.312 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.50605962000000004, 2024-09-18T10:15:56.312 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20280843200000001, 2024-09-18T10:15:56.312 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:56.312 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:56.312 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:56.313 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.313 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:56.313 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:55.723156+0000", 2024-09-18T10:15:56.313 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:56.313 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.313 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.313 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:56.313 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:55.723156+0000", 2024-09-18T10:15:56.313 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:56.313 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.313 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.313 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:56.313 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:55.723156+0000", 2024-09-18T10:15:56.313 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:56.314 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.314 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.314 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:56.314 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:55.723215+0000", 2024-09-18T10:15:56.314 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9318999999999999e-05 2024-09-18T10:15:56.314 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.314 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.314 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:56.314 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:55.723243+0000", 2024-09-18T10:15:56.314 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.8506000000000001e-05 2024-09-18T10:15:56.314 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.314 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.314 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:56.314 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:55.823360+0000", 2024-09-18T10:15:56.315 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100116356 2024-09-18T10:15:56.315 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.315 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.315 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:56.315 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:55.923621+0000", 2024-09-18T10:15:56.315 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10026093599999999 2024-09-18T10:15:56.315 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.315 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.315 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:56.315 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:55.923673+0000", 2024-09-18T10:15:56.315 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.2445999999999999e-05 2024-09-18T10:15:56.315 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.315 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.315 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:56.316 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:55.925909+0000", 2024-09-18T10:15:56.316 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0022360869999999999 2024-09-18T10:15:56.316 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.316 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.316 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:56.316 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:55.925965+0000", 2024-09-18T10:15:56.316 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.5439000000000002e-05 2024-09-18T10:15:56.316 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:56.316 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:56.316 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:56.316 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.316 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.316 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 100/40 ECSubWriteReply(tid=53, last_complete=100'38, committed=1, applied=1))", 2024-09-18T10:15:56.316 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:55.831180+0000", 2024-09-18T10:15:56.317 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.398035796, 2024-09-18T10:15:56.317 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20294891000000001, 2024-09-18T10:15:56.317 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:56.317 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:56.317 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:56.317 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.317 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:56.317 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:55.831180+0000", 2024-09-18T10:15:56.317 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:56.317 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.317 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.317 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:56.317 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:55.831180+0000", 2024-09-18T10:15:56.317 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:56.318 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.318 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.318 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:56.318 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:55.831177+0000", 2024-09-18T10:15:56.318 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999971 2024-09-18T10:15:56.318 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.318 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.318 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:56.318 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:55.831194+0000", 2024-09-18T10:15:56.318 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6552999999999999e-05 2024-09-18T10:15:56.318 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.318 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.318 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:56.318 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:55.831225+0000", 2024-09-18T10:15:56.318 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1473000000000002e-05 2024-09-18T10:15:56.319 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.319 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.319 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:56.319 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:55.931322+0000", 2024-09-18T10:15:56.319 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100096437 2024-09-18T10:15:56.319 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.319 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.319 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:56.319 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:56.032873+0000", 2024-09-18T10:15:56.319 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.101551321 2024-09-18T10:15:56.319 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.319 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.319 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:56.319 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:56.034129+0000", 2024-09-18T10:15:56.320 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0012560430000000001 2024-09-18T10:15:56.320 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:56.320 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:56.320 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:56.320 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:56.320 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:56.320 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-18T10:15:56.322 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T10:15:56.322 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-18T10:15:56.315+0000 7f14c6a1a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 22138) UID: 0 2024-09-18T10:15:56.422 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-18T10:15:56.419+0000 7f578c52f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22134) UID: 0 2024-09-18T10:15:56.431 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-18T10:15:56.431 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [ 2024-09-18T10:15:56.431 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.431 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDPGRecoveryDeleteReply(3.9s0 e100,94 3:93224e57:::smithi18925896-10:1a,93'97)", 2024-09-18T10:15:56.431 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:56.344556+0000", 2024-09-18T10:15:56.431 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.085832602999999993, 2024-09-18T10:15:56.431 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.085874224999999998, 2024-09-18T10:15:56.432 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:56.432 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "queued for pg", 2024-09-18T10:15:56.432 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:56.432 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.432 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:56.432 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:56.344556+0000", 2024-09-18T10:15:56.432 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:56.432 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.432 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.432 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:56.432 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:56.344556+0000", 2024-09-18T10:15:56.432 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:56.432 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.433 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.433 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:56.433 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:56.344555+0000", 2024-09-18T10:15:56.433 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:56.433 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.433 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.433 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:56.433 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:56.344569+0000", 2024-09-18T10:15:56.433 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.47e-05 2024-09-18T10:15:56.433 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.433 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.433 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:56.433 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:56.344610+0000", 2024-09-18T10:15:56.434 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.0661999999999998e-05 2024-09-18T10:15:56.434 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.434 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.434 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:56.434 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:56.344625+0000", 2024-09-18T10:15:56.434 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4966999999999999e-05 2024-09-18T10:15:56.434 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:56.434 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:56.434 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:56.434 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.434 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.434 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpRead(3.19s0 100/95 ECSubRead(tid=124, to_read={3:9bcb6eef:::smithi18925896-11:head=0,262144,0}, subchunks={3:9bcb6eef:::smithi18925896-11:head=[0,1]}, attrs_to_read=3:9bcb6eef:::smithi18925896-11:head))", 2024-09-18T10:15:56.434 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:56.376880+0000", 2024-09-18T10:15:56.435 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.053508600000000003, 2024-09-18T10:15:56.435 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.053642942999999998, 2024-09-18T10:15:56.435 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:56.435 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "queued for pg", 2024-09-18T10:15:56.435 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:56.435 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.435 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:56.435 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:56.376880+0000", 2024-09-18T10:15:56.435 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:56.435 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.435 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.435 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:56.435 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:56.376880+0000", 2024-09-18T10:15:56.436 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:56.436 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.436 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.436 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:56.436 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:56.376878+0000", 2024-09-18T10:15:56.436 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:56.436 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.436 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.436 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:56.436 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:56.376902+0000", 2024-09-18T10:15:56.436 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4096000000000001e-05 2024-09-18T10:15:56.436 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.436 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.437 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:56.437 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:56.376942+0000", 2024-09-18T10:15:56.437 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.0025999999999999e-05 2024-09-18T10:15:56.437 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.437 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.437 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:56.437 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:56.376958+0000", 2024-09-18T10:15:56.437 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.615e-05 2024-09-18T10:15:56.437 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:56.437 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:56.437 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:56.437 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:56.437 INFO:teuthology.orchestra.run.smithi189.stdout: ], 2024-09-18T10:15:56.438 INFO:teuthology.orchestra.run.smithi189.stdout: "num_ops": 2 2024-09-18T10:15:56.438 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-18T10:15:56.441 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T10:15:56.524 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-18T10:15:56.519+0000 7f00046a8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 22136) UID: 0 2024-09-18T10:15:56.536 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-18T10:15:56.536 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-18T10:15:56.537 INFO:teuthology.orchestra.run.smithi189.stdout: "complaint_time": 30, 2024-09-18T10:15:56.537 INFO:teuthology.orchestra.run.smithi189.stdout: "num_blocked_ops": 0 2024-09-18T10:15:56.537 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-18T10:15:56.545 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T10:15:56.624 INFO:tasks.ceph.osd.3.smithi189.stderr:2024-09-18T10:15:56.618+0000 7f2440fc7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22396) UID: 0 2024-09-18T10:15:56.650 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-18T10:15:56.650 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 20, 2024-09-18T10:15:56.650 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 600, 2024-09-18T10:15:56.650 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [ 2024-09-18T10:15:56.650 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.650 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:114 3.9s0 3:9a1a774c:::smithi18925896-38:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:56.650 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:50.816650+0000", 2024-09-18T10:15:56.651 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 65.829491759999996, 2024-09-18T10:15:56.651 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.41853457900000002, 2024-09-18T10:15:56.651 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:56.651 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:15:56.651 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:56.651 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:56.651 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:56.651 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 114 2024-09-18T10:15:56.651 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.651 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:56.651 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.651 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:56.651 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.816650+0000", 2024-09-18T10:15:56.652 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:56.652 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.652 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.652 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:56.652 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.816650+0000", 2024-09-18T10:15:56.652 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:56.652 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.652 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.652 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:56.652 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.816648+0000", 2024-09-18T10:15:56.652 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:56.652 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.653 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.653 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:56.653 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.816661+0000", 2024-09-18T10:15:56.653 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2619000000000001e-05 2024-09-18T10:15:56.653 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.653 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.653 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:56.653 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.816696+0000", 2024-09-18T10:15:56.653 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4885999999999998e-05 2024-09-18T10:15:56.653 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.653 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.653 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:56.654 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.816708+0000", 2024-09-18T10:15:56.654 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2843000000000001e-05 2024-09-18T10:15:56.654 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.654 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.654 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:56.654 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.818228+0000", 2024-09-18T10:15:56.654 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001519882 2024-09-18T10:15:56.654 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.654 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.654 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for rw locks", 2024-09-18T10:15:56.654 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.818557+0000", 2024-09-18T10:15:56.654 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00032873700000000001 2024-09-18T10:15:56.654 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.655 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.655 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:56.655 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.925488+0000", 2024-09-18T10:15:56.655 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.106930502 2024-09-18T10:15:56.655 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.655 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.655 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:56.655 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.925563+0000", 2024-09-18T10:15:56.655 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.5386e-05 2024-09-18T10:15:56.655 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.655 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.655 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:56.655 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.235185+0000", 2024-09-18T10:15:56.655 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.30962182399999999 2024-09-18T10:15:56.656 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:56.656 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:56.656 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:56.656 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.656 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.656 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:138 3.9s0 3:9ab7c4c0:::smithi18925896-46:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:56.656 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:50.921054+0000", 2024-09-18T10:15:56.656 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 65.725087462000005, 2024-09-18T10:15:56.656 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.61288337900000001, 2024-09-18T10:15:56.656 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:56.656 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:15:56.656 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:56.656 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:56.657 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:56.657 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 138 2024-09-18T10:15:56.657 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.657 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:56.657 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.657 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:56.657 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.921054+0000", 2024-09-18T10:15:56.657 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:56.657 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.657 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.657 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:56.657 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.921054+0000", 2024-09-18T10:15:56.657 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:56.658 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.658 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.658 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:56.658 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.921054+0000", 2024-09-18T10:15:56.658 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:56.658 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.658 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.658 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:56.658 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.921057+0000", 2024-09-18T10:15:56.658 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1829999999999998e-06 2024-09-18T10:15:56.658 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.658 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.658 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:56.659 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.921090+0000", 2024-09-18T10:15:56.659 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3281999999999997e-05 2024-09-18T10:15:56.659 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.659 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.659 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:56.659 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.921096+0000", 2024-09-18T10:15:56.659 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.0519999999999998e-06 2024-09-18T10:15:56.659 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.659 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.659 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:56.659 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.127704+0000", 2024-09-18T10:15:56.659 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20660720499999999 2024-09-18T10:15:56.659 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.660 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.660 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for rw locks", 2024-09-18T10:15:56.660 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.127851+0000", 2024-09-18T10:15:56.660 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00014710999999999999 2024-09-18T10:15:56.660 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.660 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.660 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:56.660 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.234300+0000", 2024-09-18T10:15:56.660 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10644893900000001 2024-09-18T10:15:56.660 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.660 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.660 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:56.660 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.234462+0000", 2024-09-18T10:15:56.660 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00016218999999999999 2024-09-18T10:15:56.661 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.661 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.661 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:56.661 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.533938+0000", 2024-09-18T10:15:56.661 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.29947596999999998 2024-09-18T10:15:56.661 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:56.661 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:56.661 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:56.661 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.661 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.661 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:143 3.9s0 3:9221fff5:::smithi18925896-48:head [append 0~376832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:56.661 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:50.932038+0000", 2024-09-18T10:15:56.661 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 65.714103860999998, 2024-09-18T10:15:56.662 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.60109878000000005, 2024-09-18T10:15:56.662 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:56.662 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:56.662 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:56.662 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:56.662 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:56.662 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 143 2024-09-18T10:15:56.662 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.662 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:56.662 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.662 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:56.662 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932038+0000", 2024-09-18T10:15:56.662 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:56.663 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.663 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.663 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:56.663 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932038+0000", 2024-09-18T10:15:56.663 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:56.663 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.663 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.663 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:56.663 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932037+0000", 2024-09-18T10:15:56.663 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:15:56.663 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.663 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.663 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:56.663 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932309+0000", 2024-09-18T10:15:56.664 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000272086 2024-09-18T10:15:56.664 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.664 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.664 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:56.664 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932323+0000", 2024-09-18T10:15:56.664 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4354e-05 2024-09-18T10:15:56.664 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.664 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.664 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:56.665 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932329+0000", 2024-09-18T10:15:56.665 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.6840000000000001e-06 2024-09-18T10:15:56.665 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.665 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.665 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:56.665 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.130012+0000", 2024-09-18T10:15:56.665 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.19768277100000001 2024-09-18T10:15:56.665 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.665 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.665 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:56.665 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.130068+0000", 2024-09-18T10:15:56.665 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.6158e-05 2024-09-18T10:15:56.665 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.666 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.666 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:56.666 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.130809+0000", 2024-09-18T10:15:56.666 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00074078700000000002 2024-09-18T10:15:56.666 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.666 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.666 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:56.666 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.132840+0000", 2024-09-18T10:15:56.666 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0020308879999999998 2024-09-18T10:15:56.666 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.666 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.666 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:56.666 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.533025+0000", 2024-09-18T10:15:56.667 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.400185491 2024-09-18T10:15:56.667 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.667 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.667 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:56.667 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.533137+0000", 2024-09-18T10:15:56.667 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000111727 2024-09-18T10:15:56.667 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:56.667 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:56.667 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:56.667 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.667 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.667 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:144 3.9s0 3:9221fff5:::smithi18925896-48:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:56.668 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:50.932336+0000", 2024-09-18T10:15:56.668 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 65.713806224999999, 2024-09-18T10:15:56.668 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.60198766699999995, 2024-09-18T10:15:56.668 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:56.668 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:56.668 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:56.668 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:56.668 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:56.668 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 144 2024-09-18T10:15:56.668 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.668 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:56.668 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.668 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:56.669 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932336+0000", 2024-09-18T10:15:56.669 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:56.669 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.669 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.672 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:56.672 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932336+0000", 2024-09-18T10:15:56.672 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:56.672 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.672 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.672 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:56.672 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932335+0000", 2024-09-18T10:15:56.672 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:15:56.673 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.673 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.673 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:56.673 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932339+0000", 2024-09-18T10:15:56.673 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8539999999999999e-06 2024-09-18T10:15:56.673 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.673 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.673 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:56.673 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932347+0000", 2024-09-18T10:15:56.673 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.2910000000000007e-06 2024-09-18T10:15:56.673 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.673 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.673 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:56.674 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932351+0000", 2024-09-18T10:15:56.674 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.5420000000000001e-06 2024-09-18T10:15:56.674 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.674 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.674 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:56.674 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.131380+0000", 2024-09-18T10:15:56.674 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.199029236 2024-09-18T10:15:56.674 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.674 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.674 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:56.674 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.131436+0000", 2024-09-18T10:15:56.674 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.6098999999999999e-05 2024-09-18T10:15:56.674 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.675 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.675 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:56.675 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.131777+0000", 2024-09-18T10:15:56.675 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00034058800000000001 2024-09-18T10:15:56.675 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.675 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.675 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:56.675 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.133245+0000", 2024-09-18T10:15:56.675 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0014682300000000001 2024-09-18T10:15:56.675 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.675 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.675 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:56.675 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.534187+0000", 2024-09-18T10:15:56.675 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40094253299999999 2024-09-18T10:15:56.676 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.676 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.676 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:56.676 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.534323+0000", 2024-09-18T10:15:56.676 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00013606299999999999 2024-09-18T10:15:56.676 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:56.676 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:56.676 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:56.676 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.676 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.676 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:145 3.9s0 3:9221fff5:::smithi18925896-48:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:56.676 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:50.932355+0000", 2024-09-18T10:15:56.677 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 65.713786741000007, 2024-09-18T10:15:56.677 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.80502307200000001, 2024-09-18T10:15:56.677 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:56.677 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:15:56.677 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:56.677 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:56.677 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:56.677 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 145 2024-09-18T10:15:56.677 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.677 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:56.677 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.677 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:56.677 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932355+0000", 2024-09-18T10:15:56.677 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:56.678 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.678 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.678 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:56.678 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932355+0000", 2024-09-18T10:15:56.678 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:56.678 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.678 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.678 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:56.678 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932355+0000", 2024-09-18T10:15:56.678 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:56.678 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.678 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.678 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:56.679 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932358+0000", 2024-09-18T10:15:56.679 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4369999999999999e-06 2024-09-18T10:15:56.679 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.679 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.679 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:56.679 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932366+0000", 2024-09-18T10:15:56.679 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.3010000000000007e-06 2024-09-18T10:15:56.679 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.679 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.679 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:56.679 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932370+0000", 2024-09-18T10:15:56.679 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.5619999999999999e-06 2024-09-18T10:15:56.679 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.680 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.680 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:56.680 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.132210+0000", 2024-09-18T10:15:56.680 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.199840145 2024-09-18T10:15:56.680 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.680 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.680 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for rw locks", 2024-09-18T10:15:56.680 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.132312+0000", 2024-09-18T10:15:56.680 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000102335 2024-09-18T10:15:56.680 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.680 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.680 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:56.681 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.535310+0000", 2024-09-18T10:15:56.681 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40299779000000002 2024-09-18T10:15:56.681 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.681 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.681 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:56.681 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.535541+0000", 2024-09-18T10:15:56.681 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00023038299999999999 2024-09-18T10:15:56.681 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.681 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.681 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:56.681 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.737378+0000", 2024-09-18T10:15:56.681 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.201837604 2024-09-18T10:15:56.681 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:56.681 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:56.682 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:56.682 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.682 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.682 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:174 3.9s0 3:9fc2eacb:::smithi18925896-17:head [read 0~278528,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:56.682 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:51.836672+0000", 2024-09-18T10:15:56.682 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 64.809469485999998, 2024-09-18T10:15:56.682 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.407996215, 2024-09-18T10:15:56.682 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:56.682 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:15:56.682 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:56.682 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:56.682 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:56.682 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 174 2024-09-18T10:15:56.682 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.683 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:56.683 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.683 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:56.683 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.836672+0000", 2024-09-18T10:15:56.683 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:56.683 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.683 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.683 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:56.683 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.836672+0000", 2024-09-18T10:15:56.683 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:56.683 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.683 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.683 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:56.684 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.836671+0000", 2024-09-18T10:15:56.684 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:56.684 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.684 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.684 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:56.684 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.836680+0000", 2024-09-18T10:15:56.684 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.5830000000000007e-06 2024-09-18T10:15:56.684 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.684 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.684 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:56.684 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.836696+0000", 2024-09-18T10:15:56.684 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6823000000000001e-05 2024-09-18T10:15:56.685 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.685 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.685 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:56.685 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.836710+0000", 2024-09-18T10:15:56.685 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.325e-05 2024-09-18T10:15:56.685 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.685 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.685 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:56.685 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.142312+0000", 2024-09-18T10:15:56.685 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.30560266400000002 2024-09-18T10:15:56.685 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.685 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.685 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:56.686 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.142549+0000", 2024-09-18T10:15:56.686 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00023630299999999999 2024-09-18T10:15:56.686 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.686 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.686 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:56.686 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.244669+0000", 2024-09-18T10:15:56.686 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10211993599999999 2024-09-18T10:15:56.686 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:56.686 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:56.686 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:56.686 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.686 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.686 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:202 3.9s0 3:9221fff5:::smithi18925896-48:head [copy-from ver 2] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:56.687 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:52.793605+0000", 2024-09-18T10:15:56.687 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 63.852536891, 2024-09-18T10:15:56.687 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.61381690899999997, 2024-09-18T10:15:56.687 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:56.687 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:56.687 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:56.687 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:56.687 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:56.687 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 202 2024-09-18T10:15:56.687 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.687 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:56.687 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.688 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:56.688 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.793605+0000", 2024-09-18T10:15:56.688 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:56.688 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.688 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.688 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:56.688 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.793605+0000", 2024-09-18T10:15:56.688 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:56.688 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.688 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.688 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:56.688 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.793604+0000", 2024-09-18T10:15:56.688 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:15:56.689 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.689 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.689 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:56.689 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.793613+0000", 2024-09-18T10:15:56.689 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.4360000000000008e-06 2024-09-18T10:15:56.689 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.689 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.689 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:56.689 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.793628+0000", 2024-09-18T10:15:56.689 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4953000000000001e-05 2024-09-18T10:15:56.689 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.689 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.689 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:56.689 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.793634+0000", 2024-09-18T10:15:56.690 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.0170000000000002e-06 2024-09-18T10:15:56.690 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.690 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.690 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:56.690 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.905359+0000", 2024-09-18T10:15:56.690 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.111724328 2024-09-18T10:15:56.690 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.690 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.690 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:56.690 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.905540+0000", 2024-09-18T10:15:56.690 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000181552 2024-09-18T10:15:56.690 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.690 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.691 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:56.691 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:53.203724+0000", 2024-09-18T10:15:56.691 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.29818339500000002 2024-09-18T10:15:56.691 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.691 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.691 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:56.691 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:53.206308+0000", 2024-09-18T10:15:56.691 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0025841620000000001 2024-09-18T10:15:56.691 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.691 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.691 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:56.691 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:53.407237+0000", 2024-09-18T10:15:56.691 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.200928739 2024-09-18T10:15:56.692 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.692 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.692 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:56.692 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:53.407422+0000", 2024-09-18T10:15:56.692 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000185384 2024-09-18T10:15:56.692 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:56.692 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:56.692 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:56.692 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.692 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.692 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:203 3.9s0 3:9221fff5:::smithi18925896-48:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:15:56.692 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:52.793641+0000", 2024-09-18T10:15:56.692 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 63.852500505999998, 2024-09-18T10:15:56.693 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.41220821099999999, 2024-09-18T10:15:56.693 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:56.693 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:15:56.693 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:56.693 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:56.693 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:56.693 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 203 2024-09-18T10:15:56.693 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.693 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:56.693 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.693 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:56.693 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.793641+0000", 2024-09-18T10:15:56.693 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:56.693 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.694 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.694 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:56.694 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.793641+0000", 2024-09-18T10:15:56.694 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:56.694 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.694 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.694 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:56.694 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.793641+0000", 2024-09-18T10:15:56.694 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:15:56.694 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.694 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.694 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:56.694 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.793644+0000", 2024-09-18T10:15:56.695 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7170000000000002e-06 2024-09-18T10:15:56.695 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.695 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.695 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:56.695 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.793658+0000", 2024-09-18T10:15:56.695 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3353000000000001e-05 2024-09-18T10:15:56.695 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.695 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.695 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:56.695 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.793663+0000", 2024-09-18T10:15:56.695 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.7729999999999996e-06 2024-09-18T10:15:56.695 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.695 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.696 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:56.696 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.905813+0000", 2024-09-18T10:15:56.696 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.112150033 2024-09-18T10:15:56.696 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.696 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.696 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for blocked object", 2024-09-18T10:15:56.696 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.905901+0000", 2024-09-18T10:15:56.696 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.7319999999999997e-05 2024-09-18T10:15:56.697 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.697 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.697 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:56.697 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:53.205317+0000", 2024-09-18T10:15:56.697 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.299415826 2024-09-18T10:15:56.697 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.697 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.697 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:56.697 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:53.205528+0000", 2024-09-18T10:15:56.697 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000211485 2024-09-18T10:15:56.697 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.697 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.697 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:56.697 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:53.205850+0000", 2024-09-18T10:15:56.698 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000321544 2024-09-18T10:15:56.698 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:56.698 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:56.698 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:56.698 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.698 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.698 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:351 3.19s0 3:9a1a774c:::smithi18925896-38:b [read 0~1638400,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-18T10:15:56.698 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:03.544302+0000", 2024-09-18T10:15:56.698 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 53.101839501999997, 2024-09-18T10:15:56.698 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.41512137599999999, 2024-09-18T10:15:56.698 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:56.698 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:15:56.698 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:56.699 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:56.699 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:56.699 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 351 2024-09-18T10:15:56.699 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.699 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:56.699 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.699 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:56.699 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.544302+0000", 2024-09-18T10:15:56.699 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:56.699 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.699 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.699 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:56.699 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.544302+0000", 2024-09-18T10:15:56.700 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:56.700 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.700 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.700 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:56.700 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.544298+0000", 2024-09-18T10:15:56.700 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999957 2024-09-18T10:15:56.700 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.700 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.700 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:56.700 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.544320+0000", 2024-09-18T10:15:56.701 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1549e-05 2024-09-18T10:15:56.701 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.701 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.701 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:56.701 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.544366+0000", 2024-09-18T10:15:56.701 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.6752999999999998e-05 2024-09-18T10:15:56.701 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.701 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.701 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:56.701 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.544387+0000", 2024-09-18T10:15:56.701 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1052000000000001e-05 2024-09-18T10:15:56.701 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.701 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.702 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:56.702 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.544758+0000", 2024-09-18T10:15:56.702 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00037069700000000002 2024-09-18T10:15:56.702 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.702 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.702 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "op must wait for map", 2024-09-18T10:15:56.702 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.544786+0000", 2024-09-18T10:15:56.702 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7472000000000001e-05 2024-09-18T10:15:56.702 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.702 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.702 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:56.702 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.649236+0000", 2024-09-18T10:15:56.702 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.104450378 2024-09-18T10:15:56.703 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.703 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.703 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:56.703 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.649492+0000", 2024-09-18T10:15:56.703 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00025605300000000001 2024-09-18T10:15:56.703 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.703 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.703 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:56.703 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.959424+0000", 2024-09-18T10:15:56.703 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.30993174899999998 2024-09-18T10:15:56.703 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:56.703 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:56.703 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:56.703 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.704 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.704 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:417 3.9s0 3:9221fff5:::smithi18925896-48:head [copy-from ver 8] snapc 10=[10,f,e,b,4] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-18T10:15:56.704 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:08.547348+0000", 2024-09-18T10:15:56.704 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 48.098794276, 2024-09-18T10:15:56.704 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.69832097199999998, 2024-09-18T10:15:56.704 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:56.704 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:56.704 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:56.704 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:56.704 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:56.704 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 417 2024-09-18T10:15:56.704 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.704 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:56.705 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.705 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:56.705 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:08.547348+0000", 2024-09-18T10:15:56.705 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:56.705 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.705 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.705 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:56.705 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:08.547348+0000", 2024-09-18T10:15:56.705 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:56.705 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.705 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.705 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:56.705 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:08.547344+0000", 2024-09-18T10:15:56.706 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999967 2024-09-18T10:15:56.706 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.706 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.706 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:56.706 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:08.547367+0000", 2024-09-18T10:15:56.706 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.2316000000000001e-05 2024-09-18T10:15:56.706 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.706 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.706 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:56.706 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:08.547415+0000", 2024-09-18T10:15:56.706 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.8541999999999997e-05 2024-09-18T10:15:56.706 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.706 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.706 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:56.707 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:08.647531+0000", 2024-09-18T10:15:56.707 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10011588 2024-09-18T10:15:56.707 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.707 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.707 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:56.707 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:08.647713+0000", 2024-09-18T10:15:56.707 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000181965 2024-09-18T10:15:56.707 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.707 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.707 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "op must wait for map", 2024-09-18T10:15:56.707 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:08.647726+0000", 2024-09-18T10:15:56.707 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3576e-05 2024-09-18T10:15:56.707 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.708 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.708 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:56.708 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:08.754339+0000", 2024-09-18T10:15:56.708 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10661229699999999 2024-09-18T10:15:56.708 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.708 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.708 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:56.708 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:08.754855+0000", 2024-09-18T10:15:56.708 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00051627900000000002 2024-09-18T10:15:56.708 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.708 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.708 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:56.708 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:09.040700+0000", 2024-09-18T10:15:56.709 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.285844922 2024-09-18T10:15:56.709 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.709 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.709 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:56.709 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:09.045961+0000", 2024-09-18T10:15:56.709 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0052605769999999998 2024-09-18T10:15:56.709 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.709 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.709 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:56.709 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:09.245481+0000", 2024-09-18T10:15:56.709 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.19952081599999999 2024-09-18T10:15:56.709 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.710 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.710 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:56.710 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:09.245669+0000", 2024-09-18T10:15:56.710 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000187245 2024-09-18T10:15:56.710 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:56.710 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:56.710 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:56.710 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.710 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.710 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.1s4 57/40 ECSubWrite(tid=112, reqid=osd.4.0:112, at_version=57'20, trim_to=0'0, roll_forward_to=56'16))", 2024-09-18T10:15:56.710 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:13.062699+0000", 2024-09-18T10:15:56.710 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 43.583442587, 2024-09-18T10:15:56.711 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40470361999999999, 2024-09-18T10:15:56.711 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:56.711 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:56.711 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:56.711 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.711 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:56.711 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.062699+0000", 2024-09-18T10:15:56.711 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:56.711 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.711 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.711 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:56.711 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.062699+0000", 2024-09-18T10:15:56.711 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:56.711 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.712 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.712 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:56.712 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.062699+0000", 2024-09-18T10:15:56.712 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:56.712 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.712 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.712 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:56.712 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.062734+0000", 2024-09-18T10:15:56.712 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.5108999999999999e-05 2024-09-18T10:15:56.712 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.712 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.712 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:56.712 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.062746+0000", 2024-09-18T10:15:56.713 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2062000000000001e-05 2024-09-18T10:15:56.713 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.713 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.713 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:56.713 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.062754+0000", 2024-09-18T10:15:56.713 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.5800000000000003e-06 2024-09-18T10:15:56.713 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.713 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.713 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:56.713 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.254656+0000", 2024-09-18T10:15:56.713 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.19190244400000001 2024-09-18T10:15:56.713 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.713 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.714 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:56.714 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.254673+0000", 2024-09-18T10:15:56.714 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7146000000000002e-05 2024-09-18T10:15:56.714 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.714 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.714 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:56.714 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.467347+0000", 2024-09-18T10:15:56.714 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.21267335500000001 2024-09-18T10:15:56.714 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.714 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.714 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:56.714 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.467403+0000", 2024-09-18T10:15:56.714 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.6443999999999997e-05 2024-09-18T10:15:56.715 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:56.715 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:56.715 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:56.715 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.715 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.715 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:499 3.9s0 3:9221fff5:::smithi18925896-48:head [read 0~704512,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T10:15:56.715 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:17.889501+0000", 2024-09-18T10:15:56.715 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 38.75664055, 2024-09-18T10:15:56.715 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.41188610399999998, 2024-09-18T10:15:56.715 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:56.715 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:15:56.715 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:56.715 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:56.716 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:56.716 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 499 2024-09-18T10:15:56.716 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.716 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:56.716 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.716 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:56.716 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889501+0000", 2024-09-18T10:15:56.716 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:56.716 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.716 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.716 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:56.716 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889501+0000", 2024-09-18T10:15:56.717 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:56.717 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.717 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.717 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:56.717 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889499+0000", 2024-09-18T10:15:56.717 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999976 2024-09-18T10:15:56.717 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.717 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.717 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:56.717 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889514+0000", 2024-09-18T10:15:56.717 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4793999999999999e-05 2024-09-18T10:15:56.717 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.717 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.717 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:56.717 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889538+0000", 2024-09-18T10:15:56.718 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4218000000000001e-05 2024-09-18T10:15:56.718 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.718 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.718 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:56.718 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889552+0000", 2024-09-18T10:15:56.718 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4021e-05 2024-09-18T10:15:56.718 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.718 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.718 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:56.718 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889705+0000", 2024-09-18T10:15:56.718 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00015229499999999999 2024-09-18T10:15:56.718 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.718 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.718 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "op must wait for map", 2024-09-18T10:15:56.718 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889725+0000", 2024-09-18T10:15:56.719 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0251999999999999e-05 2024-09-18T10:15:56.719 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.719 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.719 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:56.719 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.894166+0000", 2024-09-18T10:15:56.719 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0044407969999999998 2024-09-18T10:15:56.719 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.719 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.719 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:56.720 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.894321+0000", 2024-09-18T10:15:56.720 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000155403 2024-09-18T10:15:56.720 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.720 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.720 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:56.720 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.301387+0000", 2024-09-18T10:15:56.720 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40706650900000002 2024-09-18T10:15:56.720 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:56.720 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:56.720 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:56.720 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.720 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.721 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:500 3.9s0 3:9221fff5:::smithi18925896-48:head [read 0~704512,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T10:15:56.721 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:17.889581+0000", 2024-09-18T10:15:56.721 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 38.756561009000002, 2024-09-18T10:15:56.721 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.41275783799999999, 2024-09-18T10:15:56.721 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:56.721 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:15:56.721 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:56.721 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:56.721 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:56.721 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 500 2024-09-18T10:15:56.721 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.721 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:56.721 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.722 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:56.722 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889581+0000", 2024-09-18T10:15:56.722 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:56.722 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.722 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.722 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:56.722 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889581+0000", 2024-09-18T10:15:56.722 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:56.722 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.722 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.722 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:56.722 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889580+0000", 2024-09-18T10:15:56.722 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:56.723 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.723 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.723 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:56.723 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889586+0000", 2024-09-18T10:15:56.723 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8340000000000004e-06 2024-09-18T10:15:56.723 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.723 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.723 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:56.723 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889594+0000", 2024-09-18T10:15:56.723 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.8320000000000006e-06 2024-09-18T10:15:56.723 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.723 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.723 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:56.724 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889599+0000", 2024-09-18T10:15:56.724 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.7890000000000002e-06 2024-09-18T10:15:56.724 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.724 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.724 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:56.724 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889906+0000", 2024-09-18T10:15:56.724 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00030686400000000001 2024-09-18T10:15:56.724 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.724 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.724 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:56.724 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889930+0000", 2024-09-18T10:15:56.724 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4318999999999999e-05 2024-09-18T10:15:56.724 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.724 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.725 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:56.725 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.895541+0000", 2024-09-18T10:15:56.725 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0056113029999999998 2024-09-18T10:15:56.725 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.725 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.725 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:56.725 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.895817+0000", 2024-09-18T10:15:56.725 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000275612 2024-09-18T10:15:56.725 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.725 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.725 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:56.725 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.302339+0000", 2024-09-18T10:15:56.725 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40652197699999998 2024-09-18T10:15:56.726 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:56.726 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:56.726 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:56.726 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.726 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.726 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:501 3.9s0 3:9221fff5:::smithi18925896-48:head [read 0~704512,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T10:15:56.726 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:17.889668+0000", 2024-09-18T10:15:56.726 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 38.756473899, 2024-09-18T10:15:56.726 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.51380426099999998, 2024-09-18T10:15:56.726 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:56.726 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:15:56.726 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:56.726 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:56.726 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:56.727 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 501 2024-09-18T10:15:56.727 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.727 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:56.727 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.727 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:56.727 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889668+0000", 2024-09-18T10:15:56.727 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:56.727 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.727 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.727 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:56.727 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889668+0000", 2024-09-18T10:15:56.727 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:56.727 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.727 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.727 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:56.728 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889667+0000", 2024-09-18T10:15:56.728 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:56.728 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.728 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.728 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:56.728 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889671+0000", 2024-09-18T10:15:56.728 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7570000000000002e-06 2024-09-18T10:15:56.728 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.728 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.728 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:56.728 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889682+0000", 2024-09-18T10:15:56.728 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0709e-05 2024-09-18T10:15:56.728 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.728 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.729 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:56.729 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889686+0000", 2024-09-18T10:15:56.729 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.437e-06 2024-09-18T10:15:56.729 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.729 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.729 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:56.729 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889996+0000", 2024-09-18T10:15:56.729 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00030937899999999998 2024-09-18T10:15:56.729 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.729 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.729 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:56.729 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.890018+0000", 2024-09-18T10:15:56.729 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.2592999999999999e-05 2024-09-18T10:15:56.729 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.730 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.730 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:56.730 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.996777+0000", 2024-09-18T10:15:56.730 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.106758192 2024-09-18T10:15:56.730 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.730 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.730 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:56.730 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.997045+0000", 2024-09-18T10:15:56.730 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00026851300000000001 2024-09-18T10:15:56.730 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.730 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.730 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:56.730 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.403472+0000", 2024-09-18T10:15:56.730 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40642724899999999 2024-09-18T10:15:56.731 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:56.731 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:56.731 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:56.731 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.731 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.731 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:502 3.19s0 3:9a1a774c:::smithi18925896-38:head [create,setxattr Rj\\mZOfQXo^b_ZWWfqN]m[iPt^aRVgsYNetVd`VO (19508) in=40b,setxattr [adnUNmYmWdN__]tR]msPnVSRiY^psh`dY\\Z (630) in=36b,setxattr cPsWXtVeXTPagdn_WVZbVcV\\UjSZUqsVlPZbY` (630) in=38b,setxattr pqbrReOSsgZlmftc]`Z`]USaXjsacf`fao_VP`eU (1190) in=40b,setxattr aYfgr^q[\\aPg_aZsakfUfsdhYbra^QXjlkrpdY\\\\ (138) in=40b,setxattr e\\XXWiVknihNQQ[OsThQUbRmtprrfYPeRX[[mRcQ (1719) in=40b,setxattr TNQYR^jgR`pPaYitmrcliV[dttobncTntcfTlQkt (8420) in=40b,setxattr W_aq[RVsaUpQtX_RpV^Vpqrg`QgQViPeVaRqZSmf (525) in=40b,setxattr RWdkeaaUP\\PZteTNscS^UVRtjo`oqqdNqe]cdZUa (245) in=40b,setxattr R^`al]TqifatYOiOU]p`ZegfXmNsXVaZcdd]b]YX (1435) in=40b,setxattr p]OS_Znl[dNa^qTn`WW[j\\dtgbNac_bijYmg (630) in=36b,setxattr n_jr[tV[VhYPqU]UenciobhtYfY^a^c\\hd^YrcQ[ (13472) in=40b,setxattr gReQSUg`cdl_ilU_lo[ggg\\fZsVQY[PgY]jPsZ_U (14664) in=40b,setxattr cNWaQrsdt^dic_qN_^mTXmc`_X]YOTXNk[WldRe` (523) in=40b,setxattr tZ]grhlaq]kUO[k^d^dr``XNio^^Oaebpj\\[Ytdq (33) in=40b,setxattr jqS_j^cefaXab`pQoPXfRkSmdako`fZ_heb^_fma (1191) in=40b,setxattr dgXPi (630) in=5b,setxattr Or_kkpbnNkTrQ\\apUaUsZR\\XW^nabhaiQ\\]gOjR\\ (525) in=40b,setxattr Rf`aR[fdtjQ\\NaqZke]l_rUssrVpOPXYWpXkl_kd (2876) in=40b,setxattr _sUSYWYUmOrlUVhZajZeYbbnsdra[]`Q`^Ogea[m (631) in=40b] snapc 19=[19,18,16,12,11,10,f,e] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T10:15:56.731 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:17.894821+0000", 2024-09-18T10:15:56.731 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 38.751320870999997, 2024-09-18T10:15:56.731 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.61297755899999995, 2024-09-18T10:15:56.731 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:56.731 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:56.731 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:56.731 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:56.732 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:56.732 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 502 2024-09-18T10:15:56.732 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.732 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:56.732 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.732 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:56.732 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.894821+0000", 2024-09-18T10:15:56.732 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:56.732 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.732 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.732 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:56.732 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.894821+0000", 2024-09-18T10:15:56.733 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:56.733 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.733 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.733 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:56.733 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.894819+0000", 2024-09-18T10:15:56.733 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:56.733 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.733 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.733 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:56.733 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.894942+0000", 2024-09-18T10:15:56.733 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00012299699999999999 2024-09-18T10:15:56.733 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.733 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.733 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:56.734 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.894987+0000", 2024-09-18T10:15:56.734 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.515e-05 2024-09-18T10:15:56.734 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.734 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.734 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:56.734 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.895021+0000", 2024-09-18T10:15:56.734 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3905000000000001e-05 2024-09-18T10:15:56.734 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.734 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.734 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:56.734 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.302420+0000", 2024-09-18T10:15:56.734 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40739912099999998 2024-09-18T10:15:56.734 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.735 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.735 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:56.735 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.302573+0000", 2024-09-18T10:15:56.735 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00015244400000000001 2024-09-18T10:15:56.735 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.735 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.735 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:56.735 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.304119+0000", 2024-09-18T10:15:56.735 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001546504 2024-09-18T10:15:56.735 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.735 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.735 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:56.736 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.306309+0000", 2024-09-18T10:15:56.736 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.002189992 2024-09-18T10:15:56.736 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.736 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.736 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:56.736 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.507653+0000", 2024-09-18T10:15:56.736 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20134429600000001 2024-09-18T10:15:56.736 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.736 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.736 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:56.736 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.507799+0000", 2024-09-18T10:15:56.736 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000145153 2024-09-18T10:15:56.736 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:56.737 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:56.737 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:56.737 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.737 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.737 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:514 3.19s0 3:9fc2eacb:::smithi18925896-17:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e63)", 2024-09-18T10:15:56.737 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:19.215577+0000", 2024-09-18T10:15:56.737 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 37.430564629999999, 2024-09-18T10:15:56.737 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.55261972699999995, 2024-09-18T10:15:56.737 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:56.737 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:15:56.737 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:56.737 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:56.737 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:56.737 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 514 2024-09-18T10:15:56.738 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.738 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:56.738 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.738 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:56.738 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.215577+0000", 2024-09-18T10:15:56.738 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:56.738 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.738 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.738 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:56.738 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.215577+0000", 2024-09-18T10:15:56.738 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:56.738 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.738 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.739 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:56.739 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.215575+0000", 2024-09-18T10:15:56.739 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:56.739 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.739 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.739 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:56.739 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.215590+0000", 2024-09-18T10:15:56.739 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4153e-05 2024-09-18T10:15:56.739 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.739 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.739 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:56.739 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.215610+0000", 2024-09-18T10:15:56.739 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9962e-05 2024-09-18T10:15:56.739 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.740 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.740 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:56.740 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.215622+0000", 2024-09-18T10:15:56.740 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2714000000000001e-05 2024-09-18T10:15:56.740 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.740 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.740 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:56.740 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.216288+0000", 2024-09-18T10:15:56.740 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00066566900000000003 2024-09-18T10:15:56.740 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.740 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.740 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for blocked object", 2024-09-18T10:15:56.740 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.216400+0000", 2024-09-18T10:15:56.741 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000112101 2024-09-18T10:15:56.741 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.741 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.741 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:56.741 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.767868+0000", 2024-09-18T10:15:56.741 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.55146753900000001 2024-09-18T10:15:56.741 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.741 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.741 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:56.741 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.768017+0000", 2024-09-18T10:15:56.741 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000149567 2024-09-18T10:15:56.741 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.741 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.742 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:56.742 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.768197+0000", 2024-09-18T10:15:56.742 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00017981200000000001 2024-09-18T10:15:56.742 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:56.742 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:56.742 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:56.742 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.742 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.742 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:561 3.19s0 3:9fc2eacb:::smithi18925896-17:head [append 0~622592] snapc 1f=[1f,1e,1a,16,12,11,10,f] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-18T10:15:56.742 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:23.955639+0000", 2024-09-18T10:15:56.742 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 32.690503284000002, 2024-09-18T10:15:56.742 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.51252563100000004, 2024-09-18T10:15:56.742 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:56.742 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:56.743 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:56.743 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:56.743 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:56.743 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 561 2024-09-18T10:15:56.743 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.743 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:56.743 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.743 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:56.743 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.955639+0000", 2024-09-18T10:15:56.743 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:56.743 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.743 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.743 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:56.743 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.955639+0000", 2024-09-18T10:15:56.744 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:56.744 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.744 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.744 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:56.744 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.955637+0000", 2024-09-18T10:15:56.744 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:56.744 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.744 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.744 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:56.744 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956211+0000", 2024-09-18T10:15:56.744 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00057413099999999997 2024-09-18T10:15:56.744 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.744 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.744 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:56.745 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956241+0000", 2024-09-18T10:15:56.745 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0244999999999999e-05 2024-09-18T10:15:56.745 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.745 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.745 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:56.745 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956253+0000", 2024-09-18T10:15:56.745 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1824000000000001e-05 2024-09-18T10:15:56.745 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.745 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.745 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:56.745 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956338+0000", 2024-09-18T10:15:56.745 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.5544999999999995e-05 2024-09-18T10:15:56.745 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.745 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.746 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:56.746 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956349+0000", 2024-09-18T10:15:56.746 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0203e-05 2024-09-18T10:15:56.746 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.746 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.746 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:56.746 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:24.162636+0000", 2024-09-18T10:15:56.746 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.206287267 2024-09-18T10:15:56.746 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.746 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.746 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:56.746 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:24.162935+0000", 2024-09-18T10:15:56.746 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00029932800000000002 2024-09-18T10:15:56.747 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.747 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.747 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:56.747 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:24.164750+0000", 2024-09-18T10:15:56.747 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0018148400000000001 2024-09-18T10:15:56.747 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.747 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.747 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:56.747 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:24.168548+0000", 2024-09-18T10:15:56.747 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0037977089999999998 2024-09-18T10:15:56.747 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.747 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.747 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:56.747 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:24.467969+0000", 2024-09-18T10:15:56.748 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.29942109700000002 2024-09-18T10:15:56.748 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.748 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.748 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:56.748 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:24.468164+0000", 2024-09-18T10:15:56.748 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00019534000000000001 2024-09-18T10:15:56.748 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:56.748 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:56.748 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:56.748 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.748 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.748 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:562 3.19s0 3:9fc2eacb:::smithi18925896-17:head [append 0~32768] snapc 1f=[1f,1e,1a,16,12,11,10,f] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-18T10:15:56.748 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:23.956275+0000", 2024-09-18T10:15:56.749 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 32.689867296999999, 2024-09-18T10:15:56.749 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.61260356900000001, 2024-09-18T10:15:56.749 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:56.749 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:56.749 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:56.749 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:56.749 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:56.749 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 562 2024-09-18T10:15:56.749 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.749 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:56.749 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.749 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:56.749 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956275+0000", 2024-09-18T10:15:56.750 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:56.750 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.750 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.750 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:56.750 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956275+0000", 2024-09-18T10:15:56.750 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:56.750 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.750 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.750 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:56.750 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956273+0000", 2024-09-18T10:15:56.750 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:56.750 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.751 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.751 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:56.751 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956355+0000", 2024-09-18T10:15:56.751 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.1661000000000003e-05 2024-09-18T10:15:56.751 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.751 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.751 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:56.751 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956366+0000", 2024-09-18T10:15:56.751 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1192000000000001e-05 2024-09-18T10:15:56.751 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.751 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.751 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:56.752 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956373+0000", 2024-09-18T10:15:56.752 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.5599999999999999e-06 2024-09-18T10:15:56.752 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.752 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.752 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:56.752 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956434+0000", 2024-09-18T10:15:56.752 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.0761999999999999e-05 2024-09-18T10:15:56.752 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.752 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.752 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:56.752 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956441+0000", 2024-09-18T10:15:56.752 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.1430000000000001e-06 2024-09-18T10:15:56.752 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.752 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.753 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:56.753 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:24.165954+0000", 2024-09-18T10:15:56.753 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20951336100000001 2024-09-18T10:15:56.753 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.753 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.753 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:56.753 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:24.166077+0000", 2024-09-18T10:15:56.753 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00012297299999999999 2024-09-18T10:15:56.753 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.753 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.753 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:56.753 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:24.166875+0000", 2024-09-18T10:15:56.754 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00079825999999999996 2024-09-18T10:15:56.754 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.754 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.754 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:56.754 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:24.168650+0000", 2024-09-18T10:15:56.754 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0017746839999999999 2024-09-18T10:15:56.754 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.754 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.754 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:56.754 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:24.568726+0000", 2024-09-18T10:15:56.754 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40007642799999998 2024-09-18T10:15:56.754 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.755 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.755 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:56.755 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:24.568878+0000", 2024-09-18T10:15:56.755 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000151803 2024-09-18T10:15:56.755 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:56.755 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:56.755 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:56.755 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.755 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.755 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:563 3.19s0 3:9fc2eacb:::smithi18925896-17:head [setxattr _header (57) in=7b] snapc 1f=[1f,1e,1a,16,12,11,10,f] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-18T10:15:56.755 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:23.956389+0000", 2024-09-18T10:15:56.755 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 32.689752951000003, 2024-09-18T10:15:56.755 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.61276698100000004, 2024-09-18T10:15:56.756 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:56.756 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:56.756 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:56.756 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:56.756 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:56.756 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 563 2024-09-18T10:15:56.756 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.756 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:56.756 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.756 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:56.756 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956389+0000", 2024-09-18T10:15:56.756 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:56.756 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.757 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.757 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:56.757 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956389+0000", 2024-09-18T10:15:56.757 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:56.757 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.757 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.757 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:56.757 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956387+0000", 2024-09-18T10:15:56.757 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:56.757 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.757 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.757 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:56.757 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956395+0000", 2024-09-18T10:15:56.758 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.537e-06 2024-09-18T10:15:56.758 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.758 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.758 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:56.758 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956405+0000", 2024-09-18T10:15:56.758 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0057999999999999e-05 2024-09-18T10:15:56.758 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.758 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.758 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:56.758 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956416+0000", 2024-09-18T10:15:56.758 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1311999999999999e-05 2024-09-18T10:15:56.758 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.758 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.759 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:56.759 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956502+0000", 2024-09-18T10:15:56.759 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.5413000000000004e-05 2024-09-18T10:15:56.759 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.759 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.759 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:56.759 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956512+0000", 2024-09-18T10:15:56.759 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0299000000000001e-05 2024-09-18T10:15:56.759 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.759 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.759 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:56.759 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:24.167493+0000", 2024-09-18T10:15:56.759 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.21098060699999999 2024-09-18T10:15:56.760 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.760 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.760 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:56.760 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:24.167618+0000", 2024-09-18T10:15:56.760 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00012526599999999999 2024-09-18T10:15:56.760 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.760 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.760 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:56.760 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:24.168157+0000", 2024-09-18T10:15:56.760 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00053891400000000004 2024-09-18T10:15:56.760 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.760 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.760 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:56.760 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:24.168870+0000", 2024-09-18T10:15:56.761 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00071273300000000001 2024-09-18T10:15:56.761 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.761 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.761 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:56.761 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:24.569055+0000", 2024-09-18T10:15:56.761 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40018569599999998 2024-09-18T10:15:56.761 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.761 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.761 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:56.761 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:24.569156+0000", 2024-09-18T10:15:56.761 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000100738 2024-09-18T10:15:56.761 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:56.761 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:56.761 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:56.762 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.762 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.762 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:564 3.19s0 3:9fc2eacb:::smithi18925896-17:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e68)", 2024-09-18T10:15:56.762 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:23.956428+0000", 2024-09-18T10:15:56.762 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 32.689713851, 2024-09-18T10:15:56.762 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.91697059800000003, 2024-09-18T10:15:56.762 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:56.762 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:15:56.762 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:56.762 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:56.762 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:56.762 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 564 2024-09-18T10:15:56.762 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.762 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:56.763 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.763 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:56.763 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956428+0000", 2024-09-18T10:15:56.763 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:56.763 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.763 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.763 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:56.763 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956428+0000", 2024-09-18T10:15:56.763 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:56.763 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.763 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.763 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:56.763 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956427+0000", 2024-09-18T10:15:56.763 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:15:56.764 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.764 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.764 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:56.764 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956432+0000", 2024-09-18T10:15:56.764 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9980000000000003e-06 2024-09-18T10:15:56.764 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.764 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.764 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:56.764 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956446+0000", 2024-09-18T10:15:56.764 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3602e-05 2024-09-18T10:15:56.764 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.764 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.764 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:56.765 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956454+0000", 2024-09-18T10:15:56.765 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.2369999999999992e-06 2024-09-18T10:15:56.765 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.765 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.765 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:56.765 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956551+0000", 2024-09-18T10:15:56.765 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.7424999999999996e-05 2024-09-18T10:15:56.765 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.765 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.765 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:15:56.765 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956558+0000", 2024-09-18T10:15:56.765 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.8050000000000001e-06 2024-09-18T10:15:56.765 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.766 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.766 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:56.766 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:24.168428+0000", 2024-09-18T10:15:56.766 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.21186939299999999 2024-09-18T10:15:56.766 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.766 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.766 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for rw locks", 2024-09-18T10:15:56.766 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:24.168531+0000", 2024-09-18T10:15:56.766 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000103707 2024-09-18T10:15:56.766 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.766 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.766 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:56.766 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:24.669928+0000", 2024-09-18T10:15:56.767 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.50139628999999997 2024-09-18T10:15:56.767 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.767 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.767 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:56.767 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:24.670206+0000", 2024-09-18T10:15:56.767 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00027815000000000001 2024-09-18T10:15:56.767 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:56.767 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:56.767 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:56.767 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:24.873399+0000", 2024-09-18T10:15:56.767 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20319285200000001 2024-09-18T10:15:56.767 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:56.767 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:56.768 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:56.768 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:56.768 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:56.768 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-18T10:15:56.770 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-09-18T10:15:56.770 INFO:tasks.rados.rados.0.smithi189.stdout:393: done (1 left) 2024-09-18T10:15:56.770 INFO:tasks.rados.rados.0.smithi189.stdout:394: snap_remove snap 19 2024-09-18T10:15:56.770 INFO:tasks.ceph.osd.11.smithi189.stderr:2024-09-18T10:15:56.766+0000 7f718b60a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 22400) UID: 0 2024-09-18T10:15:56.870 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-18T10:15:56.865+0000 7f302f2c3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22187) UID: 0 2024-09-18T10:15:56.924 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-18T10:15:56.924 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-18T10:15:56.925 INFO:teuthology.orchestra.run.smithi189.stdout: "num_ops": 0 2024-09-18T10:15:56.925 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-18T10:15:56.935 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-09-18T10:15:56.971 INFO:tasks.ceph.osd.6.smithi154.stderr:2024-09-18T10:15:56.970+0000 7f0abc9e1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22089) UID: 0 2024-09-18T10:15:57.027 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-18T10:15:57.028 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-18T10:15:57.028 INFO:teuthology.orchestra.run.smithi189.stdout: "complaint_time": 30, 2024-09-18T10:15:57.028 INFO:teuthology.orchestra.run.smithi189.stdout: "num_blocked_ops": 0 2024-09-18T10:15:57.028 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-18T10:15:57.037 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-09-18T10:15:57.072 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-18T10:15:57.067+0000 7f00046a8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 22136) UID: 0 2024-09-18T10:15:57.131 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-18T10:15:57.131 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 20, 2024-09-18T10:15:57.131 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 600, 2024-09-18T10:15:57.131 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [ 2024-09-18T10:15:57.132 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.132 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:146 3.es0 3:78aad6f5:::smithi18925896-49:head [append 0~442368,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:57.132 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:51.117496+0000", 2024-09-18T10:15:57.132 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 66.010023121000003, 2024-09-18T10:15:57.132 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.61437150200000001, 2024-09-18T10:15:57.132 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:57.132 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:57.132 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:57.132 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:57.132 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:57.132 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 146 2024-09-18T10:15:57.132 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.132 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:57.133 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.133 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:57.133 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.117496+0000", 2024-09-18T10:15:57.133 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:57.133 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.133 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.133 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:57.133 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.117496+0000", 2024-09-18T10:15:57.133 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:57.133 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.133 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.133 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:57.134 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.117493+0000", 2024-09-18T10:15:57.134 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999976 2024-09-18T10:15:57.134 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.134 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.134 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:57.134 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.117787+0000", 2024-09-18T10:15:57.134 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00029378100000000001 2024-09-18T10:15:57.134 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.134 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.134 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:57.134 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.117820+0000", 2024-09-18T10:15:57.134 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3073e-05 2024-09-18T10:15:57.135 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.135 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.135 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:57.135 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.117834+0000", 2024-09-18T10:15:57.135 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3369e-05 2024-09-18T10:15:57.135 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.135 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.135 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:57.135 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.127090+0000", 2024-09-18T10:15:57.135 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0092563769999999997 2024-09-18T10:15:57.135 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.135 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.135 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:57.136 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.127435+0000", 2024-09-18T10:15:57.136 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00034522300000000003 2024-09-18T10:15:57.136 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.136 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.136 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:57.136 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.129115+0000", 2024-09-18T10:15:57.136 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0016792980000000001 2024-09-18T10:15:57.136 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.136 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.136 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:57.136 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.430505+0000", 2024-09-18T10:15:57.136 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.30139002300000001 2024-09-18T10:15:57.136 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.137 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.137 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:57.137 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.731753+0000", 2024-09-18T10:15:57.137 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.30124798800000002 2024-09-18T10:15:57.137 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.137 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.137 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:57.137 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.731867+0000", 2024-09-18T10:15:57.137 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000114764 2024-09-18T10:15:57.137 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:57.137 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:57.137 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:57.137 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.137 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.138 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:147 3.es0 3:78aad6f5:::smithi18925896-49:head [append 0~376832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:57.138 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:51.122092+0000", 2024-09-18T10:15:57.138 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 66.005426702999998, 2024-09-18T10:15:57.138 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.61047882200000003, 2024-09-18T10:15:57.138 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:57.138 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:57.138 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:57.138 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:57.138 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:57.138 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 147 2024-09-18T10:15:57.138 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.138 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:57.138 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.139 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:57.139 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122092+0000", 2024-09-18T10:15:57.139 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:57.139 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.139 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.139 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:57.139 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122092+0000", 2024-09-18T10:15:57.139 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:57.139 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.139 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.139 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:57.139 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122091+0000", 2024-09-18T10:15:57.140 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:57.140 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.140 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.140 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:57.140 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122504+0000", 2024-09-18T10:15:57.140 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00041322100000000002 2024-09-18T10:15:57.140 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.140 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.140 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:57.140 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122527+0000", 2024-09-18T10:15:57.140 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3504999999999999e-05 2024-09-18T10:15:57.140 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.141 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.141 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:57.141 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122537+0000", 2024-09-18T10:15:57.141 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.8460000000000003e-06 2024-09-18T10:15:57.141 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.141 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.141 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:57.141 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.327415+0000", 2024-09-18T10:15:57.141 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20487799700000001 2024-09-18T10:15:57.141 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.141 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.141 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:57.141 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.327614+0000", 2024-09-18T10:15:57.142 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00019855400000000001 2024-09-18T10:15:57.142 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.142 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.142 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:57.142 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.328920+0000", 2024-09-18T10:15:57.142 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001306263 2024-09-18T10:15:57.142 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.142 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.142 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:57.142 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.430570+0000", 2024-09-18T10:15:57.142 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.101650508 2024-09-18T10:15:57.142 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.142 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.143 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:57.143 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.732477+0000", 2024-09-18T10:15:57.143 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.30190700100000001 2024-09-18T10:15:57.143 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.143 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.143 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:57.143 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.732571+0000", 2024-09-18T10:15:57.143 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.3580999999999997e-05 2024-09-18T10:15:57.143 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:57.143 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:57.143 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:57.143 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.143 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.143 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:148 3.es0 3:78aad6f5:::smithi18925896-49:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:57.144 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:51.122547+0000", 2024-09-18T10:15:57.144 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 66.004971611000002, 2024-09-18T10:15:57.144 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.91462763400000002, 2024-09-18T10:15:57.144 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:57.144 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:57.144 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:57.144 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:57.144 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:57.144 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 148 2024-09-18T10:15:57.144 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.144 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:57.144 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.144 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:57.144 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122547+0000", 2024-09-18T10:15:57.145 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:57.145 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.145 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.145 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:57.145 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122547+0000", 2024-09-18T10:15:57.145 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:57.145 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.145 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.145 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:57.145 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122546+0000", 2024-09-18T10:15:57.145 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:15:57.145 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.145 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.146 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:57.146 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122552+0000", 2024-09-18T10:15:57.146 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3569999999999997e-06 2024-09-18T10:15:57.146 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.146 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.146 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:57.146 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122560+0000", 2024-09-18T10:15:57.146 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.191e-06 2024-09-18T10:15:57.146 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.146 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.146 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:57.146 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122569+0000", 2024-09-18T10:15:57.146 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.8230000000000001e-06 2024-09-18T10:15:57.147 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.147 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.147 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:57.147 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.834492+0000", 2024-09-18T10:15:57.147 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.71192345599999995 2024-09-18T10:15:57.147 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.147 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.147 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:57.147 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.834669+0000", 2024-09-18T10:15:57.147 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000177035 2024-09-18T10:15:57.147 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.147 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.147 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:57.147 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.835442+0000", 2024-09-18T10:15:57.148 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00077327799999999996 2024-09-18T10:15:57.148 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.148 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.148 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:57.148 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.936574+0000", 2024-09-18T10:15:57.148 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.101131163 2024-09-18T10:15:57.148 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.148 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.148 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:57.148 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.037028+0000", 2024-09-18T10:15:57.148 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100454026 2024-09-18T10:15:57.148 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.148 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.149 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:57.149 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.037175+0000", 2024-09-18T10:15:57.149 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000147305 2024-09-18T10:15:57.149 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:57.149 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:57.149 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:57.149 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.149 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.149 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:149 3.es0 3:78aad6f5:::smithi18925896-49:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:15:57.149 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:51.122575+0000", 2024-09-18T10:15:57.149 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 66.004944402999996, 2024-09-18T10:15:57.149 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.016813872, 2024-09-18T10:15:57.149 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:57.150 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:15:57.150 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:57.150 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:57.150 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:57.150 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 149 2024-09-18T10:15:57.150 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.150 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:57.150 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.150 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:57.150 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122575+0000", 2024-09-18T10:15:57.150 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:57.150 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.150 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.151 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:57.151 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122575+0000", 2024-09-18T10:15:57.151 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:57.151 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.151 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.151 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:57.151 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122574+0000", 2024-09-18T10:15:57.151 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:15:57.151 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.151 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.151 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:57.151 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122579+0000", 2024-09-18T10:15:57.151 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3650000000000003e-06 2024-09-18T10:15:57.152 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.152 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.152 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:57.152 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122587+0000", 2024-09-18T10:15:57.152 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.3000000000000002e-06 2024-09-18T10:15:57.152 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.152 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.152 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:57.152 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122592+0000", 2024-09-18T10:15:57.152 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9080000000000003e-06 2024-09-18T10:15:57.152 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.152 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.152 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:57.153 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.935931+0000", 2024-09-18T10:15:57.153 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.81333878199999998 2024-09-18T10:15:57.153 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.153 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.153 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for rw locks", 2024-09-18T10:15:57.153 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.936216+0000", 2024-09-18T10:15:57.153 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00028463400000000003 2024-09-18T10:15:57.153 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.153 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.153 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:57.153 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.038007+0000", 2024-09-18T10:15:57.153 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.101791539 2024-09-18T10:15:57.153 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.153 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.154 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:57.154 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.038254+0000", 2024-09-18T10:15:57.154 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000246932 2024-09-18T10:15:57.154 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.154 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.154 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:57.154 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.139388+0000", 2024-09-18T10:15:57.154 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10113417199999999 2024-09-18T10:15:57.154 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:57.154 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:57.154 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:57.154 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.154 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.155 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpRead(3.es0 36/32 ECSubRead(tid=3, to_read={3:7255ddf4:::smithi18925896-39:head=0,4096,0}, subchunks={3:7255ddf4:::smithi18925896-39:head=[0,1]}, attrs_to_read=))", 2024-09-18T10:15:57.155 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:51.126838+0000", 2024-09-18T10:15:57.155 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 66.000680979999998, 2024-09-18T10:15:57.155 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.707585825, 2024-09-18T10:15:57.155 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:57.155 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:57.155 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:57.155 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.155 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:57.155 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.126838+0000", 2024-09-18T10:15:57.155 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:57.155 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.156 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.156 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:57.156 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.126838+0000", 2024-09-18T10:15:57.156 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:57.156 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.156 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.156 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:57.156 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.126838+0000", 2024-09-18T10:15:57.156 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:57.156 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.156 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.156 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:57.156 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.126838+0000", 2024-09-18T10:15:57.157 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:57.157 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.157 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.157 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:57.157 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.127004+0000", 2024-09-18T10:15:57.157 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00016626499999999999 2024-09-18T10:15:57.157 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.157 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.157 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:57.157 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.227100+0000", 2024-09-18T10:15:57.157 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.1000955 2024-09-18T10:15:57.157 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.157 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.158 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:57.158 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.834083+0000", 2024-09-18T10:15:57.158 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.60698300100000002 2024-09-18T10:15:57.158 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.158 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.158 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:57.158 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.834424+0000", 2024-09-18T10:15:57.158 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000341059 2024-09-18T10:15:57.158 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:57.158 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:57.158 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:57.158 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.158 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.159 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpReadReply(3.es0 36/32 ECSubReadReply(tid=3, attrs_read=0))", 2024-09-18T10:15:57.159 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:51.214895+0000", 2024-09-18T10:15:57.159 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 65.912624303000001, 2024-09-18T10:15:57.159 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.61857800100000004, 2024-09-18T10:15:57.159 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:57.159 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:57.159 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:57.159 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.159 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:57.159 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.214895+0000", 2024-09-18T10:15:57.159 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:57.159 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.159 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.160 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:57.160 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.214895+0000", 2024-09-18T10:15:57.160 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:57.160 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.160 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.160 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:57.160 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.214893+0000", 2024-09-18T10:15:57.160 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:57.160 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.160 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.160 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:57.160 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.214927+0000", 2024-09-18T10:15:57.161 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3841999999999999e-05 2024-09-18T10:15:57.161 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.161 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.161 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:57.161 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.214953+0000", 2024-09-18T10:15:57.161 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.6519000000000001e-05 2024-09-18T10:15:57.161 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.161 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.161 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:57.161 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.214971+0000", 2024-09-18T10:15:57.161 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7535999999999999e-05 2024-09-18T10:15:57.161 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.162 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.162 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:57.162 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.833397+0000", 2024-09-18T10:15:57.162 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.61842642599999997 2024-09-18T10:15:57.162 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.162 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.162 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:57.162 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.833473+0000", 2024-09-18T10:15:57.162 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.5621e-05 2024-09-18T10:15:57.162 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:57.162 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:57.162 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:57.163 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.163 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.163 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpReadReply(3.es0 36/32 ECSubReadReply(tid=3, attrs_read=0))", 2024-09-18T10:15:57.163 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:51.227596+0000", 2024-09-18T10:15:57.163 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 65.899922751999995, 2024-09-18T10:15:57.163 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.60597977400000003, 2024-09-18T10:15:57.163 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:57.163 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:57.163 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:57.163 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.163 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:57.163 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.227596+0000", 2024-09-18T10:15:57.163 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:57.164 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.164 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.164 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:57.164 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.227596+0000", 2024-09-18T10:15:57.164 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:57.164 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.164 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.164 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:57.164 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.227595+0000", 2024-09-18T10:15:57.164 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:15:57.164 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.164 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.164 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:57.164 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.227605+0000", 2024-09-18T10:15:57.165 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.7040000000000006e-06 2024-09-18T10:15:57.165 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.165 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.165 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:57.165 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.227621+0000", 2024-09-18T10:15:57.165 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6107999999999999e-05 2024-09-18T10:15:57.165 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.165 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.165 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:57.165 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.227637+0000", 2024-09-18T10:15:57.165 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5282999999999999e-05 2024-09-18T10:15:57.165 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.166 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.166 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:57.166 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.833511+0000", 2024-09-18T10:15:57.166 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.60587403699999998 2024-09-18T10:15:57.166 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.166 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.166 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:57.166 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.833576+0000", 2024-09-18T10:15:57.166 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.5392e-05 2024-09-18T10:15:57.166 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:57.166 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:57.166 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:57.166 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.167 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.167 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.es0 36/32 ECSubWriteReply(tid=4, last_complete=36'3, committed=1, applied=1))", 2024-09-18T10:15:57.167 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:51.233227+0000", 2024-09-18T10:15:57.167 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 65.894291738000007, 2024-09-18T10:15:57.167 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.39820466900000001, 2024-09-18T10:15:57.167 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:57.167 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:57.167 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:57.167 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.167 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:57.167 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.233227+0000", 2024-09-18T10:15:57.167 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:57.167 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.167 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.168 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:57.168 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.233227+0000", 2024-09-18T10:15:57.168 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:57.168 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.168 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.168 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:57.168 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.233226+0000", 2024-09-18T10:15:57.168 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:15:57.168 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.168 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.168 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:57.168 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.233232+0000", 2024-09-18T10:15:57.168 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9370000000000001e-06 2024-09-18T10:15:57.169 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.169 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.169 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:57.169 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.233248+0000", 2024-09-18T10:15:57.169 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5574999999999999e-05 2024-09-18T10:15:57.169 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.169 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.169 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:57.169 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.233254+0000", 2024-09-18T10:15:57.169 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.3659999999999997e-06 2024-09-18T10:15:57.169 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.169 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.169 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:57.170 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.631420+0000", 2024-09-18T10:15:57.170 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.39816591699999998 2024-09-18T10:15:57.170 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.170 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.170 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:57.170 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.631432+0000", 2024-09-18T10:15:57.170 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1737e-05 2024-09-18T10:15:57.170 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:57.170 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:57.170 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:57.170 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.170 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.170 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.es0 36/32 ECSubWriteReply(tid=5, last_complete=36'4, committed=1, applied=1))", 2024-09-18T10:15:57.171 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:51.331048+0000", 2024-09-18T10:15:57.171 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 65.796471311000005, 2024-09-18T10:15:57.171 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40123123900000002, 2024-09-18T10:15:57.171 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:57.171 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:57.171 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:57.171 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.171 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:57.171 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.331048+0000", 2024-09-18T10:15:57.171 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:57.171 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.171 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.171 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:57.171 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.331048+0000", 2024-09-18T10:15:57.172 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:57.172 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.172 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.172 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:57.172 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.331046+0000", 2024-09-18T10:15:57.172 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:57.172 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.172 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.172 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:57.172 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.331057+0000", 2024-09-18T10:15:57.172 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1683e-05 2024-09-18T10:15:57.172 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.173 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.173 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:57.173 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.331090+0000", 2024-09-18T10:15:57.173 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2576000000000001e-05 2024-09-18T10:15:57.173 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.173 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.173 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:57.173 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.331107+0000", 2024-09-18T10:15:57.173 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7382999999999999e-05 2024-09-18T10:15:57.173 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.173 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.174 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:57.174 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.732241+0000", 2024-09-18T10:15:57.174 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40113375200000001 2024-09-18T10:15:57.174 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.174 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.174 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:57.174 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.732279+0000", 2024-09-18T10:15:57.174 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7719000000000003e-05 2024-09-18T10:15:57.174 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:57.174 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:57.174 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:57.174 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.175 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.175 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.es0 36/32 ECSubWriteReply(tid=5, last_complete=36'4, committed=1, applied=1))", 2024-09-18T10:15:57.175 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:51.331214+0000", 2024-09-18T10:15:57.175 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 65.796304917000001, 2024-09-18T10:15:57.175 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40084109800000001, 2024-09-18T10:15:57.175 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:57.175 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:57.175 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:57.175 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.175 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:57.175 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.331214+0000", 2024-09-18T10:15:57.175 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:57.175 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.176 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.176 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:57.176 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.331214+0000", 2024-09-18T10:15:57.176 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:57.176 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.176 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.176 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:57.176 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.331213+0000", 2024-09-18T10:15:57.176 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:15:57.176 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.177 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.177 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:57.177 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.331220+0000", 2024-09-18T10:15:57.177 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.1369999999999996e-06 2024-09-18T10:15:57.177 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.177 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.177 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:57.177 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.331235+0000", 2024-09-18T10:15:57.177 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4849e-05 2024-09-18T10:15:57.177 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.177 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.178 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:57.178 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.331243+0000", 2024-09-18T10:15:57.178 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.157e-06 2024-09-18T10:15:57.178 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.178 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.178 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:57.178 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.732010+0000", 2024-09-18T10:15:57.178 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40076711300000001 2024-09-18T10:15:57.178 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.179 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.179 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:57.179 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.732055+0000", 2024-09-18T10:15:57.179 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4821999999999999e-05 2024-09-18T10:15:57.179 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:57.179 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:57.179 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:57.179 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.179 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.179 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:347 3.es0 3:7255ddf4:::smithi18925896-39:head [append 0~376832] snapc b=[b,4] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-18T10:15:57.179 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:02.898980+0000", 2024-09-18T10:15:57.179 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 54.228538893, 2024-09-18T10:15:57.179 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.50845237799999998, 2024-09-18T10:15:57.180 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:57.180 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:57.180 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:57.180 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:57.180 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:57.180 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 347 2024-09-18T10:15:57.180 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.180 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:57.180 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.180 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:57.180 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.898980+0000", 2024-09-18T10:15:57.180 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:57.180 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.181 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.181 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:57.181 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.898980+0000", 2024-09-18T10:15:57.181 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:57.181 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.181 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.181 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:57.181 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.898978+0000", 2024-09-18T10:15:57.181 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:57.181 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.181 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.181 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:57.182 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.899378+0000", 2024-09-18T10:15:57.182 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000399465 2024-09-18T10:15:57.182 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.182 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.182 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:57.182 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.899397+0000", 2024-09-18T10:15:57.182 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9219999999999999e-05 2024-09-18T10:15:57.182 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.182 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.182 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:57.182 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.899410+0000", 2024-09-18T10:15:57.182 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2486e-05 2024-09-18T10:15:57.183 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.183 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.183 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:57.183 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.103248+0000", 2024-09-18T10:15:57.183 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20383807200000001 2024-09-18T10:15:57.183 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.183 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.183 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:57.183 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.103447+0000", 2024-09-18T10:15:57.183 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00019970400000000001 2024-09-18T10:15:57.183 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.183 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.183 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:57.184 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.104894+0000", 2024-09-18T10:15:57.184 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001447083 2024-09-18T10:15:57.184 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.184 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.184 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:57.184 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.209378+0000", 2024-09-18T10:15:57.184 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.104483942 2024-09-18T10:15:57.184 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.184 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.184 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:57.184 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.407310+0000", 2024-09-18T10:15:57.185 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.197931406 2024-09-18T10:15:57.185 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.185 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.185 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:57.185 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.407432+0000", 2024-09-18T10:15:57.185 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00012266999999999999 2024-09-18T10:15:57.185 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:57.185 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:57.185 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:57.185 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.185 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.186 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:348 3.es0 3:7255ddf4:::smithi18925896-39:head [setxattr _header (56) in=7b] snapc b=[b,4] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-18T10:15:57.186 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:02.899423+0000", 2024-09-18T10:15:57.186 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 54.228095465000003, 2024-09-18T10:15:57.186 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.50859120400000002, 2024-09-18T10:15:57.186 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:57.186 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:57.186 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:57.186 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:57.186 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:57.186 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 348 2024-09-18T10:15:57.186 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.186 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:57.187 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.187 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:57.187 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.899423+0000", 2024-09-18T10:15:57.187 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:57.187 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.187 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.187 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:57.187 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.899423+0000", 2024-09-18T10:15:57.187 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:57.187 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.187 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.187 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:57.187 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.899422+0000", 2024-09-18T10:15:57.187 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:57.188 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.188 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.188 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:57.188 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.899431+0000", 2024-09-18T10:15:57.188 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.5960000000000004e-06 2024-09-18T10:15:57.188 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.188 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.188 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:57.189 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.899451+0000", 2024-09-18T10:15:57.189 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0089999999999999e-05 2024-09-18T10:15:57.189 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.189 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.189 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:57.189 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.899461+0000", 2024-09-18T10:15:57.189 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0373e-05 2024-09-18T10:15:57.189 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.189 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.189 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:57.189 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.106191+0000", 2024-09-18T10:15:57.189 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20672991700000001 2024-09-18T10:15:57.190 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.190 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.190 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:57.190 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.106370+0000", 2024-09-18T10:15:57.190 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00017880500000000001 2024-09-18T10:15:57.190 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.190 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.190 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:57.190 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.107175+0000", 2024-09-18T10:15:57.190 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00080489699999999995 2024-09-18T10:15:57.190 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.190 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.190 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:57.190 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.209383+0000", 2024-09-18T10:15:57.191 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10220804999999999 2024-09-18T10:15:57.191 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.191 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.191 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:57.191 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.407801+0000", 2024-09-18T10:15:57.191 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.19841824199999999 2024-09-18T10:15:57.191 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.191 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.191 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:57.191 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.408015+0000", 2024-09-18T10:15:57.191 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00021364100000000001 2024-09-18T10:15:57.192 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:57.192 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:57.192 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:57.192 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.192 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.192 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:349 3.es0 3:7255ddf4:::smithi18925896-39:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e47)", 2024-09-18T10:15:57.192 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:02.899474+0000", 2024-09-18T10:15:57.192 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 54.228045254000001, 2024-09-18T10:15:57.192 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.81294787599999996, 2024-09-18T10:15:57.192 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:57.192 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:15:57.192 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:57.193 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:57.193 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:57.193 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 349 2024-09-18T10:15:57.193 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.193 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:57.193 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.193 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:57.193 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.899474+0000", 2024-09-18T10:15:57.193 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:57.193 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.193 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.193 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:57.193 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.899474+0000", 2024-09-18T10:15:57.193 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:57.194 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.194 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.194 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:57.194 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.899472+0000", 2024-09-18T10:15:57.194 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:57.194 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.194 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.194 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:57.194 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.899480+0000", 2024-09-18T10:15:57.194 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.0129999999999993e-06 2024-09-18T10:15:57.194 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.194 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.195 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:57.195 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.899501+0000", 2024-09-18T10:15:57.195 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0959999999999999e-05 2024-09-18T10:15:57.195 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.195 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.195 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:57.195 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.899513+0000", 2024-09-18T10:15:57.195 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1736000000000001e-05 2024-09-18T10:15:57.195 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.195 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.195 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:57.195 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.208436+0000", 2024-09-18T10:15:57.195 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.30892317800000002 2024-09-18T10:15:57.195 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.196 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.196 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for rw locks", 2024-09-18T10:15:57.196 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.208762+0000", 2024-09-18T10:15:57.196 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00032566099999999998 2024-09-18T10:15:57.196 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.196 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.196 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:57.196 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.408786+0000", 2024-09-18T10:15:57.196 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.200024389 2024-09-18T10:15:57.196 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.196 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.196 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:57.196 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.409088+0000", 2024-09-18T10:15:57.197 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000301272 2024-09-18T10:15:57.197 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.197 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.197 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:57.197 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.712422+0000", 2024-09-18T10:15:57.197 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.303333937 2024-09-18T10:15:57.197 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:57.197 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:57.197 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:57.197 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.197 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.197 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.17s1 57/40 ECSubWrite(tid=106, reqid=client.4632.0:441, at_version=57'22, trim_to=0'0, roll_forward_to=53'18))", 2024-09-18T10:15:57.197 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:12.963189+0000", 2024-09-18T10:15:57.198 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 44.164329537999997, 2024-09-18T10:15:57.198 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.404840753, 2024-09-18T10:15:57.198 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:57.198 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:57.198 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:57.198 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.198 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:57.198 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:12.963189+0000", 2024-09-18T10:15:57.198 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:57.198 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.198 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.198 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:57.198 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:12.963189+0000", 2024-09-18T10:15:57.199 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:57.199 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.199 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.199 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:57.199 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:12.963189+0000", 2024-09-18T10:15:57.199 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:15:57.199 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.199 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.199 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:57.199 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:12.963511+0000", 2024-09-18T10:15:57.199 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00032195800000000001 2024-09-18T10:15:57.199 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.199 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.200 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:57.200 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:12.963536+0000", 2024-09-18T10:15:57.200 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5795e-05 2024-09-18T10:15:57.200 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.200 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.200 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:57.200 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:12.963548+0000", 2024-09-18T10:15:57.200 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1235e-05 2024-09-18T10:15:57.200 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.200 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.200 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:57.200 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.164194+0000", 2024-09-18T10:15:57.200 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.200646044 2024-09-18T10:15:57.200 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.201 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.201 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:57.201 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.164208+0000", 2024-09-18T10:15:57.201 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4107000000000001e-05 2024-09-18T10:15:57.201 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.201 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.201 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:57.201 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.367962+0000", 2024-09-18T10:15:57.201 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20375375800000001 2024-09-18T10:15:57.201 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.201 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.201 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:57.201 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.368030+0000", 2024-09-18T10:15:57.202 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.8664999999999994e-05 2024-09-18T10:15:57.202 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:57.202 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:57.202 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:57.202 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.202 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.202 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.17s1 57/40 ECSubWrite(tid=107, reqid=client.4632.0:442, at_version=57'23, trim_to=0'0, roll_forward_to=53'18))", 2024-09-18T10:15:57.202 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:12.963562+0000", 2024-09-18T10:15:57.202 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 44.163956818999999, 2024-09-18T10:15:57.202 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40594254800000001, 2024-09-18T10:15:57.202 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:57.203 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:57.203 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:57.203 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.203 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:57.203 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:12.963562+0000", 2024-09-18T10:15:57.203 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:57.203 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.203 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.203 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:57.203 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:12.963562+0000", 2024-09-18T10:15:57.203 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:57.203 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.204 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.204 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:57.204 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:12.963561+0000", 2024-09-18T10:15:57.204 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:57.204 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.204 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.204 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:57.204 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:12.963869+0000", 2024-09-18T10:15:57.204 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00030785799999999999 2024-09-18T10:15:57.204 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.204 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.204 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:57.205 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:12.963886+0000", 2024-09-18T10:15:57.205 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6582000000000001e-05 2024-09-18T10:15:57.205 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.205 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.205 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:57.205 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.063959+0000", 2024-09-18T10:15:57.205 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10007302699999999 2024-09-18T10:15:57.205 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.205 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.205 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:57.205 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.166049+0000", 2024-09-18T10:15:57.205 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.102089845 2024-09-18T10:15:57.205 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.206 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.206 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:57.206 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.166064+0000", 2024-09-18T10:15:57.206 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5287000000000001e-05 2024-09-18T10:15:57.206 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.206 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.206 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:57.206 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.369452+0000", 2024-09-18T10:15:57.206 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20338777 2024-09-18T10:15:57.206 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.206 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.206 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:57.207 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.369505+0000", 2024-09-18T10:15:57.207 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.2868000000000003e-05 2024-09-18T10:15:57.207 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:57.207 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:57.207 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:57.207 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.207 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.207 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:495 3.es0 3:7255ddf4:::smithi18925896-39:head [rmxattr R^`al]TqifatYOiOU]p`ZegfXmNsXVaZcdd]b]YX in=40b,rmxattr TNQYR^jgR`pPaYitmrcliV[dttobncTntcfTlQkt in=40b,rmxattr W_aq[RVsaUpQtX_RpV^Vpqrg`QgQViPeVaRqZSmf in=40b,rmxattr aYfgr^q[\\aPg_aZsakfUfsdhYbra^QXjlkrpdY\\\\ in=40b,rmxattr dgXPi in=5b,rmxattr p]OS_Znl[dNa^qTn`WW[j\\dtgbNac_bijYmg in=36b,rmxattr tZ]grhlaq]kUO[k^d^dr``XNio^^Oaebpj\\[Ytdq in=40b] snapc 19=[19,18,16,12,11,10,f,e] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T10:15:57.207 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:17.889335+0000", 2024-09-18T10:15:57.207 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 39.238183585999998, 2024-09-18T10:15:57.207 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.42846536899999998, 2024-09-18T10:15:57.207 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:57.208 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:57.208 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:57.208 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:57.208 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:57.208 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 495 2024-09-18T10:15:57.208 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.208 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:57.208 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.208 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:57.208 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889335+0000", 2024-09-18T10:15:57.208 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:57.208 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.208 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.209 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:57.209 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889335+0000", 2024-09-18T10:15:57.209 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:57.209 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.209 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.209 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:57.209 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889333+0000", 2024-09-18T10:15:57.209 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999976 2024-09-18T10:15:57.209 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.209 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.209 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:57.209 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889351+0000", 2024-09-18T10:15:57.210 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7795000000000002e-05 2024-09-18T10:15:57.210 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.210 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.210 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:57.210 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889383+0000", 2024-09-18T10:15:57.210 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2740999999999997e-05 2024-09-18T10:15:57.210 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.210 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.210 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:57.210 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889396+0000", 2024-09-18T10:15:57.210 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2926e-05 2024-09-18T10:15:57.210 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.211 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.211 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:57.211 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.989725+0000", 2024-09-18T10:15:57.211 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10032845999999999 2024-09-18T10:15:57.211 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.211 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.211 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "op must wait for map", 2024-09-18T10:15:57.211 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.989751+0000", 2024-09-18T10:15:57.211 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5819000000000001e-05 2024-09-18T10:15:57.211 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.211 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.211 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:57.212 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.999619+0000", 2024-09-18T10:15:57.212 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0098683920000000001 2024-09-18T10:15:57.212 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.212 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.212 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:57.212 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.999776+0000", 2024-09-18T10:15:57.212 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000157336 2024-09-18T10:15:57.212 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.212 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.212 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:57.212 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.001374+0000", 2024-09-18T10:15:57.212 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0015982419999999999 2024-09-18T10:15:57.212 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.213 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.213 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:57.213 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.003908+0000", 2024-09-18T10:15:57.213 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0025331870000000001 2024-09-18T10:15:57.213 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.213 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.213 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:57.213 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.317708+0000", 2024-09-18T10:15:57.213 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.31380064800000002 2024-09-18T10:15:57.213 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.213 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.213 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:57.213 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.317801+0000", 2024-09-18T10:15:57.214 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.2427000000000001e-05 2024-09-18T10:15:57.214 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:57.214 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:57.214 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:57.214 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.214 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.214 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:646 3.es0 3:78aad6f5:::smithi18925896-49:head [append 0~0] snapc 21=[20,1f,1e,1a,16,11,10] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-18T10:15:57.214 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:29.784316+0000", 2024-09-18T10:15:57.214 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 27.343202504000001, 2024-09-18T10:15:57.214 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.38893830699999998, 2024-09-18T10:15:57.214 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:57.214 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:57.215 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:57.215 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:57.215 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:57.215 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 646 2024-09-18T10:15:57.215 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.215 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:57.215 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.215 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:57.215 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:29.784316+0000", 2024-09-18T10:15:57.215 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:57.215 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.215 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.215 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:57.216 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:29.784316+0000", 2024-09-18T10:15:57.216 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:57.216 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.216 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.216 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:57.216 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:29.784314+0000", 2024-09-18T10:15:57.216 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999976 2024-09-18T10:15:57.216 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.216 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.216 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:57.216 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:29.784325+0000", 2024-09-18T10:15:57.216 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0869e-05 2024-09-18T10:15:57.217 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.217 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.217 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:57.217 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:29.784359+0000", 2024-09-18T10:15:57.217 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4496000000000003e-05 2024-09-18T10:15:57.217 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.217 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.217 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:57.217 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:29.784374+0000", 2024-09-18T10:15:57.217 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4945e-05 2024-09-18T10:15:57.217 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.217 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.218 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:57.218 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:29.870174+0000", 2024-09-18T10:15:57.218 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.085799399999999998 2024-09-18T10:15:57.218 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.218 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.218 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:57.218 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:29.870387+0000", 2024-09-18T10:15:57.218 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00021327799999999999 2024-09-18T10:15:57.218 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.218 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.218 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:57.218 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:29.871229+0000", 2024-09-18T10:15:57.219 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00084239500000000001 2024-09-18T10:15:57.219 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.219 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.219 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:57.219 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.172680+0000", 2024-09-18T10:15:57.219 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.301450944 2024-09-18T10:15:57.219 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.219 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.219 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:57.219 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.173021+0000", 2024-09-18T10:15:57.219 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00034057199999999998 2024-09-18T10:15:57.219 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.219 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.220 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:57.220 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.173255+0000", 2024-09-18T10:15:57.220 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000233945 2024-09-18T10:15:57.220 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:57.220 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:57.220 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:57.220 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.220 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.220 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:647 3.es0 3:78aad6f5:::smithi18925896-49:head [rollback 10] snapc 21=[20,1f,1e,1a,16,11,10] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-18T10:15:57.220 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:29.784393+0000", 2024-09-18T10:15:57.220 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 27.343125591, 2024-09-18T10:15:57.220 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.38955653200000001, 2024-09-18T10:15:57.221 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:57.221 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:57.221 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:57.221 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:57.221 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:57.221 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 647 2024-09-18T10:15:57.221 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.221 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:57.221 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.221 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:57.221 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:29.784393+0000", 2024-09-18T10:15:57.221 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:57.221 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.221 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.222 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:57.222 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:29.784393+0000", 2024-09-18T10:15:57.222 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:57.222 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.222 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.222 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:57.222 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:29.784393+0000", 2024-09-18T10:15:57.222 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:15:57.222 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.222 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.222 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:57.222 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:29.784397+0000", 2024-09-18T10:15:57.222 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4290000000000003e-06 2024-09-18T10:15:57.223 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.223 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.223 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:57.223 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:29.784409+0000", 2024-09-18T10:15:57.223 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2208000000000001e-05 2024-09-18T10:15:57.223 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.223 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.223 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:57.223 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:29.784418+0000", 2024-09-18T10:15:57.223 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.1060000000000006e-06 2024-09-18T10:15:57.223 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.223 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.223 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:57.223 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:29.871905+0000", 2024-09-18T10:15:57.224 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.087486918999999996 2024-09-18T10:15:57.224 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.224 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.224 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:57.224 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:29.872024+0000", 2024-09-18T10:15:57.224 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000119152 2024-09-18T10:15:57.224 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.224 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.224 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:57.224 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:29.872743+0000", 2024-09-18T10:15:57.224 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00071839500000000004 2024-09-18T10:15:57.224 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.225 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.225 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:57.225 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.172683+0000", 2024-09-18T10:15:57.225 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.29994042399999998 2024-09-18T10:15:57.225 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.225 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.225 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:57.225 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.173719+0000", 2024-09-18T10:15:57.225 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0010353199999999999 2024-09-18T10:15:57.225 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.225 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.225 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:57.225 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.173950+0000", 2024-09-18T10:15:57.226 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00023138499999999999 2024-09-18T10:15:57.226 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:57.226 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:57.226 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:57.226 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.226 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.226 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:648 3.es0 3:78aad6f5:::smithi18925896-49:head [append 0~0] snapc 21=[20,1f,1e,1a,16,11,10] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-18T10:15:57.226 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:29.784506+0000", 2024-09-18T10:15:57.226 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 27.343012927, 2024-09-18T10:15:57.226 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.79270480700000001, 2024-09-18T10:15:57.226 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:57.226 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:57.226 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:57.227 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:57.227 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:57.227 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 648 2024-09-18T10:15:57.227 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.227 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:57.227 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.227 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:57.227 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:29.784506+0000", 2024-09-18T10:15:57.227 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:57.227 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.227 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.227 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:57.227 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:29.784506+0000", 2024-09-18T10:15:57.228 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:57.228 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.228 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.228 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:57.228 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:29.784505+0000", 2024-09-18T10:15:57.228 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:15:57.228 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.228 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.228 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:57.228 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:29.784510+0000", 2024-09-18T10:15:57.228 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9540000000000003e-06 2024-09-18T10:15:57.228 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.229 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.229 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:57.229 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:29.784523+0000", 2024-09-18T10:15:57.229 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2791e-05 2024-09-18T10:15:57.229 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.229 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.229 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:57.229 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:29.884663+0000", 2024-09-18T10:15:57.229 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10014081499999999 2024-09-18T10:15:57.229 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.229 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.229 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:57.229 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.175196+0000", 2024-09-18T10:15:57.230 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.29053214300000002 2024-09-18T10:15:57.230 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.230 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.230 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:57.230 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.175436+0000", 2024-09-18T10:15:57.230 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000240084 2024-09-18T10:15:57.230 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.230 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.230 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:57.230 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.176117+0000", 2024-09-18T10:15:57.230 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00068175400000000002 2024-09-18T10:15:57.230 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.230 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.231 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:57.231 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.277609+0000", 2024-09-18T10:15:57.231 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10149114100000001 2024-09-18T10:15:57.231 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.231 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.231 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:57.231 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.577146+0000", 2024-09-18T10:15:57.231 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.29953759600000002 2024-09-18T10:15:57.231 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.231 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.231 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:57.231 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.577211+0000", 2024-09-18T10:15:57.232 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.4630999999999998e-05 2024-09-18T10:15:57.232 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:57.232 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:57.232 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:57.232 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.232 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.232 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:694 3.es0 3:78aad6f5:::smithi18925896-49:head [stat,delete] snapc 2a=[2a,29,28,26,20,1f,1e,1a,11] ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-09-18T10:15:57.232 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:35.579838+0000", 2024-09-18T10:15:57.232 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 21.547681072, 2024-09-18T10:15:57.232 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40589915300000001, 2024-09-18T10:15:57.232 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:57.232 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:57.233 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:57.233 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:57.233 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:57.233 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 694 2024-09-18T10:15:57.233 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.233 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:57.233 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.233 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:57.233 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:35.579838+0000", 2024-09-18T10:15:57.233 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:57.233 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.233 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.234 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:57.234 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:35.579838+0000", 2024-09-18T10:15:57.234 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:57.234 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.234 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.234 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:57.234 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:35.579835+0000", 2024-09-18T10:15:57.234 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999971 2024-09-18T10:15:57.234 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.234 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.234 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:57.234 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:35.579849+0000", 2024-09-18T10:15:57.235 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3786e-05 2024-09-18T10:15:57.235 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.235 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.235 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:57.235 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:35.579886+0000", 2024-09-18T10:15:57.235 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7183e-05 2024-09-18T10:15:57.235 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.235 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.235 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:57.235 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:35.680021+0000", 2024-09-18T10:15:57.235 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10013464599999999 2024-09-18T10:15:57.236 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.236 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.236 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:57.236 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:35.686371+0000", 2024-09-18T10:15:57.236 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0063500789999999998 2024-09-18T10:15:57.236 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.236 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.236 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "op must wait for map", 2024-09-18T10:15:57.236 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:35.686388+0000", 2024-09-18T10:15:57.236 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7662000000000001e-05 2024-09-18T10:15:57.236 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.236 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.236 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:57.237 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:35.880602+0000", 2024-09-18T10:15:57.237 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.19421374999999999 2024-09-18T10:15:57.237 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.237 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.237 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:57.237 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:35.880897+0000", 2024-09-18T10:15:57.237 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00029453999999999998 2024-09-18T10:15:57.237 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.237 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.237 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:57.237 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:35.882269+0000", 2024-09-18T10:15:57.237 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00137261 2024-09-18T10:15:57.237 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.238 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.238 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:57.238 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:35.884030+0000", 2024-09-18T10:15:57.238 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0017602449999999999 2024-09-18T10:15:57.238 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.238 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.238 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:57.238 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:35.985612+0000", 2024-09-18T10:15:57.238 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.101582165 2024-09-18T10:15:57.238 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.238 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.238 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:57.238 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:35.985737+0000", 2024-09-18T10:15:57.238 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00012524699999999999 2024-09-18T10:15:57.239 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:57.239 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:57.239 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:57.239 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:57.239 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:57.239 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-18T10:15:57.241 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_ops_in_flight 2024-09-18T10:15:57.241 INFO:tasks.ceph.osd.14.smithi154.stderr:2024-09-18T10:15:57.238+0000 7f75a19e7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 22093) UID: 0 2024-09-18T10:15:57.341 INFO:tasks.ceph.osd.8.smithi029.stderr:2024-09-18T10:15:57.333+0000 7f2555c27700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 22192) UID: 0 2024-09-18T10:15:57.350 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-18T10:15:57.350 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-18T10:15:57.350 INFO:teuthology.orchestra.run.smithi189.stdout: "num_ops": 0 2024-09-18T10:15:57.351 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-18T10:15:57.360 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_blocked_ops 2024-09-18T10:15:57.442 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-18T10:15:57.439+0000 7f1981873700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22132) UID: 0 2024-09-18T10:15:57.458 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-18T10:15:57.458 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-18T10:15:57.458 INFO:teuthology.orchestra.run.smithi189.stdout: "complaint_time": 30, 2024-09-18T10:15:57.458 INFO:teuthology.orchestra.run.smithi189.stdout: "num_blocked_ops": 0 2024-09-18T10:15:57.459 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-18T10:15:57.467 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_historic_ops 2024-09-18T10:15:57.529 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T10:15:57.543 INFO:tasks.ceph.osd.10.smithi154.stderr:2024-09-18T10:15:57.538+0000 7fb73221a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 22091) UID: 0 2024-09-18T10:15:57.563 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-18T10:15:57.563 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 20, 2024-09-18T10:15:57.563 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 600, 2024-09-18T10:15:57.563 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [ 2024-09-18T10:15:57.563 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.563 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4257.0:19 1.0 e24/23 1:30306672:devicehealth::main.db.0000000000000000:head v 24'9, mlcod=24'7)", 2024-09-18T10:15:57.563 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:33.346192+0000", 2024-09-18T10:15:57.564 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 84.215248536999994, 2024-09-18T10:15:57.564 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20178458699999999, 2024-09-18T10:15:57.564 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:57.564 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:57.564 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:57.564 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.564 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:57.564 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:33.346192+0000", 2024-09-18T10:15:57.564 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:57.564 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.564 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.564 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:57.565 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:33.346192+0000", 2024-09-18T10:15:57.565 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:57.565 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.565 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.565 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:57.565 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:33.346190+0000", 2024-09-18T10:15:57.565 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:57.565 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.565 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.565 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:57.565 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:33.346206+0000", 2024-09-18T10:15:57.565 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6257e-05 2024-09-18T10:15:57.565 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.565 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.566 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:57.566 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:33.346237+0000", 2024-09-18T10:15:57.566 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0471999999999999e-05 2024-09-18T10:15:57.566 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.566 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.566 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:57.566 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:33.446377+0000", 2024-09-18T10:15:57.566 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10014044499999999 2024-09-18T10:15:57.566 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.567 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.567 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:57.567 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:33.546722+0000", 2024-09-18T10:15:57.567 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100344593 2024-09-18T10:15:57.567 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.567 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.567 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:57.567 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:33.546787+0000", 2024-09-18T10:15:57.567 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.5347999999999998e-05 2024-09-18T10:15:57.568 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.568 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.568 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:57.568 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:33.547891+0000", 2024-09-18T10:15:57.568 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001104379 2024-09-18T10:15:57.568 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.568 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.568 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:57.568 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:33.547976+0000", 2024-09-18T10:15:57.568 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.5023000000000002e-05 2024-09-18T10:15:57.568 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:57.568 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:57.568 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:57.569 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.569 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.569 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4257.0:48 1.0 e26/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 26'30, mlcod=26'22)", 2024-09-18T10:15:57.569 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:35.294144+0000", 2024-09-18T10:15:57.569 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 82.267296393999999, 2024-09-18T10:15:57.569 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20346468400000001, 2024-09-18T10:15:57.569 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:57.569 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:57.569 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:57.569 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.570 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:57.570 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:35.294144+0000", 2024-09-18T10:15:57.570 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:57.570 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.570 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.570 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:57.570 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:35.294144+0000", 2024-09-18T10:15:57.570 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:57.570 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.570 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.570 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:57.570 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:35.294142+0000", 2024-09-18T10:15:57.570 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:57.570 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.571 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.571 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:57.571 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:35.294156+0000", 2024-09-18T10:15:57.571 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4059e-05 2024-09-18T10:15:57.571 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.571 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.571 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:57.571 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:35.294209+0000", 2024-09-18T10:15:57.571 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.2164999999999999e-05 2024-09-18T10:15:57.571 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.572 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.572 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:57.572 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:35.394305+0000", 2024-09-18T10:15:57.572 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10009605000000001 2024-09-18T10:15:57.572 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.572 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.572 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:57.572 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:35.394422+0000", 2024-09-18T10:15:57.572 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000117093 2024-09-18T10:15:57.572 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.572 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.572 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:57.572 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:35.394460+0000", 2024-09-18T10:15:57.573 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8037999999999997e-05 2024-09-18T10:15:57.573 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.573 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.573 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:57.573 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:35.497437+0000", 2024-09-18T10:15:57.573 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.102977206 2024-09-18T10:15:57.573 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.573 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.573 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:57.573 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:35.497609+0000", 2024-09-18T10:15:57.573 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00017165299999999999 2024-09-18T10:15:57.573 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:57.574 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:57.574 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:57.574 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.574 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.574 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4257.0:96 1.0 e29/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 29'70, mlcod=29'68)", 2024-09-18T10:15:57.574 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:39.427232+0000", 2024-09-18T10:15:57.574 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 78.134208270000002, 2024-09-18T10:15:57.574 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20408627300000001, 2024-09-18T10:15:57.574 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:57.574 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:57.574 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:57.574 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.574 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:57.575 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.427232+0000", 2024-09-18T10:15:57.575 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:57.575 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.575 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.575 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:57.575 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.427232+0000", 2024-09-18T10:15:57.575 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:57.575 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.575 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.575 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:57.575 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.427230+0000", 2024-09-18T10:15:57.576 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:57.576 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.576 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.576 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:57.576 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.427399+0000", 2024-09-18T10:15:57.576 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000168787 2024-09-18T10:15:57.576 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.576 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.576 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:57.576 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.427431+0000", 2024-09-18T10:15:57.576 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2372000000000001e-05 2024-09-18T10:15:57.576 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.576 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.577 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:57.577 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.427444+0000", 2024-09-18T10:15:57.577 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2303000000000001e-05 2024-09-18T10:15:57.577 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.577 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.577 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:57.577 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.427580+0000", 2024-09-18T10:15:57.577 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000136019 2024-09-18T10:15:57.577 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.577 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.577 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:57.578 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.427622+0000", 2024-09-18T10:15:57.578 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.2551e-05 2024-09-18T10:15:57.578 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.578 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.578 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:57.578 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.631238+0000", 2024-09-18T10:15:57.578 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20361580100000001 2024-09-18T10:15:57.578 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.578 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.578 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:57.578 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.631318+0000", 2024-09-18T10:15:57.578 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.0370000000000005e-05 2024-09-18T10:15:57.579 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:57.579 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:57.579 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:57.579 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.579 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.579 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4257.0:97 1.0 e29/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 29'71, mlcod=29'68)", 2024-09-18T10:15:57.579 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:39.428800+0000", 2024-09-18T10:15:57.579 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 78.132640327999994, 2024-09-18T10:15:57.579 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.202566529, 2024-09-18T10:15:57.579 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:57.579 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:57.579 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:57.579 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.579 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:57.580 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.428800+0000", 2024-09-18T10:15:57.580 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:57.580 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.580 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.580 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:57.580 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.428800+0000", 2024-09-18T10:15:57.580 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:57.580 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.580 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.580 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:57.581 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.428798+0000", 2024-09-18T10:15:57.581 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:57.581 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.581 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.581 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:57.581 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.428814+0000", 2024-09-18T10:15:57.581 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5262999999999999e-05 2024-09-18T10:15:57.581 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.581 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.581 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:57.581 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.428836+0000", 2024-09-18T10:15:57.581 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.2096e-05 2024-09-18T10:15:57.581 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.581 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.582 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:57.582 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.428847+0000", 2024-09-18T10:15:57.582 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1272e-05 2024-09-18T10:15:57.582 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.582 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.582 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:57.582 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.529081+0000", 2024-09-18T10:15:57.582 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100233853 2024-09-18T10:15:57.582 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.582 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.582 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:57.582 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.529115+0000", 2024-09-18T10:15:57.582 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3862000000000002e-05 2024-09-18T10:15:57.583 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.583 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.583 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:57.583 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.631329+0000", 2024-09-18T10:15:57.583 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.102214465 2024-09-18T10:15:57.583 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.583 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.583 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:57.583 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.631367+0000", 2024-09-18T10:15:57.583 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7450999999999998e-05 2024-09-18T10:15:57.583 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:57.583 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:57.583 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:57.583 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.584 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.584 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4257.0:113 1.0 e29/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 29'83, mlcod=29'80)", 2024-09-18T10:15:57.584 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:40.801786+0000", 2024-09-18T10:15:57.584 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 76.759654818000001, 2024-09-18T10:15:57.584 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.203408961, 2024-09-18T10:15:57.584 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:57.584 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:57.584 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:57.584 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.584 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:57.584 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.801786+0000", 2024-09-18T10:15:57.584 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:57.584 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.584 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.585 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:57.585 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.801786+0000", 2024-09-18T10:15:57.585 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:57.585 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.585 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.585 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:57.585 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.801784+0000", 2024-09-18T10:15:57.585 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:57.585 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.585 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.585 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:57.585 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.801801+0000", 2024-09-18T10:15:57.585 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7223000000000001e-05 2024-09-18T10:15:57.586 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.586 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.586 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:57.586 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.801823+0000", 2024-09-18T10:15:57.586 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1469000000000001e-05 2024-09-18T10:15:57.586 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.586 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.586 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:57.586 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.801835+0000", 2024-09-18T10:15:57.586 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2033e-05 2024-09-18T10:15:57.586 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.586 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.586 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:57.586 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.902049+0000", 2024-09-18T10:15:57.587 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100213659 2024-09-18T10:15:57.587 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.587 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.587 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:57.587 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.902082+0000", 2024-09-18T10:15:57.587 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3668999999999999e-05 2024-09-18T10:15:57.587 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.587 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.587 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:57.587 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.005131+0000", 2024-09-18T10:15:57.587 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.103048379 2024-09-18T10:15:57.587 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.587 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.588 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:57.588 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.005195+0000", 2024-09-18T10:15:57.588 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.3805000000000003e-05 2024-09-18T10:15:57.588 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:57.588 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:57.588 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:57.588 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.589 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.589 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4257.0:114 1.0 e29/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 29'84, mlcod=29'80)", 2024-09-18T10:15:57.589 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:40.802657+0000", 2024-09-18T10:15:57.589 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 76.758783683999994, 2024-09-18T10:15:57.589 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20256802600000001, 2024-09-18T10:15:57.589 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:57.589 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:57.589 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:57.589 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.590 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:57.590 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.802657+0000", 2024-09-18T10:15:57.590 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:57.590 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.590 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.590 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:57.590 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.802657+0000", 2024-09-18T10:15:57.590 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:57.590 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.590 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.591 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:57.591 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.802656+0000", 2024-09-18T10:15:57.591 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:15:57.591 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.591 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.591 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:57.591 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.802824+0000", 2024-09-18T10:15:57.591 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000168784 2024-09-18T10:15:57.591 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.591 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.591 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:57.591 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.802844+0000", 2024-09-18T10:15:57.591 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0058999999999999e-05 2024-09-18T10:15:57.591 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.592 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.592 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:57.592 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.802857+0000", 2024-09-18T10:15:57.592 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2163e-05 2024-09-18T10:15:57.592 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.592 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.592 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:57.592 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.902531+0000", 2024-09-18T10:15:57.592 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.099674591000000007 2024-09-18T10:15:57.592 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.592 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.592 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:57.592 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.902559+0000", 2024-09-18T10:15:57.593 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7483000000000001e-05 2024-09-18T10:15:57.593 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.593 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.593 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:57.593 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.005198+0000", 2024-09-18T10:15:57.593 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.102638853 2024-09-18T10:15:57.593 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.593 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.593 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:57.593 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.005225+0000", 2024-09-18T10:15:57.593 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7206e-05 2024-09-18T10:15:57.593 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:57.594 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:57.594 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:57.594 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.594 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.594 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4257.0:128 1.0 e29/23 1:30306672:devicehealth::main.db.0000000000000000:head v 29'98, mlcod=29'95)", 2024-09-18T10:15:57.594 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:41.222806+0000", 2024-09-18T10:15:57.594 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 76.338634417999998, 2024-09-18T10:15:57.594 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20200015099999999, 2024-09-18T10:15:57.594 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:57.594 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:57.594 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:57.594 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.594 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:57.594 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.222806+0000", 2024-09-18T10:15:57.595 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:57.595 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.595 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.595 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:57.595 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.222806+0000", 2024-09-18T10:15:57.595 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:57.595 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.595 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.595 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:57.595 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.222804+0000", 2024-09-18T10:15:57.595 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:57.595 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.595 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.595 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:57.596 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.222885+0000", 2024-09-18T10:15:57.596 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.0140000000000002e-05 2024-09-18T10:15:57.596 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.596 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.596 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:57.596 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.222915+0000", 2024-09-18T10:15:57.596 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0168999999999999e-05 2024-09-18T10:15:57.596 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.596 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.596 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:57.596 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.323060+0000", 2024-09-18T10:15:57.596 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100145475 2024-09-18T10:15:57.596 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.597 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.597 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:57.597 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.423304+0000", 2024-09-18T10:15:57.597 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100243924 2024-09-18T10:15:57.597 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.597 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.597 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:57.597 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.423338+0000", 2024-09-18T10:15:57.597 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3819000000000003e-05 2024-09-18T10:15:57.597 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.597 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.597 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:57.597 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.424724+0000", 2024-09-18T10:15:57.598 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0013862690000000001 2024-09-18T10:15:57.598 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.598 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.598 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:57.598 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.424806+0000", 2024-09-18T10:15:57.598 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.1901000000000001e-05 2024-09-18T10:15:57.598 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:57.598 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:57.598 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:57.598 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.598 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.598 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.4s1 41/40 ECSubWrite(tid=17, reqid=osd.8.0:17, at_version=41'10, trim_to=0'0, roll_forward_to=0'0))", 2024-09-18T10:15:57.599 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:57.445149+0000", 2024-09-18T10:15:57.599 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 60.116291855999997, 2024-09-18T10:15:57.599 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20394425099999999, 2024-09-18T10:15:57.599 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:57.599 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:57.599 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:57.599 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.599 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:57.599 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.445149+0000", 2024-09-18T10:15:57.599 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:57.599 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.599 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.599 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:57.599 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.445149+0000", 2024-09-18T10:15:57.600 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:57.600 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.600 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.600 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:57.600 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.445147+0000", 2024-09-18T10:15:57.600 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:57.600 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.600 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.600 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:57.600 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.445221+0000", 2024-09-18T10:15:57.600 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.4140999999999994e-05 2024-09-18T10:15:57.600 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.600 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.600 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:57.601 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.445260+0000", 2024-09-18T10:15:57.601 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8998999999999997e-05 2024-09-18T10:15:57.601 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.601 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.601 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:57.601 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.545386+0000", 2024-09-18T10:15:57.601 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10012621300000001 2024-09-18T10:15:57.601 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.601 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.601 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:57.601 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.645963+0000", 2024-09-18T10:15:57.601 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10057662000000001 2024-09-18T10:15:57.601 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.602 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.602 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:57.602 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.645988+0000", 2024-09-18T10:15:57.602 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5449e-05 2024-09-18T10:15:57.602 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.602 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.602 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:57.602 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.649028+0000", 2024-09-18T10:15:57.602 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0030390840000000001 2024-09-18T10:15:57.602 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.602 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.602 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:57.602 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.649093+0000", 2024-09-18T10:15:57.602 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.5258000000000004e-05 2024-09-18T10:15:57.603 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:57.603 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:57.603 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:57.603 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.603 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.603 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.6s6 41/40 ECSubWrite(tid=31, reqid=client.4632.0:249, at_version=41'13, trim_to=0'0, roll_forward_to=0'0))", 2024-09-18T10:15:57.603 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:57.550336+0000", 2024-09-18T10:15:57.603 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 60.011104817000003, 2024-09-18T10:15:57.603 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.203590574, 2024-09-18T10:15:57.603 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:57.603 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:57.603 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:57.603 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.603 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:57.603 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.550336+0000", 2024-09-18T10:15:57.604 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:57.604 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.604 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.604 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:57.604 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.550336+0000", 2024-09-18T10:15:57.604 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:57.604 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.604 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.604 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:57.604 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.550329+0000", 2024-09-18T10:15:57.604 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999933 2024-09-18T10:15:57.604 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.604 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.604 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:57.605 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.550932+0000", 2024-09-18T10:15:57.605 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00060276999999999998 2024-09-18T10:15:57.605 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.605 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.605 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:57.605 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.550955+0000", 2024-09-18T10:15:57.605 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.2884999999999999e-05 2024-09-18T10:15:57.605 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.605 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.605 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:57.605 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.651052+0000", 2024-09-18T10:15:57.605 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100097174 2024-09-18T10:15:57.605 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.605 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.606 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:57.606 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.651160+0000", 2024-09-18T10:15:57.606 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000108483 2024-09-18T10:15:57.606 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.606 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.606 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:57.606 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.651172+0000", 2024-09-18T10:15:57.606 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1790999999999999e-05 2024-09-18T10:15:57.606 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.606 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.606 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:57.606 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.753875+0000", 2024-09-18T10:15:57.606 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10270337 2024-09-18T10:15:57.607 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.607 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.607 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:57.607 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.753926+0000", 2024-09-18T10:15:57.607 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0767999999999999e-05 2024-09-18T10:15:57.607 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:57.607 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:57.607 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:57.607 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.607 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.607 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.8s4 42/40 ECSubWrite(tid=133, reqid=osd.6.0:132, at_version=0'0, trim_to=0'0, roll_forward_to=41'13))", 2024-09-18T10:15:57.607 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:58.049763+0000", 2024-09-18T10:15:57.607 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 59.511677081000002, 2024-09-18T10:15:57.608 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.21446268399999999, 2024-09-18T10:15:57.608 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:57.608 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:57.608 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:57.608 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.608 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:57.608 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:58.049763+0000", 2024-09-18T10:15:57.608 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:57.608 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.608 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.608 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:57.608 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:58.049763+0000", 2024-09-18T10:15:57.608 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:57.609 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.609 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.609 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:57.609 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:58.049763+0000", 2024-09-18T10:15:57.609 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:57.609 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.609 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.609 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:57.609 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:58.049787+0000", 2024-09-18T10:15:57.609 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3819e-05 2024-09-18T10:15:57.609 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.609 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.609 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:57.610 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:58.049817+0000", 2024-09-18T10:15:57.610 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0749e-05 2024-09-18T10:15:57.610 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.610 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.610 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:57.610 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:58.049836+0000", 2024-09-18T10:15:57.610 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9182999999999999e-05 2024-09-18T10:15:57.610 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.610 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.610 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:57.610 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:58.163409+0000", 2024-09-18T10:15:57.610 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.11357204899999999 2024-09-18T10:15:57.610 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.611 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.611 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:57.611 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:58.163424+0000", 2024-09-18T10:15:57.611 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5403000000000001e-05 2024-09-18T10:15:57.611 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.611 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.611 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:57.611 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:58.264161+0000", 2024-09-18T10:15:57.611 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100736942 2024-09-18T10:15:57.611 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.611 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.611 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:57.611 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:58.264226+0000", 2024-09-18T10:15:57.612 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.5085000000000005e-05 2024-09-18T10:15:57.612 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:57.612 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:57.612 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:57.612 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.612 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.612 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.ds3 47/32 ECSubWrite(tid=56, reqid=client.4632.0:333, at_version=47'9, trim_to=0'0, roll_forward_to=41'6))", 2024-09-18T10:15:57.612 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:02.801124+0000", 2024-09-18T10:15:57.612 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 54.760315923, 2024-09-18T10:15:57.612 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20185083000000001, 2024-09-18T10:15:57.612 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:57.612 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:57.612 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:57.613 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.613 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:57.613 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.801124+0000", 2024-09-18T10:15:57.613 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:57.613 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.613 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.613 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:57.613 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.801124+0000", 2024-09-18T10:15:57.613 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:57.613 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.613 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.613 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:57.613 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.801103+0000", 2024-09-18T10:15:57.614 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999785 2024-09-18T10:15:57.614 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.614 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.614 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:57.614 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.801366+0000", 2024-09-18T10:15:57.614 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00026269299999999999 2024-09-18T10:15:57.614 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.614 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.614 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:57.614 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.801395+0000", 2024-09-18T10:15:57.614 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.9572000000000001e-05 2024-09-18T10:15:57.614 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.614 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.615 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:57.615 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.801412+0000", 2024-09-18T10:15:57.615 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6872999999999999e-05 2024-09-18T10:15:57.615 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.615 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.615 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:57.615 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.000543+0000", 2024-09-18T10:15:57.615 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.19913046700000001 2024-09-18T10:15:57.615 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.615 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.615 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:57.615 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.000579+0000", 2024-09-18T10:15:57.615 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6649000000000001e-05 2024-09-18T10:15:57.616 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.616 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.616 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:57.616 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.002922+0000", 2024-09-18T10:15:57.616 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0023426139999999998 2024-09-18T10:15:57.616 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.616 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.616 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:57.616 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.002975+0000", 2024-09-18T10:15:57.616 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3232000000000001e-05 2024-09-18T10:15:57.616 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:57.616 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:57.616 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:57.616 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.617 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.617 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.15s1 63/60 ECSubWrite(tid=54, reqid=client.4632.0:512, at_version=0'0, trim_to=0'0, roll_forward_to=63'4))", 2024-09-18T10:15:57.617 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:19.215356+0000", 2024-09-18T10:15:57.617 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 38.346084060999999, 2024-09-18T10:15:57.617 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20187551500000001, 2024-09-18T10:15:57.617 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:57.617 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:57.617 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:57.617 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.617 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:57.617 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.215356+0000", 2024-09-18T10:15:57.617 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:57.617 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.617 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.618 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:57.618 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.215356+0000", 2024-09-18T10:15:57.618 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:57.618 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.618 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.618 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:57.618 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.215355+0000", 2024-09-18T10:15:57.618 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:57.618 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.618 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.618 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:57.618 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.215386+0000", 2024-09-18T10:15:57.618 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0809000000000003e-05 2024-09-18T10:15:57.618 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.619 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.619 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:57.619 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.215434+0000", 2024-09-18T10:15:57.619 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.8751999999999997e-05 2024-09-18T10:15:57.619 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.619 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.619 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:57.619 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.315532+0000", 2024-09-18T10:15:57.619 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100097281 2024-09-18T10:15:57.619 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.619 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.619 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:57.619 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.415795+0000", 2024-09-18T10:15:57.619 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10026387 2024-09-18T10:15:57.620 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.620 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.620 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:57.620 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.415823+0000", 2024-09-18T10:15:57.620 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7469000000000001e-05 2024-09-18T10:15:57.620 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.620 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.620 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:57.620 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.417173+0000", 2024-09-18T10:15:57.620 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0013505030000000001 2024-09-18T10:15:57.620 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.620 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.620 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:57.620 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.417232+0000", 2024-09-18T10:15:57.621 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8421000000000002e-05 2024-09-18T10:15:57.621 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:57.621 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:57.621 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:57.621 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.621 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.621 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.4s1 72/40 ECSubWrite(tid=72, reqid=osd.8.0:72, at_version=72'53, trim_to=0'0, roll_forward_to=71'51))", 2024-09-18T10:15:57.621 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:28.190950+0000", 2024-09-18T10:15:57.621 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 29.370490236999999, 2024-09-18T10:15:57.621 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20228795599999999, 2024-09-18T10:15:57.621 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:57.621 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:57.621 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:57.621 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.622 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:57.622 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.190950+0000", 2024-09-18T10:15:57.622 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:57.622 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.622 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.622 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:57.622 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.190950+0000", 2024-09-18T10:15:57.622 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:57.622 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.622 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.622 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:57.622 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.190949+0000", 2024-09-18T10:15:57.622 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:57.622 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.623 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.623 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:57.623 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.191001+0000", 2024-09-18T10:15:57.623 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.2581999999999999e-05 2024-09-18T10:15:57.623 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.623 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.623 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:57.623 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.191022+0000", 2024-09-18T10:15:57.623 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0792999999999999e-05 2024-09-18T10:15:57.623 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.623 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.623 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:57.623 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.191034+0000", 2024-09-18T10:15:57.623 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1603000000000001e-05 2024-09-18T10:15:57.623 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.624 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.624 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:57.624 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.391648+0000", 2024-09-18T10:15:57.624 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20061451299999999 2024-09-18T10:15:57.624 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.624 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.624 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:57.624 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.391678+0000", 2024-09-18T10:15:57.624 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0199e-05 2024-09-18T10:15:57.624 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.624 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.624 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:57.625 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.393192+0000", 2024-09-18T10:15:57.625 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0015135599999999999 2024-09-18T10:15:57.625 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.625 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.625 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:57.625 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.393238+0000", 2024-09-18T10:15:57.625 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.6128999999999998e-05 2024-09-18T10:15:57.625 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:57.625 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:57.625 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:57.625 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.625 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.625 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.4s1 72/40 ECSubWrite(tid=74, reqid=osd.8.0:72, at_version=0'0, trim_to=0'0, roll_forward_to=72'53))", 2024-09-18T10:15:57.625 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:28.496056+0000", 2024-09-18T10:15:57.626 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 29.065384154, 2024-09-18T10:15:57.626 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20153745300000001, 2024-09-18T10:15:57.626 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:57.626 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:57.626 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:57.626 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.626 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:57.626 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.496056+0000", 2024-09-18T10:15:57.626 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:57.626 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.626 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.626 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:57.626 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.496056+0000", 2024-09-18T10:15:57.626 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:57.627 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.627 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.627 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:57.627 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.496055+0000", 2024-09-18T10:15:57.627 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:57.627 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.627 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.627 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:57.627 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.496088+0000", 2024-09-18T10:15:57.627 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3411999999999998e-05 2024-09-18T10:15:57.627 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.627 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.627 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:57.628 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.496120+0000", 2024-09-18T10:15:57.628 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1745999999999997e-05 2024-09-18T10:15:57.628 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.628 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.628 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:57.628 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.596220+0000", 2024-09-18T10:15:57.628 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100100381 2024-09-18T10:15:57.628 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.628 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.628 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:57.628 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.596320+0000", 2024-09-18T10:15:57.628 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.9745000000000001e-05 2024-09-18T10:15:57.628 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.628 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.629 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:57.629 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.596344+0000", 2024-09-18T10:15:57.629 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4598999999999999e-05 2024-09-18T10:15:57.629 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.629 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.629 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:57.629 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.697512+0000", 2024-09-18T10:15:57.629 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10116728699999999 2024-09-18T10:15:57.629 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.629 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.629 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:57.629 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.697594+0000", 2024-09-18T10:15:57.629 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.1972999999999996e-05 2024-09-18T10:15:57.630 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:57.630 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:57.630 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:57.630 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.630 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.630 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.4s1 75/40 ECSubWrite(tid=80, reqid=osd.8.0:80, at_version=75'61, trim_to=0'0, roll_forward_to=74'59))", 2024-09-18T10:15:57.630 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:30.566285+0000", 2024-09-18T10:15:57.630 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 26.995154994, 2024-09-18T10:15:57.630 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.201584069, 2024-09-18T10:15:57.630 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:57.630 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:57.630 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:57.630 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.630 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:57.630 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.566285+0000", 2024-09-18T10:15:57.631 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:57.631 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.631 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.631 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:57.631 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.566285+0000", 2024-09-18T10:15:57.631 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:57.631 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.631 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.631 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:57.631 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.566285+0000", 2024-09-18T10:15:57.631 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:57.631 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.631 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.631 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:57.631 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.566310+0000", 2024-09-18T10:15:57.632 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5250999999999999e-05 2024-09-18T10:15:57.632 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.632 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.632 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:57.632 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.566319+0000", 2024-09-18T10:15:57.632 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.5159999999999994e-06 2024-09-18T10:15:57.632 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.632 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.632 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:57.632 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.666393+0000", 2024-09-18T10:15:57.632 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10007392399999999 2024-09-18T10:15:57.632 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.632 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.632 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:57.633 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.666455+0000", 2024-09-18T10:15:57.633 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.2372999999999994e-05 2024-09-18T10:15:57.633 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.633 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.633 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:57.633 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.666465+0000", 2024-09-18T10:15:57.633 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.7820000000000005e-06 2024-09-18T10:15:57.633 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.633 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.633 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:57.633 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.767822+0000", 2024-09-18T10:15:57.633 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10135730900000001 2024-09-18T10:15:57.633 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.634 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.634 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:57.634 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.767869+0000", 2024-09-18T10:15:57.634 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.7181000000000001e-05 2024-09-18T10:15:57.634 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:57.634 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:57.634 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:57.634 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.634 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.634 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.fs2 89/32 ECSubWrite(tid=187, reqid=osd.13.0:187, at_version=89'40, trim_to=0'0, roll_forward_to=81'37))", 2024-09-18T10:15:57.634 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:44.898707+0000", 2024-09-18T10:15:57.634 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 12.662733535999999, 2024-09-18T10:15:57.634 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.203487005, 2024-09-18T10:15:57.634 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:57.635 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:57.635 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:57.635 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.635 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:57.635 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:44.898707+0000", 2024-09-18T10:15:57.635 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:57.635 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.635 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.635 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:57.635 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:44.898707+0000", 2024-09-18T10:15:57.635 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:57.635 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.635 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.635 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:57.636 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:44.898706+0000", 2024-09-18T10:15:57.636 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:15:57.636 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.636 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.636 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:57.636 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:44.898768+0000", 2024-09-18T10:15:57.636 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.2237999999999996e-05 2024-09-18T10:15:57.636 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.636 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.636 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:57.636 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:44.898791+0000", 2024-09-18T10:15:57.636 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.2713000000000001e-05 2024-09-18T10:15:57.637 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.637 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.637 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:57.637 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:44.898808+0000", 2024-09-18T10:15:57.637 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7356000000000001e-05 2024-09-18T10:15:57.637 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.637 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.637 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:57.637 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:45.099963+0000", 2024-09-18T10:15:57.637 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20115494 2024-09-18T10:15:57.637 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.637 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.637 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:57.637 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:45.100014+0000", 2024-09-18T10:15:57.638 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.1298999999999998e-05 2024-09-18T10:15:57.638 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.638 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.638 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:57.638 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:45.102121+0000", 2024-09-18T10:15:57.638 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0021067479999999999 2024-09-18T10:15:57.638 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.638 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.638 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:57.638 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:45.102194+0000", 2024-09-18T10:15:57.638 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.2700999999999994e-05 2024-09-18T10:15:57.638 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:57.638 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:57.638 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:57.639 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.639 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.639 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:942 3.4s1 3:2619bf65:::smithi18925896-16:head [delete] snapc 35=[35,34,33,30,2f,28,20,1f,11] RETRY=2 ondisk+retry+write+known_if_redirected+full_force+supports_pool_eio e95)", 2024-09-18T10:15:57.639 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:50.784970+0000", 2024-09-18T10:15:57.639 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 6.7764699469999998, 2024-09-18T10:15:57.639 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3075152910000001, 2024-09-18T10:15:57.639 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:57.639 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:15:57.639 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:15:57.639 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:15:57.639 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:15:57.639 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 942 2024-09-18T10:15:57.639 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.640 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:57.640 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.640 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:57.640 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:50.784970+0000", 2024-09-18T10:15:57.640 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:57.640 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.640 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.640 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:57.640 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:50.784970+0000", 2024-09-18T10:15:57.640 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:57.640 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.640 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.641 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:57.641 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:50.784967+0000", 2024-09-18T10:15:57.641 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999967 2024-09-18T10:15:57.641 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.641 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.641 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:57.641 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:50.784983+0000", 2024-09-18T10:15:57.641 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5849999999999999e-05 2024-09-18T10:15:57.641 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.641 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.641 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:57.641 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:50.785021+0000", 2024-09-18T10:15:57.641 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8039e-05 2024-09-18T10:15:57.641 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.642 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.642 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:57.642 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:50.785036+0000", 2024-09-18T10:15:57.642 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4790000000000001e-05 2024-09-18T10:15:57.642 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.642 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.642 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:57.642 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:50.983583+0000", 2024-09-18T10:15:57.642 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.19854741400000001 2024-09-18T10:15:57.642 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.642 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.642 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-18T10:15:57.642 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:50.983588+0000", 2024-09-18T10:15:57.642 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1200000000000004e-06 2024-09-18T10:15:57.643 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.643 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.643 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:57.643 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:51.886817+0000", 2024-09-18T10:15:57.643 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.90322920900000003 2024-09-18T10:15:57.643 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.643 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.643 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:15:57.643 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:51.887374+0000", 2024-09-18T10:15:57.643 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00055698399999999995 2024-09-18T10:15:57.643 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.643 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.643 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:15:57.643 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:51.889200+0000", 2024-09-18T10:15:57.644 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001826016 2024-09-18T10:15:57.644 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.644 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.644 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:15:57.644 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:51.891521+0000", 2024-09-18T10:15:57.644 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.002321301 2024-09-18T10:15:57.644 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.644 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.644 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:15:57.644 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:52.091827+0000", 2024-09-18T10:15:57.644 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.200305603 2024-09-18T10:15:57.644 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.644 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.645 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:57.645 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:52.092486+0000", 2024-09-18T10:15:57.645 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00065908799999999995 2024-09-18T10:15:57.645 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:57.645 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:57.645 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:57.645 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.645 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.645 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpRead(3.cs1 95/94 ECSubRead(tid=206, to_read={3:351da1bf:::smithi18925896-6:2f=0,262144,0}, subchunks={3:351da1bf:::smithi18925896-6:2f=[0,1]}, attrs_to_read=))", 2024-09-18T10:15:57.645 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:51.508874+0000", 2024-09-18T10:15:57.645 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 6.0525668379999997, 2024-09-18T10:15:57.645 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.27511936599999998, 2024-09-18T10:15:57.645 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:57.645 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:57.645 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:57.646 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.646 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:57.646 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:51.508874+0000", 2024-09-18T10:15:57.646 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:57.646 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.646 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.646 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:57.646 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:51.508874+0000", 2024-09-18T10:15:57.646 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:57.646 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.646 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.646 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:57.646 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:51.508872+0000", 2024-09-18T10:15:57.646 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:15:57.647 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.647 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.647 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:57.647 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:51.508890+0000", 2024-09-18T10:15:57.647 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.8298999999999999e-05 2024-09-18T10:15:57.647 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.647 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.647 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:57.647 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:51.508930+0000", 2024-09-18T10:15:57.647 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9190000000000003e-05 2024-09-18T10:15:57.647 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.647 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.647 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:57.648 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:51.609029+0000", 2024-09-18T10:15:57.648 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100099382 2024-09-18T10:15:57.648 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.648 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.648 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:57.648 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:51.783094+0000", 2024-09-18T10:15:57.648 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.174064892 2024-09-18T10:15:57.648 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.648 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.648 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:57.648 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:51.783993+0000", 2024-09-18T10:15:57.648 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00089909300000000001 2024-09-18T10:15:57.648 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:57.648 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:57.649 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:57.649 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.649 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.649 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpRead(3.cs1 96/94 ECSubRead(tid=207, to_read={3:351da1bf:::smithi18925896-6:head=0,262144,0}, subchunks={3:351da1bf:::smithi18925896-6:head=[0,1]}, attrs_to_read=))", 2024-09-18T10:15:57.649 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:51.677052+0000", 2024-09-18T10:15:57.649 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 5.8843885910000004, 2024-09-18T10:15:57.649 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.31682236800000002, 2024-09-18T10:15:57.649 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:57.649 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:57.649 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:57.649 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.649 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:57.649 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:51.677052+0000", 2024-09-18T10:15:57.649 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:57.650 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.650 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.650 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:57.650 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:51.677052+0000", 2024-09-18T10:15:57.650 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:57.650 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.650 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.650 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:57.650 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:51.677051+0000", 2024-09-18T10:15:57.650 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999995 2024-09-18T10:15:57.650 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.650 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.650 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:57.650 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:51.677069+0000", 2024-09-18T10:15:57.651 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.8182999999999999e-05 2024-09-18T10:15:57.651 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.651 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.651 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:57.651 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:51.677114+0000", 2024-09-18T10:15:57.651 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5018999999999997e-05 2024-09-18T10:15:57.651 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.651 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.651 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:57.651 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:51.777230+0000", 2024-09-18T10:15:57.651 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100115924 2024-09-18T10:15:57.651 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.651 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.652 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:57.652 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:51.993689+0000", 2024-09-18T10:15:57.652 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.216458762 2024-09-18T10:15:57.652 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.652 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.652 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:57.652 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:51.993874+0000", 2024-09-18T10:15:57.652 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00018503000000000001 2024-09-18T10:15:57.652 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:57.652 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:57.652 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:57.652 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.652 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.652 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpRead(3.cs1 96/94 ECSubRead(tid=208, to_read={3:351da1bf:::smithi18925896-6:2f=262144,262144,0}, subchunks={3:351da1bf:::smithi18925896-6:2f=[0,1]}, attrs_to_read=))", 2024-09-18T10:15:57.653 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:51.789038+0000", 2024-09-18T10:15:57.653 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 5.7724020569999999, 2024-09-18T10:15:57.653 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20205589600000001, 2024-09-18T10:15:57.653 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:15:57.653 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:15:57.653 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:15:57.653 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.653 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:15:57.653 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:51.789038+0000", 2024-09-18T10:15:57.653 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:57.653 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.653 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.653 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:15:57.654 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:51.789038+0000", 2024-09-18T10:15:57.654 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:15:57.654 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.654 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.654 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:15:57.654 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:51.789037+0000", 2024-09-18T10:15:57.654 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:15:57.654 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.654 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.654 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:15:57.654 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:51.789058+0000", 2024-09-18T10:15:57.654 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1498999999999999e-05 2024-09-18T10:15:57.654 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.654 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.655 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:15:57.655 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:51.789091+0000", 2024-09-18T10:15:57.655 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3030000000000001e-05 2024-09-18T10:15:57.655 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.655 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.655 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:15:57.655 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:51.789106+0000", 2024-09-18T10:15:57.655 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4566000000000001e-05 2024-09-18T10:15:57.655 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.655 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.655 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:15:57.655 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:51.990519+0000", 2024-09-18T10:15:57.655 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20141335699999999 2024-09-18T10:15:57.656 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:15:57.656 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:15:57.656 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:15:57.656 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:51.991094+0000", 2024-09-18T10:15:57.656 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00057515099999999998 2024-09-18T10:15:57.656 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:57.656 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:57.656 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:57.656 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:15:57.656 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:15:57.656 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-18T10:15:57.658 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T10:15:57.659 INFO:tasks.ceph.osd.12.smithi029.stderr:2024-09-18T10:15:57.653+0000 7f8e4e627700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 22190) UID: 0 2024-09-18T10:15:57.677 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-18T10:15:57.717 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-18T10:15:57.725 INFO:tasks.rados.rados.0.smithi189.stdout:394: done (1 left) 2024-09-18T10:15:57.725 INFO:tasks.rados.rados.0.smithi189.stdout:395: snap_remove snap 18 2024-09-18T10:15:57.742 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-18T10:15:57.752 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-18T10:15:57.758 INFO:tasks.ceph.osd.4.smithi029.stderr:2024-09-18T10:15:57.753+0000 7f550cf6e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22189) UID: 0 2024-09-18T10:15:57.777 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-18T10:15:57.796 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:57.797 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:57.810 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T10:15:57.859 INFO:tasks.ceph.osd.6.smithi154.stderr:2024-09-18T10:15:57.854+0000 7f0abc9e1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22089) UID: 0 2024-09-18T10:15:57.944 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-18T10:15:57.960 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-18T10:15:57.955+0000 7f00046a8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 22136) UID: 0 2024-09-18T10:15:57.965 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:57.966 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:57.975 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T10:15:58.061 INFO:tasks.ceph.osd.3.smithi189.stderr:2024-09-18T10:15:58.053+0000 7f2440fc7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22396) UID: 0 2024-09-18T10:15:58.063 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-18T10:15:58.072 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:58.072 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:58.081 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T10:15:58.161 INFO:teuthology.orchestra.run.smithi029.stdout:osd.6: {} 2024-09-18T10:15:58.162 INFO:tasks.ceph.osd.15.smithi189.stderr:2024-09-18T10:15:58.157+0000 7fd0e725e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 22401) UID: 0 2024-09-18T10:15:58.174 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:58.174 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:58.183 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T10:15:58.262 INFO:teuthology.orchestra.run.smithi029.stdout:osd.7: {} 2024-09-18T10:15:58.263 INFO:tasks.ceph.osd.10.smithi154.stderr:2024-09-18T10:15:58.258+0000 7fb73221a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 22091) UID: 0 2024-09-18T10:15:58.279 INFO:teuthology.orchestra.run.smithi029.stdout:osd.8: {} 2024-09-18T10:15:58.281 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:58.282 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:58.291 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T10:15:58.300 INFO:teuthology.orchestra.run.smithi029.stdout:osd.9: {} 2024-09-18T10:15:58.323 INFO:teuthology.orchestra.run.smithi029.stdout:osd.10: {} 2024-09-18T10:15:58.346 INFO:teuthology.orchestra.run.smithi029.stdout:osd.11: {} 2024-09-18T10:15:58.363 INFO:tasks.ceph.osd.6.smithi154.stderr:2024-09-18T10:15:58.362+0000 7f0abc9e1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22089) UID: 0 2024-09-18T10:15:58.364 INFO:teuthology.orchestra.run.smithi029.stdout:osd.12: {} 2024-09-18T10:15:58.388 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:58.388 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:58.398 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-09-18T10:15:58.464 INFO:tasks.ceph.osd.3.smithi189.stderr:2024-09-18T10:15:58.457+0000 7f2440fc7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22396) UID: 0 2024-09-18T10:15:58.466 INFO:teuthology.orchestra.run.smithi029.stdout:osd.13: {} 2024-09-18T10:15:58.488 INFO:teuthology.orchestra.run.smithi029.stdout:osd.14: {} 2024-09-18T10:15:58.494 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:58.494 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:58.504 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-09-18T10:15:58.512 INFO:teuthology.orchestra.run.smithi029.stdout:osd.15: {} 2024-09-18T10:15:58.513 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T10:15:58.513 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T10:15:58.513 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T10:15:58.513 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T10:15:58.513 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T10:15:58.513 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T10:15:58.513 INFO:teuthology.orchestra.run.smithi029.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T10:15:58.513 INFO:teuthology.orchestra.run.smithi029.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T10:15:58.513 INFO:teuthology.orchestra.run.smithi029.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-18T10:15:58.513 INFO:teuthology.orchestra.run.smithi029.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-18T10:15:58.513 INFO:teuthology.orchestra.run.smithi029.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-18T10:15:58.513 INFO:teuthology.orchestra.run.smithi029.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-18T10:15:58.514 INFO:teuthology.orchestra.run.smithi029.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-18T10:15:58.514 INFO:teuthology.orchestra.run.smithi029.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-18T10:15:58.514 INFO:teuthology.orchestra.run.smithi029.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-18T10:15:58.514 INFO:teuthology.orchestra.run.smithi029.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-18T10:15:58.564 INFO:tasks.rados.rados.0.smithi189.stdout:395: done (1 left) 2024-09-18T10:15:58.564 INFO:tasks.rados.rados.0.smithi189.stdout:396: setattr oid 21 current snap is 32 2024-09-18T10:15:58.564 INFO:tasks.rados.rados.0.smithi189.stdout:391: finishing write tid 1 to smithi18925896-12 2024-09-18T10:15:58.564 INFO:tasks.rados.rados.0.smithi189.stdout:391: finishing write tid 2 to smithi18925896-12 2024-09-18T10:15:58.564 INFO:tasks.rados.rados.0.smithi189.stdout:391: finishing write tid 3 to smithi18925896-12 2024-09-18T10:15:58.564 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 12 v 94 (ObjNum 148 snap 32 seq_num 148) dirty exists 2024-09-18T10:15:58.564 INFO:tasks.rados.rados.0.smithi189.stdout:391: left oid 12 (ObjNum 148 snap 32 seq_num 148) 2024-09-18T10:15:58.566 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-09-18T10:15:58.562+0000 7fdfa8698700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22087) UID: 0 2024-09-18T10:15:58.570 INFO:teuthology.orchestra.run.smithi029.stderr:nodeep-scrub is set 2024-09-18T10:15:58.574 INFO:tasks.rados.rados.0.smithi189.stdout:391: done (1 left) 2024-09-18T10:15:58.574 INFO:tasks.rados.rados.0.smithi189.stdout:397: read oid 27 snap -1 2024-09-18T10:15:58.574 INFO:tasks.rados.rados.0.smithi189.stdout:397: expect (ObjNum 26 snap 0 seq_num 26) 2024-09-18T10:15:58.574 INFO:tasks.rados.rados.0.smithi189.stdout:append oid 32 current snap is 32 2024-09-18T10:15:58.574 INFO:tasks.rados.rados.0.smithi189.stdout:398: seq_num 149 ranges {606208=98304} 2024-09-18T10:15:58.578 INFO:tasks.rados.rados.0.smithi189.stdout:398: writing smithi18925896-32 from 606208 to 704512 tid 1 2024-09-18T10:15:58.578 INFO:tasks.rados.rados.0.smithi189.stdout:399: setattr oid 8 current snap is 32 2024-09-18T10:15:58.580 INFO:tasks.rados.rados.0.smithi189.stdout: waiting on 4 2024-09-18T10:15:58.604 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:58.604 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:58.613 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-09-18T10:15:58.667 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-18T10:15:58.661+0000 7f302f2c3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22187) UID: 0 2024-09-18T10:15:58.712 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:58.712 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:58.722 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_ops_in_flight 2024-09-18T10:15:58.769 INFO:tasks.ceph.osd.10.smithi154.stderr:2024-09-18T10:15:58.766+0000 7fb73221a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 22091) UID: 0 2024-09-18T10:15:58.820 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:58.821 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:58.831 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_blocked_ops 2024-09-18T10:15:58.870 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-18T10:15:58.867+0000 7f00046a8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 22136) UID: 0 2024-09-18T10:15:58.930 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:58.930 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:58.940 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_historic_ops 2024-09-18T10:15:58.971 INFO:tasks.ceph.osd.10.smithi154.stderr:2024-09-18T10:15:58.966+0000 7fb73221a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 22091) UID: 0 2024-09-18T10:15:59.035 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:59.035 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:59.044 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T10:15:59.049 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 21 v 103 (ObjNum 36 snap 0 seq_num 36) dirty exists 2024-09-18T10:15:59.049 INFO:tasks.rados.rados.0.smithi189.stdout:396: done (3 left) 2024-09-18T10:15:59.049 INFO:tasks.rados.rados.0.smithi189.stdout: waiting on 3 2024-09-18T10:15:59.071 INFO:tasks.ceph.osd.6.smithi154.stderr:2024-09-18T10:15:59.070+0000 7f0abc9e1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22089) UID: 0 2024-09-18T10:15:59.079 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 8 v 112 (ObjNum 7 snap 0 seq_num 7) dirty exists 2024-09-18T10:15:59.079 INFO:tasks.rados.rados.0.smithi189.stdout:399: done (2 left) 2024-09-18T10:15:59.079 INFO:tasks.rados.rados.0.smithi189.stdout: waiting on 2 2024-09-18T10:15:59.087 INFO:tasks.rados.rados.0.smithi189.stdout:397: expect (ObjNum 26 snap 0 seq_num 26) 2024-09-18T10:15:59.105 INFO:tasks.rados.rados.0.smithi189.stdout:397: done (1 left) 2024-09-18T10:15:59.105 INFO:tasks.rados.rados.0.smithi189.stdout: waiting on 1 2024-09-18T10:15:59.153 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:59.153 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:59.163 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T10:15:59.172 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-18T10:15:59.165+0000 7f302f2c3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22187) UID: 0 2024-09-18T10:15:59.262 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:59.262 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:59.271 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T10:15:59.273 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-18T10:15:59.265+0000 7f302f2c3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22187) UID: 0 2024-09-18T10:15:59.367 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:59.367 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:59.374 INFO:tasks.ceph.osd.4.smithi029.stderr:2024-09-18T10:15:59.369+0000 7f550cf6e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22189) UID: 0 2024-09-18T10:15:59.376 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T10:15:59.472 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:59.472 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:59.475 INFO:tasks.ceph.osd.3.smithi189.stderr:2024-09-18T10:15:59.469+0000 7f2440fc7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22396) UID: 0 2024-09-18T10:15:59.481 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T10:15:59.576 INFO:tasks.ceph.osd.15.smithi189.stderr:2024-09-18T10:15:59.569+0000 7fd0e725e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 22401) UID: 0 2024-09-18T10:15:59.578 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:59.579 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:59.588 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T10:15:59.677 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-18T10:15:59.669+0000 7f302f2c3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22187) UID: 0 2024-09-18T10:15:59.686 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:59.687 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:59.696 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-09-18T10:15:59.711 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7, 9, 10, 11, 12, 13, 14, 15, 8] out_osds: [] dead_osds: [] live_osds: [0, 4, 8, 12, 1, 5, 9, 13, 2, 6, 10, 14, 3, 7, 11, 15] 2024-09-18T10:15:59.711 INFO:tasks.thrashosds.thrasher:choose_action: min_in 8 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T10:15:59.711 INFO:tasks.thrashosds.thrasher:Growing pool unique_pool_0 2024-09-18T10:15:59.711 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T10:15:59.777 INFO:tasks.ceph.osd.10.smithi154.stderr:2024-09-18T10:15:59.774+0000 7fb73221a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 22091) UID: 0 2024-09-18T10:15:59.798 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:59.799 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:59.812 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-09-18T10:15:59.878 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-09-18T10:15:59.874+0000 7fdfa8698700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22087) UID: 0 2024-09-18T10:15:59.911 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-18T10:15:59.911 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:15:59.920 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-09-18T10:15:59.980 INFO:tasks.ceph.osd.10.smithi154.stderr:2024-09-18T10:15:59.978+0000 7fb73221a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 22091) UID: 0 2024-09-18T10:16:00.020 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-18T10:16:00.020 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:16:00.030 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_ops_in_flight 2024-09-18T10:16:00.050 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-18T10:16:00.052 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-09-18T10:16:00.067 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":134,"stamp":"2024-09-18T10:15:59.561829+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":131138099,"num_objects":140,"num_object_clones":88,"num_object_copies":960,"num_objects_missing_on_primary":6,"num_objects_missing":6,"num_objects_degraded":7,"num_objects_misplaced":18,"num_objects_unfound":0,"num_objects_dirty":140,"num_whiteouts":9,"num_read":341,"num_read_kb":161878,"num_write":716,"num_write_kb":104322,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":120,"num_bytes_recovered":143212544,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1673,"ondisk_log_size":1673,"up":200,"acting":200,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":215,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":16,"kb":1509949440,"kb_used":5106660,"kb_used_data":286524,"kb_used_omap":26,"kb_used_meta":4819877,"kb_avail":1504842780,"statfs":{"total":1546188226560,"available":1540959006720,"internally_reserved":0,"allocated":293400576,"data_stored":401305588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":27264,"internal_metadata":4935554432},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,2,0,0,1],"upper_bound":2048},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1540096,"num_objects":2,"num_object_clones":3,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-9,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":-1,"num_read":13,"num_read_kb":17784,"num_write":13,"num_write_kb":3264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":9371648,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.847253"},"pg_stats":[{"pgid":"3.19","version":"90'122","reported_seq":139,"reported_epoch":98,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-18T10:15:53.756378+0000","last_change":"2024-09-18T10:15:51.990076+0000","last_active":"2024-09-18T10:15:53.756378+0000","last_peered":"2024-09-18T10:15:53.756378+0000","last_clean":"2024-09-18T10:15:28.330327+0000","last_became_active":"2024-09-18T10:15:51.892167+0000","last_became_peered":"2024-09-18T10:15:51.892167+0000","last_unstale":"2024-09-18T10:15:53.756378+0000","last_undegraded":"2024-09-18T10:15:51.673131+0000","last_fullsized":"2024-09-18T10:15:53.756378+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":5,"last_scrub":"68'92","last_scrub_stamp":"2024-09-18T10:15:27.485910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:27.485910+0000","objects_scrubbed":10,"log_size":122,"log_dups_size":0,"ondisk_log_size":122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":10,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:39.070888+0000","scrub_duration":0.69568305100000005,"objects_trimmed":0,"snaptrim_duration":0.41041204199999998,"stat_sum":{"num_bytes":16777216,"num_objects":14,"num_object_clones":10,"num_object_copies":98,"num_objects_missing_on_primary":6,"num_objects_missing":6,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":11,"num_read_kb":16756,"num_write":17,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":14385152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,1,2,13,9,14],"acting":[8,4,1,2,13,9,14],"avail_no_missing":["1(2)","2(1)","2(3)","3(6)","4(1)","5(4)","7(0)","8(5)","9(5)","10(2)","11(3)","13(4)","14(6)"],"object_location_counts":[{"shards":"1(2),2(1),2(3),3(6),4(1),5(4),7(0),8(5),9(5),10(2),11(0),11(3),13(4),14(6)","objects":3},{"shards":"1(2),2(3),4(1),8(0),9(5),13(4),14(6)","objects":8},{"shards":"1(2),2(3),4(1),9(5),13(4),14(6)","objects":3}],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.18","version":"93'61","reported_seq":173,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-18T10:15:54.923982+0000","last_change":"2024-09-18T10:15:54.925047+0000","last_active":"2024-09-18T10:15:54.923982+0000","last_peered":"2024-09-18T10:15:54.923982+0000","last_clean":"2024-09-18T10:15:54.923982+0000","last_became_active":"2024-09-18T10:15:29.028750+0000","last_became_peered":"2024-09-18T10:15:29.028750+0000","last_unstale":"2024-09-18T10:15:54.923982+0000","last_undegraded":"2024-09-18T10:15:54.923982+0000","last_fullsized":"2024-09-18T10:15:54.923982+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":5,"last_scrub":"93'61","last_scrub_stamp":"2024-09-18T10:15:51.095148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:51.095148+0000","objects_scrubbed":5,"log_size":61,"log_dups_size":0,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:56.109718+0000","scrub_duration":0.91822677900000005,"objects_trimmed":0,"snaptrim_duration":0.0029654389999999998,"stat_sum":{"num_bytes":6291456,"num_objects":5,"num_object_clones":3,"num_object_copies":35,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,11,0,4,6,13,12],"acting":[10,11,0,4,6,13,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[{"start":"10","length":"1"},{"start":"12","length":"1"},{"start":"16","length":"1"},{"start":"1a","length":"1"},{"start":"1e","length":"1"},{"start":"26","length":"1"},{"start":"29","length":"2"},{"start":"2d","length":"1"},{"start":"2f","length":"1"}]},{"pgid":"3.17","version":"99'77","reported_seq":208,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-18T10:15:55.127364+0000","last_change":"2024-09-18T10:15:55.130806+0000","last_active":"2024-09-18T10:15:55.127364+0000","last_peered":"2024-09-18T10:15:55.127364+0000","last_clean":"2024-09-18T10:15:55.127364+0000","last_became_active":"2024-09-18T10:15:21.323733+0000","last_became_peered":"2024-09-18T10:15:21.323733+0000","last_unstale":"2024-09-18T10:15:55.127364+0000","last_undegraded":"2024-09-18T10:15:55.127364+0000","last_fullsized":"2024-09-18T10:15:55.127364+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":5,"last_scrub":"74'45","last_scrub_stamp":"2024-09-18T10:15:31.583098+0000","last_deep_scrub":"74'45","last_deep_scrub_stamp":"2024-09-18T10:15:31.583098+0000","last_clean_scrub_stamp":"2024-09-18T10:15:31.583098+0000","objects_scrubbed":5,"log_size":77,"log_dups_size":0,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:52.458457+0000","scrub_duration":2.494961714,"objects_trimmed":1,"snaptrim_duration":0.30861842299999998,"stat_sum":{"num_bytes":6537216,"num_objects":5,"num_object_clones":3,"num_object_copies":35,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":11,"num_write_kb":3920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,13,3,14,2,11,6],"acting":[10,13,3,14,2,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[{"start":"10","length":"1"},{"start":"12","length":"1"},{"start":"16","length":"1"},{"start":"1a","length":"1"},{"start":"1e","length":"1"},{"start":"26","length":"1"},{"start":"29","length":"2"},{"start":"2d","length":"1"},{"start":"2f","length":"1"}]},{"pgid":"3.16","version":"103'49","reported_seq":235,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-18T10:15:58.947189+0000","last_change":"2024-09-18T10:15:59.149421+0000","last_active":"2024-09-18T10:15:58.947189+0000","last_peered":"2024-09-18T10:15:58.947189+0000","last_clean":"2024-09-18T10:15:58.947189+0000","last_became_active":"2024-09-18T10:15:50.782912+0000","last_became_peered":"2024-09-18T10:15:50.782912+0000","last_unstale":"2024-09-18T10:15:58.947189+0000","last_undegraded":"2024-09-18T10:15:58.947189+0000","last_fullsized":"2024-09-18T10:15:58.947189+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":5,"last_scrub":"36'11","last_scrub_stamp":"2024-09-18T10:15:07.445786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:07.445786+0000","objects_scrubbed":1,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:14.115122+0000","scrub_duration":0.30645798400000002,"objects_trimmed":0,"snaptrim_duration":0.47534329800000003,"stat_sum":{"num_bytes":8486912,"num_objects":4,"num_object_clones":3,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":3,"num_read_kb":1,"num_write":9,"num_write_kb":7616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":13729792,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,6,4,8,2,14,10],"acting":[9,6,4,8,2,14,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[{"start":"10","length":"1"},{"start":"12","length":"1"},{"start":"16","length":"1"},{"start":"1a","length":"1"},{"start":"1e","length":"3"},{"start":"26","length":"1"},{"start":"29","length":"2"},{"start":"2d","length":"1"},{"start":"2f","length":"2"},{"start":"35","length":"1"}]},{"pgid":"3.15","version":"100'31","reported_seq":174,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-18T10:15:56.033543+0000","last_change":"2024-09-18T10:15:56.035491+0000","last_active":"2024-09-18T10:15:56.033543+0000","last_peered":"2024-09-18T10:15:56.033543+0000","last_clean":"2024-09-18T10:15:56.033543+0000","last_became_active":"2024-09-18T10:15:17.127771+0000","last_became_peered":"2024-09-18T10:15:17.127771+0000","last_unstale":"2024-09-18T10:15:56.033543+0000","last_undegraded":"2024-09-18T10:15:56.033543+0000","last_fullsized":"2024-09-18T10:15:56.033543+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":5,"last_scrub":"66'10","last_scrub_stamp":"2024-09-18T10:15:27.270892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:27.270892+0000","objects_scrubbed":2,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:57.241541+0000","scrub_duration":0.77475592100000001,"objects_trimmed":1,"snaptrim_duration":0.314871609,"stat_sum":{"num_bytes":2408448,"num_objects":3,"num_object_clones":2,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":1,"num_read":2,"num_read_kb":1,"num_write":26,"num_write_kb":2288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,15,13,10,1,7,9],"acting":[14,15,13,10,1,7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"10","length":"1"},{"start":"12","length":"1"},{"start":"16","length":"1"},{"start":"1a","length":"1"},{"start":"1e","length":"1"},{"start":"26","length":"1"},{"start":"29","length":"2"},{"start":"2d","length":"1"},{"start":"2f","length":"1"},{"start":"35","length":"1"}]},{"pgid":"3.14","version":"37'8","reported_seq":134,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-18T10:15:54.817482+0000","last_change":"2024-09-18T10:15:54.818733+0000","last_active":"2024-09-18T10:15:54.817482+0000","last_peered":"2024-09-18T10:15:54.817482+0000","last_clean":"2024-09-18T10:15:54.817482+0000","last_became_active":"2024-09-18T10:15:14.767770+0000","last_became_peered":"2024-09-18T10:15:14.767770+0000","last_unstale":"2024-09-18T10:15:54.817482+0000","last_undegraded":"2024-09-18T10:15:54.817482+0000","last_fullsized":"2024-09-18T10:15:54.817482+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":5,"last_scrub":"37'8","last_scrub_stamp":"2024-09-18T10:15:17.041590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:17.041590+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:32.741813+0000","scrub_duration":0.60018000800000004,"objects_trimmed":0,"snaptrim_duration":0.0019885419999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,14,9,10,5,13,4],"acting":[1,14,9,10,5,13,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"10","length":"1"},{"start":"12","length":"1"},{"start":"16","length":"1"},{"start":"1a","length":"1"},{"start":"1e","length":"1"},{"start":"26","length":"1"},{"start":"29","length":"2"},{"start":"2d","length":"1"},{"start":"2f","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":85,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-18T10:15:54.822773+0000","last_change":"2024-09-18T10:14:37.427722+0000","last_active":"2024-09-18T10:15:54.822773+0000","last_peered":"2024-09-18T10:15:54.822773+0000","last_clean":"2024-09-18T10:15:54.822773+0000","last_became_active":"2024-09-18T10:14:35.348943+0000","last_became_peered":"2024-09-18T10:14:35.348943+0000","last_unstale":"2024-09-18T10:15:54.822773+0000","last_undegraded":"2024-09-18T10:15:54.822773+0000","last_fullsized":"2024-09-18T10:15:54.822773+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:14:34.288596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:03.161727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060442999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,10],"acting":[13,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"3.6","version":"93'50","reported_seq":275,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-18T10:15:53.755800+0000","last_change":"2024-09-18T10:15:48.875458+0000","last_active":"2024-09-18T10:15:53.755800+0000","last_peered":"2024-09-18T10:15:53.755800+0000","last_clean":"2024-09-18T10:15:53.755800+0000","last_became_active":"2024-09-18T10:14:57.139786+0000","last_became_peered":"2024-09-18T10:14:57.139786+0000","last_unstale":"2024-09-18T10:15:53.755800+0000","last_undegraded":"2024-09-18T10:15:53.755800+0000","last_fullsized":"2024-09-18T10:15:53.755800+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'38","last_scrub_stamp":"2024-09-18T10:15:29.667178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:29.667178+0000","objects_scrubbed":5,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:51.483631+0000","scrub_duration":0.41099292199999998,"objects_trimmed":0,"snaptrim_duration":0.20802078299999999,"stat_sum":{"num_bytes":1310720,"num_objects":5,"num_object_clones":1,"num_object_copies":35,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":193,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,14,6,2,5,9,15],"acting":[0,14,6,2,5,9,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"10","length":"1"},{"start":"12","length":"1"},{"start":"16","length":"1"},{"start":"1a","length":"1"},{"start":"1e","length":"1"},{"start":"26","length":"1"},{"start":"29","length":"2"},{"start":"2d","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":13,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-18T10:15:53.768645+0000","last_change":"2024-09-18T10:15:50.978910+0000","last_active":"2024-09-18T10:15:53.768645+0000","last_peered":"2024-09-18T10:15:53.768645+0000","last_clean":"2024-09-18T10:15:53.768645+0000","last_became_active":"2024-09-18T10:15:50.978055+0000","last_became_peered":"2024-09-18T10:15:50.978055+0000","last_unstale":"2024-09-18T10:15:53.768645+0000","last_undegraded":"2024-09-18T10:15:53.768645+0000","last_fullsized":"2024-09-18T10:15:53.768645+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:14:34.288596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11],"acting":[8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.7","version":"99'70","reported_seq":284,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-18T10:15:55.354387+0000","last_change":"2024-09-18T10:15:55.356947+0000","last_active":"2024-09-18T10:15:55.354387+0000","last_peered":"2024-09-18T10:15:55.354387+0000","last_clean":"2024-09-18T10:15:55.354387+0000","last_became_active":"2024-09-18T10:14:57.137776+0000","last_became_peered":"2024-09-18T10:14:57.137776+0000","last_unstale":"2024-09-18T10:15:55.354387+0000","last_undegraded":"2024-09-18T10:15:55.354387+0000","last_fullsized":"2024-09-18T10:15:55.354387+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"60'30","last_scrub_stamp":"2024-09-18T10:15:17.136535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:17.136535+0000","objects_scrubbed":3,"log_size":70,"log_dups_size":0,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:35.499009+0000","scrub_duration":0.60680385400000003,"objects_trimmed":0,"snaptrim_duration":0.52790119999999996,"stat_sum":{"num_bytes":7340032,"num_objects":6,"num_object_clones":4,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":6,"num_read_kb":8019,"num_write":32,"num_write_kb":3440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6,0,14,10,13],"acting":[4,11,6,0,14,10,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"10","length":"1"},{"start":"12","length":"1"},{"start":"16","length":"1"},{"start":"1a","length":"1"},{"start":"1e","length":"1"},{"start":"26","length":"1"},{"start":"29","length":"2"},{"start":"2d","length":"1"},{"start":"2f","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":93,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-18T10:15:53.807339+0000","last_change":"2024-09-18T10:15:50.883382+0000","last_active":"2024-09-18T10:15:53.807339+0000","last_peered":"2024-09-18T10:15:53.807339+0000","last_clean":"2024-09-18T10:15:53.807339+0000","last_became_active":"2024-09-18T10:14:35.329198+0000","last_became_peered":"2024-09-18T10:14:35.329198+0000","last_unstale":"2024-09-18T10:15:53.807339+0000","last_undegraded":"2024-09-18T10:15:53.807339+0000","last_fullsized":"2024-09-18T10:15:53.807339+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:15:50.883147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:15:50.883147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:17:15.846467+0000","scrub_duration":0.153222372,"objects_trimmed":0,"snaptrim_duration":0.00062415699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"96'109","reported_seq":119,"reported_epoch":99,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-18T10:15:54.884282+0000","last_change":"2024-09-18T10:15:55.465372+0000","last_active":"2024-09-18T10:15:54.884282+0000","last_peered":"2024-09-18T10:15:54.884282+0000","last_clean":"2024-09-18T10:15:49.272797+0000","last_became_active":"2024-09-18T10:15:51.785137+0000","last_became_peered":"2024-09-18T10:15:51.785137+0000","last_unstale":"2024-09-18T10:15:54.884282+0000","last_undegraded":"2024-09-18T10:15:51.575717+0000","last_fullsized":"2024-09-18T10:15:51.572936+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"60'34","last_scrub_stamp":"2024-09-18T10:15:17.325003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:17.325003+0000","objects_scrubbed":6,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:45.470605+0000","scrub_duration":0.82233752999999998,"objects_trimmed":0,"snaptrim_duration":0.94050503500000004,"stat_sum":{"num_bytes":10108928,"num_objects":12,"num_object_clones":9,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":11,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":1,"num_read":16,"num_read_kb":15010,"num_write":19,"num_write_kb":10880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":12435456,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,15,4,2,6,10,7],"acting":[2147483647,15,4,2,6,10,7],"avail_no_missing":["15(1)","2(3)","4(2)","6(4)","7(6)","10(5)","12(0)"],"object_location_counts":[{"shards":"2(3),4(2),6(4),7(6),10(5),12(0),15(1)","objects":11},{"shards":"2(3),4(2),6(4),7(6),10(5),15(1)","objects":1}],"blocked_by":[],"up_primary":8,"acting_primary":15,"purged_snaps":[{"start":"10","length":"1"},{"start":"12","length":"1"},{"start":"16","length":"1"},{"start":"1a","length":"1"},{"start":"1e","length":"1"},{"start":"26","length":"1"},{"start":"29","length":"2"},{"start":"2d","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":84,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-18T10:15:53.705275+0000","last_change":"2024-09-18T10:14:37.426192+0000","last_active":"2024-09-18T10:15:53.705275+0000","last_peered":"2024-09-18T10:15:53.705275+0000","last_clean":"2024-09-18T10:15:53.705275+0000","last_became_active":"2024-09-18T10:14:35.524102+0000","last_became_peered":"2024-09-18T10:14:35.524102+0000","last_unstale":"2024-09-18T10:15:53.705275+0000","last_undegraded":"2024-09-18T10:15:53.705275+0000","last_fullsized":"2024-09-18T10:15:53.705275+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:14:34.288596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:15:59.190171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.10077985,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.5","version":"35'2","reported_seq":233,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-18T10:15:54.810024+0000","last_change":"2024-09-18T10:15:54.810960+0000","last_active":"2024-09-18T10:15:54.810024+0000","last_peered":"2024-09-18T10:15:54.810024+0000","last_clean":"2024-09-18T10:15:54.810024+0000","last_became_active":"2024-09-18T10:15:50.780770+0000","last_became_peered":"2024-09-18T10:15:50.780770+0000","last_unstale":"2024-09-18T10:15:54.810024+0000","last_undegraded":"2024-09-18T10:15:54.810024+0000","last_fullsized":"2024-09-18T10:15:54.810024+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"35'2","last_scrub_stamp":"2024-09-18T10:15:03.886316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:03.886316+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:16.496741+0000","scrub_duration":0.35694589500000001,"objects_trimmed":0,"snaptrim_duration":0.001368141,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,4,9,12,0,11],"acting":[5,8,4,9,12,0,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"10","length":"1"},{"start":"12","length":"1"},{"start":"16","length":"1"},{"start":"1a","length":"1"},{"start":"1e","length":"1"},{"start":"26","length":"1"},{"start":"29","length":"2"},{"start":"2d","length":"1"},{"start":"2f","length":"1"}]},{"pgid":"3.3","version":"96'29","reported_seq":240,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-18T10:15:54.822588+0000","last_change":"2024-09-18T10:15:54.923719+0000","last_active":"2024-09-18T10:15:54.822588+0000","last_peered":"2024-09-18T10:15:54.822588+0000","last_clean":"2024-09-18T10:15:54.822588+0000","last_became_active":"2024-09-18T10:14:57.138546+0000","last_became_peered":"2024-09-18T10:14:57.138546+0000","last_unstale":"2024-09-18T10:15:54.822588+0000","last_undegraded":"2024-09-18T10:15:54.822588+0000","last_fullsized":"2024-09-18T10:15:54.822588+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'23","last_scrub_stamp":"2024-09-18T10:15:28.483344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:28.483344+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:47.612849+0000","scrub_duration":0.40619440600000001,"objects_trimmed":0,"snaptrim_duration":0.10133315900000001,"stat_sum":{"num_bytes":606208,"num_objects":1,"num_object_clones":0,"num_object_copies":7,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":1,"num_write":6,"num_write_kb":2864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4,13,11,2,5],"acting":[10,9,4,13,11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[{"start":"10","length":"1"},{"start":"12","length":"1"},{"start":"16","length":"1"},{"start":"1a","length":"1"},{"start":"1e","length":"1"},{"start":"26","length":"1"},{"start":"29","length":"2"},{"start":"2d","length":"1"},{"start":"2f","length":"1"}]},{"pgid":"2.2","version":"29'2","reported_seq":87,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-18T10:15:54.822425+0000","last_change":"2024-09-18T10:14:37.429583+0000","last_active":"2024-09-18T10:15:54.822425+0000","last_peered":"2024-09-18T10:15:54.822425+0000","last_clean":"2024-09-18T10:15:54.822425+0000","last_became_active":"2024-09-18T10:14:35.715374+0000","last_became_peered":"2024-09-18T10:14:35.715374+0000","last_unstale":"2024-09-18T10:15:54.822425+0000","last_undegraded":"2024-09-18T10:15:54.822425+0000","last_fullsized":"2024-09-18T10:15:54.822425+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:14:34.288596+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:03.520789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002421224,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,12],"acting":[13,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"3.0","version":"99'36","reported_seq":248,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-18T10:15:55.227851+0000","last_change":"2024-09-18T10:15:55.229783+0000","last_active":"2024-09-18T10:15:55.227851+0000","last_peered":"2024-09-18T10:15:55.227851+0000","last_clean":"2024-09-18T10:15:55.227851+0000","last_became_active":"2024-09-18T10:14:57.444374+0000","last_became_peered":"2024-09-18T10:14:57.444374+0000","last_unstale":"2024-09-18T10:15:55.227851+0000","last_undegraded":"2024-09-18T10:15:55.227851+0000","last_fullsized":"2024-09-18T10:15:55.227851+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'4","last_scrub_stamp":"2024-09-18T10:15:05.668933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:05.668933+0000","objects_scrubbed":1,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:18.433780+0000","scrub_duration":2.6328342170000001,"objects_trimmed":0,"snaptrim_duration":0.40087085099999997,"stat_sum":{"num_bytes":1982464,"num_objects":3,"num_object_clones":2,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":1,"num_read":4,"num_read_kb":1,"num_write":7,"num_write_kb":1936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,9,2,6,15,4],"acting":[3,7,9,2,6,15,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"10","length":"1"},{"start":"12","length":"1"},{"start":"16","length":"1"},{"start":"1a","length":"1"},{"start":"1e","length":"1"},{"start":"26","length":"1"},{"start":"29","length":"2"},{"start":"2d","length":"1"},{"start":"2f","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":86,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-18T10:15:54.815628+0000","last_change":"2024-09-18T10:14:37.601046+0000","last_active":"2024-09-18T10:15:54.815628+0000","last_peered":"2024-09-18T10:15:54.815628+0000","last_clean":"2024-09-18T10:15:54.815628+0000","last_became_active":"2024-09-18T10:14:35.348262+0000","last_became_peered":"2024-09-18T10:14:35.348262+0000","last_unstale":"2024-09-18T10:15:54.815628+0000","last_undegraded":"2024-09-18T10:15:54.815628+0000","last_fullsized":"2024-09-18T10:15:54.815628+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:14:34.288596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:03.520789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063430400000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,0],"acting":[14,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"3.1","version":"99'94","reported_seq":303,"reported_epoch":99,"state":"active+clean+snaptrim","last_fresh":"2024-09-18T10:15:55.353312+0000","last_change":"2024-09-18T10:15:54.828519+0000","last_active":"2024-09-18T10:15:55.353312+0000","last_peered":"2024-09-18T10:15:55.353312+0000","last_clean":"2024-09-18T10:15:55.353312+0000","last_became_active":"2024-09-18T10:14:57.038503+0000","last_became_peered":"2024-09-18T10:14:57.038503+0000","last_unstale":"2024-09-18T10:15:55.353312+0000","last_undegraded":"2024-09-18T10:15:55.353312+0000","last_fullsized":"2024-09-18T10:15:55.353312+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'85","last_scrub_stamp":"2024-09-18T10:15:51.347579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:51.347579+0000","objects_scrubbed":9,"log_size":94,"log_dups_size":0,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:17:08.459990+0000","scrub_duration":0.71137397400000002,"objects_trimmed":1,"snaptrim_duration":0.84492282100000005,"stat_sum":{"num_bytes":3522560,"num_objects":10,"num_object_clones":7,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":1,"num_read":9,"num_read_kb":14690,"num_write":3,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,13,14,10,7,3,2],"acting":[4,13,14,10,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"10","length":"1"},{"start":"12","length":"1"},{"start":"16","length":"1"},{"start":"1a","length":"1"},{"start":"1e","length":"1"},{"start":"26","length":"1"},{"start":"29","length":"2"},{"start":"2d","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":89,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-18T10:15:57.732339+0000","last_change":"2024-09-18T10:14:37.600953+0000","last_active":"2024-09-18T10:15:57.732339+0000","last_peered":"2024-09-18T10:15:57.732339+0000","last_clean":"2024-09-18T10:15:57.732339+0000","last_became_active":"2024-09-18T10:14:35.349203+0000","last_became_peered":"2024-09-18T10:14:35.349203+0000","last_unstale":"2024-09-18T10:15:57.732339+0000","last_undegraded":"2024-09-18T10:15:57.732339+0000","last_fullsized":"2024-09-18T10:15:57.732339+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:14:34.288596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060130600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3],"acting":[9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2","version":"100'117","reported_seq":338,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-18T10:15:56.032262+0000","last_change":"2024-09-18T10:15:56.034297+0000","last_active":"2024-09-18T10:15:56.032262+0000","last_peered":"2024-09-18T10:15:56.032262+0000","last_clean":"2024-09-18T10:15:56.032262+0000","last_became_active":"2024-09-18T10:14:57.227520+0000","last_became_peered":"2024-09-18T10:14:57.227520+0000","last_unstale":"2024-09-18T10:15:56.032262+0000","last_undegraded":"2024-09-18T10:15:56.032262+0000","last_fullsized":"2024-09-18T10:15:56.032262+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'18","last_scrub_stamp":"2024-09-18T10:15:03.202100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:03.202100+0000","objects_scrubbed":6,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:12.722351+0000","scrub_duration":0.63960403700000001,"objects_trimmed":0,"snaptrim_duration":0.31412853099999999,"stat_sum":{"num_bytes":8814592,"num_objects":11,"num_object_clones":7,"num_object_copies":77,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":1,"num_read":26,"num_read_kb":5884,"num_write":42,"num_write_kb":6432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,3,13,1,5,10,7],"acting":[14,3,13,1,5,10,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"10","length":"1"},{"start":"12","length":"1"},{"start":"16","length":"1"},{"start":"1a","length":"1"},{"start":"1e","length":"1"},{"start":"26","length":"1"},{"start":"29","length":"2"},{"start":"2d","length":"1"},{"start":"2f","length":"1"},{"start":"35","length":"1"}]},{"pgid":"2.3","version":"27'1","reported_seq":87,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-18T10:15:54.819043+0000","last_change":"2024-09-18T10:14:37.801252+0000","last_active":"2024-09-18T10:15:54.819043+0000","last_peered":"2024-09-18T10:15:54.819043+0000","last_clean":"2024-09-18T10:15:54.819043+0000","last_became_active":"2024-09-18T10:14:35.487746+0000","last_became_peered":"2024-09-18T10:14:35.487746+0000","last_unstale":"2024-09-18T10:15:54.819043+0000","last_undegraded":"2024-09-18T10:15:54.819043+0000","last_fullsized":"2024-09-18T10:15:54.819043+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:14:34.288596+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:03.161727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.10072764100000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,5],"acting":[14,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"1.0","version":"29'112","reported_seq":219,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-18T10:15:53.756008+0000","last_change":"2024-09-18T10:14:32.509942+0000","last_active":"2024-09-18T10:15:53.756008+0000","last_peered":"2024-09-18T10:15:53.756008+0000","last_clean":"2024-09-18T10:15:53.756008+0000","last_became_active":"2024-09-18T10:14:32.508037+0000","last_became_peered":"2024-09-18T10:14:32.508037+0000","last_unstale":"2024-09-18T10:15:53.756008+0000","last_undegraded":"2024-09-18T10:15:53.756008+0000","last_fullsized":"2024-09-18T10:15:53.756008+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:31.267741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:31.267741+0000","last_clean_scrub_stamp":"2024-09-18T10:14:31.267741+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:15:59.262610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,15],"acting":[6,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.8","version":"99'61","reported_seq":261,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-18T10:15:55.334670+0000","last_change":"2024-09-18T10:15:55.337693+0000","last_active":"2024-09-18T10:15:55.334670+0000","last_peered":"2024-09-18T10:15:55.334670+0000","last_clean":"2024-09-18T10:15:55.334670+0000","last_became_active":"2024-09-18T10:14:57.443754+0000","last_became_peered":"2024-09-18T10:14:57.443754+0000","last_unstale":"2024-09-18T10:15:55.334670+0000","last_undegraded":"2024-09-18T10:15:55.334670+0000","last_fullsized":"2024-09-18T10:15:55.334670+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"41'13","last_scrub_stamp":"2024-09-18T10:15:05.373444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:05.373444+0000","objects_scrubbed":2,"log_size":61,"log_dups_size":0,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:31.932850+0000","scrub_duration":2.8745114210000002,"objects_trimmed":0,"snaptrim_duration":0.52295428399999999,"stat_sum":{"num_bytes":3899392,"num_objects":7,"num_object_clones":5,"num_object_copies":49,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":7,"num_read_kb":706,"num_write":9,"num_write_kb":2640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,11,3,15,1,7],"acting":[6,0,11,3,15,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"10","length":"1"},{"start":"12","length":"1"},{"start":"16","length":"1"},{"start":"1a","length":"1"},{"start":"1e","length":"1"},{"start":"26","length":"1"},{"start":"29","length":"2"},{"start":"2d","length":"1"},{"start":"2f","length":"1"}]},{"pgid":"3.9","version":"93'98","reported_seq":373,"reported_epoch":99,"state":"active+recovering","last_fresh":"2024-09-18T10:15:55.124735+0000","last_change":"2024-09-18T10:15:54.251695+0000","last_active":"2024-09-18T10:15:55.124735+0000","last_peered":"2024-09-18T10:15:55.124735+0000","last_clean":"2024-09-18T10:15:48.982962+0000","last_became_active":"2024-09-18T10:15:51.188859+0000","last_became_peered":"2024-09-18T10:15:51.188859+0000","last_unstale":"2024-09-18T10:15:55.124735+0000","last_undegraded":"2024-09-18T10:15:55.124735+0000","last_fullsized":"2024-09-18T10:15:55.124735+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'35","last_scrub_stamp":"2024-09-18T10:15:07.077517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:07.077517+0000","objects_scrubbed":4,"log_size":98,"log_dups_size":0,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:18.140951+0000","scrub_duration":0.71488955499999995,"objects_trimmed":1,"snaptrim_duration":0.40991788099999998,"stat_sum":{"num_bytes":4325376,"num_objects":6,"num_object_clones":4,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":6,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":1,"num_read":21,"num_read_kb":21422,"num_write":50,"num_write_kb":5392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":7782400,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,10,11,5,8,3],"acting":[7,2,10,11,5,8,3],"avail_no_missing":["7(0)","2(1)","3(6)","4(5)","5(4)","10(2)","11(3)"],"object_location_counts":[{"shards":"2(1),3(6),4(5),5(4),7(0),10(2),11(3)","objects":6}],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"10","length":"1"},{"start":"12","length":"1"},{"start":"16","length":"1"},{"start":"1a","length":"1"},{"start":"1e","length":"1"},{"start":"26","length":"1"},{"start":"29","length":"2"},{"start":"2d","length":"1"}]},{"pgid":"3.a","version":"99'68","reported_seq":274,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-18T10:15:55.727250+0000","last_change":"2024-09-18T10:15:55.829874+0000","last_active":"2024-09-18T10:15:55.727250+0000","last_peered":"2024-09-18T10:15:55.727250+0000","last_clean":"2024-09-18T10:15:55.727250+0000","last_became_active":"2024-09-18T10:14:46.765703+0000","last_became_peered":"2024-09-18T10:14:46.765703+0000","last_unstale":"2024-09-18T10:15:55.727250+0000","last_undegraded":"2024-09-18T10:15:55.727250+0000","last_fullsized":"2024-09-18T10:15:55.727250+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:14:45.576168+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":0,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:08.721290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.103045204,"stat_sum":{"num_bytes":3538944,"num_objects":5,"num_object_clones":3,"num_object_copies":35,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":42,"num_read_kb":16745,"num_write":42,"num_write_kb":6032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,1,6,5,3,15,4],"acting":[13,1,6,5,3,15,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"10","length":"1"},{"start":"12","length":"1"},{"start":"16","length":"1"},{"start":"1a","length":"1"},{"start":"1e","length":"1"},{"start":"26","length":"1"},{"start":"29","length":"2"},{"start":"2d","length":"1"},{"start":"2f","length":"1"},{"start":"35","length":"1"}]},{"pgid":"3.b","version":"99'46","reported_seq":222,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-18T10:15:55.535552+0000","last_change":"2024-09-18T10:15:55.536917+0000","last_active":"2024-09-18T10:15:55.535552+0000","last_peered":"2024-09-18T10:15:55.535552+0000","last_clean":"2024-09-18T10:15:55.535552+0000","last_became_active":"2024-09-18T10:14:46.802834+0000","last_became_peered":"2024-09-18T10:14:46.802834+0000","last_unstale":"2024-09-18T10:15:55.535552+0000","last_undegraded":"2024-09-18T10:15:55.535552+0000","last_fullsized":"2024-09-18T10:15:55.535552+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:14:45.576168+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:04.590511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.70805742199999999,"stat_sum":{"num_bytes":4177920,"num_objects":5,"num_object_clones":3,"num_object_copies":35,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":7,"num_read_kb":3,"num_write":55,"num_write_kb":4368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,10,12,11,1,14],"acting":[4,0,10,12,11,1,14],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"10","length":"1"},{"start":"12","length":"1"},{"start":"16","length":"1"},{"start":"1a","length":"1"},{"start":"1e","length":"1"},{"start":"26","length":"1"},{"start":"29","length":"2"},{"start":"2d","length":"1"},{"start":"2f","length":"1"}]},{"pgid":"3.c","version":"99'150","reported_seq":409,"reported_epoch":99,"state":"active+clean+snaptrim","last_fresh":"2024-09-18T10:15:55.360576+0000","last_change":"2024-09-18T10:15:55.360624+0000","last_active":"2024-09-18T10:15:55.360576+0000","last_peered":"2024-09-18T10:15:55.360576+0000","last_clean":"2024-09-18T10:15:55.360576+0000","last_became_active":"2024-09-18T10:15:50.784815+0000","last_became_peered":"2024-09-18T10:15:50.784815+0000","last_unstale":"2024-09-18T10:15:55.360576+0000","last_undegraded":"2024-09-18T10:15:55.360576+0000","last_fullsized":"2024-09-18T10:15:55.360576+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:14:45.576168+0000","objects_scrubbed":0,"log_size":150,"log_dups_size":0,"ondisk_log_size":150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":4,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.61717861799999996,"stat_sum":{"num_bytes":17154048,"num_objects":15,"num_object_clones":10,"num_object_copies":105,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":1,"num_read":42,"num_read_kb":24573,"num_write":110,"num_write_kb":17424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":55410688,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,13,3,7,12,14],"acting":[5,8,13,3,7,12,14],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"10","length":"1"},{"start":"12","length":"1"},{"start":"16","length":"1"},{"start":"1a","length":"1"},{"start":"1e","length":"1"},{"start":"26","length":"1"}]},{"pgid":"3.d","version":"100'43","reported_seq":284,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-18T10:15:56.136262+0000","last_change":"2024-09-18T10:15:56.138266+0000","last_active":"2024-09-18T10:15:56.136262+0000","last_peered":"2024-09-18T10:15:56.136262+0000","last_clean":"2024-09-18T10:15:56.136262+0000","last_became_active":"2024-09-18T10:15:50.985282+0000","last_became_peered":"2024-09-18T10:15:50.985282+0000","last_unstale":"2024-09-18T10:15:56.136262+0000","last_undegraded":"2024-09-18T10:15:56.136262+0000","last_fullsized":"2024-09-18T10:15:56.136262+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:14:45.576168+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:03.900063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.31071810500000002,"stat_sum":{"num_bytes":3817472,"num_objects":4,"num_object_clones":3,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":19,"num_read_kb":17526,"num_write":12,"num_write_kb":3968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":8388608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,1,11,15,9,8,4],"acting":[13,1,11,15,9,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"10","length":"1"},{"start":"12","length":"1"},{"start":"16","length":"1"},{"start":"1a","length":"1"},{"start":"1e","length":"1"},{"start":"26","length":"1"},{"start":"29","length":"2"},{"start":"2d","length":"1"},{"start":"2f","length":"1"},{"start":"35","length":"1"}]},{"pgid":"3.e","version":"103'74","reported_seq":340,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-18T10:15:58.768854+0000","last_change":"2024-09-18T10:15:58.770364+0000","last_active":"2024-09-18T10:15:58.768854+0000","last_peered":"2024-09-18T10:15:58.768854+0000","last_clean":"2024-09-18T10:15:58.768854+0000","last_became_active":"2024-09-18T10:15:51.077393+0000","last_became_peered":"2024-09-18T10:15:51.077393+0000","last_unstale":"2024-09-18T10:15:58.768854+0000","last_undegraded":"2024-09-18T10:15:58.768854+0000","last_fullsized":"2024-09-18T10:15:58.768854+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:14:45.576168+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.20657598799999999,"stat_sum":{"num_bytes":5423104,"num_objects":5,"num_object_clones":3,"num_object_copies":35,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":2,"num_read":14,"num_read_kb":2406,"num_write":46,"num_write_kb":5296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":15237120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,14,6,13,8,3,12],"acting":[11,14,6,13,8,3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"10","length":"1"},{"start":"12","length":"1"},{"start":"16","length":"1"},{"start":"1a","length":"1"},{"start":"1e","length":"3"},{"start":"26","length":"1"},{"start":"29","length":"2"},{"start":"2d","length":"1"},{"start":"2f","length":"2"},{"start":"35","length":"1"}]},{"pgid":"3.f","version":"100'58","reported_seq":244,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-18T10:15:56.131672+0000","last_change":"2024-09-18T10:15:56.133377+0000","last_active":"2024-09-18T10:15:56.131672+0000","last_peered":"2024-09-18T10:15:56.131672+0000","last_clean":"2024-09-18T10:15:56.131672+0000","last_became_active":"2024-09-18T10:14:46.803296+0000","last_became_peered":"2024-09-18T10:14:46.803296+0000","last_unstale":"2024-09-18T10:15:56.131672+0000","last_undegraded":"2024-09-18T10:15:56.131672+0000","last_fullsized":"2024-09-18T10:15:56.131672+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:14:45.576168+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:11.624416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.40615378299999999,"stat_sum":{"num_bytes":6815744,"num_objects":5,"num_object_clones":3,"num_object_copies":35,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":11,"num_read_kb":1782,"num_write":42,"num_write_kb":7216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,3,15,14,2,11,0],"acting":[13,3,15,14,2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"10","length":"1"},{"start":"12","length":"1"},{"start":"16","length":"1"},{"start":"1a","length":"1"},{"start":"1e","length":"1"},{"start":"26","length":"1"},{"start":"29","length":"2"},{"start":"2d","length":"1"},{"start":"2f","length":"1"},{"start":"35","length":"1"}]},{"pgid":"3.10","version":"93'39","reported_seq":251,"reported_epoch":99,"state":"active+recovering","last_fresh":"2024-09-18T10:15:55.334003+0000","last_change":"2024-09-18T10:15:53.666346+0000","last_active":"2024-09-18T10:15:55.334003+0000","last_peered":"2024-09-18T10:15:55.334003+0000","last_clean":"2024-09-18T10:15:49.182849+0000","last_became_active":"2024-09-18T10:15:50.787275+0000","last_became_peered":"2024-09-18T10:15:50.787275+0000","last_unstale":"2024-09-18T10:15:55.334003+0000","last_undegraded":"2024-09-18T10:15:55.334003+0000","last_fullsized":"2024-09-18T10:15:55.334003+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'19","last_scrub_stamp":"2024-09-18T10:15:29.719467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:29.719467+0000","objects_scrubbed":1,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:30.348118+0000","scrub_duration":2.0403324629999999,"objects_trimmed":0,"snaptrim_duration":0.31074911999999999,"stat_sum":{"num_bytes":5488640,"num_objects":3,"num_object_clones":2,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":6,"num_read_kb":16083,"num_write":9,"num_write_kb":3712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,5,4,8,0,9],"acting":[6,7,5,4,8,0,9],"avail_no_missing":["6(0)","0(5)","1(4)","4(3)","5(2)","7(1)","9(6)"],"object_location_counts":[{"shards":"0(5),1(4),4(3),5(2),6(0),7(1),9(6)","objects":1},{"shards":"0(5),4(3),5(2),6(0),7(1),8(4),9(6)","objects":2}],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"10","length":"1"},{"start":"12","length":"1"},{"start":"16","length":"1"},{"start":"1a","length":"1"},{"start":"1e","length":"1"},{"start":"26","length":"1"},{"start":"29","length":"2"},{"start":"2d","length":"1"}]},{"pgid":"3.11","version":"36'6","reported_seq":154,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-18T10:15:54.828889+0000","last_change":"2024-09-18T10:15:54.829955+0000","last_active":"2024-09-18T10:15:54.828889+0000","last_peered":"2024-09-18T10:15:54.828889+0000","last_clean":"2024-09-18T10:15:54.828889+0000","last_became_active":"2024-09-18T10:15:04.955869+0000","last_became_peered":"2024-09-18T10:15:04.955869+0000","last_unstale":"2024-09-18T10:15:54.828889+0000","last_undegraded":"2024-09-18T10:15:54.828889+0000","last_fullsized":"2024-09-18T10:15:54.828889+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":5,"last_scrub":"36'6","last_scrub_stamp":"2024-09-18T10:15:07.755163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:07.755163+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:13.113623+0000","scrub_duration":0.60886176599999997,"objects_trimmed":0,"snaptrim_duration":0.001249167,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,13,10,0,14,2],"acting":[3,5,13,10,0,14,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"10","length":"1"},{"start":"12","length":"1"},{"start":"16","length":"1"},{"start":"1a","length":"1"},{"start":"1e","length":"1"},{"start":"26","length":"1"},{"start":"29","length":"2"},{"start":"2d","length":"1"},{"start":"2f","length":"1"}]},{"pgid":"3.12","version":"36'8","reported_seq":262,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-18T10:15:55.720790+0000","last_change":"2024-09-18T10:15:55.821837+0000","last_active":"2024-09-18T10:15:55.720790+0000","last_peered":"2024-09-18T10:15:55.720790+0000","last_clean":"2024-09-18T10:15:55.720790+0000","last_became_active":"2024-09-18T10:15:50.682909+0000","last_became_peered":"2024-09-18T10:15:50.682909+0000","last_unstale":"2024-09-18T10:15:55.720790+0000","last_undegraded":"2024-09-18T10:15:55.720790+0000","last_fullsized":"2024-09-18T10:15:55.720790+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":5,"last_scrub":"36'8","last_scrub_stamp":"2024-09-18T10:15:04.353687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:04.353687+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:11.257401+0000","scrub_duration":0.76511709400000005,"objects_trimmed":0,"snaptrim_duration":0.10180410300000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,5,8,3,4,11,1],"acting":[14,5,8,3,4,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"10","length":"1"},{"start":"12","length":"1"},{"start":"16","length":"1"},{"start":"1a","length":"1"},{"start":"1e","length":"1"},{"start":"26","length":"1"},{"start":"29","length":"2"},{"start":"2d","length":"1"},{"start":"2f","length":"1"},{"start":"35","length":"1"}]},{"pgid":"3.13","version":"93'52","reported_seq":224,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-18T10:15:54.716756+0000","last_change":"2024-09-18T10:15:54.717589+0000","last_active":"2024-09-18T10:15:54.716756+0000","last_peered":"2024-09-18T10:15:54.716756+0000","last_clean":"2024-09-18T10:15:54.716756+0000","last_became_active":"2024-09-18T10:15:13.263306+0000","last_became_peered":"2024-09-18T10:15:13.263306+0000","last_unstale":"2024-09-18T10:15:54.716756+0000","last_undegraded":"2024-09-18T10:15:54.716756+0000","last_fullsized":"2024-09-18T10:15:54.716756+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":5,"last_scrub":"60'33","last_scrub_stamp":"2024-09-18T10:15:19.140411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:19.140411+0000","objects_scrubbed":1,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:39.399176+0000","scrub_duration":0.706777345,"objects_trimmed":0,"snaptrim_duration":0.0011237230000000001,"stat_sum":{"num_bytes":1720320,"num_objects":2,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,13,2,6,0,4,15],"acting":[1,13,2,6,0,4,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"10","length":"1"},{"start":"12","length":"1"},{"start":"16","length":"1"},{"start":"1a","length":"1"},{"start":"1e","length":"1"},{"start":"26","length":"1"},{"start":"29","length":"2"},{"start":"2d","length":"1"},{"start":"2f","length":"1"}]}],"pool_stats":[{"poolid":3,"num_pg":26,"stat_sum":{"num_bytes":130547712,"num_objects":136,"num_object_clones":88,"num_object_copies":952,"num_objects_missing_on_primary":6,"num_objects_missing":6,"num_objects_degraded":7,"num_objects_misplaced":18,"num_objects_unfound":0,"num_objects_dirty":136,"num_whiteouts":9,"num_read":255,"num_read_kb":161805,"num_write":569,"num_write_kb":101648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":120,"num_bytes_recovered":143212544,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":260292608,"data_stored":373972992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1558,"ondisk_log_size":1558,"up":182,"acting":182,"num_store_stats":16},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":13},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":112,"ondisk_log_size":112,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":15,"up_from":22,"seq":94489280531,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312032,"kb_used_data":10512,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94059808,"statfs":{"total":96636764160,"available":96317243392,"internally_reserved":0,"allocated":10764288,"data_stored":14945651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,4,6,7,8,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:15:31 2024","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.74399999999999999},{"interface":"front","average":{"1min":6.7190000000000003,"5min":6.7190000000000003,"15min":6.7190000000000003},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":101.291,"5min":101.291,"15min":101.291},"last":0.79300000000000004}]},{"osd":1,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.70199999999999996},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.931,"5min":1.931,"15min":1.931},"last":0.67300000000000004}]},{"osd":2,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.417,"5min":1.417,"15min":1.417},"last":1.179},{"interface":"front","average":{"1min":5.5389999999999997,"5min":5.5389999999999997,"15min":5.5389999999999997},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":78.242999999999995,"5min":78.242999999999995,"15min":78.242999999999995},"last":1.1299999999999999}]},{"osd":3,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":3.3029999999999999,"5min":3.3029999999999999,"15min":3.3029999999999999},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":41.720999999999997,"5min":41.720999999999997,"15min":41.720999999999997},"last":0.55400000000000005},{"interface":"front","average":{"1min":1.014,"5min":1.014,"15min":1.014},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.49,"5min":1.49,"15min":1.49},"last":0.89500000000000002}]},{"osd":4,"last update":"Wed Sep 18 10:15:31 2024","interfaces":[{"interface":"back","average":{"1min":11.021000000000001,"5min":11.021000000000001,"15min":11.021000000000001},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":99.745999999999995,"5min":99.745999999999995,"15min":99.745999999999995},"last":1.089},{"interface":"front","average":{"1min":11.140000000000001,"5min":11.140000000000001,"15min":11.140000000000001},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":99.781000000000006,"5min":99.781000000000006,"15min":99.781000000000006},"last":1.0609999999999999}]},{"osd":5,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":1.028},{"interface":"front","average":{"1min":1.7869999999999999,"5min":1.7869999999999999,"15min":1.7869999999999999},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":10.874000000000001,"5min":10.874000000000001,"15min":10.874000000000001},"last":0.92400000000000004}]},{"osd":6,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":1.9710000000000001,"5min":1.9710000000000001,"15min":1.9710000000000001},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":12.367000000000001,"5min":12.367000000000001,"15min":12.367000000000001},"last":0.92900000000000005},{"interface":"front","average":{"1min":5.6109999999999998,"5min":5.6109999999999998,"15min":5.6109999999999998},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":74.156000000000006,"5min":74.156000000000006,"15min":74.156000000000006},"last":1.603}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2729999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.232}]},{"osd":10,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":6.3680000000000003,"5min":6.3680000000000003,"15min":6.3680000000000003},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":86.546999999999997,"5min":86.546999999999997,"15min":86.546999999999997},"last":0.97699999999999998},{"interface":"front","average":{"1min":6.375,"5min":6.375,"15min":6.375},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":86.497,"5min":86.497,"15min":86.497},"last":0.84199999999999997}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4590000000000001}]},{"osd":13,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":6.0819999999999999,"5min":6.0819999999999999,"15min":6.0819999999999999},"min":{"1min":1.093,"5min":1.093,"15min":1.093},"max":{"1min":80.197000000000003,"5min":80.197000000000003,"15min":80.197000000000003},"last":1.585},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.8420000000000001,"5min":1.8420000000000001,"15min":1.8420000000000001},"last":1.7}]},{"osd":14,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":1.444,"5min":1.444,"15min":1.444},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":2.661,"5min":2.661,"15min":2.661},"last":1.5309999999999999},{"interface":"front","average":{"1min":10.750999999999999,"5min":10.750999999999999,"15min":10.750999999999999},"min":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"max":{"1min":85.006,"5min":85.006,"15min":85.006},"last":1.486}]}]},{"osd":14,"up_from":22,"seq":94489280531,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":325752,"kb_used_data":24240,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94046088,"statfs":{"total":96636764160,"available":96303194112,"internally_reserved":0,"allocated":24821760,"data_stored":36260691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,3,4,5,7,8,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":15.391,"5min":15.391,"15min":15.391},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":189.88800000000001,"5min":189.88800000000001,"15min":189.88800000000001},"last":1.2250000000000001},{"interface":"front","average":{"1min":19.533000000000001,"5min":19.533000000000001,"15min":19.533000000000001},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":180.09999999999999,"5min":180.09999999999999,"15min":180.09999999999999},"last":71.900999999999996}]},{"osd":1,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":9.1829999999999998,"5min":9.1829999999999998,"15min":9.1829999999999998},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":79.626000000000005,"5min":79.626000000000005,"15min":79.626000000000005},"last":0.94499999999999995},{"interface":"front","average":{"1min":15.465,"5min":15.465,"15min":15.465},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":189.762,"5min":189.762,"15min":189.762},"last":1.044}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.417}]},{"osd":3,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":15.247,"5min":15.247,"15min":15.247},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":180.19900000000001,"5min":180.19900000000001,"15min":180.19900000000001},"last":0.98699999999999999},{"interface":"front","average":{"1min":21.341999999999999,"5min":21.341999999999999,"15min":21.341999999999999},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":180.851,"5min":180.851,"15min":180.851},"last":1.248}]},{"osd":4,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":25.399999999999999,"5min":25.399999999999999,"15min":25.399999999999999},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":189.809,"5min":189.809,"15min":189.809},"last":1.6499999999999999},{"interface":"front","average":{"1min":15.281000000000001,"5min":15.281000000000001,"15min":15.281000000000001},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":180.31700000000001,"5min":180.31700000000001,"15min":180.31700000000001},"last":1.2809999999999999}]},{"osd":5,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":26.425999999999998,"5min":26.425999999999998,"15min":26.425999999999998},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":277.41199999999998,"5min":277.41199999999998,"15min":277.41199999999998},"last":1.1699999999999999},{"interface":"front","average":{"1min":15.683,"5min":15.683,"15min":15.683},"min":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":189.64699999999999,"5min":189.64699999999999,"15min":189.64699999999999},"last":1.7450000000000001}]},{"osd":7,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":15.460000000000001,"5min":15.460000000000001,"15min":15.460000000000001},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":180.268,"5min":180.268,"15min":180.268},"last":1.4750000000000001},{"interface":"front","average":{"1min":15.154,"5min":15.154,"15min":15.154},"min":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":180.631,"5min":180.631,"15min":180.631},"last":1.4319999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3680000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8300000000000001}]},{"osd":10,"last update":"Wed Sep 18 10:15:49 2024","interfaces":[{"interface":"back","average":{"1min":17.039000000000001,"5min":17.039000000000001,"15min":17.039000000000001},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":180.15899999999999,"5min":180.15899999999999,"15min":180.15899999999999},"last":1.6279999999999999},{"interface":"front","average":{"1min":31.004000000000001,"5min":31.004000000000001,"15min":31.004000000000001},"min":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":269.423,"5min":269.423,"15min":269.423},"last":1.9239999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.887}]},{"osd":12,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":15.855,"5min":15.855,"15min":15.855},"min":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"max":{"1min":189.84999999999999,"5min":189.84999999999999,"15min":189.84999999999999},"last":1.857},{"interface":"front","average":{"1min":15.666,"5min":15.666,"15min":15.666},"min":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"max":{"1min":180.381,"5min":180.381,"15min":180.381},"last":1.776}]},{"osd":13,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":26.244,"5min":26.244,"15min":26.244},"min":{"1min":0.875,"5min":0.875,"15min":0.875},"max":{"1min":180.59100000000001,"5min":180.59100000000001,"15min":180.59100000000001},"last":1.8049999999999999},{"interface":"front","average":{"1min":25.079000000000001,"5min":25.079000000000001,"15min":25.079000000000001},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":189.71299999999999,"5min":189.71299999999999,"15min":189.71299999999999},"last":2.0009999999999999}]},{"osd":15,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":15.680999999999999,"5min":15.680999999999999,"15min":15.680999999999999},"min":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"max":{"1min":180.44800000000001,"5min":180.44800000000001,"15min":180.44800000000001},"last":1.6819999999999999},{"interface":"front","average":{"1min":18.472000000000001,"5min":18.472000000000001,"15min":18.472000000000001},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":180.53,"5min":180.53,"15min":180.53},"last":1.6000000000000001}]}]},{"osd":13,"up_from":22,"seq":94489280532,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":326484,"kb_used_data":24968,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94045356,"statfs":{"total":96636764160,"available":96302444544,"internally_reserved":0,"allocated":25567232,"data_stored":37055334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,6,8,9,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":20.742000000000001,"5min":20.742000000000001,"15min":20.742000000000001},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":176.66800000000001,"5min":176.66800000000001,"15min":176.66800000000001},"last":1.3160000000000001},{"interface":"front","average":{"1min":30.433,"5min":30.433,"15min":30.433},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":199.732,"5min":199.732,"15min":199.732},"last":0.92800000000000005}]},{"osd":1,"last update":"Wed Sep 18 10:15:48 2024","interfaces":[{"interface":"back","average":{"1min":1.986,"5min":1.986,"15min":1.986},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":18.471,"5min":18.471,"15min":18.471},"last":1.1080000000000001},{"interface":"front","average":{"1min":20.716999999999999,"5min":20.716999999999999,"15min":20.716999999999999},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":176.56800000000001,"5min":176.56800000000001,"15min":176.56800000000001},"last":0.97799999999999998}]},{"osd":2,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":11.984,"5min":11.984,"15min":11.984},"min":{"1min":0.751,"5min":0.751,"15min":0.751},"max":{"1min":196.59200000000001,"5min":196.59200000000001,"15min":196.59200000000001},"last":1.0069999999999999},{"interface":"front","average":{"1min":26.109999999999999,"5min":26.109999999999999,"15min":26.109999999999999},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":177.03,"5min":177.03,"15min":177.03},"last":1.2130000000000001}]},{"osd":3,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":25.431999999999999,"5min":25.431999999999999,"15min":25.431999999999999},"min":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"max":{"1min":168.27699999999999,"5min":168.27699999999999,"15min":168.27699999999999},"last":1.268},{"interface":"front","average":{"1min":12.045999999999999,"5min":12.045999999999999,"15min":12.045999999999999},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":196.643,"5min":196.643,"15min":196.643},"last":0.88300000000000001}]},{"osd":4,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":29.802,"5min":29.802,"15min":29.802},"min":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"max":{"1min":168.48099999999999,"5min":168.48099999999999,"15min":168.48099999999999},"last":1.05},{"interface":"front","average":{"1min":8.1780000000000008,"5min":8.1780000000000008,"15min":8.1780000000000008},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":65.188000000000002,"5min":65.188000000000002,"15min":65.188000000000002},"last":1.1830000000000001}]},{"osd":5,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":26.152999999999999,"5min":26.152999999999999,"15min":26.152999999999999},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":176.48099999999999,"5min":176.48099999999999,"15min":176.48099999999999},"last":1.5089999999999999},{"interface":"front","average":{"1min":24.885999999999999,"5min":24.885999999999999,"15min":24.885999999999999},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":167.62299999999999,"5min":167.62299999999999,"15min":167.62299999999999},"last":1.407}]},{"osd":6,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":29.355,"5min":29.355,"15min":29.355},"min":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":176.81200000000001,"5min":176.81200000000001,"15min":176.81200000000001},"last":1.4179999999999999},{"interface":"front","average":{"1min":27.855,"5min":27.855,"15min":27.855},"min":{"1min":1.24,"5min":1.24,"15min":1.24},"max":{"1min":167.947,"5min":167.947,"15min":167.947},"last":1.3640000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4390000000000001}]},{"osd":9,"last update":"Wed Sep 18 10:15:48 2024","interfaces":[{"interface":"back","average":{"1min":21.055,"5min":21.055,"15min":21.055},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":176.73599999999999,"5min":176.73599999999999,"15min":176.73599999999999},"last":1.6279999999999999},{"interface":"front","average":{"1min":23.465,"5min":23.465,"15min":23.465},"min":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"max":{"1min":167.84899999999999,"5min":167.84899999999999,"15min":167.84899999999999},"last":1.4890000000000001}]},{"osd":10,"last update":"Wed Sep 18 10:15:39 2024","interfaces":[{"interface":"back","average":{"1min":22.678999999999998,"5min":22.678999999999998,"15min":22.678999999999998},"min":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"max":{"1min":168.07900000000001,"5min":168.07900000000001,"15min":168.07900000000001},"last":1.5329999999999999},{"interface":"front","average":{"1min":31.972000000000001,"5min":31.972000000000001,"15min":31.972000000000001},"min":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"max":{"1min":196.37700000000001,"5min":196.37700000000001,"15min":196.37700000000001},"last":1.778}]},{"osd":11,"last update":"Wed Sep 18 10:15:48 2024","interfaces":[{"interface":"back","average":{"1min":1.611,"5min":1.611,"15min":1.611},"min":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"max":{"1min":2.391,"5min":2.391,"15min":2.391},"last":1.853},{"interface":"front","average":{"1min":26.463999999999999,"5min":26.463999999999999,"15min":26.463999999999999},"min":{"1min":1.028,"5min":1.028,"15min":1.028},"max":{"1min":176.94999999999999,"5min":176.94999999999999,"15min":176.94999999999999},"last":1.726}]},{"osd":12,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":1.5820000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"min":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"max":{"1min":2.7879999999999998,"5min":2.7879999999999998,"15min":2.7879999999999998},"last":1.895},{"interface":"front","average":{"1min":21.914999999999999,"5min":21.914999999999999,"15min":21.914999999999999},"min":{"1min":1.296,"5min":1.296,"15min":1.296},"max":{"1min":168.34399999999999,"5min":168.34399999999999,"15min":168.34399999999999},"last":1.657}]},{"osd":14,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":6.694,"5min":6.694,"15min":6.694},"min":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"max":{"1min":87.427000000000007,"5min":87.427000000000007,"15min":87.427000000000007},"last":1.972},{"interface":"front","average":{"1min":26.231999999999999,"5min":26.231999999999999,"15min":26.231999999999999},"min":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"max":{"1min":176.87700000000001,"5min":176.87700000000001,"15min":176.87700000000001},"last":1.804}]},{"osd":15,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":20.334,"5min":20.334,"15min":20.334},"min":{"1min":1.266,"5min":1.266,"15min":1.266},"max":{"1min":168.40700000000001,"5min":168.40700000000001,"15min":168.40700000000001},"last":1.6080000000000001},{"interface":"front","average":{"1min":16.350999999999999,"5min":16.350999999999999,"15min":16.350999999999999},"min":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"max":{"1min":196.512,"5min":196.512,"15min":196.512},"last":1.9330000000000001}]}]},{"osd":0,"up_from":21,"seq":90194313235,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311688,"kb_used_data":10164,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94060152,"statfs":{"total":96636764160,"available":96317595648,"internally_reserved":0,"allocated":10407936,"data_stored":14066315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,4,5,6,9,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 18 10:15:34 2024","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.341,"5min":1.341,"15min":1.341},"last":1.0720000000000001},{"interface":"front","average":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.4630000000000001,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":1.131}]},{"osd":2,"last update":"Wed Sep 18 10:15:34 2024","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":3.7269999999999999,"5min":3.7269999999999999,"15min":3.7269999999999999},"last":46.220999999999997},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.528,"5min":1.528,"15min":1.528},"last":0.85199999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.196}]},{"osd":4,"last update":"Wed Sep 18 10:15:29 2024","interfaces":[{"interface":"back","average":{"1min":15.4,"5min":15.4,"15min":15.4},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":130.28,"5min":130.28,"15min":130.28},"last":142.20699999999999},{"interface":"front","average":{"1min":16,"5min":16,"15min":16},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":79.914000000000001,"5min":79.914000000000001,"15min":79.914000000000001},"last":0.80900000000000005}]},{"osd":5,"last update":"Wed Sep 18 10:15:34 2024","interfaces":[{"interface":"back","average":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"min":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"max":{"1min":1.6819999999999999,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":1.2270000000000001},{"interface":"front","average":{"1min":9.7759999999999998,"5min":9.7759999999999998,"15min":9.7759999999999998},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":130.39599999999999,"5min":130.39599999999999,"15min":130.39599999999999},"last":0.90500000000000003}]},{"osd":6,"last update":"Wed Sep 18 10:15:47 2024","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"min":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"max":{"1min":1.776,"5min":1.776,"15min":1.776},"last":1.167},{"interface":"front","average":{"1min":2.2069999999999999,"5min":2.2069999999999999,"15min":2.2069999999999999},"min":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"max":{"1min":15.250999999999999,"5min":15.250999999999999,"15min":15.250999999999999},"last":1.0429999999999999}]},{"osd":9,"last update":"Wed Sep 18 10:15:29 2024","interfaces":[{"interface":"back","average":{"1min":1.407,"5min":1.407,"15min":1.407},"min":{"1min":0.879,"5min":0.879,"15min":0.879},"max":{"1min":1.855,"5min":1.855,"15min":1.855},"last":1.3240000000000001},{"interface":"front","average":{"1min":10.532999999999999,"5min":10.532999999999999,"15min":10.532999999999999},"min":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"max":{"1min":130.441,"5min":130.441,"15min":130.441},"last":1.024}]},{"osd":11,"last update":"Wed Sep 18 10:15:29 2024","interfaces":[{"interface":"back","average":{"1min":1.54,"5min":1.54,"15min":1.54},"min":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"max":{"1min":2.0209999999999999,"5min":2.0209999999999999,"15min":2.0209999999999999},"last":1.359},{"interface":"front","average":{"1min":5.327,"5min":5.327,"15min":5.327},"min":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":54.805999999999997,"5min":54.805999999999997,"15min":54.805999999999997},"last":1.4159999999999999}]},{"osd":12,"last update":"Wed Sep 18 10:15:29 2024","interfaces":[{"interface":"back","average":{"1min":10.949999999999999,"5min":10.949999999999999,"15min":10.949999999999999},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":130.34800000000001,"5min":130.34800000000001,"15min":130.34800000000001},"last":0.98599999999999999},{"interface":"front","average":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.9139999999999999,"5min":1.9139999999999999,"15min":1.9139999999999999},"last":1.25}]},{"osd":14,"last update":"Wed Sep 18 10:15:34 2024","interfaces":[{"interface":"back","average":{"1min":10.039999999999999,"5min":10.039999999999999,"15min":10.039999999999999},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":130.48500000000001,"5min":130.48500000000001,"15min":130.48500000000001},"last":1.2909999999999999},{"interface":"front","average":{"1min":1.5600000000000001,"5min":1.5600000000000001,"15min":1.5600000000000001},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":2.1819999999999999,"5min":2.1819999999999999,"15min":2.1819999999999999},"last":1.4470000000000001}]},{"osd":15,"last update":"Wed Sep 18 10:15:34 2024","interfaces":[{"interface":"back","average":{"1min":3.0750000000000002,"5min":3.0750000000000002,"15min":3.0750000000000002},"min":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":23.806999999999999,"5min":23.806999999999999,"15min":23.806999999999999},"last":1.498},{"interface":"front","average":{"1min":10.122,"5min":10.122,"15min":10.122},"min":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":130.52799999999999,"5min":130.52799999999999,"15min":130.52799999999999},"last":1.395}]}]},{"osd":4,"up_from":21,"seq":90194313237,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":323280,"kb_used_data":21764,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94048560,"statfs":{"total":96636764160,"available":96305725440,"internally_reserved":0,"allocated":22286336,"data_stored":29813587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,5,6,7,8,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,1],"upper_bound":2048},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:15:29 2024","interfaces":[{"interface":"back","average":{"1min":11.151,"5min":11.151,"15min":11.151},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":89.266999999999996,"5min":89.266999999999996,"15min":89.266999999999996},"last":0.68799999999999994},{"interface":"front","average":{"1min":5.0049999999999999,"5min":5.0049999999999999,"15min":5.0049999999999999},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":46.898000000000003,"5min":46.898000000000003,"15min":46.898000000000003},"last":0.443}]},{"osd":1,"last update":"Wed Sep 18 10:15:34 2024","interfaces":[{"interface":"back","average":{"1min":14.653,"5min":14.653,"15min":14.653},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":164.21199999999999,"5min":164.21199999999999,"15min":164.21199999999999},"last":1.363},{"interface":"front","average":{"1min":11.363,"5min":11.363,"15min":11.363},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":89.331000000000003,"5min":89.331000000000003,"15min":89.331000000000003},"last":1.2310000000000001}]},{"osd":2,"last update":"Wed Sep 18 10:15:47 2024","interfaces":[{"interface":"back","average":{"1min":23.617000000000001,"5min":23.617000000000001,"15min":23.617000000000001},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":192.136,"5min":192.136,"15min":192.136},"last":1.0660000000000001},{"interface":"front","average":{"1min":11.888999999999999,"5min":11.888999999999999,"15min":11.888999999999999},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":198.87700000000001,"5min":198.87700000000001,"15min":198.87700000000001},"last":1.333}]},{"osd":3,"last update":"Wed Sep 18 10:15:34 2024","interfaces":[{"interface":"back","average":{"1min":18.170999999999999,"5min":18.170999999999999,"15min":18.170999999999999},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":137.053,"5min":137.053,"15min":137.053},"last":1.4630000000000001},{"interface":"front","average":{"1min":27.619,"5min":27.619,"15min":27.619},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":203.87899999999999,"5min":203.87899999999999,"15min":203.87899999999999},"last":1.4450000000000001}]},{"osd":5,"last update":"Wed Sep 18 10:15:34 2024","interfaces":[{"interface":"back","average":{"1min":18.940000000000001,"5min":18.940000000000001,"15min":18.940000000000001},"min":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"max":{"1min":189.55600000000001,"5min":189.55600000000001,"15min":189.55600000000001},"last":0.84399999999999997},{"interface":"front","average":{"1min":3.2469999999999999,"5min":3.2469999999999999,"15min":3.2469999999999999},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":47.518000000000001,"5min":47.518000000000001,"15min":47.518000000000001},"last":1.4019999999999999}]},{"osd":6,"last update":"Wed Sep 18 10:15:34 2024","interfaces":[{"interface":"back","average":{"1min":32.25,"5min":32.25,"15min":32.25},"min":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":197.33199999999999,"5min":197.33199999999999,"15min":197.33199999999999},"last":1.151},{"interface":"front","average":{"1min":18.963000000000001,"5min":18.963000000000001,"15min":18.963000000000001},"min":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"max":{"1min":179.19200000000001,"5min":179.19200000000001,"15min":179.19200000000001},"last":1.518}]},{"osd":7,"last update":"Wed Sep 18 10:15:34 2024","interfaces":[{"interface":"back","average":{"1min":27.597000000000001,"5min":27.597000000000001,"15min":27.597000000000001},"min":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"max":{"1min":290.036,"5min":290.036,"15min":290.036},"last":1.536},{"interface":"front","average":{"1min":17.425999999999998,"5min":17.425999999999998,"15min":17.425999999999998},"min":{"1min":1.173,"5min":1.173,"15min":1.173},"max":{"1min":139.696,"5min":139.696,"15min":139.696},"last":1.109}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3069999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6419999999999999}]},{"osd":10,"last update":"Wed Sep 18 10:15:46 2024","interfaces":[{"interface":"back","average":{"1min":22.786000000000001,"5min":22.786000000000001,"15min":22.786000000000001},"min":{"1min":1.079,"5min":1.079,"15min":1.079},"max":{"1min":164.43899999999999,"5min":164.43899999999999,"15min":164.43899999999999},"last":1.593},{"interface":"front","average":{"1min":12.59,"5min":12.59,"15min":12.59},"min":{"1min":0.878,"5min":0.878,"15min":0.878},"max":{"1min":89.442999999999998,"5min":89.442999999999998,"15min":89.442999999999998},"last":1.27}]},{"osd":11,"last update":"Wed Sep 18 10:15:29 2024","interfaces":[{"interface":"back","average":{"1min":10.817,"5min":10.817,"15min":10.817},"min":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"max":{"1min":101.666,"5min":101.666,"15min":101.666},"last":1.5589999999999999},{"interface":"front","average":{"1min":24.919,"5min":24.919,"15min":24.919},"min":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"max":{"1min":164.733,"5min":164.733,"15min":164.733},"last":1.486}]},{"osd":12,"last update":"Wed Sep 18 10:15:29 2024","interfaces":[{"interface":"back","average":{"1min":11.863,"5min":11.863,"15min":11.863},"min":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":89.400000000000006,"5min":89.400000000000006,"15min":89.400000000000006},"last":0.96199999999999997},{"interface":"front","average":{"1min":8.1699999999999999,"5min":8.1699999999999999,"15min":8.1699999999999999},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":101.699,"5min":101.699,"15min":101.699},"last":1.613}]},{"osd":13,"last update":"Wed Sep 18 10:15:47 2024","interfaces":[{"interface":"back","average":{"1min":25.314,"5min":25.314,"15min":25.314},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":199.37799999999999,"5min":199.37799999999999,"15min":199.37799999999999},"last":1.702},{"interface":"front","average":{"1min":23.385000000000002,"5min":23.385000000000002,"15min":23.385000000000002},"min":{"1min":1.022,"5min":1.022,"15min":1.022},"max":{"1min":164.53,"5min":164.53,"15min":164.53},"last":1.7250000000000001}]},{"osd":14,"last update":"Wed Sep 18 10:15:34 2024","interfaces":[{"interface":"back","average":{"1min":21.658000000000001,"5min":21.658000000000001,"15min":21.658000000000001},"min":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"max":{"1min":202.191,"5min":202.191,"15min":202.191},"last":1.6559999999999999},{"interface":"front","average":{"1min":19.306999999999999,"5min":19.306999999999999,"15min":19.306999999999999},"min":{"1min":1.163,"5min":1.163,"15min":1.163},"max":{"1min":164.35400000000001,"5min":164.35400000000001,"15min":164.35400000000001},"last":1.6830000000000001}]}]},{"osd":9,"up_from":21,"seq":90194313236,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":314160,"kb_used_data":12640,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94057680,"statfs":{"total":96636764160,"available":96315064320,"internally_reserved":0,"allocated":12943360,"data_stored":15711869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,4,5,6,8,10,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":7.2450000000000001,"5min":7.2450000000000001,"15min":7.2450000000000001},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":127.902,"5min":127.902,"15min":127.902},"last":0.97399999999999998},{"interface":"front","average":{"1min":8.3109999999999999,"5min":8.3109999999999999,"15min":8.3109999999999999},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":96.203000000000003,"5min":96.203000000000003,"15min":96.203000000000003},"last":0.81200000000000006}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77000000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.048}]},{"osd":3,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":6.226,"5min":6.226,"15min":6.226},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":98.888999999999996,"5min":98.888999999999996,"15min":98.888999999999996},"last":0.73399999999999999},{"interface":"front","average":{"1min":6.2320000000000002,"5min":6.2320000000000002,"15min":6.2320000000000002},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":98.947999999999993,"5min":98.947999999999993,"15min":98.947999999999993},"last":0.85599999999999998}]},{"osd":4,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"min":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"max":{"1min":1.5780000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.82999999999999996},{"interface":"front","average":{"1min":9.5090000000000003,"5min":9.5090000000000003,"15min":9.5090000000000003},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":71.366,"5min":71.366,"15min":71.366},"last":1.238}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.01}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.355}]},{"osd":8,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":4.5910000000000002,"5min":4.5910000000000002,"15min":4.5910000000000002},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":64.912999999999997,"5min":64.912999999999997,"15min":64.912999999999997},"last":1.2629999999999999},{"interface":"front","average":{"1min":6.2309999999999999,"5min":6.2309999999999999,"15min":6.2309999999999999},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":57.664000000000001,"5min":57.664000000000001,"15min":57.664000000000001},"last":1.417}]},{"osd":10,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":1.302,"5min":1.302,"15min":1.302},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":2.109,"5min":2.109,"15min":2.109},"last":1.391},{"interface":"front","average":{"1min":3.7799999999999998,"5min":3.7799999999999998,"15min":3.7799999999999998},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":54.167999999999999,"5min":54.167999999999999,"15min":54.167999999999999},"last":1.444}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.669}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.446}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":96}]},{"osd":14,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":10.006,"5min":10.006,"15min":10.006},"min":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":62.899000000000001,"5min":62.899000000000001,"15min":62.899000000000001},"last":1.4930000000000001},{"interface":"front","average":{"1min":15.250999999999999,"5min":15.250999999999999,"15min":15.250999999999999},"min":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"max":{"1min":155.61000000000001,"5min":155.61000000000001,"15min":155.61000000000001},"last":1.47}]},{"osd":15,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":1.5,"5min":1.5,"15min":1.5},"min":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"max":{"1min":1.9970000000000001,"5min":1.9970000000000001,"15min":1.9970000000000001},"last":1.538},{"interface":"front","average":{"1min":4.077,"5min":4.077,"15min":4.077},"min":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":54.335999999999999,"5min":54.335999999999999,"15min":54.335999999999999},"last":60.289000000000001}]}]},{"osd":11,"up_from":21,"seq":90194313236,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":321132,"kb_used_data":19612,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94050708,"statfs":{"total":96636764160,"available":96307924992,"internally_reserved":0,"allocated":20082688,"data_stored":24903293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,4,5,6,7,8,9,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0049999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88}]},{"osd":2,"last update":"Wed Sep 18 10:15:33 2024","interfaces":[{"interface":"back","average":{"1min":10.189,"5min":10.189,"15min":10.189},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":99.718000000000004,"5min":99.718000000000004,"15min":99.718000000000004},"last":0.97699999999999998},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":1.2150000000000001}]},{"osd":3,"last update":"Wed Sep 18 10:15:46 2024","interfaces":[{"interface":"back","average":{"1min":5.0720000000000001,"5min":5.0720000000000001,"15min":5.0720000000000001},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":85.963999999999999,"5min":85.963999999999999,"15min":85.963999999999999},"last":1.0209999999999999},{"interface":"front","average":{"1min":22.759,"5min":22.759,"15min":22.759},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":125.672,"5min":125.672,"15min":125.672},"last":1.05}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.343}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":97.091999999999999}]},{"osd":6,"last update":"Wed Sep 18 10:15:46 2024","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.157,"15min":1.157},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":5.04,"5min":5.04,"15min":5.04},"last":1.593},{"interface":"front","average":{"1min":6.9880000000000004,"5min":6.9880000000000004,"15min":6.9880000000000004},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":94.162000000000006,"5min":94.162000000000006,"15min":94.162000000000006},"last":1.4079999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4399999999999999}]},{"osd":8,"last update":"Wed Sep 18 10:15:33 2024","interfaces":[{"interface":"back","average":{"1min":6.04,"5min":6.04,"15min":6.04},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":85.900000000000006,"5min":85.900000000000006,"15min":85.900000000000006},"last":1.7470000000000001},{"interface":"front","average":{"1min":9.9849999999999994,"5min":9.9849999999999994,"15min":9.9849999999999994},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":99.765000000000001,"5min":99.765000000000001,"15min":99.765000000000001},"last":1.472}]},{"osd":9,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":5.8239999999999998,"5min":5.8239999999999998,"15min":5.8239999999999998},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":85.838999999999999,"5min":85.838999999999999,"15min":85.838999999999999},"last":1.643},{"interface":"front","average":{"1min":8.1709999999999994,"5min":8.1709999999999994,"15min":8.1709999999999994},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":99.810000000000002,"5min":99.810000000000002,"15min":99.810000000000002},"last":1.6200000000000001}]},{"osd":10,"last update":"Wed Sep 18 10:15:33 2024","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.097,"15min":1.097},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.371,"5min":1.371,"15min":1.371},"last":1.6879999999999999},{"interface":"front","average":{"1min":7.6859999999999999,"5min":7.6859999999999999,"15min":7.6859999999999999},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":89.290000000000006,"5min":89.290000000000006,"15min":89.290000000000006},"last":1.885}]},{"osd":12,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":1.125,"15min":1.125},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.6259999999999999,"5min":1.6259999999999999,"15min":1.6259999999999999},"last":1.7290000000000001},{"interface":"front","average":{"1min":10.385999999999999,"5min":10.385999999999999,"15min":10.385999999999999},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":86.025999999999996,"5min":86.025999999999996,"15min":86.025999999999996},"last":1.823}]},{"osd":13,"last update":"Wed Sep 18 10:15:33 2024","interfaces":[{"interface":"back","average":{"1min":30.405000000000001,"5min":30.405000000000001,"15min":30.405000000000001},"min":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"max":{"1min":189.25899999999999,"5min":189.25899999999999,"15min":189.25899999999999},"last":1.669},{"interface":"front","average":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":2.1720000000000002,"5min":2.1720000000000002,"15min":2.1720000000000002},"last":1.8080000000000001}]},{"osd":14,"last update":"Wed Sep 18 10:15:33 2024","interfaces":[{"interface":"back","average":{"1min":6.423,"5min":6.423,"15min":6.423},"min":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":86.109999999999999,"5min":86.109999999999999,"15min":86.109999999999999},"last":1.948},{"interface":"front","average":{"1min":8.9819999999999993,"5min":8.9819999999999993,"15min":8.9819999999999993},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":102.625,"5min":102.625,"15min":102.625},"last":1.7150000000000001}]},{"osd":15,"last update":"Wed Sep 18 10:15:33 2024","interfaces":[{"interface":"back","average":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":2.0249999999999999,"5min":2.0249999999999999,"15min":2.0249999999999999},"last":1.7689999999999999},{"interface":"front","average":{"1min":6.484,"5min":6.484,"15min":6.484},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":86.165999999999997,"5min":86.165999999999997,"15min":86.165999999999997},"last":2.0099999999999998}]}]},{"osd":12,"up_from":21,"seq":90194313237,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311636,"kb_used_data":14276,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94060204,"statfs":{"total":96636764160,"available":96317648896,"internally_reserved":0,"allocated":14618624,"data_stored":19623568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[2,3,4,6,7,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":2.3839999999999999,"5min":2.3839999999999999,"15min":2.3839999999999999},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":32.476999999999997,"5min":32.476999999999997,"15min":32.476999999999997},"last":0.64200000000000002},{"interface":"front","average":{"1min":2.625,"5min":2.625,"15min":2.625},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":32.509,"5min":32.509,"15min":32.509},"last":0.89300000000000002}]},{"osd":1,"last update":"Wed Sep 18 10:15:32 2024","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.6399999999999999,"5min":1.6399999999999999,"15min":1.6399999999999999},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.905,"5min":1.905,"15min":1.905},"last":0.81899999999999995}]},{"osd":2,"last update":"Wed Sep 18 10:15:32 2024","interfaces":[{"interface":"back","average":{"1min":1.002,"5min":1.002,"15min":1.002},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.96299999999999997},{"interface":"front","average":{"1min":4.1189999999999998,"5min":4.1189999999999998,"15min":4.1189999999999998},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":62.780999999999999,"5min":62.780999999999999,"15min":62.780999999999999},"last":1.367}]},{"osd":3,"last update":"Wed Sep 18 10:15:32 2024","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":1.145,"15min":1.145},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.772,"5min":1.772,"15min":1.772},"last":0.93799999999999994},{"interface":"front","average":{"1min":1.151,"5min":1.151,"15min":1.151},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.4710000000000001,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":1.1040000000000001}]},{"osd":4,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":1.052,"15min":1.052},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.6759999999999999,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":0.80000000000000004},{"interface":"front","average":{"1min":1.222,"5min":1.222,"15min":1.222},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.774,"5min":1.774,"15min":1.774},"last":0.874}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1850000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0449999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":94.382000000000005}]},{"osd":9,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":1.268,"5min":1.268,"15min":1.268},"min":{"1min":0.875,"5min":0.875,"15min":0.875},"max":{"1min":1.9079999999999999,"5min":1.9079999999999999,"15min":1.9079999999999999},"last":1.2569999999999999},{"interface":"front","average":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.8460000000000001,"5min":1.8460000000000001,"15min":1.8460000000000001},"last":1.4379999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.077}]},{"osd":11,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":1.365,"5min":1.365,"15min":1.365},"min":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":2.0539999999999998,"5min":2.0539999999999998,"15min":2.0539999999999998},"last":1.387},{"interface":"front","average":{"1min":1.383,"5min":1.383,"15min":1.383},"min":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"max":{"1min":1.98,"5min":1.98,"15min":1.98},"last":1.6879999999999999}]},{"osd":13,"last update":"Wed Sep 18 10:15:32 2024","interfaces":[{"interface":"back","average":{"1min":1.333,"5min":1.333,"15min":1.333},"min":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":2.016,"5min":2.016,"15min":2.016},"last":1.603},{"interface":"front","average":{"1min":1.429,"5min":1.429,"15min":1.429},"min":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"max":{"1min":1.9379999999999999,"5min":1.9379999999999999,"15min":1.9379999999999999},"last":1.5089999999999999}]},{"osd":14,"last update":"Wed Sep 18 10:15:32 2024","interfaces":[{"interface":"back","average":{"1min":1.4299999999999999,"5min":1.4299999999999999,"15min":1.4299999999999999},"min":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"max":{"1min":2.0619999999999998,"5min":2.0619999999999998,"15min":2.0619999999999998},"last":1.401},{"interface":"front","average":{"1min":1.391,"5min":1.391,"15min":1.391},"min":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":2.1339999999999999,"5min":2.1339999999999999,"15min":2.1339999999999999},"last":1.6619999999999999}]},{"osd":15,"last update":"Wed Sep 18 10:15:32 2024","interfaces":[{"interface":"back","average":{"1min":1.494,"5min":1.494,"15min":1.494},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":2.1179999999999999,"5min":2.1179999999999999,"15min":2.1179999999999999},"last":1.345},{"interface":"front","average":{"1min":1.48,"5min":1.48,"15min":1.48},"min":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"max":{"1min":2.1600000000000001,"5min":2.1600000000000001,"15min":2.1600000000000001},"last":1.5429999999999999}]}]},{"osd":1,"up_from":22,"seq":94489280531,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":317420,"kb_used_data":15904,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94054420,"statfs":{"total":96636764160,"available":96311726080,"internally_reserved":0,"allocated":16285696,"data_stored":18082643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,4,5,6,9,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:15:35 2024","interfaces":[{"interface":"back","average":{"1min":2.8700000000000001,"5min":2.8700000000000001,"15min":2.8700000000000001},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":36.268000000000001,"5min":36.268000000000001,"15min":36.268000000000001},"last":0.72899999999999998},{"interface":"front","average":{"1min":17.715,"5min":17.715,"15min":17.715},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":154.67599999999999,"5min":154.67599999999999,"15min":154.67599999999999},"last":0.78900000000000003}]},{"osd":2,"last update":"Wed Sep 18 10:15:35 2024","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.343,"5min":1.343,"15min":1.343},"last":1.0269999999999999},{"interface":"front","average":{"1min":2.9449999999999998,"5min":2.9449999999999998,"15min":2.9449999999999998},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":36.323999999999998,"5min":36.323999999999998,"15min":36.323999999999998},"last":0.93799999999999994}]},{"osd":3,"last update":"Wed Sep 18 10:15:35 2024","interfaces":[{"interface":"back","average":{"1min":12.128,"5min":12.128,"15min":12.128},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":150.642,"5min":150.642,"15min":150.642},"last":1.0720000000000001},{"interface":"front","average":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.6930000000000001,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":0.82999999999999996}]},{"osd":4,"last update":"Wed Sep 18 10:15:35 2024","interfaces":[{"interface":"back","average":{"1min":3.3439999999999999,"5min":3.3439999999999999,"15min":3.3439999999999999},"min":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":40.783000000000001,"5min":40.783000000000001,"15min":40.783000000000001},"last":0.98199999999999998},{"interface":"front","average":{"1min":4.1699999999999999,"5min":4.1699999999999999,"15min":4.1699999999999999},"min":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":50.747,"5min":50.747,"15min":50.747},"last":0.89900000000000002}]},{"osd":5,"last update":"Wed Sep 18 10:15:35 2024","interfaces":[{"interface":"back","average":{"1min":5.9720000000000004,"5min":5.9720000000000004,"15min":5.9720000000000004},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":52.317999999999998,"5min":52.317999999999998,"15min":52.317999999999998},"last":1.099},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.748,"5min":1.748,"15min":1.748},"last":1.196}]},{"osd":6,"last update":"Wed Sep 18 10:15:35 2024","interfaces":[{"interface":"back","average":{"1min":8.3490000000000002,"5min":8.3490000000000002,"15min":8.3490000000000002},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":84.939999999999998,"5min":84.939999999999998,"15min":84.939999999999998},"last":1.1279999999999999},{"interface":"front","average":{"1min":10.352,"5min":10.352,"15min":10.352},"min":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"max":{"1min":85.045000000000002,"5min":85.045000000000002,"15min":85.045000000000002},"last":1.22}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":118.43600000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7789999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3340000000000001}]},{"osd":10,"last update":"Wed Sep 18 10:15:35 2024","interfaces":[{"interface":"back","average":{"1min":3.4300000000000002,"5min":3.4300000000000002,"15min":3.4300000000000002},"min":{"1min":1.073,"5min":1.073,"15min":1.073},"max":{"1min":36.436,"5min":36.436,"15min":36.436},"last":1.4239999999999999},{"interface":"front","average":{"1min":11.914999999999999,"5min":11.914999999999999,"15min":11.914999999999999},"min":{"1min":0.88,"5min":0.88,"15min":0.88},"max":{"1min":100.402,"5min":100.402,"15min":100.402},"last":1.2769999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.825}]},{"osd":12,"last update":"Wed Sep 18 10:15:35 2024","interfaces":[{"interface":"back","average":{"1min":4.2130000000000001,"5min":4.2130000000000001,"15min":4.2130000000000001},"min":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"max":{"1min":49.976999999999997,"5min":49.976999999999997,"15min":49.976999999999997},"last":1.5640000000000001},{"interface":"front","average":{"1min":8.4000000000000004,"5min":8.4000000000000004,"15min":8.4000000000000004},"min":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"max":{"1min":124.78100000000001,"5min":124.78100000000001,"15min":124.78100000000001},"last":1.516}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3700000000000001}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.458}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5880000000000001}]}]},{"osd":2,"up_from":22,"seq":94489280531,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":323628,"kb_used_data":22108,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94048212,"statfs":{"total":96636764160,"available":96305369088,"internally_reserved":0,"allocated":22638592,"data_stored":31249035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,3,4,5,6,7,8,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 18 10:15:31 2024","interfaces":[{"interface":"back","average":{"1min":26.495000000000001,"5min":26.495000000000001,"15min":26.495000000000001},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":419.05700000000002,"5min":419.05700000000002,"15min":419.05700000000002},"last":0.83699999999999997},{"interface":"front","average":{"1min":4.9059999999999997,"5min":4.9059999999999997,"15min":4.9059999999999997},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":98.254000000000005,"5min":98.254000000000005,"15min":98.254000000000005},"last":35.899999999999999}]},{"osd":3,"last update":"Wed Sep 18 10:15:31 2024","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":9.0969999999999995},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.79100000000000004}]},{"osd":4,"last update":"Wed Sep 18 10:15:31 2024","interfaces":[{"interface":"back","average":{"1min":1.925,"5min":1.925,"15min":1.925},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":24.888999999999999,"5min":24.888999999999999,"15min":24.888999999999999},"last":35.683},{"interface":"front","average":{"1min":4.5389999999999997,"5min":4.5389999999999997,"15min":4.5389999999999997},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":84.647000000000006,"5min":84.647000000000006,"15min":84.647000000000006},"last":9.1780000000000008}]},{"osd":5,"last update":"Wed Sep 18 10:15:31 2024","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.478,"5min":1.478,"15min":1.478},"last":0.88900000000000001},{"interface":"front","average":{"1min":1.054,"5min":1.054,"15min":1.054},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.415,"5min":1.415,"15min":1.415},"last":36.119}]},{"osd":6,"last update":"Wed Sep 18 10:15:31 2024","interfaces":[{"interface":"back","average":{"1min":4.758,"5min":4.758,"15min":4.758},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":71.451999999999998,"5min":71.451999999999998,"15min":71.451999999999998},"last":8.9529999999999994},{"interface":"front","average":{"1min":4.8730000000000002,"5min":4.8730000000000002,"15min":4.8730000000000002},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":90.188999999999993,"5min":90.188999999999993,"15min":90.188999999999993},"last":0.64400000000000002}]},{"osd":7,"last update":"Wed Sep 18 10:15:31 2024","interfaces":[{"interface":"back","average":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":1.2050000000000001},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.724,"5min":1.724,"15min":1.724},"last":35.960999999999999},{"interface":"front","average":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":3.016,"5min":3.016,"15min":3.016},"last":9.2379999999999995}]},{"osd":8,"last update":"Wed Sep 18 10:15:31 2024","interfaces":[{"interface":"back","average":{"1min":5.1020000000000003,"5min":5.1020000000000003,"15min":5.1020000000000003},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":92.578999999999994,"5min":92.578999999999994,"15min":92.578999999999994},"last":0.93899999999999995},{"interface":"front","average":{"1min":2.7639999999999998,"5min":2.7639999999999998,"15min":2.7639999999999998},"min":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"max":{"1min":35.271000000000001,"5min":35.271000000000001,"15min":35.271000000000001},"last":36.088000000000001}]},{"osd":12,"last update":"Wed Sep 18 10:15:31 2024","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"min":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"max":{"1min":1.696,"5min":1.696,"15min":1.696},"last":36.014000000000003},{"interface":"front","average":{"1min":2.056,"5min":2.056,"15min":2.056},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":16.878,"5min":16.878,"15min":16.878},"last":9.359}]},{"osd":13,"last update":"Wed Sep 18 10:15:31 2024","interfaces":[{"interface":"back","average":{"1min":7.718,"5min":7.718,"15min":7.718},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":141.84200000000001,"5min":141.84200000000001,"15min":141.84200000000001},"last":0.97999999999999998},{"interface":"front","average":{"1min":5.4589999999999996,"5min":5.4589999999999996,"15min":5.4589999999999996},"min":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":97.441999999999993,"5min":97.441999999999993,"15min":97.441999999999993},"last":35.841000000000001}]},{"osd":15,"last update":"Wed Sep 18 10:15:31 2024","interfaces":[{"interface":"back","average":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":3.1480000000000001,"5min":3.1480000000000001,"15min":3.1480000000000001},"last":9.4190000000000005},{"interface":"front","average":{"1min":3.911,"5min":3.911,"15min":3.911},"min":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":61.694000000000003,"5min":61.694000000000003,"15min":61.694000000000003},"last":1.1419999999999999}]}]},{"osd":3,"up_from":22,"seq":94489280531,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":321672,"kb_used_data":20156,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94050168,"statfs":{"total":96636764160,"available":96307372032,"internally_reserved":0,"allocated":20639744,"data_stored":32013139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,4,5,6,7,9,10,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:15:32 2024","interfaces":[{"interface":"back","average":{"1min":1.5169999999999999,"5min":1.5169999999999999,"15min":1.5169999999999999},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":13.680999999999999,"5min":13.680999999999999,"15min":13.680999999999999},"last":0.83399999999999996},{"interface":"front","average":{"1min":10.788,"5min":10.788,"15min":10.788},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":71.316999999999993,"5min":71.316999999999993,"15min":71.316999999999993},"last":1.095}]},{"osd":2,"last update":"Wed Sep 18 10:15:32 2024","interfaces":[{"interface":"back","average":{"1min":5.3029999999999999,"5min":5.3029999999999999,"15min":5.3029999999999999},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":56.523000000000003,"5min":56.523000000000003,"15min":56.523000000000003},"last":1.0229999999999999},{"interface":"front","average":{"1min":1.109,"5min":1.109,"15min":1.109},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":3.327,"5min":3.327,"15min":3.327},"last":1.181}]},{"osd":4,"last update":"Wed Sep 18 10:15:32 2024","interfaces":[{"interface":"back","average":{"1min":6.8739999999999997,"5min":6.8739999999999997,"15min":6.8739999999999997},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":71.135000000000005,"5min":71.135000000000005,"15min":71.135000000000005},"last":1.248},{"interface":"front","average":{"1min":4.3579999999999997,"5min":4.3579999999999997,"15min":4.3579999999999997},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":56.253999999999998,"5min":56.253999999999998,"15min":56.253999999999998},"last":1.1559999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91400000000000003}]},{"osd":6,"last update":"Wed Sep 18 10:15:32 2024","interfaces":[{"interface":"back","average":{"1min":1.131,"5min":1.131,"15min":1.131},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.744,"5min":1.744,"15min":1.744},"last":1.2869999999999999},{"interface":"front","average":{"1min":8.7520000000000007,"5min":8.7520000000000007,"15min":8.7520000000000007},"min":{"1min":0.749,"5min":0.749,"15min":0.749},"max":{"1min":81.210999999999999,"5min":81.210999999999999,"15min":81.210999999999999},"last":1.3939999999999999}]},{"osd":7,"last update":"Wed Sep 18 10:15:32 2024","interfaces":[{"interface":"back","average":{"1min":16.132000000000001,"5min":16.132000000000001,"15min":16.132000000000001},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":129.68299999999999,"5min":129.68299999999999,"15min":129.68299999999999},"last":1.22},{"interface":"front","average":{"1min":13.532999999999999,"5min":13.532999999999999,"15min":13.532999999999999},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":129.726,"5min":129.726,"15min":129.726},"last":1.468}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0620000000000001}]},{"osd":9,"last update":"Wed Sep 18 10:15:32 2024","interfaces":[{"interface":"back","average":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.9990000000000001,"5min":1.9990000000000001,"15min":1.9990000000000001},"last":1.635},{"interface":"front","average":{"1min":6.109,"5min":6.109,"15min":6.109},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":81.171000000000006,"5min":81.171000000000006,"15min":81.171000000000006},"last":1.5940000000000001}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.913}]},{"osd":13,"last update":"Wed Sep 18 10:15:32 2024","interfaces":[{"interface":"back","average":{"1min":1.5189999999999999,"5min":1.5189999999999999,"15min":1.5189999999999999},"min":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":2.6859999999999999,"5min":2.6859999999999999,"15min":2.6859999999999999},"last":1.5529999999999999},{"interface":"front","average":{"1min":6.282,"5min":6.282,"15min":6.282},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":71.447000000000003,"5min":71.447000000000003,"15min":71.447000000000003},"last":1.7769999999999999}]},{"osd":14,"last update":"Wed Sep 18 10:15:32 2024","interfaces":[{"interface":"back","average":{"1min":5.6900000000000004,"5min":5.6900000000000004,"15min":5.6900000000000004},"min":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"max":{"1min":56.664999999999999,"5min":56.664999999999999,"15min":56.664999999999999},"last":1.6599999999999999},{"interface":"front","average":{"1min":2.363,"5min":2.363,"15min":2.363},"min":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"max":{"1min":18.523,"5min":18.523,"15min":18.523},"last":1.956}]},{"osd":15,"last update":"Wed Sep 18 10:15:45 2024","interfaces":[{"interface":"back","average":{"1min":4.5830000000000002,"5min":4.5830000000000002,"15min":4.5830000000000002},"min":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"max":{"1min":56.357999999999997,"5min":56.357999999999997,"15min":56.357999999999997},"last":4.3289999999999997},{"interface":"front","average":{"1min":10.225,"5min":10.225,"15min":10.225},"min":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":122.727,"5min":122.727,"15min":122.727},"last":1.8089999999999999}]}]},{"osd":5,"up_from":22,"seq":94489280531,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":315812,"kb_used_data":14292,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94056028,"statfs":{"total":96636764160,"available":96313372672,"internally_reserved":0,"allocated":14635008,"data_stored":23134859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,4,6,7,8,9,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:15:47 2024","interfaces":[{"interface":"back","average":{"1min":7.8860000000000001,"5min":7.8860000000000001,"15min":7.8860000000000001},"min":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":84.430999999999997,"5min":84.430999999999997,"15min":84.430999999999997},"last":1.1020000000000001},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.96599999999999997}]},{"osd":2,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":9.8339999999999996,"5min":9.8339999999999996,"15min":9.8339999999999996},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":84.566000000000003,"5min":84.566000000000003,"15min":84.566000000000003},"last":0.89300000000000002},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":9.5069999999999997,"5min":9.5069999999999997,"15min":9.5069999999999997},"last":1.2929999999999999}]},{"osd":3,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":8.5210000000000008,"5min":8.5210000000000008,"15min":8.5210000000000008},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":84.358999999999995,"5min":84.358999999999995,"15min":84.358999999999995},"last":1.345},{"interface":"front","average":{"1min":8.2230000000000008,"5min":8.2230000000000008,"15min":8.2230000000000008},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":97.549999999999997,"5min":97.549999999999997,"15min":97.549999999999997},"last":1.196}]},{"osd":4,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":31.829999999999998,"5min":31.829999999999998,"15min":31.829999999999998},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":266.911,"5min":266.911,"15min":266.911},"last":1.018},{"interface":"front","average":{"1min":22.798999999999999,"5min":22.798999999999999,"15min":22.798999999999999},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":260.53500000000003,"5min":260.53500000000003,"15min":260.53500000000003},"last":1.2270000000000001}]},{"osd":6,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":9.1940000000000008,"5min":9.1940000000000008,"15min":9.1940000000000008},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":58.171999999999997,"5min":58.171999999999997,"15min":58.171999999999997},"last":1.0640000000000001},{"interface":"front","average":{"1min":21.391999999999999,"5min":21.391999999999999,"15min":21.391999999999999},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":227.48099999999999,"5min":227.48099999999999,"15min":227.48099999999999},"last":20.27}]},{"osd":7,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":12.013,"5min":12.013,"15min":12.013},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":123.111,"5min":123.111,"15min":123.111},"last":1.409},{"interface":"front","average":{"1min":13.994,"5min":13.994,"15min":13.994},"min":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"max":{"1min":84.617000000000004,"5min":84.617000000000004,"15min":84.617000000000004},"last":1.4379999999999999}]},{"osd":8,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":20.352,"5min":20.352,"15min":20.352},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":260.44200000000001,"5min":260.44200000000001,"15min":260.44200000000001},"last":1.5529999999999999},{"interface":"front","average":{"1min":7.0430000000000001,"5min":7.0430000000000001,"15min":7.0430000000000001},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":106.818,"5min":106.818,"15min":106.818},"last":1.3740000000000001}]},{"osd":9,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":5.6470000000000002,"5min":5.6470000000000002,"15min":5.6470000000000002},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":80.272000000000006,"5min":80.272000000000006,"15min":80.272000000000006},"last":1.26},{"interface":"front","average":{"1min":1.8600000000000001,"5min":1.8600000000000001,"15min":1.8600000000000001},"min":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"max":{"1min":9.5690000000000008,"5min":9.5690000000000008,"15min":9.5690000000000008},"last":1.488}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6479999999999999}]},{"osd":11,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":9.3919999999999995,"5min":9.3919999999999995,"15min":9.3919999999999995},"min":{"1min":0.997,"5min":0.997,"15min":0.997},"max":{"1min":123.158,"5min":123.158,"15min":123.158},"last":1.752},{"interface":"front","average":{"1min":11.367000000000001,"5min":11.367000000000001,"15min":11.367000000000001},"min":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"max":{"1min":84.522999999999996,"5min":84.522999999999996,"15min":84.522999999999996},"last":1.792}]},{"osd":12,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":1.7789999999999999,"5min":1.7789999999999999,"15min":1.7789999999999999},"last":1.675},{"interface":"front","average":{"1min":1.9750000000000001,"5min":1.9750000000000001,"15min":1.9750000000000001},"min":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"max":{"1min":9.7710000000000008,"5min":9.7710000000000008,"15min":9.7710000000000008},"last":1.8500000000000001}]},{"osd":13,"last update":"Wed Sep 18 10:15:47 2024","interfaces":[{"interface":"back","average":{"1min":24.271000000000001,"5min":24.271000000000001,"15min":24.271000000000001},"min":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"max":{"1min":140.273,"5min":140.273,"15min":140.273},"last":1.617},{"interface":"front","average":{"1min":24.221,"5min":24.221,"15min":24.221},"min":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"max":{"1min":170.86000000000001,"5min":170.86000000000001,"15min":170.86000000000001},"last":1.72}]},{"osd":14,"last update":"Wed Sep 18 10:15:47 2024","interfaces":[{"interface":"back","average":{"1min":13.257,"5min":13.257,"15min":13.257},"min":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"max":{"1min":101.733,"5min":101.733,"15min":101.733},"last":1.5920000000000001},{"interface":"front","average":{"1min":9.5190000000000001,"5min":9.5190000000000001,"15min":9.5190000000000001},"min":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"max":{"1min":122.952,"5min":122.952,"15min":122.952},"last":1.9199999999999999}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8839999999999999}]}]},{"osd":6,"up_from":22,"seq":94489280531,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":319292,"kb_used_data":17768,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94052548,"statfs":{"total":96636764160,"available":96309809152,"internally_reserved":0,"allocated":18194432,"data_stored":24468851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,3,4,5,7,8,9,11,12,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:15:47 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":76.555999999999997}]},{"osd":1,"last update":"Wed Sep 18 10:15:47 2024","interfaces":[{"interface":"back","average":{"1min":4.5549999999999997,"5min":4.5549999999999997,"15min":4.5549999999999997},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":52.332999999999998,"5min":52.332999999999998,"15min":52.332999999999998},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":1.111}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.343}]},{"osd":3,"last update":"Wed Sep 18 10:15:47 2024","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.492,"5min":1.492,"15min":1.492},"last":76.891999999999996},{"interface":"front","average":{"1min":14.007,"5min":14.007,"15min":14.007},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":169.244,"5min":169.244,"15min":169.244},"last":1.083}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.161}]},{"osd":5,"last update":"Wed Sep 18 10:15:32 2024","interfaces":[{"interface":"back","average":{"1min":11.228999999999999,"5min":11.228999999999999,"15min":11.228999999999999},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":119.032,"5min":119.032,"15min":119.032},"last":1.23},{"interface":"front","average":{"1min":3.629,"5min":3.629,"15min":3.629},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":54.731000000000002,"5min":54.731000000000002,"15min":54.731000000000002},"last":83.858000000000004}]},{"osd":7,"last update":"Wed Sep 18 10:15:32 2024","interfaces":[{"interface":"back","average":{"1min":6.7569999999999997,"5min":6.7569999999999997,"15min":6.7569999999999997},"min":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":52.204999999999998,"5min":52.204999999999998,"15min":52.204999999999998},"last":1.266},{"interface":"front","average":{"1min":3.0550000000000002,"5min":3.0550000000000002,"15min":3.0550000000000002},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":40.848999999999997,"5min":40.848999999999997,"15min":40.848999999999997},"last":1.397}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":76.766000000000005}]},{"osd":9,"last update":"Wed Sep 18 10:15:32 2024","interfaces":[{"interface":"back","average":{"1min":8.2799999999999994,"5min":8.2799999999999994,"15min":8.2799999999999994},"min":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"max":{"1min":95.691999999999993,"5min":95.691999999999993,"15min":95.691999999999993},"last":76.997},{"interface":"front","average":{"1min":12.766999999999999,"5min":12.766999999999999,"15min":12.766999999999999},"min":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"max":{"1min":165.31100000000001,"5min":165.31100000000001,"15min":165.31100000000001},"last":1.4219999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.329}]},{"osd":11,"last update":"Wed Sep 18 10:15:32 2024","interfaces":[{"interface":"back","average":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.7749999999999999,"5min":1.7749999999999999,"15min":1.7749999999999999},"last":1.4470000000000001},{"interface":"front","average":{"1min":3.883,"5min":3.883,"15min":3.883},"min":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"max":{"1min":54.823,"5min":54.823,"15min":54.823},"last":77.158000000000001}]},{"osd":12,"last update":"Wed Sep 18 10:15:32 2024","interfaces":[{"interface":"back","average":{"1min":5.0469999999999997,"5min":5.0469999999999997,"15min":5.0469999999999997},"min":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":52.378,"5min":52.378,"15min":52.378},"last":1.46},{"interface":"front","average":{"1min":4.4379999999999997,"5min":4.4379999999999997,"15min":4.4379999999999997},"min":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":64.760999999999996,"5min":64.760999999999996,"15min":64.760999999999996},"last":1.5229999999999999}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2050000000000001}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2669999999999999}]},{"osd":15,"last update":"Wed Sep 18 10:15:32 2024","interfaces":[{"interface":"back","average":{"1min":1.474,"5min":1.474,"15min":1.474},"min":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"max":{"1min":3.1840000000000002,"5min":3.1840000000000002,"15min":3.1840000000000002},"last":1.484},{"interface":"front","average":{"1min":3.9729999999999999,"5min":3.9729999999999999,"15min":3.9729999999999999},"min":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"max":{"1min":54.869999999999997,"5min":54.869999999999997,"15min":54.869999999999997},"last":77.081999999999994}]}]},{"osd":7,"up_from":22,"seq":94489280531,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":319332,"kb_used_data":17812,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94052508,"statfs":{"total":96636764160,"available":96309768192,"internally_reserved":0,"allocated":18239488,"data_stored":28988043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,4,5,6,8,10,11,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:15:33 2024","interfaces":[{"interface":"back","average":{"1min":12.721,"5min":12.721,"15min":12.721},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":139.58600000000001,"5min":139.58600000000001,"15min":139.58600000000001},"last":1.532},{"interface":"front","average":{"1min":0.999,"5min":0.999,"15min":0.999},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":1.335}]},{"osd":2,"last update":"Wed Sep 18 10:15:46 2024","interfaces":[{"interface":"back","average":{"1min":15.436,"5min":15.436,"15min":15.436},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":139.36799999999999,"5min":139.36799999999999,"15min":139.36799999999999},"last":1.202},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":1.02}]},{"osd":3,"last update":"Wed Sep 18 10:15:46 2024","interfaces":[{"interface":"back","average":{"1min":17.327999999999999,"5min":17.327999999999999,"15min":17.327999999999999},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":139.27199999999999,"5min":139.27199999999999,"15min":139.27199999999999},"last":0.93100000000000005},{"interface":"front","average":{"1min":3.964,"5min":3.964,"15min":3.964},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":23.748999999999999,"5min":23.748999999999999,"15min":23.748999999999999},"last":0.85599999999999998}]},{"osd":4,"last update":"Wed Sep 18 10:15:37 2024","interfaces":[{"interface":"back","average":{"1min":19.146999999999998,"5min":19.146999999999998,"15min":19.146999999999998},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":139.51599999999999,"5min":139.51599999999999,"15min":139.51599999999999},"last":1.3779999999999999},{"interface":"front","average":{"1min":4.9989999999999997,"5min":4.9989999999999997,"15min":4.9989999999999997},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":27.882999999999999,"5min":27.882999999999999,"15min":27.882999999999999},"last":1.091}]},{"osd":5,"last update":"Wed Sep 18 10:15:46 2024","interfaces":[{"interface":"back","average":{"1min":1.099,"5min":1.099,"15min":1.099},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":2.3620000000000001,"5min":2.3620000000000001,"15min":2.3620000000000001},"last":73.328999999999994},{"interface":"front","average":{"1min":17.736999999999998,"5min":17.736999999999998,"15min":17.736999999999998},"min":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"max":{"1min":139.44300000000001,"5min":139.44300000000001,"15min":139.44300000000001},"last":1.673}]},{"osd":6,"last update":"Wed Sep 18 10:15:33 2024","interfaces":[{"interface":"back","average":{"1min":17.375,"5min":17.375,"15min":17.375},"min":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":183.001,"5min":183.001,"15min":183.001},"last":1.7569999999999999},{"interface":"front","average":{"1min":4.3109999999999999,"5min":4.3109999999999999,"15min":4.3109999999999999},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":23.978999999999999,"5min":23.978999999999999,"15min":23.978999999999999},"last":1.4990000000000001}]},{"osd":8,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.25,"15min":1.25},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":2.6139999999999999,"5min":2.6139999999999999,"15min":2.6139999999999999},"last":1.786},{"interface":"front","average":{"1min":15.906000000000001,"5min":15.906000000000001,"15min":15.906000000000001},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":139.798,"5min":139.798,"15min":139.798},"last":1.895}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.861}]},{"osd":10,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":6.5039999999999996,"5min":6.5039999999999996,"15min":6.5039999999999996},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":32.539999999999999,"5min":32.539999999999999,"15min":32.539999999999999},"last":1.5780000000000001},{"interface":"front","average":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":2.254,"5min":2.254,"15min":2.254},"last":1.635}]},{"osd":11,"last update":"Wed Sep 18 10:15:33 2024","interfaces":[{"interface":"back","average":{"1min":5.5469999999999997,"5min":5.5469999999999997,"15min":5.5469999999999997},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":62.356000000000002,"5min":62.356000000000002,"15min":62.356000000000002},"last":1.4199999999999999},{"interface":"front","average":{"1min":1.411,"5min":1.411,"15min":1.411},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":2.738,"5min":2.738,"15min":2.738},"last":1.728}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.885}]},{"osd":13,"last update":"Wed Sep 18 10:15:33 2024","interfaces":[{"interface":"back","average":{"1min":18.379000000000001,"5min":18.379000000000001,"15min":18.379000000000001},"min":{"1min":0.874,"5min":0.874,"15min":0.874},"max":{"1min":139.655,"5min":139.655,"15min":139.655},"last":1.8240000000000001},{"interface":"front","average":{"1min":19.879000000000001,"5min":19.879000000000001,"15min":19.879000000000001},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":173.113,"5min":173.113,"15min":173.113},"last":1.7030000000000001}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5900000000000001}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5509999999999999}]}]},{"osd":8,"up_from":22,"seq":94489280531,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":322596,"kb_used_data":21080,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94049244,"statfs":{"total":96636764160,"available":96306425856,"internally_reserved":0,"allocated":21585920,"data_stored":24165203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,4,5,6,7,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93300000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2}]},{"osd":2,"last update":"Wed Sep 18 10:15:46 2024","interfaces":[{"interface":"back","average":{"1min":6.5880000000000001,"5min":6.5880000000000001,"15min":6.5880000000000001},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":77.435000000000002,"5min":77.435000000000002,"15min":77.435000000000002},"last":0.97899999999999998},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.526,"5min":1.526,"15min":1.526},"last":0.878}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.306}]},{"osd":4,"last update":"Wed Sep 18 10:15:46 2024","interfaces":[{"interface":"back","average":{"1min":11.678000000000001,"5min":11.678000000000001,"15min":11.678000000000001},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":90.438000000000002,"5min":90.438000000000002,"15min":90.438000000000002},"last":0.92900000000000005},{"interface":"front","average":{"1min":13.071999999999999,"5min":13.071999999999999,"15min":13.071999999999999},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":93.939999999999998,"5min":93.939999999999998,"15min":93.939999999999998},"last":1.0820000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4199999999999999}]},{"osd":6,"last update":"Wed Sep 18 10:15:46 2024","interfaces":[{"interface":"back","average":{"1min":18.465,"5min":18.465,"15min":18.465},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":293.26900000000001,"5min":293.26900000000001,"15min":293.26900000000001},"last":1.4930000000000001},{"interface":"front","average":{"1min":6.1559999999999997,"5min":6.1559999999999997,"15min":6.1559999999999997},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":70.203000000000003,"5min":70.203000000000003,"15min":70.203000000000003},"last":1.4590000000000001}]},{"osd":7,"last update":"Wed Sep 18 10:15:33 2024","interfaces":[{"interface":"back","average":{"1min":2.71,"5min":2.71,"15min":2.71},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":26.361000000000001,"5min":26.361000000000001,"15min":26.361000000000001},"last":80.635000000000005},{"interface":"front","average":{"1min":6.0709999999999997,"5min":6.0709999999999997,"15min":6.0709999999999997},"min":{"1min":0.878,"5min":0.878,"15min":0.878},"max":{"1min":79.186999999999998,"5min":79.186999999999998,"15min":79.186999999999998},"last":1.752}]},{"osd":9,"last update":"Wed Sep 18 10:15:33 2024","interfaces":[{"interface":"back","average":{"1min":2.5459999999999998,"5min":2.5459999999999998,"15min":2.5459999999999998},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":23.934000000000001,"5min":23.934000000000001,"15min":23.934000000000001},"last":1.651},{"interface":"front","average":{"1min":8.9060000000000006,"5min":8.9060000000000006,"15min":8.9060000000000006},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":98.914000000000001,"5min":98.914000000000001,"15min":98.914000000000001},"last":1.8700000000000001}]},{"osd":10,"last update":"Wed Sep 18 10:15:33 2024","interfaces":[{"interface":"back","average":{"1min":6.9119999999999999,"5min":6.9119999999999999,"15min":6.9119999999999999},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":90.751000000000005,"5min":90.751000000000005,"15min":90.751000000000005},"last":1.8},{"interface":"front","average":{"1min":9.3390000000000004,"5min":9.3390000000000004,"15min":9.3390000000000004},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":83.795000000000002,"5min":83.795000000000002,"15min":83.795000000000002},"last":1.8280000000000001}]},{"osd":11,"last update":"Wed Sep 18 10:15:33 2024","interfaces":[{"interface":"back","average":{"1min":2.9009999999999998,"5min":2.9009999999999998,"15min":2.9009999999999998},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":26.462,"5min":26.462,"15min":26.462},"last":1.95},{"interface":"front","average":{"1min":1.998,"5min":1.998,"15min":1.998},"min":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"max":{"1min":12.475,"5min":12.475,"15min":12.475},"last":1.9730000000000001}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7729999999999999}]},{"osd":13,"last update":"Wed Sep 18 10:15:33 2024","interfaces":[{"interface":"back","average":{"1min":9.1539999999999999,"5min":9.1539999999999999,"15min":9.1539999999999999},"min":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"max":{"1min":94.222999999999999,"5min":94.222999999999999,"15min":94.222999999999999},"last":1.895},{"interface":"front","average":{"1min":2.7240000000000002,"5min":2.7240000000000002,"15min":2.7240000000000002},"min":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"max":{"1min":23.983000000000001,"5min":23.983000000000001,"15min":23.983000000000001},"last":1.7250000000000001}]},{"osd":14,"last update":"Wed Sep 18 10:15:33 2024","interfaces":[{"interface":"back","average":{"1min":20.768999999999998,"5min":20.768999999999998,"15min":20.768999999999998},"min":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":150.93600000000001,"5min":150.93600000000001,"15min":150.93600000000001},"last":2.048},{"interface":"front","average":{"1min":3.5979999999999999,"5min":3.5979999999999999,"15min":3.5979999999999999},"min":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"max":{"1min":23.006,"5min":23.006,"15min":23.006},"last":1.706}]},{"osd":15,"last update":"Wed Sep 18 10:15:33 2024","interfaces":[{"interface":"back","average":{"1min":2.7949999999999999,"5min":2.7949999999999999,"15min":2.7949999999999999},"min":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"max":{"1min":24.100000000000001,"5min":24.100000000000001,"15min":24.100000000000001},"last":1.5640000000000001},{"interface":"front","average":{"1min":3.056,"5min":3.056,"15min":3.056},"min":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"max":{"1min":26.414000000000001,"5min":26.414000000000001,"15min":26.414000000000001},"last":2.0009999999999999}]}]},{"osd":10,"up_from":22,"seq":94489280532,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":320744,"kb_used_data":19228,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94051096,"statfs":{"total":96636764160,"available":96308322304,"internally_reserved":0,"allocated":19689472,"data_stored":26823507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,6,7,9,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":145.12799999999999}]},{"osd":1,"last update":"Wed Sep 18 10:15:33 2024","interfaces":[{"interface":"back","average":{"1min":10.933,"5min":10.933,"15min":10.933},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":84.203999999999994,"5min":84.203999999999994,"15min":84.203999999999994},"last":145.20699999999999},{"interface":"front","average":{"1min":17.222999999999999,"5min":17.222999999999999,"15min":17.222999999999999},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":99.995999999999995,"5min":99.995999999999995,"15min":99.995999999999995},"last":38.338000000000001}]},{"osd":2,"last update":"Wed Sep 18 10:15:33 2024","interfaces":[{"interface":"back","average":{"1min":10.893000000000001,"5min":10.893000000000001,"15min":10.893000000000001},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":84.072000000000003,"5min":84.072000000000003,"15min":84.072000000000003},"last":145.17599999999999},{"interface":"front","average":{"1min":11.550000000000001,"5min":11.550000000000001,"15min":11.550000000000001},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":99.798000000000002,"5min":99.798000000000002,"15min":99.798000000000002},"last":0.71699999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89500000000000002}]},{"osd":4,"last update":"Wed Sep 18 10:15:49 2024","interfaces":[{"interface":"back","average":{"1min":25.588999999999999,"5min":25.588999999999999,"15min":25.588999999999999},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":125.23099999999999,"5min":125.23099999999999,"15min":125.23099999999999},"last":0.92300000000000004},{"interface":"front","average":{"1min":31.167999999999999,"5min":31.167999999999999,"15min":31.167999999999999},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":295.93200000000002,"5min":295.93200000000002,"15min":295.93200000000002},"last":0.93999999999999995}]},{"osd":5,"last update":"Wed Sep 18 10:15:49 2024","interfaces":[{"interface":"back","average":{"1min":24.577999999999999,"5min":24.577999999999999,"15min":24.577999999999999},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":215.51400000000001,"5min":215.51400000000001,"15min":215.51400000000001},"last":145.24000000000001},{"interface":"front","average":{"1min":27.329999999999998,"5min":27.329999999999998,"15min":27.329999999999998},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":252.69300000000001,"5min":252.69300000000001,"15min":252.69300000000001},"last":0.99399999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":45.667999999999999}]},{"osd":7,"last update":"Wed Sep 18 10:15:33 2024","interfaces":[{"interface":"back","average":{"1min":3.0590000000000002,"5min":3.0590000000000002,"15min":3.0590000000000002},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":34.558,"5min":34.558,"15min":34.558},"last":1.087},{"interface":"front","average":{"1min":13.959,"5min":13.959,"15min":13.959},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":84.123000000000005,"5min":84.123000000000005,"15min":84.123000000000005},"last":145.5}]},{"osd":9,"last update":"Wed Sep 18 10:15:33 2024","interfaces":[{"interface":"back","average":{"1min":11.747,"5min":11.747,"15min":11.747},"min":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":100.214,"5min":100.214,"15min":100.214},"last":1.139},{"interface":"front","average":{"1min":12.548,"5min":12.548,"15min":12.548},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":97.385999999999996,"5min":97.385999999999996,"15min":97.385999999999996},"last":1.224}]},{"osd":11,"last update":"Wed Sep 18 10:15:33 2024","interfaces":[{"interface":"back","average":{"1min":11.829000000000001,"5min":11.829000000000001,"15min":11.829000000000001},"min":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"max":{"1min":84.322999999999993,"5min":84.322999999999993,"15min":84.322999999999993},"last":145.26900000000001},{"interface":"front","average":{"1min":23.379000000000001,"5min":23.379000000000001,"15min":23.379000000000001},"min":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":295.70400000000001,"5min":295.70400000000001,"15min":295.70400000000001},"last":44.121000000000002}]},{"osd":12,"last update":"Wed Sep 18 10:15:33 2024","interfaces":[{"interface":"back","average":{"1min":3.1179999999999999,"5min":3.1179999999999999,"15min":3.1179999999999999},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":34.625999999999998,"5min":34.625999999999998,"15min":34.625999999999998},"last":1.1599999999999999},{"interface":"front","average":{"1min":11.198,"5min":11.198,"15min":11.198},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":84.242999999999995,"5min":84.242999999999995,"15min":84.242999999999995},"last":145.358}]},{"osd":13,"last update":"Wed Sep 18 10:15:33 2024","interfaces":[{"interface":"back","average":{"1min":19.885000000000002,"5min":19.885000000000002,"15min":19.885000000000002},"min":{"1min":0.877,"5min":0.877,"15min":0.877},"max":{"1min":100.31,"5min":100.31,"15min":100.31},"last":1.1990000000000001},{"interface":"front","average":{"1min":16.280000000000001,"5min":16.280000000000001,"15min":16.280000000000001},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":85.879999999999995,"5min":85.879999999999995,"15min":85.879999999999995},"last":1.2689999999999999}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":145.46700000000001}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1080000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8962048,"data_stored":13230080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":14823424,"data_stored":17235968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20729856,"data_stored":29462528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":18206720,"data_stored":29134848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20824064,"data_stored":28966912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12681216,"data_stored":21217280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16199680,"data_stored":23093248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16814080,"data_stored":28225536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20385792,"data_stored":23580672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":11415552,"data_stored":14823424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":18247680,"data_stored":26624000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":11685888,"data_stored":15036416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":13086720,"data_stored":18735104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":24100864,"data_stored":36208640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":23359488,"data_stored":35414016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":8769536,"data_stored":12984320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T10:16:00.134 INFO:tasks.ceph.ceph_manager.ceph:increase pool size by 10 2024-09-18T10:16:00.134 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set unique_pool_0 pg_num 36 2024-09-18T10:16:00.136 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-18T10:16:00.136 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:16:00.136 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_blocked_ops 2024-09-18T10:16:00.137 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-18T10:16:00.131+0000 7f578c52f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22134) UID: 0 2024-09-18T10:16:00.236 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-18T10:16:00.236 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:16:00.237 INFO:tasks.ceph.osd.8.smithi029.stderr:2024-09-18T10:16:00.229+0000 7f2555c27700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 22192) UID: 0 2024-09-18T10:16:00.246 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_historic_ops 2024-09-18T10:16:00.337 INFO:tasks.ceph.osd.11.smithi189.stderr:2024-09-18T10:16:00.333+0000 7f718b60a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 22400) UID: 0 2024-09-18T10:16:00.341 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-18T10:16:00.341 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:16:00.350 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T10:16:00.401 INFO:tasks.rados.rados.0.smithi189.stdout:398: finishing write tid 1 to smithi18925896-32 2024-09-18T10:16:00.438 INFO:tasks.ceph.osd.4.smithi029.stderr:2024-09-18T10:16:00.433+0000 7f550cf6e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22189) UID: 0 2024-09-18T10:16:00.458 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-09-18T10:16:00.458 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:16:00.467 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T10:16:00.524 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T10:16:00.566 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-09-18T10:16:00.566 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:16:00.574 INFO:tasks.ceph.osd.8.smithi029.stderr:2024-09-18T10:16:00.569+0000 7f2555c27700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 22192) UID: 0 2024-09-18T10:16:00.575 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T10:16:00.585 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-18T10:16:00.640 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-09-18T10:16:00.638+0000 7fdfa8698700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22087) UID: 0 2024-09-18T10:16:00.670 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-09-18T10:16:00.670 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:16:00.670 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-18T10:16:00.679 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T10:16:00.694 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-18T10:16:00.713 INFO:tasks.rados.rados.0.smithi189.stdout:398: finishing write tid 2 to smithi18925896-32 2024-09-18T10:16:00.721 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-18T10:16:00.741 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-09-18T10:16:00.738+0000 7fdfa8698700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22087) UID: 0 2024-09-18T10:16:00.747 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-18T10:16:00.766 INFO:teuthology.orchestra.run.smithi029.stderr:set pool 3 pg_num to 36 2024-09-18T10:16:00.769 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-18T10:16:00.791 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-18T10:16:00.824 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-09-18T10:16:00.824 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:16:00.833 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T10:16:00.842 INFO:tasks.ceph.osd.6.smithi154.stderr:2024-09-18T10:16:00.838+0000 7f0abc9e1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22089) UID: 0 2024-09-18T10:16:00.886 INFO:teuthology.orchestra.run.smithi029.stdout:osd.6: {} 2024-09-18T10:16:00.909 INFO:teuthology.orchestra.run.smithi029.stdout:osd.7: {} 2024-09-18T10:16:00.930 INFO:teuthology.orchestra.run.smithi029.stdout:osd.8: {} 2024-09-18T10:16:00.930 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-09-18T10:16:00.930 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-09-18T10:16:00.930 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-09-18T10:16:00.930 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-09-18T10:16:00.930 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-09-18T10:16:00.939 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T10:16:00.952 INFO:teuthology.orchestra.run.smithi029.stdout:osd.9: {} 2024-09-18T10:16:00.973 INFO:teuthology.orchestra.run.smithi029.stdout:osd.10: {} 2024-09-18T10:16:00.984 INFO:tasks.ceph.osd.6.smithi154.stderr:2024-09-18T10:16:00.982+0000 7f0abc9e1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22089) UID: 0 2024-09-18T10:16:00.995 INFO:teuthology.orchestra.run.smithi029.stdout:osd.11: {} 2024-09-18T10:16:01.016 INFO:tasks.rados.rados.0.smithi189.stdout:398: finishing write tid 3 to smithi18925896-32 2024-09-18T10:16:01.016 INFO:tasks.rados.rados.0.smithi189.stdout:update_object_version oid 32 v 116 (ObjNum 149 snap 32 seq_num 149) dirty exists 2024-09-18T10:16:01.016 INFO:tasks.rados.rados.0.smithi189.stdout:398: left oid 32 (ObjNum 149 snap 32 seq_num 149) 2024-09-18T10:16:01.016 INFO:tasks.rados.rados.0.smithi189.stdout:398: done (0 left) 2024-09-18T10:16:01.019 INFO:teuthology.orchestra.run.smithi029.stdout:osd.12: {} 2024-09-18T10:16:01.023 INFO:tasks.rados.rados.0.smithi189.stderr:0 errors. 2024-09-18T10:16:01.023 INFO:tasks.rados.rados.0.smithi189.stderr: 2024-09-18T10:16:01.029 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-18T10:16:01.040 INFO:teuthology.orchestra.run.smithi029.stdout:osd.13: {} 2024-09-18T10:16:01.043 INFO:tasks.ceph.osd.10.smithi154.stderr:2024-09-18T10:16:01.042+0000 7fb73221a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 22091) UID: 0 2024-09-18T10:16:01.067 INFO:teuthology.orchestra.run.smithi029.stdout:osd.14: {} 2024-09-18T10:16:01.087 INFO:teuthology.orchestra.run.smithi029.stdout:osd.15: {} 2024-09-18T10:16:01.091 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-09-18T10:16:01.091 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-09-18T10:16:01.091 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-09-18T10:16:01.091 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-09-18T10:16:01.091 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.091 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T10:16:01.091 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:36.786936+0000", 2024-09-18T10:16:01.091 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 84.299614770999995, 2024-09-18T10:16:01.091 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.71130157999999999, 2024-09-18T10:16:01.092 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:01.092 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:01.092 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:01.092 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:16:01.092 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:16:01.092 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 72 2024-09-18T10:16:01.092 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.092 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:01.092 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.092 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:01.092 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.786936+0000", 2024-09-18T10:16:01.092 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:01.092 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.092 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.093 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:01.093 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.786936+0000", 2024-09-18T10:16:01.093 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:01.093 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.093 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.093 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:01.093 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.786935+0000", 2024-09-18T10:16:01.093 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999995 2024-09-18T10:16:01.093 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.093 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.093 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:01.093 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787018+0000", 2024-09-18T10:16:01.094 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.2511999999999999e-05 2024-09-18T10:16:01.094 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.094 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.094 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:01.094 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787028+0000", 2024-09-18T10:16:01.094 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0709e-05 2024-09-18T10:16:01.094 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.094 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.094 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:01.095 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787034+0000", 2024-09-18T10:16:01.095 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.7769999999999999e-06 2024-09-18T10:16:01.095 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.095 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.095 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:01.095 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.295095+0000", 2024-09-18T10:16:01.095 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.50806109099999996 2024-09-18T10:16:01.096 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.096 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.096 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:01.096 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.295221+0000", 2024-09-18T10:16:01.096 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00012563599999999999 2024-09-18T10:16:01.096 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.096 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.096 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:16:01.096 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.295453+0000", 2024-09-18T10:16:01.096 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000232387 2024-09-18T10:16:01.097 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.097 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.097 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:16:01.097 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.301373+0000", 2024-09-18T10:16:01.097 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0059202279999999996 2024-09-18T10:16:01.097 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.097 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.097 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:16:01.097 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.498044+0000", 2024-09-18T10:16:01.097 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.19667104699999999 2024-09-18T10:16:01.097 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.097 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.098 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:01.098 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.498133+0000", 2024-09-18T10:16:01.098 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.9044999999999999e-05 2024-09-18T10:16:01.098 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.098 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.098 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:01.098 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.498237+0000", 2024-09-18T10:16:01.098 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00010385100000000001 2024-09-18T10:16:01.098 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:01.098 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:01.098 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:01.098 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.099 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.099 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T10:16:01.099 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:36.787042+0000", 2024-09-18T10:16:01.099 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 84.299508063999994, 2024-09-18T10:16:01.099 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.71152325100000002, 2024-09-18T10:16:01.099 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:01.099 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:01.099 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:01.099 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:16:01.099 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:16:01.100 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 73 2024-09-18T10:16:01.100 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.100 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:01.100 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.100 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:01.100 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787042+0000", 2024-09-18T10:16:01.100 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:01.100 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.100 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.100 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:01.101 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787042+0000", 2024-09-18T10:16:01.101 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:01.101 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.101 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.101 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:01.101 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787042+0000", 2024-09-18T10:16:01.101 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:16:01.101 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.101 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.101 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:01.102 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787048+0000", 2024-09-18T10:16:01.102 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.8900000000000001e-06 2024-09-18T10:16:01.102 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.102 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.102 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:01.102 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787057+0000", 2024-09-18T10:16:01.103 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.4430000000000003e-06 2024-09-18T10:16:01.103 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.103 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.103 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:01.103 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787063+0000", 2024-09-18T10:16:01.103 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.6459999999999998e-06 2024-09-18T10:16:01.103 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.103 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.103 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:01.103 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.297548+0000", 2024-09-18T10:16:01.103 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.51048589499999997 2024-09-18T10:16:01.103 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.103 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.104 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:01.104 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.297668+0000", 2024-09-18T10:16:01.104 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00011917 2024-09-18T10:16:01.104 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.104 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.104 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:16:01.104 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.297891+0000", 2024-09-18T10:16:01.104 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00022301999999999999 2024-09-18T10:16:01.104 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.104 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.104 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:16:01.104 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.301383+0000", 2024-09-18T10:16:01.104 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.003492762 2024-09-18T10:16:01.105 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.105 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.105 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:16:01.105 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.498374+0000", 2024-09-18T10:16:01.105 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.19699053899999999 2024-09-18T10:16:01.105 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.105 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.105 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:01.105 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.498506+0000", 2024-09-18T10:16:01.105 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00013246899999999999 2024-09-18T10:16:01.105 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.105 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.105 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:01.105 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.498566+0000", 2024-09-18T10:16:01.106 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.9274000000000002e-05 2024-09-18T10:16:01.106 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:01.106 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:01.106 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:01.106 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.106 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.106 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T10:16:01.106 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:36.787128+0000", 2024-09-18T10:16:01.106 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 84.299422755999998, 2024-09-18T10:16:01.106 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.71169541800000002, 2024-09-18T10:16:01.106 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:01.106 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:01.106 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:01.107 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:16:01.107 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:16:01.107 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 74 2024-09-18T10:16:01.107 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.107 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:01.107 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.107 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:01.107 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787128+0000", 2024-09-18T10:16:01.107 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:01.107 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.107 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.107 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:01.107 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787128+0000", 2024-09-18T10:16:01.107 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:01.107 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.108 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.108 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:01.108 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787127+0000", 2024-09-18T10:16:01.108 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:16:01.108 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.108 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.108 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:01.108 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787133+0000", 2024-09-18T10:16:01.108 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.6930000000000004e-06 2024-09-18T10:16:01.108 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.108 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.108 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:01.108 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787142+0000", 2024-09-18T10:16:01.109 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.1959999999999997e-06 2024-09-18T10:16:01.109 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.109 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.109 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:01.109 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787150+0000", 2024-09-18T10:16:01.109 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.6399999999999997e-06 2024-09-18T10:16:01.109 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.109 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.109 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:01.109 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.298925+0000", 2024-09-18T10:16:01.110 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.51177523599999997 2024-09-18T10:16:01.110 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.110 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.110 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:01.110 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.299064+0000", 2024-09-18T10:16:01.110 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000139622 2024-09-18T10:16:01.110 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.110 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.110 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:16:01.110 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.299287+0000", 2024-09-18T10:16:01.110 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00022309700000000001 2024-09-18T10:16:01.110 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.111 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.111 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:16:01.111 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.301389+0000", 2024-09-18T10:16:01.111 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0021018759999999999 2024-09-18T10:16:01.111 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.111 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.111 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:16:01.111 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.498648+0000", 2024-09-18T10:16:01.111 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.19725873999999999 2024-09-18T10:16:01.111 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.112 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.112 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:01.112 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.498731+0000", 2024-09-18T10:16:01.112 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.2429000000000006e-05 2024-09-18T10:16:01.112 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.112 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.112 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:01.112 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.498823+0000", 2024-09-18T10:16:01.112 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.2655e-05 2024-09-18T10:16:01.112 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:01.112 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:01.112 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:01.113 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.113 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.113 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T10:16:01.113 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:36.787157+0000", 2024-09-18T10:16:01.113 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 84.299393531000007, 2024-09-18T10:16:01.113 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.81204889999999996, 2024-09-18T10:16:01.113 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:01.113 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:01.113 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:01.113 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:16:01.113 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:16:01.113 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 75 2024-09-18T10:16:01.113 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.114 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:01.114 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.114 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:01.114 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787157+0000", 2024-09-18T10:16:01.114 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:01.114 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.114 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.114 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:01.114 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787157+0000", 2024-09-18T10:16:01.114 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:01.114 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.114 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.114 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:01.115 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787156+0000", 2024-09-18T10:16:01.115 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:16:01.115 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.115 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.115 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:01.115 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787236+0000", 2024-09-18T10:16:01.115 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.0175999999999999e-05 2024-09-18T10:16:01.115 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.115 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.115 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:01.116 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787247+0000", 2024-09-18T10:16:01.116 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0448999999999999e-05 2024-09-18T10:16:01.116 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.116 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.116 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:01.116 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787253+0000", 2024-09-18T10:16:01.116 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.2469999999999997e-06 2024-09-18T10:16:01.116 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.116 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.116 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:01.116 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.299974+0000", 2024-09-18T10:16:01.116 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.51272114499999999 2024-09-18T10:16:01.116 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.117 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.117 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:01.117 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.300115+0000", 2024-09-18T10:16:01.117 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000141452 2024-09-18T10:16:01.117 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.117 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.117 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:16:01.117 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.300349+0000", 2024-09-18T10:16:01.117 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00023363999999999999 2024-09-18T10:16:01.117 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.117 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.117 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:16:01.117 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.301947+0000", 2024-09-18T10:16:01.118 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0015976510000000001 2024-09-18T10:16:01.118 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.118 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.118 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:16:01.118 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.599027+0000", 2024-09-18T10:16:01.118 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.29708051600000002 2024-09-18T10:16:01.118 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.118 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.118 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:01.118 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.599106+0000", 2024-09-18T10:16:01.118 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.9033e-05 2024-09-18T10:16:01.118 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.118 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.119 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:01.119 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.599206+0000", 2024-09-18T10:16:01.119 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.9561000000000004e-05 2024-09-18T10:16:01.119 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:01.119 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:01.119 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:01.119 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.119 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.119 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T10:16:01.119 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:36.787261+0000", 2024-09-18T10:16:01.119 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 84.299289880000003, 2024-09-18T10:16:01.119 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.81223649499999995, 2024-09-18T10:16:01.119 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:01.120 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:01.120 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:01.120 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:16:01.120 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:16:01.120 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 76 2024-09-18T10:16:01.120 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.120 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:01.120 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.120 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:01.120 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787261+0000", 2024-09-18T10:16:01.121 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:01.121 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.121 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.121 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:01.121 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787261+0000", 2024-09-18T10:16:01.121 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:01.121 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.121 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.121 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:01.121 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787260+0000", 2024-09-18T10:16:01.121 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:16:01.122 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.122 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.122 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:01.122 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787265+0000", 2024-09-18T10:16:01.122 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1590000000000001e-06 2024-09-18T10:16:01.122 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.122 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.122 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:01.122 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787273+0000", 2024-09-18T10:16:01.122 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.4100000000000008e-06 2024-09-18T10:16:01.122 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.123 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.123 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:01.123 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787279+0000", 2024-09-18T10:16:01.123 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.3630000000000002e-06 2024-09-18T10:16:01.123 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.123 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.123 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:01.123 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.496955+0000", 2024-09-18T10:16:01.123 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.70967585099999997 2024-09-18T10:16:01.123 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.123 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.123 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:01.123 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.497107+0000", 2024-09-18T10:16:01.123 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000152044 2024-09-18T10:16:01.124 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.124 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.124 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:16:01.124 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.497345+0000", 2024-09-18T10:16:01.124 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000238693 2024-09-18T10:16:01.124 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.124 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.124 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:16:01.124 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.599248+0000", 2024-09-18T10:16:01.124 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.101902836 2024-09-18T10:16:01.124 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.124 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.124 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:16:01.125 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.599329+0000", 2024-09-18T10:16:01.125 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.1064999999999997e-05 2024-09-18T10:16:01.125 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.125 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.125 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:01.125 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.599441+0000", 2024-09-18T10:16:01.125 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000111797 2024-09-18T10:16:01.125 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.125 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.125 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:01.125 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.599497+0000", 2024-09-18T10:16:01.125 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.6119999999999998e-05 2024-09-18T10:16:01.125 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:01.126 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:01.126 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:01.126 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.126 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.126 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:93 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T10:16:01.126 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:38.616472+0000", 2024-09-18T10:16:01.126 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 82.470078596999997, 2024-09-18T10:16:01.126 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.60940747100000003, 2024-09-18T10:16:01.126 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:01.126 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:01.126 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:01.126 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:16:01.126 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:16:01.127 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 93 2024-09-18T10:16:01.127 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.127 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:01.127 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.127 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:01.127 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.616472+0000", 2024-09-18T10:16:01.127 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:01.127 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.127 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.127 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:01.127 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.616472+0000", 2024-09-18T10:16:01.127 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:01.127 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.128 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.128 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:01.128 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.616470+0000", 2024-09-18T10:16:01.128 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:16:01.128 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.128 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.128 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:01.129 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.616644+0000", 2024-09-18T10:16:01.129 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00017372000000000001 2024-09-18T10:16:01.129 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.129 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.129 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:01.130 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.616695+0000", 2024-09-18T10:16:01.130 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.0991e-05 2024-09-18T10:16:01.130 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.130 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.130 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:01.131 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.616709+0000", 2024-09-18T10:16:01.131 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4008999999999999e-05 2024-09-18T10:16:01.131 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.131 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.131 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:01.131 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.720960+0000", 2024-09-18T10:16:01.131 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.104250624 2024-09-18T10:16:01.131 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.131 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.131 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:01.132 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.721068+0000", 2024-09-18T10:16:01.132 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00010814700000000001 2024-09-18T10:16:01.132 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.132 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.132 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:16:01.132 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.721270+0000", 2024-09-18T10:16:01.133 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00020184200000000001 2024-09-18T10:16:01.133 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.133 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.133 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:16:01.133 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.024691+0000", 2024-09-18T10:16:01.133 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.30342105600000002 2024-09-18T10:16:01.133 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.133 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.133 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:16:01.133 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.225725+0000", 2024-09-18T10:16:01.134 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20103395099999999 2024-09-18T10:16:01.134 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.134 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.134 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:01.134 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.225816+0000", 2024-09-18T10:16:01.134 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.1928999999999993e-05 2024-09-18T10:16:01.134 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.134 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.134 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:01.134 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.225879+0000", 2024-09-18T10:16:01.134 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.2918999999999998e-05 2024-09-18T10:16:01.134 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:01.134 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:01.134 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:01.135 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.135 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.135 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:94 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T10:16:01.135 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:38.616729+0000", 2024-09-18T10:16:01.135 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 82.469821572000001, 2024-09-18T10:16:01.135 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.60943170999999996, 2024-09-18T10:16:01.135 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:01.135 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:01.135 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:01.135 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:16:01.135 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:16:01.136 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 94 2024-09-18T10:16:01.136 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.136 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:01.136 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.136 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:01.136 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.616729+0000", 2024-09-18T10:16:01.136 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:01.136 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.136 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.136 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:01.136 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.616729+0000", 2024-09-18T10:16:01.136 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:01.136 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.136 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.137 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:01.137 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.616727+0000", 2024-09-18T10:16:01.137 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:16:01.137 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.137 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.137 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:01.137 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.616738+0000", 2024-09-18T10:16:01.137 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0776e-05 2024-09-18T10:16:01.137 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.137 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.137 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:01.137 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.616756+0000", 2024-09-18T10:16:01.137 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8079000000000001e-05 2024-09-18T10:16:01.138 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.138 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.138 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:01.138 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.716882+0000", 2024-09-18T10:16:01.138 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10012612999999999 2024-09-18T10:16:01.138 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.138 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.138 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:01.138 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.822315+0000", 2024-09-18T10:16:01.138 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.105433162 2024-09-18T10:16:01.138 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.138 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.138 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:01.138 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.822453+0000", 2024-09-18T10:16:01.138 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000137396 2024-09-18T10:16:01.139 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.139 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.139 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:16:01.139 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.822674+0000", 2024-09-18T10:16:01.139 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00022154400000000001 2024-09-18T10:16:01.139 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.139 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.139 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:16:01.139 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.024888+0000", 2024-09-18T10:16:01.139 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20221397199999999 2024-09-18T10:16:01.139 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.139 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.139 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:16:01.140 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.226011+0000", 2024-09-18T10:16:01.140 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20112316299999999 2024-09-18T10:16:01.140 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.140 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.140 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:01.140 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.226094+0000", 2024-09-18T10:16:01.140 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.2155000000000002e-05 2024-09-18T10:16:01.140 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.140 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.140 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:01.140 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.226161+0000", 2024-09-18T10:16:01.141 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.7138999999999995e-05 2024-09-18T10:16:01.141 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:01.141 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:01.141 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:01.141 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.141 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.141 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:95 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T10:16:01.141 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:38.716909+0000", 2024-09-18T10:16:01.141 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 82.369641857999994, 2024-09-18T10:16:01.141 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.60960763600000001, 2024-09-18T10:16:01.142 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:01.142 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:01.142 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:01.142 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:16:01.142 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:16:01.142 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 95 2024-09-18T10:16:01.142 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.142 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:01.142 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.142 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:01.142 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.716909+0000", 2024-09-18T10:16:01.142 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:01.143 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.143 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.143 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:01.143 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.716909+0000", 2024-09-18T10:16:01.143 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:01.143 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.143 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.143 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:01.143 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.716907+0000", 2024-09-18T10:16:01.143 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:16:01.143 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.143 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.143 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:01.144 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.716921+0000", 2024-09-18T10:16:01.144 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4572e-05 2024-09-18T10:16:01.144 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.144 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.144 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:01.144 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.716939+0000", 2024-09-18T10:16:01.144 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7149000000000001e-05 2024-09-18T10:16:01.144 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.144 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.144 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:01.144 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.716948+0000", 2024-09-18T10:16:01.144 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.7000000000000003e-06 2024-09-18T10:16:01.144 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.145 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.145 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:01.145 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.923430+0000", 2024-09-18T10:16:01.145 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20648133299999999 2024-09-18T10:16:01.145 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.145 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.145 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:01.145 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.923566+0000", 2024-09-18T10:16:01.145 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00013657599999999999 2024-09-18T10:16:01.145 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.145 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.145 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:16:01.145 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.923782+0000", 2024-09-18T10:16:01.146 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00021599400000000001 2024-09-18T10:16:01.146 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.146 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.146 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:16:01.146 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.024899+0000", 2024-09-18T10:16:01.146 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10111666800000001 2024-09-18T10:16:01.146 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.147 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.147 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:16:01.147 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.326379+0000", 2024-09-18T10:16:01.147 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.30148048799999999 2024-09-18T10:16:01.147 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.147 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.147 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:01.147 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.326454+0000", 2024-09-18T10:16:01.147 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.4988999999999996e-05 2024-09-18T10:16:01.147 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.148 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.148 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:01.148 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.326516+0000", 2024-09-18T10:16:01.148 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.1902999999999993e-05 2024-09-18T10:16:01.148 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:01.148 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:01.148 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:01.148 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.148 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.148 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:96 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T10:16:01.148 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:38.716972+0000", 2024-09-18T10:16:01.148 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 82.369578868000005, 2024-09-18T10:16:01.148 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.91539460800000005, 2024-09-18T10:16:01.149 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:01.149 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:01.149 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:01.149 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:16:01.149 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:16:01.149 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 96 2024-09-18T10:16:01.149 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.149 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:01.149 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.149 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:01.149 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.716972+0000", 2024-09-18T10:16:01.149 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:01.149 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.149 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.150 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:01.150 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.716972+0000", 2024-09-18T10:16:01.150 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:01.150 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.150 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.150 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:01.150 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.716971+0000", 2024-09-18T10:16:01.150 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:16:01.150 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.150 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.150 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:01.150 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717061+0000", 2024-09-18T10:16:01.150 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.0744999999999999e-05 2024-09-18T10:16:01.150 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.151 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.151 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:01.151 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717073+0000", 2024-09-18T10:16:01.151 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1943000000000001e-05 2024-09-18T10:16:01.151 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.151 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.151 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:01.151 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717080+0000", 2024-09-18T10:16:01.151 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0060000000000003e-06 2024-09-18T10:16:01.151 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.152 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.152 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:01.152 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.426708+0000", 2024-09-18T10:16:01.152 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.70962730399999996 2024-09-18T10:16:01.152 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.152 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.152 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:01.152 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.426854+0000", 2024-09-18T10:16:01.152 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00014654199999999999 2024-09-18T10:16:01.152 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.152 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.152 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:16:01.152 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.427106+0000", 2024-09-18T10:16:01.152 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00025226600000000001 2024-09-18T10:16:01.153 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.153 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.153 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:16:01.153 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.433087+0000", 2024-09-18T10:16:01.153 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0059804860000000001 2024-09-18T10:16:01.153 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.153 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.153 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:16:01.153 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.632096+0000", 2024-09-18T10:16:01.153 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.19900922099999999 2024-09-18T10:16:01.153 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.153 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.153 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:01.153 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.632257+0000", 2024-09-18T10:16:01.154 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00016058399999999999 2024-09-18T10:16:01.154 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.154 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.154 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:01.154 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.632366+0000", 2024-09-18T10:16:01.154 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000109601 2024-09-18T10:16:01.154 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:01.154 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:01.154 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:01.154 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.154 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.154 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:97 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T10:16:01.155 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:38.717093+0000", 2024-09-18T10:16:01.155 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 82.369457057999995, 2024-09-18T10:16:01.155 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.91561460900000002, 2024-09-18T10:16:01.155 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:01.155 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:01.155 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:01.155 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:16:01.155 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:16:01.155 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 97 2024-09-18T10:16:01.155 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.155 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:01.155 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.155 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:01.155 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717093+0000", 2024-09-18T10:16:01.156 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:01.156 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.156 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.156 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:01.156 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717093+0000", 2024-09-18T10:16:01.156 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:01.156 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.156 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.156 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:01.156 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717092+0000", 2024-09-18T10:16:01.156 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:16:01.156 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.156 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.156 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:01.157 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717099+0000", 2024-09-18T10:16:01.157 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.246e-06 2024-09-18T10:16:01.157 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.157 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.157 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:01.157 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717109+0000", 2024-09-18T10:16:01.157 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0499999999999999e-05 2024-09-18T10:16:01.157 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.157 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.157 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:01.158 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717115+0000", 2024-09-18T10:16:01.158 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.7690000000000002e-06 2024-09-18T10:16:01.158 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.158 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.158 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:01.158 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.428509+0000", 2024-09-18T10:16:01.158 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.71139444299999999 2024-09-18T10:16:01.158 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.158 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.158 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:01.158 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.428621+0000", 2024-09-18T10:16:01.158 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00011156 2024-09-18T10:16:01.158 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.158 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.159 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:16:01.159 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.428805+0000", 2024-09-18T10:16:01.159 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000183597 2024-09-18T10:16:01.159 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.159 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.159 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:16:01.159 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.433106+0000", 2024-09-18T10:16:01.159 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0043015019999999996 2024-09-18T10:16:01.159 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.159 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.159 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:16:01.159 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.632474+0000", 2024-09-18T10:16:01.159 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.199368092 2024-09-18T10:16:01.160 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.160 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.160 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:01.160 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.632605+0000", 2024-09-18T10:16:01.160 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00013110899999999999 2024-09-18T10:16:01.160 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.160 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.160 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:01.160 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.632708+0000", 2024-09-18T10:16:01.160 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000102748 2024-09-18T10:16:01.160 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:01.161 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:01.161 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:01.161 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.161 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.161 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:98 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T10:16:01.161 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:38.717126+0000", 2024-09-18T10:16:01.161 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 82.369424796000004, 2024-09-18T10:16:01.161 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.015084141, 2024-09-18T10:16:01.161 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:01.162 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:01.162 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:01.162 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:16:01.162 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:16:01.162 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 98 2024-09-18T10:16:01.162 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.162 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:01.162 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.162 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:01.162 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717126+0000", 2024-09-18T10:16:01.163 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:01.163 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.163 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.163 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:01.163 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717126+0000", 2024-09-18T10:16:01.163 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:01.163 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.163 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.163 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:01.163 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717125+0000", 2024-09-18T10:16:01.163 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:16:01.163 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.163 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.164 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:01.164 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717130+0000", 2024-09-18T10:16:01.164 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.4160000000000003e-06 2024-09-18T10:16:01.164 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.164 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.164 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:01.164 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717139+0000", 2024-09-18T10:16:01.164 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.6230000000000003e-06 2024-09-18T10:16:01.164 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.164 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.164 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:01.164 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717144+0000", 2024-09-18T10:16:01.164 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5330000000000002e-06 2024-09-18T10:16:01.165 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.165 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.165 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:01.165 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.429491+0000", 2024-09-18T10:16:01.165 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.71234629500000002 2024-09-18T10:16:01.168 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.168 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.168 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:01.168 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.429741+0000", 2024-09-18T10:16:01.168 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000250516 2024-09-18T10:16:01.168 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.169 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.169 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:16:01.169 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.429989+0000", 2024-09-18T10:16:01.169 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00024811599999999999 2024-09-18T10:16:01.169 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.169 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.169 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:16:01.169 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.435056+0000", 2024-09-18T10:16:01.169 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0050669670000000003 2024-09-18T10:16:01.169 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.169 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.169 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:16:01.169 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.732068+0000", 2024-09-18T10:16:01.169 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.29701180900000002 2024-09-18T10:16:01.170 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.170 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.170 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:01.170 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.732145+0000", 2024-09-18T10:16:01.170 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.7152999999999998e-05 2024-09-18T10:16:01.170 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.170 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.170 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:01.170 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.732210+0000", 2024-09-18T10:16:01.170 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.4503000000000002e-05 2024-09-18T10:16:01.170 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:01.170 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:01.170 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:01.170 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.171 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.171 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:99 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T10:16:01.171 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:38.717230+0000", 2024-09-18T10:16:01.171 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 82.369320235999993, 2024-09-18T10:16:01.171 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0157905819999999, 2024-09-18T10:16:01.171 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:01.171 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:01.171 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:01.171 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:16:01.171 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:16:01.171 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 99 2024-09-18T10:16:01.172 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.172 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:01.172 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.172 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:01.172 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717230+0000", 2024-09-18T10:16:01.172 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:01.172 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.173 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.173 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:01.173 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717230+0000", 2024-09-18T10:16:01.173 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:01.173 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.173 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.173 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:01.173 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717229+0000", 2024-09-18T10:16:01.173 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:16:01.173 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.174 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.174 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:01.174 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717318+0000", 2024-09-18T10:16:01.174 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.8197999999999998e-05 2024-09-18T10:16:01.174 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.174 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.174 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:01.174 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717342+0000", 2024-09-18T10:16:01.174 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4445000000000001e-05 2024-09-18T10:16:01.174 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.174 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.174 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:01.174 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717360+0000", 2024-09-18T10:16:01.175 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7782999999999999e-05 2024-09-18T10:16:01.175 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.175 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.175 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:01.175 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.430650+0000", 2024-09-18T10:16:01.175 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.71329020200000004 2024-09-18T10:16:01.175 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.175 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.175 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:01.175 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.430805+0000", 2024-09-18T10:16:01.176 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00015474900000000001 2024-09-18T10:16:01.176 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.176 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.176 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:16:01.176 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.431021+0000", 2024-09-18T10:16:01.176 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000215911 2024-09-18T10:16:01.176 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.176 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.176 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:16:01.176 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.435099+0000", 2024-09-18T10:16:01.176 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0040787439999999996 2024-09-18T10:16:01.176 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.176 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.177 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:16:01.177 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.732758+0000", 2024-09-18T10:16:01.177 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.297658637 2024-09-18T10:16:01.177 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.177 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.177 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:01.177 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.732909+0000", 2024-09-18T10:16:01.177 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00015069899999999999 2024-09-18T10:16:01.177 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.177 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.177 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:01.177 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.733021+0000", 2024-09-18T10:16:01.177 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00011207 2024-09-18T10:16:01.178 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:01.178 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:01.178 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:01.178 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.178 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.178 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:100 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T10:16:01.178 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:38.717370+0000", 2024-09-18T10:16:01.178 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 82.369180674000006, 2024-09-18T10:16:01.178 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1158324850000001, 2024-09-18T10:16:01.178 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:01.178 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:01.178 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:01.178 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:16:01.179 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:16:01.179 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 100 2024-09-18T10:16:01.179 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.179 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:01.179 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.179 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:01.179 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717370+0000", 2024-09-18T10:16:01.179 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:01.179 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.179 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.179 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:01.179 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717370+0000", 2024-09-18T10:16:01.179 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:01.180 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.180 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.180 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:01.180 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717369+0000", 2024-09-18T10:16:01.180 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:16:01.180 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.180 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.180 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:01.180 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717378+0000", 2024-09-18T10:16:01.180 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.3559999999999999e-06 2024-09-18T10:16:01.180 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.180 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.180 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:01.181 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717398+0000", 2024-09-18T10:16:01.181 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.9962e-05 2024-09-18T10:16:01.181 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.181 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.181 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:01.181 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717407+0000", 2024-09-18T10:16:01.181 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.2699999999999993e-06 2024-09-18T10:16:01.181 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.181 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.181 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:01.181 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.432184+0000", 2024-09-18T10:16:01.181 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.71477630299999995 2024-09-18T10:16:01.181 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.182 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.182 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:01.182 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.432316+0000", 2024-09-18T10:16:01.182 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000132082 2024-09-18T10:16:01.182 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.182 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.182 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:16:01.182 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.432526+0000", 2024-09-18T10:16:01.182 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000210659 2024-09-18T10:16:01.182 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.182 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.182 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:16:01.182 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.435651+0000", 2024-09-18T10:16:01.183 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.003124169 2024-09-18T10:16:01.183 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.183 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.183 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:16:01.183 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.832931+0000", 2024-09-18T10:16:01.183 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.39728084899999999 2024-09-18T10:16:01.183 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.183 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.183 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:01.183 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.833058+0000", 2024-09-18T10:16:01.183 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000126969 2024-09-18T10:16:01.183 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.183 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.184 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:01.184 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.833202+0000", 2024-09-18T10:16:01.184 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000143922 2024-09-18T10:16:01.184 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:01.184 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:01.184 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:01.184 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.184 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.184 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:141 3.8s0 3:1c02e63c:::smithi18925896-47:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:16:01.184 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:51.119430+0000", 2024-09-18T10:16:01.184 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 69.967120469999998, 2024-09-18T10:16:01.184 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.60306919000000003, 2024-09-18T10:16:01.184 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:01.185 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "started", 2024-09-18T10:16:01.185 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:01.185 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:16:01.185 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:01.185 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 141 2024-09-18T10:16:01.185 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.185 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:01.185 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.185 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:01.185 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.119430+0000", 2024-09-18T10:16:01.185 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:01.185 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.185 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.186 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:01.186 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.119430+0000", 2024-09-18T10:16:01.186 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:01.186 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.186 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.186 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:01.186 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.119430+0000", 2024-09-18T10:16:01.186 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999995 2024-09-18T10:16:01.186 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.186 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.186 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:01.186 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.119432+0000", 2024-09-18T10:16:01.186 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6240000000000002e-06 2024-09-18T10:16:01.187 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.187 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.187 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:01.187 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.119439+0000", 2024-09-18T10:16:01.187 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.8329999999999996e-06 2024-09-18T10:16:01.187 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.187 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.187 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:01.187 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.119444+0000", 2024-09-18T10:16:01.187 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.485e-06 2024-09-18T10:16:01.187 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.187 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.187 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:01.187 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.620626+0000", 2024-09-18T10:16:01.187 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.50118172299999997 2024-09-18T10:16:01.188 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.188 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.188 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:01.188 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.620781+0000", 2024-09-18T10:16:01.188 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00015495 2024-09-18T10:16:01.188 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.188 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.188 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:01.188 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.722499+0000", 2024-09-18T10:16:01.188 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.101718085 2024-09-18T10:16:01.188 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:01.188 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:01.188 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:01.188 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.189 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.189 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:154 3.8s0 3:1d386431:::smithi18925896-2:head [copy-from ver 4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:16:01.189 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:51.211083+0000", 2024-09-18T10:16:01.189 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 69.875467244999996, 2024-09-18T10:16:01.189 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.73193688499999998, 2024-09-18T10:16:01.189 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:01.189 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:01.189 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:01.189 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:16:01.189 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:01.189 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 154 2024-09-18T10:16:01.189 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.190 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:01.190 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.190 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:01.190 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.211083+0000", 2024-09-18T10:16:01.190 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:01.190 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.190 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.190 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:01.190 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.211083+0000", 2024-09-18T10:16:01.190 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:01.190 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.190 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.190 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:01.190 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.211080+0000", 2024-09-18T10:16:01.191 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999971 2024-09-18T10:16:01.191 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.191 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.191 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:01.191 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.211096+0000", 2024-09-18T10:16:01.191 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.5699999999999999e-05 2024-09-18T10:16:01.191 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.191 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.191 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:01.191 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.211134+0000", 2024-09-18T10:16:01.191 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.7694000000000002e-05 2024-09-18T10:16:01.191 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.191 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.191 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:01.192 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.211149+0000", 2024-09-18T10:16:01.192 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4746000000000001e-05 2024-09-18T10:16:01.192 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.192 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.192 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:01.192 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.621341+0000", 2024-09-18T10:16:01.192 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.41019203799999998 2024-09-18T10:16:01.192 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.192 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.192 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:01.192 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.621505+0000", 2024-09-18T10:16:01.192 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00016443299999999999 2024-09-18T10:16:01.192 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.193 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.193 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:01.193 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.840064+0000", 2024-09-18T10:16:01.193 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.21855904700000001 2024-09-18T10:16:01.193 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.193 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.193 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:01.193 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.842430+0000", 2024-09-18T10:16:01.193 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0023664100000000002 2024-09-18T10:16:01.193 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.193 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.193 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:01.193 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.942864+0000", 2024-09-18T10:16:01.194 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10043373 2024-09-18T10:16:01.194 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.194 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.194 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:01.194 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.943020+0000", 2024-09-18T10:16:01.194 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00015598000000000001 2024-09-18T10:16:01.194 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:01.194 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:01.194 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:01.194 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.194 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.194 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:155 3.8s0 3:1d386431:::smithi18925896-2:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:16:01.194 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:51.211164+0000", 2024-09-18T10:16:01.195 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 69.875386875000004, 2024-09-18T10:16:01.195 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.630957349, 2024-09-18T10:16:01.195 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:01.195 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "started", 2024-09-18T10:16:01.195 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:01.195 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:16:01.195 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:01.195 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 155 2024-09-18T10:16:01.195 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.195 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:01.195 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.195 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:01.195 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.211164+0000", 2024-09-18T10:16:01.195 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:01.196 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.196 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.196 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:01.196 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.211164+0000", 2024-09-18T10:16:01.196 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:01.196 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.196 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.196 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:01.196 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.211163+0000", 2024-09-18T10:16:01.196 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:16:01.196 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.196 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.196 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:01.197 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.211167+0000", 2024-09-18T10:16:01.197 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.6299999999999997e-06 2024-09-18T10:16:01.197 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.197 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.197 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:01.197 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.211178+0000", 2024-09-18T10:16:01.197 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0798999999999999e-05 2024-09-18T10:16:01.197 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.197 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.197 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:01.197 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.211186+0000", 2024-09-18T10:16:01.197 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.5669999999999997e-06 2024-09-18T10:16:01.197 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.198 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.198 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:01.198 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.623024+0000", 2024-09-18T10:16:01.198 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.411838433 2024-09-18T10:16:01.198 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.198 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.198 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for blocked object", 2024-09-18T10:16:01.198 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.623104+0000", 2024-09-18T10:16:01.198 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.0136000000000007e-05 2024-09-18T10:16:01.198 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.198 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.198 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:01.198 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.841600+0000", 2024-09-18T10:16:01.198 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.21849595399999999 2024-09-18T10:16:01.199 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.199 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.199 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:01.199 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.841868+0000", 2024-09-18T10:16:01.199 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00026825800000000001 2024-09-18T10:16:01.199 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.199 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.199 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:01.199 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.842121+0000", 2024-09-18T10:16:01.199 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00025266500000000001 2024-09-18T10:16:01.199 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:01.199 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:01.199 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:01.200 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.200 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.200 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:619 3.8s0 3:1162852a:::smithi18925896-44:head [copy-from ver 14] snapc 21=[20,1f,1e,1a,16,12,11,10] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-18T10:16:01.200 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:27.977392+0000", 2024-09-18T10:16:01.200 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 33.109158620999999, 2024-09-18T10:16:01.200 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.71139311599999999, 2024-09-18T10:16:01.200 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:01.200 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:01.200 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:01.200 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:16:01.200 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:01.200 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 619 2024-09-18T10:16:01.200 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.201 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:01.201 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.201 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:01.201 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:27.977392+0000", 2024-09-18T10:16:01.201 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:01.201 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.201 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.201 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:01.201 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:27.977392+0000", 2024-09-18T10:16:01.201 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:01.201 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.201 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.201 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:01.201 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:27.977390+0000", 2024-09-18T10:16:01.202 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:16:01.202 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.202 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.202 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:01.202 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:27.977399+0000", 2024-09-18T10:16:01.202 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.1239999999999994e-06 2024-09-18T10:16:01.202 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.202 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.202 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:01.202 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:27.977422+0000", 2024-09-18T10:16:01.202 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.2734999999999998e-05 2024-09-18T10:16:01.203 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.203 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.203 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:01.203 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.077547+0000", 2024-09-18T10:16:01.203 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100124728 2024-09-18T10:16:01.203 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.203 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.203 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:01.204 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.077713+0000", 2024-09-18T10:16:01.204 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000166515 2024-09-18T10:16:01.204 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.204 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.204 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op must wait for map", 2024-09-18T10:16:01.204 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.077728+0000", 2024-09-18T10:16:01.204 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.5036e-05 2024-09-18T10:16:01.204 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.204 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.204 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:01.204 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.275819+0000", 2024-09-18T10:16:01.204 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.19809117600000001 2024-09-18T10:16:01.204 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.204 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.205 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:01.205 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.276220+0000", 2024-09-18T10:16:01.205 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000400426 2024-09-18T10:16:01.205 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.205 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.205 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:01.205 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.383764+0000", 2024-09-18T10:16:01.205 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.107543814 2024-09-18T10:16:01.205 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.205 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.205 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:01.205 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.486402+0000", 2024-09-18T10:16:01.205 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.102638484 2024-09-18T10:16:01.205 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.206 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.206 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:01.206 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.688521+0000", 2024-09-18T10:16:01.206 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.202118873 2024-09-18T10:16:01.206 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.206 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.206 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:01.206 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.688785+0000", 2024-09-18T10:16:01.206 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000264133 2024-09-18T10:16:01.206 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:01.206 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:01.206 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:01.206 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.206 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.207 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:626 3.10s0 3:0a83d6b3:::smithi18925896-20:head [append 0~638976] snapc 21=[20,1f,1e,1a,16,12,11,10] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-18T10:16:01.207 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:28.708261+0000", 2024-09-18T10:16:01.207 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 32.378289056, 2024-09-18T10:16:01.207 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.70096385699999997, 2024-09-18T10:16:01.207 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:01.207 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:01.207 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:01.207 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:16:01.207 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:01.207 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 626 2024-09-18T10:16:01.207 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.207 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:01.207 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.208 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:01.208 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.708261+0000", 2024-09-18T10:16:01.208 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:01.208 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.208 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.208 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:01.208 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.708261+0000", 2024-09-18T10:16:01.208 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:01.208 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.208 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.208 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:01.208 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.708259+0000", 2024-09-18T10:16:01.208 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:16:01.208 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.209 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.209 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:01.209 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.709175+0000", 2024-09-18T10:16:01.209 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00091571199999999995 2024-09-18T10:16:01.209 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.209 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.209 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:01.209 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.709199+0000", 2024-09-18T10:16:01.209 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.3785e-05 2024-09-18T10:16:01.209 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.209 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.209 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:01.209 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.809328+0000", 2024-09-18T10:16:01.209 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100129494 2024-09-18T10:16:01.210 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.210 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.210 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:01.210 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.996826+0000", 2024-09-18T10:16:01.210 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.18749712700000001 2024-09-18T10:16:01.210 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.210 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.210 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:01.210 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.997086+0000", 2024-09-18T10:16:01.210 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00026047499999999998 2024-09-18T10:16:01.210 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.210 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.210 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:01.210 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.998987+0000", 2024-09-18T10:16:01.211 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019014889999999999 2024-09-18T10:16:01.211 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.211 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.211 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:01.211 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.104118+0000", 2024-09-18T10:16:01.211 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10513012100000001 2024-09-18T10:16:01.211 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.211 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.211 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:01.211 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.409085+0000", 2024-09-18T10:16:01.211 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.30496710599999999 2024-09-18T10:16:01.211 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.211 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.212 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:01.212 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.409225+0000", 2024-09-18T10:16:01.212 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00014059899999999999 2024-09-18T10:16:01.212 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:01.212 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:01.212 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:01.212 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.212 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.212 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:629 3.10s0 3:0a83d6b3:::smithi18925896-20:head [setxattr _header (57) in=7b] snapc 21=[20,1f,1e,1a,16,12,11,10] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-18T10:16:01.212 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:28.910973+0000", 2024-09-18T10:16:01.212 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 32.175577938000004, 2024-09-18T10:16:01.212 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.80485965999999998, 2024-09-18T10:16:01.212 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:01.212 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:01.213 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:01.213 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:16:01.213 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:01.213 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 629 2024-09-18T10:16:01.213 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.213 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:01.213 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.213 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:01.213 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.910973+0000", 2024-09-18T10:16:01.213 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:01.213 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.213 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.213 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:01.213 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.910973+0000", 2024-09-18T10:16:01.214 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:01.214 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.214 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.214 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:01.214 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.910971+0000", 2024-09-18T10:16:01.214 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:16:01.214 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.214 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.214 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:01.214 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.910976+0000", 2024-09-18T10:16:01.214 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.0069999999999997e-06 2024-09-18T10:16:01.214 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.214 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.215 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:01.215 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.910984+0000", 2024-09-18T10:16:01.215 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.7109999999999995e-06 2024-09-18T10:16:01.215 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.215 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.215 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:01.215 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.910989+0000", 2024-09-18T10:16:01.215 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1549999999999998e-06 2024-09-18T10:16:01.215 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.215 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.215 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:01.215 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.411082+0000", 2024-09-18T10:16:01.215 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.50009252199999998 2024-09-18T10:16:01.215 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.216 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.216 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:01.216 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.411264+0000", 2024-09-18T10:16:01.216 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00018190799999999999 2024-09-18T10:16:01.216 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.216 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.216 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:01.216 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.412262+0000", 2024-09-18T10:16:01.216 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00099852199999999995 2024-09-18T10:16:01.216 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.216 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.216 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:01.216 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.513736+0000", 2024-09-18T10:16:01.216 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.101474077 2024-09-18T10:16:01.217 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.217 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.217 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:01.217 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.715730+0000", 2024-09-18T10:16:01.217 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20199325600000001 2024-09-18T10:16:01.217 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.217 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.217 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:01.217 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.715832+0000", 2024-09-18T10:16:01.217 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000102633 2024-09-18T10:16:01.217 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:01.217 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:01.217 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:01.217 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.218 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.218 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:630 3.10s0 3:0a83d6b3:::smithi18925896-20:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e72)", 2024-09-18T10:16:01.218 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:28.910995+0000", 2024-09-18T10:16:01.218 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 32.175555907000003, 2024-09-18T10:16:01.218 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2560086020000001, 2024-09-18T10:16:01.218 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:01.218 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "started", 2024-09-18T10:16:01.218 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:01.218 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:16:01.218 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:01.218 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 630 2024-09-18T10:16:01.218 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.218 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:01.218 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.219 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:01.219 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.910995+0000", 2024-09-18T10:16:01.219 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:01.219 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.219 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.219 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:01.219 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.910995+0000", 2024-09-18T10:16:01.219 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:01.219 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.219 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.219 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:01.219 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.910994+0000", 2024-09-18T10:16:01.219 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:16:01.220 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.220 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.220 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:01.220 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.910997+0000", 2024-09-18T10:16:01.220 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2820000000000001e-06 2024-09-18T10:16:01.220 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.220 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.220 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:01.220 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.911008+0000", 2024-09-18T10:16:01.220 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0617e-05 2024-09-18T10:16:01.220 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.220 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.220 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:01.221 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.911013+0000", 2024-09-18T10:16:01.221 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.8160000000000001e-06 2024-09-18T10:16:01.221 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.221 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.221 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:01.221 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.413034+0000", 2024-09-18T10:16:01.221 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.50202176600000004 2024-09-18T10:16:01.221 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.221 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.221 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for rw locks", 2024-09-18T10:16:01.221 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.413303+0000", 2024-09-18T10:16:01.221 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00026880600000000001 2024-09-18T10:16:01.221 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.222 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.222 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:01.222 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.716533+0000", 2024-09-18T10:16:01.222 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.30323016000000003 2024-09-18T10:16:01.222 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.222 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.222 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:01.222 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.716647+0000", 2024-09-18T10:16:01.222 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00011372899999999999 2024-09-18T10:16:01.222 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.222 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.222 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:01.222 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:30.167003+0000", 2024-09-18T10:16:01.222 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.450356172 2024-09-18T10:16:01.223 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:01.223 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:01.223 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:01.223 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:01.223 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:01.223 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-09-18T10:16:01.225 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T10:16:01.225 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T10:16:01.225 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T10:16:01.225 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T10:16:01.225 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T10:16:01.225 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T10:16:01.225 INFO:teuthology.orchestra.run.smithi029.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T10:16:01.226 INFO:teuthology.orchestra.run.smithi029.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T10:16:01.226 INFO:teuthology.orchestra.run.smithi029.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-18T10:16:01.226 INFO:teuthology.orchestra.run.smithi029.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-18T10:16:01.226 INFO:teuthology.orchestra.run.smithi029.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-18T10:16:01.226 INFO:teuthology.orchestra.run.smithi029.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-18T10:16:01.226 INFO:teuthology.orchestra.run.smithi029.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-18T10:16:01.226 INFO:teuthology.orchestra.run.smithi029.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-18T10:16:01.226 INFO:teuthology.orchestra.run.smithi029.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-18T10:16:01.226 INFO:teuthology.orchestra.run.smithi029.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-18T10:16:01.227 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-09-18T10:16:01.227 INFO:tasks.ceph.osd.6.smithi154.stderr:2024-09-18T10:16:01.222+0000 7f0abc9e1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22089) UID: 0 2024-09-18T10:16:01.319 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-18T10:16:01.320 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":105,"fsid":"ab6d6beb-0627-4b7f-8cdb-4e0a44970f10","created":"2024-09-18T10:14:18.827644+0000","modified":"2024-09-18T10:16:00.742882+0000","last_up_change":"2024-09-18T10:14:30.228591+0000","last_in_change":"2024-09-18T10:15:49.554826+0000","flags":"noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5806080,"flags_set":["nodeep-scrub","noscrub","pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-18T10:14:31.266113+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-18T10:14:33.358485+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":29,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"unique_pool_0","create_time":"2024-09-18T10:14:44.624320+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":3,"size":7,"min_size":5,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":26,"pg_placement_num":26,"pg_placement_num_target":36,"pg_num_target":36,"pg_num_pending":26,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"40","auid":0,"snap_mode":"selfmanaged","snap_seq":60,"snap_epoch":103,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"shecprofile","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":16384,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rados":{}}}],"osds":[{"osd":0,"uuid":"39cf0511-0009-433f-92f2-e5686f33d713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":22197},{"type":"v1","addr":"172.21.15.29:6809","nonce":22197}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":22197},{"type":"v1","addr":"172.21.15.29:6811","nonce":22197}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":22197},{"type":"v1","addr":"172.21.15.29:6815","nonce":22197}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":22197},{"type":"v1","addr":"172.21.15.29:6813","nonce":22197}]},"public_addr":"172.21.15.29:6809/22197","cluster_addr":"172.21.15.29:6811/22197","heartbeat_back_addr":"172.21.15.29:6815/22197","heartbeat_front_addr":"172.21.15.29:6813/22197","state":["exists","up"]},{"osd":1,"uuid":"a0bf06ab-b98e-4130-91d2-7d4283540c08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":22140},{"type":"v1","addr":"172.21.15.136:6809","nonce":22140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":22140},{"type":"v1","addr":"172.21.15.136:6811","nonce":22140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":22140},{"type":"v1","addr":"172.21.15.136:6815","nonce":22140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":22140},{"type":"v1","addr":"172.21.15.136:6813","nonce":22140}]},"public_addr":"172.21.15.136:6809/22140","cluster_addr":"172.21.15.136:6811/22140","heartbeat_back_addr":"172.21.15.136:6815/22140","heartbeat_front_addr":"172.21.15.136:6813/22140","state":["exists","up"]},{"osd":2,"uuid":"742a786e-6129-4fbc-bb5b-e2e3c03c2d77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":22095},{"type":"v1","addr":"172.21.15.154:6825","nonce":22095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":22095},{"type":"v1","addr":"172.21.15.154:6827","nonce":22095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":22095},{"type":"v1","addr":"172.21.15.154:6831","nonce":22095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":22095},{"type":"v1","addr":"172.21.15.154:6829","nonce":22095}]},"public_addr":"172.21.15.154:6825/22095","cluster_addr":"172.21.15.154:6827/22095","heartbeat_back_addr":"172.21.15.154:6831/22095","heartbeat_front_addr":"172.21.15.154:6829/22095","state":["exists","up"]},{"osd":3,"uuid":"3aa77114-24ed-49ab-86ff-a0600794cbf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":22404},{"type":"v1","addr":"172.21.15.189:6803","nonce":22404}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":22404},{"type":"v1","addr":"172.21.15.189:6805","nonce":22404}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":22404},{"type":"v1","addr":"172.21.15.189:6809","nonce":22404}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":22404},{"type":"v1","addr":"172.21.15.189:6807","nonce":22404}]},"public_addr":"172.21.15.189:6803/22404","cluster_addr":"172.21.15.189:6805/22404","heartbeat_back_addr":"172.21.15.189:6809/22404","heartbeat_front_addr":"172.21.15.189:6807/22404","state":["exists","up"]},{"osd":4,"uuid":"426567ad-b0be-41d4-afcf-dc56ec3974cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":22195},{"type":"v1","addr":"172.21.15.29:6801","nonce":22195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":22195},{"type":"v1","addr":"172.21.15.29:6803","nonce":22195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":22195},{"type":"v1","addr":"172.21.15.29:6807","nonce":22195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":22195},{"type":"v1","addr":"172.21.15.29:6805","nonce":22195}]},"public_addr":"172.21.15.29:6801/22195","cluster_addr":"172.21.15.29:6803/22195","heartbeat_back_addr":"172.21.15.29:6807/22195","heartbeat_front_addr":"172.21.15.29:6805/22195","state":["exists","up"]},{"osd":5,"uuid":"d9559e42-1f4e-4373-8392-a536a0afd871","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6817","nonce":22141},{"type":"v1","addr":"172.21.15.136:6819","nonce":22141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6821","nonce":22141},{"type":"v1","addr":"172.21.15.136:6823","nonce":22141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6829","nonce":22141},{"type":"v1","addr":"172.21.15.136:6831","nonce":22141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6825","nonce":22141},{"type":"v1","addr":"172.21.15.136:6827","nonce":22141}]},"public_addr":"172.21.15.136:6819/22141","cluster_addr":"172.21.15.136:6823/22141","heartbeat_back_addr":"172.21.15.136:6831/22141","heartbeat_front_addr":"172.21.15.136:6827/22141","state":["exists","up"]},{"osd":6,"uuid":"d50ef1e2-8bc5-4a90-9be3-365a0db2be6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":22096},{"type":"v1","addr":"172.21.15.154:6801","nonce":22096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":22096},{"type":"v1","addr":"172.21.15.154:6803","nonce":22096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":22096},{"type":"v1","addr":"172.21.15.154:6807","nonce":22096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":22096},{"type":"v1","addr":"172.21.15.154:6805","nonce":22096}]},"public_addr":"172.21.15.154:6801/22096","cluster_addr":"172.21.15.154:6803/22096","heartbeat_back_addr":"172.21.15.154:6807/22096","heartbeat_front_addr":"172.21.15.154:6805/22096","state":["exists","up"]},{"osd":7,"uuid":"bd0b85e4-cfee-4d19-90a5-79854794b48b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":22405},{"type":"v1","addr":"172.21.15.189:6819","nonce":22405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":22405},{"type":"v1","addr":"172.21.15.189:6821","nonce":22405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":22405},{"type":"v1","addr":"172.21.15.189:6825","nonce":22405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":22405},{"type":"v1","addr":"172.21.15.189:6823","nonce":22405}]},"public_addr":"172.21.15.189:6819/22405","cluster_addr":"172.21.15.189:6821/22405","heartbeat_back_addr":"172.21.15.189:6825/22405","heartbeat_front_addr":"172.21.15.189:6823/22405","state":["exists","up"]},{"osd":8,"uuid":"cd333f56-c165-45d7-8c8b-48d34bc4ddc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":22200},{"type":"v1","addr":"172.21.15.29:6825","nonce":22200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":22200},{"type":"v1","addr":"172.21.15.29:6827","nonce":22200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":22200},{"type":"v1","addr":"172.21.15.29:6831","nonce":22200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":22200},{"type":"v1","addr":"172.21.15.29:6829","nonce":22200}]},"public_addr":"172.21.15.29:6825/22200","cluster_addr":"172.21.15.29:6827/22200","heartbeat_back_addr":"172.21.15.29:6831/22200","heartbeat_front_addr":"172.21.15.29:6829/22200","state":["exists","up"]},{"osd":9,"uuid":"1e079e4c-8801-4754-9d0d-d0b68695d32e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":22142},{"type":"v1","addr":"172.21.15.136:6801","nonce":22142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":22142},{"type":"v1","addr":"172.21.15.136:6803","nonce":22142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":22142},{"type":"v1","addr":"172.21.15.136:6807","nonce":22142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":22142},{"type":"v1","addr":"172.21.15.136:6805","nonce":22142}]},"public_addr":"172.21.15.136:6801/22142","cluster_addr":"172.21.15.136:6803/22142","heartbeat_back_addr":"172.21.15.136:6807/22142","heartbeat_front_addr":"172.21.15.136:6805/22142","state":["exists","up"]},{"osd":10,"uuid":"92ee68d4-371c-4849-930f-a8fa18d64b99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":22097},{"type":"v1","addr":"172.21.15.154:6809","nonce":22097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":22097},{"type":"v1","addr":"172.21.15.154:6811","nonce":22097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":22097},{"type":"v1","addr":"172.21.15.154:6815","nonce":22097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":22097},{"type":"v1","addr":"172.21.15.154:6813","nonce":22097}]},"public_addr":"172.21.15.154:6809/22097","cluster_addr":"172.21.15.154:6811/22097","heartbeat_back_addr":"172.21.15.154:6815/22097","heartbeat_front_addr":"172.21.15.154:6813/22097","state":["exists","up"]},{"osd":11,"uuid":"634ddaf5-f7b8-4b27-a46a-04e21b73fcf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":22416},{"type":"v1","addr":"172.21.15.189:6827","nonce":22416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":22416},{"type":"v1","addr":"172.21.15.189:6829","nonce":22416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":22416},{"type":"v1","addr":"172.21.15.189:6833","nonce":22416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":22416},{"type":"v1","addr":"172.21.15.189:6831","nonce":22416}]},"public_addr":"172.21.15.189:6827/22416","cluster_addr":"172.21.15.189:6829/22416","heartbeat_back_addr":"172.21.15.189:6833/22416","heartbeat_front_addr":"172.21.15.189:6831/22416","state":["exists","up"]},{"osd":12,"uuid":"bc58137b-5948-400d-b377-193b15824d8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":22210},{"type":"v1","addr":"172.21.15.29:6817","nonce":22210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":22210},{"type":"v1","addr":"172.21.15.29:6819","nonce":22210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":22210},{"type":"v1","addr":"172.21.15.29:6823","nonce":22210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":22210},{"type":"v1","addr":"172.21.15.29:6821","nonce":22210}]},"public_addr":"172.21.15.29:6817/22210","cluster_addr":"172.21.15.29:6819/22210","heartbeat_back_addr":"172.21.15.29:6823/22210","heartbeat_front_addr":"172.21.15.29:6821/22210","state":["exists","up"]},{"osd":13,"uuid":"a8b753a7-ab69-4853-ac36-66d9b85c2e20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":22143},{"type":"v1","addr":"172.21.15.136:6818","nonce":22143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":22143},{"type":"v1","addr":"172.21.15.136:6822","nonce":22143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":22143},{"type":"v1","addr":"172.21.15.136:6830","nonce":22143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":22143},{"type":"v1","addr":"172.21.15.136:6826","nonce":22143}]},"public_addr":"172.21.15.136:6818/22143","cluster_addr":"172.21.15.136:6822/22143","heartbeat_back_addr":"172.21.15.136:6830/22143","heartbeat_front_addr":"172.21.15.136:6826/22143","state":["exists","up"]},{"osd":14,"uuid":"8ad0848f-0de4-4a75-9c63-8c4a4a7b8524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":22098},{"type":"v1","addr":"172.21.15.154:6817","nonce":22098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":22098},{"type":"v1","addr":"172.21.15.154:6819","nonce":22098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":22098},{"type":"v1","addr":"172.21.15.154:6823","nonce":22098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":22098},{"type":"v1","addr":"172.21.15.154:6821","nonce":22098}]},"public_addr":"172.21.15.154:6817/22098","cluster_addr":"172.21.15.154:6819/22098","heartbeat_back_addr":"172.21.15.154:6823/22098","heartbeat_front_addr":"172.21.15.154:6821/22098","state":["exists","up"]},{"osd":15,"uuid":"069099b3-ec1a-483b-b967-98cddbf2f678","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":22415},{"type":"v1","addr":"172.21.15.189:6811","nonce":22415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":22415},{"type":"v1","addr":"172.21.15.189:6813","nonce":22415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":22415},{"type":"v1","addr":"172.21.15.189:6817","nonce":22415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":22415},{"type":"v1","addr":"172.21.15.189:6815","nonce":22415}]},"public_addr":"172.21.15.189:6811/22415","cluster_addr":"172.21.15.189:6813/22415","heartbeat_back_addr":"172.21.15.189:6817/22415","heartbeat_front_addr":"172.21.15.189:6815/22415","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:15:56.126704+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:15:32.497046+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:15:55.189067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:15:44.298040+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:15:56.991035+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:15:51.718897+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:15:41.309784+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:15:36.256262+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:15:42.972647+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:15:36.896626+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:15:54.542105+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:15:47.318375+0000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:15:31.985850+0000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:15:36.748402+0000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:15:52.801215+0000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:15:59.018997+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"},"shecprofile":{"c":"2","crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","k":"4","m":"3","name":"shecprofile","plugin":"shec","technique":"multiple"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":16,"length":1},{"begin":18,"length":1},{"begin":22,"length":1},{"begin":26,"length":1},{"begin":30,"length":3},{"begin":38,"length":1},{"begin":41,"length":2},{"begin":45,"length":1},{"begin":47,"length":2},{"begin":53,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-18T10:16:01.325 INFO:tasks.ceph.osd.14.smithi154.stderr:2024-09-18T10:16:01.322+0000 7f75a19e7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 22093) UID: 0 2024-09-18T10:16:01.334 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T10:16:01.375 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-09-18T10:16:01.375 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-09-18T10:16:01.375 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-09-18T10:16:01.375 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-09-18T10:16:01.384 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-09-18T10:16:01.426 INFO:tasks.ceph.osd.15.smithi189.stderr:2024-09-18T10:16:01.421+0000 7fd0e725e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 22401) UID: 0 2024-09-18T10:16:01.485 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-09-18T10:16:01.485 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-09-18T10:16:01.486 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-09-18T10:16:01.486 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-09-18T10:16:01.486 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-09-18T10:16:01.494 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-09-18T10:16:01.526 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-18T10:16:01.523+0000 7f578c52f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22134) UID: 0 2024-09-18T10:16:01.598 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-09-18T10:16:01.598 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-09-18T10:16:01.598 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-09-18T10:16:01.598 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-09-18T10:16:01.598 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.598 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:189 3.3s0 3:ce8164ce:::smithi18925896-7:head [copy-from ver 6] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:16:01.599 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:52.694279+0000", 2024-09-18T10:16:01.599 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 68.900439148999993, 2024-09-18T10:16:01.599 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.51539283599999997, 2024-09-18T10:16:01.599 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:01.599 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:01.599 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:01.599 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:16:01.599 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:01.599 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 189 2024-09-18T10:16:01.599 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.600 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:01.600 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.601 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:01.601 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694279+0000", 2024-09-18T10:16:01.601 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:01.601 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.601 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.601 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:01.601 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694279+0000", 2024-09-18T10:16:01.601 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:01.601 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.601 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.601 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:01.601 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694277+0000", 2024-09-18T10:16:01.601 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999971 2024-09-18T10:16:01.602 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.602 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.602 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:01.602 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694290+0000", 2024-09-18T10:16:01.602 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3862e-05 2024-09-18T10:16:01.602 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.602 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.602 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:01.602 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694311+0000", 2024-09-18T10:16:01.602 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.0273999999999999e-05 2024-09-18T10:16:01.602 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.602 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.602 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:01.602 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694322+0000", 2024-09-18T10:16:01.603 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1722999999999999e-05 2024-09-18T10:16:01.603 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.603 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.603 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:01.603 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.794494+0000", 2024-09-18T10:16:01.603 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100171653 2024-09-18T10:16:01.603 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.603 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.603 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:01.603 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.794705+0000", 2024-09-18T10:16:01.603 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00021126000000000001 2024-09-18T10:16:01.603 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.603 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.603 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:01.604 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.096113+0000", 2024-09-18T10:16:01.604 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.30140739100000002 2024-09-18T10:16:01.604 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.604 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.604 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:01.604 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.098602+0000", 2024-09-18T10:16:01.604 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.002489593 2024-09-18T10:16:01.604 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.604 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.604 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:01.604 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.209538+0000", 2024-09-18T10:16:01.604 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.110935482 2024-09-18T10:16:01.605 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.605 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.605 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:01.605 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.209672+0000", 2024-09-18T10:16:01.605 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00013435399999999999 2024-09-18T10:16:01.605 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:01.605 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:01.605 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:01.605 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.605 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.605 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:194 3.3s0 3:c9ce6440:::smithi18925896-24:head [copy-from ver 6] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:16:01.605 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:52.694388+0000", 2024-09-18T10:16:01.605 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 68.900330285999999, 2024-09-18T10:16:01.605 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.92118689200000003, 2024-09-18T10:16:01.606 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:01.606 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:01.606 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:01.606 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:16:01.606 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:01.606 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 194 2024-09-18T10:16:01.606 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.606 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:01.606 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.606 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:01.606 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694388+0000", 2024-09-18T10:16:01.606 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:01.606 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.607 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.607 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:01.607 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694388+0000", 2024-09-18T10:16:01.607 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:01.607 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.607 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.607 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:01.607 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694388+0000", 2024-09-18T10:16:01.607 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:16:01.607 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.607 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.607 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:01.607 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694393+0000", 2024-09-18T10:16:01.607 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.2299999999999999e-06 2024-09-18T10:16:01.608 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.608 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.608 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:01.608 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694411+0000", 2024-09-18T10:16:01.608 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7802999999999999e-05 2024-09-18T10:16:01.608 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.608 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.608 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:01.608 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694417+0000", 2024-09-18T10:16:01.608 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.4640000000000003e-06 2024-09-18T10:16:01.608 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.608 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.608 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:01.609 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.795289+0000", 2024-09-18T10:16:01.609 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100872288 2024-09-18T10:16:01.609 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.609 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.609 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:01.609 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.795548+0000", 2024-09-18T10:16:01.609 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00025869199999999998 2024-09-18T10:16:01.609 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.609 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.609 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:01.609 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.412215+0000", 2024-09-18T10:16:01.609 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.61666667100000006 2024-09-18T10:16:01.609 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.609 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.610 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:01.610 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.514838+0000", 2024-09-18T10:16:01.610 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10262323399999999 2024-09-18T10:16:01.610 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.610 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.610 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:01.610 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.615386+0000", 2024-09-18T10:16:01.610 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100548106 2024-09-18T10:16:01.610 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.610 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.610 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:01.610 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.615575+0000", 2024-09-18T10:16:01.610 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000189128 2024-09-18T10:16:01.610 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:01.610 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:01.611 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:01.611 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.611 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.611 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:195 3.3s0 3:c9ce6440:::smithi18925896-24:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:16:01.611 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:52.694424+0000", 2024-09-18T10:16:01.611 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 68.900294549999998, 2024-09-18T10:16:01.611 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.82009685499999996, 2024-09-18T10:16:01.611 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:01.611 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "started", 2024-09-18T10:16:01.611 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:01.611 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:16:01.611 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:01.611 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 195 2024-09-18T10:16:01.612 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.612 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:01.612 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.612 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:01.612 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694424+0000", 2024-09-18T10:16:01.612 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:01.612 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.612 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.612 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:01.612 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694424+0000", 2024-09-18T10:16:01.612 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:01.612 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.612 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.612 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:01.613 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694423+0000", 2024-09-18T10:16:01.613 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999995 2024-09-18T10:16:01.613 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.613 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.613 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:01.613 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694427+0000", 2024-09-18T10:16:01.613 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2650000000000001e-06 2024-09-18T10:16:01.613 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.613 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.613 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:01.613 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694434+0000", 2024-09-18T10:16:01.613 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.5410000000000003e-06 2024-09-18T10:16:01.613 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.613 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.614 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:01.614 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694441+0000", 2024-09-18T10:16:01.614 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.187e-06 2024-09-18T10:16:01.614 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.614 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.614 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:01.614 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.795870+0000", 2024-09-18T10:16:01.614 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.101428617 2024-09-18T10:16:01.614 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.614 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.614 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for blocked object", 2024-09-18T10:16:01.614 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.796040+0000", 2024-09-18T10:16:01.614 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00017008 2024-09-18T10:16:01.615 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.615 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.615 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:01.615 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.514130+0000", 2024-09-18T10:16:01.615 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.71809022199999994 2024-09-18T10:16:01.615 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.615 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.615 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:01.615 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.514348+0000", 2024-09-18T10:16:01.615 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000217488 2024-09-18T10:16:01.615 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.615 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.615 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:01.615 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.514521+0000", 2024-09-18T10:16:01.616 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00017300900000000001 2024-09-18T10:16:01.616 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:01.616 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:01.616 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:01.616 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.616 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.616 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.bs2 41/32 ECSubWrite(tid=58, reqid=osd.4.0:58, at_version=41'14, trim_to=0'0, roll_forward_to=39'12))", 2024-09-18T10:16:01.616 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:56.829698+0000", 2024-09-18T10:16:01.616 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 64.765020456000002, 2024-09-18T10:16:01.616 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.71711602600000002, 2024-09-18T10:16:01.616 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:01.616 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:16:01.617 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:01.617 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.617 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:01.617 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.829698+0000", 2024-09-18T10:16:01.617 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:01.617 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.617 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.617 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:01.617 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.829698+0000", 2024-09-18T10:16:01.617 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:01.617 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.617 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.617 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:01.617 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.829697+0000", 2024-09-18T10:16:01.618 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:16:01.618 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.618 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.618 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:01.618 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.829733+0000", 2024-09-18T10:16:01.618 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.6217999999999998e-05 2024-09-18T10:16:01.618 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.618 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.618 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:01.618 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.829757+0000", 2024-09-18T10:16:01.618 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.3915000000000001e-05 2024-09-18T10:16:01.618 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.618 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.618 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:01.619 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.829769+0000", 2024-09-18T10:16:01.619 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1754e-05 2024-09-18T10:16:01.619 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.619 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.619 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:01.619 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.342965+0000", 2024-09-18T10:16:01.619 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.51319593799999996 2024-09-18T10:16:01.619 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.619 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.619 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:01.619 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.343020+0000", 2024-09-18T10:16:01.619 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5367e-05 2024-09-18T10:16:01.619 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.619 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.620 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:01.620 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.546772+0000", 2024-09-18T10:16:01.620 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.203751933 2024-09-18T10:16:01.620 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.620 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.620 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:01.620 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.546814+0000", 2024-09-18T10:16:01.620 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.2196000000000001e-05 2024-09-18T10:16:01.620 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:01.620 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:01.620 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:01.620 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.620 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.620 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDPGPushReply(3.17s0 67/64 [PushReplyOp(3:ee962131:::smithi18925896-29:10)])", 2024-09-18T10:16:01.621 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:23.671965+0000", 2024-09-18T10:16:01.621 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 37.922754005999998, 2024-09-18T10:16:01.621 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.60673808500000004, 2024-09-18T10:16:01.621 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:01.621 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:16:01.621 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:01.621 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.621 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:01.621 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.671965+0000", 2024-09-18T10:16:01.621 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:01.621 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.621 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.621 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:01.621 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.671965+0000", 2024-09-18T10:16:01.622 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:01.622 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.622 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.622 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:01.622 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.671963+0000", 2024-09-18T10:16:01.622 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:16:01.622 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.622 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.622 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:01.622 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.671977+0000", 2024-09-18T10:16:01.622 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4093e-05 2024-09-18T10:16:01.622 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.622 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.622 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:01.623 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.672009+0000", 2024-09-18T10:16:01.623 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1939e-05 2024-09-18T10:16:01.623 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.623 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.623 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:01.623 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.772102+0000", 2024-09-18T10:16:01.623 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10009319 2024-09-18T10:16:01.623 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.623 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.623 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:01.623 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.278591+0000", 2024-09-18T10:16:01.623 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.50648915999999999 2024-09-18T10:16:01.623 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.623 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.624 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:01.624 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.278703+0000", 2024-09-18T10:16:01.624 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000111433 2024-09-18T10:16:01.624 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:01.624 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:01.624 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:01.624 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.624 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.624 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDPGPushReply(3.17s0 67/64 [PushReplyOp(3:ee962131:::smithi18925896-29:10)])", 2024-09-18T10:16:01.624 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:23.772436+0000", 2024-09-18T10:16:01.624 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 37.822282770000001, 2024-09-18T10:16:01.624 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.50655292500000004, 2024-09-18T10:16:01.624 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:01.625 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:16:01.625 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:01.625 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.625 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:01.625 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.772436+0000", 2024-09-18T10:16:01.625 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:01.625 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.625 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.625 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:01.625 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.772436+0000", 2024-09-18T10:16:01.625 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:01.625 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.625 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.625 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:01.626 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.772434+0000", 2024-09-18T10:16:01.626 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:16:01.626 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.626 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.626 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:01.626 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.772448+0000", 2024-09-18T10:16:01.626 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3682e-05 2024-09-18T10:16:01.626 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.626 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.626 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:01.626 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.772477+0000", 2024-09-18T10:16:01.626 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9255999999999999e-05 2024-09-18T10:16:01.626 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.626 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.627 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:01.627 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.772490+0000", 2024-09-18T10:16:01.627 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2962999999999999e-05 2024-09-18T10:16:01.627 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.627 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.627 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:01.627 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.278888+0000", 2024-09-18T10:16:01.627 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.50639794699999996 2024-09-18T10:16:01.627 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.627 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.627 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:01.627 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.278989+0000", 2024-09-18T10:16:01.627 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000100777 2024-09-18T10:16:01.627 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:01.628 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:01.628 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:01.628 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.628 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.628 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDPGPushReply(3.17s0 67/64 [PushReplyOp(3:ee962131:::smithi18925896-29:10)])", 2024-09-18T10:16:01.628 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:23.772911+0000", 2024-09-18T10:16:01.628 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 37.821807491000001, 2024-09-18T10:16:01.628 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.50636473400000004, 2024-09-18T10:16:01.628 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:01.628 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:16:01.628 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:01.628 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.628 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:01.629 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.772911+0000", 2024-09-18T10:16:01.629 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:01.629 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.629 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.629 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:01.629 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.772911+0000", 2024-09-18T10:16:01.629 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:01.629 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.629 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.629 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:01.629 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.772909+0000", 2024-09-18T10:16:01.629 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:16:01.629 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.630 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.630 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:01.630 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.772923+0000", 2024-09-18T10:16:01.630 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3957e-05 2024-09-18T10:16:01.630 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.630 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.630 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:01.630 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.772956+0000", 2024-09-18T10:16:01.630 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2925000000000001e-05 2024-09-18T10:16:01.630 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.630 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.630 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:01.630 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.772969+0000", 2024-09-18T10:16:01.630 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2516e-05 2024-09-18T10:16:01.631 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.631 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.631 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:01.631 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.279192+0000", 2024-09-18T10:16:01.631 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.50622349799999999 2024-09-18T10:16:01.631 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.631 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.631 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:01.631 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.279276+0000", 2024-09-18T10:16:01.631 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.3498000000000006e-05 2024-09-18T10:16:01.631 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:01.631 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:01.631 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:01.631 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.632 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.632 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDPGPushReply(3.17s0 67/64 [PushReplyOp(3:ead53b0f:::smithi18925896-25:13)])", 2024-09-18T10:16:01.632 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:23.875511+0000", 2024-09-18T10:16:01.632 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 37.719207865999998, 2024-09-18T10:16:01.632 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.50638906400000006, 2024-09-18T10:16:01.632 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:01.632 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:16:01.632 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:01.632 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.632 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:01.632 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.875511+0000", 2024-09-18T10:16:01.632 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:01.632 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.633 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.633 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:01.633 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.875511+0000", 2024-09-18T10:16:01.633 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:01.633 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.633 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.633 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:01.633 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.875510+0000", 2024-09-18T10:16:01.633 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999995 2024-09-18T10:16:01.633 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.633 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.633 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:01.633 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.875516+0000", 2024-09-18T10:16:01.634 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.8930000000000002e-06 2024-09-18T10:16:01.634 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.634 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.634 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:01.634 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.875526+0000", 2024-09-18T10:16:01.634 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0183000000000001e-05 2024-09-18T10:16:01.634 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.634 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.634 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:01.634 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.975617+0000", 2024-09-18T10:16:01.634 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10009069299999999 2024-09-18T10:16:01.634 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.634 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.635 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:01.635 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.381839+0000", 2024-09-18T10:16:01.635 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.40622223600000001 2024-09-18T10:16:01.635 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.635 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.635 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:01.635 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.381900+0000", 2024-09-18T10:16:01.635 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.0572000000000002e-05 2024-09-18T10:16:01.635 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:01.635 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:01.635 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:01.635 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.635 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.635 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:635 3.17s0 3:ead53b0f:::smithi18925896-25:head [append 0~655360] snapc 21=[20,1f,1e,1a,16,11,10] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-18T10:16:01.636 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:29.755594+0000", 2024-09-18T10:16:01.636 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 31.839124945999998, 2024-09-18T10:16:01.636 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.62129057300000001, 2024-09-18T10:16:01.636 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:01.636 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:01.636 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:01.636 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:16:01.636 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:01.636 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 635 2024-09-18T10:16:01.636 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.636 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:01.636 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.636 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:01.636 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.755594+0000", 2024-09-18T10:16:01.637 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:01.637 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.637 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.637 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:01.637 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.755594+0000", 2024-09-18T10:16:01.637 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:01.637 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.637 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.637 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:01.637 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.755590+0000", 2024-09-18T10:16:01.637 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999967 2024-09-18T10:16:01.637 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.637 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.638 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:01.638 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.756465+0000", 2024-09-18T10:16:01.638 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00087483000000000005 2024-09-18T10:16:01.638 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.638 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.638 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:01.638 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.756502+0000", 2024-09-18T10:16:01.638 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.7067999999999998e-05 2024-09-18T10:16:01.638 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.638 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.638 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:01.638 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.756516+0000", 2024-09-18T10:16:01.638 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4071e-05 2024-09-18T10:16:01.638 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.639 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.639 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:01.639 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.760022+0000", 2024-09-18T10:16:01.639 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0035059570000000001 2024-09-18T10:16:01.639 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.639 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.639 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:01.639 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.760234+0000", 2024-09-18T10:16:01.639 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00021161199999999999 2024-09-18T10:16:01.639 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.639 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.639 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:01.639 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.762520+0000", 2024-09-18T10:16:01.639 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0022855760000000001 2024-09-18T10:16:01.640 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.640 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.640 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:01.640 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.770533+0000", 2024-09-18T10:16:01.640 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0080138269999999994 2024-09-18T10:16:01.640 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.640 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.640 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:01.640 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:30.376766+0000", 2024-09-18T10:16:01.640 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.60623257500000005 2024-09-18T10:16:01.640 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.640 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.640 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:01.640 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:30.376884+0000", 2024-09-18T10:16:01.641 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00011819699999999999 2024-09-18T10:16:01.641 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:01.641 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:01.641 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:01.641 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.641 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.641 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:636 3.17s0 3:ead53b0f:::smithi18925896-25:head [append 0~606208] snapc 21=[20,1f,1e,1a,16,11,10] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-18T10:16:01.641 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:29.758851+0000", 2024-09-18T10:16:01.641 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 31.835867293, 2024-09-18T10:16:01.641 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.61852410899999999, 2024-09-18T10:16:01.641 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:01.641 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:01.641 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:01.641 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:16:01.642 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:01.642 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 636 2024-09-18T10:16:01.642 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.642 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:01.642 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.642 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:01.642 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.758851+0000", 2024-09-18T10:16:01.642 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:01.642 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.642 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.642 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:01.642 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.758851+0000", 2024-09-18T10:16:01.642 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:01.643 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.643 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.643 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:01.643 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.758849+0000", 2024-09-18T10:16:01.643 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999976 2024-09-18T10:16:01.643 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.643 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.643 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:01.643 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.759643+0000", 2024-09-18T10:16:01.643 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00079385800000000004 2024-09-18T10:16:01.643 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.643 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.643 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:01.644 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.759677+0000", 2024-09-18T10:16:01.644 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.4504e-05 2024-09-18T10:16:01.644 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.644 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.644 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:01.644 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.759691+0000", 2024-09-18T10:16:01.644 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3586e-05 2024-09-18T10:16:01.644 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.644 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.644 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:01.644 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.765309+0000", 2024-09-18T10:16:01.644 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0056179460000000004 2024-09-18T10:16:01.644 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.644 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.645 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:01.645 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.765656+0000", 2024-09-18T10:16:01.645 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000347783 2024-09-18T10:16:01.645 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.645 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.645 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:01.645 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.766735+0000", 2024-09-18T10:16:01.645 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0010785460000000001 2024-09-18T10:16:01.645 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.645 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.645 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:01.645 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.973532+0000", 2024-09-18T10:16:01.645 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20679750599999999 2024-09-18T10:16:01.645 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.646 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.646 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:01.646 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:30.377254+0000", 2024-09-18T10:16:01.646 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.40372181400000001 2024-09-18T10:16:01.646 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.646 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.646 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:01.646 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:30.377375+0000", 2024-09-18T10:16:01.646 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000121037 2024-09-18T10:16:01.646 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:01.646 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:01.646 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:01.646 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.646 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.647 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:637 3.17s0 3:ead53b0f:::smithi18925896-25:head [append 0~475136] snapc 21=[20,1f,1e,1a,16,11,10] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-18T10:16:01.647 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:29.761784+0000", 2024-09-18T10:16:01.647 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 31.832934439999999, 2024-09-18T10:16:01.647 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.61605535199999995, 2024-09-18T10:16:01.647 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:01.647 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:01.647 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:01.647 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:16:01.647 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:01.647 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 637 2024-09-18T10:16:01.647 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.647 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:01.647 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.647 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:01.648 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.761784+0000", 2024-09-18T10:16:01.648 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:01.648 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.648 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.648 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:01.648 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.761784+0000", 2024-09-18T10:16:01.648 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:01.648 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.648 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.648 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:01.648 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.761776+0000", 2024-09-18T10:16:01.648 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999919 2024-09-18T10:16:01.648 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.649 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.649 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:01.649 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.762424+0000", 2024-09-18T10:16:01.649 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00064803899999999999 2024-09-18T10:16:01.649 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.649 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.649 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:01.649 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.762455+0000", 2024-09-18T10:16:01.649 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1316000000000003e-05 2024-09-18T10:16:01.649 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.649 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.649 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:01.649 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.762471+0000", 2024-09-18T10:16:01.649 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.6174000000000001e-05 2024-09-18T10:16:01.650 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.650 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.650 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:01.650 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.767732+0000", 2024-09-18T10:16:01.650 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0052603190000000003 2024-09-18T10:16:01.650 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.650 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.650 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:01.650 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.767873+0000", 2024-09-18T10:16:01.650 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000140949 2024-09-18T10:16:01.650 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.650 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.650 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:01.650 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.768566+0000", 2024-09-18T10:16:01.651 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00069325799999999998 2024-09-18T10:16:01.651 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.651 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.651 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:01.651 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.973546+0000", 2024-09-18T10:16:01.651 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.204980144 2024-09-18T10:16:01.651 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.651 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.651 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:01.651 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:30.377732+0000", 2024-09-18T10:16:01.651 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.404185558 2024-09-18T10:16:01.651 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.652 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.652 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:01.652 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:30.377839+0000", 2024-09-18T10:16:01.652 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000107909 2024-09-18T10:16:01.652 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:01.652 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:01.652 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:01.652 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.652 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.652 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:638 3.17s0 3:ead53b0f:::smithi18925896-25:head [append 0~81920] snapc 21=[20,1f,1e,1a,16,11,10] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-18T10:16:01.652 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:29.762493+0000", 2024-09-18T10:16:01.652 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 31.83222542, 2024-09-18T10:16:01.652 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.61563624900000002, 2024-09-18T10:16:01.652 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:01.653 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:01.653 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:01.653 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:16:01.653 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:01.653 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 638 2024-09-18T10:16:01.653 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.653 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:01.653 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.653 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:01.653 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.762493+0000", 2024-09-18T10:16:01.653 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:01.653 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.653 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.653 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:01.654 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.762493+0000", 2024-09-18T10:16:01.654 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:01.654 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.654 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.654 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:01.654 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.762492+0000", 2024-09-18T10:16:01.654 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:16:01.654 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.654 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.654 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:01.654 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.762566+0000", 2024-09-18T10:16:01.654 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.3977e-05 2024-09-18T10:16:01.654 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.655 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.655 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:01.655 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.762585+0000", 2024-09-18T10:16:01.655 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.9074000000000001e-05 2024-09-18T10:16:01.655 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.655 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.655 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:01.655 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.762595+0000", 2024-09-18T10:16:01.655 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0913999999999999e-05 2024-09-18T10:16:01.655 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.655 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.655 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:01.655 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.769730+0000", 2024-09-18T10:16:01.655 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0071346680000000003 2024-09-18T10:16:01.656 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.656 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.656 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:01.656 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.769816+0000", 2024-09-18T10:16:01.656 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.5422999999999998e-05 2024-09-18T10:16:01.656 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.656 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.656 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:01.656 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.770214+0000", 2024-09-18T10:16:01.656 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00039818999999999998 2024-09-18T10:16:01.656 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.656 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.656 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:01.656 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.973818+0000", 2024-09-18T10:16:01.657 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20360445499999999 2024-09-18T10:16:01.657 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.657 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.657 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:01.657 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:30.378035+0000", 2024-09-18T10:16:01.657 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.40421707600000001 2024-09-18T10:16:01.657 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.657 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.657 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:01.657 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:30.378129+0000", 2024-09-18T10:16:01.657 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.4056999999999997e-05 2024-09-18T10:16:01.657 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:01.657 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:01.657 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:01.658 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.658 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.658 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:639 3.17s0 3:ead53b0f:::smithi18925896-25:head [setxattr _header (57) in=7b] snapc 21=[20,1f,1e,1a,16,11,10] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-18T10:16:01.658 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:29.762609+0000", 2024-09-18T10:16:01.658 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 31.832109266, 2024-09-18T10:16:01.658 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.61577652400000005, 2024-09-18T10:16:01.658 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:01.658 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:01.658 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:01.658 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:16:01.658 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:01.658 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 639 2024-09-18T10:16:01.658 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.658 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:01.659 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.659 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:01.659 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.762609+0000", 2024-09-18T10:16:01.659 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:01.659 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.659 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.659 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:01.659 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.762609+0000", 2024-09-18T10:16:01.659 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:01.659 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.659 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.659 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:01.659 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.762608+0000", 2024-09-18T10:16:01.659 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:16:01.660 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.660 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.660 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:01.660 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.762615+0000", 2024-09-18T10:16:01.660 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.2339999999999997e-06 2024-09-18T10:16:01.660 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.660 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.660 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:01.660 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.762628+0000", 2024-09-18T10:16:01.660 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2848e-05 2024-09-18T10:16:01.660 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.660 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.660 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:01.660 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.762636+0000", 2024-09-18T10:16:01.661 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.0970000000000006e-06 2024-09-18T10:16:01.661 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.661 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.661 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:01.661 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.870703+0000", 2024-09-18T10:16:01.661 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.108067121 2024-09-18T10:16:01.661 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.661 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.661 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:01.661 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.870949+0000", 2024-09-18T10:16:01.661 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00024526700000000002 2024-09-18T10:16:01.661 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.661 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.662 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:01.662 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.871876+0000", 2024-09-18T10:16:01.662 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00092680400000000004 2024-09-18T10:16:01.662 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.662 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.662 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:01.662 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.973821+0000", 2024-09-18T10:16:01.662 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10194552599999999 2024-09-18T10:16:01.662 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.662 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.662 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:01.662 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:30.378300+0000", 2024-09-18T10:16:01.662 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.40447920900000001 2024-09-18T10:16:01.662 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.663 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.663 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:01.663 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:30.378386+0000", 2024-09-18T10:16:01.663 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.5531999999999994e-05 2024-09-18T10:16:01.663 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:01.663 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:01.663 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:01.663 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.663 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.663 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.7s5 74/40 ECSubWrite(tid=168, reqid=osd.4.0:166, at_version=0'0, trim_to=0'0, roll_forward_to=74'47))", 2024-09-18T10:16:01.663 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:30.378493+0000", 2024-09-18T10:16:01.663 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 31.216225622, 2024-09-18T10:16:01.663 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.605881475, 2024-09-18T10:16:01.663 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:01.663 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:16:01.664 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:01.664 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.664 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:01.664 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:30.378493+0000", 2024-09-18T10:16:01.664 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:01.664 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.664 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.664 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:01.664 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:30.378493+0000", 2024-09-18T10:16:01.664 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:01.664 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.664 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.664 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:01.665 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:30.378492+0000", 2024-09-18T10:16:01.665 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:16:01.665 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.665 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.665 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:01.665 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:30.378513+0000", 2024-09-18T10:16:01.665 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.1427e-05 2024-09-18T10:16:01.665 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.665 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.665 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:01.665 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:30.378550+0000", 2024-09-18T10:16:01.665 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.7351999999999997e-05 2024-09-18T10:16:01.665 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.666 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.666 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:01.666 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:30.478641+0000", 2024-09-18T10:16:01.666 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100090965 2024-09-18T10:16:01.666 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.666 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.666 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:01.666 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:30.882297+0000", 2024-09-18T10:16:01.666 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.40365594799999999 2024-09-18T10:16:01.666 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.666 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.666 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:01.666 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:30.882320+0000", 2024-09-18T10:16:01.666 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.3127999999999999e-05 2024-09-18T10:16:01.667 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.667 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.667 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:01.667 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:30.984327+0000", 2024-09-18T10:16:01.667 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10200699100000001 2024-09-18T10:16:01.667 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.667 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.667 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:01.667 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:30.984374+0000", 2024-09-18T10:16:01.667 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.6953000000000003e-05 2024-09-18T10:16:01.667 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:01.667 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:01.667 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:01.667 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.668 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.668 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:758 3.17s0 3:ee962131:::smithi18925896-29:head [append 0~524288] snapc 2f=[2f,29,28,20,1f,1e,1a,11] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-18T10:16:01.668 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:40.612706+0000", 2024-09-18T10:16:01.668 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 20.982012869999998, 2024-09-18T10:16:01.668 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.60348796500000002, 2024-09-18T10:16:01.668 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:01.668 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:01.668 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:01.668 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:16:01.668 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:01.668 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 758 2024-09-18T10:16:01.668 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.668 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:01.668 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.669 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:01.669 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.612706+0000", 2024-09-18T10:16:01.669 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:01.669 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.669 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.669 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:01.669 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.612706+0000", 2024-09-18T10:16:01.669 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:01.669 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.669 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.669 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:01.669 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.612702+0000", 2024-09-18T10:16:01.669 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999967 2024-09-18T10:16:01.670 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.670 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.670 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:01.670 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.613684+0000", 2024-09-18T10:16:01.670 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00098185200000000007 2024-09-18T10:16:01.670 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.670 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.670 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:01.670 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.613722+0000", 2024-09-18T10:16:01.670 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.7490000000000002e-05 2024-09-18T10:16:01.670 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.670 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.670 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:01.671 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.613741+0000", 2024-09-18T10:16:01.671 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.9707999999999998e-05 2024-09-18T10:16:01.671 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.671 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.671 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:01.671 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.613931+0000", 2024-09-18T10:16:01.671 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00018992799999999999 2024-09-18T10:16:01.671 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.671 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.671 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op must wait for map", 2024-09-18T10:16:01.671 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.613950+0000", 2024-09-18T10:16:01.671 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8909000000000001e-05 2024-09-18T10:16:01.671 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.672 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.672 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:01.672 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.726459+0000", 2024-09-18T10:16:01.672 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.112508555 2024-09-18T10:16:01.672 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.672 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.672 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:01.672 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.726722+0000", 2024-09-18T10:16:01.672 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00026331999999999999 2024-09-18T10:16:01.672 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.672 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.672 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:01.672 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.728892+0000", 2024-09-18T10:16:01.673 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0021697040000000002 2024-09-18T10:16:01.673 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.673 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.673 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:01.673 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.741279+0000", 2024-09-18T10:16:01.673 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.012386856 2024-09-18T10:16:01.673 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.673 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.673 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:01.673 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:41.215936+0000", 2024-09-18T10:16:01.673 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.474657153 2024-09-18T10:16:01.673 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.673 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.673 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:01.674 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:41.216194+0000", 2024-09-18T10:16:01.674 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00025781799999999999 2024-09-18T10:16:01.674 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:01.674 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:01.674 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:01.674 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.674 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.674 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:759 3.17s0 3:ee962131:::smithi18925896-29:head [append 0~573440] snapc 2f=[2f,29,28,20,1f,1e,1a,11] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-18T10:16:01.674 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:40.620305+0000", 2024-09-18T10:16:01.674 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 20.974413209000002, 2024-09-18T10:16:01.674 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.59650130599999995, 2024-09-18T10:16:01.674 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:01.674 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:01.675 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:01.675 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:16:01.675 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:01.675 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 759 2024-09-18T10:16:01.675 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.675 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:01.675 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.675 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:01.675 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.620305+0000", 2024-09-18T10:16:01.675 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:01.675 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.675 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.675 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:01.676 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.620305+0000", 2024-09-18T10:16:01.676 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:01.676 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.676 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.676 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:01.676 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.620302+0000", 2024-09-18T10:16:01.676 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999971 2024-09-18T10:16:01.676 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.676 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.676 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:01.676 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.620967+0000", 2024-09-18T10:16:01.676 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00066427800000000003 2024-09-18T10:16:01.676 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.677 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.677 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:01.677 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.620994+0000", 2024-09-18T10:16:01.677 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.76e-05 2024-09-18T10:16:01.677 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.677 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.677 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:01.677 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.621004+0000", 2024-09-18T10:16:01.677 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0098000000000001e-05 2024-09-18T10:16:01.677 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.677 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.677 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:01.677 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.621152+0000", 2024-09-18T10:16:01.678 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00014735799999999999 2024-09-18T10:16:01.678 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.678 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.678 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:16:01.678 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.621173+0000", 2024-09-18T10:16:01.678 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.1223e-05 2024-09-18T10:16:01.678 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.678 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.678 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:01.678 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.730351+0000", 2024-09-18T10:16:01.678 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.109177487 2024-09-18T10:16:01.678 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.678 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.678 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:01.679 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.730553+0000", 2024-09-18T10:16:01.679 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00020222600000000001 2024-09-18T10:16:01.679 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.679 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.679 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:01.679 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.732189+0000", 2024-09-18T10:16:01.679 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001636153 2024-09-18T10:16:01.679 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.679 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.679 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:01.679 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.741283+0000", 2024-09-18T10:16:01.679 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0090935889999999991 2024-09-18T10:16:01.679 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.680 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.680 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:01.680 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:41.216570+0000", 2024-09-18T10:16:01.680 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.475287444 2024-09-18T10:16:01.680 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.680 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.680 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:01.680 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:41.216807+0000", 2024-09-18T10:16:01.680 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00023667999999999999 2024-09-18T10:16:01.680 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:01.680 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:01.680 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:01.680 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.680 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.681 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:760 3.17s0 3:ee962131:::smithi18925896-29:head [append 0~688128] snapc 2f=[2f,29,28,20,1f,1e,1a,11] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-18T10:16:01.681 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:40.624963+0000", 2024-09-18T10:16:01.681 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 20.969755704000001, 2024-09-18T10:16:01.681 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.592485437, 2024-09-18T10:16:01.681 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:01.681 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:01.681 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:01.681 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:16:01.681 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:01.681 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 760 2024-09-18T10:16:01.681 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.681 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:01.681 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.682 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:01.682 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.624963+0000", 2024-09-18T10:16:01.682 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:01.682 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.682 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.682 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:01.682 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.624963+0000", 2024-09-18T10:16:01.682 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:01.682 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.682 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.682 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:01.682 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.624960+0000", 2024-09-18T10:16:01.682 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999971 2024-09-18T10:16:01.682 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.683 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.683 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:01.683 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.625766+0000", 2024-09-18T10:16:01.683 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00080539100000000003 2024-09-18T10:16:01.683 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.683 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.683 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:01.683 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.625792+0000", 2024-09-18T10:16:01.683 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5989000000000001e-05 2024-09-18T10:16:01.683 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.683 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.683 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:01.683 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.625801+0000", 2024-09-18T10:16:01.683 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.4080000000000004e-06 2024-09-18T10:16:01.684 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.684 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.684 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:01.684 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.627383+0000", 2024-09-18T10:16:01.684 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0015816440000000001 2024-09-18T10:16:01.684 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.684 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.684 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:16:01.684 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.627397+0000", 2024-09-18T10:16:01.684 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3973e-05 2024-09-18T10:16:01.684 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.684 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.684 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:01.684 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.733560+0000", 2024-09-18T10:16:01.685 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.106163805 2024-09-18T10:16:01.685 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.685 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.685 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:01.685 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.733770+0000", 2024-09-18T10:16:01.685 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00020980900000000001 2024-09-18T10:16:01.685 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.685 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.685 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:01.685 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.735644+0000", 2024-09-18T10:16:01.685 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001874164 2024-09-18T10:16:01.685 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.685 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.686 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:01.686 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.741750+0000", 2024-09-18T10:16:01.686 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0061058379999999997 2024-09-18T10:16:01.686 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.686 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.686 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:01.686 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:41.217140+0000", 2024-09-18T10:16:01.686 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.47538995699999997 2024-09-18T10:16:01.686 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.686 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.686 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:01.686 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:41.217448+0000", 2024-09-18T10:16:01.686 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000308089 2024-09-18T10:16:01.687 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:01.687 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:01.687 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:01.687 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.687 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.687 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:761 3.17s0 3:ee962131:::smithi18925896-29:head [append 0~294912] snapc 2f=[2f,29,28,20,1f,1e,1a,11] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-18T10:16:01.687 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:40.626751+0000", 2024-09-18T10:16:01.687 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 20.967967436999999, 2024-09-18T10:16:01.687 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.59137154800000002, 2024-09-18T10:16:01.687 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:01.687 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:01.687 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:01.687 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:16:01.687 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:01.688 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 761 2024-09-18T10:16:01.688 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.688 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:01.688 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.688 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:01.688 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.626751+0000", 2024-09-18T10:16:01.688 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:01.688 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.688 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.688 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:01.688 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.626751+0000", 2024-09-18T10:16:01.688 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:01.689 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.689 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.689 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:01.689 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.626749+0000", 2024-09-18T10:16:01.689 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:16:01.689 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.689 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.689 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:01.689 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.627187+0000", 2024-09-18T10:16:01.689 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000437682 2024-09-18T10:16:01.689 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.689 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.689 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:01.690 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.627209+0000", 2024-09-18T10:16:01.690 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.2195999999999999e-05 2024-09-18T10:16:01.690 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.690 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.690 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:01.690 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.627218+0000", 2024-09-18T10:16:01.690 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.4829999999999999e-06 2024-09-18T10:16:01.690 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.690 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.690 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:01.690 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.627438+0000", 2024-09-18T10:16:01.690 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00022063399999999999 2024-09-18T10:16:01.690 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.690 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.691 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:16:01.691 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.627447+0000", 2024-09-18T10:16:01.691 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.687e-06 2024-09-18T10:16:01.691 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.691 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.691 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:01.691 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.737040+0000", 2024-09-18T10:16:01.691 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10959300800000001 2024-09-18T10:16:01.691 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.691 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.691 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:01.691 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.737281+0000", 2024-09-18T10:16:01.691 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00024113100000000001 2024-09-18T10:16:01.692 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.692 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.692 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:01.692 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.738816+0000", 2024-09-18T10:16:01.692 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0015353649999999999 2024-09-18T10:16:01.692 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.692 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.692 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:01.692 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.741752+0000", 2024-09-18T10:16:01.692 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0029357239999999998 2024-09-18T10:16:01.692 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.692 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.692 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:01.693 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:41.217885+0000", 2024-09-18T10:16:01.693 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.476132522 2024-09-18T10:16:01.693 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.693 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.693 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:01.693 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:41.218123+0000", 2024-09-18T10:16:01.693 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000237968 2024-09-18T10:16:01.693 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:01.693 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:01.693 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:01.693 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.693 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.693 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:762 3.17s0 3:ee962131:::smithi18925896-29:head [setxattr _header (57) in=7b] snapc 2f=[2f,29,28,20,1f,1e,1a,11] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-18T10:16:01.693 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:40.627233+0000", 2024-09-18T10:16:01.694 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 20.967485605, 2024-09-18T10:16:01.694 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.68868450599999997, 2024-09-18T10:16:01.694 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:01.694 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:01.694 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:01.694 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:16:01.694 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:01.694 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 762 2024-09-18T10:16:01.694 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.694 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:01.694 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.694 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:01.694 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.627233+0000", 2024-09-18T10:16:01.694 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:01.694 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.695 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.695 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:01.695 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.627233+0000", 2024-09-18T10:16:01.695 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:01.695 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.695 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.695 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:01.695 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.627232+0000", 2024-09-18T10:16:01.695 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:16:01.695 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.695 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.695 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:01.695 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.627238+0000", 2024-09-18T10:16:01.695 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.7570000000000001e-06 2024-09-18T10:16:01.696 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.696 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.696 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:01.696 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.627246+0000", 2024-09-18T10:16:01.696 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.0199999999999994e-06 2024-09-18T10:16:01.696 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.696 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.696 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:01.696 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.627251+0000", 2024-09-18T10:16:01.696 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9599999999999999e-06 2024-09-18T10:16:01.696 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.696 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.696 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:01.697 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.627481+0000", 2024-09-18T10:16:01.697 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00023046199999999999 2024-09-18T10:16:01.697 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.697 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.697 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:16:01.697 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.627493+0000", 2024-09-18T10:16:01.697 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2173e-05 2024-09-18T10:16:01.697 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.697 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.697 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:01.697 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.740060+0000", 2024-09-18T10:16:01.697 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.112566686 2024-09-18T10:16:01.697 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.698 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.698 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:01.698 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.740191+0000", 2024-09-18T10:16:01.698 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00013140999999999999 2024-09-18T10:16:01.698 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.698 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.698 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:01.698 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.740901+0000", 2024-09-18T10:16:01.698 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00070978199999999995 2024-09-18T10:16:01.698 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.698 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.698 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:01.698 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.741853+0000", 2024-09-18T10:16:01.698 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00095228399999999996 2024-09-18T10:16:01.699 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.699 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.699 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:01.699 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:41.315753+0000", 2024-09-18T10:16:01.699 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.57389904999999997 2024-09-18T10:16:01.699 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.699 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.699 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:01.699 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:41.315917+0000", 2024-09-18T10:16:01.699 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00016491699999999999 2024-09-18T10:16:01.699 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:01.699 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:01.699 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:01.699 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.700 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.700 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:763 3.17s0 3:ee962131:::smithi18925896-29:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e85)", 2024-09-18T10:16:01.700 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:40.627257+0000", 2024-09-18T10:16:01.700 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 20.967461319999998, 2024-09-18T10:16:01.700 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.89335734899999997, 2024-09-18T10:16:01.700 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:01.700 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "started", 2024-09-18T10:16:01.700 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:01.700 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:16:01.700 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:01.700 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 763 2024-09-18T10:16:01.701 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.701 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:01.701 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.701 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:01.702 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.627257+0000", 2024-09-18T10:16:01.702 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:01.702 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.702 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.702 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:01.702 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.627257+0000", 2024-09-18T10:16:01.702 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:01.702 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.702 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.702 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:01.702 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.627256+0000", 2024-09-18T10:16:01.702 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:16:01.702 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.702 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.703 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:01.703 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.627260+0000", 2024-09-18T10:16:01.703 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.7450000000000001e-06 2024-09-18T10:16:01.703 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.703 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.703 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:01.703 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.627269+0000", 2024-09-18T10:16:01.703 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.4400000000000005e-06 2024-09-18T10:16:01.703 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.703 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.703 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:01.703 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.627273+0000", 2024-09-18T10:16:01.703 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.8600000000000001e-06 2024-09-18T10:16:01.703 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.704 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.704 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:01.704 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.717888+0000", 2024-09-18T10:16:01.704 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.090614586999999996 2024-09-18T10:16:01.704 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.704 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.704 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:16:01.704 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.717909+0000", 2024-09-18T10:16:01.704 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.0942999999999999e-05 2024-09-18T10:16:01.704 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.704 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.704 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:01.704 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.741375+0000", 2024-09-18T10:16:01.705 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.023465604000000001 2024-09-18T10:16:01.705 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.705 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.705 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for rw locks", 2024-09-18T10:16:01.705 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.741656+0000", 2024-09-18T10:16:01.705 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00028103600000000002 2024-09-18T10:16:01.705 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.705 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.705 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:01.705 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:41.317563+0000", 2024-09-18T10:16:01.705 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.57590774499999997 2024-09-18T10:16:01.705 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.705 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.705 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:01.706 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:41.317798+0000", 2024-09-18T10:16:01.706 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00023456099999999999 2024-09-18T10:16:01.706 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:01.706 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:01.706 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:01.706 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:41.520615+0000", 2024-09-18T10:16:01.706 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20281658499999999 2024-09-18T10:16:01.706 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:01.706 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:01.706 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:01.706 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:01.706 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:01.706 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-09-18T10:16:01.709 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_ops_in_flight 2024-09-18T10:16:01.711 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-18T10:16:01.713 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-18T10:16:01.703+0000 7f00046a8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 22136) UID: 0 2024-09-18T10:16:01.713 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-09-18T10:16:01.714 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":136,"stamp":"2024-09-18T10:16:01.222731+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":128254515,"num_objects":135,"num_object_clones":83,"num_object_copies":925,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":135,"num_whiteouts":9,"num_read":346,"num_read_kb":163176,"num_write":758,"num_write_kb":104418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":134,"num_bytes_recovered":168099840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1814,"ondisk_log_size":1814,"up":200,"acting":200,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":209,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":16,"kb":1509949440,"kb_used":5091476,"kb_used_data":271292,"kb_used_omap":26,"kb_used_meta":4819877,"kb_avail":1504857964,"statfs":{"total":1546188226560,"available":1540974555136,"internally_reserved":0,"allocated":277803008,"data_stored":364443876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":27264,"internal_metadata":4935554432},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-2883584,"num_objects":-5,"num_object_clones":-5,"num_object_copies":-35,"num_objects_missing_on_primary":-6,"num_objects_missing":-6,"num_objects_degraded":-7,"num_objects_misplaced":-18,"num_objects_unfound":0,"num_objects_dirty":-5,"num_whiteouts":0,"num_read":5,"num_read_kb":1298,"num_write":42,"num_write_kb":96,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":24887296,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001372"},"pg_stats":[{"pgid":"3.19","version":"104'134","reported_seq":188,"reported_epoch":104,"state":"active+clean+snaptrim","last_fresh":"2024-09-18T10:16:00.711344+0000","last_change":"2024-09-18T10:16:00.711459+0000","last_active":"2024-09-18T10:16:00.711344+0000","last_peered":"2024-09-18T10:16:00.711344+0000","last_clean":"2024-09-18T10:16:00.711344+0000","last_became_active":"2024-09-18T10:15:51.892167+0000","last_became_peered":"2024-09-18T10:15:51.892167+0000","last_unstale":"2024-09-18T10:16:00.711344+0000","last_undegraded":"2024-09-18T10:16:00.711344+0000","last_fullsized":"2024-09-18T10:16:00.711344+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":5,"last_scrub":"68'92","last_scrub_stamp":"2024-09-18T10:15:27.485910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:27.485910+0000","objects_scrubbed":10,"log_size":134,"log_dups_size":0,"ondisk_log_size":134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":10,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:39.070888+0000","scrub_duration":0.69568305100000005,"objects_trimmed":1,"snaptrim_duration":0.84685460899999998,"stat_sum":{"num_bytes":16646144,"num_objects":13,"num_object_clones":9,"num_object_copies":91,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":11,"num_read_kb":16756,"num_write":17,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":28753920,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,1,2,13,9,14],"acting":[8,4,1,2,13,9,14],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[{"start":"10","length":"1"},{"start":"12","length":"1"},{"start":"16","length":"1"},{"start":"1a","length":"1"}]},{"pgid":"3.18","version":"103'67","reported_seq":199,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-18T10:15:58.945752+0000","last_change":"2024-09-18T10:15:58.945752+0000","last_active":"2024-09-18T10:15:58.945752+0000","last_peered":"2024-09-18T10:15:58.945752+0000","last_clean":"2024-09-18T10:15:58.945752+0000","last_became_active":"2024-09-18T10:15:29.028750+0000","last_became_peered":"2024-09-18T10:15:29.028750+0000","last_unstale":"2024-09-18T10:15:58.945752+0000","last_undegraded":"2024-09-18T10:15:58.945752+0000","last_fullsized":"2024-09-18T10:15:58.945752+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":5,"last_scrub":"93'61","last_scrub_stamp":"2024-09-18T10:15:51.095148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:51.095148+0000","objects_scrubbed":5,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:56.109718+0000","scrub_duration":0.91822677900000005,"objects_trimmed":1,"snaptrim_duration":0.30585581699999997,"stat_sum":{"num_bytes":6291456,"num_objects":4,"num_object_clones":2,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,11,0,4,6,13,12],"acting":[10,11,0,4,6,13,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[{"start":"10","length":"1"},{"start":"12","length":"1"},{"start":"16","length":"1"},{"start":"1a","length":"1"},{"start":"1e","length":"3"},{"start":"26","length":"1"},{"start":"29","length":"2"},{"start":"2d","length":"1"},{"start":"2f","length":"2"},{"start":"35","length":"1"}]},{"pgid":"3.17","version":"103'81","reported_seq":233,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-18T10:15:59.065040+0000","last_change":"2024-09-18T10:15:59.065040+0000","last_active":"2024-09-18T10:15:59.065040+0000","last_peered":"2024-09-18T10:15:59.065040+0000","last_clean":"2024-09-18T10:15:59.065040+0000","last_became_active":"2024-09-18T10:15:21.323733+0000","last_became_peered":"2024-09-18T10:15:21.323733+0000","last_unstale":"2024-09-18T10:15:59.065040+0000","last_undegraded":"2024-09-18T10:15:59.065040+0000","last_fullsized":"2024-09-18T10:15:59.065040+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":5,"last_scrub":"74'45","last_scrub_stamp":"2024-09-18T10:15:31.583098+0000","last_deep_scrub":"74'45","last_deep_scrub_stamp":"2024-09-18T10:15:31.583098+0000","last_clean_scrub_stamp":"2024-09-18T10:15:31.583098+0000","objects_scrubbed":5,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:52.458457+0000","scrub_duration":2.494961714,"objects_trimmed":1,"snaptrim_duration":0.49059750099999999,"stat_sum":{"num_bytes":6537216,"num_objects":4,"num_object_clones":2,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":11,"num_write_kb":3920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,13,3,14,2,11,6],"acting":[10,13,3,14,2,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[{"start":"10","length":"1"},{"start":"12","length":"1"},{"start":"16","length":"1"},{"start":"1a","length":"1"},{"start":"1e","length":"3"},{"start":"26","length":"1"},{"start":"29","length":"2"},{"start":"2d","length":"1"},{"start":"2f","length":"2"},{"start":"35","length":"1"}]},{"pgid":"3.16","version":"103'49","reported_seq":235,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-18T10:15:58.947189+0000","last_change":"2024-09-18T10:15:59.149421+0000","last_active":"2024-09-18T10:15:58.947189+0000","last_peered":"2024-09-18T10:15:58.947189+0000","last_clean":"2024-09-18T10:15:58.947189+0000","last_became_active":"2024-09-18T10:15:50.782912+0000","last_became_peered":"2024-09-18T10:15:50.782912+0000","last_unstale":"2024-09-18T10:15:58.947189+0000","last_undegraded":"2024-09-18T10:15:58.947189+0000","last_fullsized":"2024-09-18T10:15:58.947189+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":5,"last_scrub":"36'11","last_scrub_stamp":"2024-09-18T10:15:07.445786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:07.445786+0000","objects_scrubbed":1,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:14.115122+0000","scrub_duration":0.30645798400000002,"objects_trimmed":0,"snaptrim_duration":0.47534329800000003,"stat_sum":{"num_bytes":8486912,"num_objects":4,"num_object_clones":3,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":3,"num_read_kb":1,"num_write":9,"num_write_kb":7616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":13729792,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,6,4,8,2,14,10],"acting":[9,6,4,8,2,14,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[{"start":"10","length":"1"},{"start":"12","length":"1"},{"start":"16","length":"1"},{"start":"1a","length":"1"},{"start":"1e","length":"3"},{"start":"26","length":"1"},{"start":"29","length":"2"},{"start":"2d","length":"1"},{"start":"2f","length":"2"},{"start":"35","length":"1"}]},{"pgid":"3.15","version":"100'31","reported_seq":191,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-18T10:15:59.665425+0000","last_change":"2024-09-18T10:15:58.565869+0000","last_active":"2024-09-18T10:15:59.665425+0000","last_peered":"2024-09-18T10:15:59.665425+0000","last_clean":"2024-09-18T10:15:59.665425+0000","last_became_active":"2024-09-18T10:15:17.127771+0000","last_became_peered":"2024-09-18T10:15:17.127771+0000","last_unstale":"2024-09-18T10:15:59.665425+0000","last_undegraded":"2024-09-18T10:15:59.665425+0000","last_fullsized":"2024-09-18T10:15:59.665425+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":5,"last_scrub":"66'10","last_scrub_stamp":"2024-09-18T10:15:27.270892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:27.270892+0000","objects_scrubbed":2,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:57.241541+0000","scrub_duration":0.77475592100000001,"objects_trimmed":0,"snaptrim_duration":0.002978016,"stat_sum":{"num_bytes":2408448,"num_objects":3,"num_object_clones":2,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":1,"num_read":2,"num_read_kb":1,"num_write":26,"num_write_kb":2288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,15,13,10,1,7,9],"acting":[14,15,13,10,1,7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"10","length":"1"},{"start":"12","length":"1"},{"start":"16","length":"1"},{"start":"1a","length":"1"},{"start":"1e","length":"3"},{"start":"26","length":"1"},{"start":"29","length":"2"},{"start":"2d","length":"1"},{"start":"2f","length":"2"},{"start":"35","length":"1"}]},{"pgid":"3.14","version":"37'8","reported_seq":155,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-18T10:15:58.576521+0000","last_change":"2024-09-18T10:15:58.576521+0000","last_active":"2024-09-18T10:15:58.576521+0000","last_peered":"2024-09-18T10:15:58.576521+0000","last_clean":"2024-09-18T10:15:58.576521+0000","last_became_active":"2024-09-18T10:15:14.767770+0000","last_became_peered":"2024-09-18T10:15:14.767770+0000","last_unstale":"2024-09-18T10:15:58.576521+0000","last_undegraded":"2024-09-18T10:15:58.576521+0000","last_fullsized":"2024-09-18T10:15:58.576521+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":5,"last_scrub":"37'8","last_scrub_stamp":"2024-09-18T10:15:17.041590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:17.041590+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:32.741813+0000","scrub_duration":0.60018000800000004,"objects_trimmed":0,"snaptrim_duration":0.0011844620000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,14,9,10,5,13,4],"acting":[1,14,9,10,5,13,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"10","length":"1"},{"start":"12","length":"1"},{"start":"16","length":"1"},{"start":"1a","length":"1"},{"start":"1e","length":"3"},{"start":"26","length":"1"},{"start":"29","length":"2"},{"start":"2d","length":"1"},{"start":"2f","length":"2"},{"start":"35","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":85,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-18T10:15:54.822773+0000","last_change":"2024-09-18T10:14:37.427722+0000","last_active":"2024-09-18T10:15:54.822773+0000","last_peered":"2024-09-18T10:15:54.822773+0000","last_clean":"2024-09-18T10:15:54.822773+0000","last_became_active":"2024-09-18T10:14:35.348943+0000","last_became_peered":"2024-09-18T10:14:35.348943+0000","last_unstale":"2024-09-18T10:15:54.822773+0000","last_undegraded":"2024-09-18T10:15:54.822773+0000","last_fullsized":"2024-09-18T10:15:54.822773+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:14:34.288596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:03.161727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060442999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,10],"acting":[13,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"3.6","version":"103'54","reported_seq":307,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-18T10:15:58.986722+0000","last_change":"2024-09-18T10:15:58.986722+0000","last_active":"2024-09-18T10:15:58.986722+0000","last_peered":"2024-09-18T10:15:58.986722+0000","last_clean":"2024-09-18T10:15:58.986722+0000","last_became_active":"2024-09-18T10:14:57.139786+0000","last_became_peered":"2024-09-18T10:14:57.139786+0000","last_unstale":"2024-09-18T10:15:58.986722+0000","last_undegraded":"2024-09-18T10:15:58.986722+0000","last_fullsized":"2024-09-18T10:15:58.986722+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'38","last_scrub_stamp":"2024-09-18T10:15:29.667178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:29.667178+0000","objects_scrubbed":5,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:51.483631+0000","scrub_duration":0.41099292199999998,"objects_trimmed":0,"snaptrim_duration":0.41307451099999998,"stat_sum":{"num_bytes":1310720,"num_objects":5,"num_object_clones":1,"num_object_copies":35,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":8,"num_read_kb":1490,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,14,6,2,5,9,15],"acting":[0,14,6,2,5,9,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"10","length":"1"},{"start":"12","length":"1"},{"start":"16","length":"1"},{"start":"1a","length":"1"},{"start":"1e","length":"3"},{"start":"26","length":"1"},{"start":"29","length":"2"},{"start":"2d","length":"1"},{"start":"2f","length":"2"},{"start":"35","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":18,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-18T10:15:58.568610+0000","last_change":"2024-09-18T10:15:50.978910+0000","last_active":"2024-09-18T10:15:58.568610+0000","last_peered":"2024-09-18T10:15:58.568610+0000","last_clean":"2024-09-18T10:15:58.568610+0000","last_became_active":"2024-09-18T10:15:50.978055+0000","last_became_peered":"2024-09-18T10:15:50.978055+0000","last_unstale":"2024-09-18T10:15:58.568610+0000","last_undegraded":"2024-09-18T10:15:58.568610+0000","last_fullsized":"2024-09-18T10:15:58.568610+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:14:34.288596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11],"acting":[8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.7","version":"103'78","reported_seq":313,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-18T10:15:59.257162+0000","last_change":"2024-09-18T10:15:59.257162+0000","last_active":"2024-09-18T10:15:59.257162+0000","last_peered":"2024-09-18T10:15:59.257162+0000","last_clean":"2024-09-18T10:15:59.257162+0000","last_became_active":"2024-09-18T10:14:57.137776+0000","last_became_peered":"2024-09-18T10:14:57.137776+0000","last_unstale":"2024-09-18T10:15:59.257162+0000","last_undegraded":"2024-09-18T10:15:59.257162+0000","last_fullsized":"2024-09-18T10:15:59.257162+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"60'30","last_scrub_stamp":"2024-09-18T10:15:17.136535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:17.136535+0000","objects_scrubbed":3,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:35.499009+0000","scrub_duration":0.60680385400000003,"objects_trimmed":1,"snaptrim_duration":0.58309704600000001,"stat_sum":{"num_bytes":4489216,"num_objects":5,"num_object_clones":3,"num_object_copies":35,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":6,"num_read_kb":8019,"num_write":32,"num_write_kb":3440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6,0,14,10,13],"acting":[4,11,6,0,14,10,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"10","length":"1"},{"start":"12","length":"1"},{"start":"16","length":"1"},{"start":"1a","length":"1"},{"start":"1e","length":"3"},{"start":"26","length":"1"},{"start":"29","length":"2"},{"start":"2d","length":"1"},{"start":"2f","length":"2"},{"start":"35","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":98,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-18T10:15:58.666484+0000","last_change":"2024-09-18T10:15:50.883382+0000","last_active":"2024-09-18T10:15:58.666484+0000","last_peered":"2024-09-18T10:15:58.666484+0000","last_clean":"2024-09-18T10:15:58.666484+0000","last_became_active":"2024-09-18T10:14:35.329198+0000","last_became_peered":"2024-09-18T10:14:35.329198+0000","last_unstale":"2024-09-18T10:15:58.666484+0000","last_undegraded":"2024-09-18T10:15:58.666484+0000","last_fullsized":"2024-09-18T10:15:58.666484+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:15:50.883147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:15:50.883147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:17:15.846467+0000","scrub_duration":0.153222372,"objects_trimmed":0,"snaptrim_duration":0.00062415699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"104'116","reported_seq":137,"reported_epoch":104,"state":"active+clean+snaptrim","last_fresh":"2024-09-18T10:16:00.303305+0000","last_change":"2024-09-18T10:16:00.101220+0000","last_active":"2024-09-18T10:16:00.303305+0000","last_peered":"2024-09-18T10:16:00.303305+0000","last_clean":"2024-09-18T10:16:00.303305+0000","last_became_active":"2024-09-18T10:15:59.893680+0000","last_became_peered":"2024-09-18T10:15:59.893680+0000","last_unstale":"2024-09-18T10:16:00.303305+0000","last_undegraded":"2024-09-18T10:16:00.303305+0000","last_fullsized":"2024-09-18T10:16:00.303305+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"60'34","last_scrub_stamp":"2024-09-18T10:15:17.325003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:17.325003+0000","objects_scrubbed":6,"log_size":116,"log_dups_size":0,"ondisk_log_size":116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":5,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:47.029587+0000","scrub_duration":0.82233752999999998,"objects_trimmed":1,"snaptrim_duration":0.94050503500000004,"stat_sum":{"num_bytes":10207232,"num_objects":12,"num_object_clones":9,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":1,"num_read":16,"num_read_kb":15010,"num_write":21,"num_write_kb":10976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":12435456,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,15,4,2,6,10,7],"acting":[8,15,4,2,6,10,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[{"start":"10","length":"1"},{"start":"12","length":"1"},{"start":"16","length":"1"},{"start":"1a","length":"1"},{"start":"1e","length":"1"},{"start":"26","length":"1"},{"start":"29","length":"2"},{"start":"2d","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":89,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-18T10:15:58.573935+0000","last_change":"2024-09-18T10:14:37.426192+0000","last_active":"2024-09-18T10:15:58.573935+0000","last_peered":"2024-09-18T10:15:58.573935+0000","last_clean":"2024-09-18T10:15:58.573935+0000","last_became_active":"2024-09-18T10:14:35.524102+0000","last_became_peered":"2024-09-18T10:14:35.524102+0000","last_unstale":"2024-09-18T10:15:58.573935+0000","last_undegraded":"2024-09-18T10:15:58.573935+0000","last_fullsized":"2024-09-18T10:15:58.573935+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:14:34.288596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.10077985,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.5","version":"35'2","reported_seq":254,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-18T10:15:58.677092+0000","last_change":"2024-09-18T10:15:58.677092+0000","last_active":"2024-09-18T10:15:58.677092+0000","last_peered":"2024-09-18T10:15:58.677092+0000","last_clean":"2024-09-18T10:15:58.677092+0000","last_became_active":"2024-09-18T10:15:50.780770+0000","last_became_peered":"2024-09-18T10:15:50.780770+0000","last_unstale":"2024-09-18T10:15:58.677092+0000","last_undegraded":"2024-09-18T10:15:58.677092+0000","last_fullsized":"2024-09-18T10:15:58.677092+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"35'2","last_scrub_stamp":"2024-09-18T10:15:03.886316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:03.886316+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:16.496741+0000","scrub_duration":0.35694589500000001,"objects_trimmed":0,"snaptrim_duration":0.0019609720000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,4,9,12,0,11],"acting":[5,8,4,9,12,0,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"10","length":"1"},{"start":"12","length":"1"},{"start":"16","length":"1"},{"start":"1a","length":"1"},{"start":"1e","length":"3"},{"start":"26","length":"1"},{"start":"29","length":"2"},{"start":"2d","length":"1"},{"start":"2f","length":"2"},{"start":"35","length":"1"}]},{"pgid":"3.3","version":"103'31","reported_seq":262,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-18T10:15:59.048958+0000","last_change":"2024-09-18T10:15:58.947168+0000","last_active":"2024-09-18T10:15:59.048958+0000","last_peered":"2024-09-18T10:15:59.048958+0000","last_clean":"2024-09-18T10:15:59.048958+0000","last_became_active":"2024-09-18T10:14:57.138546+0000","last_became_peered":"2024-09-18T10:14:57.138546+0000","last_unstale":"2024-09-18T10:15:59.048958+0000","last_undegraded":"2024-09-18T10:15:59.048958+0000","last_fullsized":"2024-09-18T10:15:59.048958+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'23","last_scrub_stamp":"2024-09-18T10:15:28.483344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:28.483344+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:47.612849+0000","scrub_duration":0.40619440600000001,"objects_trimmed":0,"snaptrim_duration":0.19894819599999999,"stat_sum":{"num_bytes":606208,"num_objects":2,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":1,"num_write":26,"num_write_kb":2864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4,13,11,2,5],"acting":[10,9,4,13,11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[{"start":"10","length":"1"},{"start":"12","length":"1"},{"start":"16","length":"1"},{"start":"1a","length":"1"},{"start":"1e","length":"3"},{"start":"26","length":"1"},{"start":"29","length":"2"},{"start":"2d","length":"1"},{"start":"2f","length":"2"},{"start":"35","length":"1"}]},{"pgid":"2.2","version":"29'2","reported_seq":87,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-18T10:15:54.822425+0000","last_change":"2024-09-18T10:14:37.429583+0000","last_active":"2024-09-18T10:15:54.822425+0000","last_peered":"2024-09-18T10:15:54.822425+0000","last_clean":"2024-09-18T10:15:54.822425+0000","last_became_active":"2024-09-18T10:14:35.715374+0000","last_became_peered":"2024-09-18T10:14:35.715374+0000","last_unstale":"2024-09-18T10:15:54.822425+0000","last_undegraded":"2024-09-18T10:15:54.822425+0000","last_fullsized":"2024-09-18T10:15:54.822425+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:14:34.288596+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:03.520789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002421224,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,12],"acting":[13,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"3.0","version":"101'40","reported_seq":273,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-18T10:15:58.572149+0000","last_change":"2024-09-18T10:15:58.572149+0000","last_active":"2024-09-18T10:15:58.572149+0000","last_peered":"2024-09-18T10:15:58.572149+0000","last_clean":"2024-09-18T10:15:58.572149+0000","last_became_active":"2024-09-18T10:14:57.444374+0000","last_became_peered":"2024-09-18T10:14:57.444374+0000","last_unstale":"2024-09-18T10:15:58.572149+0000","last_undegraded":"2024-09-18T10:15:58.572149+0000","last_fullsized":"2024-09-18T10:15:58.572149+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'4","last_scrub_stamp":"2024-09-18T10:15:05.668933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:05.668933+0000","objects_scrubbed":1,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:18.433780+0000","scrub_duration":2.6328342170000001,"objects_trimmed":0,"snaptrim_duration":0.0020690270000000002,"stat_sum":{"num_bytes":1982464,"num_objects":3,"num_object_clones":2,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":1,"num_read":4,"num_read_kb":1,"num_write":7,"num_write_kb":1936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,9,2,6,15,4],"acting":[3,7,9,2,6,15,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"10","length":"1"},{"start":"12","length":"1"},{"start":"16","length":"1"},{"start":"1a","length":"1"},{"start":"1e","length":"3"},{"start":"26","length":"1"},{"start":"29","length":"2"},{"start":"2d","length":"1"},{"start":"2f","length":"2"},{"start":"35","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":91,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-18T10:15:59.665502+0000","last_change":"2024-09-18T10:14:37.601046+0000","last_active":"2024-09-18T10:15:59.665502+0000","last_peered":"2024-09-18T10:15:59.665502+0000","last_clean":"2024-09-18T10:15:59.665502+0000","last_became_active":"2024-09-18T10:14:35.348262+0000","last_became_peered":"2024-09-18T10:14:35.348262+0000","last_unstale":"2024-09-18T10:15:59.665502+0000","last_undegraded":"2024-09-18T10:15:59.665502+0000","last_fullsized":"2024-09-18T10:15:59.665502+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:14:34.288596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:03.520789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063430400000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,0],"acting":[14,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"3.1","version":"103'114","reported_seq":342,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-18T10:15:59.253229+0000","last_change":"2024-09-18T10:15:59.253229+0000","last_active":"2024-09-18T10:15:59.253229+0000","last_peered":"2024-09-18T10:15:59.253229+0000","last_clean":"2024-09-18T10:15:59.253229+0000","last_became_active":"2024-09-18T10:14:57.038503+0000","last_became_peered":"2024-09-18T10:14:57.038503+0000","last_unstale":"2024-09-18T10:15:59.253229+0000","last_undegraded":"2024-09-18T10:15:59.253229+0000","last_fullsized":"2024-09-18T10:15:59.253229+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"93'85","last_scrub_stamp":"2024-09-18T10:15:51.347579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:51.347579+0000","objects_scrubbed":9,"log_size":114,"log_dups_size":0,"ondisk_log_size":114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:17:08.459990+0000","scrub_duration":0.71137397400000002,"objects_trimmed":1,"snaptrim_duration":0.67973042699999997,"stat_sum":{"num_bytes":3522560,"num_objects":10,"num_object_clones":7,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":1,"num_read":10,"num_read_kb":14691,"num_write":23,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,13,14,10,7,3,2],"acting":[4,13,14,10,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"10","length":"1"},{"start":"12","length":"1"},{"start":"16","length":"1"},{"start":"1a","length":"1"},{"start":"1e","length":"3"},{"start":"26","length":"1"},{"start":"29","length":"2"},{"start":"2d","length":"1"},{"start":"2f","length":"2"},{"start":"35","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":89,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-18T10:15:57.732339+0000","last_change":"2024-09-18T10:14:37.600953+0000","last_active":"2024-09-18T10:15:57.732339+0000","last_peered":"2024-09-18T10:15:57.732339+0000","last_clean":"2024-09-18T10:15:57.732339+0000","last_became_active":"2024-09-18T10:14:35.349203+0000","last_became_peered":"2024-09-18T10:14:35.349203+0000","last_unstale":"2024-09-18T10:15:57.732339+0000","last_undegraded":"2024-09-18T10:15:57.732339+0000","last_fullsized":"2024-09-18T10:15:57.732339+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:14:34.288596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060130600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3],"acting":[9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2","version":"103'133","reported_seq":368,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-18T10:15:59.665368+0000","last_change":"2024-09-18T10:15:59.291746+0000","last_active":"2024-09-18T10:15:59.665368+0000","last_peered":"2024-09-18T10:15:59.665368+0000","last_clean":"2024-09-18T10:15:59.665368+0000","last_became_active":"2024-09-18T10:14:57.227520+0000","last_became_peered":"2024-09-18T10:14:57.227520+0000","last_unstale":"2024-09-18T10:15:59.665368+0000","last_undegraded":"2024-09-18T10:15:59.665368+0000","last_fullsized":"2024-09-18T10:15:59.665368+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'18","last_scrub_stamp":"2024-09-18T10:15:03.202100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:03.202100+0000","objects_scrubbed":6,"log_size":133,"log_dups_size":0,"ondisk_log_size":133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:12.722351+0000","scrub_duration":0.63960403700000001,"objects_trimmed":0,"snaptrim_duration":0.54067356499999997,"stat_sum":{"num_bytes":8814592,"num_objects":11,"num_object_clones":7,"num_object_copies":77,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":1,"num_read":26,"num_read_kb":5884,"num_write":42,"num_write_kb":6432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,3,13,1,5,10,7],"acting":[14,3,13,1,5,10,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"10","length":"1"},{"start":"12","length":"1"},{"start":"16","length":"1"},{"start":"1a","length":"1"},{"start":"1e","length":"3"},{"start":"26","length":"1"},{"start":"29","length":"2"},{"start":"2d","length":"1"},{"start":"2f","length":"2"},{"start":"35","length":"1"}]},{"pgid":"2.3","version":"27'1","reported_seq":92,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-18T10:15:59.665283+0000","last_change":"2024-09-18T10:14:37.801252+0000","last_active":"2024-09-18T10:15:59.665283+0000","last_peered":"2024-09-18T10:15:59.665283+0000","last_clean":"2024-09-18T10:15:59.665283+0000","last_became_active":"2024-09-18T10:14:35.487746+0000","last_became_peered":"2024-09-18T10:14:35.487746+0000","last_unstale":"2024-09-18T10:15:59.665283+0000","last_undegraded":"2024-09-18T10:15:59.665283+0000","last_fullsized":"2024-09-18T10:15:59.665283+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:14:34.288596+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:03.161727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.10072764100000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,5],"acting":[14,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"1.0","version":"29'112","reported_seq":224,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-18T10:15:58.569598+0000","last_change":"2024-09-18T10:14:32.509942+0000","last_active":"2024-09-18T10:15:58.569598+0000","last_peered":"2024-09-18T10:15:58.569598+0000","last_clean":"2024-09-18T10:15:58.569598+0000","last_became_active":"2024-09-18T10:14:32.508037+0000","last_became_peered":"2024-09-18T10:14:32.508037+0000","last_unstale":"2024-09-18T10:15:58.569598+0000","last_undegraded":"2024-09-18T10:15:58.569598+0000","last_fullsized":"2024-09-18T10:15:58.569598+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:31.267741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:31.267741+0000","last_clean_scrub_stamp":"2024-09-18T10:14:31.267741+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,15],"acting":[6,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.8","version":"103'69","reported_seq":288,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-18T10:15:59.179962+0000","last_change":"2024-09-18T10:15:59.179962+0000","last_active":"2024-09-18T10:15:59.179962+0000","last_peered":"2024-09-18T10:15:59.179962+0000","last_clean":"2024-09-18T10:15:59.179962+0000","last_became_active":"2024-09-18T10:14:57.443754+0000","last_became_peered":"2024-09-18T10:14:57.443754+0000","last_unstale":"2024-09-18T10:15:59.179962+0000","last_undegraded":"2024-09-18T10:15:59.179962+0000","last_fullsized":"2024-09-18T10:15:59.179962+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"41'13","last_scrub_stamp":"2024-09-18T10:15:05.373444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:05.373444+0000","objects_scrubbed":2,"log_size":69,"log_dups_size":0,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:31.932850+0000","scrub_duration":2.8745114210000002,"objects_trimmed":0,"snaptrim_duration":0.510307387,"stat_sum":{"num_bytes":3899392,"num_objects":7,"num_object_clones":5,"num_object_copies":49,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":7,"num_read_kb":706,"num_write":9,"num_write_kb":2640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,11,3,15,1,7],"acting":[6,0,11,3,15,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"10","length":"1"},{"start":"12","length":"1"},{"start":"16","length":"1"},{"start":"1a","length":"1"},{"start":"1e","length":"3"},{"start":"26","length":"1"},{"start":"29","length":"2"},{"start":"2d","length":"1"},{"start":"2f","length":"2"},{"start":"35","length":"1"}]},{"pgid":"3.9","version":"103'102","reported_seq":419,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-18T10:15:58.779408+0000","last_change":"2024-09-18T10:15:58.779408+0000","last_active":"2024-09-18T10:15:58.779408+0000","last_peered":"2024-09-18T10:15:58.779408+0000","last_clean":"2024-09-18T10:15:58.779408+0000","last_became_active":"2024-09-18T10:15:51.188859+0000","last_became_peered":"2024-09-18T10:15:51.188859+0000","last_unstale":"2024-09-18T10:15:58.779408+0000","last_undegraded":"2024-09-18T10:15:58.779408+0000","last_fullsized":"2024-09-18T10:15:58.779408+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'35","last_scrub_stamp":"2024-09-18T10:15:07.077517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:07.077517+0000","objects_scrubbed":4,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:18.140951+0000","scrub_duration":0.71488955499999995,"objects_trimmed":1,"snaptrim_duration":0.205162702,"stat_sum":{"num_bytes":4325376,"num_objects":5,"num_object_clones":3,"num_object_copies":35,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":1,"num_read":21,"num_read_kb":21422,"num_write":50,"num_write_kb":5392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":12812288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,10,11,5,8,3],"acting":[7,2,10,11,5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"10","length":"1"},{"start":"12","length":"1"},{"start":"16","length":"1"},{"start":"1a","length":"1"},{"start":"1e","length":"3"},{"start":"26","length":"1"},{"start":"29","length":"2"},{"start":"2d","length":"1"},{"start":"2f","length":"2"},{"start":"35","length":"1"}]},{"pgid":"3.a","version":"99'68","reported_seq":274,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-18T10:15:55.727250+0000","last_change":"2024-09-18T10:15:55.829874+0000","last_active":"2024-09-18T10:15:55.727250+0000","last_peered":"2024-09-18T10:15:55.727250+0000","last_clean":"2024-09-18T10:15:55.727250+0000","last_became_active":"2024-09-18T10:14:46.765703+0000","last_became_peered":"2024-09-18T10:14:46.765703+0000","last_unstale":"2024-09-18T10:15:55.727250+0000","last_undegraded":"2024-09-18T10:15:55.727250+0000","last_fullsized":"2024-09-18T10:15:55.727250+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:14:45.576168+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":0,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:08.721290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.103045204,"stat_sum":{"num_bytes":3538944,"num_objects":5,"num_object_clones":3,"num_object_copies":35,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":42,"num_read_kb":16745,"num_write":42,"num_write_kb":6032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,1,6,5,3,15,4],"acting":[13,1,6,5,3,15,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"10","length":"1"},{"start":"12","length":"1"},{"start":"16","length":"1"},{"start":"1a","length":"1"},{"start":"1e","length":"1"},{"start":"26","length":"1"},{"start":"29","length":"2"},{"start":"2d","length":"1"},{"start":"2f","length":"1"},{"start":"35","length":"1"}]},{"pgid":"3.b","version":"103'54","reported_seq":250,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-18T10:15:59.047977+0000","last_change":"2024-09-18T10:15:59.047977+0000","last_active":"2024-09-18T10:15:59.047977+0000","last_peered":"2024-09-18T10:15:59.047977+0000","last_clean":"2024-09-18T10:15:59.047977+0000","last_became_active":"2024-09-18T10:14:46.802834+0000","last_became_peered":"2024-09-18T10:14:46.802834+0000","last_unstale":"2024-09-18T10:15:59.047977+0000","last_undegraded":"2024-09-18T10:15:59.047977+0000","last_fullsized":"2024-09-18T10:15:59.047977+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:14:45.576168+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:04.590511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.47418787099999998,"stat_sum":{"num_bytes":4177920,"num_objects":5,"num_object_clones":3,"num_object_copies":35,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":7,"num_read_kb":3,"num_write":55,"num_write_kb":4368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,10,12,11,1,14],"acting":[4,0,10,12,11,1,14],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"10","length":"1"},{"start":"12","length":"1"},{"start":"16","length":"1"},{"start":"1a","length":"1"},{"start":"1e","length":"3"},{"start":"26","length":"1"},{"start":"29","length":"2"},{"start":"2d","length":"1"},{"start":"2f","length":"2"},{"start":"35","length":"1"}]},{"pgid":"3.c","version":"104'184","reported_seq":471,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-18T10:16:00.401467+0000","last_change":"2024-09-18T10:16:00.403225+0000","last_active":"2024-09-18T10:16:00.401467+0000","last_peered":"2024-09-18T10:16:00.401467+0000","last_clean":"2024-09-18T10:16:00.401467+0000","last_became_active":"2024-09-18T10:15:50.784815+0000","last_became_peered":"2024-09-18T10:15:50.784815+0000","last_unstale":"2024-09-18T10:16:00.401467+0000","last_undegraded":"2024-09-18T10:16:00.401467+0000","last_fullsized":"2024-09-18T10:16:00.401467+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:14:45.576168+0000","objects_scrubbed":0,"log_size":184,"log_dups_size":0,"ondisk_log_size":184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.50700858199999999,"stat_sum":{"num_bytes":17154048,"num_objects":14,"num_object_clones":9,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":1,"num_read":42,"num_read_kb":24573,"num_write":110,"num_write_kb":17424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":55410688,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,13,3,7,12,14],"acting":[5,8,13,3,7,12,14],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"10","length":"1"},{"start":"12","length":"1"},{"start":"16","length":"1"},{"start":"1a","length":"1"},{"start":"1e","length":"3"},{"start":"26","length":"1"},{"start":"29","length":"2"},{"start":"2d","length":"1"},{"start":"2f","length":"2"},{"start":"35","length":"1"}]},{"pgid":"3.d","version":"100'43","reported_seq":284,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-18T10:15:56.136262+0000","last_change":"2024-09-18T10:15:56.138266+0000","last_active":"2024-09-18T10:15:56.136262+0000","last_peered":"2024-09-18T10:15:56.136262+0000","last_clean":"2024-09-18T10:15:56.136262+0000","last_became_active":"2024-09-18T10:15:50.985282+0000","last_became_peered":"2024-09-18T10:15:50.985282+0000","last_unstale":"2024-09-18T10:15:56.136262+0000","last_undegraded":"2024-09-18T10:15:56.136262+0000","last_fullsized":"2024-09-18T10:15:56.136262+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:14:45.576168+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:03.900063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.31071810500000002,"stat_sum":{"num_bytes":3817472,"num_objects":4,"num_object_clones":3,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":19,"num_read_kb":17526,"num_write":12,"num_write_kb":3968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":8388608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,1,11,15,9,8,4],"acting":[13,1,11,15,9,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"10","length":"1"},{"start":"12","length":"1"},{"start":"16","length":"1"},{"start":"1a","length":"1"},{"start":"1e","length":"1"},{"start":"26","length":"1"},{"start":"29","length":"2"},{"start":"2d","length":"1"},{"start":"2f","length":"1"},{"start":"35","length":"1"}]},{"pgid":"3.e","version":"103'74","reported_seq":340,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-18T10:15:58.768854+0000","last_change":"2024-09-18T10:15:58.770364+0000","last_active":"2024-09-18T10:15:58.768854+0000","last_peered":"2024-09-18T10:15:58.768854+0000","last_clean":"2024-09-18T10:15:58.768854+0000","last_became_active":"2024-09-18T10:15:51.077393+0000","last_became_peered":"2024-09-18T10:15:51.077393+0000","last_unstale":"2024-09-18T10:15:58.768854+0000","last_undegraded":"2024-09-18T10:15:58.768854+0000","last_fullsized":"2024-09-18T10:15:58.768854+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:14:45.576168+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.20657598799999999,"stat_sum":{"num_bytes":5423104,"num_objects":5,"num_object_clones":3,"num_object_copies":35,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":2,"num_read":14,"num_read_kb":2406,"num_write":46,"num_write_kb":5296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":15237120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,14,6,13,8,3,12],"acting":[11,14,6,13,8,3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"10","length":"1"},{"start":"12","length":"1"},{"start":"16","length":"1"},{"start":"1a","length":"1"},{"start":"1e","length":"3"},{"start":"26","length":"1"},{"start":"29","length":"2"},{"start":"2d","length":"1"},{"start":"2f","length":"2"},{"start":"35","length":"1"}]},{"pgid":"3.f","version":"100'58","reported_seq":244,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-18T10:15:56.131672+0000","last_change":"2024-09-18T10:15:56.133377+0000","last_active":"2024-09-18T10:15:56.131672+0000","last_peered":"2024-09-18T10:15:56.131672+0000","last_clean":"2024-09-18T10:15:56.131672+0000","last_became_active":"2024-09-18T10:14:46.803296+0000","last_became_peered":"2024-09-18T10:14:46.803296+0000","last_unstale":"2024-09-18T10:15:56.131672+0000","last_undegraded":"2024-09-18T10:15:56.131672+0000","last_fullsized":"2024-09-18T10:15:56.131672+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:14:45.576168+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:11.624416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.40615378299999999,"stat_sum":{"num_bytes":6815744,"num_objects":5,"num_object_clones":3,"num_object_copies":35,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":11,"num_read_kb":1782,"num_write":42,"num_write_kb":7216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,3,15,14,2,11,0],"acting":[13,3,15,14,2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"10","length":"1"},{"start":"12","length":"1"},{"start":"16","length":"1"},{"start":"1a","length":"1"},{"start":"1e","length":"1"},{"start":"26","length":"1"},{"start":"29","length":"2"},{"start":"2d","length":"1"},{"start":"2f","length":"1"},{"start":"35","length":"1"}]},{"pgid":"3.10","version":"103'43","reported_seq":285,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-18T10:15:58.682011+0000","last_change":"2024-09-18T10:15:58.682011+0000","last_active":"2024-09-18T10:15:58.682011+0000","last_peered":"2024-09-18T10:15:58.682011+0000","last_clean":"2024-09-18T10:15:58.682011+0000","last_became_active":"2024-09-18T10:15:50.787275+0000","last_became_peered":"2024-09-18T10:15:50.787275+0000","last_unstale":"2024-09-18T10:15:58.682011+0000","last_undegraded":"2024-09-18T10:15:58.682011+0000","last_fullsized":"2024-09-18T10:15:58.682011+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'19","last_scrub_stamp":"2024-09-18T10:15:29.719467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:29.719467+0000","objects_scrubbed":1,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:30.348118+0000","scrub_duration":2.0403324629999999,"objects_trimmed":0,"snaptrim_duration":0.112609244,"stat_sum":{"num_bytes":5488640,"num_objects":3,"num_object_clones":2,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":6,"num_read_kb":16083,"num_write":9,"num_write_kb":3712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":21331968,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,5,4,8,0,9],"acting":[6,7,5,4,8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"10","length":"1"},{"start":"12","length":"1"},{"start":"16","length":"1"},{"start":"1a","length":"1"},{"start":"1e","length":"3"},{"start":"26","length":"1"},{"start":"29","length":"2"},{"start":"2d","length":"1"},{"start":"2f","length":"2"},{"start":"35","length":"1"}]},{"pgid":"3.11","version":"36'6","reported_seq":175,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-18T10:15:58.771680+0000","last_change":"2024-09-18T10:15:58.771680+0000","last_active":"2024-09-18T10:15:58.771680+0000","last_peered":"2024-09-18T10:15:58.771680+0000","last_clean":"2024-09-18T10:15:58.771680+0000","last_became_active":"2024-09-18T10:15:04.955869+0000","last_became_peered":"2024-09-18T10:15:04.955869+0000","last_unstale":"2024-09-18T10:15:58.771680+0000","last_undegraded":"2024-09-18T10:15:58.771680+0000","last_fullsized":"2024-09-18T10:15:58.771680+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":5,"last_scrub":"36'6","last_scrub_stamp":"2024-09-18T10:15:07.755163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:07.755163+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:13.113623+0000","scrub_duration":0.60886176599999997,"objects_trimmed":0,"snaptrim_duration":0.101449467,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,13,10,0,14,2],"acting":[3,5,13,10,0,14,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"10","length":"1"},{"start":"12","length":"1"},{"start":"16","length":"1"},{"start":"1a","length":"1"},{"start":"1e","length":"3"},{"start":"26","length":"1"},{"start":"29","length":"2"},{"start":"2d","length":"1"},{"start":"2f","length":"2"},{"start":"35","length":"1"}]},{"pgid":"3.12","version":"36'8","reported_seq":279,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-18T10:15:59.665537+0000","last_change":"2024-09-18T10:15:58.564395+0000","last_active":"2024-09-18T10:15:59.665537+0000","last_peered":"2024-09-18T10:15:59.665537+0000","last_clean":"2024-09-18T10:15:59.665537+0000","last_became_active":"2024-09-18T10:15:50.682909+0000","last_became_peered":"2024-09-18T10:15:50.682909+0000","last_unstale":"2024-09-18T10:15:59.665537+0000","last_undegraded":"2024-09-18T10:15:59.665537+0000","last_fullsized":"2024-09-18T10:15:59.665537+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":5,"last_scrub":"36'8","last_scrub_stamp":"2024-09-18T10:15:04.353687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:04.353687+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:11.257401+0000","scrub_duration":0.76511709400000005,"objects_trimmed":0,"snaptrim_duration":0.001712454,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,5,8,3,4,11,1],"acting":[14,5,8,3,4,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"10","length":"1"},{"start":"12","length":"1"},{"start":"16","length":"1"},{"start":"1a","length":"1"},{"start":"1e","length":"3"},{"start":"26","length":"1"},{"start":"29","length":"2"},{"start":"2d","length":"1"},{"start":"2f","length":"2"},{"start":"35","length":"1"}]},{"pgid":"3.13","version":"93'52","reported_seq":245,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-18T10:15:58.576553+0000","last_change":"2024-09-18T10:15:58.576553+0000","last_active":"2024-09-18T10:15:58.576553+0000","last_peered":"2024-09-18T10:15:58.576553+0000","last_clean":"2024-09-18T10:15:58.576553+0000","last_became_active":"2024-09-18T10:15:13.263306+0000","last_became_peered":"2024-09-18T10:15:13.263306+0000","last_unstale":"2024-09-18T10:15:58.576553+0000","last_undegraded":"2024-09-18T10:15:58.576553+0000","last_fullsized":"2024-09-18T10:15:58.576553+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":5,"last_scrub":"60'33","last_scrub_stamp":"2024-09-18T10:15:19.140411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:19.140411+0000","objects_scrubbed":1,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:39.399176+0000","scrub_duration":0.706777345,"objects_trimmed":0,"snaptrim_duration":0.001165826,"stat_sum":{"num_bytes":1720320,"num_objects":2,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,13,2,6,0,4,15],"acting":[1,13,2,6,0,4,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"10","length":"1"},{"start":"12","length":"1"},{"start":"16","length":"1"},{"start":"1a","length":"1"},{"start":"1e","length":"3"},{"start":"26","length":"1"},{"start":"29","length":"2"},{"start":"2d","length":"1"},{"start":"2f","length":"2"},{"start":"35","length":"1"}]}],"pool_stats":[{"poolid":3,"num_pg":26,"stat_sum":{"num_bytes":127664128,"num_objects":131,"num_object_clones":83,"num_object_copies":917,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":131,"num_whiteouts":9,"num_read":260,"num_read_kb":163103,"num_write":611,"num_write_kb":101744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":134,"num_bytes_recovered":168099840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":245366784,"data_stored":340729856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1699,"ondisk_log_size":1699,"up":182,"acting":182,"num_store_stats":16},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":13},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":112,"ondisk_log_size":112,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":15,"up_from":22,"seq":94489280532,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312120,"kb_used_data":10592,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94059720,"statfs":{"total":96636764160,"available":96317153280,"internally_reserved":0,"allocated":10846208,"data_stored":13879221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,4,6,7,8,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:15:31 2024","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":1.161},{"interface":"front","average":{"1min":6.7190000000000003,"5min":6.7190000000000003,"15min":6.7190000000000003},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":101.291,"5min":101.291,"15min":101.291},"last":1.119}]},{"osd":1,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.70199999999999996},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.931,"5min":1.931,"15min":1.931},"last":0.67300000000000004}]},{"osd":2,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.417,"5min":1.417,"15min":1.417},"last":0.79700000000000004},{"interface":"front","average":{"1min":5.5389999999999997,"5min":5.5389999999999997,"15min":5.5389999999999997},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":78.242999999999995,"5min":78.242999999999995,"15min":78.242999999999995},"last":0.85099999999999998}]},{"osd":3,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":3.3029999999999999,"5min":3.3029999999999999,"15min":3.3029999999999999},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":41.720999999999997,"5min":41.720999999999997,"15min":41.720999999999997},"last":0.55400000000000005},{"interface":"front","average":{"1min":1.014,"5min":1.014,"15min":1.014},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.49,"5min":1.49,"15min":1.49},"last":0.89500000000000002}]},{"osd":4,"last update":"Wed Sep 18 10:15:31 2024","interfaces":[{"interface":"back","average":{"1min":11.021000000000001,"5min":11.021000000000001,"15min":11.021000000000001},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":99.745999999999995,"5min":99.745999999999995,"15min":99.745999999999995},"last":1.032},{"interface":"front","average":{"1min":11.140000000000001,"5min":11.140000000000001,"15min":11.140000000000001},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":99.781000000000006,"5min":99.781000000000006,"15min":99.781000000000006},"last":1.369}]},{"osd":5,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":1.028},{"interface":"front","average":{"1min":1.7869999999999999,"5min":1.7869999999999999,"15min":1.7869999999999999},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":10.874000000000001,"5min":10.874000000000001,"15min":10.874000000000001},"last":0.92400000000000004}]},{"osd":6,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":1.9710000000000001,"5min":1.9710000000000001,"15min":1.9710000000000001},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":12.367000000000001,"5min":12.367000000000001,"15min":12.367000000000001},"last":0.93300000000000005},{"interface":"front","average":{"1min":5.6109999999999998,"5min":5.6109999999999998,"15min":5.6109999999999998},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":74.156000000000006,"5min":74.156000000000006,"15min":74.156000000000006},"last":0.98899999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.276}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90000000000000002}]},{"osd":10,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":6.3680000000000003,"5min":6.3680000000000003,"15min":6.3680000000000003},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":86.546999999999997,"5min":86.546999999999997,"15min":86.546999999999997},"last":1.2310000000000001},{"interface":"front","average":{"1min":6.375,"5min":6.375,"15min":6.375},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":86.497,"5min":86.497,"15min":86.497},"last":1.093}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3280000000000001}]},{"osd":13,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":6.0819999999999999,"5min":6.0819999999999999,"15min":6.0819999999999999},"min":{"1min":1.093,"5min":1.093,"15min":1.093},"max":{"1min":80.197000000000003,"5min":80.197000000000003,"15min":80.197000000000003},"last":1.5069999999999999},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.8420000000000001,"5min":1.8420000000000001,"15min":1.8420000000000001},"last":1.421}]},{"osd":14,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":1.444,"5min":1.444,"15min":1.444},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":2.661,"5min":2.661,"15min":2.661},"last":1.1419999999999999},{"interface":"front","average":{"1min":10.750999999999999,"5min":10.750999999999999,"15min":10.750999999999999},"min":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"max":{"1min":85.006,"5min":85.006,"15min":85.006},"last":1.554}]}]},{"osd":14,"up_from":22,"seq":94489280532,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":325144,"kb_used_data":23624,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94046696,"statfs":{"total":96636764160,"available":96303816704,"internally_reserved":0,"allocated":24190976,"data_stored":33359253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,3,4,5,7,8,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":15.391,"5min":15.391,"15min":15.391},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":189.88800000000001,"5min":189.88800000000001,"15min":189.88800000000001},"last":0.94199999999999995},{"interface":"front","average":{"1min":19.533000000000001,"5min":19.533000000000001,"15min":19.533000000000001},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":180.09999999999999,"5min":180.09999999999999,"15min":180.09999999999999},"last":0.98099999999999998}]},{"osd":1,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":9.1829999999999998,"5min":9.1829999999999998,"15min":9.1829999999999998},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":79.626000000000005,"5min":79.626000000000005,"15min":79.626000000000005},"last":21.334},{"interface":"front","average":{"1min":15.465,"5min":15.465,"15min":15.465},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":189.762,"5min":189.762,"15min":189.762},"last":1.1890000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.417}]},{"osd":3,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":15.247,"5min":15.247,"15min":15.247},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":180.19900000000001,"5min":180.19900000000001,"15min":180.19900000000001},"last":1.0660000000000001},{"interface":"front","average":{"1min":21.341999999999999,"5min":21.341999999999999,"15min":21.341999999999999},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":180.851,"5min":180.851,"15min":180.851},"last":38.560000000000002}]},{"osd":4,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":25.399999999999999,"5min":25.399999999999999,"15min":25.399999999999999},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":189.809,"5min":189.809,"15min":189.809},"last":1.091},{"interface":"front","average":{"1min":15.281000000000001,"5min":15.281000000000001,"15min":15.281000000000001},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":180.31700000000001,"5min":180.31700000000001,"15min":180.31700000000001},"last":133.244}]},{"osd":5,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":26.425999999999998,"5min":26.425999999999998,"15min":26.425999999999998},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":277.41199999999998,"5min":277.41199999999998,"15min":277.41199999999998},"last":1.0269999999999999},{"interface":"front","average":{"1min":15.683,"5min":15.683,"15min":15.683},"min":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":189.64699999999999,"5min":189.64699999999999,"15min":189.64699999999999},"last":1.3120000000000001}]},{"osd":7,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":15.460000000000001,"5min":15.460000000000001,"15min":15.460000000000001},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":180.268,"5min":180.268,"15min":180.268},"last":1.417},{"interface":"front","average":{"1min":15.154,"5min":15.154,"15min":15.154},"min":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":180.631,"5min":180.631,"15min":180.631},"last":1.23}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.552}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.754}]},{"osd":10,"last update":"Wed Sep 18 10:15:49 2024","interfaces":[{"interface":"back","average":{"1min":17.039000000000001,"5min":17.039000000000001,"15min":17.039000000000001},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":180.15899999999999,"5min":180.15899999999999,"15min":180.15899999999999},"last":1.6859999999999999},{"interface":"front","average":{"1min":31.004000000000001,"5min":31.004000000000001,"15min":31.004000000000001},"min":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":269.423,"5min":269.423,"15min":269.423},"last":1.853}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":40.415999999999997}]},{"osd":12,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":15.855,"5min":15.855,"15min":15.855},"min":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"max":{"1min":189.84999999999999,"5min":189.84999999999999,"15min":189.84999999999999},"last":1.661},{"interface":"front","average":{"1min":15.666,"5min":15.666,"15min":15.666},"min":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"max":{"1min":180.381,"5min":180.381,"15min":180.381},"last":1.8049999999999999}]},{"osd":13,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":26.244,"5min":26.244,"15min":26.244},"min":{"1min":0.875,"5min":0.875,"15min":0.875},"max":{"1min":180.59100000000001,"5min":180.59100000000001,"15min":180.59100000000001},"last":1.5840000000000001},{"interface":"front","average":{"1min":25.079000000000001,"5min":25.079000000000001,"15min":25.079000000000001},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":189.71299999999999,"5min":189.71299999999999,"15min":189.71299999999999},"last":1.7270000000000001}]},{"osd":15,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":15.680999999999999,"5min":15.680999999999999,"15min":15.680999999999999},"min":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"max":{"1min":180.44800000000001,"5min":180.44800000000001,"15min":180.44800000000001},"last":1.8959999999999999},{"interface":"front","average":{"1min":18.472000000000001,"5min":18.472000000000001,"15min":18.472000000000001},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":180.53,"5min":180.53,"15min":180.53},"last":1.6379999999999999}]}]},{"osd":13,"up_from":22,"seq":94489280532,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":326484,"kb_used_data":24968,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94045356,"statfs":{"total":96636764160,"available":96302444544,"internally_reserved":0,"allocated":25567232,"data_stored":37055334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,6,8,9,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":20.742000000000001,"5min":20.742000000000001,"15min":20.742000000000001},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":176.66800000000001,"5min":176.66800000000001,"15min":176.66800000000001},"last":1.3160000000000001},{"interface":"front","average":{"1min":30.433,"5min":30.433,"15min":30.433},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":199.732,"5min":199.732,"15min":199.732},"last":0.92800000000000005}]},{"osd":1,"last update":"Wed Sep 18 10:15:48 2024","interfaces":[{"interface":"back","average":{"1min":1.986,"5min":1.986,"15min":1.986},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":18.471,"5min":18.471,"15min":18.471},"last":1.1080000000000001},{"interface":"front","average":{"1min":20.716999999999999,"5min":20.716999999999999,"15min":20.716999999999999},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":176.56800000000001,"5min":176.56800000000001,"15min":176.56800000000001},"last":0.97799999999999998}]},{"osd":2,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":11.984,"5min":11.984,"15min":11.984},"min":{"1min":0.751,"5min":0.751,"15min":0.751},"max":{"1min":196.59200000000001,"5min":196.59200000000001,"15min":196.59200000000001},"last":1.0069999999999999},{"interface":"front","average":{"1min":26.109999999999999,"5min":26.109999999999999,"15min":26.109999999999999},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":177.03,"5min":177.03,"15min":177.03},"last":1.2130000000000001}]},{"osd":3,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":25.431999999999999,"5min":25.431999999999999,"15min":25.431999999999999},"min":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"max":{"1min":168.27699999999999,"5min":168.27699999999999,"15min":168.27699999999999},"last":1.268},{"interface":"front","average":{"1min":12.045999999999999,"5min":12.045999999999999,"15min":12.045999999999999},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":196.643,"5min":196.643,"15min":196.643},"last":0.88300000000000001}]},{"osd":4,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":29.802,"5min":29.802,"15min":29.802},"min":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"max":{"1min":168.48099999999999,"5min":168.48099999999999,"15min":168.48099999999999},"last":1.05},{"interface":"front","average":{"1min":8.1780000000000008,"5min":8.1780000000000008,"15min":8.1780000000000008},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":65.188000000000002,"5min":65.188000000000002,"15min":65.188000000000002},"last":1.1830000000000001}]},{"osd":5,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":26.152999999999999,"5min":26.152999999999999,"15min":26.152999999999999},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":176.48099999999999,"5min":176.48099999999999,"15min":176.48099999999999},"last":1.5089999999999999},{"interface":"front","average":{"1min":24.885999999999999,"5min":24.885999999999999,"15min":24.885999999999999},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":167.62299999999999,"5min":167.62299999999999,"15min":167.62299999999999},"last":1.407}]},{"osd":6,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":29.355,"5min":29.355,"15min":29.355},"min":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":176.81200000000001,"5min":176.81200000000001,"15min":176.81200000000001},"last":1.4179999999999999},{"interface":"front","average":{"1min":27.855,"5min":27.855,"15min":27.855},"min":{"1min":1.24,"5min":1.24,"15min":1.24},"max":{"1min":167.947,"5min":167.947,"15min":167.947},"last":1.3640000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4390000000000001}]},{"osd":9,"last update":"Wed Sep 18 10:15:48 2024","interfaces":[{"interface":"back","average":{"1min":21.055,"5min":21.055,"15min":21.055},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":176.73599999999999,"5min":176.73599999999999,"15min":176.73599999999999},"last":1.6279999999999999},{"interface":"front","average":{"1min":23.465,"5min":23.465,"15min":23.465},"min":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"max":{"1min":167.84899999999999,"5min":167.84899999999999,"15min":167.84899999999999},"last":1.4890000000000001}]},{"osd":10,"last update":"Wed Sep 18 10:15:39 2024","interfaces":[{"interface":"back","average":{"1min":22.678999999999998,"5min":22.678999999999998,"15min":22.678999999999998},"min":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"max":{"1min":168.07900000000001,"5min":168.07900000000001,"15min":168.07900000000001},"last":1.5329999999999999},{"interface":"front","average":{"1min":31.972000000000001,"5min":31.972000000000001,"15min":31.972000000000001},"min":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"max":{"1min":196.37700000000001,"5min":196.37700000000001,"15min":196.37700000000001},"last":1.778}]},{"osd":11,"last update":"Wed Sep 18 10:15:48 2024","interfaces":[{"interface":"back","average":{"1min":1.611,"5min":1.611,"15min":1.611},"min":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"max":{"1min":2.391,"5min":2.391,"15min":2.391},"last":1.853},{"interface":"front","average":{"1min":26.463999999999999,"5min":26.463999999999999,"15min":26.463999999999999},"min":{"1min":1.028,"5min":1.028,"15min":1.028},"max":{"1min":176.94999999999999,"5min":176.94999999999999,"15min":176.94999999999999},"last":1.726}]},{"osd":12,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":1.5820000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"min":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"max":{"1min":2.7879999999999998,"5min":2.7879999999999998,"15min":2.7879999999999998},"last":1.895},{"interface":"front","average":{"1min":21.914999999999999,"5min":21.914999999999999,"15min":21.914999999999999},"min":{"1min":1.296,"5min":1.296,"15min":1.296},"max":{"1min":168.34399999999999,"5min":168.34399999999999,"15min":168.34399999999999},"last":1.657}]},{"osd":14,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":6.694,"5min":6.694,"15min":6.694},"min":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"max":{"1min":87.427000000000007,"5min":87.427000000000007,"15min":87.427000000000007},"last":1.972},{"interface":"front","average":{"1min":26.231999999999999,"5min":26.231999999999999,"15min":26.231999999999999},"min":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"max":{"1min":176.87700000000001,"5min":176.87700000000001,"15min":176.87700000000001},"last":1.804}]},{"osd":15,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":20.334,"5min":20.334,"15min":20.334},"min":{"1min":1.266,"5min":1.266,"15min":1.266},"max":{"1min":168.40700000000001,"5min":168.40700000000001,"15min":168.40700000000001},"last":1.6080000000000001},{"interface":"front","average":{"1min":16.350999999999999,"5min":16.350999999999999,"15min":16.350999999999999},"min":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"max":{"1min":196.512,"5min":196.512,"15min":196.512},"last":1.9330000000000001}]}]},{"osd":0,"up_from":21,"seq":90194313236,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311068,"kb_used_data":9548,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94060772,"statfs":{"total":96636764160,"available":96318230528,"internally_reserved":0,"allocated":9777152,"data_stored":12516989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,4,5,6,9,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 18 10:15:34 2024","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.341,"5min":1.341,"15min":1.341},"last":1.1279999999999999},{"interface":"front","average":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.4630000000000001,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":0.89200000000000002}]},{"osd":2,"last update":"Wed Sep 18 10:15:34 2024","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":3.7269999999999999,"5min":3.7269999999999999,"15min":3.7269999999999999},"last":0.82899999999999996},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.528,"5min":1.528,"15min":1.528},"last":1.1950000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.196}]},{"osd":4,"last update":"Wed Sep 18 10:15:29 2024","interfaces":[{"interface":"back","average":{"1min":15.4,"5min":15.4,"15min":15.4},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":130.28,"5min":130.28,"15min":130.28},"last":0.94799999999999995},{"interface":"front","average":{"1min":16,"5min":16,"15min":16},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":79.914000000000001,"5min":79.914000000000001,"15min":79.914000000000001},"last":1.3029999999999999}]},{"osd":5,"last update":"Wed Sep 18 10:15:34 2024","interfaces":[{"interface":"back","average":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"min":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"max":{"1min":1.6819999999999999,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":1.3520000000000001},{"interface":"front","average":{"1min":9.7759999999999998,"5min":9.7759999999999998,"15min":9.7759999999999998},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":130.39599999999999,"5min":130.39599999999999,"15min":130.39599999999999},"last":1.242}]},{"osd":6,"last update":"Wed Sep 18 10:15:47 2024","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"min":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"max":{"1min":1.776,"5min":1.776,"15min":1.776},"last":1.5349999999999999},{"interface":"front","average":{"1min":2.2069999999999999,"5min":2.2069999999999999,"15min":2.2069999999999999},"min":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"max":{"1min":15.250999999999999,"5min":15.250999999999999,"15min":15.250999999999999},"last":1.4690000000000001}]},{"osd":9,"last update":"Wed Sep 18 10:15:29 2024","interfaces":[{"interface":"back","average":{"1min":1.407,"5min":1.407,"15min":1.407},"min":{"1min":0.879,"5min":0.879,"15min":0.879},"max":{"1min":1.855,"5min":1.855,"15min":1.855},"last":1.617},{"interface":"front","average":{"1min":10.532999999999999,"5min":10.532999999999999,"15min":10.532999999999999},"min":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"max":{"1min":130.441,"5min":130.441,"15min":130.441},"last":1.3919999999999999}]},{"osd":11,"last update":"Wed Sep 18 10:15:29 2024","interfaces":[{"interface":"back","average":{"1min":1.54,"5min":1.54,"15min":1.54},"min":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"max":{"1min":2.0209999999999999,"5min":2.0209999999999999,"15min":2.0209999999999999},"last":1.758},{"interface":"front","average":{"1min":5.327,"5min":5.327,"15min":5.327},"min":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":54.805999999999997,"5min":54.805999999999997,"15min":54.805999999999997},"last":1.8149999999999999}]},{"osd":12,"last update":"Wed Sep 18 10:15:29 2024","interfaces":[{"interface":"back","average":{"1min":10.949999999999999,"5min":10.949999999999999,"15min":10.949999999999999},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":130.34800000000001,"5min":130.34800000000001,"15min":130.34800000000001},"last":1.504},{"interface":"front","average":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.9139999999999999,"5min":1.9139999999999999,"15min":1.9139999999999999},"last":1.655}]},{"osd":14,"last update":"Wed Sep 18 10:15:34 2024","interfaces":[{"interface":"back","average":{"1min":10.039999999999999,"5min":10.039999999999999,"15min":10.039999999999999},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":130.48500000000001,"5min":130.48500000000001,"15min":130.48500000000001},"last":1.5920000000000001},{"interface":"front","average":{"1min":1.5600000000000001,"5min":1.5600000000000001,"15min":1.5600000000000001},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":2.1819999999999999,"5min":2.1819999999999999,"15min":2.1819999999999999},"last":1.8440000000000001}]},{"osd":15,"last update":"Wed Sep 18 10:15:34 2024","interfaces":[{"interface":"back","average":{"1min":3.0750000000000002,"5min":3.0750000000000002,"15min":3.0750000000000002},"min":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":23.806999999999999,"5min":23.806999999999999,"15min":23.806999999999999},"last":1.7250000000000001},{"interface":"front","average":{"1min":10.122,"5min":10.122,"15min":10.122},"min":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":130.52799999999999,"5min":130.52799999999999,"15min":130.52799999999999},"last":1.6879999999999999}]}]},{"osd":4,"up_from":21,"seq":90194313238,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":321444,"kb_used_data":19920,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94050396,"statfs":{"total":96636764160,"available":96307605504,"internally_reserved":0,"allocated":20398080,"data_stored":27592085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,5,6,7,8,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:15:29 2024","interfaces":[{"interface":"back","average":{"1min":11.151,"5min":11.151,"15min":11.151},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":89.266999999999996,"5min":89.266999999999996,"15min":89.266999999999996},"last":0.95099999999999996},{"interface":"front","average":{"1min":5.0049999999999999,"5min":5.0049999999999999,"15min":5.0049999999999999},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":46.898000000000003,"5min":46.898000000000003,"15min":46.898000000000003},"last":0.995}]},{"osd":1,"last update":"Wed Sep 18 10:15:34 2024","interfaces":[{"interface":"back","average":{"1min":14.653,"5min":14.653,"15min":14.653},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":164.21199999999999,"5min":164.21199999999999,"15min":164.21199999999999},"last":0.90900000000000003},{"interface":"front","average":{"1min":11.363,"5min":11.363,"15min":11.363},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":89.331000000000003,"5min":89.331000000000003,"15min":89.331000000000003},"last":1.288}]},{"osd":2,"last update":"Wed Sep 18 10:15:47 2024","interfaces":[{"interface":"back","average":{"1min":23.617000000000001,"5min":23.617000000000001,"15min":23.617000000000001},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":192.136,"5min":192.136,"15min":192.136},"last":58.140000000000001},{"interface":"front","average":{"1min":11.888999999999999,"5min":11.888999999999999,"15min":11.888999999999999},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":198.87700000000001,"5min":198.87700000000001,"15min":198.87700000000001},"last":1.429}]},{"osd":3,"last update":"Wed Sep 18 10:15:34 2024","interfaces":[{"interface":"back","average":{"1min":18.170999999999999,"5min":18.170999999999999,"15min":18.170999999999999},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":137.053,"5min":137.053,"15min":137.053},"last":99.194999999999993},{"interface":"front","average":{"1min":27.619,"5min":27.619,"15min":27.619},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":203.87899999999999,"5min":203.87899999999999,"15min":203.87899999999999},"last":1.236}]},{"osd":5,"last update":"Wed Sep 18 10:15:34 2024","interfaces":[{"interface":"back","average":{"1min":18.940000000000001,"5min":18.940000000000001,"15min":18.940000000000001},"min":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"max":{"1min":189.55600000000001,"5min":189.55600000000001,"15min":189.55600000000001},"last":1.5009999999999999},{"interface":"front","average":{"1min":3.2469999999999999,"5min":3.2469999999999999,"15min":3.2469999999999999},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":47.518000000000001,"5min":47.518000000000001,"15min":47.518000000000001},"last":1.5409999999999999}]},{"osd":6,"last update":"Wed Sep 18 10:15:34 2024","interfaces":[{"interface":"back","average":{"1min":32.25,"5min":32.25,"15min":32.25},"min":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":197.33199999999999,"5min":197.33199999999999,"15min":197.33199999999999},"last":1.6519999999999999},{"interface":"front","average":{"1min":18.963000000000001,"5min":18.963000000000001,"15min":18.963000000000001},"min":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"max":{"1min":179.19200000000001,"5min":179.19200000000001,"15min":179.19200000000001},"last":1.597}]},{"osd":7,"last update":"Wed Sep 18 10:15:34 2024","interfaces":[{"interface":"back","average":{"1min":27.597000000000001,"5min":27.597000000000001,"15min":27.597000000000001},"min":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"max":{"1min":290.036,"5min":290.036,"15min":290.036},"last":1.3779999999999999},{"interface":"front","average":{"1min":17.425999999999998,"5min":17.425999999999998,"15min":17.425999999999998},"min":{"1min":1.173,"5min":1.173,"15min":1.173},"max":{"1min":139.696,"5min":139.696,"15min":139.696},"last":1.833}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.784}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5680000000000001}]},{"osd":10,"last update":"Wed Sep 18 10:15:46 2024","interfaces":[{"interface":"back","average":{"1min":22.786000000000001,"5min":22.786000000000001,"15min":22.786000000000001},"min":{"1min":1.079,"5min":1.079,"15min":1.079},"max":{"1min":164.43899999999999,"5min":164.43899999999999,"15min":164.43899999999999},"last":1.619},{"interface":"front","average":{"1min":12.59,"5min":12.59,"15min":12.59},"min":{"1min":0.878,"5min":0.878,"15min":0.878},"max":{"1min":89.442999999999998,"5min":89.442999999999998,"15min":89.442999999999998},"last":59.408999999999999}]},{"osd":11,"last update":"Wed Sep 18 10:15:29 2024","interfaces":[{"interface":"back","average":{"1min":10.817,"5min":10.817,"15min":10.817},"min":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"max":{"1min":101.666,"5min":101.666,"15min":101.666},"last":1.7010000000000001},{"interface":"front","average":{"1min":24.919,"5min":24.919,"15min":24.919},"min":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"max":{"1min":164.733,"5min":164.733,"15min":164.733},"last":1.736}]},{"osd":12,"last update":"Wed Sep 18 10:15:29 2024","interfaces":[{"interface":"back","average":{"1min":11.863,"5min":11.863,"15min":11.863},"min":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":89.400000000000006,"5min":89.400000000000006,"15min":89.400000000000006},"last":1.8779999999999999},{"interface":"front","average":{"1min":8.1699999999999999,"5min":8.1699999999999999,"15min":8.1699999999999999},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":101.699,"5min":101.699,"15min":101.699},"last":92.698999999999998}]},{"osd":13,"last update":"Wed Sep 18 10:15:47 2024","interfaces":[{"interface":"back","average":{"1min":25.314,"5min":25.314,"15min":25.314},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":199.37799999999999,"5min":199.37799999999999,"15min":199.37799999999999},"last":1.76},{"interface":"front","average":{"1min":23.385000000000002,"5min":23.385000000000002,"15min":23.385000000000002},"min":{"1min":1.022,"5min":1.022,"15min":1.022},"max":{"1min":164.53,"5min":164.53,"15min":164.53},"last":1.6830000000000001}]},{"osd":14,"last update":"Wed Sep 18 10:15:34 2024","interfaces":[{"interface":"back","average":{"1min":21.658000000000001,"5min":21.658000000000001,"15min":21.658000000000001},"min":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"max":{"1min":202.191,"5min":202.191,"15min":202.191},"last":1.8089999999999999},{"interface":"front","average":{"1min":19.306999999999999,"5min":19.306999999999999,"15min":19.306999999999999},"min":{"1min":1.163,"5min":1.163,"15min":1.163},"max":{"1min":164.35400000000001,"5min":164.35400000000001,"15min":164.35400000000001},"last":101.492}]}]},{"osd":9,"up_from":21,"seq":90194313236,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":314160,"kb_used_data":12640,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94057680,"statfs":{"total":96636764160,"available":96315064320,"internally_reserved":0,"allocated":12943360,"data_stored":15711869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,4,5,6,8,10,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":7.2450000000000001,"5min":7.2450000000000001,"15min":7.2450000000000001},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":127.902,"5min":127.902,"15min":127.902},"last":0.97399999999999998},{"interface":"front","average":{"1min":8.3109999999999999,"5min":8.3109999999999999,"15min":8.3109999999999999},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":96.203000000000003,"5min":96.203000000000003,"15min":96.203000000000003},"last":0.81200000000000006}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77000000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.048}]},{"osd":3,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":6.226,"5min":6.226,"15min":6.226},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":98.888999999999996,"5min":98.888999999999996,"15min":98.888999999999996},"last":0.73399999999999999},{"interface":"front","average":{"1min":6.2320000000000002,"5min":6.2320000000000002,"15min":6.2320000000000002},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":98.947999999999993,"5min":98.947999999999993,"15min":98.947999999999993},"last":0.85599999999999998}]},{"osd":4,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"min":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"max":{"1min":1.5780000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.82999999999999996},{"interface":"front","average":{"1min":9.5090000000000003,"5min":9.5090000000000003,"15min":9.5090000000000003},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":71.366,"5min":71.366,"15min":71.366},"last":1.238}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.01}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.355}]},{"osd":8,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":4.5910000000000002,"5min":4.5910000000000002,"15min":4.5910000000000002},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":64.912999999999997,"5min":64.912999999999997,"15min":64.912999999999997},"last":1.2629999999999999},{"interface":"front","average":{"1min":6.2309999999999999,"5min":6.2309999999999999,"15min":6.2309999999999999},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":57.664000000000001,"5min":57.664000000000001,"15min":57.664000000000001},"last":1.417}]},{"osd":10,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":1.302,"5min":1.302,"15min":1.302},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":2.109,"5min":2.109,"15min":2.109},"last":1.391},{"interface":"front","average":{"1min":3.7799999999999998,"5min":3.7799999999999998,"15min":3.7799999999999998},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":54.167999999999999,"5min":54.167999999999999,"15min":54.167999999999999},"last":1.444}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.669}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.446}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":96}]},{"osd":14,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":10.006,"5min":10.006,"15min":10.006},"min":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":62.899000000000001,"5min":62.899000000000001,"15min":62.899000000000001},"last":1.4930000000000001},{"interface":"front","average":{"1min":15.250999999999999,"5min":15.250999999999999,"15min":15.250999999999999},"min":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"max":{"1min":155.61000000000001,"5min":155.61000000000001,"15min":155.61000000000001},"last":1.47}]},{"osd":15,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":1.5,"5min":1.5,"15min":1.5},"min":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"max":{"1min":1.9970000000000001,"5min":1.9970000000000001,"15min":1.9970000000000001},"last":1.538},{"interface":"front","average":{"1min":4.077,"5min":4.077,"15min":4.077},"min":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":54.335999999999999,"5min":54.335999999999999,"15min":54.335999999999999},"last":60.289000000000001}]}]},{"osd":11,"up_from":21,"seq":90194313236,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":321132,"kb_used_data":19612,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94050708,"statfs":{"total":96636764160,"available":96307924992,"internally_reserved":0,"allocated":20082688,"data_stored":24903293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,4,5,6,7,8,9,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0049999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88}]},{"osd":2,"last update":"Wed Sep 18 10:15:33 2024","interfaces":[{"interface":"back","average":{"1min":10.189,"5min":10.189,"15min":10.189},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":99.718000000000004,"5min":99.718000000000004,"15min":99.718000000000004},"last":0.97699999999999998},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":1.2150000000000001}]},{"osd":3,"last update":"Wed Sep 18 10:15:46 2024","interfaces":[{"interface":"back","average":{"1min":5.0720000000000001,"5min":5.0720000000000001,"15min":5.0720000000000001},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":85.963999999999999,"5min":85.963999999999999,"15min":85.963999999999999},"last":1.0209999999999999},{"interface":"front","average":{"1min":22.759,"5min":22.759,"15min":22.759},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":125.672,"5min":125.672,"15min":125.672},"last":1.05}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.343}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":97.091999999999999}]},{"osd":6,"last update":"Wed Sep 18 10:15:46 2024","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.157,"15min":1.157},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":5.04,"5min":5.04,"15min":5.04},"last":1.593},{"interface":"front","average":{"1min":6.9880000000000004,"5min":6.9880000000000004,"15min":6.9880000000000004},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":94.162000000000006,"5min":94.162000000000006,"15min":94.162000000000006},"last":1.4079999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4399999999999999}]},{"osd":8,"last update":"Wed Sep 18 10:15:33 2024","interfaces":[{"interface":"back","average":{"1min":6.04,"5min":6.04,"15min":6.04},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":85.900000000000006,"5min":85.900000000000006,"15min":85.900000000000006},"last":1.7470000000000001},{"interface":"front","average":{"1min":9.9849999999999994,"5min":9.9849999999999994,"15min":9.9849999999999994},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":99.765000000000001,"5min":99.765000000000001,"15min":99.765000000000001},"last":1.472}]},{"osd":9,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":5.8239999999999998,"5min":5.8239999999999998,"15min":5.8239999999999998},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":85.838999999999999,"5min":85.838999999999999,"15min":85.838999999999999},"last":1.643},{"interface":"front","average":{"1min":8.1709999999999994,"5min":8.1709999999999994,"15min":8.1709999999999994},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":99.810000000000002,"5min":99.810000000000002,"15min":99.810000000000002},"last":1.6200000000000001}]},{"osd":10,"last update":"Wed Sep 18 10:15:33 2024","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.097,"15min":1.097},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.371,"5min":1.371,"15min":1.371},"last":1.6879999999999999},{"interface":"front","average":{"1min":7.6859999999999999,"5min":7.6859999999999999,"15min":7.6859999999999999},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":89.290000000000006,"5min":89.290000000000006,"15min":89.290000000000006},"last":1.885}]},{"osd":12,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":1.125,"15min":1.125},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.6259999999999999,"5min":1.6259999999999999,"15min":1.6259999999999999},"last":1.7290000000000001},{"interface":"front","average":{"1min":10.385999999999999,"5min":10.385999999999999,"15min":10.385999999999999},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":86.025999999999996,"5min":86.025999999999996,"15min":86.025999999999996},"last":1.823}]},{"osd":13,"last update":"Wed Sep 18 10:15:33 2024","interfaces":[{"interface":"back","average":{"1min":30.405000000000001,"5min":30.405000000000001,"15min":30.405000000000001},"min":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"max":{"1min":189.25899999999999,"5min":189.25899999999999,"15min":189.25899999999999},"last":1.669},{"interface":"front","average":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":2.1720000000000002,"5min":2.1720000000000002,"15min":2.1720000000000002},"last":1.8080000000000001}]},{"osd":14,"last update":"Wed Sep 18 10:15:33 2024","interfaces":[{"interface":"back","average":{"1min":6.423,"5min":6.423,"15min":6.423},"min":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":86.109999999999999,"5min":86.109999999999999,"15min":86.109999999999999},"last":1.948},{"interface":"front","average":{"1min":8.9819999999999993,"5min":8.9819999999999993,"15min":8.9819999999999993},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":102.625,"5min":102.625,"15min":102.625},"last":1.7150000000000001}]},{"osd":15,"last update":"Wed Sep 18 10:15:33 2024","interfaces":[{"interface":"back","average":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":2.0249999999999999,"5min":2.0249999999999999,"15min":2.0249999999999999},"last":1.7689999999999999},{"interface":"front","average":{"1min":6.484,"5min":6.484,"15min":6.484},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":86.165999999999997,"5min":86.165999999999997,"15min":86.165999999999997},"last":2.0099999999999998}]}]},{"osd":12,"up_from":21,"seq":90194313237,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311636,"kb_used_data":14276,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94060204,"statfs":{"total":96636764160,"available":96317648896,"internally_reserved":0,"allocated":14618624,"data_stored":19623568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[2,3,4,6,7,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":2.3839999999999999,"5min":2.3839999999999999,"15min":2.3839999999999999},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":32.476999999999997,"5min":32.476999999999997,"15min":32.476999999999997},"last":0.64200000000000002},{"interface":"front","average":{"1min":2.625,"5min":2.625,"15min":2.625},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":32.509,"5min":32.509,"15min":32.509},"last":0.89300000000000002}]},{"osd":1,"last update":"Wed Sep 18 10:15:32 2024","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.6399999999999999,"5min":1.6399999999999999,"15min":1.6399999999999999},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.905,"5min":1.905,"15min":1.905},"last":0.81899999999999995}]},{"osd":2,"last update":"Wed Sep 18 10:15:32 2024","interfaces":[{"interface":"back","average":{"1min":1.002,"5min":1.002,"15min":1.002},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.96299999999999997},{"interface":"front","average":{"1min":4.1189999999999998,"5min":4.1189999999999998,"15min":4.1189999999999998},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":62.780999999999999,"5min":62.780999999999999,"15min":62.780999999999999},"last":1.367}]},{"osd":3,"last update":"Wed Sep 18 10:15:32 2024","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":1.145,"15min":1.145},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.772,"5min":1.772,"15min":1.772},"last":0.93799999999999994},{"interface":"front","average":{"1min":1.151,"5min":1.151,"15min":1.151},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.4710000000000001,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":1.1040000000000001}]},{"osd":4,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":1.052,"15min":1.052},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.6759999999999999,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":0.80000000000000004},{"interface":"front","average":{"1min":1.222,"5min":1.222,"15min":1.222},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.774,"5min":1.774,"15min":1.774},"last":0.874}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1850000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0449999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":94.382000000000005}]},{"osd":9,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":1.268,"5min":1.268,"15min":1.268},"min":{"1min":0.875,"5min":0.875,"15min":0.875},"max":{"1min":1.9079999999999999,"5min":1.9079999999999999,"15min":1.9079999999999999},"last":1.2569999999999999},{"interface":"front","average":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.8460000000000001,"5min":1.8460000000000001,"15min":1.8460000000000001},"last":1.4379999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.077}]},{"osd":11,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":1.365,"5min":1.365,"15min":1.365},"min":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":2.0539999999999998,"5min":2.0539999999999998,"15min":2.0539999999999998},"last":1.387},{"interface":"front","average":{"1min":1.383,"5min":1.383,"15min":1.383},"min":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"max":{"1min":1.98,"5min":1.98,"15min":1.98},"last":1.6879999999999999}]},{"osd":13,"last update":"Wed Sep 18 10:15:32 2024","interfaces":[{"interface":"back","average":{"1min":1.333,"5min":1.333,"15min":1.333},"min":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":2.016,"5min":2.016,"15min":2.016},"last":1.603},{"interface":"front","average":{"1min":1.429,"5min":1.429,"15min":1.429},"min":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"max":{"1min":1.9379999999999999,"5min":1.9379999999999999,"15min":1.9379999999999999},"last":1.5089999999999999}]},{"osd":14,"last update":"Wed Sep 18 10:15:32 2024","interfaces":[{"interface":"back","average":{"1min":1.4299999999999999,"5min":1.4299999999999999,"15min":1.4299999999999999},"min":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"max":{"1min":2.0619999999999998,"5min":2.0619999999999998,"15min":2.0619999999999998},"last":1.401},{"interface":"front","average":{"1min":1.391,"5min":1.391,"15min":1.391},"min":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":2.1339999999999999,"5min":2.1339999999999999,"15min":2.1339999999999999},"last":1.6619999999999999}]},{"osd":15,"last update":"Wed Sep 18 10:15:32 2024","interfaces":[{"interface":"back","average":{"1min":1.494,"5min":1.494,"15min":1.494},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":2.1179999999999999,"5min":2.1179999999999999,"15min":2.1179999999999999},"last":1.345},{"interface":"front","average":{"1min":1.48,"5min":1.48,"15min":1.48},"min":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"max":{"1min":2.1600000000000001,"5min":2.1600000000000001,"15min":2.1600000000000001},"last":1.5429999999999999}]}]},{"osd":1,"up_from":22,"seq":94489280532,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":314904,"kb_used_data":13380,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94056936,"statfs":{"total":96636764160,"available":96314302464,"internally_reserved":0,"allocated":13701120,"data_stored":15467925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,4,5,6,9,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:15:35 2024","interfaces":[{"interface":"back","average":{"1min":2.8700000000000001,"5min":2.8700000000000001,"15min":2.8700000000000001},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":36.268000000000001,"5min":36.268000000000001,"15min":36.268000000000001},"last":0.65800000000000003},{"interface":"front","average":{"1min":17.715,"5min":17.715,"15min":17.715},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":154.67599999999999,"5min":154.67599999999999,"15min":154.67599999999999},"last":66.534999999999997}]},{"osd":2,"last update":"Wed Sep 18 10:15:35 2024","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.343,"5min":1.343,"15min":1.343},"last":0.71499999999999997},{"interface":"front","average":{"1min":2.9449999999999998,"5min":2.9449999999999998,"15min":2.9449999999999998},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":36.323999999999998,"5min":36.323999999999998,"15min":36.323999999999998},"last":0.81399999999999995}]},{"osd":3,"last update":"Wed Sep 18 10:15:35 2024","interfaces":[{"interface":"back","average":{"1min":12.128,"5min":12.128,"15min":12.128},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":150.642,"5min":150.642,"15min":150.642},"last":66.344999999999999},{"interface":"front","average":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.6930000000000001,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":0.99299999999999999}]},{"osd":4,"last update":"Wed Sep 18 10:15:35 2024","interfaces":[{"interface":"back","average":{"1min":3.3439999999999999,"5min":3.3439999999999999,"15min":3.3439999999999999},"min":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":40.783000000000001,"5min":40.783000000000001,"15min":40.783000000000001},"last":66.442999999999998},{"interface":"front","average":{"1min":4.1699999999999999,"5min":4.1699999999999999,"15min":4.1699999999999999},"min":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":50.747,"5min":50.747,"15min":50.747},"last":96.209999999999994}]},{"osd":5,"last update":"Wed Sep 18 10:15:35 2024","interfaces":[{"interface":"back","average":{"1min":5.9720000000000004,"5min":5.9720000000000004,"15min":5.9720000000000004},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":52.317999999999998,"5min":52.317999999999998,"15min":52.317999999999998},"last":110.154},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.748,"5min":1.748,"15min":1.748},"last":66.611000000000004}]},{"osd":6,"last update":"Wed Sep 18 10:15:35 2024","interfaces":[{"interface":"back","average":{"1min":8.3490000000000002,"5min":8.3490000000000002,"15min":8.3490000000000002},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":84.939999999999998,"5min":84.939999999999998,"15min":84.939999999999998},"last":1.1990000000000001},{"interface":"front","average":{"1min":10.352,"5min":10.352,"15min":10.352},"min":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"max":{"1min":85.045000000000002,"5min":85.045000000000002,"15min":85.045000000000002},"last":0.97199999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":118.43600000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7789999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89300000000000002}]},{"osd":10,"last update":"Wed Sep 18 10:15:35 2024","interfaces":[{"interface":"back","average":{"1min":3.4300000000000002,"5min":3.4300000000000002,"15min":3.4300000000000002},"min":{"1min":1.073,"5min":1.073,"15min":1.073},"max":{"1min":36.436,"5min":36.436,"15min":36.436},"last":1.0449999999999999},{"interface":"front","average":{"1min":11.914999999999999,"5min":11.914999999999999,"15min":11.914999999999999},"min":{"1min":0.88,"5min":0.88,"15min":0.88},"max":{"1min":100.402,"5min":100.402,"15min":100.402},"last":66.673000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.825}]},{"osd":12,"last update":"Wed Sep 18 10:15:35 2024","interfaces":[{"interface":"back","average":{"1min":4.2130000000000001,"5min":4.2130000000000001,"15min":4.2130000000000001},"min":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"max":{"1min":49.976999999999997,"5min":49.976999999999997,"15min":49.976999999999997},"last":1.101},{"interface":"front","average":{"1min":8.4000000000000004,"5min":8.4000000000000004,"15min":8.4000000000000004},"min":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"max":{"1min":124.78100000000001,"5min":124.78100000000001,"15min":124.78100000000001},"last":8.4049999999999994}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":66.867000000000004}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":66.825999999999993}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.165}]}]},{"osd":2,"up_from":22,"seq":94489280532,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":319716,"kb_used_data":18192,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94052124,"statfs":{"total":96636764160,"available":96309374976,"internally_reserved":0,"allocated":18628608,"data_stored":24282517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,3,4,5,6,7,8,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 18 10:15:31 2024","interfaces":[{"interface":"back","average":{"1min":26.495000000000001,"5min":26.495000000000001,"15min":26.495000000000001},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":419.05700000000002,"5min":419.05700000000002,"15min":419.05700000000002},"last":0.499},{"interface":"front","average":{"1min":4.9059999999999997,"5min":4.9059999999999997,"15min":4.9059999999999997},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":98.254000000000005,"5min":98.254000000000005,"15min":98.254000000000005},"last":0.63300000000000001}]},{"osd":3,"last update":"Wed Sep 18 10:15:31 2024","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.68000000000000005}]},{"osd":4,"last update":"Wed Sep 18 10:15:31 2024","interfaces":[{"interface":"back","average":{"1min":1.925,"5min":1.925,"15min":1.925},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":24.888999999999999,"5min":24.888999999999999,"15min":24.888999999999999},"last":0.55600000000000005},{"interface":"front","average":{"1min":4.5389999999999997,"5min":4.5389999999999997,"15min":4.5389999999999997},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":84.647000000000006,"5min":84.647000000000006,"15min":84.647000000000006},"last":0.46700000000000003}]},{"osd":5,"last update":"Wed Sep 18 10:15:31 2024","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.478,"5min":1.478,"15min":1.478},"last":0.90600000000000003},{"interface":"front","average":{"1min":1.054,"5min":1.054,"15min":1.054},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.415,"5min":1.415,"15min":1.415},"last":0.80000000000000004}]},{"osd":6,"last update":"Wed Sep 18 10:15:31 2024","interfaces":[{"interface":"back","average":{"1min":4.758,"5min":4.758,"15min":4.758},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":71.451999999999998,"5min":71.451999999999998,"15min":71.451999999999998},"last":0.65300000000000002},{"interface":"front","average":{"1min":4.8730000000000002,"5min":4.8730000000000002,"15min":4.8730000000000002},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":90.188999999999993,"5min":90.188999999999993,"15min":90.188999999999993},"last":0.753}]},{"osd":7,"last update":"Wed Sep 18 10:15:31 2024","interfaces":[{"interface":"back","average":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":1.2050000000000001},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.724,"5min":1.724,"15min":1.724},"last":88.534000000000006},{"interface":"front","average":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":3.016,"5min":3.016,"15min":3.016},"last":88.484999999999999}]},{"osd":8,"last update":"Wed Sep 18 10:15:31 2024","interfaces":[{"interface":"back","average":{"1min":5.1020000000000003,"5min":5.1020000000000003,"15min":5.1020000000000003},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":92.578999999999994,"5min":92.578999999999994,"15min":92.578999999999994},"last":0.82899999999999996},{"interface":"front","average":{"1min":2.7639999999999998,"5min":2.7639999999999998,"15min":2.7639999999999998},"min":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"max":{"1min":35.271000000000001,"5min":35.271000000000001,"15min":35.271000000000001},"last":0.70699999999999996}]},{"osd":12,"last update":"Wed Sep 18 10:15:31 2024","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"min":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"max":{"1min":1.696,"5min":1.696,"15min":1.696},"last":0.872},{"interface":"front","average":{"1min":2.056,"5min":2.056,"15min":2.056},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":16.878,"5min":16.878,"15min":16.878},"last":0.73199999999999998}]},{"osd":13,"last update":"Wed Sep 18 10:15:31 2024","interfaces":[{"interface":"back","average":{"1min":7.718,"5min":7.718,"15min":7.718},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":141.84200000000001,"5min":141.84200000000001,"15min":141.84200000000001},"last":80.450000000000003},{"interface":"front","average":{"1min":5.4589999999999996,"5min":5.4589999999999996,"15min":5.4589999999999996},"min":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":97.441999999999993,"5min":97.441999999999993,"15min":97.441999999999993},"last":0.94099999999999995}]},{"osd":15,"last update":"Wed Sep 18 10:15:31 2024","interfaces":[{"interface":"back","average":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":3.1480000000000001,"5min":3.1480000000000001,"15min":3.1480000000000001},"last":82.069999999999993},{"interface":"front","average":{"1min":3.911,"5min":3.911,"15min":3.911},"min":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":61.694000000000003,"5min":61.694000000000003,"15min":61.694000000000003},"last":0.96999999999999997}]}]},{"osd":3,"up_from":22,"seq":94489280532,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":318232,"kb_used_data":16712,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94053608,"statfs":{"total":96636764160,"available":96310894592,"internally_reserved":0,"allocated":17113088,"data_stored":23830141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,4,5,6,7,9,10,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:15:32 2024","interfaces":[{"interface":"back","average":{"1min":1.5169999999999999,"5min":1.5169999999999999,"15min":1.5169999999999999},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":13.680999999999999,"5min":13.680999999999999,"15min":13.680999999999999},"last":0.89600000000000002},{"interface":"front","average":{"1min":10.788,"5min":10.788,"15min":10.788},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":71.316999999999993,"5min":71.316999999999993,"15min":71.316999999999993},"last":1.752}]},{"osd":2,"last update":"Wed Sep 18 10:15:32 2024","interfaces":[{"interface":"back","average":{"1min":5.3029999999999999,"5min":5.3029999999999999,"15min":5.3029999999999999},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":56.523000000000003,"5min":56.523000000000003,"15min":56.523000000000003},"last":1.089},{"interface":"front","average":{"1min":1.109,"5min":1.109,"15min":1.109},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":3.327,"5min":3.327,"15min":3.327},"last":1.5640000000000001}]},{"osd":4,"last update":"Wed Sep 18 10:15:32 2024","interfaces":[{"interface":"back","average":{"1min":6.8739999999999997,"5min":6.8739999999999997,"15min":6.8739999999999997},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":71.135000000000005,"5min":71.135000000000005,"15min":71.135000000000005},"last":1.6399999999999999},{"interface":"front","average":{"1min":4.3579999999999997,"5min":4.3579999999999997,"15min":4.3579999999999997},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":56.253999999999998,"5min":56.253999999999998,"15min":56.253999999999998},"last":1.7849999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.044}]},{"osd":6,"last update":"Wed Sep 18 10:15:32 2024","interfaces":[{"interface":"back","average":{"1min":1.131,"5min":1.131,"15min":1.131},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.744,"5min":1.744,"15min":1.744},"last":1.728},{"interface":"front","average":{"1min":8.7520000000000007,"5min":8.7520000000000007,"15min":8.7520000000000007},"min":{"1min":0.749,"5min":0.749,"15min":0.749},"max":{"1min":81.210999999999999,"5min":81.210999999999999,"15min":81.210999999999999},"last":1.337}]},{"osd":7,"last update":"Wed Sep 18 10:15:32 2024","interfaces":[{"interface":"back","average":{"1min":16.132000000000001,"5min":16.132000000000001,"15min":16.132000000000001},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":129.68299999999999,"5min":129.68299999999999,"15min":129.68299999999999},"last":1.698},{"interface":"front","average":{"1min":13.532999999999999,"5min":13.532999999999999,"15min":13.532999999999999},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":129.726,"5min":129.726,"15min":129.726},"last":1.845}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0620000000000001}]},{"osd":9,"last update":"Wed Sep 18 10:15:32 2024","interfaces":[{"interface":"back","average":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.9990000000000001,"5min":1.9990000000000001,"15min":1.9990000000000001},"last":2.0209999999999999},{"interface":"front","average":{"1min":6.109,"5min":6.109,"15min":6.109},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":81.171000000000006,"5min":81.171000000000006,"15min":81.171000000000006},"last":94.546999999999997}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0659999999999998}]},{"osd":13,"last update":"Wed Sep 18 10:15:32 2024","interfaces":[{"interface":"back","average":{"1min":1.5189999999999999,"5min":1.5189999999999999,"15min":1.5189999999999999},"min":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":2.6859999999999999,"5min":2.6859999999999999,"15min":2.6859999999999999},"last":1.9279999999999999},{"interface":"front","average":{"1min":6.282,"5min":6.282,"15min":6.282},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":71.447000000000003,"5min":71.447000000000003,"15min":71.447000000000003},"last":1.885}]},{"osd":14,"last update":"Wed Sep 18 10:15:32 2024","interfaces":[{"interface":"back","average":{"1min":5.6900000000000004,"5min":5.6900000000000004,"15min":5.6900000000000004},"min":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"max":{"1min":56.664999999999999,"5min":56.664999999999999,"15min":56.664999999999999},"last":1.863},{"interface":"front","average":{"1min":2.363,"5min":2.363,"15min":2.363},"min":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"max":{"1min":18.523,"5min":18.523,"15min":18.523},"last":1.9770000000000001}]},{"osd":15,"last update":"Wed Sep 18 10:15:45 2024","interfaces":[{"interface":"back","average":{"1min":4.5830000000000002,"5min":4.5830000000000002,"15min":4.5830000000000002},"min":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"max":{"1min":56.357999999999997,"5min":56.357999999999997,"15min":56.357999999999997},"last":1.907},{"interface":"front","average":{"1min":10.225,"5min":10.225,"15min":10.225},"min":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":122.727,"5min":122.727,"15min":122.727},"last":2.1110000000000002}]}]},{"osd":5,"up_from":22,"seq":94489280532,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312836,"kb_used_data":11316,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94059004,"statfs":{"total":96636764160,"available":96316420096,"internally_reserved":0,"allocated":11587584,"data_stored":17354365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,4,6,7,8,9,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:15:47 2024","interfaces":[{"interface":"back","average":{"1min":7.8860000000000001,"5min":7.8860000000000001,"15min":7.8860000000000001},"min":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":84.430999999999997,"5min":84.430999999999997,"15min":84.430999999999997},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":41.832999999999998}]},{"osd":2,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":9.8339999999999996,"5min":9.8339999999999996,"15min":9.8339999999999996},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":84.566000000000003,"5min":84.566000000000003,"15min":84.566000000000003},"last":0.90600000000000003},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":9.5069999999999997,"5min":9.5069999999999997,"15min":9.5069999999999997},"last":0.93700000000000006}]},{"osd":3,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":8.5210000000000008,"5min":8.5210000000000008,"15min":8.5210000000000008},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":84.358999999999995,"5min":84.358999999999995,"15min":84.358999999999995},"last":0.83499999999999996},{"interface":"front","average":{"1min":8.2230000000000008,"5min":8.2230000000000008,"15min":8.2230000000000008},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":97.549999999999997,"5min":97.549999999999997,"15min":97.549999999999997},"last":41.780000000000001}]},{"osd":4,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":31.829999999999998,"5min":31.829999999999998,"15min":31.829999999999998},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":266.911,"5min":266.911,"15min":266.911},"last":1.0309999999999999},{"interface":"front","average":{"1min":22.798999999999999,"5min":22.798999999999999,"15min":22.798999999999999},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":260.53500000000003,"5min":260.53500000000003,"15min":260.53500000000003},"last":0.76200000000000001}]},{"osd":6,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":9.1940000000000008,"5min":9.1940000000000008,"15min":9.1940000000000008},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":58.171999999999997,"5min":58.171999999999997,"15min":58.171999999999997},"last":41.886000000000003},{"interface":"front","average":{"1min":21.391999999999999,"5min":21.391999999999999,"15min":21.391999999999999},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":227.48099999999999,"5min":227.48099999999999,"15min":227.48099999999999},"last":0.86199999999999999}]},{"osd":7,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":12.013,"5min":12.013,"15min":12.013},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":123.111,"5min":123.111,"15min":123.111},"last":1.2769999999999999},{"interface":"front","average":{"1min":13.994,"5min":13.994,"15min":13.994},"min":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"max":{"1min":84.617000000000004,"5min":84.617000000000004,"15min":84.617000000000004},"last":1.1259999999999999}]},{"osd":8,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":20.352,"5min":20.352,"15min":20.352},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":260.44200000000001,"5min":260.44200000000001,"15min":260.44200000000001},"last":0.98599999999999999},{"interface":"front","average":{"1min":7.0430000000000001,"5min":7.0430000000000001,"15min":7.0430000000000001},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":106.818,"5min":106.818,"15min":106.818},"last":41.975000000000001}]},{"osd":9,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":5.6470000000000002,"5min":5.6470000000000002,"15min":5.6470000000000002},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":80.272000000000006,"5min":80.272000000000006,"15min":80.272000000000006},"last":41.652000000000001},{"interface":"front","average":{"1min":1.8600000000000001,"5min":1.8600000000000001,"15min":1.8600000000000001},"min":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"max":{"1min":9.5690000000000008,"5min":9.5690000000000008,"15min":9.5690000000000008},"last":0.70499999999999996}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6479999999999999}]},{"osd":11,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":9.3919999999999995,"5min":9.3919999999999995,"15min":9.3919999999999995},"min":{"1min":0.997,"5min":0.997,"15min":0.997},"max":{"1min":123.158,"5min":123.158,"15min":123.158},"last":1.1000000000000001},{"interface":"front","average":{"1min":11.367000000000001,"5min":11.367000000000001,"15min":11.367000000000001},"min":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"max":{"1min":84.522999999999996,"5min":84.522999999999996,"15min":84.522999999999996},"last":1.056}]},{"osd":12,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":1.7789999999999999,"5min":1.7789999999999999,"15min":1.7789999999999999},"last":41.929000000000002},{"interface":"front","average":{"1min":1.9750000000000001,"5min":1.9750000000000001,"15min":1.9750000000000001},"min":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"max":{"1min":9.7710000000000008,"5min":9.7710000000000008,"15min":9.7710000000000008},"last":1.232}]},{"osd":13,"last update":"Wed Sep 18 10:15:47 2024","interfaces":[{"interface":"back","average":{"1min":24.271000000000001,"5min":24.271000000000001,"15min":24.271000000000001},"min":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"max":{"1min":140.273,"5min":140.273,"15min":140.273},"last":1.171},{"interface":"front","average":{"1min":24.221,"5min":24.221,"15min":24.221},"min":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"max":{"1min":170.86000000000001,"5min":170.86000000000001,"15min":170.86000000000001},"last":1.194}]},{"osd":14,"last update":"Wed Sep 18 10:15:47 2024","interfaces":[{"interface":"back","average":{"1min":13.257,"5min":13.257,"15min":13.257},"min":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"max":{"1min":101.733,"5min":101.733,"15min":101.733},"last":42.021000000000001},{"interface":"front","average":{"1min":9.5190000000000001,"5min":9.5190000000000001,"15min":9.5190000000000001},"min":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"max":{"1min":122.952,"5min":122.952,"15min":122.952},"last":1.321}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8839999999999999}]}]},{"osd":6,"up_from":22,"seq":94489280532,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":318692,"kb_used_data":17176,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94053148,"statfs":{"total":96636764160,"available":96310423552,"internally_reserved":0,"allocated":17588224,"data_stored":22968245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,3,4,5,7,8,9,11,12,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:15:47 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":67.674000000000007},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.79600000000000004}]},{"osd":1,"last update":"Wed Sep 18 10:15:47 2024","interfaces":[{"interface":"back","average":{"1min":4.5549999999999997,"5min":4.5549999999999997,"15min":4.5549999999999997},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":52.332999999999998,"5min":52.332999999999998,"15min":52.332999999999998},"last":0.91900000000000004},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":67.763000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.343}]},{"osd":3,"last update":"Wed Sep 18 10:15:47 2024","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.492,"5min":1.492,"15min":1.492},"last":0.88},{"interface":"front","average":{"1min":14.007,"5min":14.007,"15min":14.007},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":169.244,"5min":169.244,"15min":169.244},"last":1.0249999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75700000000000001}]},{"osd":5,"last update":"Wed Sep 18 10:15:32 2024","interfaces":[{"interface":"back","average":{"1min":11.228999999999999,"5min":11.228999999999999,"15min":11.228999999999999},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":119.032,"5min":119.032,"15min":119.032},"last":68.025999999999996},{"interface":"front","average":{"1min":3.629,"5min":3.629,"15min":3.629},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":54.731000000000002,"5min":54.731000000000002,"15min":54.731000000000002},"last":1.0569999999999999}]},{"osd":7,"last update":"Wed Sep 18 10:15:32 2024","interfaces":[{"interface":"back","average":{"1min":6.7569999999999997,"5min":6.7569999999999997,"15min":6.7569999999999997},"min":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":52.204999999999998,"5min":52.204999999999998,"15min":52.204999999999998},"last":69.171000000000006},{"interface":"front","average":{"1min":3.0550000000000002,"5min":3.0550000000000002,"15min":3.0550000000000002},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":40.848999999999997,"5min":40.848999999999997,"15min":40.848999999999997},"last":69.239000000000004}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1}]},{"osd":9,"last update":"Wed Sep 18 10:15:32 2024","interfaces":[{"interface":"back","average":{"1min":8.2799999999999994,"5min":8.2799999999999994,"15min":8.2799999999999994},"min":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"max":{"1min":95.691999999999993,"5min":95.691999999999993,"15min":95.691999999999993},"last":1.1000000000000001},{"interface":"front","average":{"1min":12.766999999999999,"5min":12.766999999999999,"15min":12.766999999999999},"min":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"max":{"1min":165.31100000000001,"5min":165.31100000000001,"15min":165.31100000000001},"last":1.1719999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.329}]},{"osd":11,"last update":"Wed Sep 18 10:15:32 2024","interfaces":[{"interface":"back","average":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.7749999999999999,"5min":1.7749999999999999,"15min":1.7749999999999999},"last":67.909999999999997},{"interface":"front","average":{"1min":3.883,"5min":3.883,"15min":3.883},"min":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"max":{"1min":54.823,"5min":54.823,"15min":54.823},"last":65.804000000000002}]},{"osd":12,"last update":"Wed Sep 18 10:15:32 2024","interfaces":[{"interface":"back","average":{"1min":5.0469999999999997,"5min":5.0469999999999997,"15min":5.0469999999999997},"min":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":52.378,"5min":52.378,"15min":52.378},"last":1.2190000000000001},{"interface":"front","average":{"1min":4.4379999999999997,"5min":4.4379999999999997,"15min":4.4379999999999997},"min":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":64.760999999999996,"5min":64.760999999999996,"15min":64.760999999999996},"last":67.835999999999999}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2050000000000001}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2669999999999999}]},{"osd":15,"last update":"Wed Sep 18 10:15:32 2024","interfaces":[{"interface":"back","average":{"1min":1.474,"5min":1.474,"15min":1.474},"min":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"max":{"1min":3.1840000000000002,"5min":3.1840000000000002,"15min":3.1840000000000002},"last":68.069999999999993},{"interface":"front","average":{"1min":3.9729999999999999,"5min":3.9729999999999999,"15min":3.9729999999999999},"min":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"max":{"1min":54.869999999999997,"5min":54.869999999999997,"15min":54.869999999999997},"last":1.302}]}]},{"osd":7,"up_from":22,"seq":94489280532,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":316892,"kb_used_data":15368,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94054948,"statfs":{"total":96636764160,"available":96312266752,"internally_reserved":0,"allocated":15736832,"data_stored":24122773,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,4,5,6,8,10,11,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:15:33 2024","interfaces":[{"interface":"back","average":{"1min":12.721,"5min":12.721,"15min":12.721},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":139.58600000000001,"5min":139.58600000000001,"15min":139.58600000000001},"last":1.0509999999999999},{"interface":"front","average":{"1min":0.999,"5min":0.999,"15min":0.999},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.98499999999999999}]},{"osd":2,"last update":"Wed Sep 18 10:15:46 2024","interfaces":[{"interface":"back","average":{"1min":15.436,"5min":15.436,"15min":15.436},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":139.36799999999999,"5min":139.36799999999999,"15min":139.36799999999999},"last":0.876},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.77200000000000002}]},{"osd":3,"last update":"Wed Sep 18 10:15:46 2024","interfaces":[{"interface":"back","average":{"1min":17.327999999999999,"5min":17.327999999999999,"15min":17.327999999999999},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":139.27199999999999,"5min":139.27199999999999,"15min":139.27199999999999},"last":0.70199999999999996},{"interface":"front","average":{"1min":3.964,"5min":3.964,"15min":3.964},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":23.748999999999999,"5min":23.748999999999999,"15min":23.748999999999999},"last":48.904000000000003}]},{"osd":4,"last update":"Wed Sep 18 10:15:37 2024","interfaces":[{"interface":"back","average":{"1min":19.146999999999998,"5min":19.146999999999998,"15min":19.146999999999998},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":139.51599999999999,"5min":139.51599999999999,"15min":139.51599999999999},"last":0.95999999999999996},{"interface":"front","average":{"1min":4.9989999999999997,"5min":4.9989999999999997,"15min":4.9989999999999997},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":27.882999999999999,"5min":27.882999999999999,"15min":27.882999999999999},"last":49.241999999999997}]},{"osd":5,"last update":"Wed Sep 18 10:15:46 2024","interfaces":[{"interface":"back","average":{"1min":1.099,"5min":1.099,"15min":1.099},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":2.3620000000000001,"5min":2.3620000000000001,"15min":2.3620000000000001},"last":0.84499999999999997},{"interface":"front","average":{"1min":17.736999999999998,"5min":17.736999999999998,"15min":17.736999999999998},"min":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"max":{"1min":139.44300000000001,"5min":139.44300000000001,"15min":139.44300000000001},"last":1.008}]},{"osd":6,"last update":"Wed Sep 18 10:15:33 2024","interfaces":[{"interface":"back","average":{"1min":17.375,"5min":17.375,"15min":17.375},"min":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":183.001,"5min":183.001,"15min":183.001},"last":40.060000000000002},{"interface":"front","average":{"1min":4.3109999999999999,"5min":4.3109999999999999,"15min":4.3109999999999999},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":23.978999999999999,"5min":23.978999999999999,"15min":23.978999999999999},"last":49.393000000000001}]},{"osd":8,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.25,"15min":1.25},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":2.6139999999999999,"5min":2.6139999999999999,"15min":2.6139999999999999},"last":0.92500000000000004},{"interface":"front","average":{"1min":15.906000000000001,"5min":15.906000000000001,"15min":15.906000000000001},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":139.798,"5min":139.798,"15min":139.798},"last":20.783999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.861}]},{"osd":10,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":6.5039999999999996,"5min":6.5039999999999996,"15min":6.5039999999999996},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":32.539999999999999,"5min":32.539999999999999,"15min":32.539999999999999},"last":49.323999999999998},{"interface":"front","average":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":2.254,"5min":2.254,"15min":2.254},"last":1.0289999999999999}]},{"osd":11,"last update":"Wed Sep 18 10:15:33 2024","interfaces":[{"interface":"back","average":{"1min":5.5469999999999997,"5min":5.5469999999999997,"15min":5.5469999999999997},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":62.356000000000002,"5min":62.356000000000002,"15min":62.356000000000002},"last":49.154000000000003},{"interface":"front","average":{"1min":1.411,"5min":1.411,"15min":1.411},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":2.738,"5min":2.738,"15min":2.738},"last":1.069}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.885}]},{"osd":13,"last update":"Wed Sep 18 10:15:33 2024","interfaces":[{"interface":"back","average":{"1min":18.379000000000001,"5min":18.379000000000001,"15min":18.379000000000001},"min":{"1min":0.874,"5min":0.874,"15min":0.874},"max":{"1min":139.655,"5min":139.655,"15min":139.655},"last":113.428},{"interface":"front","average":{"1min":19.879000000000001,"5min":19.879000000000001,"15min":19.879000000000001},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":173.113,"5min":173.113,"15min":173.113},"last":49.457999999999998}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5900000000000001}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5509999999999999}]}]},{"osd":8,"up_from":22,"seq":94489280532,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":329396,"kb_used_data":27872,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94042444,"statfs":{"total":96636764160,"available":96299462656,"internally_reserved":0,"allocated":28540928,"data_stored":29820309,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,4,6,7,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93300000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99399999999999999}]},{"osd":2,"last update":"Wed Sep 18 10:15:46 2024","interfaces":[{"interface":"back","average":{"1min":6.5880000000000001,"5min":6.5880000000000001,"15min":6.5880000000000001},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":77.435000000000002,"5min":77.435000000000002,"15min":77.435000000000002},"last":1.3080000000000001},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.526,"5min":1.526,"15min":1.526},"last":1.272}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.306}]},{"osd":4,"last update":"Wed Sep 18 10:15:46 2024","interfaces":[{"interface":"back","average":{"1min":11.678000000000001,"5min":11.678000000000001,"15min":11.678000000000001},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":90.438000000000002,"5min":90.438000000000002,"15min":90.438000000000002},"last":0.93999999999999995},{"interface":"front","average":{"1min":13.071999999999999,"5min":13.071999999999999,"15min":13.071999999999999},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":93.939999999999998,"5min":93.939999999999998,"15min":93.939999999999998},"last":1.1890000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4199999999999999}]},{"osd":6,"last update":"Wed Sep 18 10:15:46 2024","interfaces":[{"interface":"back","average":{"1min":18.465,"5min":18.465,"15min":18.465},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":293.26900000000001,"5min":293.26900000000001,"15min":293.26900000000001},"last":1.3600000000000001},{"interface":"front","average":{"1min":6.1559999999999997,"5min":6.1559999999999997,"15min":6.1559999999999997},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":70.203000000000003,"5min":70.203000000000003,"15min":70.203000000000003},"last":1.2270000000000001}]},{"osd":7,"last update":"Wed Sep 18 10:15:33 2024","interfaces":[{"interface":"back","average":{"1min":2.71,"5min":2.71,"15min":2.71},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":26.361000000000001,"5min":26.361000000000001,"15min":26.361000000000001},"last":1.393},{"interface":"front","average":{"1min":6.0709999999999997,"5min":6.0709999999999997,"15min":6.0709999999999997},"min":{"1min":0.878,"5min":0.878,"15min":0.878},"max":{"1min":79.186999999999998,"5min":79.186999999999998,"15min":79.186999999999998},"last":1.516}]},{"osd":9,"last update":"Wed Sep 18 10:15:33 2024","interfaces":[{"interface":"back","average":{"1min":2.5459999999999998,"5min":2.5459999999999998,"15min":2.5459999999999998},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":23.934000000000001,"5min":23.934000000000001,"15min":23.934000000000001},"last":1.6180000000000001},{"interface":"front","average":{"1min":8.9060000000000006,"5min":8.9060000000000006,"15min":8.9060000000000006},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":98.914000000000001,"5min":98.914000000000001,"15min":98.914000000000001},"last":1.4630000000000001}]},{"osd":10,"last update":"Wed Sep 18 10:15:33 2024","interfaces":[{"interface":"back","average":{"1min":6.9119999999999999,"5min":6.9119999999999999,"15min":6.9119999999999999},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":90.751000000000005,"5min":90.751000000000005,"15min":90.751000000000005},"last":1.853},{"interface":"front","average":{"1min":9.3390000000000004,"5min":9.3390000000000004,"15min":9.3390000000000004},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":83.795000000000002,"5min":83.795000000000002,"15min":83.795000000000002},"last":1.744}]},{"osd":11,"last update":"Wed Sep 18 10:15:33 2024","interfaces":[{"interface":"back","average":{"1min":2.9009999999999998,"5min":2.9009999999999998,"15min":2.9009999999999998},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":26.462,"5min":26.462,"15min":26.462},"last":1.548},{"interface":"front","average":{"1min":1.998,"5min":1.998,"15min":1.998},"min":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"max":{"1min":12.475,"5min":12.475,"15min":12.475},"last":1.5960000000000001}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6679999999999999}]},{"osd":13,"last update":"Wed Sep 18 10:15:33 2024","interfaces":[{"interface":"back","average":{"1min":9.1539999999999999,"5min":9.1539999999999999,"15min":9.1539999999999999},"min":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"max":{"1min":94.222999999999999,"5min":94.222999999999999,"15min":94.222999999999999},"last":1.778},{"interface":"front","average":{"1min":2.7240000000000002,"5min":2.7240000000000002,"15min":2.7240000000000002},"min":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"max":{"1min":23.983000000000001,"5min":23.983000000000001,"15min":23.983000000000001},"last":1.4239999999999999}]},{"osd":14,"last update":"Wed Sep 18 10:15:33 2024","interfaces":[{"interface":"back","average":{"1min":20.768999999999998,"5min":20.768999999999998,"15min":20.768999999999998},"min":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":150.93600000000001,"5min":150.93600000000001,"15min":150.93600000000001},"last":1.887},{"interface":"front","average":{"1min":3.5979999999999999,"5min":3.5979999999999999,"15min":3.5979999999999999},"min":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"max":{"1min":23.006,"5min":23.006,"15min":23.006},"last":2.012}]},{"osd":15,"last update":"Wed Sep 18 10:15:33 2024","interfaces":[{"interface":"back","average":{"1min":2.7949999999999999,"5min":2.7949999999999999,"15min":2.7949999999999999},"min":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"max":{"1min":24.100000000000001,"5min":24.100000000000001,"15min":24.100000000000001},"last":1.7949999999999999},{"interface":"front","average":{"1min":3.056,"5min":3.056,"15min":3.056},"min":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"max":{"1min":26.414000000000001,"5min":26.414000000000001,"15min":26.414000000000001},"last":1.8129999999999999}]}]},{"osd":10,"up_from":22,"seq":94489280533,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":317620,"kb_used_data":16096,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94054220,"statfs":{"total":96636764160,"available":96311521280,"internally_reserved":0,"allocated":16482304,"data_stored":21955989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,6,7,9,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1639999999999999}]},{"osd":1,"last update":"Wed Sep 18 10:15:33 2024","interfaces":[{"interface":"back","average":{"1min":10.933,"5min":10.933,"15min":10.933},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":84.203999999999994,"5min":84.203999999999994,"15min":84.203999999999994},"last":1.2689999999999999},{"interface":"front","average":{"1min":17.222999999999999,"5min":17.222999999999999,"15min":17.222999999999999},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":99.995999999999995,"5min":99.995999999999995,"15min":99.995999999999995},"last":0.995}]},{"osd":2,"last update":"Wed Sep 18 10:15:33 2024","interfaces":[{"interface":"back","average":{"1min":10.893000000000001,"5min":10.893000000000001,"15min":10.893000000000001},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":84.072000000000003,"5min":84.072000000000003,"15min":84.072000000000003},"last":1.038},{"interface":"front","average":{"1min":11.550000000000001,"5min":11.550000000000001,"15min":11.550000000000001},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":99.798000000000002,"5min":99.798000000000002,"15min":99.798000000000002},"last":1.298}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.214}]},{"osd":4,"last update":"Wed Sep 18 10:15:49 2024","interfaces":[{"interface":"back","average":{"1min":25.588999999999999,"5min":25.588999999999999,"15min":25.588999999999999},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":125.23099999999999,"5min":125.23099999999999,"15min":125.23099999999999},"last":1.407},{"interface":"front","average":{"1min":31.167999999999999,"5min":31.167999999999999,"15min":31.167999999999999},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":295.93200000000002,"5min":295.93200000000002,"15min":295.93200000000002},"last":1.341}]},{"osd":5,"last update":"Wed Sep 18 10:15:49 2024","interfaces":[{"interface":"back","average":{"1min":24.577999999999999,"5min":24.577999999999999,"15min":24.577999999999999},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":215.51400000000001,"5min":215.51400000000001,"15min":215.51400000000001},"last":1.623},{"interface":"front","average":{"1min":27.329999999999998,"5min":27.329999999999998,"15min":27.329999999999998},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":252.69300000000001,"5min":252.69300000000001,"15min":252.69300000000001},"last":1.5029999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6850000000000001}]},{"osd":7,"last update":"Wed Sep 18 10:15:33 2024","interfaces":[{"interface":"back","average":{"1min":3.0590000000000002,"5min":3.0590000000000002,"15min":3.0590000000000002},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":34.558,"5min":34.558,"15min":34.558},"last":1.4690000000000001},{"interface":"front","average":{"1min":13.959,"5min":13.959,"15min":13.959},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":84.123000000000005,"5min":84.123000000000005,"15min":84.123000000000005},"last":1.9710000000000001}]},{"osd":9,"last update":"Wed Sep 18 10:15:33 2024","interfaces":[{"interface":"back","average":{"1min":11.747,"5min":11.747,"15min":11.747},"min":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":100.214,"5min":100.214,"15min":100.214},"last":1.7949999999999999},{"interface":"front","average":{"1min":12.548,"5min":12.548,"15min":12.548},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":97.385999999999996,"5min":97.385999999999996,"15min":97.385999999999996},"last":1.744}]},{"osd":11,"last update":"Wed Sep 18 10:15:33 2024","interfaces":[{"interface":"back","average":{"1min":11.829000000000001,"5min":11.829000000000001,"15min":11.829000000000001},"min":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"max":{"1min":84.322999999999993,"5min":84.322999999999993,"15min":84.322999999999993},"last":1.891},{"interface":"front","average":{"1min":23.379000000000001,"5min":23.379000000000001,"15min":23.379000000000001},"min":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":295.70400000000001,"5min":295.70400000000001,"15min":295.70400000000001},"last":1.597}]},{"osd":12,"last update":"Wed Sep 18 10:15:33 2024","interfaces":[{"interface":"back","average":{"1min":3.1179999999999999,"5min":3.1179999999999999,"15min":3.1179999999999999},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":34.625999999999998,"5min":34.625999999999998,"15min":34.625999999999998},"last":1.659},{"interface":"front","average":{"1min":11.198,"5min":11.198,"15min":11.198},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":84.242999999999995,"5min":84.242999999999995,"15min":84.242999999999995},"last":1.714}]},{"osd":13,"last update":"Wed Sep 18 10:15:33 2024","interfaces":[{"interface":"back","average":{"1min":19.885000000000002,"5min":19.885000000000002,"15min":19.885000000000002},"min":{"1min":0.877,"5min":0.877,"15min":0.877},"max":{"1min":100.31,"5min":100.31,"15min":100.31},"last":1.915},{"interface":"front","average":{"1min":16.280000000000001,"5min":16.280000000000001,"15min":16.280000000000001},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":85.879999999999995,"5min":85.879999999999995,"15min":85.879999999999995},"last":1.855}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8260000000000001}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1080000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8249344,"data_stored":11628544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12156928,"data_stored":14569472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":17084416,"data_stored":23384064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":15585280,"data_stored":22941696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":18878464,"data_stored":26869760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":10059776,"data_stored":16465920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":15511552,"data_stored":21540864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":14217216,"data_stored":23400448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":26988544,"data_stored":29065216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":11415552,"data_stored":14823424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":14938112,"data_stored":21057536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":11685888,"data_stored":15036416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":13086720,"data_stored":18735104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":24100864,"data_stored":36208640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":22614016,"data_stored":32374784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":8794112,"data_stored":12627968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T10:16:01.719 INFO:tasks.ceph.ceph_manager.ceph:pg 3.19 in trimming, state: active+clean+snaptrim 2024-09-18T10:16:01.719 INFO:tasks.ceph.ceph_manager.ceph:pg 3.4 in trimming, state: active+clean+snaptrim 2024-09-18T10:16:01.719 INFO:tasks.ceph.ceph_manager.ceph:unique_pool_0 still trimming, waiting 2024-09-18T10:16:01.773 INFO:teuthology.orchestra.run.smithi029.stderr:noscrub is unset 2024-09-18T10:16:01.808 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-09-18T10:16:01.806+0000 7fdfa8698700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22087) UID: 0 2024-09-18T10:16:01.830 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-09-18T10:16:01.830 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-09-18T10:16:01.830 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-09-18T10:16:01.830 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-09-18T10:16:01.840 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_blocked_ops 2024-09-18T10:16:01.909 INFO:tasks.ceph.osd.3.smithi189.stderr:2024-09-18T10:16:01.905+0000 7f2440fc7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22396) UID: 0 2024-09-18T10:16:01.981 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-09-18T10:16:01.981 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-09-18T10:16:01.981 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-09-18T10:16:01.981 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-09-18T10:16:01.981 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-09-18T10:16:01.990 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_historic_ops 2024-09-18T10:16:02.010 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-18T10:16:02.003+0000 7f00046a8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 22136) UID: 0 2024-09-18T10:16:02.095 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-09-18T10:16:02.095 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-09-18T10:16:02.095 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-09-18T10:16:02.095 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-09-18T10:16:02.095 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.095 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4341.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-18T10:16:02.095 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:35.324351+0000", 2024-09-18T10:16:02.095 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 86.767162662999993, 2024-09-18T10:16:02.096 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.36640713699999999, 2024-09-18T10:16:02.096 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:02.096 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:02.096 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:02.096 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4341", 2024-09-18T10:16:02.096 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.29:0/3001591505", 2024-09-18T10:16:02.096 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 1 2024-09-18T10:16:02.096 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.096 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:02.096 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.096 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:02.096 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.324351+0000", 2024-09-18T10:16:02.096 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:02.096 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.097 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.097 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:02.097 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.324351+0000", 2024-09-18T10:16:02.097 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:02.097 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.097 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.097 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:02.097 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.324348+0000", 2024-09-18T10:16:02.097 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999976 2024-09-18T10:16:02.097 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.097 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.097 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:02.097 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.324359+0000", 2024-09-18T10:16:02.097 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1048e-05 2024-09-18T10:16:02.098 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.098 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.098 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:02.098 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.324383+0000", 2024-09-18T10:16:02.098 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.3813000000000001e-05 2024-09-18T10:16:02.098 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.098 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.098 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:02.098 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.324421+0000", 2024-09-18T10:16:02.098 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8012000000000001e-05 2024-09-18T10:16:02.098 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.098 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.098 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:02.098 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.324548+0000", 2024-09-18T10:16:02.099 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00012702199999999999 2024-09-18T10:16:02.099 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.099 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.099 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for peered", 2024-09-18T10:16:02.099 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.324565+0000", 2024-09-18T10:16:02.099 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.6325e-05 2024-09-18T10:16:02.099 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.099 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.099 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:02.099 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.588380+0000", 2024-09-18T10:16:02.099 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.263815458 2024-09-18T10:16:02.099 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.099 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.099 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:02.100 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.588626+0000", 2024-09-18T10:16:02.100 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00024633499999999998 2024-09-18T10:16:02.100 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.100 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.100 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 5", 2024-09-18T10:16:02.100 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.588803+0000", 2024-09-18T10:16:02.100 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00017640599999999999 2024-09-18T10:16:02.100 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.100 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.100 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:16:02.100 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.589943+0000", 2024-09-18T10:16:02.100 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0011400049999999999 2024-09-18T10:16:02.101 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.101 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.101 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:16:02.101 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.690623+0000", 2024-09-18T10:16:02.101 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100680032 2024-09-18T10:16:02.101 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.101 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.101 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:02.101 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.690702+0000", 2024-09-18T10:16:02.101 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.9342e-05 2024-09-18T10:16:02.101 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.101 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.101 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:02.102 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.690758+0000", 2024-09-18T10:16:02.102 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5639e-05 2024-09-18T10:16:02.102 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:02.102 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:02.102 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:02.102 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.102 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.102 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:264 3.2s0 3:46b2871c:::smithi18925896-34:head [append 0~606208] snapc 5=[5,4,3] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-18T10:16:02.102 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:56.739704+0000", 2024-09-18T10:16:02.102 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 65.351809677999995, 2024-09-18T10:16:02.102 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.79624036899999995, 2024-09-18T10:16:02.102 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:02.102 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:02.102 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:02.102 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:16:02.103 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:02.103 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 264 2024-09-18T10:16:02.103 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.103 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:02.103 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.103 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:02.103 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.739704+0000", 2024-09-18T10:16:02.103 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:02.103 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.103 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.103 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:02.103 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.739704+0000", 2024-09-18T10:16:02.103 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:02.104 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.104 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.104 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:02.104 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.739702+0000", 2024-09-18T10:16:02.104 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:16:02.104 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.104 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.104 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:02.104 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.740467+0000", 2024-09-18T10:16:02.104 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00076493100000000005 2024-09-18T10:16:02.104 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.104 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.104 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:02.104 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.740485+0000", 2024-09-18T10:16:02.105 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8516e-05 2024-09-18T10:16:02.105 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.105 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.105 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:02.105 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.740495+0000", 2024-09-18T10:16:02.105 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.2669999999999995e-06 2024-09-18T10:16:02.105 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.105 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.105 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:02.105 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.028948+0000", 2024-09-18T10:16:02.105 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.28845313500000003 2024-09-18T10:16:02.105 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.105 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.105 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for peered", 2024-09-18T10:16:02.106 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.028969+0000", 2024-09-18T10:16:02.106 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.1369000000000002e-05 2024-09-18T10:16:02.106 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.106 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.106 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:02.106 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.228324+0000", 2024-09-18T10:16:02.106 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.19935476699999999 2024-09-18T10:16:02.106 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.106 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.107 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:02.107 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.228558+0000", 2024-09-18T10:16:02.107 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00023393099999999999 2024-09-18T10:16:02.107 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.107 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.107 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:02.107 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.230390+0000", 2024-09-18T10:16:02.107 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018326970000000001 2024-09-18T10:16:02.107 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.107 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.107 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:02.107 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.239491+0000", 2024-09-18T10:16:02.107 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0091007950000000001 2024-09-18T10:16:02.107 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.108 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.108 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:02.108 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.535834+0000", 2024-09-18T10:16:02.108 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.29634234100000001 2024-09-18T10:16:02.108 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.108 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.108 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:02.108 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.535944+0000", 2024-09-18T10:16:02.108 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000110401 2024-09-18T10:16:02.108 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:02.108 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:02.108 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:02.108 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.109 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.109 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:265 3.2s0 3:46b2871c:::smithi18925896-34:head [append 0~376832] snapc 5=[5,4,3] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-18T10:16:02.109 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:56.741882+0000", 2024-09-18T10:16:02.109 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 65.349631656, 2024-09-18T10:16:02.109 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.80071129500000005, 2024-09-18T10:16:02.109 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:02.109 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:02.109 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:02.109 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:16:02.109 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:02.109 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 265 2024-09-18T10:16:02.109 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.109 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:02.110 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.110 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:02.110 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.741882+0000", 2024-09-18T10:16:02.110 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:02.110 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.110 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.110 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:02.110 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.741882+0000", 2024-09-18T10:16:02.110 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:02.110 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.110 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.110 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:02.110 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.741880+0000", 2024-09-18T10:16:02.110 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:16:02.111 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.111 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.111 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:02.111 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742251+0000", 2024-09-18T10:16:02.111 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00037050499999999998 2024-09-18T10:16:02.111 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.111 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.111 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:02.111 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742264+0000", 2024-09-18T10:16:02.111 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2787e-05 2024-09-18T10:16:02.111 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.111 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.111 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:02.111 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742269+0000", 2024-09-18T10:16:02.112 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.0509999999999996e-06 2024-09-18T10:16:02.112 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.112 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.112 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:02.112 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.234342+0000", 2024-09-18T10:16:02.112 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.492073712 2024-09-18T10:16:02.112 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.112 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.112 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:02.112 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.234556+0000", 2024-09-18T10:16:02.112 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00021380699999999999 2024-09-18T10:16:02.112 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.112 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.113 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:02.113 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.236316+0000", 2024-09-18T10:16:02.113 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0017597100000000001 2024-09-18T10:16:02.113 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.113 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.113 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:02.113 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.241476+0000", 2024-09-18T10:16:02.113 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0051604449999999996 2024-09-18T10:16:02.113 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.113 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.113 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:02.113 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.542502+0000", 2024-09-18T10:16:02.113 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.30102573700000002 2024-09-18T10:16:02.114 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.114 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.114 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:02.114 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.542593+0000", 2024-09-18T10:16:02.114 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.0859000000000005e-05 2024-09-18T10:16:02.114 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:02.114 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:02.114 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:02.114 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.114 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.114 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:266 3.2s0 3:46b2871c:::smithi18925896-34:head [setxattr _header (56) in=7b] snapc 5=[5,4,3] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-18T10:16:02.114 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:56.742274+0000", 2024-09-18T10:16:02.114 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 65.349239025000003, 2024-09-18T10:16:02.115 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.80056005299999999, 2024-09-18T10:16:02.115 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:02.115 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:02.115 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:02.115 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:16:02.115 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:02.115 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 266 2024-09-18T10:16:02.115 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.115 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:02.115 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.115 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:02.115 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742274+0000", 2024-09-18T10:16:02.115 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:02.116 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.116 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.116 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:02.116 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742274+0000", 2024-09-18T10:16:02.116 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:02.116 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.116 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.116 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:02.116 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742274+0000", 2024-09-18T10:16:02.116 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999995 2024-09-18T10:16:02.116 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.116 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.116 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:02.117 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742278+0000", 2024-09-18T10:16:02.117 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.0840000000000002e-06 2024-09-18T10:16:02.117 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.117 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.117 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:02.117 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742284+0000", 2024-09-18T10:16:02.117 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.1090000000000003e-06 2024-09-18T10:16:02.117 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.117 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.117 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:02.117 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742289+0000", 2024-09-18T10:16:02.117 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.4890000000000003e-06 2024-09-18T10:16:02.118 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.118 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.118 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:02.118 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.237660+0000", 2024-09-18T10:16:02.118 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.49537068299999998 2024-09-18T10:16:02.118 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.118 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.118 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:02.118 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.237924+0000", 2024-09-18T10:16:02.118 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00026387300000000003 2024-09-18T10:16:02.118 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.118 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.119 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:02.119 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.238979+0000", 2024-09-18T10:16:02.119 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0010549380000000001 2024-09-18T10:16:02.119 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.119 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.119 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:02.119 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.243571+0000", 2024-09-18T10:16:02.119 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0045919749999999999 2024-09-18T10:16:02.119 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.119 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.120 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:02.120 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.542742+0000", 2024-09-18T10:16:02.120 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.29917095999999999 2024-09-18T10:16:02.120 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.120 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.120 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:02.120 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.542834+0000", 2024-09-18T10:16:02.120 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.2536999999999998e-05 2024-09-18T10:16:02.120 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:02.120 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:02.120 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:02.120 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.120 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.121 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:267 3.2s0 3:46b2871c:::smithi18925896-34:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e41)", 2024-09-18T10:16:02.121 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:56.742292+0000", 2024-09-18T10:16:02.121 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 65.349221186999998, 2024-09-18T10:16:02.121 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.98973012599999999, 2024-09-18T10:16:02.121 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:02.121 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "started", 2024-09-18T10:16:02.121 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:02.121 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:16:02.121 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:02.121 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 267 2024-09-18T10:16:02.121 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.122 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:02.122 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.122 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:02.122 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742292+0000", 2024-09-18T10:16:02.122 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:02.122 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.122 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.122 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:02.122 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742292+0000", 2024-09-18T10:16:02.122 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:02.122 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.122 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.122 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:02.122 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742292+0000", 2024-09-18T10:16:02.123 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999995 2024-09-18T10:16:02.123 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.123 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.123 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:02.123 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742294+0000", 2024-09-18T10:16:02.123 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.103e-06 2024-09-18T10:16:02.123 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.123 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.123 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:02.123 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742298+0000", 2024-09-18T10:16:02.123 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8070000000000001e-06 2024-09-18T10:16:02.123 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.123 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.124 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:02.124 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742300+0000", 2024-09-18T10:16:02.124 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5890000000000001e-06 2024-09-18T10:16:02.124 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.124 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.124 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:02.124 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.239711+0000", 2024-09-18T10:16:02.124 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.497410625 2024-09-18T10:16:02.124 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.124 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.124 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for rw locks", 2024-09-18T10:16:02.124 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.239786+0000", 2024-09-18T10:16:02.124 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.4827999999999996e-05 2024-09-18T10:16:02.124 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.125 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.125 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:02.125 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.543234+0000", 2024-09-18T10:16:02.125 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.30344799099999997 2024-09-18T10:16:02.125 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.125 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.125 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:02.125 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.543335+0000", 2024-09-18T10:16:02.125 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000101625 2024-09-18T10:16:02.125 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.125 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.125 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:02.125 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.732022+0000", 2024-09-18T10:16:02.126 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.18868686700000001 2024-09-18T10:16:02.126 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:02.126 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:02.126 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:02.126 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.126 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.126 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:496 3.2s0 3:44cb9e27:::smithi18925896-5:head [read 0~81920,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T10:16:02.126 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:17.889532+0000", 2024-09-18T10:16:02.126 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 44.201981568000001, 2024-09-18T10:16:02.126 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.32688004999999998, 2024-09-18T10:16:02.126 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:02.126 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "started", 2024-09-18T10:16:02.126 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:02.126 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:16:02.127 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:02.127 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 496 2024-09-18T10:16:02.127 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.127 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:02.127 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.127 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:02.127 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889532+0000", 2024-09-18T10:16:02.127 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:02.127 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.127 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.127 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:02.127 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889532+0000", 2024-09-18T10:16:02.127 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:02.127 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.128 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.128 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:02.128 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889529+0000", 2024-09-18T10:16:02.128 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999971 2024-09-18T10:16:02.128 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.128 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.128 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:02.128 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889546+0000", 2024-09-18T10:16:02.128 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.6690000000000001e-05 2024-09-18T10:16:02.128 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.128 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.128 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:02.128 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889580+0000", 2024-09-18T10:16:02.128 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.4638999999999998e-05 2024-09-18T10:16:02.129 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.129 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.129 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:02.129 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889596+0000", 2024-09-18T10:16:02.129 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.5571999999999999e-05 2024-09-18T10:16:02.129 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.129 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.129 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:02.129 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.989817+0000", 2024-09-18T10:16:02.129 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100221713 2024-09-18T10:16:02.129 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.129 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.129 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op must wait for map", 2024-09-18T10:16:02.129 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.989836+0000", 2024-09-18T10:16:02.130 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8516e-05 2024-09-18T10:16:02.130 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.130 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.130 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:02.130 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.990420+0000", 2024-09-18T10:16:02.130 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000583918 2024-09-18T10:16:02.130 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.130 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.130 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:02.130 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.990693+0000", 2024-09-18T10:16:02.130 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00027301499999999999 2024-09-18T10:16:02.130 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.130 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.130 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:02.130 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:18.216412+0000", 2024-09-18T10:16:02.131 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.22571880699999999 2024-09-18T10:16:02.131 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:02.131 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:02.131 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:02.131 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.131 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.131 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:497 3.2s0 3:44cb9e27:::smithi18925896-5:head [read 0~81920,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T10:16:02.131 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:17.889627+0000", 2024-09-18T10:16:02.131 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 44.201886651000002, 2024-09-18T10:16:02.131 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.32738385800000003, 2024-09-18T10:16:02.131 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:02.131 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "started", 2024-09-18T10:16:02.131 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:02.132 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:16:02.132 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:02.132 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 497 2024-09-18T10:16:02.132 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.132 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:02.132 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.132 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:02.132 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889627+0000", 2024-09-18T10:16:02.132 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:02.132 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.132 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.132 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:02.132 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889627+0000", 2024-09-18T10:16:02.132 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:02.133 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.133 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.133 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:02.133 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889625+0000", 2024-09-18T10:16:02.133 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:16:02.133 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.133 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.133 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:02.133 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889633+0000", 2024-09-18T10:16:02.133 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.4170000000000003e-06 2024-09-18T10:16:02.133 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.133 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.133 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:02.133 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889664+0000", 2024-09-18T10:16:02.134 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0589000000000001e-05 2024-09-18T10:16:02.134 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.134 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.134 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:02.134 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889672+0000", 2024-09-18T10:16:02.134 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.3729999999999993e-06 2024-09-18T10:16:02.134 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.134 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.134 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:02.134 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.990017+0000", 2024-09-18T10:16:02.134 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100345608 2024-09-18T10:16:02.134 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.134 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.134 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:16:02.135 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.990046+0000", 2024-09-18T10:16:02.135 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8912999999999999e-05 2024-09-18T10:16:02.135 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.135 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.135 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:02.135 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.991459+0000", 2024-09-18T10:16:02.135 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0014122340000000001 2024-09-18T10:16:02.135 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.135 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.135 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:02.135 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.991631+0000", 2024-09-18T10:16:02.135 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000172029 2024-09-18T10:16:02.135 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.135 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.135 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:02.136 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:18.217010+0000", 2024-09-18T10:16:02.136 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.22537980499999999 2024-09-18T10:16:02.136 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:02.136 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:02.136 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:02.136 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.136 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.136 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:498 3.2s0 3:44cb9e27:::smithi18925896-5:head [read 0~81920,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T10:16:02.136 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:17.889709+0000", 2024-09-18T10:16:02.136 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 44.201804293000002, 2024-09-18T10:16:02.136 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.32775850600000001, 2024-09-18T10:16:02.136 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:02.136 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "started", 2024-09-18T10:16:02.136 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:02.137 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:16:02.137 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:02.137 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 498 2024-09-18T10:16:02.137 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.137 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:02.137 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.137 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:02.137 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889709+0000", 2024-09-18T10:16:02.137 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:02.137 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.137 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.137 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:02.137 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889709+0000", 2024-09-18T10:16:02.137 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:02.138 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.138 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.138 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:02.138 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889708+0000", 2024-09-18T10:16:02.138 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:16:02.138 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.138 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.138 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:02.138 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889716+0000", 2024-09-18T10:16:02.138 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.0989999999999999e-06 2024-09-18T10:16:02.138 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.138 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.138 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:02.138 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889732+0000", 2024-09-18T10:16:02.138 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.6093e-05 2024-09-18T10:16:02.139 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.139 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.139 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:02.139 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889742+0000", 2024-09-18T10:16:02.139 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.4599999999999992e-06 2024-09-18T10:16:02.139 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.139 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.139 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:02.139 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.992175+0000", 2024-09-18T10:16:02.139 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10243377200000001 2024-09-18T10:16:02.139 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.139 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.139 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:02.140 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.992333+0000", 2024-09-18T10:16:02.140 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00015753499999999999 2024-09-18T10:16:02.140 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.140 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.140 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:02.140 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:18.217467+0000", 2024-09-18T10:16:02.140 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.22513459999999999 2024-09-18T10:16:02.140 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:02.140 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:02.140 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:02.140 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.140 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.140 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:516 3.2s0 3:44cb9e27:::smithi18925896-5:head [append 0~425984] snapc 1a=[1a,19,18,16,12,11,10,f,e] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-18T10:16:02.140 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:19.430973+0000", 2024-09-18T10:16:02.141 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 42.660539903999997, 2024-09-18T10:16:02.141 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.59812562800000002, 2024-09-18T10:16:02.141 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:02.141 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:02.141 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:02.141 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:16:02.141 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:02.141 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 516 2024-09-18T10:16:02.141 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.141 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:02.141 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.141 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:02.141 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.430973+0000", 2024-09-18T10:16:02.141 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:02.142 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.142 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.142 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:02.142 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.430973+0000", 2024-09-18T10:16:02.142 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:02.142 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.142 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.142 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:02.142 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.430970+0000", 2024-09-18T10:16:02.142 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999967 2024-09-18T10:16:02.142 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.142 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.142 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:02.142 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.431995+0000", 2024-09-18T10:16:02.143 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001024896 2024-09-18T10:16:02.143 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.143 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.143 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:02.143 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.432030+0000", 2024-09-18T10:16:02.143 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5241999999999999e-05 2024-09-18T10:16:02.143 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.143 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.143 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:02.143 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.432044+0000", 2024-09-18T10:16:02.143 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.411e-05 2024-09-18T10:16:02.143 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.143 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.143 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:02.144 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.432226+0000", 2024-09-18T10:16:02.144 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000181841 2024-09-18T10:16:02.144 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.144 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.144 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:02.144 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.432421+0000", 2024-09-18T10:16:02.144 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000194527 2024-09-18T10:16:02.144 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.144 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.144 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:02.144 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.434150+0000", 2024-09-18T10:16:02.144 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001729742 2024-09-18T10:16:02.144 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.145 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.145 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:02.145 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.436513+0000", 2024-09-18T10:16:02.145 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0023622299999999999 2024-09-18T10:16:02.145 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.145 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.145 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:02.145 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:20.029005+0000", 2024-09-18T10:16:02.145 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.59249230100000005 2024-09-18T10:16:02.145 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.145 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.145 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:02.145 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:20.029099+0000", 2024-09-18T10:16:02.146 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.4042000000000005e-05 2024-09-18T10:16:02.146 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:02.146 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:02.147 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:02.147 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.147 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.147 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:517 3.2s0 3:44cb9e27:::smithi18925896-5:head [append 0~786432] snapc 1a=[1a,19,18,16,12,11,10,f,e] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-18T10:16:02.147 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:19.439831+0000", 2024-09-18T10:16:02.147 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 42.651682364999999, 2024-09-18T10:16:02.147 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.58958403800000003, 2024-09-18T10:16:02.147 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:02.147 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:02.147 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:02.147 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:16:02.147 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:02.147 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 517 2024-09-18T10:16:02.147 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.148 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:02.148 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.148 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:02.148 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.439831+0000", 2024-09-18T10:16:02.148 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:02.148 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.148 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.148 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:02.148 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.439831+0000", 2024-09-18T10:16:02.148 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:02.148 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.148 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.148 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:02.149 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.439828+0000", 2024-09-18T10:16:02.149 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999976 2024-09-18T10:16:02.149 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.149 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.149 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:02.149 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.441109+0000", 2024-09-18T10:16:02.149 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001280988 2024-09-18T10:16:02.149 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.149 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.149 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:02.149 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.441168+0000", 2024-09-18T10:16:02.149 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.8869000000000002e-05 2024-09-18T10:16:02.149 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.149 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.150 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:02.150 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.541302+0000", 2024-09-18T10:16:02.150 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100134055 2024-09-18T10:16:02.150 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.150 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.150 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:02.150 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.637962+0000", 2024-09-18T10:16:02.150 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.096659682999999996 2024-09-18T10:16:02.150 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.150 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.150 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:02.150 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.638164+0000", 2024-09-18T10:16:02.150 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00020226700000000001 2024-09-18T10:16:02.150 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.150 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.151 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:02.151 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.639933+0000", 2024-09-18T10:16:02.151 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001768501 2024-09-18T10:16:02.151 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.151 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.151 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:02.151 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.646572+0000", 2024-09-18T10:16:02.151 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.006639069 2024-09-18T10:16:02.151 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.151 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.151 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:02.151 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:20.029341+0000", 2024-09-18T10:16:02.151 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.38276939599999998 2024-09-18T10:16:02.152 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.152 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.152 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:02.152 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:20.029415+0000", 2024-09-18T10:16:02.152 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.3726000000000005e-05 2024-09-18T10:16:02.152 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:02.152 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:02.152 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:02.152 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.152 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.152 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:518 3.2s0 3:44cb9e27:::smithi18925896-5:head [append 0~376832] snapc 1a=[1a,19,18,16,12,11,10,f,e] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-18T10:16:02.152 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:19.541352+0000", 2024-09-18T10:16:02.152 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 42.550161326999998, 2024-09-18T10:16:02.153 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.48828847600000003, 2024-09-18T10:16:02.153 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:02.153 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:02.153 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:02.153 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:16:02.153 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:02.153 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 518 2024-09-18T10:16:02.153 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.153 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:02.153 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.153 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:02.153 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.541352+0000", 2024-09-18T10:16:02.153 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:02.154 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.154 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.154 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:02.154 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.541352+0000", 2024-09-18T10:16:02.154 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:02.154 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.154 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.154 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:02.154 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.541350+0000", 2024-09-18T10:16:02.154 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:16:02.154 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.154 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.154 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:02.154 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.541834+0000", 2024-09-18T10:16:02.154 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00048402500000000002 2024-09-18T10:16:02.155 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.155 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.155 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:02.155 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.541861+0000", 2024-09-18T10:16:02.155 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.7333000000000001e-05 2024-09-18T10:16:02.155 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.155 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.155 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:02.155 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.541875+0000", 2024-09-18T10:16:02.155 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4025999999999999e-05 2024-09-18T10:16:02.155 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.155 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.155 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:02.156 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.641684+0000", 2024-09-18T10:16:02.156 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.099808872000000007 2024-09-18T10:16:02.156 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.156 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.156 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:02.156 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.641873+0000", 2024-09-18T10:16:02.156 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00018933200000000001 2024-09-18T10:16:02.156 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.156 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.156 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:02.156 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.643401+0000", 2024-09-18T10:16:02.156 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001527104 2024-09-18T10:16:02.157 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.157 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.157 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:02.157 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.646854+0000", 2024-09-18T10:16:02.157 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0034532069999999998 2024-09-18T10:16:02.157 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.157 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.157 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:02.157 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:20.029569+0000", 2024-09-18T10:16:02.157 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.38271509599999998 2024-09-18T10:16:02.157 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.158 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.158 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:02.158 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:20.029640+0000", 2024-09-18T10:16:02.158 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.1594000000000006e-05 2024-09-18T10:16:02.158 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:02.158 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:02.158 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:02.158 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.158 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.158 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:519 3.2s0 3:44cb9e27:::smithi18925896-5:head [setxattr _header (56) in=7b] snapc 1a=[1a,19,18,16,12,11,10,f,e] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-18T10:16:02.158 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:19.541886+0000", 2024-09-18T10:16:02.158 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 42.549627639999997, 2024-09-18T10:16:02.159 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.488165768, 2024-09-18T10:16:02.159 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:02.159 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:02.159 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:02.159 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:16:02.159 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:02.159 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 519 2024-09-18T10:16:02.159 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.159 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:02.159 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.159 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:02.159 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.541886+0000", 2024-09-18T10:16:02.159 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:02.160 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.160 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.160 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:02.160 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.541886+0000", 2024-09-18T10:16:02.160 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:02.160 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.160 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.160 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:02.160 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.541884+0000", 2024-09-18T10:16:02.160 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:16:02.160 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.160 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.160 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:02.161 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.541890+0000", 2024-09-18T10:16:02.161 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.0589999999999999e-06 2024-09-18T10:16:02.161 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.161 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.161 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:02.161 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.541899+0000", 2024-09-18T10:16:02.161 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.02e-06 2024-09-18T10:16:02.161 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.161 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.161 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:02.161 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.541905+0000", 2024-09-18T10:16:02.161 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.9399999999999999e-06 2024-09-18T10:16:02.162 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.162 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.162 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:02.162 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.644561+0000", 2024-09-18T10:16:02.162 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.102655408 2024-09-18T10:16:02.162 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.162 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.162 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:02.162 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.644869+0000", 2024-09-18T10:16:02.162 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00030766700000000002 2024-09-18T10:16:02.162 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.162 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.162 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:02.162 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.645745+0000", 2024-09-18T10:16:02.163 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000876314 2024-09-18T10:16:02.163 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.163 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.163 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:02.163 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.647092+0000", 2024-09-18T10:16:02.163 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0013469269999999999 2024-09-18T10:16:02.163 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.163 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.163 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:02.163 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:20.029901+0000", 2024-09-18T10:16:02.163 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.38280895199999998 2024-09-18T10:16:02.163 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.163 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.164 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:02.164 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:20.030051+0000", 2024-09-18T10:16:02.164 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00015067699999999999 2024-09-18T10:16:02.164 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:02.164 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:02.164 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:02.164 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.164 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.164 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:573 3.2s0 3:46b2871c:::smithi18925896-34:head [read 0~557056,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-18T10:16:02.164 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:24.942342+0000", 2024-09-18T10:16:02.164 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 37.149170925999996, 2024-09-18T10:16:02.164 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.36312128700000001, 2024-09-18T10:16:02.164 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:02.164 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "started", 2024-09-18T10:16:02.165 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:02.165 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:16:02.165 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:02.165 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 573 2024-09-18T10:16:02.165 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.165 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:02.165 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.165 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:02.165 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.942342+0000", 2024-09-18T10:16:02.165 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:02.165 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.165 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.165 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:02.166 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.942342+0000", 2024-09-18T10:16:02.166 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:02.166 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.166 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.166 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:02.166 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.942341+0000", 2024-09-18T10:16:02.166 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:16:02.166 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.166 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.166 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:02.166 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.942363+0000", 2024-09-18T10:16:02.166 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.1406000000000001e-05 2024-09-18T10:16:02.166 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.166 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.167 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:02.167 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.942374+0000", 2024-09-18T10:16:02.167 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1466e-05 2024-09-18T10:16:02.167 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.167 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.167 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:02.167 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.942381+0000", 2024-09-18T10:16:02.167 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.9530000000000002e-06 2024-09-18T10:16:02.167 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.167 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.167 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:02.167 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.942654+0000", 2024-09-18T10:16:02.167 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000272692 2024-09-18T10:16:02.168 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.168 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.168 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:16:02.168 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.942672+0000", 2024-09-18T10:16:02.168 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7643e-05 2024-09-18T10:16:02.168 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.168 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.168 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:02.168 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:25.101730+0000", 2024-09-18T10:16:02.168 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.15905815800000001 2024-09-18T10:16:02.168 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.168 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.168 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:02.168 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:25.101881+0000", 2024-09-18T10:16:02.169 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000151299 2024-09-18T10:16:02.169 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.169 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.169 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:02.169 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:25.305464+0000", 2024-09-18T10:16:02.169 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.203582656 2024-09-18T10:16:02.169 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:02.169 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:02.169 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:02.169 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.169 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.169 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:574 3.2s0 3:46b2871c:::smithi18925896-34:head [read 0~557056,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-18T10:16:02.169 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:24.942392+0000", 2024-09-18T10:16:02.170 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 37.149121129999997, 2024-09-18T10:16:02.170 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.36415850300000002, 2024-09-18T10:16:02.170 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:02.170 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "started", 2024-09-18T10:16:02.170 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:02.170 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:16:02.170 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:02.170 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 574 2024-09-18T10:16:02.170 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.170 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:02.170 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.170 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:02.170 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.942392+0000", 2024-09-18T10:16:02.171 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:02.171 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.171 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.171 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:02.171 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.942392+0000", 2024-09-18T10:16:02.171 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:02.171 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.171 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.171 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:02.171 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.942391+0000", 2024-09-18T10:16:02.171 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:16:02.171 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.171 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.172 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:02.172 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.942396+0000", 2024-09-18T10:16:02.172 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.6759999999999998e-06 2024-09-18T10:16:02.172 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.172 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.172 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:02.172 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.942407+0000", 2024-09-18T10:16:02.172 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0686999999999999e-05 2024-09-18T10:16:02.172 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.172 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.172 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:02.172 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.942413+0000", 2024-09-18T10:16:02.172 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.1659999999999999e-06 2024-09-18T10:16:02.173 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.173 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.173 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:02.173 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.942750+0000", 2024-09-18T10:16:02.173 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00033731700000000001 2024-09-18T10:16:02.173 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.173 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.173 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:16:02.173 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.942776+0000", 2024-09-18T10:16:02.173 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6342999999999998e-05 2024-09-18T10:16:02.173 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.173 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.173 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:02.174 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:25.102396+0000", 2024-09-18T10:16:02.174 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.15961924199999999 2024-09-18T10:16:02.174 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.174 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.174 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:02.174 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:25.102659+0000", 2024-09-18T10:16:02.174 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00026355500000000001 2024-09-18T10:16:02.174 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.174 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.174 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:02.174 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:25.306551+0000", 2024-09-18T10:16:02.174 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20389143700000001 2024-09-18T10:16:02.174 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:02.174 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:02.175 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:02.175 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.175 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.175 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWriteReply(3.2s0 89/40 ECSubWriteReply(tid=135, last_complete=89'96, committed=1, applied=1))", 2024-09-18T10:16:02.175 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:45.106385+0000", 2024-09-18T10:16:02.175 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 16.985128356000001, 2024-09-18T10:16:02.175 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.35616392200000002, 2024-09-18T10:16:02.175 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:02.175 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:16:02.175 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:02.175 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.175 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:02.175 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:45.106385+0000", 2024-09-18T10:16:02.176 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:02.176 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.176 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.176 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:02.176 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:45.106385+0000", 2024-09-18T10:16:02.176 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:02.176 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.176 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.176 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:02.176 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:45.106384+0000", 2024-09-18T10:16:02.176 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999995 2024-09-18T10:16:02.176 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.176 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.176 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:02.177 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:45.106388+0000", 2024-09-18T10:16:02.177 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.6100000000000002e-06 2024-09-18T10:16:02.177 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.177 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.177 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:02.177 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:45.106395+0000", 2024-09-18T10:16:02.177 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.8399999999999997e-06 2024-09-18T10:16:02.177 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.177 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.177 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:02.177 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:45.106399+0000", 2024-09-18T10:16:02.177 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.1899999999999997e-06 2024-09-18T10:16:02.177 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.177 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.178 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:02.178 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:45.461488+0000", 2024-09-18T10:16:02.178 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.35508847199999999 2024-09-18T10:16:02.178 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.178 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.178 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:02.178 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:45.462549+0000", 2024-09-18T10:16:02.178 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0010612760000000001 2024-09-18T10:16:02.178 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:02.178 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:02.178 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:02.178 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.178 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.179 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:891 3.2s0 3:427b2c0b:::smithi18925896-42:head [append 0~0] snapc 34=[34,33,30,2f,28,20,1f,1a,11] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-18T10:16:02.179 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:46.655855+0000", 2024-09-18T10:16:02.179 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 15.435658628000001, 2024-09-18T10:16:02.179 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.45749477399999999, 2024-09-18T10:16:02.179 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:02.179 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:02.179 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:02.179 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:16:02.179 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:02.179 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 891 2024-09-18T10:16:02.179 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.179 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:02.179 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.179 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:02.179 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.655855+0000", 2024-09-18T10:16:02.180 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:02.180 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.180 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.180 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:02.180 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.655855+0000", 2024-09-18T10:16:02.180 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:02.180 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.180 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.180 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:02.180 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.655852+0000", 2024-09-18T10:16:02.180 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999971 2024-09-18T10:16:02.180 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.180 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.180 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:02.181 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.655866+0000", 2024-09-18T10:16:02.181 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4842e-05 2024-09-18T10:16:02.181 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.181 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.181 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:02.181 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.655906+0000", 2024-09-18T10:16:02.181 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.9465000000000002e-05 2024-09-18T10:16:02.181 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.181 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.181 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:02.181 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.756059+0000", 2024-09-18T10:16:02.181 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100153544 2024-09-18T10:16:02.181 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.181 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.182 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:02.182 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.804067+0000", 2024-09-18T10:16:02.182 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.048007288000000002 2024-09-18T10:16:02.182 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.182 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.182 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:02.182 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.804374+0000", 2024-09-18T10:16:02.182 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00030716700000000001 2024-09-18T10:16:02.182 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.182 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.182 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:02.182 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.807008+0000", 2024-09-18T10:16:02.182 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.002633799 2024-09-18T10:16:02.183 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.183 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.183 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:02.183 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.911478+0000", 2024-09-18T10:16:02.183 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.104469986 2024-09-18T10:16:02.183 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.183 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.183 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:02.183 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:47.113216+0000", 2024-09-18T10:16:02.183 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.201737789 2024-09-18T10:16:02.183 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.183 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.183 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:02.184 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:47.113349+0000", 2024-09-18T10:16:02.184 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00013389700000000001 2024-09-18T10:16:02.184 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:02.184 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:02.184 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:02.184 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.184 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.184 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:892 3.2s0 3:427b2c0b:::smithi18925896-42:head [rollback 28] snapc 34=[34,33,30,2f,28,20,1f,1a,11] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-18T10:16:02.184 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:46.756090+0000", 2024-09-18T10:16:02.184 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 15.335423548, 2024-09-18T10:16:02.184 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.35768046799999997, 2024-09-18T10:16:02.184 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:02.184 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:02.184 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:02.185 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:16:02.185 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:02.185 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 892 2024-09-18T10:16:02.185 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.185 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:02.185 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.185 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:02.185 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.756090+0000", 2024-09-18T10:16:02.185 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:02.185 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.185 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.185 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:02.185 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.756090+0000", 2024-09-18T10:16:02.186 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:02.186 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.186 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.186 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:02.186 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.756088+0000", 2024-09-18T10:16:02.186 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:16:02.186 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.186 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.186 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:02.186 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.756099+0000", 2024-09-18T10:16:02.186 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0879999999999999e-05 2024-09-18T10:16:02.186 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.186 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.187 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:02.187 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.756116+0000", 2024-09-18T10:16:02.187 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7849000000000001e-05 2024-09-18T10:16:02.187 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.187 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.187 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:02.187 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.756136+0000", 2024-09-18T10:16:02.187 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.9720000000000001e-05 2024-09-18T10:16:02.187 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.187 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.187 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:02.187 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.808713+0000", 2024-09-18T10:16:02.187 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.052577089 2024-09-18T10:16:02.187 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.188 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.188 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:02.188 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.808963+0000", 2024-09-18T10:16:02.188 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000249824 2024-09-18T10:16:02.188 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.188 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.188 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:02.188 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.810124+0000", 2024-09-18T10:16:02.188 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0011610100000000001 2024-09-18T10:16:02.188 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.188 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.188 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:02.189 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:47.011739+0000", 2024-09-18T10:16:02.189 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20161468900000001 2024-09-18T10:16:02.189 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.189 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.189 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:02.189 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:47.113617+0000", 2024-09-18T10:16:02.189 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10187808700000001 2024-09-18T10:16:02.189 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.189 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.189 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:02.189 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:47.113770+0000", 2024-09-18T10:16:02.189 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00015334300000000001 2024-09-18T10:16:02.189 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:02.189 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:02.190 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:02.190 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.190 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.190 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:893 3.2s0 3:427b2c0b:::smithi18925896-42:head [append 0~0] snapc 34=[34,33,30,2f,28,20,1f,1a,11] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-18T10:16:02.190 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:46.756179+0000", 2024-09-18T10:16:02.190 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 15.335334639999999, 2024-09-18T10:16:02.190 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.66260898499999998, 2024-09-18T10:16:02.190 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:02.190 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:02.190 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:02.190 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:16:02.190 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:02.190 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 893 2024-09-18T10:16:02.191 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.191 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:02.191 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.191 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:02.191 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.756179+0000", 2024-09-18T10:16:02.191 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:02.191 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.191 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.191 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:02.191 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.756179+0000", 2024-09-18T10:16:02.191 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:02.191 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.191 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.192 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:02.192 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.756177+0000", 2024-09-18T10:16:02.192 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:16:02.192 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.192 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.192 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:02.192 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.756185+0000", 2024-09-18T10:16:02.192 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.32e-06 2024-09-18T10:16:02.192 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.192 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.192 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:02.192 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.756214+0000", 2024-09-18T10:16:02.192 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8992999999999998e-05 2024-09-18T10:16:02.193 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.193 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.193 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:02.193 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.856357+0000", 2024-09-18T10:16:02.193 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100142493 2024-09-18T10:16:02.193 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.193 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.193 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:02.193 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:47.114791+0000", 2024-09-18T10:16:02.193 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.25843391799999998 2024-09-18T10:16:02.193 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.193 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.193 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:02.194 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:47.114948+0000", 2024-09-18T10:16:02.194 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00015682700000000001 2024-09-18T10:16:02.194 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.194 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.194 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:02.194 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:47.116043+0000", 2024-09-18T10:16:02.194 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0010952309999999999 2024-09-18T10:16:02.194 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.194 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.194 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:02.194 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:47.117006+0000", 2024-09-18T10:16:02.194 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00096323899999999996 2024-09-18T10:16:02.194 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.194 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.195 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:02.195 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:47.418568+0000", 2024-09-18T10:16:02.196 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.30156222300000002 2024-09-18T10:16:02.196 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.196 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.196 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:02.196 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:47.418788+0000", 2024-09-18T10:16:02.196 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00021923799999999999 2024-09-18T10:16:02.196 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:02.197 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:02.197 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:02.197 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.197 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.197 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(osd.6.22:5 3.2s0 3:47202842:::smithi18925896-22:head [copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-18T10:16:02.197 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:46.756797+0000", 2024-09-18T10:16:02.197 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 15.334716126, 2024-09-18T10:16:02.197 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.35575880999999998, 2024-09-18T10:16:02.197 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:02.197 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:16:02.197 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:02.197 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.197 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:02.198 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.756797+0000", 2024-09-18T10:16:02.198 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:02.198 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.198 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.198 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:02.198 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.756797+0000", 2024-09-18T10:16:02.198 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:02.198 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.198 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.198 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:02.198 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.756795+0000", 2024-09-18T10:16:02.198 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:16:02.198 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.199 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.199 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:02.199 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.756807+0000", 2024-09-18T10:16:02.199 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1970000000000001e-05 2024-09-18T10:16:02.199 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.199 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.199 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:02.199 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.756837+0000", 2024-09-18T10:16:02.199 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9706999999999999e-05 2024-09-18T10:16:02.199 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.199 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.199 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:02.199 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.756849+0000", 2024-09-18T10:16:02.200 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1904e-05 2024-09-18T10:16:02.200 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.200 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.200 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:02.200 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:47.112101+0000", 2024-09-18T10:16:02.200 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.35525225900000001 2024-09-18T10:16:02.200 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.200 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.200 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:02.200 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:47.112556+0000", 2024-09-18T10:16:02.200 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00045505300000000002 2024-09-18T10:16:02.200 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:02.200 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:02.201 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:02.201 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.201 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.201 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:953 3.2s0 3:46b2871c:::smithi18925896-34:head [read 0~557056,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-18T10:16:02.201 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:52.128760+0000", 2024-09-18T10:16:02.201 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 9.9627529789999993, 2024-09-18T10:16:02.201 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.40564792700000002, 2024-09-18T10:16:02.201 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:02.201 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "started", 2024-09-18T10:16:02.201 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:02.201 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:16:02.201 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:02.201 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 953 2024-09-18T10:16:02.201 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.202 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:02.202 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.202 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:02.202 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:52.128760+0000", 2024-09-18T10:16:02.202 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:02.202 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.202 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.202 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:02.202 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:52.128760+0000", 2024-09-18T10:16:02.202 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:02.202 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.202 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.202 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:02.203 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:52.128757+0000", 2024-09-18T10:16:02.203 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999967 2024-09-18T10:16:02.203 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.203 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.203 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:02.203 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:52.128774+0000", 2024-09-18T10:16:02.203 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.6912e-05 2024-09-18T10:16:02.203 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.203 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.203 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:02.203 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:52.128809+0000", 2024-09-18T10:16:02.203 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.4913000000000003e-05 2024-09-18T10:16:02.203 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.203 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.204 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:02.204 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:52.228946+0000", 2024-09-18T10:16:02.204 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100137698 2024-09-18T10:16:02.204 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.204 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.204 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:02.204 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:52.229076+0000", 2024-09-18T10:16:02.204 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00012982 2024-09-18T10:16:02.204 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.204 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.204 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:02.204 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:52.229281+0000", 2024-09-18T10:16:02.204 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000204592 2024-09-18T10:16:02.205 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:02.205 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:02.205 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:02.205 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:52.534408+0000", 2024-09-18T10:16:02.205 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.30512741500000001 2024-09-18T10:16:02.205 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:02.205 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:02.205 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:02.205 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:02.205 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:02.205 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-09-18T10:16:02.207 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T10:16:02.207 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-18T10:16:02.203+0000 7f14c6a1a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 22138) UID: 0 2024-09-18T10:16:02.307 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-18T10:16:02.303+0000 7f578c52f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22134) UID: 0 2024-09-18T10:16:02.316 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-18T10:16:02.316 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-18T10:16:02.316 INFO:teuthology.orchestra.run.smithi189.stdout: "num_ops": 0 2024-09-18T10:16:02.316 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-18T10:16:02.326 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T10:16:02.408 INFO:tasks.ceph.osd.12.smithi029.stderr:2024-09-18T10:16:02.401+0000 7f8e4e627700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 22190) UID: 0 2024-09-18T10:16:02.422 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-18T10:16:02.423 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-18T10:16:02.423 INFO:teuthology.orchestra.run.smithi189.stdout: "complaint_time": 30, 2024-09-18T10:16:02.423 INFO:teuthology.orchestra.run.smithi189.stdout: "num_blocked_ops": 0 2024-09-18T10:16:02.423 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-18T10:16:02.432 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T10:16:02.509 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-09-18T10:16:02.506+0000 7fdfa8698700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22087) UID: 0 2024-09-18T10:16:02.533 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-18T10:16:02.533 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 20, 2024-09-18T10:16:02.533 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 600, 2024-09-18T10:16:02.533 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [ 2024-09-18T10:16:02.533 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.533 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 36/32 ECSubWriteReply(tid=5, last_complete=36'4, committed=1, applied=1))", 2024-09-18T10:16:02.533 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:50.303102+0000", 2024-09-18T10:16:02.533 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 72.226397047999995, 2024-09-18T10:16:02.533 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20304619600000001, 2024-09-18T10:16:02.533 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:02.533 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:16:02.534 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:02.534 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.534 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:02.534 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.303102+0000", 2024-09-18T10:16:02.534 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:02.534 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.534 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.534 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:02.534 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.303102+0000", 2024-09-18T10:16:02.534 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:02.534 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.534 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.534 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:02.534 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.303100+0000", 2024-09-18T10:16:02.535 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:16:02.535 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.535 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.535 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:02.535 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.303114+0000", 2024-09-18T10:16:02.535 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3866000000000001e-05 2024-09-18T10:16:02.535 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.535 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.535 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:02.535 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.303149+0000", 2024-09-18T10:16:02.535 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.5249000000000001e-05 2024-09-18T10:16:02.535 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.535 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.536 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:02.536 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.303174+0000", 2024-09-18T10:16:02.536 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4785000000000001e-05 2024-09-18T10:16:02.536 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.536 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.536 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:02.536 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.506129+0000", 2024-09-18T10:16:02.536 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20295469199999999 2024-09-18T10:16:02.536 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.536 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.536 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:02.537 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.506148+0000", 2024-09-18T10:16:02.537 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9720000000000001e-05 2024-09-18T10:16:02.537 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:02.537 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:02.537 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:02.537 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.537 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.537 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 36/32 ECSubWriteReply(tid=5, last_complete=36'4, committed=1, applied=1))", 2024-09-18T10:16:02.537 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:50.305623+0000", 2024-09-18T10:16:02.537 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 72.223876075999996, 2024-09-18T10:16:02.537 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40085840299999997, 2024-09-18T10:16:02.537 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:02.538 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:16:02.538 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:02.538 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.538 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:02.538 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.305623+0000", 2024-09-18T10:16:02.538 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:02.538 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.538 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.538 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:02.538 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.305623+0000", 2024-09-18T10:16:02.538 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:02.538 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.538 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.539 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:02.539 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.305621+0000", 2024-09-18T10:16:02.539 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999976 2024-09-18T10:16:02.539 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.539 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.539 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:02.539 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.305635+0000", 2024-09-18T10:16:02.539 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4283e-05 2024-09-18T10:16:02.539 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.539 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.539 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:02.539 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.305663+0000", 2024-09-18T10:16:02.539 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7475999999999999e-05 2024-09-18T10:16:02.539 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.540 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.540 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:02.540 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.305680+0000", 2024-09-18T10:16:02.540 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7609e-05 2024-09-18T10:16:02.540 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.540 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.540 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:02.540 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.706451+0000", 2024-09-18T10:16:02.540 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40077027399999998 2024-09-18T10:16:02.540 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.540 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.540 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:02.540 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.706481+0000", 2024-09-18T10:16:02.541 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0935000000000001e-05 2024-09-18T10:16:02.541 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:02.541 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:02.541 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:02.541 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.541 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.541 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 36/32 ECSubWriteReply(tid=5, last_complete=36'4, committed=1, applied=1))", 2024-09-18T10:16:02.541 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:50.305688+0000", 2024-09-18T10:16:02.541 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 72.223810712000002, 2024-09-18T10:16:02.541 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40093562900000002, 2024-09-18T10:16:02.541 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:02.541 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:16:02.541 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:02.542 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.542 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:02.542 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.305688+0000", 2024-09-18T10:16:02.542 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:02.542 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.542 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.542 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:02.542 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.305688+0000", 2024-09-18T10:16:02.542 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:02.542 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.542 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.542 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:02.542 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.305687+0000", 2024-09-18T10:16:02.542 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:16:02.543 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.543 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.543 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:02.543 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.305698+0000", 2024-09-18T10:16:02.543 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1763000000000001e-05 2024-09-18T10:16:02.543 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.543 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.543 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:02.543 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.305729+0000", 2024-09-18T10:16:02.543 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0502e-05 2024-09-18T10:16:02.543 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.543 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.543 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:02.544 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.305741+0000", 2024-09-18T10:16:02.544 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2363e-05 2024-09-18T10:16:02.544 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.544 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.544 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:02.544 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.706585+0000", 2024-09-18T10:16:02.544 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.400843265 2024-09-18T10:16:02.544 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.544 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.544 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:02.544 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.706624+0000", 2024-09-18T10:16:02.544 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9459000000000003e-05 2024-09-18T10:16:02.544 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:02.544 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:02.545 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:02.545 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.545 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.545 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 36/32 ECSubWriteReply(tid=4, last_complete=36'3, committed=1, applied=1))", 2024-09-18T10:16:02.545 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:50.404544+0000", 2024-09-18T10:16:02.545 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 72.124955197000006, 2024-09-18T10:16:02.545 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.30239659400000002, 2024-09-18T10:16:02.545 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:02.545 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:16:02.545 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:02.545 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.545 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:02.545 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.404544+0000", 2024-09-18T10:16:02.546 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:02.546 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.546 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.546 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:02.546 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.404544+0000", 2024-09-18T10:16:02.546 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:02.546 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.546 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.546 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:02.546 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.404542+0000", 2024-09-18T10:16:02.546 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:16:02.546 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.547 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.547 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:02.547 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.404555+0000", 2024-09-18T10:16:02.547 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3096e-05 2024-09-18T10:16:02.547 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.547 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.547 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:02.547 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.404576+0000", 2024-09-18T10:16:02.547 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1165000000000001e-05 2024-09-18T10:16:02.547 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.547 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.547 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:02.547 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.404587+0000", 2024-09-18T10:16:02.548 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0549e-05 2024-09-18T10:16:02.548 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.548 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.548 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:02.548 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.706657+0000", 2024-09-18T10:16:02.548 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.30206971999999999 2024-09-18T10:16:02.548 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.548 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.548 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:02.548 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.706941+0000", 2024-09-18T10:16:02.548 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00028382100000000002 2024-09-18T10:16:02.548 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:02.549 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:02.549 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:02.549 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.549 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.549 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 36/32 ECSubWriteReply(tid=5, last_complete=36'4, committed=1, applied=1))", 2024-09-18T10:16:02.549 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:50.406036+0000", 2024-09-18T10:16:02.549 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 72.123462979999999, 2024-09-18T10:16:02.549 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.30215917399999997, 2024-09-18T10:16:02.549 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:02.549 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:16:02.550 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:02.550 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.550 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:02.550 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.406036+0000", 2024-09-18T10:16:02.550 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:02.550 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.550 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.550 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:02.550 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.406036+0000", 2024-09-18T10:16:02.550 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:02.551 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.551 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.551 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:02.551 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.406035+0000", 2024-09-18T10:16:02.551 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:16:02.551 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.551 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.551 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:02.551 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.406046+0000", 2024-09-18T10:16:02.551 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1704e-05 2024-09-18T10:16:02.551 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.551 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.551 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:02.551 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.406065+0000", 2024-09-18T10:16:02.552 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.8448999999999999e-05 2024-09-18T10:16:02.552 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.552 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.552 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:02.552 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.406076+0000", 2024-09-18T10:16:02.552 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1286e-05 2024-09-18T10:16:02.552 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.552 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.552 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:02.552 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.707005+0000", 2024-09-18T10:16:02.552 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.30092866899999998 2024-09-18T10:16:02.552 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.553 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.553 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:02.553 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.708195+0000", 2024-09-18T10:16:02.553 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001190546 2024-09-18T10:16:02.553 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:02.553 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:02.553 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:02.553 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.553 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.553 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.cs3 36/32 ECSubWrite(tid=14, reqid=client.4632.0:92, at_version=0'0, trim_to=0'0, roll_forward_to=36'6))", 2024-09-18T10:16:02.553 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:50.504842+0000", 2024-09-18T10:16:02.553 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 72.024657578000003, 2024-09-18T10:16:02.553 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.202186116, 2024-09-18T10:16:02.554 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:02.554 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:16:02.554 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:02.554 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.554 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:02.554 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.504842+0000", 2024-09-18T10:16:02.554 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:02.554 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.554 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.554 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:02.554 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.504842+0000", 2024-09-18T10:16:02.554 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:02.554 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.554 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.555 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:02.555 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.504840+0000", 2024-09-18T10:16:02.555 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:16:02.555 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.555 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.555 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:02.555 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.504868+0000", 2024-09-18T10:16:02.555 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7809000000000001e-05 2024-09-18T10:16:02.555 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.555 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.555 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:02.555 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.504900+0000", 2024-09-18T10:16:02.555 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1921999999999997e-05 2024-09-18T10:16:02.556 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.556 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.556 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:02.556 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.605012+0000", 2024-09-18T10:16:02.556 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10011272 2024-09-18T10:16:02.556 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.556 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.556 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:02.556 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.605115+0000", 2024-09-18T10:16:02.556 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000103003 2024-09-18T10:16:02.556 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.556 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.556 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:16:02.557 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.605132+0000", 2024-09-18T10:16:02.557 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6359999999999999e-05 2024-09-18T10:16:02.557 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.557 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.557 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:16:02.557 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.706978+0000", 2024-09-18T10:16:02.557 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.101846672 2024-09-18T10:16:02.557 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.557 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.557 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:02.557 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.707028+0000", 2024-09-18T10:16:02.557 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9443000000000001e-05 2024-09-18T10:16:02.557 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:02.558 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:02.558 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:02.558 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.558 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.558 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:179 3.0s0 3:02f544f0:::smithi18925896-30:head [read 0~491520,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:16:02.558 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:52.680315+0000", 2024-09-18T10:16:02.558 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 69.849184449000006, 2024-09-18T10:16:02.558 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20435319499999999, 2024-09-18T10:16:02.558 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:02.559 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:16:02.559 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:16:02.559 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:16:02.559 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:02.559 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 179 2024-09-18T10:16:02.559 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.559 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:02.559 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.559 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:02.559 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.680315+0000", 2024-09-18T10:16:02.559 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:02.559 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.559 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.559 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:02.560 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.680315+0000", 2024-09-18T10:16:02.560 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:02.560 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.560 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.560 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:02.560 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.680313+0000", 2024-09-18T10:16:02.560 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:16:02.560 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.560 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.560 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:02.560 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.680322+0000", 2024-09-18T10:16:02.561 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.9299999999999992e-06 2024-09-18T10:16:02.561 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.561 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.561 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:02.561 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.680343+0000", 2024-09-18T10:16:02.561 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0556000000000001e-05 2024-09-18T10:16:02.561 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.561 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.561 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:02.561 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.680373+0000", 2024-09-18T10:16:02.561 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0592000000000001e-05 2024-09-18T10:16:02.562 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.562 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.562 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:02.562 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.681360+0000", 2024-09-18T10:16:02.562 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00098610400000000002 2024-09-18T10:16:02.562 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.562 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.562 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:16:02.562 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.681624+0000", 2024-09-18T10:16:02.563 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000263992 2024-09-18T10:16:02.563 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.563 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.563 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:02.563 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.884668+0000", 2024-09-18T10:16:02.563 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.203044321 2024-09-18T10:16:02.563 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:02.563 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:02.563 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:02.563 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.563 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.563 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:180 3.0s0 3:02f544f0:::smithi18925896-30:head [read 0~491520,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:16:02.564 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:52.680394+0000", 2024-09-18T10:16:02.564 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 69.849105370999993, 2024-09-18T10:16:02.564 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20540486599999999, 2024-09-18T10:16:02.564 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:02.564 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:16:02.564 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:16:02.564 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:16:02.564 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:02.564 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 180 2024-09-18T10:16:02.564 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.564 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:02.564 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.564 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:02.565 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.680394+0000", 2024-09-18T10:16:02.565 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:02.565 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.565 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.565 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:02.565 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.680394+0000", 2024-09-18T10:16:02.565 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:02.565 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.565 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.565 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:02.565 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.680393+0000", 2024-09-18T10:16:02.565 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:16:02.565 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.566 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.566 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:02.566 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.680401+0000", 2024-09-18T10:16:02.566 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.4169999999999992e-06 2024-09-18T10:16:02.566 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.566 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.566 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:02.566 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.680422+0000", 2024-09-18T10:16:02.566 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1186e-05 2024-09-18T10:16:02.566 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.566 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.566 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:02.566 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.680433+0000", 2024-09-18T10:16:02.567 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1103e-05 2024-09-18T10:16:02.567 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.567 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.567 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:02.567 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.783888+0000", 2024-09-18T10:16:02.567 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.103454746 2024-09-18T10:16:02.567 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.567 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.567 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:16:02.567 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.784067+0000", 2024-09-18T10:16:02.567 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00017865399999999999 2024-09-18T10:16:02.567 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.567 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.567 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:02.568 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.885799+0000", 2024-09-18T10:16:02.568 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10173201699999999 2024-09-18T10:16:02.568 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:02.568 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:02.568 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:02.568 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.568 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.568 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:191 3.0s0 3:0a83d6b3:::smithi18925896-20:2 [read 0~327680,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:16:02.568 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:52.694194+0000", 2024-09-18T10:16:02.568 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 69.835304930999996, 2024-09-18T10:16:02.568 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.29753027399999998, 2024-09-18T10:16:02.568 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:02.568 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:16:02.569 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:16:02.569 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:16:02.569 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:02.569 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 191 2024-09-18T10:16:02.569 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.569 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:02.569 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.569 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:02.569 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.694194+0000", 2024-09-18T10:16:02.569 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:02.569 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.569 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.569 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:02.569 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.694194+0000", 2024-09-18T10:16:02.570 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:02.570 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.570 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.570 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:02.570 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.694192+0000", 2024-09-18T10:16:02.570 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:16:02.570 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.570 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.570 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:02.571 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.694204+0000", 2024-09-18T10:16:02.571 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2233e-05 2024-09-18T10:16:02.571 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.571 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.571 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:02.572 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.694280+0000", 2024-09-18T10:16:02.572 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.5584e-05 2024-09-18T10:16:02.572 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.572 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.572 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:02.572 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.694292+0000", 2024-09-18T10:16:02.573 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2478e-05 2024-09-18T10:16:02.573 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.573 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.573 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:02.573 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.885869+0000", 2024-09-18T10:16:02.574 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.191576304 2024-09-18T10:16:02.574 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.574 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.574 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:16:02.574 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.886008+0000", 2024-09-18T10:16:02.574 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00013880300000000001 2024-09-18T10:16:02.574 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.574 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.575 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:02.575 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.991725+0000", 2024-09-18T10:16:02.575 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10571691 2024-09-18T10:16:02.575 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:02.575 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:02.575 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:02.575 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.575 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.575 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:192 3.0s0 3:0a83d6b3:::smithi18925896-20:2 [read 0~327680,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:16:02.575 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:52.694306+0000", 2024-09-18T10:16:02.575 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 69.835192981000006, 2024-09-18T10:16:02.575 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.41051474799999998, 2024-09-18T10:16:02.575 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:02.575 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:16:02.576 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:16:02.576 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:16:02.576 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:02.576 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 192 2024-09-18T10:16:02.576 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.577 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:02.577 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.577 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:02.577 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.694306+0000", 2024-09-18T10:16:02.577 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:02.577 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.578 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.578 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:02.578 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.694306+0000", 2024-09-18T10:16:02.578 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:02.578 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.578 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.578 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:02.579 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.694305+0000", 2024-09-18T10:16:02.579 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:16:02.579 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.579 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.579 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:02.579 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.694312+0000", 2024-09-18T10:16:02.579 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.5129999999999999e-06 2024-09-18T10:16:02.580 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.580 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.580 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:02.580 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.694328+0000", 2024-09-18T10:16:02.580 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5766000000000001e-05 2024-09-18T10:16:02.580 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.580 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.581 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:02.581 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.694349+0000", 2024-09-18T10:16:02.581 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0990000000000001e-05 2024-09-18T10:16:02.581 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.581 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.581 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:02.582 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.887405+0000", 2024-09-18T10:16:02.582 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.193055791 2024-09-18T10:16:02.582 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.582 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.582 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:16:02.582 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.887535+0000", 2024-09-18T10:16:02.583 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00013043000000000001 2024-09-18T10:16:02.583 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.583 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.583 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:02.583 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:53.104821+0000", 2024-09-18T10:16:02.583 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.217285486 2024-09-18T10:16:02.584 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:02.584 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:02.584 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:02.584 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.584 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.584 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:193 3.0s0 3:0a83d6b3:::smithi18925896-20:2 [read 0~327680,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:16:02.584 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:52.694359+0000", 2024-09-18T10:16:02.585 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 69.835139799999993, 2024-09-18T10:16:02.585 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.411282918, 2024-09-18T10:16:02.585 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:02.585 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:16:02.585 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:16:02.585 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:16:02.585 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:02.586 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 193 2024-09-18T10:16:02.586 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.586 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:02.586 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.586 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:02.586 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.694359+0000", 2024-09-18T10:16:02.586 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:02.586 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.586 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.586 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:02.586 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.694359+0000", 2024-09-18T10:16:02.586 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:02.586 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.586 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.587 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:02.587 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.694358+0000", 2024-09-18T10:16:02.587 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:16:02.587 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.587 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.587 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:02.587 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.694364+0000", 2024-09-18T10:16:02.587 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8089999999999998e-06 2024-09-18T10:16:02.587 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.587 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.587 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:02.587 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.694381+0000", 2024-09-18T10:16:02.588 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6949e-05 2024-09-18T10:16:02.588 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.588 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.588 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:02.588 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.694388+0000", 2024-09-18T10:16:02.588 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.8079999999999999e-06 2024-09-18T10:16:02.588 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.588 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.588 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:02.588 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.988184+0000", 2024-09-18T10:16:02.589 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.29379642900000003 2024-09-18T10:16:02.589 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.589 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.589 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:16:02.589 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.988399+0000", 2024-09-18T10:16:02.589 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000214537 2024-09-18T10:16:02.589 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.589 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.589 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:02.590 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:53.105642+0000", 2024-09-18T10:16:02.590 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.11724332699999999 2024-09-18T10:16:02.590 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:02.590 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:02.590 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:02.590 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.590 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.590 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.10s0 45/40 ECSubWriteReply(tid=18, last_complete=45'6, committed=1, applied=1))", 2024-09-18T10:16:02.590 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:01.742819+0000", 2024-09-18T10:16:02.590 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 60.786680197000003, 2024-09-18T10:16:02.590 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.30317059800000001, 2024-09-18T10:16:02.590 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:02.590 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:16:02.591 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:02.591 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.591 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:02.591 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.742819+0000", 2024-09-18T10:16:02.591 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:02.591 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.591 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.591 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:02.591 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.742819+0000", 2024-09-18T10:16:02.591 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:02.591 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.591 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.592 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:02.592 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.742817+0000", 2024-09-18T10:16:02.592 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:16:02.592 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.592 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.592 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:02.592 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.742832+0000", 2024-09-18T10:16:02.592 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.466e-05 2024-09-18T10:16:02.592 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.592 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.592 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:02.592 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.742866+0000", 2024-09-18T10:16:02.592 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4242000000000002e-05 2024-09-18T10:16:02.592 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.593 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.593 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:02.593 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.843033+0000", 2024-09-18T10:16:02.593 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100166479 2024-09-18T10:16:02.593 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.593 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.593 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:02.593 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.045959+0000", 2024-09-18T10:16:02.593 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.202926 2024-09-18T10:16:02.593 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.593 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.593 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:02.594 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.045990+0000", 2024-09-18T10:16:02.594 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0889999999999997e-05 2024-09-18T10:16:02.594 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:02.594 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:02.594 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:02.594 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.594 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.594 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.10s0 45/40 ECSubWriteReply(tid=18, last_complete=45'6, committed=1, applied=1))", 2024-09-18T10:16:02.594 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:01.843113+0000", 2024-09-18T10:16:02.594 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 60.686386351000003, 2024-09-18T10:16:02.594 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.203080129, 2024-09-18T10:16:02.594 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:02.594 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:16:02.595 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:02.595 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.595 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:02.595 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.843113+0000", 2024-09-18T10:16:02.595 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:02.595 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.595 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.595 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:02.595 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.843113+0000", 2024-09-18T10:16:02.595 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:02.596 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.596 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.596 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:02.596 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.843111+0000", 2024-09-18T10:16:02.596 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:16:02.596 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.596 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.596 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:02.596 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.843122+0000", 2024-09-18T10:16:02.596 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0199e-05 2024-09-18T10:16:02.596 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.596 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.596 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:02.597 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.843147+0000", 2024-09-18T10:16:02.597 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5113000000000002e-05 2024-09-18T10:16:02.597 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.597 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.597 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:02.597 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.943289+0000", 2024-09-18T10:16:02.597 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100141813 2024-09-18T10:16:02.597 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.597 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.597 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:02.597 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.046146+0000", 2024-09-18T10:16:02.598 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.102857768 2024-09-18T10:16:02.598 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.598 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.598 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:02.598 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.046193+0000", 2024-09-18T10:16:02.598 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.6566000000000001e-05 2024-09-18T10:16:02.598 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:02.598 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:02.598 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:02.598 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.598 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.598 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.1s5 57/40 ECSubWrite(tid=111, reqid=osd.4.0:111, at_version=57'18, trim_to=0'0, roll_forward_to=56'16))", 2024-09-18T10:16:02.599 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:13.054749+0000", 2024-09-18T10:16:02.599 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 49.474749928000001, 2024-09-18T10:16:02.599 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.201940535, 2024-09-18T10:16:02.599 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:02.599 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:16:02.599 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:02.599 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.599 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:02.599 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.054749+0000", 2024-09-18T10:16:02.599 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:02.599 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.599 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.599 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:02.600 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.054749+0000", 2024-09-18T10:16:02.600 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:02.600 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.600 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.600 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:02.600 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.054749+0000", 2024-09-18T10:16:02.600 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999995 2024-09-18T10:16:02.600 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.600 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.600 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:02.600 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.054785+0000", 2024-09-18T10:16:02.600 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6480999999999999e-05 2024-09-18T10:16:02.600 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.600 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.601 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:02.601 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.054808+0000", 2024-09-18T10:16:02.601 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.2351e-05 2024-09-18T10:16:02.601 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.601 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.601 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:02.601 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.154914+0000", 2024-09-18T10:16:02.601 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100106862 2024-09-18T10:16:02.601 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.601 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.601 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:02.601 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.155065+0000", 2024-09-18T10:16:02.602 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000150762 2024-09-18T10:16:02.602 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.602 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.602 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:16:02.602 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.155083+0000", 2024-09-18T10:16:02.602 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.8207e-05 2024-09-18T10:16:02.602 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.602 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.602 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:16:02.602 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.256244+0000", 2024-09-18T10:16:02.602 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10116046200000001 2024-09-18T10:16:02.602 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.602 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.602 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:02.602 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.256690+0000", 2024-09-18T10:16:02.603 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00044597000000000001 2024-09-18T10:16:02.603 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:02.603 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:02.603 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:02.603 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.603 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.603 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:511 3.0s0 3:02f544f0:::smithi18925896-30:head [stat,delete] snapc 1a=[1a,19,18,16,12,11,10,f,e] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-18T10:16:02.603 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:18.899317+0000", 2024-09-18T10:16:02.603 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 43.630182550999997, 2024-09-18T10:16:02.603 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.21051439599999999, 2024-09-18T10:16:02.603 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:02.603 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:02.603 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:16:02.604 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:16:02.604 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:02.604 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 511 2024-09-18T10:16:02.604 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.604 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:02.604 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.604 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:02.604 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.899317+0000", 2024-09-18T10:16:02.604 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:02.604 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.604 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.604 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:02.604 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.899317+0000", 2024-09-18T10:16:02.604 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:02.605 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.605 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.605 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:02.605 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.899314+0000", 2024-09-18T10:16:02.605 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999971 2024-09-18T10:16:02.605 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.605 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.605 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:02.606 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.899329+0000", 2024-09-18T10:16:02.606 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5160000000000001e-05 2024-09-18T10:16:02.606 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.606 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.606 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:02.606 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.899364+0000", 2024-09-18T10:16:02.606 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.5456000000000001e-05 2024-09-18T10:16:02.606 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.606 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.606 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:02.606 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.899385+0000", 2024-09-18T10:16:02.606 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0279e-05 2024-09-18T10:16:02.606 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.607 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.607 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:02.607 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.899497+0000", 2024-09-18T10:16:02.607 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000111773 2024-09-18T10:16:02.607 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.607 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.607 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "op must wait for map", 2024-09-18T10:16:02.607 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.899508+0000", 2024-09-18T10:16:02.607 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1177e-05 2024-09-18T10:16:02.607 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.607 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.607 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:02.607 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.903698+0000", 2024-09-18T10:16:02.607 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0041902850000000002 2024-09-18T10:16:02.608 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.608 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.608 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:16:02.608 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.903878+0000", 2024-09-18T10:16:02.608 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00018009499999999999 2024-09-18T10:16:02.608 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.608 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.608 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:16:02.608 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.905286+0000", 2024-09-18T10:16:02.608 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0014077720000000001 2024-09-18T10:16:02.609 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.609 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.609 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:16:02.609 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.003449+0000", 2024-09-18T10:16:02.609 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.098163081999999999 2024-09-18T10:16:02.609 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.609 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.609 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:16:02.609 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.109709+0000", 2024-09-18T10:16:02.610 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10625963400000001 2024-09-18T10:16:02.610 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.610 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.610 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:02.610 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.109831+0000", 2024-09-18T10:16:02.610 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00012246699999999999 2024-09-18T10:16:02.610 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:02.610 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:02.610 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:02.610 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.610 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.610 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.19s6 72/40 ECSubWrite(tid=176, reqid=osd.7.0:174, at_version=0'0, trim_to=0'0, roll_forward_to=71'100))", 2024-09-18T10:16:02.610 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:28.330736+0000", 2024-09-18T10:16:02.610 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 34.198763085000003, 2024-09-18T10:16:02.611 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20302320200000001, 2024-09-18T10:16:02.611 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:02.611 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:16:02.611 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:02.611 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.611 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:02.611 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.330736+0000", 2024-09-18T10:16:02.611 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:02.612 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.612 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.612 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:02.612 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.330736+0000", 2024-09-18T10:16:02.612 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:02.612 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.612 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.612 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:02.612 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.330735+0000", 2024-09-18T10:16:02.612 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:16:02.612 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.612 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.613 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:02.613 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.330759+0000", 2024-09-18T10:16:02.613 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4549000000000002e-05 2024-09-18T10:16:02.613 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.613 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.613 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:02.613 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.330816+0000", 2024-09-18T10:16:02.613 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.6693000000000001e-05 2024-09-18T10:16:02.613 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.613 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.613 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:02.613 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.430957+0000", 2024-09-18T10:16:02.613 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10014150199999999 2024-09-18T10:16:02.614 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.614 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.614 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:02.614 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.531236+0000", 2024-09-18T10:16:02.614 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100278201 2024-09-18T10:16:02.614 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.614 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.614 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:16:02.614 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.531263+0000", 2024-09-18T10:16:02.614 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7172999999999999e-05 2024-09-18T10:16:02.614 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.614 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.614 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:16:02.614 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.533701+0000", 2024-09-18T10:16:02.615 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0024382760000000001 2024-09-18T10:16:02.615 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.615 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.615 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:02.615 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.533759+0000", 2024-09-18T10:16:02.615 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8267999999999998e-05 2024-09-18T10:16:02.615 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:02.615 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:02.615 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:02.615 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.615 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.616 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.2s1 100/40 ECSubWrite(tid=175, reqid=osd.14.0:175, at_version=100'117, trim_to=0'0, roll_forward_to=99'115))", 2024-09-18T10:16:02.616 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:55.723156+0000", 2024-09-18T10:16:02.616 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 6.8063430050000004, 2024-09-18T10:16:02.616 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20280843200000001, 2024-09-18T10:16:02.616 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:02.616 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:16:02.616 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:02.616 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.616 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:02.617 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:55.723156+0000", 2024-09-18T10:16:02.617 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:02.617 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.617 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.617 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:02.617 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:55.723156+0000", 2024-09-18T10:16:02.617 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:02.617 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.617 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.617 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:02.617 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:55.723156+0000", 2024-09-18T10:16:02.617 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999995 2024-09-18T10:16:02.617 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.617 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.618 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:02.618 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:55.723215+0000", 2024-09-18T10:16:02.618 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9318999999999999e-05 2024-09-18T10:16:02.618 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.618 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.618 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:02.618 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:55.723243+0000", 2024-09-18T10:16:02.618 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.8506000000000001e-05 2024-09-18T10:16:02.618 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.618 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.618 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:02.618 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:55.823360+0000", 2024-09-18T10:16:02.619 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100116356 2024-09-18T10:16:02.619 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.619 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.619 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:02.619 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:55.923621+0000", 2024-09-18T10:16:02.619 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10026093599999999 2024-09-18T10:16:02.619 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.619 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.619 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:16:02.619 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:55.923673+0000", 2024-09-18T10:16:02.619 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.2445999999999999e-05 2024-09-18T10:16:02.620 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.620 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.620 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:16:02.620 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:55.925909+0000", 2024-09-18T10:16:02.620 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0022360869999999999 2024-09-18T10:16:02.620 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.620 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.620 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:02.620 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:55.925965+0000", 2024-09-18T10:16:02.620 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.5439000000000002e-05 2024-09-18T10:16:02.621 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:02.621 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:02.621 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:02.621 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.621 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.621 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 100/40 ECSubWriteReply(tid=53, last_complete=100'38, committed=1, applied=1))", 2024-09-18T10:16:02.621 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:55.831180+0000", 2024-09-18T10:16:02.621 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 6.6983191810000005, 2024-09-18T10:16:02.621 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20294891000000001, 2024-09-18T10:16:02.621 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:02.622 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:16:02.622 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:02.622 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.622 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:02.622 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:55.831180+0000", 2024-09-18T10:16:02.622 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:02.622 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.622 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.622 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:02.622 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:55.831180+0000", 2024-09-18T10:16:02.622 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:02.622 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.622 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.623 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:02.623 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:55.831177+0000", 2024-09-18T10:16:02.623 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999971 2024-09-18T10:16:02.623 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.623 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.623 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:02.623 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:55.831194+0000", 2024-09-18T10:16:02.623 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6552999999999999e-05 2024-09-18T10:16:02.623 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.623 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.623 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:02.623 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:55.831225+0000", 2024-09-18T10:16:02.623 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1473000000000002e-05 2024-09-18T10:16:02.623 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.624 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.624 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:02.624 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:55.931322+0000", 2024-09-18T10:16:02.624 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100096437 2024-09-18T10:16:02.624 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.624 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.624 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:02.624 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:56.032873+0000", 2024-09-18T10:16:02.624 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.101551321 2024-09-18T10:16:02.624 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.624 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.624 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:02.624 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:56.034129+0000", 2024-09-18T10:16:02.625 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0012560430000000001 2024-09-18T10:16:02.625 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:02.625 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:02.625 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:02.625 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.625 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.625 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.1s5 101/40 ECSubWrite(tid=322, reqid=osd.4.0:322, at_version=101'100, trim_to=0'0, roll_forward_to=100'98))", 2024-09-18T10:16:02.625 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:56.721945+0000", 2024-09-18T10:16:02.625 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 5.8075538980000001, 2024-09-18T10:16:02.625 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20405271899999999, 2024-09-18T10:16:02.625 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:02.625 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:16:02.626 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:02.626 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.626 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:02.626 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:56.721945+0000", 2024-09-18T10:16:02.626 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:02.626 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.626 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.626 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:02.626 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:56.721945+0000", 2024-09-18T10:16:02.627 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:02.627 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.627 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.627 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:02.627 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:56.721945+0000", 2024-09-18T10:16:02.627 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999995 2024-09-18T10:16:02.627 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.627 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.627 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:02.627 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:56.721982+0000", 2024-09-18T10:16:02.627 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7251e-05 2024-09-18T10:16:02.627 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.627 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.627 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:02.628 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:56.722005+0000", 2024-09-18T10:16:02.628 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.2481999999999999e-05 2024-09-18T10:16:02.628 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.628 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.628 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:02.628 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:56.722017+0000", 2024-09-18T10:16:02.628 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2651e-05 2024-09-18T10:16:02.628 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.628 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.628 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:02.628 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:56.822239+0000", 2024-09-18T10:16:02.628 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10022138 2024-09-18T10:16:02.628 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.629 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.629 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:16:02.629 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:56.822255+0000", 2024-09-18T10:16:02.629 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6067e-05 2024-09-18T10:16:02.629 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.629 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.629 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:16:02.629 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:56.925947+0000", 2024-09-18T10:16:02.629 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.103692507 2024-09-18T10:16:02.629 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.629 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.629 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:02.630 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:56.925998+0000", 2024-09-18T10:16:02.630 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0682e-05 2024-09-18T10:16:02.630 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:02.630 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:02.630 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:02.630 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.630 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.630 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.1s5 101/40 ECSubWrite(tid=323, reqid=osd.4.0:323, at_version=101'102, trim_to=0'0, roll_forward_to=100'98))", 2024-09-18T10:16:02.630 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:56.724570+0000", 2024-09-18T10:16:02.630 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 5.8049292440000002, 2024-09-18T10:16:02.630 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20195624700000001, 2024-09-18T10:16:02.630 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:02.630 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:16:02.630 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:02.631 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.631 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:02.631 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:56.724570+0000", 2024-09-18T10:16:02.631 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:02.631 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.631 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.631 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:02.631 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:56.724570+0000", 2024-09-18T10:16:02.631 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:02.631 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.632 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.632 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:02.632 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:56.724569+0000", 2024-09-18T10:16:02.632 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:16:02.632 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.632 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.632 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:02.632 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:56.724618+0000", 2024-09-18T10:16:02.632 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.8810999999999997e-05 2024-09-18T10:16:02.633 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.633 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.633 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:02.633 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:56.724635+0000", 2024-09-18T10:16:02.633 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7465e-05 2024-09-18T10:16:02.633 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.633 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.633 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:02.633 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:56.724644+0000", 2024-09-18T10:16:02.633 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.833e-06 2024-09-18T10:16:02.633 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.633 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.633 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:02.634 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:56.924891+0000", 2024-09-18T10:16:02.634 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20024745699999999 2024-09-18T10:16:02.634 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.634 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.634 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:16:02.634 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:56.924934+0000", 2024-09-18T10:16:02.634 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.2976000000000003e-05 2024-09-18T10:16:02.634 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.634 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.634 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:16:02.634 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:56.926493+0000", 2024-09-18T10:16:02.634 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0015585029999999999 2024-09-18T10:16:02.634 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:02.634 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:02.635 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:02.635 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:56.926526+0000", 2024-09-18T10:16:02.635 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3376e-05 2024-09-18T10:16:02.635 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:02.635 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:02.635 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:02.635 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:02.635 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:02.635 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-18T10:16:02.637 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T10:16:02.637 INFO:tasks.ceph.osd.3.smithi189.stderr:2024-09-18T10:16:02.629+0000 7f2440fc7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22396) UID: 0 2024-09-18T10:16:02.738 INFO:tasks.ceph.osd.11.smithi189.stderr:2024-09-18T10:16:02.733+0000 7f718b60a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 22400) UID: 0 2024-09-18T10:16:02.742 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-18T10:16:02.742 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-18T10:16:02.742 INFO:teuthology.orchestra.run.smithi189.stdout: "num_ops": 0 2024-09-18T10:16:02.742 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-18T10:16:02.753 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T10:16:02.838 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-18T10:16:02.833+0000 7f14dcc5b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22398) UID: 0 2024-09-18T10:16:02.897 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-18T10:16:02.898 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-18T10:16:02.898 INFO:teuthology.orchestra.run.smithi189.stdout: "complaint_time": 30, 2024-09-18T10:16:02.898 INFO:teuthology.orchestra.run.smithi189.stdout: "num_blocked_ops": 0 2024-09-18T10:16:02.898 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-18T10:16:02.907 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T10:16:02.939 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-18T10:16:02.935+0000 7f00046a8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 22136) UID: 0 2024-09-18T10:16:03.008 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-18T10:16:03.008 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 20, 2024-09-18T10:16:03.008 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 600, 2024-09-18T10:16:03.008 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [ 2024-09-18T10:16:03.008 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.008 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:114 3.9s0 3:9a1a774c:::smithi18925896-38:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:16:03.008 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:50.816650+0000", 2024-09-18T10:16:03.009 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 72.187040461999999, 2024-09-18T10:16:03.009 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.41853457900000002, 2024-09-18T10:16:03.009 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:03.009 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:16:03.009 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:16:03.009 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:16:03.009 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:03.009 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 114 2024-09-18T10:16:03.009 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.009 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:03.009 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.009 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:03.009 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.816650+0000", 2024-09-18T10:16:03.010 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:03.010 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.010 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.010 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:03.010 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.816650+0000", 2024-09-18T10:16:03.010 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:03.010 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.010 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.010 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:03.010 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.816648+0000", 2024-09-18T10:16:03.010 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:16:03.011 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.011 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.011 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:03.011 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.816661+0000", 2024-09-18T10:16:03.011 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2619000000000001e-05 2024-09-18T10:16:03.011 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.011 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.011 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:03.011 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.816696+0000", 2024-09-18T10:16:03.011 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4885999999999998e-05 2024-09-18T10:16:03.011 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.011 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.012 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:03.012 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.816708+0000", 2024-09-18T10:16:03.012 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2843000000000001e-05 2024-09-18T10:16:03.012 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.012 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.012 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:03.012 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.818228+0000", 2024-09-18T10:16:03.012 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001519882 2024-09-18T10:16:03.012 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.012 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.012 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for rw locks", 2024-09-18T10:16:03.012 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.818557+0000", 2024-09-18T10:16:03.012 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00032873700000000001 2024-09-18T10:16:03.012 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.013 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.013 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:03.013 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.925488+0000", 2024-09-18T10:16:03.013 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.106930502 2024-09-18T10:16:03.013 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.013 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.013 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:16:03.017 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.925563+0000", 2024-09-18T10:16:03.017 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.5386e-05 2024-09-18T10:16:03.017 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.017 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.017 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:03.017 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.235185+0000", 2024-09-18T10:16:03.017 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.30962182399999999 2024-09-18T10:16:03.017 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:03.017 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:03.017 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:03.017 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.017 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.018 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:138 3.9s0 3:9ab7c4c0:::smithi18925896-46:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:16:03.018 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:50.921054+0000", 2024-09-18T10:16:03.018 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 72.082636163999993, 2024-09-18T10:16:03.018 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.61288337900000001, 2024-09-18T10:16:03.018 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:03.018 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:16:03.018 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:16:03.018 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:16:03.018 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:03.018 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 138 2024-09-18T10:16:03.018 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.019 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:03.019 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.019 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:03.019 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.921054+0000", 2024-09-18T10:16:03.019 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:03.019 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.019 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.019 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:03.019 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.921054+0000", 2024-09-18T10:16:03.019 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:03.020 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.020 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.020 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:03.020 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.921054+0000", 2024-09-18T10:16:03.020 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999995 2024-09-18T10:16:03.020 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.020 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.020 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:03.020 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.921057+0000", 2024-09-18T10:16:03.020 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1829999999999998e-06 2024-09-18T10:16:03.020 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.020 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.021 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:03.021 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.921090+0000", 2024-09-18T10:16:03.021 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3281999999999997e-05 2024-09-18T10:16:03.021 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.021 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.021 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:03.021 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.921096+0000", 2024-09-18T10:16:03.021 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.0519999999999998e-06 2024-09-18T10:16:03.021 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.021 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.021 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:03.021 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.127704+0000", 2024-09-18T10:16:03.021 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20660720499999999 2024-09-18T10:16:03.021 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.021 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.022 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for rw locks", 2024-09-18T10:16:03.022 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.127851+0000", 2024-09-18T10:16:03.022 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00014710999999999999 2024-09-18T10:16:03.022 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.022 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.022 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:03.022 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.234300+0000", 2024-09-18T10:16:03.022 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10644893900000001 2024-09-18T10:16:03.022 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.022 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.022 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:16:03.023 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.234462+0000", 2024-09-18T10:16:03.023 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00016218999999999999 2024-09-18T10:16:03.023 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.023 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.023 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:03.023 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.533938+0000", 2024-09-18T10:16:03.023 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.29947596999999998 2024-09-18T10:16:03.023 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:03.023 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:03.023 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:03.023 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.023 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.023 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:143 3.9s0 3:9221fff5:::smithi18925896-48:head [append 0~376832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:16:03.024 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:50.932038+0000", 2024-09-18T10:16:03.024 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 72.071652563000001, 2024-09-18T10:16:03.024 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.60109878000000005, 2024-09-18T10:16:03.024 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:03.024 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:03.024 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:16:03.024 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:16:03.024 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:03.024 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 143 2024-09-18T10:16:03.024 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.024 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:03.024 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.024 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:03.024 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932038+0000", 2024-09-18T10:16:03.025 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:03.025 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.025 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.025 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:03.025 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932038+0000", 2024-09-18T10:16:03.025 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:03.025 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.025 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.025 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:03.025 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932037+0000", 2024-09-18T10:16:03.025 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:16:03.025 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.025 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.025 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:03.026 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932309+0000", 2024-09-18T10:16:03.026 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000272086 2024-09-18T10:16:03.026 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.026 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.026 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:03.026 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932323+0000", 2024-09-18T10:16:03.026 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4354e-05 2024-09-18T10:16:03.026 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.026 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.026 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:03.026 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932329+0000", 2024-09-18T10:16:03.026 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.6840000000000001e-06 2024-09-18T10:16:03.026 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.027 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.027 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:03.027 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.130012+0000", 2024-09-18T10:16:03.027 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.19768277100000001 2024-09-18T10:16:03.027 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.027 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.027 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:16:03.027 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.130068+0000", 2024-09-18T10:16:03.027 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.6158e-05 2024-09-18T10:16:03.027 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.027 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.027 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:16:03.027 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.130809+0000", 2024-09-18T10:16:03.028 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00074078700000000002 2024-09-18T10:16:03.028 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.028 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.028 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:16:03.028 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.132840+0000", 2024-09-18T10:16:03.028 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0020308879999999998 2024-09-18T10:16:03.028 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.028 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.028 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:16:03.028 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.533025+0000", 2024-09-18T10:16:03.028 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.400185491 2024-09-18T10:16:03.028 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.028 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.028 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:03.029 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.533137+0000", 2024-09-18T10:16:03.029 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000111727 2024-09-18T10:16:03.029 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:03.029 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:03.029 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:03.029 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.029 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.029 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:144 3.9s0 3:9221fff5:::smithi18925896-48:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:16:03.029 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:50.932336+0000", 2024-09-18T10:16:03.029 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 72.071354927000002, 2024-09-18T10:16:03.029 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.60198766699999995, 2024-09-18T10:16:03.029 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:03.029 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:03.030 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:16:03.030 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:16:03.030 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:03.030 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 144 2024-09-18T10:16:03.030 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.030 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:03.030 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.030 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:03.030 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932336+0000", 2024-09-18T10:16:03.030 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:03.030 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.030 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.030 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:03.030 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932336+0000", 2024-09-18T10:16:03.031 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:03.031 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.031 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.031 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:03.031 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932335+0000", 2024-09-18T10:16:03.031 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:16:03.031 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.031 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.031 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:03.031 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932339+0000", 2024-09-18T10:16:03.031 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8539999999999999e-06 2024-09-18T10:16:03.031 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.031 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.032 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:03.032 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932347+0000", 2024-09-18T10:16:03.032 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.2910000000000007e-06 2024-09-18T10:16:03.032 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.032 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.032 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:03.032 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932351+0000", 2024-09-18T10:16:03.032 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.5420000000000001e-06 2024-09-18T10:16:03.032 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.032 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.032 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:03.032 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.131380+0000", 2024-09-18T10:16:03.033 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.199029236 2024-09-18T10:16:03.033 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.033 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.033 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:16:03.033 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.131436+0000", 2024-09-18T10:16:03.033 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.6098999999999999e-05 2024-09-18T10:16:03.033 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.033 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.033 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:16:03.033 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.131777+0000", 2024-09-18T10:16:03.033 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00034058800000000001 2024-09-18T10:16:03.033 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.033 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.033 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:16:03.034 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.133245+0000", 2024-09-18T10:16:03.034 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0014682300000000001 2024-09-18T10:16:03.034 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.034 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.034 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:16:03.034 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.534187+0000", 2024-09-18T10:16:03.034 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40094253299999999 2024-09-18T10:16:03.034 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.034 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.034 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:03.034 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.534323+0000", 2024-09-18T10:16:03.034 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00013606299999999999 2024-09-18T10:16:03.034 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:03.034 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:03.035 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:03.035 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.035 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.035 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:145 3.9s0 3:9221fff5:::smithi18925896-48:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:16:03.035 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:50.932355+0000", 2024-09-18T10:16:03.035 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 72.071335442999995, 2024-09-18T10:16:03.035 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.80502307200000001, 2024-09-18T10:16:03.035 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:03.035 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:16:03.035 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:16:03.035 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:16:03.035 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:03.035 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 145 2024-09-18T10:16:03.036 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.036 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:03.036 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.036 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:03.036 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932355+0000", 2024-09-18T10:16:03.036 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:03.036 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.036 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.036 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:03.036 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932355+0000", 2024-09-18T10:16:03.036 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:03.036 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.036 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.037 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:03.037 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932355+0000", 2024-09-18T10:16:03.037 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999995 2024-09-18T10:16:03.037 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.037 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.037 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:03.037 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932358+0000", 2024-09-18T10:16:03.037 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4369999999999999e-06 2024-09-18T10:16:03.037 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.037 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.037 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:03.037 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932366+0000", 2024-09-18T10:16:03.038 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.3010000000000007e-06 2024-09-18T10:16:03.038 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.038 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.038 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:03.038 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932370+0000", 2024-09-18T10:16:03.038 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.5619999999999999e-06 2024-09-18T10:16:03.038 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.038 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.038 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:03.038 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.132210+0000", 2024-09-18T10:16:03.038 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.199840145 2024-09-18T10:16:03.038 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.039 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.039 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for rw locks", 2024-09-18T10:16:03.039 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.132312+0000", 2024-09-18T10:16:03.039 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000102335 2024-09-18T10:16:03.039 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.039 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.039 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:03.039 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.535310+0000", 2024-09-18T10:16:03.039 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40299779000000002 2024-09-18T10:16:03.039 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.039 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.039 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:16:03.040 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.535541+0000", 2024-09-18T10:16:03.040 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00023038299999999999 2024-09-18T10:16:03.040 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.040 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.040 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:03.040 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.737378+0000", 2024-09-18T10:16:03.040 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.201837604 2024-09-18T10:16:03.040 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:03.040 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:03.040 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:03.040 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.041 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.041 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:174 3.9s0 3:9fc2eacb:::smithi18925896-17:head [read 0~278528,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:16:03.041 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:51.836672+0000", 2024-09-18T10:16:03.041 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 71.167018188, 2024-09-18T10:16:03.041 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.407996215, 2024-09-18T10:16:03.041 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:03.041 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:16:03.041 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:16:03.041 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:16:03.041 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:03.041 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 174 2024-09-18T10:16:03.041 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.041 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:03.042 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.042 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:03.042 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.836672+0000", 2024-09-18T10:16:03.042 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:03.042 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.042 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.042 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:03.042 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.836672+0000", 2024-09-18T10:16:03.042 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:03.042 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.042 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.042 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:03.042 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.836671+0000", 2024-09-18T10:16:03.043 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:16:03.043 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.043 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.043 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:03.043 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.836680+0000", 2024-09-18T10:16:03.043 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.5830000000000007e-06 2024-09-18T10:16:03.043 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.043 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.043 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:03.043 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.836696+0000", 2024-09-18T10:16:03.043 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6823000000000001e-05 2024-09-18T10:16:03.043 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.044 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.044 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:03.044 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.836710+0000", 2024-09-18T10:16:03.044 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.325e-05 2024-09-18T10:16:03.044 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.044 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.044 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:03.044 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.142312+0000", 2024-09-18T10:16:03.044 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.30560266400000002 2024-09-18T10:16:03.044 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.044 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.044 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:16:03.044 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.142549+0000", 2024-09-18T10:16:03.045 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00023630299999999999 2024-09-18T10:16:03.045 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.045 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.045 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:03.045 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.244669+0000", 2024-09-18T10:16:03.045 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10211993599999999 2024-09-18T10:16:03.045 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:03.045 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:03.045 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:03.045 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.045 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.046 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:202 3.9s0 3:9221fff5:::smithi18925896-48:head [copy-from ver 2] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:16:03.046 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:52.793605+0000", 2024-09-18T10:16:03.046 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 70.210085593000002, 2024-09-18T10:16:03.046 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.61381690899999997, 2024-09-18T10:16:03.046 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:03.046 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:03.046 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:16:03.046 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:16:03.046 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:03.046 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 202 2024-09-18T10:16:03.046 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.046 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:03.046 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.046 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:03.047 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.793605+0000", 2024-09-18T10:16:03.047 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:03.047 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.047 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.047 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:03.047 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.793605+0000", 2024-09-18T10:16:03.047 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:03.047 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.047 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.047 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:03.047 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.793604+0000", 2024-09-18T10:16:03.047 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:16:03.047 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.047 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.048 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:03.048 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.793613+0000", 2024-09-18T10:16:03.048 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.4360000000000008e-06 2024-09-18T10:16:03.048 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.048 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.048 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:03.048 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.793628+0000", 2024-09-18T10:16:03.048 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4953000000000001e-05 2024-09-18T10:16:03.048 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.048 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.048 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:03.048 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.793634+0000", 2024-09-18T10:16:03.048 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.0170000000000002e-06 2024-09-18T10:16:03.048 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.049 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.049 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:03.049 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.905359+0000", 2024-09-18T10:16:03.049 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.111724328 2024-09-18T10:16:03.049 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.049 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.049 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:16:03.049 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.905540+0000", 2024-09-18T10:16:03.049 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000181552 2024-09-18T10:16:03.049 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.049 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.049 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:16:03.049 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:53.203724+0000", 2024-09-18T10:16:03.050 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.29818339500000002 2024-09-18T10:16:03.050 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.050 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.050 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:16:03.050 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:53.206308+0000", 2024-09-18T10:16:03.050 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0025841620000000001 2024-09-18T10:16:03.050 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.050 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.050 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:16:03.050 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:53.407237+0000", 2024-09-18T10:16:03.050 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.200928739 2024-09-18T10:16:03.051 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.051 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.051 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:03.051 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:53.407422+0000", 2024-09-18T10:16:03.051 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000185384 2024-09-18T10:16:03.051 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:03.051 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:03.051 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:03.051 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.051 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.051 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:203 3.9s0 3:9221fff5:::smithi18925896-48:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:16:03.051 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:52.793641+0000", 2024-09-18T10:16:03.051 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 70.210049208000001, 2024-09-18T10:16:03.052 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.41220821099999999, 2024-09-18T10:16:03.052 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:03.052 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:16:03.052 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:16:03.052 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:16:03.052 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:03.052 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 203 2024-09-18T10:16:03.052 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.052 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:03.052 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.052 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:03.052 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.793641+0000", 2024-09-18T10:16:03.052 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:03.053 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.053 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.053 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:03.053 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.793641+0000", 2024-09-18T10:16:03.053 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:03.053 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.053 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.053 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:03.053 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.793641+0000", 2024-09-18T10:16:03.053 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:16:03.053 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.053 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.054 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:03.054 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.793644+0000", 2024-09-18T10:16:03.054 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7170000000000002e-06 2024-09-18T10:16:03.054 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.054 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.054 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:03.054 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.793658+0000", 2024-09-18T10:16:03.054 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3353000000000001e-05 2024-09-18T10:16:03.054 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.054 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.054 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:03.055 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.793663+0000", 2024-09-18T10:16:03.055 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.7729999999999996e-06 2024-09-18T10:16:03.055 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.055 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.055 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:03.055 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.905813+0000", 2024-09-18T10:16:03.055 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.112150033 2024-09-18T10:16:03.055 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.055 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.055 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for blocked object", 2024-09-18T10:16:03.055 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.905901+0000", 2024-09-18T10:16:03.055 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.7319999999999997e-05 2024-09-18T10:16:03.055 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.055 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.056 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:03.056 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:53.205317+0000", 2024-09-18T10:16:03.056 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.299415826 2024-09-18T10:16:03.056 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.056 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.056 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:16:03.056 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:53.205528+0000", 2024-09-18T10:16:03.056 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000211485 2024-09-18T10:16:03.056 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.056 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.056 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:03.056 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:53.205850+0000", 2024-09-18T10:16:03.056 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000321544 2024-09-18T10:16:03.057 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:03.057 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:03.057 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:03.057 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.057 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.057 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:351 3.19s0 3:9a1a774c:::smithi18925896-38:b [read 0~1638400,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-18T10:16:03.057 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:03.544302+0000", 2024-09-18T10:16:03.057 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 59.459388204, 2024-09-18T10:16:03.057 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.41512137599999999, 2024-09-18T10:16:03.058 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:03.058 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:16:03.058 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:16:03.058 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:16:03.058 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:03.058 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 351 2024-09-18T10:16:03.058 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.058 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:03.058 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.058 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:03.059 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.544302+0000", 2024-09-18T10:16:03.059 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:03.059 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.059 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.059 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:03.059 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.544302+0000", 2024-09-18T10:16:03.059 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:03.059 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.059 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.059 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:03.060 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.544298+0000", 2024-09-18T10:16:03.060 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999957 2024-09-18T10:16:03.060 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.060 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.060 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:03.060 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.544320+0000", 2024-09-18T10:16:03.060 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1549e-05 2024-09-18T10:16:03.060 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.060 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.060 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:03.061 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.544366+0000", 2024-09-18T10:16:03.061 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.6752999999999998e-05 2024-09-18T10:16:03.061 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.061 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.061 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:03.061 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.544387+0000", 2024-09-18T10:16:03.061 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1052000000000001e-05 2024-09-18T10:16:03.061 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.061 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.061 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:03.061 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.544758+0000", 2024-09-18T10:16:03.061 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00037069700000000002 2024-09-18T10:16:03.061 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.062 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.062 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "op must wait for map", 2024-09-18T10:16:03.062 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.544786+0000", 2024-09-18T10:16:03.062 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7472000000000001e-05 2024-09-18T10:16:03.062 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.062 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.062 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:03.062 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.649236+0000", 2024-09-18T10:16:03.062 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.104450378 2024-09-18T10:16:03.062 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.062 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.063 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:16:03.063 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.649492+0000", 2024-09-18T10:16:03.063 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00025605300000000001 2024-09-18T10:16:03.063 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.063 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.063 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:03.063 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.959424+0000", 2024-09-18T10:16:03.063 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.30993174899999998 2024-09-18T10:16:03.063 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:03.063 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:03.063 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:03.063 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.063 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.063 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:417 3.9s0 3:9221fff5:::smithi18925896-48:head [copy-from ver 8] snapc 10=[10,f,e,b,4] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-18T10:16:03.064 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:08.547348+0000", 2024-09-18T10:16:03.064 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 54.456342978000002, 2024-09-18T10:16:03.064 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.69832097199999998, 2024-09-18T10:16:03.064 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:03.064 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:03.064 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:16:03.064 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:16:03.064 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:03.064 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 417 2024-09-18T10:16:03.064 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.064 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:03.064 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.064 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:03.065 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:08.547348+0000", 2024-09-18T10:16:03.065 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:03.065 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.065 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.065 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:03.065 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:08.547348+0000", 2024-09-18T10:16:03.065 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:03.065 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.065 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.065 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:03.065 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:08.547344+0000", 2024-09-18T10:16:03.065 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999967 2024-09-18T10:16:03.065 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.065 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.066 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:03.066 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:08.547367+0000", 2024-09-18T10:16:03.066 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.2316000000000001e-05 2024-09-18T10:16:03.066 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.066 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.066 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:03.066 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:08.547415+0000", 2024-09-18T10:16:03.066 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.8541999999999997e-05 2024-09-18T10:16:03.066 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.066 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.066 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:03.066 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:08.647531+0000", 2024-09-18T10:16:03.066 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10011588 2024-09-18T10:16:03.066 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.066 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.067 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:03.067 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:08.647713+0000", 2024-09-18T10:16:03.067 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000181965 2024-09-18T10:16:03.067 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.067 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.067 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "op must wait for map", 2024-09-18T10:16:03.067 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:08.647726+0000", 2024-09-18T10:16:03.067 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3576e-05 2024-09-18T10:16:03.067 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.067 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.067 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:03.067 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:08.754339+0000", 2024-09-18T10:16:03.067 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10661229699999999 2024-09-18T10:16:03.067 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.067 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.067 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:16:03.068 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:08.754855+0000", 2024-09-18T10:16:03.068 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00051627900000000002 2024-09-18T10:16:03.068 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.068 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.068 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:16:03.068 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:09.040700+0000", 2024-09-18T10:16:03.068 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.285844922 2024-09-18T10:16:03.068 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.068 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.068 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:16:03.068 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:09.045961+0000", 2024-09-18T10:16:03.068 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0052605769999999998 2024-09-18T10:16:03.068 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.068 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.069 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:16:03.069 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:09.245481+0000", 2024-09-18T10:16:03.069 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.19952081599999999 2024-09-18T10:16:03.069 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.069 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.069 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:03.069 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:09.245669+0000", 2024-09-18T10:16:03.069 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000187245 2024-09-18T10:16:03.069 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:03.069 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:03.069 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:03.070 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.070 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.070 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.1s4 57/40 ECSubWrite(tid=112, reqid=osd.4.0:112, at_version=57'20, trim_to=0'0, roll_forward_to=56'16))", 2024-09-18T10:16:03.070 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:13.062699+0000", 2024-09-18T10:16:03.070 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 49.940991289000003, 2024-09-18T10:16:03.070 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40470361999999999, 2024-09-18T10:16:03.070 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:03.070 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:16:03.070 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:03.070 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.070 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:03.070 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.062699+0000", 2024-09-18T10:16:03.070 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:03.071 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.071 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.071 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:03.071 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.062699+0000", 2024-09-18T10:16:03.071 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:03.071 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.071 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.071 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:03.071 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.062699+0000", 2024-09-18T10:16:03.071 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999995 2024-09-18T10:16:03.071 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.071 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.071 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:03.071 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.062734+0000", 2024-09-18T10:16:03.072 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.5108999999999999e-05 2024-09-18T10:16:03.072 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.072 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.072 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:03.072 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.062746+0000", 2024-09-18T10:16:03.072 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2062000000000001e-05 2024-09-18T10:16:03.072 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.072 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.072 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:03.072 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.062754+0000", 2024-09-18T10:16:03.072 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.5800000000000003e-06 2024-09-18T10:16:03.072 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.072 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.072 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:03.072 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.254656+0000", 2024-09-18T10:16:03.073 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.19190244400000001 2024-09-18T10:16:03.073 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.073 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.073 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:16:03.073 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.254673+0000", 2024-09-18T10:16:03.073 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7146000000000002e-05 2024-09-18T10:16:03.073 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.073 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.073 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:16:03.073 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.467347+0000", 2024-09-18T10:16:03.073 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.21267335500000001 2024-09-18T10:16:03.073 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.074 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.074 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:03.074 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.467403+0000", 2024-09-18T10:16:03.074 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.6443999999999997e-05 2024-09-18T10:16:03.074 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:03.074 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:03.074 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:03.074 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.074 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.074 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:499 3.9s0 3:9221fff5:::smithi18925896-48:head [read 0~704512,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T10:16:03.074 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:17.889501+0000", 2024-09-18T10:16:03.074 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 45.114189252000003, 2024-09-18T10:16:03.074 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.41188610399999998, 2024-09-18T10:16:03.074 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:03.075 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:16:03.075 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:16:03.075 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:16:03.075 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:03.075 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 499 2024-09-18T10:16:03.075 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.075 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:03.075 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.075 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:03.075 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889501+0000", 2024-09-18T10:16:03.075 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:03.075 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.076 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.076 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:03.076 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889501+0000", 2024-09-18T10:16:03.076 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:03.076 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.076 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.076 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:03.076 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889499+0000", 2024-09-18T10:16:03.076 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999976 2024-09-18T10:16:03.076 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.076 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.077 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:03.077 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889514+0000", 2024-09-18T10:16:03.077 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4793999999999999e-05 2024-09-18T10:16:03.077 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.078 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.078 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:03.078 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889538+0000", 2024-09-18T10:16:03.078 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4218000000000001e-05 2024-09-18T10:16:03.078 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.078 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.078 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:03.078 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889552+0000", 2024-09-18T10:16:03.078 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4021e-05 2024-09-18T10:16:03.078 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.078 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.078 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:03.078 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889705+0000", 2024-09-18T10:16:03.079 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00015229499999999999 2024-09-18T10:16:03.079 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.079 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.079 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "op must wait for map", 2024-09-18T10:16:03.079 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889725+0000", 2024-09-18T10:16:03.079 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0251999999999999e-05 2024-09-18T10:16:03.079 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.079 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.079 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:03.079 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.894166+0000", 2024-09-18T10:16:03.079 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0044407969999999998 2024-09-18T10:16:03.079 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.079 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.079 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:16:03.080 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.894321+0000", 2024-09-18T10:16:03.080 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000155403 2024-09-18T10:16:03.080 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.080 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.080 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:03.080 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.301387+0000", 2024-09-18T10:16:03.080 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40706650900000002 2024-09-18T10:16:03.080 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:03.080 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:03.080 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:03.080 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.080 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.080 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:500 3.9s0 3:9221fff5:::smithi18925896-48:head [read 0~704512,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T10:16:03.081 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:17.889581+0000", 2024-09-18T10:16:03.081 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 45.114109710999998, 2024-09-18T10:16:03.081 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.41275783799999999, 2024-09-18T10:16:03.081 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:03.081 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:16:03.081 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:16:03.081 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:16:03.081 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:03.081 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 500 2024-09-18T10:16:03.081 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.081 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:03.081 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.081 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:03.082 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889581+0000", 2024-09-18T10:16:03.082 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:03.082 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.082 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.082 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:03.082 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889581+0000", 2024-09-18T10:16:03.082 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:03.082 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.082 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.082 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:03.082 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889580+0000", 2024-09-18T10:16:03.082 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999995 2024-09-18T10:16:03.082 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.082 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.083 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:03.083 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889586+0000", 2024-09-18T10:16:03.083 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8340000000000004e-06 2024-09-18T10:16:03.083 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.083 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.083 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:03.083 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889594+0000", 2024-09-18T10:16:03.083 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.8320000000000006e-06 2024-09-18T10:16:03.083 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.083 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.083 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:03.083 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889599+0000", 2024-09-18T10:16:03.083 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.7890000000000002e-06 2024-09-18T10:16:03.083 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.084 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.084 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:03.084 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889906+0000", 2024-09-18T10:16:03.084 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00030686400000000001 2024-09-18T10:16:03.084 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.084 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.084 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:16:03.084 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889930+0000", 2024-09-18T10:16:03.084 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4318999999999999e-05 2024-09-18T10:16:03.084 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.084 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.084 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:03.084 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.895541+0000", 2024-09-18T10:16:03.084 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0056113029999999998 2024-09-18T10:16:03.085 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.085 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.085 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:16:03.085 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.895817+0000", 2024-09-18T10:16:03.085 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000275612 2024-09-18T10:16:03.085 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.085 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.085 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:03.085 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.302339+0000", 2024-09-18T10:16:03.085 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40652197699999998 2024-09-18T10:16:03.085 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:03.085 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:03.085 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:03.085 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.086 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.086 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:501 3.9s0 3:9221fff5:::smithi18925896-48:head [read 0~704512,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T10:16:03.086 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:17.889668+0000", 2024-09-18T10:16:03.086 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 45.114022601000002, 2024-09-18T10:16:03.086 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.51380426099999998, 2024-09-18T10:16:03.086 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:03.086 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:16:03.086 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:16:03.086 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:16:03.086 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:03.086 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 501 2024-09-18T10:16:03.086 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.086 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:03.087 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.087 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:03.087 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889668+0000", 2024-09-18T10:16:03.087 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:03.087 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.087 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.087 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:03.087 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889668+0000", 2024-09-18T10:16:03.087 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:03.087 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.087 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.087 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:03.087 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889667+0000", 2024-09-18T10:16:03.087 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999995 2024-09-18T10:16:03.087 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.088 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.088 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:03.088 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889671+0000", 2024-09-18T10:16:03.088 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7570000000000002e-06 2024-09-18T10:16:03.088 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.088 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.088 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:03.088 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889682+0000", 2024-09-18T10:16:03.088 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0709e-05 2024-09-18T10:16:03.088 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.088 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.088 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:03.089 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889686+0000", 2024-09-18T10:16:03.089 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.437e-06 2024-09-18T10:16:03.089 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.089 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.089 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:03.089 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889996+0000", 2024-09-18T10:16:03.089 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00030937899999999998 2024-09-18T10:16:03.089 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.089 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.089 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:16:03.090 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.890018+0000", 2024-09-18T10:16:03.090 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.2592999999999999e-05 2024-09-18T10:16:03.090 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.090 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.090 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:03.090 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.996777+0000", 2024-09-18T10:16:03.090 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.106758192 2024-09-18T10:16:03.090 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.090 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.090 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:16:03.090 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.997045+0000", 2024-09-18T10:16:03.090 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00026851300000000001 2024-09-18T10:16:03.090 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.091 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.091 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:03.091 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.403472+0000", 2024-09-18T10:16:03.091 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40642724899999999 2024-09-18T10:16:03.091 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:03.091 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:03.091 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:03.091 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.091 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.091 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:502 3.19s0 3:9a1a774c:::smithi18925896-38:head [create,setxattr Rj\\mZOfQXo^b_ZWWfqN]m[iPt^aRVgsYNetVd`VO (19508) in=40b,setxattr [adnUNmYmWdN__]tR]msPnVSRiY^psh`dY\\Z (630) in=36b,setxattr cPsWXtVeXTPagdn_WVZbVcV\\UjSZUqsVlPZbY` (630) in=38b,setxattr pqbrReOSsgZlmftc]`Z`]USaXjsacf`fao_VP`eU (1190) in=40b,setxattr aYfgr^q[\\aPg_aZsakfUfsdhYbra^QXjlkrpdY\\\\ (138) in=40b,setxattr e\\XXWiVknihNQQ[OsThQUbRmtprrfYPeRX[[mRcQ (1719) in=40b,setxattr TNQYR^jgR`pPaYitmrcliV[dttobncTntcfTlQkt (8420) in=40b,setxattr W_aq[RVsaUpQtX_RpV^Vpqrg`QgQViPeVaRqZSmf (525) in=40b,setxattr RWdkeaaUP\\PZteTNscS^UVRtjo`oqqdNqe]cdZUa (245) in=40b,setxattr R^`al]TqifatYOiOU]p`ZegfXmNsXVaZcdd]b]YX (1435) in=40b,setxattr p]OS_Znl[dNa^qTn`WW[j\\dtgbNac_bijYmg (630) in=36b,setxattr n_jr[tV[VhYPqU]UenciobhtYfY^a^c\\hd^YrcQ[ (13472) in=40b,setxattr gReQSUg`cdl_ilU_lo[ggg\\fZsVQY[PgY]jPsZ_U (14664) in=40b,setxattr cNWaQrsdt^dic_qN_^mTXmc`_X]YOTXNk[WldRe` (523) in=40b,setxattr tZ]grhlaq]kUO[k^d^dr``XNio^^Oaebpj\\[Ytdq (33) in=40b,setxattr jqS_j^cefaXab`pQoPXfRkSmdako`fZ_heb^_fma (1191) in=40b,setxattr dgXPi (630) in=5b,setxattr Or_kkpbnNkTrQ\\apUaUsZR\\XW^nabhaiQ\\]gOjR\\ (525) in=40b,setxattr Rf`aR[fdtjQ\\NaqZke]l_rUssrVpOPXYWpXkl_kd (2876) in=40b,setxattr _sUSYWYUmOrlUVhZajZeYbbnsdra[]`Q`^Ogea[m (631) in=40b] snapc 19=[19,18,16,12,11,10,f,e] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T10:16:03.091 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:17.894821+0000", 2024-09-18T10:16:03.091 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 45.108869573, 2024-09-18T10:16:03.091 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.61297755899999995, 2024-09-18T10:16:03.091 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:03.092 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:03.092 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:16:03.092 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:16:03.092 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:03.092 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 502 2024-09-18T10:16:03.092 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.092 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:03.092 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.092 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:03.092 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.894821+0000", 2024-09-18T10:16:03.092 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:03.092 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.092 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.093 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:03.093 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.894821+0000", 2024-09-18T10:16:03.093 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:03.093 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.093 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.093 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:03.093 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.894819+0000", 2024-09-18T10:16:03.093 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:16:03.093 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.093 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.093 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:03.093 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.894942+0000", 2024-09-18T10:16:03.093 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00012299699999999999 2024-09-18T10:16:03.093 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.094 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.094 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:03.094 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.894987+0000", 2024-09-18T10:16:03.094 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.515e-05 2024-09-18T10:16:03.094 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.094 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.094 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:03.094 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.895021+0000", 2024-09-18T10:16:03.094 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3905000000000001e-05 2024-09-18T10:16:03.094 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.094 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.094 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:03.094 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.302420+0000", 2024-09-18T10:16:03.094 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40739912099999998 2024-09-18T10:16:03.095 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.095 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.095 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:16:03.095 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.302573+0000", 2024-09-18T10:16:03.095 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00015244400000000001 2024-09-18T10:16:03.095 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.095 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.095 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:16:03.095 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.304119+0000", 2024-09-18T10:16:03.095 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001546504 2024-09-18T10:16:03.095 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.095 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.095 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:16:03.095 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.306309+0000", 2024-09-18T10:16:03.096 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.002189992 2024-09-18T10:16:03.096 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.096 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.096 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:16:03.096 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.507653+0000", 2024-09-18T10:16:03.096 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20134429600000001 2024-09-18T10:16:03.096 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.096 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.096 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:03.096 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.507799+0000", 2024-09-18T10:16:03.096 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000145153 2024-09-18T10:16:03.096 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:03.096 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:03.096 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:03.097 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.097 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.097 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:514 3.19s0 3:9fc2eacb:::smithi18925896-17:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e63)", 2024-09-18T10:16:03.097 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:19.215577+0000", 2024-09-18T10:16:03.097 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 43.788113332000002, 2024-09-18T10:16:03.097 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.55261972699999995, 2024-09-18T10:16:03.097 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:03.097 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:16:03.097 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:16:03.097 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:16:03.097 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:03.097 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 514 2024-09-18T10:16:03.097 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.097 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:03.098 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.098 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:03.098 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.215577+0000", 2024-09-18T10:16:03.098 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:03.098 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.098 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.098 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:03.098 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.215577+0000", 2024-09-18T10:16:03.098 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:03.098 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.098 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.098 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:03.098 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.215575+0000", 2024-09-18T10:16:03.098 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:16:03.099 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.099 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.099 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:03.099 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.215590+0000", 2024-09-18T10:16:03.099 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4153e-05 2024-09-18T10:16:03.099 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.099 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.099 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:03.099 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.215610+0000", 2024-09-18T10:16:03.099 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9962e-05 2024-09-18T10:16:03.099 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.099 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.099 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:03.099 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.215622+0000", 2024-09-18T10:16:03.100 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2714000000000001e-05 2024-09-18T10:16:03.100 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.100 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.100 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:03.100 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.216288+0000", 2024-09-18T10:16:03.100 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00066566900000000003 2024-09-18T10:16:03.100 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.100 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.100 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for blocked object", 2024-09-18T10:16:03.100 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.216400+0000", 2024-09-18T10:16:03.100 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000112101 2024-09-18T10:16:03.100 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.100 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.101 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:03.101 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.767868+0000", 2024-09-18T10:16:03.101 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.55146753900000001 2024-09-18T10:16:03.101 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.101 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.101 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:16:03.101 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.768017+0000", 2024-09-18T10:16:03.101 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000149567 2024-09-18T10:16:03.101 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.101 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.101 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:03.101 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.768197+0000", 2024-09-18T10:16:03.101 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00017981200000000001 2024-09-18T10:16:03.101 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:03.102 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:03.102 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:03.102 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.102 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.102 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:561 3.19s0 3:9fc2eacb:::smithi18925896-17:head [append 0~622592] snapc 1f=[1f,1e,1a,16,12,11,10,f] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-18T10:16:03.102 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:23.955639+0000", 2024-09-18T10:16:03.102 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 39.048051985999997, 2024-09-18T10:16:03.102 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.51252563100000004, 2024-09-18T10:16:03.102 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:03.102 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:03.102 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:16:03.102 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:16:03.102 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:03.102 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 561 2024-09-18T10:16:03.103 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.103 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:03.103 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.103 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:03.103 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.955639+0000", 2024-09-18T10:16:03.103 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:03.103 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.103 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.103 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:03.103 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.955639+0000", 2024-09-18T10:16:03.103 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:03.103 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.103 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.103 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:03.104 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.955637+0000", 2024-09-18T10:16:03.104 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:16:03.104 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.104 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.104 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:03.104 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956211+0000", 2024-09-18T10:16:03.104 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00057413099999999997 2024-09-18T10:16:03.104 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.104 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.104 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:03.104 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956241+0000", 2024-09-18T10:16:03.104 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0244999999999999e-05 2024-09-18T10:16:03.104 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.105 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.105 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:03.105 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956253+0000", 2024-09-18T10:16:03.105 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1824000000000001e-05 2024-09-18T10:16:03.105 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.105 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.105 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:03.105 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956338+0000", 2024-09-18T10:16:03.105 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.5544999999999995e-05 2024-09-18T10:16:03.105 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.105 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.105 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:16:03.105 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956349+0000", 2024-09-18T10:16:03.105 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0203e-05 2024-09-18T10:16:03.105 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.106 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.106 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:03.106 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:24.162636+0000", 2024-09-18T10:16:03.106 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.206287267 2024-09-18T10:16:03.106 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.106 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.106 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:16:03.106 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:24.162935+0000", 2024-09-18T10:16:03.106 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00029932800000000002 2024-09-18T10:16:03.106 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.106 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.106 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:16:03.106 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:24.164750+0000", 2024-09-18T10:16:03.107 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0018148400000000001 2024-09-18T10:16:03.107 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.107 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.107 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:16:03.107 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:24.168548+0000", 2024-09-18T10:16:03.107 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0037977089999999998 2024-09-18T10:16:03.107 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.107 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.107 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:16:03.107 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:24.467969+0000", 2024-09-18T10:16:03.107 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.29942109700000002 2024-09-18T10:16:03.107 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.107 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.108 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:03.108 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:24.468164+0000", 2024-09-18T10:16:03.108 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00019534000000000001 2024-09-18T10:16:03.108 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:03.108 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:03.108 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:03.108 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.108 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.108 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:562 3.19s0 3:9fc2eacb:::smithi18925896-17:head [append 0~32768] snapc 1f=[1f,1e,1a,16,12,11,10,f] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-18T10:16:03.108 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:23.956275+0000", 2024-09-18T10:16:03.108 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 39.047415999000002, 2024-09-18T10:16:03.108 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.61260356900000001, 2024-09-18T10:16:03.108 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:03.109 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:03.109 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:16:03.109 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:16:03.109 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:03.109 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 562 2024-09-18T10:16:03.109 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.109 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:03.109 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.109 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:03.109 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956275+0000", 2024-09-18T10:16:03.109 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:03.109 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.109 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.109 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:03.110 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956275+0000", 2024-09-18T10:16:03.110 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:03.110 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.110 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.110 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:03.110 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956273+0000", 2024-09-18T10:16:03.110 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:16:03.110 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.110 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.110 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:03.110 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956355+0000", 2024-09-18T10:16:03.110 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.1661000000000003e-05 2024-09-18T10:16:03.110 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.110 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.111 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:03.111 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956366+0000", 2024-09-18T10:16:03.111 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1192000000000001e-05 2024-09-18T10:16:03.111 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.111 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.111 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:03.111 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956373+0000", 2024-09-18T10:16:03.111 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.5599999999999999e-06 2024-09-18T10:16:03.111 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.111 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.111 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:03.111 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956434+0000", 2024-09-18T10:16:03.111 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.0761999999999999e-05 2024-09-18T10:16:03.111 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.111 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.112 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:16:03.112 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956441+0000", 2024-09-18T10:16:03.112 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.1430000000000001e-06 2024-09-18T10:16:03.112 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.112 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.112 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:03.112 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:24.165954+0000", 2024-09-18T10:16:03.112 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20951336100000001 2024-09-18T10:16:03.112 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.112 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.112 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:16:03.112 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:24.166077+0000", 2024-09-18T10:16:03.112 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00012297299999999999 2024-09-18T10:16:03.112 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.113 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.113 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:16:03.113 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:24.166875+0000", 2024-09-18T10:16:03.113 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00079825999999999996 2024-09-18T10:16:03.113 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.113 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.113 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:16:03.113 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:24.168650+0000", 2024-09-18T10:16:03.113 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0017746839999999999 2024-09-18T10:16:03.113 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.113 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.113 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:16:03.113 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:24.568726+0000", 2024-09-18T10:16:03.113 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40007642799999998 2024-09-18T10:16:03.114 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.114 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.114 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:03.114 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:24.568878+0000", 2024-09-18T10:16:03.114 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000151803 2024-09-18T10:16:03.114 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:03.114 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:03.114 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:03.114 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.114 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.114 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:563 3.19s0 3:9fc2eacb:::smithi18925896-17:head [setxattr _header (57) in=7b] snapc 1f=[1f,1e,1a,16,12,11,10,f] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-18T10:16:03.114 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:23.956389+0000", 2024-09-18T10:16:03.114 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 39.047301652999998, 2024-09-18T10:16:03.115 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.61276698100000004, 2024-09-18T10:16:03.115 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:03.115 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:03.115 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:16:03.115 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:16:03.115 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:03.115 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 563 2024-09-18T10:16:03.115 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.115 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:03.115 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.115 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:03.115 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956389+0000", 2024-09-18T10:16:03.115 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:03.115 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.116 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.116 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:03.116 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956389+0000", 2024-09-18T10:16:03.116 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:03.116 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.116 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.116 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:03.116 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956387+0000", 2024-09-18T10:16:03.116 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:16:03.116 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.116 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.116 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:03.116 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956395+0000", 2024-09-18T10:16:03.117 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.537e-06 2024-09-18T10:16:03.117 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.117 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.117 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:03.117 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956405+0000", 2024-09-18T10:16:03.117 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0057999999999999e-05 2024-09-18T10:16:03.117 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.117 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.117 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:03.117 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956416+0000", 2024-09-18T10:16:03.117 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1311999999999999e-05 2024-09-18T10:16:03.117 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.117 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.118 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:03.118 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956502+0000", 2024-09-18T10:16:03.118 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.5413000000000004e-05 2024-09-18T10:16:03.118 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.118 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.118 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:16:03.118 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956512+0000", 2024-09-18T10:16:03.118 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0299000000000001e-05 2024-09-18T10:16:03.118 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.118 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.118 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:03.118 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:24.167493+0000", 2024-09-18T10:16:03.118 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.21098060699999999 2024-09-18T10:16:03.118 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.119 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.119 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:16:03.119 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:24.167618+0000", 2024-09-18T10:16:03.119 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00012526599999999999 2024-09-18T10:16:03.119 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.119 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.119 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:16:03.119 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:24.168157+0000", 2024-09-18T10:16:03.119 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00053891400000000004 2024-09-18T10:16:03.119 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.119 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.119 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:16:03.119 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:24.168870+0000", 2024-09-18T10:16:03.120 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00071273300000000001 2024-09-18T10:16:03.120 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.120 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.120 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:16:03.120 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:24.569055+0000", 2024-09-18T10:16:03.120 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40018569599999998 2024-09-18T10:16:03.120 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.120 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.120 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:03.120 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:24.569156+0000", 2024-09-18T10:16:03.120 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000100738 2024-09-18T10:16:03.120 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:03.120 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:03.120 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:03.121 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.121 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.121 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:564 3.19s0 3:9fc2eacb:::smithi18925896-17:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e68)", 2024-09-18T10:16:03.121 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:23.956428+0000", 2024-09-18T10:16:03.121 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 39.047262553000003, 2024-09-18T10:16:03.121 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.91697059800000003, 2024-09-18T10:16:03.121 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:03.121 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:16:03.121 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:16:03.121 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:16:03.121 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:03.121 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 564 2024-09-18T10:16:03.121 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.122 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:03.122 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.122 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:03.122 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956428+0000", 2024-09-18T10:16:03.122 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:03.122 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.122 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.122 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:03.122 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956428+0000", 2024-09-18T10:16:03.122 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:03.122 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.122 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.122 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:03.122 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956427+0000", 2024-09-18T10:16:03.123 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:16:03.123 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.123 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.123 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:03.123 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956432+0000", 2024-09-18T10:16:03.123 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9980000000000003e-06 2024-09-18T10:16:03.123 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.123 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.123 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:03.123 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956446+0000", 2024-09-18T10:16:03.123 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3602e-05 2024-09-18T10:16:03.123 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.123 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.123 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:03.123 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956454+0000", 2024-09-18T10:16:03.124 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.2369999999999992e-06 2024-09-18T10:16:03.124 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.124 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.124 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:03.124 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956551+0000", 2024-09-18T10:16:03.124 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.7424999999999996e-05 2024-09-18T10:16:03.124 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.124 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.124 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:16:03.124 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956558+0000", 2024-09-18T10:16:03.124 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.8050000000000001e-06 2024-09-18T10:16:03.124 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.124 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.124 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:03.125 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:24.168428+0000", 2024-09-18T10:16:03.125 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.21186939299999999 2024-09-18T10:16:03.125 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.125 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.125 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for rw locks", 2024-09-18T10:16:03.125 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:24.168531+0000", 2024-09-18T10:16:03.125 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000103707 2024-09-18T10:16:03.125 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.125 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.125 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:03.125 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:24.669928+0000", 2024-09-18T10:16:03.125 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.50139628999999997 2024-09-18T10:16:03.125 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.125 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.126 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:16:03.126 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:24.670206+0000", 2024-09-18T10:16:03.126 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00027815000000000001 2024-09-18T10:16:03.126 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.126 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.126 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:03.126 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:24.873399+0000", 2024-09-18T10:16:03.126 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20319285200000001 2024-09-18T10:16:03.126 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:03.126 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:03.126 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:03.126 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:03.126 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:03.127 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-18T10:16:03.128 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-09-18T10:16:03.129 INFO:tasks.ceph.osd.11.smithi189.stderr:2024-09-18T10:16:03.121+0000 7f718b60a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 22400) UID: 0 2024-09-18T10:16:03.228 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T10:16:03.229 INFO:tasks.ceph.osd.4.smithi029.stderr:2024-09-18T10:16:03.221+0000 7f550cf6e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22189) UID: 0 2024-09-18T10:16:03.274 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-18T10:16:03.274 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-18T10:16:03.274 INFO:teuthology.orchestra.run.smithi189.stdout: "num_ops": 0 2024-09-18T10:16:03.274 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-18T10:16:03.283 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-09-18T10:16:03.330 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-18T10:16:03.323+0000 7f14c6a1a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 22138) UID: 0 2024-09-18T10:16:03.374 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-18T10:16:03.374 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-18T10:16:03.374 INFO:teuthology.orchestra.run.smithi189.stdout: "complaint_time": 30, 2024-09-18T10:16:03.374 INFO:teuthology.orchestra.run.smithi189.stdout: "num_blocked_ops": 0 2024-09-18T10:16:03.374 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-18T10:16:03.377 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-18T10:16:03.383 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-09-18T10:16:03.402 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-18T10:16:03.425 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-18T10:16:03.430 INFO:tasks.ceph.osd.6.smithi154.stderr:2024-09-18T10:16:03.426+0000 7f0abc9e1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22089) UID: 0 2024-09-18T10:16:03.447 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-18T10:16:03.465 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-18T10:16:03.480 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-18T10:16:03.480 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 20, 2024-09-18T10:16:03.480 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 600, 2024-09-18T10:16:03.480 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [ 2024-09-18T10:16:03.480 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.480 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:146 3.es0 3:78aad6f5:::smithi18925896-49:head [append 0~442368,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:16:03.480 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:51.117496+0000", 2024-09-18T10:16:03.481 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 72.360406488999999, 2024-09-18T10:16:03.481 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.61437150200000001, 2024-09-18T10:16:03.481 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:03.481 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:03.481 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:16:03.481 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:16:03.481 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:03.481 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 146 2024-09-18T10:16:03.481 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.481 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:03.481 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.481 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:03.481 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.117496+0000", 2024-09-18T10:16:03.482 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:03.482 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.482 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.482 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:03.482 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.117496+0000", 2024-09-18T10:16:03.482 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:03.482 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.482 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.482 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:03.482 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.117493+0000", 2024-09-18T10:16:03.482 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999976 2024-09-18T10:16:03.482 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.482 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.482 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:03.483 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.117787+0000", 2024-09-18T10:16:03.483 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00029378100000000001 2024-09-18T10:16:03.483 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.483 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.483 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:03.483 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.117820+0000", 2024-09-18T10:16:03.483 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3073e-05 2024-09-18T10:16:03.483 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.483 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.483 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:03.483 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.117834+0000", 2024-09-18T10:16:03.483 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3369e-05 2024-09-18T10:16:03.483 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.484 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.484 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:03.484 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.127090+0000", 2024-09-18T10:16:03.484 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0092563769999999997 2024-09-18T10:16:03.484 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.484 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.484 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:16:03.484 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.127435+0000", 2024-09-18T10:16:03.484 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00034522300000000003 2024-09-18T10:16:03.484 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.484 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.484 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:16:03.484 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.129115+0000", 2024-09-18T10:16:03.484 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0016792980000000001 2024-09-18T10:16:03.485 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.485 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.485 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:16:03.485 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.430505+0000", 2024-09-18T10:16:03.485 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.30139002300000001 2024-09-18T10:16:03.485 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.485 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.485 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:16:03.485 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.731753+0000", 2024-09-18T10:16:03.485 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.30124798800000002 2024-09-18T10:16:03.485 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.485 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.486 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:03.486 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.731867+0000", 2024-09-18T10:16:03.486 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000114764 2024-09-18T10:16:03.486 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:03.486 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:03.486 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:03.486 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.486 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.486 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:147 3.es0 3:78aad6f5:::smithi18925896-49:head [append 0~376832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:16:03.486 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:51.122092+0000", 2024-09-18T10:16:03.486 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 72.355810070999993, 2024-09-18T10:16:03.486 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.61047882200000003, 2024-09-18T10:16:03.486 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:03.486 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:03.487 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:16:03.487 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:16:03.487 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:03.487 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 147 2024-09-18T10:16:03.487 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.487 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:03.487 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.487 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:03.487 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122092+0000", 2024-09-18T10:16:03.487 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:03.487 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.487 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.487 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:03.488 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122092+0000", 2024-09-18T10:16:03.488 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:03.488 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.488 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.488 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:03.488 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122091+0000", 2024-09-18T10:16:03.488 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:16:03.488 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.488 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.488 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:03.488 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122504+0000", 2024-09-18T10:16:03.488 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00041322100000000002 2024-09-18T10:16:03.488 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.488 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.489 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:03.489 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122527+0000", 2024-09-18T10:16:03.489 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3504999999999999e-05 2024-09-18T10:16:03.489 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.489 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.489 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:03.489 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122537+0000", 2024-09-18T10:16:03.489 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.8460000000000003e-06 2024-09-18T10:16:03.489 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.489 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.489 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:03.489 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.327415+0000", 2024-09-18T10:16:03.489 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20487799700000001 2024-09-18T10:16:03.490 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.490 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.490 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:16:03.490 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.327614+0000", 2024-09-18T10:16:03.490 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00019855400000000001 2024-09-18T10:16:03.490 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.490 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.490 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:16:03.490 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.328920+0000", 2024-09-18T10:16:03.490 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001306263 2024-09-18T10:16:03.490 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.490 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.490 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:16:03.491 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.430570+0000", 2024-09-18T10:16:03.491 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.101650508 2024-09-18T10:16:03.491 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.491 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.491 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:16:03.491 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.732477+0000", 2024-09-18T10:16:03.491 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.30190700100000001 2024-09-18T10:16:03.491 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.491 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.491 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:03.491 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.732571+0000", 2024-09-18T10:16:03.491 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.3580999999999997e-05 2024-09-18T10:16:03.491 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:03.491 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:03.492 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:03.492 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.492 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.492 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:148 3.es0 3:78aad6f5:::smithi18925896-49:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:16:03.492 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:51.122547+0000", 2024-09-18T10:16:03.492 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 72.355354978999998, 2024-09-18T10:16:03.492 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.91462763400000002, 2024-09-18T10:16:03.492 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:03.492 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:03.492 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:16:03.492 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:16:03.492 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:03.492 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 148 2024-09-18T10:16:03.493 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.493 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:03.493 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.493 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:03.493 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122547+0000", 2024-09-18T10:16:03.493 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:03.493 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.493 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.493 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:03.493 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122547+0000", 2024-09-18T10:16:03.493 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:03.493 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.493 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.493 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:03.494 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122546+0000", 2024-09-18T10:16:03.494 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:16:03.494 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.494 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.494 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:03.494 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122552+0000", 2024-09-18T10:16:03.494 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3569999999999997e-06 2024-09-18T10:16:03.494 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.494 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.494 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:03.494 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122560+0000", 2024-09-18T10:16:03.494 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.191e-06 2024-09-18T10:16:03.494 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.495 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.495 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:03.495 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122569+0000", 2024-09-18T10:16:03.495 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.8230000000000001e-06 2024-09-18T10:16:03.495 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.495 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.495 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:03.495 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.834492+0000", 2024-09-18T10:16:03.495 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.71192345599999995 2024-09-18T10:16:03.495 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.495 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.495 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:16:03.495 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.834669+0000", 2024-09-18T10:16:03.496 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000177035 2024-09-18T10:16:03.496 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.496 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.496 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:16:03.496 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.835442+0000", 2024-09-18T10:16:03.496 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00077327799999999996 2024-09-18T10:16:03.496 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.496 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.496 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:16:03.496 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.936574+0000", 2024-09-18T10:16:03.496 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.101131163 2024-09-18T10:16:03.496 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.497 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.497 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:16:03.497 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.037028+0000", 2024-09-18T10:16:03.497 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100454026 2024-09-18T10:16:03.497 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.497 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.497 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:03.497 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.037175+0000", 2024-09-18T10:16:03.497 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000147305 2024-09-18T10:16:03.497 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:03.497 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:03.497 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:03.497 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.498 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.498 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:149 3.es0 3:78aad6f5:::smithi18925896-49:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:16:03.498 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:51.122575+0000", 2024-09-18T10:16:03.498 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 72.355327771000006, 2024-09-18T10:16:03.498 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.016813872, 2024-09-18T10:16:03.498 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:03.498 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:16:03.498 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:16:03.498 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:16:03.498 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:03.498 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 149 2024-09-18T10:16:03.498 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.498 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:03.499 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.499 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:03.499 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122575+0000", 2024-09-18T10:16:03.499 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:03.499 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.499 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.499 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:03.499 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122575+0000", 2024-09-18T10:16:03.499 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:03.499 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.499 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.499 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:03.499 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122574+0000", 2024-09-18T10:16:03.500 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:16:03.500 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.500 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.500 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:03.500 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122579+0000", 2024-09-18T10:16:03.500 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3650000000000003e-06 2024-09-18T10:16:03.500 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.500 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.500 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:03.500 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122587+0000", 2024-09-18T10:16:03.500 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.3000000000000002e-06 2024-09-18T10:16:03.500 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.500 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.500 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:03.501 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122592+0000", 2024-09-18T10:16:03.501 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9080000000000003e-06 2024-09-18T10:16:03.501 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.501 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.501 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:03.501 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.935931+0000", 2024-09-18T10:16:03.501 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.81333878199999998 2024-09-18T10:16:03.501 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.501 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.501 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for rw locks", 2024-09-18T10:16:03.501 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.936216+0000", 2024-09-18T10:16:03.501 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00028463400000000003 2024-09-18T10:16:03.501 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.502 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.502 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:03.502 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.038007+0000", 2024-09-18T10:16:03.502 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.101791539 2024-09-18T10:16:03.502 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.502 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.502 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:16:03.502 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.038254+0000", 2024-09-18T10:16:03.502 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000246932 2024-09-18T10:16:03.502 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.502 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.502 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:03.502 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.139388+0000", 2024-09-18T10:16:03.502 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10113417199999999 2024-09-18T10:16:03.503 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:03.503 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:03.503 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:03.503 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.503 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.503 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpRead(3.es0 36/32 ECSubRead(tid=3, to_read={3:7255ddf4:::smithi18925896-39:head=0,4096,0}, subchunks={3:7255ddf4:::smithi18925896-39:head=[0,1]}, attrs_to_read=))", 2024-09-18T10:16:03.503 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:51.126838+0000", 2024-09-18T10:16:03.503 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 72.351064347999994, 2024-09-18T10:16:03.503 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.707585825, 2024-09-18T10:16:03.503 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:03.503 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:16:03.503 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:03.503 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.504 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:03.504 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.126838+0000", 2024-09-18T10:16:03.504 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:03.504 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.504 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.504 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:03.504 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.126838+0000", 2024-09-18T10:16:03.504 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:03.504 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.504 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.504 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:03.504 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.126838+0000", 2024-09-18T10:16:03.504 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:03.505 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.505 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.505 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:03.505 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.126838+0000", 2024-09-18T10:16:03.505 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:03.505 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.505 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.505 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:03.505 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.127004+0000", 2024-09-18T10:16:03.505 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00016626499999999999 2024-09-18T10:16:03.505 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.505 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.505 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:03.506 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.227100+0000", 2024-09-18T10:16:03.506 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.1000955 2024-09-18T10:16:03.506 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.506 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.506 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:03.506 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.834083+0000", 2024-09-18T10:16:03.506 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.60698300100000002 2024-09-18T10:16:03.506 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.506 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.506 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:03.506 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.834424+0000", 2024-09-18T10:16:03.506 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000341059 2024-09-18T10:16:03.506 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:03.507 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:03.507 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:03.507 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.507 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.507 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpReadReply(3.es0 36/32 ECSubReadReply(tid=3, attrs_read=0))", 2024-09-18T10:16:03.507 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:51.214895+0000", 2024-09-18T10:16:03.507 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 72.263007670999997, 2024-09-18T10:16:03.507 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.61857800100000004, 2024-09-18T10:16:03.507 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:03.507 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:16:03.507 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:03.507 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.507 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:03.507 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.214895+0000", 2024-09-18T10:16:03.508 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:03.508 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.508 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.508 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:03.508 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.214895+0000", 2024-09-18T10:16:03.508 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:03.508 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.508 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.508 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:03.508 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.214893+0000", 2024-09-18T10:16:03.508 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:16:03.508 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.508 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.508 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:03.509 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.214927+0000", 2024-09-18T10:16:03.509 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3841999999999999e-05 2024-09-18T10:16:03.509 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.509 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.509 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:03.509 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.214953+0000", 2024-09-18T10:16:03.509 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.6519000000000001e-05 2024-09-18T10:16:03.509 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.509 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.509 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:03.509 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.214971+0000", 2024-09-18T10:16:03.509 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7535999999999999e-05 2024-09-18T10:16:03.509 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.510 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.510 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:03.510 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.833397+0000", 2024-09-18T10:16:03.510 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.61842642599999997 2024-09-18T10:16:03.510 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.510 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.510 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:03.510 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.833473+0000", 2024-09-18T10:16:03.510 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.5621e-05 2024-09-18T10:16:03.510 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:03.510 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:03.510 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:03.510 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.511 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.511 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpReadReply(3.es0 36/32 ECSubReadReply(tid=3, attrs_read=0))", 2024-09-18T10:16:03.511 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:51.227596+0000", 2024-09-18T10:16:03.511 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 72.250306120000005, 2024-09-18T10:16:03.511 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.60597977400000003, 2024-09-18T10:16:03.511 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:03.511 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:16:03.511 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:03.511 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.511 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:03.511 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.227596+0000", 2024-09-18T10:16:03.511 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:03.511 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.512 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.512 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:03.512 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.227596+0000", 2024-09-18T10:16:03.512 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:03.512 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.512 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.512 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:03.512 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.227595+0000", 2024-09-18T10:16:03.513 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:16:03.513 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.513 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.513 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:03.513 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.227605+0000", 2024-09-18T10:16:03.513 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.7040000000000006e-06 2024-09-18T10:16:03.513 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.513 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.513 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:03.513 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.227621+0000", 2024-09-18T10:16:03.513 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6107999999999999e-05 2024-09-18T10:16:03.513 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.513 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.514 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:03.514 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.227637+0000", 2024-09-18T10:16:03.514 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5282999999999999e-05 2024-09-18T10:16:03.514 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.514 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.514 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:03.514 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.833511+0000", 2024-09-18T10:16:03.514 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.60587403699999998 2024-09-18T10:16:03.514 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.514 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.514 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:03.514 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.833576+0000", 2024-09-18T10:16:03.514 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.5392e-05 2024-09-18T10:16:03.514 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:03.515 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:03.515 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:03.515 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.515 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.515 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.es0 36/32 ECSubWriteReply(tid=4, last_complete=36'3, committed=1, applied=1))", 2024-09-18T10:16:03.515 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:51.233227+0000", 2024-09-18T10:16:03.515 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 72.244675106000003, 2024-09-18T10:16:03.515 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.39820466900000001, 2024-09-18T10:16:03.515 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:03.515 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:16:03.515 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:03.515 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.515 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:03.516 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.233227+0000", 2024-09-18T10:16:03.516 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:03.516 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.516 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.516 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:03.516 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.233227+0000", 2024-09-18T10:16:03.516 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:03.516 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.516 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.516 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:03.516 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.233226+0000", 2024-09-18T10:16:03.516 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:16:03.516 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.517 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.517 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:03.517 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.233232+0000", 2024-09-18T10:16:03.517 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9370000000000001e-06 2024-09-18T10:16:03.517 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.517 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.517 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:03.517 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.233248+0000", 2024-09-18T10:16:03.517 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5574999999999999e-05 2024-09-18T10:16:03.517 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.517 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.517 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:03.517 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.233254+0000", 2024-09-18T10:16:03.518 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.3659999999999997e-06 2024-09-18T10:16:03.518 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.518 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.518 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:03.518 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.631420+0000", 2024-09-18T10:16:03.518 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.39816591699999998 2024-09-18T10:16:03.518 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.518 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.518 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:03.518 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.631432+0000", 2024-09-18T10:16:03.518 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1737e-05 2024-09-18T10:16:03.518 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:03.518 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:03.519 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:03.519 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.519 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.519 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.es0 36/32 ECSubWriteReply(tid=5, last_complete=36'4, committed=1, applied=1))", 2024-09-18T10:16:03.519 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:51.331048+0000", 2024-09-18T10:16:03.519 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 72.146854679, 2024-09-18T10:16:03.519 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40123123900000002, 2024-09-18T10:16:03.519 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:03.519 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:16:03.519 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:03.519 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.519 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:03.519 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.331048+0000", 2024-09-18T10:16:03.520 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:03.520 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.520 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.520 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:03.520 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.331048+0000", 2024-09-18T10:16:03.520 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:03.520 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.520 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.520 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:03.520 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.331046+0000", 2024-09-18T10:16:03.520 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:16:03.520 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.520 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.521 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:03.521 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.331057+0000", 2024-09-18T10:16:03.521 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1683e-05 2024-09-18T10:16:03.521 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.521 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.521 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:03.521 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.331090+0000", 2024-09-18T10:16:03.521 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2576000000000001e-05 2024-09-18T10:16:03.521 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.521 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.521 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:03.521 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.331107+0000", 2024-09-18T10:16:03.521 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7382999999999999e-05 2024-09-18T10:16:03.521 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.522 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.522 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:03.522 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.732241+0000", 2024-09-18T10:16:03.522 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40113375200000001 2024-09-18T10:16:03.522 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.522 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.522 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:03.522 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.732279+0000", 2024-09-18T10:16:03.522 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7719000000000003e-05 2024-09-18T10:16:03.522 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:03.522 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:03.522 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:03.522 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.523 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.523 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.es0 36/32 ECSubWriteReply(tid=5, last_complete=36'4, committed=1, applied=1))", 2024-09-18T10:16:03.523 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:51.331214+0000", 2024-09-18T10:16:03.523 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 72.146688284999996, 2024-09-18T10:16:03.523 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40084109800000001, 2024-09-18T10:16:03.523 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:03.523 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:16:03.523 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:03.523 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.523 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:03.523 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.331214+0000", 2024-09-18T10:16:03.523 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:03.523 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.524 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.524 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:03.524 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.331214+0000", 2024-09-18T10:16:03.524 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:03.524 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.524 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.524 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:03.524 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.331213+0000", 2024-09-18T10:16:03.524 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:16:03.524 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.524 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.524 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:03.524 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.331220+0000", 2024-09-18T10:16:03.524 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.1369999999999996e-06 2024-09-18T10:16:03.525 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.525 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.525 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:03.525 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.331235+0000", 2024-09-18T10:16:03.525 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4849e-05 2024-09-18T10:16:03.525 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.525 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.525 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:03.525 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.331243+0000", 2024-09-18T10:16:03.525 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.157e-06 2024-09-18T10:16:03.525 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.525 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.525 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:03.526 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.732010+0000", 2024-09-18T10:16:03.526 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40076711300000001 2024-09-18T10:16:03.526 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.526 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.526 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:03.526 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.732055+0000", 2024-09-18T10:16:03.526 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4821999999999999e-05 2024-09-18T10:16:03.526 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:03.526 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:03.526 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:03.526 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.526 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.526 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:347 3.es0 3:7255ddf4:::smithi18925896-39:head [append 0~376832] snapc b=[b,4] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-18T10:16:03.527 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:02.898980+0000", 2024-09-18T10:16:03.527 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 60.578922261000002, 2024-09-18T10:16:03.527 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.50845237799999998, 2024-09-18T10:16:03.527 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:03.527 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:03.527 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:16:03.527 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:16:03.527 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:03.527 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 347 2024-09-18T10:16:03.527 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.527 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:03.527 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.527 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:03.527 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.898980+0000", 2024-09-18T10:16:03.528 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:03.528 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.528 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.528 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:03.528 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.898980+0000", 2024-09-18T10:16:03.528 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:03.528 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.528 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.528 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:03.528 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.898978+0000", 2024-09-18T10:16:03.528 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:16:03.528 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.528 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.529 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:03.529 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.899378+0000", 2024-09-18T10:16:03.529 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000399465 2024-09-18T10:16:03.529 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.529 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.529 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:03.529 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.899397+0000", 2024-09-18T10:16:03.529 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9219999999999999e-05 2024-09-18T10:16:03.529 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.529 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.529 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:03.529 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.899410+0000", 2024-09-18T10:16:03.529 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2486e-05 2024-09-18T10:16:03.529 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.530 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.530 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:03.530 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.103248+0000", 2024-09-18T10:16:03.530 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20383807200000001 2024-09-18T10:16:03.530 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.530 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.530 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:16:03.530 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.103447+0000", 2024-09-18T10:16:03.530 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00019970400000000001 2024-09-18T10:16:03.530 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.530 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.530 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:16:03.530 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.104894+0000", 2024-09-18T10:16:03.531 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001447083 2024-09-18T10:16:03.531 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.531 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.531 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:16:03.531 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.209378+0000", 2024-09-18T10:16:03.531 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.104483942 2024-09-18T10:16:03.531 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.531 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.531 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:16:03.531 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.407310+0000", 2024-09-18T10:16:03.531 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.197931406 2024-09-18T10:16:03.531 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.531 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.532 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:03.532 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.407432+0000", 2024-09-18T10:16:03.532 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00012266999999999999 2024-09-18T10:16:03.532 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:03.532 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:03.532 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:03.532 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.532 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.532 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:348 3.es0 3:7255ddf4:::smithi18925896-39:head [setxattr _header (56) in=7b] snapc b=[b,4] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-18T10:16:03.532 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:02.899423+0000", 2024-09-18T10:16:03.532 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 60.578478832999998, 2024-09-18T10:16:03.532 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.50859120400000002, 2024-09-18T10:16:03.532 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:03.532 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:03.533 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:16:03.533 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:16:03.533 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:03.533 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 348 2024-09-18T10:16:03.533 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.533 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:03.533 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.533 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:03.533 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.899423+0000", 2024-09-18T10:16:03.533 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:03.533 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.533 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.533 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:03.533 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.899423+0000", 2024-09-18T10:16:03.534 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:03.534 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.534 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.534 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:03.534 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.899422+0000", 2024-09-18T10:16:03.534 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:16:03.534 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.534 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.534 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:03.534 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.899431+0000", 2024-09-18T10:16:03.534 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.5960000000000004e-06 2024-09-18T10:16:03.534 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.534 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.534 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:03.535 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.899451+0000", 2024-09-18T10:16:03.535 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0089999999999999e-05 2024-09-18T10:16:03.535 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.535 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.535 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:03.535 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.899461+0000", 2024-09-18T10:16:03.535 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0373e-05 2024-09-18T10:16:03.535 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.535 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.535 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:03.535 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.106191+0000", 2024-09-18T10:16:03.535 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20672991700000001 2024-09-18T10:16:03.535 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.535 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.536 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:16:03.536 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.106370+0000", 2024-09-18T10:16:03.536 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00017880500000000001 2024-09-18T10:16:03.536 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.536 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.536 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:16:03.536 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.107175+0000", 2024-09-18T10:16:03.536 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00080489699999999995 2024-09-18T10:16:03.536 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.536 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.536 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:16:03.536 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.209383+0000", 2024-09-18T10:16:03.536 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10220804999999999 2024-09-18T10:16:03.537 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.537 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.537 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:16:03.537 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.407801+0000", 2024-09-18T10:16:03.537 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.19841824199999999 2024-09-18T10:16:03.537 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.537 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.537 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:03.537 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.408015+0000", 2024-09-18T10:16:03.537 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00021364100000000001 2024-09-18T10:16:03.537 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:03.537 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:03.537 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:03.537 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.538 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.538 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:349 3.es0 3:7255ddf4:::smithi18925896-39:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e47)", 2024-09-18T10:16:03.538 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:02.899474+0000", 2024-09-18T10:16:03.538 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 60.578428621999997, 2024-09-18T10:16:03.538 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.81294787599999996, 2024-09-18T10:16:03.538 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:03.538 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:16:03.538 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:16:03.538 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:16:03.538 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:03.538 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 349 2024-09-18T10:16:03.538 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.538 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:03.539 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.539 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:03.539 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.899474+0000", 2024-09-18T10:16:03.539 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:03.539 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.539 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.539 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:03.539 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.899474+0000", 2024-09-18T10:16:03.539 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:03.539 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.539 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.540 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:03.540 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.899472+0000", 2024-09-18T10:16:03.540 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:16:03.540 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.540 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.540 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:03.540 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.899480+0000", 2024-09-18T10:16:03.540 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.0129999999999993e-06 2024-09-18T10:16:03.540 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.540 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.540 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:03.540 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.899501+0000", 2024-09-18T10:16:03.540 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0959999999999999e-05 2024-09-18T10:16:03.541 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.541 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.541 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:03.541 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.899513+0000", 2024-09-18T10:16:03.541 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1736000000000001e-05 2024-09-18T10:16:03.541 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.541 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.541 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:03.541 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.208436+0000", 2024-09-18T10:16:03.541 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.30892317800000002 2024-09-18T10:16:03.541 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.541 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.541 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for rw locks", 2024-09-18T10:16:03.541 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.208762+0000", 2024-09-18T10:16:03.542 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00032566099999999998 2024-09-18T10:16:03.542 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.542 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.542 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:03.542 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.408786+0000", 2024-09-18T10:16:03.542 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.200024389 2024-09-18T10:16:03.542 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.542 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.542 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:16:03.542 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.409088+0000", 2024-09-18T10:16:03.542 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000301272 2024-09-18T10:16:03.542 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.542 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.542 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:03.543 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.712422+0000", 2024-09-18T10:16:03.543 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.303333937 2024-09-18T10:16:03.543 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:03.543 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:03.543 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:03.543 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.543 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.543 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.17s1 57/40 ECSubWrite(tid=106, reqid=client.4632.0:441, at_version=57'22, trim_to=0'0, roll_forward_to=53'18))", 2024-09-18T10:16:03.543 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:12.963189+0000", 2024-09-18T10:16:03.543 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 50.514712906, 2024-09-18T10:16:03.543 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.404840753, 2024-09-18T10:16:03.543 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:03.543 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:16:03.544 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:03.544 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.544 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:03.544 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:12.963189+0000", 2024-09-18T10:16:03.544 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:03.544 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.544 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.544 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:03.544 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:12.963189+0000", 2024-09-18T10:16:03.544 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:03.544 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.544 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.544 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:03.544 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:12.963189+0000", 2024-09-18T10:16:03.545 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:16:03.545 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.545 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.545 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:03.545 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:12.963511+0000", 2024-09-18T10:16:03.545 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00032195800000000001 2024-09-18T10:16:03.545 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.545 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.545 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:03.545 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:12.963536+0000", 2024-09-18T10:16:03.545 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5795e-05 2024-09-18T10:16:03.545 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.545 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.546 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:03.546 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:12.963548+0000", 2024-09-18T10:16:03.546 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1235e-05 2024-09-18T10:16:03.546 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.546 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.546 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:03.546 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.164194+0000", 2024-09-18T10:16:03.546 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.200646044 2024-09-18T10:16:03.546 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.546 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.546 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:16:03.546 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.164208+0000", 2024-09-18T10:16:03.546 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4107000000000001e-05 2024-09-18T10:16:03.547 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.547 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.547 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:16:03.547 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.367962+0000", 2024-09-18T10:16:03.547 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20375375800000001 2024-09-18T10:16:03.547 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.547 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.547 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:03.547 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.368030+0000", 2024-09-18T10:16:03.547 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.8664999999999994e-05 2024-09-18T10:16:03.547 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:03.547 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:03.547 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:03.547 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.548 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.548 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.17s1 57/40 ECSubWrite(tid=107, reqid=client.4632.0:442, at_version=57'23, trim_to=0'0, roll_forward_to=53'18))", 2024-09-18T10:16:03.548 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:12.963562+0000", 2024-09-18T10:16:03.548 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 50.514340187000002, 2024-09-18T10:16:03.548 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40594254800000001, 2024-09-18T10:16:03.548 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:03.548 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:16:03.548 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:03.548 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.548 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:03.548 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:12.963562+0000", 2024-09-18T10:16:03.548 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:03.548 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.548 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.549 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:03.549 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:12.963562+0000", 2024-09-18T10:16:03.549 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:03.549 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.549 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.549 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:03.549 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:12.963561+0000", 2024-09-18T10:16:03.549 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999995 2024-09-18T10:16:03.549 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.549 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.549 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:03.549 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:12.963869+0000", 2024-09-18T10:16:03.549 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00030785799999999999 2024-09-18T10:16:03.550 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.550 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.550 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:03.550 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:12.963886+0000", 2024-09-18T10:16:03.550 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6582000000000001e-05 2024-09-18T10:16:03.550 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.550 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.550 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:03.550 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.063959+0000", 2024-09-18T10:16:03.550 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10007302699999999 2024-09-18T10:16:03.550 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.550 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.550 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:03.551 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.166049+0000", 2024-09-18T10:16:03.551 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.102089845 2024-09-18T10:16:03.551 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.551 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.551 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:16:03.551 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.166064+0000", 2024-09-18T10:16:03.551 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5287000000000001e-05 2024-09-18T10:16:03.551 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.551 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.551 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:16:03.551 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.369452+0000", 2024-09-18T10:16:03.551 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20338777 2024-09-18T10:16:03.551 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.551 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.552 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:03.552 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.369505+0000", 2024-09-18T10:16:03.552 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.2868000000000003e-05 2024-09-18T10:16:03.552 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:03.552 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:03.552 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:03.552 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.552 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.552 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:495 3.es0 3:7255ddf4:::smithi18925896-39:head [rmxattr R^`al]TqifatYOiOU]p`ZegfXmNsXVaZcdd]b]YX in=40b,rmxattr TNQYR^jgR`pPaYitmrcliV[dttobncTntcfTlQkt in=40b,rmxattr W_aq[RVsaUpQtX_RpV^Vpqrg`QgQViPeVaRqZSmf in=40b,rmxattr aYfgr^q[\\aPg_aZsakfUfsdhYbra^QXjlkrpdY\\\\ in=40b,rmxattr dgXPi in=5b,rmxattr p]OS_Znl[dNa^qTn`WW[j\\dtgbNac_bijYmg in=36b,rmxattr tZ]grhlaq]kUO[k^d^dr``XNio^^Oaebpj\\[Ytdq in=40b] snapc 19=[19,18,16,12,11,10,f,e] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T10:16:03.552 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:17.889335+0000", 2024-09-18T10:16:03.552 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 45.588566954000001, 2024-09-18T10:16:03.552 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.42846536899999998, 2024-09-18T10:16:03.552 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:03.553 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:03.553 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:16:03.553 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:16:03.553 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:03.553 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 495 2024-09-18T10:16:03.553 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.553 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:03.553 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.553 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:03.553 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889335+0000", 2024-09-18T10:16:03.553 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:03.553 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.553 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.554 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:03.554 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889335+0000", 2024-09-18T10:16:03.554 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:03.554 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.554 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.554 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:03.554 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889333+0000", 2024-09-18T10:16:03.554 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999976 2024-09-18T10:16:03.554 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.554 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.554 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:03.554 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889351+0000", 2024-09-18T10:16:03.554 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7795000000000002e-05 2024-09-18T10:16:03.555 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.555 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.555 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:03.555 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889383+0000", 2024-09-18T10:16:03.555 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2740999999999997e-05 2024-09-18T10:16:03.555 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.555 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.555 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:03.555 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889396+0000", 2024-09-18T10:16:03.555 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2926e-05 2024-09-18T10:16:03.555 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.555 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.555 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:03.555 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.989725+0000", 2024-09-18T10:16:03.556 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10032845999999999 2024-09-18T10:16:03.556 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.556 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.556 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "op must wait for map", 2024-09-18T10:16:03.556 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.989751+0000", 2024-09-18T10:16:03.556 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5819000000000001e-05 2024-09-18T10:16:03.556 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.556 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.556 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:03.556 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.999619+0000", 2024-09-18T10:16:03.556 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0098683920000000001 2024-09-18T10:16:03.556 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.556 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.556 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:16:03.557 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.999776+0000", 2024-09-18T10:16:03.557 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000157336 2024-09-18T10:16:03.557 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.557 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.557 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:16:03.557 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.001374+0000", 2024-09-18T10:16:03.557 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0015982419999999999 2024-09-18T10:16:03.557 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.557 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.557 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:16:03.557 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.003908+0000", 2024-09-18T10:16:03.557 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0025331870000000001 2024-09-18T10:16:03.557 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.558 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.558 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:16:03.558 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.317708+0000", 2024-09-18T10:16:03.558 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.31380064800000002 2024-09-18T10:16:03.558 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.558 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.558 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:03.558 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.317801+0000", 2024-09-18T10:16:03.558 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.2427000000000001e-05 2024-09-18T10:16:03.558 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:03.558 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:03.558 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:03.558 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.559 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.559 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:646 3.es0 3:78aad6f5:::smithi18925896-49:head [append 0~0] snapc 21=[20,1f,1e,1a,16,11,10] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-18T10:16:03.559 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:29.784316+0000", 2024-09-18T10:16:03.559 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 33.693585872, 2024-09-18T10:16:03.559 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.38893830699999998, 2024-09-18T10:16:03.559 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:03.559 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:03.559 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:16:03.559 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:16:03.559 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:03.559 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 646 2024-09-18T10:16:03.559 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.560 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:03.560 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.560 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:03.560 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:29.784316+0000", 2024-09-18T10:16:03.560 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:03.560 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.560 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.560 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:03.560 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:29.784316+0000", 2024-09-18T10:16:03.560 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:03.560 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.560 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.560 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:03.560 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:29.784314+0000", 2024-09-18T10:16:03.561 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999976 2024-09-18T10:16:03.561 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.561 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.561 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:03.561 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:29.784325+0000", 2024-09-18T10:16:03.561 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0869e-05 2024-09-18T10:16:03.561 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.561 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.561 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:03.561 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:29.784359+0000", 2024-09-18T10:16:03.561 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4496000000000003e-05 2024-09-18T10:16:03.561 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.561 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.562 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:03.562 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:29.784374+0000", 2024-09-18T10:16:03.562 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4945e-05 2024-09-18T10:16:03.562 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.562 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.562 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:03.562 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:29.870174+0000", 2024-09-18T10:16:03.562 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.085799399999999998 2024-09-18T10:16:03.562 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.562 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.562 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:16:03.562 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:29.870387+0000", 2024-09-18T10:16:03.562 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00021327799999999999 2024-09-18T10:16:03.562 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.563 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.563 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:16:03.563 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:29.871229+0000", 2024-09-18T10:16:03.563 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00084239500000000001 2024-09-18T10:16:03.563 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.563 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.563 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:16:03.563 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.172680+0000", 2024-09-18T10:16:03.563 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.301450944 2024-09-18T10:16:03.563 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.563 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.563 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:16:03.563 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.173021+0000", 2024-09-18T10:16:03.563 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00034057199999999998 2024-09-18T10:16:03.564 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.564 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.564 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:03.564 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.173255+0000", 2024-09-18T10:16:03.564 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000233945 2024-09-18T10:16:03.564 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:03.564 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:03.564 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:03.564 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.564 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.564 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:647 3.es0 3:78aad6f5:::smithi18925896-49:head [rollback 10] snapc 21=[20,1f,1e,1a,16,11,10] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-18T10:16:03.564 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:29.784393+0000", 2024-09-18T10:16:03.564 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 33.693508958999999, 2024-09-18T10:16:03.565 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.38955653200000001, 2024-09-18T10:16:03.565 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:03.565 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:03.565 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:16:03.565 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:16:03.565 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:03.565 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 647 2024-09-18T10:16:03.565 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.565 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:03.565 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.565 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:03.565 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:29.784393+0000", 2024-09-18T10:16:03.565 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:03.565 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.566 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.566 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:03.566 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:29.784393+0000", 2024-09-18T10:16:03.566 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:03.566 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.566 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.566 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:03.566 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:29.784393+0000", 2024-09-18T10:16:03.566 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:16:03.566 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.566 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.566 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:03.566 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:29.784397+0000", 2024-09-18T10:16:03.567 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4290000000000003e-06 2024-09-18T10:16:03.567 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.567 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.567 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:03.567 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:29.784409+0000", 2024-09-18T10:16:03.567 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2208000000000001e-05 2024-09-18T10:16:03.567 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.567 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.567 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:03.567 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:29.784418+0000", 2024-09-18T10:16:03.567 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.1060000000000006e-06 2024-09-18T10:16:03.567 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.567 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.567 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:03.568 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:29.871905+0000", 2024-09-18T10:16:03.568 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.087486918999999996 2024-09-18T10:16:03.568 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.568 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.568 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:16:03.568 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:29.872024+0000", 2024-09-18T10:16:03.568 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000119152 2024-09-18T10:16:03.568 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.568 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.568 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:16:03.568 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:29.872743+0000", 2024-09-18T10:16:03.568 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00071839500000000004 2024-09-18T10:16:03.568 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.569 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.569 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:16:03.569 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.172683+0000", 2024-09-18T10:16:03.569 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.29994042399999998 2024-09-18T10:16:03.569 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.569 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.569 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:16:03.569 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.173719+0000", 2024-09-18T10:16:03.569 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0010353199999999999 2024-09-18T10:16:03.569 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.569 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.569 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:03.569 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.173950+0000", 2024-09-18T10:16:03.569 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00023138499999999999 2024-09-18T10:16:03.570 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:03.570 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:03.570 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:03.570 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.570 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.570 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:648 3.es0 3:78aad6f5:::smithi18925896-49:head [append 0~0] snapc 21=[20,1f,1e,1a,16,11,10] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-18T10:16:03.570 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:29.784506+0000", 2024-09-18T10:16:03.570 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 33.693396294999999, 2024-09-18T10:16:03.570 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.79270480700000001, 2024-09-18T10:16:03.570 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:03.570 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:03.570 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:16:03.570 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:16:03.570 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:03.571 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 648 2024-09-18T10:16:03.571 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.571 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:03.571 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.571 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:03.571 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:29.784506+0000", 2024-09-18T10:16:03.571 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:03.571 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.571 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.571 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:03.571 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:29.784506+0000", 2024-09-18T10:16:03.571 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:03.571 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.571 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.572 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:03.572 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:29.784505+0000", 2024-09-18T10:16:03.572 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:16:03.572 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.572 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.572 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:03.572 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:29.784510+0000", 2024-09-18T10:16:03.572 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9540000000000003e-06 2024-09-18T10:16:03.572 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.572 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.572 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:03.572 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:29.784523+0000", 2024-09-18T10:16:03.572 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2791e-05 2024-09-18T10:16:03.573 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.573 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.573 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:03.573 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:29.884663+0000", 2024-09-18T10:16:03.573 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10014081499999999 2024-09-18T10:16:03.573 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.573 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.573 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:03.573 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.175196+0000", 2024-09-18T10:16:03.573 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.29053214300000002 2024-09-18T10:16:03.573 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.573 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.573 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:16:03.573 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.175436+0000", 2024-09-18T10:16:03.574 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000240084 2024-09-18T10:16:03.574 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.574 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.574 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:16:03.574 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.176117+0000", 2024-09-18T10:16:03.574 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00068175400000000002 2024-09-18T10:16:03.574 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.574 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.574 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:16:03.574 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.277609+0000", 2024-09-18T10:16:03.574 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10149114100000001 2024-09-18T10:16:03.574 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.574 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.575 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:16:03.575 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.577146+0000", 2024-09-18T10:16:03.575 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.29953759600000002 2024-09-18T10:16:03.575 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.575 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.575 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:03.575 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.577211+0000", 2024-09-18T10:16:03.575 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.4630999999999998e-05 2024-09-18T10:16:03.575 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:03.575 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:03.575 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:03.575 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.575 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.575 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:694 3.es0 3:78aad6f5:::smithi18925896-49:head [stat,delete] snapc 2a=[2a,29,28,26,20,1f,1e,1a,11] ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-09-18T10:16:03.576 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:35.579838+0000", 2024-09-18T10:16:03.576 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 27.898064439999999, 2024-09-18T10:16:03.576 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40589915300000001, 2024-09-18T10:16:03.576 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:03.576 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:03.576 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:16:03.576 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:16:03.576 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:03.576 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 694 2024-09-18T10:16:03.576 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.576 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:03.576 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.576 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:03.576 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:35.579838+0000", 2024-09-18T10:16:03.577 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:03.577 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.577 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.577 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:03.577 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:35.579838+0000", 2024-09-18T10:16:03.577 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:03.577 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.577 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.577 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:03.577 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:35.579835+0000", 2024-09-18T10:16:03.577 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999971 2024-09-18T10:16:03.577 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.577 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.577 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:03.578 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:35.579849+0000", 2024-09-18T10:16:03.578 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3786e-05 2024-09-18T10:16:03.578 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.578 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.578 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:03.578 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:35.579886+0000", 2024-09-18T10:16:03.578 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7183e-05 2024-09-18T10:16:03.578 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.578 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.578 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:03.578 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:35.680021+0000", 2024-09-18T10:16:03.578 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10013464599999999 2024-09-18T10:16:03.578 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.578 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.579 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:03.579 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:35.686371+0000", 2024-09-18T10:16:03.579 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0063500789999999998 2024-09-18T10:16:03.579 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.579 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.579 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "op must wait for map", 2024-09-18T10:16:03.579 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:35.686388+0000", 2024-09-18T10:16:03.579 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7662000000000001e-05 2024-09-18T10:16:03.579 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.579 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.579 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:03.579 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:35.880602+0000", 2024-09-18T10:16:03.579 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.19421374999999999 2024-09-18T10:16:03.579 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.580 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.580 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:16:03.580 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:35.880897+0000", 2024-09-18T10:16:03.580 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00029453999999999998 2024-09-18T10:16:03.580 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.580 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.580 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:16:03.580 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:35.882269+0000", 2024-09-18T10:16:03.580 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00137261 2024-09-18T10:16:03.580 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.580 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.580 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:16:03.580 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:35.884030+0000", 2024-09-18T10:16:03.581 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0017602449999999999 2024-09-18T10:16:03.581 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.581 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.581 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:16:03.581 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:35.985612+0000", 2024-09-18T10:16:03.581 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.101582165 2024-09-18T10:16:03.581 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:03.581 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:03.581 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:03.581 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:35.985737+0000", 2024-09-18T10:16:03.581 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00012524699999999999 2024-09-18T10:16:03.581 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:03.581 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:03.581 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:03.582 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:03.582 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:03.582 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-18T10:16:03.583 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-18T10:16:03.584 INFO:teuthology.orchestra.run.smithi029.stdout:osd.6: {} 2024-09-18T10:16:03.584 INFO:teuthology.orchestra.run.smithi029.stdout:osd.7: {} 2024-09-18T10:16:03.584 INFO:teuthology.orchestra.run.smithi029.stdout:osd.8: {} 2024-09-18T10:16:03.584 INFO:teuthology.orchestra.run.smithi029.stdout:osd.9: {} 2024-09-18T10:16:03.584 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_ops_in_flight 2024-09-18T10:16:03.585 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-18T10:16:03.579+0000 7f00046a8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 22136) UID: 0 2024-09-18T10:16:03.613 INFO:teuthology.orchestra.run.smithi029.stdout:osd.10: {} 2024-09-18T10:16:03.632 INFO:teuthology.orchestra.run.smithi029.stdout:osd.11: {} 2024-09-18T10:16:03.655 INFO:teuthology.orchestra.run.smithi029.stdout:osd.12: {} 2024-09-18T10:16:03.678 INFO:teuthology.orchestra.run.smithi029.stdout:osd.13: {} 2024-09-18T10:16:03.684 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-18T10:16:03.679+0000 7f14c6a1a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 22138) UID: 0 2024-09-18T10:16:03.688 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-18T10:16:03.688 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-18T10:16:03.688 INFO:teuthology.orchestra.run.smithi189.stdout: "num_ops": 0 2024-09-18T10:16:03.688 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-18T10:16:03.698 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_blocked_ops 2024-09-18T10:16:03.703 INFO:teuthology.orchestra.run.smithi029.stdout:osd.14: {} 2024-09-18T10:16:03.722 INFO:teuthology.orchestra.run.smithi029.stdout:osd.15: {} 2024-09-18T10:16:03.725 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T10:16:03.725 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T10:16:03.725 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T10:16:03.725 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T10:16:03.725 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T10:16:03.725 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T10:16:03.725 INFO:teuthology.orchestra.run.smithi029.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T10:16:03.725 INFO:teuthology.orchestra.run.smithi029.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T10:16:03.725 INFO:teuthology.orchestra.run.smithi029.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-18T10:16:03.725 INFO:teuthology.orchestra.run.smithi029.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-18T10:16:03.726 INFO:teuthology.orchestra.run.smithi029.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-18T10:16:03.726 INFO:teuthology.orchestra.run.smithi029.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-18T10:16:03.726 INFO:teuthology.orchestra.run.smithi029.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-18T10:16:03.726 INFO:teuthology.orchestra.run.smithi029.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-18T10:16:03.726 INFO:teuthology.orchestra.run.smithi029.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-18T10:16:03.726 INFO:teuthology.orchestra.run.smithi029.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-18T10:16:03.785 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-18T10:16:03.779+0000 7f578c52f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22134) UID: 0 2024-09-18T10:16:03.791 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-18T10:16:03.791 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:16:03.795 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-18T10:16:03.801 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_historic_ops 2024-09-18T10:16:03.886 INFO:tasks.ceph.osd.10.smithi154.stderr:2024-09-18T10:16:03.882+0000 7fb73221a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 22091) UID: 0 2024-09-18T10:16:03.909 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-18T10:16:03.909 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:16:03.918 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T10:16:03.987 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-18T10:16:03.983+0000 7f1981873700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22132) UID: 0 2024-09-18T10:16:04.018 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:16:04.018 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:16:04.028 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T10:16:04.088 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-18T10:16:04.081+0000 7f302f2c3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22187) UID: 0 2024-09-18T10:16:04.126 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:16:04.126 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:16:04.136 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T10:16:04.189 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-18T10:16:04.185+0000 7f14dcc5b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22398) UID: 0 2024-09-18T10:16:04.282 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:16:04.282 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:16:04.290 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-18T10:16:04.283+0000 7f1981873700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22132) UID: 0 2024-09-18T10:16:04.291 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T10:16:04.388 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:16:04.388 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:16:04.391 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-18T10:16:04.387+0000 7f578c52f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22134) UID: 0 2024-09-18T10:16:04.398 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T10:16:04.491 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-18T10:16:04.485+0000 7f302f2c3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22187) UID: 0 2024-09-18T10:16:04.501 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:16:04.501 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:16:04.511 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T10:16:04.593 INFO:tasks.ceph.osd.3.smithi189.stderr:2024-09-18T10:16:04.585+0000 7f2440fc7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22396) UID: 0 2024-09-18T10:16:04.659 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:16:04.659 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:16:04.669 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-09-18T10:16:04.694 INFO:tasks.ceph.osd.6.smithi154.stderr:2024-09-18T10:16:04.690+0000 7f0abc9e1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22089) UID: 0 2024-09-18T10:16:04.770 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:16:04.771 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:16:04.780 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-09-18T10:16:04.797 INFO:teuthology.orchestra.run.smithi029.stderr:nodeep-scrub is unset 2024-09-18T10:16:04.797 INFO:tasks.ceph.osd.12.smithi029.stderr:2024-09-18T10:16:04.793+0000 7f8e4e627700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 22190) UID: 0 2024-09-18T10:16:04.876 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:16:04.876 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:16:04.886 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-09-18T10:16:04.895 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-18T10:16:04.891+0000 7f14c6a1a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 22138) UID: 0 2024-09-18T10:16:04.984 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:16:04.984 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:16:04.993 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_ops_in_flight 2024-09-18T10:16:04.997 INFO:tasks.ceph.osd.4.smithi029.stderr:2024-09-18T10:16:04.989+0000 7f550cf6e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22189) UID: 0 2024-09-18T10:16:05.092 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:16:05.092 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:16:05.097 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-09-18T10:16:05.094+0000 7fdfa8698700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22087) UID: 0 2024-09-18T10:16:05.102 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_blocked_ops 2024-09-18T10:16:05.198 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-18T10:16:05.195+0000 7f1981873700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22132) UID: 0 2024-09-18T10:16:05.202 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:16:05.202 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:16:05.212 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_historic_ops 2024-09-18T10:16:05.299 INFO:tasks.ceph.osd.4.smithi029.stderr:2024-09-18T10:16:05.293+0000 7f550cf6e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22189) UID: 0 2024-09-18T10:16:05.314 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:16:05.314 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:16:05.323 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T10:16:05.399 INFO:tasks.ceph.osd.8.smithi029.stderr:2024-09-18T10:16:05.397+0000 7f2555c27700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 22192) UID: 0 2024-09-18T10:16:05.430 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-18T10:16:05.430 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:16:05.440 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T10:16:05.500 INFO:tasks.ceph.osd.6.smithi154.stderr:2024-09-18T10:16:05.498+0000 7f0abc9e1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22089) UID: 0 2024-09-18T10:16:05.534 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-18T10:16:05.534 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:16:05.543 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T10:16:05.601 INFO:tasks.ceph.osd.14.smithi154.stderr:2024-09-18T10:16:05.598+0000 7f75a19e7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 22093) UID: 0 2024-09-18T10:16:05.643 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-18T10:16:05.643 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:16:05.652 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T10:16:05.702 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-18T10:16:05.699+0000 7f00046a8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 22136) UID: 0 2024-09-18T10:16:05.737 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T10:16:05.751 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-18T10:16:05.751 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:16:05.760 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T10:16:05.780 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7, 9, 10, 11, 12, 13, 14, 15, 8] out_osds: [] dead_osds: [] live_osds: [0, 4, 8, 12, 1, 5, 9, 13, 2, 6, 10, 14, 3, 7, 11, 15] 2024-09-18T10:16:05.780 INFO:tasks.thrashosds.thrasher:choose_action: min_in 8 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T10:16:05.780 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-18T10:16:05.780 INFO:tasks.thrashosds.thrasher:Setting osd 14 primary_affinity to 1.000000 2024-09-18T10:16:05.780 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 14 1 2024-09-18T10:16:05.802 INFO:tasks.ceph.osd.6.smithi154.stderr:2024-09-18T10:16:05.798+0000 7f0abc9e1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22089) UID: 0 2024-09-18T10:16:05.857 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-18T10:16:05.857 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:16:05.866 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T10:16:05.884 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-18T10:16:05.903 INFO:tasks.ceph.osd.3.smithi189.stderr:2024-09-18T10:16:05.897+0000 7f2440fc7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22396) UID: 0 2024-09-18T10:16:05.905 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-18T10:16:05.934 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-18T10:16:05.961 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-18T10:16:05.962 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-18T10:16:05.962 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:16:05.971 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-09-18T10:16:05.978 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-18T10:16:06.004 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-09-18T10:16:06.002+0000 7fdfa8698700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22087) UID: 0 2024-09-18T10:16:06.062 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-18T10:16:06.063 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:16:06.071 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-09-18T10:16:06.091 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-18T10:16:06.105 INFO:tasks.ceph.osd.10.smithi154.stderr:2024-09-18T10:16:06.102+0000 7fb73221a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 22091) UID: 0 2024-09-18T10:16:06.119 INFO:teuthology.orchestra.run.smithi029.stdout:osd.6: {} 2024-09-18T10:16:06.141 INFO:teuthology.orchestra.run.smithi029.stdout:osd.7: {} 2024-09-18T10:16:06.163 INFO:teuthology.orchestra.run.smithi029.stdout:osd.8: {} 2024-09-18T10:16:06.172 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-18T10:16:06.172 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:16:06.181 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-09-18T10:16:06.187 INFO:teuthology.orchestra.run.smithi029.stdout:osd.9: {} 2024-09-18T10:16:06.206 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-18T10:16:06.201+0000 7f14dcc5b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22398) UID: 0 2024-09-18T10:16:06.210 INFO:teuthology.orchestra.run.smithi029.stdout:osd.10: {} 2024-09-18T10:16:06.235 INFO:teuthology.orchestra.run.smithi029.stdout:osd.11: {} 2024-09-18T10:16:06.258 INFO:teuthology.orchestra.run.smithi029.stdout:osd.12: {} 2024-09-18T10:16:06.278 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-18T10:16:06.278 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-18T10:16:06.278 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-18T10:16:06.278 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-18T10:16:06.279 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.279 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.ds4 41/32 ECSubWrite(tid=41, reqid=osd.13.0:41, at_version=41'6, trim_to=0'0, roll_forward_to=37'4))", 2024-09-18T10:16:06.279 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:57.138491+0000", 2024-09-18T10:16:06.279 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 69.137711386999996, 2024-09-18T10:16:06.279 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.200128009, 2024-09-18T10:16:06.279 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:06.279 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:16:06.279 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:06.279 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.279 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:06.279 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.138491+0000", 2024-09-18T10:16:06.279 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:06.279 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.279 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.280 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:06.280 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.138491+0000", 2024-09-18T10:16:06.280 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:06.280 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.280 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.280 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:06.280 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.138490+0000", 2024-09-18T10:16:06.280 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:16:06.280 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.280 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.280 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:06.280 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.138555+0000", 2024-09-18T10:16:06.280 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4571000000000002e-05 2024-09-18T10:16:06.281 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.281 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.281 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:06.281 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.138581+0000", 2024-09-18T10:16:06.281 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6069000000000001e-05 2024-09-18T10:16:06.281 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.281 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.281 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:06.281 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.138598+0000", 2024-09-18T10:16:06.281 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7059e-05 2024-09-18T10:16:06.281 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.281 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.281 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:06.282 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.336697+0000", 2024-09-18T10:16:06.282 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.198099517 2024-09-18T10:16:06.282 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.282 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.282 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:16:06.282 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.336733+0000", 2024-09-18T10:16:06.282 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5369e-05 2024-09-18T10:16:06.282 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.282 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.282 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:16:06.282 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.338552+0000", 2024-09-18T10:16:06.282 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001819773 2024-09-18T10:16:06.282 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.283 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.283 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:06.283 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.338619+0000", 2024-09-18T10:16:06.283 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6390999999999995e-05 2024-09-18T10:16:06.283 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:06.283 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:06.283 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:06.283 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.283 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.283 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.ds4 41/32 ECSubWrite(tid=43, reqid=osd.13.0:41, at_version=0'0, trim_to=0'0, roll_forward_to=41'6))", 2024-09-18T10:16:06.283 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:57.441404+0000", 2024-09-18T10:16:06.283 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 68.834797777999995, 2024-09-18T10:16:06.283 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30108198600000002, 2024-09-18T10:16:06.283 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:06.283 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:16:06.284 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:06.284 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.284 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:06.284 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.441404+0000", 2024-09-18T10:16:06.284 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:06.284 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.284 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.284 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:06.284 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.441404+0000", 2024-09-18T10:16:06.284 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:06.284 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.284 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.284 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:06.285 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.441403+0000", 2024-09-18T10:16:06.285 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999981 2024-09-18T10:16:06.285 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.285 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.285 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:06.285 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.441435+0000", 2024-09-18T10:16:06.285 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2502000000000002e-05 2024-09-18T10:16:06.285 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.285 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.285 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:06.285 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.441473+0000", 2024-09-18T10:16:06.285 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8302e-05 2024-09-18T10:16:06.285 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.286 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.286 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:06.286 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.441524+0000", 2024-09-18T10:16:06.286 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0679e-05 2024-09-18T10:16:06.286 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.286 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.286 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:06.286 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.641739+0000", 2024-09-18T10:16:06.286 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20021524299999999 2024-09-18T10:16:06.286 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.286 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.286 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:16:06.286 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.641766+0000", 2024-09-18T10:16:06.286 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7081999999999999e-05 2024-09-18T10:16:06.287 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.287 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.287 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:16:06.287 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.742432+0000", 2024-09-18T10:16:06.287 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10066531300000001 2024-09-18T10:16:06.287 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.287 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.287 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:06.287 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.742486+0000", 2024-09-18T10:16:06.287 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4691999999999997e-05 2024-09-18T10:16:06.287 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:06.287 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:06.287 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:06.287 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.288 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.288 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:509 3.16s0 3:6c64c4fa:::smithi18925896-41:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e62)", 2024-09-18T10:16:06.288 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:17.914565+0000", 2024-09-18T10:16:06.288 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 48.361637170999998, 2024-09-18T10:16:06.288 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.059666403, 2024-09-18T10:16:06.288 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:06.288 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-18T10:16:06.288 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:16:06.288 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:16:06.288 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:06.288 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 509 2024-09-18T10:16:06.288 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.288 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:06.289 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.289 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:06.289 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.914565+0000", 2024-09-18T10:16:06.289 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:06.289 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.289 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.289 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:06.289 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.914565+0000", 2024-09-18T10:16:06.289 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:06.289 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.289 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.289 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:06.289 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.914564+0000", 2024-09-18T10:16:06.290 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:16:06.290 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.290 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.290 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:06.290 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.914570+0000", 2024-09-18T10:16:06.290 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3940000000000001e-06 2024-09-18T10:16:06.290 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.290 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.290 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:06.290 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.914585+0000", 2024-09-18T10:16:06.290 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4348000000000001e-05 2024-09-18T10:16:06.290 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.290 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.291 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:06.291 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.914591+0000", 2024-09-18T10:16:06.291 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4450000000000001e-06 2024-09-18T10:16:06.291 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.291 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.291 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:06.291 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.101342+0000", 2024-09-18T10:16:06.291 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.186750576 2024-09-18T10:16:06.291 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.291 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.291 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-18T10:16:06.291 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.101343+0000", 2024-09-18T10:16:06.291 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4300000000000001e-07 2024-09-18T10:16:06.291 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.292 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.292 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:06.292 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.334525+0000", 2024-09-18T10:16:06.292 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2331822159999999 2024-09-18T10:16:06.292 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.292 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.292 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for missing object", 2024-09-18T10:16:06.292 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.334706+0000", 2024-09-18T10:16:06.292 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00018068500000000001 2024-09-18T10:16:06.292 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.292 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.292 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:06.292 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.838318+0000", 2024-09-18T10:16:06.293 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.50361270300000005 2024-09-18T10:16:06.293 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.293 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.293 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2024-09-18T10:16:06.293 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.838493+0000", 2024-09-18T10:16:06.293 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017465199999999999 2024-09-18T10:16:06.293 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.293 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.293 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:06.293 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.972913+0000", 2024-09-18T10:16:06.293 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.134419977 2024-09-18T10:16:06.293 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.293 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.293 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:16:06.294 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.973261+0000", 2024-09-18T10:16:06.294 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00034786399999999997 2024-09-18T10:16:06.294 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.294 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.294 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:06.294 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.974231+0000", 2024-09-18T10:16:06.294 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00097053099999999995 2024-09-18T10:16:06.294 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:06.294 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:06.294 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:06.294 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.294 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.294 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDScrubReserve(3.6s5 REQUEST e63)", 2024-09-18T10:16:06.295 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:19.068302+0000", 2024-09-18T10:16:06.295 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 47.207900643000002, 2024-09-18T10:16:06.295 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.26520614300000001, 2024-09-18T10:16:06.295 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:06.295 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-18T10:16:06.295 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:06.295 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.295 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:06.295 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.068302+0000", 2024-09-18T10:16:06.295 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:06.295 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.295 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.295 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:06.295 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.068302+0000", 2024-09-18T10:16:06.296 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:06.296 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.296 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.296 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:06.296 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.068300+0000", 2024-09-18T10:16:06.296 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999981 2024-09-18T10:16:06.296 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.296 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.296 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:06.296 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.068311+0000", 2024-09-18T10:16:06.296 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1005999999999999e-05 2024-09-18T10:16:06.296 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.296 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.297 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:06.297 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.068345+0000", 2024-09-18T10:16:06.297 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3983000000000003e-05 2024-09-18T10:16:06.297 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.297 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.297 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:06.297 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.068372+0000", 2024-09-18T10:16:06.297 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7385999999999998e-05 2024-09-18T10:16:06.297 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.297 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.297 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:06.297 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.333409+0000", 2024-09-18T10:16:06.297 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.26503692400000001 2024-09-18T10:16:06.297 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.298 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.298 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:16:06.298 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.333444+0000", 2024-09-18T10:16:06.298 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4916000000000002e-05 2024-09-18T10:16:06.298 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.298 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.298 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:06.298 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.333508+0000", 2024-09-18T10:16:06.298 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3658000000000006e-05 2024-09-18T10:16:06.298 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:06.298 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:06.298 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:06.298 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.298 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.299 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDPGPushReply(3.16s0 63/62 [PushReplyOp(3:6c64c4fa:::smithi18925896-41:head)])", 2024-09-18T10:16:06.299 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:19.524127+0000", 2024-09-18T10:16:06.299 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 46.752075208000001, 2024-09-18T10:16:06.299 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20108187499999999, 2024-09-18T10:16:06.299 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:06.299 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:16:06.299 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:06.299 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.299 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:06.299 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.524127+0000", 2024-09-18T10:16:06.299 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:06.299 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.299 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.299 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:06.300 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.524127+0000", 2024-09-18T10:16:06.300 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:06.300 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.300 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.300 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:06.300 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.524117+0000", 2024-09-18T10:16:06.300 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999905 2024-09-18T10:16:06.300 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.300 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.300 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:06.300 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.524143+0000", 2024-09-18T10:16:06.300 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5534999999999999e-05 2024-09-18T10:16:06.300 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.300 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.301 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:06.301 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.524209+0000", 2024-09-18T10:16:06.301 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6192000000000006e-05 2024-09-18T10:16:06.301 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.301 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.301 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:06.301 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.524227+0000", 2024-09-18T10:16:06.301 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7986e-05 2024-09-18T10:16:06.301 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.301 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.301 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:06.301 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.724925+0000", 2024-09-18T10:16:06.301 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.200697662 2024-09-18T10:16:06.301 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.302 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.302 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:06.302 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.725209+0000", 2024-09-18T10:16:06.302 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000284102 2024-09-18T10:16:06.302 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:06.302 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:06.302 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:06.302 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.302 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.302 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDPGPushReply(3.16s0 63/62 [PushReplyOp(3:6c64c4fa:::smithi18925896-41:head)])", 2024-09-18T10:16:06.302 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:19.524280+0000", 2024-09-18T10:16:06.302 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 46.751921865, 2024-09-18T10:16:06.302 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.201340348, 2024-09-18T10:16:06.302 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:06.303 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:16:06.303 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:06.303 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.303 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:06.303 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.524280+0000", 2024-09-18T10:16:06.303 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:06.303 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.303 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.303 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:06.303 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.524280+0000", 2024-09-18T10:16:06.303 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:06.303 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.303 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.303 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:06.304 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.524274+0000", 2024-09-18T10:16:06.304 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999933 2024-09-18T10:16:06.304 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.304 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.304 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:06.304 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.524289+0000", 2024-09-18T10:16:06.304 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5722999999999998e-05 2024-09-18T10:16:06.304 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.304 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.304 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:06.304 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.524315+0000", 2024-09-18T10:16:06.304 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5188999999999999e-05 2024-09-18T10:16:06.304 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.304 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.305 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:06.305 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.524325+0000", 2024-09-18T10:16:06.305 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0736e-05 2024-09-18T10:16:06.305 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.305 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.305 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:06.305 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.725489+0000", 2024-09-18T10:16:06.305 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.201163377 2024-09-18T10:16:06.305 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.305 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.305 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:06.305 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.725621+0000", 2024-09-18T10:16:06.305 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013202600000000001 2024-09-18T10:16:06.306 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:06.306 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:06.306 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:06.306 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.306 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.306 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDPGPushReply(3.16s0 63/62 [PushReplyOp(3:6c64c4fa:::smithi18925896-41:head)])", 2024-09-18T10:16:06.306 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:19.524382+0000", 2024-09-18T10:16:06.306 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 46.751820211000002, 2024-09-18T10:16:06.306 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.21126635699999999, 2024-09-18T10:16:06.306 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:06.306 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:16:06.306 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:06.306 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.306 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:06.307 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.524382+0000", 2024-09-18T10:16:06.307 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:06.307 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.307 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.307 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:06.307 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.524382+0000", 2024-09-18T10:16:06.307 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:06.307 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.307 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.307 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:06.307 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.524381+0000", 2024-09-18T10:16:06.307 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999995 2024-09-18T10:16:06.307 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.307 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.308 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:06.308 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.524388+0000", 2024-09-18T10:16:06.308 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4860000000000002e-06 2024-09-18T10:16:06.308 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.308 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.308 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:06.308 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.524420+0000", 2024-09-18T10:16:06.308 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2623000000000003e-05 2024-09-18T10:16:06.308 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.308 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.308 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:06.308 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.624550+0000", 2024-09-18T10:16:06.308 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100129128 2024-09-18T10:16:06.309 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.309 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.309 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:06.309 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.726236+0000", 2024-09-18T10:16:06.309 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10168595699999999 2024-09-18T10:16:06.309 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.309 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.309 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:06.309 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.735648+0000", 2024-09-18T10:16:06.309 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0094127829999999992 2024-09-18T10:16:06.309 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:06.309 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:06.309 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:06.309 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.310 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.310 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.10s6 73/46 ECSubWrite(tid=194, reqid=client.4632.0:626, at_version=73'16, trim_to=0'0, roll_forward_to=60'14))", 2024-09-18T10:16:06.310 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:29.000022+0000", 2024-09-18T10:16:06.310 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 37.276180637000003, 2024-09-18T10:16:06.310 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20370728099999999, 2024-09-18T10:16:06.310 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:06.310 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:16:06.310 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:06.310 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.310 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:06.310 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.000022+0000", 2024-09-18T10:16:06.310 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:06.310 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.311 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.311 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:06.311 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.000022+0000", 2024-09-18T10:16:06.311 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:06.311 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.311 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.311 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:06.311 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.000020+0000", 2024-09-18T10:16:06.311 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999981 2024-09-18T10:16:06.311 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.311 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.311 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:06.311 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.000428+0000", 2024-09-18T10:16:06.312 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000408689 2024-09-18T10:16:06.312 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.312 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.312 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:06.312 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.000464+0000", 2024-09-18T10:16:06.312 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5768e-05 2024-09-18T10:16:06.312 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.312 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.312 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:06.312 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.100621+0000", 2024-09-18T10:16:06.312 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100157223 2024-09-18T10:16:06.312 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.312 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.313 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:06.313 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.200904+0000", 2024-09-18T10:16:06.313 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10028234900000001 2024-09-18T10:16:06.313 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.313 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.313 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:16:06.313 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.200921+0000", 2024-09-18T10:16:06.313 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7217000000000001e-05 2024-09-18T10:16:06.313 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.313 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.313 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:16:06.313 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.203672+0000", 2024-09-18T10:16:06.313 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0027511879999999999 2024-09-18T10:16:06.313 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.314 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.314 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:06.314 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.203729+0000", 2024-09-18T10:16:06.314 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6823999999999997e-05 2024-09-18T10:16:06.314 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:06.314 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:06.314 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:06.314 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.314 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.314 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.10s6 73/46 ECSubWrite(tid=195, reqid=client.4632.0:627, at_version=73'17, trim_to=0'0, roll_forward_to=60'14))", 2024-09-18T10:16:06.314 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:29.103724+0000", 2024-09-18T10:16:06.314 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 37.172478554000001, 2024-09-18T10:16:06.314 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.202940394, 2024-09-18T10:16:06.315 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:06.315 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:16:06.315 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:06.315 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.315 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:06.315 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.103724+0000", 2024-09-18T10:16:06.315 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:06.315 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.315 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.315 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:06.315 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.103724+0000", 2024-09-18T10:16:06.315 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:06.315 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.315 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.316 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:06.316 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.103722+0000", 2024-09-18T10:16:06.316 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999981 2024-09-18T10:16:06.316 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.316 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.316 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:06.316 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.104024+0000", 2024-09-18T10:16:06.316 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00030194100000000001 2024-09-18T10:16:06.316 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.316 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.316 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:06.316 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.104067+0000", 2024-09-18T10:16:06.316 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2840999999999999e-05 2024-09-18T10:16:06.316 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.317 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.317 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:06.317 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.204203+0000", 2024-09-18T10:16:06.317 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100136243 2024-09-18T10:16:06.317 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.317 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.317 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:06.317 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.304498+0000", 2024-09-18T10:16:06.317 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100294809 2024-09-18T10:16:06.317 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.317 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.317 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:16:06.317 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.304514+0000", 2024-09-18T10:16:06.317 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6163000000000001e-05 2024-09-18T10:16:06.318 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.318 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.318 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:16:06.318 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.306599+0000", 2024-09-18T10:16:06.318 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0020848759999999998 2024-09-18T10:16:06.318 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.318 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.318 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:06.318 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.306664+0000", 2024-09-18T10:16:06.318 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5331000000000002e-05 2024-09-18T10:16:06.318 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:06.318 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:06.318 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:06.318 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.319 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.319 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.6s5 75/40 ECSubWrite(tid=66, reqid=osd.0.0:66, at_version=75'42, trim_to=0'0, roll_forward_to=74'40))", 2024-09-18T10:16:06.319 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:30.668776+0000", 2024-09-18T10:16:06.319 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 35.607425892999998, 2024-09-18T10:16:06.319 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30218223300000002, 2024-09-18T10:16:06.319 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:06.319 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:16:06.319 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:06.319 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.319 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:06.319 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:30.668776+0000", 2024-09-18T10:16:06.319 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:06.319 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.319 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.320 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:06.320 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:30.668776+0000", 2024-09-18T10:16:06.320 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:06.320 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.320 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.320 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:06.320 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:30.668775+0000", 2024-09-18T10:16:06.320 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999986 2024-09-18T10:16:06.320 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.320 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.320 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:06.320 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:30.668821+0000", 2024-09-18T10:16:06.320 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6147999999999999e-05 2024-09-18T10:16:06.321 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.321 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.321 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:06.321 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:30.668864+0000", 2024-09-18T10:16:06.321 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3004999999999999e-05 2024-09-18T10:16:06.321 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.321 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.321 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:06.321 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:30.768961+0000", 2024-09-18T10:16:06.321 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10009678499999999 2024-09-18T10:16:06.321 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.321 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.321 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:06.321 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:30.969322+0000", 2024-09-18T10:16:06.322 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20036179000000001 2024-09-18T10:16:06.322 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.322 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.322 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:16:06.322 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:30.969349+0000", 2024-09-18T10:16:06.322 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6795999999999999e-05 2024-09-18T10:16:06.322 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.322 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.322 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:16:06.322 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:30.970891+0000", 2024-09-18T10:16:06.322 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0015418459999999999 2024-09-18T10:16:06.322 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.322 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.323 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:06.323 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:30.970959+0000", 2024-09-18T10:16:06.323 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7490000000000006e-05 2024-09-18T10:16:06.323 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:06.323 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:06.323 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:06.323 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.323 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.323 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:784 3.16s0 3:6c64c4fa:::smithi18925896-41:head [append 0~557056 in=557056b] snapc 2f=[2f,29,28,20,1f,1e,1a,11] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-18T10:16:06.323 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:40.660850+0000", 2024-09-18T10:16:06.323 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 25.615352224999999, 2024-09-18T10:16:06.323 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.97869106100000003, 2024-09-18T10:16:06.323 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:06.323 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:06.324 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:16:06.324 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:16:06.324 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:06.324 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 784 2024-09-18T10:16:06.324 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.324 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:06.324 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.324 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:06.324 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.660850+0000", 2024-09-18T10:16:06.324 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:06.324 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.324 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.324 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:06.325 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.660850+0000", 2024-09-18T10:16:06.325 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:06.325 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.325 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.325 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:06.325 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.660847+0000", 2024-09-18T10:16:06.325 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999976 2024-09-18T10:16:06.325 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.325 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.325 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:06.325 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.661846+0000", 2024-09-18T10:16:06.325 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00099824000000000006 2024-09-18T10:16:06.325 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.325 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.326 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:06.326 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.661878+0000", 2024-09-18T10:16:06.326 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2224000000000002e-05 2024-09-18T10:16:06.326 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.326 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.326 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:06.326 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.761995+0000", 2024-09-18T10:16:06.326 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100117568 2024-09-18T10:16:06.326 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.326 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.326 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:06.326 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.762894+0000", 2024-09-18T10:16:06.327 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00089831700000000004 2024-09-18T10:16:06.327 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.327 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.327 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for degraded object", 2024-09-18T10:16:06.327 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.764145+0000", 2024-09-18T10:16:06.327 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001251203 2024-09-18T10:16:06.327 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.327 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.327 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:06.327 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:41.428859+0000", 2024-09-18T10:16:06.327 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.664713848 2024-09-18T10:16:06.327 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.327 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.327 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:16:06.328 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:41.429140+0000", 2024-09-18T10:16:06.328 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00028127799999999999 2024-09-18T10:16:06.328 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.328 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.328 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:16:06.328 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:41.431693+0000", 2024-09-18T10:16:06.328 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0025525169999999998 2024-09-18T10:16:06.328 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.328 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.328 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:16:06.328 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:41.538557+0000", 2024-09-18T10:16:06.328 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.106864548 2024-09-18T10:16:06.328 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.329 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.329 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:16:06.329 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:41.639179+0000", 2024-09-18T10:16:06.329 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100622194 2024-09-18T10:16:06.329 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.329 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.329 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:06.329 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:41.639541+0000", 2024-09-18T10:16:06.329 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00036170400000000002 2024-09-18T10:16:06.329 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:06.329 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:06.329 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:06.329 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.329 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.330 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWriteReply(3.16s0 85/82 ECSubWriteReply(tid=17, last_complete=85'28, committed=1, applied=1))", 2024-09-18T10:16:06.330 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:41.437048+0000", 2024-09-18T10:16:06.330 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 24.839154569000002, 2024-09-18T10:16:06.330 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20252846199999999, 2024-09-18T10:16:06.330 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:06.330 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:16:06.330 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:06.330 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.330 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:06.330 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:41.437048+0000", 2024-09-18T10:16:06.330 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:06.330 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.330 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.330 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:06.331 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:41.437048+0000", 2024-09-18T10:16:06.331 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:06.331 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.331 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.331 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:06.331 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:41.437047+0000", 2024-09-18T10:16:06.331 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:16:06.331 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.331 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.331 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:06.331 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:41.437054+0000", 2024-09-18T10:16:06.331 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1670000000000002e-06 2024-09-18T10:16:06.331 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.331 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.332 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:06.332 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:41.437068+0000", 2024-09-18T10:16:06.332 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3628999999999999e-05 2024-09-18T10:16:06.332 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.332 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.332 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:06.332 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:41.437074+0000", 2024-09-18T10:16:06.332 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5860000000000001e-06 2024-09-18T10:16:06.332 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.332 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.332 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:06.332 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:41.639023+0000", 2024-09-18T10:16:06.332 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.201948668 2024-09-18T10:16:06.332 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.333 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.333 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:06.333 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:41.639576+0000", 2024-09-18T10:16:06.333 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00055334500000000001 2024-09-18T10:16:06.333 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:06.333 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:06.333 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:06.333 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.333 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.333 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWriteReply(3.16s0 85/82 ECSubWriteReply(tid=17, last_complete=85'28, committed=1, applied=1))", 2024-09-18T10:16:06.333 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:41.437537+0000", 2024-09-18T10:16:06.333 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 24.838664787999999, 2024-09-18T10:16:06.333 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.201442223, 2024-09-18T10:16:06.334 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:06.334 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:16:06.334 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:06.334 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.334 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:06.334 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:41.437537+0000", 2024-09-18T10:16:06.334 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:06.334 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.334 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.334 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:06.334 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:41.437537+0000", 2024-09-18T10:16:06.334 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:06.334 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.334 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.334 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:06.335 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:41.437537+0000", 2024-09-18T10:16:06.335 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:16:06.335 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.335 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.335 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:06.335 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:41.437542+0000", 2024-09-18T10:16:06.335 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5310000000000001e-06 2024-09-18T10:16:06.335 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.335 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.335 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:06.335 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:41.437554+0000", 2024-09-18T10:16:06.335 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2381000000000001e-05 2024-09-18T10:16:06.335 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.335 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.336 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:06.336 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:41.437564+0000", 2024-09-18T10:16:06.336 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8700000000000004e-06 2024-09-18T10:16:06.336 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.336 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.336 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:06.336 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:41.638935+0000", 2024-09-18T10:16:06.336 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20137052899999999 2024-09-18T10:16:06.336 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.336 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.336 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:06.336 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:41.638980+0000", 2024-09-18T10:16:06.336 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4743999999999997e-05 2024-09-18T10:16:06.337 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:06.337 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:06.337 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:06.337 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.337 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.337 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWriteReply(3.16s0 85/82 ECSubWriteReply(tid=19, last_complete=85'30, committed=1, applied=1))", 2024-09-18T10:16:06.337 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:41.438463+0000", 2024-09-18T10:16:06.337 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 24.837738854000001, 2024-09-18T10:16:06.337 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30126321900000003, 2024-09-18T10:16:06.337 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:06.337 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:16:06.337 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:06.337 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.337 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:06.338 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:41.438463+0000", 2024-09-18T10:16:06.338 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:06.338 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.338 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.338 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:06.338 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:41.438463+0000", 2024-09-18T10:16:06.338 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:06.338 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.338 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.338 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:06.338 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:41.438463+0000", 2024-09-18T10:16:06.338 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999995 2024-09-18T10:16:06.338 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.338 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.338 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:06.339 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:41.438468+0000", 2024-09-18T10:16:06.339 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9529999999999998e-06 2024-09-18T10:16:06.339 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.339 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.339 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:06.339 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:41.438479+0000", 2024-09-18T10:16:06.339 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1167e-05 2024-09-18T10:16:06.339 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.339 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.339 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:06.339 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:41.538570+0000", 2024-09-18T10:16:06.339 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10009143400000001 2024-09-18T10:16:06.339 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.340 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.340 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:06.340 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:41.739692+0000", 2024-09-18T10:16:06.340 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20112211599999999 2024-09-18T10:16:06.340 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.340 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.340 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:06.340 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:41.739727+0000", 2024-09-18T10:16:06.340 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4236999999999998e-05 2024-09-18T10:16:06.340 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:06.340 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:06.340 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:06.340 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.341 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.341 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWriteReply(3.16s0 85/82 ECSubWriteReply(tid=18, last_complete=85'29, committed=1, applied=1))", 2024-09-18T10:16:06.341 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:41.438999+0000", 2024-09-18T10:16:06.341 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 24.837202878999999, 2024-09-18T10:16:06.341 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.300815683, 2024-09-18T10:16:06.341 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:06.341 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:16:06.341 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:06.341 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.341 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:06.341 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:41.438999+0000", 2024-09-18T10:16:06.341 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:06.342 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.342 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.342 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:06.342 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:41.438999+0000", 2024-09-18T10:16:06.342 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:06.342 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.342 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.342 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:06.342 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:41.438999+0000", 2024-09-18T10:16:06.342 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999995 2024-09-18T10:16:06.342 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.342 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.342 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:06.342 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:41.439004+0000", 2024-09-18T10:16:06.343 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8169999999999998e-06 2024-09-18T10:16:06.343 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.343 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.343 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:06.343 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:41.439035+0000", 2024-09-18T10:16:06.343 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1405000000000002e-05 2024-09-18T10:16:06.343 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.343 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.343 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:06.343 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:41.439046+0000", 2024-09-18T10:16:06.343 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1367e-05 2024-09-18T10:16:06.343 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.343 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.343 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:06.344 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:41.739792+0000", 2024-09-18T10:16:06.344 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30074592900000002 2024-09-18T10:16:06.344 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.344 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.344 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:06.344 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:41.739815+0000", 2024-09-18T10:16:06.344 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2798000000000001e-05 2024-09-18T10:16:06.344 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:06.344 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:06.344 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:06.344 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.344 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.344 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWriteReply(3.16s0 89/82 ECSubWriteReply(tid=30, last_complete=89'37, committed=1, applied=1))", 2024-09-18T10:16:06.345 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:44.908040+0000", 2024-09-18T10:16:06.345 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 21.368162611999999, 2024-09-18T10:16:06.345 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.199742948, 2024-09-18T10:16:06.345 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:06.345 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:16:06.345 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:06.345 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.345 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:06.345 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:44.908040+0000", 2024-09-18T10:16:06.345 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:06.345 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.345 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.345 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:06.346 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:44.908040+0000", 2024-09-18T10:16:06.346 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:06.346 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.346 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.346 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:06.346 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:44.908038+0000", 2024-09-18T10:16:06.346 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:16:06.346 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.346 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.346 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:06.346 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:44.908048+0000", 2024-09-18T10:16:06.346 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3749999999999992e-06 2024-09-18T10:16:06.346 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.346 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.347 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:06.347 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:44.908069+0000", 2024-09-18T10:16:06.347 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0817000000000001e-05 2024-09-18T10:16:06.347 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.347 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.347 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:06.347 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:44.908077+0000", 2024-09-18T10:16:06.347 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7369999999999995e-06 2024-09-18T10:16:06.347 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.347 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.347 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:06.347 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.107754+0000", 2024-09-18T10:16:06.347 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.19967654400000001 2024-09-18T10:16:06.347 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.348 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.348 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:06.348 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.107783+0000", 2024-09-18T10:16:06.348 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8609e-05 2024-09-18T10:16:06.348 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:06.348 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:06.348 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:06.348 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.348 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.348 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.6s5 89/40 ECSubWrite(tid=78, reqid=osd.0.0:77, at_version=0'0, trim_to=0'0, roll_forward_to=89'48))", 2024-09-18T10:16:06.348 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:44.997782+0000", 2024-09-18T10:16:06.348 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 21.278419874000001, 2024-09-18T10:16:06.348 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.21032350799999999, 2024-09-18T10:16:06.349 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:06.349 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:16:06.349 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:06.349 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.349 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:06.349 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:44.997782+0000", 2024-09-18T10:16:06.349 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:06.349 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.349 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.349 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:06.349 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:44.997782+0000", 2024-09-18T10:16:06.349 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:06.349 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.349 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.350 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:06.350 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:44.997781+0000", 2024-09-18T10:16:06.350 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999981 2024-09-18T10:16:06.350 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.350 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.350 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:06.350 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:44.997812+0000", 2024-09-18T10:16:06.350 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1829000000000003e-05 2024-09-18T10:16:06.350 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.350 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.350 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:06.350 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:44.997854+0000", 2024-09-18T10:16:06.350 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1486000000000002e-05 2024-09-18T10:16:06.350 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.351 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.351 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:06.351 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.097955+0000", 2024-09-18T10:16:06.351 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10010105399999999 2024-09-18T10:16:06.351 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.351 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.351 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:06.351 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.108057+0000", 2024-09-18T10:16:06.351 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010101941999999999 2024-09-18T10:16:06.351 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.351 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.351 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:16:06.351 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.108069+0000", 2024-09-18T10:16:06.351 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2216e-05 2024-09-18T10:16:06.352 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.352 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.352 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:16:06.352 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.208058+0000", 2024-09-18T10:16:06.352 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.099989073999999997 2024-09-18T10:16:06.352 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.352 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.352 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:06.352 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.208106+0000", 2024-09-18T10:16:06.352 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7604e-05 2024-09-18T10:16:06.352 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:06.352 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:06.352 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:06.353 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.353 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.353 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:879 3.16s0 3:6c64c4fa:::smithi18925896-41:head [copy-from ver 84] snapc 33=[33,30,2f,28,20,1f,1a,11] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-18T10:16:06.353 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:45.696039+0000", 2024-09-18T10:16:06.353 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 20.580163456000001, 2024-09-18T10:16:06.353 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.003875471, 2024-09-18T10:16:06.353 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:06.353 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:06.353 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:16:06.353 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:16:06.353 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:06.353 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 879 2024-09-18T10:16:06.353 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.354 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:06.354 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.354 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:06.354 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.696039+0000", 2024-09-18T10:16:06.354 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:06.354 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.354 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.354 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:06.354 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.696039+0000", 2024-09-18T10:16:06.354 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:06.354 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.354 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.354 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:06.354 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.696035+0000", 2024-09-18T10:16:06.355 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999962 2024-09-18T10:16:06.355 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.355 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.355 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:06.355 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.696060+0000", 2024-09-18T10:16:06.355 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5219e-05 2024-09-18T10:16:06.355 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.355 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.355 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:06.355 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.696108+0000", 2024-09-18T10:16:06.355 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7435000000000002e-05 2024-09-18T10:16:06.355 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.355 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.356 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:06.356 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.696124+0000", 2024-09-18T10:16:06.356 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5990000000000001e-05 2024-09-18T10:16:06.356 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.356 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.356 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:06.356 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.696244+0000", 2024-09-18T10:16:06.356 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000120049 2024-09-18T10:16:06.356 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.356 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.356 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-09-18T10:16:06.356 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.696259+0000", 2024-09-18T10:16:06.356 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5265999999999999e-05 2024-09-18T10:16:06.357 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.357 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.357 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:06.357 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.700259+0000", 2024-09-18T10:16:06.357 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0039999570000000002 2024-09-18T10:16:06.357 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.357 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.357 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:16:06.357 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.700367+0000", 2024-09-18T10:16:06.357 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010851199999999999 2024-09-18T10:16:06.357 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.357 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.357 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:16:06.357 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:46.586082+0000", 2024-09-18T10:16:06.358 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.88571493199999995 2024-09-18T10:16:06.358 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.358 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.358 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:16:06.358 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:46.590196+0000", 2024-09-18T10:16:06.358 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0041135549999999996 2024-09-18T10:16:06.358 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.358 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.358 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:16:06.358 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:46.699714+0000", 2024-09-18T10:16:06.358 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.109518436 2024-09-18T10:16:06.358 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.358 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.358 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:06.359 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:46.699914+0000", 2024-09-18T10:16:06.359 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019992 2024-09-18T10:16:06.359 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:06.359 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:06.359 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:06.359 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.359 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.359 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:880 3.16s0 3:6c64c4fa:::smithi18925896-41:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e90)", 2024-09-18T10:16:06.359 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:45.696157+0000", 2024-09-18T10:16:06.359 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 20.580045207000001, 2024-09-18T10:16:06.359 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.89392841599999995, 2024-09-18T10:16:06.359 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:06.359 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-18T10:16:06.359 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:16:06.360 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:16:06.360 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:06.360 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 880 2024-09-18T10:16:06.360 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.360 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:06.360 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.360 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:06.360 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.696157+0000", 2024-09-18T10:16:06.360 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:06.360 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.360 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.360 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:06.360 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.696157+0000", 2024-09-18T10:16:06.360 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:06.361 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.361 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.361 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:06.361 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.696156+0000", 2024-09-18T10:16:06.361 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999986 2024-09-18T10:16:06.361 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.361 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.361 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:06.361 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.696164+0000", 2024-09-18T10:16:06.361 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4470000000000006e-06 2024-09-18T10:16:06.361 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.361 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.361 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:06.361 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.696186+0000", 2024-09-18T10:16:06.362 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2302000000000001e-05 2024-09-18T10:16:06.362 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.362 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.362 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:06.362 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.696236+0000", 2024-09-18T10:16:06.362 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9146e-05 2024-09-18T10:16:06.362 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.362 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.362 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:06.362 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.696374+0000", 2024-09-18T10:16:06.362 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013882200000000001 2024-09-18T10:16:06.362 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.362 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.362 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:16:06.363 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.696390+0000", 2024-09-18T10:16:06.363 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5421999999999999e-05 2024-09-18T10:16:06.363 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.363 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.363 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:06.363 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.700572+0000", 2024-09-18T10:16:06.363 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0041817460000000001 2024-09-18T10:16:06.363 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.363 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.363 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2024-09-18T10:16:06.363 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.700674+0000", 2024-09-18T10:16:06.363 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000102512 2024-09-18T10:16:06.363 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.363 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.364 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:06.364 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:46.589461+0000", 2024-09-18T10:16:06.364 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.88878669300000002 2024-09-18T10:16:06.364 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.364 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.364 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:16:06.364 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:46.589768+0000", 2024-09-18T10:16:06.364 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00030681000000000003 2024-09-18T10:16:06.364 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.364 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.364 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:06.364 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:46.590085+0000", 2024-09-18T10:16:06.364 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00031782300000000001 2024-09-18T10:16:06.365 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:06.365 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:06.365 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:06.365 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.365 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.365 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.ds4 100/94 ECSubWrite(tid=241, reqid=osd.13.0:239, at_version=0'0, trim_to=0'0, roll_forward_to=100'43))", 2024-09-18T10:16:06.365 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:56.235861+0000", 2024-09-18T10:16:06.365 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.040341365, 2024-09-18T10:16:06.365 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.201709472, 2024-09-18T10:16:06.365 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:06.365 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:16:06.365 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:06.365 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.365 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:06.366 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:56.235861+0000", 2024-09-18T10:16:06.366 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:06.366 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.366 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.366 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:06.366 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:56.235861+0000", 2024-09-18T10:16:06.366 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:06.366 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.366 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.366 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:06.366 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:56.235860+0000", 2024-09-18T10:16:06.366 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:16:06.366 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.366 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.367 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:06.367 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:56.235913+0000", 2024-09-18T10:16:06.367 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3347999999999998e-05 2024-09-18T10:16:06.367 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.367 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.367 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:06.367 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:56.235940+0000", 2024-09-18T10:16:06.367 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6846e-05 2024-09-18T10:16:06.367 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.367 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.367 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:06.367 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:56.336070+0000", 2024-09-18T10:16:06.367 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100129629 2024-09-18T10:16:06.367 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.367 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.368 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:06.368 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:56.436495+0000", 2024-09-18T10:16:06.368 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100425555 2024-09-18T10:16:06.368 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.368 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.368 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:16:06.368 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:56.436514+0000", 2024-09-18T10:16:06.368 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8963000000000001e-05 2024-09-18T10:16:06.368 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.368 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.368 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:16:06.368 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:56.437522+0000", 2024-09-18T10:16:06.368 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001007806 2024-09-18T10:16:06.368 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.368 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.369 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:06.369 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:56.437570+0000", 2024-09-18T10:16:06.369 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8291999999999998e-05 2024-09-18T10:16:06.369 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:06.369 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:06.369 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:06.369 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:06.369 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:06.369 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-18T10:16:06.371 INFO:teuthology.orchestra.run.smithi029.stdout:osd.13: {} 2024-09-18T10:16:06.371 INFO:teuthology.orchestra.run.smithi029.stdout:osd.14: {} 2024-09-18T10:16:06.371 INFO:teuthology.orchestra.run.smithi029.stdout:osd.15: {} 2024-09-18T10:16:06.372 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_ops_in_flight 2024-09-18T10:16:06.372 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T10:16:06.373 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T10:16:06.373 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T10:16:06.373 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T10:16:06.373 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T10:16:06.373 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T10:16:06.373 INFO:teuthology.orchestra.run.smithi029.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T10:16:06.373 INFO:teuthology.orchestra.run.smithi029.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T10:16:06.373 INFO:teuthology.orchestra.run.smithi029.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-18T10:16:06.373 INFO:teuthology.orchestra.run.smithi029.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-18T10:16:06.373 INFO:teuthology.orchestra.run.smithi029.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-18T10:16:06.373 INFO:teuthology.orchestra.run.smithi029.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-18T10:16:06.373 INFO:teuthology.orchestra.run.smithi029.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-18T10:16:06.373 INFO:teuthology.orchestra.run.smithi029.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-18T10:16:06.374 INFO:teuthology.orchestra.run.smithi029.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-18T10:16:06.374 INFO:teuthology.orchestra.run.smithi029.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-18T10:16:06.374 INFO:tasks.ceph.osd.12.smithi029.stderr:2024-09-18T10:16:06.365+0000 7f8e4e627700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 22190) UID: 0 2024-09-18T10:16:06.471 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-18T10:16:06.465+0000 7f14dcc5b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22398) UID: 0 2024-09-18T10:16:06.481 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-18T10:16:06.481 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-18T10:16:06.481 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-09-18T10:16:06.481 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-18T10:16:06.490 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_blocked_ops 2024-09-18T10:16:06.572 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-18T10:16:06.569+0000 7f14dcc5b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22398) UID: 0 2024-09-18T10:16:06.586 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-18T10:16:06.587 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-18T10:16:06.587 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-18T10:16:06.587 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-18T10:16:06.587 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-18T10:16:06.595 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_historic_ops 2024-09-18T10:16:06.673 INFO:tasks.ceph.osd.10.smithi154.stderr:2024-09-18T10:16:06.670+0000 7fb73221a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 22091) UID: 0 2024-09-18T10:16:06.697 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-18T10:16:06.697 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-18T10:16:06.697 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-18T10:16:06.697 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-18T10:16:06.697 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.697 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:201 3.as0 3:5dc9cbed:::smithi18925896-13:2 [read 0~212992,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:16:06.697 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:52.694813+0000", 2024-09-18T10:16:06.697 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 73.997637686000004, 2024-09-18T10:16:06.697 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.50652904899999995, 2024-09-18T10:16:06.697 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:06.698 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-18T10:16:06.698 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:16:06.698 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:16:06.698 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:06.698 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 201 2024-09-18T10:16:06.698 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.698 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:06.698 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.698 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:06.698 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694813+0000", 2024-09-18T10:16:06.698 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:06.698 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.698 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.699 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:06.699 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694813+0000", 2024-09-18T10:16:06.699 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:06.699 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.699 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.699 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:06.699 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694812+0000", 2024-09-18T10:16:06.699 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999995 2024-09-18T10:16:06.699 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.699 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.699 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:06.699 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694816+0000", 2024-09-18T10:16:06.699 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7720000000000001e-06 2024-09-18T10:16:06.699 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.700 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.700 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:06.700 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694822+0000", 2024-09-18T10:16:06.700 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5529999999999998e-06 2024-09-18T10:16:06.700 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.700 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.700 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:06.700 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694827+0000", 2024-09-18T10:16:06.700 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1500000000000001e-06 2024-09-18T10:16:06.700 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.700 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.700 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:06.700 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.797100+0000", 2024-09-18T10:16:06.701 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.102273424 2024-09-18T10:16:06.701 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.701 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.701 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:16:06.701 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.797329+0000", 2024-09-18T10:16:06.701 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00022870200000000001 2024-09-18T10:16:06.701 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.701 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.701 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:06.701 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.201342+0000", 2024-09-18T10:16:06.701 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.40401293799999999 2024-09-18T10:16:06.701 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:06.702 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:06.702 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:06.702 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.702 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.702 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:207 3.as0 3:5c566071:::smithi18925896-28:head [copy-from ver 6] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:16:06.702 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:52.694855+0000", 2024-09-18T10:16:06.702 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 73.997594883999994, 2024-09-18T10:16:06.702 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.81394701899999999, 2024-09-18T10:16:06.702 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:06.702 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:06.702 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:16:06.704 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:16:06.704 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:06.704 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 207 2024-09-18T10:16:06.704 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.704 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:06.704 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.704 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:06.704 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694855+0000", 2024-09-18T10:16:06.704 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:06.704 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.704 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.705 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:06.705 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694855+0000", 2024-09-18T10:16:06.705 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:06.705 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.705 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.705 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:06.705 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694852+0000", 2024-09-18T10:16:06.705 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999967 2024-09-18T10:16:06.705 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.705 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.705 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:06.705 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694859+0000", 2024-09-18T10:16:06.705 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.08e-06 2024-09-18T10:16:06.706 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.706 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.706 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:06.706 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694866+0000", 2024-09-18T10:16:06.706 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9249999999999998e-06 2024-09-18T10:16:06.706 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.706 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.706 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:06.706 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694870+0000", 2024-09-18T10:16:06.706 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0330000000000002e-06 2024-09-18T10:16:06.706 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.706 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.706 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:06.707 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.900743+0000", 2024-09-18T10:16:06.707 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20587325400000001 2024-09-18T10:16:06.707 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.707 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.707 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:16:06.707 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.901026+0000", 2024-09-18T10:16:06.707 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00028313600000000002 2024-09-18T10:16:06.707 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.707 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.707 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:16:06.707 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.204074+0000", 2024-09-18T10:16:06.707 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30304722299999998 2024-09-18T10:16:06.707 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.708 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.708 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:16:06.708 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.207280+0000", 2024-09-18T10:16:06.708 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0032067049999999998 2024-09-18T10:16:06.708 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.708 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.708 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:16:06.708 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.508672+0000", 2024-09-18T10:16:06.708 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30139174800000001 2024-09-18T10:16:06.709 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.709 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.709 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:06.709 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.508802+0000", 2024-09-18T10:16:06.709 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013037500000000001 2024-09-18T10:16:06.709 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:06.709 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:06.709 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:06.709 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.709 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.709 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:208 3.as0 3:5c566071:::smithi18925896-28:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:16:06.709 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:52.694875+0000", 2024-09-18T10:16:06.709 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 73.997575722999997, 2024-09-18T10:16:06.710 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.51199464299999997, 2024-09-18T10:16:06.710 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:06.710 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-18T10:16:06.710 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:16:06.710 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:16:06.710 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:06.710 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 208 2024-09-18T10:16:06.710 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.710 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:06.710 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.710 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:06.710 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694875+0000", 2024-09-18T10:16:06.711 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:06.711 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.711 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.711 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:06.711 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694875+0000", 2024-09-18T10:16:06.711 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:06.711 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.711 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.711 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:06.712 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694874+0000", 2024-09-18T10:16:06.712 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999995 2024-09-18T10:16:06.712 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.712 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.712 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:06.712 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694876+0000", 2024-09-18T10:16:06.712 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2369999999999999e-06 2024-09-18T10:16:06.712 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.712 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.712 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:06.712 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694881+0000", 2024-09-18T10:16:06.712 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0950000000000003e-06 2024-09-18T10:16:06.713 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.713 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.713 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:06.713 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694887+0000", 2024-09-18T10:16:06.713 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5690000000000004e-06 2024-09-18T10:16:06.713 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.713 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.713 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:06.713 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.901475+0000", 2024-09-18T10:16:06.713 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20658803000000001 2024-09-18T10:16:06.713 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.713 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.713 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2024-09-18T10:16:06.714 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.901550+0000", 2024-09-18T10:16:06.714 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5020999999999999e-05 2024-09-18T10:16:06.714 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.714 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.714 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:06.714 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.206415+0000", 2024-09-18T10:16:06.714 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30486486000000002 2024-09-18T10:16:06.714 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.714 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.715 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:16:06.715 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.206634+0000", 2024-09-18T10:16:06.715 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00021925499999999999 2024-09-18T10:16:06.715 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.715 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.715 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:06.715 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.206869+0000", 2024-09-18T10:16:06.715 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00023503599999999999 2024-09-18T10:16:06.715 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:06.715 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:06.715 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:06.715 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.715 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.715 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:335 3.ds0 3:b1a14543:::smithi18925896-15:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e47)", 2024-09-18T10:16:06.716 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:02.793201+0000", 2024-09-18T10:16:06.716 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 63.899249451000003, 2024-09-18T10:16:06.716 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.51366298600000004, 2024-09-18T10:16:06.716 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:06.716 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-18T10:16:06.716 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:16:06.716 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:16:06.716 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:06.716 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 335 2024-09-18T10:16:06.716 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.716 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:06.716 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.716 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:06.717 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.793201+0000", 2024-09-18T10:16:06.717 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:06.717 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.717 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.717 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:06.717 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.793201+0000", 2024-09-18T10:16:06.717 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:06.717 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.717 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.717 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:06.717 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.793200+0000", 2024-09-18T10:16:06.717 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999986 2024-09-18T10:16:06.718 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.718 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.718 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:06.718 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.793206+0000", 2024-09-18T10:16:06.718 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2199999999999997e-06 2024-09-18T10:16:06.718 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.718 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.718 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:06.718 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.793221+0000", 2024-09-18T10:16:06.718 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5092000000000001e-05 2024-09-18T10:16:06.718 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.718 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.718 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:06.719 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.793229+0000", 2024-09-18T10:16:06.719 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1820000000000006e-06 2024-09-18T10:16:06.719 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.719 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.719 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:06.719 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.793367+0000", 2024-09-18T10:16:06.719 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000138057 2024-09-18T10:16:06.719 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.719 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.719 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:16:06.719 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.793384+0000", 2024-09-18T10:16:06.719 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6731e-05 2024-09-18T10:16:06.719 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.720 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.720 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:06.720 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.804064+0000", 2024-09-18T10:16:06.720 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010680185 2024-09-18T10:16:06.720 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.720 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.720 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2024-09-18T10:16:06.720 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.804199+0000", 2024-09-18T10:16:06.720 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013532999999999999 2024-09-18T10:16:06.720 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.720 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.720 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:06.720 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:03.202193+0000", 2024-09-18T10:16:06.721 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.39799325099999999 2024-09-18T10:16:06.721 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.721 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.721 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:16:06.721 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:03.202292+0000", 2024-09-18T10:16:06.721 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9628000000000002e-05 2024-09-18T10:16:06.721 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.721 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.721 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:06.721 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:03.306864+0000", 2024-09-18T10:16:06.721 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.104571572 2024-09-18T10:16:06.721 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:06.721 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:06.721 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:06.722 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.722 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.722 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:503 3.fs0 3:feb35297:::smithi18925896-37:head [append 0~753664] snapc 19=[19,18,16,12,11,10,f,e] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T10:16:06.722 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:17.908895+0000", 2024-09-18T10:16:06.722 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 48.783555470000003, 2024-09-18T10:16:06.722 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.51525800200000005, 2024-09-18T10:16:06.722 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:06.722 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:06.722 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:16:06.722 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:16:06.722 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:06.722 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 503 2024-09-18T10:16:06.722 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.722 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:06.723 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.723 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:06.723 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.908895+0000", 2024-09-18T10:16:06.723 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:06.723 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.723 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.723 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:06.723 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.908895+0000", 2024-09-18T10:16:06.723 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:06.723 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.723 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.723 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:06.723 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.908892+0000", 2024-09-18T10:16:06.723 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999976 2024-09-18T10:16:06.724 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.724 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.724 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:06.724 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.910143+0000", 2024-09-18T10:16:06.724 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0012507040000000001 2024-09-18T10:16:06.724 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.724 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.724 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:06.724 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.910167+0000", 2024-09-18T10:16:06.724 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4176999999999999e-05 2024-09-18T10:16:06.725 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.725 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.725 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:06.725 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.910178+0000", 2024-09-18T10:16:06.725 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0669999999999999e-05 2024-09-18T10:16:06.725 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.725 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.725 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:06.725 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.910329+0000", 2024-09-18T10:16:06.725 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000150746 2024-09-18T10:16:06.725 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.725 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.725 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-09-18T10:16:06.726 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.910346+0000", 2024-09-18T10:16:06.726 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7680999999999999e-05 2024-09-18T10:16:06.726 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.726 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.726 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:06.726 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.015009+0000", 2024-09-18T10:16:06.726 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.104662914 2024-09-18T10:16:06.726 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.726 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.727 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:16:06.727 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.015304+0000", 2024-09-18T10:16:06.727 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00029439899999999998 2024-09-18T10:16:06.727 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.727 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.727 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:16:06.727 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.017669+0000", 2024-09-18T10:16:06.727 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0023647910000000002 2024-09-18T10:16:06.728 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.728 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.728 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:16:06.728 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.126654+0000", 2024-09-18T10:16:06.728 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.108985055 2024-09-18T10:16:06.728 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.728 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.728 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:16:06.728 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.423961+0000", 2024-09-18T10:16:06.728 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.29730786399999998 2024-09-18T10:16:06.728 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.728 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.728 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:06.729 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.424153+0000", 2024-09-18T10:16:06.729 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019128099999999999 2024-09-18T10:16:06.729 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:06.729 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:06.729 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:06.729 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.729 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.729 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:504 3.fs0 3:feb35297:::smithi18925896-37:head [append 0~720896] snapc 19=[19,18,16,12,11,10,f,e] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T10:16:06.729 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:17.912996+0000", 2024-09-18T10:16:06.729 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 48.779454434999998, 2024-09-18T10:16:06.729 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.61195014999999997, 2024-09-18T10:16:06.729 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:06.729 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:06.730 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:16:06.730 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:16:06.730 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:06.730 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 504 2024-09-18T10:16:06.730 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.730 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:06.730 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.730 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:06.730 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.912996+0000", 2024-09-18T10:16:06.730 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:06.730 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.730 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.730 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:06.731 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.912996+0000", 2024-09-18T10:16:06.731 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:06.731 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.731 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.731 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:06.731 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.912994+0000", 2024-09-18T10:16:06.731 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999981 2024-09-18T10:16:06.731 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.731 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.731 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:06.731 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.913907+0000", 2024-09-18T10:16:06.731 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00091254300000000003 2024-09-18T10:16:06.731 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.732 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.732 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:06.732 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.913950+0000", 2024-09-18T10:16:06.732 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3019000000000003e-05 2024-09-18T10:16:06.732 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.732 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.732 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:06.732 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.913960+0000", 2024-09-18T10:16:06.732 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0305e-05 2024-09-18T10:16:06.732 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.732 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.732 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:06.732 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.914067+0000", 2024-09-18T10:16:06.733 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000106696 2024-09-18T10:16:06.733 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.733 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.733 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:16:06.733 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.914079+0000", 2024-09-18T10:16:06.733 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2330000000000001e-05 2024-09-18T10:16:06.733 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.733 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.733 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:06.733 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.119322+0000", 2024-09-18T10:16:06.733 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20524277899999999 2024-09-18T10:16:06.733 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.733 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.733 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:16:06.734 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.119508+0000", 2024-09-18T10:16:06.734 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00018648800000000001 2024-09-18T10:16:06.734 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.734 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.734 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:16:06.734 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.121315+0000", 2024-09-18T10:16:06.734 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001806483 2024-09-18T10:16:06.734 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.734 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.734 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:16:06.734 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.127304+0000", 2024-09-18T10:16:06.734 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0059895599999999997 2024-09-18T10:16:06.734 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.735 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.735 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:16:06.735 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.524828+0000", 2024-09-18T10:16:06.735 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.39752343600000001 2024-09-18T10:16:06.735 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.735 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.735 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:06.735 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.524946+0000", 2024-09-18T10:16:06.735 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000118193 2024-09-18T10:16:06.735 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:06.735 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:06.735 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:06.735 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.736 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.736 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:505 3.fs0 3:feb35297:::smithi18925896-37:head [append 0~49152] snapc 19=[19,18,16,12,11,10,f,e] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T10:16:06.736 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:17.913979+0000", 2024-09-18T10:16:06.736 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 48.778470956, 2024-09-18T10:16:06.736 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.71150136900000005, 2024-09-18T10:16:06.736 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:06.736 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:06.736 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:16:06.736 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:16:06.736 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:06.736 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 505 2024-09-18T10:16:06.736 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.736 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:06.737 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.737 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:06.737 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.913979+0000", 2024-09-18T10:16:06.737 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:06.737 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.737 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.737 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:06.737 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.913979+0000", 2024-09-18T10:16:06.737 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:06.737 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.737 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.737 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:06.737 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.913978+0000", 2024-09-18T10:16:06.738 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999986 2024-09-18T10:16:06.738 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.738 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.738 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:06.738 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.914061+0000", 2024-09-18T10:16:06.738 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2533000000000004e-05 2024-09-18T10:16:06.738 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.738 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.738 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:06.738 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.914077+0000", 2024-09-18T10:16:06.738 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6277e-05 2024-09-18T10:16:06.738 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.738 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.739 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:06.739 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.914087+0000", 2024-09-18T10:16:06.739 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0282000000000001e-05 2024-09-18T10:16:06.739 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.739 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.739 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:06.739 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.914144+0000", 2024-09-18T10:16:06.739 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6784999999999999e-05 2024-09-18T10:16:06.739 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.739 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.739 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:16:06.739 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.914151+0000", 2024-09-18T10:16:06.739 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.968e-06 2024-09-18T10:16:06.739 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.740 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.740 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:06.740 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.122495+0000", 2024-09-18T10:16:06.740 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.208343742 2024-09-18T10:16:06.740 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.740 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.740 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:16:06.740 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.122788+0000", 2024-09-18T10:16:06.740 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00029334199999999998 2024-09-18T10:16:06.740 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.740 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.740 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:16:06.740 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.124367+0000", 2024-09-18T10:16:06.741 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0015794699999999999 2024-09-18T10:16:06.741 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.741 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.741 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:16:06.741 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.127307+0000", 2024-09-18T10:16:06.741 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0029394780000000001 2024-09-18T10:16:06.741 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.741 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.741 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:16:06.741 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.625281+0000", 2024-09-18T10:16:06.742 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.497974166 2024-09-18T10:16:06.742 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.742 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.742 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:06.742 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.625481+0000", 2024-09-18T10:16:06.742 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019956100000000001 2024-09-18T10:16:06.742 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:06.742 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:06.742 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:06.742 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.742 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.742 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:506 3.fs0 3:feb35297:::smithi18925896-37:head [setxattr _header (57) in=7b] snapc 19=[19,18,16,12,11,10,f,e] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T10:16:06.742 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:17.914106+0000", 2024-09-18T10:16:06.742 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 48.778344689000001, 2024-09-18T10:16:06.743 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.71184716599999998, 2024-09-18T10:16:06.743 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:06.743 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:06.743 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:16:06.743 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:16:06.743 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:06.743 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 506 2024-09-18T10:16:06.743 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.743 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:06.743 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.743 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:06.743 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.914106+0000", 2024-09-18T10:16:06.743 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:06.744 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.744 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.744 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:06.744 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.914106+0000", 2024-09-18T10:16:06.744 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:06.744 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.744 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.744 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:06.744 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.914105+0000", 2024-09-18T10:16:06.744 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:16:06.744 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.744 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.744 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:06.745 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.914110+0000", 2024-09-18T10:16:06.745 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.045e-06 2024-09-18T10:16:06.745 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.745 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.745 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:06.745 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.914117+0000", 2024-09-18T10:16:06.745 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6510000000000001e-06 2024-09-18T10:16:06.745 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.745 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.745 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:06.745 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.914122+0000", 2024-09-18T10:16:06.745 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5839999999999996e-06 2024-09-18T10:16:06.745 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.746 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.746 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:06.746 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.014305+0000", 2024-09-18T10:16:06.746 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100183091 2024-09-18T10:16:06.746 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.746 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.746 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:16:06.746 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.014325+0000", 2024-09-18T10:16:06.746 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0032000000000001e-05 2024-09-18T10:16:06.746 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.746 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.746 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:06.746 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.125511+0000", 2024-09-18T10:16:06.746 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.111185942 2024-09-18T10:16:06.747 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.747 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.747 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:16:06.747 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.125760+0000", 2024-09-18T10:16:06.747 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00024864100000000002 2024-09-18T10:16:06.747 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.747 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.747 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:16:06.747 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.126375+0000", 2024-09-18T10:16:06.747 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00061536299999999996 2024-09-18T10:16:06.747 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.747 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.747 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:16:06.747 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.227656+0000", 2024-09-18T10:16:06.748 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.101281097 2024-09-18T10:16:06.748 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.748 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.748 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:16:06.748 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.625791+0000", 2024-09-18T10:16:06.748 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.39813470299999998 2024-09-18T10:16:06.748 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.748 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.748 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:06.748 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.625953+0000", 2024-09-18T10:16:06.748 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00016192999999999999 2024-09-18T10:16:06.748 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:06.748 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:06.749 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:06.749 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.749 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.749 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:507 3.fs0 3:feb35297:::smithi18925896-37:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e62)", 2024-09-18T10:16:06.749 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:17.914159+0000", 2024-09-18T10:16:06.749 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 48.778291572999997, 2024-09-18T10:16:06.749 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0151776809999999, 2024-09-18T10:16:06.749 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:06.749 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-18T10:16:06.749 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:16:06.749 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:16:06.749 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:06.749 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 507 2024-09-18T10:16:06.750 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.750 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:06.750 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.750 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:06.750 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.914159+0000", 2024-09-18T10:16:06.750 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:06.750 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.750 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.750 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:06.750 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.914159+0000", 2024-09-18T10:16:06.750 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:06.750 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.750 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.751 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:06.751 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.914158+0000", 2024-09-18T10:16:06.751 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:16:06.751 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.751 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.751 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:06.751 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.914162+0000", 2024-09-18T10:16:06.751 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0729999999999998e-06 2024-09-18T10:16:06.751 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.751 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.751 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:06.751 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.914176+0000", 2024-09-18T10:16:06.751 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4258e-05 2024-09-18T10:16:06.752 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.752 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.752 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:06.752 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.014273+0000", 2024-09-18T10:16:06.752 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10009702400000001 2024-09-18T10:16:06.752 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.752 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.752 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:06.752 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.126754+0000", 2024-09-18T10:16:06.752 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.112480581 2024-09-18T10:16:06.752 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.752 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.752 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2024-09-18T10:16:06.752 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.127051+0000", 2024-09-18T10:16:06.753 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00029721199999999999 2024-09-18T10:16:06.753 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.753 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.753 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:06.753 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.626655+0000", 2024-09-18T10:16:06.753 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.49960442399999999 2024-09-18T10:16:06.753 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.753 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.753 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:16:06.753 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.626826+0000", 2024-09-18T10:16:06.753 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000170459 2024-09-18T10:16:06.753 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.753 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.753 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:06.754 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.929336+0000", 2024-09-18T10:16:06.754 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30251042900000003 2024-09-18T10:16:06.754 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:06.754 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:06.754 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:06.754 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.754 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.754 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:524 3.ds0 3:b1a14543:::smithi18925896-15:head [append 0~442368] snapc 1a=[1a,19,18,16,12,11,10,f,e] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-18T10:16:06.754 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:19.452015+0000", 2024-09-18T10:16:06.754 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 47.240435591000001, 2024-09-18T10:16:06.754 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.55248773200000001, 2024-09-18T10:16:06.754 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:06.754 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:06.755 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:16:06.755 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:16:06.755 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:06.755 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 524 2024-09-18T10:16:06.755 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.755 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:06.755 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.755 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:06.755 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.452015+0000", 2024-09-18T10:16:06.755 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:06.755 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.755 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.755 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:06.755 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.452015+0000", 2024-09-18T10:16:06.756 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:06.756 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.756 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.756 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:06.756 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.452012+0000", 2024-09-18T10:16:06.756 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999976 2024-09-18T10:16:06.756 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.756 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.756 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:06.756 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.452793+0000", 2024-09-18T10:16:06.756 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00078048600000000005 2024-09-18T10:16:06.756 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.756 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.756 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:06.757 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.452825+0000", 2024-09-18T10:16:06.757 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2672999999999997e-05 2024-09-18T10:16:06.757 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.757 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.757 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:06.757 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.552970+0000", 2024-09-18T10:16:06.757 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100144728 2024-09-18T10:16:06.757 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.757 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.757 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:06.757 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.553127+0000", 2024-09-18T10:16:06.757 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000157202 2024-09-18T10:16:06.757 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.758 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.758 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:16:06.758 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.553439+0000", 2024-09-18T10:16:06.758 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00031139500000000001 2024-09-18T10:16:06.758 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.758 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.758 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:16:06.758 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.556240+0000", 2024-09-18T10:16:06.758 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0028012509999999998 2024-09-18T10:16:06.758 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.758 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.758 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:16:06.758 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.665529+0000", 2024-09-18T10:16:06.759 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10928916700000001 2024-09-18T10:16:06.759 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.759 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.759 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:16:06.759 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:20.004378+0000", 2024-09-18T10:16:06.759 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.33884916300000001 2024-09-18T10:16:06.759 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.759 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.759 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:06.759 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:20.004502+0000", 2024-09-18T10:16:06.759 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012407999999999999 2024-09-18T10:16:06.759 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:06.759 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:06.760 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:06.760 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.760 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.760 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:525 3.ds0 3:b1a14543:::smithi18925896-15:head [append 0~540672] snapc 1a=[1a,19,18,16,12,11,10,f,e] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-18T10:16:06.760 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:19.553028+0000", 2024-09-18T10:16:06.760 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 47.139421751999997, 2024-09-18T10:16:06.760 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.45756244000000001, 2024-09-18T10:16:06.760 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:06.760 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:06.760 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:16:06.760 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:16:06.760 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:06.760 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 525 2024-09-18T10:16:06.761 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.761 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:06.761 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.761 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:06.761 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.553028+0000", 2024-09-18T10:16:06.761 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:06.761 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.761 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.761 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:06.761 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.553028+0000", 2024-09-18T10:16:06.761 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:06.761 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.761 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.761 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:06.762 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.553026+0000", 2024-09-18T10:16:06.762 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999981 2024-09-18T10:16:06.762 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.762 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.762 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:06.762 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.553796+0000", 2024-09-18T10:16:06.762 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00076915200000000005 2024-09-18T10:16:06.762 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.762 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.762 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:06.762 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.553819+0000", 2024-09-18T10:16:06.762 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3008999999999999e-05 2024-09-18T10:16:06.762 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.762 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.763 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:06.763 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.553829+0000", 2024-09-18T10:16:06.763 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0543999999999999e-05 2024-09-18T10:16:06.763 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.763 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.763 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:06.763 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.558075+0000", 2024-09-18T10:16:06.763 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.004245932 2024-09-18T10:16:06.763 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.763 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.763 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:16:06.763 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.558384+0000", 2024-09-18T10:16:06.763 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00030926100000000002 2024-09-18T10:16:06.764 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.764 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.764 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:16:06.764 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.559950+0000", 2024-09-18T10:16:06.764 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001565615 2024-09-18T10:16:06.764 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.764 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.764 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:16:06.764 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.665535+0000", 2024-09-18T10:16:06.764 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.105584707 2024-09-18T10:16:06.764 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.764 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.764 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:16:06.764 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:20.010428+0000", 2024-09-18T10:16:06.765 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.34489341800000001 2024-09-18T10:16:06.765 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.765 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.765 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:06.765 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:20.010591+0000", 2024-09-18T10:16:06.765 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000162845 2024-09-18T10:16:06.765 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:06.765 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:06.765 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:06.765 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.765 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.765 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:526 3.ds0 3:b1a14543:::smithi18925896-15:head [append 0~622592] snapc 1a=[1a,19,18,16,12,11,10,f,e] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-18T10:16:06.765 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:19.553845+0000", 2024-09-18T10:16:06.766 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 47.138605036999998, 2024-09-18T10:16:06.766 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.55723060999999996, 2024-09-18T10:16:06.766 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:06.766 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:06.766 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:16:06.766 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:16:06.766 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:06.766 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 526 2024-09-18T10:16:06.766 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.766 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:06.766 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.766 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:06.766 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.553845+0000", 2024-09-18T10:16:06.766 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:06.767 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.767 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.767 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:06.767 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.553845+0000", 2024-09-18T10:16:06.767 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:06.767 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.767 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.767 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:06.767 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.553844+0000", 2024-09-18T10:16:06.767 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999986 2024-09-18T10:16:06.767 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.767 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.767 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:06.767 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.554666+0000", 2024-09-18T10:16:06.768 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00082239100000000001 2024-09-18T10:16:06.768 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.768 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.768 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:06.768 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.554679+0000", 2024-09-18T10:16:06.768 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.271e-05 2024-09-18T10:16:06.768 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.768 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.768 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:06.768 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.554685+0000", 2024-09-18T10:16:06.768 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4230000000000002e-06 2024-09-18T10:16:06.768 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.768 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.769 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:06.769 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.561235+0000", 2024-09-18T10:16:06.769 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0065498179999999998 2024-09-18T10:16:06.769 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.769 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.769 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:16:06.769 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.561523+0000", 2024-09-18T10:16:06.769 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000287862 2024-09-18T10:16:06.769 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.769 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.769 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:16:06.769 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.562904+0000", 2024-09-18T10:16:06.769 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0013810090000000001 2024-09-18T10:16:06.769 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.770 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.770 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:16:06.770 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.666083+0000", 2024-09-18T10:16:06.770 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.103178495 2024-09-18T10:16:06.770 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.770 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.770 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:16:06.770 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:20.110956+0000", 2024-09-18T10:16:06.770 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.44487371999999997 2024-09-18T10:16:06.770 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.770 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.770 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:06.770 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:20.111076+0000", 2024-09-18T10:16:06.770 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000119436 2024-09-18T10:16:06.771 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:06.771 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:06.771 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:06.771 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.771 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.771 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:527 3.ds0 3:b1a14543:::smithi18925896-15:head [setxattr _header (57) in=7b] snapc 1a=[1a,19,18,16,12,11,10,f,e] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-18T10:16:06.771 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:19.554694+0000", 2024-09-18T10:16:06.771 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 47.137756670000002, 2024-09-18T10:16:06.771 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.55695282099999999, 2024-09-18T10:16:06.771 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:06.771 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:06.771 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:16:06.771 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:16:06.771 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:06.772 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 527 2024-09-18T10:16:06.772 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.772 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:06.772 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.772 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:06.772 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.554694+0000", 2024-09-18T10:16:06.772 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:06.772 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.772 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.772 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:06.772 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.554694+0000", 2024-09-18T10:16:06.772 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:06.772 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.772 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.773 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:06.773 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.554692+0000", 2024-09-18T10:16:06.773 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:16:06.773 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.773 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.773 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:06.773 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.554698+0000", 2024-09-18T10:16:06.773 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8599999999999998e-06 2024-09-18T10:16:06.773 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.773 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.773 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:06.773 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.554707+0000", 2024-09-18T10:16:06.773 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4090000000000003e-06 2024-09-18T10:16:06.774 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.774 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.774 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:06.774 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.554712+0000", 2024-09-18T10:16:06.774 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4500000000000003e-06 2024-09-18T10:16:06.774 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.774 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.774 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:06.774 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.663747+0000", 2024-09-18T10:16:06.774 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10903526400000001 2024-09-18T10:16:06.774 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.774 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.774 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:16:06.774 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.664066+0000", 2024-09-18T10:16:06.775 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00031885099999999999 2024-09-18T10:16:06.775 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.775 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.775 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:16:06.775 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.665080+0000", 2024-09-18T10:16:06.775 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010141200000000001 2024-09-18T10:16:06.775 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.775 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.775 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:16:06.775 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.666476+0000", 2024-09-18T10:16:06.775 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001395456 2024-09-18T10:16:06.775 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.775 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.776 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:16:06.776 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:20.111512+0000", 2024-09-18T10:16:06.776 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.445035929 2024-09-18T10:16:06.776 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.776 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.776 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:06.776 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:20.111646+0000", 2024-09-18T10:16:06.776 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000134542 2024-09-18T10:16:06.776 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:06.776 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:06.776 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:06.776 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.776 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.776 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:687 3.ds0 3:b1a14543:::smithi18925896-15:head [read 0~2899968,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-18T10:16:06.777 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:35.580026+0000", 2024-09-18T10:16:06.777 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 31.112424558000001, 2024-09-18T10:16:06.777 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.51409843, 2024-09-18T10:16:06.777 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:06.777 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-18T10:16:06.777 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:16:06.777 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:16:06.777 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:06.777 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 687 2024-09-18T10:16:06.777 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.777 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:06.777 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.777 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:06.778 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:35.580026+0000", 2024-09-18T10:16:06.778 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:06.778 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.778 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.778 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:06.778 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:35.580026+0000", 2024-09-18T10:16:06.778 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:06.778 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.778 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.778 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:06.778 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:35.580025+0000", 2024-09-18T10:16:06.778 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:16:06.778 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.778 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.779 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:06.779 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:35.580032+0000", 2024-09-18T10:16:06.779 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1500000000000002e-06 2024-09-18T10:16:06.779 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.779 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.779 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:06.779 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:35.580045+0000", 2024-09-18T10:16:06.779 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3086e-05 2024-09-18T10:16:06.779 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.779 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.779 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:06.779 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:35.580054+0000", 2024-09-18T10:16:06.779 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9770000000000006e-06 2024-09-18T10:16:06.780 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.780 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.780 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:06.780 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:35.580361+0000", 2024-09-18T10:16:06.780 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00030747400000000002 2024-09-18T10:16:06.780 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.780 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.780 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:16:06.780 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:35.580372+0000", 2024-09-18T10:16:06.780 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0550000000000001e-05 2024-09-18T10:16:06.780 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.780 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.780 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:06.780 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:35.686406+0000", 2024-09-18T10:16:06.781 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10603446 2024-09-18T10:16:06.781 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.781 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.781 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:16:06.781 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:35.686592+0000", 2024-09-18T10:16:06.781 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00018559300000000001 2024-09-18T10:16:06.781 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.781 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.781 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:06.781 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:36.094124+0000", 2024-09-18T10:16:06.781 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.40753219699999998 2024-09-18T10:16:06.781 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:06.781 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:06.782 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:06.782 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.782 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.782 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:767 3.as0 3:5c566071:::smithi18925896-28:head [read 0~557056,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T10:16:06.782 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:40.627001+0000", 2024-09-18T10:16:06.782 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 26.065448880000002, 2024-09-18T10:16:06.782 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.53039000400000003, 2024-09-18T10:16:06.782 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:06.782 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-18T10:16:06.782 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:16:06.782 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:16:06.782 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:06.782 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 767 2024-09-18T10:16:06.783 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.783 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:06.783 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.783 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:06.783 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.627001+0000", 2024-09-18T10:16:06.783 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:06.783 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.783 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.783 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:06.783 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.627001+0000", 2024-09-18T10:16:06.783 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:06.783 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.783 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.783 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:06.784 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.626999+0000", 2024-09-18T10:16:06.784 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999971 2024-09-18T10:16:06.784 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.784 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.784 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:06.784 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.627016+0000", 2024-09-18T10:16:06.784 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7051999999999999e-05 2024-09-18T10:16:06.784 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.784 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.784 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:06.784 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.627055+0000", 2024-09-18T10:16:06.784 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9099e-05 2024-09-18T10:16:06.784 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.785 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.785 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:06.785 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.727187+0000", 2024-09-18T10:16:06.785 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100131951 2024-09-18T10:16:06.785 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.785 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.785 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:06.785 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.727299+0000", 2024-09-18T10:16:06.785 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011210899999999999 2024-09-18T10:16:06.785 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.785 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.785 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-09-18T10:16:06.785 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.727308+0000", 2024-09-18T10:16:06.786 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7539999999999995e-06 2024-09-18T10:16:06.786 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.786 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.786 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:06.786 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.832882+0000", 2024-09-18T10:16:06.786 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.105574662 2024-09-18T10:16:06.786 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.786 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.786 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:16:06.786 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.833170+0000", 2024-09-18T10:16:06.786 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00028771299999999999 2024-09-18T10:16:06.786 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.786 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.786 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:06.787 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:41.157391+0000", 2024-09-18T10:16:06.787 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.324221431 2024-09-18T10:16:06.787 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:06.787 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:06.787 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:06.787 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.787 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.787 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:779 3.as0 3:5dc9cbed:::smithi18925896-13:head [copy-from ver 47 in=55b] snapc 2f=[2f,29,28,20,1f,1e,1a,11] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-18T10:16:06.787 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:40.727404+0000", 2024-09-18T10:16:06.787 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 25.965045907, 2024-09-18T10:16:06.787 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.73144490900000003, 2024-09-18T10:16:06.787 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:06.787 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:06.787 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:16:06.788 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:16:06.788 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:06.788 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 779 2024-09-18T10:16:06.788 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.788 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:06.788 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.788 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:06.788 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.727404+0000", 2024-09-18T10:16:06.788 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:06.788 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.788 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.788 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:06.788 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.727404+0000", 2024-09-18T10:16:06.789 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:06.789 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.789 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.789 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:06.789 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.727403+0000", 2024-09-18T10:16:06.789 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999986 2024-09-18T10:16:06.789 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.789 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.789 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:06.789 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.727413+0000", 2024-09-18T10:16:06.789 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6940000000000007e-06 2024-09-18T10:16:06.789 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.789 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.790 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:06.790 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.727426+0000", 2024-09-18T10:16:06.790 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2896e-05 2024-09-18T10:16:06.790 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.790 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.790 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:06.790 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.727440+0000", 2024-09-18T10:16:06.790 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4285e-05 2024-09-18T10:16:06.790 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.790 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.790 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:06.790 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.727569+0000", 2024-09-18T10:16:06.790 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000128906 2024-09-18T10:16:06.790 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.791 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.791 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:16:06.791 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.727580+0000", 2024-09-18T10:16:06.791 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0720000000000001e-05 2024-09-18T10:16:06.791 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.791 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.791 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:06.791 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.936548+0000", 2024-09-18T10:16:06.791 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20896883299999999 2024-09-18T10:16:06.791 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.791 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.791 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:16:06.791 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.936872+0000", 2024-09-18T10:16:06.792 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00032318499999999998 2024-09-18T10:16:06.792 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.792 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.792 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:16:06.792 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:41.066546+0000", 2024-09-18T10:16:06.792 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.12967457600000001 2024-09-18T10:16:06.792 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.792 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.792 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:16:06.792 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:41.068954+0000", 2024-09-18T10:16:06.792 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002408154 2024-09-18T10:16:06.792 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.793 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.793 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:16:06.793 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:41.458701+0000", 2024-09-18T10:16:06.793 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.38974719899999999 2024-09-18T10:16:06.793 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.793 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.793 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:06.793 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:41.458849+0000", 2024-09-18T10:16:06.793 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014777799999999999 2024-09-18T10:16:06.793 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:06.793 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:06.793 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:06.793 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.794 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.794 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:872 3.fs0 3:f1c2a816:::smithi18925896-35:head [append 0~688128] snapc 33=[33,30,2f,28,20,1f,1a,11] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-18T10:16:06.794 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:45.694098+0000", 2024-09-18T10:16:06.794 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 20.998352566000001, 2024-09-18T10:16:06.794 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.45094221000000001, 2024-09-18T10:16:06.794 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:06.794 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:06.794 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:16:06.794 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:16:06.794 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:06.794 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 872 2024-09-18T10:16:06.794 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.794 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:06.794 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.795 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:06.795 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.694098+0000", 2024-09-18T10:16:06.795 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:06.795 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.795 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.795 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:06.795 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.694098+0000", 2024-09-18T10:16:06.795 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:06.795 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.795 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.795 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:06.795 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.694096+0000", 2024-09-18T10:16:06.795 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999981 2024-09-18T10:16:06.796 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.796 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.796 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:06.796 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.694986+0000", 2024-09-18T10:16:06.796 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00089013899999999997 2024-09-18T10:16:06.796 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.796 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.796 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:06.796 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.695010+0000", 2024-09-18T10:16:06.796 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4301999999999999e-05 2024-09-18T10:16:06.796 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.796 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.796 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:06.796 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.695020+0000", 2024-09-18T10:16:06.797 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0118999999999999e-05 2024-09-18T10:16:06.797 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.797 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.797 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:06.797 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.896885+0000", 2024-09-18T10:16:06.797 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20186479600000001 2024-09-18T10:16:06.797 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.797 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.797 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:16:06.797 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.897089+0000", 2024-09-18T10:16:06.797 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00020354699999999999 2024-09-18T10:16:06.797 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.797 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.798 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:16:06.798 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.898797+0000", 2024-09-18T10:16:06.798 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001708892 2024-09-18T10:16:06.798 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.798 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.798 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:16:06.798 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:46.003761+0000", 2024-09-18T10:16:06.798 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.104963166 2024-09-18T10:16:06.798 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.798 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.798 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:16:06.798 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:46.144837+0000", 2024-09-18T10:16:06.798 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.14107618899999999 2024-09-18T10:16:06.799 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.799 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.799 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:06.799 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:46.145040+0000", 2024-09-18T10:16:06.799 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00020301999999999999 2024-09-18T10:16:06.799 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:06.799 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:06.799 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:06.799 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.799 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.799 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:873 3.fs0 3:f1c2a816:::smithi18925896-35:head [append 0~245760] snapc 33=[33,30,2f,28,20,1f,1a,11] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-18T10:16:06.799 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:45.695912+0000", 2024-09-18T10:16:06.799 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 20.996537896, 2024-09-18T10:16:06.799 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.44955376499999999, 2024-09-18T10:16:06.800 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:06.800 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:06.800 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:16:06.800 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:16:06.800 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:06.800 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 873 2024-09-18T10:16:06.800 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.800 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:06.800 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.800 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:06.800 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.695912+0000", 2024-09-18T10:16:06.800 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:06.800 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.801 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.801 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:06.801 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.695912+0000", 2024-09-18T10:16:06.801 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:06.801 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.801 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.801 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:06.801 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.695911+0000", 2024-09-18T10:16:06.801 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999986 2024-09-18T10:16:06.801 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.801 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.801 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:06.801 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.696222+0000", 2024-09-18T10:16:06.802 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00031110699999999997 2024-09-18T10:16:06.802 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.802 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.802 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:06.802 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.696246+0000", 2024-09-18T10:16:06.802 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4116000000000001e-05 2024-09-18T10:16:06.802 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.802 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.802 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:06.802 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.696257+0000", 2024-09-18T10:16:06.802 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1304999999999999e-05 2024-09-18T10:16:06.802 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.802 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.803 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:06.803 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.900046+0000", 2024-09-18T10:16:06.803 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20378872100000001 2024-09-18T10:16:06.803 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.803 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.803 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:16:06.803 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.900178+0000", 2024-09-18T10:16:06.803 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013230100000000001 2024-09-18T10:16:06.803 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.803 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.803 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:16:06.803 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.901011+0000", 2024-09-18T10:16:06.803 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00083270399999999997 2024-09-18T10:16:06.803 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.804 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.804 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:16:06.804 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:46.003763+0000", 2024-09-18T10:16:06.804 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.102751922 2024-09-18T10:16:06.804 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.804 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.804 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:16:06.804 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:46.145329+0000", 2024-09-18T10:16:06.804 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.141565937 2024-09-18T10:16:06.804 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.804 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.804 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:06.804 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:46.145466+0000", 2024-09-18T10:16:06.804 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013717199999999999 2024-09-18T10:16:06.805 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:06.805 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:06.805 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:06.805 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.805 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.805 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:874 3.fs0 3:f1c2a816:::smithi18925896-35:head [setxattr _header (57) in=7b] snapc 33=[33,30,2f,28,20,1f,1a,11] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-18T10:16:06.805 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:45.696272+0000", 2024-09-18T10:16:06.805 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 20.996178599, 2024-09-18T10:16:06.805 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.64978053899999999, 2024-09-18T10:16:06.805 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:06.805 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:06.805 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:16:06.805 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:16:06.806 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:06.806 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 874 2024-09-18T10:16:06.806 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.806 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:06.806 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.806 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:06.806 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.696272+0000", 2024-09-18T10:16:06.806 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:06.806 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.806 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.806 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:06.806 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.696272+0000", 2024-09-18T10:16:06.806 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:06.806 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.807 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.807 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:06.807 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.696270+0000", 2024-09-18T10:16:06.807 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999986 2024-09-18T10:16:06.807 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.807 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.807 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:06.807 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.696278+0000", 2024-09-18T10:16:06.807 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9799999999999998e-06 2024-09-18T10:16:06.807 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.807 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.807 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:06.807 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.696293+0000", 2024-09-18T10:16:06.807 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4842e-05 2024-09-18T10:16:06.808 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.808 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.808 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:06.808 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.696303+0000", 2024-09-18T10:16:06.808 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8879999999999992e-06 2024-09-18T10:16:06.808 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.808 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.808 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:06.808 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:46.001771+0000", 2024-09-18T10:16:06.808 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30546843499999998 2024-09-18T10:16:06.808 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.808 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.808 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:16:06.809 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:46.001984+0000", 2024-09-18T10:16:06.809 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000212456 2024-09-18T10:16:06.809 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.809 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.809 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:16:06.809 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:46.002868+0000", 2024-09-18T10:16:06.809 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00088443599999999995 2024-09-18T10:16:06.809 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.809 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.809 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:16:06.809 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:46.004417+0000", 2024-09-18T10:16:06.809 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001548724 2024-09-18T10:16:06.809 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.809 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.810 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:16:06.810 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:46.345827+0000", 2024-09-18T10:16:06.810 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.34141017200000001 2024-09-18T10:16:06.810 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.810 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.810 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:06.810 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:46.346052+0000", 2024-09-18T10:16:06.810 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00022503200000000001 2024-09-18T10:16:06.810 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:06.810 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:06.810 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:06.810 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.810 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.810 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:875 3.fs0 3:f1c2a816:::smithi18925896-35:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e90)", 2024-09-18T10:16:06.811 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:45.696315+0000", 2024-09-18T10:16:06.811 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 20.996135563999999, 2024-09-18T10:16:06.811 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.95397443599999998, 2024-09-18T10:16:06.811 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:06.811 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-18T10:16:06.811 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:16:06.811 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:16:06.811 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:06.811 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 875 2024-09-18T10:16:06.811 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.811 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:06.811 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.811 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:06.811 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.696315+0000", 2024-09-18T10:16:06.812 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:06.812 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.812 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.812 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:06.812 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.696315+0000", 2024-09-18T10:16:06.812 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:06.812 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.812 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.812 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:06.812 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.696313+0000", 2024-09-18T10:16:06.812 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:16:06.812 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.812 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.812 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:06.813 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.696320+0000", 2024-09-18T10:16:06.813 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0759999999999999e-06 2024-09-18T10:16:06.813 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.813 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.813 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:06.813 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.696340+0000", 2024-09-18T10:16:06.813 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0465000000000001e-05 2024-09-18T10:16:06.813 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.813 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.813 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:06.813 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.696352+0000", 2024-09-18T10:16:06.813 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1732e-05 2024-09-18T10:16:06.813 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.814 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.814 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:06.814 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:46.105544+0000", 2024-09-18T10:16:06.814 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.40919195000000003 2024-09-18T10:16:06.814 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.814 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.814 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2024-09-18T10:16:06.814 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:46.105758+0000", 2024-09-18T10:16:06.814 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00021451599999999999 2024-09-18T10:16:06.814 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.814 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.814 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:06.814 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:46.347069+0000", 2024-09-18T10:16:06.814 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.24131116899999999 2024-09-18T10:16:06.815 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.815 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.815 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:16:06.815 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:46.347236+0000", 2024-09-18T10:16:06.815 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00016639399999999999 2024-09-18T10:16:06.815 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:06.815 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:06.815 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:06.815 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:46.650289+0000", 2024-09-18T10:16:06.815 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30305331899999999 2024-09-18T10:16:06.815 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:06.815 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:06.815 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:06.815 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:06.816 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:06.816 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-18T10:16:06.816 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-18T10:16:06.817 INFO:teuthology.orchestra.run.smithi029.stderr:set osd.14 primary-affinity to 1 (8655362) 2024-09-18T10:16:06.818 INFO:tasks.ceph.osd.14.smithi154.stderr:2024-09-18T10:16:06.814+0000 7f75a19e7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 22093) UID: 0 2024-09-18T10:16:06.819 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T10:16:06.918 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-09-18T10:16:06.914+0000 7fdfa8698700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22087) UID: 0 2024-09-18T10:16:06.926 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-09-18T10:16:06.926 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-09-18T10:16:06.927 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-09-18T10:16:06.927 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-09-18T10:16:06.935 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T10:16:07.019 INFO:tasks.ceph.osd.8.smithi029.stderr:2024-09-18T10:16:07.013+0000 7f2555c27700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 22192) UID: 0 2024-09-18T10:16:07.036 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-09-18T10:16:07.036 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-09-18T10:16:07.036 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-09-18T10:16:07.036 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-09-18T10:16:07.036 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-09-18T10:16:07.045 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T10:16:07.121 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-18T10:16:07.117+0000 7f14dcc5b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22398) UID: 0 2024-09-18T10:16:07.145 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-09-18T10:16:07.145 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-09-18T10:16:07.145 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-09-18T10:16:07.145 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-09-18T10:16:07.145 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.145 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.9s1 36/32 ECSubWrite(tid=16, reqid=client.4632.0:143, at_version=36'12, trim_to=0'0, roll_forward_to=36'9))", 2024-09-18T10:16:07.145 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:51.130949+0000", 2024-09-18T10:16:07.145 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 76.009903492999996, 2024-09-18T10:16:07.146 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.202020333, 2024-09-18T10:16:07.146 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:07.146 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:16:07.146 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:07.146 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.146 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:07.146 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.130949+0000", 2024-09-18T10:16:07.146 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:07.146 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.146 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.146 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:07.146 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.130949+0000", 2024-09-18T10:16:07.146 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:07.146 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.147 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.147 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:07.147 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.130947+0000", 2024-09-18T10:16:07.147 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:16:07.147 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.147 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.147 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:07.147 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.131149+0000", 2024-09-18T10:16:07.147 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00020114800000000001 2024-09-18T10:16:07.147 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.147 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.147 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:07.147 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.131182+0000", 2024-09-18T10:16:07.148 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.3844000000000003e-05 2024-09-18T10:16:07.148 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.148 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.148 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:07.148 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.131204+0000", 2024-09-18T10:16:07.148 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.1753999999999999e-05 2024-09-18T10:16:07.148 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.148 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.148 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:07.148 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.231367+0000", 2024-09-18T10:16:07.148 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100162472 2024-09-18T10:16:07.148 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.148 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.148 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:07.149 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.231380+0000", 2024-09-18T10:16:07.149 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3624e-05 2024-09-18T10:16:07.149 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.149 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.149 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:07.149 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.332934+0000", 2024-09-18T10:16:07.149 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.101553714 2024-09-18T10:16:07.149 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.149 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.149 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:07.151 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.332969+0000", 2024-09-18T10:16:07.151 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5525999999999998e-05 2024-09-18T10:16:07.151 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:07.151 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:07.151 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:07.151 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.151 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.152 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.9s1 36/32 ECSubWrite(tid=17, reqid=client.4632.0:144, at_version=36'13, trim_to=0'0, roll_forward_to=36'9))", 2024-09-18T10:16:07.152 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:51.131916+0000", 2024-09-18T10:16:07.152 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 76.008936665999997, 2024-09-18T10:16:07.152 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20110955899999999, 2024-09-18T10:16:07.152 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:07.152 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:16:07.152 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:07.152 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.152 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:07.152 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.131916+0000", 2024-09-18T10:16:07.152 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:07.152 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.152 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.153 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:07.153 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.131916+0000", 2024-09-18T10:16:07.153 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:07.153 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.153 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.153 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:07.153 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.131915+0000", 2024-09-18T10:16:07.153 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:16:07.153 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.153 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.153 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:07.153 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.131949+0000", 2024-09-18T10:16:07.153 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.4576999999999998e-05 2024-09-18T10:16:07.154 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.154 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.154 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:07.154 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.131979+0000", 2024-09-18T10:16:07.154 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9454e-05 2024-09-18T10:16:07.154 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.154 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.154 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:07.154 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.131992+0000", 2024-09-18T10:16:07.154 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3029e-05 2024-09-18T10:16:07.154 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.154 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.154 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:07.155 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.332112+0000", 2024-09-18T10:16:07.155 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.200120295 2024-09-18T10:16:07.155 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.155 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.155 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:07.156 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.332126+0000", 2024-09-18T10:16:07.156 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4494999999999999e-05 2024-09-18T10:16:07.156 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.156 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.156 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:07.156 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.332996+0000", 2024-09-18T10:16:07.156 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00086989000000000001 2024-09-18T10:16:07.156 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.156 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.156 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:07.156 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.333026+0000", 2024-09-18T10:16:07.156 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9159999999999999e-05 2024-09-18T10:16:07.157 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:07.157 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:07.157 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:07.157 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.158 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.158 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpRead(3.4s3 36/32 ECSubRead(tid=11, to_read={3:2650be24:::smithi18925896-45:head=0,4096,0}, subchunks={3:2650be24:::smithi18925896-45:head=[0,1]}, attrs_to_read=))", 2024-09-18T10:16:07.158 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:51.425236+0000", 2024-09-18T10:16:07.158 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 75.715616578999999, 2024-09-18T10:16:07.158 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.200812289, 2024-09-18T10:16:07.158 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:07.159 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:16:07.159 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:07.159 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.160 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:07.160 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.425236+0000", 2024-09-18T10:16:07.160 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:07.160 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.160 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.160 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:07.160 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.425236+0000", 2024-09-18T10:16:07.160 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:07.160 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.160 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.160 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:07.160 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.425234+0000", 2024-09-18T10:16:07.160 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:16:07.161 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.161 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.161 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:07.161 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.425253+0000", 2024-09-18T10:16:07.161 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8468999999999999e-05 2024-09-18T10:16:07.161 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.161 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.161 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:07.161 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.425289+0000", 2024-09-18T10:16:07.161 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.6514999999999999e-05 2024-09-18T10:16:07.161 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.161 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.161 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:07.162 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.525424+0000", 2024-09-18T10:16:07.162 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100135157 2024-09-18T10:16:07.162 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.162 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.162 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:07.162 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.625672+0000", 2024-09-18T10:16:07.162 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100247555 2024-09-18T10:16:07.162 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.162 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.162 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:07.162 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.626048+0000", 2024-09-18T10:16:07.162 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00037635000000000001 2024-09-18T10:16:07.162 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:07.163 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:07.163 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:07.163 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.163 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.163 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.9s1 37/32 ECSubWrite(tid=30, reqid=client.4632.0:187, at_version=0'0, trim_to=0'0, roll_forward_to=37'17))", 2024-09-18T10:16:07.163 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:53.090512+0000", 2024-09-18T10:16:07.163 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 74.050340724999998, 2024-09-18T10:16:07.163 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.21478031, 2024-09-18T10:16:07.163 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:07.163 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:16:07.163 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:07.163 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.163 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:07.163 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.090512+0000", 2024-09-18T10:16:07.164 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:07.164 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.164 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.164 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:07.164 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.090512+0000", 2024-09-18T10:16:07.164 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:07.164 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.164 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.164 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:07.164 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.090510+0000", 2024-09-18T10:16:07.164 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:16:07.164 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.164 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.165 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:07.165 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.090535+0000", 2024-09-18T10:16:07.165 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4848999999999999e-05 2024-09-18T10:16:07.165 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.165 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.165 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:07.165 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.090566+0000", 2024-09-18T10:16:07.165 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1096000000000002e-05 2024-09-18T10:16:07.165 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.165 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.166 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:07.166 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.190713+0000", 2024-09-18T10:16:07.166 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10014709300000001 2024-09-18T10:16:07.166 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.166 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.166 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:07.166 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.290966+0000", 2024-09-18T10:16:07.166 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10025297399999999 2024-09-18T10:16:07.166 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.166 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.166 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:07.166 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.290981+0000", 2024-09-18T10:16:07.167 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4960000000000001e-05 2024-09-18T10:16:07.167 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.167 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.167 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:07.167 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.305252+0000", 2024-09-18T10:16:07.167 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.014270738999999999 2024-09-18T10:16:07.167 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.167 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.167 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:07.167 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.305292+0000", 2024-09-18T10:16:07.167 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.0092000000000002e-05 2024-09-18T10:16:07.167 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:07.168 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:07.168 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:07.168 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.168 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.168 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.6s3 41/40 ECSubWrite(tid=32, reqid=client.4632.0:250, at_version=41'14, trim_to=0'0, roll_forward_to=0'0))", 2024-09-18T10:16:07.168 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:57.552410+0000", 2024-09-18T10:16:07.168 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 69.588442619999995, 2024-09-18T10:16:07.168 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20275494299999999, 2024-09-18T10:16:07.168 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:07.168 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:16:07.169 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:07.169 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.169 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:07.169 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.552410+0000", 2024-09-18T10:16:07.169 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:07.169 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.169 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.169 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:07.169 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.552410+0000", 2024-09-18T10:16:07.169 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:07.169 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.169 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.169 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:07.169 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.552409+0000", 2024-09-18T10:16:07.170 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999995 2024-09-18T10:16:07.170 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.170 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.170 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:07.170 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.552478+0000", 2024-09-18T10:16:07.170 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.9084999999999993e-05 2024-09-18T10:16:07.170 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.170 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.170 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:07.170 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.552488+0000", 2024-09-18T10:16:07.170 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.7990000000000005e-06 2024-09-18T10:16:07.170 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.171 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.171 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:07.171 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.552496+0000", 2024-09-18T10:16:07.171 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.3200000000000002e-06 2024-09-18T10:16:07.171 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.171 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.171 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:07.171 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.653130+0000", 2024-09-18T10:16:07.171 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100634549 2024-09-18T10:16:07.171 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.171 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.171 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:07.171 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.653144+0000", 2024-09-18T10:16:07.171 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3619e-05 2024-09-18T10:16:07.172 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.172 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.172 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:07.172 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.755128+0000", 2024-09-18T10:16:07.172 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.101983924 2024-09-18T10:16:07.172 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.172 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.172 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:07.172 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.755165+0000", 2024-09-18T10:16:07.172 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.7361999999999999e-05 2024-09-18T10:16:07.172 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:07.172 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:07.172 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:07.172 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.173 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.173 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.6s3 41/40 ECSubWrite(tid=33, reqid=client.4632.0:251, at_version=41'15, trim_to=0'0, roll_forward_to=0'0))", 2024-09-18T10:16:07.173 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:57.553805+0000", 2024-09-18T10:16:07.173 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 69.587048039999999, 2024-09-18T10:16:07.173 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20199056600000001, 2024-09-18T10:16:07.173 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:07.173 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:16:07.173 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:07.173 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.173 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:07.173 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.553805+0000", 2024-09-18T10:16:07.173 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:07.173 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.173 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.174 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:07.174 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.553805+0000", 2024-09-18T10:16:07.174 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:07.174 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.174 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.174 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:07.174 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.553804+0000", 2024-09-18T10:16:07.174 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:16:07.174 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.174 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.174 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:07.174 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.553827+0000", 2024-09-18T10:16:07.174 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.3516e-05 2024-09-18T10:16:07.174 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.175 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.175 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:07.175 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.553845+0000", 2024-09-18T10:16:07.175 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7881999999999999e-05 2024-09-18T10:16:07.175 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.175 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.175 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:07.175 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.553853+0000", 2024-09-18T10:16:07.175 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.7109999999999995e-06 2024-09-18T10:16:07.175 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.175 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.175 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:07.175 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.754527+0000", 2024-09-18T10:16:07.175 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.200674198 2024-09-18T10:16:07.175 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.176 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.176 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:07.176 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.754543+0000", 2024-09-18T10:16:07.176 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.5682e-05 2024-09-18T10:16:07.176 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.176 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.176 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:07.176 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.755758+0000", 2024-09-18T10:16:07.176 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0012155860000000001 2024-09-18T10:16:07.176 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.176 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.176 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:07.176 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.755795+0000", 2024-09-18T10:16:07.177 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.6808999999999999e-05 2024-09-18T10:16:07.177 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:07.177 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:07.177 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:07.177 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.177 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.177 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.19s1 51/40 ECSubWrite(tid=77, reqid=client.4632.0:407, at_version=51'45, trim_to=0'0, roll_forward_to=50'39))", 2024-09-18T10:16:07.177 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:06.877639+0000", 2024-09-18T10:16:07.177 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 60.263213415999999, 2024-09-18T10:16:07.177 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.301778663, 2024-09-18T10:16:07.177 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:07.177 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:16:07.177 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:07.177 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.177 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:07.178 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:06.877639+0000", 2024-09-18T10:16:07.178 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:07.178 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.178 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.178 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:07.178 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:06.877639+0000", 2024-09-18T10:16:07.178 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:07.178 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.178 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.178 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:07.178 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:06.877638+0000", 2024-09-18T10:16:07.178 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:16:07.178 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.178 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.178 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:07.179 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:06.877679+0000", 2024-09-18T10:16:07.179 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.0908000000000002e-05 2024-09-18T10:16:07.179 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.179 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.179 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:07.179 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:06.877717+0000", 2024-09-18T10:16:07.179 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.7988999999999998e-05 2024-09-18T10:16:07.179 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.179 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.179 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:07.179 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:06.977811+0000", 2024-09-18T10:16:07.179 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100094533 2024-09-18T10:16:07.179 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.180 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.180 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:07.180 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:06.977916+0000", 2024-09-18T10:16:07.180 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000104823 2024-09-18T10:16:07.180 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.180 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.180 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:07.180 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:06.977932+0000", 2024-09-18T10:16:07.180 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.6099999999999998e-05 2024-09-18T10:16:07.180 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.180 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.180 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:07.181 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:07.179377+0000", 2024-09-18T10:16:07.181 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.201444861 2024-09-18T10:16:07.181 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.181 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.181 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:07.181 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:07.179418+0000", 2024-09-18T10:16:07.181 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.0645999999999997e-05 2024-09-18T10:16:07.181 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:07.181 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:07.181 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:07.181 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.181 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.181 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.19s1 51/40 ECSubWrite(tid=78, reqid=client.4632.0:408, at_version=51'46, trim_to=0'0, roll_forward_to=50'39))", 2024-09-18T10:16:07.182 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:06.977840+0000", 2024-09-18T10:16:07.182 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 60.163012711, 2024-09-18T10:16:07.182 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.201619306, 2024-09-18T10:16:07.182 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:07.182 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:16:07.182 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:07.182 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.182 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:07.182 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:06.977840+0000", 2024-09-18T10:16:07.182 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:07.182 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.182 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.182 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:07.182 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:06.977840+0000", 2024-09-18T10:16:07.183 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:07.183 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.183 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.183 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:07.183 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:06.977839+0000", 2024-09-18T10:16:07.183 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:16:07.183 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.183 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.183 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:07.183 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:06.977887+0000", 2024-09-18T10:16:07.183 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.8470999999999997e-05 2024-09-18T10:16:07.183 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.183 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.183 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:07.184 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:06.977924+0000", 2024-09-18T10:16:07.184 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.6729e-05 2024-09-18T10:16:07.184 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.184 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.184 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:07.184 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:06.977937+0000", 2024-09-18T10:16:07.184 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2476e-05 2024-09-18T10:16:07.184 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.184 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.184 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:07.184 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:07.078614+0000", 2024-09-18T10:16:07.184 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100677007 2024-09-18T10:16:07.184 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.184 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.185 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:07.185 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:07.078634+0000", 2024-09-18T10:16:07.185 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.0134000000000001e-05 2024-09-18T10:16:07.185 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.185 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.185 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:07.185 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:07.179429+0000", 2024-09-18T10:16:07.185 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100795736 2024-09-18T10:16:07.185 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.185 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.185 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:07.185 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:07.179459+0000", 2024-09-18T10:16:07.185 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9788999999999999e-05 2024-09-18T10:16:07.185 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:07.186 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:07.186 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:07.186 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.186 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.186 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.1s6 58/40 ECSubWrite(tid=113, reqid=osd.4.0:112, at_version=0'0, trim_to=0'0, roll_forward_to=57'20))", 2024-09-18T10:16:07.186 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:13.649564+0000", 2024-09-18T10:16:07.186 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 53.491288251999997, 2024-09-18T10:16:07.186 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.204868989, 2024-09-18T10:16:07.186 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:07.186 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:16:07.186 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:07.186 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.186 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:07.186 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.649564+0000", 2024-09-18T10:16:07.186 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:07.187 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.187 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.187 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:07.187 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.649564+0000", 2024-09-18T10:16:07.187 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:07.187 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.187 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.187 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:07.187 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.649563+0000", 2024-09-18T10:16:07.187 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:16:07.187 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.187 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.187 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:07.187 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.649600+0000", 2024-09-18T10:16:07.188 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.7224999999999997e-05 2024-09-18T10:16:07.188 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.188 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.188 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:07.188 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.649651+0000", 2024-09-18T10:16:07.188 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.0905000000000001e-05 2024-09-18T10:16:07.188 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.188 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.188 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:07.188 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.649669+0000", 2024-09-18T10:16:07.188 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8522999999999999e-05 2024-09-18T10:16:07.188 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.188 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.189 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:07.189 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.750460+0000", 2024-09-18T10:16:07.189 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100791013 2024-09-18T10:16:07.189 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.189 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.189 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:07.189 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.750487+0000", 2024-09-18T10:16:07.189 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6389000000000001e-05 2024-09-18T10:16:07.189 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.189 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.189 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:07.189 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.854315+0000", 2024-09-18T10:16:07.189 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.103828238 2024-09-18T10:16:07.189 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.190 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.190 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:07.190 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.854433+0000", 2024-09-18T10:16:07.190 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000118449 2024-09-18T10:16:07.190 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:07.190 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:07.190 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:07.190 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.190 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.190 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.19s1 58/40 ECSubWrite(tid=97, reqid=osd.7.0:96, at_version=0'0, trim_to=0'0, roll_forward_to=57'54))", 2024-09-18T10:16:07.190 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:13.672411+0000", 2024-09-18T10:16:07.190 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 53.468441822999999, 2024-09-18T10:16:07.191 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.28289547100000001, 2024-09-18T10:16:07.191 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:07.191 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:16:07.191 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:07.191 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.191 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:07.191 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.672411+0000", 2024-09-18T10:16:07.191 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:07.191 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.191 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.191 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:07.191 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.672411+0000", 2024-09-18T10:16:07.191 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:07.192 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.192 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.192 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:07.192 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.672410+0000", 2024-09-18T10:16:07.192 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999995 2024-09-18T10:16:07.192 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.192 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.192 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:07.192 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.672428+0000", 2024-09-18T10:16:07.192 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7116e-05 2024-09-18T10:16:07.192 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.192 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.192 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:07.193 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.672451+0000", 2024-09-18T10:16:07.193 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.3611000000000001e-05 2024-09-18T10:16:07.193 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.193 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.193 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:07.193 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.672465+0000", 2024-09-18T10:16:07.193 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3818999999999999e-05 2024-09-18T10:16:07.193 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.193 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.193 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:07.193 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.853423+0000", 2024-09-18T10:16:07.193 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.180957599 2024-09-18T10:16:07.193 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.194 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.194 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:07.194 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.853438+0000", 2024-09-18T10:16:07.194 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.5446000000000001e-05 2024-09-18T10:16:07.194 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.194 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.194 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:07.194 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.955241+0000", 2024-09-18T10:16:07.194 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.101803053 2024-09-18T10:16:07.194 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.194 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.194 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:07.194 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.955306+0000", 2024-09-18T10:16:07.195 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.5215000000000005e-05 2024-09-18T10:16:07.195 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:07.195 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:07.195 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:07.195 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.195 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.195 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.1s6 58/40 ECSubWrite(tid=120, reqid=osd.4.0:117, at_version=0'0, trim_to=0'0, roll_forward_to=58'22))", 2024-09-18T10:16:07.195 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:14.170897+0000", 2024-09-18T10:16:07.195 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 52.969955693999999, 2024-09-18T10:16:07.195 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20147283599999999, 2024-09-18T10:16:07.195 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:07.195 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:16:07.195 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:07.195 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.196 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:07.196 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:14.170897+0000", 2024-09-18T10:16:07.196 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:07.196 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.196 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.196 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:07.196 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:14.170897+0000", 2024-09-18T10:16:07.196 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:07.196 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.196 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.196 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:07.196 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:14.170895+0000", 2024-09-18T10:16:07.196 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:16:07.197 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.197 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.197 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:07.197 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:14.170921+0000", 2024-09-18T10:16:07.197 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5848999999999999e-05 2024-09-18T10:16:07.197 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.197 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.197 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:07.197 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:14.170998+0000", 2024-09-18T10:16:07.197 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.7341000000000004e-05 2024-09-18T10:16:07.197 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.197 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.198 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:07.198 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:14.171019+0000", 2024-09-18T10:16:07.198 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.0443000000000001e-05 2024-09-18T10:16:07.198 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.198 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.198 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:07.198 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:14.271283+0000", 2024-09-18T10:16:07.198 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100263848 2024-09-18T10:16:07.198 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.198 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.198 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:07.198 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:14.271316+0000", 2024-09-18T10:16:07.198 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.3516000000000002e-05 2024-09-18T10:16:07.199 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.199 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.199 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:07.199 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:14.372304+0000", 2024-09-18T10:16:07.199 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100987524 2024-09-18T10:16:07.199 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.199 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.199 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:07.199 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:14.372370+0000", 2024-09-18T10:16:07.199 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.6107999999999998e-05 2024-09-18T10:16:07.199 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:07.199 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:07.199 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:07.200 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.200 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.200 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.9s1 63/40 ECSubWrite(tid=118, reqid=client.4632.0:531, at_version=63'56, trim_to=0'0, roll_forward_to=60'52))", 2024-09-18T10:16:07.200 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:19.565882+0000", 2024-09-18T10:16:07.200 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 47.574970749999999, 2024-09-18T10:16:07.200 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.201669185, 2024-09-18T10:16:07.200 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:07.200 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:16:07.200 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:07.200 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.200 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:07.200 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.565882+0000", 2024-09-18T10:16:07.200 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:07.201 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.201 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.201 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:07.201 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.565882+0000", 2024-09-18T10:16:07.201 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:07.201 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.201 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.201 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:07.201 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.565881+0000", 2024-09-18T10:16:07.201 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:16:07.201 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.201 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.201 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:07.202 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.565919+0000", 2024-09-18T10:16:07.202 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8254999999999999e-05 2024-09-18T10:16:07.202 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.202 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.202 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:07.202 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.565952+0000", 2024-09-18T10:16:07.202 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.3219000000000002e-05 2024-09-18T10:16:07.202 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.202 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.202 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:07.202 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.565965+0000", 2024-09-18T10:16:07.202 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.272e-05 2024-09-18T10:16:07.202 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.202 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.203 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:07.203 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.766254+0000", 2024-09-18T10:16:07.203 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.200289512 2024-09-18T10:16:07.203 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.203 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.203 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:07.203 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.766272+0000", 2024-09-18T10:16:07.203 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7306e-05 2024-09-18T10:16:07.203 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.203 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.203 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:07.203 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.767501+0000", 2024-09-18T10:16:07.203 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001229794 2024-09-18T10:16:07.204 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.204 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.204 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:07.204 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.767551+0000", 2024-09-18T10:16:07.204 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9662e-05 2024-09-18T10:16:07.204 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:07.204 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:07.204 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:07.204 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.204 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.204 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.4s3 66/40 ECSubWrite(tid=57, reqid=osd.8.0:57, at_version=66'40, trim_to=0'0, roll_forward_to=66'38))", 2024-09-18T10:16:07.204 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:22.543395+0000", 2024-09-18T10:16:07.205 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 44.597457976000001, 2024-09-18T10:16:07.205 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20190205, 2024-09-18T10:16:07.205 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:07.205 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:16:07.205 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:07.205 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.205 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:07.205 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:22.543395+0000", 2024-09-18T10:16:07.205 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:07.205 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.205 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.205 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:07.205 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:22.543395+0000", 2024-09-18T10:16:07.205 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:07.206 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.206 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.206 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:07.206 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:22.543393+0000", 2024-09-18T10:16:07.206 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:16:07.206 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.206 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.206 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:07.206 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:22.543438+0000", 2024-09-18T10:16:07.206 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.5504999999999999e-05 2024-09-18T10:16:07.206 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.206 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.207 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:07.207 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:22.543471+0000", 2024-09-18T10:16:07.207 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2969000000000002e-05 2024-09-18T10:16:07.207 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.207 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.207 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:07.207 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:22.643568+0000", 2024-09-18T10:16:07.207 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100096327 2024-09-18T10:16:07.207 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.207 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.207 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:07.207 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:22.743796+0000", 2024-09-18T10:16:07.207 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100228206 2024-09-18T10:16:07.207 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.208 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.208 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:07.208 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:22.743818+0000", 2024-09-18T10:16:07.208 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.1778999999999999e-05 2024-09-18T10:16:07.208 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.208 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.208 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:07.208 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:22.745263+0000", 2024-09-18T10:16:07.208 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0014448729999999999 2024-09-18T10:16:07.208 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.208 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.209 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:07.209 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:22.745297+0000", 2024-09-18T10:16:07.209 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.4190999999999998e-05 2024-09-18T10:16:07.209 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:07.209 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:07.209 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:07.209 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.209 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.209 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.6s3 72/40 ECSubWrite(tid=62, reqid=osd.0.0:62, at_version=72'38, trim_to=0'0, roll_forward_to=71'36))", 2024-09-18T10:16:07.209 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:28.189402+0000", 2024-09-18T10:16:07.209 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 38.951450434000002, 2024-09-18T10:16:07.209 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.201226193, 2024-09-18T10:16:07.209 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:07.210 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:16:07.210 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:07.210 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.210 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:07.210 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.189402+0000", 2024-09-18T10:16:07.210 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:07.210 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.210 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.210 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:07.210 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.189402+0000", 2024-09-18T10:16:07.210 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:07.210 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.211 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.211 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:07.211 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.189402+0000", 2024-09-18T10:16:07.211 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999995 2024-09-18T10:16:07.211 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.211 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.211 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:07.211 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.189461+0000", 2024-09-18T10:16:07.211 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.9077999999999999e-05 2024-09-18T10:16:07.211 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.211 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.211 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:07.211 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.189484+0000", 2024-09-18T10:16:07.211 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.3353000000000002e-05 2024-09-18T10:16:07.212 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.212 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.212 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:07.212 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.289578+0000", 2024-09-18T10:16:07.212 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100094161 2024-09-18T10:16:07.212 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.212 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.212 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:07.212 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.389741+0000", 2024-09-18T10:16:07.212 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100162923 2024-09-18T10:16:07.212 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.212 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.212 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:07.213 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.389749+0000", 2024-09-18T10:16:07.213 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.2889999999999998e-06 2024-09-18T10:16:07.213 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.213 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.213 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:07.213 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.390593+0000", 2024-09-18T10:16:07.213 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00084384499999999997 2024-09-18T10:16:07.213 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.213 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.213 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:07.213 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.390628+0000", 2024-09-18T10:16:07.213 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5120999999999998e-05 2024-09-18T10:16:07.213 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:07.214 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:07.214 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:07.214 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.214 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.214 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.6s3 74/40 ECSubWrite(tid=64, reqid=osd.0.0:64, at_version=74'40, trim_to=0'0, roll_forward_to=72'38))", 2024-09-18T10:16:07.214 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:29.762852+0000", 2024-09-18T10:16:07.214 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 37.378000311999998, 2024-09-18T10:16:07.214 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.202373787, 2024-09-18T10:16:07.214 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:07.214 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:16:07.214 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:07.214 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.214 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:07.214 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.762852+0000", 2024-09-18T10:16:07.215 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:07.215 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.215 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.215 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:07.215 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.762852+0000", 2024-09-18T10:16:07.215 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:07.215 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.215 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.215 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:07.215 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.762852+0000", 2024-09-18T10:16:07.215 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999995 2024-09-18T10:16:07.215 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.215 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.216 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:07.216 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.762877+0000", 2024-09-18T10:16:07.216 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4759000000000001e-05 2024-09-18T10:16:07.216 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.216 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.216 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:07.216 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.762885+0000", 2024-09-18T10:16:07.216 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.7549999999999994e-06 2024-09-18T10:16:07.216 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.216 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.216 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:07.216 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.762900+0000", 2024-09-18T10:16:07.216 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.5364999999999999e-05 2024-09-18T10:16:07.217 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.217 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.217 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:07.217 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.963772+0000", 2024-09-18T10:16:07.217 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20087190899999999 2024-09-18T10:16:07.217 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.217 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.217 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:07.217 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.963781+0000", 2024-09-18T10:16:07.217 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.516e-06 2024-09-18T10:16:07.217 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.217 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.217 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:07.217 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.965178+0000", 2024-09-18T10:16:07.218 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0013961959999999999 2024-09-18T10:16:07.218 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.218 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.218 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:07.218 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.965226+0000", 2024-09-18T10:16:07.218 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.8578000000000001e-05 2024-09-18T10:16:07.218 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:07.218 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:07.218 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:07.218 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.218 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.218 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.17s4 85/64 ECSubWrite(tid=100, reqid=client.4632.0:758, at_version=85'61, trim_to=0'0, roll_forward_to=82'60))", 2024-09-18T10:16:07.218 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:40.728936+0000", 2024-09-18T10:16:07.218 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 26.411916391999998, 2024-09-18T10:16:07.218 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20385209100000001, 2024-09-18T10:16:07.218 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:07.219 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:16:07.219 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:07.219 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.219 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:07.219 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.728936+0000", 2024-09-18T10:16:07.219 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:07.219 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.219 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.219 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:07.219 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.728936+0000", 2024-09-18T10:16:07.219 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:07.219 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.219 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.219 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:07.220 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.728936+0000", 2024-09-18T10:16:07.220 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999995 2024-09-18T10:16:07.220 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.220 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.220 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:07.220 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.729098+0000", 2024-09-18T10:16:07.220 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00016212099999999999 2024-09-18T10:16:07.220 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.220 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.220 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:07.220 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.729129+0000", 2024-09-18T10:16:07.220 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0626000000000001e-05 2024-09-18T10:16:07.220 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.221 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.221 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:07.221 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.829269+0000", 2024-09-18T10:16:07.221 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100140541 2024-09-18T10:16:07.221 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.221 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.221 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:07.221 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.829412+0000", 2024-09-18T10:16:07.221 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00014265099999999999 2024-09-18T10:16:07.221 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.221 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.221 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:07.222 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.829431+0000", 2024-09-18T10:16:07.222 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8935000000000001e-05 2024-09-18T10:16:07.222 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.222 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.222 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:07.222 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.932702+0000", 2024-09-18T10:16:07.222 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.103271298 2024-09-18T10:16:07.222 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.222 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.223 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:07.223 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.932788+0000", 2024-09-18T10:16:07.223 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.6337999999999999e-05 2024-09-18T10:16:07.223 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:07.223 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:07.223 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:07.223 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.223 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.223 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpRead(3.19s3 85/83 ECSubRead(tid=220, to_read={3:9ab7c4c0:::smithi18925896-46:1a=0,512000,0}, subchunks={3:9ab7c4c0:::smithi18925896-46:1a=[0,1]}, attrs_to_read=))", 2024-09-18T10:16:07.223 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:41.177225+0000", 2024-09-18T10:16:07.223 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 25.963627172999999, 2024-09-18T10:16:07.223 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20067484299999999, 2024-09-18T10:16:07.223 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:07.223 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:16:07.223 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:07.223 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.224 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:07.224 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:41.177225+0000", 2024-09-18T10:16:07.224 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:07.224 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.224 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.224 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:07.224 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:41.177225+0000", 2024-09-18T10:16:07.224 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:07.224 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.224 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.224 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:07.224 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:41.177224+0000", 2024-09-18T10:16:07.224 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:16:07.224 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.224 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.225 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:07.225 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:41.177243+0000", 2024-09-18T10:16:07.225 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.9130000000000001e-05 2024-09-18T10:16:07.225 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.225 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.225 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:07.225 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:41.177280+0000", 2024-09-18T10:16:07.225 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.6646000000000001e-05 2024-09-18T10:16:07.225 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.225 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.225 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:07.225 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:41.277374+0000", 2024-09-18T10:16:07.225 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10009406899999999 2024-09-18T10:16:07.225 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.225 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.226 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:07.226 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:41.377630+0000", 2024-09-18T10:16:07.226 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10025651100000001 2024-09-18T10:16:07.226 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.226 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.226 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:07.226 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:41.377900+0000", 2024-09-18T10:16:07.226 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00027003699999999999 2024-09-18T10:16:07.226 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:07.226 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:07.226 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:07.226 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.226 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.227 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.4s3 85/82 ECSubWrite(tid=17, reqid=client.4632.0:748, at_version=0'0, trim_to=0'0, roll_forward_to=85'84))", 2024-09-18T10:16:07.227 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:41.216977+0000", 2024-09-18T10:16:07.227 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 25.923875452000001, 2024-09-18T10:16:07.227 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.203185543, 2024-09-18T10:16:07.227 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:07.227 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:16:07.227 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:07.227 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.227 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:07.227 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:41.216977+0000", 2024-09-18T10:16:07.227 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:07.227 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.227 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.228 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:07.228 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:41.216977+0000", 2024-09-18T10:16:07.228 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:07.228 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.228 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.228 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:07.228 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:41.216975+0000", 2024-09-18T10:16:07.228 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:16:07.228 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.228 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.228 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:07.228 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:41.217039+0000", 2024-09-18T10:16:07.228 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.4134999999999995e-05 2024-09-18T10:16:07.229 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.229 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.229 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:07.229 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:41.217084+0000", 2024-09-18T10:16:07.229 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.5092000000000001e-05 2024-09-18T10:16:07.229 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.229 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.229 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:07.229 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:41.217104+0000", 2024-09-18T10:16:07.229 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.9562999999999999e-05 2024-09-18T10:16:07.229 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.229 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.229 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:07.229 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:41.317676+0000", 2024-09-18T10:16:07.230 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10057176600000001 2024-09-18T10:16:07.230 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.230 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.230 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:07.230 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:41.317706+0000", 2024-09-18T10:16:07.230 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0426e-05 2024-09-18T10:16:07.230 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.230 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.230 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:07.230 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:41.420137+0000", 2024-09-18T10:16:07.230 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.102431151 2024-09-18T10:16:07.230 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.230 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.230 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:07.230 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:41.420163+0000", 2024-09-18T10:16:07.231 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5522999999999999e-05 2024-09-18T10:16:07.231 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:07.231 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:07.231 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:07.231 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.231 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.231 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.4s3 89/82 ECSubWrite(tid=32, reqid=osd.12.0:32, at_version=89'92, trim_to=0'0, roll_forward_to=88'90))", 2024-09-18T10:16:07.231 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:44.897313+0000", 2024-09-18T10:16:07.231 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 22.243539423000001, 2024-09-18T10:16:07.231 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.202844106, 2024-09-18T10:16:07.231 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:07.231 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:16:07.231 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:07.231 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.231 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:07.232 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:44.897313+0000", 2024-09-18T10:16:07.232 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:07.232 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.232 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.232 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:07.232 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:44.897313+0000", 2024-09-18T10:16:07.232 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:07.232 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.232 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.232 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:07.232 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:44.897313+0000", 2024-09-18T10:16:07.232 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999995 2024-09-18T10:16:07.232 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.232 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.232 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:07.233 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:44.897371+0000", 2024-09-18T10:16:07.233 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.8105e-05 2024-09-18T10:16:07.233 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.233 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.233 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:07.233 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:44.897400+0000", 2024-09-18T10:16:07.233 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9246000000000001e-05 2024-09-18T10:16:07.233 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.233 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.233 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:07.233 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:44.997512+0000", 2024-09-18T10:16:07.233 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10011223599999999 2024-09-18T10:16:07.233 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.233 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.234 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:07.234 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:45.097821+0000", 2024-09-18T10:16:07.234 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10030881899999999 2024-09-18T10:16:07.234 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.234 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.234 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:07.234 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:45.097855+0000", 2024-09-18T10:16:07.234 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.4421999999999997e-05 2024-09-18T10:16:07.234 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.234 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.234 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:07.234 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:45.100104+0000", 2024-09-18T10:16:07.234 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0022489089999999999 2024-09-18T10:16:07.234 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.234 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.234 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:07.235 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:45.100157+0000", 2024-09-18T10:16:07.235 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.3049e-05 2024-09-18T10:16:07.235 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:07.235 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:07.235 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:07.235 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.235 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.235 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.19s3 104/95 ECSubWrite(tid=158, reqid=osd.8.0:158, at_version=104'138, trim_to=0'0, roll_forward_to=104'136))", 2024-09-18T10:16:07.235 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:16:01.116991+0000", 2024-09-18T10:16:07.235 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 6.0238616450000002, 2024-09-18T10:16:07.235 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.200810707, 2024-09-18T10:16:07.235 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:07.235 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:16:07.235 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:07.235 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.236 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:07.236 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:16:01.116991+0000", 2024-09-18T10:16:07.236 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:07.236 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.236 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.236 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:07.236 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:16:01.116991+0000", 2024-09-18T10:16:07.236 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:07.236 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.236 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.236 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:07.236 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:16:01.116991+0000", 2024-09-18T10:16:07.236 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999995 2024-09-18T10:16:07.236 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.237 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.237 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:07.237 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:16:01.117020+0000", 2024-09-18T10:16:07.237 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9003999999999999e-05 2024-09-18T10:16:07.237 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.237 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.237 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:07.237 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:16:01.117029+0000", 2024-09-18T10:16:07.237 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.1789999999999997e-06 2024-09-18T10:16:07.237 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.237 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.237 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:07.237 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:16:01.117039+0000", 2024-09-18T10:16:07.237 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0463e-05 2024-09-18T10:16:07.237 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.238 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.238 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:07.238 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:16:01.119741+0000", 2024-09-18T10:16:07.238 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.002701813 2024-09-18T10:16:07.238 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.238 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.238 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:07.238 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:16:01.119759+0000", 2024-09-18T10:16:07.238 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7768999999999999e-05 2024-09-18T10:16:07.238 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.238 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.238 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:07.238 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:16:01.317738+0000", 2024-09-18T10:16:07.238 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.19797904199999999 2024-09-18T10:16:07.239 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.239 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.239 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:07.239 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:16:01.317802+0000", 2024-09-18T10:16:07.239 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.3738999999999994e-05 2024-09-18T10:16:07.239 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:07.239 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:07.239 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:07.239 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:07.239 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:07.239 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-09-18T10:16:07.241 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T10:16:07.241 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-18T10:16:07.235+0000 7f578c52f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22134) UID: 0 2024-09-18T10:16:07.342 INFO:tasks.ceph.osd.15.smithi189.stderr:2024-09-18T10:16:07.337+0000 7fd0e725e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 22401) UID: 0 2024-09-18T10:16:07.350 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-09-18T10:16:07.350 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-09-18T10:16:07.350 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-09-18T10:16:07.350 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-09-18T10:16:07.360 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T10:16:07.443 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-18T10:16:07.439+0000 7f00046a8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 22136) UID: 0 2024-09-18T10:16:07.462 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-09-18T10:16:07.462 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-09-18T10:16:07.462 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-09-18T10:16:07.462 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-09-18T10:16:07.462 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-09-18T10:16:07.471 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T10:16:07.543 INFO:tasks.ceph.osd.11.smithi189.stderr:2024-09-18T10:16:07.537+0000 7f718b60a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 22400) UID: 0 2024-09-18T10:16:07.572 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-09-18T10:16:07.572 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-09-18T10:16:07.572 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-09-18T10:16:07.572 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-09-18T10:16:07.572 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.572 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T10:16:07.572 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:36.786936+0000", 2024-09-18T10:16:07.572 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 90.780961500999993, 2024-09-18T10:16:07.572 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.71130157999999999, 2024-09-18T10:16:07.573 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:07.573 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:07.573 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:07.573 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:16:07.573 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:16:07.573 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 72 2024-09-18T10:16:07.573 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.573 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:07.573 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.573 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:07.573 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.786936+0000", 2024-09-18T10:16:07.574 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:07.574 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.574 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.574 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:07.574 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.786936+0000", 2024-09-18T10:16:07.574 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:07.574 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.574 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.574 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:07.574 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.786935+0000", 2024-09-18T10:16:07.574 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999995 2024-09-18T10:16:07.575 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.575 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.575 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:07.575 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787018+0000", 2024-09-18T10:16:07.575 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.2511999999999999e-05 2024-09-18T10:16:07.575 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.575 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.575 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:07.575 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787028+0000", 2024-09-18T10:16:07.575 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0709e-05 2024-09-18T10:16:07.575 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.575 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.576 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:07.576 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787034+0000", 2024-09-18T10:16:07.576 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.7769999999999999e-06 2024-09-18T10:16:07.576 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.576 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.576 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:07.576 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.295095+0000", 2024-09-18T10:16:07.576 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.50806109099999996 2024-09-18T10:16:07.576 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.576 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.576 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:07.576 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.295221+0000", 2024-09-18T10:16:07.577 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00012563599999999999 2024-09-18T10:16:07.577 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.577 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.577 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:16:07.577 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.295453+0000", 2024-09-18T10:16:07.577 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000232387 2024-09-18T10:16:07.577 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.577 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.577 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:16:07.577 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.301373+0000", 2024-09-18T10:16:07.577 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0059202279999999996 2024-09-18T10:16:07.577 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.578 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.578 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:16:07.578 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.498044+0000", 2024-09-18T10:16:07.578 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.19667104699999999 2024-09-18T10:16:07.578 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.578 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.578 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:07.578 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.498133+0000", 2024-09-18T10:16:07.578 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.9044999999999999e-05 2024-09-18T10:16:07.578 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.578 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.578 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:07.578 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.498237+0000", 2024-09-18T10:16:07.579 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00010385100000000001 2024-09-18T10:16:07.579 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:07.579 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:07.579 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:07.579 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.579 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.579 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T10:16:07.579 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:36.787042+0000", 2024-09-18T10:16:07.579 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 90.780854794000007, 2024-09-18T10:16:07.579 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.71152325100000002, 2024-09-18T10:16:07.579 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:07.580 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:07.580 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:07.580 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:16:07.580 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:16:07.580 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 73 2024-09-18T10:16:07.580 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.580 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:07.580 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.580 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:07.580 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787042+0000", 2024-09-18T10:16:07.580 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:07.580 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.581 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.581 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:07.581 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787042+0000", 2024-09-18T10:16:07.581 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:07.581 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.581 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.581 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:07.581 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787042+0000", 2024-09-18T10:16:07.581 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:16:07.581 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.581 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.581 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:07.581 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787048+0000", 2024-09-18T10:16:07.582 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.8900000000000001e-06 2024-09-18T10:16:07.582 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.582 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.582 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:07.582 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787057+0000", 2024-09-18T10:16:07.582 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.4430000000000003e-06 2024-09-18T10:16:07.582 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.582 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.582 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:07.582 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787063+0000", 2024-09-18T10:16:07.582 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.6459999999999998e-06 2024-09-18T10:16:07.582 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.583 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.583 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:07.583 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.297548+0000", 2024-09-18T10:16:07.583 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.51048589499999997 2024-09-18T10:16:07.583 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.583 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.583 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:07.583 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.297668+0000", 2024-09-18T10:16:07.583 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00011917 2024-09-18T10:16:07.583 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.583 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.583 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:16:07.584 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.297891+0000", 2024-09-18T10:16:07.584 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00022301999999999999 2024-09-18T10:16:07.584 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.584 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.584 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:16:07.584 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.301383+0000", 2024-09-18T10:16:07.584 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.003492762 2024-09-18T10:16:07.584 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.584 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.584 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:16:07.584 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.498374+0000", 2024-09-18T10:16:07.584 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.19699053899999999 2024-09-18T10:16:07.585 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.585 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.585 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:07.585 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.498506+0000", 2024-09-18T10:16:07.585 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00013246899999999999 2024-09-18T10:16:07.585 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.585 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.585 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:07.585 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.498566+0000", 2024-09-18T10:16:07.585 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.9274000000000002e-05 2024-09-18T10:16:07.585 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:07.585 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:07.586 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:07.586 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.586 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.586 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T10:16:07.586 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:36.787128+0000", 2024-09-18T10:16:07.586 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 90.780769485999997, 2024-09-18T10:16:07.586 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.71169541800000002, 2024-09-18T10:16:07.586 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:07.586 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:07.586 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:07.586 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:16:07.586 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:16:07.587 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 74 2024-09-18T10:16:07.587 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.587 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:07.587 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.587 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:07.587 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787128+0000", 2024-09-18T10:16:07.587 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:07.587 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.587 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.587 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:07.587 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787128+0000", 2024-09-18T10:16:07.587 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:07.587 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.588 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.588 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:07.588 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787127+0000", 2024-09-18T10:16:07.588 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:16:07.588 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.588 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.588 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:07.588 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787133+0000", 2024-09-18T10:16:07.588 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.6930000000000004e-06 2024-09-18T10:16:07.588 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.588 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.588 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:07.589 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787142+0000", 2024-09-18T10:16:07.589 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.1959999999999997e-06 2024-09-18T10:16:07.589 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.589 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.589 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:07.589 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787150+0000", 2024-09-18T10:16:07.589 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.6399999999999997e-06 2024-09-18T10:16:07.589 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.589 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.589 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:07.589 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.298925+0000", 2024-09-18T10:16:07.589 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.51177523599999997 2024-09-18T10:16:07.590 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.590 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.590 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:07.590 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.299064+0000", 2024-09-18T10:16:07.590 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000139622 2024-09-18T10:16:07.590 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.590 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.590 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:16:07.590 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.299287+0000", 2024-09-18T10:16:07.590 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00022309700000000001 2024-09-18T10:16:07.590 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.590 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.591 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:16:07.591 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.301389+0000", 2024-09-18T10:16:07.591 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0021018759999999999 2024-09-18T10:16:07.591 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.591 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.591 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:16:07.591 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.498648+0000", 2024-09-18T10:16:07.591 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.19725873999999999 2024-09-18T10:16:07.591 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.591 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.591 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:07.591 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.498731+0000", 2024-09-18T10:16:07.591 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.2429000000000006e-05 2024-09-18T10:16:07.592 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.592 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.592 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:07.592 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.498823+0000", 2024-09-18T10:16:07.592 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.2655e-05 2024-09-18T10:16:07.592 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:07.592 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:07.592 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:07.592 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.592 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.592 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T10:16:07.592 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:36.787157+0000", 2024-09-18T10:16:07.593 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 90.780740261000005, 2024-09-18T10:16:07.593 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.81204889999999996, 2024-09-18T10:16:07.593 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:07.593 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:07.593 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:07.593 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:16:07.593 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:16:07.593 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 75 2024-09-18T10:16:07.593 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.593 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:07.593 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.593 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:07.594 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787157+0000", 2024-09-18T10:16:07.594 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:07.594 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.594 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.594 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:07.594 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787157+0000", 2024-09-18T10:16:07.594 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:07.594 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.594 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.594 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:07.594 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787156+0000", 2024-09-18T10:16:07.594 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:16:07.595 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.595 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.595 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:07.595 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787236+0000", 2024-09-18T10:16:07.595 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.0175999999999999e-05 2024-09-18T10:16:07.595 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.595 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.595 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:07.595 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787247+0000", 2024-09-18T10:16:07.595 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0448999999999999e-05 2024-09-18T10:16:07.595 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.596 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.596 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:07.596 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787253+0000", 2024-09-18T10:16:07.596 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.2469999999999997e-06 2024-09-18T10:16:07.596 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.596 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.596 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:07.596 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.299974+0000", 2024-09-18T10:16:07.596 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.51272114499999999 2024-09-18T10:16:07.596 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.596 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.596 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:07.596 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.300115+0000", 2024-09-18T10:16:07.597 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000141452 2024-09-18T10:16:07.597 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.597 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.597 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:16:07.597 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.300349+0000", 2024-09-18T10:16:07.597 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00023363999999999999 2024-09-18T10:16:07.597 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.597 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.597 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:16:07.597 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.301947+0000", 2024-09-18T10:16:07.597 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0015976510000000001 2024-09-18T10:16:07.597 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.598 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.598 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:16:07.598 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.599027+0000", 2024-09-18T10:16:07.598 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.29708051600000002 2024-09-18T10:16:07.598 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.598 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.598 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:07.598 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.599106+0000", 2024-09-18T10:16:07.598 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.9033e-05 2024-09-18T10:16:07.598 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.598 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.598 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:07.599 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.599206+0000", 2024-09-18T10:16:07.599 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.9561000000000004e-05 2024-09-18T10:16:07.599 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:07.599 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:07.599 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:07.599 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.599 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.599 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T10:16:07.599 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:36.787261+0000", 2024-09-18T10:16:07.599 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 90.780636610000002, 2024-09-18T10:16:07.599 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.81223649499999995, 2024-09-18T10:16:07.599 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:07.600 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:07.600 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:07.600 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:16:07.600 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:16:07.600 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 76 2024-09-18T10:16:07.600 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.600 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:07.600 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.600 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:07.600 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787261+0000", 2024-09-18T10:16:07.600 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:07.601 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.601 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.601 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:07.601 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787261+0000", 2024-09-18T10:16:07.601 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:07.601 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.601 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.601 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:07.601 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787260+0000", 2024-09-18T10:16:07.601 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:16:07.601 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.601 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.601 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:07.602 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787265+0000", 2024-09-18T10:16:07.602 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1590000000000001e-06 2024-09-18T10:16:07.602 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.602 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.602 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:07.602 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787273+0000", 2024-09-18T10:16:07.602 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.4100000000000008e-06 2024-09-18T10:16:07.602 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.602 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.602 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:07.602 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787279+0000", 2024-09-18T10:16:07.602 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.3630000000000002e-06 2024-09-18T10:16:07.602 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.602 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.603 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:07.603 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.496955+0000", 2024-09-18T10:16:07.603 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.70967585099999997 2024-09-18T10:16:07.603 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.603 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.603 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:07.603 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.497107+0000", 2024-09-18T10:16:07.603 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000152044 2024-09-18T10:16:07.603 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.603 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.603 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:16:07.604 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.497345+0000", 2024-09-18T10:16:07.604 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000238693 2024-09-18T10:16:07.604 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.604 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.604 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:16:07.604 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.599248+0000", 2024-09-18T10:16:07.604 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.101902836 2024-09-18T10:16:07.604 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.604 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.604 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:16:07.604 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.599329+0000", 2024-09-18T10:16:07.604 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.1064999999999997e-05 2024-09-18T10:16:07.605 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.605 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.605 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:07.605 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.599441+0000", 2024-09-18T10:16:07.605 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000111797 2024-09-18T10:16:07.605 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.605 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.605 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:07.605 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.599497+0000", 2024-09-18T10:16:07.605 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.6119999999999998e-05 2024-09-18T10:16:07.605 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:07.605 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:07.605 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:07.605 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.606 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.606 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:93 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T10:16:07.606 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:38.616472+0000", 2024-09-18T10:16:07.606 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 88.951425326999995, 2024-09-18T10:16:07.606 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.60940747100000003, 2024-09-18T10:16:07.606 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:07.606 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:07.606 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:07.606 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:16:07.606 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:16:07.606 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 93 2024-09-18T10:16:07.606 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.607 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:07.607 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.607 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:07.607 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.616472+0000", 2024-09-18T10:16:07.607 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:07.607 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.607 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.607 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:07.607 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.616472+0000", 2024-09-18T10:16:07.607 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:07.607 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.607 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.607 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:07.608 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.616470+0000", 2024-09-18T10:16:07.608 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:16:07.608 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.608 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.608 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:07.608 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.616644+0000", 2024-09-18T10:16:07.608 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00017372000000000001 2024-09-18T10:16:07.608 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.608 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.608 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:07.608 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.616695+0000", 2024-09-18T10:16:07.608 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.0991e-05 2024-09-18T10:16:07.608 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.609 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.609 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:07.609 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.616709+0000", 2024-09-18T10:16:07.609 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4008999999999999e-05 2024-09-18T10:16:07.609 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.609 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.609 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:07.609 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.720960+0000", 2024-09-18T10:16:07.609 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.104250624 2024-09-18T10:16:07.609 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.609 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.609 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:07.610 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.721068+0000", 2024-09-18T10:16:07.610 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00010814700000000001 2024-09-18T10:16:07.610 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.610 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.610 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:16:07.610 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.721270+0000", 2024-09-18T10:16:07.610 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00020184200000000001 2024-09-18T10:16:07.610 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.610 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.610 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:16:07.610 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.024691+0000", 2024-09-18T10:16:07.610 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.30342105600000002 2024-09-18T10:16:07.611 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.611 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.611 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:16:07.611 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.225725+0000", 2024-09-18T10:16:07.611 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20103395099999999 2024-09-18T10:16:07.611 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.611 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.611 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:07.611 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.225816+0000", 2024-09-18T10:16:07.611 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.1928999999999993e-05 2024-09-18T10:16:07.611 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.611 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.612 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:07.612 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.225879+0000", 2024-09-18T10:16:07.612 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.2918999999999998e-05 2024-09-18T10:16:07.612 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:07.612 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:07.612 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:07.612 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.612 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.612 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:94 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T10:16:07.612 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:38.616729+0000", 2024-09-18T10:16:07.612 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 88.951168301999999, 2024-09-18T10:16:07.612 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.60943170999999996, 2024-09-18T10:16:07.612 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:07.613 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:07.613 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:07.613 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:16:07.613 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:16:07.613 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 94 2024-09-18T10:16:07.613 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.613 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:07.613 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.613 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:07.613 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.616729+0000", 2024-09-18T10:16:07.613 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:07.613 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.613 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.614 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:07.614 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.616729+0000", 2024-09-18T10:16:07.614 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:07.614 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.614 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.614 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:07.614 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.616727+0000", 2024-09-18T10:16:07.614 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:16:07.614 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.614 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.614 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:07.614 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.616738+0000", 2024-09-18T10:16:07.614 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0776e-05 2024-09-18T10:16:07.615 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.615 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.615 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:07.615 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.616756+0000", 2024-09-18T10:16:07.615 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8079000000000001e-05 2024-09-18T10:16:07.615 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.615 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.615 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:07.615 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.716882+0000", 2024-09-18T10:16:07.615 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10012612999999999 2024-09-18T10:16:07.615 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.615 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.615 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:07.616 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.822315+0000", 2024-09-18T10:16:07.616 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.105433162 2024-09-18T10:16:07.616 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.616 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.616 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:07.616 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.822453+0000", 2024-09-18T10:16:07.616 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000137396 2024-09-18T10:16:07.616 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.616 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.616 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:16:07.616 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.822674+0000", 2024-09-18T10:16:07.616 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00022154400000000001 2024-09-18T10:16:07.616 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.617 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.617 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:16:07.617 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.024888+0000", 2024-09-18T10:16:07.617 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20221397199999999 2024-09-18T10:16:07.617 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.617 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.617 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:16:07.617 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.226011+0000", 2024-09-18T10:16:07.617 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20112316299999999 2024-09-18T10:16:07.617 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.617 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.617 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:07.617 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.226094+0000", 2024-09-18T10:16:07.617 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.2155000000000002e-05 2024-09-18T10:16:07.618 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.618 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.618 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:07.618 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.226161+0000", 2024-09-18T10:16:07.618 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.7138999999999995e-05 2024-09-18T10:16:07.618 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:07.618 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:07.618 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:07.618 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.618 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.618 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:95 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T10:16:07.618 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:38.716909+0000", 2024-09-18T10:16:07.619 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 88.850988588000007, 2024-09-18T10:16:07.619 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.60960763600000001, 2024-09-18T10:16:07.619 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:07.619 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:07.619 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:07.619 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:16:07.619 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:16:07.619 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 95 2024-09-18T10:16:07.619 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.619 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:07.619 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.619 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:07.619 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.716909+0000", 2024-09-18T10:16:07.619 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:07.620 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.620 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.620 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:07.620 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.716909+0000", 2024-09-18T10:16:07.620 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:07.620 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.620 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.620 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:07.620 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.716907+0000", 2024-09-18T10:16:07.620 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:16:07.620 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.620 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.621 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:07.621 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.716921+0000", 2024-09-18T10:16:07.621 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4572e-05 2024-09-18T10:16:07.621 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.621 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.621 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:07.621 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.716939+0000", 2024-09-18T10:16:07.621 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7149000000000001e-05 2024-09-18T10:16:07.621 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.621 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.621 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:07.621 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.716948+0000", 2024-09-18T10:16:07.621 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.7000000000000003e-06 2024-09-18T10:16:07.622 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.622 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.622 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:07.622 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.923430+0000", 2024-09-18T10:16:07.622 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20648133299999999 2024-09-18T10:16:07.622 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.622 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.622 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:07.622 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.923566+0000", 2024-09-18T10:16:07.622 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00013657599999999999 2024-09-18T10:16:07.622 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.622 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.622 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:16:07.622 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.923782+0000", 2024-09-18T10:16:07.623 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00021599400000000001 2024-09-18T10:16:07.623 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.623 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.623 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:16:07.623 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.024899+0000", 2024-09-18T10:16:07.623 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10111666800000001 2024-09-18T10:16:07.623 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.623 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.623 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:16:07.623 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.326379+0000", 2024-09-18T10:16:07.623 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.30148048799999999 2024-09-18T10:16:07.623 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.623 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.624 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:07.624 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.326454+0000", 2024-09-18T10:16:07.624 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.4988999999999996e-05 2024-09-18T10:16:07.624 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.624 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.624 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:07.624 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.326516+0000", 2024-09-18T10:16:07.624 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.1902999999999993e-05 2024-09-18T10:16:07.624 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:07.624 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:07.624 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:07.624 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.624 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.625 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:96 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T10:16:07.625 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:38.716972+0000", 2024-09-18T10:16:07.625 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 88.850925598000003, 2024-09-18T10:16:07.625 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.91539460800000005, 2024-09-18T10:16:07.625 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:07.625 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:07.625 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:07.625 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:16:07.625 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:16:07.625 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 96 2024-09-18T10:16:07.625 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.625 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:07.625 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.626 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:07.626 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.716972+0000", 2024-09-18T10:16:07.626 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:07.626 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.626 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.626 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:07.626 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.716972+0000", 2024-09-18T10:16:07.626 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:07.626 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.626 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.626 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:07.627 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.716971+0000", 2024-09-18T10:16:07.627 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:16:07.627 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.627 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.627 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:07.627 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717061+0000", 2024-09-18T10:16:07.627 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.0744999999999999e-05 2024-09-18T10:16:07.627 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.627 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.627 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:07.627 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717073+0000", 2024-09-18T10:16:07.628 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1943000000000001e-05 2024-09-18T10:16:07.628 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.628 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.628 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:07.628 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717080+0000", 2024-09-18T10:16:07.628 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0060000000000003e-06 2024-09-18T10:16:07.628 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.628 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.628 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:07.628 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.426708+0000", 2024-09-18T10:16:07.628 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.70962730399999996 2024-09-18T10:16:07.628 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.628 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.629 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:07.629 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.426854+0000", 2024-09-18T10:16:07.629 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00014654199999999999 2024-09-18T10:16:07.629 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.629 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.629 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:16:07.629 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.427106+0000", 2024-09-18T10:16:07.629 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00025226600000000001 2024-09-18T10:16:07.629 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.629 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.629 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:16:07.629 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.433087+0000", 2024-09-18T10:16:07.629 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0059804860000000001 2024-09-18T10:16:07.630 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.630 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.630 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:16:07.630 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.632096+0000", 2024-09-18T10:16:07.630 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.19900922099999999 2024-09-18T10:16:07.630 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.630 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.630 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:07.630 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.632257+0000", 2024-09-18T10:16:07.630 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00016058399999999999 2024-09-18T10:16:07.630 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.630 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.630 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:07.631 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.632366+0000", 2024-09-18T10:16:07.631 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000109601 2024-09-18T10:16:07.631 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:07.631 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:07.631 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:07.631 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.631 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.631 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:97 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T10:16:07.631 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:38.717093+0000", 2024-09-18T10:16:07.631 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 88.850803787999993, 2024-09-18T10:16:07.631 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.91561460900000002, 2024-09-18T10:16:07.631 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:07.631 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:07.631 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:07.631 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:16:07.632 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:16:07.632 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 97 2024-09-18T10:16:07.632 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.632 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:07.632 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.632 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:07.632 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717093+0000", 2024-09-18T10:16:07.632 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:07.632 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.632 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.632 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:07.632 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717093+0000", 2024-09-18T10:16:07.633 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:07.633 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.633 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.633 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:07.633 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717092+0000", 2024-09-18T10:16:07.633 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:16:07.633 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.633 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.633 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:07.633 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717099+0000", 2024-09-18T10:16:07.633 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.246e-06 2024-09-18T10:16:07.633 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.633 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.634 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:07.634 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717109+0000", 2024-09-18T10:16:07.634 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0499999999999999e-05 2024-09-18T10:16:07.634 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.634 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.634 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:07.634 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717115+0000", 2024-09-18T10:16:07.634 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.7690000000000002e-06 2024-09-18T10:16:07.634 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.634 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.634 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:07.634 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.428509+0000", 2024-09-18T10:16:07.634 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.71139444299999999 2024-09-18T10:16:07.635 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.635 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.635 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:07.635 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.428621+0000", 2024-09-18T10:16:07.635 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00011156 2024-09-18T10:16:07.635 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.635 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.635 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:16:07.635 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.428805+0000", 2024-09-18T10:16:07.635 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000183597 2024-09-18T10:16:07.635 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.635 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.635 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:16:07.635 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.433106+0000", 2024-09-18T10:16:07.636 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0043015019999999996 2024-09-18T10:16:07.636 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.636 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.636 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:16:07.636 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.632474+0000", 2024-09-18T10:16:07.636 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.199368092 2024-09-18T10:16:07.636 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.636 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.636 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:07.636 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.632605+0000", 2024-09-18T10:16:07.636 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00013110899999999999 2024-09-18T10:16:07.636 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.636 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.636 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:07.637 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.632708+0000", 2024-09-18T10:16:07.637 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000102748 2024-09-18T10:16:07.637 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:07.637 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:07.637 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:07.637 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.637 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.637 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:98 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T10:16:07.637 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:38.717126+0000", 2024-09-18T10:16:07.637 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 88.850771526000003, 2024-09-18T10:16:07.637 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.015084141, 2024-09-18T10:16:07.637 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:07.637 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:07.638 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:07.638 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:16:07.638 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:16:07.638 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 98 2024-09-18T10:16:07.638 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.638 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:07.638 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.638 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:07.638 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717126+0000", 2024-09-18T10:16:07.638 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:07.638 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.638 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.638 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:07.638 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717126+0000", 2024-09-18T10:16:07.639 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:07.639 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.639 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.639 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:07.639 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717125+0000", 2024-09-18T10:16:07.639 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:16:07.639 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.639 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.639 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:07.639 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717130+0000", 2024-09-18T10:16:07.639 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.4160000000000003e-06 2024-09-18T10:16:07.639 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.639 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.640 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:07.640 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717139+0000", 2024-09-18T10:16:07.640 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.6230000000000003e-06 2024-09-18T10:16:07.640 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.640 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.640 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:07.640 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717144+0000", 2024-09-18T10:16:07.640 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5330000000000002e-06 2024-09-18T10:16:07.640 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.640 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.640 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:07.640 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.429491+0000", 2024-09-18T10:16:07.640 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.71234629500000002 2024-09-18T10:16:07.640 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.641 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.641 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:07.641 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.429741+0000", 2024-09-18T10:16:07.641 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000250516 2024-09-18T10:16:07.641 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.641 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.641 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:16:07.641 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.429989+0000", 2024-09-18T10:16:07.641 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00024811599999999999 2024-09-18T10:16:07.641 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.641 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.641 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:16:07.642 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.435056+0000", 2024-09-18T10:16:07.642 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0050669670000000003 2024-09-18T10:16:07.642 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.642 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.642 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:16:07.642 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.732068+0000", 2024-09-18T10:16:07.642 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.29701180900000002 2024-09-18T10:16:07.642 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.642 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.642 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:07.642 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.732145+0000", 2024-09-18T10:16:07.642 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.7152999999999998e-05 2024-09-18T10:16:07.642 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.642 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.643 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:07.643 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.732210+0000", 2024-09-18T10:16:07.643 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.4503000000000002e-05 2024-09-18T10:16:07.643 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:07.643 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:07.643 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:07.643 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.643 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.643 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:99 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T10:16:07.643 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:38.717230+0000", 2024-09-18T10:16:07.643 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 88.850666966000006, 2024-09-18T10:16:07.643 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0157905819999999, 2024-09-18T10:16:07.643 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:07.643 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:07.643 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:07.644 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:16:07.644 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:16:07.644 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 99 2024-09-18T10:16:07.644 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.644 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:07.644 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.644 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:07.644 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717230+0000", 2024-09-18T10:16:07.644 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:07.644 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.644 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.644 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:07.644 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717230+0000", 2024-09-18T10:16:07.645 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:07.645 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.645 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.645 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:07.645 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717229+0000", 2024-09-18T10:16:07.645 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:16:07.645 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.645 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.645 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:07.645 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717318+0000", 2024-09-18T10:16:07.645 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.8197999999999998e-05 2024-09-18T10:16:07.645 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.646 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.646 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:07.646 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717342+0000", 2024-09-18T10:16:07.646 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4445000000000001e-05 2024-09-18T10:16:07.646 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.646 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.646 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:07.646 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717360+0000", 2024-09-18T10:16:07.646 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7782999999999999e-05 2024-09-18T10:16:07.646 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.646 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.646 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:07.647 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.430650+0000", 2024-09-18T10:16:07.647 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.71329020200000004 2024-09-18T10:16:07.647 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.647 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.647 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:07.647 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.430805+0000", 2024-09-18T10:16:07.647 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00015474900000000001 2024-09-18T10:16:07.647 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.647 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.647 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:16:07.647 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.431021+0000", 2024-09-18T10:16:07.647 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000215911 2024-09-18T10:16:07.648 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.648 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.648 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:16:07.648 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.435099+0000", 2024-09-18T10:16:07.648 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0040787439999999996 2024-09-18T10:16:07.648 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.648 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.648 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:16:07.648 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.732758+0000", 2024-09-18T10:16:07.648 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.297658637 2024-09-18T10:16:07.648 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.648 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.649 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:07.649 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.732909+0000", 2024-09-18T10:16:07.649 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00015069899999999999 2024-09-18T10:16:07.649 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.649 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.649 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:07.649 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.733021+0000", 2024-09-18T10:16:07.649 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00011207 2024-09-18T10:16:07.649 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:07.649 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:07.649 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:07.649 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.649 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.649 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:100 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T10:16:07.650 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:38.717370+0000", 2024-09-18T10:16:07.650 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 88.850527404000005, 2024-09-18T10:16:07.650 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1158324850000001, 2024-09-18T10:16:07.650 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:07.650 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:07.650 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:07.650 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:16:07.650 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:16:07.650 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 100 2024-09-18T10:16:07.650 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.650 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:07.650 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.650 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:07.650 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717370+0000", 2024-09-18T10:16:07.651 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:07.651 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.651 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.651 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:07.651 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717370+0000", 2024-09-18T10:16:07.651 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:07.651 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.651 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.651 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:07.651 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717369+0000", 2024-09-18T10:16:07.651 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:16:07.651 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.652 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.652 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:07.652 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717378+0000", 2024-09-18T10:16:07.652 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.3559999999999999e-06 2024-09-18T10:16:07.652 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.652 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.652 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:07.652 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717398+0000", 2024-09-18T10:16:07.652 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.9962e-05 2024-09-18T10:16:07.652 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.652 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.652 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:07.653 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717407+0000", 2024-09-18T10:16:07.653 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.2699999999999993e-06 2024-09-18T10:16:07.653 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.653 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.653 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:07.653 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.432184+0000", 2024-09-18T10:16:07.653 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.71477630299999995 2024-09-18T10:16:07.653 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.653 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.653 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:07.653 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.432316+0000", 2024-09-18T10:16:07.653 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000132082 2024-09-18T10:16:07.654 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.654 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.654 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:16:07.654 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.432526+0000", 2024-09-18T10:16:07.654 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000210659 2024-09-18T10:16:07.654 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.654 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.654 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:16:07.654 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.435651+0000", 2024-09-18T10:16:07.654 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.003124169 2024-09-18T10:16:07.654 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.654 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.654 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:16:07.655 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.832931+0000", 2024-09-18T10:16:07.655 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.39728084899999999 2024-09-18T10:16:07.655 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.655 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.655 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:07.655 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.833058+0000", 2024-09-18T10:16:07.655 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000126969 2024-09-18T10:16:07.655 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.655 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.655 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:07.655 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.833202+0000", 2024-09-18T10:16:07.656 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000143922 2024-09-18T10:16:07.656 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:07.656 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:07.656 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:07.656 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.656 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.656 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:141 3.8s0 3:1c02e63c:::smithi18925896-47:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:16:07.656 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:51.119430+0000", 2024-09-18T10:16:07.656 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 76.448467199999996, 2024-09-18T10:16:07.656 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.60306919000000003, 2024-09-18T10:16:07.656 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:07.656 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "started", 2024-09-18T10:16:07.657 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:07.657 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:16:07.657 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:07.657 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 141 2024-09-18T10:16:07.657 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.657 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:07.657 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.657 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:07.657 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.119430+0000", 2024-09-18T10:16:07.657 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:07.657 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.657 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.657 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:07.658 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.119430+0000", 2024-09-18T10:16:07.658 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:07.658 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.658 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.658 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:07.658 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.119430+0000", 2024-09-18T10:16:07.658 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999995 2024-09-18T10:16:07.658 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.658 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.658 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:07.658 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.119432+0000", 2024-09-18T10:16:07.658 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6240000000000002e-06 2024-09-18T10:16:07.658 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.659 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.659 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:07.659 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.119439+0000", 2024-09-18T10:16:07.659 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.8329999999999996e-06 2024-09-18T10:16:07.659 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.659 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.659 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:07.659 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.119444+0000", 2024-09-18T10:16:07.659 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.485e-06 2024-09-18T10:16:07.659 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.659 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.659 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:07.659 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.620626+0000", 2024-09-18T10:16:07.660 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.50118172299999997 2024-09-18T10:16:07.660 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.660 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.660 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:07.660 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.620781+0000", 2024-09-18T10:16:07.660 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00015495 2024-09-18T10:16:07.660 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.660 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.660 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:07.660 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.722499+0000", 2024-09-18T10:16:07.660 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.101718085 2024-09-18T10:16:07.660 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:07.660 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:07.661 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:07.661 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.661 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.661 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:154 3.8s0 3:1d386431:::smithi18925896-2:head [copy-from ver 4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:16:07.661 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:51.211083+0000", 2024-09-18T10:16:07.661 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 76.356813974999994, 2024-09-18T10:16:07.661 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.73193688499999998, 2024-09-18T10:16:07.661 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:07.661 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:07.661 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:07.661 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:16:07.661 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:07.661 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 154 2024-09-18T10:16:07.661 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.662 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:07.662 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.662 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:07.662 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.211083+0000", 2024-09-18T10:16:07.662 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:07.662 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.662 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.662 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:07.662 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.211083+0000", 2024-09-18T10:16:07.662 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:07.662 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.662 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.662 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:07.663 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.211080+0000", 2024-09-18T10:16:07.663 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999971 2024-09-18T10:16:07.663 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.663 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.663 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:07.663 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.211096+0000", 2024-09-18T10:16:07.663 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.5699999999999999e-05 2024-09-18T10:16:07.663 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.663 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.663 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:07.663 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.211134+0000", 2024-09-18T10:16:07.663 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.7694000000000002e-05 2024-09-18T10:16:07.663 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.663 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.664 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:07.664 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.211149+0000", 2024-09-18T10:16:07.664 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4746000000000001e-05 2024-09-18T10:16:07.664 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.664 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.664 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:07.664 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.621341+0000", 2024-09-18T10:16:07.664 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.41019203799999998 2024-09-18T10:16:07.664 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.664 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.664 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:07.664 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.621505+0000", 2024-09-18T10:16:07.664 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00016443299999999999 2024-09-18T10:16:07.665 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.665 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.665 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:07.665 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.840064+0000", 2024-09-18T10:16:07.665 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.21855904700000001 2024-09-18T10:16:07.665 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.665 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.665 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:07.665 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.842430+0000", 2024-09-18T10:16:07.665 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0023664100000000002 2024-09-18T10:16:07.665 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.665 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.665 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:07.665 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.942864+0000", 2024-09-18T10:16:07.666 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10043373 2024-09-18T10:16:07.666 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.666 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.666 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:07.666 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.943020+0000", 2024-09-18T10:16:07.666 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00015598000000000001 2024-09-18T10:16:07.666 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:07.666 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:07.666 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:07.666 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.666 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.666 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:155 3.8s0 3:1d386431:::smithi18925896-2:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:16:07.666 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:51.211164+0000", 2024-09-18T10:16:07.667 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 76.356733605000002, 2024-09-18T10:16:07.667 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.630957349, 2024-09-18T10:16:07.667 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:07.667 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "started", 2024-09-18T10:16:07.667 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:07.667 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:16:07.667 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:07.667 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 155 2024-09-18T10:16:07.667 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.667 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:07.667 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.667 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:07.667 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.211164+0000", 2024-09-18T10:16:07.667 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:07.668 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.668 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.668 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:07.668 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.211164+0000", 2024-09-18T10:16:07.668 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:07.668 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.668 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.668 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:07.668 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.211163+0000", 2024-09-18T10:16:07.668 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:16:07.668 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.668 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.668 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:07.668 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.211167+0000", 2024-09-18T10:16:07.669 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.6299999999999997e-06 2024-09-18T10:16:07.669 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.669 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.669 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:07.669 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.211178+0000", 2024-09-18T10:16:07.669 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0798999999999999e-05 2024-09-18T10:16:07.669 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.669 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.669 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:07.669 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.211186+0000", 2024-09-18T10:16:07.669 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.5669999999999997e-06 2024-09-18T10:16:07.669 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.669 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.669 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:07.670 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.623024+0000", 2024-09-18T10:16:07.670 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.411838433 2024-09-18T10:16:07.670 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.670 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.670 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for blocked object", 2024-09-18T10:16:07.670 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.623104+0000", 2024-09-18T10:16:07.670 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.0136000000000007e-05 2024-09-18T10:16:07.670 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.670 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.670 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:07.670 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.841600+0000", 2024-09-18T10:16:07.670 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.21849595399999999 2024-09-18T10:16:07.670 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.670 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.671 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:07.671 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.841868+0000", 2024-09-18T10:16:07.671 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00026825800000000001 2024-09-18T10:16:07.671 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.671 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.671 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:07.671 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.842121+0000", 2024-09-18T10:16:07.671 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00025266500000000001 2024-09-18T10:16:07.671 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:07.671 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:07.671 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:07.671 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.671 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.671 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:619 3.8s0 3:1162852a:::smithi18925896-44:head [copy-from ver 14] snapc 21=[20,1f,1e,1a,16,12,11,10] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-18T10:16:07.671 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:27.977392+0000", 2024-09-18T10:16:07.672 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 39.590505350999997, 2024-09-18T10:16:07.672 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.71139311599999999, 2024-09-18T10:16:07.672 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:07.672 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:07.672 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:07.672 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:16:07.672 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:07.672 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 619 2024-09-18T10:16:07.672 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.672 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:07.672 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.672 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:07.673 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:27.977392+0000", 2024-09-18T10:16:07.673 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:07.673 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.673 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.673 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:07.673 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:27.977392+0000", 2024-09-18T10:16:07.673 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:07.673 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.673 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.673 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:07.673 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:27.977390+0000", 2024-09-18T10:16:07.673 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:16:07.674 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.674 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.674 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:07.674 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:27.977399+0000", 2024-09-18T10:16:07.674 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.1239999999999994e-06 2024-09-18T10:16:07.674 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.674 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.674 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:07.674 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:27.977422+0000", 2024-09-18T10:16:07.674 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.2734999999999998e-05 2024-09-18T10:16:07.674 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.674 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.674 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:07.675 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.077547+0000", 2024-09-18T10:16:07.675 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100124728 2024-09-18T10:16:07.675 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.675 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.675 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:07.675 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.077713+0000", 2024-09-18T10:16:07.675 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000166515 2024-09-18T10:16:07.675 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.675 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.675 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op must wait for map", 2024-09-18T10:16:07.675 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.077728+0000", 2024-09-18T10:16:07.675 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.5036e-05 2024-09-18T10:16:07.675 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.675 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.676 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:07.676 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.275819+0000", 2024-09-18T10:16:07.676 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.19809117600000001 2024-09-18T10:16:07.676 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.676 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.676 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:07.676 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.276220+0000", 2024-09-18T10:16:07.676 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000400426 2024-09-18T10:16:07.676 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.676 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.676 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:07.676 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.383764+0000", 2024-09-18T10:16:07.676 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.107543814 2024-09-18T10:16:07.676 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.677 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.677 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:07.677 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.486402+0000", 2024-09-18T10:16:07.677 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.102638484 2024-09-18T10:16:07.677 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.677 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.677 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:07.677 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.688521+0000", 2024-09-18T10:16:07.677 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.202118873 2024-09-18T10:16:07.677 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.677 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.677 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:07.677 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.688785+0000", 2024-09-18T10:16:07.678 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000264133 2024-09-18T10:16:07.678 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:07.678 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:07.678 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:07.678 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.678 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.678 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:626 3.10s0 3:0a83d6b3:::smithi18925896-20:head [append 0~638976] snapc 21=[20,1f,1e,1a,16,12,11,10] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-18T10:16:07.678 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:28.708261+0000", 2024-09-18T10:16:07.678 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 38.859635785999998, 2024-09-18T10:16:07.678 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.70096385699999997, 2024-09-18T10:16:07.678 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:07.678 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:07.678 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:07.678 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:16:07.679 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:07.679 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 626 2024-09-18T10:16:07.679 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.679 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:07.679 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.679 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:07.679 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.708261+0000", 2024-09-18T10:16:07.679 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:07.679 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.679 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.679 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:07.679 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.708261+0000", 2024-09-18T10:16:07.679 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:07.679 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.680 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.680 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:07.680 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.708259+0000", 2024-09-18T10:16:07.680 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:16:07.680 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.680 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.680 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:07.680 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.709175+0000", 2024-09-18T10:16:07.680 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00091571199999999995 2024-09-18T10:16:07.680 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.680 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.680 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:07.680 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.709199+0000", 2024-09-18T10:16:07.681 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.3785e-05 2024-09-18T10:16:07.681 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.681 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.681 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:07.681 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.809328+0000", 2024-09-18T10:16:07.681 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100129494 2024-09-18T10:16:07.681 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.681 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.681 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:07.681 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.996826+0000", 2024-09-18T10:16:07.681 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.18749712700000001 2024-09-18T10:16:07.681 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.681 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.681 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:07.681 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.997086+0000", 2024-09-18T10:16:07.682 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00026047499999999998 2024-09-18T10:16:07.682 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.682 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.682 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:07.682 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.998987+0000", 2024-09-18T10:16:07.682 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019014889999999999 2024-09-18T10:16:07.682 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.682 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.682 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:07.682 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.104118+0000", 2024-09-18T10:16:07.682 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10513012100000001 2024-09-18T10:16:07.682 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.682 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.683 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:07.683 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.409085+0000", 2024-09-18T10:16:07.683 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.30496710599999999 2024-09-18T10:16:07.683 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.683 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.683 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:07.683 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.409225+0000", 2024-09-18T10:16:07.683 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00014059899999999999 2024-09-18T10:16:07.683 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:07.683 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:07.683 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:07.683 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.683 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.684 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:629 3.10s0 3:0a83d6b3:::smithi18925896-20:head [setxattr _header (57) in=7b] snapc 21=[20,1f,1e,1a,16,12,11,10] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-18T10:16:07.684 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:28.910973+0000", 2024-09-18T10:16:07.684 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 38.656924668000002, 2024-09-18T10:16:07.684 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.80485965999999998, 2024-09-18T10:16:07.684 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:07.684 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:07.684 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:07.684 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:16:07.684 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:07.684 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 629 2024-09-18T10:16:07.684 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.684 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:07.684 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.684 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:07.684 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.910973+0000", 2024-09-18T10:16:07.685 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:07.685 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.685 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.685 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:07.685 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.910973+0000", 2024-09-18T10:16:07.685 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:07.685 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.685 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.685 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:07.685 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.910971+0000", 2024-09-18T10:16:07.685 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:16:07.685 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.685 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.685 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:07.685 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.910976+0000", 2024-09-18T10:16:07.686 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.0069999999999997e-06 2024-09-18T10:16:07.686 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.686 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.686 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:07.686 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.910984+0000", 2024-09-18T10:16:07.686 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.7109999999999995e-06 2024-09-18T10:16:07.686 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.686 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.686 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:07.686 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.910989+0000", 2024-09-18T10:16:07.686 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1549999999999998e-06 2024-09-18T10:16:07.686 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.686 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.686 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:07.686 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.411082+0000", 2024-09-18T10:16:07.687 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.50009252199999998 2024-09-18T10:16:07.687 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.687 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.687 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:07.687 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.411264+0000", 2024-09-18T10:16:07.687 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00018190799999999999 2024-09-18T10:16:07.687 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.687 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.687 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:07.687 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.412262+0000", 2024-09-18T10:16:07.687 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00099852199999999995 2024-09-18T10:16:07.687 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.687 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.688 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:07.688 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.513736+0000", 2024-09-18T10:16:07.688 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.101474077 2024-09-18T10:16:07.688 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.688 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.688 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:07.688 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.715730+0000", 2024-09-18T10:16:07.688 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20199325600000001 2024-09-18T10:16:07.688 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.688 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.688 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:07.688 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.715832+0000", 2024-09-18T10:16:07.688 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000102633 2024-09-18T10:16:07.689 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:07.689 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:07.689 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:07.689 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.689 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.689 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:630 3.10s0 3:0a83d6b3:::smithi18925896-20:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e72)", 2024-09-18T10:16:07.689 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:28.910995+0000", 2024-09-18T10:16:07.689 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 38.656902637000002, 2024-09-18T10:16:07.689 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2560086020000001, 2024-09-18T10:16:07.689 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:07.689 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "started", 2024-09-18T10:16:07.689 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:07.689 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:16:07.690 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:07.690 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 630 2024-09-18T10:16:07.690 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.690 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:07.690 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.690 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:07.690 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.910995+0000", 2024-09-18T10:16:07.690 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:07.690 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.690 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.690 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:07.690 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.910995+0000", 2024-09-18T10:16:07.690 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:07.690 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.691 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.691 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:07.691 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.910994+0000", 2024-09-18T10:16:07.691 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:16:07.691 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.691 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.691 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:07.691 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.910997+0000", 2024-09-18T10:16:07.691 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2820000000000001e-06 2024-09-18T10:16:07.691 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.691 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.691 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:07.692 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.911008+0000", 2024-09-18T10:16:07.692 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0617e-05 2024-09-18T10:16:07.692 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.692 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.692 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:07.692 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.911013+0000", 2024-09-18T10:16:07.692 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.8160000000000001e-06 2024-09-18T10:16:07.692 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.692 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.692 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:07.692 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.413034+0000", 2024-09-18T10:16:07.692 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.50202176600000004 2024-09-18T10:16:07.692 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.693 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.693 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for rw locks", 2024-09-18T10:16:07.693 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.413303+0000", 2024-09-18T10:16:07.693 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00026880600000000001 2024-09-18T10:16:07.693 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.693 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.693 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:07.693 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.716533+0000", 2024-09-18T10:16:07.693 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.30323016000000003 2024-09-18T10:16:07.693 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.693 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.693 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:07.693 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.716647+0000", 2024-09-18T10:16:07.693 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00011372899999999999 2024-09-18T10:16:07.694 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:07.694 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:07.694 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:07.694 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:30.167003+0000", 2024-09-18T10:16:07.694 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.450356172 2024-09-18T10:16:07.694 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:07.694 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:07.694 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:07.694 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:07.694 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:07.694 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-09-18T10:16:07.696 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-09-18T10:16:07.697 INFO:tasks.ceph.osd.3.smithi189.stderr:2024-09-18T10:16:07.689+0000 7f2440fc7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22396) UID: 0 2024-09-18T10:16:07.796 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-18T10:16:07.791+0000 7f1981873700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22132) UID: 0 2024-09-18T10:16:07.801 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-09-18T10:16:07.801 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-09-18T10:16:07.801 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-09-18T10:16:07.801 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-09-18T10:16:07.811 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-09-18T10:16:07.824 INFO:teuthology.orchestra.run.smithi029.stderr:noscrub is set 2024-09-18T10:16:07.897 INFO:tasks.ceph.osd.6.smithi154.stderr:2024-09-18T10:16:07.894+0000 7f0abc9e1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22089) UID: 0 2024-09-18T10:16:07.905 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-09-18T10:16:07.905 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-09-18T10:16:07.906 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-09-18T10:16:07.906 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-09-18T10:16:07.906 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-09-18T10:16:07.915 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-09-18T10:16:07.998 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-18T10:16:07.995+0000 7f14c6a1a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 22138) UID: 0 2024-09-18T10:16:08.017 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-09-18T10:16:08.017 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-09-18T10:16:08.017 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-09-18T10:16:08.017 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-09-18T10:16:08.017 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.017 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:189 3.3s0 3:ce8164ce:::smithi18925896-7:head [copy-from ver 6] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:16:08.017 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:52.694279+0000", 2024-09-18T10:16:08.017 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 75.318322252000002, 2024-09-18T10:16:08.017 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.51539283599999997, 2024-09-18T10:16:08.018 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:08.018 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:08.018 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:08.018 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:16:08.018 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:08.018 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 189 2024-09-18T10:16:08.018 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.018 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:08.018 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.018 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:08.018 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694279+0000", 2024-09-18T10:16:08.018 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:08.018 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.018 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.019 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:08.019 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694279+0000", 2024-09-18T10:16:08.019 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:08.019 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.019 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.019 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:08.019 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694277+0000", 2024-09-18T10:16:08.019 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999971 2024-09-18T10:16:08.019 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.019 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.019 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:08.019 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694290+0000", 2024-09-18T10:16:08.019 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3862e-05 2024-09-18T10:16:08.019 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.020 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.020 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:08.020 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694311+0000", 2024-09-18T10:16:08.020 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.0273999999999999e-05 2024-09-18T10:16:08.020 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.020 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.020 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:08.020 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694322+0000", 2024-09-18T10:16:08.020 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1722999999999999e-05 2024-09-18T10:16:08.020 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.020 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.020 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:08.020 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.794494+0000", 2024-09-18T10:16:08.021 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100171653 2024-09-18T10:16:08.021 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.021 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.021 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:08.021 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.794705+0000", 2024-09-18T10:16:08.021 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00021126000000000001 2024-09-18T10:16:08.021 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.021 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.021 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:08.021 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.096113+0000", 2024-09-18T10:16:08.021 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.30140739100000002 2024-09-18T10:16:08.021 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.021 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.021 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:08.022 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.098602+0000", 2024-09-18T10:16:08.022 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.002489593 2024-09-18T10:16:08.022 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.022 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.022 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:08.022 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.209538+0000", 2024-09-18T10:16:08.022 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.110935482 2024-09-18T10:16:08.022 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.022 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.022 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:08.022 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.209672+0000", 2024-09-18T10:16:08.022 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00013435399999999999 2024-09-18T10:16:08.022 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:08.023 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:08.023 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:08.023 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.023 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.023 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:194 3.3s0 3:c9ce6440:::smithi18925896-24:head [copy-from ver 6] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:16:08.023 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:52.694388+0000", 2024-09-18T10:16:08.023 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 75.318213388999993, 2024-09-18T10:16:08.023 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.92118689200000003, 2024-09-18T10:16:08.023 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:08.023 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:08.023 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:08.023 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:16:08.023 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:08.024 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 194 2024-09-18T10:16:08.024 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.024 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:08.024 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.024 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:08.024 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694388+0000", 2024-09-18T10:16:08.024 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:08.024 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.024 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.024 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:08.024 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694388+0000", 2024-09-18T10:16:08.024 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:08.025 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.025 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.025 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:08.025 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694388+0000", 2024-09-18T10:16:08.025 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:16:08.025 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.025 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.025 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:08.025 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694393+0000", 2024-09-18T10:16:08.025 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.2299999999999999e-06 2024-09-18T10:16:08.025 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.025 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.025 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:08.026 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694411+0000", 2024-09-18T10:16:08.026 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7802999999999999e-05 2024-09-18T10:16:08.026 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.026 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.026 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:08.026 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694417+0000", 2024-09-18T10:16:08.026 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.4640000000000003e-06 2024-09-18T10:16:08.026 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.026 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.026 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:08.026 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.795289+0000", 2024-09-18T10:16:08.026 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100872288 2024-09-18T10:16:08.026 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.026 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.027 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:08.027 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.795548+0000", 2024-09-18T10:16:08.027 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00025869199999999998 2024-09-18T10:16:08.027 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.027 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.027 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:08.027 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.412215+0000", 2024-09-18T10:16:08.027 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.61666667100000006 2024-09-18T10:16:08.027 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.027 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.028 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:08.028 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.514838+0000", 2024-09-18T10:16:08.028 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10262323399999999 2024-09-18T10:16:08.028 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.028 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.028 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:08.028 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.615386+0000", 2024-09-18T10:16:08.028 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100548106 2024-09-18T10:16:08.028 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.028 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.028 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:08.029 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.615575+0000", 2024-09-18T10:16:08.029 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000189128 2024-09-18T10:16:08.029 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:08.029 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:08.029 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:08.029 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.029 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.029 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:195 3.3s0 3:c9ce6440:::smithi18925896-24:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:16:08.029 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:52.694424+0000", 2024-09-18T10:16:08.029 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 75.318177653000006, 2024-09-18T10:16:08.029 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.82009685499999996, 2024-09-18T10:16:08.029 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:08.030 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "started", 2024-09-18T10:16:08.030 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:08.030 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:16:08.030 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:08.030 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 195 2024-09-18T10:16:08.030 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.030 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:08.030 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.030 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:08.030 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694424+0000", 2024-09-18T10:16:08.030 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:08.031 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.031 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.031 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:08.031 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694424+0000", 2024-09-18T10:16:08.031 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:08.031 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.031 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.031 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:08.031 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694423+0000", 2024-09-18T10:16:08.031 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999995 2024-09-18T10:16:08.031 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.031 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.031 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:08.031 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694427+0000", 2024-09-18T10:16:08.032 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2650000000000001e-06 2024-09-18T10:16:08.032 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.032 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.032 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:08.032 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694434+0000", 2024-09-18T10:16:08.032 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.5410000000000003e-06 2024-09-18T10:16:08.032 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.032 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.032 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:08.032 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694441+0000", 2024-09-18T10:16:08.032 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.187e-06 2024-09-18T10:16:08.032 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.032 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.033 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:08.033 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.795870+0000", 2024-09-18T10:16:08.033 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.101428617 2024-09-18T10:16:08.033 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.033 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.033 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for blocked object", 2024-09-18T10:16:08.033 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.796040+0000", 2024-09-18T10:16:08.034 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00017008 2024-09-18T10:16:08.034 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.034 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.034 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:08.034 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.514130+0000", 2024-09-18T10:16:08.034 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.71809022199999994 2024-09-18T10:16:08.035 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.035 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.035 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:08.035 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.514348+0000", 2024-09-18T10:16:08.035 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000217488 2024-09-18T10:16:08.035 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.036 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.036 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:08.036 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.514521+0000", 2024-09-18T10:16:08.036 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00017300900000000001 2024-09-18T10:16:08.036 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:08.036 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:08.036 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:08.036 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.036 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.037 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.bs2 41/32 ECSubWrite(tid=58, reqid=osd.4.0:58, at_version=41'14, trim_to=0'0, roll_forward_to=39'12))", 2024-09-18T10:16:08.037 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:56.829698+0000", 2024-09-18T10:16:08.037 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 71.182903558999996, 2024-09-18T10:16:08.037 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.71711602600000002, 2024-09-18T10:16:08.037 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:08.037 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:16:08.037 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:08.037 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.037 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:08.038 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.829698+0000", 2024-09-18T10:16:08.038 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:08.038 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.038 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.038 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:08.038 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.829698+0000", 2024-09-18T10:16:08.038 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:08.038 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.038 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.038 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:08.038 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.829697+0000", 2024-09-18T10:16:08.038 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:16:08.038 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.039 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.039 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:08.039 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.829733+0000", 2024-09-18T10:16:08.039 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.6217999999999998e-05 2024-09-18T10:16:08.039 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.039 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.039 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:08.039 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.829757+0000", 2024-09-18T10:16:08.039 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.3915000000000001e-05 2024-09-18T10:16:08.039 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.039 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.039 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:08.039 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.829769+0000", 2024-09-18T10:16:08.040 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1754e-05 2024-09-18T10:16:08.040 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.040 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.040 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:08.040 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.342965+0000", 2024-09-18T10:16:08.040 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.51319593799999996 2024-09-18T10:16:08.040 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.040 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.040 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:08.040 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.343020+0000", 2024-09-18T10:16:08.040 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5367e-05 2024-09-18T10:16:08.040 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.040 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.041 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:08.041 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.546772+0000", 2024-09-18T10:16:08.041 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.203751933 2024-09-18T10:16:08.041 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.041 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.041 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:08.041 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.546814+0000", 2024-09-18T10:16:08.041 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.2196000000000001e-05 2024-09-18T10:16:08.041 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:08.041 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:08.041 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:08.041 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.041 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.041 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDPGPushReply(3.17s0 67/64 [PushReplyOp(3:ee962131:::smithi18925896-29:10)])", 2024-09-18T10:16:08.042 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:23.671965+0000", 2024-09-18T10:16:08.042 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 44.340637108999999, 2024-09-18T10:16:08.042 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.60673808500000004, 2024-09-18T10:16:08.042 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:08.042 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:16:08.042 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:08.042 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.042 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:08.042 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.671965+0000", 2024-09-18T10:16:08.042 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:08.042 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.042 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.042 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:08.043 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.671965+0000", 2024-09-18T10:16:08.043 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:08.043 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.043 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.043 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:08.043 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.671963+0000", 2024-09-18T10:16:08.043 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:16:08.043 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.043 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.043 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:08.043 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.671977+0000", 2024-09-18T10:16:08.043 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4093e-05 2024-09-18T10:16:08.043 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.044 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.044 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:08.044 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.672009+0000", 2024-09-18T10:16:08.044 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1939e-05 2024-09-18T10:16:08.044 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.044 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.044 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:08.044 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.772102+0000", 2024-09-18T10:16:08.044 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10009319 2024-09-18T10:16:08.044 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.044 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.044 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:08.044 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.278591+0000", 2024-09-18T10:16:08.045 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.50648915999999999 2024-09-18T10:16:08.045 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.045 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.045 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:08.045 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.278703+0000", 2024-09-18T10:16:08.045 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000111433 2024-09-18T10:16:08.045 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:08.045 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:08.045 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:08.045 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.045 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.045 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDPGPushReply(3.17s0 67/64 [PushReplyOp(3:ee962131:::smithi18925896-29:10)])", 2024-09-18T10:16:08.045 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:23.772436+0000", 2024-09-18T10:16:08.046 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 44.240165873000002, 2024-09-18T10:16:08.046 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.50655292500000004, 2024-09-18T10:16:08.046 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:08.046 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:16:08.046 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:08.046 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.046 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:08.046 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.772436+0000", 2024-09-18T10:16:08.046 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:08.046 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.046 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.046 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:08.046 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.772436+0000", 2024-09-18T10:16:08.047 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:08.047 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.047 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.047 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:08.047 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.772434+0000", 2024-09-18T10:16:08.047 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:16:08.047 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.047 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.047 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:08.047 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.772448+0000", 2024-09-18T10:16:08.047 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3682e-05 2024-09-18T10:16:08.047 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.047 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.047 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:08.048 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.772477+0000", 2024-09-18T10:16:08.048 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9255999999999999e-05 2024-09-18T10:16:08.048 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.048 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.048 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:08.048 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.772490+0000", 2024-09-18T10:16:08.048 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2962999999999999e-05 2024-09-18T10:16:08.048 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.048 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.048 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:08.048 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.278888+0000", 2024-09-18T10:16:08.048 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.50639794699999996 2024-09-18T10:16:08.048 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.048 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.049 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:08.049 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.278989+0000", 2024-09-18T10:16:08.049 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000100777 2024-09-18T10:16:08.049 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:08.049 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:08.049 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:08.049 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.049 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.049 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDPGPushReply(3.17s0 67/64 [PushReplyOp(3:ee962131:::smithi18925896-29:10)])", 2024-09-18T10:16:08.049 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:23.772911+0000", 2024-09-18T10:16:08.049 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 44.239690594000002, 2024-09-18T10:16:08.049 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.50636473400000004, 2024-09-18T10:16:08.049 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:08.049 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:16:08.050 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:08.050 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.050 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:08.050 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.772911+0000", 2024-09-18T10:16:08.050 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:08.050 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.050 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.050 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:08.050 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.772911+0000", 2024-09-18T10:16:08.050 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:08.050 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.050 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.050 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:08.051 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.772909+0000", 2024-09-18T10:16:08.051 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:16:08.051 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.051 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.051 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:08.051 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.772923+0000", 2024-09-18T10:16:08.051 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3957e-05 2024-09-18T10:16:08.051 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.051 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.051 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:08.051 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.772956+0000", 2024-09-18T10:16:08.051 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2925000000000001e-05 2024-09-18T10:16:08.051 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.052 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.052 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:08.052 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.772969+0000", 2024-09-18T10:16:08.052 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2516e-05 2024-09-18T10:16:08.052 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.052 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.052 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:08.052 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.279192+0000", 2024-09-18T10:16:08.052 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.50622349799999999 2024-09-18T10:16:08.052 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.052 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.052 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:08.052 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.279276+0000", 2024-09-18T10:16:08.052 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.3498000000000006e-05 2024-09-18T10:16:08.053 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:08.053 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:08.053 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:08.053 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.053 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.053 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDPGPushReply(3.17s0 67/64 [PushReplyOp(3:ead53b0f:::smithi18925896-25:13)])", 2024-09-18T10:16:08.053 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:23.875511+0000", 2024-09-18T10:16:08.053 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 44.137090968999999, 2024-09-18T10:16:08.053 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.50638906400000006, 2024-09-18T10:16:08.053 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:08.053 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:16:08.053 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:08.053 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.054 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:08.054 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.875511+0000", 2024-09-18T10:16:08.054 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:08.054 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.054 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.054 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:08.054 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.875511+0000", 2024-09-18T10:16:08.054 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:08.054 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.054 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.054 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:08.054 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.875510+0000", 2024-09-18T10:16:08.054 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999995 2024-09-18T10:16:08.054 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.055 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.055 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:08.055 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.875516+0000", 2024-09-18T10:16:08.055 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.8930000000000002e-06 2024-09-18T10:16:08.055 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.055 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.055 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:08.055 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.875526+0000", 2024-09-18T10:16:08.055 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0183000000000001e-05 2024-09-18T10:16:08.055 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.055 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.055 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:08.055 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.975617+0000", 2024-09-18T10:16:08.055 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10009069299999999 2024-09-18T10:16:08.056 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.056 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.056 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:08.056 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.381839+0000", 2024-09-18T10:16:08.056 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.40622223600000001 2024-09-18T10:16:08.056 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.056 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.056 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:08.056 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.381900+0000", 2024-09-18T10:16:08.056 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.0572000000000002e-05 2024-09-18T10:16:08.056 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:08.056 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:08.056 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:08.057 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.057 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.057 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:635 3.17s0 3:ead53b0f:::smithi18925896-25:head [append 0~655360] snapc 21=[20,1f,1e,1a,16,11,10] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-18T10:16:08.057 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:29.755594+0000", 2024-09-18T10:16:08.057 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 38.257008049, 2024-09-18T10:16:08.057 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.62129057300000001, 2024-09-18T10:16:08.057 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:08.057 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:08.057 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:08.057 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:16:08.057 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:08.057 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 635 2024-09-18T10:16:08.057 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.058 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:08.058 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.058 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:08.058 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.755594+0000", 2024-09-18T10:16:08.058 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:08.058 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.058 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.058 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:08.058 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.755594+0000", 2024-09-18T10:16:08.058 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:08.058 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.058 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.058 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:08.059 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.755590+0000", 2024-09-18T10:16:08.059 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999967 2024-09-18T10:16:08.059 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.059 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.059 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:08.059 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.756465+0000", 2024-09-18T10:16:08.059 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00087483000000000005 2024-09-18T10:16:08.059 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.059 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.059 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:08.059 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.756502+0000", 2024-09-18T10:16:08.059 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.7067999999999998e-05 2024-09-18T10:16:08.059 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.060 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.060 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:08.060 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.756516+0000", 2024-09-18T10:16:08.060 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4071e-05 2024-09-18T10:16:08.060 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.060 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.060 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:08.060 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.760022+0000", 2024-09-18T10:16:08.060 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0035059570000000001 2024-09-18T10:16:08.060 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.060 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.060 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:08.060 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.760234+0000", 2024-09-18T10:16:08.060 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00021161199999999999 2024-09-18T10:16:08.061 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.061 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.061 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:08.061 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.762520+0000", 2024-09-18T10:16:08.061 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0022855760000000001 2024-09-18T10:16:08.061 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.061 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.061 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:08.061 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.770533+0000", 2024-09-18T10:16:08.061 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0080138269999999994 2024-09-18T10:16:08.061 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.061 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.061 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:08.062 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:30.376766+0000", 2024-09-18T10:16:08.062 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.60623257500000005 2024-09-18T10:16:08.062 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.062 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.062 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:08.062 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:30.376884+0000", 2024-09-18T10:16:08.062 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00011819699999999999 2024-09-18T10:16:08.062 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:08.062 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:08.062 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:08.062 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.062 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.062 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:636 3.17s0 3:ead53b0f:::smithi18925896-25:head [append 0~606208] snapc 21=[20,1f,1e,1a,16,11,10] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-18T10:16:08.063 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:29.758851+0000", 2024-09-18T10:16:08.063 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 38.253750396000001, 2024-09-18T10:16:08.063 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.61852410899999999, 2024-09-18T10:16:08.063 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:08.063 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:08.063 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:08.063 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:16:08.063 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:08.063 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 636 2024-09-18T10:16:08.063 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.063 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:08.063 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.063 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:08.063 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.758851+0000", 2024-09-18T10:16:08.064 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:08.064 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.064 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.064 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:08.064 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.758851+0000", 2024-09-18T10:16:08.064 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:08.064 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.064 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.064 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:08.064 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.758849+0000", 2024-09-18T10:16:08.064 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999976 2024-09-18T10:16:08.064 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.064 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.065 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:08.065 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.759643+0000", 2024-09-18T10:16:08.065 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00079385800000000004 2024-09-18T10:16:08.065 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.065 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.065 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:08.065 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.759677+0000", 2024-09-18T10:16:08.065 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.4504e-05 2024-09-18T10:16:08.065 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.065 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.065 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:08.065 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.759691+0000", 2024-09-18T10:16:08.065 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3586e-05 2024-09-18T10:16:08.066 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.066 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.066 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:08.066 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.765309+0000", 2024-09-18T10:16:08.066 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0056179460000000004 2024-09-18T10:16:08.066 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.066 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.066 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:08.066 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.765656+0000", 2024-09-18T10:16:08.066 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000347783 2024-09-18T10:16:08.066 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.066 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.066 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:08.066 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.766735+0000", 2024-09-18T10:16:08.067 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0010785460000000001 2024-09-18T10:16:08.067 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.067 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.067 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:08.067 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.973532+0000", 2024-09-18T10:16:08.067 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20679750599999999 2024-09-18T10:16:08.067 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.067 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.067 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:08.067 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:30.377254+0000", 2024-09-18T10:16:08.067 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.40372181400000001 2024-09-18T10:16:08.067 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.067 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.067 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:08.068 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:30.377375+0000", 2024-09-18T10:16:08.068 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000121037 2024-09-18T10:16:08.068 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:08.068 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:08.068 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:08.068 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.068 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.068 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:637 3.17s0 3:ead53b0f:::smithi18925896-25:head [append 0~475136] snapc 21=[20,1f,1e,1a,16,11,10] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-18T10:16:08.068 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:29.761784+0000", 2024-09-18T10:16:08.068 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 38.250817542999997, 2024-09-18T10:16:08.068 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.61605535199999995, 2024-09-18T10:16:08.068 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:08.068 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:08.069 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:08.069 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:16:08.069 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:08.069 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 637 2024-09-18T10:16:08.069 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.069 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:08.069 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.069 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:08.069 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.761784+0000", 2024-09-18T10:16:08.069 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:08.069 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.069 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.069 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:08.069 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.761784+0000", 2024-09-18T10:16:08.070 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:08.070 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.070 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.070 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:08.070 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.761776+0000", 2024-09-18T10:16:08.070 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999919 2024-09-18T10:16:08.070 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.070 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.070 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:08.070 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.762424+0000", 2024-09-18T10:16:08.070 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00064803899999999999 2024-09-18T10:16:08.070 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.070 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.070 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:08.071 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.762455+0000", 2024-09-18T10:16:08.071 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1316000000000003e-05 2024-09-18T10:16:08.071 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.071 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.071 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:08.071 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.762471+0000", 2024-09-18T10:16:08.071 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.6174000000000001e-05 2024-09-18T10:16:08.071 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.071 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.071 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:08.071 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.767732+0000", 2024-09-18T10:16:08.071 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0052603190000000003 2024-09-18T10:16:08.071 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.071 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.071 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:08.072 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.767873+0000", 2024-09-18T10:16:08.072 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000140949 2024-09-18T10:16:08.072 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.072 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.072 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:08.072 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.768566+0000", 2024-09-18T10:16:08.072 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00069325799999999998 2024-09-18T10:16:08.072 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.072 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.072 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:08.072 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.973546+0000", 2024-09-18T10:16:08.072 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.204980144 2024-09-18T10:16:08.072 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.073 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.073 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:08.073 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:30.377732+0000", 2024-09-18T10:16:08.073 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.404185558 2024-09-18T10:16:08.073 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.073 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.073 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:08.073 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:30.377839+0000", 2024-09-18T10:16:08.073 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000107909 2024-09-18T10:16:08.073 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:08.073 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:08.073 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:08.073 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.073 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.074 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:638 3.17s0 3:ead53b0f:::smithi18925896-25:head [append 0~81920] snapc 21=[20,1f,1e,1a,16,11,10] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-18T10:16:08.074 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:29.762493+0000", 2024-09-18T10:16:08.074 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 38.250108523000002, 2024-09-18T10:16:08.074 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.61563624900000002, 2024-09-18T10:16:08.074 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:08.074 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:08.074 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:08.074 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:16:08.074 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:08.074 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 638 2024-09-18T10:16:08.074 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.074 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:08.074 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.074 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:08.075 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.762493+0000", 2024-09-18T10:16:08.075 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:08.075 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.075 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.075 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:08.075 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.762493+0000", 2024-09-18T10:16:08.075 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:08.075 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.075 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.075 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:08.075 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.762492+0000", 2024-09-18T10:16:08.075 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:16:08.075 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.075 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.076 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:08.076 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.762566+0000", 2024-09-18T10:16:08.076 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.3977e-05 2024-09-18T10:16:08.076 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.076 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.076 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:08.076 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.762585+0000", 2024-09-18T10:16:08.076 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.9074000000000001e-05 2024-09-18T10:16:08.076 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.077 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.077 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:08.077 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.762595+0000", 2024-09-18T10:16:08.077 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0913999999999999e-05 2024-09-18T10:16:08.077 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.077 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.077 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:08.077 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.769730+0000", 2024-09-18T10:16:08.077 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0071346680000000003 2024-09-18T10:16:08.077 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.077 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.077 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:08.077 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.769816+0000", 2024-09-18T10:16:08.077 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.5422999999999998e-05 2024-09-18T10:16:08.078 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.078 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.078 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:08.078 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.770214+0000", 2024-09-18T10:16:08.078 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00039818999999999998 2024-09-18T10:16:08.078 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.078 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.078 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:08.078 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.973818+0000", 2024-09-18T10:16:08.078 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20360445499999999 2024-09-18T10:16:08.078 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.078 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.078 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:08.078 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:30.378035+0000", 2024-09-18T10:16:08.079 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.40421707600000001 2024-09-18T10:16:08.079 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.079 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.079 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:08.079 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:30.378129+0000", 2024-09-18T10:16:08.079 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.4056999999999997e-05 2024-09-18T10:16:08.079 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:08.079 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:08.079 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:08.079 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.079 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.079 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:639 3.17s0 3:ead53b0f:::smithi18925896-25:head [setxattr _header (57) in=7b] snapc 21=[20,1f,1e,1a,16,11,10] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-18T10:16:08.079 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:29.762609+0000", 2024-09-18T10:16:08.079 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 38.249992368999997, 2024-09-18T10:16:08.080 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.61577652400000005, 2024-09-18T10:16:08.080 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:08.080 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:08.080 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:08.080 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:16:08.080 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:08.080 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 639 2024-09-18T10:16:08.080 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.080 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:08.080 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.080 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:08.080 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.762609+0000", 2024-09-18T10:16:08.080 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:08.080 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.081 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.081 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:08.081 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.762609+0000", 2024-09-18T10:16:08.081 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:08.081 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.081 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.081 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:08.081 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.762608+0000", 2024-09-18T10:16:08.081 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:16:08.081 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.081 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.081 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:08.081 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.762615+0000", 2024-09-18T10:16:08.081 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.2339999999999997e-06 2024-09-18T10:16:08.082 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.082 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.082 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:08.082 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.762628+0000", 2024-09-18T10:16:08.082 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2848e-05 2024-09-18T10:16:08.082 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.082 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.082 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:08.082 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.762636+0000", 2024-09-18T10:16:08.082 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.0970000000000006e-06 2024-09-18T10:16:08.082 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.082 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.082 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:08.083 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.870703+0000", 2024-09-18T10:16:08.083 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.108067121 2024-09-18T10:16:08.083 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.083 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.083 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:08.083 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.870949+0000", 2024-09-18T10:16:08.083 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00024526700000000002 2024-09-18T10:16:08.083 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.083 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.083 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:08.083 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.871876+0000", 2024-09-18T10:16:08.083 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00092680400000000004 2024-09-18T10:16:08.083 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.083 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.084 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:08.084 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.973821+0000", 2024-09-18T10:16:08.084 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10194552599999999 2024-09-18T10:16:08.084 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.084 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.084 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:08.084 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:30.378300+0000", 2024-09-18T10:16:08.084 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.40447920900000001 2024-09-18T10:16:08.084 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.084 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.084 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:08.084 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:30.378386+0000", 2024-09-18T10:16:08.084 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.5531999999999994e-05 2024-09-18T10:16:08.084 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:08.085 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:08.085 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:08.085 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.085 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.085 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.7s5 74/40 ECSubWrite(tid=168, reqid=osd.4.0:166, at_version=0'0, trim_to=0'0, roll_forward_to=74'47))", 2024-09-18T10:16:08.085 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:30.378493+0000", 2024-09-18T10:16:08.085 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 37.634108724999997, 2024-09-18T10:16:08.085 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.605881475, 2024-09-18T10:16:08.085 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:08.085 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:16:08.085 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:08.085 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.085 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:08.086 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:30.378493+0000", 2024-09-18T10:16:08.086 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:08.086 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.086 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.086 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:08.086 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:30.378493+0000", 2024-09-18T10:16:08.086 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:08.086 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.086 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.086 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:08.086 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:30.378492+0000", 2024-09-18T10:16:08.086 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:16:08.087 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.087 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.087 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:08.087 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:30.378513+0000", 2024-09-18T10:16:08.087 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.1427e-05 2024-09-18T10:16:08.087 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.087 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.087 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:08.087 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:30.378550+0000", 2024-09-18T10:16:08.087 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.7351999999999997e-05 2024-09-18T10:16:08.087 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.087 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.087 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:08.087 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:30.478641+0000", 2024-09-18T10:16:08.087 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100090965 2024-09-18T10:16:08.088 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.088 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.088 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:08.088 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:30.882297+0000", 2024-09-18T10:16:08.088 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.40365594799999999 2024-09-18T10:16:08.088 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.088 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.088 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:08.088 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:30.882320+0000", 2024-09-18T10:16:08.088 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.3127999999999999e-05 2024-09-18T10:16:08.088 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.088 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.088 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:08.089 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:30.984327+0000", 2024-09-18T10:16:08.089 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10200699100000001 2024-09-18T10:16:08.089 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.089 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.089 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:08.089 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:30.984374+0000", 2024-09-18T10:16:08.089 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.6953000000000003e-05 2024-09-18T10:16:08.089 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:08.089 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:08.089 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:08.089 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.089 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.089 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:758 3.17s0 3:ee962131:::smithi18925896-29:head [append 0~524288] snapc 2f=[2f,29,28,20,1f,1e,1a,11] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-18T10:16:08.089 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:40.612706+0000", 2024-09-18T10:16:08.090 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 27.399895973, 2024-09-18T10:16:08.090 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.60348796500000002, 2024-09-18T10:16:08.090 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:08.090 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:08.090 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:08.090 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:16:08.090 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:08.090 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 758 2024-09-18T10:16:08.090 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.090 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:08.090 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.090 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:08.090 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.612706+0000", 2024-09-18T10:16:08.090 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:08.091 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.091 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.091 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:08.091 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.612706+0000", 2024-09-18T10:16:08.091 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:08.091 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.091 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.091 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:08.091 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.612702+0000", 2024-09-18T10:16:08.091 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999967 2024-09-18T10:16:08.091 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.091 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.091 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:08.091 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.613684+0000", 2024-09-18T10:16:08.092 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00098185200000000007 2024-09-18T10:16:08.092 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.092 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.092 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:08.092 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.613722+0000", 2024-09-18T10:16:08.092 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.7490000000000002e-05 2024-09-18T10:16:08.092 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.092 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.092 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:08.092 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.613741+0000", 2024-09-18T10:16:08.092 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.9707999999999998e-05 2024-09-18T10:16:08.092 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.092 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.092 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:08.093 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.613931+0000", 2024-09-18T10:16:08.093 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00018992799999999999 2024-09-18T10:16:08.093 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.093 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.093 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op must wait for map", 2024-09-18T10:16:08.093 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.613950+0000", 2024-09-18T10:16:08.093 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8909000000000001e-05 2024-09-18T10:16:08.093 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.093 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.093 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:08.093 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.726459+0000", 2024-09-18T10:16:08.093 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.112508555 2024-09-18T10:16:08.093 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.093 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.094 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:08.094 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.726722+0000", 2024-09-18T10:16:08.094 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00026331999999999999 2024-09-18T10:16:08.094 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.094 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.094 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:08.094 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.728892+0000", 2024-09-18T10:16:08.094 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0021697040000000002 2024-09-18T10:16:08.094 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.094 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.094 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:08.094 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.741279+0000", 2024-09-18T10:16:08.094 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.012386856 2024-09-18T10:16:08.094 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.095 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.095 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:08.095 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:41.215936+0000", 2024-09-18T10:16:08.095 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.474657153 2024-09-18T10:16:08.095 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.095 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.095 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:08.095 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:41.216194+0000", 2024-09-18T10:16:08.095 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00025781799999999999 2024-09-18T10:16:08.095 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:08.095 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:08.095 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:08.095 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.095 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.095 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:759 3.17s0 3:ee962131:::smithi18925896-29:head [append 0~573440] snapc 2f=[2f,29,28,20,1f,1e,1a,11] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-18T10:16:08.096 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:40.620305+0000", 2024-09-18T10:16:08.096 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 27.392296311999999, 2024-09-18T10:16:08.096 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.59650130599999995, 2024-09-18T10:16:08.096 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:08.096 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:08.096 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:08.096 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:16:08.096 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:08.096 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 759 2024-09-18T10:16:08.096 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.096 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:08.096 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.096 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:08.097 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.620305+0000", 2024-09-18T10:16:08.097 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:08.097 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.097 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.097 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:08.097 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.620305+0000", 2024-09-18T10:16:08.097 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:08.097 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.097 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.097 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:08.097 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.620302+0000", 2024-09-18T10:16:08.097 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999971 2024-09-18T10:16:08.097 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.098 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.098 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:08.098 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.620967+0000", 2024-09-18T10:16:08.098 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00066427800000000003 2024-09-18T10:16:08.098 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.098 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.098 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:08.098 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.620994+0000", 2024-09-18T10:16:08.098 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.76e-05 2024-09-18T10:16:08.098 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.098 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.098 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:08.098 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.621004+0000", 2024-09-18T10:16:08.098 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0098000000000001e-05 2024-09-18T10:16:08.099 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.099 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.099 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:08.099 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.621152+0000", 2024-09-18T10:16:08.099 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00014735799999999999 2024-09-18T10:16:08.099 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.099 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.099 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:16:08.099 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.621173+0000", 2024-09-18T10:16:08.099 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.1223e-05 2024-09-18T10:16:08.099 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.099 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.099 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:08.100 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.730351+0000", 2024-09-18T10:16:08.100 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.109177487 2024-09-18T10:16:08.100 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.100 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.100 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:08.100 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.730553+0000", 2024-09-18T10:16:08.100 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00020222600000000001 2024-09-18T10:16:08.100 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.100 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.100 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:08.100 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.732189+0000", 2024-09-18T10:16:08.100 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001636153 2024-09-18T10:16:08.100 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.100 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.101 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:08.101 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.741283+0000", 2024-09-18T10:16:08.101 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0090935889999999991 2024-09-18T10:16:08.101 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.101 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.101 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:08.101 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:41.216570+0000", 2024-09-18T10:16:08.101 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.475287444 2024-09-18T10:16:08.101 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.101 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.101 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:08.101 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:41.216807+0000", 2024-09-18T10:16:08.101 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00023667999999999999 2024-09-18T10:16:08.101 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:08.102 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:08.102 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:08.102 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.102 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.102 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:760 3.17s0 3:ee962131:::smithi18925896-29:head [append 0~688128] snapc 2f=[2f,29,28,20,1f,1e,1a,11] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-18T10:16:08.102 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:40.624963+0000", 2024-09-18T10:16:08.102 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 27.387638806999998, 2024-09-18T10:16:08.102 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.592485437, 2024-09-18T10:16:08.102 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:08.102 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:08.102 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:08.102 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:16:08.102 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:08.102 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 760 2024-09-18T10:16:08.103 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.103 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:08.103 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.103 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:08.103 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.624963+0000", 2024-09-18T10:16:08.103 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:08.103 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.103 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.103 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:08.103 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.624963+0000", 2024-09-18T10:16:08.103 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:08.103 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.103 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.103 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:08.103 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.624960+0000", 2024-09-18T10:16:08.104 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999971 2024-09-18T10:16:08.104 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.104 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.104 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:08.104 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.625766+0000", 2024-09-18T10:16:08.104 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00080539100000000003 2024-09-18T10:16:08.104 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.104 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.104 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:08.104 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.625792+0000", 2024-09-18T10:16:08.104 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5989000000000001e-05 2024-09-18T10:16:08.104 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.104 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.104 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:08.105 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.625801+0000", 2024-09-18T10:16:08.105 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.4080000000000004e-06 2024-09-18T10:16:08.105 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.105 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.105 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:08.105 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.627383+0000", 2024-09-18T10:16:08.105 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0015816440000000001 2024-09-18T10:16:08.105 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.105 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.105 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:16:08.105 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.627397+0000", 2024-09-18T10:16:08.105 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3973e-05 2024-09-18T10:16:08.105 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.105 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.106 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:08.106 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.733560+0000", 2024-09-18T10:16:08.106 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.106163805 2024-09-18T10:16:08.106 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.106 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.106 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:08.106 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.733770+0000", 2024-09-18T10:16:08.106 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00020980900000000001 2024-09-18T10:16:08.106 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.106 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.106 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:08.106 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.735644+0000", 2024-09-18T10:16:08.106 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001874164 2024-09-18T10:16:08.107 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.107 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.107 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:08.107 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.741750+0000", 2024-09-18T10:16:08.107 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0061058379999999997 2024-09-18T10:16:08.107 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.107 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.107 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:08.107 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:41.217140+0000", 2024-09-18T10:16:08.107 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.47538995699999997 2024-09-18T10:16:08.107 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.107 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.107 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:08.107 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:41.217448+0000", 2024-09-18T10:16:08.107 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000308089 2024-09-18T10:16:08.107 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:08.108 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:08.108 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:08.108 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.108 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.108 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:761 3.17s0 3:ee962131:::smithi18925896-29:head [append 0~294912] snapc 2f=[2f,29,28,20,1f,1e,1a,11] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-18T10:16:08.108 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:40.626751+0000", 2024-09-18T10:16:08.108 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 27.38585054, 2024-09-18T10:16:08.108 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.59137154800000002, 2024-09-18T10:16:08.108 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:08.108 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:08.108 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:08.108 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:16:08.108 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:08.109 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 761 2024-09-18T10:16:08.109 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.109 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:08.109 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.109 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:08.109 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.626751+0000", 2024-09-18T10:16:08.109 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:08.109 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.109 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.109 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:08.109 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.626751+0000", 2024-09-18T10:16:08.109 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:08.109 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.110 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.110 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:08.110 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.626749+0000", 2024-09-18T10:16:08.110 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:16:08.110 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.110 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.110 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:08.110 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.627187+0000", 2024-09-18T10:16:08.110 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000437682 2024-09-18T10:16:08.110 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.110 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.110 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:08.110 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.627209+0000", 2024-09-18T10:16:08.110 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.2195999999999999e-05 2024-09-18T10:16:08.110 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.111 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.111 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:08.111 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.627218+0000", 2024-09-18T10:16:08.111 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.4829999999999999e-06 2024-09-18T10:16:08.111 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.111 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.111 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:08.111 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.627438+0000", 2024-09-18T10:16:08.111 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00022063399999999999 2024-09-18T10:16:08.111 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.111 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.111 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:16:08.111 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.627447+0000", 2024-09-18T10:16:08.111 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.687e-06 2024-09-18T10:16:08.112 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.112 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.112 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:08.112 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.737040+0000", 2024-09-18T10:16:08.112 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10959300800000001 2024-09-18T10:16:08.112 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.112 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.112 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:08.112 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.737281+0000", 2024-09-18T10:16:08.112 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00024113100000000001 2024-09-18T10:16:08.112 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.112 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.112 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:08.112 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.738816+0000", 2024-09-18T10:16:08.112 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0015353649999999999 2024-09-18T10:16:08.113 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.113 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.113 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:08.113 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.741752+0000", 2024-09-18T10:16:08.113 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0029357239999999998 2024-09-18T10:16:08.113 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.113 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.113 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:08.113 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:41.217885+0000", 2024-09-18T10:16:08.113 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.476132522 2024-09-18T10:16:08.113 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.113 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.113 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:08.113 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:41.218123+0000", 2024-09-18T10:16:08.114 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000237968 2024-09-18T10:16:08.114 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:08.114 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:08.114 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:08.114 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.114 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.114 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:762 3.17s0 3:ee962131:::smithi18925896-29:head [setxattr _header (57) in=7b] snapc 2f=[2f,29,28,20,1f,1e,1a,11] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-18T10:16:08.114 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:40.627233+0000", 2024-09-18T10:16:08.114 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 27.385368708000001, 2024-09-18T10:16:08.114 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.68868450599999997, 2024-09-18T10:16:08.114 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:08.114 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:08.114 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:08.114 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:16:08.114 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:08.115 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 762 2024-09-18T10:16:08.115 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.115 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:08.115 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.115 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:08.115 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.627233+0000", 2024-09-18T10:16:08.115 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:08.115 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.115 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.115 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:08.115 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.627233+0000", 2024-09-18T10:16:08.115 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:08.115 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.116 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.116 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:08.116 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.627232+0000", 2024-09-18T10:16:08.116 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:16:08.116 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.116 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.116 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:08.116 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.627238+0000", 2024-09-18T10:16:08.116 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.7570000000000001e-06 2024-09-18T10:16:08.116 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.116 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.116 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:08.116 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.627246+0000", 2024-09-18T10:16:08.116 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.0199999999999994e-06 2024-09-18T10:16:08.117 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.117 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.117 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:08.117 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.627251+0000", 2024-09-18T10:16:08.117 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9599999999999999e-06 2024-09-18T10:16:08.117 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.117 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.117 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:08.117 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.627481+0000", 2024-09-18T10:16:08.117 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00023046199999999999 2024-09-18T10:16:08.117 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.117 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.117 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:16:08.118 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.627493+0000", 2024-09-18T10:16:08.118 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2173e-05 2024-09-18T10:16:08.118 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.118 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.118 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:08.118 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.740060+0000", 2024-09-18T10:16:08.118 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.112566686 2024-09-18T10:16:08.118 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.118 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.118 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:08.118 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.740191+0000", 2024-09-18T10:16:08.118 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00013140999999999999 2024-09-18T10:16:08.118 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.118 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.118 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:08.119 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.740901+0000", 2024-09-18T10:16:08.119 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00070978199999999995 2024-09-18T10:16:08.119 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.119 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.119 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:08.119 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.741853+0000", 2024-09-18T10:16:08.119 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00095228399999999996 2024-09-18T10:16:08.119 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.119 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.119 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:08.119 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:41.315753+0000", 2024-09-18T10:16:08.119 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.57389904999999997 2024-09-18T10:16:08.119 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.119 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.119 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:08.120 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:41.315917+0000", 2024-09-18T10:16:08.120 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00016491699999999999 2024-09-18T10:16:08.120 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:08.120 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:08.120 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:08.120 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.120 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.120 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:763 3.17s0 3:ee962131:::smithi18925896-29:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e85)", 2024-09-18T10:16:08.120 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:40.627257+0000", 2024-09-18T10:16:08.120 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 27.385344422999999, 2024-09-18T10:16:08.120 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.89335734899999997, 2024-09-18T10:16:08.120 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:08.120 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "started", 2024-09-18T10:16:08.120 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:08.121 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:16:08.121 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:08.121 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 763 2024-09-18T10:16:08.121 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.121 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:08.121 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.121 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:08.121 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.627257+0000", 2024-09-18T10:16:08.121 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:08.121 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.121 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.121 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:08.121 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.627257+0000", 2024-09-18T10:16:08.121 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:08.122 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.122 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.122 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:08.122 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.627256+0000", 2024-09-18T10:16:08.122 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:16:08.122 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.122 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.122 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:08.122 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.627260+0000", 2024-09-18T10:16:08.122 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.7450000000000001e-06 2024-09-18T10:16:08.122 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.122 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.122 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:08.123 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.627269+0000", 2024-09-18T10:16:08.123 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.4400000000000005e-06 2024-09-18T10:16:08.123 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.123 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.123 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:08.123 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.627273+0000", 2024-09-18T10:16:08.123 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.8600000000000001e-06 2024-09-18T10:16:08.123 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.123 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.123 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:08.123 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.717888+0000", 2024-09-18T10:16:08.123 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.090614586999999996 2024-09-18T10:16:08.123 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.123 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.123 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:16:08.124 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.717909+0000", 2024-09-18T10:16:08.124 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.0942999999999999e-05 2024-09-18T10:16:08.124 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.124 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.124 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:08.124 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.741375+0000", 2024-09-18T10:16:08.124 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.023465604000000001 2024-09-18T10:16:08.124 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.124 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.124 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for rw locks", 2024-09-18T10:16:08.124 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.741656+0000", 2024-09-18T10:16:08.124 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00028103600000000002 2024-09-18T10:16:08.124 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.125 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.125 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:08.125 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:41.317563+0000", 2024-09-18T10:16:08.125 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.57590774499999997 2024-09-18T10:16:08.125 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.125 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.125 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:08.125 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:41.317798+0000", 2024-09-18T10:16:08.125 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00023456099999999999 2024-09-18T10:16:08.125 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.125 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.125 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:08.125 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:41.520615+0000", 2024-09-18T10:16:08.125 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20281658499999999 2024-09-18T10:16:08.125 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:08.126 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:08.126 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:08.126 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:08.126 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:08.126 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-09-18T10:16:08.127 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_ops_in_flight 2024-09-18T10:16:08.128 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-18T10:16:08.123+0000 7f14c6a1a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 22138) UID: 0 2024-09-18T10:16:08.228 INFO:tasks.ceph.osd.3.smithi189.stderr:2024-09-18T10:16:08.221+0000 7f2440fc7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22396) UID: 0 2024-09-18T10:16:08.231 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-09-18T10:16:08.231 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-09-18T10:16:08.231 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-09-18T10:16:08.231 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-09-18T10:16:08.240 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_blocked_ops 2024-09-18T10:16:08.329 INFO:tasks.ceph.osd.11.smithi189.stderr:2024-09-18T10:16:08.325+0000 7f718b60a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 22400) UID: 0 2024-09-18T10:16:08.340 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-09-18T10:16:08.340 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-09-18T10:16:08.341 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-09-18T10:16:08.341 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-09-18T10:16:08.341 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-09-18T10:16:08.349 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_historic_ops 2024-09-18T10:16:08.375 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T10:16:08.430 INFO:tasks.ceph.osd.15.smithi189.stderr:2024-09-18T10:16:08.425+0000 7fd0e725e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 22401) UID: 0 2024-09-18T10:16:08.454 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-09-18T10:16:08.454 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-09-18T10:16:08.455 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-09-18T10:16:08.455 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-09-18T10:16:08.455 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.455 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4341.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-18T10:16:08.455 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:35.324351+0000", 2024-09-18T10:16:08.455 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 93.127340317000005, 2024-09-18T10:16:08.455 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.36640713699999999, 2024-09-18T10:16:08.455 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:08.455 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:08.455 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:08.455 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4341", 2024-09-18T10:16:08.455 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.29:0/3001591505", 2024-09-18T10:16:08.456 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 1 2024-09-18T10:16:08.456 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.456 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:08.456 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.456 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:08.456 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.324351+0000", 2024-09-18T10:16:08.456 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:08.456 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.456 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.456 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:08.456 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.324351+0000", 2024-09-18T10:16:08.456 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:08.457 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.457 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.457 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:08.457 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.324348+0000", 2024-09-18T10:16:08.457 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999976 2024-09-18T10:16:08.457 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.457 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.457 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:08.457 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.324359+0000", 2024-09-18T10:16:08.457 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1048e-05 2024-09-18T10:16:08.457 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.457 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.457 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:08.458 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.324383+0000", 2024-09-18T10:16:08.458 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.3813000000000001e-05 2024-09-18T10:16:08.458 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.458 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.458 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:08.458 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.324421+0000", 2024-09-18T10:16:08.458 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8012000000000001e-05 2024-09-18T10:16:08.458 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.458 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.458 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:08.458 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.324548+0000", 2024-09-18T10:16:08.458 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00012702199999999999 2024-09-18T10:16:08.459 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.459 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.459 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for peered", 2024-09-18T10:16:08.459 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.324565+0000", 2024-09-18T10:16:08.459 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.6325e-05 2024-09-18T10:16:08.459 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.459 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.459 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:08.459 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.588380+0000", 2024-09-18T10:16:08.459 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.263815458 2024-09-18T10:16:08.459 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.459 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.459 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:08.460 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.588626+0000", 2024-09-18T10:16:08.460 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00024633499999999998 2024-09-18T10:16:08.460 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.460 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.460 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 5", 2024-09-18T10:16:08.460 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.588803+0000", 2024-09-18T10:16:08.460 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00017640599999999999 2024-09-18T10:16:08.460 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.460 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.460 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:16:08.460 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.589943+0000", 2024-09-18T10:16:08.460 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0011400049999999999 2024-09-18T10:16:08.460 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.461 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.461 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:16:08.461 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.690623+0000", 2024-09-18T10:16:08.461 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100680032 2024-09-18T10:16:08.461 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.461 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.461 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:08.461 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.690702+0000", 2024-09-18T10:16:08.461 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.9342e-05 2024-09-18T10:16:08.461 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.461 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.461 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:08.461 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.690758+0000", 2024-09-18T10:16:08.462 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5639e-05 2024-09-18T10:16:08.462 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:08.462 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:08.462 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:08.462 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.462 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.462 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:264 3.2s0 3:46b2871c:::smithi18925896-34:head [append 0~606208] snapc 5=[5,4,3] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-18T10:16:08.462 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:56.739704+0000", 2024-09-18T10:16:08.462 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 71.711987332000007, 2024-09-18T10:16:08.462 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.79624036899999995, 2024-09-18T10:16:08.462 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:08.462 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:08.462 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:08.463 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:16:08.463 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:08.463 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 264 2024-09-18T10:16:08.463 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.463 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:08.463 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.463 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:08.463 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.739704+0000", 2024-09-18T10:16:08.463 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:08.463 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.463 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.463 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:08.463 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.739704+0000", 2024-09-18T10:16:08.464 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:08.464 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.464 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.464 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:08.464 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.739702+0000", 2024-09-18T10:16:08.464 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:16:08.464 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.464 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.464 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:08.464 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.740467+0000", 2024-09-18T10:16:08.464 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00076493100000000005 2024-09-18T10:16:08.464 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.464 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.465 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:08.465 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.740485+0000", 2024-09-18T10:16:08.465 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8516e-05 2024-09-18T10:16:08.465 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.465 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.465 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:08.465 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.740495+0000", 2024-09-18T10:16:08.465 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.2669999999999995e-06 2024-09-18T10:16:08.465 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.465 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.465 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:08.465 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.028948+0000", 2024-09-18T10:16:08.465 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.28845313500000003 2024-09-18T10:16:08.466 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.466 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.466 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for peered", 2024-09-18T10:16:08.466 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.028969+0000", 2024-09-18T10:16:08.466 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.1369000000000002e-05 2024-09-18T10:16:08.466 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.466 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.466 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:08.466 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.228324+0000", 2024-09-18T10:16:08.466 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.19935476699999999 2024-09-18T10:16:08.466 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.466 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.466 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:08.467 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.228558+0000", 2024-09-18T10:16:08.467 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00023393099999999999 2024-09-18T10:16:08.467 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.467 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.467 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:08.467 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.230390+0000", 2024-09-18T10:16:08.467 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018326970000000001 2024-09-18T10:16:08.467 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.467 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.467 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:08.467 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.239491+0000", 2024-09-18T10:16:08.467 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0091007950000000001 2024-09-18T10:16:08.467 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.468 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.468 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:08.468 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.535834+0000", 2024-09-18T10:16:08.468 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.29634234100000001 2024-09-18T10:16:08.468 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.468 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.468 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:08.468 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.535944+0000", 2024-09-18T10:16:08.468 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000110401 2024-09-18T10:16:08.468 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:08.468 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:08.468 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:08.468 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.468 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.469 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:265 3.2s0 3:46b2871c:::smithi18925896-34:head [append 0~376832] snapc 5=[5,4,3] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-18T10:16:08.469 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:56.741882+0000", 2024-09-18T10:16:08.469 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 71.709809309999997, 2024-09-18T10:16:08.469 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.80071129500000005, 2024-09-18T10:16:08.469 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:08.469 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:08.469 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:08.469 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:16:08.469 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:08.469 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 265 2024-09-18T10:16:08.469 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.469 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:08.469 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.470 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:08.470 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.741882+0000", 2024-09-18T10:16:08.470 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:08.470 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.470 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.470 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:08.470 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.741882+0000", 2024-09-18T10:16:08.470 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:08.470 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.470 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.470 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:08.470 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.741880+0000", 2024-09-18T10:16:08.471 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:16:08.471 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.471 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.471 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:08.471 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742251+0000", 2024-09-18T10:16:08.471 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00037050499999999998 2024-09-18T10:16:08.471 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.471 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.471 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:08.471 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742264+0000", 2024-09-18T10:16:08.471 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2787e-05 2024-09-18T10:16:08.471 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.471 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.472 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:08.472 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742269+0000", 2024-09-18T10:16:08.472 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.0509999999999996e-06 2024-09-18T10:16:08.472 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.472 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.472 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:08.472 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.234342+0000", 2024-09-18T10:16:08.472 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.492073712 2024-09-18T10:16:08.472 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.472 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.472 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:08.472 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.234556+0000", 2024-09-18T10:16:08.472 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00021380699999999999 2024-09-18T10:16:08.473 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.473 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.473 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:08.473 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.236316+0000", 2024-09-18T10:16:08.473 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0017597100000000001 2024-09-18T10:16:08.473 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.473 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.473 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:08.473 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.241476+0000", 2024-09-18T10:16:08.473 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0051604449999999996 2024-09-18T10:16:08.473 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.474 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.474 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:08.474 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.542502+0000", 2024-09-18T10:16:08.474 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.30102573700000002 2024-09-18T10:16:08.474 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.474 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.474 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:08.474 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.542593+0000", 2024-09-18T10:16:08.474 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.0859000000000005e-05 2024-09-18T10:16:08.474 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:08.474 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:08.474 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:08.475 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.475 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.475 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:266 3.2s0 3:46b2871c:::smithi18925896-34:head [setxattr _header (56) in=7b] snapc 5=[5,4,3] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-18T10:16:08.475 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:56.742274+0000", 2024-09-18T10:16:08.475 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 71.709416679, 2024-09-18T10:16:08.475 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.80056005299999999, 2024-09-18T10:16:08.475 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:08.475 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:08.475 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:08.475 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:16:08.475 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:08.475 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 266 2024-09-18T10:16:08.475 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.475 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:08.476 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.476 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:08.476 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742274+0000", 2024-09-18T10:16:08.476 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:08.476 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.476 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.476 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:08.476 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742274+0000", 2024-09-18T10:16:08.476 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:08.476 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.476 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.476 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:08.477 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742274+0000", 2024-09-18T10:16:08.477 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999995 2024-09-18T10:16:08.477 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.477 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.477 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:08.477 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742278+0000", 2024-09-18T10:16:08.477 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.0840000000000002e-06 2024-09-18T10:16:08.477 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.477 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.477 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:08.477 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742284+0000", 2024-09-18T10:16:08.477 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.1090000000000003e-06 2024-09-18T10:16:08.477 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.477 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.478 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:08.478 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742289+0000", 2024-09-18T10:16:08.478 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.4890000000000003e-06 2024-09-18T10:16:08.478 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.478 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.478 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:08.478 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.237660+0000", 2024-09-18T10:16:08.478 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.49537068299999998 2024-09-18T10:16:08.478 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.478 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.478 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:08.478 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.237924+0000", 2024-09-18T10:16:08.478 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00026387300000000003 2024-09-18T10:16:08.479 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.479 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.479 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:08.479 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.238979+0000", 2024-09-18T10:16:08.479 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0010549380000000001 2024-09-18T10:16:08.479 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.479 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.479 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:08.479 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.243571+0000", 2024-09-18T10:16:08.479 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0045919749999999999 2024-09-18T10:16:08.479 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.479 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.479 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:08.480 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.542742+0000", 2024-09-18T10:16:08.480 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.29917095999999999 2024-09-18T10:16:08.480 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.480 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.480 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:08.480 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.542834+0000", 2024-09-18T10:16:08.480 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.2536999999999998e-05 2024-09-18T10:16:08.480 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:08.480 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:08.480 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:08.480 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.480 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.480 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:267 3.2s0 3:46b2871c:::smithi18925896-34:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e41)", 2024-09-18T10:16:08.481 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:56.742292+0000", 2024-09-18T10:16:08.481 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 71.709398840999995, 2024-09-18T10:16:08.481 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.98973012599999999, 2024-09-18T10:16:08.481 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:08.481 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "started", 2024-09-18T10:16:08.481 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:08.481 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:16:08.481 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:08.481 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 267 2024-09-18T10:16:08.481 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.481 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:08.481 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.481 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:08.481 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742292+0000", 2024-09-18T10:16:08.482 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:08.482 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.482 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.482 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:08.482 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742292+0000", 2024-09-18T10:16:08.482 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:08.482 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.482 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.482 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:08.482 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742292+0000", 2024-09-18T10:16:08.482 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999995 2024-09-18T10:16:08.482 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.482 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.482 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:08.483 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742294+0000", 2024-09-18T10:16:08.483 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.103e-06 2024-09-18T10:16:08.483 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.483 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.483 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:08.483 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742298+0000", 2024-09-18T10:16:08.483 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8070000000000001e-06 2024-09-18T10:16:08.483 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.483 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.483 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:08.483 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742300+0000", 2024-09-18T10:16:08.483 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5890000000000001e-06 2024-09-18T10:16:08.483 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.483 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.484 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:08.484 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.239711+0000", 2024-09-18T10:16:08.484 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.497410625 2024-09-18T10:16:08.484 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.484 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.484 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for rw locks", 2024-09-18T10:16:08.484 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.239786+0000", 2024-09-18T10:16:08.484 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.4827999999999996e-05 2024-09-18T10:16:08.484 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.484 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.484 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:08.484 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.543234+0000", 2024-09-18T10:16:08.485 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.30344799099999997 2024-09-18T10:16:08.485 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.485 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.485 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:08.485 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.543335+0000", 2024-09-18T10:16:08.485 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000101625 2024-09-18T10:16:08.485 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.485 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.485 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:08.485 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.732022+0000", 2024-09-18T10:16:08.485 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.18868686700000001 2024-09-18T10:16:08.485 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:08.485 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:08.485 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:08.486 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.486 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.486 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:496 3.2s0 3:44cb9e27:::smithi18925896-5:head [read 0~81920,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T10:16:08.486 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:17.889532+0000", 2024-09-18T10:16:08.486 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 50.562159221999998, 2024-09-18T10:16:08.486 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.32688004999999998, 2024-09-18T10:16:08.486 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:08.486 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "started", 2024-09-18T10:16:08.486 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:08.486 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:16:08.486 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:08.486 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 496 2024-09-18T10:16:08.486 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.487 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:08.487 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.487 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:08.487 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889532+0000", 2024-09-18T10:16:08.487 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:08.487 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.487 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.487 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:08.487 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889532+0000", 2024-09-18T10:16:08.487 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:08.487 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.487 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.487 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:08.488 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889529+0000", 2024-09-18T10:16:08.488 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999971 2024-09-18T10:16:08.488 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.488 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.488 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:08.488 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889546+0000", 2024-09-18T10:16:08.488 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.6690000000000001e-05 2024-09-18T10:16:08.488 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.488 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.488 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:08.488 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889580+0000", 2024-09-18T10:16:08.488 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.4638999999999998e-05 2024-09-18T10:16:08.488 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.489 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.489 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:08.489 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889596+0000", 2024-09-18T10:16:08.489 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.5571999999999999e-05 2024-09-18T10:16:08.489 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.489 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.489 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:08.489 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.989817+0000", 2024-09-18T10:16:08.489 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100221713 2024-09-18T10:16:08.489 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.489 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.489 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op must wait for map", 2024-09-18T10:16:08.489 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.989836+0000", 2024-09-18T10:16:08.489 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8516e-05 2024-09-18T10:16:08.490 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.490 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.490 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:08.490 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.990420+0000", 2024-09-18T10:16:08.490 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000583918 2024-09-18T10:16:08.490 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.490 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.490 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:08.490 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.990693+0000", 2024-09-18T10:16:08.490 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00027301499999999999 2024-09-18T10:16:08.490 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.490 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.490 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:08.491 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:18.216412+0000", 2024-09-18T10:16:08.491 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.22571880699999999 2024-09-18T10:16:08.491 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:08.491 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:08.491 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:08.491 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.491 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.491 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:497 3.2s0 3:44cb9e27:::smithi18925896-5:head [read 0~81920,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T10:16:08.491 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:17.889627+0000", 2024-09-18T10:16:08.491 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 50.562064305, 2024-09-18T10:16:08.491 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.32738385800000003, 2024-09-18T10:16:08.491 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:08.491 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "started", 2024-09-18T10:16:08.491 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:08.492 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:16:08.492 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:08.492 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 497 2024-09-18T10:16:08.492 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.492 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:08.492 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.492 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:08.492 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889627+0000", 2024-09-18T10:16:08.492 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:08.492 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.492 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.492 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:08.492 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889627+0000", 2024-09-18T10:16:08.493 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:08.493 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.493 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.493 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:08.493 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889625+0000", 2024-09-18T10:16:08.493 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:16:08.493 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.493 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.493 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:08.493 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889633+0000", 2024-09-18T10:16:08.493 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.4170000000000003e-06 2024-09-18T10:16:08.493 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.493 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.494 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:08.494 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889664+0000", 2024-09-18T10:16:08.494 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0589000000000001e-05 2024-09-18T10:16:08.494 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.494 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.494 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:08.494 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889672+0000", 2024-09-18T10:16:08.494 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.3729999999999993e-06 2024-09-18T10:16:08.494 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.494 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.494 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:08.494 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.990017+0000", 2024-09-18T10:16:08.494 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100345608 2024-09-18T10:16:08.495 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.495 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.495 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:16:08.495 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.990046+0000", 2024-09-18T10:16:08.495 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8912999999999999e-05 2024-09-18T10:16:08.495 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.495 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.495 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:08.495 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.991459+0000", 2024-09-18T10:16:08.495 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0014122340000000001 2024-09-18T10:16:08.495 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.495 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.495 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:08.496 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.991631+0000", 2024-09-18T10:16:08.496 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000172029 2024-09-18T10:16:08.496 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.496 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.496 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:08.496 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:18.217010+0000", 2024-09-18T10:16:08.496 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.22537980499999999 2024-09-18T10:16:08.496 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:08.496 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:08.496 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:08.496 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.496 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.496 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:498 3.2s0 3:44cb9e27:::smithi18925896-5:head [read 0~81920,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T10:16:08.497 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:17.889709+0000", 2024-09-18T10:16:08.497 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 50.561981947, 2024-09-18T10:16:08.497 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.32775850600000001, 2024-09-18T10:16:08.497 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:08.497 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "started", 2024-09-18T10:16:08.497 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:08.497 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:16:08.497 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:08.497 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 498 2024-09-18T10:16:08.497 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.497 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:08.497 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.497 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:08.497 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889709+0000", 2024-09-18T10:16:08.498 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:08.498 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.498 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.498 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:08.498 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889709+0000", 2024-09-18T10:16:08.498 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:08.498 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.498 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.498 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:08.498 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889708+0000", 2024-09-18T10:16:08.498 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:16:08.498 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.498 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.499 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:08.499 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889716+0000", 2024-09-18T10:16:08.499 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.0989999999999999e-06 2024-09-18T10:16:08.499 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.499 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.499 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:08.499 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889732+0000", 2024-09-18T10:16:08.499 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.6093e-05 2024-09-18T10:16:08.499 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.499 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.499 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:08.499 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889742+0000", 2024-09-18T10:16:08.499 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.4599999999999992e-06 2024-09-18T10:16:08.500 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.500 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.500 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:08.500 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.992175+0000", 2024-09-18T10:16:08.500 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10243377200000001 2024-09-18T10:16:08.500 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.500 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.500 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:08.500 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.992333+0000", 2024-09-18T10:16:08.500 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00015753499999999999 2024-09-18T10:16:08.500 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.500 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.501 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:08.501 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:18.217467+0000", 2024-09-18T10:16:08.501 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.22513459999999999 2024-09-18T10:16:08.501 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:08.501 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:08.501 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:08.501 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.501 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.501 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:516 3.2s0 3:44cb9e27:::smithi18925896-5:head [append 0~425984] snapc 1a=[1a,19,18,16,12,11,10,f,e] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-18T10:16:08.501 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:19.430973+0000", 2024-09-18T10:16:08.501 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 49.020717558000001, 2024-09-18T10:16:08.501 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.59812562800000002, 2024-09-18T10:16:08.501 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:08.501 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:08.502 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:08.502 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:16:08.502 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:08.502 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 516 2024-09-18T10:16:08.502 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.502 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:08.502 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.502 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:08.502 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.430973+0000", 2024-09-18T10:16:08.502 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:08.502 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.502 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.503 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:08.503 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.430973+0000", 2024-09-18T10:16:08.503 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:08.503 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.503 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.503 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:08.503 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.430970+0000", 2024-09-18T10:16:08.503 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999967 2024-09-18T10:16:08.503 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.503 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.503 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:08.503 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.431995+0000", 2024-09-18T10:16:08.503 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001024896 2024-09-18T10:16:08.503 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.504 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.504 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:08.504 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.432030+0000", 2024-09-18T10:16:08.504 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5241999999999999e-05 2024-09-18T10:16:08.504 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.504 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.504 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:08.504 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.432044+0000", 2024-09-18T10:16:08.504 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.411e-05 2024-09-18T10:16:08.504 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.504 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.504 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:08.504 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.432226+0000", 2024-09-18T10:16:08.505 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000181841 2024-09-18T10:16:08.505 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.505 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.505 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:08.505 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.432421+0000", 2024-09-18T10:16:08.505 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000194527 2024-09-18T10:16:08.505 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.505 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.505 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:08.505 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.434150+0000", 2024-09-18T10:16:08.505 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001729742 2024-09-18T10:16:08.505 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.505 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.505 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:08.506 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.436513+0000", 2024-09-18T10:16:08.506 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0023622299999999999 2024-09-18T10:16:08.506 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.506 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.506 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:08.506 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:20.029005+0000", 2024-09-18T10:16:08.506 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.59249230100000005 2024-09-18T10:16:08.506 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.506 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.506 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:08.506 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:20.029099+0000", 2024-09-18T10:16:08.506 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.4042000000000005e-05 2024-09-18T10:16:08.507 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:08.507 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:08.507 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:08.507 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.507 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.507 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:517 3.2s0 3:44cb9e27:::smithi18925896-5:head [append 0~786432] snapc 1a=[1a,19,18,16,12,11,10,f,e] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-18T10:16:08.507 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:19.439831+0000", 2024-09-18T10:16:08.507 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 49.011860018999997, 2024-09-18T10:16:08.507 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.58958403800000003, 2024-09-18T10:16:08.507 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:08.507 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:08.507 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:08.507 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:16:08.507 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:08.508 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 517 2024-09-18T10:16:08.508 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.508 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:08.508 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.508 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:08.508 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.439831+0000", 2024-09-18T10:16:08.508 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:08.508 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.508 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.508 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:08.508 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.439831+0000", 2024-09-18T10:16:08.508 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:08.508 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.508 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.509 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:08.509 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.439828+0000", 2024-09-18T10:16:08.509 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999976 2024-09-18T10:16:08.509 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.509 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.509 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:08.509 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.441109+0000", 2024-09-18T10:16:08.509 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001280988 2024-09-18T10:16:08.509 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.509 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.509 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:08.509 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.441168+0000", 2024-09-18T10:16:08.510 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.8869000000000002e-05 2024-09-18T10:16:08.510 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.510 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.510 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:08.510 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.541302+0000", 2024-09-18T10:16:08.510 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100134055 2024-09-18T10:16:08.510 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.510 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.510 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:08.510 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.637962+0000", 2024-09-18T10:16:08.510 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.096659682999999996 2024-09-18T10:16:08.510 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.510 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.511 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:08.511 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.638164+0000", 2024-09-18T10:16:08.511 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00020226700000000001 2024-09-18T10:16:08.511 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.511 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.511 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:08.511 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.639933+0000", 2024-09-18T10:16:08.511 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001768501 2024-09-18T10:16:08.511 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.511 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.511 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:08.511 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.646572+0000", 2024-09-18T10:16:08.511 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.006639069 2024-09-18T10:16:08.512 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.512 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.512 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:08.512 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:20.029341+0000", 2024-09-18T10:16:08.512 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.38276939599999998 2024-09-18T10:16:08.512 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.512 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.512 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:08.512 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:20.029415+0000", 2024-09-18T10:16:08.512 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.3726000000000005e-05 2024-09-18T10:16:08.512 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:08.512 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:08.512 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:08.513 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.513 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.513 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:518 3.2s0 3:44cb9e27:::smithi18925896-5:head [append 0~376832] snapc 1a=[1a,19,18,16,12,11,10,f,e] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-18T10:16:08.513 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:19.541352+0000", 2024-09-18T10:16:08.513 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 48.910338981000002, 2024-09-18T10:16:08.513 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.48828847600000003, 2024-09-18T10:16:08.513 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:08.513 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:08.513 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:08.513 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:16:08.513 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:08.513 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 518 2024-09-18T10:16:08.513 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.514 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:08.514 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.514 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:08.514 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.541352+0000", 2024-09-18T10:16:08.514 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:08.514 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.514 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.514 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:08.514 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.541352+0000", 2024-09-18T10:16:08.514 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:08.514 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.514 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.515 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:08.515 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.541350+0000", 2024-09-18T10:16:08.515 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:16:08.515 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.515 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.515 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:08.515 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.541834+0000", 2024-09-18T10:16:08.515 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00048402500000000002 2024-09-18T10:16:08.515 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.515 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.515 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:08.515 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.541861+0000", 2024-09-18T10:16:08.515 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.7333000000000001e-05 2024-09-18T10:16:08.515 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.515 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.516 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:08.516 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.541875+0000", 2024-09-18T10:16:08.516 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4025999999999999e-05 2024-09-18T10:16:08.516 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.516 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.516 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:08.516 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.641684+0000", 2024-09-18T10:16:08.516 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.099808872000000007 2024-09-18T10:16:08.516 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.516 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.516 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:08.516 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.641873+0000", 2024-09-18T10:16:08.516 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00018933200000000001 2024-09-18T10:16:08.517 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.517 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.517 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:08.517 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.643401+0000", 2024-09-18T10:16:08.517 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001527104 2024-09-18T10:16:08.517 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.517 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.517 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:08.517 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.646854+0000", 2024-09-18T10:16:08.517 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0034532069999999998 2024-09-18T10:16:08.517 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.517 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.518 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:08.518 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:20.029569+0000", 2024-09-18T10:16:08.518 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.38271509599999998 2024-09-18T10:16:08.518 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.518 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.518 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:08.518 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:20.029640+0000", 2024-09-18T10:16:08.518 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.1594000000000006e-05 2024-09-18T10:16:08.518 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:08.518 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:08.518 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:08.518 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.518 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.519 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:519 3.2s0 3:44cb9e27:::smithi18925896-5:head [setxattr _header (56) in=7b] snapc 1a=[1a,19,18,16,12,11,10,f,e] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-18T10:16:08.519 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:19.541886+0000", 2024-09-18T10:16:08.519 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 48.909805294000002, 2024-09-18T10:16:08.519 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.488165768, 2024-09-18T10:16:08.519 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:08.519 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:08.519 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:08.519 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:16:08.519 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:08.519 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 519 2024-09-18T10:16:08.519 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.519 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:08.519 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.519 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:08.520 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.541886+0000", 2024-09-18T10:16:08.520 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:08.520 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.520 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.520 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:08.520 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.541886+0000", 2024-09-18T10:16:08.520 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:08.520 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.520 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.520 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:08.520 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.541884+0000", 2024-09-18T10:16:08.520 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:16:08.521 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.521 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.521 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:08.521 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.541890+0000", 2024-09-18T10:16:08.521 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.0589999999999999e-06 2024-09-18T10:16:08.521 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.521 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.521 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:08.521 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.541899+0000", 2024-09-18T10:16:08.521 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.02e-06 2024-09-18T10:16:08.521 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.521 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.521 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:08.522 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.541905+0000", 2024-09-18T10:16:08.522 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.9399999999999999e-06 2024-09-18T10:16:08.522 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.522 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.522 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:08.522 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.644561+0000", 2024-09-18T10:16:08.522 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.102655408 2024-09-18T10:16:08.522 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.522 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.522 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:08.522 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.644869+0000", 2024-09-18T10:16:08.522 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00030766700000000002 2024-09-18T10:16:08.523 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.523 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.523 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:08.523 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.645745+0000", 2024-09-18T10:16:08.523 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000876314 2024-09-18T10:16:08.523 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.523 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.523 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:08.523 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.647092+0000", 2024-09-18T10:16:08.523 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0013469269999999999 2024-09-18T10:16:08.523 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.523 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.523 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:08.524 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:20.029901+0000", 2024-09-18T10:16:08.524 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.38280895199999998 2024-09-18T10:16:08.524 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.524 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.524 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:08.524 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:20.030051+0000", 2024-09-18T10:16:08.524 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00015067699999999999 2024-09-18T10:16:08.525 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:08.525 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:08.525 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:08.525 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.525 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.525 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:573 3.2s0 3:46b2871c:::smithi18925896-34:head [read 0~557056,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-18T10:16:08.525 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:24.942342+0000", 2024-09-18T10:16:08.525 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 43.509348580000001, 2024-09-18T10:16:08.525 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.36312128700000001, 2024-09-18T10:16:08.525 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:08.526 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "started", 2024-09-18T10:16:08.526 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:08.526 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:16:08.526 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:08.526 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 573 2024-09-18T10:16:08.526 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.526 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:08.526 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.526 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:08.526 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.942342+0000", 2024-09-18T10:16:08.526 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:08.526 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.526 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.527 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:08.527 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.942342+0000", 2024-09-18T10:16:08.527 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:08.527 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.527 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.527 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:08.527 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.942341+0000", 2024-09-18T10:16:08.527 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:16:08.527 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.527 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.527 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:08.527 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.942363+0000", 2024-09-18T10:16:08.527 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.1406000000000001e-05 2024-09-18T10:16:08.527 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.528 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.528 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:08.528 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.942374+0000", 2024-09-18T10:16:08.528 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1466e-05 2024-09-18T10:16:08.528 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.528 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.528 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:08.528 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.942381+0000", 2024-09-18T10:16:08.528 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.9530000000000002e-06 2024-09-18T10:16:08.528 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.528 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.528 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:08.529 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.942654+0000", 2024-09-18T10:16:08.529 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000272692 2024-09-18T10:16:08.529 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.529 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.529 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:16:08.529 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.942672+0000", 2024-09-18T10:16:08.529 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7643e-05 2024-09-18T10:16:08.529 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.529 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.529 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:08.529 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:25.101730+0000", 2024-09-18T10:16:08.529 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.15905815800000001 2024-09-18T10:16:08.529 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.530 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.530 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:08.530 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:25.101881+0000", 2024-09-18T10:16:08.530 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000151299 2024-09-18T10:16:08.530 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.530 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.530 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:08.530 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:25.305464+0000", 2024-09-18T10:16:08.530 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.203582656 2024-09-18T10:16:08.530 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:08.530 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:08.531 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:08.531 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.531 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.531 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:574 3.2s0 3:46b2871c:::smithi18925896-34:head [read 0~557056,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-18T10:16:08.531 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:24.942392+0000", 2024-09-18T10:16:08.531 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 43.509298784000002, 2024-09-18T10:16:08.531 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.36415850300000002, 2024-09-18T10:16:08.531 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:08.531 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "started", 2024-09-18T10:16:08.531 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:08.531 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:16:08.531 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:08.531 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 574 2024-09-18T10:16:08.532 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.532 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:08.532 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.532 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:08.532 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.942392+0000", 2024-09-18T10:16:08.532 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:08.532 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.532 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.532 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:08.532 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.942392+0000", 2024-09-18T10:16:08.532 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:08.532 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.533 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.533 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:08.533 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.942391+0000", 2024-09-18T10:16:08.533 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:16:08.533 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.533 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.533 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:08.533 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.942396+0000", 2024-09-18T10:16:08.533 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.6759999999999998e-06 2024-09-18T10:16:08.533 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.533 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.533 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:08.533 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.942407+0000", 2024-09-18T10:16:08.534 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0686999999999999e-05 2024-09-18T10:16:08.534 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.534 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.534 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:08.534 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.942413+0000", 2024-09-18T10:16:08.534 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.1659999999999999e-06 2024-09-18T10:16:08.534 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.534 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.534 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:08.534 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.942750+0000", 2024-09-18T10:16:08.534 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00033731700000000001 2024-09-18T10:16:08.534 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.535 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.535 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:16:08.535 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.942776+0000", 2024-09-18T10:16:08.535 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6342999999999998e-05 2024-09-18T10:16:08.535 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.535 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.535 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:08.535 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:25.102396+0000", 2024-09-18T10:16:08.535 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.15961924199999999 2024-09-18T10:16:08.535 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.535 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.535 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:08.536 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:25.102659+0000", 2024-09-18T10:16:08.536 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00026355500000000001 2024-09-18T10:16:08.536 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.536 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.536 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:08.536 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:25.306551+0000", 2024-09-18T10:16:08.536 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20389143700000001 2024-09-18T10:16:08.536 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:08.536 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:08.536 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:08.536 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.536 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.536 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWriteReply(3.2s0 89/40 ECSubWriteReply(tid=135, last_complete=89'96, committed=1, applied=1))", 2024-09-18T10:16:08.537 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:45.106385+0000", 2024-09-18T10:16:08.537 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 23.345306010000002, 2024-09-18T10:16:08.537 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.35616392200000002, 2024-09-18T10:16:08.537 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:08.537 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:16:08.537 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:08.537 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.537 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:08.537 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:45.106385+0000", 2024-09-18T10:16:08.537 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:08.537 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.537 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.537 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:08.538 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:45.106385+0000", 2024-09-18T10:16:08.538 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:08.538 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.538 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.538 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:08.538 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:45.106384+0000", 2024-09-18T10:16:08.538 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999995 2024-09-18T10:16:08.538 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.538 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.538 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:08.538 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:45.106388+0000", 2024-09-18T10:16:08.538 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.6100000000000002e-06 2024-09-18T10:16:08.538 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.538 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.539 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:08.539 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:45.106395+0000", 2024-09-18T10:16:08.539 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.8399999999999997e-06 2024-09-18T10:16:08.539 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.539 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.539 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:08.539 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:45.106399+0000", 2024-09-18T10:16:08.539 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.1899999999999997e-06 2024-09-18T10:16:08.539 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.539 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.539 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:08.539 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:45.461488+0000", 2024-09-18T10:16:08.539 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.35508847199999999 2024-09-18T10:16:08.540 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.540 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.540 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:08.540 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:45.462549+0000", 2024-09-18T10:16:08.540 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0010612760000000001 2024-09-18T10:16:08.540 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:08.540 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:08.540 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:08.540 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.540 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.540 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:891 3.2s0 3:427b2c0b:::smithi18925896-42:head [append 0~0] snapc 34=[34,33,30,2f,28,20,1f,1a,11] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-18T10:16:08.541 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:46.655855+0000", 2024-09-18T10:16:08.541 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 21.795836282, 2024-09-18T10:16:08.541 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.45749477399999999, 2024-09-18T10:16:08.541 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:08.541 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:08.541 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:08.541 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:16:08.541 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:08.541 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 891 2024-09-18T10:16:08.541 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.541 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:08.541 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.542 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:08.542 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.655855+0000", 2024-09-18T10:16:08.542 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:08.542 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.542 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.542 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:08.542 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.655855+0000", 2024-09-18T10:16:08.542 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:08.542 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.542 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.542 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:08.542 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.655852+0000", 2024-09-18T10:16:08.543 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999971 2024-09-18T10:16:08.543 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.543 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.543 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:08.543 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.655866+0000", 2024-09-18T10:16:08.543 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4842e-05 2024-09-18T10:16:08.543 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.543 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.543 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:08.543 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.655906+0000", 2024-09-18T10:16:08.543 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.9465000000000002e-05 2024-09-18T10:16:08.543 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.543 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.543 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:08.544 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.756059+0000", 2024-09-18T10:16:08.544 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100153544 2024-09-18T10:16:08.544 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.544 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.544 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:08.544 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.804067+0000", 2024-09-18T10:16:08.544 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.048007288000000002 2024-09-18T10:16:08.544 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.544 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.544 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:08.544 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.804374+0000", 2024-09-18T10:16:08.544 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00030716700000000001 2024-09-18T10:16:08.545 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.545 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.545 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:08.545 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.807008+0000", 2024-09-18T10:16:08.545 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.002633799 2024-09-18T10:16:08.545 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.545 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.545 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:08.545 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.911478+0000", 2024-09-18T10:16:08.545 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.104469986 2024-09-18T10:16:08.545 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.546 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.546 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:08.546 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:47.113216+0000", 2024-09-18T10:16:08.546 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.201737789 2024-09-18T10:16:08.546 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.546 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.546 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:08.546 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:47.113349+0000", 2024-09-18T10:16:08.546 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00013389700000000001 2024-09-18T10:16:08.546 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:08.546 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:08.546 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:08.547 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.547 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.547 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:892 3.2s0 3:427b2c0b:::smithi18925896-42:head [rollback 28] snapc 34=[34,33,30,2f,28,20,1f,1a,11] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-18T10:16:08.547 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:46.756090+0000", 2024-09-18T10:16:08.547 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 21.695601201999999, 2024-09-18T10:16:08.547 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.35768046799999997, 2024-09-18T10:16:08.547 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:08.547 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:08.547 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:08.547 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:16:08.547 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:08.547 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 892 2024-09-18T10:16:08.547 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.547 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:08.548 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.548 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:08.548 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.756090+0000", 2024-09-18T10:16:08.548 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:08.548 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.548 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.548 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:08.548 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.756090+0000", 2024-09-18T10:16:08.548 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:08.548 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.548 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.548 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:08.548 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.756088+0000", 2024-09-18T10:16:08.549 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:16:08.549 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.549 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.549 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:08.549 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.756099+0000", 2024-09-18T10:16:08.549 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0879999999999999e-05 2024-09-18T10:16:08.549 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.549 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.549 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:08.549 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.756116+0000", 2024-09-18T10:16:08.549 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7849000000000001e-05 2024-09-18T10:16:08.549 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.549 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.549 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:08.550 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.756136+0000", 2024-09-18T10:16:08.550 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.9720000000000001e-05 2024-09-18T10:16:08.550 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.550 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.550 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:08.550 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.808713+0000", 2024-09-18T10:16:08.550 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.052577089 2024-09-18T10:16:08.550 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.550 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.550 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:08.550 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.808963+0000", 2024-09-18T10:16:08.550 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000249824 2024-09-18T10:16:08.550 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.551 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.551 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:08.551 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.810124+0000", 2024-09-18T10:16:08.551 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0011610100000000001 2024-09-18T10:16:08.551 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.551 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.551 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:08.551 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:47.011739+0000", 2024-09-18T10:16:08.551 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20161468900000001 2024-09-18T10:16:08.551 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.551 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.551 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:08.551 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:47.113617+0000", 2024-09-18T10:16:08.552 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10187808700000001 2024-09-18T10:16:08.552 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.552 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.552 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:08.552 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:47.113770+0000", 2024-09-18T10:16:08.552 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00015334300000000001 2024-09-18T10:16:08.552 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:08.552 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:08.552 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:08.552 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.552 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.552 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:893 3.2s0 3:427b2c0b:::smithi18925896-42:head [append 0~0] snapc 34=[34,33,30,2f,28,20,1f,1a,11] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-18T10:16:08.552 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:46.756179+0000", 2024-09-18T10:16:08.553 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 21.695512294, 2024-09-18T10:16:08.553 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.66260898499999998, 2024-09-18T10:16:08.553 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:08.553 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:08.553 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:08.553 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:16:08.553 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:08.553 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 893 2024-09-18T10:16:08.553 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.553 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:08.553 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.553 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:08.553 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.756179+0000", 2024-09-18T10:16:08.553 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:08.554 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.554 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.554 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:08.554 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.756179+0000", 2024-09-18T10:16:08.554 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:08.554 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.554 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.554 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:08.554 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.756177+0000", 2024-09-18T10:16:08.554 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:16:08.554 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.554 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.555 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:08.555 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.756185+0000", 2024-09-18T10:16:08.555 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.32e-06 2024-09-18T10:16:08.555 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.555 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.555 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:08.555 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.756214+0000", 2024-09-18T10:16:08.555 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8992999999999998e-05 2024-09-18T10:16:08.555 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.555 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.555 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:08.555 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.856357+0000", 2024-09-18T10:16:08.555 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100142493 2024-09-18T10:16:08.555 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.556 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.556 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:08.556 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:47.114791+0000", 2024-09-18T10:16:08.556 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.25843391799999998 2024-09-18T10:16:08.556 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.556 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.556 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:08.556 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:47.114948+0000", 2024-09-18T10:16:08.556 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00015682700000000001 2024-09-18T10:16:08.556 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.556 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.556 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:08.556 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:47.116043+0000", 2024-09-18T10:16:08.556 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0010952309999999999 2024-09-18T10:16:08.557 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.557 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.557 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:08.557 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:47.117006+0000", 2024-09-18T10:16:08.557 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00096323899999999996 2024-09-18T10:16:08.557 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.557 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.557 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:08.557 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:47.418568+0000", 2024-09-18T10:16:08.557 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.30156222300000002 2024-09-18T10:16:08.557 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.557 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.557 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:08.557 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:47.418788+0000", 2024-09-18T10:16:08.558 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00021923799999999999 2024-09-18T10:16:08.558 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:08.558 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:08.558 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:08.558 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.558 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.558 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(osd.6.22:5 3.2s0 3:47202842:::smithi18925896-22:head [copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-18T10:16:08.558 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:46.756797+0000", 2024-09-18T10:16:08.558 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 21.694893780000001, 2024-09-18T10:16:08.558 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.35575880999999998, 2024-09-18T10:16:08.558 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:08.558 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:16:08.559 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:08.559 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.559 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:08.559 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.756797+0000", 2024-09-18T10:16:08.559 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:08.559 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.559 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.559 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:08.559 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.756797+0000", 2024-09-18T10:16:08.559 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:08.559 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.559 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.559 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:08.560 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.756795+0000", 2024-09-18T10:16:08.560 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:16:08.560 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.560 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.560 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:08.560 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.756807+0000", 2024-09-18T10:16:08.560 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1970000000000001e-05 2024-09-18T10:16:08.560 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.560 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.560 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:08.560 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.756837+0000", 2024-09-18T10:16:08.560 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9706999999999999e-05 2024-09-18T10:16:08.561 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.561 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.561 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:08.561 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.756849+0000", 2024-09-18T10:16:08.561 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1904e-05 2024-09-18T10:16:08.561 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.561 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.561 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:08.561 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:47.112101+0000", 2024-09-18T10:16:08.561 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.35525225900000001 2024-09-18T10:16:08.561 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.561 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.562 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:08.562 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:47.112556+0000", 2024-09-18T10:16:08.562 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00045505300000000002 2024-09-18T10:16:08.562 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:08.562 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:08.562 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:08.562 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.562 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.562 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:953 3.2s0 3:46b2871c:::smithi18925896-34:head [read 0~557056,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-18T10:16:08.562 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:52.128760+0000", 2024-09-18T10:16:08.562 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 16.322930632999999, 2024-09-18T10:16:08.562 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.40564792700000002, 2024-09-18T10:16:08.562 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:08.563 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "started", 2024-09-18T10:16:08.563 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:08.563 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:16:08.563 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:08.563 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 953 2024-09-18T10:16:08.563 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.563 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:08.563 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.563 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:08.563 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:52.128760+0000", 2024-09-18T10:16:08.563 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:08.563 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.563 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.564 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:08.564 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:52.128760+0000", 2024-09-18T10:16:08.564 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:08.564 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.564 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.564 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:08.564 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:52.128757+0000", 2024-09-18T10:16:08.564 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999967 2024-09-18T10:16:08.564 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.564 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.564 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:08.564 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:52.128774+0000", 2024-09-18T10:16:08.564 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.6912e-05 2024-09-18T10:16:08.565 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.565 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.565 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:08.565 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:52.128809+0000", 2024-09-18T10:16:08.565 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.4913000000000003e-05 2024-09-18T10:16:08.565 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.565 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.566 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:08.566 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:52.228946+0000", 2024-09-18T10:16:08.566 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100137698 2024-09-18T10:16:08.566 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.566 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.566 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:08.566 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:52.229076+0000", 2024-09-18T10:16:08.566 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00012982 2024-09-18T10:16:08.566 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.566 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.566 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:08.566 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:52.229281+0000", 2024-09-18T10:16:08.567 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000204592 2024-09-18T10:16:08.567 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:08.567 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:08.567 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:08.567 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:52.534408+0000", 2024-09-18T10:16:08.567 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.30512741500000001 2024-09-18T10:16:08.567 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:08.567 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:08.567 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:08.567 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:08.567 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:08.567 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-09-18T10:16:08.569 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-18T10:16:08.569 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-18T10:16:08.569 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T10:16:08.570 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-18T10:16:08.563+0000 7f14c6a1a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 22138) UID: 0 2024-09-18T10:16:08.571 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-18T10:16:08.602 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-18T10:16:08.623 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-18T10:16:08.669 INFO:tasks.ceph.osd.14.smithi154.stderr:2024-09-18T10:16:08.666+0000 7f75a19e7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 22093) UID: 0 2024-09-18T10:16:08.670 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-18T10:16:08.671 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:16:08.680 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T10:16:08.708 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-18T10:16:08.732 INFO:teuthology.orchestra.run.smithi029.stdout:osd.6: {} 2024-09-18T10:16:08.752 INFO:teuthology.orchestra.run.smithi029.stdout:osd.7: {} 2024-09-18T10:16:08.771 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-18T10:16:08.765+0000 7f302f2c3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22187) UID: 0 2024-09-18T10:16:08.771 INFO:teuthology.orchestra.run.smithi029.stdout:osd.8: {} 2024-09-18T10:16:08.777 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-18T10:16:08.777 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:16:08.787 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T10:16:08.793 INFO:teuthology.orchestra.run.smithi029.stdout:osd.9: {} 2024-09-18T10:16:08.817 INFO:teuthology.orchestra.run.smithi029.stdout:osd.10: {} 2024-09-18T10:16:08.837 INFO:teuthology.orchestra.run.smithi029.stdout:osd.11: {} 2024-09-18T10:16:08.872 INFO:tasks.ceph.osd.12.smithi029.stderr:2024-09-18T10:16:08.865+0000 7f8e4e627700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 22190) UID: 0 2024-09-18T10:16:08.892 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-18T10:16:08.892 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:16:08.892 INFO:teuthology.orchestra.run.smithi029.stdout:osd.12: {} 2024-09-18T10:16:08.901 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T10:16:08.914 INFO:teuthology.orchestra.run.smithi029.stdout:osd.13: {} 2024-09-18T10:16:08.938 INFO:teuthology.orchestra.run.smithi029.stdout:osd.14: {} 2024-09-18T10:16:08.960 INFO:teuthology.orchestra.run.smithi029.stdout:osd.15: {} 2024-09-18T10:16:08.962 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T10:16:08.962 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T10:16:08.963 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T10:16:08.963 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T10:16:08.963 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T10:16:08.963 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T10:16:08.963 INFO:teuthology.orchestra.run.smithi029.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T10:16:08.963 INFO:teuthology.orchestra.run.smithi029.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T10:16:08.963 INFO:teuthology.orchestra.run.smithi029.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-18T10:16:08.963 INFO:teuthology.orchestra.run.smithi029.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-18T10:16:08.963 INFO:teuthology.orchestra.run.smithi029.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-18T10:16:08.963 INFO:teuthology.orchestra.run.smithi029.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-18T10:16:08.963 INFO:teuthology.orchestra.run.smithi029.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-18T10:16:08.963 INFO:teuthology.orchestra.run.smithi029.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-18T10:16:08.963 INFO:teuthology.orchestra.run.smithi029.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-18T10:16:08.964 INFO:teuthology.orchestra.run.smithi029.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-18T10:16:08.973 INFO:tasks.ceph.osd.11.smithi189.stderr:2024-09-18T10:16:08.969+0000 7f718b60a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 22400) UID: 0 2024-09-18T10:16:09.005 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-18T10:16:09.005 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:16:09.014 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T10:16:09.074 INFO:tasks.ceph.osd.4.smithi029.stderr:2024-09-18T10:16:09.069+0000 7f550cf6e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22189) UID: 0 2024-09-18T10:16:09.114 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-18T10:16:09.114 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:16:09.123 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T10:16:09.175 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-18T10:16:09.171+0000 7f578c52f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22134) UID: 0 2024-09-18T10:16:09.222 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-18T10:16:09.222 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:16:09.232 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-09-18T10:16:09.275 INFO:tasks.ceph.osd.11.smithi189.stderr:2024-09-18T10:16:09.269+0000 7f718b60a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 22400) UID: 0 2024-09-18T10:16:09.335 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-18T10:16:09.335 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:16:09.344 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-09-18T10:16:09.377 INFO:tasks.ceph.osd.12.smithi029.stderr:2024-09-18T10:16:09.369+0000 7f8e4e627700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 22190) UID: 0 2024-09-18T10:16:09.478 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-18T10:16:09.475+0000 7f00046a8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 22136) UID: 0 2024-09-18T10:16:09.486 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-18T10:16:09.486 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:16:09.495 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-09-18T10:16:09.579 INFO:tasks.ceph.osd.14.smithi154.stderr:2024-09-18T10:16:09.578+0000 7f75a19e7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 22093) UID: 0 2024-09-18T10:16:09.598 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-18T10:16:09.598 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:16:09.607 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_ops_in_flight 2024-09-18T10:16:09.680 INFO:tasks.ceph.osd.8.smithi029.stderr:2024-09-18T10:16:09.677+0000 7f2555c27700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 22192) UID: 0 2024-09-18T10:16:09.704 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-18T10:16:09.705 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:16:09.713 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_blocked_ops 2024-09-18T10:16:09.781 INFO:tasks.ceph.osd.11.smithi189.stderr:2024-09-18T10:16:09.777+0000 7f718b60a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 22400) UID: 0 2024-09-18T10:16:09.813 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-18T10:16:09.813 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:16:09.822 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_historic_ops 2024-09-18T10:16:09.838 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-18T10:16:09.883 INFO:tasks.ceph.osd.11.smithi189.stderr:2024-09-18T10:16:09.877+0000 7f718b60a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 22400) UID: 0 2024-09-18T10:16:09.923 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-18T10:16:09.923 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:16:09.932 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T10:16:09.986 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-18T10:16:09.981+0000 7f14dcc5b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22398) UID: 0 2024-09-18T10:16:10.031 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:16:10.031 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:16:10.040 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T10:16:10.097 INFO:tasks.ceph.osd.15.smithi189.stderr:2024-09-18T10:16:10.093+0000 7fd0e725e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 22401) UID: 0 2024-09-18T10:16:10.133 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:16:10.133 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:16:10.142 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T10:16:10.188 INFO:tasks.ceph.osd.15.smithi189.stderr:2024-09-18T10:16:10.181+0000 7fd0e725e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 22401) UID: 0 2024-09-18T10:16:10.241 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:16:10.241 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:16:10.250 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T10:16:10.289 INFO:tasks.ceph.osd.10.smithi154.stderr:2024-09-18T10:16:10.286+0000 7fb73221a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 22091) UID: 0 2024-09-18T10:16:10.353 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:16:10.353 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:16:10.362 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T10:16:10.393 INFO:tasks.ceph.osd.14.smithi154.stderr:2024-09-18T10:16:10.390+0000 7f75a19e7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 22093) UID: 0 2024-09-18T10:16:10.459 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:16:10.459 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:16:10.468 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T10:16:10.514 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-18T10:16:10.509+0000 7f302f2c3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22187) UID: 0 2024-09-18T10:16:10.571 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:16:10.571 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:16:10.580 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-09-18T10:16:10.598 INFO:tasks.ceph.osd.15.smithi189.stderr:2024-09-18T10:16:10.593+0000 7fd0e725e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 22401) UID: 0 2024-09-18T10:16:10.678 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:16:10.678 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:16:10.687 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-09-18T10:16:10.698 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-18T10:16:10.697+0000 7f14dcc5b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22398) UID: 0 2024-09-18T10:16:10.791 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:16:10.791 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:16:10.800 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-09-18T10:16:10.801 INFO:tasks.ceph.osd.6.smithi154.stderr:2024-09-18T10:16:10.798+0000 7f0abc9e1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22089) UID: 0 2024-09-18T10:16:10.833 INFO:teuthology.orchestra.run.smithi029.stderr:nodeep-scrub is set 2024-09-18T10:16:10.901 INFO:tasks.ceph.osd.3.smithi189.stderr:2024-09-18T10:16:10.897+0000 7f2440fc7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22396) UID: 0 2024-09-18T10:16:10.907 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:16:10.907 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:16:10.917 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_ops_in_flight 2024-09-18T10:16:10.974 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T10:16:11.002 INFO:tasks.ceph.osd.11.smithi189.stderr:2024-09-18T10:16:10.997+0000 7f718b60a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 22400) UID: 0 2024-09-18T10:16:11.012 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:16:11.012 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:16:11.022 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_blocked_ops 2024-09-18T10:16:11.103 INFO:tasks.ceph.osd.6.smithi154.stderr:2024-09-18T10:16:11.098+0000 7f0abc9e1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22089) UID: 0 2024-09-18T10:16:11.117 INFO:teuthology.orchestra.run.smithi029.stdout:osd.0: {} 2024-09-18T10:16:11.142 INFO:teuthology.orchestra.run.smithi029.stdout:osd.1: {} 2024-09-18T10:16:11.164 INFO:teuthology.orchestra.run.smithi029.stdout:osd.2: {} 2024-09-18T10:16:11.166 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:16:11.166 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:16:11.177 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_historic_ops 2024-09-18T10:16:11.194 INFO:teuthology.orchestra.run.smithi029.stdout:osd.3: {} 2024-09-18T10:16:11.203 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-18T10:16:11.197+0000 7f302f2c3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22187) UID: 0 2024-09-18T10:16:11.216 INFO:teuthology.orchestra.run.smithi029.stdout:osd.4: {} 2024-09-18T10:16:11.237 INFO:teuthology.orchestra.run.smithi029.stdout:osd.5: {} 2024-09-18T10:16:11.259 INFO:teuthology.orchestra.run.smithi029.stdout:osd.6: {} 2024-09-18T10:16:11.275 INFO:teuthology.orchestra.run.smithi029.stdout:ERROR: (22) Invalid argument 2024-09-18T10:16:11.275 INFO:teuthology.orchestra.run.smithi029.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T10:16:11.279 INFO:teuthology.orchestra.run.smithi029.stdout:osd.7: {} 2024-09-18T10:16:11.285 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T10:16:11.299 INFO:teuthology.orchestra.run.smithi029.stdout:osd.8: {} 2024-09-18T10:16:11.303 INFO:tasks.ceph.osd.12.smithi029.stderr:2024-09-18T10:16:11.297+0000 7f8e4e627700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 22190) UID: 0 2024-09-18T10:16:11.321 INFO:teuthology.orchestra.run.smithi029.stdout:osd.9: {} 2024-09-18T10:16:11.343 INFO:teuthology.orchestra.run.smithi029.stdout:osd.10: {} 2024-09-18T10:16:11.365 INFO:teuthology.orchestra.run.smithi029.stdout:osd.11: {} 2024-09-18T10:16:11.385 INFO:teuthology.orchestra.run.smithi029.stdout:osd.12: {} 2024-09-18T10:16:11.395 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-18T10:16:11.395 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-18T10:16:11.395 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.395 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpRead(3.1as1 114/111 ECSubRead(tid=229, to_read={3:5c566071:::smithi18925896-28:13=0,262144,0}, subchunks={3:5c566071:::smithi18925896-28:13=[0,1]}, attrs_to_read=3:5c566071:::smithi18925896-28:13))", 2024-09-18T10:16:11.396 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:16:11.387178+0000", 2024-09-18T10:16:11.396 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.0076579650000000001, 2024-09-18T10:16:11.396 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0077032639999999996, 2024-09-18T10:16:11.396 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:11.396 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "no flag points reached", 2024-09-18T10:16:11.396 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:11.396 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.396 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:11.397 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:16:11.387178+0000", 2024-09-18T10:16:11.397 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:11.397 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.397 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.397 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:11.397 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:16:11.387178+0000", 2024-09-18T10:16:11.397 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:11.397 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.397 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.397 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:11.397 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:16:11.387176+0000", 2024-09-18T10:16:11.397 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999981 2024-09-18T10:16:11.397 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.398 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.398 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:11.398 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:16:11.387197+0000", 2024-09-18T10:16:11.398 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1189999999999999e-05 2024-09-18T10:16:11.398 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.398 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.398 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:11.398 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:16:11.387248+0000", 2024-09-18T10:16:11.398 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1208000000000002e-05 2024-09-18T10:16:11.398 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:11.398 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:11.398 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:11.398 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:11.398 INFO:teuthology.orchestra.run.smithi136.stdout: ], 2024-09-18T10:16:11.399 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 1 2024-09-18T10:16:11.399 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-18T10:16:11.404 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-18T10:16:11.399+0000 7f00046a8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 22136) UID: 0 2024-09-18T10:16:11.405 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T10:16:11.409 INFO:teuthology.orchestra.run.smithi029.stdout:osd.13: {} 2024-09-18T10:16:11.432 INFO:teuthology.orchestra.run.smithi029.stdout:osd.14: {} 2024-09-18T10:16:11.454 INFO:teuthology.orchestra.run.smithi029.stdout:osd.15: {} 2024-09-18T10:16:11.457 INFO:teuthology.orchestra.run.smithi029.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T10:16:11.457 INFO:teuthology.orchestra.run.smithi029.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T10:16:11.457 INFO:teuthology.orchestra.run.smithi029.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T10:16:11.457 INFO:teuthology.orchestra.run.smithi029.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T10:16:11.457 INFO:teuthology.orchestra.run.smithi029.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T10:16:11.457 INFO:teuthology.orchestra.run.smithi029.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T10:16:11.457 INFO:teuthology.orchestra.run.smithi029.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T10:16:11.457 INFO:teuthology.orchestra.run.smithi029.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T10:16:11.458 INFO:teuthology.orchestra.run.smithi029.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-18T10:16:11.458 INFO:teuthology.orchestra.run.smithi029.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-18T10:16:11.458 INFO:teuthology.orchestra.run.smithi029.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-18T10:16:11.458 INFO:teuthology.orchestra.run.smithi029.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-18T10:16:11.458 INFO:teuthology.orchestra.run.smithi029.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-18T10:16:11.458 INFO:teuthology.orchestra.run.smithi029.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-18T10:16:11.458 INFO:teuthology.orchestra.run.smithi029.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-18T10:16:11.458 INFO:teuthology.orchestra.run.smithi029.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-18T10:16:11.505 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-18T10:16:11.499+0000 7f14c6a1a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 22138) UID: 0 2024-09-18T10:16:11.507 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-18T10:16:11.507 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-18T10:16:11.507 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-18T10:16:11.507 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-18T10:16:11.507 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-18T10:16:11.516 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T10:16:11.605 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-18T10:16:11.599+0000 7f1981873700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22132) UID: 0 2024-09-18T10:16:11.664 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-18T10:16:11.664 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-18T10:16:11.664 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-18T10:16:11.665 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-18T10:16:11.665 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.665 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.bs5 36/32 ECSubWrite(tid=26, reqid=client.4632.0:119, at_version=36'4, trim_to=0'0, roll_forward_to=35'2))", 2024-09-18T10:16:11.665 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:50.812131+0000", 2024-09-18T10:16:11.665 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 80.848510008999995, 2024-09-18T10:16:11.665 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.201739738, 2024-09-18T10:16:11.665 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:11.665 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:16:11.665 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:11.665 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.666 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:11.666 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.812131+0000", 2024-09-18T10:16:11.666 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:11.666 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.666 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.666 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:11.666 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.812131+0000", 2024-09-18T10:16:11.666 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:11.666 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.666 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.666 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:11.666 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.812129+0000", 2024-09-18T10:16:11.666 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999986 2024-09-18T10:16:11.666 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.667 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.667 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:11.667 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.812166+0000", 2024-09-18T10:16:11.667 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6309000000000001e-05 2024-09-18T10:16:11.667 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.667 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.667 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:11.667 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.812199+0000", 2024-09-18T10:16:11.667 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3219000000000002e-05 2024-09-18T10:16:11.667 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.667 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.667 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:11.667 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:50.812225+0000", 2024-09-18T10:16:11.668 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6302999999999999e-05 2024-09-18T10:16:11.668 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.668 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.668 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:11.668 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.012537+0000", 2024-09-18T10:16:11.668 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20031156899999999 2024-09-18T10:16:11.668 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.668 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.668 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:16:11.668 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.012564+0000", 2024-09-18T10:16:11.668 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7165000000000002e-05 2024-09-18T10:16:11.668 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.668 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.668 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:16:11.669 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.013814+0000", 2024-09-18T10:16:11.669 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001249836 2024-09-18T10:16:11.669 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.669 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.669 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:11.669 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.013871+0000", 2024-09-18T10:16:11.669 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6894000000000001e-05 2024-09-18T10:16:11.669 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:11.669 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:11.669 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:11.669 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.669 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.669 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.2s3 41/40 ECSubWrite(tid=23, reqid=client.4632.0:265, at_version=41'11, trim_to=0'0, roll_forward_to=0'0))", 2024-09-18T10:16:11.670 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:57.236544+0000", 2024-09-18T10:16:11.670 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 74.424096757000001, 2024-09-18T10:16:11.670 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20209979, 2024-09-18T10:16:11.670 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:11.670 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:16:11.670 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:11.670 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.670 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:11.670 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.236544+0000", 2024-09-18T10:16:11.670 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:11.670 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.670 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.670 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:11.671 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.236544+0000", 2024-09-18T10:16:11.671 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:11.671 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.671 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.671 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:11.671 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.236543+0000", 2024-09-18T10:16:11.671 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999986 2024-09-18T10:16:11.671 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.671 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.671 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:11.671 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.236820+0000", 2024-09-18T10:16:11.671 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000277068 2024-09-18T10:16:11.671 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.672 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.672 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:11.672 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.236876+0000", 2024-09-18T10:16:11.672 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6555000000000003e-05 2024-09-18T10:16:11.672 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.672 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.672 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:11.672 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.336982+0000", 2024-09-18T10:16:11.672 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100105679 2024-09-18T10:16:11.672 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.672 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.672 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:11.672 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.337122+0000", 2024-09-18T10:16:11.673 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000139777 2024-09-18T10:16:11.673 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.673 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.673 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:16:11.673 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.337135+0000", 2024-09-18T10:16:11.673 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3559e-05 2024-09-18T10:16:11.673 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.673 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.673 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:16:11.673 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.438600+0000", 2024-09-18T10:16:11.674 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.101464572 2024-09-18T10:16:11.674 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.674 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.674 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:11.674 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.438644+0000", 2024-09-18T10:16:11.674 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4085000000000002e-05 2024-09-18T10:16:11.674 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:11.675 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:11.675 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:11.675 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.675 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.675 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.2s3 41/40 ECSubWrite(tid=25, reqid=client.4632.0:266, at_version=0'0, trim_to=0'0, roll_forward_to=41'12))", 2024-09-18T10:16:11.675 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:57.543345+0000", 2024-09-18T10:16:11.675 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 74.117296347000007, 2024-09-18T10:16:11.675 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.18858536100000001, 2024-09-18T10:16:11.675 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:11.675 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:16:11.675 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:11.675 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.676 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:11.676 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.543345+0000", 2024-09-18T10:16:11.676 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:11.676 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.676 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.676 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:11.676 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.543345+0000", 2024-09-18T10:16:11.676 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:11.676 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.676 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.676 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:11.676 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.543343+0000", 2024-09-18T10:16:11.676 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999986 2024-09-18T10:16:11.677 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.677 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.677 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:11.677 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.543385+0000", 2024-09-18T10:16:11.677 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2487999999999997e-05 2024-09-18T10:16:11.677 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.677 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.677 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:11.677 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.543428+0000", 2024-09-18T10:16:11.677 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2315999999999999e-05 2024-09-18T10:16:11.677 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.678 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.678 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:11.678 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.543446+0000", 2024-09-18T10:16:11.678 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8189000000000002e-05 2024-09-18T10:16:11.678 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.678 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.678 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:11.678 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.730560+0000", 2024-09-18T10:16:11.678 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.187114473 2024-09-18T10:16:11.678 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.678 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.678 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:16:11.678 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.730585+0000", 2024-09-18T10:16:11.679 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4652e-05 2024-09-18T10:16:11.679 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.679 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.679 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:16:11.679 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.731859+0000", 2024-09-18T10:16:11.679 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001274425 2024-09-18T10:16:11.679 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.679 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.679 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:11.679 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.731930+0000", 2024-09-18T10:16:11.679 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0474000000000003e-05 2024-09-18T10:16:11.679 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:11.679 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:11.679 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:11.680 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.680 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.680 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.as1 57/32 ECSubWrite(tid=68, reqid=osd.13.0:68, at_version=57'16, trim_to=0'0, roll_forward_to=56'14))", 2024-09-18T10:16:11.680 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:12.949505+0000", 2024-09-18T10:16:11.680 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 58.711136160000002, 2024-09-18T10:16:11.680 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.29432398300000001, 2024-09-18T10:16:11.680 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:11.680 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:16:11.680 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:11.680 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.680 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:11.680 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:12.949505+0000", 2024-09-18T10:16:11.680 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:11.681 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.681 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.681 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:11.681 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:12.949505+0000", 2024-09-18T10:16:11.681 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:11.681 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.681 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.681 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:11.681 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:12.949503+0000", 2024-09-18T10:16:11.681 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999981 2024-09-18T10:16:11.681 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.681 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.681 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:11.682 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:12.949684+0000", 2024-09-18T10:16:11.682 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00018156800000000001 2024-09-18T10:16:11.682 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.682 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.682 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:11.682 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:12.950018+0000", 2024-09-18T10:16:11.682 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00033348699999999999 2024-09-18T10:16:11.682 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.682 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.682 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:11.682 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:12.950036+0000", 2024-09-18T10:16:11.682 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7669e-05 2024-09-18T10:16:11.682 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.682 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.683 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:11.683 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:13.143164+0000", 2024-09-18T10:16:11.683 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.19312819000000001 2024-09-18T10:16:11.683 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.683 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.683 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:16:11.683 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:13.143178+0000", 2024-09-18T10:16:11.683 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3974999999999999e-05 2024-09-18T10:16:11.683 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.683 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.683 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:16:11.683 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:13.243785+0000", 2024-09-18T10:16:11.683 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10060730499999999 2024-09-18T10:16:11.683 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.684 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.684 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:11.684 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:13.243829+0000", 2024-09-18T10:16:11.684 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3634999999999998e-05 2024-09-18T10:16:11.684 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:11.684 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:11.684 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:11.684 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.684 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.684 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.2s3 57/40 ECSubWrite(tid=43, reqid=osd.14.0:42, at_version=0'0, trim_to=0'0, roll_forward_to=57'26))", 2024-09-18T10:16:11.684 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:13.057362+0000", 2024-09-18T10:16:11.684 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 58.603279278999999, 2024-09-18T10:16:11.684 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.286917694, 2024-09-18T10:16:11.684 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:11.685 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:16:11.685 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:11.685 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.685 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:11.685 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:13.057362+0000", 2024-09-18T10:16:11.685 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:11.685 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.685 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.685 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:11.685 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:13.057362+0000", 2024-09-18T10:16:11.686 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:11.686 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.686 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.686 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:11.686 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:13.057359+0000", 2024-09-18T10:16:11.686 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999976 2024-09-18T10:16:11.686 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.686 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.686 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:11.686 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:13.057392+0000", 2024-09-18T10:16:11.687 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2836999999999997e-05 2024-09-18T10:16:11.687 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.687 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.688 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:11.688 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:13.057432+0000", 2024-09-18T10:16:11.688 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0195999999999999e-05 2024-09-18T10:16:11.688 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.688 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.688 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:11.688 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:13.057450+0000", 2024-09-18T10:16:11.688 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8156e-05 2024-09-18T10:16:11.688 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.688 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.688 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:11.688 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:13.243225+0000", 2024-09-18T10:16:11.689 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.18577438499999999 2024-09-18T10:16:11.689 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.689 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.689 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:16:11.689 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:13.243241+0000", 2024-09-18T10:16:11.689 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.643e-05 2024-09-18T10:16:11.689 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.689 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.689 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:16:11.689 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:13.344203+0000", 2024-09-18T10:16:11.689 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100961541 2024-09-18T10:16:11.689 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.689 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.690 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:11.690 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:13.344279+0000", 2024-09-18T10:16:11.690 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6487000000000001e-05 2024-09-18T10:16:11.690 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:11.690 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:11.690 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:11.690 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.690 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.690 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.2s3 58/40 ECSubWrite(tid=44, reqid=client.4632.0:452, at_version=58'28, trim_to=0'0, roll_forward_to=57'26))", 2024-09-18T10:16:11.690 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:13.563698+0000", 2024-09-18T10:16:11.690 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 58.096942738000003, 2024-09-18T10:16:11.690 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20210718, 2024-09-18T10:16:11.690 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:11.690 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:16:11.691 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:11.691 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.691 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:11.691 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:13.563698+0000", 2024-09-18T10:16:11.691 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:11.691 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.691 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.691 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:11.691 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:13.563698+0000", 2024-09-18T10:16:11.691 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:11.691 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.691 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.691 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:11.692 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:13.563697+0000", 2024-09-18T10:16:11.692 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999995 2024-09-18T10:16:11.692 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.692 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.692 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:11.692 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:13.563729+0000", 2024-09-18T10:16:11.692 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1358e-05 2024-09-18T10:16:11.692 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.692 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.692 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:11.692 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:13.563740+0000", 2024-09-18T10:16:11.692 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.083e-05 2024-09-18T10:16:11.692 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.693 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.693 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:11.693 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:13.563747+0000", 2024-09-18T10:16:11.693 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6520000000000006e-06 2024-09-18T10:16:11.693 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.693 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.693 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:11.693 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:13.762333+0000", 2024-09-18T10:16:11.693 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.19858611700000001 2024-09-18T10:16:11.693 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.693 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.693 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:16:11.694 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:13.762350+0000", 2024-09-18T10:16:11.694 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6796e-05 2024-09-18T10:16:11.694 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.694 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.694 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:16:11.694 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:13.765695+0000", 2024-09-18T10:16:11.694 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0033452540000000002 2024-09-18T10:16:11.694 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.694 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.694 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:11.694 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:13.765805+0000", 2024-09-18T10:16:11.694 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000109843 2024-09-18T10:16:11.694 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:11.694 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:11.695 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:11.695 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.695 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.695 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDScrubReserve(3.18s5 REQUEST e61)", 2024-09-18T10:16:11.695 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:17.424550+0000", 2024-09-18T10:16:11.695 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 54.236091201000001, 2024-09-18T10:16:11.695 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.200564715, 2024-09-18T10:16:11.695 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:11.695 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-18T10:16:11.695 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:11.695 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.695 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:11.695 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.424550+0000", 2024-09-18T10:16:11.696 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:11.696 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.696 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.696 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:11.696 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.424550+0000", 2024-09-18T10:16:11.696 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:11.696 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.696 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.696 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:11.696 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.424548+0000", 2024-09-18T10:16:11.696 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999981 2024-09-18T10:16:11.696 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.696 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.697 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:11.697 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.424560+0000", 2024-09-18T10:16:11.697 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2859e-05 2024-09-18T10:16:11.697 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.697 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.697 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:11.697 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.424595+0000", 2024-09-18T10:16:11.697 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4576000000000002e-05 2024-09-18T10:16:11.697 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.697 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.697 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:11.697 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.524738+0000", 2024-09-18T10:16:11.697 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100143278 2024-09-18T10:16:11.698 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.698 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.698 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:11.698 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.625032+0000", 2024-09-18T10:16:11.698 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10029397399999999 2024-09-18T10:16:11.698 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.698 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.698 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:16:11.698 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.625064+0000", 2024-09-18T10:16:11.698 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1848999999999999e-05 2024-09-18T10:16:11.698 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.698 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.698 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:11.699 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.625114+0000", 2024-09-18T10:16:11.699 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0305e-05 2024-09-18T10:16:11.699 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:11.699 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:11.699 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:11.699 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.699 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.699 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:515 3.13s0 3:c9ce6440:::smithi18925896-24:head [delete] snapc 1a=[1a,19,18,16,12,11,10,f,e] ondisk+write+known_if_redirected+full_force+supports_pool_eio e63)", 2024-09-18T10:16:11.699 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:19.217452+0000", 2024-09-18T10:16:11.699 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 52.443189269999998, 2024-09-18T10:16:11.699 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20492691099999999, 2024-09-18T10:16:11.699 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:11.699 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:11.699 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:16:11.699 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:16:11.700 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:11.700 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 515 2024-09-18T10:16:11.700 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.700 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:11.700 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.700 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:11.700 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.217452+0000", 2024-09-18T10:16:11.700 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:11.700 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.700 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.700 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:11.700 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.217452+0000", 2024-09-18T10:16:11.700 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:11.700 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.701 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.701 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:11.701 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.217448+0000", 2024-09-18T10:16:11.701 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999967 2024-09-18T10:16:11.701 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.701 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.701 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:11.701 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.217464+0000", 2024-09-18T10:16:11.701 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5763000000000001e-05 2024-09-18T10:16:11.701 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.702 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.702 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:11.702 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.217500+0000", 2024-09-18T10:16:11.702 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5536e-05 2024-09-18T10:16:11.702 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.702 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.702 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:11.702 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.217528+0000", 2024-09-18T10:16:11.702 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8609e-05 2024-09-18T10:16:11.702 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.702 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.702 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:11.702 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.217647+0000", 2024-09-18T10:16:11.702 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000118917 2024-09-18T10:16:11.703 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.703 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.703 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:16:11.703 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.217866+0000", 2024-09-18T10:16:11.703 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00021885399999999999 2024-09-18T10:16:11.703 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.703 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.703 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:16:11.703 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.219038+0000", 2024-09-18T10:16:11.703 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001172125 2024-09-18T10:16:11.703 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.703 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.704 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:16:11.704 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.220845+0000", 2024-09-18T10:16:11.704 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001806385 2024-09-18T10:16:11.704 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.704 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.704 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:16:11.704 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.422256+0000", 2024-09-18T10:16:11.704 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20141129799999999 2024-09-18T10:16:11.704 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.704 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.704 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:11.704 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.422379+0000", 2024-09-18T10:16:11.704 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012262699999999999 2024-09-18T10:16:11.704 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:11.705 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:11.705 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:11.705 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.705 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.705 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.bs5 72/32 ECSubWrite(tid=153, reqid=client.4632.0:624, at_version=72'23, trim_to=0'0, roll_forward_to=60'21))", 2024-09-18T10:16:11.705 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:28.105186+0000", 2024-09-18T10:16:11.705 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 43.555454757, 2024-09-18T10:16:11.705 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.38192188100000002, 2024-09-18T10:16:11.705 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:11.705 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:16:11.705 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:11.706 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.706 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:11.706 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:28.105186+0000", 2024-09-18T10:16:11.706 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:11.706 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.706 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.706 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:11.706 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:28.105186+0000", 2024-09-18T10:16:11.706 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:11.706 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.706 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.706 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:11.706 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:28.105186+0000", 2024-09-18T10:16:11.706 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999995 2024-09-18T10:16:11.707 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.707 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.707 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:11.707 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:28.105214+0000", 2024-09-18T10:16:11.707 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7985000000000001e-05 2024-09-18T10:16:11.707 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.707 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.707 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:11.707 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:28.105228+0000", 2024-09-18T10:16:11.707 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3930999999999999e-05 2024-09-18T10:16:11.707 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.707 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.707 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:11.707 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:28.205370+0000", 2024-09-18T10:16:11.708 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10014197800000001 2024-09-18T10:16:11.708 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.708 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.708 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:11.708 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:28.305658+0000", 2024-09-18T10:16:11.708 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.1002884 2024-09-18T10:16:11.708 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.708 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.708 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:16:11.708 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:28.305679+0000", 2024-09-18T10:16:11.708 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0812e-05 2024-09-18T10:16:11.708 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.708 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.709 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:16:11.709 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:28.487015+0000", 2024-09-18T10:16:11.709 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.18133600499999999 2024-09-18T10:16:11.709 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.709 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.709 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:11.709 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:28.487108+0000", 2024-09-18T10:16:11.709 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3011999999999996e-05 2024-09-18T10:16:11.709 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:11.709 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:11.709 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:11.709 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.709 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.709 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.2s3 72/40 ECSubWrite(tid=93, reqid=osd.14.0:92, at_version=0'0, trim_to=0'0, roll_forward_to=72'55))", 2024-09-18T10:16:11.710 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:28.389120+0000", 2024-09-18T10:16:11.710 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 43.271520522000003, 2024-09-18T10:16:11.710 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20149926400000001, 2024-09-18T10:16:11.710 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:11.710 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:16:11.710 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:11.710 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.710 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:11.710 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:28.389120+0000", 2024-09-18T10:16:11.710 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:11.710 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.710 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.710 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:11.710 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:28.389120+0000", 2024-09-18T10:16:11.711 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:11.711 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.711 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.711 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:11.711 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:28.389112+0000", 2024-09-18T10:16:11.711 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999919 2024-09-18T10:16:11.711 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.711 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.711 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:11.711 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:28.389153+0000", 2024-09-18T10:16:11.711 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0138000000000001e-05 2024-09-18T10:16:11.711 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.711 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.712 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:11.712 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:28.389182+0000", 2024-09-18T10:16:11.712 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9113000000000001e-05 2024-09-18T10:16:11.712 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.712 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.712 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:11.712 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:28.489269+0000", 2024-09-18T10:16:11.712 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100087491 2024-09-18T10:16:11.712 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.712 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.712 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:11.712 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:28.589445+0000", 2024-09-18T10:16:11.713 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100175537 2024-09-18T10:16:11.713 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.713 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.713 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:16:11.713 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:28.589468+0000", 2024-09-18T10:16:11.713 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2781000000000001e-05 2024-09-18T10:16:11.713 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.713 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.713 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:16:11.713 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:28.590550+0000", 2024-09-18T10:16:11.713 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001082251 2024-09-18T10:16:11.713 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.713 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.714 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:11.714 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:28.590620+0000", 2024-09-18T10:16:11.714 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9822999999999999e-05 2024-09-18T10:16:11.714 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:11.714 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:11.714 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:11.714 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.714 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.714 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.8s5 74/40 ECSubWrite(tid=202, reqid=osd.6.0:202, at_version=74'27, trim_to=0'0, roll_forward_to=72'23))", 2024-09-18T10:16:11.714 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:29.719110+0000", 2024-09-18T10:16:11.714 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 41.941530966000002, 2024-09-18T10:16:11.714 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.19912816699999999, 2024-09-18T10:16:11.714 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:11.714 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:16:11.715 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:11.715 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.715 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:11.715 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.719110+0000", 2024-09-18T10:16:11.715 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:11.715 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.715 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.715 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:11.715 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.719110+0000", 2024-09-18T10:16:11.715 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:11.715 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.715 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.715 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:11.716 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.719109+0000", 2024-09-18T10:16:11.716 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:16:11.716 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.716 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.716 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:11.716 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.719133+0000", 2024-09-18T10:16:11.716 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4409999999999998e-05 2024-09-18T10:16:11.716 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.716 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.716 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:11.716 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.719151+0000", 2024-09-18T10:16:11.716 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7441000000000001e-05 2024-09-18T10:16:11.716 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.717 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.717 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:11.717 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.719159+0000", 2024-09-18T10:16:11.717 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1370000000000002e-06 2024-09-18T10:16:11.717 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.717 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.717 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:11.717 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.916605+0000", 2024-09-18T10:16:11.717 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.19744605700000001 2024-09-18T10:16:11.717 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.717 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.717 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:16:11.717 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.916621+0000", 2024-09-18T10:16:11.718 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6059e-05 2024-09-18T10:16:11.718 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.718 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.718 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:16:11.718 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.918184+0000", 2024-09-18T10:16:11.718 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001562765 2024-09-18T10:16:11.718 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.718 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.718 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:11.718 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.918238+0000", 2024-09-18T10:16:11.718 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4348999999999998e-05 2024-09-18T10:16:11.718 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:11.718 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:11.718 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:11.719 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.719 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.719 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWriteReply(3.13s0 74/56 ECSubWriteReply(tid=23, last_complete=74'46, committed=1, applied=1))", 2024-09-18T10:16:11.719 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:29.774402+0000", 2024-09-18T10:16:11.719 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 41.886239082000003, 2024-09-18T10:16:11.719 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.19560270900000001, 2024-09-18T10:16:11.719 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:11.719 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:16:11.719 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:11.719 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.719 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:11.719 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.774402+0000", 2024-09-18T10:16:11.720 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:11.720 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.720 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.720 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:11.720 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.774402+0000", 2024-09-18T10:16:11.720 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:11.720 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.720 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.721 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:11.721 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.774401+0000", 2024-09-18T10:16:11.721 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:16:11.721 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.721 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.721 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:11.721 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.774407+0000", 2024-09-18T10:16:11.721 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.173e-06 2024-09-18T10:16:11.721 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.721 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.721 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:11.721 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.774423+0000", 2024-09-18T10:16:11.722 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6568000000000001e-05 2024-09-18T10:16:11.722 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.722 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.722 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:11.722 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.774431+0000", 2024-09-18T10:16:11.722 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4649999999999997e-06 2024-09-18T10:16:11.722 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.722 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.722 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:11.722 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.969971+0000", 2024-09-18T10:16:11.722 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.19554042199999999 2024-09-18T10:16:11.722 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.722 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.723 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:11.723 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.970004+0000", 2024-09-18T10:16:11.723 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3151999999999997e-05 2024-09-18T10:16:11.723 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:11.723 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:11.723 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:11.723 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.723 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.723 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.bs5 74/32 ECSubWrite(tid=163, reqid=client.4632.0:650, at_version=74'27, trim_to=0'0, roll_forward_to=72'23))", 2024-09-18T10:16:11.723 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:30.069481+0000", 2024-09-18T10:16:11.723 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 41.591159623000003, 2024-09-18T10:16:11.723 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30291274699999998, 2024-09-18T10:16:11.723 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:11.723 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:16:11.724 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:11.724 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.724 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:11.724 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:30.069481+0000", 2024-09-18T10:16:11.724 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:11.724 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.724 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.724 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:11.724 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:30.069481+0000", 2024-09-18T10:16:11.724 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:11.724 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.724 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.724 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:11.725 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:30.069479+0000", 2024-09-18T10:16:11.725 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999981 2024-09-18T10:16:11.725 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.725 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.725 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:11.725 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:30.069535+0000", 2024-09-18T10:16:11.725 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5674999999999998e-05 2024-09-18T10:16:11.725 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.725 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.725 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:11.725 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:30.069577+0000", 2024-09-18T10:16:11.726 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1588000000000002e-05 2024-09-18T10:16:11.726 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.726 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.726 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:11.726 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:30.069595+0000", 2024-09-18T10:16:11.726 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8128999999999999e-05 2024-09-18T10:16:11.726 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.726 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.726 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:11.726 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:30.269862+0000", 2024-09-18T10:16:11.726 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20026767000000001 2024-09-18T10:16:11.726 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.727 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.727 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:16:11.727 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:30.269881+0000", 2024-09-18T10:16:11.727 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8402000000000001e-05 2024-09-18T10:16:11.727 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.727 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.727 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:16:11.727 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:30.372285+0000", 2024-09-18T10:16:11.727 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10240445099999999 2024-09-18T10:16:11.727 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.727 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.727 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:11.728 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:30.372394+0000", 2024-09-18T10:16:11.728 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010880200000000001 2024-09-18T10:16:11.728 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:11.728 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:11.728 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:11.728 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.728 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.728 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDPGPush(3.19s2 79/74 [PushOp(3:9a1a774c:::smithi18925896-38:19, version: 71'93, data_included: [0~262144], data_size: 262144, omap_header_size: 0, omap_entries_size: 0, attrset_size: 3, recovery_info: ObjectRecoveryInfo(3:9a1a774c:::smithi18925896-38:19@71'93, size: 1638400, copy_subset: [], clone_subset: {}, snapset: 19=[]:{19=[16,12,11,10]}, object_exist: 1), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:1048576, data_complete:false, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:true, error:false))])", 2024-09-18T10:16:11.728 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:35.501041+0000", 2024-09-18T10:16:11.728 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 36.159599886000002, 2024-09-18T10:16:11.728 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30191898, 2024-09-18T10:16:11.728 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:11.729 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:16:11.729 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:11.729 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.729 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:11.729 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:35.501041+0000", 2024-09-18T10:16:11.729 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:11.729 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.729 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.729 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:11.729 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:35.501041+0000", 2024-09-18T10:16:11.729 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:11.729 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.729 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.729 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:11.729 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:35.501039+0000", 2024-09-18T10:16:11.730 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999981 2024-09-18T10:16:11.730 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.730 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.730 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:11.730 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:35.501446+0000", 2024-09-18T10:16:11.730 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00040681000000000002 2024-09-18T10:16:11.730 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.730 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.730 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:11.730 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:35.501490+0000", 2024-09-18T10:16:11.730 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4162000000000002e-05 2024-09-18T10:16:11.730 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.730 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.730 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:11.730 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:35.601651+0000", 2024-09-18T10:16:11.731 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100160545 2024-09-18T10:16:11.731 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.731 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.731 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:11.731 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:35.801995+0000", 2024-09-18T10:16:11.731 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20034463699999999 2024-09-18T10:16:11.731 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.731 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.731 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:11.731 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:35.802960+0000", 2024-09-18T10:16:11.731 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00096465600000000004 2024-09-18T10:16:11.731 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:11.731 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:11.732 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:11.732 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.732 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.732 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.8s5 85/40 ECSubWrite(tid=238, reqid=client.4632.0:791, at_version=0'0, trim_to=0'0, roll_forward_to=85'37))", 2024-09-18T10:16:11.732 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:41.215795+0000", 2024-09-18T10:16:11.732 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 30.444845472000001, 2024-09-18T10:16:11.732 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20151841100000001, 2024-09-18T10:16:11.732 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:11.732 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:16:11.732 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:11.732 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.732 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:11.732 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:41.215795+0000", 2024-09-18T10:16:11.732 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:11.733 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.733 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.733 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:11.733 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:41.215795+0000", 2024-09-18T10:16:11.733 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:11.733 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.733 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.733 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:11.733 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:41.215794+0000", 2024-09-18T10:16:11.733 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:16:11.733 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.733 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.733 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:11.733 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:41.215816+0000", 2024-09-18T10:16:11.734 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.139e-05 2024-09-18T10:16:11.734 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.734 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.734 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:11.734 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:41.215837+0000", 2024-09-18T10:16:11.734 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1817000000000001e-05 2024-09-18T10:16:11.734 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.734 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.734 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:11.734 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:41.315944+0000", 2024-09-18T10:16:11.734 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100106606 2024-09-18T10:16:11.734 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.734 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.735 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:11.735 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:41.416182+0000", 2024-09-18T10:16:11.735 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100237783 2024-09-18T10:16:11.735 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.735 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.735 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:16:11.735 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:41.416201+0000", 2024-09-18T10:16:11.735 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8836000000000001e-05 2024-09-18T10:16:11.735 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.735 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.735 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:16:11.735 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:41.417233+0000", 2024-09-18T10:16:11.735 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001032023 2024-09-18T10:16:11.736 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.736 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.736 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:11.736 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:41.417314+0000", 2024-09-18T10:16:11.736 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1139999999999999e-05 2024-09-18T10:16:11.736 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:11.736 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:11.736 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:11.736 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.736 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.736 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.bs5 89/32 ECSubWrite(tid=253, reqid=osd.4.0:251, at_version=0'0, trim_to=0'0, roll_forward_to=89'38))", 2024-09-18T10:16:11.736 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:45.306704+0000", 2024-09-18T10:16:11.736 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 26.353936714, 2024-09-18T10:16:11.736 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20066322, 2024-09-18T10:16:11.737 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:11.737 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:16:11.737 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:11.737 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.737 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:11.737 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.306704+0000", 2024-09-18T10:16:11.737 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:11.737 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.737 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.737 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:11.737 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.306704+0000", 2024-09-18T10:16:11.737 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:11.737 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.738 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.738 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:11.738 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.306703+0000", 2024-09-18T10:16:11.738 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999986 2024-09-18T10:16:11.738 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.738 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.738 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:11.738 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.306728+0000", 2024-09-18T10:16:11.739 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5499000000000001e-05 2024-09-18T10:16:11.739 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.739 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.739 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:11.739 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.306760+0000", 2024-09-18T10:16:11.739 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2194999999999999e-05 2024-09-18T10:16:11.739 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.739 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.739 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:11.739 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.306773+0000", 2024-09-18T10:16:11.739 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.287e-05 2024-09-18T10:16:11.739 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.739 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.739 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:11.740 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.308165+0000", 2024-09-18T10:16:11.740 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001391883 2024-09-18T10:16:11.740 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.740 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.740 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:16:11.740 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.308182+0000", 2024-09-18T10:16:11.740 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7076e-05 2024-09-18T10:16:11.740 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.740 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.740 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:16:11.740 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.507275+0000", 2024-09-18T10:16:11.740 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.199093296 2024-09-18T10:16:11.740 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.740 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.741 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:11.741 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.507367+0000", 2024-09-18T10:16:11.741 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1996999999999993e-05 2024-09-18T10:16:11.741 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:11.741 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:11.741 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:11.741 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.741 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.741 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.2s3 94/40 ECSubWrite(tid=166, reqid=client.4632.0:940, at_version=0'0, trim_to=0'0, roll_forward_to=94'113))", 2024-09-18T10:16:11.741 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:50.313166+0000", 2024-09-18T10:16:11.741 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 21.347475295999999, 2024-09-18T10:16:11.741 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20138013199999999, 2024-09-18T10:16:11.741 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:11.741 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:16:11.742 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:11.742 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.742 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:11.742 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:50.313166+0000", 2024-09-18T10:16:11.742 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:11.742 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.742 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.742 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:11.742 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:50.313166+0000", 2024-09-18T10:16:11.742 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:11.742 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.742 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.742 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:11.743 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:50.313164+0000", 2024-09-18T10:16:11.743 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999981 2024-09-18T10:16:11.743 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.743 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.743 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:11.743 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:50.313190+0000", 2024-09-18T10:16:11.743 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6236000000000001e-05 2024-09-18T10:16:11.743 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.743 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.743 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:11.743 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:50.313222+0000", 2024-09-18T10:16:11.743 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2215000000000002e-05 2024-09-18T10:16:11.744 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.744 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.744 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:11.744 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:50.413362+0000", 2024-09-18T10:16:11.744 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100139597 2024-09-18T10:16:11.744 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.744 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.744 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:11.744 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:50.413520+0000", 2024-09-18T10:16:11.744 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015814799999999999 2024-09-18T10:16:11.744 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.744 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.744 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:16:11.745 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:50.413544+0000", 2024-09-18T10:16:11.745 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4236e-05 2024-09-18T10:16:11.745 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.745 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.745 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:16:11.745 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:50.514518+0000", 2024-09-18T10:16:11.745 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10097379099999999 2024-09-18T10:16:11.745 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.745 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.745 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:11.745 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:50.514546+0000", 2024-09-18T10:16:11.745 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7878999999999999e-05 2024-09-18T10:16:11.745 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:11.745 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:11.746 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:11.746 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.746 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.746 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpRead(3.10s4 100/94 ECSubRead(tid=280, to_read={3:0a83d6b3:::smithi18925896-20:head=524288,262144,0}, subchunks={3:0a83d6b3:::smithi18925896-20:head=[0,1]}, attrs_to_read=))", 2024-09-18T10:16:11.746 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:55.963056+0000", 2024-09-18T10:16:11.746 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 15.697584446, 2024-09-18T10:16:11.746 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.201442542, 2024-09-18T10:16:11.746 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:11.746 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:16:11.746 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:11.746 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.746 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:11.746 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:55.963056+0000", 2024-09-18T10:16:11.747 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:11.747 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.747 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.747 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:11.747 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:55.963056+0000", 2024-09-18T10:16:11.747 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:11.747 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.747 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.747 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:11.747 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:55.963055+0000", 2024-09-18T10:16:11.747 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999986 2024-09-18T10:16:11.747 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.747 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.748 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:11.748 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:55.963075+0000", 2024-09-18T10:16:11.748 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0015999999999999e-05 2024-09-18T10:16:11.748 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.748 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.748 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:11.748 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:55.963112+0000", 2024-09-18T10:16:11.748 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7536000000000001e-05 2024-09-18T10:16:11.748 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.748 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.748 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:11.748 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:56.063280+0000", 2024-09-18T10:16:11.748 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100168034 2024-09-18T10:16:11.749 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.749 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.749 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:11.749 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:56.163574+0000", 2024-09-18T10:16:11.749 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100294024 2024-09-18T10:16:11.749 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.749 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.749 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:11.749 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:56.164499+0000", 2024-09-18T10:16:11.749 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00092454899999999999 2024-09-18T10:16:11.749 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:11.749 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:11.749 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:11.749 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.750 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.750 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.ds1 100/94 ECSubWrite(tid=241, reqid=osd.13.0:239, at_version=0'0, trim_to=0'0, roll_forward_to=100'43))", 2024-09-18T10:16:11.750 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:56.136834+0000", 2024-09-18T10:16:11.750 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 15.523806472, 2024-09-18T10:16:11.750 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20084928199999999, 2024-09-18T10:16:11.750 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:11.750 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:16:11.750 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:11.750 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.750 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:11.750 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:56.136834+0000", 2024-09-18T10:16:11.750 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:11.750 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.750 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.751 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:11.751 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:56.136834+0000", 2024-09-18T10:16:11.751 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:11.751 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.751 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.751 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:11.751 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:56.136832+0000", 2024-09-18T10:16:11.751 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999981 2024-09-18T10:16:11.751 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.751 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.751 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:11.751 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:56.136867+0000", 2024-09-18T10:16:11.751 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4876000000000003e-05 2024-09-18T10:16:11.751 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.752 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.752 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:11.752 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:56.136914+0000", 2024-09-18T10:16:11.752 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6891999999999998e-05 2024-09-18T10:16:11.752 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.752 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.752 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:11.752 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:56.237018+0000", 2024-09-18T10:16:11.752 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100103909 2024-09-18T10:16:11.752 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.752 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.752 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:11.752 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:56.237297+0000", 2024-09-18T10:16:11.753 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000278941 2024-09-18T10:16:11.753 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.753 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.753 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:16:11.753 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:56.237333+0000", 2024-09-18T10:16:11.753 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5716000000000002e-05 2024-09-18T10:16:11.753 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.753 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.753 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:16:11.753 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:56.337620+0000", 2024-09-18T10:16:11.753 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100287853 2024-09-18T10:16:11.753 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.753 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.753 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:11.754 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:56.337684+0000", 2024-09-18T10:16:11.754 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3212000000000004e-05 2024-09-18T10:16:11.754 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:11.754 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:11.754 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:11.754 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.754 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.754 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpRead(3.1as1 114/111 ECSubRead(tid=229, to_read={3:5c566071:::smithi18925896-28:13=0,262144,0}, subchunks={3:5c566071:::smithi18925896-28:13=[0,1]}, attrs_to_read=3:5c566071:::smithi18925896-28:13))", 2024-09-18T10:16:11.754 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:16:11.387178+0000", 2024-09-18T10:16:11.754 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.27346309800000002, 2024-09-18T10:16:11.754 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20104681599999999, 2024-09-18T10:16:11.754 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:11.754 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:16:11.755 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:11.755 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.755 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:11.755 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:16:11.387178+0000", 2024-09-18T10:16:11.755 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:11.755 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.755 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.755 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:11.755 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:16:11.387178+0000", 2024-09-18T10:16:11.755 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:11.755 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.755 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.755 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:11.756 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:16:11.387176+0000", 2024-09-18T10:16:11.756 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999981 2024-09-18T10:16:11.756 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.756 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.756 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:11.756 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:16:11.387197+0000", 2024-09-18T10:16:11.756 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1189999999999999e-05 2024-09-18T10:16:11.756 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.756 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.756 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:11.756 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:16:11.387248+0000", 2024-09-18T10:16:11.756 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1208000000000002e-05 2024-09-18T10:16:11.756 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.757 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.757 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:11.757 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:16:11.487396+0000", 2024-09-18T10:16:11.757 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100148052 2024-09-18T10:16:11.757 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.757 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.757 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:11.757 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:16:11.587653+0000", 2024-09-18T10:16:11.757 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100256604 2024-09-18T10:16:11.757 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:11.757 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:11.757 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:11.757 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:16:11.588225+0000", 2024-09-18T10:16:11.758 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000571785 2024-09-18T10:16:11.758 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:11.758 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:11.758 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:11.758 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:11.758 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:11.758 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-18T10:16:11.759 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T10:16:11.760 INFO:tasks.ceph.osd.14.smithi154.stderr:2024-09-18T10:16:11.758+0000 7f75a19e7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 22093) UID: 0 2024-09-18T10:16:11.761 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T10:16:11.821 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7, 9, 10, 11, 12, 13, 14, 15, 8] out_osds: [] dead_osds: [] live_osds: [0, 4, 8, 12, 1, 5, 9, 13, 2, 6, 10, 14, 3, 7, 11, 15] 2024-09-18T10:16:11.821 INFO:tasks.thrashosds.thrasher:choose_action: min_in 8 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T10:16:11.824 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-18T10:16:11.824 INFO:tasks.thrashosds.thrasher:Setting osd 0 primary_affinity to 1.000000 2024-09-18T10:16:11.825 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 0 1 2024-09-18T10:16:11.860 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-18T10:16:11.853+0000 7f302f2c3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22187) UID: 0 2024-09-18T10:16:11.868 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-18T10:16:11.868 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-18T10:16:11.868 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-09-18T10:16:11.868 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-18T10:16:11.877 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T10:16:11.961 INFO:tasks.ceph.osd.15.smithi189.stderr:2024-09-18T10:16:11.957+0000 7fd0e725e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 22401) UID: 0 2024-09-18T10:16:11.974 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-18T10:16:11.974 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-18T10:16:11.974 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-18T10:16:11.974 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-18T10:16:11.974 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-18T10:16:11.983 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T10:16:12.062 INFO:tasks.ceph.osd.15.smithi189.stderr:2024-09-18T10:16:12.057+0000 7fd0e725e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 22401) UID: 0 2024-09-18T10:16:12.079 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-18T10:16:12.079 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-18T10:16:12.080 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-18T10:16:12.080 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-18T10:16:12.080 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.080 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:150 3.cs0 3:3972e5c3:::smithi18925896-50:head [append 0~458752,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:16:12.080 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:51.125225+0000", 2024-09-18T10:16:12.080 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 80.949550739000003, 2024-09-18T10:16:12.080 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.40871674099999999, 2024-09-18T10:16:12.080 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:12.080 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:12.080 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:16:12.080 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:16:12.080 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:12.081 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 150 2024-09-18T10:16:12.081 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.081 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:12.081 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.081 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:12.081 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.125225+0000", 2024-09-18T10:16:12.081 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:12.081 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.081 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.081 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:12.081 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.125225+0000", 2024-09-18T10:16:12.081 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:12.082 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.082 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.082 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:12.082 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.125222+0000", 2024-09-18T10:16:12.082 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999967 2024-09-18T10:16:12.082 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.082 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.082 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:12.082 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.126038+0000", 2024-09-18T10:16:12.082 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00081693399999999995 2024-09-18T10:16:12.082 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.082 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.082 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:12.082 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.126072+0000", 2024-09-18T10:16:12.083 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4038999999999997e-05 2024-09-18T10:16:12.083 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.083 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.083 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:12.083 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.126084+0000", 2024-09-18T10:16:12.083 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1908e-05 2024-09-18T10:16:12.083 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.083 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.083 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:12.083 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.126184+0000", 2024-09-18T10:16:12.083 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9868e-05 2024-09-18T10:16:12.083 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.083 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.084 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:16:12.084 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.126439+0000", 2024-09-18T10:16:12.084 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00025457799999999999 2024-09-18T10:16:12.084 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.084 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.084 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:16:12.084 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.128215+0000", 2024-09-18T10:16:12.084 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0017764499999999999 2024-09-18T10:16:12.084 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.084 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.084 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:16:12.084 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.232589+0000", 2024-09-18T10:16:12.084 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.104374015 2024-09-18T10:16:12.084 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.085 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.085 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:16:12.085 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.533834+0000", 2024-09-18T10:16:12.085 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30124498 2024-09-18T10:16:12.085 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.085 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.085 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:12.085 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.533941+0000", 2024-09-18T10:16:12.085 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010718299999999999 2024-09-18T10:16:12.085 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:12.085 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:12.085 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:12.086 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.086 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.086 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:152 3.cs0 3:3972e5c3:::smithi18925896-50:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:16:12.086 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:51.128123+0000", 2024-09-18T10:16:12.086 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 80.946652067000002, 2024-09-18T10:16:12.086 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.80287000200000003, 2024-09-18T10:16:12.086 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:12.086 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:12.086 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:16:12.086 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:16:12.086 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:12.086 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 152 2024-09-18T10:16:12.086 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.086 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:12.087 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.087 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:12.087 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.128123+0000", 2024-09-18T10:16:12.087 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:12.087 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.087 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.087 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:12.087 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.128123+0000", 2024-09-18T10:16:12.087 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:12.087 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.087 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.087 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:12.087 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.128122+0000", 2024-09-18T10:16:12.087 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999986 2024-09-18T10:16:12.088 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.088 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.088 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:12.088 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.128129+0000", 2024-09-18T10:16:12.088 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6730000000000003e-06 2024-09-18T10:16:12.088 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.088 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.088 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:12.088 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.128156+0000", 2024-09-18T10:16:12.088 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7489000000000001e-05 2024-09-18T10:16:12.088 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.088 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.088 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:12.089 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.128173+0000", 2024-09-18T10:16:12.089 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7102999999999999e-05 2024-09-18T10:16:12.089 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.089 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.089 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:12.089 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.535139+0000", 2024-09-18T10:16:12.089 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.40696592599999998 2024-09-18T10:16:12.089 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.089 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.089 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:16:12.089 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.535380+0000", 2024-09-18T10:16:12.089 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00024023600000000001 2024-09-18T10:16:12.089 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.090 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.090 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:16:12.090 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.536784+0000", 2024-09-18T10:16:12.090 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0014046950000000001 2024-09-18T10:16:12.090 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.090 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.090 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:16:12.090 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.537964+0000", 2024-09-18T10:16:12.090 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0011793050000000001 2024-09-18T10:16:12.090 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.090 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.090 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:16:12.090 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.930841+0000", 2024-09-18T10:16:12.090 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.39287700599999997 2024-09-18T10:16:12.091 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.091 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.091 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:12.091 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.930993+0000", 2024-09-18T10:16:12.091 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015280799999999999 2024-09-18T10:16:12.091 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:12.091 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:12.091 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:12.091 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.091 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.091 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:153 3.cs0 3:3972e5c3:::smithi18925896-50:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:16:12.091 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:51.128181+0000", 2024-09-18T10:16:12.091 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 80.946594172000005, 2024-09-18T10:16:12.091 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2060332149999999, 2024-09-18T10:16:12.092 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:12.092 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-18T10:16:12.092 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:16:12.092 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:16:12.092 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:12.092 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 153 2024-09-18T10:16:12.092 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.092 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:12.092 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.092 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:12.092 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.128181+0000", 2024-09-18T10:16:12.092 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:12.092 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.092 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.093 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:12.093 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.128181+0000", 2024-09-18T10:16:12.093 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:12.093 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.093 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.093 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:12.093 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.128180+0000", 2024-09-18T10:16:12.093 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:16:12.093 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.093 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.093 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:12.093 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.128186+0000", 2024-09-18T10:16:12.093 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.31e-06 2024-09-18T10:16:12.093 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.094 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.094 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:12.094 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.128195+0000", 2024-09-18T10:16:12.094 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5540000000000003e-06 2024-09-18T10:16:12.094 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.094 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.094 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:12.094 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.128201+0000", 2024-09-18T10:16:12.094 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2129999999999996e-06 2024-09-18T10:16:12.094 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.094 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.094 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:12.094 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.537382+0000", 2024-09-18T10:16:12.095 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.40918055399999997 2024-09-18T10:16:12.095 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.095 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.095 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2024-09-18T10:16:12.095 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.537571+0000", 2024-09-18T10:16:12.095 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00018903299999999999 2024-09-18T10:16:12.095 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.095 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.095 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:12.095 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.931789+0000", 2024-09-18T10:16:12.095 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.39421780499999998 2024-09-18T10:16:12.095 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.095 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.095 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:16:12.096 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.932023+0000", 2024-09-18T10:16:12.096 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000233868 2024-09-18T10:16:12.096 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.096 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.096 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:12.096 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.334215+0000", 2024-09-18T10:16:12.096 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.40219177499999997 2024-09-18T10:16:12.096 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:12.096 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:12.096 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:12.096 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.096 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.096 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:163 3.cs0 3:34d26c88:::smithi18925896-36:head [read 0~589824,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:16:12.096 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:51.532633+0000", 2024-09-18T10:16:12.097 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 80.542142584999993, 2024-09-18T10:16:12.097 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.70024756899999996, 2024-09-18T10:16:12.097 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:12.097 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-18T10:16:12.097 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:16:12.097 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:16:12.097 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:12.097 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 163 2024-09-18T10:16:12.097 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.097 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:12.097 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.097 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:12.097 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.532633+0000", 2024-09-18T10:16:12.098 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:12.098 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.098 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.098 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:12.098 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.532633+0000", 2024-09-18T10:16:12.098 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:12.098 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.098 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.098 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:12.098 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.532632+0000", 2024-09-18T10:16:12.098 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:16:12.098 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.098 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.098 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:12.099 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.532636+0000", 2024-09-18T10:16:12.099 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3529999999999997e-06 2024-09-18T10:16:12.099 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.099 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.099 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:12.099 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.532645+0000", 2024-09-18T10:16:12.099 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7099999999999996e-06 2024-09-18T10:16:12.099 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.099 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.099 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:12.099 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.532650+0000", 2024-09-18T10:16:12.099 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3029999999999999e-06 2024-09-18T10:16:12.099 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.099 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.100 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:12.100 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.930003+0000", 2024-09-18T10:16:12.100 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.397352333 2024-09-18T10:16:12.100 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.100 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.100 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:16:12.100 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:51.930174+0000", 2024-09-18T10:16:12.100 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017110000000000001 2024-09-18T10:16:12.100 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.100 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.100 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:12.100 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.232880+0000", 2024-09-18T10:16:12.100 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30270652300000001 2024-09-18T10:16:12.101 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:12.101 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:12.101 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:12.101 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.101 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.101 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:181 3.cs0 3:3a4fded2:::smithi18925896-23:head [append 0~524288] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:16:12.101 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:52.691563+0000", 2024-09-18T10:16:12.101 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 79.383212353999994, 2024-09-18T10:16:12.101 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.418704519, 2024-09-18T10:16:12.101 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:12.101 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:12.101 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:16:12.101 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:16:12.101 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:12.102 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 181 2024-09-18T10:16:12.102 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.102 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:12.102 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.102 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:12.102 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.691563+0000", 2024-09-18T10:16:12.102 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:12.102 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.102 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.102 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:12.102 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.691563+0000", 2024-09-18T10:16:12.102 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:12.102 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.102 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.103 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:12.103 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.691560+0000", 2024-09-18T10:16:12.103 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999971 2024-09-18T10:16:12.103 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.103 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.103 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:12.103 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.692740+0000", 2024-09-18T10:16:12.103 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001180195 2024-09-18T10:16:12.103 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.103 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.103 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:12.103 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.692779+0000", 2024-09-18T10:16:12.103 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8535000000000003e-05 2024-09-18T10:16:12.103 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.104 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.104 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:12.104 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.692794+0000", 2024-09-18T10:16:12.104 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4807000000000001e-05 2024-09-18T10:16:12.104 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.104 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.104 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:12.104 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.793038+0000", 2024-09-18T10:16:12.104 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100244729 2024-09-18T10:16:12.104 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.104 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.104 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-09-18T10:16:12.104 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.793050+0000", 2024-09-18T10:16:12.104 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1776e-05 2024-09-18T10:16:12.105 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.105 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.105 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:12.105 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.893883+0000", 2024-09-18T10:16:12.105 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100833383 2024-09-18T10:16:12.105 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.105 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.105 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:16:12.105 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.894056+0000", 2024-09-18T10:16:12.105 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017296999999999999 2024-09-18T10:16:12.105 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.105 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.105 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:16:12.105 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.896702+0000", 2024-09-18T10:16:12.106 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002645728 2024-09-18T10:16:12.106 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.106 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.106 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:16:12.106 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.001443+0000", 2024-09-18T10:16:12.106 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.104740824 2024-09-18T10:16:12.106 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.106 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.106 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:16:12.106 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.110173+0000", 2024-09-18T10:16:12.106 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.108729645 2024-09-18T10:16:12.106 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.106 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.106 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:12.106 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.110268+0000", 2024-09-18T10:16:12.107 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4987000000000004e-05 2024-09-18T10:16:12.107 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:12.107 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:12.107 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:12.107 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.107 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.107 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:182 3.cs0 3:3a4fded2:::smithi18925896-23:head [append 0~376832] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:16:12.107 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:52.693795+0000", 2024-09-18T10:16:12.107 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 79.380980625999996, 2024-09-18T10:16:12.107 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.51656788200000003, 2024-09-18T10:16:12.107 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:12.107 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:12.107 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:16:12.107 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:16:12.108 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:12.108 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 182 2024-09-18T10:16:12.108 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.108 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:12.108 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.108 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:12.108 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.693795+0000", 2024-09-18T10:16:12.108 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:12.108 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.108 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.108 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:12.108 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.693795+0000", 2024-09-18T10:16:12.108 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:12.108 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.109 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.109 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:12.109 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.693787+0000", 2024-09-18T10:16:12.109 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999919 2024-09-18T10:16:12.109 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.109 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.109 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:12.109 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694491+0000", 2024-09-18T10:16:12.109 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00070477199999999999 2024-09-18T10:16:12.109 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.109 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.109 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:12.109 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694526+0000", 2024-09-18T10:16:12.109 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4304999999999998e-05 2024-09-18T10:16:12.110 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.110 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.110 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:12.110 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694539+0000", 2024-09-18T10:16:12.110 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3107e-05 2024-09-18T10:16:12.110 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.110 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.110 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:12.110 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.793145+0000", 2024-09-18T10:16:12.110 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.098605719999999994 2024-09-18T10:16:12.110 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.110 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.110 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:16:12.110 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.793156+0000", 2024-09-18T10:16:12.111 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1892000000000001e-05 2024-09-18T10:16:12.111 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.111 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.111 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:12.111 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.898472+0000", 2024-09-18T10:16:12.111 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10531558000000001 2024-09-18T10:16:12.111 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.111 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.111 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:16:12.111 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.898643+0000", 2024-09-18T10:16:12.111 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017116399999999999 2024-09-18T10:16:12.111 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.111 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.111 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:16:12.112 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.900074+0000", 2024-09-18T10:16:12.112 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0014304299999999999 2024-09-18T10:16:12.112 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.112 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.112 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:16:12.112 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.001446+0000", 2024-09-18T10:16:12.112 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.101371934 2024-09-18T10:16:12.112 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.112 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.112 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:16:12.112 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.210292+0000", 2024-09-18T10:16:12.112 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.208846693 2024-09-18T10:16:12.112 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.113 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.113 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:12.113 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.210363+0000", 2024-09-18T10:16:12.113 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0470999999999997e-05 2024-09-18T10:16:12.113 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:12.113 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:12.113 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:12.113 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.113 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.113 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:183 3.cs0 3:3a4fded2:::smithi18925896-23:head [setxattr _header (56) in=7b] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:16:12.113 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:52.694557+0000", 2024-09-18T10:16:12.113 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 79.380218065999998, 2024-09-18T10:16:12.113 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.51641106000000003, 2024-09-18T10:16:12.113 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:12.114 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:12.114 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:16:12.114 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:16:12.114 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:12.114 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 183 2024-09-18T10:16:12.114 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.114 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:12.114 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.114 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:12.114 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694557+0000", 2024-09-18T10:16:12.114 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:12.114 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.114 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.114 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:12.115 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694557+0000", 2024-09-18T10:16:12.115 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:12.115 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.115 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.115 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:12.115 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694556+0000", 2024-09-18T10:16:12.115 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999986 2024-09-18T10:16:12.115 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.115 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.115 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:12.115 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694570+0000", 2024-09-18T10:16:12.115 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4249e-05 2024-09-18T10:16:12.115 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.115 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.116 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:12.116 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694580+0000", 2024-09-18T10:16:12.116 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7459999999999995e-06 2024-09-18T10:16:12.116 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.116 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.116 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:12.116 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694601+0000", 2024-09-18T10:16:12.116 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1583e-05 2024-09-18T10:16:12.116 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.116 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.116 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:12.116 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.793251+0000", 2024-09-18T10:16:12.116 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.098649465000000006 2024-09-18T10:16:12.116 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.117 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.117 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:16:12.117 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.793262+0000", 2024-09-18T10:16:12.117 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1113e-05 2024-09-18T10:16:12.117 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.117 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.117 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:12.117 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.001995+0000", 2024-09-18T10:16:12.117 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20873336200000001 2024-09-18T10:16:12.117 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.117 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.117 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:16:12.117 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.002285+0000", 2024-09-18T10:16:12.117 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00028924300000000001 2024-09-18T10:16:12.118 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.118 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.118 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:16:12.118 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.003993+0000", 2024-09-18T10:16:12.118 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0017085410000000001 2024-09-18T10:16:12.118 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.118 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.118 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:16:12.118 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.109904+0000", 2024-09-18T10:16:12.118 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.105911038 2024-09-18T10:16:12.118 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.118 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.118 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:16:12.118 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.210762+0000", 2024-09-18T10:16:12.119 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100858146 2024-09-18T10:16:12.119 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.119 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.119 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:12.119 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.210968+0000", 2024-09-18T10:16:12.119 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00020613000000000001 2024-09-18T10:16:12.119 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:12.119 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:12.119 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:12.119 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.119 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.119 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:184 3.cs0 3:3a4fded2:::smithi18925896-23:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:16:12.119 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:52.694648+0000", 2024-09-18T10:16:12.120 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 79.380127841999993, 2024-09-18T10:16:12.120 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.72409976600000003, 2024-09-18T10:16:12.120 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:12.120 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-18T10:16:12.120 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:16:12.120 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:16:12.120 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:12.120 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 184 2024-09-18T10:16:12.120 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.120 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:12.120 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.120 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:12.120 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694648+0000", 2024-09-18T10:16:12.121 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:12.121 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.121 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.121 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:12.121 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694648+0000", 2024-09-18T10:16:12.121 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:12.121 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.121 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.121 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:12.121 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694647+0000", 2024-09-18T10:16:12.121 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:16:12.121 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.121 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.121 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:12.122 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694652+0000", 2024-09-18T10:16:12.122 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1170000000000003e-06 2024-09-18T10:16:12.122 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.122 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.122 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:12.122 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694668+0000", 2024-09-18T10:16:12.122 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6019e-05 2024-09-18T10:16:12.122 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.122 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.122 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:12.122 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694674+0000", 2024-09-18T10:16:12.122 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7000000000000002e-06 2024-09-18T10:16:12.122 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.123 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.123 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:12.123 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.893402+0000", 2024-09-18T10:16:12.123 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.19872805700000001 2024-09-18T10:16:12.123 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.123 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.123 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:16:12.123 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.893413+0000", 2024-09-18T10:16:12.123 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0329999999999999e-05 2024-09-18T10:16:12.123 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.123 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.123 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:12.123 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.004678+0000", 2024-09-18T10:16:12.123 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.111265369 2024-09-18T10:16:12.124 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.124 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.124 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2024-09-18T10:16:12.124 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.004893+0000", 2024-09-18T10:16:12.124 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00021510499999999999 2024-09-18T10:16:12.124 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.124 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.124 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:12.124 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.211096+0000", 2024-09-18T10:16:12.124 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.206202899 2024-09-18T10:16:12.124 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.124 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.124 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:16:12.124 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.211230+0000", 2024-09-18T10:16:12.125 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000133907 2024-09-18T10:16:12.125 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.125 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.125 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:12.125 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.418747+0000", 2024-09-18T10:16:12.125 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20751733 2024-09-18T10:16:12.125 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:12.125 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:12.125 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:12.125 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.125 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.125 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:196 3.cs0 3:351da1bf:::smithi18925896-6:head [append 0~0] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:16:12.125 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:52.694697+0000", 2024-09-18T10:16:12.126 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 79.380078744000002, 2024-09-18T10:16:12.126 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.61758450099999995, 2024-09-18T10:16:12.126 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:12.126 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:12.126 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:16:12.126 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:16:12.126 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:12.126 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 196 2024-09-18T10:16:12.126 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.126 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:12.126 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.126 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:12.126 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694697+0000", 2024-09-18T10:16:12.126 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:12.127 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.127 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.127 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:12.127 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694697+0000", 2024-09-18T10:16:12.127 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:12.127 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.127 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.127 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:12.127 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694696+0000", 2024-09-18T10:16:12.127 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:16:12.127 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.127 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.127 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:12.127 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694705+0000", 2024-09-18T10:16:12.128 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7799999999999995e-06 2024-09-18T10:16:12.128 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.128 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.128 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:12.128 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694721+0000", 2024-09-18T10:16:12.128 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5102e-05 2024-09-18T10:16:12.128 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.128 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.128 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:12.128 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694727+0000", 2024-09-18T10:16:12.128 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2570000000000004e-06 2024-09-18T10:16:12.128 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.128 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.129 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:12.129 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.893457+0000", 2024-09-18T10:16:12.129 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.198729824 2024-09-18T10:16:12.129 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.129 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.129 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:16:12.129 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.893464+0000", 2024-09-18T10:16:12.129 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6860000000000006e-06 2024-09-18T10:16:12.129 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.129 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.129 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:12.129 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.105086+0000", 2024-09-18T10:16:12.129 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.21162193000000001 2024-09-18T10:16:12.129 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.129 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.130 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:16:12.130 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.105357+0000", 2024-09-18T10:16:12.130 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00027109699999999998 2024-09-18T10:16:12.130 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.130 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.130 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:16:12.130 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.107161+0000", 2024-09-18T10:16:12.130 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001803727 2024-09-18T10:16:12.130 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.130 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.130 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:16:12.130 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.109961+0000", 2024-09-18T10:16:12.130 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0028003080000000001 2024-09-18T10:16:12.130 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.131 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.131 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:16:12.131 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.312174+0000", 2024-09-18T10:16:12.131 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20221219700000001 2024-09-18T10:16:12.131 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.131 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.131 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:12.131 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.312281+0000", 2024-09-18T10:16:12.131 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000107656 2024-09-18T10:16:12.131 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:12.131 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:12.131 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:12.131 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.131 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.132 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:197 3.cs0 3:351da1bf:::smithi18925896-6:head [rollback 2] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:16:12.132 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:52.694734+0000", 2024-09-18T10:16:12.132 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 79.380041702, 2024-09-18T10:16:12.132 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.61830483899999999, 2024-09-18T10:16:12.132 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:12.132 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:12.132 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:16:12.132 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:16:12.132 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:12.132 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 197 2024-09-18T10:16:12.132 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.132 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:12.132 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.132 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:12.133 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694734+0000", 2024-09-18T10:16:12.133 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:12.133 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.133 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.133 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:12.133 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694734+0000", 2024-09-18T10:16:12.133 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:12.133 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.133 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.133 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:12.133 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694733+0000", 2024-09-18T10:16:12.133 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:16:12.133 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.133 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.134 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:12.134 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694737+0000", 2024-09-18T10:16:12.134 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.636e-06 2024-09-18T10:16:12.134 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.134 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.134 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:12.134 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694752+0000", 2024-09-18T10:16:12.134 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5792999999999999e-05 2024-09-18T10:16:12.134 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.134 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.134 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:12.134 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694758+0000", 2024-09-18T10:16:12.134 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2660000000000001e-06 2024-09-18T10:16:12.134 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.135 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.135 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:12.135 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.108286+0000", 2024-09-18T10:16:12.135 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.41352792700000002 2024-09-18T10:16:12.135 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.135 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.135 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:16:12.135 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.108540+0000", 2024-09-18T10:16:12.135 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000253928 2024-09-18T10:16:12.135 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.135 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.135 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:16:12.135 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.109284+0000", 2024-09-18T10:16:12.135 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00074488600000000005 2024-09-18T10:16:12.136 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.136 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.136 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:16:12.136 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.210174+0000", 2024-09-18T10:16:12.136 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10088973699999999 2024-09-18T10:16:12.136 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.136 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.136 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:16:12.136 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.312857+0000", 2024-09-18T10:16:12.136 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10268295600000001 2024-09-18T10:16:12.136 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.136 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.136 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:12.137 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.313039+0000", 2024-09-18T10:16:12.137 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000181476 2024-09-18T10:16:12.137 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:12.137 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:12.137 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:12.137 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.137 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.137 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:633 3.cs0 3:3972e5c3:::smithi18925896-50:head [read 0~3981312,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-18T10:16:12.137 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:28.815454+0000", 2024-09-18T10:16:12.137 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 43.259321768, 2024-09-18T10:16:12.137 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.41958616100000001, 2024-09-18T10:16:12.137 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:12.137 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-18T10:16:12.137 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:16:12.138 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:16:12.138 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:12.138 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 633 2024-09-18T10:16:12.138 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.138 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:12.138 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.138 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:12.138 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:28.815454+0000", 2024-09-18T10:16:12.138 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:12.138 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.138 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.138 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:12.138 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:28.815454+0000", 2024-09-18T10:16:12.138 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:12.139 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.139 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.139 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:12.139 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:28.815453+0000", 2024-09-18T10:16:12.139 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:16:12.139 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.139 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.139 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:12.139 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:28.815458+0000", 2024-09-18T10:16:12.139 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1760000000000001e-06 2024-09-18T10:16:12.139 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.139 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.139 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:12.140 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:28.815467+0000", 2024-09-18T10:16:12.140 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1300000000000007e-06 2024-09-18T10:16:12.140 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.140 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.140 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:12.140 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:28.815484+0000", 2024-09-18T10:16:12.140 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6722e-05 2024-09-18T10:16:12.140 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.140 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.140 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:12.140 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.021577+0000", 2024-09-18T10:16:12.140 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20609329200000001 2024-09-18T10:16:12.140 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.140 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.141 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:16:12.141 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.021788+0000", 2024-09-18T10:16:12.141 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000211115 2024-09-18T10:16:12.141 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.141 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.141 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:12.141 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.235040+0000", 2024-09-18T10:16:12.141 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.213251576 2024-09-18T10:16:12.141 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:12.141 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:12.141 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:12.141 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.141 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.141 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:750 3.cs0 3:3a4fded2:::smithi18925896-23:head [append 0~507904] snapc 2d=[29,28,20,1f,1e,1a,11] ondisk+write+known_if_redirected+supports_pool_eio e84)", 2024-09-18T10:16:12.142 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:39.699368+0000", 2024-09-18T10:16:12.142 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 32.375407928000001, 2024-09-18T10:16:12.142 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.71043271500000005, 2024-09-18T10:16:12.142 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:12.142 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:12.142 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:16:12.142 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:16:12.142 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:12.142 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 750 2024-09-18T10:16:12.142 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.142 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:12.142 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.142 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:12.142 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.699368+0000", 2024-09-18T10:16:12.143 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:12.143 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.143 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.143 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:12.143 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.699368+0000", 2024-09-18T10:16:12.143 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:12.143 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.143 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.143 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:12.143 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.699365+0000", 2024-09-18T10:16:12.143 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999971 2024-09-18T10:16:12.143 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.143 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.143 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:12.144 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.700330+0000", 2024-09-18T10:16:12.144 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00096560400000000001 2024-09-18T10:16:12.144 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.144 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.144 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:12.144 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.700363+0000", 2024-09-18T10:16:12.144 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2335999999999997e-05 2024-09-18T10:16:12.144 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.144 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.144 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:12.144 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.700375+0000", 2024-09-18T10:16:12.144 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2319e-05 2024-09-18T10:16:12.144 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.144 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.145 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:12.145 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.880022+0000", 2024-09-18T10:16:12.145 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.17964736000000001 2024-09-18T10:16:12.145 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.145 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.145 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:16:12.145 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.880560+0000", 2024-09-18T10:16:12.145 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00053748100000000003 2024-09-18T10:16:12.145 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.145 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.145 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:16:12.145 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.883059+0000", 2024-09-18T10:16:12.145 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002499631 2024-09-18T10:16:12.145 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.146 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.146 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:16:12.146 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.990736+0000", 2024-09-18T10:16:12.146 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10767652 2024-09-18T10:16:12.146 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.146 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.146 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:16:12.146 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.408641+0000", 2024-09-18T10:16:12.146 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.41790554299999999 2024-09-18T10:16:12.146 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.146 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.147 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:12.147 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.409800+0000", 2024-09-18T10:16:12.147 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001158854 2024-09-18T10:16:12.147 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:12.147 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:12.147 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:12.147 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.147 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.147 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:751 3.cs0 3:3a4fded2:::smithi18925896-23:head [append 0~425984] snapc 2d=[29,28,20,1f,1e,1a,11] ondisk+write+known_if_redirected+supports_pool_eio e84)", 2024-09-18T10:16:12.147 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:39.701793+0000", 2024-09-18T10:16:12.147 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 32.372982837000002, 2024-09-18T10:16:12.147 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.70943514100000005, 2024-09-18T10:16:12.147 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:12.148 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:12.148 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:16:12.148 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:16:12.148 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:12.148 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 751 2024-09-18T10:16:12.148 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.148 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:12.148 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.148 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:12.148 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.701793+0000", 2024-09-18T10:16:12.148 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:12.148 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.149 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.149 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:12.149 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.701793+0000", 2024-09-18T10:16:12.149 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:12.149 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.149 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.149 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:12.149 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.701791+0000", 2024-09-18T10:16:12.149 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999981 2024-09-18T10:16:12.149 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.149 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.149 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:12.150 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.702524+0000", 2024-09-18T10:16:12.150 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00073345599999999995 2024-09-18T10:16:12.150 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.150 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.150 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:12.150 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.702550+0000", 2024-09-18T10:16:12.150 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5989000000000001e-05 2024-09-18T10:16:12.150 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.150 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.150 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:12.150 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.702560+0000", 2024-09-18T10:16:12.150 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7270000000000002e-06 2024-09-18T10:16:12.150 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.151 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.151 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:12.151 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.986833+0000", 2024-09-18T10:16:12.151 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.28427270399999999 2024-09-18T10:16:12.151 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.151 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.151 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:16:12.151 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.987210+0000", 2024-09-18T10:16:12.151 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00037690899999999998 2024-09-18T10:16:12.151 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.152 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.152 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:16:12.152 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.989054+0000", 2024-09-18T10:16:12.152 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001844856 2024-09-18T10:16:12.152 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.152 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.152 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:16:12.152 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.102805+0000", 2024-09-18T10:16:12.152 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.113750928 2024-09-18T10:16:12.152 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.152 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.152 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:16:12.152 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.410623+0000", 2024-09-18T10:16:12.152 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30781788199999999 2024-09-18T10:16:12.153 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.153 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.153 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:12.153 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.411228+0000", 2024-09-18T10:16:12.153 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00060455000000000003 2024-09-18T10:16:12.153 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:12.153 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:12.153 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:12.153 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.153 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.153 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:752 3.cs0 3:3a4fded2:::smithi18925896-23:head [append 0~442368] snapc 2d=[29,28,20,1f,1e,1a,11] ondisk+write+known_if_redirected+supports_pool_eio e84)", 2024-09-18T10:16:12.153 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:39.704347+0000", 2024-09-18T10:16:12.153 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 32.370428711000002, 2024-09-18T10:16:12.154 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.70797012699999995, 2024-09-18T10:16:12.154 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:12.154 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:12.154 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:16:12.154 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:16:12.154 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:12.154 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 752 2024-09-18T10:16:12.154 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.154 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:12.154 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.154 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:12.154 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.704347+0000", 2024-09-18T10:16:12.154 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:12.154 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.155 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.155 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:12.155 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.704347+0000", 2024-09-18T10:16:12.155 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:12.155 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.155 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.155 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:12.155 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.704339+0000", 2024-09-18T10:16:12.155 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999928 2024-09-18T10:16:12.155 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.155 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.155 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:12.155 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.705024+0000", 2024-09-18T10:16:12.155 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00068492100000000001 2024-09-18T10:16:12.156 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.156 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.156 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:12.156 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.705059+0000", 2024-09-18T10:16:12.156 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4739000000000001e-05 2024-09-18T10:16:12.156 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.156 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.156 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:12.156 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.705072+0000", 2024-09-18T10:16:12.156 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3125999999999999e-05 2024-09-18T10:16:12.156 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.156 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.156 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:12.157 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.991070+0000", 2024-09-18T10:16:12.157 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.285997483 2024-09-18T10:16:12.157 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.157 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.157 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:16:12.157 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.991414+0000", 2024-09-18T10:16:12.157 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00034466500000000002 2024-09-18T10:16:12.157 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.157 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.157 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:16:12.157 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.993220+0000", 2024-09-18T10:16:12.157 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001805891 2024-09-18T10:16:12.157 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.157 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.158 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:16:12.158 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.206261+0000", 2024-09-18T10:16:12.158 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.21304089300000001 2024-09-18T10:16:12.158 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.158 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.158 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:16:12.158 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.411747+0000", 2024-09-18T10:16:12.158 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.205486321 2024-09-18T10:16:12.158 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.158 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.158 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:12.158 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.412317+0000", 2024-09-18T10:16:12.158 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00056940799999999998 2024-09-18T10:16:12.158 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:12.159 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:12.159 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:12.159 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.159 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.159 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:753 3.cs0 3:3a4fded2:::smithi18925896-23:head [append 0~540672 in=540672b] snapc 2d=[29,28,20,1f,1e,1a,11] ondisk+write+known_if_redirected+supports_pool_eio e84)", 2024-09-18T10:16:12.159 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:39.707399+0000", 2024-09-18T10:16:12.159 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 32.367376770999996, 2024-09-18T10:16:12.159 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.80469576600000003, 2024-09-18T10:16:12.159 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:12.159 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:12.159 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:16:12.159 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:16:12.159 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:12.159 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 753 2024-09-18T10:16:12.160 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.160 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:12.160 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.160 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:12.160 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.707399+0000", 2024-09-18T10:16:12.160 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:12.160 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.160 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.160 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:12.160 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.707399+0000", 2024-09-18T10:16:12.160 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:12.160 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.160 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.161 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:12.161 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.707391+0000", 2024-09-18T10:16:12.161 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999928 2024-09-18T10:16:12.161 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.161 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.161 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:12.161 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.708412+0000", 2024-09-18T10:16:12.161 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010203759999999999 2024-09-18T10:16:12.161 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.161 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.161 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:12.161 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.708445+0000", 2024-09-18T10:16:12.161 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3674999999999999e-05 2024-09-18T10:16:12.161 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.162 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.162 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:12.162 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.708458+0000", 2024-09-18T10:16:12.162 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2943e-05 2024-09-18T10:16:12.162 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.162 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.162 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:12.162 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.994678+0000", 2024-09-18T10:16:12.162 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.28621953700000002 2024-09-18T10:16:12.162 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.162 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.162 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:16:12.162 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.995068+0000", 2024-09-18T10:16:12.162 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000390338 2024-09-18T10:16:12.163 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.163 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.163 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:16:12.163 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.997188+0000", 2024-09-18T10:16:12.163 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0021194159999999998 2024-09-18T10:16:12.163 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.163 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.163 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:16:12.163 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.206267+0000", 2024-09-18T10:16:12.163 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20907888599999999 2024-09-18T10:16:12.163 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.163 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.163 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:16:12.163 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.510874+0000", 2024-09-18T10:16:12.164 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30460767599999999 2024-09-18T10:16:12.164 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.164 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.164 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:12.164 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.512094+0000", 2024-09-18T10:16:12.164 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001220215 2024-09-18T10:16:12.164 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:12.164 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:12.164 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:12.164 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.164 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.164 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:754 3.cs0 3:3a4fded2:::smithi18925896-23:head [append 0~147456 in=147456b] snapc 2d=[29,28,20,1f,1e,1a,11] ondisk+write+known_if_redirected+supports_pool_eio e84)", 2024-09-18T10:16:12.164 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:39.708472+0000", 2024-09-18T10:16:12.164 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 32.366303813000002, 2024-09-18T10:16:12.165 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.90559658799999998, 2024-09-18T10:16:12.165 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:12.165 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:12.165 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:16:12.165 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:16:12.165 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:12.165 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 754 2024-09-18T10:16:12.165 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.165 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:12.165 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.165 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:12.165 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.708472+0000", 2024-09-18T10:16:12.165 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:12.165 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.166 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.166 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:12.166 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.708472+0000", 2024-09-18T10:16:12.166 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:12.166 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.166 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.166 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:12.166 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.708470+0000", 2024-09-18T10:16:12.166 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999986 2024-09-18T10:16:12.166 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.166 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.166 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:12.166 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.708676+0000", 2024-09-18T10:16:12.167 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00020609599999999999 2024-09-18T10:16:12.167 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.167 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.167 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:12.167 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.708704+0000", 2024-09-18T10:16:12.167 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7138999999999999e-05 2024-09-18T10:16:12.167 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.167 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.167 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:12.167 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.708713+0000", 2024-09-18T10:16:12.167 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1500000000000005e-06 2024-09-18T10:16:12.167 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.167 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.167 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:12.168 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.099648+0000", 2024-09-18T10:16:12.168 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.39093543800000002 2024-09-18T10:16:12.168 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.168 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.168 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:16:12.168 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.100061+0000", 2024-09-18T10:16:12.168 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000412958 2024-09-18T10:16:12.168 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.168 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.168 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:16:12.168 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.101562+0000", 2024-09-18T10:16:12.168 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001500758 2024-09-18T10:16:12.168 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.168 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.169 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:16:12.169 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.206942+0000", 2024-09-18T10:16:12.169 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10538027 2024-09-18T10:16:12.169 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.169 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.169 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:16:12.169 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.612979+0000", 2024-09-18T10:16:12.169 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.40603637599999998 2024-09-18T10:16:12.169 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.169 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.169 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:12.169 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.614068+0000", 2024-09-18T10:16:12.169 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001089717 2024-09-18T10:16:12.169 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:12.170 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:12.170 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:12.170 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.170 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.170 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:755 3.cs0 3:3a4fded2:::smithi18925896-23:head [setxattr _header (57) in=64b] snapc 2d=[29,28,20,1f,1e,1a,11] ondisk+write+known_if_redirected+supports_pool_eio e84)", 2024-09-18T10:16:12.170 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:39.708729+0000", 2024-09-18T10:16:12.170 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 32.366046599999997, 2024-09-18T10:16:12.170 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.095273111, 2024-09-18T10:16:12.170 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:12.170 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:12.170 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:16:12.170 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:16:12.170 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:12.170 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 755 2024-09-18T10:16:12.171 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.171 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:12.171 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.171 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:12.171 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.708729+0000", 2024-09-18T10:16:12.171 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:12.171 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.171 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.171 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:12.171 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.708729+0000", 2024-09-18T10:16:12.171 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:12.171 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.171 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.171 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:12.172 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.708728+0000", 2024-09-18T10:16:12.172 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:16:12.172 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.172 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.172 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:12.172 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.708738+0000", 2024-09-18T10:16:12.172 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.01e-05 2024-09-18T10:16:12.172 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.172 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.172 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:12.172 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.708752+0000", 2024-09-18T10:16:12.172 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3713000000000001e-05 2024-09-18T10:16:12.172 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.172 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.173 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:12.173 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.708762+0000", 2024-09-18T10:16:12.173 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0631999999999999e-05 2024-09-18T10:16:12.173 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.173 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.173 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:12.173 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.203162+0000", 2024-09-18T10:16:12.173 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.49439976200000002 2024-09-18T10:16:12.173 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.173 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.173 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:16:12.173 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.203686+0000", 2024-09-18T10:16:12.173 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00052412100000000001 2024-09-18T10:16:12.173 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.174 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.174 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:16:12.174 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.205437+0000", 2024-09-18T10:16:12.174 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001750759 2024-09-18T10:16:12.174 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.174 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.174 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:16:12.174 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.307503+0000", 2024-09-18T10:16:12.174 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10206599199999999 2024-09-18T10:16:12.174 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.174 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.174 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:16:12.174 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.802731+0000", 2024-09-18T10:16:12.174 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.49522788200000001 2024-09-18T10:16:12.175 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.175 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.175 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:12.175 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.804002+0000", 2024-09-18T10:16:12.175 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001271063 2024-09-18T10:16:12.175 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:12.175 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:12.175 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:12.175 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.175 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.176 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:756 3.cs0 3:3a4fded2:::smithi18925896-23:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e84)", 2024-09-18T10:16:12.176 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:39.708769+0000", 2024-09-18T10:16:12.176 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 32.366006048000003, 2024-09-18T10:16:12.176 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.917282216, 2024-09-18T10:16:12.176 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:12.176 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-18T10:16:12.176 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:16:12.176 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:16:12.176 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:12.176 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 756 2024-09-18T10:16:12.176 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.176 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:12.176 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.177 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:12.177 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.708769+0000", 2024-09-18T10:16:12.177 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:12.177 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.177 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.177 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:12.177 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.708769+0000", 2024-09-18T10:16:12.177 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:12.177 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.177 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.177 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:12.177 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.708769+0000", 2024-09-18T10:16:12.177 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:16:12.177 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.177 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.178 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:12.178 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.708773+0000", 2024-09-18T10:16:12.178 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9929999999999997e-06 2024-09-18T10:16:12.178 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.178 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.178 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:12.178 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.708786+0000", 2024-09-18T10:16:12.178 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3606e-05 2024-09-18T10:16:12.178 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.178 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.179 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:12.179 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:39.708792+0000", 2024-09-18T10:16:12.179 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7559999999999996e-06 2024-09-18T10:16:12.179 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.179 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.179 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:12.179 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.206394+0000", 2024-09-18T10:16:12.179 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.49760259400000001 2024-09-18T10:16:12.179 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.179 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.179 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2024-09-18T10:16:12.179 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.206929+0000", 2024-09-18T10:16:12.179 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00053430700000000003 2024-09-18T10:16:12.180 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.180 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.180 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:12.180 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.905920+0000", 2024-09-18T10:16:12.180 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.69899167699999998 2024-09-18T10:16:12.180 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.180 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.180 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:16:12.180 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.906439+0000", 2024-09-18T10:16:12.180 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00051873700000000002 2024-09-18T10:16:12.180 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.180 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.180 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:12.181 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:41.626052+0000", 2024-09-18T10:16:12.181 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.71961241200000003 2024-09-18T10:16:12.181 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:12.181 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:12.181 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:12.181 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.181 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.181 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:894 3.cs0 3:3972e5c3:::smithi18925896-50:head [append 0~0] snapc 34=[34,33,30,2f,28,20,1f,1a,11] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-18T10:16:12.181 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:46.655936+0000", 2024-09-18T10:16:12.181 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 25.418839611999999, 2024-09-18T10:16:12.181 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.41569194399999998, 2024-09-18T10:16:12.181 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:12.181 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:12.181 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:16:12.182 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:16:12.182 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:12.182 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 894 2024-09-18T10:16:12.182 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.182 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:12.182 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.182 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:12.182 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:46.655936+0000", 2024-09-18T10:16:12.182 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:12.182 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.182 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.182 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:12.182 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:46.655936+0000", 2024-09-18T10:16:12.182 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:12.183 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.183 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.183 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:12.183 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:46.655933+0000", 2024-09-18T10:16:12.183 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999967 2024-09-18T10:16:12.183 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.183 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.183 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:12.183 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:46.655950+0000", 2024-09-18T10:16:12.183 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7655999999999998e-05 2024-09-18T10:16:12.183 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.183 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.183 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:12.183 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:46.655989+0000", 2024-09-18T10:16:12.184 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8785000000000002e-05 2024-09-18T10:16:12.184 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.184 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.184 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:12.184 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:46.656005+0000", 2024-09-18T10:16:12.184 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5699999999999999e-05 2024-09-18T10:16:12.184 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.184 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.184 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:12.184 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:46.667772+0000", 2024-09-18T10:16:12.184 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011766999 2024-09-18T10:16:12.184 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.184 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.184 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-09-18T10:16:12.185 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:46.667799+0000", 2024-09-18T10:16:12.185 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7529e-05 2024-09-18T10:16:12.185 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.185 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.185 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:12.185 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:46.772915+0000", 2024-09-18T10:16:12.185 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.105115785 2024-09-18T10:16:12.185 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.185 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.185 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:16:12.185 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:46.773182+0000", 2024-09-18T10:16:12.185 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000267211 2024-09-18T10:16:12.185 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.185 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.186 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:16:12.186 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:46.775225+0000", 2024-09-18T10:16:12.186 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002042814 2024-09-18T10:16:12.186 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.186 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.186 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:16:12.186 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:46.970216+0000", 2024-09-18T10:16:12.186 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.194991465 2024-09-18T10:16:12.186 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.186 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.186 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:16:12.186 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:47.070954+0000", 2024-09-18T10:16:12.186 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100737702 2024-09-18T10:16:12.186 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.187 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.187 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:12.187 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:47.071628+0000", 2024-09-18T10:16:12.187 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00067362399999999995 2024-09-18T10:16:12.187 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:12.187 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:12.187 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:12.187 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.187 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.187 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWriteReply(3.cs0 100/94 ECSubWriteReply(tid=249, last_complete=100'156, committed=1, applied=1))", 2024-09-18T10:16:12.187 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:56.181007+0000", 2024-09-18T10:16:12.187 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 15.893768069, 2024-09-18T10:16:12.187 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.502082272, 2024-09-18T10:16:12.187 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:12.188 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:16:12.188 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:12.188 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.188 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:12.188 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:56.181007+0000", 2024-09-18T10:16:12.188 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:12.188 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.188 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.188 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:12.188 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:56.181007+0000", 2024-09-18T10:16:12.188 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:12.188 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.188 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.189 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:12.189 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:56.181007+0000", 2024-09-18T10:16:12.189 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999995 2024-09-18T10:16:12.189 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.189 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.189 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:12.189 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:56.181013+0000", 2024-09-18T10:16:12.189 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4799999999999998e-06 2024-09-18T10:16:12.189 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.189 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.189 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:12.189 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:56.181069+0000", 2024-09-18T10:16:12.189 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6051999999999998e-05 2024-09-18T10:16:12.190 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.190 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.190 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:12.190 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:56.281222+0000", 2024-09-18T10:16:12.190 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100153097 2024-09-18T10:16:12.190 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.190 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.190 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:12.190 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:56.681934+0000", 2024-09-18T10:16:12.190 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.400711769 2024-09-18T10:16:12.190 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.190 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.190 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:12.190 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:56.683090+0000", 2024-09-18T10:16:12.191 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0011555179999999999 2024-09-18T10:16:12.191 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:12.191 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:12.191 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:12.191 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:12.191 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:12.191 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-18T10:16:12.194 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-09-18T10:16:12.196 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-18T10:16:12.198 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-09-18T10:16:12.199 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":150,"stamp":"2024-09-18T10:16:11.226441+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":125338163,"num_objects":131,"num_object_clones":80,"num_object_copies":897,"num_objects_missing_on_primary":7,"num_objects_missing":7,"num_objects_degraded":0,"num_objects_misplaced":35,"num_objects_unfound":0,"num_objects_dirty":131,"num_whiteouts":11,"num_read":219,"num_read_kb":104941,"num_write":470,"num_write_kb":63466,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":93257728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2607,"ondisk_log_size":2607,"up":270,"acting":270,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":274,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":16,"kb":1509949440,"kb_used":5081168,"kb_used_data":261012,"kb_used_omap":26,"kb_used_meta":4819877,"kb_avail":1504868272,"statfs":{"total":1546188226560,"available":1540985110528,"internally_reserved":0,"allocated":267276288,"data_stored":340282376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":27264,"internal_metadata":4935554432},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-2097152,"num_objects":1,"num_object_clones":1,"num_object_copies":7,"num_objects_missing_on_primary":7,"num_objects_missing":7,"num_objects_degraded":0,"num_objects_misplaced":35,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":-36,"num_read_kb":-12695,"num_write":-106,"num_write_kb":-10736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-19,"num_bytes_recovered":-15237120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001652"},"pg_stats":[{"pgid":"3.23","version":"103'31","reported_seq":287,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-18T10:16:08.751728+0000","last_change":"2024-09-18T10:16:04.107991+0000","last_active":"2024-09-18T10:16:08.751728+0000","last_peered":"2024-09-18T10:16:08.751728+0000","last_clean":"2024-09-18T10:16:08.751728+0000","last_became_active":"2024-09-18T10:16:04.105399+0000","last_became_peered":"2024-09-18T10:16:04.105399+0000","last_unstale":"2024-09-18T10:16:08.751728+0000","last_undegraded":"2024-09-18T10:16:08.751728+0000","last_fullsized":"2024-09-18T10:16:08.751728+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":6,"last_scrub":"65'23","last_scrub_stamp":"2024-09-18T10:15:28.483344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:28.483344+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.40619440600000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":303104,"num_objects":1,"num_object_clones":1,"num_object_copies":7,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13,"num_write_kb":1432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4,13,11,2,5],"acting":[10,9,4,13,11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.22","version":"103'133","reported_seq":403,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-18T10:16:09.937189+0000","last_change":"2024-09-18T10:16:06.495375+0000","last_active":"2024-09-18T10:16:09.937189+0000","last_peered":"2024-09-18T10:16:09.937189+0000","last_clean":"2024-09-18T10:16:09.937189+0000","last_became_active":"2024-09-18T10:16:03.497692+0000","last_became_peered":"2024-09-18T10:16:03.497692+0000","last_unstale":"2024-09-18T10:16:09.937189+0000","last_undegraded":"2024-09-18T10:16:09.937189+0000","last_fullsized":"2024-09-18T10:16:09.937189+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":6,"last_scrub":"103'133","last_scrub_stamp":"2024-09-18T10:16:06.495097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:16:06.495097+0000","objects_scrubbed":8,"log_size":133,"log_dups_size":0,"ondisk_log_size":133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:17:09.897131+0000","scrub_duration":0.91069030399999995,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5226496,"num_objects":8,"num_object_clones":5,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":1,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,3,13,1,5,10,7],"acting":[14,3,13,1,5,10,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"3.21","version":"103'114","reported_seq":367,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-18T10:16:08.581073+0000","last_change":"2024-09-18T10:16:03.498733+0000","last_active":"2024-09-18T10:16:08.581073+0000","last_peered":"2024-09-18T10:16:08.581073+0000","last_clean":"2024-09-18T10:16:08.581073+0000","last_became_active":"2024-09-18T10:16:03.398090+0000","last_became_peered":"2024-09-18T10:16:03.398090+0000","last_unstale":"2024-09-18T10:16:08.581073+0000","last_undegraded":"2024-09-18T10:16:08.581073+0000","last_fullsized":"2024-09-18T10:16:08.581073+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":6,"last_scrub":"93'85","last_scrub_stamp":"2024-09-18T10:15:51.347579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:51.347579+0000","objects_scrubbed":9,"log_size":114,"log_dups_size":0,"ondisk_log_size":114,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.71137397400000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1761280,"num_objects":5,"num_object_clones":4,"num_object_copies":35,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":1,"num_read":5,"num_read_kb":7346,"num_write":12,"num_write_kb":40,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,13,14,10,7,3,2],"acting":[4,13,14,10,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.20","version":"101'40","reported_seq":297,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-18T10:16:08.581270+0000","last_change":"2024-09-18T10:16:03.570126+0000","last_active":"2024-09-18T10:16:08.581270+0000","last_peered":"2024-09-18T10:16:08.581270+0000","last_clean":"2024-09-18T10:16:08.581270+0000","last_became_active":"2024-09-18T10:16:03.396081+0000","last_became_peered":"2024-09-18T10:16:03.396081+0000","last_unstale":"2024-09-18T10:16:08.581270+0000","last_undegraded":"2024-09-18T10:16:08.581270+0000","last_fullsized":"2024-09-18T10:16:08.581270+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":6,"last_scrub":"36'4","last_scrub_stamp":"2024-09-18T10:15:05.668933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:05.668933+0000","objects_scrubbed":1,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"queued for scrub","scrub_duration":2.6328342170000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":991232,"num_objects":2,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":1,"num_read":2,"num_read_kb":1,"num_write":4,"num_write_kb":968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,9,2,6,15,4],"acting":[3,7,9,2,6,15,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"103'64","reported_seq":285,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-18T10:16:03.994244+0000","last_change":"2024-09-18T10:16:03.993246+0000","last_active":"2024-09-18T10:16:03.994244+0000","last_peered":"2024-09-18T10:16:03.994244+0000","last_clean":"2024-09-18T10:16:03.994244+0000","last_became_active":"2024-09-18T10:16:03.991332+0000","last_became_peered":"2024-09-18T10:16:03.991332+0000","last_unstale":"2024-09-18T10:16:03.994244+0000","last_undegraded":"2024-09-18T10:16:03.994244+0000","last_fullsized":"2024-09-18T10:16:03.994244+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:14:45.576168+0000","objects_scrubbed":0,"log_size":64,"log_dups_size":0,"ondisk_log_size":64,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3407872,"num_objects":2,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":891,"num_write":21,"num_write_kb":3608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,3,15,14,2,11,0],"acting":[13,3,15,14,2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"1e","length":"3"},{"start":"26","length":"1"},{"start":"29","length":"2"},{"start":"2d","length":"1"},{"start":"2f","length":"2"},{"start":"35","length":"1"}]},{"pgid":"3.1e","version":"103'74","reported_seq":375,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-18T10:16:08.650469+0000","last_change":"2024-09-18T10:16:07.432361+0000","last_active":"2024-09-18T10:16:08.650469+0000","last_peered":"2024-09-18T10:16:08.650469+0000","last_clean":"2024-09-18T10:16:08.650469+0000","last_became_active":"2024-09-18T10:16:03.597888+0000","last_became_peered":"2024-09-18T10:16:03.597888+0000","last_unstale":"2024-09-18T10:16:08.650469+0000","last_undegraded":"2024-09-18T10:16:08.650469+0000","last_fullsized":"2024-09-18T10:16:08.650469+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":5,"last_scrub":"103'74","last_scrub_stamp":"2024-09-18T10:16:07.432111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:16:07.432111+0000","objects_scrubbed":3,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:17:14.383341+0000","scrub_duration":0.30753098099999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2670592,"num_objects":3,"num_object_clones":2,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":1,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,14,6,13,8,3,12],"acting":[11,14,6,13,8,3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"103'49","reported_seq":325,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-18T10:16:03.573535+0000","last_change":"2024-09-18T10:16:03.191306+0000","last_active":"2024-09-18T10:16:03.573535+0000","last_peered":"2024-09-18T10:16:03.573535+0000","last_clean":"2024-09-18T10:16:03.573535+0000","last_became_active":"2024-09-18T10:16:03.187475+0000","last_became_peered":"2024-09-18T10:16:03.187475+0000","last_unstale":"2024-09-18T10:16:03.573535+0000","last_undegraded":"2024-09-18T10:16:03.573535+0000","last_fullsized":"2024-09-18T10:16:03.573535+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:14:45.576168+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1908736,"num_objects":2,"num_object_clones":2,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":8763,"num_write":6,"num_write_kb":1984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4194304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,1,11,15,9,8,4],"acting":[13,1,11,15,9,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"1e","length":"3"},{"start":"26","length":"1"},{"start":"29","length":"2"},{"start":"2d","length":"1"},{"start":"2f","length":"2"},{"start":"35","length":"1"}]},{"pgid":"3.1c","version":"104'184","reported_seq":506,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-18T10:16:08.781095+0000","last_change":"2024-09-18T10:16:04.854468+0000","last_active":"2024-09-18T10:16:08.781095+0000","last_peered":"2024-09-18T10:16:08.781095+0000","last_clean":"2024-09-18T10:16:08.781095+0000","last_became_active":"2024-09-18T10:16:03.395545+0000","last_became_peered":"2024-09-18T10:16:03.395545+0000","last_unstale":"2024-09-18T10:16:08.781095+0000","last_undegraded":"2024-09-18T10:16:08.781095+0000","last_fullsized":"2024-09-18T10:16:08.781095+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":5,"last_scrub":"104'184","last_scrub_stamp":"2024-09-18T10:16:04.854323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:16:04.854323+0000","objects_scrubbed":10,"log_size":184,"log_dups_size":0,"ondisk_log_size":184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:17:32.847713+0000","scrub_duration":0.41471314100000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13729792,"num_objects":10,"num_object_clones":7,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,13,3,7,12,14],"acting":[5,8,13,3,7,12,14],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"peering","last_fresh":"2024-09-18T10:16:09.931371+0000","last_change":"2024-09-18T10:16:09.931371+0000","last_active":"0.000000","last_peered":"0.000000","last_clean":"0.000000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-18T10:16:09.931371+0000","last_undegraded":"2024-09-18T10:16:09.931371+0000","last_fullsized":"2024-09-18T10:16:09.931371+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:14:45.576168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,5,14,8,12,2,13],"acting":[15,5,14,8,12,2,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":15,"acting_primary":15,"purged_snaps":[]},{"pgid":"3.1a","version":"103'74","reported_seq":23,"reported_epoch":113,"state":"active+recovering+degraded","last_fresh":"2024-09-18T10:16:10.123471+0000","last_change":"2024-09-18T10:16:10.116722+0000","last_active":"2024-09-18T10:16:10.123471+0000","last_peered":"2024-09-18T10:16:10.123471+0000","last_clean":"2024-09-18T10:15:58.857010+0000","last_became_active":"2024-09-18T10:16:09.096161+0000","last_became_peered":"2024-09-18T10:16:09.096161+0000","last_unstale":"2024-09-18T10:16:10.123471+0000","last_undegraded":"2024-09-18T10:16:10.014895+0000","last_fullsized":"2024-09-18T10:16:10.123471+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:14:45.576168+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1769472,"num_objects":3,"num_object_clones":2,"num_object_copies":21,"num_objects_missing_on_primary":7,"num_objects_missing":7,"num_objects_degraded":0,"num_objects_misplaced":35,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":21,"num_read_kb":8373,"num_write":21,"num_write_kb":3016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,12,15,2,14,1,13],"acting":[7,12,15,2,14,1,13],"avail_no_missing":["1(1)","3(4)","4(6)","5(3)","6(2)","13(0)","15(5)"],"object_location_counts":[{"shards":"1(1),3(4),4(6),5(3),6(2),13(0),15(5)","objects":5},{"shards":"1(5),2(3),7(0),12(1),13(6),14(4),15(2)","objects":-2}],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"105'140","reported_seq":242,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-18T10:16:08.709515+0000","last_change":"2024-09-18T10:16:01.833607+0000","last_active":"2024-09-18T10:16:08.709515+0000","last_peered":"2024-09-18T10:16:08.709515+0000","last_clean":"2024-09-18T10:16:08.709515+0000","last_became_active":"2024-09-18T10:15:51.892167+0000","last_became_peered":"2024-09-18T10:15:51.892167+0000","last_unstale":"2024-09-18T10:16:08.709515+0000","last_undegraded":"2024-09-18T10:16:08.709515+0000","last_fullsized":"2024-09-18T10:16:08.709515+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":5,"last_scrub":"68'92","last_scrub_stamp":"2024-09-18T10:15:27.485910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:27.485910+0000","objects_scrubbed":10,"log_size":140,"log_dups_size":0,"ondisk_log_size":140,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:39.070888+0000","scrub_duration":0.69568305100000005,"objects_trimmed":0,"snaptrim_duration":0.81620175800000005,"stat_sum":{"num_bytes":16646144,"num_objects":11,"num_object_clones":7,"num_object_copies":77,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":11,"num_read_kb":16756,"num_write":17,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":28753920,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,1,2,13,9,14],"acting":[8,4,1,2,13,9,14],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.18","version":"103'67","reported_seq":210,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-18T10:16:08.851534+0000","last_change":"2024-09-18T10:15:58.945752+0000","last_active":"2024-09-18T10:16:08.851534+0000","last_peered":"2024-09-18T10:16:08.851534+0000","last_clean":"2024-09-18T10:16:08.851534+0000","last_became_active":"2024-09-18T10:15:29.028750+0000","last_became_peered":"2024-09-18T10:15:29.028750+0000","last_unstale":"2024-09-18T10:16:08.851534+0000","last_undegraded":"2024-09-18T10:16:08.851534+0000","last_fullsized":"2024-09-18T10:16:08.851534+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":5,"last_scrub":"93'61","last_scrub_stamp":"2024-09-18T10:15:51.095148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:51.095148+0000","objects_scrubbed":5,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:56.109718+0000","scrub_duration":0.91822677900000005,"objects_trimmed":1,"snaptrim_duration":0.30585581699999997,"stat_sum":{"num_bytes":6291456,"num_objects":4,"num_object_clones":2,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,11,0,4,6,13,12],"acting":[10,11,0,4,6,13,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.17","version":"103'81","reported_seq":244,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-18T10:16:08.752347+0000","last_change":"2024-09-18T10:15:59.065040+0000","last_active":"2024-09-18T10:16:08.752347+0000","last_peered":"2024-09-18T10:16:08.752347+0000","last_clean":"2024-09-18T10:16:08.752347+0000","last_became_active":"2024-09-18T10:15:21.323733+0000","last_became_peered":"2024-09-18T10:15:21.323733+0000","last_unstale":"2024-09-18T10:16:08.752347+0000","last_undegraded":"2024-09-18T10:16:08.752347+0000","last_fullsized":"2024-09-18T10:16:08.752347+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":5,"last_scrub":"74'45","last_scrub_stamp":"2024-09-18T10:15:31.583098+0000","last_deep_scrub":"74'45","last_deep_scrub_stamp":"2024-09-18T10:15:31.583098+0000","last_clean_scrub_stamp":"2024-09-18T10:15:31.583098+0000","objects_scrubbed":5,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:52.458457+0000","scrub_duration":2.494961714,"objects_trimmed":1,"snaptrim_duration":0.49059750099999999,"stat_sum":{"num_bytes":6537216,"num_objects":4,"num_object_clones":2,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":11,"num_write_kb":3920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,13,3,14,2,11,6],"acting":[10,13,3,14,2,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"103'49","reported_seq":246,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-18T10:16:08.891915+0000","last_change":"2024-09-18T10:15:59.149421+0000","last_active":"2024-09-18T10:16:08.891915+0000","last_peered":"2024-09-18T10:16:08.891915+0000","last_clean":"2024-09-18T10:16:08.891915+0000","last_became_active":"2024-09-18T10:15:50.782912+0000","last_became_peered":"2024-09-18T10:15:50.782912+0000","last_unstale":"2024-09-18T10:16:08.891915+0000","last_undegraded":"2024-09-18T10:16:08.891915+0000","last_fullsized":"2024-09-18T10:16:08.891915+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":5,"last_scrub":"36'11","last_scrub_stamp":"2024-09-18T10:15:07.445786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:07.445786+0000","objects_scrubbed":1,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:14.115122+0000","scrub_duration":0.30645798400000002,"objects_trimmed":0,"snaptrim_duration":0.47534329800000003,"stat_sum":{"num_bytes":8486912,"num_objects":4,"num_object_clones":3,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":3,"num_read_kb":1,"num_write":9,"num_write_kb":7616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":13729792,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,6,4,8,2,14,10],"acting":[9,6,4,8,2,14,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.15","version":"100'31","reported_seq":202,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-18T10:16:09.937430+0000","last_change":"2024-09-18T10:15:58.565869+0000","last_active":"2024-09-18T10:16:09.937430+0000","last_peered":"2024-09-18T10:16:09.937430+0000","last_clean":"2024-09-18T10:16:09.937430+0000","last_became_active":"2024-09-18T10:15:17.127771+0000","last_became_peered":"2024-09-18T10:15:17.127771+0000","last_unstale":"2024-09-18T10:16:09.937430+0000","last_undegraded":"2024-09-18T10:16:09.937430+0000","last_fullsized":"2024-09-18T10:16:09.937430+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":5,"last_scrub":"66'10","last_scrub_stamp":"2024-09-18T10:15:27.270892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:27.270892+0000","objects_scrubbed":2,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:57.241541+0000","scrub_duration":0.77475592100000001,"objects_trimmed":0,"snaptrim_duration":0.002978016,"stat_sum":{"num_bytes":2408448,"num_objects":3,"num_object_clones":2,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":1,"num_read":2,"num_read_kb":1,"num_write":26,"num_write_kb":2288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,15,13,10,1,7,9],"acting":[14,15,13,10,1,7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"3.14","version":"37'8","reported_seq":166,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-18T10:16:08.580790+0000","last_change":"2024-09-18T10:15:58.576521+0000","last_active":"2024-09-18T10:16:08.580790+0000","last_peered":"2024-09-18T10:16:08.580790+0000","last_clean":"2024-09-18T10:16:08.580790+0000","last_became_active":"2024-09-18T10:15:14.767770+0000","last_became_peered":"2024-09-18T10:15:14.767770+0000","last_unstale":"2024-09-18T10:16:08.580790+0000","last_undegraded":"2024-09-18T10:16:08.580790+0000","last_fullsized":"2024-09-18T10:16:08.580790+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":5,"last_scrub":"37'8","last_scrub_stamp":"2024-09-18T10:15:17.041590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:17.041590+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:32.741813+0000","scrub_duration":0.60018000800000004,"objects_trimmed":0,"snaptrim_duration":0.0011844620000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,14,9,10,5,13,4],"acting":[1,14,9,10,5,13,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":94,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-18T10:16:03.586892+0000","last_change":"2024-09-18T10:14:37.427722+0000","last_active":"2024-09-18T10:16:03.586892+0000","last_peered":"2024-09-18T10:16:03.586892+0000","last_clean":"2024-09-18T10:16:03.586892+0000","last_became_active":"2024-09-18T10:14:35.348943+0000","last_became_peered":"2024-09-18T10:14:35.348943+0000","last_unstale":"2024-09-18T10:16:03.586892+0000","last_undegraded":"2024-09-18T10:16:03.586892+0000","last_fullsized":"2024-09-18T10:16:03.586892+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:14:34.288596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060442999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,10],"acting":[13,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"3.6","version":"103'54","reported_seq":318,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-18T10:16:08.649762+0000","last_change":"2024-09-18T10:15:58.986722+0000","last_active":"2024-09-18T10:16:08.649762+0000","last_peered":"2024-09-18T10:16:08.649762+0000","last_clean":"2024-09-18T10:16:08.649762+0000","last_became_active":"2024-09-18T10:14:57.139786+0000","last_became_peered":"2024-09-18T10:14:57.139786+0000","last_unstale":"2024-09-18T10:16:08.649762+0000","last_undegraded":"2024-09-18T10:16:08.649762+0000","last_fullsized":"2024-09-18T10:16:08.649762+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'38","last_scrub_stamp":"2024-09-18T10:15:29.667178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:29.667178+0000","objects_scrubbed":5,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:51.483631+0000","scrub_duration":0.41099292199999998,"objects_trimmed":0,"snaptrim_duration":0.41307451099999998,"stat_sum":{"num_bytes":1310720,"num_objects":5,"num_object_clones":1,"num_object_copies":35,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":8,"num_read_kb":1490,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,14,6,2,5,9,15],"acting":[0,14,6,2,5,9,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":33,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-18T10:16:08.609400+0000","last_change":"2024-09-18T10:16:08.016334+0000","last_active":"2024-09-18T10:16:08.609400+0000","last_peered":"2024-09-18T10:16:08.609400+0000","last_clean":"2024-09-18T10:16:08.609400+0000","last_became_active":"2024-09-18T10:15:50.978055+0000","last_became_peered":"2024-09-18T10:15:50.978055+0000","last_unstale":"2024-09-18T10:16:08.609400+0000","last_undegraded":"2024-09-18T10:16:08.609400+0000","last_fullsized":"2024-09-18T10:16:08.609400+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:14:34.288596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11],"acting":[8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.7","version":"103'78","reported_seq":324,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-18T10:16:08.581859+0000","last_change":"2024-09-18T10:15:59.257162+0000","last_active":"2024-09-18T10:16:08.581859+0000","last_peered":"2024-09-18T10:16:08.581859+0000","last_clean":"2024-09-18T10:16:08.581859+0000","last_became_active":"2024-09-18T10:14:57.137776+0000","last_became_peered":"2024-09-18T10:14:57.137776+0000","last_unstale":"2024-09-18T10:16:08.581859+0000","last_undegraded":"2024-09-18T10:16:08.581859+0000","last_fullsized":"2024-09-18T10:16:08.581859+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"60'30","last_scrub_stamp":"2024-09-18T10:15:17.136535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:17.136535+0000","objects_scrubbed":3,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:35.499009+0000","scrub_duration":0.60680385400000003,"objects_trimmed":1,"snaptrim_duration":0.58309704600000001,"stat_sum":{"num_bytes":4489216,"num_objects":5,"num_object_clones":3,"num_object_copies":35,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":6,"num_read_kb":8019,"num_write":32,"num_write_kb":3440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6,0,14,10,13],"acting":[4,11,6,0,14,10,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":107,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-18T10:16:08.672981+0000","last_change":"2024-09-18T10:15:50.883382+0000","last_active":"2024-09-18T10:16:08.672981+0000","last_peered":"2024-09-18T10:16:08.672981+0000","last_clean":"2024-09-18T10:16:08.672981+0000","last_became_active":"2024-09-18T10:14:35.329198+0000","last_became_peered":"2024-09-18T10:14:35.329198+0000","last_unstale":"2024-09-18T10:16:08.672981+0000","last_undegraded":"2024-09-18T10:16:08.672981+0000","last_fullsized":"2024-09-18T10:16:08.672981+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:15:50.883147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:15:50.883147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:17:15.846467+0000","scrub_duration":0.153222372,"objects_trimmed":0,"snaptrim_duration":0.00062415699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"105'122","reported_seq":175,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-18T10:16:08.609580+0000","last_change":"2024-09-18T10:16:02.144743+0000","last_active":"2024-09-18T10:16:08.609580+0000","last_peered":"2024-09-18T10:16:08.609580+0000","last_clean":"2024-09-18T10:16:08.609580+0000","last_became_active":"2024-09-18T10:15:59.893680+0000","last_became_peered":"2024-09-18T10:15:59.893680+0000","last_unstale":"2024-09-18T10:16:08.609580+0000","last_undegraded":"2024-09-18T10:16:08.609580+0000","last_fullsized":"2024-09-18T10:16:08.609580+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"60'34","last_scrub_stamp":"2024-09-18T10:15:17.325003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:17.325003+0000","objects_scrubbed":6,"log_size":122,"log_dups_size":0,"ondisk_log_size":122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:47.029587+0000","scrub_duration":0.82233752999999998,"objects_trimmed":0,"snaptrim_duration":0.30900760199999999,"stat_sum":{"num_bytes":10207232,"num_objects":11,"num_object_clones":8,"num_object_copies":77,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":1,"num_read":17,"num_read_kb":15011,"num_write":21,"num_write_kb":10976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":12435456,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,15,4,2,6,10,7],"acting":[8,15,4,2,6,10,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":108,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-18T10:16:08.581904+0000","last_change":"2024-09-18T10:16:02.749341+0000","last_active":"2024-09-18T10:16:08.581904+0000","last_peered":"2024-09-18T10:16:08.581904+0000","last_clean":"2024-09-18T10:16:08.581904+0000","last_became_active":"2024-09-18T10:14:35.524102+0000","last_became_peered":"2024-09-18T10:14:35.524102+0000","last_unstale":"2024-09-18T10:16:08.581904+0000","last_undegraded":"2024-09-18T10:16:08.581904+0000","last_fullsized":"2024-09-18T10:16:08.581904+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:16:02.749150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:16:02.749150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:17:04.725472+0000","scrub_duration":0.20368988499999999,"objects_trimmed":0,"snaptrim_duration":0.10077985,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.5","version":"35'2","reported_seq":265,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-18T10:16:08.781316+0000","last_change":"2024-09-18T10:15:58.677092+0000","last_active":"2024-09-18T10:16:08.781316+0000","last_peered":"2024-09-18T10:16:08.781316+0000","last_clean":"2024-09-18T10:16:08.781316+0000","last_became_active":"2024-09-18T10:15:50.780770+0000","last_became_peered":"2024-09-18T10:15:50.780770+0000","last_unstale":"2024-09-18T10:16:08.781316+0000","last_undegraded":"2024-09-18T10:16:08.781316+0000","last_fullsized":"2024-09-18T10:16:08.781316+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"35'2","last_scrub_stamp":"2024-09-18T10:15:03.886316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:03.886316+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:16.496741+0000","scrub_duration":0.35694589500000001,"objects_trimmed":0,"snaptrim_duration":0.0019609720000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,4,9,12,0,11],"acting":[5,8,4,9,12,0,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"103'31","reported_seq":287,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-18T10:16:08.752571+0000","last_change":"2024-09-18T10:16:03.292682+0000","last_active":"2024-09-18T10:16:08.752571+0000","last_peered":"2024-09-18T10:16:08.752571+0000","last_clean":"2024-09-18T10:16:08.752571+0000","last_became_active":"2024-09-18T10:16:03.288737+0000","last_became_peered":"2024-09-18T10:16:03.288737+0000","last_unstale":"2024-09-18T10:16:08.752571+0000","last_undegraded":"2024-09-18T10:16:08.752571+0000","last_fullsized":"2024-09-18T10:16:08.752571+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'23","last_scrub_stamp":"2024-09-18T10:15:28.483344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:28.483344+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.40619440600000001,"objects_trimmed":0,"snaptrim_duration":0.19894819599999999,"stat_sum":{"num_bytes":303104,"num_objects":1,"num_object_clones":0,"num_object_copies":7,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":13,"num_write_kb":1432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4,13,11,2,5],"acting":[10,9,4,13,11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"29'2","reported_seq":96,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-18T10:16:03.704139+0000","last_change":"2024-09-18T10:14:37.429583+0000","last_active":"2024-09-18T10:16:03.704139+0000","last_peered":"2024-09-18T10:16:03.704139+0000","last_clean":"2024-09-18T10:16:03.704139+0000","last_became_active":"2024-09-18T10:14:35.715374+0000","last_became_peered":"2024-09-18T10:14:35.715374+0000","last_unstale":"2024-09-18T10:16:03.704139+0000","last_undegraded":"2024-09-18T10:16:03.704139+0000","last_fullsized":"2024-09-18T10:16:03.704139+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:14:34.288596+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002421224,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,12],"acting":[13,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"3.0","version":"101'40","reported_seq":309,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-18T10:16:08.583665+0000","last_change":"2024-09-18T10:16:04.232027+0000","last_active":"2024-09-18T10:16:08.583665+0000","last_peered":"2024-09-18T10:16:08.583665+0000","last_clean":"2024-09-18T10:16:08.583665+0000","last_became_active":"2024-09-18T10:16:03.292835+0000","last_became_peered":"2024-09-18T10:16:03.292835+0000","last_unstale":"2024-09-18T10:16:08.583665+0000","last_undegraded":"2024-09-18T10:16:08.583665+0000","last_fullsized":"2024-09-18T10:16:08.583665+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"101'40","last_scrub_stamp":"2024-09-18T10:16:04.231663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:16:04.231663+0000","objects_scrubbed":3,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:17:26.955713+0000","scrub_duration":0.41188470300000002,"objects_trimmed":0,"snaptrim_duration":0.0020690270000000002,"stat_sum":{"num_bytes":1982464,"num_objects":3,"num_object_clones":2,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":1,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,9,2,6,15,4],"acting":[3,7,9,2,6,15,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":106,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-18T10:16:09.937382+0000","last_change":"2024-09-18T10:16:07.883194+0000","last_active":"2024-09-18T10:16:09.937382+0000","last_peered":"2024-09-18T10:16:09.937382+0000","last_clean":"2024-09-18T10:16:09.937382+0000","last_became_active":"2024-09-18T10:14:35.348262+0000","last_became_peered":"2024-09-18T10:14:35.348262+0000","last_unstale":"2024-09-18T10:16:09.937382+0000","last_undegraded":"2024-09-18T10:16:09.937382+0000","last_fullsized":"2024-09-18T10:16:09.937382+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:14:34.288596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063430400000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,0],"acting":[14,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"3.1","version":"103'114","reported_seq":378,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-18T10:16:08.581504+0000","last_change":"2024-09-18T10:16:07.139502+0000","last_active":"2024-09-18T10:16:08.581504+0000","last_peered":"2024-09-18T10:16:08.581504+0000","last_clean":"2024-09-18T10:16:08.581504+0000","last_became_active":"2024-09-18T10:16:03.093351+0000","last_became_peered":"2024-09-18T10:16:03.093351+0000","last_unstale":"2024-09-18T10:16:08.581504+0000","last_undegraded":"2024-09-18T10:16:08.581504+0000","last_fullsized":"2024-09-18T10:16:08.581504+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"103'114","last_scrub_stamp":"2024-09-18T10:16:07.139042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:16:07.139042+0000","objects_scrubbed":7,"log_size":114,"log_dups_size":0,"ondisk_log_size":114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:17:33.960661+0000","scrub_duration":0.51157593999999995,"objects_trimmed":1,"snaptrim_duration":0.67973042699999997,"stat_sum":{"num_bytes":933888,"num_objects":7,"num_object_clones":5,"num_object_copies":49,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":1,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,13,14,10,7,3,2],"acting":[4,13,14,10,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":98,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-18T10:16:08.070359+0000","last_change":"2024-09-18T10:14:37.600953+0000","last_active":"2024-09-18T10:16:08.070359+0000","last_peered":"2024-09-18T10:16:08.070359+0000","last_clean":"2024-09-18T10:16:08.070359+0000","last_became_active":"2024-09-18T10:14:35.349203+0000","last_became_peered":"2024-09-18T10:14:35.349203+0000","last_unstale":"2024-09-18T10:16:08.070359+0000","last_undegraded":"2024-09-18T10:16:08.070359+0000","last_fullsized":"2024-09-18T10:16:08.070359+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:14:34.288596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060130600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3],"acting":[9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2","version":"103'133","reported_seq":404,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-18T10:16:09.937519+0000","last_change":"2024-09-18T10:16:04.210460+0000","last_active":"2024-09-18T10:16:09.937519+0000","last_peered":"2024-09-18T10:16:09.937519+0000","last_clean":"2024-09-18T10:16:09.937519+0000","last_became_active":"2024-09-18T10:16:03.186816+0000","last_became_peered":"2024-09-18T10:16:03.186816+0000","last_unstale":"2024-09-18T10:16:09.937519+0000","last_undegraded":"2024-09-18T10:16:09.937519+0000","last_fullsized":"2024-09-18T10:16:09.937519+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"103'133","last_scrub_stamp":"2024-09-18T10:16:04.210203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:16:04.210203+0000","objects_scrubbed":3,"log_size":133,"log_dups_size":0,"ondisk_log_size":133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:17:25.269088+0000","scrub_duration":0.60847910000000005,"objects_trimmed":0,"snaptrim_duration":0.54067356499999997,"stat_sum":{"num_bytes":3588096,"num_objects":3,"num_object_clones":2,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,3,13,1,5,10,7],"acting":[14,3,13,1,5,10,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"2.3","version":"27'1","reported_seq":111,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-18T10:16:09.938477+0000","last_change":"2024-09-18T10:16:06.900858+0000","last_active":"2024-09-18T10:16:09.938477+0000","last_peered":"2024-09-18T10:16:09.938477+0000","last_clean":"2024-09-18T10:16:09.938477+0000","last_became_active":"2024-09-18T10:14:35.487746+0000","last_became_peered":"2024-09-18T10:14:35.487746+0000","last_unstale":"2024-09-18T10:16:09.938477+0000","last_undegraded":"2024-09-18T10:16:09.938477+0000","last_fullsized":"2024-09-18T10:16:09.938477+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'1","last_scrub_stamp":"2024-09-18T10:16:06.900601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:16:06.900601+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:17:25.247813+0000","scrub_duration":0.27078197700000001,"objects_trimmed":0,"snaptrim_duration":0.10072764100000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,5],"acting":[14,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"1.0","version":"29'112","reported_seq":243,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-18T10:16:08.571751+0000","last_change":"2024-09-18T10:16:03.315273+0000","last_active":"2024-09-18T10:16:08.571751+0000","last_peered":"2024-09-18T10:16:08.571751+0000","last_clean":"2024-09-18T10:16:08.571751+0000","last_became_active":"2024-09-18T10:14:32.508037+0000","last_became_peered":"2024-09-18T10:14:32.508037+0000","last_unstale":"2024-09-18T10:16:08.571751+0000","last_undegraded":"2024-09-18T10:16:08.571751+0000","last_fullsized":"2024-09-18T10:16:08.571751+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'112","last_scrub_stamp":"2024-09-18T10:16:03.315126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:31.267741+0000","last_clean_scrub_stamp":"2024-09-18T10:16:03.315126+0000","objects_scrubbed":2,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:17:16.808627+0000","scrub_duration":0.10318229800000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,15],"acting":[6,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.8","version":"103'69","reported_seq":299,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-18T10:16:08.571244+0000","last_change":"2024-09-18T10:15:59.179962+0000","last_active":"2024-09-18T10:16:08.571244+0000","last_peered":"2024-09-18T10:16:08.571244+0000","last_clean":"2024-09-18T10:16:08.571244+0000","last_became_active":"2024-09-18T10:14:57.443754+0000","last_became_peered":"2024-09-18T10:14:57.443754+0000","last_unstale":"2024-09-18T10:16:08.571244+0000","last_undegraded":"2024-09-18T10:16:08.571244+0000","last_fullsized":"2024-09-18T10:16:08.571244+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"41'13","last_scrub_stamp":"2024-09-18T10:15:05.373444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:05.373444+0000","objects_scrubbed":2,"log_size":69,"log_dups_size":0,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:31.932850+0000","scrub_duration":2.8745114210000002,"objects_trimmed":0,"snaptrim_duration":0.510307387,"stat_sum":{"num_bytes":3899392,"num_objects":7,"num_object_clones":5,"num_object_copies":49,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":7,"num_read_kb":706,"num_write":9,"num_write_kb":2640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,11,3,15,1,7],"acting":[6,0,11,3,15,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.9","version":"103'102","reported_seq":430,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-18T10:16:08.682736+0000","last_change":"2024-09-18T10:15:58.779408+0000","last_active":"2024-09-18T10:16:08.682736+0000","last_peered":"2024-09-18T10:16:08.682736+0000","last_clean":"2024-09-18T10:16:08.682736+0000","last_became_active":"2024-09-18T10:15:51.188859+0000","last_became_peered":"2024-09-18T10:15:51.188859+0000","last_unstale":"2024-09-18T10:16:08.682736+0000","last_undegraded":"2024-09-18T10:16:08.682736+0000","last_fullsized":"2024-09-18T10:16:08.682736+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'35","last_scrub_stamp":"2024-09-18T10:15:07.077517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:07.077517+0000","objects_scrubbed":4,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:18.140951+0000","scrub_duration":0.71488955499999995,"objects_trimmed":1,"snaptrim_duration":0.205162702,"stat_sum":{"num_bytes":4325376,"num_objects":5,"num_object_clones":3,"num_object_copies":35,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":1,"num_read":21,"num_read_kb":21422,"num_write":50,"num_write_kb":5392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":12812288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,10,11,5,8,3],"acting":[7,2,10,11,5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"103'74","reported_seq":326,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-18T10:16:06.096918+0000","last_change":"2024-09-18T10:16:06.096555+0000","last_active":"2024-09-18T10:16:06.096918+0000","last_peered":"2024-09-18T10:16:06.096918+0000","last_clean":"2024-09-18T10:16:06.096918+0000","last_became_active":"2024-09-18T10:16:03.093176+0000","last_became_peered":"2024-09-18T10:16:03.093176+0000","last_unstale":"2024-09-18T10:16:06.096918+0000","last_undegraded":"2024-09-18T10:16:06.096918+0000","last_fullsized":"2024-09-18T10:16:06.096918+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"103'74","last_scrub_stamp":"2024-09-18T10:16:06.096260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:16:06.096260+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:17:09.563444+0000","scrub_duration":0.40458393300000001,"objects_trimmed":0,"snaptrim_duration":0.28956907799999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,1,6,5,3,15,4],"acting":[13,1,6,5,3,15,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"1e","length":"3"},{"start":"26","length":"1"},{"start":"29","length":"2"},{"start":"2d","length":"1"},{"start":"2f","length":"2"},{"start":"35","length":"1"}]},{"pgid":"3.b","version":"103'54","reported_seq":282,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-18T10:16:08.582317+0000","last_change":"2024-09-18T10:16:08.031527+0000","last_active":"2024-09-18T10:16:08.582317+0000","last_peered":"2024-09-18T10:16:08.582317+0000","last_clean":"2024-09-18T10:16:08.582317+0000","last_became_active":"2024-09-18T10:16:03.587034+0000","last_became_peered":"2024-09-18T10:16:03.587034+0000","last_unstale":"2024-09-18T10:16:08.582317+0000","last_undegraded":"2024-09-18T10:16:08.582317+0000","last_fullsized":"2024-09-18T10:16:08.582317+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:14:45.576168+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.47418787099999998,"stat_sum":{"num_bytes":2088960,"num_objects":2,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":1,"num_write":27,"num_write_kb":2184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,10,12,11,1,14],"acting":[4,0,10,12,11,1,14],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.c","version":"104'184","reported_seq":507,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-18T10:16:08.681020+0000","last_change":"2024-09-18T10:16:04.308126+0000","last_active":"2024-09-18T10:16:08.681020+0000","last_peered":"2024-09-18T10:16:08.681020+0000","last_clean":"2024-09-18T10:16:08.681020+0000","last_became_active":"2024-09-18T10:16:03.192034+0000","last_became_peered":"2024-09-18T10:16:03.192034+0000","last_unstale":"2024-09-18T10:16:08.681020+0000","last_undegraded":"2024-09-18T10:16:08.681020+0000","last_fullsized":"2024-09-18T10:16:08.681020+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"104'184","last_scrub_stamp":"2024-09-18T10:16:04.307715+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:16:04.307715+0000","objects_scrubbed":4,"log_size":184,"log_dups_size":0,"ondisk_log_size":184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:17:27.469248+0000","scrub_duration":0.60788976500000003,"objects_trimmed":0,"snaptrim_duration":0.50700858199999999,"stat_sum":{"num_bytes":3424256,"num_objects":4,"num_object_clones":2,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":1,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,13,3,7,12,14],"acting":[5,8,13,3,7,12,14],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"103'49","reported_seq":337,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-18T10:16:04.309479+0000","last_change":"2024-09-18T10:16:04.309056+0000","last_active":"2024-09-18T10:16:04.309479+0000","last_peered":"2024-09-18T10:16:04.309479+0000","last_clean":"2024-09-18T10:16:04.309479+0000","last_became_active":"2024-09-18T10:16:03.284499+0000","last_became_peered":"2024-09-18T10:16:03.284499+0000","last_unstale":"2024-09-18T10:16:04.309479+0000","last_undegraded":"2024-09-18T10:16:04.309479+0000","last_fullsized":"2024-09-18T10:16:04.309479+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"103'49","last_scrub_stamp":"2024-09-18T10:16:04.308714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:16:04.308714+0000","objects_scrubbed":4,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:17:29.880655+0000","scrub_duration":0.61971236900000004,"objects_trimmed":0,"snaptrim_duration":0.58533100999999998,"stat_sum":{"num_bytes":3817472,"num_objects":4,"num_object_clones":3,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,1,11,15,9,8,4],"acting":[13,1,11,15,9,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"1e","length":"3"},{"start":"26","length":"1"},{"start":"29","length":"2"},{"start":"2d","length":"1"},{"start":"2f","length":"2"},{"start":"35","length":"1"}]},{"pgid":"3.e","version":"103'74","reported_seq":376,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-18T10:16:08.651365+0000","last_change":"2024-09-18T10:16:04.898485+0000","last_active":"2024-09-18T10:16:08.651365+0000","last_peered":"2024-09-18T10:16:08.651365+0000","last_clean":"2024-09-18T10:16:08.651365+0000","last_became_active":"2024-09-18T10:16:03.289917+0000","last_became_peered":"2024-09-18T10:16:03.289917+0000","last_unstale":"2024-09-18T10:16:08.651365+0000","last_undegraded":"2024-09-18T10:16:08.651365+0000","last_fullsized":"2024-09-18T10:16:08.651365+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"103'74","last_scrub_stamp":"2024-09-18T10:16:04.898232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:16:04.898232+0000","objects_scrubbed":2,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:17:28.205150+0000","scrub_duration":0.60700314200000005,"objects_trimmed":0,"snaptrim_duration":0.20657598799999999,"stat_sum":{"num_bytes":2752512,"num_objects":2,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":1,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,14,6,13,8,3,12],"acting":[11,14,6,13,8,3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.f","version":"103'64","reported_seq":297,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-18T10:16:05.001575+0000","last_change":"2024-09-18T10:16:05.001256+0000","last_active":"2024-09-18T10:16:05.001575+0000","last_peered":"2024-09-18T10:16:05.001575+0000","last_clean":"2024-09-18T10:16:05.001575+0000","last_became_active":"2024-09-18T10:16:03.586315+0000","last_became_peered":"2024-09-18T10:16:03.586315+0000","last_unstale":"2024-09-18T10:16:05.001575+0000","last_undegraded":"2024-09-18T10:16:05.001575+0000","last_fullsized":"2024-09-18T10:16:05.001575+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"103'64","last_scrub_stamp":"2024-09-18T10:16:05.001024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:16:05.001024+0000","objects_scrubbed":1,"log_size":64,"log_dups_size":0,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:17:26.257200+0000","scrub_duration":0.31694162199999998,"objects_trimmed":1,"snaptrim_duration":0.22163957500000001,"stat_sum":{"num_bytes":2277376,"num_objects":1,"num_object_clones":0,"num_object_copies":7,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,3,15,14,2,11,0],"acting":[13,3,15,14,2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"1e","length":"3"},{"start":"26","length":"1"},{"start":"29","length":"2"},{"start":"2d","length":"1"},{"start":"2f","length":"2"},{"start":"35","length":"1"}]},{"pgid":"3.10","version":"103'43","reported_seq":296,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-18T10:16:08.571619+0000","last_change":"2024-09-18T10:15:58.682011+0000","last_active":"2024-09-18T10:16:08.571619+0000","last_peered":"2024-09-18T10:16:08.571619+0000","last_clean":"2024-09-18T10:16:08.571619+0000","last_became_active":"2024-09-18T10:15:50.787275+0000","last_became_peered":"2024-09-18T10:15:50.787275+0000","last_unstale":"2024-09-18T10:16:08.571619+0000","last_undegraded":"2024-09-18T10:16:08.571619+0000","last_fullsized":"2024-09-18T10:16:08.571619+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'19","last_scrub_stamp":"2024-09-18T10:15:29.719467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:29.719467+0000","objects_scrubbed":1,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:30.348118+0000","scrub_duration":2.0403324629999999,"objects_trimmed":0,"snaptrim_duration":0.112609244,"stat_sum":{"num_bytes":5488640,"num_objects":3,"num_object_clones":2,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":6,"num_read_kb":16083,"num_write":9,"num_write_kb":3712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":21331968,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,5,4,8,0,9],"acting":[6,7,5,4,8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"36'6","reported_seq":186,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-18T10:16:08.681483+0000","last_change":"2024-09-18T10:15:58.771680+0000","last_active":"2024-09-18T10:16:08.681483+0000","last_peered":"2024-09-18T10:16:08.681483+0000","last_clean":"2024-09-18T10:16:08.681483+0000","last_became_active":"2024-09-18T10:15:04.955869+0000","last_became_peered":"2024-09-18T10:15:04.955869+0000","last_unstale":"2024-09-18T10:16:08.681483+0000","last_undegraded":"2024-09-18T10:16:08.681483+0000","last_fullsized":"2024-09-18T10:16:08.681483+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":5,"last_scrub":"36'6","last_scrub_stamp":"2024-09-18T10:15:07.755163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:07.755163+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:13.113623+0000","scrub_duration":0.60886176599999997,"objects_trimmed":0,"snaptrim_duration":0.101449467,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,13,10,0,14,2],"acting":[3,5,13,10,0,14,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"36'8","reported_seq":290,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-18T10:16:09.937402+0000","last_change":"2024-09-18T10:15:58.564395+0000","last_active":"2024-09-18T10:16:09.937402+0000","last_peered":"2024-09-18T10:16:09.937402+0000","last_clean":"2024-09-18T10:16:09.937402+0000","last_became_active":"2024-09-18T10:15:50.682909+0000","last_became_peered":"2024-09-18T10:15:50.682909+0000","last_unstale":"2024-09-18T10:16:09.937402+0000","last_undegraded":"2024-09-18T10:16:09.937402+0000","last_fullsized":"2024-09-18T10:16:09.937402+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":5,"last_scrub":"36'8","last_scrub_stamp":"2024-09-18T10:15:04.353687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:04.353687+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:11.257401+0000","scrub_duration":0.76511709400000005,"objects_trimmed":0,"snaptrim_duration":0.001712454,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,5,8,3,4,11,1],"acting":[14,5,8,3,4,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"3.13","version":"93'52","reported_seq":256,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-18T10:16:08.582096+0000","last_change":"2024-09-18T10:15:58.576553+0000","last_active":"2024-09-18T10:16:08.582096+0000","last_peered":"2024-09-18T10:16:08.582096+0000","last_clean":"2024-09-18T10:16:08.582096+0000","last_became_active":"2024-09-18T10:15:13.263306+0000","last_became_peered":"2024-09-18T10:15:13.263306+0000","last_unstale":"2024-09-18T10:16:08.582096+0000","last_undegraded":"2024-09-18T10:16:08.582096+0000","last_fullsized":"2024-09-18T10:16:08.582096+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":5,"last_scrub":"60'33","last_scrub_stamp":"2024-09-18T10:15:19.140411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:45.576168+0000","last_clean_scrub_stamp":"2024-09-18T10:15:19.140411+0000","objects_scrubbed":1,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:39.399176+0000","scrub_duration":0.706777345,"objects_trimmed":0,"snaptrim_duration":0.001165826,"stat_sum":{"num_bytes":1720320,"num_objects":2,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,13,2,6,0,4,15],"acting":[1,13,2,6,0,4,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":36,"stat_sum":{"num_bytes":124747776,"num_objects":127,"num_object_clones":80,"num_object_copies":889,"num_objects_missing_on_primary":7,"num_objects_missing":7,"num_objects_degraded":0,"num_objects_misplaced":35,"num_objects_unfound":0,"num_objects_dirty":127,"num_whiteouts":11,"num_read":133,"num_read_kb":104868,"num_write":323,"num_write_kb":60792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":93257728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":240689152,"data_stored":324870144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2492,"ondisk_log_size":2492,"up":252,"acting":252,"num_store_stats":16},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":13},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":112,"ondisk_log_size":112,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":15,"up_from":22,"seq":94489280534,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312276,"kb_used_data":10760,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94059564,"statfs":{"total":96636764160,"available":96316993536,"internally_reserved":0,"allocated":11018240,"data_stored":13458279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,4,6,7,8,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:15:31 2024","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":1.3939999999999999},{"interface":"front","average":{"1min":6.7190000000000003,"5min":6.7190000000000003,"15min":6.7190000000000003},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":101.291,"5min":101.291,"15min":101.291},"last":0.876}]},{"osd":1,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.70199999999999996},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.931,"5min":1.931,"15min":1.931},"last":0.67300000000000004}]},{"osd":2,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.417,"5min":1.417,"15min":1.417},"last":0.96799999999999997},{"interface":"front","average":{"1min":5.5389999999999997,"5min":5.5389999999999997,"15min":5.5389999999999997},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":78.242999999999995,"5min":78.242999999999995,"15min":78.242999999999995},"last":1.046}]},{"osd":3,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":3.3029999999999999,"5min":3.3029999999999999,"15min":3.3029999999999999},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":41.720999999999997,"5min":41.720999999999997,"15min":41.720999999999997},"last":0.55400000000000005},{"interface":"front","average":{"1min":1.014,"5min":1.014,"15min":1.014},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.49,"5min":1.49,"15min":1.49},"last":0.89500000000000002}]},{"osd":4,"last update":"Wed Sep 18 10:15:31 2024","interfaces":[{"interface":"back","average":{"1min":11.021000000000001,"5min":11.021000000000001,"15min":11.021000000000001},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":99.745999999999995,"5min":99.745999999999995,"15min":99.745999999999995},"last":0.82399999999999995},{"interface":"front","average":{"1min":11.140000000000001,"5min":11.140000000000001,"15min":11.140000000000001},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":99.781000000000006,"5min":99.781000000000006,"15min":99.781000000000006},"last":1.2789999999999999}]},{"osd":5,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":1.028},{"interface":"front","average":{"1min":1.7869999999999999,"5min":1.7869999999999999,"15min":1.7869999999999999},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":10.874000000000001,"5min":10.874000000000001,"15min":10.874000000000001},"last":0.92400000000000004}]},{"osd":6,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":1.9710000000000001,"5min":1.9710000000000001,"15min":1.9710000000000001},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":12.367000000000001,"5min":12.367000000000001,"15min":12.367000000000001},"last":1.085},{"interface":"front","average":{"1min":5.6109999999999998,"5min":5.6109999999999998,"15min":5.6109999999999998},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":74.156000000000006,"5min":74.156000000000006,"15min":74.156000000000006},"last":1.2230000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.464}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.014}]},{"osd":10,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":6.3680000000000003,"5min":6.3680000000000003,"15min":6.3680000000000003},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":86.546999999999997,"5min":86.546999999999997,"15min":86.546999999999997},"last":1.4410000000000001},{"interface":"front","average":{"1min":6.375,"5min":6.375,"15min":6.375},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":86.497,"5min":86.497,"15min":86.497},"last":1.1100000000000001}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.502}]},{"osd":13,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":6.0819999999999999,"5min":6.0819999999999999,"15min":6.0819999999999999},"min":{"1min":1.093,"5min":1.093,"15min":1.093},"max":{"1min":80.197000000000003,"5min":80.197000000000003,"15min":80.197000000000003},"last":1.659},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.8420000000000001,"5min":1.8420000000000001,"15min":1.8420000000000001},"last":1.589}]},{"osd":14,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":1.444,"5min":1.444,"15min":1.444},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":2.661,"5min":2.661,"15min":2.661},"last":1.3120000000000001},{"interface":"front","average":{"1min":10.750999999999999,"5min":10.750999999999999,"15min":10.750999999999999},"min":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"max":{"1min":85.006,"5min":85.006,"15min":85.006},"last":1.611}]}]},{"osd":14,"up_from":22,"seq":94489280534,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":325508,"kb_used_data":23992,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94046332,"statfs":{"total":96636764160,"available":96303443968,"internally_reserved":0,"allocated":24567808,"data_stored":33147207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,3,4,5,7,8,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":15.391,"5min":15.391,"15min":15.391},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":189.88800000000001,"5min":189.88800000000001,"15min":189.88800000000001},"last":1.3919999999999999},{"interface":"front","average":{"1min":19.533000000000001,"5min":19.533000000000001,"15min":19.533000000000001},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":180.09999999999999,"5min":180.09999999999999,"15min":180.09999999999999},"last":0.84199999999999997}]},{"osd":1,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":9.1829999999999998,"5min":9.1829999999999998,"15min":9.1829999999999998},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":79.626000000000005,"5min":79.626000000000005,"15min":79.626000000000005},"last":1.3160000000000001},{"interface":"front","average":{"1min":15.465,"5min":15.465,"15min":15.465},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":189.762,"5min":189.762,"15min":189.762},"last":1.2509999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.417}]},{"osd":3,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":15.247,"5min":15.247,"15min":15.247},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":180.19900000000001,"5min":180.19900000000001,"15min":180.19900000000001},"last":1.131},{"interface":"front","average":{"1min":21.341999999999999,"5min":21.341999999999999,"15min":21.341999999999999},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":180.851,"5min":180.851,"15min":180.851},"last":0.95399999999999996}]},{"osd":4,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":25.399999999999999,"5min":25.399999999999999,"15min":25.399999999999999},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":189.809,"5min":189.809,"15min":189.809},"last":0.91300000000000003},{"interface":"front","average":{"1min":15.281000000000001,"5min":15.281000000000001,"15min":15.281000000000001},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":180.31700000000001,"5min":180.31700000000001,"15min":180.31700000000001},"last":1.4630000000000001}]},{"osd":5,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":26.425999999999998,"5min":26.425999999999998,"15min":26.425999999999998},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":277.41199999999998,"5min":277.41199999999998,"15min":277.41199999999998},"last":1.5069999999999999},{"interface":"front","average":{"1min":15.683,"5min":15.683,"15min":15.683},"min":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":189.64699999999999,"5min":189.64699999999999,"15min":189.64699999999999},"last":1.016}]},{"osd":7,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":15.460000000000001,"5min":15.460000000000001,"15min":15.460000000000001},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":180.268,"5min":180.268,"15min":180.268},"last":1.552},{"interface":"front","average":{"1min":15.154,"5min":15.154,"15min":15.154},"min":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":180.631,"5min":180.631,"15min":180.631},"last":1.2070000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.633}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7090000000000001}]},{"osd":10,"last update":"Wed Sep 18 10:15:49 2024","interfaces":[{"interface":"back","average":{"1min":17.039000000000001,"5min":17.039000000000001,"15min":17.039000000000001},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":180.15899999999999,"5min":180.15899999999999,"15min":180.15899999999999},"last":1.802},{"interface":"front","average":{"1min":31.004000000000001,"5min":31.004000000000001,"15min":31.004000000000001},"min":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":269.423,"5min":269.423,"15min":269.423},"last":1.9239999999999999}]},{"osd":11,"last update":"Wed Sep 18 10:16:10 2024","interfaces":[{"interface":"back","average":{"1min":9.6890000000000001,"5min":9.6890000000000001,"15min":9.6890000000000001},"min":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":91.111999999999995,"5min":91.111999999999995,"15min":91.111999999999995},"last":1.675},{"interface":"front","average":{"1min":8.1460000000000008,"5min":8.1460000000000008,"15min":8.1460000000000008},"min":{"1min":1.181,"5min":1.181,"15min":1.181},"max":{"1min":91.165000000000006,"5min":91.165000000000006,"15min":91.165000000000006},"last":1.589}]},{"osd":12,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":15.855,"5min":15.855,"15min":15.855},"min":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"max":{"1min":189.84999999999999,"5min":189.84999999999999,"15min":189.84999999999999},"last":1.8480000000000001},{"interface":"front","average":{"1min":15.666,"5min":15.666,"15min":15.666},"min":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"max":{"1min":180.381,"5min":180.381,"15min":180.381},"last":1.9710000000000001}]},{"osd":13,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":26.244,"5min":26.244,"15min":26.244},"min":{"1min":0.875,"5min":0.875,"15min":0.875},"max":{"1min":180.59100000000001,"5min":180.59100000000001,"15min":180.59100000000001},"last":1.772},{"interface":"front","average":{"1min":25.079000000000001,"5min":25.079000000000001,"15min":25.079000000000001},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":189.71299999999999,"5min":189.71299999999999,"15min":189.71299999999999},"last":1.736}]},{"osd":15,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":15.680999999999999,"5min":15.680999999999999,"15min":15.680999999999999},"min":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"max":{"1min":180.44800000000001,"5min":180.44800000000001,"15min":180.44800000000001},"last":1.8740000000000001},{"interface":"front","average":{"1min":18.472000000000001,"5min":18.472000000000001,"15min":18.472000000000001},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":180.53,"5min":180.53,"15min":180.53},"last":1.829}]}]},{"osd":13,"up_from":22,"seq":94489280534,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":325916,"kb_used_data":24400,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94045924,"statfs":{"total":96636764160,"available":96303026176,"internally_reserved":0,"allocated":24985600,"data_stored":33614612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,6,8,9,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":20.742000000000001,"5min":20.742000000000001,"15min":20.742000000000001},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":176.66800000000001,"5min":176.66800000000001,"15min":176.66800000000001},"last":1.0329999999999999},{"interface":"front","average":{"1min":30.433,"5min":30.433,"15min":30.433},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":199.732,"5min":199.732,"15min":199.732},"last":1.0509999999999999}]},{"osd":1,"last update":"Wed Sep 18 10:15:48 2024","interfaces":[{"interface":"back","average":{"1min":1.986,"5min":1.986,"15min":1.986},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":18.471,"5min":18.471,"15min":18.471},"last":1.1140000000000001},{"interface":"front","average":{"1min":20.716999999999999,"5min":20.716999999999999,"15min":20.716999999999999},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":176.56800000000001,"5min":176.56800000000001,"15min":176.56800000000001},"last":1.274}]},{"osd":2,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":11.984,"5min":11.984,"15min":11.984},"min":{"1min":0.751,"5min":0.751,"15min":0.751},"max":{"1min":196.59200000000001,"5min":196.59200000000001,"15min":196.59200000000001},"last":1.0069999999999999},{"interface":"front","average":{"1min":26.109999999999999,"5min":26.109999999999999,"15min":26.109999999999999},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":177.03,"5min":177.03,"15min":177.03},"last":1.135}]},{"osd":3,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":25.431999999999999,"5min":25.431999999999999,"15min":25.431999999999999},"min":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"max":{"1min":168.27699999999999,"5min":168.27699999999999,"15min":168.27699999999999},"last":0.77200000000000002},{"interface":"front","average":{"1min":12.045999999999999,"5min":12.045999999999999,"15min":12.045999999999999},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":196.643,"5min":196.643,"15min":196.643},"last":0.84499999999999997}]},{"osd":4,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":29.802,"5min":29.802,"15min":29.802},"min":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"max":{"1min":168.48099999999999,"5min":168.48099999999999,"15min":168.48099999999999},"last":1.3109999999999999},{"interface":"front","average":{"1min":8.1780000000000008,"5min":8.1780000000000008,"15min":8.1780000000000008},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":65.188000000000002,"5min":65.188000000000002,"15min":65.188000000000002},"last":1.298}]},{"osd":5,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":26.152999999999999,"5min":26.152999999999999,"15min":26.152999999999999},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":176.48099999999999,"5min":176.48099999999999,"15min":176.48099999999999},"last":0.89500000000000002},{"interface":"front","average":{"1min":24.885999999999999,"5min":24.885999999999999,"15min":24.885999999999999},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":167.62299999999999,"5min":167.62299999999999,"15min":167.62299999999999},"last":0.93899999999999995}]},{"osd":6,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":29.355,"5min":29.355,"15min":29.355},"min":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":176.81200000000001,"5min":176.81200000000001,"15min":176.81200000000001},"last":1.2},{"interface":"front","average":{"1min":27.855,"5min":27.855,"15min":27.855},"min":{"1min":1.24,"5min":1.24,"15min":1.24},"max":{"1min":167.947,"5min":167.947,"15min":167.947},"last":1.375}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.238}]},{"osd":9,"last update":"Wed Sep 18 10:15:48 2024","interfaces":[{"interface":"back","average":{"1min":21.055,"5min":21.055,"15min":21.055},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":176.73599999999999,"5min":176.73599999999999,"15min":176.73599999999999},"last":1.341},{"interface":"front","average":{"1min":23.465,"5min":23.465,"15min":23.465},"min":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"max":{"1min":167.84899999999999,"5min":167.84899999999999,"15min":167.84899999999999},"last":1.4510000000000001}]},{"osd":10,"last update":"Wed Sep 18 10:15:39 2024","interfaces":[{"interface":"back","average":{"1min":22.678999999999998,"5min":22.678999999999998,"15min":22.678999999999998},"min":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"max":{"1min":168.07900000000001,"5min":168.07900000000001,"15min":168.07900000000001},"last":1.5089999999999999},{"interface":"front","average":{"1min":31.972000000000001,"5min":31.972000000000001,"15min":31.972000000000001},"min":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"max":{"1min":196.37700000000001,"5min":196.37700000000001,"15min":196.37700000000001},"last":1.3600000000000001}]},{"osd":11,"last update":"Wed Sep 18 10:15:48 2024","interfaces":[{"interface":"back","average":{"1min":1.611,"5min":1.611,"15min":1.611},"min":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"max":{"1min":2.391,"5min":2.391,"15min":2.391},"last":1.482},{"interface":"front","average":{"1min":26.463999999999999,"5min":26.463999999999999,"15min":26.463999999999999},"min":{"1min":1.028,"5min":1.028,"15min":1.028},"max":{"1min":176.94999999999999,"5min":176.94999999999999,"15min":176.94999999999999},"last":1.401}]},{"osd":12,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":1.5820000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"min":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"max":{"1min":2.7879999999999998,"5min":2.7879999999999998,"15min":2.7879999999999998},"last":1.419},{"interface":"front","average":{"1min":21.914999999999999,"5min":21.914999999999999,"15min":21.914999999999999},"min":{"1min":1.296,"5min":1.296,"15min":1.296},"max":{"1min":168.34399999999999,"5min":168.34399999999999,"15min":168.34399999999999},"last":1.573}]},{"osd":14,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":6.694,"5min":6.694,"15min":6.694},"min":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"max":{"1min":87.427000000000007,"5min":87.427000000000007,"15min":87.427000000000007},"last":1.54},{"interface":"front","average":{"1min":26.231999999999999,"5min":26.231999999999999,"15min":26.231999999999999},"min":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"max":{"1min":176.87700000000001,"5min":176.87700000000001,"15min":176.87700000000001},"last":1.4410000000000001}]},{"osd":15,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":20.334,"5min":20.334,"15min":20.334},"min":{"1min":1.266,"5min":1.266,"15min":1.266},"max":{"1min":168.40700000000001,"5min":168.40700000000001,"15min":168.40700000000001},"last":1.6419999999999999},{"interface":"front","average":{"1min":16.350999999999999,"5min":16.350999999999999,"15min":16.350999999999999},"min":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"max":{"1min":196.512,"5min":196.512,"15min":196.512},"last":1.6180000000000001}]}]},{"osd":0,"up_from":21,"seq":90194313238,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311216,"kb_used_data":9692,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94060624,"statfs":{"total":96636764160,"available":96318078976,"internally_reserved":0,"allocated":9924608,"data_stored":12607862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,4,5,6,9,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 18 10:15:34 2024","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.341,"5min":1.341,"15min":1.341},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.4630000000000001,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":1.0069999999999999}]},{"osd":2,"last update":"Wed Sep 18 10:15:34 2024","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":3.7269999999999999,"5min":3.7269999999999999,"15min":3.7269999999999999},"last":1.0449999999999999},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.528,"5min":1.528,"15min":1.528},"last":0.85999999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.196}]},{"osd":4,"last update":"Wed Sep 18 10:15:29 2024","interfaces":[{"interface":"back","average":{"1min":15.4,"5min":15.4,"15min":15.4},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":130.28,"5min":130.28,"15min":130.28},"last":0.71499999999999997},{"interface":"front","average":{"1min":16,"5min":16,"15min":16},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":79.914000000000001,"5min":79.914000000000001,"15min":79.914000000000001},"last":1.167}]},{"osd":5,"last update":"Wed Sep 18 10:15:34 2024","interfaces":[{"interface":"back","average":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"min":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"max":{"1min":1.6819999999999999,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":1.333},{"interface":"front","average":{"1min":9.7759999999999998,"5min":9.7759999999999998,"15min":9.7759999999999998},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":130.39599999999999,"5min":130.39599999999999,"15min":130.39599999999999},"last":0.99099999999999999}]},{"osd":6,"last update":"Wed Sep 18 10:15:47 2024","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"min":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"max":{"1min":1.776,"5min":1.776,"15min":1.776},"last":1.2589999999999999},{"interface":"front","average":{"1min":2.2069999999999999,"5min":2.2069999999999999,"15min":2.2069999999999999},"min":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"max":{"1min":15.250999999999999,"5min":15.250999999999999,"15min":15.250999999999999},"last":1.214}]},{"osd":9,"last update":"Wed Sep 18 10:15:29 2024","interfaces":[{"interface":"back","average":{"1min":1.407,"5min":1.407,"15min":1.407},"min":{"1min":0.879,"5min":0.879,"15min":0.879},"max":{"1min":1.855,"5min":1.855,"15min":1.855},"last":1.4450000000000001},{"interface":"front","average":{"1min":10.532999999999999,"5min":10.532999999999999,"15min":10.532999999999999},"min":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"max":{"1min":130.441,"5min":130.441,"15min":130.441},"last":0.93600000000000005}]},{"osd":11,"last update":"Wed Sep 18 10:15:29 2024","interfaces":[{"interface":"back","average":{"1min":1.54,"5min":1.54,"15min":1.54},"min":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"max":{"1min":2.0209999999999999,"5min":2.0209999999999999,"15min":2.0209999999999999},"last":1.4850000000000001},{"interface":"front","average":{"1min":5.327,"5min":5.327,"15min":5.327},"min":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":54.805999999999997,"5min":54.805999999999997,"15min":54.805999999999997},"last":1.544}]},{"osd":12,"last update":"Wed Sep 18 10:15:29 2024","interfaces":[{"interface":"back","average":{"1min":10.949999999999999,"5min":10.949999999999999,"15min":10.949999999999999},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":130.34800000000001,"5min":130.34800000000001,"15min":130.34800000000001},"last":1.0940000000000001},{"interface":"front","average":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.9139999999999999,"5min":1.9139999999999999,"15min":1.9139999999999999},"last":1.3740000000000001}]},{"osd":14,"last update":"Wed Sep 18 10:15:34 2024","interfaces":[{"interface":"back","average":{"1min":10.039999999999999,"5min":10.039999999999999,"15min":10.039999999999999},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":130.48500000000001,"5min":130.48500000000001,"15min":130.48500000000001},"last":1.149},{"interface":"front","average":{"1min":1.5600000000000001,"5min":1.5600000000000001,"15min":1.5600000000000001},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":2.1819999999999999,"5min":2.1819999999999999,"15min":2.1819999999999999},"last":1.4219999999999999}]},{"osd":15,"last update":"Wed Sep 18 10:15:34 2024","interfaces":[{"interface":"back","average":{"1min":3.0750000000000002,"5min":3.0750000000000002,"15min":3.0750000000000002},"min":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":23.806999999999999,"5min":23.806999999999999,"15min":23.806999999999999},"last":1.621},{"interface":"front","average":{"1min":10.122,"5min":10.122,"15min":10.122},"min":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":130.52799999999999,"5min":130.52799999999999,"15min":130.52799999999999},"last":1.3069999999999999}]}]},{"osd":4,"up_from":21,"seq":90194313240,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":321572,"kb_used_data":20056,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94050268,"statfs":{"total":96636764160,"available":96307474432,"internally_reserved":0,"allocated":20537344,"data_stored":26605895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,5,6,7,8,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:15:29 2024","interfaces":[{"interface":"back","average":{"1min":11.151,"5min":11.151,"15min":11.151},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":89.266999999999996,"5min":89.266999999999996,"15min":89.266999999999996},"last":0.91700000000000004},{"interface":"front","average":{"1min":5.0049999999999999,"5min":5.0049999999999999,"15min":5.0049999999999999},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":46.898000000000003,"5min":46.898000000000003,"15min":46.898000000000003},"last":0.99399999999999999}]},{"osd":1,"last update":"Wed Sep 18 10:15:34 2024","interfaces":[{"interface":"back","average":{"1min":14.653,"5min":14.653,"15min":14.653},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":164.21199999999999,"5min":164.21199999999999,"15min":164.21199999999999},"last":1.0549999999999999},{"interface":"front","average":{"1min":11.363,"5min":11.363,"15min":11.363},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":89.331000000000003,"5min":89.331000000000003,"15min":89.331000000000003},"last":1.379}]},{"osd":2,"last update":"Wed Sep 18 10:15:47 2024","interfaces":[{"interface":"back","average":{"1min":23.617000000000001,"5min":23.617000000000001,"15min":23.617000000000001},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":192.136,"5min":192.136,"15min":192.136},"last":1.585},{"interface":"front","average":{"1min":11.888999999999999,"5min":11.888999999999999,"15min":11.888999999999999},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":198.87700000000001,"5min":198.87700000000001,"15min":198.87700000000001},"last":1.6930000000000001}]},{"osd":3,"last update":"Wed Sep 18 10:15:34 2024","interfaces":[{"interface":"back","average":{"1min":18.170999999999999,"5min":18.170999999999999,"15min":18.170999999999999},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":137.053,"5min":137.053,"15min":137.053},"last":1.474},{"interface":"front","average":{"1min":27.619,"5min":27.619,"15min":27.619},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":203.87899999999999,"5min":203.87899999999999,"15min":203.87899999999999},"last":1.349}]},{"osd":5,"last update":"Wed Sep 18 10:15:34 2024","interfaces":[{"interface":"back","average":{"1min":18.940000000000001,"5min":18.940000000000001,"15min":18.940000000000001},"min":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"max":{"1min":189.55600000000001,"5min":189.55600000000001,"15min":189.55600000000001},"last":1.9730000000000001},{"interface":"front","average":{"1min":3.2469999999999999,"5min":3.2469999999999999,"15min":3.2469999999999999},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":47.518000000000001,"5min":47.518000000000001,"15min":47.518000000000001},"last":1.8129999999999999}]},{"osd":6,"last update":"Wed Sep 18 10:15:34 2024","interfaces":[{"interface":"back","average":{"1min":32.25,"5min":32.25,"15min":32.25},"min":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":197.33199999999999,"5min":197.33199999999999,"15min":197.33199999999999},"last":1.889},{"interface":"front","average":{"1min":18.963000000000001,"5min":18.963000000000001,"15min":18.963000000000001},"min":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"max":{"1min":179.19200000000001,"5min":179.19200000000001,"15min":179.19200000000001},"last":1.923}]},{"osd":7,"last update":"Wed Sep 18 10:15:34 2024","interfaces":[{"interface":"back","average":{"1min":27.597000000000001,"5min":27.597000000000001,"15min":27.597000000000001},"min":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"max":{"1min":290.036,"5min":290.036,"15min":290.036},"last":1.516},{"interface":"front","average":{"1min":17.425999999999998,"5min":17.425999999999998,"15min":17.425999999999998},"min":{"1min":1.173,"5min":1.173,"15min":1.173},"max":{"1min":139.696,"5min":139.696,"15min":139.696},"last":2.0979999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.772}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.008}]},{"osd":10,"last update":"Wed Sep 18 10:15:46 2024","interfaces":[{"interface":"back","average":{"1min":22.786000000000001,"5min":22.786000000000001,"15min":22.786000000000001},"min":{"1min":1.079,"5min":1.079,"15min":1.079},"max":{"1min":164.43899999999999,"5min":164.43899999999999,"15min":164.43899999999999},"last":1.744},{"interface":"front","average":{"1min":12.59,"5min":12.59,"15min":12.59},"min":{"1min":0.878,"5min":0.878,"15min":0.878},"max":{"1min":89.442999999999998,"5min":89.442999999999998,"15min":89.442999999999998},"last":2.181}]},{"osd":11,"last update":"Wed Sep 18 10:15:29 2024","interfaces":[{"interface":"back","average":{"1min":10.817,"5min":10.817,"15min":10.817},"min":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"max":{"1min":101.666,"5min":101.666,"15min":101.666},"last":2.1469999999999998},{"interface":"front","average":{"1min":24.919,"5min":24.919,"15min":24.919},"min":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"max":{"1min":164.733,"5min":164.733,"15min":164.733},"last":1.655}]},{"osd":12,"last update":"Wed Sep 18 10:15:29 2024","interfaces":[{"interface":"back","average":{"1min":11.863,"5min":11.863,"15min":11.863},"min":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":89.400000000000006,"5min":89.400000000000006,"15min":89.400000000000006},"last":2.2959999999999998},{"interface":"front","average":{"1min":8.1699999999999999,"5min":8.1699999999999999,"15min":8.1699999999999999},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":101.699,"5min":101.699,"15min":101.699},"last":2.2229999999999999}]},{"osd":13,"last update":"Wed Sep 18 10:15:47 2024","interfaces":[{"interface":"back","average":{"1min":25.314,"5min":25.314,"15min":25.314},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":199.37799999999999,"5min":199.37799999999999,"15min":199.37799999999999},"last":2.3130000000000002},{"interface":"front","average":{"1min":23.385000000000002,"5min":23.385000000000002,"15min":23.385000000000002},"min":{"1min":1.022,"5min":1.022,"15min":1.022},"max":{"1min":164.53,"5min":164.53,"15min":164.53},"last":1.8480000000000001}]},{"osd":14,"last update":"Wed Sep 18 10:15:34 2024","interfaces":[{"interface":"back","average":{"1min":21.658000000000001,"5min":21.658000000000001,"15min":21.658000000000001},"min":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"max":{"1min":202.191,"5min":202.191,"15min":202.191},"last":2.2709999999999999},{"interface":"front","average":{"1min":19.306999999999999,"5min":19.306999999999999,"15min":19.306999999999999},"min":{"1min":1.163,"5min":1.163,"15min":1.163},"max":{"1min":164.35400000000001,"5min":164.35400000000001,"15min":164.35400000000001},"last":2.121}]}]},{"osd":9,"up_from":21,"seq":90194313238,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":314276,"kb_used_data":12752,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94057564,"statfs":{"total":96636764160,"available":96314945536,"internally_reserved":0,"allocated":13058048,"data_stored":15237494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,4,5,6,8,10,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":7.2450000000000001,"5min":7.2450000000000001,"15min":7.2450000000000001},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":127.902,"5min":127.902,"15min":127.902},"last":0.97399999999999998},{"interface":"front","average":{"1min":8.3109999999999999,"5min":8.3109999999999999,"15min":8.3109999999999999},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":96.203000000000003,"5min":96.203000000000003,"15min":96.203000000000003},"last":0.81200000000000006}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70599999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92500000000000004}]},{"osd":3,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":6.226,"5min":6.226,"15min":6.226},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":98.888999999999996,"5min":98.888999999999996,"15min":98.888999999999996},"last":1.133},{"interface":"front","average":{"1min":6.2320000000000002,"5min":6.2320000000000002,"15min":6.2320000000000002},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":98.947999999999993,"5min":98.947999999999993,"15min":98.947999999999993},"last":1.1719999999999999}]},{"osd":4,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"min":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"max":{"1min":1.5780000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.86799999999999999},{"interface":"front","average":{"1min":9.5090000000000003,"5min":9.5090000000000003,"15min":9.5090000000000003},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":71.366,"5min":71.366,"15min":71.366},"last":1.2709999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0329999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.075}]},{"osd":8,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":4.5910000000000002,"5min":4.5910000000000002,"15min":4.5910000000000002},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":64.912999999999997,"5min":64.912999999999997,"15min":64.912999999999997},"last":1.2350000000000001},{"interface":"front","average":{"1min":6.2309999999999999,"5min":6.2309999999999999,"15min":6.2309999999999999},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":57.664000000000001,"5min":57.664000000000001,"15min":57.664000000000001},"last":1.349}]},{"osd":10,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":1.302,"5min":1.302,"15min":1.302},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":2.109,"5min":2.109,"15min":2.109},"last":1.494},{"interface":"front","average":{"1min":3.7799999999999998,"5min":3.7799999999999998,"15min":3.7799999999999998},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":54.167999999999999,"5min":54.167999999999999,"15min":54.167999999999999},"last":1.0029999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.669}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.446}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":96}]},{"osd":14,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":10.006,"5min":10.006,"15min":10.006},"min":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":62.899000000000001,"5min":62.899000000000001,"15min":62.899000000000001},"last":1.3919999999999999},{"interface":"front","average":{"1min":15.250999999999999,"5min":15.250999999999999,"15min":15.250999999999999},"min":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"max":{"1min":155.61000000000001,"5min":155.61000000000001,"15min":155.61000000000001},"last":1.415}]},{"osd":15,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":1.5,"5min":1.5,"15min":1.5},"min":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"max":{"1min":1.9970000000000001,"5min":1.9970000000000001,"15min":1.9970000000000001},"last":1.569},{"interface":"front","average":{"1min":4.077,"5min":4.077,"15min":4.077},"min":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":54.335999999999999,"5min":54.335999999999999,"15min":54.335999999999999},"last":1.1950000000000001}]}]},{"osd":11,"up_from":21,"seq":90194313238,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":314572,"kb_used_data":13048,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94057268,"statfs":{"total":96636764160,"available":96314642432,"internally_reserved":0,"allocated":13361152,"data_stored":16015734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[2,3,6,8,9,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0049999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86599999999999999}]},{"osd":2,"last update":"Wed Sep 18 10:15:33 2024","interfaces":[{"interface":"back","average":{"1min":10.189,"5min":10.189,"15min":10.189},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":99.718000000000004,"5min":99.718000000000004,"15min":99.718000000000004},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.82399999999999995}]},{"osd":3,"last update":"Wed Sep 18 10:15:46 2024","interfaces":[{"interface":"back","average":{"1min":5.0720000000000001,"5min":5.0720000000000001,"15min":5.0720000000000001},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":85.963999999999999,"5min":85.963999999999999,"15min":85.963999999999999},"last":0.65100000000000002},{"interface":"front","average":{"1min":22.759,"5min":22.759,"15min":22.759},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":125.672,"5min":125.672,"15min":125.672},"last":1.002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.444}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5980000000000001}]},{"osd":6,"last update":"Wed Sep 18 10:15:46 2024","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.157,"15min":1.157},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":5.04,"5min":5.04,"15min":5.04},"last":0.70599999999999996},{"interface":"front","average":{"1min":6.9880000000000004,"5min":6.9880000000000004,"15min":6.9880000000000004},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":94.162000000000006,"5min":94.162000000000006,"15min":94.162000000000006},"last":0.89000000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.524}]},{"osd":8,"last update":"Wed Sep 18 10:15:33 2024","interfaces":[{"interface":"back","average":{"1min":6.04,"5min":6.04,"15min":6.04},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":85.900000000000006,"5min":85.900000000000006,"15min":85.900000000000006},"last":1.044},{"interface":"front","average":{"1min":9.9849999999999994,"5min":9.9849999999999994,"15min":9.9849999999999994},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":99.765000000000001,"5min":99.765000000000001,"15min":99.765000000000001},"last":0.753}]},{"osd":9,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":5.8239999999999998,"5min":5.8239999999999998,"15min":5.8239999999999998},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":85.838999999999999,"5min":85.838999999999999,"15min":85.838999999999999},"last":1.204},{"interface":"front","average":{"1min":8.1709999999999994,"5min":8.1709999999999994,"15min":8.1709999999999994},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":99.810000000000002,"5min":99.810000000000002,"15min":99.810000000000002},"last":0.85699999999999998}]},{"osd":10,"last update":"Wed Sep 18 10:15:33 2024","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.097,"15min":1.097},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.371,"5min":1.371,"15min":1.371},"last":1.0289999999999999},{"interface":"front","average":{"1min":7.6859999999999999,"5min":7.6859999999999999,"15min":7.6859999999999999},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":89.290000000000006,"5min":89.290000000000006,"15min":89.290000000000006},"last":1.4970000000000001}]},{"osd":12,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":1.125,"15min":1.125},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.6259999999999999,"5min":1.6259999999999999,"15min":1.6259999999999999},"last":0.96599999999999997},{"interface":"front","average":{"1min":10.385999999999999,"5min":10.385999999999999,"15min":10.385999999999999},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":86.025999999999996,"5min":86.025999999999996,"15min":86.025999999999996},"last":1.3620000000000001}]},{"osd":13,"last update":"Wed Sep 18 10:15:33 2024","interfaces":[{"interface":"back","average":{"1min":30.405000000000001,"5min":30.405000000000001,"15min":30.405000000000001},"min":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"max":{"1min":189.25899999999999,"5min":189.25899999999999,"15min":189.25899999999999},"last":1.1379999999999999},{"interface":"front","average":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":2.1720000000000002,"5min":2.1720000000000002,"15min":2.1720000000000002},"last":1.119}]},{"osd":14,"last update":"Wed Sep 18 10:15:33 2024","interfaces":[{"interface":"back","average":{"1min":6.423,"5min":6.423,"15min":6.423},"min":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":86.109999999999999,"5min":86.109999999999999,"15min":86.109999999999999},"last":1.282},{"interface":"front","average":{"1min":8.9819999999999993,"5min":8.9819999999999993,"15min":8.9819999999999993},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":102.625,"5min":102.625,"15min":102.625},"last":1.1839999999999999}]},{"osd":15,"last update":"Wed Sep 18 10:15:33 2024","interfaces":[{"interface":"back","average":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":2.0249999999999999,"5min":2.0249999999999999,"15min":2.0249999999999999},"last":1.1659999999999999},{"interface":"front","average":{"1min":6.484,"5min":6.484,"15min":6.484},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":86.165999999999997,"5min":86.165999999999997,"15min":86.165999999999997},"last":1.4299999999999999}]}]},{"osd":12,"up_from":21,"seq":90194313239,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306908,"kb_used_data":9544,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94064932,"statfs":{"total":96636764160,"available":96322490368,"internally_reserved":0,"allocated":9773056,"data_stored":13607305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[2,3,4,6,7,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":2.3839999999999999,"5min":2.3839999999999999,"15min":2.3839999999999999},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":32.476999999999997,"5min":32.476999999999997,"15min":32.476999999999997},"last":0.64200000000000002},{"interface":"front","average":{"1min":2.625,"5min":2.625,"15min":2.625},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":32.509,"5min":32.509,"15min":32.509},"last":0.89300000000000002}]},{"osd":1,"last update":"Wed Sep 18 10:15:32 2024","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.6399999999999999,"5min":1.6399999999999999,"15min":1.6399999999999999},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.905,"5min":1.905,"15min":1.905},"last":0.81899999999999995}]},{"osd":2,"last update":"Wed Sep 18 10:15:32 2024","interfaces":[{"interface":"back","average":{"1min":1.002,"5min":1.002,"15min":1.002},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.85699999999999998},{"interface":"front","average":{"1min":4.1189999999999998,"5min":4.1189999999999998,"15min":4.1189999999999998},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":62.780999999999999,"5min":62.780999999999999,"15min":62.780999999999999},"last":1.1279999999999999}]},{"osd":3,"last update":"Wed Sep 18 10:15:32 2024","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":1.145,"15min":1.145},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.772,"5min":1.772,"15min":1.772},"last":0.93899999999999995},{"interface":"front","average":{"1min":1.151,"5min":1.151,"15min":1.151},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.4710000000000001,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":1.1759999999999999}]},{"osd":4,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":1.052,"15min":1.052},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.6759999999999999,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":0.89700000000000002},{"interface":"front","average":{"1min":1.222,"5min":1.222,"15min":1.222},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.774,"5min":1.774,"15min":1.774},"last":1.472}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2929999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.411}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":94.382000000000005}]},{"osd":9,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":1.268,"5min":1.268,"15min":1.268},"min":{"1min":0.875,"5min":0.875,"15min":0.875},"max":{"1min":1.9079999999999999,"5min":1.9079999999999999,"15min":1.9079999999999999},"last":1.3280000000000001},{"interface":"front","average":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.8460000000000001,"5min":1.8460000000000001,"15min":1.8460000000000001},"last":1.51}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5940000000000001}]},{"osd":11,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":1.365,"5min":1.365,"15min":1.365},"min":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":2.0539999999999998,"5min":2.0539999999999998,"15min":2.0539999999999998},"last":1.7090000000000001},{"interface":"front","average":{"1min":1.383,"5min":1.383,"15min":1.383},"min":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"max":{"1min":1.98,"5min":1.98,"15min":1.98},"last":1.552}]},{"osd":13,"last update":"Wed Sep 18 10:15:32 2024","interfaces":[{"interface":"back","average":{"1min":1.333,"5min":1.333,"15min":1.333},"min":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":2.016,"5min":2.016,"15min":2.016},"last":1.651},{"interface":"front","average":{"1min":1.429,"5min":1.429,"15min":1.429},"min":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"max":{"1min":1.9379999999999999,"5min":1.9379999999999999,"15min":1.9379999999999999},"last":1.8049999999999999}]},{"osd":14,"last update":"Wed Sep 18 10:15:32 2024","interfaces":[{"interface":"back","average":{"1min":1.4299999999999999,"5min":1.4299999999999999,"15min":1.4299999999999999},"min":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"max":{"1min":2.0619999999999998,"5min":2.0619999999999998,"15min":2.0619999999999998},"last":1.633},{"interface":"front","average":{"1min":1.391,"5min":1.391,"15min":1.391},"min":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":2.1339999999999999,"5min":2.1339999999999999,"15min":2.1339999999999999},"last":1.746}]},{"osd":15,"last update":"Wed Sep 18 10:15:32 2024","interfaces":[{"interface":"back","average":{"1min":1.494,"5min":1.494,"15min":1.494},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":2.1179999999999999,"5min":2.1179999999999999,"15min":2.1179999999999999},"last":1.8520000000000001},{"interface":"front","average":{"1min":1.48,"5min":1.48,"15min":1.48},"min":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"max":{"1min":2.1600000000000001,"5min":2.1600000000000001,"15min":2.1600000000000001},"last":1.6850000000000001}]}]},{"osd":1,"up_from":22,"seq":94489280534,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":315000,"kb_used_data":13476,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94056840,"statfs":{"total":96636764160,"available":96314204160,"internally_reserved":0,"allocated":13799424,"data_stored":14983542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,4,5,6,9,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:15:35 2024","interfaces":[{"interface":"back","average":{"1min":2.8700000000000001,"5min":2.8700000000000001,"15min":2.8700000000000001},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":36.268000000000001,"5min":36.268000000000001,"15min":36.268000000000001},"last":0.83199999999999996},{"interface":"front","average":{"1min":17.715,"5min":17.715,"15min":17.715},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":154.67599999999999,"5min":154.67599999999999,"15min":154.67599999999999},"last":0.88200000000000001}]},{"osd":2,"last update":"Wed Sep 18 10:15:35 2024","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.343,"5min":1.343,"15min":1.343},"last":1.048},{"interface":"front","average":{"1min":2.9449999999999998,"5min":2.9449999999999998,"15min":2.9449999999999998},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":36.323999999999998,"5min":36.323999999999998,"15min":36.323999999999998},"last":1.0860000000000001}]},{"osd":3,"last update":"Wed Sep 18 10:15:35 2024","interfaces":[{"interface":"back","average":{"1min":12.128,"5min":12.128,"15min":12.128},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":150.642,"5min":150.642,"15min":150.642},"last":1.2529999999999999},{"interface":"front","average":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.6930000000000001,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":0.93600000000000005}]},{"osd":4,"last update":"Wed Sep 18 10:15:35 2024","interfaces":[{"interface":"back","average":{"1min":3.3439999999999999,"5min":3.3439999999999999,"15min":3.3439999999999999},"min":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":40.783000000000001,"5min":40.783000000000001,"15min":40.783000000000001},"last":1.1839999999999999},{"interface":"front","average":{"1min":4.1699999999999999,"5min":4.1699999999999999,"15min":4.1699999999999999},"min":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":50.747,"5min":50.747,"15min":50.747},"last":1.288}]},{"osd":5,"last update":"Wed Sep 18 10:15:35 2024","interfaces":[{"interface":"back","average":{"1min":5.9720000000000004,"5min":5.9720000000000004,"15min":5.9720000000000004},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":52.317999999999998,"5min":52.317999999999998,"15min":52.317999999999998},"last":1.212},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.748,"5min":1.748,"15min":1.748},"last":1.337}]},{"osd":6,"last update":"Wed Sep 18 10:15:35 2024","interfaces":[{"interface":"back","average":{"1min":8.3490000000000002,"5min":8.3490000000000002,"15min":8.3490000000000002},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":84.939999999999998,"5min":84.939999999999998,"15min":84.939999999999998},"last":1.3660000000000001},{"interface":"front","average":{"1min":10.352,"5min":10.352,"15min":10.352},"min":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"max":{"1min":85.045000000000002,"5min":85.045000000000002,"15min":85.045000000000002},"last":1.444}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":118.43600000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7789999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6020000000000001}]},{"osd":10,"last update":"Wed Sep 18 10:15:35 2024","interfaces":[{"interface":"back","average":{"1min":3.4300000000000002,"5min":3.4300000000000002,"15min":3.4300000000000002},"min":{"1min":1.073,"5min":1.073,"15min":1.073},"max":{"1min":36.436,"5min":36.436,"15min":36.436},"last":1.544},{"interface":"front","average":{"1min":11.914999999999999,"5min":11.914999999999999,"15min":11.914999999999999},"min":{"1min":0.88,"5min":0.88,"15min":0.88},"max":{"1min":100.402,"5min":100.402,"15min":100.402},"last":1.4079999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.825}]},{"osd":12,"last update":"Wed Sep 18 10:15:35 2024","interfaces":[{"interface":"back","average":{"1min":4.2130000000000001,"5min":4.2130000000000001,"15min":4.2130000000000001},"min":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"max":{"1min":49.976999999999997,"5min":49.976999999999997,"15min":49.976999999999997},"last":1.51},{"interface":"front","average":{"1min":8.4000000000000004,"5min":8.4000000000000004,"15min":8.4000000000000004},"min":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"max":{"1min":124.78100000000001,"5min":124.78100000000001,"15min":124.78100000000001},"last":1.647}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.484}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.575}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6970000000000001}]}]},{"osd":2,"up_from":22,"seq":94489280534,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":320076,"kb_used_data":18560,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94051764,"statfs":{"total":96636764160,"available":96309006336,"internally_reserved":0,"allocated":19005440,"data_stored":23382343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,3,4,5,6,7,8,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 18 10:15:31 2024","interfaces":[{"interface":"back","average":{"1min":26.495000000000001,"5min":26.495000000000001,"15min":26.495000000000001},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":419.05700000000002,"5min":419.05700000000002,"15min":419.05700000000002},"last":0.77000000000000002},{"interface":"front","average":{"1min":4.9059999999999997,"5min":4.9059999999999997,"15min":4.9059999999999997},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":98.254000000000005,"5min":98.254000000000005,"15min":98.254000000000005},"last":0.82599999999999996}]},{"osd":3,"last update":"Wed Sep 18 10:15:31 2024","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.94599999999999995}]},{"osd":4,"last update":"Wed Sep 18 10:15:31 2024","interfaces":[{"interface":"back","average":{"1min":1.925,"5min":1.925,"15min":1.925},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":24.888999999999999,"5min":24.888999999999999,"15min":24.888999999999999},"last":1.1160000000000001},{"interface":"front","average":{"1min":4.5389999999999997,"5min":4.5389999999999997,"15min":4.5389999999999997},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":84.647000000000006,"5min":84.647000000000006,"15min":84.647000000000006},"last":1.2589999999999999}]},{"osd":5,"last update":"Wed Sep 18 10:15:31 2024","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.478,"5min":1.478,"15min":1.478},"last":1.02},{"interface":"front","average":{"1min":1.054,"5min":1.054,"15min":1.054},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.415,"5min":1.415,"15min":1.415},"last":1.22}]},{"osd":6,"last update":"Wed Sep 18 10:15:31 2024","interfaces":[{"interface":"back","average":{"1min":4.758,"5min":4.758,"15min":4.758},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":71.451999999999998,"5min":71.451999999999998,"15min":71.451999999999998},"last":1.161},{"interface":"front","average":{"1min":4.8730000000000002,"5min":4.8730000000000002,"15min":4.8730000000000002},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":90.188999999999993,"5min":90.188999999999993,"15min":90.188999999999993},"last":1.0920000000000001}]},{"osd":7,"last update":"Wed Sep 18 10:15:31 2024","interfaces":[{"interface":"back","average":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":1.2050000000000001},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.724,"5min":1.724,"15min":1.724},"last":1.3480000000000001},{"interface":"front","average":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":3.016,"5min":3.016,"15min":3.016},"last":1.514}]},{"osd":8,"last update":"Wed Sep 18 10:15:31 2024","interfaces":[{"interface":"back","average":{"1min":5.1020000000000003,"5min":5.1020000000000003,"15min":5.1020000000000003},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":92.578999999999994,"5min":92.578999999999994,"15min":92.578999999999994},"last":1.143},{"interface":"front","average":{"1min":2.7639999999999998,"5min":2.7639999999999998,"15min":2.7639999999999998},"min":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"max":{"1min":35.271000000000001,"5min":35.271000000000001,"15min":35.271000000000001},"last":1.49}]},{"osd":12,"last update":"Wed Sep 18 10:15:31 2024","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"min":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"max":{"1min":1.696,"5min":1.696,"15min":1.696},"last":1.5489999999999999},{"interface":"front","average":{"1min":2.056,"5min":2.056,"15min":2.056},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":16.878,"5min":16.878,"15min":16.878},"last":1.321}]},{"osd":13,"last update":"Wed Sep 18 10:15:31 2024","interfaces":[{"interface":"back","average":{"1min":7.718,"5min":7.718,"15min":7.718},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":141.84200000000001,"5min":141.84200000000001,"15min":141.84200000000001},"last":1.248},{"interface":"front","average":{"1min":5.4589999999999996,"5min":5.4589999999999996,"15min":5.4589999999999996},"min":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":97.441999999999993,"5min":97.441999999999993,"15min":97.441999999999993},"last":1.4159999999999999}]},{"osd":15,"last update":"Wed Sep 18 10:15:31 2024","interfaces":[{"interface":"back","average":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":3.1480000000000001,"5min":3.1480000000000001,"15min":3.1480000000000001},"last":1.448},{"interface":"front","average":{"1min":3.911,"5min":3.911,"15min":3.911},"min":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":61.694000000000003,"5min":61.694000000000003,"15min":61.694000000000003},"last":1.196}]}]},{"osd":3,"up_from":22,"seq":94489280534,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":318380,"kb_used_data":16856,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94053460,"statfs":{"total":96636764160,"available":96310743040,"internally_reserved":0,"allocated":17260544,"data_stored":23921014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,4,5,6,7,9,10,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:15:32 2024","interfaces":[{"interface":"back","average":{"1min":1.5169999999999999,"5min":1.5169999999999999,"15min":1.5169999999999999},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":13.680999999999999,"5min":13.680999999999999,"15min":13.680999999999999},"last":0.91900000000000004},{"interface":"front","average":{"1min":10.788,"5min":10.788,"15min":10.788},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":71.316999999999993,"5min":71.316999999999993,"15min":71.316999999999993},"last":1.0700000000000001}]},{"osd":2,"last update":"Wed Sep 18 10:15:32 2024","interfaces":[{"interface":"back","average":{"1min":5.3029999999999999,"5min":5.3029999999999999,"15min":5.3029999999999999},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":56.523000000000003,"5min":56.523000000000003,"15min":56.523000000000003},"last":1.1519999999999999},{"interface":"front","average":{"1min":1.109,"5min":1.109,"15min":1.109},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":3.327,"5min":3.327,"15min":3.327},"last":1.3520000000000001}]},{"osd":4,"last update":"Wed Sep 18 10:15:32 2024","interfaces":[{"interface":"back","average":{"1min":6.8739999999999997,"5min":6.8739999999999997,"15min":6.8739999999999997},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":71.135000000000005,"5min":71.135000000000005,"15min":71.135000000000005},"last":1.0009999999999999},{"interface":"front","average":{"1min":4.3579999999999997,"5min":4.3579999999999997,"15min":4.3579999999999997},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":56.253999999999998,"5min":56.253999999999998,"15min":56.253999999999998},"last":1.413}]},{"osd":5,"last update":"Wed Sep 18 10:16:06 2024","interfaces":[{"interface":"back","average":{"1min":13.997,"5min":13.997,"15min":13.997},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":125.42400000000001,"5min":125.42400000000001,"15min":125.42400000000001},"last":1.266},{"interface":"front","average":{"1min":10.323,"5min":10.323,"15min":10.323},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":112.033,"5min":112.033,"15min":112.033},"last":1.2989999999999999}]},{"osd":6,"last update":"Wed Sep 18 10:15:32 2024","interfaces":[{"interface":"back","average":{"1min":1.131,"5min":1.131,"15min":1.131},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.744,"5min":1.744,"15min":1.744},"last":1.177},{"interface":"front","average":{"1min":8.7520000000000007,"5min":8.7520000000000007,"15min":8.7520000000000007},"min":{"1min":0.749,"5min":0.749,"15min":0.749},"max":{"1min":81.210999999999999,"5min":81.210999999999999,"15min":81.210999999999999},"last":1.109}]},{"osd":7,"last update":"Wed Sep 18 10:15:32 2024","interfaces":[{"interface":"back","average":{"1min":16.132000000000001,"5min":16.132000000000001,"15min":16.132000000000001},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":129.68299999999999,"5min":129.68299999999999,"15min":129.68299999999999},"last":0.96299999999999997},{"interface":"front","average":{"1min":13.532999999999999,"5min":13.532999999999999,"15min":13.532999999999999},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":129.726,"5min":129.726,"15min":129.726},"last":1.524}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0620000000000001}]},{"osd":9,"last update":"Wed Sep 18 10:15:32 2024","interfaces":[{"interface":"back","average":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.9990000000000001,"5min":1.9990000000000001,"15min":1.9990000000000001},"last":1.4570000000000001},{"interface":"front","average":{"1min":6.109,"5min":6.109,"15min":6.109},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":81.171000000000006,"5min":81.171000000000006,"15min":81.171000000000006},"last":1.3380000000000001}]},{"osd":10,"last update":"Wed Sep 18 10:16:06 2024","interfaces":[{"interface":"back","average":{"1min":1.4870000000000001,"5min":1.4870000000000001,"15min":1.4870000000000001},"min":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":2.0659999999999998,"5min":2.0659999999999998,"15min":2.0659999999999998},"last":1.599},{"interface":"front","average":{"1min":4.9349999999999996,"5min":4.9349999999999996,"15min":4.9349999999999996},"min":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":71.668999999999997,"5min":71.668999999999997,"15min":71.668999999999997},"last":1.4450000000000001}]},{"osd":13,"last update":"Wed Sep 18 10:15:32 2024","interfaces":[{"interface":"back","average":{"1min":1.5189999999999999,"5min":1.5189999999999999,"15min":1.5189999999999999},"min":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":2.6859999999999999,"5min":2.6859999999999999,"15min":2.6859999999999999},"last":1.6439999999999999},{"interface":"front","average":{"1min":6.282,"5min":6.282,"15min":6.282},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":71.447000000000003,"5min":71.447000000000003,"15min":71.447000000000003},"last":1.4790000000000001}]},{"osd":14,"last update":"Wed Sep 18 10:15:32 2024","interfaces":[{"interface":"back","average":{"1min":5.6900000000000004,"5min":5.6900000000000004,"15min":5.6900000000000004},"min":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"max":{"1min":56.664999999999999,"5min":56.664999999999999,"15min":56.664999999999999},"last":1.5649999999999999},{"interface":"front","average":{"1min":2.363,"5min":2.363,"15min":2.363},"min":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"max":{"1min":18.523,"5min":18.523,"15min":18.523},"last":1.7170000000000001}]},{"osd":15,"last update":"Wed Sep 18 10:15:45 2024","interfaces":[{"interface":"back","average":{"1min":4.5830000000000002,"5min":4.5830000000000002,"15min":4.5830000000000002},"min":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"max":{"1min":56.357999999999997,"5min":56.357999999999997,"15min":56.357999999999997},"last":1.496},{"interface":"front","average":{"1min":10.225,"5min":10.225,"15min":10.225},"min":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":122.727,"5min":122.727,"15min":122.727},"last":1.681}]}]},{"osd":5,"up_from":22,"seq":94489280534,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312984,"kb_used_data":11460,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94058856,"statfs":{"total":96636764160,"available":96316268544,"internally_reserved":0,"allocated":11735040,"data_stored":17445238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,4,6,7,8,9,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:15:47 2024","interfaces":[{"interface":"back","average":{"1min":7.8860000000000001,"5min":7.8860000000000001,"15min":7.8860000000000001},"min":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":84.430999999999997,"5min":84.430999999999997,"15min":84.430999999999997},"last":1.262},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":1.3260000000000001}]},{"osd":2,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":9.8339999999999996,"5min":9.8339999999999996,"15min":9.8339999999999996},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":84.566000000000003,"5min":84.566000000000003,"15min":84.566000000000003},"last":0.75900000000000001},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":9.5069999999999997,"5min":9.5069999999999997,"15min":9.5069999999999997},"last":0.60999999999999999}]},{"osd":3,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":8.5210000000000008,"5min":8.5210000000000008,"15min":8.5210000000000008},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":84.358999999999995,"5min":84.358999999999995,"15min":84.358999999999995},"last":1.4419999999999999},{"interface":"front","average":{"1min":8.2230000000000008,"5min":8.2230000000000008,"15min":8.2230000000000008},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":97.549999999999997,"5min":97.549999999999997,"15min":97.549999999999997},"last":1.0820000000000001}]},{"osd":4,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":31.829999999999998,"5min":31.829999999999998,"15min":31.829999999999998},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":266.911,"5min":266.911,"15min":266.911},"last":1.0249999999999999},{"interface":"front","average":{"1min":22.798999999999999,"5min":22.798999999999999,"15min":22.798999999999999},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":260.53500000000003,"5min":260.53500000000003,"15min":260.53500000000003},"last":1.0589999999999999}]},{"osd":6,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":9.1940000000000008,"5min":9.1940000000000008,"15min":9.1940000000000008},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":58.171999999999997,"5min":58.171999999999997,"15min":58.171999999999997},"last":0.88200000000000001},{"interface":"front","average":{"1min":21.391999999999999,"5min":21.391999999999999,"15min":21.391999999999999},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":227.48099999999999,"5min":227.48099999999999,"15min":227.48099999999999},"last":0.92600000000000005}]},{"osd":7,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":12.013,"5min":12.013,"15min":12.013},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":123.111,"5min":123.111,"15min":123.111},"last":1.105},{"interface":"front","average":{"1min":13.994,"5min":13.994,"15min":13.994},"min":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"max":{"1min":84.617000000000004,"5min":84.617000000000004,"15min":84.617000000000004},"last":0.97199999999999998}]},{"osd":8,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":20.352,"5min":20.352,"15min":20.352},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":260.44200000000001,"5min":260.44200000000001,"15min":260.44200000000001},"last":1.157},{"interface":"front","average":{"1min":7.0430000000000001,"5min":7.0430000000000001,"15min":7.0430000000000001},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":106.818,"5min":106.818,"15min":106.818},"last":1.228}]},{"osd":9,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":5.6470000000000002,"5min":5.6470000000000002,"15min":5.6470000000000002},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":80.272000000000006,"5min":80.272000000000006,"15min":80.272000000000006},"last":1.1359999999999999},{"interface":"front","average":{"1min":1.8600000000000001,"5min":1.8600000000000001,"15min":1.8600000000000001},"min":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"max":{"1min":9.5690000000000008,"5min":9.5690000000000008,"15min":9.5690000000000008},"last":1.1919999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6479999999999999}]},{"osd":11,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":9.3919999999999995,"5min":9.3919999999999995,"15min":9.3919999999999995},"min":{"1min":0.997,"5min":0.997,"15min":0.997},"max":{"1min":123.158,"5min":123.158,"15min":123.158},"last":1.2909999999999999},{"interface":"front","average":{"1min":11.367000000000001,"5min":11.367000000000001,"15min":11.367000000000001},"min":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"max":{"1min":84.522999999999996,"5min":84.522999999999996,"15min":84.522999999999996},"last":1.3959999999999999}]},{"osd":12,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":1.7789999999999999,"5min":1.7789999999999999,"15min":1.7789999999999999},"last":1.419},{"interface":"front","average":{"1min":1.9750000000000001,"5min":1.9750000000000001,"15min":1.9750000000000001},"min":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"max":{"1min":9.7710000000000008,"5min":9.7710000000000008,"15min":9.7710000000000008},"last":1.4630000000000001}]},{"osd":13,"last update":"Wed Sep 18 10:15:47 2024","interfaces":[{"interface":"back","average":{"1min":24.271000000000001,"5min":24.271000000000001,"15min":24.271000000000001},"min":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"max":{"1min":140.273,"5min":140.273,"15min":140.273},"last":1.3700000000000001},{"interface":"front","average":{"1min":24.221,"5min":24.221,"15min":24.221},"min":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"max":{"1min":170.86000000000001,"5min":170.86000000000001,"15min":170.86000000000001},"last":1.353}]},{"osd":14,"last update":"Wed Sep 18 10:15:47 2024","interfaces":[{"interface":"back","average":{"1min":13.257,"5min":13.257,"15min":13.257},"min":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"max":{"1min":101.733,"5min":101.733,"15min":101.733},"last":1.494},{"interface":"front","average":{"1min":9.5190000000000001,"5min":9.5190000000000001,"15min":9.5190000000000001},"min":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"max":{"1min":122.952,"5min":122.952,"15min":122.952},"last":1.54}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8839999999999999}]}]},{"osd":6,"up_from":22,"seq":94489280534,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":318844,"kb_used_data":17320,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94052996,"statfs":{"total":96636764160,"available":96310267904,"internally_reserved":0,"allocated":17735680,"data_stored":22371175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,3,4,5,7,8,9,11,12,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:15:47 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.91900000000000004},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.82799999999999996}]},{"osd":1,"last update":"Wed Sep 18 10:15:47 2024","interfaces":[{"interface":"back","average":{"1min":4.5549999999999997,"5min":4.5549999999999997,"15min":4.5549999999999997},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":52.332999999999998,"5min":52.332999999999998,"15min":52.332999999999998},"last":1.2150000000000001},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":1.133}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.343}]},{"osd":3,"last update":"Wed Sep 18 10:15:47 2024","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.492,"5min":1.492,"15min":1.492},"last":1.1639999999999999},{"interface":"front","average":{"1min":14.007,"5min":14.007,"15min":14.007},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":169.244,"5min":169.244,"15min":169.244},"last":0.96999999999999997}]},{"osd":4,"last update":"Wed Sep 18 10:16:02 2024","interfaces":[{"interface":"back","average":{"1min":11.111000000000001,"5min":11.111000000000001,"15min":11.111000000000001},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":128.053,"5min":128.053,"15min":128.053},"last":1.3640000000000001},{"interface":"front","average":{"1min":4.6189999999999998,"5min":4.6189999999999998,"15min":4.6189999999999998},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":67.960999999999999,"5min":67.960999999999999,"15min":67.960999999999999},"last":1.2609999999999999}]},{"osd":5,"last update":"Wed Sep 18 10:15:32 2024","interfaces":[{"interface":"back","average":{"1min":11.228999999999999,"5min":11.228999999999999,"15min":11.228999999999999},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":119.032,"5min":119.032,"15min":119.032},"last":1.49},{"interface":"front","average":{"1min":3.629,"5min":3.629,"15min":3.629},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":54.731000000000002,"5min":54.731000000000002,"15min":54.731000000000002},"last":1.29}]},{"osd":7,"last update":"Wed Sep 18 10:15:32 2024","interfaces":[{"interface":"back","average":{"1min":6.7569999999999997,"5min":6.7569999999999997,"15min":6.7569999999999997},"min":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":52.204999999999998,"5min":52.204999999999998,"15min":52.204999999999998},"last":1.429},{"interface":"front","average":{"1min":3.0550000000000002,"5min":3.0550000000000002,"15min":3.0550000000000002},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":40.848999999999997,"5min":40.848999999999997,"15min":40.848999999999997},"last":1.395}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4450000000000001}]},{"osd":9,"last update":"Wed Sep 18 10:15:32 2024","interfaces":[{"interface":"back","average":{"1min":8.2799999999999994,"5min":8.2799999999999994,"15min":8.2799999999999994},"min":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"max":{"1min":95.691999999999993,"5min":95.691999999999993,"15min":95.691999999999993},"last":1.534},{"interface":"front","average":{"1min":12.766999999999999,"5min":12.766999999999999,"15min":12.766999999999999},"min":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"max":{"1min":165.31100000000001,"5min":165.31100000000001,"15min":165.31100000000001},"last":1.52}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.329}]},{"osd":11,"last update":"Wed Sep 18 10:15:32 2024","interfaces":[{"interface":"back","average":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.7749999999999999,"5min":1.7749999999999999,"15min":1.7749999999999999},"last":1.6439999999999999},{"interface":"front","average":{"1min":3.883,"5min":3.883,"15min":3.883},"min":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"max":{"1min":54.823,"5min":54.823,"15min":54.823},"last":1.611}]},{"osd":12,"last update":"Wed Sep 18 10:15:32 2024","interfaces":[{"interface":"back","average":{"1min":5.0469999999999997,"5min":5.0469999999999997,"15min":5.0469999999999997},"min":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":52.378,"5min":52.378,"15min":52.378},"last":1.5920000000000001},{"interface":"front","average":{"1min":4.4379999999999997,"5min":4.4379999999999997,"15min":4.4379999999999997},"min":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":64.760999999999996,"5min":64.760999999999996,"15min":64.760999999999996},"last":1.5680000000000001}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2050000000000001}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2669999999999999}]},{"osd":15,"last update":"Wed Sep 18 10:15:32 2024","interfaces":[{"interface":"back","average":{"1min":1.474,"5min":1.474,"15min":1.474},"min":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"max":{"1min":3.1840000000000002,"5min":3.1840000000000002,"15min":3.1840000000000002},"last":1.72},{"interface":"front","average":{"1min":3.9729999999999999,"5min":3.9729999999999999,"15min":3.9729999999999999},"min":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"max":{"1min":54.869999999999997,"5min":54.869999999999997,"15min":54.869999999999997},"last":1.6839999999999999}]}]},{"osd":7,"up_from":22,"seq":94489280534,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":317032,"kb_used_data":15520,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94054808,"statfs":{"total":96636764160,"available":96312123392,"internally_reserved":0,"allocated":15892480,"data_stored":23691638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,6,8,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:15:33 2024","interfaces":[{"interface":"back","average":{"1min":12.721,"5min":12.721,"15min":12.721},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":139.58600000000001,"5min":139.58600000000001,"15min":139.58600000000001},"last":1.6499999999999999},{"interface":"front","average":{"1min":0.999,"5min":0.999,"15min":0.999},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":1.528}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.321}]},{"osd":2,"last update":"Wed Sep 18 10:15:46 2024","interfaces":[{"interface":"back","average":{"1min":15.436,"5min":15.436,"15min":15.436},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":139.36799999999999,"5min":139.36799999999999,"15min":139.36799999999999},"last":1.0069999999999999},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":1.3600000000000001}]},{"osd":3,"last update":"Wed Sep 18 10:15:46 2024","interfaces":[{"interface":"back","average":{"1min":17.327999999999999,"5min":17.327999999999999,"15min":17.327999999999999},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":139.27199999999999,"5min":139.27199999999999,"15min":139.27199999999999},"last":1.105},{"interface":"front","average":{"1min":3.964,"5min":3.964,"15min":3.964},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":23.748999999999999,"5min":23.748999999999999,"15min":23.748999999999999},"last":1.1519999999999999}]},{"osd":4,"last update":"Wed Sep 18 10:15:37 2024","interfaces":[{"interface":"back","average":{"1min":19.146999999999998,"5min":19.146999999999998,"15min":19.146999999999998},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":139.51599999999999,"5min":139.51599999999999,"15min":139.51599999999999},"last":1.3959999999999999},{"interface":"front","average":{"1min":4.9989999999999997,"5min":4.9989999999999997,"15min":4.9989999999999997},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":27.882999999999999,"5min":27.882999999999999,"15min":27.882999999999999},"last":0.92700000000000005}]},{"osd":5,"last update":"Wed Sep 18 10:15:46 2024","interfaces":[{"interface":"back","average":{"1min":1.099,"5min":1.099,"15min":1.099},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":2.3620000000000001,"5min":2.3620000000000001,"15min":2.3620000000000001},"last":0.96799999999999997},{"interface":"front","average":{"1min":17.736999999999998,"5min":17.736999999999998,"15min":17.736999999999998},"min":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"max":{"1min":139.44300000000001,"5min":139.44300000000001,"15min":139.44300000000001},"last":1.502}]},{"osd":6,"last update":"Wed Sep 18 10:15:33 2024","interfaces":[{"interface":"back","average":{"1min":17.375,"5min":17.375,"15min":17.375},"min":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":183.001,"5min":183.001,"15min":183.001},"last":1.579},{"interface":"front","average":{"1min":4.3109999999999999,"5min":4.3109999999999999,"15min":4.3109999999999999},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":23.978999999999999,"5min":23.978999999999999,"15min":23.978999999999999},"last":1.246}]},{"osd":8,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.25,"15min":1.25},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":2.6139999999999999,"5min":2.6139999999999999,"15min":2.6139999999999999},"last":1.484},{"interface":"front","average":{"1min":15.906000000000001,"5min":15.906000000000001,"15min":15.906000000000001},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":139.798,"5min":139.798,"15min":139.798},"last":1.724}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.861}]},{"osd":10,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":6.5039999999999996,"5min":6.5039999999999996,"15min":6.5039999999999996},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":32.539999999999999,"5min":32.539999999999999,"15min":32.539999999999999},"last":1.4550000000000001},{"interface":"front","average":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":2.254,"5min":2.254,"15min":2.254},"last":1.5660000000000001}]},{"osd":11,"last update":"Wed Sep 18 10:15:33 2024","interfaces":[{"interface":"back","average":{"1min":5.5469999999999997,"5min":5.5469999999999997,"15min":5.5469999999999997},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":62.356000000000002,"5min":62.356000000000002,"15min":62.356000000000002},"last":1.542},{"interface":"front","average":{"1min":1.411,"5min":1.411,"15min":1.411},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":2.738,"5min":2.738,"15min":2.738},"last":1.7430000000000001}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.611}]},{"osd":13,"last update":"Wed Sep 18 10:15:33 2024","interfaces":[{"interface":"back","average":{"1min":18.379000000000001,"5min":18.379000000000001,"15min":18.379000000000001},"min":{"1min":0.874,"5min":0.874,"15min":0.874},"max":{"1min":139.655,"5min":139.655,"15min":139.655},"last":1.774},{"interface":"front","average":{"1min":19.879000000000001,"5min":19.879000000000001,"15min":19.879000000000001},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":173.113,"5min":173.113,"15min":173.113},"last":1.7010000000000001}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8180000000000001}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6020000000000001}]}]},{"osd":8,"up_from":22,"seq":94489280534,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":328852,"kb_used_data":27336,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94042988,"statfs":{"total":96636764160,"available":96300019712,"internally_reserved":0,"allocated":27992064,"data_stored":28834119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,4,6,7,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93300000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.464}]},{"osd":2,"last update":"Wed Sep 18 10:15:46 2024","interfaces":[{"interface":"back","average":{"1min":6.5880000000000001,"5min":6.5880000000000001,"15min":6.5880000000000001},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":77.435000000000002,"5min":77.435000000000002,"15min":77.435000000000002},"last":1.8},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.526,"5min":1.526,"15min":1.526},"last":1.903}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.306}]},{"osd":4,"last update":"Wed Sep 18 10:15:46 2024","interfaces":[{"interface":"back","average":{"1min":11.678000000000001,"5min":11.678000000000001,"15min":11.678000000000001},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":90.438000000000002,"5min":90.438000000000002,"15min":90.438000000000002},"last":1.5589999999999999},{"interface":"front","average":{"1min":13.071999999999999,"5min":13.071999999999999,"15min":13.071999999999999},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":93.939999999999998,"5min":93.939999999999998,"15min":93.939999999999998},"last":2.1309999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4199999999999999}]},{"osd":6,"last update":"Wed Sep 18 10:15:46 2024","interfaces":[{"interface":"back","average":{"1min":18.465,"5min":18.465,"15min":18.465},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":293.26900000000001,"5min":293.26900000000001,"15min":293.26900000000001},"last":2.0350000000000001},{"interface":"front","average":{"1min":6.1559999999999997,"5min":6.1559999999999997,"15min":6.1559999999999997},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":70.203000000000003,"5min":70.203000000000003,"15min":70.203000000000003},"last":1.839}]},{"osd":7,"last update":"Wed Sep 18 10:15:33 2024","interfaces":[{"interface":"back","average":{"1min":2.71,"5min":2.71,"15min":2.71},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":26.361000000000001,"5min":26.361000000000001,"15min":26.361000000000001},"last":2.2810000000000001},{"interface":"front","average":{"1min":6.0709999999999997,"5min":6.0709999999999997,"15min":6.0709999999999997},"min":{"1min":0.878,"5min":0.878,"15min":0.878},"max":{"1min":79.186999999999998,"5min":79.186999999999998,"15min":79.186999999999998},"last":2.3170000000000002}]},{"osd":9,"last update":"Wed Sep 18 10:15:33 2024","interfaces":[{"interface":"back","average":{"1min":2.5459999999999998,"5min":2.5459999999999998,"15min":2.5459999999999998},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":23.934000000000001,"5min":23.934000000000001,"15min":23.934000000000001},"last":2.411},{"interface":"front","average":{"1min":8.9060000000000006,"5min":8.9060000000000006,"15min":8.9060000000000006},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":98.914000000000001,"5min":98.914000000000001,"15min":98.914000000000001},"last":2.5190000000000001}]},{"osd":10,"last update":"Wed Sep 18 10:15:33 2024","interfaces":[{"interface":"back","average":{"1min":6.9119999999999999,"5min":6.9119999999999999,"15min":6.9119999999999999},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":90.751000000000005,"5min":90.751000000000005,"15min":90.751000000000005},"last":2.3679999999999999},{"interface":"front","average":{"1min":9.3390000000000004,"5min":9.3390000000000004,"15min":9.3390000000000004},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":83.795000000000002,"5min":83.795000000000002,"15min":83.795000000000002},"last":2.3340000000000001}]},{"osd":11,"last update":"Wed Sep 18 10:15:33 2024","interfaces":[{"interface":"back","average":{"1min":2.9009999999999998,"5min":2.9009999999999998,"15min":2.9009999999999998},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":26.462,"5min":26.462,"15min":26.462},"last":2.383},{"interface":"front","average":{"1min":1.998,"5min":1.998,"15min":1.998},"min":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"max":{"1min":12.475,"5min":12.475,"15min":12.475},"last":2.2519999999999998}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.641}]},{"osd":13,"last update":"Wed Sep 18 10:15:33 2024","interfaces":[{"interface":"back","average":{"1min":9.1539999999999999,"5min":9.1539999999999999,"15min":9.1539999999999999},"min":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"max":{"1min":94.222999999999999,"5min":94.222999999999999,"15min":94.222999999999999},"last":2.4910000000000001},{"interface":"front","average":{"1min":2.7240000000000002,"5min":2.7240000000000002,"15min":2.7240000000000002},"min":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"max":{"1min":23.983000000000001,"5min":23.983000000000001,"15min":23.983000000000001},"last":2.4689999999999999}]},{"osd":14,"last update":"Wed Sep 18 10:15:33 2024","interfaces":[{"interface":"back","average":{"1min":20.768999999999998,"5min":20.768999999999998,"15min":20.768999999999998},"min":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":150.93600000000001,"5min":150.93600000000001,"15min":150.93600000000001},"last":2.4500000000000002},{"interface":"front","average":{"1min":3.5979999999999999,"5min":3.5979999999999999,"15min":3.5979999999999999},"min":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"max":{"1min":23.006,"5min":23.006,"15min":23.006},"last":2.4340000000000002}]},{"osd":15,"last update":"Wed Sep 18 10:15:33 2024","interfaces":[{"interface":"back","average":{"1min":2.7949999999999999,"5min":2.7949999999999999,"15min":2.7949999999999999},"min":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"max":{"1min":24.100000000000001,"5min":24.100000000000001,"15min":24.100000000000001},"last":2.206},{"interface":"front","average":{"1min":3.056,"5min":3.056,"15min":3.056},"min":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"max":{"1min":26.414000000000001,"5min":26.414000000000001,"15min":26.414000000000001},"last":2.5859999999999999}]}]},{"osd":10,"up_from":22,"seq":94489280535,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":317756,"kb_used_data":16240,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94054084,"statfs":{"total":96636764160,"available":96311382016,"internally_reserved":0,"allocated":16629760,"data_stored":21358919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,6,7,9,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4710000000000001}]},{"osd":1,"last update":"Wed Sep 18 10:15:33 2024","interfaces":[{"interface":"back","average":{"1min":10.933,"5min":10.933,"15min":10.933},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":84.203999999999994,"5min":84.203999999999994,"15min":84.203999999999994},"last":1.637},{"interface":"front","average":{"1min":17.222999999999999,"5min":17.222999999999999,"15min":17.222999999999999},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":99.995999999999995,"5min":99.995999999999995,"15min":99.995999999999995},"last":0.995}]},{"osd":2,"last update":"Wed Sep 18 10:15:33 2024","interfaces":[{"interface":"back","average":{"1min":10.893000000000001,"5min":10.893000000000001,"15min":10.893000000000001},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":84.072000000000003,"5min":84.072000000000003,"15min":84.072000000000003},"last":1.0369999999999999},{"interface":"front","average":{"1min":11.550000000000001,"5min":11.550000000000001,"15min":11.550000000000001},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":99.798000000000002,"5min":99.798000000000002,"15min":99.798000000000002},"last":1.4199999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2310000000000001}]},{"osd":4,"last update":"Wed Sep 18 10:15:49 2024","interfaces":[{"interface":"back","average":{"1min":25.588999999999999,"5min":25.588999999999999,"15min":25.588999999999999},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":125.23099999999999,"5min":125.23099999999999,"15min":125.23099999999999},"last":1.26},{"interface":"front","average":{"1min":31.167999999999999,"5min":31.167999999999999,"15min":31.167999999999999},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":295.93200000000002,"5min":295.93200000000002,"15min":295.93200000000002},"last":1.371}]},{"osd":5,"last update":"Wed Sep 18 10:15:49 2024","interfaces":[{"interface":"back","average":{"1min":24.577999999999999,"5min":24.577999999999999,"15min":24.577999999999999},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":215.51400000000001,"5min":215.51400000000001,"15min":215.51400000000001},"last":1.8169999999999999},{"interface":"front","average":{"1min":27.329999999999998,"5min":27.329999999999998,"15min":27.329999999999998},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":252.69300000000001,"5min":252.69300000000001,"15min":252.69300000000001},"last":1.5169999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.744}]},{"osd":7,"last update":"Wed Sep 18 10:15:33 2024","interfaces":[{"interface":"back","average":{"1min":3.0590000000000002,"5min":3.0590000000000002,"15min":3.0590000000000002},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":34.558,"5min":34.558,"15min":34.558},"last":1.8600000000000001},{"interface":"front","average":{"1min":13.959,"5min":13.959,"15min":13.959},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":84.123000000000005,"5min":84.123000000000005,"15min":84.123000000000005},"last":1.99}]},{"osd":9,"last update":"Wed Sep 18 10:15:33 2024","interfaces":[{"interface":"back","average":{"1min":11.747,"5min":11.747,"15min":11.747},"min":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":100.214,"5min":100.214,"15min":100.214},"last":1.8879999999999999},{"interface":"front","average":{"1min":12.548,"5min":12.548,"15min":12.548},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":97.385999999999996,"5min":97.385999999999996,"15min":97.385999999999996},"last":2.1150000000000002}]},{"osd":11,"last update":"Wed Sep 18 10:15:33 2024","interfaces":[{"interface":"back","average":{"1min":11.829000000000001,"5min":11.829000000000001,"15min":11.829000000000001},"min":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"max":{"1min":84.322999999999993,"5min":84.322999999999993,"15min":84.322999999999993},"last":2.2160000000000002},{"interface":"front","average":{"1min":23.379000000000001,"5min":23.379000000000001,"15min":23.379000000000001},"min":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":295.70400000000001,"5min":295.70400000000001,"15min":295.70400000000001},"last":1.609}]},{"osd":12,"last update":"Wed Sep 18 10:15:33 2024","interfaces":[{"interface":"back","average":{"1min":3.1179999999999999,"5min":3.1179999999999999,"15min":3.1179999999999999},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":34.625999999999998,"5min":34.625999999999998,"15min":34.625999999999998},"last":2.2559999999999998},{"interface":"front","average":{"1min":11.198,"5min":11.198,"15min":11.198},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":84.242999999999995,"5min":84.242999999999995,"15min":84.242999999999995},"last":2.0699999999999998}]},{"osd":13,"last update":"Wed Sep 18 10:15:33 2024","interfaces":[{"interface":"back","average":{"1min":19.885000000000002,"5min":19.885000000000002,"15min":19.885000000000002},"min":{"1min":0.877,"5min":0.877,"15min":0.877},"max":{"1min":100.31,"5min":100.31,"15min":100.31},"last":1.7889999999999999},{"interface":"front","average":{"1min":16.280000000000001,"5min":16.280000000000001,"15min":16.280000000000001},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":85.879999999999995,"5min":85.879999999999995,"15min":85.879999999999995},"last":2.1869999999999998}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1459999999999999}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1080000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8249344,"data_stored":11628544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12386304,"data_stored":14266368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":17313792,"data_stored":22392832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":15585280,"data_stored":22941696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":18845696,"data_stored":25616384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":10059776,"data_stored":16465920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":15511552,"data_stored":20852736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":14479360,"data_stored":22974464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":26300416,"data_stored":27844608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":11382784,"data_stored":14258176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":14938112,"data_stored":20369408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":11685888,"data_stored":15036416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":8093696,"data_stored":12627968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":23355392,"data_stored":32665600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":23445504,"data_stored":32727040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":9056256,"data_stored":12201984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T10:16:12.205 INFO:tasks.ceph.ceph_manager.ceph:removing pool_name unique_pool_0 2024-09-18T10:16:12.205 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm unique_pool_0 unique_pool_0 --yes-i-really-really-mean-it 2024-09-18T10:16:12.205 INFO:tasks.ceph.osd.12.smithi029.stderr:2024-09-18T10:16:12.185+0000 7f8e4e627700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 22190) UID: 0 2024-09-18T10:16:12.293 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-18T10:16:12.289+0000 7f14dcc5b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22398) UID: 0 2024-09-18T10:16:12.304 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-18T10:16:12.304 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-18T10:16:12.304 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-09-18T10:16:12.304 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-18T10:16:12.313 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-09-18T10:16:12.394 INFO:tasks.ceph.osd.6.smithi154.stderr:2024-09-18T10:16:12.390+0000 7f0abc9e1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22089) UID: 0 2024-09-18T10:16:12.416 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-18T10:16:12.416 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-18T10:16:12.416 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-18T10:16:12.416 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-18T10:16:12.416 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-18T10:16:12.425 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-09-18T10:16:12.495 INFO:tasks.ceph.osd.15.smithi189.stderr:2024-09-18T10:16:12.489+0000 7fd0e725e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 22401) UID: 0 2024-09-18T10:16:12.530 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-18T10:16:12.530 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-18T10:16:12.530 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-18T10:16:12.531 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-18T10:16:12.531 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.531 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.ds4 41/32 ECSubWrite(tid=41, reqid=osd.13.0:41, at_version=41'6, trim_to=0'0, roll_forward_to=37'4))", 2024-09-18T10:16:12.531 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:57.138491+0000", 2024-09-18T10:16:12.531 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 75.387631071000001, 2024-09-18T10:16:12.531 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.200128009, 2024-09-18T10:16:12.531 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:12.531 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:16:12.531 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:12.531 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.532 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:12.532 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.138491+0000", 2024-09-18T10:16:12.532 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:12.532 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.532 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.532 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:12.532 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.138491+0000", 2024-09-18T10:16:12.532 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:12.532 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.532 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.533 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:12.533 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.138490+0000", 2024-09-18T10:16:12.533 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:16:12.533 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.533 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.533 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:12.533 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.138555+0000", 2024-09-18T10:16:12.533 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4571000000000002e-05 2024-09-18T10:16:12.533 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.533 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.533 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:12.534 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.138581+0000", 2024-09-18T10:16:12.534 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6069000000000001e-05 2024-09-18T10:16:12.534 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.534 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.534 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:12.534 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.138598+0000", 2024-09-18T10:16:12.534 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7059e-05 2024-09-18T10:16:12.534 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.534 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.534 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:12.534 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.336697+0000", 2024-09-18T10:16:12.534 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.198099517 2024-09-18T10:16:12.535 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.535 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.535 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:16:12.535 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.336733+0000", 2024-09-18T10:16:12.535 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5369e-05 2024-09-18T10:16:12.535 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.535 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.535 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:16:12.535 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.338552+0000", 2024-09-18T10:16:12.535 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001819773 2024-09-18T10:16:12.535 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.536 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.536 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:12.536 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.338619+0000", 2024-09-18T10:16:12.536 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6390999999999995e-05 2024-09-18T10:16:12.536 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:12.536 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:12.536 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:12.536 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.536 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.536 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.ds4 41/32 ECSubWrite(tid=43, reqid=osd.13.0:41, at_version=0'0, trim_to=0'0, roll_forward_to=41'6))", 2024-09-18T10:16:12.536 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:57.441404+0000", 2024-09-18T10:16:12.536 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 75.084717462, 2024-09-18T10:16:12.537 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30108198600000002, 2024-09-18T10:16:12.537 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:12.537 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:16:12.537 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:12.537 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.537 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:12.537 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.441404+0000", 2024-09-18T10:16:12.537 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:12.537 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.538 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.538 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:12.538 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.441404+0000", 2024-09-18T10:16:12.538 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:12.538 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.538 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.538 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:12.538 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.441403+0000", 2024-09-18T10:16:12.538 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999981 2024-09-18T10:16:12.538 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.538 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.539 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:12.539 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.441435+0000", 2024-09-18T10:16:12.539 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2502000000000002e-05 2024-09-18T10:16:12.539 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.539 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.539 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:12.539 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.441473+0000", 2024-09-18T10:16:12.539 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8302e-05 2024-09-18T10:16:12.539 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.539 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.539 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:12.540 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.441524+0000", 2024-09-18T10:16:12.540 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0679e-05 2024-09-18T10:16:12.540 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.540 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.540 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:12.540 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.641739+0000", 2024-09-18T10:16:12.540 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20021524299999999 2024-09-18T10:16:12.540 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.540 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.540 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:16:12.541 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.641766+0000", 2024-09-18T10:16:12.541 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7081999999999999e-05 2024-09-18T10:16:12.541 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.541 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.541 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:16:12.541 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.742432+0000", 2024-09-18T10:16:12.541 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10066531300000001 2024-09-18T10:16:12.541 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.541 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.541 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:12.541 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:57.742486+0000", 2024-09-18T10:16:12.541 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4691999999999997e-05 2024-09-18T10:16:12.542 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:12.542 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:12.542 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:12.542 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.542 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.542 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:509 3.16s0 3:6c64c4fa:::smithi18925896-41:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e62)", 2024-09-18T10:16:12.542 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:17.914565+0000", 2024-09-18T10:16:12.542 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 54.611556855000003, 2024-09-18T10:16:12.543 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.059666403, 2024-09-18T10:16:12.543 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:12.543 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-18T10:16:12.543 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:16:12.543 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:16:12.543 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:12.543 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 509 2024-09-18T10:16:12.543 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.543 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:12.543 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.543 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:12.544 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.914565+0000", 2024-09-18T10:16:12.544 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:12.544 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.544 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.544 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:12.544 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.914565+0000", 2024-09-18T10:16:12.544 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:12.544 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.544 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.544 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:12.544 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.914564+0000", 2024-09-18T10:16:12.544 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:16:12.545 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.545 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.545 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:12.545 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.914570+0000", 2024-09-18T10:16:12.545 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3940000000000001e-06 2024-09-18T10:16:12.545 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.545 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.545 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:12.545 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.914585+0000", 2024-09-18T10:16:12.545 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4348000000000001e-05 2024-09-18T10:16:12.545 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.546 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.546 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:12.546 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.914591+0000", 2024-09-18T10:16:12.546 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4450000000000001e-06 2024-09-18T10:16:12.546 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.546 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.546 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:12.546 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.101342+0000", 2024-09-18T10:16:12.546 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.186750576 2024-09-18T10:16:12.546 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.546 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.546 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-18T10:16:12.547 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.101343+0000", 2024-09-18T10:16:12.547 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4300000000000001e-07 2024-09-18T10:16:12.547 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.547 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.547 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:12.547 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.334525+0000", 2024-09-18T10:16:12.547 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2331822159999999 2024-09-18T10:16:12.547 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.547 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.547 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for missing object", 2024-09-18T10:16:12.547 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.334706+0000", 2024-09-18T10:16:12.547 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00018068500000000001 2024-09-18T10:16:12.548 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.548 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.548 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:12.548 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.838318+0000", 2024-09-18T10:16:12.548 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.50361270300000005 2024-09-18T10:16:12.548 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.548 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.548 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2024-09-18T10:16:12.548 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.838493+0000", 2024-09-18T10:16:12.548 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017465199999999999 2024-09-18T10:16:12.548 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.549 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.549 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:12.549 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.972913+0000", 2024-09-18T10:16:12.549 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.134419977 2024-09-18T10:16:12.549 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.549 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.549 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:16:12.549 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.973261+0000", 2024-09-18T10:16:12.549 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00034786399999999997 2024-09-18T10:16:12.549 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.550 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.550 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:12.550 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.974231+0000", 2024-09-18T10:16:12.550 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00097053099999999995 2024-09-18T10:16:12.550 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:12.550 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:12.550 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:12.550 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.550 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.550 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDScrubReserve(3.6s5 REQUEST e63)", 2024-09-18T10:16:12.550 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:19.068302+0000", 2024-09-18T10:16:12.551 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 53.457820327, 2024-09-18T10:16:12.551 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.26520614300000001, 2024-09-18T10:16:12.551 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:12.551 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-18T10:16:12.551 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:12.551 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.551 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:12.551 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.068302+0000", 2024-09-18T10:16:12.551 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:12.551 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.551 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.551 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:12.552 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.068302+0000", 2024-09-18T10:16:12.552 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:12.552 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.552 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.552 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:12.552 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.068300+0000", 2024-09-18T10:16:12.552 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999981 2024-09-18T10:16:12.552 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.552 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.552 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:12.552 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.068311+0000", 2024-09-18T10:16:12.553 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1005999999999999e-05 2024-09-18T10:16:12.553 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.553 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.553 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:12.553 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.068345+0000", 2024-09-18T10:16:12.553 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3983000000000003e-05 2024-09-18T10:16:12.553 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.553 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.553 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:12.553 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.068372+0000", 2024-09-18T10:16:12.553 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7385999999999998e-05 2024-09-18T10:16:12.554 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.554 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.554 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:12.554 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.333409+0000", 2024-09-18T10:16:12.554 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.26503692400000001 2024-09-18T10:16:12.554 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.554 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.554 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:16:12.554 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.333444+0000", 2024-09-18T10:16:12.554 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4916000000000002e-05 2024-09-18T10:16:12.554 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.554 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.555 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:12.555 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.333508+0000", 2024-09-18T10:16:12.555 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3658000000000006e-05 2024-09-18T10:16:12.555 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:12.555 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:12.555 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:12.555 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.555 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.555 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDPGPushReply(3.16s0 63/62 [PushReplyOp(3:6c64c4fa:::smithi18925896-41:head)])", 2024-09-18T10:16:12.555 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:19.524127+0000", 2024-09-18T10:16:12.555 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 53.001994891999999, 2024-09-18T10:16:12.556 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20108187499999999, 2024-09-18T10:16:12.556 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:12.556 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:16:12.556 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:12.556 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.556 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:12.556 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.524127+0000", 2024-09-18T10:16:12.556 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:12.556 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.556 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.556 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:12.557 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.524127+0000", 2024-09-18T10:16:12.557 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:12.557 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.557 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.557 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:12.557 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.524117+0000", 2024-09-18T10:16:12.557 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999905 2024-09-18T10:16:12.557 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.557 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.557 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:12.558 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.524143+0000", 2024-09-18T10:16:12.558 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5534999999999999e-05 2024-09-18T10:16:12.558 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.558 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.558 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:12.558 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.524209+0000", 2024-09-18T10:16:12.558 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6192000000000006e-05 2024-09-18T10:16:12.558 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.558 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.558 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:12.559 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.524227+0000", 2024-09-18T10:16:12.559 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7986e-05 2024-09-18T10:16:12.559 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.559 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.559 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:12.559 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.724925+0000", 2024-09-18T10:16:12.559 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.200697662 2024-09-18T10:16:12.559 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.559 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.559 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:12.559 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.725209+0000", 2024-09-18T10:16:12.560 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000284102 2024-09-18T10:16:12.560 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:12.560 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:12.560 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:12.560 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.560 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.560 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDPGPushReply(3.16s0 63/62 [PushReplyOp(3:6c64c4fa:::smithi18925896-41:head)])", 2024-09-18T10:16:12.560 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:19.524280+0000", 2024-09-18T10:16:12.560 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 53.001841548999998, 2024-09-18T10:16:12.560 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.201340348, 2024-09-18T10:16:12.560 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:12.561 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:16:12.561 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:12.561 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.561 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:12.561 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.524280+0000", 2024-09-18T10:16:12.561 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:12.561 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.561 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.561 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:12.561 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.524280+0000", 2024-09-18T10:16:12.561 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:12.562 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.562 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.562 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:12.562 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.524274+0000", 2024-09-18T10:16:12.562 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999933 2024-09-18T10:16:12.562 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.562 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.562 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:12.562 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.524289+0000", 2024-09-18T10:16:12.562 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5722999999999998e-05 2024-09-18T10:16:12.562 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.562 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.563 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:12.563 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.524315+0000", 2024-09-18T10:16:12.563 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5188999999999999e-05 2024-09-18T10:16:12.563 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.563 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.563 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:12.563 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.524325+0000", 2024-09-18T10:16:12.563 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0736e-05 2024-09-18T10:16:12.563 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.563 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.563 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:12.563 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.725489+0000", 2024-09-18T10:16:12.564 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.201163377 2024-09-18T10:16:12.564 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.564 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.564 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:12.564 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.725621+0000", 2024-09-18T10:16:12.564 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013202600000000001 2024-09-18T10:16:12.564 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:12.564 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:12.564 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:12.564 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.565 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.565 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDPGPushReply(3.16s0 63/62 [PushReplyOp(3:6c64c4fa:::smithi18925896-41:head)])", 2024-09-18T10:16:12.565 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:19.524382+0000", 2024-09-18T10:16:12.565 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 53.001739895, 2024-09-18T10:16:12.565 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.21126635699999999, 2024-09-18T10:16:12.565 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:12.565 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:16:12.565 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:12.565 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.565 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:12.565 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.524382+0000", 2024-09-18T10:16:12.565 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:12.566 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.566 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.566 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:12.566 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.524382+0000", 2024-09-18T10:16:12.566 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:12.566 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.566 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.566 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:12.566 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.524381+0000", 2024-09-18T10:16:12.566 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999995 2024-09-18T10:16:12.566 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.566 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.567 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:12.567 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.524388+0000", 2024-09-18T10:16:12.567 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4860000000000002e-06 2024-09-18T10:16:12.567 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.567 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.567 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:12.567 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.524420+0000", 2024-09-18T10:16:12.567 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2623000000000003e-05 2024-09-18T10:16:12.567 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.567 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.567 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:12.568 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.624550+0000", 2024-09-18T10:16:12.568 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100129128 2024-09-18T10:16:12.568 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.568 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.568 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:12.568 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.726236+0000", 2024-09-18T10:16:12.568 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10168595699999999 2024-09-18T10:16:12.568 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.568 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.568 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:12.568 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.735648+0000", 2024-09-18T10:16:12.568 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0094127829999999992 2024-09-18T10:16:12.569 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:12.569 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:12.569 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:12.569 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.569 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.569 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.10s6 73/46 ECSubWrite(tid=194, reqid=client.4632.0:626, at_version=73'16, trim_to=0'0, roll_forward_to=60'14))", 2024-09-18T10:16:12.569 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:29.000022+0000", 2024-09-18T10:16:12.569 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 43.526100321000001, 2024-09-18T10:16:12.569 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20370728099999999, 2024-09-18T10:16:12.569 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:12.570 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:16:12.570 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:12.570 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.570 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:12.570 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.000022+0000", 2024-09-18T10:16:12.570 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:12.570 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.570 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.570 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:12.571 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.000022+0000", 2024-09-18T10:16:12.571 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:12.571 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.571 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.571 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:12.571 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.000020+0000", 2024-09-18T10:16:12.571 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999981 2024-09-18T10:16:12.571 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.571 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.571 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:12.572 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.000428+0000", 2024-09-18T10:16:12.572 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000408689 2024-09-18T10:16:12.572 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.572 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.572 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:12.572 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.000464+0000", 2024-09-18T10:16:12.572 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5768e-05 2024-09-18T10:16:12.572 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.572 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.572 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:12.572 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.100621+0000", 2024-09-18T10:16:12.573 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100157223 2024-09-18T10:16:12.573 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.573 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.573 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:12.573 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.200904+0000", 2024-09-18T10:16:12.573 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10028234900000001 2024-09-18T10:16:12.573 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.573 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.573 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:16:12.573 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.200921+0000", 2024-09-18T10:16:12.573 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7217000000000001e-05 2024-09-18T10:16:12.574 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.574 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.574 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:16:12.574 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.203672+0000", 2024-09-18T10:16:12.574 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0027511879999999999 2024-09-18T10:16:12.574 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.574 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.574 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:12.574 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.203729+0000", 2024-09-18T10:16:12.574 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6823999999999997e-05 2024-09-18T10:16:12.574 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:12.574 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:12.575 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:12.575 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.575 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.575 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.10s6 73/46 ECSubWrite(tid=195, reqid=client.4632.0:627, at_version=73'17, trim_to=0'0, roll_forward_to=60'14))", 2024-09-18T10:16:12.575 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:29.103724+0000", 2024-09-18T10:16:12.575 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 43.422398238, 2024-09-18T10:16:12.575 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.202940394, 2024-09-18T10:16:12.575 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:12.575 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:16:12.575 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:12.575 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.575 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:12.576 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.103724+0000", 2024-09-18T10:16:12.576 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:12.576 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.576 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.576 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:12.576 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.103724+0000", 2024-09-18T10:16:12.576 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:12.576 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.576 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.576 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:12.576 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.103722+0000", 2024-09-18T10:16:12.576 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999981 2024-09-18T10:16:12.577 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.577 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.577 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:12.577 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.104024+0000", 2024-09-18T10:16:12.577 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00030194100000000001 2024-09-18T10:16:12.577 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.577 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.577 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:12.577 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.104067+0000", 2024-09-18T10:16:12.577 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2840999999999999e-05 2024-09-18T10:16:12.577 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.577 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.578 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:12.578 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.204203+0000", 2024-09-18T10:16:12.578 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100136243 2024-09-18T10:16:12.578 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.578 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.578 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:12.578 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.304498+0000", 2024-09-18T10:16:12.578 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100294809 2024-09-18T10:16:12.578 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.578 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.578 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:16:12.578 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.304514+0000", 2024-09-18T10:16:12.579 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6163000000000001e-05 2024-09-18T10:16:12.579 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.579 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.579 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:16:12.579 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.306599+0000", 2024-09-18T10:16:12.579 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0020848759999999998 2024-09-18T10:16:12.579 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.579 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.579 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:12.579 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:29.306664+0000", 2024-09-18T10:16:12.579 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5331000000000002e-05 2024-09-18T10:16:12.579 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:12.579 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:12.580 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:12.580 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.580 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.580 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.6s5 75/40 ECSubWrite(tid=66, reqid=osd.0.0:66, at_version=75'42, trim_to=0'0, roll_forward_to=74'40))", 2024-09-18T10:16:12.580 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:30.668776+0000", 2024-09-18T10:16:12.580 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 41.857345576999997, 2024-09-18T10:16:12.580 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30218223300000002, 2024-09-18T10:16:12.580 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:12.580 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:16:12.580 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:12.580 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.581 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:12.581 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:30.668776+0000", 2024-09-18T10:16:12.581 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:12.581 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.581 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.581 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:12.581 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:30.668776+0000", 2024-09-18T10:16:12.581 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:12.581 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.581 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.581 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:12.581 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:30.668775+0000", 2024-09-18T10:16:12.582 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999986 2024-09-18T10:16:12.582 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.582 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.582 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:12.582 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:30.668821+0000", 2024-09-18T10:16:12.582 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6147999999999999e-05 2024-09-18T10:16:12.582 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.582 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.582 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:12.582 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:30.668864+0000", 2024-09-18T10:16:12.582 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3004999999999999e-05 2024-09-18T10:16:12.582 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.583 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.583 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:12.583 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:30.768961+0000", 2024-09-18T10:16:12.583 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10009678499999999 2024-09-18T10:16:12.583 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.583 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.583 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:12.583 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:30.969322+0000", 2024-09-18T10:16:12.583 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20036179000000001 2024-09-18T10:16:12.583 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.583 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.583 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:16:12.584 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:30.969349+0000", 2024-09-18T10:16:12.584 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6795999999999999e-05 2024-09-18T10:16:12.584 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.584 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.584 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:16:12.584 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:30.970891+0000", 2024-09-18T10:16:12.584 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0015418459999999999 2024-09-18T10:16:12.584 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.584 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.584 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:12.585 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:30.970959+0000", 2024-09-18T10:16:12.585 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7490000000000006e-05 2024-09-18T10:16:12.585 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:12.585 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:12.585 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:12.585 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.585 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.585 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:784 3.16s0 3:6c64c4fa:::smithi18925896-41:head [append 0~557056 in=557056b] snapc 2f=[2f,29,28,20,1f,1e,1a,11] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-18T10:16:12.585 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:40.660850+0000", 2024-09-18T10:16:12.585 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 31.865271909000001, 2024-09-18T10:16:12.586 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.97869106100000003, 2024-09-18T10:16:12.586 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:12.586 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:12.586 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:16:12.586 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:16:12.586 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:12.586 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 784 2024-09-18T10:16:12.586 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.586 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:12.586 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.586 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:12.586 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.660850+0000", 2024-09-18T10:16:12.587 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:12.587 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.587 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.587 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:12.587 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.660850+0000", 2024-09-18T10:16:12.587 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:12.587 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.587 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.587 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:12.587 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.660847+0000", 2024-09-18T10:16:12.587 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999976 2024-09-18T10:16:12.587 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.588 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.588 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:12.588 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.661846+0000", 2024-09-18T10:16:12.588 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00099824000000000006 2024-09-18T10:16:12.588 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.588 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.588 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:12.588 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.661878+0000", 2024-09-18T10:16:12.589 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2224000000000002e-05 2024-09-18T10:16:12.589 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.589 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.589 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:12.589 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.761995+0000", 2024-09-18T10:16:12.589 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100117568 2024-09-18T10:16:12.589 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.589 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.589 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:12.589 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.762894+0000", 2024-09-18T10:16:12.589 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00089831700000000004 2024-09-18T10:16:12.590 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.590 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.590 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for degraded object", 2024-09-18T10:16:12.590 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.764145+0000", 2024-09-18T10:16:12.590 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001251203 2024-09-18T10:16:12.590 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.590 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.590 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:12.590 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:41.428859+0000", 2024-09-18T10:16:12.590 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.664713848 2024-09-18T10:16:12.590 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.591 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.591 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:16:12.591 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:41.429140+0000", 2024-09-18T10:16:12.591 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00028127799999999999 2024-09-18T10:16:12.591 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.591 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.591 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:16:12.591 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:41.431693+0000", 2024-09-18T10:16:12.591 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0025525169999999998 2024-09-18T10:16:12.591 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.591 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.592 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:16:12.592 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:41.538557+0000", 2024-09-18T10:16:12.592 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.106864548 2024-09-18T10:16:12.592 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.592 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.592 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:16:12.592 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:41.639179+0000", 2024-09-18T10:16:12.592 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100622194 2024-09-18T10:16:12.592 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.592 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.592 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:12.592 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:41.639541+0000", 2024-09-18T10:16:12.593 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00036170400000000002 2024-09-18T10:16:12.593 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:12.593 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:12.593 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:12.593 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.593 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.593 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWriteReply(3.16s0 85/82 ECSubWriteReply(tid=17, last_complete=85'28, committed=1, applied=1))", 2024-09-18T10:16:12.593 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:41.437048+0000", 2024-09-18T10:16:12.593 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 31.089074253, 2024-09-18T10:16:12.593 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20252846199999999, 2024-09-18T10:16:12.594 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:12.594 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:16:12.594 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:12.594 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.594 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:12.594 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:41.437048+0000", 2024-09-18T10:16:12.594 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:12.594 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.594 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.594 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:12.595 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:41.437048+0000", 2024-09-18T10:16:12.595 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:12.595 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.595 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.595 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:12.595 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:41.437047+0000", 2024-09-18T10:16:12.595 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:16:12.595 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.595 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.595 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:12.595 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:41.437054+0000", 2024-09-18T10:16:12.596 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1670000000000002e-06 2024-09-18T10:16:12.596 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.596 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.596 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:12.596 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:41.437068+0000", 2024-09-18T10:16:12.596 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3628999999999999e-05 2024-09-18T10:16:12.596 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.596 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.596 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:12.596 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:41.437074+0000", 2024-09-18T10:16:12.596 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5860000000000001e-06 2024-09-18T10:16:12.596 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.597 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.597 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:12.597 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:41.639023+0000", 2024-09-18T10:16:12.597 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.201948668 2024-09-18T10:16:12.597 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.597 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.597 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:12.597 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:41.639576+0000", 2024-09-18T10:16:12.597 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00055334500000000001 2024-09-18T10:16:12.597 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:12.598 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:12.598 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:12.598 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.598 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.598 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWriteReply(3.16s0 85/82 ECSubWriteReply(tid=17, last_complete=85'28, committed=1, applied=1))", 2024-09-18T10:16:12.598 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:41.437537+0000", 2024-09-18T10:16:12.598 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 31.088584472000001, 2024-09-18T10:16:12.598 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.201442223, 2024-09-18T10:16:12.598 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:12.599 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:16:12.599 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:12.599 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.599 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:12.599 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:41.437537+0000", 2024-09-18T10:16:12.599 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:12.599 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.599 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.599 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:12.600 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:41.437537+0000", 2024-09-18T10:16:12.600 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:12.600 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.600 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.600 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:12.600 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:41.437537+0000", 2024-09-18T10:16:12.600 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:16:12.600 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.600 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.600 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:12.601 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:41.437542+0000", 2024-09-18T10:16:12.601 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5310000000000001e-06 2024-09-18T10:16:12.601 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.601 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.601 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:12.601 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:41.437554+0000", 2024-09-18T10:16:12.601 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2381000000000001e-05 2024-09-18T10:16:12.601 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.601 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.601 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:12.601 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:41.437564+0000", 2024-09-18T10:16:12.602 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8700000000000004e-06 2024-09-18T10:16:12.602 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.602 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.602 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:12.602 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:41.638935+0000", 2024-09-18T10:16:12.602 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20137052899999999 2024-09-18T10:16:12.602 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.602 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.602 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:12.602 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:41.638980+0000", 2024-09-18T10:16:12.602 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4743999999999997e-05 2024-09-18T10:16:12.603 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:12.603 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:12.603 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:12.603 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.603 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.603 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWriteReply(3.16s0 85/82 ECSubWriteReply(tid=19, last_complete=85'30, committed=1, applied=1))", 2024-09-18T10:16:12.603 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:41.438463+0000", 2024-09-18T10:16:12.603 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 31.087658537999999, 2024-09-18T10:16:12.603 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30126321900000003, 2024-09-18T10:16:12.603 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:12.604 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:16:12.604 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:12.604 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.604 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:12.604 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:41.438463+0000", 2024-09-18T10:16:12.604 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:12.604 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.604 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.604 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:12.604 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:41.438463+0000", 2024-09-18T10:16:12.605 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:12.605 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.605 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.605 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:12.605 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:41.438463+0000", 2024-09-18T10:16:12.605 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999995 2024-09-18T10:16:12.605 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.605 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.605 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:12.605 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:41.438468+0000", 2024-09-18T10:16:12.605 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9529999999999998e-06 2024-09-18T10:16:12.606 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.606 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.606 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:12.606 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:41.438479+0000", 2024-09-18T10:16:12.606 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1167e-05 2024-09-18T10:16:12.606 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.606 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.606 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:12.606 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:41.538570+0000", 2024-09-18T10:16:12.606 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10009143400000001 2024-09-18T10:16:12.606 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.607 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.607 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:12.607 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:41.739692+0000", 2024-09-18T10:16:12.607 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20112211599999999 2024-09-18T10:16:12.607 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.607 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.607 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:12.607 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:41.739727+0000", 2024-09-18T10:16:12.607 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4236999999999998e-05 2024-09-18T10:16:12.607 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:12.607 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:12.607 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:12.608 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.608 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.608 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWriteReply(3.16s0 85/82 ECSubWriteReply(tid=18, last_complete=85'29, committed=1, applied=1))", 2024-09-18T10:16:12.608 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:41.438999+0000", 2024-09-18T10:16:12.608 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 31.087122563000001, 2024-09-18T10:16:12.608 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.300815683, 2024-09-18T10:16:12.608 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:12.608 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:16:12.608 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:12.608 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.608 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:12.609 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:41.438999+0000", 2024-09-18T10:16:12.609 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:12.609 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.609 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.609 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:12.609 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:41.438999+0000", 2024-09-18T10:16:12.609 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:12.609 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.609 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.609 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:12.609 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:41.438999+0000", 2024-09-18T10:16:12.610 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999995 2024-09-18T10:16:12.610 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.610 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.610 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:12.610 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:41.439004+0000", 2024-09-18T10:16:12.610 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8169999999999998e-06 2024-09-18T10:16:12.610 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.610 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.610 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:12.611 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:41.439035+0000", 2024-09-18T10:16:12.611 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1405000000000002e-05 2024-09-18T10:16:12.611 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.611 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.611 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:12.611 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:41.439046+0000", 2024-09-18T10:16:12.611 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1367e-05 2024-09-18T10:16:12.611 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.611 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.611 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:12.612 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:41.739792+0000", 2024-09-18T10:16:12.612 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30074592900000002 2024-09-18T10:16:12.612 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.612 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.612 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:12.612 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:41.739815+0000", 2024-09-18T10:16:12.612 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2798000000000001e-05 2024-09-18T10:16:12.612 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:12.612 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:12.612 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:12.612 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.613 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.613 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWriteReply(3.16s0 89/82 ECSubWriteReply(tid=30, last_complete=89'37, committed=1, applied=1))", 2024-09-18T10:16:12.613 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:44.908040+0000", 2024-09-18T10:16:12.613 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 27.618082296000001, 2024-09-18T10:16:12.613 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.199742948, 2024-09-18T10:16:12.613 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:12.613 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:16:12.613 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:12.613 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.614 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:12.614 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:44.908040+0000", 2024-09-18T10:16:12.614 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:12.614 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.614 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.614 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:12.614 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:44.908040+0000", 2024-09-18T10:16:12.614 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:12.614 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.614 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.615 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:12.615 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:44.908038+0000", 2024-09-18T10:16:12.615 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:16:12.615 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.615 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.615 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:12.615 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:44.908048+0000", 2024-09-18T10:16:12.615 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3749999999999992e-06 2024-09-18T10:16:12.615 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.615 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.616 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:12.616 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:44.908069+0000", 2024-09-18T10:16:12.616 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0817000000000001e-05 2024-09-18T10:16:12.616 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.616 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.616 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:12.616 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:44.908077+0000", 2024-09-18T10:16:12.616 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7369999999999995e-06 2024-09-18T10:16:12.616 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.616 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.616 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:12.617 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.107754+0000", 2024-09-18T10:16:12.617 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.19967654400000001 2024-09-18T10:16:12.617 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.617 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.617 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:12.617 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.107783+0000", 2024-09-18T10:16:12.617 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8609e-05 2024-09-18T10:16:12.617 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:12.617 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:12.617 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:12.617 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.618 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.618 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.6s5 89/40 ECSubWrite(tid=78, reqid=osd.0.0:77, at_version=0'0, trim_to=0'0, roll_forward_to=89'48))", 2024-09-18T10:16:12.618 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:44.997782+0000", 2024-09-18T10:16:12.618 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 27.528339557999999, 2024-09-18T10:16:12.618 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.21032350799999999, 2024-09-18T10:16:12.618 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:12.618 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:16:12.618 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:12.618 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.618 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:12.618 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:44.997782+0000", 2024-09-18T10:16:12.618 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:12.619 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.619 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.619 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:12.619 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:44.997782+0000", 2024-09-18T10:16:12.619 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:12.619 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.619 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.619 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:12.619 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:44.997781+0000", 2024-09-18T10:16:12.619 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999981 2024-09-18T10:16:12.619 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.620 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.620 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:12.620 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:44.997812+0000", 2024-09-18T10:16:12.620 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1829000000000003e-05 2024-09-18T10:16:12.620 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.620 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.620 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:12.620 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:44.997854+0000", 2024-09-18T10:16:12.620 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1486000000000002e-05 2024-09-18T10:16:12.620 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.620 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.620 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:12.621 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.097955+0000", 2024-09-18T10:16:12.621 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10010105399999999 2024-09-18T10:16:12.621 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.621 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.621 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:12.621 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.108057+0000", 2024-09-18T10:16:12.621 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010101941999999999 2024-09-18T10:16:12.621 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.621 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.621 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:16:12.621 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.108069+0000", 2024-09-18T10:16:12.621 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2216e-05 2024-09-18T10:16:12.622 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.622 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.622 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:16:12.622 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.208058+0000", 2024-09-18T10:16:12.622 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.099989073999999997 2024-09-18T10:16:12.622 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.622 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.622 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:12.623 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.208106+0000", 2024-09-18T10:16:12.623 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7604e-05 2024-09-18T10:16:12.623 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:12.623 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:12.623 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:12.623 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.623 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.623 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:879 3.16s0 3:6c64c4fa:::smithi18925896-41:head [copy-from ver 84] snapc 33=[33,30,2f,28,20,1f,1a,11] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-18T10:16:12.623 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:45.696039+0000", 2024-09-18T10:16:12.623 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 26.830083139999999, 2024-09-18T10:16:12.623 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.003875471, 2024-09-18T10:16:12.624 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:12.624 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:12.624 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:16:12.624 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:16:12.624 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:12.624 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 879 2024-09-18T10:16:12.624 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.624 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:12.624 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.624 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:12.624 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.696039+0000", 2024-09-18T10:16:12.625 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:12.625 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.625 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.625 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:12.625 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.696039+0000", 2024-09-18T10:16:12.625 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:12.625 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.625 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.625 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:12.625 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.696035+0000", 2024-09-18T10:16:12.625 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999962 2024-09-18T10:16:12.625 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.626 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.626 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:12.626 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.696060+0000", 2024-09-18T10:16:12.626 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5219e-05 2024-09-18T10:16:12.626 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.626 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.626 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:12.626 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.696108+0000", 2024-09-18T10:16:12.626 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7435000000000002e-05 2024-09-18T10:16:12.626 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.626 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.627 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:12.627 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.696124+0000", 2024-09-18T10:16:12.627 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5990000000000001e-05 2024-09-18T10:16:12.627 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.627 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.627 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:12.627 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.696244+0000", 2024-09-18T10:16:12.627 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000120049 2024-09-18T10:16:12.627 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.627 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.627 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-09-18T10:16:12.628 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.696259+0000", 2024-09-18T10:16:12.628 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5265999999999999e-05 2024-09-18T10:16:12.628 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.628 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.628 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:12.628 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.700259+0000", 2024-09-18T10:16:12.628 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0039999570000000002 2024-09-18T10:16:12.628 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.628 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.628 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:16:12.628 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.700367+0000", 2024-09-18T10:16:12.629 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010851199999999999 2024-09-18T10:16:12.629 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.629 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.629 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:16:12.629 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:46.586082+0000", 2024-09-18T10:16:12.629 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.88571493199999995 2024-09-18T10:16:12.629 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.629 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.629 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:16:12.629 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:46.590196+0000", 2024-09-18T10:16:12.630 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0041135549999999996 2024-09-18T10:16:12.630 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.630 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.630 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:16:12.630 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:46.699714+0000", 2024-09-18T10:16:12.630 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.109518436 2024-09-18T10:16:12.630 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.630 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.630 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:12.630 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:46.699914+0000", 2024-09-18T10:16:12.630 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019992 2024-09-18T10:16:12.630 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:12.631 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:12.631 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:12.631 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.631 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.631 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:880 3.16s0 3:6c64c4fa:::smithi18925896-41:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e90)", 2024-09-18T10:16:12.631 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:45.696157+0000", 2024-09-18T10:16:12.631 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 26.829964890999999, 2024-09-18T10:16:12.631 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.89392841599999995, 2024-09-18T10:16:12.631 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:12.631 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-18T10:16:12.631 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:16:12.631 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:16:12.631 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:12.632 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 880 2024-09-18T10:16:12.632 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.632 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:12.632 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.632 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:12.632 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.696157+0000", 2024-09-18T10:16:12.632 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:12.632 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.632 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.632 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:12.632 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.696157+0000", 2024-09-18T10:16:12.633 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:12.633 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.633 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.633 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:12.633 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.696156+0000", 2024-09-18T10:16:12.633 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999986 2024-09-18T10:16:12.633 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.633 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.633 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:12.633 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.696164+0000", 2024-09-18T10:16:12.633 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4470000000000006e-06 2024-09-18T10:16:12.633 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.634 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.634 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:12.634 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.696186+0000", 2024-09-18T10:16:12.634 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2302000000000001e-05 2024-09-18T10:16:12.634 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.634 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.634 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:12.634 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.696236+0000", 2024-09-18T10:16:12.634 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9146e-05 2024-09-18T10:16:12.634 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.634 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.635 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:12.635 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.696374+0000", 2024-09-18T10:16:12.635 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013882200000000001 2024-09-18T10:16:12.635 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.635 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.635 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:16:12.635 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.696390+0000", 2024-09-18T10:16:12.635 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5421999999999999e-05 2024-09-18T10:16:12.635 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.635 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.635 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:12.635 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.700572+0000", 2024-09-18T10:16:12.636 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0041817460000000001 2024-09-18T10:16:12.636 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.636 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.636 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2024-09-18T10:16:12.636 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.700674+0000", 2024-09-18T10:16:12.636 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000102512 2024-09-18T10:16:12.636 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.636 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.636 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:12.636 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:46.589461+0000", 2024-09-18T10:16:12.636 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.88878669300000002 2024-09-18T10:16:12.636 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.637 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.637 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:16:12.637 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:46.589768+0000", 2024-09-18T10:16:12.637 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00030681000000000003 2024-09-18T10:16:12.637 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.637 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.637 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:12.637 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:46.590085+0000", 2024-09-18T10:16:12.637 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00031782300000000001 2024-09-18T10:16:12.637 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:12.637 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:12.638 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:12.638 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.638 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.638 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.ds4 100/94 ECSubWrite(tid=241, reqid=osd.13.0:239, at_version=0'0, trim_to=0'0, roll_forward_to=100'43))", 2024-09-18T10:16:12.638 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:56.235861+0000", 2024-09-18T10:16:12.638 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 16.290261049000001, 2024-09-18T10:16:12.638 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.201709472, 2024-09-18T10:16:12.638 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:12.638 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-18T10:16:12.638 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:12.639 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.639 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:12.639 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:56.235861+0000", 2024-09-18T10:16:12.639 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:12.639 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.639 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.639 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:12.639 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:56.235861+0000", 2024-09-18T10:16:12.639 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:12.639 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.639 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.640 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:12.640 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:56.235860+0000", 2024-09-18T10:16:12.640 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:16:12.640 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.640 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.640 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:12.640 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:56.235913+0000", 2024-09-18T10:16:12.640 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3347999999999998e-05 2024-09-18T10:16:12.640 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.640 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.640 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:12.641 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:56.235940+0000", 2024-09-18T10:16:12.641 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6846e-05 2024-09-18T10:16:12.641 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.641 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.641 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:12.641 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:56.336070+0000", 2024-09-18T10:16:12.641 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100129629 2024-09-18T10:16:12.641 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.641 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.642 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:12.642 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:56.436495+0000", 2024-09-18T10:16:12.642 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100425555 2024-09-18T10:16:12.642 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.642 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.642 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:16:12.642 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:56.436514+0000", 2024-09-18T10:16:12.642 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8963000000000001e-05 2024-09-18T10:16:12.642 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.642 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.643 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:16:12.643 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:56.437522+0000", 2024-09-18T10:16:12.643 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001007806 2024-09-18T10:16:12.643 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.643 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.643 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:12.643 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:56.437570+0000", 2024-09-18T10:16:12.643 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8291999999999998e-05 2024-09-18T10:16:12.643 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:12.643 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:12.643 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:12.644 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:12.644 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:12.644 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-18T10:16:12.645 INFO:tasks.ceph.osd.6.smithi154.stderr:2024-09-18T10:16:12.642+0000 7f0abc9e1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22089) UID: 0 2024-09-18T10:16:12.646 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_ops_in_flight 2024-09-18T10:16:12.746 INFO:tasks.ceph.osd.4.smithi029.stderr:2024-09-18T10:16:12.741+0000 7f550cf6e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22189) UID: 0 2024-09-18T10:16:12.751 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-18T10:16:12.752 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-18T10:16:12.752 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-09-18T10:16:12.752 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-18T10:16:12.761 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_blocked_ops 2024-09-18T10:16:12.847 INFO:tasks.ceph.osd.0.smithi029.stderr:2024-09-18T10:16:12.841+0000 7f302f2c3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22187) UID: 0 2024-09-18T10:16:12.848 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-18T10:16:12.853 INFO:teuthology.orchestra.run.smithi029.stderr:set osd.0 primary-affinity to 1 (8655362) 2024-09-18T10:16:12.854 INFO:teuthology.orchestra.run.smithi029.stderr:pool 'unique_pool_0' removed 2024-09-18T10:16:12.861 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-18T10:16:12.861 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-18T10:16:12.861 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-18T10:16:12.861 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-18T10:16:12.861 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-18T10:16:12.869 DEBUG:teuthology.run_tasks:Unwinding manager thrashosds 2024-09-18T10:16:12.878 INFO:tasks.thrashosds:joining thrashosds 2024-09-18T10:16:12.880 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_historic_ops 2024-09-18T10:16:12.986 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-18T10:16:12.987 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-18T10:16:12.987 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-18T10:16:12.987 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-18T10:16:12.987 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.987 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:201 3.as0 3:5dc9cbed:::smithi18925896-13:2 [read 0~212992,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:16:12.987 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:52.694813+0000", 2024-09-18T10:16:12.987 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 80.288432509000003, 2024-09-18T10:16:12.987 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.50652904899999995, 2024-09-18T10:16:12.987 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:12.987 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-18T10:16:12.987 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:16:12.988 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:16:12.988 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:12.988 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 201 2024-09-18T10:16:12.988 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.988 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:12.988 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.988 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:12.988 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694813+0000", 2024-09-18T10:16:12.988 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:12.988 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.988 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.988 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:12.988 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694813+0000", 2024-09-18T10:16:12.989 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:12.989 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.989 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.989 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:12.989 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694812+0000", 2024-09-18T10:16:12.989 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999995 2024-09-18T10:16:12.989 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.989 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.989 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:12.989 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694816+0000", 2024-09-18T10:16:12.989 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7720000000000001e-06 2024-09-18T10:16:12.989 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.989 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.989 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:12.990 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694822+0000", 2024-09-18T10:16:12.990 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5529999999999998e-06 2024-09-18T10:16:12.990 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.990 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.990 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:12.990 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694827+0000", 2024-09-18T10:16:12.990 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1500000000000001e-06 2024-09-18T10:16:12.990 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.990 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.990 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:12.990 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.797100+0000", 2024-09-18T10:16:12.990 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.102273424 2024-09-18T10:16:12.990 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.991 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.991 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:16:12.991 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.797329+0000", 2024-09-18T10:16:12.991 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00022870200000000001 2024-09-18T10:16:12.991 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.991 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.991 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:12.992 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.201342+0000", 2024-09-18T10:16:12.992 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.40401293799999999 2024-09-18T10:16:12.992 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:12.992 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:12.992 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:12.992 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.992 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.992 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:207 3.as0 3:5c566071:::smithi18925896-28:head [copy-from ver 6] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:16:12.992 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:52.694855+0000", 2024-09-18T10:16:12.993 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 80.288389706999993, 2024-09-18T10:16:12.993 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.81394701899999999, 2024-09-18T10:16:12.993 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:12.993 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:12.993 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:16:12.993 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:16:12.993 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:12.993 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 207 2024-09-18T10:16:12.993 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.993 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:12.993 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.993 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:12.993 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694855+0000", 2024-09-18T10:16:12.994 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:12.994 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.994 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.994 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:12.994 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694855+0000", 2024-09-18T10:16:12.994 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:12.994 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.994 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.994 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:12.994 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694852+0000", 2024-09-18T10:16:12.994 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999967 2024-09-18T10:16:12.994 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.994 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.994 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:12.995 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694859+0000", 2024-09-18T10:16:12.995 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.08e-06 2024-09-18T10:16:12.995 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.995 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.995 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:12.995 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694866+0000", 2024-09-18T10:16:12.995 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9249999999999998e-06 2024-09-18T10:16:12.995 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.995 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.995 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:12.995 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694870+0000", 2024-09-18T10:16:12.995 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0330000000000002e-06 2024-09-18T10:16:12.995 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.995 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.996 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:12.996 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.900743+0000", 2024-09-18T10:16:12.996 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20587325400000001 2024-09-18T10:16:12.996 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.996 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.996 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:16:12.996 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.901026+0000", 2024-09-18T10:16:12.996 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00028313600000000002 2024-09-18T10:16:12.996 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.996 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.996 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:16:12.996 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.204074+0000", 2024-09-18T10:16:12.996 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30304722299999998 2024-09-18T10:16:12.997 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.997 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.997 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:16:12.997 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.207280+0000", 2024-09-18T10:16:12.997 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0032067049999999998 2024-09-18T10:16:12.997 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.997 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.997 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:16:12.997 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.508672+0000", 2024-09-18T10:16:12.997 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30139174800000001 2024-09-18T10:16:12.997 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.997 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.997 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:12.998 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.508802+0000", 2024-09-18T10:16:12.998 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013037500000000001 2024-09-18T10:16:12.998 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:12.998 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:12.998 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:12.998 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.998 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.998 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:208 3.as0 3:5c566071:::smithi18925896-28:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:16:12.998 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:14:52.694875+0000", 2024-09-18T10:16:12.998 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 80.288370545999996, 2024-09-18T10:16:12.998 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.51199464299999997, 2024-09-18T10:16:12.998 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:12.998 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-18T10:16:12.998 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:16:12.999 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:16:12.999 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:12.999 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 208 2024-09-18T10:16:12.999 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.999 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:12.999 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.999 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:12.999 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694875+0000", 2024-09-18T10:16:12.999 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:12.999 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:12.999 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:12.999 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:12.999 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694875+0000", 2024-09-18T10:16:12.999 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:13.000 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.000 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.000 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:13.000 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694874+0000", 2024-09-18T10:16:13.000 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999995 2024-09-18T10:16:13.000 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.000 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.000 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:13.000 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694876+0000", 2024-09-18T10:16:13.000 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2369999999999999e-06 2024-09-18T10:16:13.000 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.000 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.000 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:13.001 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694881+0000", 2024-09-18T10:16:13.001 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0950000000000003e-06 2024-09-18T10:16:13.001 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.001 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.001 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:13.001 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.694887+0000", 2024-09-18T10:16:13.001 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5690000000000004e-06 2024-09-18T10:16:13.001 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.001 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.001 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:13.001 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.901475+0000", 2024-09-18T10:16:13.001 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20658803000000001 2024-09-18T10:16:13.001 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.002 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.002 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2024-09-18T10:16:13.002 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:52.901550+0000", 2024-09-18T10:16:13.002 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5020999999999999e-05 2024-09-18T10:16:13.002 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.002 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.002 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:13.002 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.206415+0000", 2024-09-18T10:16:13.002 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30486486000000002 2024-09-18T10:16:13.002 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.002 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.003 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:16:13.003 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.206634+0000", 2024-09-18T10:16:13.003 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00021925499999999999 2024-09-18T10:16:13.003 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.003 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.003 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:13.003 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:14:53.206869+0000", 2024-09-18T10:16:13.003 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00023503599999999999 2024-09-18T10:16:13.003 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:13.003 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:13.004 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:13.004 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.004 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.004 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:335 3.ds0 3:b1a14543:::smithi18925896-15:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e47)", 2024-09-18T10:16:13.004 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:02.793201+0000", 2024-09-18T10:16:13.004 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 70.190044274000002, 2024-09-18T10:16:13.004 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.51366298600000004, 2024-09-18T10:16:13.004 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:13.004 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-18T10:16:13.004 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:16:13.005 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:16:13.005 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:13.005 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 335 2024-09-18T10:16:13.005 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.005 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:13.005 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.005 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:13.005 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.793201+0000", 2024-09-18T10:16:13.005 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:13.005 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.005 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.005 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:13.005 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.793201+0000", 2024-09-18T10:16:13.006 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:13.006 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.006 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.006 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:13.006 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.793200+0000", 2024-09-18T10:16:13.006 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999986 2024-09-18T10:16:13.006 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.006 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.006 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:13.006 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.793206+0000", 2024-09-18T10:16:13.006 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2199999999999997e-06 2024-09-18T10:16:13.006 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.006 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.007 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:13.007 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.793221+0000", 2024-09-18T10:16:13.007 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5092000000000001e-05 2024-09-18T10:16:13.007 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.007 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.007 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:13.007 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.793229+0000", 2024-09-18T10:16:13.007 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1820000000000006e-06 2024-09-18T10:16:13.007 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.007 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.007 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:13.007 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.793367+0000", 2024-09-18T10:16:13.007 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000138057 2024-09-18T10:16:13.008 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.008 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.008 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:16:13.008 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.793384+0000", 2024-09-18T10:16:13.008 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6731e-05 2024-09-18T10:16:13.008 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.008 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.008 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:13.008 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.804064+0000", 2024-09-18T10:16:13.008 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010680185 2024-09-18T10:16:13.008 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.008 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.008 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2024-09-18T10:16:13.009 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:02.804199+0000", 2024-09-18T10:16:13.009 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013532999999999999 2024-09-18T10:16:13.009 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.009 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.009 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:13.009 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:03.202193+0000", 2024-09-18T10:16:13.009 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.39799325099999999 2024-09-18T10:16:13.009 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.009 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.009 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:16:13.009 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:03.202292+0000", 2024-09-18T10:16:13.009 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9628000000000002e-05 2024-09-18T10:16:13.009 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.009 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.010 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:13.010 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:03.306864+0000", 2024-09-18T10:16:13.010 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.104571572 2024-09-18T10:16:13.010 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:13.010 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:13.010 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:13.010 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.010 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.010 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:503 3.fs0 3:feb35297:::smithi18925896-37:head [append 0~753664] snapc 19=[19,18,16,12,11,10,f,e] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T10:16:13.010 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:17.908895+0000", 2024-09-18T10:16:13.010 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 55.074350293000002, 2024-09-18T10:16:13.010 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.51525800200000005, 2024-09-18T10:16:13.011 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:13.011 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:13.011 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:16:13.011 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:16:13.011 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:13.011 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 503 2024-09-18T10:16:13.011 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.011 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:13.011 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.011 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:13.011 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.908895+0000", 2024-09-18T10:16:13.011 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:13.012 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.012 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.012 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:13.012 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.908895+0000", 2024-09-18T10:16:13.012 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:13.012 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.012 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.012 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:13.012 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.908892+0000", 2024-09-18T10:16:13.012 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999976 2024-09-18T10:16:13.012 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.012 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.012 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:13.013 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.910143+0000", 2024-09-18T10:16:13.013 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0012507040000000001 2024-09-18T10:16:13.013 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.013 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.013 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:13.013 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.910167+0000", 2024-09-18T10:16:13.013 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4176999999999999e-05 2024-09-18T10:16:13.013 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.013 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.013 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:13.013 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.910178+0000", 2024-09-18T10:16:13.013 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0669999999999999e-05 2024-09-18T10:16:13.013 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.013 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.014 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:13.014 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.910329+0000", 2024-09-18T10:16:13.014 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000150746 2024-09-18T10:16:13.014 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.014 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.014 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-09-18T10:16:13.014 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.910346+0000", 2024-09-18T10:16:13.014 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7680999999999999e-05 2024-09-18T10:16:13.014 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.014 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.014 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:13.014 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.015009+0000", 2024-09-18T10:16:13.014 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.104662914 2024-09-18T10:16:13.014 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.015 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.015 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:16:13.015 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.015304+0000", 2024-09-18T10:16:13.015 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00029439899999999998 2024-09-18T10:16:13.015 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.015 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.015 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:16:13.015 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.017669+0000", 2024-09-18T10:16:13.015 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0023647910000000002 2024-09-18T10:16:13.015 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.015 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.015 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:16:13.015 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.126654+0000", 2024-09-18T10:16:13.016 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.108985055 2024-09-18T10:16:13.016 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.016 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.016 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:16:13.016 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.423961+0000", 2024-09-18T10:16:13.016 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.29730786399999998 2024-09-18T10:16:13.016 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.016 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.016 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:13.016 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.424153+0000", 2024-09-18T10:16:13.016 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019128099999999999 2024-09-18T10:16:13.016 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:13.016 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:13.016 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:13.017 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.017 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.017 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:504 3.fs0 3:feb35297:::smithi18925896-37:head [append 0~720896] snapc 19=[19,18,16,12,11,10,f,e] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T10:16:13.017 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:17.912996+0000", 2024-09-18T10:16:13.017 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 55.070249257999997, 2024-09-18T10:16:13.017 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.61195014999999997, 2024-09-18T10:16:13.017 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:13.017 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:13.017 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:16:13.017 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:16:13.017 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:13.017 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 504 2024-09-18T10:16:13.017 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.018 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:13.018 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.018 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:13.018 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.912996+0000", 2024-09-18T10:16:13.018 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:13.018 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.018 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.018 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:13.018 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.912996+0000", 2024-09-18T10:16:13.018 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:13.018 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.018 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.018 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:13.018 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.912994+0000", 2024-09-18T10:16:13.019 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999981 2024-09-18T10:16:13.019 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.019 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.019 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:13.019 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.913907+0000", 2024-09-18T10:16:13.019 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00091254300000000003 2024-09-18T10:16:13.019 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.019 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.019 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:13.019 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.913950+0000", 2024-09-18T10:16:13.019 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3019000000000003e-05 2024-09-18T10:16:13.019 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.019 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.020 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:13.020 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.913960+0000", 2024-09-18T10:16:13.020 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0305e-05 2024-09-18T10:16:13.020 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.020 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.020 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:13.020 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.914067+0000", 2024-09-18T10:16:13.020 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000106696 2024-09-18T10:16:13.020 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.020 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.020 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:16:13.020 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.914079+0000", 2024-09-18T10:16:13.020 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2330000000000001e-05 2024-09-18T10:16:13.021 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.021 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.021 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:13.021 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.119322+0000", 2024-09-18T10:16:13.021 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20524277899999999 2024-09-18T10:16:13.021 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.021 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.021 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:16:13.021 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.119508+0000", 2024-09-18T10:16:13.021 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00018648800000000001 2024-09-18T10:16:13.022 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.022 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.022 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:16:13.022 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.121315+0000", 2024-09-18T10:16:13.022 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001806483 2024-09-18T10:16:13.023 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.023 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.023 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:16:13.023 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.127304+0000", 2024-09-18T10:16:13.023 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0059895599999999997 2024-09-18T10:16:13.023 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.023 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.024 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:16:13.024 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.524828+0000", 2024-09-18T10:16:13.024 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.39752343600000001 2024-09-18T10:16:13.024 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.024 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.024 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:13.024 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.524946+0000", 2024-09-18T10:16:13.024 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000118193 2024-09-18T10:16:13.024 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:13.024 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:13.024 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:13.024 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.024 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.025 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:505 3.fs0 3:feb35297:::smithi18925896-37:head [append 0~49152] snapc 19=[19,18,16,12,11,10,f,e] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T10:16:13.025 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:17.913979+0000", 2024-09-18T10:16:13.025 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 55.069265778999998, 2024-09-18T10:16:13.025 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.71150136900000005, 2024-09-18T10:16:13.025 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:13.025 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:13.025 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:16:13.025 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:16:13.025 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:13.025 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 505 2024-09-18T10:16:13.025 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.025 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:13.025 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.026 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:13.026 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.913979+0000", 2024-09-18T10:16:13.026 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:13.026 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.026 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.026 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:13.026 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.913979+0000", 2024-09-18T10:16:13.026 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:13.026 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.026 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.026 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:13.026 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.913978+0000", 2024-09-18T10:16:13.026 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999986 2024-09-18T10:16:13.026 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.027 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.027 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:13.027 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.914061+0000", 2024-09-18T10:16:13.027 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2533000000000004e-05 2024-09-18T10:16:13.027 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.027 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.027 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:13.027 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.914077+0000", 2024-09-18T10:16:13.027 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6277e-05 2024-09-18T10:16:13.027 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.027 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.027 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:13.028 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.914087+0000", 2024-09-18T10:16:13.028 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0282000000000001e-05 2024-09-18T10:16:13.028 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.028 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.028 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:13.028 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.914144+0000", 2024-09-18T10:16:13.028 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6784999999999999e-05 2024-09-18T10:16:13.028 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.028 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.028 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:16:13.028 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.914151+0000", 2024-09-18T10:16:13.028 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.968e-06 2024-09-18T10:16:13.028 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.029 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.029 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:13.029 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.122495+0000", 2024-09-18T10:16:13.029 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.208343742 2024-09-18T10:16:13.029 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.029 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.029 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:16:13.029 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.122788+0000", 2024-09-18T10:16:13.029 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00029334199999999998 2024-09-18T10:16:13.029 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.029 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.029 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:16:13.030 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.124367+0000", 2024-09-18T10:16:13.030 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0015794699999999999 2024-09-18T10:16:13.030 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.030 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.030 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:16:13.030 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.127307+0000", 2024-09-18T10:16:13.030 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0029394780000000001 2024-09-18T10:16:13.030 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.030 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.030 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:16:13.030 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.625281+0000", 2024-09-18T10:16:13.030 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.497974166 2024-09-18T10:16:13.030 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.031 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.031 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:13.031 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.625481+0000", 2024-09-18T10:16:13.031 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019956100000000001 2024-09-18T10:16:13.031 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:13.031 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:13.031 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:13.031 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.031 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.031 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:506 3.fs0 3:feb35297:::smithi18925896-37:head [setxattr _header (57) in=7b] snapc 19=[19,18,16,12,11,10,f,e] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T10:16:13.031 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:17.914106+0000", 2024-09-18T10:16:13.031 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 55.069139512, 2024-09-18T10:16:13.031 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.71184716599999998, 2024-09-18T10:16:13.032 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:13.032 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:13.032 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:16:13.032 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:16:13.032 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:13.032 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 506 2024-09-18T10:16:13.032 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.032 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:13.032 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.032 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:13.032 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.914106+0000", 2024-09-18T10:16:13.033 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:13.033 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.033 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.033 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:13.033 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.914106+0000", 2024-09-18T10:16:13.033 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:13.033 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.033 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.033 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:13.034 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.914105+0000", 2024-09-18T10:16:13.034 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:16:13.034 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.034 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.034 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:13.034 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.914110+0000", 2024-09-18T10:16:13.034 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.045e-06 2024-09-18T10:16:13.034 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.034 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.034 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:13.034 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.914117+0000", 2024-09-18T10:16:13.034 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6510000000000001e-06 2024-09-18T10:16:13.035 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.035 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.035 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:13.035 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.914122+0000", 2024-09-18T10:16:13.035 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5839999999999996e-06 2024-09-18T10:16:13.035 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.035 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.035 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:13.035 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.014305+0000", 2024-09-18T10:16:13.035 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100183091 2024-09-18T10:16:13.035 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.035 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.035 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:16:13.036 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.014325+0000", 2024-09-18T10:16:13.036 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0032000000000001e-05 2024-09-18T10:16:13.036 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.036 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.036 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:13.036 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.125511+0000", 2024-09-18T10:16:13.036 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.111185942 2024-09-18T10:16:13.036 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.036 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.036 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:16:13.036 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.125760+0000", 2024-09-18T10:16:13.036 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00024864100000000002 2024-09-18T10:16:13.037 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.037 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.037 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:16:13.037 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.126375+0000", 2024-09-18T10:16:13.037 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00061536299999999996 2024-09-18T10:16:13.037 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.037 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.037 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:16:13.037 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.227656+0000", 2024-09-18T10:16:13.037 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.101281097 2024-09-18T10:16:13.037 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.037 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.037 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:16:13.037 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.625791+0000", 2024-09-18T10:16:13.038 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.39813470299999998 2024-09-18T10:16:13.038 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.038 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.038 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:13.038 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.625953+0000", 2024-09-18T10:16:13.038 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00016192999999999999 2024-09-18T10:16:13.038 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:13.038 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:13.038 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:13.038 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.038 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.038 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:507 3.fs0 3:feb35297:::smithi18925896-37:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e62)", 2024-09-18T10:16:13.038 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:17.914159+0000", 2024-09-18T10:16:13.038 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 55.069086396000003, 2024-09-18T10:16:13.039 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0151776809999999, 2024-09-18T10:16:13.039 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:13.039 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-18T10:16:13.039 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:16:13.039 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:16:13.039 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:13.039 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 507 2024-09-18T10:16:13.039 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.039 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:13.039 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.039 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:13.039 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.914159+0000", 2024-09-18T10:16:13.039 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:13.039 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.040 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.040 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:13.040 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.914159+0000", 2024-09-18T10:16:13.040 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:13.040 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.040 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.040 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:13.040 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.914158+0000", 2024-09-18T10:16:13.040 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:16:13.040 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.041 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.041 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:13.041 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.914162+0000", 2024-09-18T10:16:13.041 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0729999999999998e-06 2024-09-18T10:16:13.041 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.041 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.041 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:13.041 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:17.914176+0000", 2024-09-18T10:16:13.041 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4258e-05 2024-09-18T10:16:13.041 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.041 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.041 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:13.041 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.014273+0000", 2024-09-18T10:16:13.041 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10009702400000001 2024-09-18T10:16:13.042 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.042 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.042 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:13.042 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.126754+0000", 2024-09-18T10:16:13.042 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.112480581 2024-09-18T10:16:13.042 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.042 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.042 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2024-09-18T10:16:13.042 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.127051+0000", 2024-09-18T10:16:13.042 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00029721199999999999 2024-09-18T10:16:13.042 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.042 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.042 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:13.043 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.626655+0000", 2024-09-18T10:16:13.043 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.49960442399999999 2024-09-18T10:16:13.043 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.043 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.043 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:16:13.043 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.626826+0000", 2024-09-18T10:16:13.043 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000170459 2024-09-18T10:16:13.043 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.043 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.043 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:13.043 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:18.929336+0000", 2024-09-18T10:16:13.043 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30251042900000003 2024-09-18T10:16:13.043 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:13.044 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:13.044 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:13.044 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.044 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.044 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:524 3.ds0 3:b1a14543:::smithi18925896-15:head [append 0~442368] snapc 1a=[1a,19,18,16,12,11,10,f,e] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-18T10:16:13.044 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:19.452015+0000", 2024-09-18T10:16:13.044 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 53.531230413999999, 2024-09-18T10:16:13.044 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.55248773200000001, 2024-09-18T10:16:13.044 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:13.044 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:13.044 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:16:13.044 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:16:13.044 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:13.045 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 524 2024-09-18T10:16:13.045 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.045 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:13.045 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.045 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:13.045 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.452015+0000", 2024-09-18T10:16:13.045 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:13.045 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.045 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.045 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:13.045 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.452015+0000", 2024-09-18T10:16:13.045 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:13.045 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.046 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.046 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:13.046 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.452012+0000", 2024-09-18T10:16:13.046 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999976 2024-09-18T10:16:13.046 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.046 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.046 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:13.046 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.452793+0000", 2024-09-18T10:16:13.046 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00078048600000000005 2024-09-18T10:16:13.046 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.046 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.046 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:13.046 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.452825+0000", 2024-09-18T10:16:13.046 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2672999999999997e-05 2024-09-18T10:16:13.047 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.047 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.047 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:13.047 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.552970+0000", 2024-09-18T10:16:13.047 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100144728 2024-09-18T10:16:13.047 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.047 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.047 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:13.047 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.553127+0000", 2024-09-18T10:16:13.047 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000157202 2024-09-18T10:16:13.047 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.047 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.047 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:16:13.047 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.553439+0000", 2024-09-18T10:16:13.048 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00031139500000000001 2024-09-18T10:16:13.048 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.048 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.048 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:16:13.048 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.556240+0000", 2024-09-18T10:16:13.048 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0028012509999999998 2024-09-18T10:16:13.048 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.048 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.048 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:16:13.048 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.665529+0000", 2024-09-18T10:16:13.048 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10928916700000001 2024-09-18T10:16:13.048 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.049 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.049 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:16:13.049 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:20.004378+0000", 2024-09-18T10:16:13.049 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.33884916300000001 2024-09-18T10:16:13.049 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.049 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.049 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:13.049 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:20.004502+0000", 2024-09-18T10:16:13.049 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012407999999999999 2024-09-18T10:16:13.049 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:13.049 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:13.049 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:13.049 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.050 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.050 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:525 3.ds0 3:b1a14543:::smithi18925896-15:head [append 0~540672] snapc 1a=[1a,19,18,16,12,11,10,f,e] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-18T10:16:13.050 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:19.553028+0000", 2024-09-18T10:16:13.050 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 53.430216575000003, 2024-09-18T10:16:13.050 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.45756244000000001, 2024-09-18T10:16:13.050 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:13.050 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:13.050 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:16:13.050 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:16:13.050 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:13.050 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 525 2024-09-18T10:16:13.050 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.050 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:13.051 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.051 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:13.051 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.553028+0000", 2024-09-18T10:16:13.051 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:13.051 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.051 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.051 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:13.051 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.553028+0000", 2024-09-18T10:16:13.051 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:13.051 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.051 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.051 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:13.052 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.553026+0000", 2024-09-18T10:16:13.052 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999981 2024-09-18T10:16:13.052 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.052 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.052 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:13.052 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.553796+0000", 2024-09-18T10:16:13.052 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00076915200000000005 2024-09-18T10:16:13.052 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.052 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.052 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:13.053 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.553819+0000", 2024-09-18T10:16:13.053 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3008999999999999e-05 2024-09-18T10:16:13.053 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.053 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.053 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:13.053 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.553829+0000", 2024-09-18T10:16:13.053 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0543999999999999e-05 2024-09-18T10:16:13.053 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.053 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.053 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:13.053 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.558075+0000", 2024-09-18T10:16:13.053 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.004245932 2024-09-18T10:16:13.053 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.054 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.054 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:16:13.054 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.558384+0000", 2024-09-18T10:16:13.054 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00030926100000000002 2024-09-18T10:16:13.054 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.054 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.054 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:16:13.054 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.559950+0000", 2024-09-18T10:16:13.054 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001565615 2024-09-18T10:16:13.054 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.054 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.055 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:16:13.055 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.665535+0000", 2024-09-18T10:16:13.055 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.105584707 2024-09-18T10:16:13.055 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.055 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.055 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:16:13.055 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:20.010428+0000", 2024-09-18T10:16:13.055 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.34489341800000001 2024-09-18T10:16:13.055 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.056 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.056 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:13.056 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:20.010591+0000", 2024-09-18T10:16:13.056 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000162845 2024-09-18T10:16:13.056 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:13.056 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:13.056 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:13.057 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.057 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.057 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:526 3.ds0 3:b1a14543:::smithi18925896-15:head [append 0~622592] snapc 1a=[1a,19,18,16,12,11,10,f,e] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-18T10:16:13.058 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:19.553845+0000", 2024-09-18T10:16:13.058 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 53.429399859999997, 2024-09-18T10:16:13.058 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.55723060999999996, 2024-09-18T10:16:13.058 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:13.058 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:13.058 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:16:13.058 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:16:13.058 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:13.058 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 526 2024-09-18T10:16:13.058 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.058 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:13.058 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.058 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:13.059 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.553845+0000", 2024-09-18T10:16:13.059 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:13.060 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.060 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.060 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:13.060 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.553845+0000", 2024-09-18T10:16:13.060 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:13.060 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.060 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.061 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:13.061 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.553844+0000", 2024-09-18T10:16:13.061 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999986 2024-09-18T10:16:13.061 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.061 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.061 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:13.061 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.554666+0000", 2024-09-18T10:16:13.061 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00082239100000000001 2024-09-18T10:16:13.061 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.061 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.061 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:13.061 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.554679+0000", 2024-09-18T10:16:13.061 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.271e-05 2024-09-18T10:16:13.061 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.062 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.062 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:13.062 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.554685+0000", 2024-09-18T10:16:13.062 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4230000000000002e-06 2024-09-18T10:16:13.062 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.062 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.062 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:13.062 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.561235+0000", 2024-09-18T10:16:13.062 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0065498179999999998 2024-09-18T10:16:13.062 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.062 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.062 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:16:13.062 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.561523+0000", 2024-09-18T10:16:13.063 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000287862 2024-09-18T10:16:13.063 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.063 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.063 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:16:13.063 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.562904+0000", 2024-09-18T10:16:13.063 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0013810090000000001 2024-09-18T10:16:13.063 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.063 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.063 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:16:13.063 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.666083+0000", 2024-09-18T10:16:13.063 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.103178495 2024-09-18T10:16:13.063 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.063 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.063 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:16:13.063 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:20.110956+0000", 2024-09-18T10:16:13.064 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.44487371999999997 2024-09-18T10:16:13.064 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.064 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.064 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:13.064 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:20.111076+0000", 2024-09-18T10:16:13.064 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000119436 2024-09-18T10:16:13.064 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:13.064 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:13.064 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:13.064 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.064 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.064 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:527 3.ds0 3:b1a14543:::smithi18925896-15:head [setxattr _header (57) in=7b] snapc 1a=[1a,19,18,16,12,11,10,f,e] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-18T10:16:13.064 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:19.554694+0000", 2024-09-18T10:16:13.065 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 53.428551493000001, 2024-09-18T10:16:13.065 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.55695282099999999, 2024-09-18T10:16:13.065 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:13.065 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:13.065 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:16:13.065 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:16:13.065 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:13.065 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 527 2024-09-18T10:16:13.065 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.065 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:13.065 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.065 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:13.065 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.554694+0000", 2024-09-18T10:16:13.066 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:13.066 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.066 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.066 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:13.066 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.554694+0000", 2024-09-18T10:16:13.066 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:13.066 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.066 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.066 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:13.066 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.554692+0000", 2024-09-18T10:16:13.066 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:16:13.066 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.066 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.066 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:13.067 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.554698+0000", 2024-09-18T10:16:13.067 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8599999999999998e-06 2024-09-18T10:16:13.067 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.067 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.067 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:13.067 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.554707+0000", 2024-09-18T10:16:13.067 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4090000000000003e-06 2024-09-18T10:16:13.067 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.067 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.067 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:13.067 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.554712+0000", 2024-09-18T10:16:13.067 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4500000000000003e-06 2024-09-18T10:16:13.067 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.068 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.068 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:13.068 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.663747+0000", 2024-09-18T10:16:13.068 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10903526400000001 2024-09-18T10:16:13.068 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.068 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.068 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:16:13.068 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.664066+0000", 2024-09-18T10:16:13.068 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00031885099999999999 2024-09-18T10:16:13.068 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.068 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.068 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:16:13.068 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.665080+0000", 2024-09-18T10:16:13.069 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010141200000000001 2024-09-18T10:16:13.069 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.069 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.069 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:16:13.069 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:19.666476+0000", 2024-09-18T10:16:13.069 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001395456 2024-09-18T10:16:13.069 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.069 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.069 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:16:13.069 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:20.111512+0000", 2024-09-18T10:16:13.069 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.445035929 2024-09-18T10:16:13.069 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.069 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.070 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:13.070 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:20.111646+0000", 2024-09-18T10:16:13.070 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000134542 2024-09-18T10:16:13.070 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:13.070 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:13.070 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:13.070 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.070 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.070 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:687 3.ds0 3:b1a14543:::smithi18925896-15:head [read 0~2899968,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-18T10:16:13.070 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:35.580026+0000", 2024-09-18T10:16:13.070 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 37.403219381, 2024-09-18T10:16:13.070 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.51409843, 2024-09-18T10:16:13.070 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:13.070 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-18T10:16:13.071 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:16:13.071 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:16:13.071 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:13.071 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 687 2024-09-18T10:16:13.071 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.071 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:13.071 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.071 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:13.071 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:35.580026+0000", 2024-09-18T10:16:13.071 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:13.071 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.071 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.071 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:13.071 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:35.580026+0000", 2024-09-18T10:16:13.072 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:13.072 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.072 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.072 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:13.072 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:35.580025+0000", 2024-09-18T10:16:13.072 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:16:13.072 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.072 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.072 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:13.072 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:35.580032+0000", 2024-09-18T10:16:13.072 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1500000000000002e-06 2024-09-18T10:16:13.072 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.072 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.072 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:13.073 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:35.580045+0000", 2024-09-18T10:16:13.073 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3086e-05 2024-09-18T10:16:13.073 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.073 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.073 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:13.073 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:35.580054+0000", 2024-09-18T10:16:13.073 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9770000000000006e-06 2024-09-18T10:16:13.073 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.073 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.073 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:13.073 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:35.580361+0000", 2024-09-18T10:16:13.073 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00030747400000000002 2024-09-18T10:16:13.073 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.074 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.074 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:16:13.074 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:35.580372+0000", 2024-09-18T10:16:13.074 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0550000000000001e-05 2024-09-18T10:16:13.074 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.074 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.074 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:13.074 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:35.686406+0000", 2024-09-18T10:16:13.074 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10603446 2024-09-18T10:16:13.074 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.074 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.074 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:16:13.074 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:35.686592+0000", 2024-09-18T10:16:13.074 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00018559300000000001 2024-09-18T10:16:13.075 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.075 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.075 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:13.075 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:36.094124+0000", 2024-09-18T10:16:13.075 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.40753219699999998 2024-09-18T10:16:13.075 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:13.075 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:13.075 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:13.075 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.075 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.075 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:767 3.as0 3:5c566071:::smithi18925896-28:head [read 0~557056,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T10:16:13.075 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:40.627001+0000", 2024-09-18T10:16:13.075 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 32.356243702999997, 2024-09-18T10:16:13.076 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.53039000400000003, 2024-09-18T10:16:13.076 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:13.076 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-18T10:16:13.076 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:16:13.076 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:16:13.076 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:13.076 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 767 2024-09-18T10:16:13.076 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.076 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:13.076 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.076 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:13.076 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.627001+0000", 2024-09-18T10:16:13.076 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:13.076 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.077 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.077 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:13.077 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.627001+0000", 2024-09-18T10:16:13.077 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:13.077 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.077 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.077 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:13.077 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.626999+0000", 2024-09-18T10:16:13.077 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999971 2024-09-18T10:16:13.077 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.077 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.077 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:13.077 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.627016+0000", 2024-09-18T10:16:13.077 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7051999999999999e-05 2024-09-18T10:16:13.078 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.078 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.078 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:13.078 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.627055+0000", 2024-09-18T10:16:13.078 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9099e-05 2024-09-18T10:16:13.078 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.078 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.078 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:13.078 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.727187+0000", 2024-09-18T10:16:13.078 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100131951 2024-09-18T10:16:13.078 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.078 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.078 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:13.078 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.727299+0000", 2024-09-18T10:16:13.079 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011210899999999999 2024-09-18T10:16:13.079 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.079 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.079 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-09-18T10:16:13.079 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.727308+0000", 2024-09-18T10:16:13.079 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7539999999999995e-06 2024-09-18T10:16:13.079 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.079 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.079 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:13.079 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.832882+0000", 2024-09-18T10:16:13.079 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.105574662 2024-09-18T10:16:13.079 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.079 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.079 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:16:13.080 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.833170+0000", 2024-09-18T10:16:13.080 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00028771299999999999 2024-09-18T10:16:13.080 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.080 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.080 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:13.080 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:41.157391+0000", 2024-09-18T10:16:13.080 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.324221431 2024-09-18T10:16:13.080 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:13.080 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:13.080 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:13.080 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.080 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.080 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:779 3.as0 3:5dc9cbed:::smithi18925896-13:head [copy-from ver 47 in=55b] snapc 2f=[2f,29,28,20,1f,1e,1a,11] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-18T10:16:13.080 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:40.727404+0000", 2024-09-18T10:16:13.081 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 32.255840730000003, 2024-09-18T10:16:13.081 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.73144490900000003, 2024-09-18T10:16:13.081 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:13.081 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:13.081 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:16:13.081 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:16:13.081 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:13.081 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 779 2024-09-18T10:16:13.081 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.081 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:13.081 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.081 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:13.081 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.727404+0000", 2024-09-18T10:16:13.081 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:13.082 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.082 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.082 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:13.082 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.727404+0000", 2024-09-18T10:16:13.082 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:13.082 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.082 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.082 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:13.082 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.727403+0000", 2024-09-18T10:16:13.082 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999986 2024-09-18T10:16:13.082 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.082 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.082 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:13.083 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.727413+0000", 2024-09-18T10:16:13.083 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6940000000000007e-06 2024-09-18T10:16:13.083 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.083 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.083 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:13.083 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.727426+0000", 2024-09-18T10:16:13.083 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2896e-05 2024-09-18T10:16:13.083 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.083 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.083 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:13.083 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.727440+0000", 2024-09-18T10:16:13.083 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4285e-05 2024-09-18T10:16:13.083 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.083 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.084 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:13.084 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.727569+0000", 2024-09-18T10:16:13.084 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000128906 2024-09-18T10:16:13.084 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.084 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.084 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:16:13.084 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.727580+0000", 2024-09-18T10:16:13.084 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0720000000000001e-05 2024-09-18T10:16:13.084 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.084 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.084 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:13.084 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.936548+0000", 2024-09-18T10:16:13.084 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20896883299999999 2024-09-18T10:16:13.084 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.085 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.085 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:16:13.085 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:40.936872+0000", 2024-09-18T10:16:13.085 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00032318499999999998 2024-09-18T10:16:13.085 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.085 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.085 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:16:13.085 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:41.066546+0000", 2024-09-18T10:16:13.085 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.12967457600000001 2024-09-18T10:16:13.085 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.085 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.085 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:16:13.085 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:41.068954+0000", 2024-09-18T10:16:13.086 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002408154 2024-09-18T10:16:13.086 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.086 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.086 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:16:13.086 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:41.458701+0000", 2024-09-18T10:16:13.086 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.38974719899999999 2024-09-18T10:16:13.086 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.086 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.086 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:13.086 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:41.458849+0000", 2024-09-18T10:16:13.086 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014777799999999999 2024-09-18T10:16:13.086 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:13.086 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:13.086 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:13.087 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.087 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.087 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:872 3.fs0 3:f1c2a816:::smithi18925896-35:head [append 0~688128] snapc 33=[33,30,2f,28,20,1f,1a,11] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-18T10:16:13.087 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:45.694098+0000", 2024-09-18T10:16:13.087 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 27.289147389, 2024-09-18T10:16:13.087 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.45094221000000001, 2024-09-18T10:16:13.087 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:13.087 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:13.087 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:16:13.087 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:16:13.087 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:13.087 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 872 2024-09-18T10:16:13.087 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.088 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:13.088 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.088 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:13.088 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.694098+0000", 2024-09-18T10:16:13.088 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:13.088 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.088 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.088 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:13.088 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.694098+0000", 2024-09-18T10:16:13.088 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:13.088 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.088 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.088 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:13.089 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.694096+0000", 2024-09-18T10:16:13.089 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999981 2024-09-18T10:16:13.089 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.089 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.089 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:13.089 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.694986+0000", 2024-09-18T10:16:13.089 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00089013899999999997 2024-09-18T10:16:13.089 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.089 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.089 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:13.089 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.695010+0000", 2024-09-18T10:16:13.089 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4301999999999999e-05 2024-09-18T10:16:13.089 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.089 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.090 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:13.090 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.695020+0000", 2024-09-18T10:16:13.090 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0118999999999999e-05 2024-09-18T10:16:13.090 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.090 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.090 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:13.090 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.896885+0000", 2024-09-18T10:16:13.090 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20186479600000001 2024-09-18T10:16:13.090 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.090 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.090 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:16:13.090 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.897089+0000", 2024-09-18T10:16:13.090 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00020354699999999999 2024-09-18T10:16:13.091 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.091 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.091 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:16:13.091 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.898797+0000", 2024-09-18T10:16:13.091 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001708892 2024-09-18T10:16:13.091 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.091 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.091 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:16:13.091 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:46.003761+0000", 2024-09-18T10:16:13.091 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.104963166 2024-09-18T10:16:13.091 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.091 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.091 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:16:13.091 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:46.144837+0000", 2024-09-18T10:16:13.092 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.14107618899999999 2024-09-18T10:16:13.092 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.092 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.092 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:13.092 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:46.145040+0000", 2024-09-18T10:16:13.092 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00020301999999999999 2024-09-18T10:16:13.092 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:13.092 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:13.092 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:13.092 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.092 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.092 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:873 3.fs0 3:f1c2a816:::smithi18925896-35:head [append 0~245760] snapc 33=[33,30,2f,28,20,1f,1a,11] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-18T10:16:13.092 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:45.695912+0000", 2024-09-18T10:16:13.093 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 27.287332718999998, 2024-09-18T10:16:13.093 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.44955376499999999, 2024-09-18T10:16:13.093 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:13.093 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:13.093 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:16:13.093 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:16:13.093 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:13.093 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 873 2024-09-18T10:16:13.093 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.093 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:13.093 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.093 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:13.093 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.695912+0000", 2024-09-18T10:16:13.093 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:13.094 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.094 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.094 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:13.094 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.695912+0000", 2024-09-18T10:16:13.094 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:13.094 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.094 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.094 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:13.094 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.695911+0000", 2024-09-18T10:16:13.094 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999986 2024-09-18T10:16:13.094 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.094 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.094 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:13.094 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.696222+0000", 2024-09-18T10:16:13.095 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00031110699999999997 2024-09-18T10:16:13.095 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.095 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.095 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:13.095 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.696246+0000", 2024-09-18T10:16:13.095 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4116000000000001e-05 2024-09-18T10:16:13.095 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.095 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.095 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:13.095 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.696257+0000", 2024-09-18T10:16:13.095 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1304999999999999e-05 2024-09-18T10:16:13.095 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.095 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.096 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:13.096 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.900046+0000", 2024-09-18T10:16:13.096 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20378872100000001 2024-09-18T10:16:13.096 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.096 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.096 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:16:13.096 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.900178+0000", 2024-09-18T10:16:13.096 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013230100000000001 2024-09-18T10:16:13.096 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.096 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.096 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:16:13.096 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.901011+0000", 2024-09-18T10:16:13.096 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00083270399999999997 2024-09-18T10:16:13.096 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.097 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.097 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:16:13.097 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:46.003763+0000", 2024-09-18T10:16:13.097 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.102751922 2024-09-18T10:16:13.097 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.097 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.097 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:16:13.097 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:46.145329+0000", 2024-09-18T10:16:13.097 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.141565937 2024-09-18T10:16:13.097 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.097 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.097 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:13.097 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:46.145466+0000", 2024-09-18T10:16:13.098 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013717199999999999 2024-09-18T10:16:13.098 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:13.098 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:13.098 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:13.098 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.098 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.098 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:874 3.fs0 3:f1c2a816:::smithi18925896-35:head [setxattr _header (57) in=7b] snapc 33=[33,30,2f,28,20,1f,1a,11] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-18T10:16:13.098 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:45.696272+0000", 2024-09-18T10:16:13.098 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 27.286973421999999, 2024-09-18T10:16:13.098 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.64978053899999999, 2024-09-18T10:16:13.098 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:13.098 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:13.098 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:16:13.098 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:16:13.099 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:13.099 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 874 2024-09-18T10:16:13.099 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.099 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:13.099 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.099 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:13.099 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.696272+0000", 2024-09-18T10:16:13.099 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:13.099 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.099 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.099 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:13.099 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.696272+0000", 2024-09-18T10:16:13.099 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:13.099 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.100 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.100 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:13.100 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.696270+0000", 2024-09-18T10:16:13.100 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.9999986 2024-09-18T10:16:13.100 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.100 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.100 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:13.100 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.696278+0000", 2024-09-18T10:16:13.100 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9799999999999998e-06 2024-09-18T10:16:13.100 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.100 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.100 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:13.100 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.696293+0000", 2024-09-18T10:16:13.101 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4842e-05 2024-09-18T10:16:13.101 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.101 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.101 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:13.101 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.696303+0000", 2024-09-18T10:16:13.101 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8879999999999992e-06 2024-09-18T10:16:13.101 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.101 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.101 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:13.101 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:46.001771+0000", 2024-09-18T10:16:13.101 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30546843499999998 2024-09-18T10:16:13.101 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.101 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.101 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:16:13.102 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:46.001984+0000", 2024-09-18T10:16:13.102 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000212456 2024-09-18T10:16:13.102 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.102 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.102 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-09-18T10:16:13.102 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:46.002868+0000", 2024-09-18T10:16:13.102 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00088443599999999995 2024-09-18T10:16:13.102 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.102 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.102 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-09-18T10:16:13.102 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:46.004417+0000", 2024-09-18T10:16:13.102 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001548724 2024-09-18T10:16:13.102 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.102 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.103 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-18T10:16:13.103 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:46.345827+0000", 2024-09-18T10:16:13.103 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.34141017200000001 2024-09-18T10:16:13.103 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.103 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.103 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:13.103 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:46.346052+0000", 2024-09-18T10:16:13.103 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00022503200000000001 2024-09-18T10:16:13.103 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:13.103 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:13.103 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:13.103 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.103 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.103 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4632.0:875 3.fs0 3:f1c2a816:::smithi18925896-35:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e90)", 2024-09-18T10:16:13.104 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-18T10:15:45.696315+0000", 2024-09-18T10:16:13.104 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 27.286930387000002, 2024-09-18T10:16:13.104 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.95397443599999998, 2024-09-18T10:16:13.104 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-18T10:16:13.104 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-18T10:16:13.104 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-18T10:16:13.104 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4632", 2024-09-18T10:16:13.104 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:13.104 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 875 2024-09-18T10:16:13.104 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.104 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-18T10:16:13.104 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.104 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-18T10:16:13.105 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.696315+0000", 2024-09-18T10:16:13.105 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:13.105 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.105 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.105 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-18T10:16:13.105 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.696315+0000", 2024-09-18T10:16:13.105 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-18T10:16:13.105 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.105 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.105 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-18T10:16:13.105 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.696313+0000", 2024-09-18T10:16:13.105 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4294967295.999999 2024-09-18T10:16:13.105 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.105 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.106 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-18T10:16:13.106 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.696320+0000", 2024-09-18T10:16:13.106 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0759999999999999e-06 2024-09-18T10:16:13.106 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.106 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.106 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-18T10:16:13.106 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.696340+0000", 2024-09-18T10:16:13.106 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0465000000000001e-05 2024-09-18T10:16:13.106 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.106 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.106 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-18T10:16:13.106 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:45.696352+0000", 2024-09-18T10:16:13.106 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1732e-05 2024-09-18T10:16:13.106 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.107 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.107 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:13.107 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:46.105544+0000", 2024-09-18T10:16:13.107 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.40919195000000003 2024-09-18T10:16:13.107 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.107 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.107 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2024-09-18T10:16:13.107 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:46.105758+0000", 2024-09-18T10:16:13.107 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00021451599999999999 2024-09-18T10:16:13.107 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.107 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.107 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-18T10:16:13.107 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:46.347069+0000", 2024-09-18T10:16:13.107 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.24131116899999999 2024-09-18T10:16:13.108 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.108 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.108 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-18T10:16:13.108 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:46.347236+0000", 2024-09-18T10:16:13.108 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00016639399999999999 2024-09-18T10:16:13.108 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-18T10:16:13.108 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-18T10:16:13.108 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-18T10:16:13.108 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-18T10:15:46.650289+0000", 2024-09-18T10:16:13.108 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30305331899999999 2024-09-18T10:16:13.108 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:13.108 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:13.108 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:13.108 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-18T10:16:13.109 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-18T10:16:13.109 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-18T10:16:13.110 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T10:16:13.219 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-09-18T10:16:13.219 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-09-18T10:16:13.219 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-09-18T10:16:13.220 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-09-18T10:16:13.229 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T10:16:13.333 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-09-18T10:16:13.333 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-09-18T10:16:13.333 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-09-18T10:16:13.333 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-09-18T10:16:13.333 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-09-18T10:16:13.342 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T10:16:13.442 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-09-18T10:16:13.442 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-09-18T10:16:13.442 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-09-18T10:16:13.442 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-09-18T10:16:13.442 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.442 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.9s1 36/32 ECSubWrite(tid=16, reqid=client.4632.0:143, at_version=36'12, trim_to=0'0, roll_forward_to=36'9))", 2024-09-18T10:16:13.442 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:51.130949+0000", 2024-09-18T10:16:13.443 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 82.307630338999999, 2024-09-18T10:16:13.443 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.202020333, 2024-09-18T10:16:13.443 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:13.443 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:16:13.443 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:13.443 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.443 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:13.443 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.130949+0000", 2024-09-18T10:16:13.443 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:13.443 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.443 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.443 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:13.443 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.130949+0000", 2024-09-18T10:16:13.444 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:13.444 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.444 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.444 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:13.444 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.130947+0000", 2024-09-18T10:16:13.444 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:16:13.444 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.444 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.444 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:13.444 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.131149+0000", 2024-09-18T10:16:13.444 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00020114800000000001 2024-09-18T10:16:13.444 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.445 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.445 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:13.445 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.131182+0000", 2024-09-18T10:16:13.445 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.3844000000000003e-05 2024-09-18T10:16:13.445 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.445 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.445 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:13.445 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.131204+0000", 2024-09-18T10:16:13.445 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.1753999999999999e-05 2024-09-18T10:16:13.445 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.445 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.445 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:13.445 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.231367+0000", 2024-09-18T10:16:13.446 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100162472 2024-09-18T10:16:13.446 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.446 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.446 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:13.446 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.231380+0000", 2024-09-18T10:16:13.446 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3624e-05 2024-09-18T10:16:13.446 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.446 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.446 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:13.446 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.332934+0000", 2024-09-18T10:16:13.446 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.101553714 2024-09-18T10:16:13.447 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.447 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.447 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:13.447 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.332969+0000", 2024-09-18T10:16:13.447 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5525999999999998e-05 2024-09-18T10:16:13.447 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:13.447 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:13.447 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:13.447 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.447 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.447 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.9s1 36/32 ECSubWrite(tid=17, reqid=client.4632.0:144, at_version=36'13, trim_to=0'0, roll_forward_to=36'9))", 2024-09-18T10:16:13.447 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:51.131916+0000", 2024-09-18T10:16:13.447 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 82.306663512, 2024-09-18T10:16:13.448 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20110955899999999, 2024-09-18T10:16:13.448 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:13.448 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:16:13.448 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:13.448 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.448 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:13.448 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.131916+0000", 2024-09-18T10:16:13.448 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:13.448 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.448 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.448 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:13.448 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.131916+0000", 2024-09-18T10:16:13.448 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:13.448 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.449 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.449 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:13.449 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.131915+0000", 2024-09-18T10:16:13.449 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:16:13.449 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.449 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.449 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:13.449 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.131949+0000", 2024-09-18T10:16:13.449 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.4576999999999998e-05 2024-09-18T10:16:13.449 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.449 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.449 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:13.450 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.131979+0000", 2024-09-18T10:16:13.450 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9454e-05 2024-09-18T10:16:13.450 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.450 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.450 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:13.450 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.131992+0000", 2024-09-18T10:16:13.450 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3029e-05 2024-09-18T10:16:13.450 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.450 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.450 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:13.450 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.332112+0000", 2024-09-18T10:16:13.450 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.200120295 2024-09-18T10:16:13.450 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.451 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.451 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:13.451 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.332126+0000", 2024-09-18T10:16:13.451 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4494999999999999e-05 2024-09-18T10:16:13.451 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.451 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.451 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:13.451 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.332996+0000", 2024-09-18T10:16:13.451 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00086989000000000001 2024-09-18T10:16:13.451 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.451 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.451 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:13.452 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.333026+0000", 2024-09-18T10:16:13.452 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9159999999999999e-05 2024-09-18T10:16:13.452 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:13.452 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:13.452 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:13.452 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.452 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.452 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpRead(3.4s3 36/32 ECSubRead(tid=11, to_read={3:2650be24:::smithi18925896-45:head=0,4096,0}, subchunks={3:2650be24:::smithi18925896-45:head=[0,1]}, attrs_to_read=))", 2024-09-18T10:16:13.452 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:51.425236+0000", 2024-09-18T10:16:13.452 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 82.013343425000002, 2024-09-18T10:16:13.452 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.200812289, 2024-09-18T10:16:13.452 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:13.453 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:16:13.453 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:13.453 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.453 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:13.453 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.425236+0000", 2024-09-18T10:16:13.453 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:13.453 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.453 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.453 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:13.453 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.425236+0000", 2024-09-18T10:16:13.453 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:13.453 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.453 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.454 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:13.454 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.425234+0000", 2024-09-18T10:16:13.454 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:16:13.454 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.454 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.454 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:13.454 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.425253+0000", 2024-09-18T10:16:13.454 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8468999999999999e-05 2024-09-18T10:16:13.454 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.454 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.454 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:13.454 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.425289+0000", 2024-09-18T10:16:13.454 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.6514999999999999e-05 2024-09-18T10:16:13.454 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.455 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.455 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:13.455 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.525424+0000", 2024-09-18T10:16:13.455 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100135157 2024-09-18T10:16:13.455 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.455 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.455 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:13.455 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.625672+0000", 2024-09-18T10:16:13.455 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100247555 2024-09-18T10:16:13.455 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.455 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.455 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:13.455 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.626048+0000", 2024-09-18T10:16:13.455 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00037635000000000001 2024-09-18T10:16:13.456 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:13.456 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:13.456 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:13.456 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.456 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.456 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.9s1 37/32 ECSubWrite(tid=30, reqid=client.4632.0:187, at_version=0'0, trim_to=0'0, roll_forward_to=37'17))", 2024-09-18T10:16:13.456 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:53.090512+0000", 2024-09-18T10:16:13.456 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 80.348067571000001, 2024-09-18T10:16:13.456 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.21478031, 2024-09-18T10:16:13.456 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:13.456 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:16:13.456 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:13.456 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.457 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:13.457 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.090512+0000", 2024-09-18T10:16:13.457 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:13.457 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.457 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.457 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:13.457 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.090512+0000", 2024-09-18T10:16:13.457 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:13.457 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.457 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.457 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:13.457 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.090510+0000", 2024-09-18T10:16:13.457 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:16:13.457 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.458 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.458 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:13.458 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.090535+0000", 2024-09-18T10:16:13.458 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4848999999999999e-05 2024-09-18T10:16:13.458 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.458 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.458 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:13.458 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.090566+0000", 2024-09-18T10:16:13.458 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1096000000000002e-05 2024-09-18T10:16:13.458 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.458 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.458 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:13.458 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.190713+0000", 2024-09-18T10:16:13.459 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10014709300000001 2024-09-18T10:16:13.459 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.459 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.459 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:13.459 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.290966+0000", 2024-09-18T10:16:13.459 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10025297399999999 2024-09-18T10:16:13.459 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.459 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.459 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:13.459 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.290981+0000", 2024-09-18T10:16:13.459 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4960000000000001e-05 2024-09-18T10:16:13.459 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.459 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.459 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:13.460 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.305252+0000", 2024-09-18T10:16:13.460 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.014270738999999999 2024-09-18T10:16:13.460 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.460 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.460 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:13.460 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.305292+0000", 2024-09-18T10:16:13.460 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.0092000000000002e-05 2024-09-18T10:16:13.460 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:13.460 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:13.460 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:13.460 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.460 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.460 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.6s3 41/40 ECSubWrite(tid=32, reqid=client.4632.0:250, at_version=41'14, trim_to=0'0, roll_forward_to=0'0))", 2024-09-18T10:16:13.460 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:57.552410+0000", 2024-09-18T10:16:13.461 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 75.886169465999998, 2024-09-18T10:16:13.461 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20275494299999999, 2024-09-18T10:16:13.461 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:13.461 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:16:13.461 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:13.461 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.461 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:13.461 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.552410+0000", 2024-09-18T10:16:13.461 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:13.461 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.461 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.461 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:13.461 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.552410+0000", 2024-09-18T10:16:13.462 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:13.462 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.462 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.462 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:13.462 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.552409+0000", 2024-09-18T10:16:13.462 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999995 2024-09-18T10:16:13.462 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.462 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.462 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:13.462 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.552478+0000", 2024-09-18T10:16:13.462 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.9084999999999993e-05 2024-09-18T10:16:13.462 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.462 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.463 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:13.463 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.552488+0000", 2024-09-18T10:16:13.463 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.7990000000000005e-06 2024-09-18T10:16:13.463 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.463 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.463 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:13.463 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.552496+0000", 2024-09-18T10:16:13.463 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.3200000000000002e-06 2024-09-18T10:16:13.463 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.463 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.463 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:13.463 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.653130+0000", 2024-09-18T10:16:13.463 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100634549 2024-09-18T10:16:13.464 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.464 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.464 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:13.464 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.653144+0000", 2024-09-18T10:16:13.464 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3619e-05 2024-09-18T10:16:13.464 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.464 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.464 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:13.464 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.755128+0000", 2024-09-18T10:16:13.464 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.101983924 2024-09-18T10:16:13.464 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.464 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.464 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:13.465 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.755165+0000", 2024-09-18T10:16:13.465 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.7361999999999999e-05 2024-09-18T10:16:13.465 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:13.465 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:13.465 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:13.465 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.465 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.465 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.6s3 41/40 ECSubWrite(tid=33, reqid=client.4632.0:251, at_version=41'15, trim_to=0'0, roll_forward_to=0'0))", 2024-09-18T10:16:13.465 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:57.553805+0000", 2024-09-18T10:16:13.465 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 75.884774886000002, 2024-09-18T10:16:13.465 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20199056600000001, 2024-09-18T10:16:13.465 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:13.465 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:16:13.465 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:13.466 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.466 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:13.468 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.553805+0000", 2024-09-18T10:16:13.468 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:13.468 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.468 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.468 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:13.468 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.553805+0000", 2024-09-18T10:16:13.468 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:13.468 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.469 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.469 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:13.469 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.553804+0000", 2024-09-18T10:16:13.469 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:16:13.469 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.469 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.469 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:13.469 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.553827+0000", 2024-09-18T10:16:13.469 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.3516e-05 2024-09-18T10:16:13.469 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.469 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.469 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:13.469 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.553845+0000", 2024-09-18T10:16:13.470 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7881999999999999e-05 2024-09-18T10:16:13.470 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.470 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.470 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:13.470 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.553853+0000", 2024-09-18T10:16:13.470 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.7109999999999995e-06 2024-09-18T10:16:13.470 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.470 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.470 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:13.470 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.754527+0000", 2024-09-18T10:16:13.470 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.200674198 2024-09-18T10:16:13.470 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.470 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.471 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:13.471 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.754543+0000", 2024-09-18T10:16:13.471 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.5682e-05 2024-09-18T10:16:13.471 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.471 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.471 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:13.471 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.755758+0000", 2024-09-18T10:16:13.471 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0012155860000000001 2024-09-18T10:16:13.471 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.471 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.471 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:13.471 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.755795+0000", 2024-09-18T10:16:13.471 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.6808999999999999e-05 2024-09-18T10:16:13.471 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:13.472 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:13.472 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:13.472 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.472 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.472 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.19s1 51/40 ECSubWrite(tid=77, reqid=client.4632.0:407, at_version=51'45, trim_to=0'0, roll_forward_to=50'39))", 2024-09-18T10:16:13.472 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:06.877639+0000", 2024-09-18T10:16:13.472 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 66.560940262000003, 2024-09-18T10:16:13.472 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.301778663, 2024-09-18T10:16:13.472 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:13.472 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:16:13.472 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:13.472 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.472 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:13.472 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:06.877639+0000", 2024-09-18T10:16:13.473 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:13.473 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.473 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.473 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:13.473 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:06.877639+0000", 2024-09-18T10:16:13.473 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:13.473 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.473 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.473 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:13.473 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:06.877638+0000", 2024-09-18T10:16:13.473 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:16:13.473 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.473 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.474 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:13.474 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:06.877679+0000", 2024-09-18T10:16:13.474 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.0908000000000002e-05 2024-09-18T10:16:13.474 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.474 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.474 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:13.474 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:06.877717+0000", 2024-09-18T10:16:13.474 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.7988999999999998e-05 2024-09-18T10:16:13.474 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.474 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.474 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:13.474 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:06.977811+0000", 2024-09-18T10:16:13.474 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100094533 2024-09-18T10:16:13.475 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.475 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.475 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:13.475 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:06.977916+0000", 2024-09-18T10:16:13.475 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000104823 2024-09-18T10:16:13.475 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.475 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.475 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:13.475 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:06.977932+0000", 2024-09-18T10:16:13.475 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.6099999999999998e-05 2024-09-18T10:16:13.475 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.475 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.475 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:13.475 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:07.179377+0000", 2024-09-18T10:16:13.476 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.201444861 2024-09-18T10:16:13.476 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.476 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.476 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:13.476 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:07.179418+0000", 2024-09-18T10:16:13.476 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.0645999999999997e-05 2024-09-18T10:16:13.476 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:13.476 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:13.476 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:13.476 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.476 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.476 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.19s1 51/40 ECSubWrite(tid=78, reqid=client.4632.0:408, at_version=51'46, trim_to=0'0, roll_forward_to=50'39))", 2024-09-18T10:16:13.476 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:06.977840+0000", 2024-09-18T10:16:13.477 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 66.460739556999997, 2024-09-18T10:16:13.477 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.201619306, 2024-09-18T10:16:13.477 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:13.477 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:16:13.477 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:13.477 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.477 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:13.477 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:06.977840+0000", 2024-09-18T10:16:13.477 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:13.477 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.477 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.477 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:13.477 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:06.977840+0000", 2024-09-18T10:16:13.477 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:13.478 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.478 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.478 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:13.478 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:06.977839+0000", 2024-09-18T10:16:13.478 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:16:13.478 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.478 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.478 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:13.478 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:06.977887+0000", 2024-09-18T10:16:13.478 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.8470999999999997e-05 2024-09-18T10:16:13.478 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.478 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.478 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:13.479 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:06.977924+0000", 2024-09-18T10:16:13.479 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.6729e-05 2024-09-18T10:16:13.479 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.479 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.479 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:13.479 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:06.977937+0000", 2024-09-18T10:16:13.479 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2476e-05 2024-09-18T10:16:13.479 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.479 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.479 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:13.479 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:07.078614+0000", 2024-09-18T10:16:13.479 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100677007 2024-09-18T10:16:13.479 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.479 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.480 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:13.480 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:07.078634+0000", 2024-09-18T10:16:13.480 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.0134000000000001e-05 2024-09-18T10:16:13.480 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.480 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.480 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:13.480 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:07.179429+0000", 2024-09-18T10:16:13.480 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100795736 2024-09-18T10:16:13.480 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.480 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.480 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:13.480 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:07.179459+0000", 2024-09-18T10:16:13.480 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9788999999999999e-05 2024-09-18T10:16:13.481 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:13.481 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:13.481 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:13.481 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.481 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.481 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.1s6 58/40 ECSubWrite(tid=113, reqid=osd.4.0:112, at_version=0'0, trim_to=0'0, roll_forward_to=57'20))", 2024-09-18T10:16:13.481 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:13.649564+0000", 2024-09-18T10:16:13.481 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 59.789015098, 2024-09-18T10:16:13.481 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.204868989, 2024-09-18T10:16:13.481 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:13.481 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:16:13.481 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:13.481 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.482 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:13.482 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.649564+0000", 2024-09-18T10:16:13.482 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:13.482 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.482 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.482 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:13.482 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.649564+0000", 2024-09-18T10:16:13.482 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:13.482 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.482 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.482 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:13.482 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.649563+0000", 2024-09-18T10:16:13.482 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:16:13.483 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.483 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.483 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:13.483 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.649600+0000", 2024-09-18T10:16:13.483 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.7224999999999997e-05 2024-09-18T10:16:13.483 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.483 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.483 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:13.483 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.649651+0000", 2024-09-18T10:16:13.483 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.0905000000000001e-05 2024-09-18T10:16:13.483 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.483 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.483 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:13.484 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.649669+0000", 2024-09-18T10:16:13.484 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8522999999999999e-05 2024-09-18T10:16:13.484 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.484 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.484 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:13.484 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.750460+0000", 2024-09-18T10:16:13.484 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100791013 2024-09-18T10:16:13.484 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.484 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.484 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:13.484 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.750487+0000", 2024-09-18T10:16:13.484 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6389000000000001e-05 2024-09-18T10:16:13.484 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.485 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.485 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:13.485 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.854315+0000", 2024-09-18T10:16:13.485 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.103828238 2024-09-18T10:16:13.485 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.485 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.485 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:13.485 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.854433+0000", 2024-09-18T10:16:13.485 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000118449 2024-09-18T10:16:13.485 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:13.485 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:13.485 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:13.485 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.485 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.486 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.19s1 58/40 ECSubWrite(tid=97, reqid=osd.7.0:96, at_version=0'0, trim_to=0'0, roll_forward_to=57'54))", 2024-09-18T10:16:13.486 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:13.672411+0000", 2024-09-18T10:16:13.486 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 59.766168669000002, 2024-09-18T10:16:13.486 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.28289547100000001, 2024-09-18T10:16:13.486 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:13.486 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:16:13.486 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:13.486 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.486 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:13.486 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.672411+0000", 2024-09-18T10:16:13.486 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:13.486 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.487 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.487 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:13.487 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.672411+0000", 2024-09-18T10:16:13.487 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:13.487 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.487 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.487 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:13.487 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.672410+0000", 2024-09-18T10:16:13.487 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999995 2024-09-18T10:16:13.487 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.487 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.487 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:13.487 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.672428+0000", 2024-09-18T10:16:13.487 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7116e-05 2024-09-18T10:16:13.488 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.488 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.488 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:13.488 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.672451+0000", 2024-09-18T10:16:13.488 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.3611000000000001e-05 2024-09-18T10:16:13.488 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.488 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.488 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:13.488 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.672465+0000", 2024-09-18T10:16:13.488 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3818999999999999e-05 2024-09-18T10:16:13.488 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.488 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.488 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:13.489 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.853423+0000", 2024-09-18T10:16:13.489 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.180957599 2024-09-18T10:16:13.489 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.489 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.489 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:13.489 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.853438+0000", 2024-09-18T10:16:13.489 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.5446000000000001e-05 2024-09-18T10:16:13.489 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.489 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.489 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:13.489 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.955241+0000", 2024-09-18T10:16:13.489 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.101803053 2024-09-18T10:16:13.490 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.490 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.490 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:13.490 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:13.955306+0000", 2024-09-18T10:16:13.490 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.5215000000000005e-05 2024-09-18T10:16:13.490 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:13.490 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:13.490 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:13.490 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.490 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.490 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.1s6 58/40 ECSubWrite(tid=120, reqid=osd.4.0:117, at_version=0'0, trim_to=0'0, roll_forward_to=58'22))", 2024-09-18T10:16:13.490 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:14.170897+0000", 2024-09-18T10:16:13.490 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 59.267682540000003, 2024-09-18T10:16:13.490 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20147283599999999, 2024-09-18T10:16:13.491 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:13.491 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:16:13.491 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:13.491 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.491 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:13.491 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:14.170897+0000", 2024-09-18T10:16:13.491 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:13.491 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.491 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.491 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:13.491 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:14.170897+0000", 2024-09-18T10:16:13.491 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:13.491 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.492 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.492 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:13.492 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:14.170895+0000", 2024-09-18T10:16:13.492 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:16:13.492 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.492 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.492 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:13.492 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:14.170921+0000", 2024-09-18T10:16:13.492 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5848999999999999e-05 2024-09-18T10:16:13.492 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.492 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.492 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:13.492 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:14.170998+0000", 2024-09-18T10:16:13.493 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.7341000000000004e-05 2024-09-18T10:16:13.493 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.493 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.493 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:13.493 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:14.171019+0000", 2024-09-18T10:16:13.493 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.0443000000000001e-05 2024-09-18T10:16:13.493 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.493 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.493 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:13.493 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:14.271283+0000", 2024-09-18T10:16:13.493 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100263848 2024-09-18T10:16:13.493 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.493 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.493 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:13.494 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:14.271316+0000", 2024-09-18T10:16:13.494 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.3516000000000002e-05 2024-09-18T10:16:13.494 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.494 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.494 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:13.494 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:14.372304+0000", 2024-09-18T10:16:13.494 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100987524 2024-09-18T10:16:13.494 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.494 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.494 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:13.494 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:14.372370+0000", 2024-09-18T10:16:13.494 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.6107999999999998e-05 2024-09-18T10:16:13.494 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:13.495 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:13.495 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:13.495 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.495 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.495 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.9s1 63/40 ECSubWrite(tid=118, reqid=client.4632.0:531, at_version=63'56, trim_to=0'0, roll_forward_to=60'52))", 2024-09-18T10:16:13.495 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:19.565882+0000", 2024-09-18T10:16:13.495 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 53.872697596000002, 2024-09-18T10:16:13.495 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.201669185, 2024-09-18T10:16:13.495 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:13.495 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:16:13.495 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:13.495 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.495 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:13.496 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.565882+0000", 2024-09-18T10:16:13.496 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:13.496 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.496 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.496 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:13.496 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.565882+0000", 2024-09-18T10:16:13.496 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:13.496 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.496 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.496 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:13.496 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.565881+0000", 2024-09-18T10:16:13.496 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:16:13.496 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.497 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.497 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:13.497 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.565919+0000", 2024-09-18T10:16:13.497 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8254999999999999e-05 2024-09-18T10:16:13.497 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.497 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.497 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:13.497 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.565952+0000", 2024-09-18T10:16:13.497 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.3219000000000002e-05 2024-09-18T10:16:13.497 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.497 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.497 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:13.497 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.565965+0000", 2024-09-18T10:16:13.498 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.272e-05 2024-09-18T10:16:13.498 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.498 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.498 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:13.498 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.766254+0000", 2024-09-18T10:16:13.498 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.200289512 2024-09-18T10:16:13.498 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.498 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.498 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:13.498 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.766272+0000", 2024-09-18T10:16:13.498 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7306e-05 2024-09-18T10:16:13.498 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.498 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.499 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:13.499 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.767501+0000", 2024-09-18T10:16:13.499 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001229794 2024-09-18T10:16:13.499 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.499 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.499 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:13.499 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.767551+0000", 2024-09-18T10:16:13.499 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9662e-05 2024-09-18T10:16:13.499 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:13.499 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:13.499 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:13.499 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.499 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.500 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.4s3 66/40 ECSubWrite(tid=57, reqid=osd.8.0:57, at_version=66'40, trim_to=0'0, roll_forward_to=66'38))", 2024-09-18T10:16:13.500 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:22.543395+0000", 2024-09-18T10:16:13.500 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 50.895184821999997, 2024-09-18T10:16:13.500 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20190205, 2024-09-18T10:16:13.500 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:13.500 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:16:13.500 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:13.500 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.500 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:13.500 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:22.543395+0000", 2024-09-18T10:16:13.500 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:13.500 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.500 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.501 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:13.501 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:22.543395+0000", 2024-09-18T10:16:13.501 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:13.501 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.501 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.501 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:13.501 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:22.543393+0000", 2024-09-18T10:16:13.501 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:16:13.501 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.501 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.501 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:13.501 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:22.543438+0000", 2024-09-18T10:16:13.501 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.5504999999999999e-05 2024-09-18T10:16:13.501 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.502 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.502 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:13.502 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:22.543471+0000", 2024-09-18T10:16:13.502 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2969000000000002e-05 2024-09-18T10:16:13.502 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.502 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.502 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:13.502 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:22.643568+0000", 2024-09-18T10:16:13.502 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100096327 2024-09-18T10:16:13.502 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.502 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.502 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:13.502 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:22.743796+0000", 2024-09-18T10:16:13.503 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100228206 2024-09-18T10:16:13.503 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.503 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.503 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:13.503 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:22.743818+0000", 2024-09-18T10:16:13.503 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.1778999999999999e-05 2024-09-18T10:16:13.503 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.503 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.503 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:13.503 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:22.745263+0000", 2024-09-18T10:16:13.503 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0014448729999999999 2024-09-18T10:16:13.503 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.503 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.504 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:13.504 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:22.745297+0000", 2024-09-18T10:16:13.504 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.4190999999999998e-05 2024-09-18T10:16:13.504 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:13.504 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:13.504 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:13.504 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.504 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.504 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.6s3 72/40 ECSubWrite(tid=62, reqid=osd.0.0:62, at_version=72'38, trim_to=0'0, roll_forward_to=71'36))", 2024-09-18T10:16:13.504 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:28.189402+0000", 2024-09-18T10:16:13.504 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 45.249177279999998, 2024-09-18T10:16:13.504 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.201226193, 2024-09-18T10:16:13.504 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:13.505 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:16:13.505 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:13.505 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.505 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:13.505 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.189402+0000", 2024-09-18T10:16:13.505 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:13.505 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.505 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.505 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:13.505 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.189402+0000", 2024-09-18T10:16:13.505 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:13.505 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.505 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.505 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:13.506 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.189402+0000", 2024-09-18T10:16:13.506 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999995 2024-09-18T10:16:13.506 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.506 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.506 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:13.506 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.189461+0000", 2024-09-18T10:16:13.506 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.9077999999999999e-05 2024-09-18T10:16:13.506 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.506 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.506 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:13.506 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.189484+0000", 2024-09-18T10:16:13.506 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.3353000000000002e-05 2024-09-18T10:16:13.506 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.507 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.507 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:13.507 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.289578+0000", 2024-09-18T10:16:13.507 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100094161 2024-09-18T10:16:13.507 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.507 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.507 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:13.507 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.389741+0000", 2024-09-18T10:16:13.507 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100162923 2024-09-18T10:16:13.507 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.507 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.507 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:13.507 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.389749+0000", 2024-09-18T10:16:13.508 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.2889999999999998e-06 2024-09-18T10:16:13.508 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.508 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.508 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:13.508 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.390593+0000", 2024-09-18T10:16:13.508 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00084384499999999997 2024-09-18T10:16:13.508 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.508 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.508 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:13.508 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.390628+0000", 2024-09-18T10:16:13.508 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5120999999999998e-05 2024-09-18T10:16:13.508 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:13.508 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:13.508 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:13.509 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.509 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.509 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.6s3 74/40 ECSubWrite(tid=64, reqid=osd.0.0:64, at_version=74'40, trim_to=0'0, roll_forward_to=72'38))", 2024-09-18T10:16:13.509 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:29.762852+0000", 2024-09-18T10:16:13.509 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 43.675727158000001, 2024-09-18T10:16:13.509 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.202373787, 2024-09-18T10:16:13.509 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:13.509 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:16:13.509 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:13.509 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.509 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:13.509 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.762852+0000", 2024-09-18T10:16:13.509 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:13.510 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.510 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.510 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:13.510 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.762852+0000", 2024-09-18T10:16:13.510 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:13.510 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.510 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.510 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:13.510 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.762852+0000", 2024-09-18T10:16:13.510 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999995 2024-09-18T10:16:13.510 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.510 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.510 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:13.511 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.762877+0000", 2024-09-18T10:16:13.511 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4759000000000001e-05 2024-09-18T10:16:13.511 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.511 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.511 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:13.511 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.762885+0000", 2024-09-18T10:16:13.511 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.7549999999999994e-06 2024-09-18T10:16:13.511 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.511 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.511 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:13.511 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.762900+0000", 2024-09-18T10:16:13.511 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.5364999999999999e-05 2024-09-18T10:16:13.511 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.512 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.512 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:13.512 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.963772+0000", 2024-09-18T10:16:13.512 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20087190899999999 2024-09-18T10:16:13.512 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.512 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.512 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:13.512 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.963781+0000", 2024-09-18T10:16:13.512 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.516e-06 2024-09-18T10:16:13.512 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.512 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.512 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:13.512 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.965178+0000", 2024-09-18T10:16:13.512 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0013961959999999999 2024-09-18T10:16:13.513 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.513 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.513 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:13.513 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.965226+0000", 2024-09-18T10:16:13.513 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.8578000000000001e-05 2024-09-18T10:16:13.513 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:13.513 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:13.513 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:13.513 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.513 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.513 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.17s4 85/64 ECSubWrite(tid=100, reqid=client.4632.0:758, at_version=85'61, trim_to=0'0, roll_forward_to=82'60))", 2024-09-18T10:16:13.513 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:40.728936+0000", 2024-09-18T10:16:13.513 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 32.709643237999998, 2024-09-18T10:16:13.513 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20385209100000001, 2024-09-18T10:16:13.514 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:13.514 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:16:13.514 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:13.514 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.514 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:13.514 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.728936+0000", 2024-09-18T10:16:13.514 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:13.514 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.514 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.514 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:13.514 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.728936+0000", 2024-09-18T10:16:13.514 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:13.514 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.515 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.515 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:13.515 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.728936+0000", 2024-09-18T10:16:13.515 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999995 2024-09-18T10:16:13.515 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.515 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.515 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:13.515 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.729098+0000", 2024-09-18T10:16:13.515 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00016212099999999999 2024-09-18T10:16:13.515 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.515 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.515 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:13.515 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.729129+0000", 2024-09-18T10:16:13.516 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0626000000000001e-05 2024-09-18T10:16:13.516 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.516 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.516 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:13.516 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.829269+0000", 2024-09-18T10:16:13.516 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100140541 2024-09-18T10:16:13.516 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.516 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.516 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:13.516 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.829412+0000", 2024-09-18T10:16:13.516 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00014265099999999999 2024-09-18T10:16:13.516 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.516 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.516 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:13.517 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.829431+0000", 2024-09-18T10:16:13.517 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8935000000000001e-05 2024-09-18T10:16:13.517 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.517 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.517 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:13.517 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.932702+0000", 2024-09-18T10:16:13.517 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.103271298 2024-09-18T10:16:13.517 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.517 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.517 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:13.517 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.932788+0000", 2024-09-18T10:16:13.517 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.6337999999999999e-05 2024-09-18T10:16:13.517 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:13.518 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:13.518 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:13.518 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.518 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.518 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpRead(3.19s3 85/83 ECSubRead(tid=220, to_read={3:9ab7c4c0:::smithi18925896-46:1a=0,512000,0}, subchunks={3:9ab7c4c0:::smithi18925896-46:1a=[0,1]}, attrs_to_read=))", 2024-09-18T10:16:13.518 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:41.177225+0000", 2024-09-18T10:16:13.518 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 32.261354019000002, 2024-09-18T10:16:13.518 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20067484299999999, 2024-09-18T10:16:13.518 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:13.518 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:16:13.518 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:13.518 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.518 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:13.518 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:41.177225+0000", 2024-09-18T10:16:13.519 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:13.519 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.519 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.519 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:13.519 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:41.177225+0000", 2024-09-18T10:16:13.519 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:13.519 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.519 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.519 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:13.519 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:41.177224+0000", 2024-09-18T10:16:13.519 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:16:13.519 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.519 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.519 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:13.520 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:41.177243+0000", 2024-09-18T10:16:13.520 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.9130000000000001e-05 2024-09-18T10:16:13.520 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.520 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.520 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:13.520 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:41.177280+0000", 2024-09-18T10:16:13.520 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.6646000000000001e-05 2024-09-18T10:16:13.520 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.520 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.520 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:13.520 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:41.277374+0000", 2024-09-18T10:16:13.520 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10009406899999999 2024-09-18T10:16:13.520 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.521 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.521 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:13.521 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:41.377630+0000", 2024-09-18T10:16:13.521 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10025651100000001 2024-09-18T10:16:13.521 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.521 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.521 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:13.521 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:41.377900+0000", 2024-09-18T10:16:13.521 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00027003699999999999 2024-09-18T10:16:13.521 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:13.521 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:13.521 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:13.521 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.522 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.522 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.4s3 85/82 ECSubWrite(tid=17, reqid=client.4632.0:748, at_version=0'0, trim_to=0'0, roll_forward_to=85'84))", 2024-09-18T10:16:13.522 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:41.216977+0000", 2024-09-18T10:16:13.522 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 32.221602298000001, 2024-09-18T10:16:13.522 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.203185543, 2024-09-18T10:16:13.522 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:13.522 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:16:13.522 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:13.522 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.522 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:13.522 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:41.216977+0000", 2024-09-18T10:16:13.522 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:13.522 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.522 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.523 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:13.523 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:41.216977+0000", 2024-09-18T10:16:13.523 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:13.523 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.523 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.523 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:13.523 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:41.216975+0000", 2024-09-18T10:16:13.523 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:16:13.523 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.523 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.523 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:13.523 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:41.217039+0000", 2024-09-18T10:16:13.523 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.4134999999999995e-05 2024-09-18T10:16:13.523 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.524 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.524 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:13.524 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:41.217084+0000", 2024-09-18T10:16:13.524 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.5092000000000001e-05 2024-09-18T10:16:13.524 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.524 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.524 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:13.524 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:41.217104+0000", 2024-09-18T10:16:13.524 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.9562999999999999e-05 2024-09-18T10:16:13.524 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.524 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.524 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:13.524 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:41.317676+0000", 2024-09-18T10:16:13.525 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10057176600000001 2024-09-18T10:16:13.525 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.525 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.525 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:13.525 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:41.317706+0000", 2024-09-18T10:16:13.525 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0426e-05 2024-09-18T10:16:13.525 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.525 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.525 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:13.525 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:41.420137+0000", 2024-09-18T10:16:13.525 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.102431151 2024-09-18T10:16:13.525 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.525 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.526 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:13.526 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:41.420163+0000", 2024-09-18T10:16:13.526 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5522999999999999e-05 2024-09-18T10:16:13.526 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:13.526 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:13.526 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:13.526 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.526 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.526 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.4s3 89/82 ECSubWrite(tid=32, reqid=osd.12.0:32, at_version=89'92, trim_to=0'0, roll_forward_to=88'90))", 2024-09-18T10:16:13.526 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:44.897313+0000", 2024-09-18T10:16:13.526 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 28.541266269000001, 2024-09-18T10:16:13.526 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.202844106, 2024-09-18T10:16:13.526 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:13.527 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:16:13.527 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:13.527 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.527 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:13.527 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:44.897313+0000", 2024-09-18T10:16:13.527 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:13.527 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.527 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.527 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:13.527 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:44.897313+0000", 2024-09-18T10:16:13.527 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:13.527 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.527 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.527 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:13.528 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:44.897313+0000", 2024-09-18T10:16:13.528 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999995 2024-09-18T10:16:13.528 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.528 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.528 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:13.528 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:44.897371+0000", 2024-09-18T10:16:13.528 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.8105e-05 2024-09-18T10:16:13.528 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.528 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.528 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:13.528 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:44.897400+0000", 2024-09-18T10:16:13.528 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9246000000000001e-05 2024-09-18T10:16:13.528 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.529 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.529 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:13.529 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:44.997512+0000", 2024-09-18T10:16:13.529 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10011223599999999 2024-09-18T10:16:13.529 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.529 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.529 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:13.529 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:45.097821+0000", 2024-09-18T10:16:13.529 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10030881899999999 2024-09-18T10:16:13.529 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.529 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.529 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:13.529 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:45.097855+0000", 2024-09-18T10:16:13.530 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.4421999999999997e-05 2024-09-18T10:16:13.530 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.530 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.530 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:13.530 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:45.100104+0000", 2024-09-18T10:16:13.530 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0022489089999999999 2024-09-18T10:16:13.530 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.530 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.530 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:13.530 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:45.100157+0000", 2024-09-18T10:16:13.530 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.3049e-05 2024-09-18T10:16:13.530 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:13.530 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:13.531 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:13.531 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.531 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.531 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.19s3 104/95 ECSubWrite(tid=158, reqid=osd.8.0:158, at_version=104'138, trim_to=0'0, roll_forward_to=104'136))", 2024-09-18T10:16:13.531 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:16:01.116991+0000", 2024-09-18T10:16:13.531 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 12.321588491, 2024-09-18T10:16:13.531 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.200810707, 2024-09-18T10:16:13.531 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:13.531 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:16:13.531 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:13.531 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.531 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:13.531 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:16:01.116991+0000", 2024-09-18T10:16:13.531 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:13.532 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.532 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.532 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:13.532 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:16:01.116991+0000", 2024-09-18T10:16:13.532 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:13.532 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.532 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.532 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:13.532 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:16:01.116991+0000", 2024-09-18T10:16:13.532 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999995 2024-09-18T10:16:13.532 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.532 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.532 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:13.532 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:16:01.117020+0000", 2024-09-18T10:16:13.533 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9003999999999999e-05 2024-09-18T10:16:13.533 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.533 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.533 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:13.533 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:16:01.117029+0000", 2024-09-18T10:16:13.533 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.1789999999999997e-06 2024-09-18T10:16:13.533 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.533 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.533 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:13.533 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:16:01.117039+0000", 2024-09-18T10:16:13.533 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0463e-05 2024-09-18T10:16:13.533 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.533 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.534 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:13.534 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:16:01.119741+0000", 2024-09-18T10:16:13.534 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.002701813 2024-09-18T10:16:13.534 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.534 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.534 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:13.534 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:16:01.119759+0000", 2024-09-18T10:16:13.534 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7768999999999999e-05 2024-09-18T10:16:13.534 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.534 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.534 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:13.534 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:16:01.317738+0000", 2024-09-18T10:16:13.534 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.19797904199999999 2024-09-18T10:16:13.535 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.535 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.535 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:13.535 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:16:01.317802+0000", 2024-09-18T10:16:13.535 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.3738999999999994e-05 2024-09-18T10:16:13.535 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:13.535 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:13.535 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:13.535 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:13.535 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:13.535 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-09-18T10:16:13.537 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T10:16:13.640 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-09-18T10:16:13.640 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-09-18T10:16:13.641 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-09-18T10:16:13.641 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-09-18T10:16:13.649 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T10:16:13.745 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-09-18T10:16:13.746 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-09-18T10:16:13.746 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-09-18T10:16:13.746 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-09-18T10:16:13.746 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-09-18T10:16:13.755 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T10:16:13.850 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-09-18T10:16:13.851 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-09-18T10:16:13.851 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-09-18T10:16:13.851 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-09-18T10:16:13.851 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.851 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T10:16:13.851 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:36.786936+0000", 2024-09-18T10:16:13.851 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 97.059747244999997, 2024-09-18T10:16:13.851 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.71130157999999999, 2024-09-18T10:16:13.851 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:13.851 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:13.851 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:13.852 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:16:13.852 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:16:13.852 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 72 2024-09-18T10:16:13.852 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.852 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:13.852 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.852 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:13.852 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.786936+0000", 2024-09-18T10:16:13.852 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:13.852 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.852 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.852 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:13.852 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.786936+0000", 2024-09-18T10:16:13.853 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:13.853 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.853 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.853 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:13.853 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.786935+0000", 2024-09-18T10:16:13.853 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999995 2024-09-18T10:16:13.853 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.853 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.853 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:13.853 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787018+0000", 2024-09-18T10:16:13.853 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.2511999999999999e-05 2024-09-18T10:16:13.853 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.853 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.854 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:13.854 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787028+0000", 2024-09-18T10:16:13.854 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0709e-05 2024-09-18T10:16:13.854 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.854 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.854 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:13.854 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787034+0000", 2024-09-18T10:16:13.854 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.7769999999999999e-06 2024-09-18T10:16:13.854 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.854 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.854 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:13.854 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.295095+0000", 2024-09-18T10:16:13.854 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.50806109099999996 2024-09-18T10:16:13.855 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.855 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.855 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:13.855 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.295221+0000", 2024-09-18T10:16:13.855 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00012563599999999999 2024-09-18T10:16:13.855 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.855 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.855 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:16:13.855 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.295453+0000", 2024-09-18T10:16:13.855 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000232387 2024-09-18T10:16:13.855 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.855 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.855 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:16:13.856 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.301373+0000", 2024-09-18T10:16:13.856 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0059202279999999996 2024-09-18T10:16:13.856 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.856 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.856 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:16:13.856 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.498044+0000", 2024-09-18T10:16:13.856 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.19667104699999999 2024-09-18T10:16:13.856 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.856 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.856 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:13.856 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.498133+0000", 2024-09-18T10:16:13.856 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.9044999999999999e-05 2024-09-18T10:16:13.856 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.857 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.857 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:13.857 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.498237+0000", 2024-09-18T10:16:13.857 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00010385100000000001 2024-09-18T10:16:13.857 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:13.857 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:13.857 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:13.857 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.857 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.857 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T10:16:13.857 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:36.787042+0000", 2024-09-18T10:16:13.857 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 97.059640537999996, 2024-09-18T10:16:13.857 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.71152325100000002, 2024-09-18T10:16:13.857 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:13.858 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:13.858 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:13.858 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:16:13.858 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:16:13.858 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 73 2024-09-18T10:16:13.858 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.858 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:13.858 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.858 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:13.858 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787042+0000", 2024-09-18T10:16:13.858 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:13.858 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.858 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.858 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:13.859 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787042+0000", 2024-09-18T10:16:13.859 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:13.859 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.859 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.859 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:13.859 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787042+0000", 2024-09-18T10:16:13.859 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:16:13.859 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.859 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.859 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:13.859 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787048+0000", 2024-09-18T10:16:13.859 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.8900000000000001e-06 2024-09-18T10:16:13.859 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.860 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.860 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:13.860 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787057+0000", 2024-09-18T10:16:13.860 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.4430000000000003e-06 2024-09-18T10:16:13.860 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.860 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.860 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:13.860 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787063+0000", 2024-09-18T10:16:13.860 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.6459999999999998e-06 2024-09-18T10:16:13.860 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.860 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.860 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:13.860 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.297548+0000", 2024-09-18T10:16:13.861 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.51048589499999997 2024-09-18T10:16:13.861 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.861 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.861 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:13.861 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.297668+0000", 2024-09-18T10:16:13.861 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00011917 2024-09-18T10:16:13.861 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.861 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.861 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:16:13.861 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.297891+0000", 2024-09-18T10:16:13.861 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00022301999999999999 2024-09-18T10:16:13.861 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.861 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.861 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:16:13.862 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.301383+0000", 2024-09-18T10:16:13.862 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.003492762 2024-09-18T10:16:13.862 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.862 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.862 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:16:13.862 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.498374+0000", 2024-09-18T10:16:13.862 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.19699053899999999 2024-09-18T10:16:13.862 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.862 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.862 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:13.862 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.498506+0000", 2024-09-18T10:16:13.862 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00013246899999999999 2024-09-18T10:16:13.862 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.862 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.863 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:13.863 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.498566+0000", 2024-09-18T10:16:13.863 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.9274000000000002e-05 2024-09-18T10:16:13.863 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:13.863 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:13.863 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:13.863 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.863 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.863 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T10:16:13.863 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:36.787128+0000", 2024-09-18T10:16:13.863 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 97.059555230000001, 2024-09-18T10:16:13.863 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.71169541800000002, 2024-09-18T10:16:13.863 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:13.864 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:13.864 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:13.864 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:16:13.864 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:16:13.864 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 74 2024-09-18T10:16:13.864 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.864 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:13.864 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.864 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:13.864 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787128+0000", 2024-09-18T10:16:13.864 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:13.864 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.864 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.864 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:13.865 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787128+0000", 2024-09-18T10:16:13.865 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:13.865 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.865 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.865 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:13.865 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787127+0000", 2024-09-18T10:16:13.865 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:16:13.865 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.865 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.865 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:13.865 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787133+0000", 2024-09-18T10:16:13.865 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.6930000000000004e-06 2024-09-18T10:16:13.865 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.866 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.866 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:13.866 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787142+0000", 2024-09-18T10:16:13.866 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.1959999999999997e-06 2024-09-18T10:16:13.866 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.866 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.866 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:13.866 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787150+0000", 2024-09-18T10:16:13.866 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.6399999999999997e-06 2024-09-18T10:16:13.866 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.866 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.866 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:13.866 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.298925+0000", 2024-09-18T10:16:13.866 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.51177523599999997 2024-09-18T10:16:13.867 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.867 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.867 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:13.867 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.299064+0000", 2024-09-18T10:16:13.867 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000139622 2024-09-18T10:16:13.867 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.867 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.867 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:16:13.867 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.299287+0000", 2024-09-18T10:16:13.867 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00022309700000000001 2024-09-18T10:16:13.867 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.867 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.867 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:16:13.868 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.301389+0000", 2024-09-18T10:16:13.868 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0021018759999999999 2024-09-18T10:16:13.868 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.868 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.868 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:16:13.868 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.498648+0000", 2024-09-18T10:16:13.868 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.19725873999999999 2024-09-18T10:16:13.868 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.868 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.868 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:13.868 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.498731+0000", 2024-09-18T10:16:13.868 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.2429000000000006e-05 2024-09-18T10:16:13.868 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.869 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.869 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:13.869 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.498823+0000", 2024-09-18T10:16:13.869 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.2655e-05 2024-09-18T10:16:13.869 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:13.869 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:13.869 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:13.869 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.869 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.869 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T10:16:13.869 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:36.787157+0000", 2024-09-18T10:16:13.869 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 97.059526004999995, 2024-09-18T10:16:13.869 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.81204889999999996, 2024-09-18T10:16:13.870 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:13.870 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:13.870 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:13.870 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:16:13.870 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:16:13.870 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 75 2024-09-18T10:16:13.870 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.870 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:13.870 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.870 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:13.870 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787157+0000", 2024-09-18T10:16:13.870 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:13.870 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.870 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.871 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:13.871 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787157+0000", 2024-09-18T10:16:13.871 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:13.871 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.871 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.871 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:13.871 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787156+0000", 2024-09-18T10:16:13.871 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:16:13.871 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.871 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.871 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:13.871 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787236+0000", 2024-09-18T10:16:13.871 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.0175999999999999e-05 2024-09-18T10:16:13.871 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.872 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.872 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:13.872 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787247+0000", 2024-09-18T10:16:13.872 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0448999999999999e-05 2024-09-18T10:16:13.872 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.872 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.872 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:13.872 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787253+0000", 2024-09-18T10:16:13.872 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.2469999999999997e-06 2024-09-18T10:16:13.872 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.872 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.872 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:13.872 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.299974+0000", 2024-09-18T10:16:13.872 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.51272114499999999 2024-09-18T10:16:13.873 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.873 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.873 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:13.873 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.300115+0000", 2024-09-18T10:16:13.873 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000141452 2024-09-18T10:16:13.873 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.873 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.873 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:16:13.873 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.300349+0000", 2024-09-18T10:16:13.873 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00023363999999999999 2024-09-18T10:16:13.873 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.873 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.874 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:16:13.874 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.301947+0000", 2024-09-18T10:16:13.874 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0015976510000000001 2024-09-18T10:16:13.874 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.874 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.874 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:16:13.874 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.599027+0000", 2024-09-18T10:16:13.874 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.29708051600000002 2024-09-18T10:16:13.874 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.874 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.874 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:13.874 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.599106+0000", 2024-09-18T10:16:13.874 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.9033e-05 2024-09-18T10:16:13.874 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.875 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.875 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:13.875 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.599206+0000", 2024-09-18T10:16:13.875 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.9561000000000004e-05 2024-09-18T10:16:13.875 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:13.875 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:13.875 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:13.875 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.875 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.875 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T10:16:13.875 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:36.787261+0000", 2024-09-18T10:16:13.875 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 97.059422354000006, 2024-09-18T10:16:13.875 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.81223649499999995, 2024-09-18T10:16:13.876 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:13.876 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:13.876 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:13.876 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:16:13.876 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:16:13.876 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 76 2024-09-18T10:16:13.876 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.876 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:13.876 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.876 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:13.876 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787261+0000", 2024-09-18T10:16:13.876 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:13.876 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.877 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.877 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:13.877 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787261+0000", 2024-09-18T10:16:13.877 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:13.877 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.877 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.877 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:13.877 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787260+0000", 2024-09-18T10:16:13.877 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:16:13.877 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.877 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.877 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:13.878 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787265+0000", 2024-09-18T10:16:13.878 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1590000000000001e-06 2024-09-18T10:16:13.878 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.878 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.878 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:13.878 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787273+0000", 2024-09-18T10:16:13.878 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.4100000000000008e-06 2024-09-18T10:16:13.878 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.878 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.878 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:13.878 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:36.787279+0000", 2024-09-18T10:16:13.878 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.3630000000000002e-06 2024-09-18T10:16:13.878 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.878 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.878 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:13.879 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.496955+0000", 2024-09-18T10:16:13.879 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.70967585099999997 2024-09-18T10:16:13.879 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.879 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.879 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:13.879 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.497107+0000", 2024-09-18T10:16:13.879 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000152044 2024-09-18T10:16:13.879 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.879 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.879 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:16:13.879 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.497345+0000", 2024-09-18T10:16:13.879 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000238693 2024-09-18T10:16:13.880 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.880 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.880 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:16:13.880 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.599248+0000", 2024-09-18T10:16:13.880 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.101902836 2024-09-18T10:16:13.880 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.880 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.880 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:16:13.880 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.599329+0000", 2024-09-18T10:16:13.880 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.1064999999999997e-05 2024-09-18T10:16:13.880 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.880 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.880 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:13.880 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.599441+0000", 2024-09-18T10:16:13.881 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000111797 2024-09-18T10:16:13.881 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.881 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.881 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:13.881 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:37.599497+0000", 2024-09-18T10:16:13.881 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.6119999999999998e-05 2024-09-18T10:16:13.881 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:13.881 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:13.881 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:13.881 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.881 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.881 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:93 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T10:16:13.881 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:38.616472+0000", 2024-09-18T10:16:13.882 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 95.230211070999999, 2024-09-18T10:16:13.882 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.60940747100000003, 2024-09-18T10:16:13.882 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:13.882 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:13.882 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:13.882 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:16:13.882 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:16:13.882 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 93 2024-09-18T10:16:13.882 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.882 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:13.882 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.882 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:13.882 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.616472+0000", 2024-09-18T10:16:13.883 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:13.883 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.883 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.883 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:13.883 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.616472+0000", 2024-09-18T10:16:13.883 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:13.883 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.883 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.883 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:13.883 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.616470+0000", 2024-09-18T10:16:13.883 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:16:13.883 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.883 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.884 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:13.884 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.616644+0000", 2024-09-18T10:16:13.884 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00017372000000000001 2024-09-18T10:16:13.884 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.884 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.884 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:13.884 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.616695+0000", 2024-09-18T10:16:13.884 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.0991e-05 2024-09-18T10:16:13.884 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.884 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.884 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:13.884 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.616709+0000", 2024-09-18T10:16:13.884 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4008999999999999e-05 2024-09-18T10:16:13.885 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.885 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.885 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:13.885 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.720960+0000", 2024-09-18T10:16:13.885 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.104250624 2024-09-18T10:16:13.885 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.885 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.885 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:13.885 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.721068+0000", 2024-09-18T10:16:13.885 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00010814700000000001 2024-09-18T10:16:13.885 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.885 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.885 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:16:13.885 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.721270+0000", 2024-09-18T10:16:13.886 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00020184200000000001 2024-09-18T10:16:13.886 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.886 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.886 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:16:13.886 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.024691+0000", 2024-09-18T10:16:13.886 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.30342105600000002 2024-09-18T10:16:13.886 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.886 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.886 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:16:13.886 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.225725+0000", 2024-09-18T10:16:13.886 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20103395099999999 2024-09-18T10:16:13.886 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.886 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.887 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:13.887 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.225816+0000", 2024-09-18T10:16:13.887 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.1928999999999993e-05 2024-09-18T10:16:13.887 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.887 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.887 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:13.887 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.225879+0000", 2024-09-18T10:16:13.887 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.2918999999999998e-05 2024-09-18T10:16:13.887 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:13.887 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:13.887 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:13.887 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.887 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.888 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:94 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T10:16:13.888 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:38.616729+0000", 2024-09-18T10:16:13.888 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 95.229954046000003, 2024-09-18T10:16:13.888 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.60943170999999996, 2024-09-18T10:16:13.888 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:13.888 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:13.888 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:13.888 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:16:13.888 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:16:13.888 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 94 2024-09-18T10:16:13.888 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.888 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:13.889 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.889 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:13.889 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.616729+0000", 2024-09-18T10:16:13.889 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:13.889 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.889 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.889 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:13.889 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.616729+0000", 2024-09-18T10:16:13.889 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:13.889 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.889 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.889 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:13.890 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.616727+0000", 2024-09-18T10:16:13.890 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:16:13.890 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.890 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.890 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:13.890 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.616738+0000", 2024-09-18T10:16:13.890 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0776e-05 2024-09-18T10:16:13.890 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.890 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.890 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:13.890 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.616756+0000", 2024-09-18T10:16:13.890 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8079000000000001e-05 2024-09-18T10:16:13.890 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.890 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.891 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:13.891 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.716882+0000", 2024-09-18T10:16:13.891 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10012612999999999 2024-09-18T10:16:13.891 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.891 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.891 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:13.891 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.822315+0000", 2024-09-18T10:16:13.891 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.105433162 2024-09-18T10:16:13.891 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.891 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.891 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:13.891 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.822453+0000", 2024-09-18T10:16:13.891 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000137396 2024-09-18T10:16:13.892 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.892 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.892 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:16:13.892 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.822674+0000", 2024-09-18T10:16:13.892 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00022154400000000001 2024-09-18T10:16:13.892 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.892 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.892 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:16:13.892 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.024888+0000", 2024-09-18T10:16:13.892 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20221397199999999 2024-09-18T10:16:13.892 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.892 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.892 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:16:13.893 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.226011+0000", 2024-09-18T10:16:13.893 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20112316299999999 2024-09-18T10:16:13.893 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.893 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.893 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:13.893 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.226094+0000", 2024-09-18T10:16:13.893 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.2155000000000002e-05 2024-09-18T10:16:13.893 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.893 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.893 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:13.893 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.226161+0000", 2024-09-18T10:16:13.893 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.7138999999999995e-05 2024-09-18T10:16:13.893 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:13.894 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:13.894 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:13.894 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.894 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.894 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:95 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T10:16:13.894 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:38.716909+0000", 2024-09-18T10:16:13.894 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 95.129774331999997, 2024-09-18T10:16:13.894 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.60960763600000001, 2024-09-18T10:16:13.894 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:13.894 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:13.894 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:13.894 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:16:13.894 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:16:13.895 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 95 2024-09-18T10:16:13.895 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.895 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:13.895 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.895 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:13.895 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.716909+0000", 2024-09-18T10:16:13.895 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:13.895 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.895 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.895 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:13.895 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.716909+0000", 2024-09-18T10:16:13.895 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:13.895 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.896 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.896 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:13.896 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.716907+0000", 2024-09-18T10:16:13.896 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:16:13.896 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.896 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.896 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:13.896 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.716921+0000", 2024-09-18T10:16:13.896 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4572e-05 2024-09-18T10:16:13.896 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.896 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.896 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:13.896 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.716939+0000", 2024-09-18T10:16:13.896 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7149000000000001e-05 2024-09-18T10:16:13.897 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.897 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.897 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:13.897 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.716948+0000", 2024-09-18T10:16:13.897 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.7000000000000003e-06 2024-09-18T10:16:13.897 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.897 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.897 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:13.897 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.923430+0000", 2024-09-18T10:16:13.897 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20648133299999999 2024-09-18T10:16:13.897 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.897 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.897 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:13.897 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.923566+0000", 2024-09-18T10:16:13.898 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00013657599999999999 2024-09-18T10:16:13.898 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.898 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.898 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:16:13.898 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.923782+0000", 2024-09-18T10:16:13.898 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00021599400000000001 2024-09-18T10:16:13.898 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.898 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.898 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:16:13.898 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.024899+0000", 2024-09-18T10:16:13.898 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10111666800000001 2024-09-18T10:16:13.898 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.898 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.899 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:16:13.899 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.326379+0000", 2024-09-18T10:16:13.899 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.30148048799999999 2024-09-18T10:16:13.899 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.899 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.899 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:13.899 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.326454+0000", 2024-09-18T10:16:13.899 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.4988999999999996e-05 2024-09-18T10:16:13.899 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.899 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.899 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:13.899 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.326516+0000", 2024-09-18T10:16:13.899 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.1902999999999993e-05 2024-09-18T10:16:13.900 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:13.900 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:13.900 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:13.900 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.900 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.900 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:96 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T10:16:13.900 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:38.716972+0000", 2024-09-18T10:16:13.900 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 95.129711341999993, 2024-09-18T10:16:13.900 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.91539460800000005, 2024-09-18T10:16:13.900 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:13.900 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:13.900 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:13.900 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:16:13.901 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:16:13.901 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 96 2024-09-18T10:16:13.901 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.901 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:13.901 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.901 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:13.901 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.716972+0000", 2024-09-18T10:16:13.901 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:13.901 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.901 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.901 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:13.901 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.716972+0000", 2024-09-18T10:16:13.901 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:13.901 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.902 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.902 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:13.902 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.716971+0000", 2024-09-18T10:16:13.902 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:16:13.902 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.902 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.902 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:13.902 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717061+0000", 2024-09-18T10:16:13.902 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.0744999999999999e-05 2024-09-18T10:16:13.902 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.902 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.902 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:13.902 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717073+0000", 2024-09-18T10:16:13.903 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1943000000000001e-05 2024-09-18T10:16:13.903 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.903 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.903 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:13.903 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717080+0000", 2024-09-18T10:16:13.903 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0060000000000003e-06 2024-09-18T10:16:13.903 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.903 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.903 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:13.903 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.426708+0000", 2024-09-18T10:16:13.903 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.70962730399999996 2024-09-18T10:16:13.903 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.903 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.903 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:13.904 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.426854+0000", 2024-09-18T10:16:13.904 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00014654199999999999 2024-09-18T10:16:13.904 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.904 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.904 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:16:13.904 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.427106+0000", 2024-09-18T10:16:13.904 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00025226600000000001 2024-09-18T10:16:13.904 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.904 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.904 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:16:13.904 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.433087+0000", 2024-09-18T10:16:13.904 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0059804860000000001 2024-09-18T10:16:13.904 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.905 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.905 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:16:13.905 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.632096+0000", 2024-09-18T10:16:13.905 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.19900922099999999 2024-09-18T10:16:13.905 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.905 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.905 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:13.905 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.632257+0000", 2024-09-18T10:16:13.905 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00016058399999999999 2024-09-18T10:16:13.905 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.905 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.905 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:13.905 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.632366+0000", 2024-09-18T10:16:13.905 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000109601 2024-09-18T10:16:13.906 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:13.906 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:13.906 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:13.906 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.906 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.906 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:97 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T10:16:13.906 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:38.717093+0000", 2024-09-18T10:16:13.906 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 95.129589531999997, 2024-09-18T10:16:13.906 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.91561460900000002, 2024-09-18T10:16:13.906 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:13.906 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:13.906 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:13.906 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:16:13.907 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:16:13.907 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 97 2024-09-18T10:16:13.907 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.907 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:13.907 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.907 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:13.907 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717093+0000", 2024-09-18T10:16:13.907 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:13.907 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.907 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.907 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:13.907 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717093+0000", 2024-09-18T10:16:13.907 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:13.908 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.908 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.908 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:13.908 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717092+0000", 2024-09-18T10:16:13.908 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:16:13.908 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.908 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.908 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:13.908 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717099+0000", 2024-09-18T10:16:13.908 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.246e-06 2024-09-18T10:16:13.908 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.908 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.908 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:13.909 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717109+0000", 2024-09-18T10:16:13.909 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0499999999999999e-05 2024-09-18T10:16:13.909 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.909 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.909 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:13.909 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717115+0000", 2024-09-18T10:16:13.909 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.7690000000000002e-06 2024-09-18T10:16:13.909 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.909 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.909 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:13.909 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.428509+0000", 2024-09-18T10:16:13.909 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.71139444299999999 2024-09-18T10:16:13.909 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.909 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.910 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:13.910 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.428621+0000", 2024-09-18T10:16:13.910 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00011156 2024-09-18T10:16:13.910 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.910 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.910 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:16:13.910 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.428805+0000", 2024-09-18T10:16:13.910 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000183597 2024-09-18T10:16:13.910 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.910 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.910 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:16:13.910 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.433106+0000", 2024-09-18T10:16:13.910 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0043015019999999996 2024-09-18T10:16:13.910 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.911 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.911 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:16:13.911 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.632474+0000", 2024-09-18T10:16:13.911 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.199368092 2024-09-18T10:16:13.911 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.911 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.911 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:13.911 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.632605+0000", 2024-09-18T10:16:13.911 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00013110899999999999 2024-09-18T10:16:13.911 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.911 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.911 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:13.911 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.632708+0000", 2024-09-18T10:16:13.912 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000102748 2024-09-18T10:16:13.912 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:13.912 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:13.912 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:13.912 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.912 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.912 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:98 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T10:16:13.912 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:38.717126+0000", 2024-09-18T10:16:13.912 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 95.129557270000006, 2024-09-18T10:16:13.912 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.015084141, 2024-09-18T10:16:13.912 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:13.912 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:13.912 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:13.913 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:16:13.913 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:16:13.913 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 98 2024-09-18T10:16:13.913 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.913 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:13.913 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.913 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:13.913 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717126+0000", 2024-09-18T10:16:13.913 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:13.913 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.913 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.913 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:13.914 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717126+0000", 2024-09-18T10:16:13.914 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:13.914 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.914 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.914 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:13.914 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717125+0000", 2024-09-18T10:16:13.914 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:16:13.914 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.914 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.914 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:13.914 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717130+0000", 2024-09-18T10:16:13.914 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.4160000000000003e-06 2024-09-18T10:16:13.915 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.915 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.915 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:13.915 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717139+0000", 2024-09-18T10:16:13.915 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.6230000000000003e-06 2024-09-18T10:16:13.915 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.915 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.915 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:13.915 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717144+0000", 2024-09-18T10:16:13.915 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5330000000000002e-06 2024-09-18T10:16:13.915 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.915 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.915 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:13.916 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.429491+0000", 2024-09-18T10:16:13.916 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.71234629500000002 2024-09-18T10:16:13.916 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.916 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.916 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:13.916 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.429741+0000", 2024-09-18T10:16:13.916 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000250516 2024-09-18T10:16:13.916 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.916 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.916 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:16:13.916 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.429989+0000", 2024-09-18T10:16:13.916 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00024811599999999999 2024-09-18T10:16:13.916 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.916 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.917 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:16:13.917 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.435056+0000", 2024-09-18T10:16:13.917 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0050669670000000003 2024-09-18T10:16:13.917 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.917 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.917 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:16:13.917 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.732068+0000", 2024-09-18T10:16:13.917 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.29701180900000002 2024-09-18T10:16:13.917 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.917 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.917 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:13.917 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.732145+0000", 2024-09-18T10:16:13.918 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.7152999999999998e-05 2024-09-18T10:16:13.918 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.918 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.918 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:13.918 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.732210+0000", 2024-09-18T10:16:13.918 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.4503000000000002e-05 2024-09-18T10:16:13.918 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:13.918 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:13.918 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:13.918 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.918 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.918 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:99 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T10:16:13.918 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:38.717230+0000", 2024-09-18T10:16:13.919 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 95.129452709999995, 2024-09-18T10:16:13.919 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0157905819999999, 2024-09-18T10:16:13.919 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:13.919 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:13.919 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:13.919 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:16:13.919 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:16:13.919 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 99 2024-09-18T10:16:13.919 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.919 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:13.919 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.919 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:13.919 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717230+0000", 2024-09-18T10:16:13.919 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:13.920 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.920 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.920 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:13.920 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717230+0000", 2024-09-18T10:16:13.920 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:13.920 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.920 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.920 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:13.920 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717229+0000", 2024-09-18T10:16:13.920 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:16:13.920 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.920 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.920 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:13.921 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717318+0000", 2024-09-18T10:16:13.921 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.8197999999999998e-05 2024-09-18T10:16:13.921 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.921 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.921 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:13.921 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717342+0000", 2024-09-18T10:16:13.921 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4445000000000001e-05 2024-09-18T10:16:13.921 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.921 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.921 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:13.921 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717360+0000", 2024-09-18T10:16:13.921 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7782999999999999e-05 2024-09-18T10:16:13.921 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.922 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.922 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:13.922 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.430650+0000", 2024-09-18T10:16:13.922 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.71329020200000004 2024-09-18T10:16:13.922 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.922 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.922 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:13.922 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.430805+0000", 2024-09-18T10:16:13.922 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00015474900000000001 2024-09-18T10:16:13.922 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.922 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.922 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:16:13.922 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.431021+0000", 2024-09-18T10:16:13.923 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000215911 2024-09-18T10:16:13.923 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.923 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.923 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:16:13.923 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.435099+0000", 2024-09-18T10:16:13.923 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0040787439999999996 2024-09-18T10:16:13.923 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.923 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.923 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:16:13.923 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.732758+0000", 2024-09-18T10:16:13.923 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.297658637 2024-09-18T10:16:13.923 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.923 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.924 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:13.924 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.732909+0000", 2024-09-18T10:16:13.924 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00015069899999999999 2024-09-18T10:16:13.924 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.924 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.924 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:13.924 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.733021+0000", 2024-09-18T10:16:13.924 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00011207 2024-09-18T10:16:13.924 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:13.924 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:13.924 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:13.924 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.924 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.925 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4257.0:100 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T10:16:13.925 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:38.717370+0000", 2024-09-18T10:16:13.925 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 95.129313147999994, 2024-09-18T10:16:13.925 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1158324850000001, 2024-09-18T10:16:13.925 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:13.925 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:13.925 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:13.925 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4257", 2024-09-18T10:16:13.925 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/3092077334", 2024-09-18T10:16:13.925 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 100 2024-09-18T10:16:13.925 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.925 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:13.926 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.926 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:13.926 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717370+0000", 2024-09-18T10:16:13.926 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:13.926 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.926 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.926 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:13.926 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717370+0000", 2024-09-18T10:16:13.926 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:13.926 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.926 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.926 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:13.926 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717369+0000", 2024-09-18T10:16:13.927 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:16:13.927 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.927 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.927 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:13.927 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717378+0000", 2024-09-18T10:16:13.927 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.3559999999999999e-06 2024-09-18T10:16:13.927 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.927 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.927 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:13.927 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717398+0000", 2024-09-18T10:16:13.927 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.9962e-05 2024-09-18T10:16:13.927 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.927 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.927 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:13.928 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:38.717407+0000", 2024-09-18T10:16:13.928 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.2699999999999993e-06 2024-09-18T10:16:13.928 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.928 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.928 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:13.928 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.432184+0000", 2024-09-18T10:16:13.929 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.71477630299999995 2024-09-18T10:16:13.929 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.929 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.929 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:13.929 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.432316+0000", 2024-09-18T10:16:13.929 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000132082 2024-09-18T10:16:13.929 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.929 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.929 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 15", 2024-09-18T10:16:13.930 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.432526+0000", 2024-09-18T10:16:13.930 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000210659 2024-09-18T10:16:13.930 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.930 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.930 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:16:13.930 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.435651+0000", 2024-09-18T10:16:13.930 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.003124169 2024-09-18T10:16:13.930 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.930 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.930 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:16:13.930 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.832931+0000", 2024-09-18T10:16:13.930 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.39728084899999999 2024-09-18T10:16:13.931 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.931 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.931 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:13.931 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.833058+0000", 2024-09-18T10:16:13.931 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000126969 2024-09-18T10:16:13.931 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.931 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.931 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:13.931 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:39.833202+0000", 2024-09-18T10:16:13.931 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000143922 2024-09-18T10:16:13.931 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:13.931 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:13.931 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:13.932 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.932 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.932 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:141 3.8s0 3:1c02e63c:::smithi18925896-47:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:16:13.932 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:51.119430+0000", 2024-09-18T10:16:13.932 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 82.727252944, 2024-09-18T10:16:13.932 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.60306919000000003, 2024-09-18T10:16:13.932 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:13.932 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "started", 2024-09-18T10:16:13.932 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:13.932 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:16:13.932 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:13.932 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 141 2024-09-18T10:16:13.932 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.933 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:13.933 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.933 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:13.933 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.119430+0000", 2024-09-18T10:16:13.933 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:13.933 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.933 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.933 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:13.933 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.119430+0000", 2024-09-18T10:16:13.933 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:13.933 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.933 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.934 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:13.934 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.119430+0000", 2024-09-18T10:16:13.934 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999995 2024-09-18T10:16:13.934 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.934 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.934 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:13.934 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.119432+0000", 2024-09-18T10:16:13.934 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6240000000000002e-06 2024-09-18T10:16:13.934 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.934 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.934 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:13.934 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.119439+0000", 2024-09-18T10:16:13.934 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.8329999999999996e-06 2024-09-18T10:16:13.934 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.935 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.935 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:13.935 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.119444+0000", 2024-09-18T10:16:13.935 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.485e-06 2024-09-18T10:16:13.935 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.935 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.935 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:13.935 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.620626+0000", 2024-09-18T10:16:13.935 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.50118172299999997 2024-09-18T10:16:13.935 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.935 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.935 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:13.936 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.620781+0000", 2024-09-18T10:16:13.936 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00015495 2024-09-18T10:16:13.936 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.936 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.936 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:13.936 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.722499+0000", 2024-09-18T10:16:13.936 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.101718085 2024-09-18T10:16:13.936 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:13.936 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:13.936 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:13.936 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.936 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.936 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:154 3.8s0 3:1d386431:::smithi18925896-2:head [copy-from ver 4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:16:13.937 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:51.211083+0000", 2024-09-18T10:16:13.937 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 82.635599718999998, 2024-09-18T10:16:13.937 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.73193688499999998, 2024-09-18T10:16:13.937 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:13.937 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:13.937 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:13.937 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:16:13.937 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:13.937 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 154 2024-09-18T10:16:13.937 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.937 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:13.938 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.938 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:13.938 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.211083+0000", 2024-09-18T10:16:13.938 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:13.938 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.938 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.938 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:13.938 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.211083+0000", 2024-09-18T10:16:13.938 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:13.938 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.938 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.938 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:13.938 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.211080+0000", 2024-09-18T10:16:13.938 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999971 2024-09-18T10:16:13.939 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.939 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.939 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:13.939 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.211096+0000", 2024-09-18T10:16:13.939 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.5699999999999999e-05 2024-09-18T10:16:13.939 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.939 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.939 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:13.939 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.211134+0000", 2024-09-18T10:16:13.939 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.7694000000000002e-05 2024-09-18T10:16:13.939 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.939 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.940 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:13.940 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.211149+0000", 2024-09-18T10:16:13.940 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4746000000000001e-05 2024-09-18T10:16:13.940 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.940 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.940 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:13.940 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.621341+0000", 2024-09-18T10:16:13.940 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.41019203799999998 2024-09-18T10:16:13.940 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.940 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.940 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:13.940 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.621505+0000", 2024-09-18T10:16:13.941 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00016443299999999999 2024-09-18T10:16:13.941 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.941 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.941 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:13.941 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.840064+0000", 2024-09-18T10:16:13.941 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.21855904700000001 2024-09-18T10:16:13.941 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.941 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.941 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:13.941 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.842430+0000", 2024-09-18T10:16:13.941 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0023664100000000002 2024-09-18T10:16:13.941 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.941 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.942 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:13.942 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.942864+0000", 2024-09-18T10:16:13.942 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10043373 2024-09-18T10:16:13.942 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.942 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.942 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:13.942 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.943020+0000", 2024-09-18T10:16:13.942 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00015598000000000001 2024-09-18T10:16:13.942 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:13.942 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:13.942 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:13.942 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.942 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.943 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:155 3.8s0 3:1d386431:::smithi18925896-2:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:16:13.943 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:51.211164+0000", 2024-09-18T10:16:13.943 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 82.635519349000006, 2024-09-18T10:16:13.943 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.630957349, 2024-09-18T10:16:13.943 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:13.943 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "started", 2024-09-18T10:16:13.943 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:13.943 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:16:13.943 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:13.943 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 155 2024-09-18T10:16:13.943 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.943 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:13.943 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.944 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:13.944 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.211164+0000", 2024-09-18T10:16:13.944 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:13.944 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.944 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.944 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:13.944 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.211164+0000", 2024-09-18T10:16:13.944 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:13.944 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.944 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.944 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:13.944 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.211163+0000", 2024-09-18T10:16:13.944 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:16:13.945 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.945 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.945 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:13.945 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.211167+0000", 2024-09-18T10:16:13.945 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.6299999999999997e-06 2024-09-18T10:16:13.945 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.945 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.945 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:13.945 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.211178+0000", 2024-09-18T10:16:13.945 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0798999999999999e-05 2024-09-18T10:16:13.945 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.945 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.946 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:13.946 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.211186+0000", 2024-09-18T10:16:13.946 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.5669999999999997e-06 2024-09-18T10:16:13.946 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.946 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.946 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:13.946 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.623024+0000", 2024-09-18T10:16:13.946 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.411838433 2024-09-18T10:16:13.946 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.946 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.946 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for blocked object", 2024-09-18T10:16:13.946 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.623104+0000", 2024-09-18T10:16:13.946 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.0136000000000007e-05 2024-09-18T10:16:13.947 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.947 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.947 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:13.947 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.841600+0000", 2024-09-18T10:16:13.947 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.21849595399999999 2024-09-18T10:16:13.947 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.947 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.947 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:13.947 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.841868+0000", 2024-09-18T10:16:13.947 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00026825800000000001 2024-09-18T10:16:13.947 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.947 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.947 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:13.947 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:51.842121+0000", 2024-09-18T10:16:13.948 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00025266500000000001 2024-09-18T10:16:13.948 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:13.948 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:13.948 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:13.948 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.948 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.948 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:619 3.8s0 3:1162852a:::smithi18925896-44:head [copy-from ver 14] snapc 21=[20,1f,1e,1a,16,12,11,10] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-18T10:16:13.948 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:27.977392+0000", 2024-09-18T10:16:13.948 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 45.869291095000001, 2024-09-18T10:16:13.948 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.71139311599999999, 2024-09-18T10:16:13.948 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:13.948 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:13.948 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:13.949 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:16:13.949 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:13.949 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 619 2024-09-18T10:16:13.949 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.949 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:13.949 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.949 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:13.949 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:27.977392+0000", 2024-09-18T10:16:13.949 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:13.949 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.949 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.949 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:13.949 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:27.977392+0000", 2024-09-18T10:16:13.950 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:13.950 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.950 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.950 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:13.950 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:27.977390+0000", 2024-09-18T10:16:13.950 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:16:13.950 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.950 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.950 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:13.950 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:27.977399+0000", 2024-09-18T10:16:13.950 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.1239999999999994e-06 2024-09-18T10:16:13.950 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.950 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.950 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:13.951 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:27.977422+0000", 2024-09-18T10:16:13.951 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.2734999999999998e-05 2024-09-18T10:16:13.951 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.951 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.951 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:13.951 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.077547+0000", 2024-09-18T10:16:13.951 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100124728 2024-09-18T10:16:13.951 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.951 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.951 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:13.951 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.077713+0000", 2024-09-18T10:16:13.951 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000166515 2024-09-18T10:16:13.951 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.951 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.952 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op must wait for map", 2024-09-18T10:16:13.952 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.077728+0000", 2024-09-18T10:16:13.952 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.5036e-05 2024-09-18T10:16:13.952 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.952 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.952 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:13.952 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.275819+0000", 2024-09-18T10:16:13.952 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.19809117600000001 2024-09-18T10:16:13.952 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.952 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.952 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:13.952 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.276220+0000", 2024-09-18T10:16:13.952 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000400426 2024-09-18T10:16:13.953 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.953 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.953 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:13.953 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.383764+0000", 2024-09-18T10:16:13.953 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.107543814 2024-09-18T10:16:13.953 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.953 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.953 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:13.953 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.486402+0000", 2024-09-18T10:16:13.953 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.102638484 2024-09-18T10:16:13.953 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.953 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.953 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:13.953 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.688521+0000", 2024-09-18T10:16:13.954 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.202118873 2024-09-18T10:16:13.954 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.954 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.954 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:13.954 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.688785+0000", 2024-09-18T10:16:13.954 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000264133 2024-09-18T10:16:13.954 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:13.954 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:13.954 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:13.954 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.954 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.954 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:626 3.10s0 3:0a83d6b3:::smithi18925896-20:head [append 0~638976] snapc 21=[20,1f,1e,1a,16,12,11,10] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-18T10:16:13.954 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:28.708261+0000", 2024-09-18T10:16:13.955 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 45.138421530000002, 2024-09-18T10:16:13.955 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.70096385699999997, 2024-09-18T10:16:13.955 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:13.955 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:13.955 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:13.955 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:16:13.955 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:13.955 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 626 2024-09-18T10:16:13.955 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.955 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:13.955 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.955 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:13.955 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.708261+0000", 2024-09-18T10:16:13.955 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:13.956 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.956 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.956 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:13.956 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.708261+0000", 2024-09-18T10:16:13.956 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:13.956 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.956 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.956 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:13.956 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.708259+0000", 2024-09-18T10:16:13.956 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:16:13.956 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.956 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.956 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:13.957 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.709175+0000", 2024-09-18T10:16:13.957 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00091571199999999995 2024-09-18T10:16:13.957 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.957 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.957 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:13.957 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.709199+0000", 2024-09-18T10:16:13.957 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.3785e-05 2024-09-18T10:16:13.957 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.957 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.957 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:13.957 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.809328+0000", 2024-09-18T10:16:13.957 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100129494 2024-09-18T10:16:13.957 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.958 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.958 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:13.958 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.996826+0000", 2024-09-18T10:16:13.958 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.18749712700000001 2024-09-18T10:16:13.958 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.958 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.958 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:13.958 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.997086+0000", 2024-09-18T10:16:13.958 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00026047499999999998 2024-09-18T10:16:13.958 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.958 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.958 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:13.958 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.998987+0000", 2024-09-18T10:16:13.959 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019014889999999999 2024-09-18T10:16:13.959 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.959 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.959 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:13.959 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.104118+0000", 2024-09-18T10:16:13.959 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10513012100000001 2024-09-18T10:16:13.959 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.959 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.959 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:13.959 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.409085+0000", 2024-09-18T10:16:13.959 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.30496710599999999 2024-09-18T10:16:13.959 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.959 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.960 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:13.960 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.409225+0000", 2024-09-18T10:16:13.960 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00014059899999999999 2024-09-18T10:16:13.960 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:13.960 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:13.960 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:13.960 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.960 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.960 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:629 3.10s0 3:0a83d6b3:::smithi18925896-20:head [setxattr _header (57) in=7b] snapc 21=[20,1f,1e,1a,16,12,11,10] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-18T10:16:13.960 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:28.910973+0000", 2024-09-18T10:16:13.960 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 44.935710411999999, 2024-09-18T10:16:13.960 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.80485965999999998, 2024-09-18T10:16:13.960 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:13.961 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:13.961 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:13.961 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:16:13.961 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:13.961 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 629 2024-09-18T10:16:13.961 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.961 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:13.961 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.961 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:13.961 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.910973+0000", 2024-09-18T10:16:13.961 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:13.961 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.961 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.962 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:13.962 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.910973+0000", 2024-09-18T10:16:13.962 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:13.962 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.962 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.962 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:13.962 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.910971+0000", 2024-09-18T10:16:13.962 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:16:13.962 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.962 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.962 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:13.962 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.910976+0000", 2024-09-18T10:16:13.962 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.0069999999999997e-06 2024-09-18T10:16:13.962 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.963 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.963 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:13.963 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.910984+0000", 2024-09-18T10:16:13.963 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.7109999999999995e-06 2024-09-18T10:16:13.963 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.963 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.963 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:13.963 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.910989+0000", 2024-09-18T10:16:13.963 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1549999999999998e-06 2024-09-18T10:16:13.963 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.963 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.963 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:13.963 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.411082+0000", 2024-09-18T10:16:13.963 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.50009252199999998 2024-09-18T10:16:13.964 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.964 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.964 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:13.964 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.411264+0000", 2024-09-18T10:16:13.964 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00018190799999999999 2024-09-18T10:16:13.964 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.964 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.964 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:13.964 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.412262+0000", 2024-09-18T10:16:13.964 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00099852199999999995 2024-09-18T10:16:13.964 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.964 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.964 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:13.964 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.513736+0000", 2024-09-18T10:16:13.965 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.101474077 2024-09-18T10:16:13.965 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.965 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.965 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:13.965 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.715730+0000", 2024-09-18T10:16:13.965 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20199325600000001 2024-09-18T10:16:13.965 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.965 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.965 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:13.965 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.715832+0000", 2024-09-18T10:16:13.965 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000102633 2024-09-18T10:16:13.965 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:13.965 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:13.966 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:13.966 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.966 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.966 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:630 3.10s0 3:0a83d6b3:::smithi18925896-20:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e72)", 2024-09-18T10:16:13.966 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:28.910995+0000", 2024-09-18T10:16:13.966 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 44.935688380999999, 2024-09-18T10:16:13.966 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2560086020000001, 2024-09-18T10:16:13.966 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:13.966 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "started", 2024-09-18T10:16:13.966 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:13.966 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:16:13.966 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:13.966 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 630 2024-09-18T10:16:13.966 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.967 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:13.967 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.967 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:13.967 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.910995+0000", 2024-09-18T10:16:13.967 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:13.967 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.967 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.967 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:13.967 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.910995+0000", 2024-09-18T10:16:13.967 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:13.967 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.967 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.967 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:13.967 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.910994+0000", 2024-09-18T10:16:13.968 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:16:13.968 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.968 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.968 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:13.968 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.910997+0000", 2024-09-18T10:16:13.968 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2820000000000001e-06 2024-09-18T10:16:13.968 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.968 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.968 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:13.968 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.911008+0000", 2024-09-18T10:16:13.968 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0617e-05 2024-09-18T10:16:13.968 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.969 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.969 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:13.969 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:28.911013+0000", 2024-09-18T10:16:13.969 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.8160000000000001e-06 2024-09-18T10:16:13.969 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.969 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.969 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:13.969 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.413034+0000", 2024-09-18T10:16:13.969 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.50202176600000004 2024-09-18T10:16:13.969 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.969 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.969 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for rw locks", 2024-09-18T10:16:13.969 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.413303+0000", 2024-09-18T10:16:13.970 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00026880600000000001 2024-09-18T10:16:13.970 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.970 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.970 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:13.970 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.716533+0000", 2024-09-18T10:16:13.970 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.30323016000000003 2024-09-18T10:16:13.970 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.970 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.970 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:13.970 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.716647+0000", 2024-09-18T10:16:13.970 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00011372899999999999 2024-09-18T10:16:13.970 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:13.970 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:13.970 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:13.971 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:30.167003+0000", 2024-09-18T10:16:13.971 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.450356172 2024-09-18T10:16:13.971 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:13.971 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:13.971 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:13.971 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:13.971 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:13.971 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-09-18T10:16:13.973 INFO:teuthology.orchestra.run.smithi029.stderr:noscrub is unset 2024-09-18T10:16:13.973 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-09-18T10:16:14.081 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-09-18T10:16:14.081 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-09-18T10:16:14.081 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-09-18T10:16:14.081 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-09-18T10:16:14.091 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-09-18T10:16:14.195 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-09-18T10:16:14.195 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-09-18T10:16:14.195 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-09-18T10:16:14.195 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-09-18T10:16:14.195 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-09-18T10:16:14.203 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-09-18T10:16:14.303 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-09-18T10:16:14.303 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-09-18T10:16:14.303 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-09-18T10:16:14.304 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-09-18T10:16:14.304 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.304 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:189 3.3s0 3:ce8164ce:::smithi18925896-7:head [copy-from ver 6] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:16:14.304 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:52.694279+0000", 2024-09-18T10:16:14.304 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 81.603360592000001, 2024-09-18T10:16:14.305 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.51539283599999997, 2024-09-18T10:16:14.305 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:14.305 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:14.305 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:14.305 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:16:14.305 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:14.306 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 189 2024-09-18T10:16:14.306 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.306 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:14.306 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.306 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:14.307 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694279+0000", 2024-09-18T10:16:14.307 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:14.307 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.307 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.308 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:14.308 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694279+0000", 2024-09-18T10:16:14.308 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:14.308 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.308 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.308 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:14.309 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694277+0000", 2024-09-18T10:16:14.309 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999971 2024-09-18T10:16:14.309 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.309 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.309 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:14.310 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694290+0000", 2024-09-18T10:16:14.310 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3862e-05 2024-09-18T10:16:14.310 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.310 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.310 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:14.310 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694311+0000", 2024-09-18T10:16:14.311 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.0273999999999999e-05 2024-09-18T10:16:14.311 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.311 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.311 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:14.311 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694322+0000", 2024-09-18T10:16:14.312 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1722999999999999e-05 2024-09-18T10:16:14.312 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.312 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.312 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:14.312 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.794494+0000", 2024-09-18T10:16:14.312 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100171653 2024-09-18T10:16:14.313 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.313 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.313 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:14.313 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.794705+0000", 2024-09-18T10:16:14.313 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00021126000000000001 2024-09-18T10:16:14.314 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.314 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.314 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:14.314 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.096113+0000", 2024-09-18T10:16:14.314 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.30140739100000002 2024-09-18T10:16:14.314 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.315 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.315 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:14.315 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.098602+0000", 2024-09-18T10:16:14.315 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.002489593 2024-09-18T10:16:14.315 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.315 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.316 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:14.316 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.209538+0000", 2024-09-18T10:16:14.316 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.110935482 2024-09-18T10:16:14.316 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.316 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.317 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:14.317 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.209672+0000", 2024-09-18T10:16:14.317 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00013435399999999999 2024-09-18T10:16:14.317 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:14.317 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:14.317 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:14.318 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.318 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.318 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:194 3.3s0 3:c9ce6440:::smithi18925896-24:head [copy-from ver 6] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:16:14.318 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:52.694388+0000", 2024-09-18T10:16:14.318 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 81.603251728999993, 2024-09-18T10:16:14.318 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.92118689200000003, 2024-09-18T10:16:14.319 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:14.319 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:14.319 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:14.319 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:16:14.319 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:14.319 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 194 2024-09-18T10:16:14.320 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.320 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:14.320 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.320 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:14.320 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694388+0000", 2024-09-18T10:16:14.321 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:14.321 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.321 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.321 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:14.321 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694388+0000", 2024-09-18T10:16:14.321 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:14.322 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.322 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.322 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:14.322 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694388+0000", 2024-09-18T10:16:14.322 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:16:14.322 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.323 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.323 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:14.323 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694393+0000", 2024-09-18T10:16:14.323 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.2299999999999999e-06 2024-09-18T10:16:14.323 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.323 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.323 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:14.324 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694411+0000", 2024-09-18T10:16:14.324 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7802999999999999e-05 2024-09-18T10:16:14.324 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.324 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.324 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:14.325 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694417+0000", 2024-09-18T10:16:14.325 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.4640000000000003e-06 2024-09-18T10:16:14.325 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.325 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.325 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:14.325 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.795289+0000", 2024-09-18T10:16:14.326 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100872288 2024-09-18T10:16:14.326 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.326 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.326 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:14.326 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.795548+0000", 2024-09-18T10:16:14.326 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00025869199999999998 2024-09-18T10:16:14.327 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.327 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.327 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:14.327 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.412215+0000", 2024-09-18T10:16:14.327 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.61666667100000006 2024-09-18T10:16:14.327 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.327 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.327 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:14.328 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.514838+0000", 2024-09-18T10:16:14.328 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10262323399999999 2024-09-18T10:16:14.328 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.328 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.328 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:14.328 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.615386+0000", 2024-09-18T10:16:14.328 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100548106 2024-09-18T10:16:14.329 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.329 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.329 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:14.329 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.615575+0000", 2024-09-18T10:16:14.329 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000189128 2024-09-18T10:16:14.329 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:14.329 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:14.329 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:14.330 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.330 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.330 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:195 3.3s0 3:c9ce6440:::smithi18925896-24:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:16:14.330 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:52.694424+0000", 2024-09-18T10:16:14.330 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 81.603215993000006, 2024-09-18T10:16:14.330 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.82009685499999996, 2024-09-18T10:16:14.330 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:14.331 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "started", 2024-09-18T10:16:14.331 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:14.331 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:16:14.331 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:14.331 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 195 2024-09-18T10:16:14.331 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.331 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:14.332 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.332 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:14.332 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694424+0000", 2024-09-18T10:16:14.332 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:14.332 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.332 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.333 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:14.333 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694424+0000", 2024-09-18T10:16:14.333 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:14.333 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.333 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.333 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:14.333 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694423+0000", 2024-09-18T10:16:14.333 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999995 2024-09-18T10:16:14.333 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.333 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.334 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:14.334 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694427+0000", 2024-09-18T10:16:14.334 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2650000000000001e-06 2024-09-18T10:16:14.334 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.334 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.334 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:14.334 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694434+0000", 2024-09-18T10:16:14.334 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.5410000000000003e-06 2024-09-18T10:16:14.334 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.335 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.335 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:14.335 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.694441+0000", 2024-09-18T10:16:14.335 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.187e-06 2024-09-18T10:16:14.335 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.335 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.335 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:14.336 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.795870+0000", 2024-09-18T10:16:14.336 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.101428617 2024-09-18T10:16:14.336 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.336 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.336 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for blocked object", 2024-09-18T10:16:14.336 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:52.796040+0000", 2024-09-18T10:16:14.337 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00017008 2024-09-18T10:16:14.337 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.337 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.337 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:14.337 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.514130+0000", 2024-09-18T10:16:14.337 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.71809022199999994 2024-09-18T10:16:14.338 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.338 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.338 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:14.338 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.514348+0000", 2024-09-18T10:16:14.338 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000217488 2024-09-18T10:16:14.338 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.339 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.339 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:14.339 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:53.514521+0000", 2024-09-18T10:16:14.339 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00017300900000000001 2024-09-18T10:16:14.339 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:14.339 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:14.340 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:14.340 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.340 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.340 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.bs2 41/32 ECSubWrite(tid=58, reqid=osd.4.0:58, at_version=41'14, trim_to=0'0, roll_forward_to=39'12))", 2024-09-18T10:16:14.340 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:56.829698+0000", 2024-09-18T10:16:14.341 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 77.467941898999996, 2024-09-18T10:16:14.341 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.71711602600000002, 2024-09-18T10:16:14.341 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:14.341 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:16:14.341 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:14.341 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.341 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:14.341 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.829698+0000", 2024-09-18T10:16:14.342 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:14.342 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.342 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.342 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:14.342 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.829698+0000", 2024-09-18T10:16:14.342 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:14.342 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.342 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.342 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:14.342 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.829697+0000", 2024-09-18T10:16:14.343 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:16:14.343 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.343 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.343 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:14.343 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.829733+0000", 2024-09-18T10:16:14.343 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.6217999999999998e-05 2024-09-18T10:16:14.343 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.344 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.344 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:14.344 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.829757+0000", 2024-09-18T10:16:14.344 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.3915000000000001e-05 2024-09-18T10:16:14.344 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.344 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.345 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:14.345 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.829769+0000", 2024-09-18T10:16:14.345 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1754e-05 2024-09-18T10:16:14.345 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.345 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.345 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:14.346 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.342965+0000", 2024-09-18T10:16:14.346 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.51319593799999996 2024-09-18T10:16:14.346 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.346 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.346 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:14.346 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.343020+0000", 2024-09-18T10:16:14.346 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5367e-05 2024-09-18T10:16:14.346 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.347 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.347 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:14.347 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.546772+0000", 2024-09-18T10:16:14.347 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.203751933 2024-09-18T10:16:14.347 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.347 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.347 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:14.348 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.546814+0000", 2024-09-18T10:16:14.348 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.2196000000000001e-05 2024-09-18T10:16:14.348 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:14.348 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:14.348 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:14.348 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.348 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.349 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDPGPushReply(3.17s0 67/64 [PushReplyOp(3:ee962131:::smithi18925896-29:10)])", 2024-09-18T10:16:14.349 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:23.671965+0000", 2024-09-18T10:16:14.349 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 50.625675448999999, 2024-09-18T10:16:14.349 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.60673808500000004, 2024-09-18T10:16:14.349 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:14.349 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:16:14.349 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:14.349 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.349 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:14.349 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.671965+0000", 2024-09-18T10:16:14.350 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:14.350 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.350 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.350 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:14.350 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.671965+0000", 2024-09-18T10:16:14.350 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:14.350 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.350 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.350 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:14.350 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.671963+0000", 2024-09-18T10:16:14.350 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:16:14.350 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.351 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.351 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:14.351 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.671977+0000", 2024-09-18T10:16:14.351 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4093e-05 2024-09-18T10:16:14.351 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.351 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.351 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:14.351 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.672009+0000", 2024-09-18T10:16:14.351 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1939e-05 2024-09-18T10:16:14.351 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.351 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.351 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:14.351 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.772102+0000", 2024-09-18T10:16:14.351 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10009319 2024-09-18T10:16:14.351 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.352 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.352 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:14.352 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.278591+0000", 2024-09-18T10:16:14.352 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.50648915999999999 2024-09-18T10:16:14.352 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.352 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.352 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:14.352 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.278703+0000", 2024-09-18T10:16:14.352 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000111433 2024-09-18T10:16:14.352 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:14.352 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:14.352 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:14.352 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.353 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.353 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDPGPushReply(3.17s0 67/64 [PushReplyOp(3:ee962131:::smithi18925896-29:10)])", 2024-09-18T10:16:14.353 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:23.772436+0000", 2024-09-18T10:16:14.353 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 50.525204213000002, 2024-09-18T10:16:14.353 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.50655292500000004, 2024-09-18T10:16:14.353 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:14.353 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:16:14.353 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:14.353 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.353 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:14.353 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.772436+0000", 2024-09-18T10:16:14.353 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:14.353 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.353 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.354 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:14.354 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.772436+0000", 2024-09-18T10:16:14.354 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:14.354 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.354 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.354 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:14.354 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.772434+0000", 2024-09-18T10:16:14.354 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:16:14.354 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.354 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.354 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:14.354 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.772448+0000", 2024-09-18T10:16:14.354 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3682e-05 2024-09-18T10:16:14.354 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.355 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.355 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:14.355 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.772477+0000", 2024-09-18T10:16:14.355 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9255999999999999e-05 2024-09-18T10:16:14.355 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.355 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.355 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:14.355 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.772490+0000", 2024-09-18T10:16:14.355 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2962999999999999e-05 2024-09-18T10:16:14.355 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.355 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.355 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:14.355 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.278888+0000", 2024-09-18T10:16:14.355 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.50639794699999996 2024-09-18T10:16:14.356 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.356 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.356 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:14.356 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.278989+0000", 2024-09-18T10:16:14.356 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000100777 2024-09-18T10:16:14.356 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:14.356 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:14.356 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:14.356 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.356 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.356 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDPGPushReply(3.17s0 67/64 [PushReplyOp(3:ee962131:::smithi18925896-29:10)])", 2024-09-18T10:16:14.356 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:23.772911+0000", 2024-09-18T10:16:14.356 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 50.524728934000002, 2024-09-18T10:16:14.356 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.50636473400000004, 2024-09-18T10:16:14.357 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:14.357 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:16:14.357 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:14.357 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.357 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:14.357 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.772911+0000", 2024-09-18T10:16:14.357 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:14.357 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.357 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.357 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:14.357 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.772911+0000", 2024-09-18T10:16:14.357 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:14.357 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.358 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.358 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:14.358 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.772909+0000", 2024-09-18T10:16:14.358 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:16:14.358 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.358 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.358 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:14.358 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.772923+0000", 2024-09-18T10:16:14.358 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3957e-05 2024-09-18T10:16:14.358 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.358 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.358 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:14.358 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.772956+0000", 2024-09-18T10:16:14.358 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2925000000000001e-05 2024-09-18T10:16:14.358 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.359 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.359 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:14.359 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.772969+0000", 2024-09-18T10:16:14.359 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2516e-05 2024-09-18T10:16:14.359 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.359 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.359 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:14.359 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.279192+0000", 2024-09-18T10:16:14.359 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.50622349799999999 2024-09-18T10:16:14.359 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.359 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.359 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:14.359 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.279276+0000", 2024-09-18T10:16:14.359 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.3498000000000006e-05 2024-09-18T10:16:14.360 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:14.360 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:14.360 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:14.360 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.360 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.360 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDPGPushReply(3.17s0 67/64 [PushReplyOp(3:ead53b0f:::smithi18925896-25:13)])", 2024-09-18T10:16:14.360 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:23.875511+0000", 2024-09-18T10:16:14.360 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 50.422129308999999, 2024-09-18T10:16:14.360 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.50638906400000006, 2024-09-18T10:16:14.360 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:14.360 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:16:14.360 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:14.360 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.361 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:14.361 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.875511+0000", 2024-09-18T10:16:14.361 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:14.361 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.361 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.361 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:14.361 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.875511+0000", 2024-09-18T10:16:14.361 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:14.361 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.361 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.361 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:14.361 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.875510+0000", 2024-09-18T10:16:14.361 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999995 2024-09-18T10:16:14.362 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.362 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.362 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:14.362 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.875516+0000", 2024-09-18T10:16:14.362 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.8930000000000002e-06 2024-09-18T10:16:14.362 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.362 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.362 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:14.362 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.875526+0000", 2024-09-18T10:16:14.362 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0183000000000001e-05 2024-09-18T10:16:14.362 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.362 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.362 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:14.362 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:23.975617+0000", 2024-09-18T10:16:14.363 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10009069299999999 2024-09-18T10:16:14.363 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.363 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.363 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:14.363 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.381839+0000", 2024-09-18T10:16:14.363 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.40622223600000001 2024-09-18T10:16:14.363 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.363 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.363 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:14.363 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.381900+0000", 2024-09-18T10:16:14.363 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.0572000000000002e-05 2024-09-18T10:16:14.363 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:14.363 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:14.363 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:14.364 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.364 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.364 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:635 3.17s0 3:ead53b0f:::smithi18925896-25:head [append 0~655360] snapc 21=[20,1f,1e,1a,16,11,10] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-18T10:16:14.364 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:29.755594+0000", 2024-09-18T10:16:14.364 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 44.542046388999999, 2024-09-18T10:16:14.364 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.62129057300000001, 2024-09-18T10:16:14.364 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:14.364 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:14.364 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:14.364 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:16:14.364 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:14.364 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 635 2024-09-18T10:16:14.364 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.365 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:14.365 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.365 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:14.365 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.755594+0000", 2024-09-18T10:16:14.365 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:14.365 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.365 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.365 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:14.365 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.755594+0000", 2024-09-18T10:16:14.365 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:14.365 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.365 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.365 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:14.366 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.755590+0000", 2024-09-18T10:16:14.366 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999967 2024-09-18T10:16:14.366 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.366 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.366 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:14.366 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.756465+0000", 2024-09-18T10:16:14.366 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00087483000000000005 2024-09-18T10:16:14.366 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.366 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.366 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:14.366 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.756502+0000", 2024-09-18T10:16:14.366 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.7067999999999998e-05 2024-09-18T10:16:14.366 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.366 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.367 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:14.367 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.756516+0000", 2024-09-18T10:16:14.367 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4071e-05 2024-09-18T10:16:14.367 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.367 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.367 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:14.367 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.760022+0000", 2024-09-18T10:16:14.367 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0035059570000000001 2024-09-18T10:16:14.367 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.367 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.367 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:14.367 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.760234+0000", 2024-09-18T10:16:14.367 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00021161199999999999 2024-09-18T10:16:14.367 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.368 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.368 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:14.368 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.762520+0000", 2024-09-18T10:16:14.368 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0022855760000000001 2024-09-18T10:16:14.368 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.368 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.368 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:14.368 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.770533+0000", 2024-09-18T10:16:14.368 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0080138269999999994 2024-09-18T10:16:14.368 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.368 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.369 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:14.369 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:30.376766+0000", 2024-09-18T10:16:14.369 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.60623257500000005 2024-09-18T10:16:14.369 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.369 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.369 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:14.369 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:30.376884+0000", 2024-09-18T10:16:14.369 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00011819699999999999 2024-09-18T10:16:14.369 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:14.369 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:14.369 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:14.369 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.369 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.370 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:636 3.17s0 3:ead53b0f:::smithi18925896-25:head [append 0~606208] snapc 21=[20,1f,1e,1a,16,11,10] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-18T10:16:14.370 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:29.758851+0000", 2024-09-18T10:16:14.370 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 44.538788736000001, 2024-09-18T10:16:14.370 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.61852410899999999, 2024-09-18T10:16:14.370 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:14.370 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:14.370 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:14.370 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:16:14.370 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:14.370 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 636 2024-09-18T10:16:14.370 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.370 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:14.370 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.371 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:14.371 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.758851+0000", 2024-09-18T10:16:14.371 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:14.371 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.371 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.371 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:14.371 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.758851+0000", 2024-09-18T10:16:14.371 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:14.371 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.371 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.371 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:14.371 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.758849+0000", 2024-09-18T10:16:14.371 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999976 2024-09-18T10:16:14.372 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.372 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.372 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:14.372 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.759643+0000", 2024-09-18T10:16:14.372 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00079385800000000004 2024-09-18T10:16:14.372 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.372 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.372 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:14.372 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.759677+0000", 2024-09-18T10:16:14.372 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.4504e-05 2024-09-18T10:16:14.372 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.372 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.372 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:14.373 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.759691+0000", 2024-09-18T10:16:14.373 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3586e-05 2024-09-18T10:16:14.373 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.373 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.373 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:14.373 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.765309+0000", 2024-09-18T10:16:14.373 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0056179460000000004 2024-09-18T10:16:14.373 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.373 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.373 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:14.373 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.765656+0000", 2024-09-18T10:16:14.373 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000347783 2024-09-18T10:16:14.373 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.373 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.373 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:14.374 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.766735+0000", 2024-09-18T10:16:14.374 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0010785460000000001 2024-09-18T10:16:14.374 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.374 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.374 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:14.374 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.973532+0000", 2024-09-18T10:16:14.374 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20679750599999999 2024-09-18T10:16:14.374 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.374 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.374 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:14.374 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:30.377254+0000", 2024-09-18T10:16:14.374 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.40372181400000001 2024-09-18T10:16:14.374 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.375 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.375 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:14.375 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:30.377375+0000", 2024-09-18T10:16:14.375 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000121037 2024-09-18T10:16:14.375 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:14.375 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:14.375 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:14.375 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.375 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.375 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:637 3.17s0 3:ead53b0f:::smithi18925896-25:head [append 0~475136] snapc 21=[20,1f,1e,1a,16,11,10] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-18T10:16:14.375 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:29.761784+0000", 2024-09-18T10:16:14.375 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 44.535855882999996, 2024-09-18T10:16:14.375 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.61605535199999995, 2024-09-18T10:16:14.376 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:14.376 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:14.376 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:14.376 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:16:14.376 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:14.376 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 637 2024-09-18T10:16:14.376 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.376 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:14.376 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.376 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:14.376 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.761784+0000", 2024-09-18T10:16:14.376 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:14.376 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.377 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.377 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:14.377 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.761784+0000", 2024-09-18T10:16:14.377 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:14.377 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.377 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.377 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:14.377 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.761776+0000", 2024-09-18T10:16:14.377 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999919 2024-09-18T10:16:14.377 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.377 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.377 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:14.377 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.762424+0000", 2024-09-18T10:16:14.378 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00064803899999999999 2024-09-18T10:16:14.378 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.378 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.378 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:14.378 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.762455+0000", 2024-09-18T10:16:14.378 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1316000000000003e-05 2024-09-18T10:16:14.378 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.378 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.378 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:14.378 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.762471+0000", 2024-09-18T10:16:14.378 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.6174000000000001e-05 2024-09-18T10:16:14.378 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.378 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.379 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:14.379 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.767732+0000", 2024-09-18T10:16:14.379 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0052603190000000003 2024-09-18T10:16:14.379 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.379 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.379 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:14.379 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.767873+0000", 2024-09-18T10:16:14.379 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000140949 2024-09-18T10:16:14.379 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.379 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.379 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:14.379 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.768566+0000", 2024-09-18T10:16:14.379 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00069325799999999998 2024-09-18T10:16:14.379 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.380 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.380 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:14.380 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.973546+0000", 2024-09-18T10:16:14.380 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.204980144 2024-09-18T10:16:14.380 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.380 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.380 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:14.380 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:30.377732+0000", 2024-09-18T10:16:14.380 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.404185558 2024-09-18T10:16:14.380 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.380 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.380 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:14.380 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:30.377839+0000", 2024-09-18T10:16:14.381 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000107909 2024-09-18T10:16:14.381 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:14.381 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:14.381 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:14.381 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.381 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.381 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:638 3.17s0 3:ead53b0f:::smithi18925896-25:head [append 0~81920] snapc 21=[20,1f,1e,1a,16,11,10] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-18T10:16:14.381 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:29.762493+0000", 2024-09-18T10:16:14.381 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 44.535146863000001, 2024-09-18T10:16:14.381 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.61563624900000002, 2024-09-18T10:16:14.381 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:14.381 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:14.381 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:14.381 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:16:14.382 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:14.382 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 638 2024-09-18T10:16:14.382 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.382 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:14.382 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.382 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:14.382 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.762493+0000", 2024-09-18T10:16:14.382 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:14.382 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.382 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.382 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:14.382 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.762493+0000", 2024-09-18T10:16:14.382 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:14.382 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.383 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.383 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:14.383 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.762492+0000", 2024-09-18T10:16:14.383 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:16:14.383 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.383 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.383 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:14.383 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.762566+0000", 2024-09-18T10:16:14.383 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.3977e-05 2024-09-18T10:16:14.383 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.383 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.383 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:14.383 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.762585+0000", 2024-09-18T10:16:14.384 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.9074000000000001e-05 2024-09-18T10:16:14.384 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.384 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.384 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:14.384 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.762595+0000", 2024-09-18T10:16:14.384 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0913999999999999e-05 2024-09-18T10:16:14.384 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.384 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.384 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:14.384 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.769730+0000", 2024-09-18T10:16:14.384 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0071346680000000003 2024-09-18T10:16:14.384 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.384 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.385 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:14.385 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.769816+0000", 2024-09-18T10:16:14.385 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.5422999999999998e-05 2024-09-18T10:16:14.385 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.385 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.385 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:14.385 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.770214+0000", 2024-09-18T10:16:14.385 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00039818999999999998 2024-09-18T10:16:14.385 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.385 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.385 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:14.385 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.973818+0000", 2024-09-18T10:16:14.385 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20360445499999999 2024-09-18T10:16:14.385 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.386 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.386 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:14.386 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:30.378035+0000", 2024-09-18T10:16:14.386 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.40421707600000001 2024-09-18T10:16:14.386 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.386 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.386 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:14.386 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:30.378129+0000", 2024-09-18T10:16:14.386 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.4056999999999997e-05 2024-09-18T10:16:14.386 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:14.386 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:14.386 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:14.386 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.387 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.387 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:639 3.17s0 3:ead53b0f:::smithi18925896-25:head [setxattr _header (57) in=7b] snapc 21=[20,1f,1e,1a,16,11,10] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-18T10:16:14.387 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:29.762609+0000", 2024-09-18T10:16:14.387 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 44.535030708999997, 2024-09-18T10:16:14.387 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.61577652400000005, 2024-09-18T10:16:14.387 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:14.387 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:14.387 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:14.387 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:16:14.387 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:14.387 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 639 2024-09-18T10:16:14.387 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.387 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:14.387 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.388 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:14.388 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.762609+0000", 2024-09-18T10:16:14.388 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:14.388 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.388 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.388 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:14.388 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.762609+0000", 2024-09-18T10:16:14.388 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:14.388 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.388 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.388 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:14.388 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.762608+0000", 2024-09-18T10:16:14.388 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:16:14.388 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.389 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.389 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:14.389 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.762615+0000", 2024-09-18T10:16:14.389 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.2339999999999997e-06 2024-09-18T10:16:14.389 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.389 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.389 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:14.389 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.762628+0000", 2024-09-18T10:16:14.389 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2848e-05 2024-09-18T10:16:14.389 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.389 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.389 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:14.389 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.762636+0000", 2024-09-18T10:16:14.389 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.0970000000000006e-06 2024-09-18T10:16:14.389 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.390 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.390 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:14.390 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.870703+0000", 2024-09-18T10:16:14.390 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.108067121 2024-09-18T10:16:14.390 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.390 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.390 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:14.390 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.870949+0000", 2024-09-18T10:16:14.390 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00024526700000000002 2024-09-18T10:16:14.390 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.390 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.390 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:14.390 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.871876+0000", 2024-09-18T10:16:14.390 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00092680400000000004 2024-09-18T10:16:14.391 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.391 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.391 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:14.391 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:29.973821+0000", 2024-09-18T10:16:14.391 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10194552599999999 2024-09-18T10:16:14.391 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.391 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.391 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:14.391 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:30.378300+0000", 2024-09-18T10:16:14.391 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.40447920900000001 2024-09-18T10:16:14.391 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.391 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.391 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:14.392 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:30.378386+0000", 2024-09-18T10:16:14.392 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.5531999999999994e-05 2024-09-18T10:16:14.392 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:14.392 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:14.392 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:14.392 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.392 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.392 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWrite(3.7s5 74/40 ECSubWrite(tid=168, reqid=osd.4.0:166, at_version=0'0, trim_to=0'0, roll_forward_to=74'47))", 2024-09-18T10:16:14.392 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:30.378493+0000", 2024-09-18T10:16:14.392 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 43.919147064999997, 2024-09-18T10:16:14.392 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.605881475, 2024-09-18T10:16:14.392 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:14.392 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:16:14.392 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:14.393 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.393 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:14.393 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:30.378493+0000", 2024-09-18T10:16:14.393 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:14.393 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.393 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.393 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:14.393 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:30.378493+0000", 2024-09-18T10:16:14.393 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:14.393 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.393 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.393 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:14.393 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:30.378492+0000", 2024-09-18T10:16:14.393 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:16:14.394 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.394 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.394 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:14.394 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:30.378513+0000", 2024-09-18T10:16:14.394 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.1427e-05 2024-09-18T10:16:14.394 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.394 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.394 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:14.394 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:30.378550+0000", 2024-09-18T10:16:14.394 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.7351999999999997e-05 2024-09-18T10:16:14.394 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.394 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.394 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:14.394 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:30.478641+0000", 2024-09-18T10:16:14.395 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100090965 2024-09-18T10:16:14.395 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.395 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.395 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:14.395 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:30.882297+0000", 2024-09-18T10:16:14.395 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.40365594799999999 2024-09-18T10:16:14.395 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.395 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.395 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:14.395 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:30.882320+0000", 2024-09-18T10:16:14.395 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.3127999999999999e-05 2024-09-18T10:16:14.395 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.395 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.395 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:14.396 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:30.984327+0000", 2024-09-18T10:16:14.396 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10200699100000001 2024-09-18T10:16:14.396 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.396 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.396 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:14.396 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:30.984374+0000", 2024-09-18T10:16:14.396 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.6953000000000003e-05 2024-09-18T10:16:14.396 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:14.396 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:14.396 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:14.396 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.396 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.396 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:758 3.17s0 3:ee962131:::smithi18925896-29:head [append 0~524288] snapc 2f=[2f,29,28,20,1f,1e,1a,11] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-18T10:16:14.396 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:40.612706+0000", 2024-09-18T10:16:14.397 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 33.684934312999999, 2024-09-18T10:16:14.397 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.60348796500000002, 2024-09-18T10:16:14.397 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:14.397 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:14.397 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:14.397 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:16:14.397 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:14.397 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 758 2024-09-18T10:16:14.397 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.397 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:14.397 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.397 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:14.397 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.612706+0000", 2024-09-18T10:16:14.397 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:14.398 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.398 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.398 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:14.398 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.612706+0000", 2024-09-18T10:16:14.398 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:14.398 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.398 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.398 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:14.398 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.612702+0000", 2024-09-18T10:16:14.398 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999967 2024-09-18T10:16:14.398 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.398 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.398 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:14.398 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.613684+0000", 2024-09-18T10:16:14.399 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00098185200000000007 2024-09-18T10:16:14.399 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.399 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.399 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:14.399 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.613722+0000", 2024-09-18T10:16:14.399 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.7490000000000002e-05 2024-09-18T10:16:14.399 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.399 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.399 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:14.399 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.613741+0000", 2024-09-18T10:16:14.399 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.9707999999999998e-05 2024-09-18T10:16:14.399 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.399 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.399 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:14.400 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.613931+0000", 2024-09-18T10:16:14.400 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00018992799999999999 2024-09-18T10:16:14.400 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.400 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.400 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op must wait for map", 2024-09-18T10:16:14.400 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.613950+0000", 2024-09-18T10:16:14.400 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8909000000000001e-05 2024-09-18T10:16:14.400 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.400 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.400 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:14.400 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.726459+0000", 2024-09-18T10:16:14.400 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.112508555 2024-09-18T10:16:14.400 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.401 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.401 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:14.401 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.726722+0000", 2024-09-18T10:16:14.401 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00026331999999999999 2024-09-18T10:16:14.401 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.401 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.401 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:14.401 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.728892+0000", 2024-09-18T10:16:14.401 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0021697040000000002 2024-09-18T10:16:14.401 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.401 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.401 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:14.401 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.741279+0000", 2024-09-18T10:16:14.402 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.012386856 2024-09-18T10:16:14.402 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.402 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.402 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:14.402 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:41.215936+0000", 2024-09-18T10:16:14.402 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.474657153 2024-09-18T10:16:14.402 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.402 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.402 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:14.402 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:41.216194+0000", 2024-09-18T10:16:14.402 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00025781799999999999 2024-09-18T10:16:14.402 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:14.402 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:14.403 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:14.403 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.403 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.403 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:759 3.17s0 3:ee962131:::smithi18925896-29:head [append 0~573440] snapc 2f=[2f,29,28,20,1f,1e,1a,11] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-18T10:16:14.403 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:40.620305+0000", 2024-09-18T10:16:14.403 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 33.677334651999999, 2024-09-18T10:16:14.403 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.59650130599999995, 2024-09-18T10:16:14.403 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:14.403 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:14.403 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:14.403 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:16:14.403 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:14.403 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 759 2024-09-18T10:16:14.403 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.404 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:14.404 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.404 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:14.404 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.620305+0000", 2024-09-18T10:16:14.404 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:14.404 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.404 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.404 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:14.404 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.620305+0000", 2024-09-18T10:16:14.404 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:14.404 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.404 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.404 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:14.404 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.620302+0000", 2024-09-18T10:16:14.405 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999971 2024-09-18T10:16:14.405 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.405 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.405 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:14.405 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.620967+0000", 2024-09-18T10:16:14.405 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00066427800000000003 2024-09-18T10:16:14.405 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.405 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.405 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:14.405 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.620994+0000", 2024-09-18T10:16:14.405 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.76e-05 2024-09-18T10:16:14.405 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.405 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.406 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:14.406 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.621004+0000", 2024-09-18T10:16:14.406 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0098000000000001e-05 2024-09-18T10:16:14.406 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.406 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.406 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:14.406 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.621152+0000", 2024-09-18T10:16:14.406 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00014735799999999999 2024-09-18T10:16:14.406 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.406 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.406 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:16:14.406 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.621173+0000", 2024-09-18T10:16:14.406 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.1223e-05 2024-09-18T10:16:14.406 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.407 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.407 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:14.407 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.730351+0000", 2024-09-18T10:16:14.407 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.109177487 2024-09-18T10:16:14.407 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.407 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.407 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:14.407 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.730553+0000", 2024-09-18T10:16:14.407 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00020222600000000001 2024-09-18T10:16:14.407 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.407 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.407 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:14.408 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.732189+0000", 2024-09-18T10:16:14.408 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001636153 2024-09-18T10:16:14.408 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.408 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.408 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:14.408 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.741283+0000", 2024-09-18T10:16:14.408 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0090935889999999991 2024-09-18T10:16:14.408 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.408 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.408 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:14.408 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:41.216570+0000", 2024-09-18T10:16:14.409 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.475287444 2024-09-18T10:16:14.409 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.409 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.409 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:14.409 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:41.216807+0000", 2024-09-18T10:16:14.409 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00023667999999999999 2024-09-18T10:16:14.409 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:14.409 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:14.409 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:14.409 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.409 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.409 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:760 3.17s0 3:ee962131:::smithi18925896-29:head [append 0~688128] snapc 2f=[2f,29,28,20,1f,1e,1a,11] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-18T10:16:14.409 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:40.624963+0000", 2024-09-18T10:16:14.409 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 33.672677147000002, 2024-09-18T10:16:14.410 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.592485437, 2024-09-18T10:16:14.410 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:14.410 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:14.410 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:14.410 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:16:14.410 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:14.410 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 760 2024-09-18T10:16:14.410 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.410 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:14.410 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.410 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:14.410 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.624963+0000", 2024-09-18T10:16:14.410 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:14.410 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.411 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.411 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:14.411 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.624963+0000", 2024-09-18T10:16:14.411 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:14.411 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.411 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.411 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:14.411 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.624960+0000", 2024-09-18T10:16:14.411 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999971 2024-09-18T10:16:14.411 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.411 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.411 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:14.411 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.625766+0000", 2024-09-18T10:16:14.411 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00080539100000000003 2024-09-18T10:16:14.412 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.412 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.412 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:14.412 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.625792+0000", 2024-09-18T10:16:14.412 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5989000000000001e-05 2024-09-18T10:16:14.412 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.412 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.412 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:14.412 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.625801+0000", 2024-09-18T10:16:14.412 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.4080000000000004e-06 2024-09-18T10:16:14.412 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.412 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.412 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:14.413 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.627383+0000", 2024-09-18T10:16:14.413 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0015816440000000001 2024-09-18T10:16:14.413 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.413 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.413 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:16:14.413 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.627397+0000", 2024-09-18T10:16:14.413 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3973e-05 2024-09-18T10:16:14.413 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.413 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.413 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:14.413 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.733560+0000", 2024-09-18T10:16:14.413 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.106163805 2024-09-18T10:16:14.413 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.413 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.414 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:14.414 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.733770+0000", 2024-09-18T10:16:14.414 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00020980900000000001 2024-09-18T10:16:14.414 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.414 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.414 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:14.414 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.735644+0000", 2024-09-18T10:16:14.414 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001874164 2024-09-18T10:16:14.414 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.414 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.414 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:14.414 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.741750+0000", 2024-09-18T10:16:14.414 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0061058379999999997 2024-09-18T10:16:14.415 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.415 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.415 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:14.415 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:41.217140+0000", 2024-09-18T10:16:14.415 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.47538995699999997 2024-09-18T10:16:14.415 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.415 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.415 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:14.415 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:41.217448+0000", 2024-09-18T10:16:14.415 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000308089 2024-09-18T10:16:14.415 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:14.415 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:14.415 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:14.415 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.416 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.416 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:761 3.17s0 3:ee962131:::smithi18925896-29:head [append 0~294912] snapc 2f=[2f,29,28,20,1f,1e,1a,11] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-18T10:16:14.416 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:40.626751+0000", 2024-09-18T10:16:14.416 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 33.67088888, 2024-09-18T10:16:14.416 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.59137154800000002, 2024-09-18T10:16:14.416 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:14.416 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:14.416 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:14.416 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:16:14.416 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:14.416 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 761 2024-09-18T10:16:14.416 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.416 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:14.416 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.417 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:14.417 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.626751+0000", 2024-09-18T10:16:14.417 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:14.417 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.417 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.417 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:14.417 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.626751+0000", 2024-09-18T10:16:14.417 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:14.417 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.417 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.417 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:14.417 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.626749+0000", 2024-09-18T10:16:14.417 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:16:14.417 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.418 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.418 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:14.418 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.627187+0000", 2024-09-18T10:16:14.418 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000437682 2024-09-18T10:16:14.418 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.418 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.418 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:14.418 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.627209+0000", 2024-09-18T10:16:14.418 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.2195999999999999e-05 2024-09-18T10:16:14.418 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.418 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.418 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:14.418 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.627218+0000", 2024-09-18T10:16:14.418 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.4829999999999999e-06 2024-09-18T10:16:14.419 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.419 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.419 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:14.419 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.627438+0000", 2024-09-18T10:16:14.419 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00022063399999999999 2024-09-18T10:16:14.419 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.419 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.419 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:16:14.419 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.627447+0000", 2024-09-18T10:16:14.419 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.687e-06 2024-09-18T10:16:14.419 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.419 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.419 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:14.420 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.737040+0000", 2024-09-18T10:16:14.420 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10959300800000001 2024-09-18T10:16:14.420 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.420 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.420 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:14.420 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.737281+0000", 2024-09-18T10:16:14.420 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00024113100000000001 2024-09-18T10:16:14.420 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.420 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.420 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:14.420 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.738816+0000", 2024-09-18T10:16:14.420 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0015353649999999999 2024-09-18T10:16:14.420 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.420 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.421 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:14.421 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.741752+0000", 2024-09-18T10:16:14.421 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0029357239999999998 2024-09-18T10:16:14.421 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.421 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.421 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:14.421 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:41.217885+0000", 2024-09-18T10:16:14.421 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.476132522 2024-09-18T10:16:14.421 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.421 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.421 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:14.421 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:41.218123+0000", 2024-09-18T10:16:14.421 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000237968 2024-09-18T10:16:14.422 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:14.422 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:14.422 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:14.422 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.422 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.422 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:762 3.17s0 3:ee962131:::smithi18925896-29:head [setxattr _header (57) in=7b] snapc 2f=[2f,29,28,20,1f,1e,1a,11] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-18T10:16:14.422 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:40.627233+0000", 2024-09-18T10:16:14.422 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 33.670407048000001, 2024-09-18T10:16:14.422 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.68868450599999997, 2024-09-18T10:16:14.422 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:14.422 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:14.422 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:14.422 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:16:14.422 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:14.423 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 762 2024-09-18T10:16:14.423 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.423 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:14.423 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.423 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:14.423 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.627233+0000", 2024-09-18T10:16:14.423 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:14.423 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.423 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.423 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:14.423 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.627233+0000", 2024-09-18T10:16:14.423 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:14.423 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.423 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.423 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:14.424 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.627232+0000", 2024-09-18T10:16:14.424 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:16:14.424 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.424 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.424 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:14.424 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.627238+0000", 2024-09-18T10:16:14.424 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.7570000000000001e-06 2024-09-18T10:16:14.424 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.424 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.424 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:14.424 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.627246+0000", 2024-09-18T10:16:14.424 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.0199999999999994e-06 2024-09-18T10:16:14.424 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.425 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.425 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:14.425 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.627251+0000", 2024-09-18T10:16:14.425 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9599999999999999e-06 2024-09-18T10:16:14.425 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.425 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.425 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:14.425 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.627481+0000", 2024-09-18T10:16:14.425 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00023046199999999999 2024-09-18T10:16:14.425 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.425 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.425 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:16:14.425 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.627493+0000", 2024-09-18T10:16:14.425 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2173e-05 2024-09-18T10:16:14.426 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.426 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.426 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:14.426 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.740060+0000", 2024-09-18T10:16:14.426 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.112566686 2024-09-18T10:16:14.426 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.426 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.426 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:14.426 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.740191+0000", 2024-09-18T10:16:14.426 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00013140999999999999 2024-09-18T10:16:14.426 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.426 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.426 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:14.426 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.740901+0000", 2024-09-18T10:16:14.427 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00070978199999999995 2024-09-18T10:16:14.427 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.427 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.427 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:14.427 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.741853+0000", 2024-09-18T10:16:14.427 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00095228399999999996 2024-09-18T10:16:14.427 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.427 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.427 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:14.427 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:41.315753+0000", 2024-09-18T10:16:14.427 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.57389904999999997 2024-09-18T10:16:14.427 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.427 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.428 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:14.428 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:41.315917+0000", 2024-09-18T10:16:14.428 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00016491699999999999 2024-09-18T10:16:14.428 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:14.428 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:14.428 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:14.428 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.428 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.428 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:763 3.17s0 3:ee962131:::smithi18925896-29:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e85)", 2024-09-18T10:16:14.428 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:40.627257+0000", 2024-09-18T10:16:14.428 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 33.670382762999999, 2024-09-18T10:16:14.428 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.89335734899999997, 2024-09-18T10:16:14.428 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:14.429 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "started", 2024-09-18T10:16:14.429 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:14.429 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:16:14.429 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:14.429 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 763 2024-09-18T10:16:14.429 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.429 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:14.429 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.429 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:14.429 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.627257+0000", 2024-09-18T10:16:14.429 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:14.429 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.429 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.429 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:14.430 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.627257+0000", 2024-09-18T10:16:14.430 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:14.430 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.430 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.430 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:14.430 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.627256+0000", 2024-09-18T10:16:14.430 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:16:14.430 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.430 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.430 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:14.430 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.627260+0000", 2024-09-18T10:16:14.430 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.7450000000000001e-06 2024-09-18T10:16:14.430 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.431 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.431 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:14.431 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.627269+0000", 2024-09-18T10:16:14.431 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.4400000000000005e-06 2024-09-18T10:16:14.431 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.431 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.431 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:14.431 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.627273+0000", 2024-09-18T10:16:14.431 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.8600000000000001e-06 2024-09-18T10:16:14.431 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.431 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.431 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:14.431 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.717888+0000", 2024-09-18T10:16:14.431 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.090614586999999996 2024-09-18T10:16:14.432 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.432 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.432 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:16:14.432 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.717909+0000", 2024-09-18T10:16:14.432 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.0942999999999999e-05 2024-09-18T10:16:14.432 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.432 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.432 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:14.432 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.741375+0000", 2024-09-18T10:16:14.432 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.023465604000000001 2024-09-18T10:16:14.432 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.432 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.432 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for rw locks", 2024-09-18T10:16:14.432 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:40.741656+0000", 2024-09-18T10:16:14.433 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00028103600000000002 2024-09-18T10:16:14.433 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.433 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.433 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:14.433 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:41.317563+0000", 2024-09-18T10:16:14.433 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.57590774499999997 2024-09-18T10:16:14.433 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.433 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.433 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:14.433 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:41.317798+0000", 2024-09-18T10:16:14.433 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00023456099999999999 2024-09-18T10:16:14.433 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.433 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.433 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:14.434 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:41.520615+0000", 2024-09-18T10:16:14.434 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20281658499999999 2024-09-18T10:16:14.434 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:14.434 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:14.434 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:14.434 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:14.434 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:14.434 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-09-18T10:16:14.435 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_ops_in_flight 2024-09-18T10:16:14.547 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-09-18T10:16:14.547 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-09-18T10:16:14.547 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-09-18T10:16:14.547 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-09-18T10:16:14.556 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_blocked_ops 2024-09-18T10:16:14.657 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-09-18T10:16:14.657 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-09-18T10:16:14.657 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-09-18T10:16:14.657 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-09-18T10:16:14.657 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-09-18T10:16:14.666 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_historic_ops 2024-09-18T10:16:14.764 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-09-18T10:16:14.765 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-09-18T10:16:14.765 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-09-18T10:16:14.765 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-09-18T10:16:14.765 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.765 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4341.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-18T10:16:14.765 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:35.324351+0000", 2024-09-18T10:16:14.765 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 99.436628717999994, 2024-09-18T10:16:14.765 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.36640713699999999, 2024-09-18T10:16:14.765 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:14.765 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:14.765 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:14.765 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4341", 2024-09-18T10:16:14.765 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.29:0/3001591505", 2024-09-18T10:16:14.766 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 1 2024-09-18T10:16:14.766 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.766 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:14.766 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.766 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:14.766 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.324351+0000", 2024-09-18T10:16:14.766 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:14.766 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.766 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.766 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:14.766 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.324351+0000", 2024-09-18T10:16:14.766 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:14.766 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.766 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.767 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:14.767 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.324348+0000", 2024-09-18T10:16:14.767 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999976 2024-09-18T10:16:14.767 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.767 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.767 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:14.767 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.324359+0000", 2024-09-18T10:16:14.767 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1048e-05 2024-09-18T10:16:14.767 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.767 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.767 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:14.767 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.324383+0000", 2024-09-18T10:16:14.767 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.3813000000000001e-05 2024-09-18T10:16:14.767 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.768 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.768 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:14.768 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.324421+0000", 2024-09-18T10:16:14.768 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8012000000000001e-05 2024-09-18T10:16:14.768 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.768 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.768 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:14.768 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.324548+0000", 2024-09-18T10:16:14.768 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00012702199999999999 2024-09-18T10:16:14.768 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.768 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.768 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for peered", 2024-09-18T10:16:14.768 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.324565+0000", 2024-09-18T10:16:14.768 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.6325e-05 2024-09-18T10:16:14.769 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.769 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.769 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:14.769 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.588380+0000", 2024-09-18T10:16:14.769 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.263815458 2024-09-18T10:16:14.769 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.769 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.769 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:14.769 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.588626+0000", 2024-09-18T10:16:14.769 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00024633499999999998 2024-09-18T10:16:14.769 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.769 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.769 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for subops from 5", 2024-09-18T10:16:14.769 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.588803+0000", 2024-09-18T10:16:14.770 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00017640599999999999 2024-09-18T10:16:14.770 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.770 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.770 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op_commit", 2024-09-18T10:16:14.770 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.589943+0000", 2024-09-18T10:16:14.770 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0011400049999999999 2024-09-18T10:16:14.770 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.770 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.770 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_commit_rec", 2024-09-18T10:16:14.770 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.690623+0000", 2024-09-18T10:16:14.770 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100680032 2024-09-18T10:16:14.770 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.770 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.770 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:14.771 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.690702+0000", 2024-09-18T10:16:14.771 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.9342e-05 2024-09-18T10:16:14.771 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.771 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.771 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:14.771 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:35.690758+0000", 2024-09-18T10:16:14.771 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5639e-05 2024-09-18T10:16:14.771 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:14.771 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:14.771 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:14.771 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.771 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.771 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:264 3.2s0 3:46b2871c:::smithi18925896-34:head [append 0~606208] snapc 5=[5,4,3] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-18T10:16:14.771 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:56.739704+0000", 2024-09-18T10:16:14.772 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 78.021275732999996, 2024-09-18T10:16:14.772 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.79624036899999995, 2024-09-18T10:16:14.772 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:14.772 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:14.772 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:14.772 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:16:14.772 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:14.772 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 264 2024-09-18T10:16:14.772 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.772 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:14.772 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.772 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:14.772 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.739704+0000", 2024-09-18T10:16:14.773 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:14.773 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.773 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.773 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:14.773 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.739704+0000", 2024-09-18T10:16:14.773 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:14.773 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.773 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.773 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:14.773 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.739702+0000", 2024-09-18T10:16:14.773 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:16:14.773 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.773 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.773 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:14.773 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.740467+0000", 2024-09-18T10:16:14.774 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00076493100000000005 2024-09-18T10:16:14.774 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.774 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.774 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:14.774 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.740485+0000", 2024-09-18T10:16:14.774 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8516e-05 2024-09-18T10:16:14.774 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.774 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.774 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:14.774 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.740495+0000", 2024-09-18T10:16:14.774 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.2669999999999995e-06 2024-09-18T10:16:14.774 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.774 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.774 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:14.775 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.028948+0000", 2024-09-18T10:16:14.775 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.28845313500000003 2024-09-18T10:16:14.775 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.775 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.775 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for peered", 2024-09-18T10:16:14.775 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.028969+0000", 2024-09-18T10:16:14.775 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.1369000000000002e-05 2024-09-18T10:16:14.775 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.775 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.775 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:14.775 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.228324+0000", 2024-09-18T10:16:14.775 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.19935476699999999 2024-09-18T10:16:14.775 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.775 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.776 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:14.776 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.228558+0000", 2024-09-18T10:16:14.776 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00023393099999999999 2024-09-18T10:16:14.776 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.776 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.776 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:14.776 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.230390+0000", 2024-09-18T10:16:14.776 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018326970000000001 2024-09-18T10:16:14.776 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.776 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.776 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:14.776 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.239491+0000", 2024-09-18T10:16:14.776 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0091007950000000001 2024-09-18T10:16:14.777 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.777 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.777 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:14.777 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.535834+0000", 2024-09-18T10:16:14.777 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.29634234100000001 2024-09-18T10:16:14.777 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.777 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.777 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:14.777 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.535944+0000", 2024-09-18T10:16:14.777 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000110401 2024-09-18T10:16:14.777 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:14.777 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:14.777 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:14.777 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.777 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.778 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:265 3.2s0 3:46b2871c:::smithi18925896-34:head [append 0~376832] snapc 5=[5,4,3] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-18T10:16:14.778 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:56.741882+0000", 2024-09-18T10:16:14.778 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 78.019097711000001, 2024-09-18T10:16:14.778 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.80071129500000005, 2024-09-18T10:16:14.778 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:14.778 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:14.778 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:14.778 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:16:14.778 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:14.778 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 265 2024-09-18T10:16:14.778 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.778 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:14.778 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.778 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:14.779 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.741882+0000", 2024-09-18T10:16:14.779 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:14.779 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.779 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.779 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:14.779 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.741882+0000", 2024-09-18T10:16:14.779 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:14.779 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.779 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.779 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:14.779 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.741880+0000", 2024-09-18T10:16:14.779 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:16:14.779 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.779 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.780 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:14.780 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742251+0000", 2024-09-18T10:16:14.780 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00037050499999999998 2024-09-18T10:16:14.780 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.780 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.780 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:14.780 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742264+0000", 2024-09-18T10:16:14.780 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2787e-05 2024-09-18T10:16:14.780 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.780 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.780 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:14.780 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742269+0000", 2024-09-18T10:16:14.780 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.0509999999999996e-06 2024-09-18T10:16:14.781 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.781 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.781 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:14.781 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.234342+0000", 2024-09-18T10:16:14.781 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.492073712 2024-09-18T10:16:14.781 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.781 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.781 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:14.781 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.234556+0000", 2024-09-18T10:16:14.781 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00021380699999999999 2024-09-18T10:16:14.781 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.781 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.781 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:14.781 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.236316+0000", 2024-09-18T10:16:14.782 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0017597100000000001 2024-09-18T10:16:14.782 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.782 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.782 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:14.782 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.241476+0000", 2024-09-18T10:16:14.782 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0051604449999999996 2024-09-18T10:16:14.782 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.782 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.782 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:14.782 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.542502+0000", 2024-09-18T10:16:14.782 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.30102573700000002 2024-09-18T10:16:14.782 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.782 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.783 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:14.783 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.542593+0000", 2024-09-18T10:16:14.783 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.0859000000000005e-05 2024-09-18T10:16:14.783 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:14.783 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:14.783 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:14.783 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.783 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.783 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:266 3.2s0 3:46b2871c:::smithi18925896-34:head [setxattr _header (56) in=7b] snapc 5=[5,4,3] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-18T10:16:14.783 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:56.742274+0000", 2024-09-18T10:16:14.783 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 78.018705080000004, 2024-09-18T10:16:14.783 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.80056005299999999, 2024-09-18T10:16:14.783 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:14.783 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:14.784 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:14.784 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:16:14.784 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:14.784 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 266 2024-09-18T10:16:14.784 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.784 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:14.784 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.784 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:14.784 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742274+0000", 2024-09-18T10:16:14.784 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:14.785 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.785 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.785 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:14.785 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742274+0000", 2024-09-18T10:16:14.785 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:14.785 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.785 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.785 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:14.785 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742274+0000", 2024-09-18T10:16:14.785 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999995 2024-09-18T10:16:14.785 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.785 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.785 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:14.785 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742278+0000", 2024-09-18T10:16:14.786 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.0840000000000002e-06 2024-09-18T10:16:14.786 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.786 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.786 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:14.786 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742284+0000", 2024-09-18T10:16:14.786 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.1090000000000003e-06 2024-09-18T10:16:14.786 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.786 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.786 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:14.786 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742289+0000", 2024-09-18T10:16:14.786 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.4890000000000003e-06 2024-09-18T10:16:14.786 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.786 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.786 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:14.786 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.237660+0000", 2024-09-18T10:16:14.787 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.49537068299999998 2024-09-18T10:16:14.787 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.787 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.787 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:14.787 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.237924+0000", 2024-09-18T10:16:14.787 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00026387300000000003 2024-09-18T10:16:14.787 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.787 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.787 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:14.787 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.238979+0000", 2024-09-18T10:16:14.787 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0010549380000000001 2024-09-18T10:16:14.787 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.787 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.787 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:14.788 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.243571+0000", 2024-09-18T10:16:14.788 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0045919749999999999 2024-09-18T10:16:14.788 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.788 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.788 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:14.788 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.542742+0000", 2024-09-18T10:16:14.788 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.29917095999999999 2024-09-18T10:16:14.788 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.788 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.788 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:14.788 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.542834+0000", 2024-09-18T10:16:14.788 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.2536999999999998e-05 2024-09-18T10:16:14.788 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:14.788 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:14.789 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:14.789 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.789 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.789 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:267 3.2s0 3:46b2871c:::smithi18925896-34:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e41)", 2024-09-18T10:16:14.789 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:14:56.742292+0000", 2024-09-18T10:16:14.789 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 78.018687241999999, 2024-09-18T10:16:14.789 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.98973012599999999, 2024-09-18T10:16:14.789 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:14.789 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "started", 2024-09-18T10:16:14.789 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:14.789 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:16:14.789 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:14.789 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 267 2024-09-18T10:16:14.789 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.790 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:14.790 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.790 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:14.790 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742292+0000", 2024-09-18T10:16:14.790 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:14.790 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.790 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.790 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:14.790 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742292+0000", 2024-09-18T10:16:14.790 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:14.790 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.790 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.790 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:14.790 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742292+0000", 2024-09-18T10:16:14.791 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999995 2024-09-18T10:16:14.791 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.791 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.791 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:14.791 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742294+0000", 2024-09-18T10:16:14.791 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.103e-06 2024-09-18T10:16:14.791 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.791 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.791 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:14.791 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742298+0000", 2024-09-18T10:16:14.791 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8070000000000001e-06 2024-09-18T10:16:14.791 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.791 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.791 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:14.791 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:56.742300+0000", 2024-09-18T10:16:14.792 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5890000000000001e-06 2024-09-18T10:16:14.792 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.792 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.792 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:14.792 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.239711+0000", 2024-09-18T10:16:14.792 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.497410625 2024-09-18T10:16:14.792 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.792 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.792 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for rw locks", 2024-09-18T10:16:14.792 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.239786+0000", 2024-09-18T10:16:14.792 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.4827999999999996e-05 2024-09-18T10:16:14.792 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.792 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.792 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:14.793 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.543234+0000", 2024-09-18T10:16:14.793 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.30344799099999997 2024-09-18T10:16:14.793 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.793 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.793 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:14.793 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.543335+0000", 2024-09-18T10:16:14.793 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000101625 2024-09-18T10:16:14.793 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.793 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.793 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:14.793 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:14:57.732022+0000", 2024-09-18T10:16:14.793 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.18868686700000001 2024-09-18T10:16:14.793 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:14.793 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:14.794 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:14.794 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.794 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.794 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:496 3.2s0 3:44cb9e27:::smithi18925896-5:head [read 0~81920,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T10:16:14.794 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:17.889532+0000", 2024-09-18T10:16:14.794 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 56.871447623000002, 2024-09-18T10:16:14.794 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.32688004999999998, 2024-09-18T10:16:14.794 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:14.794 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "started", 2024-09-18T10:16:14.794 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:14.794 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:16:14.794 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:14.794 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 496 2024-09-18T10:16:14.794 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.795 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:14.795 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.795 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:14.795 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889532+0000", 2024-09-18T10:16:14.795 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:14.795 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.795 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.795 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:14.795 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889532+0000", 2024-09-18T10:16:14.795 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:14.795 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.795 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.795 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:14.795 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889529+0000", 2024-09-18T10:16:14.796 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999971 2024-09-18T10:16:14.796 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.796 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.796 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:14.796 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889546+0000", 2024-09-18T10:16:14.796 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.6690000000000001e-05 2024-09-18T10:16:14.796 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.796 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.796 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:14.796 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889580+0000", 2024-09-18T10:16:14.796 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.4638999999999998e-05 2024-09-18T10:16:14.796 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.796 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.796 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:14.797 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889596+0000", 2024-09-18T10:16:14.797 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.5571999999999999e-05 2024-09-18T10:16:14.797 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.797 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.797 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:14.797 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.989817+0000", 2024-09-18T10:16:14.797 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100221713 2024-09-18T10:16:14.797 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.797 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.797 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "op must wait for map", 2024-09-18T10:16:14.797 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.989836+0000", 2024-09-18T10:16:14.797 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8516e-05 2024-09-18T10:16:14.797 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.797 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.798 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:14.798 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.990420+0000", 2024-09-18T10:16:14.798 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000583918 2024-09-18T10:16:14.798 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.798 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.798 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:14.798 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.990693+0000", 2024-09-18T10:16:14.798 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00027301499999999999 2024-09-18T10:16:14.798 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.798 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.798 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:14.798 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:18.216412+0000", 2024-09-18T10:16:14.798 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.22571880699999999 2024-09-18T10:16:14.798 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:14.799 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:14.799 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:14.799 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.799 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.799 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:497 3.2s0 3:44cb9e27:::smithi18925896-5:head [read 0~81920,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T10:16:14.799 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:17.889627+0000", 2024-09-18T10:16:14.799 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 56.871352706000003, 2024-09-18T10:16:14.799 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.32738385800000003, 2024-09-18T10:16:14.799 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:14.799 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "started", 2024-09-18T10:16:14.799 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:14.799 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:16:14.799 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:14.799 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 497 2024-09-18T10:16:14.800 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.800 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:14.800 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.800 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:14.800 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889627+0000", 2024-09-18T10:16:14.800 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:14.800 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.800 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.800 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:14.800 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889627+0000", 2024-09-18T10:16:14.800 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:14.800 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.800 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.800 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:14.801 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889625+0000", 2024-09-18T10:16:14.801 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:16:14.801 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.801 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.801 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:14.801 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889633+0000", 2024-09-18T10:16:14.801 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.4170000000000003e-06 2024-09-18T10:16:14.801 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.801 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.801 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:14.801 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889664+0000", 2024-09-18T10:16:14.801 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0589000000000001e-05 2024-09-18T10:16:14.801 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.802 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.802 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:14.802 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889672+0000", 2024-09-18T10:16:14.802 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.3729999999999993e-06 2024-09-18T10:16:14.802 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.802 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.802 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:14.802 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.990017+0000", 2024-09-18T10:16:14.802 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100345608 2024-09-18T10:16:14.802 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.802 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.802 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:16:14.802 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.990046+0000", 2024-09-18T10:16:14.802 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8912999999999999e-05 2024-09-18T10:16:14.803 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.803 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.803 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:14.803 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.991459+0000", 2024-09-18T10:16:14.803 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0014122340000000001 2024-09-18T10:16:14.803 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.803 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.803 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:14.803 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.991631+0000", 2024-09-18T10:16:14.803 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000172029 2024-09-18T10:16:14.803 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.803 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.803 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:14.803 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:18.217010+0000", 2024-09-18T10:16:14.803 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.22537980499999999 2024-09-18T10:16:14.804 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:14.804 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:14.804 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:14.804 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.804 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.804 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:498 3.2s0 3:44cb9e27:::smithi18925896-5:head [read 0~81920,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T10:16:14.804 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:17.889709+0000", 2024-09-18T10:16:14.804 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 56.871270348000003, 2024-09-18T10:16:14.804 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.32775850600000001, 2024-09-18T10:16:14.804 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:14.804 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "started", 2024-09-18T10:16:14.804 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:14.804 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:16:14.804 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:14.805 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 498 2024-09-18T10:16:14.805 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.805 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:14.805 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.805 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:14.805 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889709+0000", 2024-09-18T10:16:14.805 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:14.805 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.805 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.805 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:14.805 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889709+0000", 2024-09-18T10:16:14.805 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:14.805 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.805 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.806 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:14.806 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889708+0000", 2024-09-18T10:16:14.806 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:16:14.806 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.806 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.806 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:14.806 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889716+0000", 2024-09-18T10:16:14.806 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.0989999999999999e-06 2024-09-18T10:16:14.806 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.806 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.806 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:14.806 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889732+0000", 2024-09-18T10:16:14.806 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.6093e-05 2024-09-18T10:16:14.806 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.807 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.807 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:14.807 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.889742+0000", 2024-09-18T10:16:14.807 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.4599999999999992e-06 2024-09-18T10:16:14.807 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.807 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.807 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:14.807 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.992175+0000", 2024-09-18T10:16:14.807 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10243377200000001 2024-09-18T10:16:14.807 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.807 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.807 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:14.807 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:17.992333+0000", 2024-09-18T10:16:14.807 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00015753499999999999 2024-09-18T10:16:14.808 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.808 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.808 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:14.808 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:18.217467+0000", 2024-09-18T10:16:14.808 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.22513459999999999 2024-09-18T10:16:14.808 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:14.808 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:14.808 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:14.808 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.808 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.808 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:516 3.2s0 3:44cb9e27:::smithi18925896-5:head [append 0~425984] snapc 1a=[1a,19,18,16,12,11,10,f,e] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-18T10:16:14.808 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:19.430973+0000", 2024-09-18T10:16:14.809 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 55.330005958999998, 2024-09-18T10:16:14.809 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.59812562800000002, 2024-09-18T10:16:14.809 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:14.809 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:14.809 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:14.809 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:16:14.809 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:14.809 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 516 2024-09-18T10:16:14.809 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.809 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:14.809 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.810 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:14.810 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.430973+0000", 2024-09-18T10:16:14.810 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:14.810 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.810 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.810 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:14.810 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.430973+0000", 2024-09-18T10:16:14.810 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:14.810 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.810 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.810 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:14.810 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.430970+0000", 2024-09-18T10:16:14.810 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999967 2024-09-18T10:16:14.810 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.811 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.811 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:14.811 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.431995+0000", 2024-09-18T10:16:14.811 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001024896 2024-09-18T10:16:14.811 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.811 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.811 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:14.811 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.432030+0000", 2024-09-18T10:16:14.811 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5241999999999999e-05 2024-09-18T10:16:14.811 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.811 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.811 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:14.811 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.432044+0000", 2024-09-18T10:16:14.811 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.411e-05 2024-09-18T10:16:14.812 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.812 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.812 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:14.812 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.432226+0000", 2024-09-18T10:16:14.812 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000181841 2024-09-18T10:16:14.812 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.812 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.812 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:14.812 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.432421+0000", 2024-09-18T10:16:14.812 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000194527 2024-09-18T10:16:14.812 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.812 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.812 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:14.812 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.434150+0000", 2024-09-18T10:16:14.813 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001729742 2024-09-18T10:16:14.813 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.813 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.813 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:14.813 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.436513+0000", 2024-09-18T10:16:14.813 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0023622299999999999 2024-09-18T10:16:14.813 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.813 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.813 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:14.813 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:20.029005+0000", 2024-09-18T10:16:14.813 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.59249230100000005 2024-09-18T10:16:14.813 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.813 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.813 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:14.814 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:20.029099+0000", 2024-09-18T10:16:14.814 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.4042000000000005e-05 2024-09-18T10:16:14.814 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:14.814 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:14.814 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:14.814 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.814 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.814 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:517 3.2s0 3:44cb9e27:::smithi18925896-5:head [append 0~786432] snapc 1a=[1a,19,18,16,12,11,10,f,e] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-18T10:16:14.814 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:19.439831+0000", 2024-09-18T10:16:14.814 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 55.32114842, 2024-09-18T10:16:14.814 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.58958403800000003, 2024-09-18T10:16:14.814 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:14.814 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:14.814 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:14.815 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:16:14.815 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:14.815 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 517 2024-09-18T10:16:14.815 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.815 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:14.815 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.815 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:14.815 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.439831+0000", 2024-09-18T10:16:14.815 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:14.815 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.815 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.815 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:14.815 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.439831+0000", 2024-09-18T10:16:14.815 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:14.815 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.816 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.816 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:14.816 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.439828+0000", 2024-09-18T10:16:14.816 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999976 2024-09-18T10:16:14.816 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.816 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.816 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:14.816 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.441109+0000", 2024-09-18T10:16:14.816 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001280988 2024-09-18T10:16:14.816 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.816 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.816 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:14.816 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.441168+0000", 2024-09-18T10:16:14.817 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.8869000000000002e-05 2024-09-18T10:16:14.817 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.817 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.817 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:14.817 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.541302+0000", 2024-09-18T10:16:14.817 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100134055 2024-09-18T10:16:14.817 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.817 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.817 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:14.817 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.637962+0000", 2024-09-18T10:16:14.817 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.096659682999999996 2024-09-18T10:16:14.817 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.817 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.817 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:14.817 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.638164+0000", 2024-09-18T10:16:14.818 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00020226700000000001 2024-09-18T10:16:14.818 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.818 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.818 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:14.818 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.639933+0000", 2024-09-18T10:16:14.818 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001768501 2024-09-18T10:16:14.818 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.818 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.818 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:14.818 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.646572+0000", 2024-09-18T10:16:14.818 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.006639069 2024-09-18T10:16:14.818 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.818 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.818 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:14.819 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:20.029341+0000", 2024-09-18T10:16:14.819 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.38276939599999998 2024-09-18T10:16:14.819 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.819 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.819 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:14.819 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:20.029415+0000", 2024-09-18T10:16:14.819 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.3726000000000005e-05 2024-09-18T10:16:14.819 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:14.819 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:14.819 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:14.819 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.819 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.819 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:518 3.2s0 3:44cb9e27:::smithi18925896-5:head [append 0~376832] snapc 1a=[1a,19,18,16,12,11,10,f,e] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-18T10:16:14.819 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:19.541352+0000", 2024-09-18T10:16:14.819 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 55.219627381999999, 2024-09-18T10:16:14.820 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.48828847600000003, 2024-09-18T10:16:14.820 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:14.820 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:14.820 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:14.820 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:16:14.820 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:14.820 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 518 2024-09-18T10:16:14.820 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.820 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:14.820 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.820 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:14.820 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.541352+0000", 2024-09-18T10:16:14.820 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:14.820 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.821 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.821 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:14.821 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.541352+0000", 2024-09-18T10:16:14.821 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:14.821 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.821 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.821 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:14.821 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.541350+0000", 2024-09-18T10:16:14.821 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:16:14.821 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.821 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.821 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:14.821 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.541834+0000", 2024-09-18T10:16:14.821 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00048402500000000002 2024-09-18T10:16:14.822 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.822 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.822 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:14.822 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.541861+0000", 2024-09-18T10:16:14.822 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.7333000000000001e-05 2024-09-18T10:16:14.822 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.822 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.822 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:14.822 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.541875+0000", 2024-09-18T10:16:14.822 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4025999999999999e-05 2024-09-18T10:16:14.822 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.822 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.822 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:14.822 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.641684+0000", 2024-09-18T10:16:14.823 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.099808872000000007 2024-09-18T10:16:14.823 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.823 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.823 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:14.823 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.641873+0000", 2024-09-18T10:16:14.823 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00018933200000000001 2024-09-18T10:16:14.823 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.823 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.823 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:14.823 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.643401+0000", 2024-09-18T10:16:14.823 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001527104 2024-09-18T10:16:14.823 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.823 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.823 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:14.823 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.646854+0000", 2024-09-18T10:16:14.824 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0034532069999999998 2024-09-18T10:16:14.824 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.824 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.824 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:14.824 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:20.029569+0000", 2024-09-18T10:16:14.824 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.38271509599999998 2024-09-18T10:16:14.824 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.824 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.824 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:14.824 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:20.029640+0000", 2024-09-18T10:16:14.824 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.1594000000000006e-05 2024-09-18T10:16:14.824 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:14.824 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:14.825 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:14.825 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.825 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.825 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:519 3.2s0 3:44cb9e27:::smithi18925896-5:head [setxattr _header (56) in=7b] snapc 1a=[1a,19,18,16,12,11,10,f,e] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-18T10:16:14.825 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:19.541886+0000", 2024-09-18T10:16:14.825 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 55.219093694999998, 2024-09-18T10:16:14.825 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.488165768, 2024-09-18T10:16:14.825 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:14.825 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:14.825 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:14.825 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:16:14.825 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:14.825 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 519 2024-09-18T10:16:14.825 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.826 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:14.826 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.826 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:14.826 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.541886+0000", 2024-09-18T10:16:14.826 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:14.826 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.826 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.826 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:14.826 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.541886+0000", 2024-09-18T10:16:14.826 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:14.826 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.826 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.826 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:14.826 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.541884+0000", 2024-09-18T10:16:14.827 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:16:14.827 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.827 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.827 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:14.827 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.541890+0000", 2024-09-18T10:16:14.827 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.0589999999999999e-06 2024-09-18T10:16:14.827 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.827 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.827 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:14.827 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.541899+0000", 2024-09-18T10:16:14.827 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.02e-06 2024-09-18T10:16:14.827 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.827 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.827 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:14.827 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.541905+0000", 2024-09-18T10:16:14.828 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.9399999999999999e-06 2024-09-18T10:16:14.828 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.828 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.828 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:14.828 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.644561+0000", 2024-09-18T10:16:14.828 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.102655408 2024-09-18T10:16:14.828 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.828 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.828 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:14.828 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.644869+0000", 2024-09-18T10:16:14.828 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00030766700000000002 2024-09-18T10:16:14.828 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.828 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.829 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:14.829 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.645745+0000", 2024-09-18T10:16:14.829 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000876314 2024-09-18T10:16:14.829 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.829 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.829 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:14.829 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:19.647092+0000", 2024-09-18T10:16:14.829 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0013469269999999999 2024-09-18T10:16:14.829 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.829 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.829 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:14.829 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:20.029901+0000", 2024-09-18T10:16:14.829 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.38280895199999998 2024-09-18T10:16:14.829 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.830 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.830 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:14.830 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:20.030051+0000", 2024-09-18T10:16:14.830 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00015067699999999999 2024-09-18T10:16:14.830 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:14.830 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:14.830 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:14.830 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.830 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.830 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:573 3.2s0 3:46b2871c:::smithi18925896-34:head [read 0~557056,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-18T10:16:14.830 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:24.942342+0000", 2024-09-18T10:16:14.830 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 49.818636980999997, 2024-09-18T10:16:14.830 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.36312128700000001, 2024-09-18T10:16:14.830 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:14.830 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "started", 2024-09-18T10:16:14.831 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:14.831 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:16:14.831 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:14.831 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 573 2024-09-18T10:16:14.831 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.831 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:14.831 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.831 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:14.831 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.942342+0000", 2024-09-18T10:16:14.831 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:14.831 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.831 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.831 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:14.832 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.942342+0000", 2024-09-18T10:16:14.832 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:14.832 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.832 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.832 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:14.832 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.942341+0000", 2024-09-18T10:16:14.832 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:16:14.832 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.832 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.832 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:14.832 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.942363+0000", 2024-09-18T10:16:14.832 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.1406000000000001e-05 2024-09-18T10:16:14.832 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.832 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.833 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:14.833 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.942374+0000", 2024-09-18T10:16:14.833 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1466e-05 2024-09-18T10:16:14.833 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.833 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.833 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:14.833 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.942381+0000", 2024-09-18T10:16:14.833 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.9530000000000002e-06 2024-09-18T10:16:14.833 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.833 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.833 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:14.833 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.942654+0000", 2024-09-18T10:16:14.833 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000272692 2024-09-18T10:16:14.833 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.833 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.834 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:16:14.834 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.942672+0000", 2024-09-18T10:16:14.834 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7643e-05 2024-09-18T10:16:14.834 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.834 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.834 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:14.834 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:25.101730+0000", 2024-09-18T10:16:14.834 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.15905815800000001 2024-09-18T10:16:14.834 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.834 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.834 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:14.834 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:25.101881+0000", 2024-09-18T10:16:14.834 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000151299 2024-09-18T10:16:14.835 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.835 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.835 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:14.835 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:25.305464+0000", 2024-09-18T10:16:14.835 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.203582656 2024-09-18T10:16:14.835 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:14.835 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:14.835 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:14.835 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.835 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.835 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:574 3.2s0 3:46b2871c:::smithi18925896-34:head [read 0~557056,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-18T10:16:14.835 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:24.942392+0000", 2024-09-18T10:16:14.835 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 49.818587184999998, 2024-09-18T10:16:14.835 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.36415850300000002, 2024-09-18T10:16:14.835 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:14.836 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "started", 2024-09-18T10:16:14.836 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:14.836 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:16:14.836 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:14.836 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 574 2024-09-18T10:16:14.836 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.836 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:14.836 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.836 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:14.836 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.942392+0000", 2024-09-18T10:16:14.836 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:14.836 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.836 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.837 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:14.837 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.942392+0000", 2024-09-18T10:16:14.837 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:14.837 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.837 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.837 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:14.837 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.942391+0000", 2024-09-18T10:16:14.837 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.999999 2024-09-18T10:16:14.837 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.837 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.837 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:14.837 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.942396+0000", 2024-09-18T10:16:14.837 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.6759999999999998e-06 2024-09-18T10:16:14.837 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.838 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.838 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:14.838 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.942407+0000", 2024-09-18T10:16:14.838 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0686999999999999e-05 2024-09-18T10:16:14.838 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.838 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.838 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:14.838 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.942413+0000", 2024-09-18T10:16:14.838 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.1659999999999999e-06 2024-09-18T10:16:14.838 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.838 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.838 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:14.838 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.942750+0000", 2024-09-18T10:16:14.839 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00033731700000000001 2024-09-18T10:16:14.839 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.839 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.839 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:16:14.839 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:24.942776+0000", 2024-09-18T10:16:14.839 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6342999999999998e-05 2024-09-18T10:16:14.839 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.839 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.839 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:14.839 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:25.102396+0000", 2024-09-18T10:16:14.839 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.15961924199999999 2024-09-18T10:16:14.839 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.839 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.839 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:14.839 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:25.102659+0000", 2024-09-18T10:16:14.840 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00026355500000000001 2024-09-18T10:16:14.840 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.840 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.840 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:14.840 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:25.306551+0000", 2024-09-18T10:16:14.840 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20389143700000001 2024-09-18T10:16:14.840 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:14.840 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:14.840 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:14.840 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.840 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.840 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "MOSDECSubOpWriteReply(3.2s0 89/40 ECSubWriteReply(tid=135, last_complete=89'96, committed=1, applied=1))", 2024-09-18T10:16:14.840 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:45.106385+0000", 2024-09-18T10:16:14.840 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 29.654594411000001, 2024-09-18T10:16:14.841 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.35616392200000002, 2024-09-18T10:16:14.841 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:14.841 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:16:14.841 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:14.841 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.841 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:14.841 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:45.106385+0000", 2024-09-18T10:16:14.841 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:14.841 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.841 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.841 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:14.841 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:45.106385+0000", 2024-09-18T10:16:14.841 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:14.841 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.842 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.842 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:14.842 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:45.106384+0000", 2024-09-18T10:16:14.842 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999995 2024-09-18T10:16:14.842 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.842 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.842 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:14.842 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:45.106388+0000", 2024-09-18T10:16:14.842 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.6100000000000002e-06 2024-09-18T10:16:14.842 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.842 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.842 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:14.842 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:45.106395+0000", 2024-09-18T10:16:14.842 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.8399999999999997e-06 2024-09-18T10:16:14.843 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.843 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.843 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:14.843 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:45.106399+0000", 2024-09-18T10:16:14.843 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.1899999999999997e-06 2024-09-18T10:16:14.843 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.843 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.843 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:14.843 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:45.461488+0000", 2024-09-18T10:16:14.843 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.35508847199999999 2024-09-18T10:16:14.843 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.843 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.843 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:14.843 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:45.462549+0000", 2024-09-18T10:16:14.844 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0010612760000000001 2024-09-18T10:16:14.844 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:14.844 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:14.844 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:14.844 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.844 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.844 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:891 3.2s0 3:427b2c0b:::smithi18925896-42:head [append 0~0] snapc 34=[34,33,30,2f,28,20,1f,1a,11] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-18T10:16:14.844 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:46.655855+0000", 2024-09-18T10:16:14.844 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 28.105124683, 2024-09-18T10:16:14.844 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.45749477399999999, 2024-09-18T10:16:14.844 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:14.844 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:14.844 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:14.844 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:16:14.845 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:14.845 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 891 2024-09-18T10:16:14.845 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.845 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:14.845 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.845 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:14.845 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.655855+0000", 2024-09-18T10:16:14.845 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:14.845 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.845 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.845 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:14.845 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.655855+0000", 2024-09-18T10:16:14.845 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:14.845 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.846 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.846 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:14.846 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.655852+0000", 2024-09-18T10:16:14.846 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999971 2024-09-18T10:16:14.846 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.846 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.846 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:14.846 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.655866+0000", 2024-09-18T10:16:14.846 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4842e-05 2024-09-18T10:16:14.846 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.846 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.846 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:14.846 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.655906+0000", 2024-09-18T10:16:14.846 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.9465000000000002e-05 2024-09-18T10:16:14.846 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.847 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.847 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:14.847 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.756059+0000", 2024-09-18T10:16:14.847 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100153544 2024-09-18T10:16:14.847 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.847 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.847 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:14.847 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.804067+0000", 2024-09-18T10:16:14.847 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.048007288000000002 2024-09-18T10:16:14.847 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.847 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.847 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:14.847 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.804374+0000", 2024-09-18T10:16:14.847 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00030716700000000001 2024-09-18T10:16:14.847 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.848 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.848 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:14.848 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.807008+0000", 2024-09-18T10:16:14.848 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.002633799 2024-09-18T10:16:14.848 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.848 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.848 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:14.848 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.911478+0000", 2024-09-18T10:16:14.848 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.104469986 2024-09-18T10:16:14.848 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.848 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.848 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:14.848 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:47.113216+0000", 2024-09-18T10:16:14.848 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.201737789 2024-09-18T10:16:14.849 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.849 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.849 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:14.849 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:47.113349+0000", 2024-09-18T10:16:14.849 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00013389700000000001 2024-09-18T10:16:14.849 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:14.849 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:14.849 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:14.849 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.849 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.849 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:892 3.2s0 3:427b2c0b:::smithi18925896-42:head [rollback 28] snapc 34=[34,33,30,2f,28,20,1f,1a,11] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-18T10:16:14.849 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:46.756090+0000", 2024-09-18T10:16:14.849 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 28.004889602999999, 2024-09-18T10:16:14.849 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.35768046799999997, 2024-09-18T10:16:14.850 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:14.850 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:14.850 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:14.850 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:16:14.850 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:14.850 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 892 2024-09-18T10:16:14.850 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.850 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:14.850 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.850 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:14.850 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.756090+0000", 2024-09-18T10:16:14.850 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:14.850 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.850 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.851 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:14.851 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.756090+0000", 2024-09-18T10:16:14.851 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:14.851 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.851 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.851 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:14.851 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.756088+0000", 2024-09-18T10:16:14.851 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:16:14.851 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.851 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.851 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:14.851 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.756099+0000", 2024-09-18T10:16:14.851 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0879999999999999e-05 2024-09-18T10:16:14.851 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.852 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.852 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:14.852 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.756116+0000", 2024-09-18T10:16:14.852 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7849000000000001e-05 2024-09-18T10:16:14.852 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.852 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.852 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:14.852 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.756136+0000", 2024-09-18T10:16:14.852 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.9720000000000001e-05 2024-09-18T10:16:14.852 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.852 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.852 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:14.852 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.808713+0000", 2024-09-18T10:16:14.852 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.052577089 2024-09-18T10:16:14.853 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.853 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.853 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:14.853 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.808963+0000", 2024-09-18T10:16:14.853 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000249824 2024-09-18T10:16:14.853 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.853 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.853 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:14.853 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.810124+0000", 2024-09-18T10:16:14.853 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0011610100000000001 2024-09-18T10:16:14.853 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.853 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.853 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:14.853 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:47.011739+0000", 2024-09-18T10:16:14.854 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.20161468900000001 2024-09-18T10:16:14.854 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.854 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.854 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:14.854 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:47.113617+0000", 2024-09-18T10:16:14.854 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.10187808700000001 2024-09-18T10:16:14.854 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.854 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.854 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:14.854 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:47.113770+0000", 2024-09-18T10:16:14.854 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00015334300000000001 2024-09-18T10:16:14.854 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:14.854 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:14.854 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:14.855 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.855 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.855 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:893 3.2s0 3:427b2c0b:::smithi18925896-42:head [append 0~0] snapc 34=[34,33,30,2f,28,20,1f,1a,11] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-18T10:16:14.855 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:46.756179+0000", 2024-09-18T10:16:14.855 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 28.004800695, 2024-09-18T10:16:14.855 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.66260898499999998, 2024-09-18T10:16:14.855 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:14.855 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:14.855 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:14.855 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:16:14.855 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:14.855 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 893 2024-09-18T10:16:14.855 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.855 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:14.856 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.856 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:14.856 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.756179+0000", 2024-09-18T10:16:14.856 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:14.856 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.856 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.856 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:14.856 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.756179+0000", 2024-09-18T10:16:14.856 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:14.856 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.856 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.856 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:14.856 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.756177+0000", 2024-09-18T10:16:14.856 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999986 2024-09-18T10:16:14.857 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.857 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.857 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:14.857 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.756185+0000", 2024-09-18T10:16:14.857 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.32e-06 2024-09-18T10:16:14.857 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.857 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.857 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:14.857 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.756214+0000", 2024-09-18T10:16:14.857 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8992999999999998e-05 2024-09-18T10:16:14.857 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.857 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.857 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:14.857 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.856357+0000", 2024-09-18T10:16:14.857 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100142493 2024-09-18T10:16:14.858 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.858 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.858 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:14.858 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:47.114791+0000", 2024-09-18T10:16:14.858 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.25843391799999998 2024-09-18T10:16:14.858 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.858 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.858 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:14.858 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:47.114948+0000", 2024-09-18T10:16:14.858 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00015682700000000001 2024-09-18T10:16:14.858 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.858 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.858 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_started", 2024-09-18T10:16:14.858 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:47.116043+0000", 2024-09-18T10:16:14.859 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0010952309999999999 2024-09-18T10:16:14.859 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.859 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.859 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "sub_op_committed", 2024-09-18T10:16:14.859 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:47.117006+0000", 2024-09-18T10:16:14.859 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00096323899999999996 2024-09-18T10:16:14.859 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.859 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.859 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-09-18T10:16:14.859 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:47.418568+0000", 2024-09-18T10:16:14.859 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.30156222300000002 2024-09-18T10:16:14.859 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.859 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.859 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:14.860 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:47.418788+0000", 2024-09-18T10:16:14.860 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00021923799999999999 2024-09-18T10:16:14.860 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:14.860 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:14.860 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:14.860 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.860 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.860 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(osd.6.22:5 3.2s0 3:47202842:::smithi18925896-22:head [copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-18T10:16:14.860 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:46.756797+0000", 2024-09-18T10:16:14.860 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 28.004182181000001, 2024-09-18T10:16:14.860 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.35575880999999998, 2024-09-18T10:16:14.860 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:14.860 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-09-18T10:16:14.860 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:14.861 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.861 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:14.861 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.756797+0000", 2024-09-18T10:16:14.861 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:14.861 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.861 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.861 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:14.861 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.756797+0000", 2024-09-18T10:16:14.861 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:14.861 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.861 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.861 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:14.861 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.756795+0000", 2024-09-18T10:16:14.861 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999981 2024-09-18T10:16:14.861 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.862 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.862 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:14.862 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.756807+0000", 2024-09-18T10:16:14.862 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1970000000000001e-05 2024-09-18T10:16:14.862 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.862 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.862 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:14.862 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.756837+0000", 2024-09-18T10:16:14.862 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9706999999999999e-05 2024-09-18T10:16:14.862 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.862 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.862 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:14.862 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:46.756849+0000", 2024-09-18T10:16:14.862 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1904e-05 2024-09-18T10:16:14.863 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.863 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.863 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:14.863 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:47.112101+0000", 2024-09-18T10:16:14.863 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.35525225900000001 2024-09-18T10:16:14.863 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.863 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.863 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:14.863 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:47.112556+0000", 2024-09-18T10:16:14.863 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00045505300000000002 2024-09-18T10:16:14.863 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:14.863 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:14.863 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:14.863 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.864 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.864 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4632.0:953 3.2s0 3:46b2871c:::smithi18925896-34:head [read 0~557056,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-18T10:16:14.864 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-09-18T10:15:52.128760+0000", 2024-09-18T10:16:14.864 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 22.632219033999998, 2024-09-18T10:16:14.864 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.40564792700000002, 2024-09-18T10:16:14.864 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-09-18T10:16:14.864 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "started", 2024-09-18T10:16:14.864 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-09-18T10:16:14.864 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4632", 2024-09-18T10:16:14.864 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:14.864 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 953 2024-09-18T10:16:14.864 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.864 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-09-18T10:16:14.864 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.865 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-09-18T10:16:14.865 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:52.128760+0000", 2024-09-18T10:16:14.865 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:14.865 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.865 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.865 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-09-18T10:16:14.865 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:52.128760+0000", 2024-09-18T10:16:14.865 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-09-18T10:16:14.865 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.865 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.865 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-09-18T10:16:14.865 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:52.128757+0000", 2024-09-18T10:16:14.865 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4294967295.9999967 2024-09-18T10:16:14.865 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.865 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.866 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-09-18T10:16:14.866 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:52.128774+0000", 2024-09-18T10:16:14.866 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.6912e-05 2024-09-18T10:16:14.866 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.866 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.866 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-09-18T10:16:14.866 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:52.128809+0000", 2024-09-18T10:16:14.866 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.4913000000000003e-05 2024-09-18T10:16:14.866 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.866 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.866 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-09-18T10:16:14.866 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:52.228946+0000", 2024-09-18T10:16:14.866 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.100137698 2024-09-18T10:16:14.866 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.867 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.867 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-09-18T10:16:14.867 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:52.229076+0000", 2024-09-18T10:16:14.867 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00012982 2024-09-18T10:16:14.867 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.867 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.867 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-09-18T10:16:14.867 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:52.229281+0000", 2024-09-18T10:16:14.867 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000204592 2024-09-18T10:16:14.867 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-18T10:16:14.867 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-09-18T10:16:14.867 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-09-18T10:16:14.867 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-09-18T10:15:52.534408+0000", 2024-09-18T10:16:14.867 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.30512741500000001 2024-09-18T10:16:14.867 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:14.868 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:14.868 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:14.868 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-18T10:16:14.868 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-18T10:16:14.868 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-09-18T10:16:14.869 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T10:16:14.982 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-18T10:16:14.982 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-18T10:16:14.982 INFO:teuthology.orchestra.run.smithi189.stdout: "num_ops": 0 2024-09-18T10:16:14.982 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-18T10:16:14.991 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T10:16:15.095 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-18T10:16:15.095 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-18T10:16:15.095 INFO:teuthology.orchestra.run.smithi189.stdout: "complaint_time": 30, 2024-09-18T10:16:15.095 INFO:teuthology.orchestra.run.smithi189.stdout: "num_blocked_ops": 0 2024-09-18T10:16:15.096 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-18T10:16:15.104 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T10:16:15.214 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-18T10:16:15.214 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 20, 2024-09-18T10:16:15.214 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 600, 2024-09-18T10:16:15.214 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [ 2024-09-18T10:16:15.214 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.214 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 36/32 ECSubWriteReply(tid=5, last_complete=36'4, committed=1, applied=1))", 2024-09-18T10:16:15.214 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:50.303102+0000", 2024-09-18T10:16:15.214 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 84.907603613000006, 2024-09-18T10:16:15.214 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20304619600000001, 2024-09-18T10:16:15.215 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:15.215 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:16:15.215 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:15.215 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.215 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:15.215 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.303102+0000", 2024-09-18T10:16:15.215 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:15.215 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.215 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.215 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:15.215 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.303102+0000", 2024-09-18T10:16:15.216 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:15.216 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.216 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.216 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:15.216 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.303100+0000", 2024-09-18T10:16:15.216 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:16:15.216 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.216 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.216 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:15.216 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.303114+0000", 2024-09-18T10:16:15.216 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3866000000000001e-05 2024-09-18T10:16:15.216 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.217 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.217 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:15.217 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.303149+0000", 2024-09-18T10:16:15.217 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.5249000000000001e-05 2024-09-18T10:16:15.217 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.217 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.217 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:15.217 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.303174+0000", 2024-09-18T10:16:15.217 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4785000000000001e-05 2024-09-18T10:16:15.217 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.217 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.217 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:15.217 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.506129+0000", 2024-09-18T10:16:15.217 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20295469199999999 2024-09-18T10:16:15.218 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.218 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.218 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:15.218 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.506148+0000", 2024-09-18T10:16:15.218 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9720000000000001e-05 2024-09-18T10:16:15.218 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:15.218 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:15.218 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:15.218 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.218 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.218 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 36/32 ECSubWriteReply(tid=5, last_complete=36'4, committed=1, applied=1))", 2024-09-18T10:16:15.218 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:50.305623+0000", 2024-09-18T10:16:15.218 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 84.905082641000007, 2024-09-18T10:16:15.219 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40085840299999997, 2024-09-18T10:16:15.219 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:15.219 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:16:15.219 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:15.219 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.219 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:15.219 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.305623+0000", 2024-09-18T10:16:15.219 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:15.219 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.219 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.219 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:15.219 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.305623+0000", 2024-09-18T10:16:15.219 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:15.219 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.220 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.220 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:15.220 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.305621+0000", 2024-09-18T10:16:15.220 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999976 2024-09-18T10:16:15.220 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.220 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.220 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:15.220 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.305635+0000", 2024-09-18T10:16:15.220 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4283e-05 2024-09-18T10:16:15.220 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.220 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.220 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:15.220 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.305663+0000", 2024-09-18T10:16:15.221 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7475999999999999e-05 2024-09-18T10:16:15.221 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.221 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.221 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:15.221 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.305680+0000", 2024-09-18T10:16:15.221 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7609e-05 2024-09-18T10:16:15.221 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.221 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.221 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:15.221 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.706451+0000", 2024-09-18T10:16:15.221 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40077027399999998 2024-09-18T10:16:15.221 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.221 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.222 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:15.222 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.706481+0000", 2024-09-18T10:16:15.222 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0935000000000001e-05 2024-09-18T10:16:15.222 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:15.222 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:15.222 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:15.222 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.222 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.222 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 36/32 ECSubWriteReply(tid=5, last_complete=36'4, committed=1, applied=1))", 2024-09-18T10:16:15.222 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:50.305688+0000", 2024-09-18T10:16:15.222 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 84.905017276999999, 2024-09-18T10:16:15.222 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40093562900000002, 2024-09-18T10:16:15.222 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:15.222 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:16:15.223 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:15.223 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.223 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:15.223 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.305688+0000", 2024-09-18T10:16:15.223 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:15.223 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.223 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.223 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:15.223 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.305688+0000", 2024-09-18T10:16:15.223 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:15.223 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.223 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.223 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:15.224 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.305687+0000", 2024-09-18T10:16:15.224 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:16:15.224 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.224 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.224 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:15.224 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.305698+0000", 2024-09-18T10:16:15.224 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1763000000000001e-05 2024-09-18T10:16:15.224 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.224 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.224 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:15.224 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.305729+0000", 2024-09-18T10:16:15.224 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0502e-05 2024-09-18T10:16:15.224 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.224 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.225 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:15.225 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.305741+0000", 2024-09-18T10:16:15.225 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2363e-05 2024-09-18T10:16:15.225 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.225 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.225 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:15.225 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.706585+0000", 2024-09-18T10:16:15.225 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.400843265 2024-09-18T10:16:15.225 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.225 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.225 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:15.225 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.706624+0000", 2024-09-18T10:16:15.225 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9459000000000003e-05 2024-09-18T10:16:15.226 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:15.226 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:15.226 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:15.226 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.226 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.226 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 36/32 ECSubWriteReply(tid=4, last_complete=36'3, committed=1, applied=1))", 2024-09-18T10:16:15.226 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:50.404544+0000", 2024-09-18T10:16:15.226 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 84.806161762000002, 2024-09-18T10:16:15.226 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.30239659400000002, 2024-09-18T10:16:15.226 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:15.226 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:16:15.226 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:15.226 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.226 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:15.227 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.404544+0000", 2024-09-18T10:16:15.227 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:15.227 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.227 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.227 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:15.227 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.404544+0000", 2024-09-18T10:16:15.227 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:15.227 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.227 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.227 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:15.227 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.404542+0000", 2024-09-18T10:16:15.227 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:16:15.227 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.227 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.228 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:15.228 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.404555+0000", 2024-09-18T10:16:15.228 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3096e-05 2024-09-18T10:16:15.228 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.228 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.228 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:15.228 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.404576+0000", 2024-09-18T10:16:15.228 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1165000000000001e-05 2024-09-18T10:16:15.228 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.228 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.228 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:15.228 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.404587+0000", 2024-09-18T10:16:15.228 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0549e-05 2024-09-18T10:16:15.229 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.229 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.229 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:15.229 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.706657+0000", 2024-09-18T10:16:15.229 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.30206971999999999 2024-09-18T10:16:15.229 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.229 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.229 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:15.229 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.706941+0000", 2024-09-18T10:16:15.229 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00028382100000000002 2024-09-18T10:16:15.229 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:15.229 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:15.229 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:15.230 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.230 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.230 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 36/32 ECSubWriteReply(tid=5, last_complete=36'4, committed=1, applied=1))", 2024-09-18T10:16:15.230 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:50.406036+0000", 2024-09-18T10:16:15.230 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 84.804669544999996, 2024-09-18T10:16:15.230 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.30215917399999997, 2024-09-18T10:16:15.230 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:15.230 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:16:15.230 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:15.230 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.230 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:15.230 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.406036+0000", 2024-09-18T10:16:15.230 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:15.231 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.231 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.231 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:15.231 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.406036+0000", 2024-09-18T10:16:15.231 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:15.231 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.231 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.231 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:15.231 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.406035+0000", 2024-09-18T10:16:15.231 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:16:15.231 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.231 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.231 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:15.231 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.406046+0000", 2024-09-18T10:16:15.232 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1704e-05 2024-09-18T10:16:15.232 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.232 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.232 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:15.232 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.406065+0000", 2024-09-18T10:16:15.232 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.8448999999999999e-05 2024-09-18T10:16:15.232 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.232 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.232 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:15.232 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.406076+0000", 2024-09-18T10:16:15.232 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1286e-05 2024-09-18T10:16:15.232 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.232 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.233 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:15.233 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.707005+0000", 2024-09-18T10:16:15.233 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.30092866899999998 2024-09-18T10:16:15.233 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.233 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.233 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:15.233 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.708195+0000", 2024-09-18T10:16:15.233 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001190546 2024-09-18T10:16:15.233 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:15.233 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:15.234 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:15.234 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.234 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.234 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.cs3 36/32 ECSubWrite(tid=14, reqid=client.4632.0:92, at_version=0'0, trim_to=0'0, roll_forward_to=36'6))", 2024-09-18T10:16:15.234 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:50.504842+0000", 2024-09-18T10:16:15.234 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 84.705864142999999, 2024-09-18T10:16:15.234 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.202186116, 2024-09-18T10:16:15.234 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:15.234 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:16:15.234 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:15.234 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.234 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:15.234 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.504842+0000", 2024-09-18T10:16:15.235 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:15.235 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.235 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.235 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:15.235 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.504842+0000", 2024-09-18T10:16:15.235 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:15.235 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.235 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.235 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:15.235 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.504840+0000", 2024-09-18T10:16:15.235 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:16:15.235 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.235 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.236 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:15.236 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.504868+0000", 2024-09-18T10:16:15.236 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7809000000000001e-05 2024-09-18T10:16:15.236 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.236 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.236 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:15.236 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.504900+0000", 2024-09-18T10:16:15.236 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1921999999999997e-05 2024-09-18T10:16:15.236 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.236 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.236 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:15.236 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.605012+0000", 2024-09-18T10:16:15.236 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10011272 2024-09-18T10:16:15.236 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.237 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.237 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:15.237 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.605115+0000", 2024-09-18T10:16:15.237 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000103003 2024-09-18T10:16:15.237 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.237 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.237 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:16:15.237 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.605132+0000", 2024-09-18T10:16:15.237 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6359999999999999e-05 2024-09-18T10:16:15.237 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.237 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.237 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:16:15.237 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.706978+0000", 2024-09-18T10:16:15.238 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.101846672 2024-09-18T10:16:15.238 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.238 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.238 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:15.238 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.707028+0000", 2024-09-18T10:16:15.238 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9443000000000001e-05 2024-09-18T10:16:15.238 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:15.238 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:15.238 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:15.238 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.238 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.238 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:179 3.0s0 3:02f544f0:::smithi18925896-30:head [read 0~491520,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:16:15.238 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:52.680315+0000", 2024-09-18T10:16:15.239 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 82.530391014000003, 2024-09-18T10:16:15.239 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20435319499999999, 2024-09-18T10:16:15.239 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:15.239 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:16:15.239 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:16:15.239 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:16:15.239 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:15.239 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 179 2024-09-18T10:16:15.239 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.239 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:15.239 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.239 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:15.239 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.680315+0000", 2024-09-18T10:16:15.239 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:15.240 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.240 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.240 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:15.240 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.680315+0000", 2024-09-18T10:16:15.240 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:15.240 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.240 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.240 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:15.240 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.680313+0000", 2024-09-18T10:16:15.240 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:16:15.240 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.240 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.240 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:15.241 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.680322+0000", 2024-09-18T10:16:15.241 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.9299999999999992e-06 2024-09-18T10:16:15.241 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.241 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.241 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:15.241 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.680343+0000", 2024-09-18T10:16:15.241 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0556000000000001e-05 2024-09-18T10:16:15.241 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.241 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.241 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:15.241 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.680373+0000", 2024-09-18T10:16:15.241 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0592000000000001e-05 2024-09-18T10:16:15.241 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.242 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.242 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:15.242 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.681360+0000", 2024-09-18T10:16:15.242 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00098610400000000002 2024-09-18T10:16:15.242 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.242 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.242 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:16:15.242 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.681624+0000", 2024-09-18T10:16:15.242 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000263992 2024-09-18T10:16:15.242 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.242 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.242 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:15.242 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.884668+0000", 2024-09-18T10:16:15.242 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.203044321 2024-09-18T10:16:15.243 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:15.243 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:15.243 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:15.243 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.243 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.243 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:180 3.0s0 3:02f544f0:::smithi18925896-30:head [read 0~491520,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:16:15.243 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:52.680394+0000", 2024-09-18T10:16:15.243 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 82.530311936000004, 2024-09-18T10:16:15.243 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20540486599999999, 2024-09-18T10:16:15.243 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:15.243 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:16:15.243 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:16:15.243 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:16:15.243 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:15.244 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 180 2024-09-18T10:16:15.244 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.244 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:15.244 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.244 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:15.244 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.680394+0000", 2024-09-18T10:16:15.244 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:15.244 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.244 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.244 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:15.244 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.680394+0000", 2024-09-18T10:16:15.244 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:15.244 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.245 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.245 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:15.245 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.680393+0000", 2024-09-18T10:16:15.245 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:16:15.245 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.245 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.245 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:15.245 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.680401+0000", 2024-09-18T10:16:15.245 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.4169999999999992e-06 2024-09-18T10:16:15.245 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.245 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.245 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:15.245 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.680422+0000", 2024-09-18T10:16:15.246 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1186e-05 2024-09-18T10:16:15.246 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.246 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.246 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:15.246 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.680433+0000", 2024-09-18T10:16:15.246 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1103e-05 2024-09-18T10:16:15.246 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.246 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.246 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:15.246 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.783888+0000", 2024-09-18T10:16:15.246 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.103454746 2024-09-18T10:16:15.246 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.246 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.247 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:16:15.247 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.784067+0000", 2024-09-18T10:16:15.247 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00017865399999999999 2024-09-18T10:16:15.247 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.247 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.247 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:15.247 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.885799+0000", 2024-09-18T10:16:15.247 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10173201699999999 2024-09-18T10:16:15.247 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:15.247 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:15.247 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:15.247 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.247 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.247 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:191 3.0s0 3:0a83d6b3:::smithi18925896-20:2 [read 0~327680,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:16:15.248 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:52.694194+0000", 2024-09-18T10:16:15.248 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 82.516511496000007, 2024-09-18T10:16:15.248 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.29753027399999998, 2024-09-18T10:16:15.248 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:15.248 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:16:15.248 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:16:15.248 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:16:15.248 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:15.248 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 191 2024-09-18T10:16:15.248 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.248 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:15.248 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.248 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:15.249 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.694194+0000", 2024-09-18T10:16:15.249 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:15.249 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.249 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.249 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:15.249 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.694194+0000", 2024-09-18T10:16:15.249 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:15.249 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.249 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.249 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:15.249 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.694192+0000", 2024-09-18T10:16:15.249 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:16:15.249 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.249 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.250 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:15.250 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.694204+0000", 2024-09-18T10:16:15.250 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2233e-05 2024-09-18T10:16:15.250 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.250 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.250 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:15.250 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.694280+0000", 2024-09-18T10:16:15.250 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.5584e-05 2024-09-18T10:16:15.250 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.250 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.250 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:15.250 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.694292+0000", 2024-09-18T10:16:15.250 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2478e-05 2024-09-18T10:16:15.251 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.251 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.251 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:15.251 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.885869+0000", 2024-09-18T10:16:15.251 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.191576304 2024-09-18T10:16:15.251 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.251 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.251 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:16:15.251 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.886008+0000", 2024-09-18T10:16:15.251 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00013880300000000001 2024-09-18T10:16:15.251 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.251 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.251 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:15.251 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.991725+0000", 2024-09-18T10:16:15.252 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10571691 2024-09-18T10:16:15.252 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:15.252 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:15.252 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:15.252 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.252 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.252 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:192 3.0s0 3:0a83d6b3:::smithi18925896-20:2 [read 0~327680,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:16:15.252 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:52.694306+0000", 2024-09-18T10:16:15.252 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 82.516399546000002, 2024-09-18T10:16:15.252 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.41051474799999998, 2024-09-18T10:16:15.252 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:15.252 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:16:15.252 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:16:15.253 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:16:15.253 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:15.253 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 192 2024-09-18T10:16:15.253 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.253 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:15.253 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.253 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:15.253 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.694306+0000", 2024-09-18T10:16:15.253 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:15.253 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.253 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.253 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:15.253 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.694306+0000", 2024-09-18T10:16:15.253 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:15.254 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.254 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.254 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:15.254 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.694305+0000", 2024-09-18T10:16:15.254 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:16:15.254 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.254 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.254 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:15.254 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.694312+0000", 2024-09-18T10:16:15.254 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.5129999999999999e-06 2024-09-18T10:16:15.254 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.254 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.254 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:15.255 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.694328+0000", 2024-09-18T10:16:15.255 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5766000000000001e-05 2024-09-18T10:16:15.255 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.255 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.255 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:15.255 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.694349+0000", 2024-09-18T10:16:15.255 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0990000000000001e-05 2024-09-18T10:16:15.255 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.255 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.255 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:15.255 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.887405+0000", 2024-09-18T10:16:15.255 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.193055791 2024-09-18T10:16:15.255 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.255 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.256 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:16:15.256 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.887535+0000", 2024-09-18T10:16:15.256 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00013043000000000001 2024-09-18T10:16:15.256 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.256 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.256 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:15.256 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:53.104821+0000", 2024-09-18T10:16:15.256 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.217285486 2024-09-18T10:16:15.256 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:15.256 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:15.257 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:15.257 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.257 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.257 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:193 3.0s0 3:0a83d6b3:::smithi18925896-20:2 [read 0~327680,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:16:15.257 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:52.694359+0000", 2024-09-18T10:16:15.257 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 82.516346365000004, 2024-09-18T10:16:15.257 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.411282918, 2024-09-18T10:16:15.257 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:15.257 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:16:15.257 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:16:15.257 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:16:15.257 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:15.257 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 193 2024-09-18T10:16:15.258 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.258 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:15.258 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.258 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:15.258 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.694359+0000", 2024-09-18T10:16:15.258 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:15.258 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.258 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.258 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:15.258 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.694359+0000", 2024-09-18T10:16:15.258 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:15.258 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.258 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.258 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:15.259 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.694358+0000", 2024-09-18T10:16:15.259 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:16:15.259 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.259 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.259 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:15.259 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.694364+0000", 2024-09-18T10:16:15.259 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8089999999999998e-06 2024-09-18T10:16:15.259 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.259 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.259 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:15.260 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.694381+0000", 2024-09-18T10:16:15.260 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6949e-05 2024-09-18T10:16:15.260 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.260 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.260 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:15.260 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.694388+0000", 2024-09-18T10:16:15.260 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.8079999999999999e-06 2024-09-18T10:16:15.260 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.260 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.260 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:15.260 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.988184+0000", 2024-09-18T10:16:15.260 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.29379642900000003 2024-09-18T10:16:15.261 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.261 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.261 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:16:15.261 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.988399+0000", 2024-09-18T10:16:15.261 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000214537 2024-09-18T10:16:15.261 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.262 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.262 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:15.262 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:53.105642+0000", 2024-09-18T10:16:15.262 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.11724332699999999 2024-09-18T10:16:15.262 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:15.262 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:15.262 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:15.262 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.262 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.262 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.10s0 45/40 ECSubWriteReply(tid=18, last_complete=45'6, committed=1, applied=1))", 2024-09-18T10:16:15.262 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:01.742819+0000", 2024-09-18T10:16:15.262 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 73.467886762000006, 2024-09-18T10:16:15.262 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.30317059800000001, 2024-09-18T10:16:15.263 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:15.263 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:16:15.263 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:15.263 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.263 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:15.263 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.742819+0000", 2024-09-18T10:16:15.263 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:15.263 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.263 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.263 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:15.263 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.742819+0000", 2024-09-18T10:16:15.263 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:15.263 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.263 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.264 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:15.264 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.742817+0000", 2024-09-18T10:16:15.264 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:16:15.264 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.264 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.264 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:15.264 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.742832+0000", 2024-09-18T10:16:15.264 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.466e-05 2024-09-18T10:16:15.264 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.264 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.264 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:15.264 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.742866+0000", 2024-09-18T10:16:15.265 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4242000000000002e-05 2024-09-18T10:16:15.265 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.265 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.265 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:15.265 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.843033+0000", 2024-09-18T10:16:15.265 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100166479 2024-09-18T10:16:15.265 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.265 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.265 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:15.265 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.045959+0000", 2024-09-18T10:16:15.265 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.202926 2024-09-18T10:16:15.265 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.265 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.266 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:15.266 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.045990+0000", 2024-09-18T10:16:15.266 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0889999999999997e-05 2024-09-18T10:16:15.266 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:15.266 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:15.266 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:15.266 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.266 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.266 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.10s0 45/40 ECSubWriteReply(tid=18, last_complete=45'6, committed=1, applied=1))", 2024-09-18T10:16:15.266 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:01.843113+0000", 2024-09-18T10:16:15.266 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 73.367592916000007, 2024-09-18T10:16:15.266 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.203080129, 2024-09-18T10:16:15.266 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:15.267 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:16:15.267 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:15.267 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.267 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:15.267 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.843113+0000", 2024-09-18T10:16:15.267 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:15.267 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.267 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.267 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:15.267 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.843113+0000", 2024-09-18T10:16:15.267 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:15.267 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.267 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.267 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:15.268 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.843111+0000", 2024-09-18T10:16:15.268 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:16:15.268 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.268 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.268 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:15.268 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.843122+0000", 2024-09-18T10:16:15.268 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0199e-05 2024-09-18T10:16:15.268 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.268 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.268 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:15.268 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.843147+0000", 2024-09-18T10:16:15.268 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5113000000000002e-05 2024-09-18T10:16:15.268 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.269 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.269 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:15.269 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:01.943289+0000", 2024-09-18T10:16:15.269 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100141813 2024-09-18T10:16:15.269 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.269 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.269 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:15.269 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.046146+0000", 2024-09-18T10:16:15.269 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.102857768 2024-09-18T10:16:15.269 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.269 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.269 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:15.269 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.046193+0000", 2024-09-18T10:16:15.270 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.6566000000000001e-05 2024-09-18T10:16:15.270 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:15.270 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:15.270 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:15.270 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.270 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.270 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.1s5 57/40 ECSubWrite(tid=111, reqid=osd.4.0:111, at_version=57'18, trim_to=0'0, roll_forward_to=56'16))", 2024-09-18T10:16:15.270 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:13.054749+0000", 2024-09-18T10:16:15.270 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 62.155956492999998, 2024-09-18T10:16:15.270 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.201940535, 2024-09-18T10:16:15.270 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:15.270 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:16:15.270 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:15.270 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.271 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:15.271 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.054749+0000", 2024-09-18T10:16:15.271 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:15.271 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.271 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.271 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:15.271 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.054749+0000", 2024-09-18T10:16:15.271 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:15.271 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.271 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.271 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:15.271 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.054749+0000", 2024-09-18T10:16:15.271 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999995 2024-09-18T10:16:15.271 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.272 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.272 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:15.272 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.054785+0000", 2024-09-18T10:16:15.272 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6480999999999999e-05 2024-09-18T10:16:15.272 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.272 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.272 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:15.272 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.054808+0000", 2024-09-18T10:16:15.272 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.2351e-05 2024-09-18T10:16:15.272 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.272 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.272 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:15.272 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.154914+0000", 2024-09-18T10:16:15.273 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100106862 2024-09-18T10:16:15.273 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.273 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.273 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:15.273 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.155065+0000", 2024-09-18T10:16:15.273 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000150762 2024-09-18T10:16:15.273 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.273 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.273 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:16:15.273 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.155083+0000", 2024-09-18T10:16:15.273 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.8207e-05 2024-09-18T10:16:15.273 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.273 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.273 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:16:15.274 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.256244+0000", 2024-09-18T10:16:15.274 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10116046200000001 2024-09-18T10:16:15.274 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.274 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.274 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:15.274 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.256690+0000", 2024-09-18T10:16:15.274 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00044597000000000001 2024-09-18T10:16:15.274 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:15.274 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:15.274 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:15.274 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.274 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.274 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:511 3.0s0 3:02f544f0:::smithi18925896-30:head [stat,delete] snapc 1a=[1a,19,18,16,12,11,10,f,e] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-18T10:16:15.274 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:18.899317+0000", 2024-09-18T10:16:15.275 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 56.311389116000001, 2024-09-18T10:16:15.275 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.21051439599999999, 2024-09-18T10:16:15.275 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:15.275 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:15.275 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:16:15.275 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:16:15.275 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:15.275 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 511 2024-09-18T10:16:15.275 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.275 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:15.275 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.275 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:15.275 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.899317+0000", 2024-09-18T10:16:15.276 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:15.276 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.276 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.276 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:15.276 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.899317+0000", 2024-09-18T10:16:15.276 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:15.276 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.276 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.276 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:15.276 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.899314+0000", 2024-09-18T10:16:15.276 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999971 2024-09-18T10:16:15.276 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.276 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.276 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:15.277 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.899329+0000", 2024-09-18T10:16:15.277 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5160000000000001e-05 2024-09-18T10:16:15.277 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.277 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.277 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:15.277 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.899364+0000", 2024-09-18T10:16:15.277 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.5456000000000001e-05 2024-09-18T10:16:15.277 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.277 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.277 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:15.277 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.899385+0000", 2024-09-18T10:16:15.277 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0279e-05 2024-09-18T10:16:15.277 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.278 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.278 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:15.278 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.899497+0000", 2024-09-18T10:16:15.278 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000111773 2024-09-18T10:16:15.278 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.278 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.278 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "op must wait for map", 2024-09-18T10:16:15.278 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.899508+0000", 2024-09-18T10:16:15.278 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1177e-05 2024-09-18T10:16:15.278 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.278 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.278 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:15.278 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.903698+0000", 2024-09-18T10:16:15.278 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0041902850000000002 2024-09-18T10:16:15.279 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.279 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.279 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:16:15.279 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.903878+0000", 2024-09-18T10:16:15.279 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00018009499999999999 2024-09-18T10:16:15.279 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.279 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.279 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:16:15.279 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.905286+0000", 2024-09-18T10:16:15.279 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0014077720000000001 2024-09-18T10:16:15.279 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.279 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.279 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:16:15.280 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.003449+0000", 2024-09-18T10:16:15.280 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.098163081999999999 2024-09-18T10:16:15.280 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.280 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.280 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:16:15.280 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.109709+0000", 2024-09-18T10:16:15.280 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10625963400000001 2024-09-18T10:16:15.280 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.280 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.280 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:15.280 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.109831+0000", 2024-09-18T10:16:15.280 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00012246699999999999 2024-09-18T10:16:15.280 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:15.280 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:15.281 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:15.281 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.281 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.281 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.19s6 72/40 ECSubWrite(tid=176, reqid=osd.7.0:174, at_version=0'0, trim_to=0'0, roll_forward_to=71'100))", 2024-09-18T10:16:15.281 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:28.330736+0000", 2024-09-18T10:16:15.281 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 46.87996965, 2024-09-18T10:16:15.281 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20302320200000001, 2024-09-18T10:16:15.281 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:15.281 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:16:15.281 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:15.281 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.281 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:15.281 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.330736+0000", 2024-09-18T10:16:15.281 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:15.282 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.282 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.282 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:15.282 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.330736+0000", 2024-09-18T10:16:15.282 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:15.282 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.282 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.282 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:15.282 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.330735+0000", 2024-09-18T10:16:15.282 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:16:15.282 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.282 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.282 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:15.283 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.330759+0000", 2024-09-18T10:16:15.283 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4549000000000002e-05 2024-09-18T10:16:15.283 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.283 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.283 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:15.283 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.330816+0000", 2024-09-18T10:16:15.283 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.6693000000000001e-05 2024-09-18T10:16:15.283 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.283 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.283 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:15.283 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.430957+0000", 2024-09-18T10:16:15.283 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10014150199999999 2024-09-18T10:16:15.283 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.283 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.284 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:15.284 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.531236+0000", 2024-09-18T10:16:15.284 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100278201 2024-09-18T10:16:15.284 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.284 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.284 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:16:15.284 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.531263+0000", 2024-09-18T10:16:15.284 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7172999999999999e-05 2024-09-18T10:16:15.284 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.284 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.284 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:16:15.284 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.533701+0000", 2024-09-18T10:16:15.284 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0024382760000000001 2024-09-18T10:16:15.285 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.285 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.285 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:15.285 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.533759+0000", 2024-09-18T10:16:15.285 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8267999999999998e-05 2024-09-18T10:16:15.285 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:15.285 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:15.285 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:15.285 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.285 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.285 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.2s1 100/40 ECSubWrite(tid=175, reqid=osd.14.0:175, at_version=100'117, trim_to=0'0, roll_forward_to=99'115))", 2024-09-18T10:16:15.285 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:55.723156+0000", 2024-09-18T10:16:15.285 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 19.487549569999999, 2024-09-18T10:16:15.285 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20280843200000001, 2024-09-18T10:16:15.286 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:15.286 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:16:15.286 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:15.286 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.286 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:15.286 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:55.723156+0000", 2024-09-18T10:16:15.286 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:15.286 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.286 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.286 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:15.286 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:55.723156+0000", 2024-09-18T10:16:15.286 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:15.286 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.286 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.287 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:15.287 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:55.723156+0000", 2024-09-18T10:16:15.287 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999995 2024-09-18T10:16:15.287 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.287 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.287 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:15.287 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:55.723215+0000", 2024-09-18T10:16:15.287 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9318999999999999e-05 2024-09-18T10:16:15.287 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.287 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.287 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:15.287 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:55.723243+0000", 2024-09-18T10:16:15.287 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.8506000000000001e-05 2024-09-18T10:16:15.287 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.288 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.288 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:15.288 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:55.823360+0000", 2024-09-18T10:16:15.288 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100116356 2024-09-18T10:16:15.288 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.288 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.288 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:15.288 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:55.923621+0000", 2024-09-18T10:16:15.288 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10026093599999999 2024-09-18T10:16:15.288 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.288 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.288 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:16:15.288 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:55.923673+0000", 2024-09-18T10:16:15.289 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.2445999999999999e-05 2024-09-18T10:16:15.289 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.289 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.289 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:16:15.289 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:55.925909+0000", 2024-09-18T10:16:15.289 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0022360869999999999 2024-09-18T10:16:15.289 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.289 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.289 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:15.289 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:55.925965+0000", 2024-09-18T10:16:15.289 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.5439000000000002e-05 2024-09-18T10:16:15.289 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:15.289 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:15.290 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:15.290 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.290 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.290 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 100/40 ECSubWriteReply(tid=53, last_complete=100'38, committed=1, applied=1))", 2024-09-18T10:16:15.290 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:55.831180+0000", 2024-09-18T10:16:15.290 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 19.379525745999999, 2024-09-18T10:16:15.290 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20294891000000001, 2024-09-18T10:16:15.290 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:15.290 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:16:15.290 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:15.290 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.290 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:15.290 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:55.831180+0000", 2024-09-18T10:16:15.290 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:15.291 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.291 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.291 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:15.291 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:55.831180+0000", 2024-09-18T10:16:15.291 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:15.291 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.291 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.291 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:15.291 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:55.831177+0000", 2024-09-18T10:16:15.291 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999971 2024-09-18T10:16:15.291 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.291 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.291 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:15.291 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:55.831194+0000", 2024-09-18T10:16:15.292 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6552999999999999e-05 2024-09-18T10:16:15.292 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.292 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.292 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:15.292 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:55.831225+0000", 2024-09-18T10:16:15.292 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1473000000000002e-05 2024-09-18T10:16:15.292 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.292 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.292 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:15.292 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:55.931322+0000", 2024-09-18T10:16:15.292 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100096437 2024-09-18T10:16:15.292 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.292 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.293 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:15.293 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:56.032873+0000", 2024-09-18T10:16:15.293 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.101551321 2024-09-18T10:16:15.293 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.293 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.293 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:15.293 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:56.034129+0000", 2024-09-18T10:16:15.293 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0012560430000000001 2024-09-18T10:16:15.293 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:15.293 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:15.293 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:15.293 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.293 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.294 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.1s5 101/40 ECSubWrite(tid=322, reqid=osd.4.0:322, at_version=101'100, trim_to=0'0, roll_forward_to=100'98))", 2024-09-18T10:16:15.294 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:56.721945+0000", 2024-09-18T10:16:15.294 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 18.488760462999998, 2024-09-18T10:16:15.294 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20405271899999999, 2024-09-18T10:16:15.294 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:15.294 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:16:15.294 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:15.294 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.294 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:15.294 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:56.721945+0000", 2024-09-18T10:16:15.294 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:15.294 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.294 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.294 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:15.295 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:56.721945+0000", 2024-09-18T10:16:15.295 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:15.295 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.295 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.295 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:15.295 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:56.721945+0000", 2024-09-18T10:16:15.295 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999995 2024-09-18T10:16:15.295 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.295 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.296 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:15.296 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:56.721982+0000", 2024-09-18T10:16:15.296 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7251e-05 2024-09-18T10:16:15.296 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.296 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.296 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:15.296 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:56.722005+0000", 2024-09-18T10:16:15.296 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.2481999999999999e-05 2024-09-18T10:16:15.296 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.296 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.296 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:15.296 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:56.722017+0000", 2024-09-18T10:16:15.296 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2651e-05 2024-09-18T10:16:15.297 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.297 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.297 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:15.297 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:56.822239+0000", 2024-09-18T10:16:15.297 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10022138 2024-09-18T10:16:15.297 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.297 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.297 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:16:15.297 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:56.822255+0000", 2024-09-18T10:16:15.297 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6067e-05 2024-09-18T10:16:15.297 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.297 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.297 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:16:15.297 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:56.925947+0000", 2024-09-18T10:16:15.298 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.103692507 2024-09-18T10:16:15.298 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.298 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.298 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:15.298 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:56.925998+0000", 2024-09-18T10:16:15.298 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0682e-05 2024-09-18T10:16:15.298 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:15.298 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:15.298 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:15.298 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.298 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.298 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.1s5 101/40 ECSubWrite(tid=323, reqid=osd.4.0:323, at_version=101'102, trim_to=0'0, roll_forward_to=100'98))", 2024-09-18T10:16:15.298 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:56.724570+0000", 2024-09-18T10:16:15.299 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 18.486135809, 2024-09-18T10:16:15.299 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20195624700000001, 2024-09-18T10:16:15.299 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:15.299 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:16:15.299 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:15.299 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.299 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:15.299 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:56.724570+0000", 2024-09-18T10:16:15.299 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:15.299 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.299 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.299 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:15.299 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:56.724570+0000", 2024-09-18T10:16:15.299 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:15.300 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.300 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.300 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:15.300 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:56.724569+0000", 2024-09-18T10:16:15.300 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:16:15.300 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.300 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.300 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:15.300 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:56.724618+0000", 2024-09-18T10:16:15.300 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.8810999999999997e-05 2024-09-18T10:16:15.300 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.300 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.300 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:15.301 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:56.724635+0000", 2024-09-18T10:16:15.301 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7465e-05 2024-09-18T10:16:15.301 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.301 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.301 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:15.301 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:56.724644+0000", 2024-09-18T10:16:15.301 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.833e-06 2024-09-18T10:16:15.301 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.301 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.301 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:15.301 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:56.924891+0000", 2024-09-18T10:16:15.301 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20024745699999999 2024-09-18T10:16:15.301 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.301 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.302 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:16:15.302 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:56.924934+0000", 2024-09-18T10:16:15.302 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.2976000000000003e-05 2024-09-18T10:16:15.302 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.302 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.302 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:16:15.302 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:56.926493+0000", 2024-09-18T10:16:15.302 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0015585029999999999 2024-09-18T10:16:15.302 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.302 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.302 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:15.302 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:56.926526+0000", 2024-09-18T10:16:15.302 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3376e-05 2024-09-18T10:16:15.303 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:15.303 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:15.303 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:15.303 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:15.303 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:15.303 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-18T10:16:15.304 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T10:16:15.412 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-18T10:16:15.412 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-18T10:16:15.412 INFO:teuthology.orchestra.run.smithi189.stdout: "num_ops": 0 2024-09-18T10:16:15.412 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-18T10:16:15.421 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T10:16:15.523 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-18T10:16:15.523 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-18T10:16:15.523 INFO:teuthology.orchestra.run.smithi189.stdout: "complaint_time": 30, 2024-09-18T10:16:15.523 INFO:teuthology.orchestra.run.smithi189.stdout: "num_blocked_ops": 0 2024-09-18T10:16:15.523 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-18T10:16:15.532 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T10:16:15.638 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-18T10:16:15.638 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 20, 2024-09-18T10:16:15.638 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 600, 2024-09-18T10:16:15.638 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [ 2024-09-18T10:16:15.638 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.639 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:114 3.9s0 3:9a1a774c:::smithi18925896-38:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:16:15.639 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:50.816650+0000", 2024-09-18T10:16:15.639 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 84.817326147000003, 2024-09-18T10:16:15.639 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.41853457900000002, 2024-09-18T10:16:15.639 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:15.639 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:16:15.639 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:16:15.639 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:16:15.639 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:15.639 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 114 2024-09-18T10:16:15.639 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.640 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:15.640 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.640 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:15.640 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.816650+0000", 2024-09-18T10:16:15.640 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:15.640 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.640 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.640 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:15.640 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.816650+0000", 2024-09-18T10:16:15.640 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:15.640 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.640 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.641 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:15.641 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.816648+0000", 2024-09-18T10:16:15.641 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:16:15.641 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.641 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.641 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:15.641 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.816661+0000", 2024-09-18T10:16:15.641 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2619000000000001e-05 2024-09-18T10:16:15.641 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.641 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.641 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:15.641 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.816696+0000", 2024-09-18T10:16:15.642 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4885999999999998e-05 2024-09-18T10:16:15.642 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.642 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.642 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:15.642 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.816708+0000", 2024-09-18T10:16:15.642 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2843000000000001e-05 2024-09-18T10:16:15.642 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.642 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.642 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:15.642 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.818228+0000", 2024-09-18T10:16:15.642 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001519882 2024-09-18T10:16:15.642 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.643 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.643 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for rw locks", 2024-09-18T10:16:15.643 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.818557+0000", 2024-09-18T10:16:15.643 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00032873700000000001 2024-09-18T10:16:15.643 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.643 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.643 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:15.643 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.925488+0000", 2024-09-18T10:16:15.643 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.106930502 2024-09-18T10:16:15.643 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.643 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.644 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:16:15.644 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.925563+0000", 2024-09-18T10:16:15.644 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.5386e-05 2024-09-18T10:16:15.644 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.644 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.644 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:15.644 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.235185+0000", 2024-09-18T10:16:15.644 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.30962182399999999 2024-09-18T10:16:15.644 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:15.644 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:15.644 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:15.644 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.645 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.645 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:138 3.9s0 3:9ab7c4c0:::smithi18925896-46:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:16:15.645 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:50.921054+0000", 2024-09-18T10:16:15.645 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 84.712921848999997, 2024-09-18T10:16:15.645 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.61288337900000001, 2024-09-18T10:16:15.645 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:15.645 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:16:15.645 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:16:15.645 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:16:15.645 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:15.645 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 138 2024-09-18T10:16:15.645 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.645 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:15.646 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.646 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:15.646 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.921054+0000", 2024-09-18T10:16:15.646 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:15.646 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.646 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.646 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:15.646 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.921054+0000", 2024-09-18T10:16:15.646 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:15.646 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.646 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.646 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:15.646 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.921054+0000", 2024-09-18T10:16:15.647 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999995 2024-09-18T10:16:15.647 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.647 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.647 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:15.647 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.921057+0000", 2024-09-18T10:16:15.647 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1829999999999998e-06 2024-09-18T10:16:15.647 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.647 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.647 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:15.648 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.921090+0000", 2024-09-18T10:16:15.648 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3281999999999997e-05 2024-09-18T10:16:15.648 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.648 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.648 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:15.648 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.921096+0000", 2024-09-18T10:16:15.649 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.0519999999999998e-06 2024-09-18T10:16:15.649 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.649 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.649 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:15.649 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.127704+0000", 2024-09-18T10:16:15.649 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20660720499999999 2024-09-18T10:16:15.649 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.649 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.649 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for rw locks", 2024-09-18T10:16:15.649 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.127851+0000", 2024-09-18T10:16:15.649 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00014710999999999999 2024-09-18T10:16:15.649 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.650 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.650 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:15.650 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.234300+0000", 2024-09-18T10:16:15.650 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10644893900000001 2024-09-18T10:16:15.650 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.650 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.650 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:16:15.650 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.234462+0000", 2024-09-18T10:16:15.650 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00016218999999999999 2024-09-18T10:16:15.650 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.650 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.650 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:15.650 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.533938+0000", 2024-09-18T10:16:15.651 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.29947596999999998 2024-09-18T10:16:15.651 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:15.651 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:15.651 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:15.651 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.651 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.651 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:143 3.9s0 3:9221fff5:::smithi18925896-48:head [append 0~376832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:16:15.651 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:50.932038+0000", 2024-09-18T10:16:15.651 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 84.701938248000005, 2024-09-18T10:16:15.651 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.60109878000000005, 2024-09-18T10:16:15.651 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:15.651 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:15.651 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:16:15.651 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:16:15.652 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:15.652 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 143 2024-09-18T10:16:15.652 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.652 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:15.652 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.652 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:15.652 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932038+0000", 2024-09-18T10:16:15.652 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:15.652 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.652 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.652 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:15.653 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932038+0000", 2024-09-18T10:16:15.653 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:15.653 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.653 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.653 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:15.653 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932037+0000", 2024-09-18T10:16:15.653 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:16:15.653 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.653 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.653 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:15.653 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932309+0000", 2024-09-18T10:16:15.653 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000272086 2024-09-18T10:16:15.653 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.653 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.653 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:15.654 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932323+0000", 2024-09-18T10:16:15.654 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4354e-05 2024-09-18T10:16:15.654 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.654 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.654 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:15.654 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932329+0000", 2024-09-18T10:16:15.654 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.6840000000000001e-06 2024-09-18T10:16:15.654 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.654 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.654 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:15.655 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.130012+0000", 2024-09-18T10:16:15.655 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.19768277100000001 2024-09-18T10:16:15.655 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.655 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.655 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:16:15.655 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.130068+0000", 2024-09-18T10:16:15.655 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.6158e-05 2024-09-18T10:16:15.655 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.655 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.655 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:16:15.655 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.130809+0000", 2024-09-18T10:16:15.656 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00074078700000000002 2024-09-18T10:16:15.656 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.656 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.656 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:16:15.656 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.132840+0000", 2024-09-18T10:16:15.656 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0020308879999999998 2024-09-18T10:16:15.656 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.656 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.656 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:16:15.656 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.533025+0000", 2024-09-18T10:16:15.656 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.400185491 2024-09-18T10:16:15.656 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.656 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.656 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:15.657 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.533137+0000", 2024-09-18T10:16:15.657 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000111727 2024-09-18T10:16:15.657 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:15.657 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:15.657 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:15.657 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.657 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.657 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:144 3.9s0 3:9221fff5:::smithi18925896-48:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:16:15.657 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:50.932336+0000", 2024-09-18T10:16:15.657 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 84.701640612000006, 2024-09-18T10:16:15.657 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.60198766699999995, 2024-09-18T10:16:15.658 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:15.658 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:15.658 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:16:15.658 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:16:15.658 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:15.658 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 144 2024-09-18T10:16:15.658 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.658 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:15.658 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.658 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:15.659 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932336+0000", 2024-09-18T10:16:15.659 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:15.659 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.659 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.659 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:15.659 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932336+0000", 2024-09-18T10:16:15.659 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:15.659 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.659 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.659 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:15.660 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932335+0000", 2024-09-18T10:16:15.660 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:16:15.660 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.660 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.660 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:15.660 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932339+0000", 2024-09-18T10:16:15.660 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8539999999999999e-06 2024-09-18T10:16:15.660 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.660 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.660 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:15.660 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932347+0000", 2024-09-18T10:16:15.660 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.2910000000000007e-06 2024-09-18T10:16:15.660 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.661 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.661 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:15.661 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932351+0000", 2024-09-18T10:16:15.661 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.5420000000000001e-06 2024-09-18T10:16:15.661 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.661 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.661 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:15.661 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.131380+0000", 2024-09-18T10:16:15.661 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.199029236 2024-09-18T10:16:15.661 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.661 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.661 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:16:15.661 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.131436+0000", 2024-09-18T10:16:15.662 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.6098999999999999e-05 2024-09-18T10:16:15.662 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.662 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.662 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:16:15.662 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.131777+0000", 2024-09-18T10:16:15.662 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00034058800000000001 2024-09-18T10:16:15.662 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.662 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.662 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:16:15.662 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.133245+0000", 2024-09-18T10:16:15.662 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0014682300000000001 2024-09-18T10:16:15.662 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.662 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.663 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:16:15.663 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.534187+0000", 2024-09-18T10:16:15.663 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40094253299999999 2024-09-18T10:16:15.663 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.663 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.663 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:15.663 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.534323+0000", 2024-09-18T10:16:15.663 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00013606299999999999 2024-09-18T10:16:15.663 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:15.663 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:15.663 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:15.663 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.663 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.664 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:145 3.9s0 3:9221fff5:::smithi18925896-48:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:16:15.664 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:50.932355+0000", 2024-09-18T10:16:15.664 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 84.701621127999999, 2024-09-18T10:16:15.664 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.80502307200000001, 2024-09-18T10:16:15.664 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:15.664 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:16:15.664 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:16:15.664 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:16:15.664 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:15.664 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 145 2024-09-18T10:16:15.664 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.664 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:15.664 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.665 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:15.665 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932355+0000", 2024-09-18T10:16:15.665 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:15.665 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.665 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.665 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:15.665 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932355+0000", 2024-09-18T10:16:15.665 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:15.665 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.665 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.665 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:15.665 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932355+0000", 2024-09-18T10:16:15.665 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999995 2024-09-18T10:16:15.665 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.666 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.666 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:15.666 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932358+0000", 2024-09-18T10:16:15.666 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4369999999999999e-06 2024-09-18T10:16:15.666 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.666 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.666 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:15.666 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932366+0000", 2024-09-18T10:16:15.666 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.3010000000000007e-06 2024-09-18T10:16:15.666 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.666 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.666 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:15.666 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:50.932370+0000", 2024-09-18T10:16:15.666 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.5619999999999999e-06 2024-09-18T10:16:15.667 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.667 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.667 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:15.667 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.132210+0000", 2024-09-18T10:16:15.667 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.199840145 2024-09-18T10:16:15.667 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.667 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.667 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for rw locks", 2024-09-18T10:16:15.667 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.132312+0000", 2024-09-18T10:16:15.667 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000102335 2024-09-18T10:16:15.667 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.667 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.667 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:15.667 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.535310+0000", 2024-09-18T10:16:15.668 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40299779000000002 2024-09-18T10:16:15.668 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.668 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.668 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:16:15.668 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.535541+0000", 2024-09-18T10:16:15.668 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00023038299999999999 2024-09-18T10:16:15.668 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.668 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.668 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:15.668 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.737378+0000", 2024-09-18T10:16:15.668 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.201837604 2024-09-18T10:16:15.668 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:15.668 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:15.668 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:15.669 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.669 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.669 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:174 3.9s0 3:9fc2eacb:::smithi18925896-17:head [read 0~278528,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:16:15.669 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:51.836672+0000", 2024-09-18T10:16:15.669 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 83.797303873000004, 2024-09-18T10:16:15.669 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.407996215, 2024-09-18T10:16:15.669 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:15.669 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:16:15.669 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:16:15.670 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:16:15.670 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:15.670 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 174 2024-09-18T10:16:15.670 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.670 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:15.670 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.670 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:15.670 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.836672+0000", 2024-09-18T10:16:15.670 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:15.670 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.671 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.671 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:15.671 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.836672+0000", 2024-09-18T10:16:15.671 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:15.671 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.671 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.671 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:15.671 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.836671+0000", 2024-09-18T10:16:15.671 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:16:15.671 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.672 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.672 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:15.672 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.836680+0000", 2024-09-18T10:16:15.672 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.5830000000000007e-06 2024-09-18T10:16:15.672 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.672 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.672 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:15.672 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.836696+0000", 2024-09-18T10:16:15.672 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6823000000000001e-05 2024-09-18T10:16:15.672 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.672 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.673 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:15.673 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.836710+0000", 2024-09-18T10:16:15.673 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.325e-05 2024-09-18T10:16:15.673 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.673 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.673 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:15.673 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.142312+0000", 2024-09-18T10:16:15.673 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.30560266400000002 2024-09-18T10:16:15.673 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.673 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.673 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:16:15.673 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.142549+0000", 2024-09-18T10:16:15.673 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00023630299999999999 2024-09-18T10:16:15.673 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.674 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.674 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:15.674 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.244669+0000", 2024-09-18T10:16:15.674 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10211993599999999 2024-09-18T10:16:15.674 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:15.674 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:15.674 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:15.674 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.674 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.674 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:202 3.9s0 3:9221fff5:::smithi18925896-48:head [copy-from ver 2] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:16:15.674 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:52.793605+0000", 2024-09-18T10:16:15.674 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 82.840371278000006, 2024-09-18T10:16:15.674 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.61381690899999997, 2024-09-18T10:16:15.674 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:15.675 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:15.675 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:16:15.675 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:16:15.675 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:15.675 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 202 2024-09-18T10:16:15.675 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.675 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:15.675 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.675 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:15.675 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.793605+0000", 2024-09-18T10:16:15.675 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:15.675 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.675 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.675 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:15.676 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.793605+0000", 2024-09-18T10:16:15.676 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:15.676 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.676 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.676 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:15.676 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.793604+0000", 2024-09-18T10:16:15.676 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:16:15.676 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.676 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.676 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:15.676 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.793613+0000", 2024-09-18T10:16:15.676 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.4360000000000008e-06 2024-09-18T10:16:15.676 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.677 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.677 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:15.677 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.793628+0000", 2024-09-18T10:16:15.677 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4953000000000001e-05 2024-09-18T10:16:15.677 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.677 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.677 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:15.677 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.793634+0000", 2024-09-18T10:16:15.677 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.0170000000000002e-06 2024-09-18T10:16:15.677 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.677 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.677 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:15.677 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.905359+0000", 2024-09-18T10:16:15.677 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.111724328 2024-09-18T10:16:15.678 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.678 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.678 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:16:15.678 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.905540+0000", 2024-09-18T10:16:15.678 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000181552 2024-09-18T10:16:15.678 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.678 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.678 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:16:15.678 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:53.203724+0000", 2024-09-18T10:16:15.678 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.29818339500000002 2024-09-18T10:16:15.678 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.678 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.678 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:16:15.679 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:53.206308+0000", 2024-09-18T10:16:15.679 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0025841620000000001 2024-09-18T10:16:15.679 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.679 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.679 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:16:15.679 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:53.407237+0000", 2024-09-18T10:16:15.679 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.200928739 2024-09-18T10:16:15.679 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.679 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.679 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:15.679 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:53.407422+0000", 2024-09-18T10:16:15.679 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000185384 2024-09-18T10:16:15.679 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:15.679 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:15.680 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:15.680 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.680 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.680 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:203 3.9s0 3:9221fff5:::smithi18925896-48:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e37)", 2024-09-18T10:16:15.680 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:52.793641+0000", 2024-09-18T10:16:15.680 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 82.840334893000005, 2024-09-18T10:16:15.680 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.41220821099999999, 2024-09-18T10:16:15.680 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:15.680 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:16:15.681 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:16:15.681 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:16:15.681 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:15.681 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 203 2024-09-18T10:16:15.681 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.681 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:15.681 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.681 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:15.681 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.793641+0000", 2024-09-18T10:16:15.681 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:15.681 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.681 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.681 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:15.681 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.793641+0000", 2024-09-18T10:16:15.682 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:15.682 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.682 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.682 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:15.682 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.793641+0000", 2024-09-18T10:16:15.682 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:16:15.682 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.682 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.682 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:15.682 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.793644+0000", 2024-09-18T10:16:15.682 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7170000000000002e-06 2024-09-18T10:16:15.682 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.682 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.682 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:15.683 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.793658+0000", 2024-09-18T10:16:15.683 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3353000000000001e-05 2024-09-18T10:16:15.683 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.683 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.683 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:15.683 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.793663+0000", 2024-09-18T10:16:15.683 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.7729999999999996e-06 2024-09-18T10:16:15.683 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.683 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.683 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:15.683 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.905813+0000", 2024-09-18T10:16:15.683 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.112150033 2024-09-18T10:16:15.683 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.683 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.684 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for blocked object", 2024-09-18T10:16:15.684 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.905901+0000", 2024-09-18T10:16:15.684 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.7319999999999997e-05 2024-09-18T10:16:15.684 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.684 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.684 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:15.684 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:53.205317+0000", 2024-09-18T10:16:15.684 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.299415826 2024-09-18T10:16:15.684 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.684 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.684 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:16:15.684 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:53.205528+0000", 2024-09-18T10:16:15.684 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000211485 2024-09-18T10:16:15.685 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.685 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.685 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:15.685 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:53.205850+0000", 2024-09-18T10:16:15.685 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000321544 2024-09-18T10:16:15.685 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:15.685 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:15.685 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:15.685 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.685 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.685 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:351 3.19s0 3:9a1a774c:::smithi18925896-38:b [read 0~1638400,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-18T10:16:15.685 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:03.544302+0000", 2024-09-18T10:16:15.685 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 72.089673888999997, 2024-09-18T10:16:15.686 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.41512137599999999, 2024-09-18T10:16:15.686 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:15.686 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:16:15.686 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:16:15.686 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:16:15.686 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:15.686 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 351 2024-09-18T10:16:15.686 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.686 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:15.686 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.686 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:15.686 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.544302+0000", 2024-09-18T10:16:15.686 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:15.687 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.687 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.687 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:15.687 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.544302+0000", 2024-09-18T10:16:15.687 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:15.687 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.687 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.687 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:15.687 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.544298+0000", 2024-09-18T10:16:15.687 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999957 2024-09-18T10:16:15.687 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.687 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.687 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:15.687 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.544320+0000", 2024-09-18T10:16:15.688 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1549e-05 2024-09-18T10:16:15.688 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.688 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.688 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:15.688 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.544366+0000", 2024-09-18T10:16:15.688 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.6752999999999998e-05 2024-09-18T10:16:15.688 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.688 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.688 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:15.688 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.544387+0000", 2024-09-18T10:16:15.688 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1052000000000001e-05 2024-09-18T10:16:15.688 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.688 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.689 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:15.689 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.544758+0000", 2024-09-18T10:16:15.689 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00037069700000000002 2024-09-18T10:16:15.689 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.689 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.689 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "op must wait for map", 2024-09-18T10:16:15.689 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.544786+0000", 2024-09-18T10:16:15.689 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7472000000000001e-05 2024-09-18T10:16:15.689 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.689 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.689 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:15.689 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.649236+0000", 2024-09-18T10:16:15.689 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.104450378 2024-09-18T10:16:15.689 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.690 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.690 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:16:15.690 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.649492+0000", 2024-09-18T10:16:15.690 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00025605300000000001 2024-09-18T10:16:15.690 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.690 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.690 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:15.690 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.959424+0000", 2024-09-18T10:16:15.690 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.30993174899999998 2024-09-18T10:16:15.691 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:15.691 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:15.691 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:15.691 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.691 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.691 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:417 3.9s0 3:9221fff5:::smithi18925896-48:head [copy-from ver 8] snapc 10=[10,f,e,b,4] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-18T10:16:15.691 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:08.547348+0000", 2024-09-18T10:16:15.691 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 67.086628662999999, 2024-09-18T10:16:15.691 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.69832097199999998, 2024-09-18T10:16:15.692 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:15.692 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:15.692 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:16:15.692 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:16:15.692 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:15.692 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 417 2024-09-18T10:16:15.692 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.692 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:15.692 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.692 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:15.692 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:08.547348+0000", 2024-09-18T10:16:15.692 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:15.692 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.693 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.693 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:15.693 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:08.547348+0000", 2024-09-18T10:16:15.693 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:15.693 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.693 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.693 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:15.693 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:08.547344+0000", 2024-09-18T10:16:15.693 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999967 2024-09-18T10:16:15.693 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.693 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.693 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:15.693 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:08.547367+0000", 2024-09-18T10:16:15.694 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.2316000000000001e-05 2024-09-18T10:16:15.694 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.694 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.694 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:15.694 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:08.547415+0000", 2024-09-18T10:16:15.694 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.8541999999999997e-05 2024-09-18T10:16:15.694 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.694 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.694 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:15.694 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:08.647531+0000", 2024-09-18T10:16:15.694 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10011588 2024-09-18T10:16:15.694 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.694 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.694 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:15.695 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:08.647713+0000", 2024-09-18T10:16:15.695 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000181965 2024-09-18T10:16:15.695 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.695 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.695 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "op must wait for map", 2024-09-18T10:16:15.695 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:08.647726+0000", 2024-09-18T10:16:15.695 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3576e-05 2024-09-18T10:16:15.695 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.695 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.695 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:15.695 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:08.754339+0000", 2024-09-18T10:16:15.695 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10661229699999999 2024-09-18T10:16:15.695 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.696 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.696 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:16:15.696 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:08.754855+0000", 2024-09-18T10:16:15.696 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00051627900000000002 2024-09-18T10:16:15.696 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.696 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.696 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:16:15.696 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:09.040700+0000", 2024-09-18T10:16:15.696 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.285844922 2024-09-18T10:16:15.696 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.696 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.696 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:16:15.697 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:09.045961+0000", 2024-09-18T10:16:15.697 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0052605769999999998 2024-09-18T10:16:15.697 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.697 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.697 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:16:15.697 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:09.245481+0000", 2024-09-18T10:16:15.697 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.19952081599999999 2024-09-18T10:16:15.697 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.697 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.697 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:15.697 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:09.245669+0000", 2024-09-18T10:16:15.698 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000187245 2024-09-18T10:16:15.698 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:15.698 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:15.698 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:15.698 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.698 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.698 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.1s4 57/40 ECSubWrite(tid=112, reqid=osd.4.0:112, at_version=57'20, trim_to=0'0, roll_forward_to=56'16))", 2024-09-18T10:16:15.698 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:13.062699+0000", 2024-09-18T10:16:15.698 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 62.571276974, 2024-09-18T10:16:15.698 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40470361999999999, 2024-09-18T10:16:15.698 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:15.698 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:16:15.698 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:15.698 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.699 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:15.699 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.062699+0000", 2024-09-18T10:16:15.699 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:15.699 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.699 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.699 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:15.699 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.062699+0000", 2024-09-18T10:16:15.699 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:15.699 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.699 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.699 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:15.699 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.062699+0000", 2024-09-18T10:16:15.699 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999995 2024-09-18T10:16:15.699 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.700 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.700 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:15.700 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.062734+0000", 2024-09-18T10:16:15.700 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.5108999999999999e-05 2024-09-18T10:16:15.700 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.700 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.700 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:15.700 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.062746+0000", 2024-09-18T10:16:15.701 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2062000000000001e-05 2024-09-18T10:16:15.701 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.701 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.701 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:15.701 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.062754+0000", 2024-09-18T10:16:15.701 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.5800000000000003e-06 2024-09-18T10:16:15.701 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.701 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.701 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:15.701 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.254656+0000", 2024-09-18T10:16:15.701 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.19190244400000001 2024-09-18T10:16:15.701 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.701 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.702 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:16:15.702 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.254673+0000", 2024-09-18T10:16:15.702 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7146000000000002e-05 2024-09-18T10:16:15.702 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.702 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.702 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:16:15.702 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.467347+0000", 2024-09-18T10:16:15.702 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.21267335500000001 2024-09-18T10:16:15.702 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.702 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.702 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:15.702 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.467403+0000", 2024-09-18T10:16:15.702 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.6443999999999997e-05 2024-09-18T10:16:15.702 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:15.703 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:15.703 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:15.703 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.703 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.703 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:499 3.9s0 3:9221fff5:::smithi18925896-48:head [read 0~704512,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T10:16:15.703 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:17.889501+0000", 2024-09-18T10:16:15.703 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 57.744474937, 2024-09-18T10:16:15.703 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.41188610399999998, 2024-09-18T10:16:15.703 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:15.703 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:16:15.703 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:16:15.703 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:16:15.703 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:15.704 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 499 2024-09-18T10:16:15.704 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.704 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:15.704 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.704 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:15.704 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889501+0000", 2024-09-18T10:16:15.704 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:15.704 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.704 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.704 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:15.704 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889501+0000", 2024-09-18T10:16:15.705 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:15.705 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.705 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.705 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:15.705 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889499+0000", 2024-09-18T10:16:15.705 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999976 2024-09-18T10:16:15.705 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.705 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.705 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:15.705 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889514+0000", 2024-09-18T10:16:15.705 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4793999999999999e-05 2024-09-18T10:16:15.705 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.705 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.705 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:15.706 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889538+0000", 2024-09-18T10:16:15.706 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4218000000000001e-05 2024-09-18T10:16:15.706 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.706 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.706 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:15.706 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889552+0000", 2024-09-18T10:16:15.706 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4021e-05 2024-09-18T10:16:15.706 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.706 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.706 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:15.706 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889705+0000", 2024-09-18T10:16:15.706 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00015229499999999999 2024-09-18T10:16:15.706 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.707 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.707 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "op must wait for map", 2024-09-18T10:16:15.707 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889725+0000", 2024-09-18T10:16:15.707 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0251999999999999e-05 2024-09-18T10:16:15.707 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.707 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.707 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:15.707 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.894166+0000", 2024-09-18T10:16:15.707 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0044407969999999998 2024-09-18T10:16:15.707 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.707 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.707 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:16:15.707 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.894321+0000", 2024-09-18T10:16:15.707 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000155403 2024-09-18T10:16:15.708 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.708 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.708 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:15.708 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.301387+0000", 2024-09-18T10:16:15.708 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40706650900000002 2024-09-18T10:16:15.708 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:15.708 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:15.708 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:15.708 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.708 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.708 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:500 3.9s0 3:9221fff5:::smithi18925896-48:head [read 0~704512,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T10:16:15.708 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:17.889581+0000", 2024-09-18T10:16:15.708 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 57.744395396000002, 2024-09-18T10:16:15.709 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.41275783799999999, 2024-09-18T10:16:15.709 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:15.709 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:16:15.709 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:16:15.709 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:16:15.709 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:15.709 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 500 2024-09-18T10:16:15.709 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.709 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:15.709 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.709 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:15.709 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889581+0000", 2024-09-18T10:16:15.709 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:15.709 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.710 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.710 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:15.710 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889581+0000", 2024-09-18T10:16:15.710 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:15.710 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.710 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.710 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:15.710 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889580+0000", 2024-09-18T10:16:15.710 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999995 2024-09-18T10:16:15.710 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.710 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.710 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:15.710 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889586+0000", 2024-09-18T10:16:15.711 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8340000000000004e-06 2024-09-18T10:16:15.711 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.711 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.711 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:15.711 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889594+0000", 2024-09-18T10:16:15.711 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.8320000000000006e-06 2024-09-18T10:16:15.711 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.711 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.711 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:15.711 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889599+0000", 2024-09-18T10:16:15.711 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.7890000000000002e-06 2024-09-18T10:16:15.711 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.711 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.711 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:15.712 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889906+0000", 2024-09-18T10:16:15.712 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00030686400000000001 2024-09-18T10:16:15.712 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.712 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.712 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:16:15.712 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889930+0000", 2024-09-18T10:16:15.712 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4318999999999999e-05 2024-09-18T10:16:15.712 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.712 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.712 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:15.712 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.895541+0000", 2024-09-18T10:16:15.712 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0056113029999999998 2024-09-18T10:16:15.712 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.713 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.713 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:16:15.713 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.895817+0000", 2024-09-18T10:16:15.713 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000275612 2024-09-18T10:16:15.713 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.713 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.713 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:15.713 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.302339+0000", 2024-09-18T10:16:15.713 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40652197699999998 2024-09-18T10:16:15.713 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:15.713 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:15.713 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:15.713 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.713 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.714 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:501 3.9s0 3:9221fff5:::smithi18925896-48:head [read 0~704512,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T10:16:15.714 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:17.889668+0000", 2024-09-18T10:16:15.714 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 57.744308285999999, 2024-09-18T10:16:15.714 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.51380426099999998, 2024-09-18T10:16:15.714 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:15.714 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:16:15.714 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:16:15.714 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:16:15.714 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:15.714 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 501 2024-09-18T10:16:15.714 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.714 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:15.714 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.714 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:15.715 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889668+0000", 2024-09-18T10:16:15.715 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:15.715 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.715 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.715 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:15.715 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889668+0000", 2024-09-18T10:16:15.715 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:15.715 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.715 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.715 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:15.715 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889667+0000", 2024-09-18T10:16:15.715 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999995 2024-09-18T10:16:15.715 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.715 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.716 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:15.716 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889671+0000", 2024-09-18T10:16:15.716 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7570000000000002e-06 2024-09-18T10:16:15.716 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.716 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.716 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:15.716 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889682+0000", 2024-09-18T10:16:15.716 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0709e-05 2024-09-18T10:16:15.716 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.716 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.716 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:15.716 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889686+0000", 2024-09-18T10:16:15.716 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.437e-06 2024-09-18T10:16:15.717 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.717 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.717 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:15.717 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889996+0000", 2024-09-18T10:16:15.717 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00030937899999999998 2024-09-18T10:16:15.717 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.717 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.717 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:16:15.717 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.890018+0000", 2024-09-18T10:16:15.717 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.2592999999999999e-05 2024-09-18T10:16:15.717 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.717 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.717 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:15.717 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.996777+0000", 2024-09-18T10:16:15.718 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.106758192 2024-09-18T10:16:15.718 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.718 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.718 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:16:15.718 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.997045+0000", 2024-09-18T10:16:15.718 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00026851300000000001 2024-09-18T10:16:15.718 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.718 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.718 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:15.718 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.403472+0000", 2024-09-18T10:16:15.718 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40642724899999999 2024-09-18T10:16:15.718 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:15.718 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:15.718 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:15.719 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.719 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.719 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:502 3.19s0 3:9a1a774c:::smithi18925896-38:head [create,setxattr Rj\\mZOfQXo^b_ZWWfqN]m[iPt^aRVgsYNetVd`VO (19508) in=40b,setxattr [adnUNmYmWdN__]tR]msPnVSRiY^psh`dY\\Z (630) in=36b,setxattr cPsWXtVeXTPagdn_WVZbVcV\\UjSZUqsVlPZbY` (630) in=38b,setxattr pqbrReOSsgZlmftc]`Z`]USaXjsacf`fao_VP`eU (1190) in=40b,setxattr aYfgr^q[\\aPg_aZsakfUfsdhYbra^QXjlkrpdY\\\\ (138) in=40b,setxattr e\\XXWiVknihNQQ[OsThQUbRmtprrfYPeRX[[mRcQ (1719) in=40b,setxattr TNQYR^jgR`pPaYitmrcliV[dttobncTntcfTlQkt (8420) in=40b,setxattr W_aq[RVsaUpQtX_RpV^Vpqrg`QgQViPeVaRqZSmf (525) in=40b,setxattr RWdkeaaUP\\PZteTNscS^UVRtjo`oqqdNqe]cdZUa (245) in=40b,setxattr R^`al]TqifatYOiOU]p`ZegfXmNsXVaZcdd]b]YX (1435) in=40b,setxattr p]OS_Znl[dNa^qTn`WW[j\\dtgbNac_bijYmg (630) in=36b,setxattr n_jr[tV[VhYPqU]UenciobhtYfY^a^c\\hd^YrcQ[ (13472) in=40b,setxattr gReQSUg`cdl_ilU_lo[ggg\\fZsVQY[PgY]jPsZ_U (14664) in=40b,setxattr cNWaQrsdt^dic_qN_^mTXmc`_X]YOTXNk[WldRe` (523) in=40b,setxattr tZ]grhlaq]kUO[k^d^dr``XNio^^Oaebpj\\[Ytdq (33) in=40b,setxattr jqS_j^cefaXab`pQoPXfRkSmdako`fZ_heb^_fma (1191) in=40b,setxattr dgXPi (630) in=5b,setxattr Or_kkpbnNkTrQ\\apUaUsZR\\XW^nabhaiQ\\]gOjR\\ (525) in=40b,setxattr Rf`aR[fdtjQ\\NaqZke]l_rUssrVpOPXYWpXkl_kd (2876) in=40b,setxattr _sUSYWYUmOrlUVhZajZeYbbnsdra[]`Q`^Ogea[m (631) in=40b] snapc 19=[19,18,16,12,11,10,f,e] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T10:16:15.719 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:17.894821+0000", 2024-09-18T10:16:15.719 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 57.739155257999997, 2024-09-18T10:16:15.719 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.61297755899999995, 2024-09-18T10:16:15.719 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:15.719 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:15.719 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:16:15.719 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:16:15.719 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:15.719 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 502 2024-09-18T10:16:15.719 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.719 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:15.720 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.720 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:15.720 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.894821+0000", 2024-09-18T10:16:15.720 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:15.720 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.720 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.720 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:15.720 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.894821+0000", 2024-09-18T10:16:15.720 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:15.720 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.720 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.720 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:15.720 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.894819+0000", 2024-09-18T10:16:15.721 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:16:15.721 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.721 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.721 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:15.721 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.894942+0000", 2024-09-18T10:16:15.721 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00012299699999999999 2024-09-18T10:16:15.721 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.721 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.721 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:15.721 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.894987+0000", 2024-09-18T10:16:15.722 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.515e-05 2024-09-18T10:16:15.722 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.722 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.722 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:15.722 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.895021+0000", 2024-09-18T10:16:15.722 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3905000000000001e-05 2024-09-18T10:16:15.722 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.722 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.722 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:15.722 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.302420+0000", 2024-09-18T10:16:15.722 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40739912099999998 2024-09-18T10:16:15.722 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.722 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.723 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:16:15.723 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.302573+0000", 2024-09-18T10:16:15.723 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00015244400000000001 2024-09-18T10:16:15.723 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.723 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.723 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:16:15.723 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.304119+0000", 2024-09-18T10:16:15.723 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001546504 2024-09-18T10:16:15.723 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.723 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.723 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:16:15.723 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.306309+0000", 2024-09-18T10:16:15.723 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.002189992 2024-09-18T10:16:15.723 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.724 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.724 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:16:15.724 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.507653+0000", 2024-09-18T10:16:15.724 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20134429600000001 2024-09-18T10:16:15.724 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.724 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.724 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:15.724 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.507799+0000", 2024-09-18T10:16:15.724 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000145153 2024-09-18T10:16:15.724 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:15.724 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:15.724 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:15.724 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.725 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.725 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:514 3.19s0 3:9fc2eacb:::smithi18925896-17:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e63)", 2024-09-18T10:16:15.725 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:19.215577+0000", 2024-09-18T10:16:15.725 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 56.418399016999999, 2024-09-18T10:16:15.725 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.55261972699999995, 2024-09-18T10:16:15.725 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:15.725 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:16:15.725 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:16:15.725 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:16:15.725 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:15.725 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 514 2024-09-18T10:16:15.725 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.725 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:15.725 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.726 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:15.726 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.215577+0000", 2024-09-18T10:16:15.726 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:15.726 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.726 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.726 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:15.726 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.215577+0000", 2024-09-18T10:16:15.726 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:15.726 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.726 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.726 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:15.726 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.215575+0000", 2024-09-18T10:16:15.726 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:16:15.726 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.727 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.727 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:15.727 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.215590+0000", 2024-09-18T10:16:15.727 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4153e-05 2024-09-18T10:16:15.727 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.727 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.727 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:15.727 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.215610+0000", 2024-09-18T10:16:15.727 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9962e-05 2024-09-18T10:16:15.727 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.727 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.727 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:15.727 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.215622+0000", 2024-09-18T10:16:15.727 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2714000000000001e-05 2024-09-18T10:16:15.728 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.728 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.728 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:15.728 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.216288+0000", 2024-09-18T10:16:15.728 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00066566900000000003 2024-09-18T10:16:15.728 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.728 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.728 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for blocked object", 2024-09-18T10:16:15.728 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.216400+0000", 2024-09-18T10:16:15.728 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000112101 2024-09-18T10:16:15.728 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.728 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.728 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:15.729 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.767868+0000", 2024-09-18T10:16:15.729 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.55146753900000001 2024-09-18T10:16:15.729 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.729 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.729 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:16:15.729 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.768017+0000", 2024-09-18T10:16:15.729 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000149567 2024-09-18T10:16:15.729 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.729 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.729 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:15.729 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.768197+0000", 2024-09-18T10:16:15.729 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00017981200000000001 2024-09-18T10:16:15.729 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:15.729 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:15.730 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:15.730 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.730 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.730 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:561 3.19s0 3:9fc2eacb:::smithi18925896-17:head [append 0~622592] snapc 1f=[1f,1e,1a,16,12,11,10,f] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-18T10:16:15.730 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:23.955639+0000", 2024-09-18T10:16:15.730 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 51.678337671000001, 2024-09-18T10:16:15.730 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.51252563100000004, 2024-09-18T10:16:15.730 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:15.730 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:15.730 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:16:15.730 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:16:15.730 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:15.730 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 561 2024-09-18T10:16:15.730 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.731 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:15.731 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.731 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:15.731 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.955639+0000", 2024-09-18T10:16:15.731 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:15.731 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.731 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.731 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:15.731 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.955639+0000", 2024-09-18T10:16:15.731 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:15.731 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.731 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.731 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:15.731 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.955637+0000", 2024-09-18T10:16:15.732 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:16:15.732 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.732 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.732 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:15.732 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956211+0000", 2024-09-18T10:16:15.732 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00057413099999999997 2024-09-18T10:16:15.732 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.732 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.732 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:15.732 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956241+0000", 2024-09-18T10:16:15.732 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0244999999999999e-05 2024-09-18T10:16:15.732 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.732 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.733 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:15.733 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956253+0000", 2024-09-18T10:16:15.733 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1824000000000001e-05 2024-09-18T10:16:15.733 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.733 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.733 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:15.733 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956338+0000", 2024-09-18T10:16:15.733 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.5544999999999995e-05 2024-09-18T10:16:15.733 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.733 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.733 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:16:15.733 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956349+0000", 2024-09-18T10:16:15.733 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0203e-05 2024-09-18T10:16:15.733 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.734 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.734 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:15.734 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:24.162636+0000", 2024-09-18T10:16:15.734 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.206287267 2024-09-18T10:16:15.734 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.734 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.734 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:16:15.734 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:24.162935+0000", 2024-09-18T10:16:15.734 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00029932800000000002 2024-09-18T10:16:15.734 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.734 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.735 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:16:15.735 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:24.164750+0000", 2024-09-18T10:16:15.735 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0018148400000000001 2024-09-18T10:16:15.735 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.735 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.735 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:16:15.735 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:24.168548+0000", 2024-09-18T10:16:15.735 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0037977089999999998 2024-09-18T10:16:15.735 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.735 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.735 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:16:15.735 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:24.467969+0000", 2024-09-18T10:16:15.735 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.29942109700000002 2024-09-18T10:16:15.736 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.736 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.736 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:15.736 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:24.468164+0000", 2024-09-18T10:16:15.736 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00019534000000000001 2024-09-18T10:16:15.736 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:15.736 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:15.736 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:15.736 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.736 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.736 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:562 3.19s0 3:9fc2eacb:::smithi18925896-17:head [append 0~32768] snapc 1f=[1f,1e,1a,16,12,11,10,f] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-18T10:16:15.736 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:23.956275+0000", 2024-09-18T10:16:15.736 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 51.677701683999999, 2024-09-18T10:16:15.736 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.61260356900000001, 2024-09-18T10:16:15.737 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:15.737 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:15.737 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:16:15.737 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:16:15.737 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:15.737 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 562 2024-09-18T10:16:15.737 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.737 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:15.737 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.737 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:15.737 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956275+0000", 2024-09-18T10:16:15.737 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:15.737 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.738 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.738 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:15.738 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956275+0000", 2024-09-18T10:16:15.738 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:15.738 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.738 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.738 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:15.738 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956273+0000", 2024-09-18T10:16:15.738 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:16:15.738 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.738 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.738 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:15.738 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956355+0000", 2024-09-18T10:16:15.738 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.1661000000000003e-05 2024-09-18T10:16:15.739 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.739 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.739 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:15.739 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956366+0000", 2024-09-18T10:16:15.739 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1192000000000001e-05 2024-09-18T10:16:15.739 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.739 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.739 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:15.739 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956373+0000", 2024-09-18T10:16:15.739 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.5599999999999999e-06 2024-09-18T10:16:15.739 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.739 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.739 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:15.739 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956434+0000", 2024-09-18T10:16:15.740 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.0761999999999999e-05 2024-09-18T10:16:15.740 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.740 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.740 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:16:15.740 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956441+0000", 2024-09-18T10:16:15.740 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.1430000000000001e-06 2024-09-18T10:16:15.740 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.740 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.740 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:15.740 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:24.165954+0000", 2024-09-18T10:16:15.740 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20951336100000001 2024-09-18T10:16:15.740 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.740 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.741 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:16:15.741 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:24.166077+0000", 2024-09-18T10:16:15.741 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00012297299999999999 2024-09-18T10:16:15.741 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.741 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.741 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:16:15.741 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:24.166875+0000", 2024-09-18T10:16:15.741 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00079825999999999996 2024-09-18T10:16:15.741 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.741 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.741 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:16:15.741 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:24.168650+0000", 2024-09-18T10:16:15.741 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0017746839999999999 2024-09-18T10:16:15.741 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.742 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.742 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:16:15.742 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:24.568726+0000", 2024-09-18T10:16:15.742 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40007642799999998 2024-09-18T10:16:15.742 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.742 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.742 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:15.742 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:24.568878+0000", 2024-09-18T10:16:15.742 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000151803 2024-09-18T10:16:15.742 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:15.742 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:15.742 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:15.742 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.742 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.743 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:563 3.19s0 3:9fc2eacb:::smithi18925896-17:head [setxattr _header (57) in=7b] snapc 1f=[1f,1e,1a,16,12,11,10,f] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-18T10:16:15.743 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:23.956389+0000", 2024-09-18T10:16:15.743 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 51.677587338000002, 2024-09-18T10:16:15.743 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.61276698100000004, 2024-09-18T10:16:15.743 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:15.743 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:15.743 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:16:15.743 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:16:15.743 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:15.743 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 563 2024-09-18T10:16:15.743 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.743 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:15.743 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.743 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:15.744 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956389+0000", 2024-09-18T10:16:15.744 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:15.744 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.744 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.744 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:15.744 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956389+0000", 2024-09-18T10:16:15.744 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:15.744 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.744 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.744 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:15.744 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956387+0000", 2024-09-18T10:16:15.745 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:16:15.745 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.745 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.745 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:15.745 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956395+0000", 2024-09-18T10:16:15.745 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.537e-06 2024-09-18T10:16:15.745 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.745 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.745 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:15.745 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956405+0000", 2024-09-18T10:16:15.745 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0057999999999999e-05 2024-09-18T10:16:15.745 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.745 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.745 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:15.746 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956416+0000", 2024-09-18T10:16:15.746 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1311999999999999e-05 2024-09-18T10:16:15.746 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.746 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.746 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:15.746 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956502+0000", 2024-09-18T10:16:15.746 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.5413000000000004e-05 2024-09-18T10:16:15.746 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.746 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.746 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:16:15.746 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956512+0000", 2024-09-18T10:16:15.746 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0299000000000001e-05 2024-09-18T10:16:15.746 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.747 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.747 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:15.747 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:24.167493+0000", 2024-09-18T10:16:15.747 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.21098060699999999 2024-09-18T10:16:15.747 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.747 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.747 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:16:15.747 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:24.167618+0000", 2024-09-18T10:16:15.747 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00012526599999999999 2024-09-18T10:16:15.747 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.747 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.747 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:16:15.747 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:24.168157+0000", 2024-09-18T10:16:15.747 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00053891400000000004 2024-09-18T10:16:15.748 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.748 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.748 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:16:15.748 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:24.168870+0000", 2024-09-18T10:16:15.748 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00071273300000000001 2024-09-18T10:16:15.748 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.748 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.748 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:16:15.748 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:24.569055+0000", 2024-09-18T10:16:15.748 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40018569599999998 2024-09-18T10:16:15.748 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.748 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.748 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:15.749 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:24.569156+0000", 2024-09-18T10:16:15.749 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000100738 2024-09-18T10:16:15.749 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:15.749 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:15.749 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:15.749 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.749 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.749 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:564 3.19s0 3:9fc2eacb:::smithi18925896-17:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e68)", 2024-09-18T10:16:15.749 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:23.956428+0000", 2024-09-18T10:16:15.749 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 51.677548238, 2024-09-18T10:16:15.749 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.91697059800000003, 2024-09-18T10:16:15.749 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:15.749 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:16:15.749 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:16:15.750 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:16:15.750 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:15.750 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 564 2024-09-18T10:16:15.750 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.750 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:15.750 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.750 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:15.750 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956428+0000", 2024-09-18T10:16:15.750 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:15.750 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.750 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.750 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:15.750 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956428+0000", 2024-09-18T10:16:15.750 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:15.751 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.751 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.751 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:15.751 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956427+0000", 2024-09-18T10:16:15.751 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:16:15.751 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.751 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.751 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:15.751 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956432+0000", 2024-09-18T10:16:15.751 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9980000000000003e-06 2024-09-18T10:16:15.751 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.751 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.751 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:15.751 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956446+0000", 2024-09-18T10:16:15.752 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3602e-05 2024-09-18T10:16:15.752 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.752 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.752 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:15.752 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956454+0000", 2024-09-18T10:16:15.752 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.2369999999999992e-06 2024-09-18T10:16:15.752 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.752 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.752 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:15.752 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956551+0000", 2024-09-18T10:16:15.752 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.7424999999999996e-05 2024-09-18T10:16:15.752 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.752 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.752 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting_for_map not empty", 2024-09-18T10:16:15.753 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:23.956558+0000", 2024-09-18T10:16:15.753 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.8050000000000001e-06 2024-09-18T10:16:15.753 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.753 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.753 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:15.753 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:24.168428+0000", 2024-09-18T10:16:15.753 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.21186939299999999 2024-09-18T10:16:15.753 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.753 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.753 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for rw locks", 2024-09-18T10:16:15.753 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:24.168531+0000", 2024-09-18T10:16:15.753 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000103707 2024-09-18T10:16:15.753 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.753 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.754 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:15.754 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:24.669928+0000", 2024-09-18T10:16:15.754 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.50139628999999997 2024-09-18T10:16:15.754 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.754 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.754 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:16:15.754 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:24.670206+0000", 2024-09-18T10:16:15.754 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00027815000000000001 2024-09-18T10:16:15.754 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:15.754 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:15.754 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:15.754 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:24.873399+0000", 2024-09-18T10:16:15.754 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20319285200000001 2024-09-18T10:16:15.754 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:15.755 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:15.755 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:15.755 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:15.755 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:15.755 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-18T10:16:15.756 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-09-18T10:16:15.861 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-18T10:16:15.861 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-18T10:16:15.861 INFO:teuthology.orchestra.run.smithi189.stdout: "num_ops": 0 2024-09-18T10:16:15.861 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-18T10:16:15.871 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-09-18T10:16:15.973 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-18T10:16:15.973 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-18T10:16:15.973 INFO:teuthology.orchestra.run.smithi189.stdout: "complaint_time": 30, 2024-09-18T10:16:15.974 INFO:teuthology.orchestra.run.smithi189.stdout: "num_blocked_ops": 0 2024-09-18T10:16:15.974 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-18T10:16:15.975 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-18T10:16:15.982 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-09-18T10:16:16.084 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-18T10:16:16.084 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 20, 2024-09-18T10:16:16.085 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 600, 2024-09-18T10:16:16.085 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [ 2024-09-18T10:16:16.085 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.085 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:146 3.es0 3:78aad6f5:::smithi18925896-49:head [append 0~442368,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:16:16.085 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:51.117496+0000", 2024-09-18T10:16:16.085 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 84.962848213000001, 2024-09-18T10:16:16.085 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.61437150200000001, 2024-09-18T10:16:16.086 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:16.086 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:16.086 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:16:16.086 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:16:16.086 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:16.086 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 146 2024-09-18T10:16:16.086 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.086 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:16.086 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.086 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:16.086 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.117496+0000", 2024-09-18T10:16:16.086 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:16.086 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.087 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.087 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:16.087 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.117496+0000", 2024-09-18T10:16:16.087 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:16.087 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.087 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.087 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:16.087 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.117493+0000", 2024-09-18T10:16:16.087 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999976 2024-09-18T10:16:16.087 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.087 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.087 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:16.087 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.117787+0000", 2024-09-18T10:16:16.087 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00029378100000000001 2024-09-18T10:16:16.088 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.088 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.088 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:16.088 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.117820+0000", 2024-09-18T10:16:16.088 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3073e-05 2024-09-18T10:16:16.088 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.088 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.088 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:16.088 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.117834+0000", 2024-09-18T10:16:16.088 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3369e-05 2024-09-18T10:16:16.088 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.088 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.088 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:16.089 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.127090+0000", 2024-09-18T10:16:16.089 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0092563769999999997 2024-09-18T10:16:16.089 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.089 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.089 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:16:16.089 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.127435+0000", 2024-09-18T10:16:16.089 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00034522300000000003 2024-09-18T10:16:16.089 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.089 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.089 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:16:16.089 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.129115+0000", 2024-09-18T10:16:16.089 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0016792980000000001 2024-09-18T10:16:16.089 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.089 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.090 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:16:16.090 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.430505+0000", 2024-09-18T10:16:16.090 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.30139002300000001 2024-09-18T10:16:16.090 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.090 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.090 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:16:16.090 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.731753+0000", 2024-09-18T10:16:16.090 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.30124798800000002 2024-09-18T10:16:16.090 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.090 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.090 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:16.090 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.731867+0000", 2024-09-18T10:16:16.090 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000114764 2024-09-18T10:16:16.090 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:16.091 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:16.091 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:16.091 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.091 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.091 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:147 3.es0 3:78aad6f5:::smithi18925896-49:head [append 0~376832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:16:16.091 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:51.122092+0000", 2024-09-18T10:16:16.091 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 84.958251794999995, 2024-09-18T10:16:16.091 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.61047882200000003, 2024-09-18T10:16:16.092 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:16.092 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:16.092 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:16:16.092 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:16:16.092 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:16.092 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 147 2024-09-18T10:16:16.092 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.092 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:16.093 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.093 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:16.093 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122092+0000", 2024-09-18T10:16:16.093 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:16.093 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.093 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.093 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:16.093 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122092+0000", 2024-09-18T10:16:16.093 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:16.093 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.094 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.094 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:16.094 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122091+0000", 2024-09-18T10:16:16.094 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:16:16.094 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.094 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.094 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:16.094 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122504+0000", 2024-09-18T10:16:16.095 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00041322100000000002 2024-09-18T10:16:16.095 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.095 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.095 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:16.096 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122527+0000", 2024-09-18T10:16:16.096 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3504999999999999e-05 2024-09-18T10:16:16.096 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.096 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.096 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:16.096 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122537+0000", 2024-09-18T10:16:16.096 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.8460000000000003e-06 2024-09-18T10:16:16.096 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.096 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.097 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:16.097 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.327415+0000", 2024-09-18T10:16:16.097 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20487799700000001 2024-09-18T10:16:16.097 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.097 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.097 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:16:16.097 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.327614+0000", 2024-09-18T10:16:16.097 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00019855400000000001 2024-09-18T10:16:16.097 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.097 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.098 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:16:16.098 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.328920+0000", 2024-09-18T10:16:16.098 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001306263 2024-09-18T10:16:16.098 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.098 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.098 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:16:16.098 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.430570+0000", 2024-09-18T10:16:16.098 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.101650508 2024-09-18T10:16:16.098 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.099 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.099 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:16:16.099 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.732477+0000", 2024-09-18T10:16:16.099 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.30190700100000001 2024-09-18T10:16:16.099 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.099 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.099 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:16.099 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.732571+0000", 2024-09-18T10:16:16.099 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.3580999999999997e-05 2024-09-18T10:16:16.099 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:16.100 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:16.100 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:16.100 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.100 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.100 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:148 3.es0 3:78aad6f5:::smithi18925896-49:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:16:16.100 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:51.122547+0000", 2024-09-18T10:16:16.100 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 84.957796703, 2024-09-18T10:16:16.100 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.91462763400000002, 2024-09-18T10:16:16.100 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:16.100 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:16.101 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:16:16.101 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:16:16.101 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:16.101 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 148 2024-09-18T10:16:16.101 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.101 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:16.101 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.103 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:16.103 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122547+0000", 2024-09-18T10:16:16.103 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:16.103 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.103 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.103 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:16.104 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122547+0000", 2024-09-18T10:16:16.104 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:16.104 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.104 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.104 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:16.104 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122546+0000", 2024-09-18T10:16:16.104 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:16:16.104 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.104 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.104 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:16.104 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122552+0000", 2024-09-18T10:16:16.104 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3569999999999997e-06 2024-09-18T10:16:16.104 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.105 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.105 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:16.105 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122560+0000", 2024-09-18T10:16:16.105 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.191e-06 2024-09-18T10:16:16.105 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.105 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.105 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:16.105 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122569+0000", 2024-09-18T10:16:16.105 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.8230000000000001e-06 2024-09-18T10:16:16.105 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.105 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.106 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:16.106 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.834492+0000", 2024-09-18T10:16:16.106 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.71192345599999995 2024-09-18T10:16:16.106 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.106 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.106 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:16:16.106 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.834669+0000", 2024-09-18T10:16:16.106 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000177035 2024-09-18T10:16:16.106 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.106 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.106 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:16:16.106 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.835442+0000", 2024-09-18T10:16:16.106 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00077327799999999996 2024-09-18T10:16:16.107 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.107 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.107 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:16:16.107 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.936574+0000", 2024-09-18T10:16:16.107 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.101131163 2024-09-18T10:16:16.107 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.107 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.107 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:16:16.107 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.037028+0000", 2024-09-18T10:16:16.107 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100454026 2024-09-18T10:16:16.107 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.108 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.108 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:16.108 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.037175+0000", 2024-09-18T10:16:16.108 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000147305 2024-09-18T10:16:16.108 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:16.108 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:16.108 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:16.108 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.108 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.108 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:149 3.es0 3:78aad6f5:::smithi18925896-49:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-09-18T10:16:16.108 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:51.122575+0000", 2024-09-18T10:16:16.108 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 84.957769494999994, 2024-09-18T10:16:16.108 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.016813872, 2024-09-18T10:16:16.109 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:16.109 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:16:16.109 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:16:16.109 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:16:16.109 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:16.109 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 149 2024-09-18T10:16:16.109 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.109 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:16.109 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.109 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:16.109 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122575+0000", 2024-09-18T10:16:16.109 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:16.109 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.109 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.110 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:16.110 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122575+0000", 2024-09-18T10:16:16.110 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:16.110 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.110 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.110 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:16.110 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122574+0000", 2024-09-18T10:16:16.110 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:16:16.110 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.110 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.110 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:16.110 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122579+0000", 2024-09-18T10:16:16.110 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3650000000000003e-06 2024-09-18T10:16:16.110 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.110 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.111 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:16.111 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122587+0000", 2024-09-18T10:16:16.111 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.3000000000000002e-06 2024-09-18T10:16:16.111 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.111 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.111 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:16.111 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.122592+0000", 2024-09-18T10:16:16.111 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9080000000000003e-06 2024-09-18T10:16:16.111 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.111 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.111 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:16.111 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.935931+0000", 2024-09-18T10:16:16.111 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.81333878199999998 2024-09-18T10:16:16.111 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.112 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.112 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for rw locks", 2024-09-18T10:16:16.112 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.936216+0000", 2024-09-18T10:16:16.112 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00028463400000000003 2024-09-18T10:16:16.112 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.112 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.112 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:16.112 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.038007+0000", 2024-09-18T10:16:16.112 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.101791539 2024-09-18T10:16:16.112 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.112 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.112 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:16:16.112 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.038254+0000", 2024-09-18T10:16:16.112 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000246932 2024-09-18T10:16:16.113 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.113 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.113 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:16.113 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:52.139388+0000", 2024-09-18T10:16:16.113 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10113417199999999 2024-09-18T10:16:16.113 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:16.113 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:16.113 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:16.113 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.113 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.114 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpRead(3.es0 36/32 ECSubRead(tid=3, to_read={3:7255ddf4:::smithi18925896-39:head=0,4096,0}, subchunks={3:7255ddf4:::smithi18925896-39:head=[0,1]}, attrs_to_read=))", 2024-09-18T10:16:16.114 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:51.126838+0000", 2024-09-18T10:16:16.114 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 84.953506071999996, 2024-09-18T10:16:16.114 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.707585825, 2024-09-18T10:16:16.114 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:16.114 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:16:16.114 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:16.114 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.114 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:16.114 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.126838+0000", 2024-09-18T10:16:16.115 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:16.115 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.115 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.115 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:16.116 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.126838+0000", 2024-09-18T10:16:16.116 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:16.116 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.116 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.116 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:16.117 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.126838+0000", 2024-09-18T10:16:16.117 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:16.117 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.117 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.117 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:16.117 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.126838+0000", 2024-09-18T10:16:16.117 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:16.117 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.117 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.117 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:16.117 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.127004+0000", 2024-09-18T10:16:16.117 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00016626499999999999 2024-09-18T10:16:16.117 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.118 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.118 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:16.118 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.227100+0000", 2024-09-18T10:16:16.118 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.1000955 2024-09-18T10:16:16.118 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.118 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.118 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:16.118 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.834083+0000", 2024-09-18T10:16:16.118 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.60698300100000002 2024-09-18T10:16:16.118 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.118 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.118 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:16.118 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.834424+0000", 2024-09-18T10:16:16.118 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000341059 2024-09-18T10:16:16.119 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:16.119 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:16.119 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:16.119 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.119 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.119 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpReadReply(3.es0 36/32 ECSubReadReply(tid=3, attrs_read=0))", 2024-09-18T10:16:16.119 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:51.214895+0000", 2024-09-18T10:16:16.119 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 84.865449394999999, 2024-09-18T10:16:16.119 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.61857800100000004, 2024-09-18T10:16:16.119 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:16.119 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:16:16.119 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:16.119 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.119 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:16.120 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.214895+0000", 2024-09-18T10:16:16.120 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:16.120 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.120 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.120 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:16.120 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.214895+0000", 2024-09-18T10:16:16.120 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:16.120 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.120 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.120 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:16.120 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.214893+0000", 2024-09-18T10:16:16.120 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:16:16.120 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.121 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.121 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:16.121 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.214927+0000", 2024-09-18T10:16:16.122 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3841999999999999e-05 2024-09-18T10:16:16.122 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.122 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.122 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:16.122 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.214953+0000", 2024-09-18T10:16:16.122 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.6519000000000001e-05 2024-09-18T10:16:16.122 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.122 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.122 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:16.122 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.214971+0000", 2024-09-18T10:16:16.122 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7535999999999999e-05 2024-09-18T10:16:16.122 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.123 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.123 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:16.123 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.833397+0000", 2024-09-18T10:16:16.123 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.61842642599999997 2024-09-18T10:16:16.123 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.123 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.123 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:16.123 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.833473+0000", 2024-09-18T10:16:16.123 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.5621e-05 2024-09-18T10:16:16.124 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:16.124 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:16.124 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:16.124 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.125 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.125 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpReadReply(3.es0 36/32 ECSubReadReply(tid=3, attrs_read=0))", 2024-09-18T10:16:16.125 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:51.227596+0000", 2024-09-18T10:16:16.125 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 84.852747844000007, 2024-09-18T10:16:16.125 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.60597977400000003, 2024-09-18T10:16:16.125 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:16.125 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:16:16.125 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:16.125 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.125 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:16.125 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.227596+0000", 2024-09-18T10:16:16.125 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:16.126 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.126 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.126 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:16.126 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.227596+0000", 2024-09-18T10:16:16.126 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:16.126 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.126 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.126 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:16.126 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.227595+0000", 2024-09-18T10:16:16.126 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:16:16.127 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.127 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.127 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:16.127 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.227605+0000", 2024-09-18T10:16:16.127 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.7040000000000006e-06 2024-09-18T10:16:16.127 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.127 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.127 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:16.127 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.227621+0000", 2024-09-18T10:16:16.127 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6107999999999999e-05 2024-09-18T10:16:16.127 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.128 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.128 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:16.128 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.227637+0000", 2024-09-18T10:16:16.128 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5282999999999999e-05 2024-09-18T10:16:16.128 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.128 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.128 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:16.128 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.833511+0000", 2024-09-18T10:16:16.128 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.60587403699999998 2024-09-18T10:16:16.128 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.128 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.129 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:16.129 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.833576+0000", 2024-09-18T10:16:16.129 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.5392e-05 2024-09-18T10:16:16.129 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:16.129 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:16.129 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:16.129 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.129 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.129 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.es0 36/32 ECSubWriteReply(tid=4, last_complete=36'3, committed=1, applied=1))", 2024-09-18T10:16:16.129 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:51.233227+0000", 2024-09-18T10:16:16.129 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 84.847116830000004, 2024-09-18T10:16:16.129 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.39820466900000001, 2024-09-18T10:16:16.130 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:16.130 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:16:16.130 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:16.130 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.130 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:16.130 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.233227+0000", 2024-09-18T10:16:16.130 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:16.130 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.130 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.130 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:16.130 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.233227+0000", 2024-09-18T10:16:16.131 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:16.131 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.131 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.131 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:16.131 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.233226+0000", 2024-09-18T10:16:16.131 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:16:16.131 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.131 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.131 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:16.132 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.233232+0000", 2024-09-18T10:16:16.132 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9370000000000001e-06 2024-09-18T10:16:16.132 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.132 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.132 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:16.132 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.233248+0000", 2024-09-18T10:16:16.132 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5574999999999999e-05 2024-09-18T10:16:16.132 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.132 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.132 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:16.132 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.233254+0000", 2024-09-18T10:16:16.133 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.3659999999999997e-06 2024-09-18T10:16:16.133 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.133 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.133 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:16.133 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.631420+0000", 2024-09-18T10:16:16.133 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.39816591699999998 2024-09-18T10:16:16.133 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.133 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.133 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:16.133 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.631432+0000", 2024-09-18T10:16:16.133 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1737e-05 2024-09-18T10:16:16.134 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:16.134 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:16.134 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:16.134 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.134 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.134 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.es0 36/32 ECSubWriteReply(tid=5, last_complete=36'4, committed=1, applied=1))", 2024-09-18T10:16:16.134 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:51.331048+0000", 2024-09-18T10:16:16.134 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 84.749296403000002, 2024-09-18T10:16:16.134 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40123123900000002, 2024-09-18T10:16:16.134 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:16.134 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:16:16.135 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:16.135 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.135 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:16.135 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.331048+0000", 2024-09-18T10:16:16.135 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:16.135 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.135 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.135 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:16.135 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.331048+0000", 2024-09-18T10:16:16.135 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:16.135 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.136 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.136 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:16.136 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.331046+0000", 2024-09-18T10:16:16.136 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:16:16.136 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.136 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.136 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:16.136 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.331057+0000", 2024-09-18T10:16:16.136 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1683e-05 2024-09-18T10:16:16.136 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.137 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.137 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:16.137 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.331090+0000", 2024-09-18T10:16:16.137 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2576000000000001e-05 2024-09-18T10:16:16.137 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.137 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.137 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:16.137 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.331107+0000", 2024-09-18T10:16:16.137 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7382999999999999e-05 2024-09-18T10:16:16.137 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.137 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.138 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:16.138 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.732241+0000", 2024-09-18T10:16:16.138 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40113375200000001 2024-09-18T10:16:16.138 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.138 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.138 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:16.138 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.732279+0000", 2024-09-18T10:16:16.138 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7719000000000003e-05 2024-09-18T10:16:16.138 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:16.138 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:16.138 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:16.138 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.139 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.139 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWriteReply(3.es0 36/32 ECSubWriteReply(tid=5, last_complete=36'4, committed=1, applied=1))", 2024-09-18T10:16:16.139 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:51.331214+0000", 2024-09-18T10:16:16.139 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 84.749130008999998, 2024-09-18T10:16:16.139 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40084109800000001, 2024-09-18T10:16:16.139 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:16.139 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:16:16.139 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:16.139 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.139 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:16.139 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.331214+0000", 2024-09-18T10:16:16.139 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:16.140 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.140 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.140 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:16.140 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.331214+0000", 2024-09-18T10:16:16.140 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:16.140 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.140 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.140 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:16.140 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.331213+0000", 2024-09-18T10:16:16.140 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:16:16.140 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.141 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.141 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:16.141 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.331220+0000", 2024-09-18T10:16:16.141 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.1369999999999996e-06 2024-09-18T10:16:16.141 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.141 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.141 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:16.141 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.331235+0000", 2024-09-18T10:16:16.141 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4849e-05 2024-09-18T10:16:16.141 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.141 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.141 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:16.142 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.331243+0000", 2024-09-18T10:16:16.142 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.157e-06 2024-09-18T10:16:16.142 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.142 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.142 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:16.142 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.732010+0000", 2024-09-18T10:16:16.142 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40076711300000001 2024-09-18T10:16:16.142 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.142 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.142 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:16.142 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:51.732055+0000", 2024-09-18T10:16:16.142 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4821999999999999e-05 2024-09-18T10:16:16.143 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:16.143 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:16.143 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:16.143 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.143 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.143 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:347 3.es0 3:7255ddf4:::smithi18925896-39:head [append 0~376832] snapc b=[b,4] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-18T10:16:16.143 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:02.898980+0000", 2024-09-18T10:16:16.143 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 73.181363985000004, 2024-09-18T10:16:16.143 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.50845237799999998, 2024-09-18T10:16:16.143 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:16.143 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:16.143 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:16:16.144 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:16:16.144 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:16.144 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 347 2024-09-18T10:16:16.144 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.144 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:16.144 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.144 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:16.144 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.898980+0000", 2024-09-18T10:16:16.144 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:16.144 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.144 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.145 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:16.145 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.898980+0000", 2024-09-18T10:16:16.145 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:16.145 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.145 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.145 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:16.145 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.898978+0000", 2024-09-18T10:16:16.145 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:16:16.145 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.145 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.145 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:16.145 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.899378+0000", 2024-09-18T10:16:16.146 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000399465 2024-09-18T10:16:16.146 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.146 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.146 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:16.146 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.899397+0000", 2024-09-18T10:16:16.146 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9219999999999999e-05 2024-09-18T10:16:16.146 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.146 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.146 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:16.146 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.899410+0000", 2024-09-18T10:16:16.146 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2486e-05 2024-09-18T10:16:16.146 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.147 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.147 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:16.147 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.103248+0000", 2024-09-18T10:16:16.147 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20383807200000001 2024-09-18T10:16:16.147 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.147 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.147 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:16:16.147 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.103447+0000", 2024-09-18T10:16:16.147 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00019970400000000001 2024-09-18T10:16:16.147 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.147 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.148 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:16:16.148 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.104894+0000", 2024-09-18T10:16:16.148 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001447083 2024-09-18T10:16:16.148 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.148 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.148 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:16:16.148 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.209378+0000", 2024-09-18T10:16:16.148 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.104483942 2024-09-18T10:16:16.148 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.148 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.148 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:16:16.148 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.407310+0000", 2024-09-18T10:16:16.149 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.197931406 2024-09-18T10:16:16.149 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.149 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.149 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:16.149 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.407432+0000", 2024-09-18T10:16:16.149 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00012266999999999999 2024-09-18T10:16:16.149 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:16.149 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:16.149 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:16.149 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.149 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.149 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:348 3.es0 3:7255ddf4:::smithi18925896-39:head [setxattr _header (56) in=7b] snapc b=[b,4] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-18T10:16:16.150 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:02.899423+0000", 2024-09-18T10:16:16.150 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 73.180920556999993, 2024-09-18T10:16:16.150 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.50859120400000002, 2024-09-18T10:16:16.150 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:16.150 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:16.150 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:16:16.150 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:16:16.150 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:16.150 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 348 2024-09-18T10:16:16.150 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.150 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:16.151 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.151 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:16.151 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.899423+0000", 2024-09-18T10:16:16.151 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:16.151 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.151 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.151 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:16.151 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.899423+0000", 2024-09-18T10:16:16.151 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:16.151 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.151 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.151 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:16.152 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.899422+0000", 2024-09-18T10:16:16.152 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:16:16.152 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.152 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.152 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:16.152 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.899431+0000", 2024-09-18T10:16:16.152 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.5960000000000004e-06 2024-09-18T10:16:16.152 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.152 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.152 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:16.153 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.899451+0000", 2024-09-18T10:16:16.153 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0089999999999999e-05 2024-09-18T10:16:16.153 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.153 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.153 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:16.153 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.899461+0000", 2024-09-18T10:16:16.153 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0373e-05 2024-09-18T10:16:16.153 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.153 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.153 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:16.153 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.106191+0000", 2024-09-18T10:16:16.154 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20672991700000001 2024-09-18T10:16:16.154 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.154 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.154 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:16:16.154 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.106370+0000", 2024-09-18T10:16:16.154 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00017880500000000001 2024-09-18T10:16:16.154 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.154 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.154 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:16:16.154 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.107175+0000", 2024-09-18T10:16:16.154 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00080489699999999995 2024-09-18T10:16:16.154 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.155 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.155 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:16:16.155 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.209383+0000", 2024-09-18T10:16:16.155 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10220804999999999 2024-09-18T10:16:16.155 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.155 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.155 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:16:16.155 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.407801+0000", 2024-09-18T10:16:16.155 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.19841824199999999 2024-09-18T10:16:16.155 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.155 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.156 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:16.156 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.408015+0000", 2024-09-18T10:16:16.156 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00021364100000000001 2024-09-18T10:16:16.156 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:16.156 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:16.156 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:16.156 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.156 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.156 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:349 3.es0 3:7255ddf4:::smithi18925896-39:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e47)", 2024-09-18T10:16:16.156 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:02.899474+0000", 2024-09-18T10:16:16.156 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 73.180870346000006, 2024-09-18T10:16:16.157 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.81294787599999996, 2024-09-18T10:16:16.157 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:16.157 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-18T10:16:16.157 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:16:16.157 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:16:16.157 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:16.157 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 349 2024-09-18T10:16:16.157 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.157 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:16.157 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.157 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:16.157 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.899474+0000", 2024-09-18T10:16:16.157 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:16.158 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.158 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.158 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:16.158 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.899474+0000", 2024-09-18T10:16:16.158 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:16.158 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.158 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.158 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:16.158 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.899472+0000", 2024-09-18T10:16:16.158 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:16:16.158 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.159 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.159 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:16.159 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.899480+0000", 2024-09-18T10:16:16.159 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.0129999999999993e-06 2024-09-18T10:16:16.159 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.159 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.159 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:16.159 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.899501+0000", 2024-09-18T10:16:16.159 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0959999999999999e-05 2024-09-18T10:16:16.159 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.159 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.159 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:16.160 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.899513+0000", 2024-09-18T10:16:16.160 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1736000000000001e-05 2024-09-18T10:16:16.160 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.160 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.160 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:16.160 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.208436+0000", 2024-09-18T10:16:16.160 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.30892317800000002 2024-09-18T10:16:16.160 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.160 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.160 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for rw locks", 2024-09-18T10:16:16.160 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.208762+0000", 2024-09-18T10:16:16.160 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00032566099999999998 2024-09-18T10:16:16.161 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.161 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.161 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:16.161 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.408786+0000", 2024-09-18T10:16:16.161 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.200024389 2024-09-18T10:16:16.161 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.161 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.161 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:16:16.161 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.409088+0000", 2024-09-18T10:16:16.161 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000301272 2024-09-18T10:16:16.161 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.161 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.162 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:16.162 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.712422+0000", 2024-09-18T10:16:16.162 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.303333937 2024-09-18T10:16:16.162 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:16.162 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:16.162 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:16.162 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.162 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.162 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.17s1 57/40 ECSubWrite(tid=106, reqid=client.4632.0:441, at_version=57'22, trim_to=0'0, roll_forward_to=53'18))", 2024-09-18T10:16:16.162 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:12.963189+0000", 2024-09-18T10:16:16.162 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 63.117154630000002, 2024-09-18T10:16:16.162 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.404840753, 2024-09-18T10:16:16.163 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:16.163 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:16:16.163 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:16.163 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.163 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:16.163 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:12.963189+0000", 2024-09-18T10:16:16.163 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:16.163 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.163 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.163 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:16.163 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:12.963189+0000", 2024-09-18T10:16:16.163 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:16.163 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.164 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.164 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:16.164 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:12.963189+0000", 2024-09-18T10:16:16.164 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:16:16.164 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.164 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.164 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:16.164 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:12.963511+0000", 2024-09-18T10:16:16.164 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00032195800000000001 2024-09-18T10:16:16.164 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.164 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.164 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:16.165 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:12.963536+0000", 2024-09-18T10:16:16.165 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5795e-05 2024-09-18T10:16:16.165 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.165 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.165 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:16.165 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:12.963548+0000", 2024-09-18T10:16:16.165 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1235e-05 2024-09-18T10:16:16.165 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.165 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.165 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:16.165 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.164194+0000", 2024-09-18T10:16:16.165 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.200646044 2024-09-18T10:16:16.166 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.166 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.166 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:16:16.166 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.164208+0000", 2024-09-18T10:16:16.166 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4107000000000001e-05 2024-09-18T10:16:16.166 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.166 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.166 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:16:16.166 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.367962+0000", 2024-09-18T10:16:16.166 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20375375800000001 2024-09-18T10:16:16.166 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.166 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.167 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:16.167 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.368030+0000", 2024-09-18T10:16:16.167 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.8664999999999994e-05 2024-09-18T10:16:16.167 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:16.167 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:16.167 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:16.167 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.167 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.167 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.17s1 57/40 ECSubWrite(tid=107, reqid=client.4632.0:442, at_version=57'23, trim_to=0'0, roll_forward_to=53'18))", 2024-09-18T10:16:16.167 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:12.963562+0000", 2024-09-18T10:16:16.167 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 63.116781910999997, 2024-09-18T10:16:16.168 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40594254800000001, 2024-09-18T10:16:16.168 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:16.168 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:16:16.168 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:16.168 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.168 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:16.168 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:12.963562+0000", 2024-09-18T10:16:16.168 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:16.168 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.168 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.168 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:16.169 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:12.963562+0000", 2024-09-18T10:16:16.169 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:16.169 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.169 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.169 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:16.169 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:12.963561+0000", 2024-09-18T10:16:16.169 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999995 2024-09-18T10:16:16.169 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.169 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.169 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:16.169 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:12.963869+0000", 2024-09-18T10:16:16.169 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00030785799999999999 2024-09-18T10:16:16.169 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.170 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.170 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:16.170 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:12.963886+0000", 2024-09-18T10:16:16.170 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6582000000000001e-05 2024-09-18T10:16:16.170 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.170 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.170 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:16.170 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.063959+0000", 2024-09-18T10:16:16.170 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10007302699999999 2024-09-18T10:16:16.170 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.170 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.170 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:16.171 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.166049+0000", 2024-09-18T10:16:16.171 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.102089845 2024-09-18T10:16:16.171 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.171 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.171 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:16:16.171 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.166064+0000", 2024-09-18T10:16:16.171 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5287000000000001e-05 2024-09-18T10:16:16.171 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.171 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.171 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:16:16.171 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.369452+0000", 2024-09-18T10:16:16.171 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20338777 2024-09-18T10:16:16.172 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.172 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.172 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:16.172 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:13.369505+0000", 2024-09-18T10:16:16.172 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.2868000000000003e-05 2024-09-18T10:16:16.172 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:16.172 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:16.172 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:16.172 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.172 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.172 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:495 3.es0 3:7255ddf4:::smithi18925896-39:head [rmxattr R^`al]TqifatYOiOU]p`ZegfXmNsXVaZcdd]b]YX in=40b,rmxattr TNQYR^jgR`pPaYitmrcliV[dttobncTntcfTlQkt in=40b,rmxattr W_aq[RVsaUpQtX_RpV^Vpqrg`QgQViPeVaRqZSmf in=40b,rmxattr aYfgr^q[\\aPg_aZsakfUfsdhYbra^QXjlkrpdY\\\\ in=40b,rmxattr dgXPi in=5b,rmxattr p]OS_Znl[dNa^qTn`WW[j\\dtgbNac_bijYmg in=36b,rmxattr tZ]grhlaq]kUO[k^d^dr``XNio^^Oaebpj\\[Ytdq in=40b] snapc 19=[19,18,16,12,11,10,f,e] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T10:16:16.172 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:17.889335+0000", 2024-09-18T10:16:16.173 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 58.191008678000003, 2024-09-18T10:16:16.173 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.42846536899999998, 2024-09-18T10:16:16.173 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:16.173 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:16.173 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:16:16.173 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:16:16.173 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:16.173 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 495 2024-09-18T10:16:16.173 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.173 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:16.173 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.173 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:16.174 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889335+0000", 2024-09-18T10:16:16.174 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:16.174 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.174 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.174 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:16.174 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889335+0000", 2024-09-18T10:16:16.174 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:16.174 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.174 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.174 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:16.174 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889333+0000", 2024-09-18T10:16:16.174 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999976 2024-09-18T10:16:16.175 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.175 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.175 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:16.175 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889351+0000", 2024-09-18T10:16:16.175 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7795000000000002e-05 2024-09-18T10:16:16.175 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.175 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.175 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:16.175 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889383+0000", 2024-09-18T10:16:16.175 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2740999999999997e-05 2024-09-18T10:16:16.175 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.175 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.176 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:16.176 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.889396+0000", 2024-09-18T10:16:16.176 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2926e-05 2024-09-18T10:16:16.176 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.176 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.176 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:16.176 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.989725+0000", 2024-09-18T10:16:16.176 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10032845999999999 2024-09-18T10:16:16.176 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.176 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.176 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "op must wait for map", 2024-09-18T10:16:16.176 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.989751+0000", 2024-09-18T10:16:16.177 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5819000000000001e-05 2024-09-18T10:16:16.177 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.177 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.177 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:16.177 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.999619+0000", 2024-09-18T10:16:16.177 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0098683920000000001 2024-09-18T10:16:16.177 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.177 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.177 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:16:16.177 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:17.999776+0000", 2024-09-18T10:16:16.177 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000157336 2024-09-18T10:16:16.178 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.178 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.178 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:16:16.178 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.001374+0000", 2024-09-18T10:16:16.178 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0015982419999999999 2024-09-18T10:16:16.178 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.178 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.178 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:16:16.178 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.003908+0000", 2024-09-18T10:16:16.178 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0025331870000000001 2024-09-18T10:16:16.178 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.178 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.179 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:16:16.179 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.317708+0000", 2024-09-18T10:16:16.179 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.31380064800000002 2024-09-18T10:16:16.179 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.179 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.179 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:16.179 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:18.317801+0000", 2024-09-18T10:16:16.179 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.2427000000000001e-05 2024-09-18T10:16:16.179 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:16.179 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:16.179 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:16.179 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.180 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.180 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:646 3.es0 3:78aad6f5:::smithi18925896-49:head [append 0~0] snapc 21=[20,1f,1e,1a,16,11,10] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-18T10:16:16.180 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:29.784316+0000", 2024-09-18T10:16:16.180 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 46.296027596000002, 2024-09-18T10:16:16.180 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.38893830699999998, 2024-09-18T10:16:16.180 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:16.180 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:16.180 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:16:16.180 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:16:16.180 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:16.181 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 646 2024-09-18T10:16:16.181 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.181 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:16.181 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.181 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:16.181 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:29.784316+0000", 2024-09-18T10:16:16.181 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:16.181 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.181 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.181 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:16.181 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:29.784316+0000", 2024-09-18T10:16:16.182 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:16.182 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.182 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.182 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:16.182 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:29.784314+0000", 2024-09-18T10:16:16.182 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999976 2024-09-18T10:16:16.182 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.182 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.182 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:16.182 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:29.784325+0000", 2024-09-18T10:16:16.183 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0869e-05 2024-09-18T10:16:16.183 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.183 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.183 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:16.183 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:29.784359+0000", 2024-09-18T10:16:16.183 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4496000000000003e-05 2024-09-18T10:16:16.183 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.183 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.183 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:16.183 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:29.784374+0000", 2024-09-18T10:16:16.184 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4945e-05 2024-09-18T10:16:16.184 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.184 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.184 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:16.184 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:29.870174+0000", 2024-09-18T10:16:16.184 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.085799399999999998 2024-09-18T10:16:16.184 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.184 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.184 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:16:16.184 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:29.870387+0000", 2024-09-18T10:16:16.184 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00021327799999999999 2024-09-18T10:16:16.185 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.185 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.185 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:16:16.185 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:29.871229+0000", 2024-09-18T10:16:16.185 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00084239500000000001 2024-09-18T10:16:16.185 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.185 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.185 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:16:16.185 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.172680+0000", 2024-09-18T10:16:16.185 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.301450944 2024-09-18T10:16:16.185 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.185 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.186 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:16:16.186 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.173021+0000", 2024-09-18T10:16:16.186 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00034057199999999998 2024-09-18T10:16:16.186 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.186 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.186 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:16.186 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.173255+0000", 2024-09-18T10:16:16.186 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000233945 2024-09-18T10:16:16.186 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:16.186 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:16.186 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:16.187 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.187 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.187 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:647 3.es0 3:78aad6f5:::smithi18925896-49:head [rollback 10] snapc 21=[20,1f,1e,1a,16,11,10] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-18T10:16:16.187 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:29.784393+0000", 2024-09-18T10:16:16.187 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 46.295950683000001, 2024-09-18T10:16:16.187 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.38955653200000001, 2024-09-18T10:16:16.187 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:16.187 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:16.187 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:16:16.187 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:16:16.188 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:16.188 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 647 2024-09-18T10:16:16.188 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.188 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:16.188 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.188 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:16.188 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:29.784393+0000", 2024-09-18T10:16:16.188 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:16.188 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.188 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.188 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:16.189 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:29.784393+0000", 2024-09-18T10:16:16.189 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:16.189 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.189 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.189 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:16.189 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:29.784393+0000", 2024-09-18T10:16:16.189 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:16:16.189 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.189 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.189 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:16.189 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:29.784397+0000", 2024-09-18T10:16:16.189 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4290000000000003e-06 2024-09-18T10:16:16.190 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.190 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.190 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:16.190 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:29.784409+0000", 2024-09-18T10:16:16.190 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2208000000000001e-05 2024-09-18T10:16:16.190 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.190 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.190 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:16.190 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:29.784418+0000", 2024-09-18T10:16:16.190 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.1060000000000006e-06 2024-09-18T10:16:16.190 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.190 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.191 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:16.191 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:29.871905+0000", 2024-09-18T10:16:16.191 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.087486918999999996 2024-09-18T10:16:16.191 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.191 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.191 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:16:16.191 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:29.872024+0000", 2024-09-18T10:16:16.191 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000119152 2024-09-18T10:16:16.191 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.191 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.191 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:16:16.192 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:29.872743+0000", 2024-09-18T10:16:16.192 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00071839500000000004 2024-09-18T10:16:16.192 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.192 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.192 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:16:16.192 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.172683+0000", 2024-09-18T10:16:16.192 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.29994042399999998 2024-09-18T10:16:16.192 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.192 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.192 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:16:16.192 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.173719+0000", 2024-09-18T10:16:16.193 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0010353199999999999 2024-09-18T10:16:16.193 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.193 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.193 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:16.193 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.173950+0000", 2024-09-18T10:16:16.193 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00023138499999999999 2024-09-18T10:16:16.193 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:16.193 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:16.193 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:16.193 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.193 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.194 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:648 3.es0 3:78aad6f5:::smithi18925896-49:head [append 0~0] snapc 21=[20,1f,1e,1a,16,11,10] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-18T10:16:16.194 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:29.784506+0000", 2024-09-18T10:16:16.194 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 46.295838019000001, 2024-09-18T10:16:16.194 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.79270480700000001, 2024-09-18T10:16:16.194 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:16.194 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:16.194 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:16:16.194 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:16:16.194 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:16.194 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 648 2024-09-18T10:16:16.194 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.194 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:16.195 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.195 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:16.195 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:29.784506+0000", 2024-09-18T10:16:16.195 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:16.195 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.195 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.195 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:16.195 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:29.784506+0000", 2024-09-18T10:16:16.195 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:16.195 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.195 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.195 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:16.196 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:29.784505+0000", 2024-09-18T10:16:16.196 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:16:16.196 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.196 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.196 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:16.196 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:29.784510+0000", 2024-09-18T10:16:16.196 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9540000000000003e-06 2024-09-18T10:16:16.196 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.196 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.196 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:16.196 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:29.784523+0000", 2024-09-18T10:16:16.196 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2791e-05 2024-09-18T10:16:16.197 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.197 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.197 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:16.197 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:29.884663+0000", 2024-09-18T10:16:16.197 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10014081499999999 2024-09-18T10:16:16.197 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.197 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.197 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:16.197 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.175196+0000", 2024-09-18T10:16:16.198 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.29053214300000002 2024-09-18T10:16:16.198 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.198 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.198 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:16:16.198 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.175436+0000", 2024-09-18T10:16:16.198 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000240084 2024-09-18T10:16:16.198 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.198 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.198 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:16:16.198 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.176117+0000", 2024-09-18T10:16:16.198 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00068175400000000002 2024-09-18T10:16:16.199 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.199 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.199 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:16:16.199 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.277609+0000", 2024-09-18T10:16:16.199 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10149114100000001 2024-09-18T10:16:16.199 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.199 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.199 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:16:16.199 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.577146+0000", 2024-09-18T10:16:16.199 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.29953759600000002 2024-09-18T10:16:16.199 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.200 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.200 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:16.200 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.577211+0000", 2024-09-18T10:16:16.200 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.4630999999999998e-05 2024-09-18T10:16:16.200 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:16.200 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:16.200 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:16.200 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.200 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.200 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:694 3.es0 3:78aad6f5:::smithi18925896-49:head [stat,delete] snapc 2a=[2a,29,28,26,20,1f,1e,1a,11] ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-09-18T10:16:16.200 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:35.579838+0000", 2024-09-18T10:16:16.201 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 40.500506164000001, 2024-09-18T10:16:16.201 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40589915300000001, 2024-09-18T10:16:16.201 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:16.201 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:16.201 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:16:16.201 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:16:16.201 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:16.201 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 694 2024-09-18T10:16:16.201 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.201 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:16.201 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.202 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:16.202 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:35.579838+0000", 2024-09-18T10:16:16.202 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:16.202 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.202 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.202 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:16.202 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:35.579838+0000", 2024-09-18T10:16:16.202 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:16.202 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.202 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.202 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:16.202 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:35.579835+0000", 2024-09-18T10:16:16.202 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999971 2024-09-18T10:16:16.203 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.203 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.203 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:16.203 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:35.579849+0000", 2024-09-18T10:16:16.203 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3786e-05 2024-09-18T10:16:16.203 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.203 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.203 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:16.203 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:35.579886+0000", 2024-09-18T10:16:16.203 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7183e-05 2024-09-18T10:16:16.203 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.203 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.204 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:16.204 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:35.680021+0000", 2024-09-18T10:16:16.204 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10013464599999999 2024-09-18T10:16:16.204 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.204 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.204 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:16.204 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:35.686371+0000", 2024-09-18T10:16:16.204 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0063500789999999998 2024-09-18T10:16:16.204 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.204 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.204 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "op must wait for map", 2024-09-18T10:16:16.204 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:35.686388+0000", 2024-09-18T10:16:16.205 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7662000000000001e-05 2024-09-18T10:16:16.205 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.205 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.205 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:16.205 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:35.880602+0000", 2024-09-18T10:16:16.205 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.19421374999999999 2024-09-18T10:16:16.205 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.205 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.205 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:16:16.205 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:35.880897+0000", 2024-09-18T10:16:16.205 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00029453999999999998 2024-09-18T10:16:16.205 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.206 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.206 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:16:16.206 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:35.882269+0000", 2024-09-18T10:16:16.206 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00137261 2024-09-18T10:16:16.206 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.206 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.206 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:16:16.206 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:35.884030+0000", 2024-09-18T10:16:16.206 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0017602449999999999 2024-09-18T10:16:16.206 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.206 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.206 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:16:16.207 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:35.985612+0000", 2024-09-18T10:16:16.207 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.101582165 2024-09-18T10:16:16.207 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.207 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.207 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:16.207 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:35.985737+0000", 2024-09-18T10:16:16.207 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00012524699999999999 2024-09-18T10:16:16.207 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:16.207 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:16.207 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:16.207 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:16.207 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:16.208 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-18T10:16:16.209 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_ops_in_flight 2024-09-18T10:16:16.316 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-18T10:16:16.316 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-18T10:16:16.316 INFO:teuthology.orchestra.run.smithi189.stdout: "num_ops": 0 2024-09-18T10:16:16.317 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-18T10:16:16.326 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_blocked_ops 2024-09-18T10:16:16.424 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-18T10:16:16.425 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-18T10:16:16.425 INFO:teuthology.orchestra.run.smithi189.stdout: "complaint_time": 30, 2024-09-18T10:16:16.425 INFO:teuthology.orchestra.run.smithi189.stdout: "num_blocked_ops": 0 2024-09-18T10:16:16.425 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-18T10:16:16.434 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_historic_ops 2024-09-18T10:16:16.539 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-18T10:16:16.539 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 20, 2024-09-18T10:16:16.539 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 600, 2024-09-18T10:16:16.539 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [ 2024-09-18T10:16:16.539 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.539 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4257.0:19 1.0 e24/23 1:30306672:devicehealth::main.db.0000000000000000:head v 24'9, mlcod=24'7)", 2024-09-18T10:16:16.540 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:33.346192+0000", 2024-09-18T10:16:16.540 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 103.189517579, 2024-09-18T10:16:16.540 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20178458699999999, 2024-09-18T10:16:16.540 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:16.540 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:16.540 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:16.540 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.540 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:16.540 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:33.346192+0000", 2024-09-18T10:16:16.540 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:16.540 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.540 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.540 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:16.541 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:33.346192+0000", 2024-09-18T10:16:16.541 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:16.541 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.541 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.541 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:16.541 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:33.346190+0000", 2024-09-18T10:16:16.541 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:16:16.541 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.541 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.541 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:16.541 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:33.346206+0000", 2024-09-18T10:16:16.541 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6257e-05 2024-09-18T10:16:16.541 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.542 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.542 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:16.542 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:33.346237+0000", 2024-09-18T10:16:16.542 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0471999999999999e-05 2024-09-18T10:16:16.542 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.542 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.542 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:16.542 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:33.446377+0000", 2024-09-18T10:16:16.542 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10014044499999999 2024-09-18T10:16:16.542 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.542 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.542 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:16.542 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:33.546722+0000", 2024-09-18T10:16:16.542 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100344593 2024-09-18T10:16:16.543 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.543 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.543 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:16:16.543 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:33.546787+0000", 2024-09-18T10:16:16.543 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.5347999999999998e-05 2024-09-18T10:16:16.543 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.543 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.543 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:16:16.543 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:33.547891+0000", 2024-09-18T10:16:16.543 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001104379 2024-09-18T10:16:16.543 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.543 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.543 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:16.543 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:33.547976+0000", 2024-09-18T10:16:16.544 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.5023000000000002e-05 2024-09-18T10:16:16.544 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:16.544 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:16.544 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:16.544 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.544 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.544 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4257.0:48 1.0 e26/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 26'30, mlcod=26'22)", 2024-09-18T10:16:16.544 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:35.294144+0000", 2024-09-18T10:16:16.544 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 101.241565436, 2024-09-18T10:16:16.544 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20346468400000001, 2024-09-18T10:16:16.544 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:16.544 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:16.544 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:16.544 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.545 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:16.545 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:35.294144+0000", 2024-09-18T10:16:16.545 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:16.545 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.545 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.545 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:16.545 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:35.294144+0000", 2024-09-18T10:16:16.545 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:16.545 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.545 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.545 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:16.545 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:35.294142+0000", 2024-09-18T10:16:16.545 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:16:16.546 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.546 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.546 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:16.546 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:35.294156+0000", 2024-09-18T10:16:16.546 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4059e-05 2024-09-18T10:16:16.546 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.546 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.546 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:16.546 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:35.294209+0000", 2024-09-18T10:16:16.546 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.2164999999999999e-05 2024-09-18T10:16:16.546 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.546 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.546 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:16.546 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:35.394305+0000", 2024-09-18T10:16:16.547 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10009605000000001 2024-09-18T10:16:16.547 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.547 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.547 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:16.547 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:35.394422+0000", 2024-09-18T10:16:16.547 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000117093 2024-09-18T10:16:16.547 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.547 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.547 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:16:16.547 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:35.394460+0000", 2024-09-18T10:16:16.547 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8037999999999997e-05 2024-09-18T10:16:16.547 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.547 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.547 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:16:16.548 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:35.497437+0000", 2024-09-18T10:16:16.548 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.102977206 2024-09-18T10:16:16.548 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.548 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.548 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:16.548 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:35.497609+0000", 2024-09-18T10:16:16.548 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00017165299999999999 2024-09-18T10:16:16.548 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:16.548 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:16.548 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:16.548 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.548 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.548 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4257.0:96 1.0 e29/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 29'70, mlcod=29'68)", 2024-09-18T10:16:16.549 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:39.427232+0000", 2024-09-18T10:16:16.549 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 97.108477312000005, 2024-09-18T10:16:16.549 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20408627300000001, 2024-09-18T10:16:16.549 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:16.549 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:16.549 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:16.549 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.549 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:16.549 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.427232+0000", 2024-09-18T10:16:16.549 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:16.549 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.549 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.549 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:16.549 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.427232+0000", 2024-09-18T10:16:16.549 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:16.550 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.550 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.550 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:16.550 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.427230+0000", 2024-09-18T10:16:16.550 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:16:16.550 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.550 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.550 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:16.550 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.427399+0000", 2024-09-18T10:16:16.550 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000168787 2024-09-18T10:16:16.550 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.550 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.550 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:16.550 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.427431+0000", 2024-09-18T10:16:16.551 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2372000000000001e-05 2024-09-18T10:16:16.551 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.551 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.551 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:16.551 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.427444+0000", 2024-09-18T10:16:16.551 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2303000000000001e-05 2024-09-18T10:16:16.551 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.551 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.551 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:16.551 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.427580+0000", 2024-09-18T10:16:16.551 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000136019 2024-09-18T10:16:16.551 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.551 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.551 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:16:16.552 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.427622+0000", 2024-09-18T10:16:16.552 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.2551e-05 2024-09-18T10:16:16.552 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.552 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.552 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:16:16.552 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.631238+0000", 2024-09-18T10:16:16.552 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20361580100000001 2024-09-18T10:16:16.552 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.552 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.552 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:16.552 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.631318+0000", 2024-09-18T10:16:16.552 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.0370000000000005e-05 2024-09-18T10:16:16.552 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:16.552 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:16.553 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:16.553 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.553 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.553 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4257.0:97 1.0 e29/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 29'71, mlcod=29'68)", 2024-09-18T10:16:16.553 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:39.428800+0000", 2024-09-18T10:16:16.553 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 97.106909369999997, 2024-09-18T10:16:16.553 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.202566529, 2024-09-18T10:16:16.553 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:16.553 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:16.553 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:16.553 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.553 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:16.553 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.428800+0000", 2024-09-18T10:16:16.554 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:16.554 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.554 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.554 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:16.554 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.428800+0000", 2024-09-18T10:16:16.554 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:16.554 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.554 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.554 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:16.554 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.428798+0000", 2024-09-18T10:16:16.554 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:16:16.554 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.554 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.554 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:16.555 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.428814+0000", 2024-09-18T10:16:16.555 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5262999999999999e-05 2024-09-18T10:16:16.555 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.555 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.555 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:16.555 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.428836+0000", 2024-09-18T10:16:16.555 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.2096e-05 2024-09-18T10:16:16.555 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.555 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.555 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:16.555 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.428847+0000", 2024-09-18T10:16:16.555 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1272e-05 2024-09-18T10:16:16.555 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.555 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.556 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:16.556 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.529081+0000", 2024-09-18T10:16:16.556 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100233853 2024-09-18T10:16:16.556 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.556 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.556 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:16:16.556 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.529115+0000", 2024-09-18T10:16:16.556 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3862000000000002e-05 2024-09-18T10:16:16.556 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.556 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.556 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:16:16.556 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.631329+0000", 2024-09-18T10:16:16.556 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.102214465 2024-09-18T10:16:16.556 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.557 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.557 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:16.557 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:39.631367+0000", 2024-09-18T10:16:16.557 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7450999999999998e-05 2024-09-18T10:16:16.557 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:16.557 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:16.557 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:16.557 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.557 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.557 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4257.0:113 1.0 e29/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 29'83, mlcod=29'80)", 2024-09-18T10:16:16.557 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:40.801786+0000", 2024-09-18T10:16:16.557 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 95.733923860000004, 2024-09-18T10:16:16.557 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.203408961, 2024-09-18T10:16:16.557 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:16.558 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:16.558 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:16.558 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.558 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:16.558 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.801786+0000", 2024-09-18T10:16:16.558 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:16.558 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.558 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.558 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:16.558 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.801786+0000", 2024-09-18T10:16:16.558 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:16.558 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.558 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.558 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:16.559 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.801784+0000", 2024-09-18T10:16:16.559 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:16:16.559 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.559 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.559 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:16.559 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.801801+0000", 2024-09-18T10:16:16.559 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7223000000000001e-05 2024-09-18T10:16:16.559 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.559 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.559 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:16.559 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.801823+0000", 2024-09-18T10:16:16.559 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1469000000000001e-05 2024-09-18T10:16:16.559 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.559 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.560 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:16.560 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.801835+0000", 2024-09-18T10:16:16.560 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2033e-05 2024-09-18T10:16:16.560 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.560 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.560 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:16.560 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.902049+0000", 2024-09-18T10:16:16.560 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100213659 2024-09-18T10:16:16.560 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.560 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.560 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:16:16.560 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.902082+0000", 2024-09-18T10:16:16.560 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3668999999999999e-05 2024-09-18T10:16:16.561 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.561 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.561 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:16:16.561 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.005131+0000", 2024-09-18T10:16:16.561 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.103048379 2024-09-18T10:16:16.561 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.561 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.561 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:16.561 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.005195+0000", 2024-09-18T10:16:16.561 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.3805000000000003e-05 2024-09-18T10:16:16.561 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:16.561 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:16.561 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:16.561 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.561 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.562 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4257.0:114 1.0 e29/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 29'84, mlcod=29'80)", 2024-09-18T10:16:16.562 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:40.802657+0000", 2024-09-18T10:16:16.562 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 95.733052725999997, 2024-09-18T10:16:16.562 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20256802600000001, 2024-09-18T10:16:16.562 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:16.562 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:16.562 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:16.562 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.562 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:16.562 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.802657+0000", 2024-09-18T10:16:16.562 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:16.562 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.562 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.562 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:16.563 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.802657+0000", 2024-09-18T10:16:16.563 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:16.563 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.563 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.563 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:16.563 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.802656+0000", 2024-09-18T10:16:16.563 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:16:16.563 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.563 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.563 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:16.563 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.802824+0000", 2024-09-18T10:16:16.563 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000168784 2024-09-18T10:16:16.563 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.563 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.564 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:16.564 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.802844+0000", 2024-09-18T10:16:16.564 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0058999999999999e-05 2024-09-18T10:16:16.564 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.564 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.564 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:16.564 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.802857+0000", 2024-09-18T10:16:16.564 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2163e-05 2024-09-18T10:16:16.564 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.564 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.564 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:16.564 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.902531+0000", 2024-09-18T10:16:16.564 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.099674591000000007 2024-09-18T10:16:16.564 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.565 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.565 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:16:16.565 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:40.902559+0000", 2024-09-18T10:16:16.565 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7483000000000001e-05 2024-09-18T10:16:16.565 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.565 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.565 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:16:16.565 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.005198+0000", 2024-09-18T10:16:16.565 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.102638853 2024-09-18T10:16:16.565 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.565 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.565 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:16.565 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.005225+0000", 2024-09-18T10:16:16.565 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7206e-05 2024-09-18T10:16:16.566 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:16.566 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:16.566 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:16.566 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.566 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.566 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4257.0:128 1.0 e29/23 1:30306672:devicehealth::main.db.0000000000000000:head v 29'98, mlcod=29'95)", 2024-09-18T10:16:16.566 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:41.222806+0000", 2024-09-18T10:16:16.566 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 95.312903460000001, 2024-09-18T10:16:16.566 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20200015099999999, 2024-09-18T10:16:16.566 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:16.566 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:16.566 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:16.566 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.567 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:16.567 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.222806+0000", 2024-09-18T10:16:16.567 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:16.567 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.567 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.567 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:16.567 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.222806+0000", 2024-09-18T10:16:16.567 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:16.567 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.567 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.567 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:16.567 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.222804+0000", 2024-09-18T10:16:16.567 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:16:16.567 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.568 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.568 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:16.568 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.222885+0000", 2024-09-18T10:16:16.568 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.0140000000000002e-05 2024-09-18T10:16:16.568 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.568 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.568 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:16.568 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.222915+0000", 2024-09-18T10:16:16.568 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0168999999999999e-05 2024-09-18T10:16:16.568 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.568 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.568 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:16.568 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.323060+0000", 2024-09-18T10:16:16.568 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100145475 2024-09-18T10:16:16.569 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.569 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.569 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:16.569 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.423304+0000", 2024-09-18T10:16:16.569 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100243924 2024-09-18T10:16:16.569 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.569 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.569 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:16:16.569 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.423338+0000", 2024-09-18T10:16:16.569 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3819000000000003e-05 2024-09-18T10:16:16.569 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.569 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.569 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:16:16.569 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.424724+0000", 2024-09-18T10:16:16.570 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0013862690000000001 2024-09-18T10:16:16.570 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.570 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.570 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:16.570 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:41.424806+0000", 2024-09-18T10:16:16.570 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.1901000000000001e-05 2024-09-18T10:16:16.570 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:16.570 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:16.570 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:16.570 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.570 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.570 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.4s1 41/40 ECSubWrite(tid=17, reqid=osd.8.0:17, at_version=41'10, trim_to=0'0, roll_forward_to=0'0))", 2024-09-18T10:16:16.570 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:57.445149+0000", 2024-09-18T10:16:16.570 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 79.090560898000007, 2024-09-18T10:16:16.571 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20394425099999999, 2024-09-18T10:16:16.571 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:16.571 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:16:16.571 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:16.571 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.571 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:16.571 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.445149+0000", 2024-09-18T10:16:16.571 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:16.571 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.571 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.571 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:16.571 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.445149+0000", 2024-09-18T10:16:16.571 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:16.572 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.572 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.572 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:16.572 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.445147+0000", 2024-09-18T10:16:16.572 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:16:16.572 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.572 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.572 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:16.572 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.445221+0000", 2024-09-18T10:16:16.572 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.4140999999999994e-05 2024-09-18T10:16:16.572 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.572 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.572 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:16.572 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.445260+0000", 2024-09-18T10:16:16.573 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8998999999999997e-05 2024-09-18T10:16:16.573 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.573 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.573 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:16.573 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.545386+0000", 2024-09-18T10:16:16.573 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10012621300000001 2024-09-18T10:16:16.573 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.573 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.573 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:16.573 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.645963+0000", 2024-09-18T10:16:16.573 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10057662000000001 2024-09-18T10:16:16.573 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.573 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.574 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:16:16.574 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.645988+0000", 2024-09-18T10:16:16.574 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5449e-05 2024-09-18T10:16:16.574 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.574 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.574 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:16:16.574 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.649028+0000", 2024-09-18T10:16:16.574 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0030390840000000001 2024-09-18T10:16:16.574 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.574 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.574 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:16.574 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.649093+0000", 2024-09-18T10:16:16.574 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.5258000000000004e-05 2024-09-18T10:16:16.574 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:16.575 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:16.575 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:16.575 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.575 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.575 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.6s6 41/40 ECSubWrite(tid=31, reqid=client.4632.0:249, at_version=41'13, trim_to=0'0, roll_forward_to=0'0))", 2024-09-18T10:16:16.575 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:57.550336+0000", 2024-09-18T10:16:16.575 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 78.985373859000006, 2024-09-18T10:16:16.575 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.203590574, 2024-09-18T10:16:16.575 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:16.575 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:16:16.575 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:16.575 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.575 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:16.575 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.550336+0000", 2024-09-18T10:16:16.576 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:16.576 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.576 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.576 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:16.576 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.550336+0000", 2024-09-18T10:16:16.576 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:16.576 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.576 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.576 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:16.576 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.550329+0000", 2024-09-18T10:16:16.576 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999933 2024-09-18T10:16:16.576 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.576 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.576 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:16.577 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.550932+0000", 2024-09-18T10:16:16.577 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00060276999999999998 2024-09-18T10:16:16.577 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.577 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.577 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:16.577 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.550955+0000", 2024-09-18T10:16:16.577 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.2884999999999999e-05 2024-09-18T10:16:16.577 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.577 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.577 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:16.577 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.651052+0000", 2024-09-18T10:16:16.577 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100097174 2024-09-18T10:16:16.577 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.577 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.578 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:16.578 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.651160+0000", 2024-09-18T10:16:16.578 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000108483 2024-09-18T10:16:16.578 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.578 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.578 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:16:16.578 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.651172+0000", 2024-09-18T10:16:16.578 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1790999999999999e-05 2024-09-18T10:16:16.578 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.578 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.578 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:16:16.578 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.753875+0000", 2024-09-18T10:16:16.578 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10270337 2024-09-18T10:16:16.578 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.579 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.579 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:16.579 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:57.753926+0000", 2024-09-18T10:16:16.579 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0767999999999999e-05 2024-09-18T10:16:16.579 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:16.579 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:16.579 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:16.579 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.579 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.579 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.8s4 42/40 ECSubWrite(tid=133, reqid=osd.6.0:132, at_version=0'0, trim_to=0'0, roll_forward_to=41'13))", 2024-09-18T10:16:16.579 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:14:58.049763+0000", 2024-09-18T10:16:16.579 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 78.485946123000005, 2024-09-18T10:16:16.579 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.21446268399999999, 2024-09-18T10:16:16.579 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:16.580 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:16:16.580 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:16.580 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.580 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:16.580 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:58.049763+0000", 2024-09-18T10:16:16.580 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:16.580 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.580 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.580 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:16.580 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:58.049763+0000", 2024-09-18T10:16:16.580 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:16.580 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.580 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.580 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:16.581 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:58.049763+0000", 2024-09-18T10:16:16.581 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999995 2024-09-18T10:16:16.581 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.581 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.581 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:16.581 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:58.049787+0000", 2024-09-18T10:16:16.581 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3819e-05 2024-09-18T10:16:16.581 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.581 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.581 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:16.581 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:58.049817+0000", 2024-09-18T10:16:16.581 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0749e-05 2024-09-18T10:16:16.581 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.581 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.582 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:16.582 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:58.049836+0000", 2024-09-18T10:16:16.582 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9182999999999999e-05 2024-09-18T10:16:16.582 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.582 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.582 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:16.582 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:58.163409+0000", 2024-09-18T10:16:16.582 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.11357204899999999 2024-09-18T10:16:16.582 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.582 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.582 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:16:16.582 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:58.163424+0000", 2024-09-18T10:16:16.582 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5403000000000001e-05 2024-09-18T10:16:16.582 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.583 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.583 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:16:16.583 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:58.264161+0000", 2024-09-18T10:16:16.583 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100736942 2024-09-18T10:16:16.583 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.583 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.583 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:16.583 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:14:58.264226+0000", 2024-09-18T10:16:16.583 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.5085000000000005e-05 2024-09-18T10:16:16.583 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:16.583 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:16.583 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:16.583 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.583 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.584 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.ds3 47/32 ECSubWrite(tid=56, reqid=client.4632.0:333, at_version=47'9, trim_to=0'0, roll_forward_to=41'6))", 2024-09-18T10:16:16.584 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:02.801124+0000", 2024-09-18T10:16:16.584 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 73.734584964999996, 2024-09-18T10:16:16.584 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20185083000000001, 2024-09-18T10:16:16.584 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:16.584 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:16:16.584 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:16.584 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.584 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:16.584 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.801124+0000", 2024-09-18T10:16:16.584 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:16.584 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.584 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.584 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:16.585 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.801124+0000", 2024-09-18T10:16:16.585 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:16.585 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.585 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.585 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:16.585 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.801103+0000", 2024-09-18T10:16:16.585 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999785 2024-09-18T10:16:16.585 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.585 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.585 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:16.585 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.801366+0000", 2024-09-18T10:16:16.585 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00026269299999999999 2024-09-18T10:16:16.585 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.585 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.586 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:16.586 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.801395+0000", 2024-09-18T10:16:16.586 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.9572000000000001e-05 2024-09-18T10:16:16.586 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.586 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.586 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:16.586 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:02.801412+0000", 2024-09-18T10:16:16.586 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6872999999999999e-05 2024-09-18T10:16:16.586 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.586 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.586 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:16.586 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.000543+0000", 2024-09-18T10:16:16.586 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.19913046700000001 2024-09-18T10:16:16.586 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.587 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.587 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:16:16.587 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.000579+0000", 2024-09-18T10:16:16.587 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6649000000000001e-05 2024-09-18T10:16:16.587 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.587 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.587 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:16:16.587 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.002922+0000", 2024-09-18T10:16:16.587 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0023426139999999998 2024-09-18T10:16:16.587 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.587 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.587 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:16.587 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:03.002975+0000", 2024-09-18T10:16:16.587 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3232000000000001e-05 2024-09-18T10:16:16.588 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:16.588 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:16.588 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:16.588 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.588 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.588 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.15s1 63/60 ECSubWrite(tid=54, reqid=client.4632.0:512, at_version=0'0, trim_to=0'0, roll_forward_to=63'4))", 2024-09-18T10:16:16.588 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:19.215356+0000", 2024-09-18T10:16:16.588 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 57.320353103000002, 2024-09-18T10:16:16.588 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20187551500000001, 2024-09-18T10:16:16.588 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:16.588 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:16:16.588 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:16.588 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.588 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:16.589 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.215356+0000", 2024-09-18T10:16:16.589 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:16.589 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.589 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.589 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:16.589 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.215356+0000", 2024-09-18T10:16:16.589 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:16.589 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.589 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.589 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:16.589 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.215355+0000", 2024-09-18T10:16:16.589 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:16:16.589 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.589 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.590 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:16.590 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.215386+0000", 2024-09-18T10:16:16.590 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0809000000000003e-05 2024-09-18T10:16:16.590 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.590 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.590 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:16.590 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.215434+0000", 2024-09-18T10:16:16.590 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.8751999999999997e-05 2024-09-18T10:16:16.590 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.590 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.590 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:16.590 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.315532+0000", 2024-09-18T10:16:16.590 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100097281 2024-09-18T10:16:16.590 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.591 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.591 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:16.591 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.415795+0000", 2024-09-18T10:16:16.591 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10026387 2024-09-18T10:16:16.591 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.591 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.591 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:16:16.591 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.415823+0000", 2024-09-18T10:16:16.591 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7469000000000001e-05 2024-09-18T10:16:16.591 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.591 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.591 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:16:16.591 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.417173+0000", 2024-09-18T10:16:16.591 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0013505030000000001 2024-09-18T10:16:16.592 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.592 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.592 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:16.592 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:19.417232+0000", 2024-09-18T10:16:16.592 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8421000000000002e-05 2024-09-18T10:16:16.592 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:16.592 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:16.592 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:16.592 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.592 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.592 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.4s1 72/40 ECSubWrite(tid=72, reqid=osd.8.0:72, at_version=72'53, trim_to=0'0, roll_forward_to=71'51))", 2024-09-18T10:16:16.592 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:28.190950+0000", 2024-09-18T10:16:16.592 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 48.344759279000002, 2024-09-18T10:16:16.592 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20228795599999999, 2024-09-18T10:16:16.593 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:16.593 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:16:16.593 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:16.593 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.593 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:16.593 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.190950+0000", 2024-09-18T10:16:16.593 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:16.593 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.593 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.593 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:16.593 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.190950+0000", 2024-09-18T10:16:16.593 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:16.593 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.593 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.594 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:16.594 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.190949+0000", 2024-09-18T10:16:16.594 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:16:16.594 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.594 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.594 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:16.594 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.191001+0000", 2024-09-18T10:16:16.594 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.2581999999999999e-05 2024-09-18T10:16:16.594 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.594 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.594 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:16.594 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.191022+0000", 2024-09-18T10:16:16.594 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0792999999999999e-05 2024-09-18T10:16:16.594 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.595 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.595 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:16.595 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.191034+0000", 2024-09-18T10:16:16.595 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1603000000000001e-05 2024-09-18T10:16:16.595 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.595 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.595 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:16.595 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.391648+0000", 2024-09-18T10:16:16.595 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20061451299999999 2024-09-18T10:16:16.595 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.595 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.595 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:16:16.595 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.391678+0000", 2024-09-18T10:16:16.595 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0199e-05 2024-09-18T10:16:16.596 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.596 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.596 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:16:16.596 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.393192+0000", 2024-09-18T10:16:16.596 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0015135599999999999 2024-09-18T10:16:16.596 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.596 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.596 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:16.596 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.393238+0000", 2024-09-18T10:16:16.596 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.6128999999999998e-05 2024-09-18T10:16:16.596 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:16.596 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:16.596 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:16.596 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.597 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.597 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.4s1 72/40 ECSubWrite(tid=74, reqid=osd.8.0:72, at_version=0'0, trim_to=0'0, roll_forward_to=72'53))", 2024-09-18T10:16:16.597 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:28.496056+0000", 2024-09-18T10:16:16.597 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 48.039653196000003, 2024-09-18T10:16:16.597 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20153745300000001, 2024-09-18T10:16:16.597 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:16.597 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:16:16.597 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:16.597 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.597 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:16.597 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.496056+0000", 2024-09-18T10:16:16.597 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:16.597 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.597 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.598 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:16.598 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.496056+0000", 2024-09-18T10:16:16.598 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:16.598 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.598 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.598 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:16.598 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.496055+0000", 2024-09-18T10:16:16.598 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:16:16.598 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.598 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.598 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:16.598 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.496088+0000", 2024-09-18T10:16:16.598 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3411999999999998e-05 2024-09-18T10:16:16.599 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.599 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.599 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:16.599 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.496120+0000", 2024-09-18T10:16:16.599 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1745999999999997e-05 2024-09-18T10:16:16.599 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.599 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.599 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:16.599 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.596220+0000", 2024-09-18T10:16:16.599 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100100381 2024-09-18T10:16:16.599 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.599 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.599 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:16.599 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.596320+0000", 2024-09-18T10:16:16.600 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.9745000000000001e-05 2024-09-18T10:16:16.600 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.600 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.600 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:16:16.600 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.596344+0000", 2024-09-18T10:16:16.600 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4598999999999999e-05 2024-09-18T10:16:16.600 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.600 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.600 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:16:16.600 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.697512+0000", 2024-09-18T10:16:16.601 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10116728699999999 2024-09-18T10:16:16.601 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.601 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.601 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:16.601 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:28.697594+0000", 2024-09-18T10:16:16.601 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.1972999999999996e-05 2024-09-18T10:16:16.601 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:16.601 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:16.601 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:16.601 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.601 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.601 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.4s1 75/40 ECSubWrite(tid=80, reqid=osd.8.0:80, at_version=75'61, trim_to=0'0, roll_forward_to=74'59))", 2024-09-18T10:16:16.601 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:30.566285+0000", 2024-09-18T10:16:16.602 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 45.969424035999999, 2024-09-18T10:16:16.602 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.201584069, 2024-09-18T10:16:16.602 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:16.602 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:16:16.602 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:16.602 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.602 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:16.602 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.566285+0000", 2024-09-18T10:16:16.602 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:16.602 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.602 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.602 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:16.603 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.566285+0000", 2024-09-18T10:16:16.603 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:16.603 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.603 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.603 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:16.603 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.566285+0000", 2024-09-18T10:16:16.603 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999995 2024-09-18T10:16:16.603 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.603 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.603 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:16.603 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.566310+0000", 2024-09-18T10:16:16.603 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5250999999999999e-05 2024-09-18T10:16:16.603 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.603 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.604 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:16.604 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.566319+0000", 2024-09-18T10:16:16.604 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.5159999999999994e-06 2024-09-18T10:16:16.604 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.604 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.604 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:16.604 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.666393+0000", 2024-09-18T10:16:16.604 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10007392399999999 2024-09-18T10:16:16.604 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.604 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.604 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:16.604 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.666455+0000", 2024-09-18T10:16:16.604 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.2372999999999994e-05 2024-09-18T10:16:16.604 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.605 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.605 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:16:16.605 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.666465+0000", 2024-09-18T10:16:16.605 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.7820000000000005e-06 2024-09-18T10:16:16.605 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.605 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.605 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:16:16.605 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.767822+0000", 2024-09-18T10:16:16.605 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.10135730900000001 2024-09-18T10:16:16.605 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.605 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.605 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:16.605 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:30.767869+0000", 2024-09-18T10:16:16.606 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.7181000000000001e-05 2024-09-18T10:16:16.606 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:16.606 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:16.606 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:16.606 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.606 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.606 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpWrite(3.fs2 89/32 ECSubWrite(tid=187, reqid=osd.13.0:187, at_version=89'40, trim_to=0'0, roll_forward_to=81'37))", 2024-09-18T10:16:16.606 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:44.898707+0000", 2024-09-18T10:16:16.606 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 31.637002578000001, 2024-09-18T10:16:16.606 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.203487005, 2024-09-18T10:16:16.606 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:16.606 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:16:16.606 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:16.606 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.606 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:16.607 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:44.898707+0000", 2024-09-18T10:16:16.607 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:16.607 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.607 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.607 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:16.607 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:44.898707+0000", 2024-09-18T10:16:16.607 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:16.607 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.607 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.607 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:16.607 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:44.898706+0000", 2024-09-18T10:16:16.607 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.999999 2024-09-18T10:16:16.607 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.607 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.608 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:16.608 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:44.898768+0000", 2024-09-18T10:16:16.608 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.2237999999999996e-05 2024-09-18T10:16:16.608 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.608 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.608 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:16.608 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:44.898791+0000", 2024-09-18T10:16:16.608 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.2713000000000001e-05 2024-09-18T10:16:16.608 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.608 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.608 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:16.608 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:44.898808+0000", 2024-09-18T10:16:16.608 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7356000000000001e-05 2024-09-18T10:16:16.609 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.609 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.609 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:16.609 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:45.099963+0000", 2024-09-18T10:16:16.609 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20115494 2024-09-18T10:16:16.609 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.609 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.609 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:16:16.609 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:45.100014+0000", 2024-09-18T10:16:16.609 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.1298999999999998e-05 2024-09-18T10:16:16.609 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.609 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.609 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:16:16.609 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:45.102121+0000", 2024-09-18T10:16:16.609 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0021067479999999999 2024-09-18T10:16:16.610 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.610 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.610 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:16.610 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:45.102194+0000", 2024-09-18T10:16:16.610 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.2700999999999994e-05 2024-09-18T10:16:16.610 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:16.610 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:16.610 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:16.610 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.610 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.610 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4632.0:942 3.4s1 3:2619bf65:::smithi18925896-16:head [delete] snapc 35=[35,34,33,30,2f,28,20,1f,11] RETRY=2 ondisk+retry+write+known_if_redirected+full_force+supports_pool_eio e95)", 2024-09-18T10:16:16.610 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:50.784970+0000", 2024-09-18T10:16:16.610 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 25.750738988999998, 2024-09-18T10:16:16.611 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3075152910000001, 2024-09-18T10:16:16.611 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:16.611 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T10:16:16.611 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-18T10:16:16.611 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4632", 2024-09-18T10:16:16.611 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.189:0/2424103459", 2024-09-18T10:16:16.611 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 942 2024-09-18T10:16:16.611 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.611 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:16.611 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.611 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:16.611 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:50.784970+0000", 2024-09-18T10:16:16.611 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:16.611 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.612 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.612 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:16.612 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:50.784970+0000", 2024-09-18T10:16:16.612 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:16.612 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.612 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.612 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:16.612 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:50.784967+0000", 2024-09-18T10:16:16.612 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999967 2024-09-18T10:16:16.612 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.612 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.612 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:16.612 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:50.784983+0000", 2024-09-18T10:16:16.612 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5849999999999999e-05 2024-09-18T10:16:16.613 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.613 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.613 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:16.613 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:50.785021+0000", 2024-09-18T10:16:16.613 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8039e-05 2024-09-18T10:16:16.613 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.613 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.613 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:16.613 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:50.785036+0000", 2024-09-18T10:16:16.613 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4790000000000001e-05 2024-09-18T10:16:16.613 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.613 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.613 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:16.613 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:50.983583+0000", 2024-09-18T10:16:16.614 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.19854741400000001 2024-09-18T10:16:16.614 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.614 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.614 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-18T10:16:16.614 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:50.983588+0000", 2024-09-18T10:16:16.614 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1200000000000004e-06 2024-09-18T10:16:16.614 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.614 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.614 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:16.614 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:51.886817+0000", 2024-09-18T10:16:16.614 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.90322920900000003 2024-09-18T10:16:16.614 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.614 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.614 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-18T10:16:16.615 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:51.887374+0000", 2024-09-18T10:16:16.615 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00055698399999999995 2024-09-18T10:16:16.615 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.615 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.615 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_started", 2024-09-18T10:16:16.615 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:51.889200+0000", 2024-09-18T10:16:16.615 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001826016 2024-09-18T10:16:16.615 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.615 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.615 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_committed", 2024-09-18T10:16:16.615 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:51.891521+0000", 2024-09-18T10:16:16.615 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.002321301 2024-09-18T10:16:16.615 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.615 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.616 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-18T10:16:16.616 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:52.091827+0000", 2024-09-18T10:16:16.616 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.200305603 2024-09-18T10:16:16.616 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.616 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.616 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:16.616 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:52.092486+0000", 2024-09-18T10:16:16.616 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00065908799999999995 2024-09-18T10:16:16.616 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:16.616 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:16.616 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:16.616 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.616 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.616 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpRead(3.cs1 95/94 ECSubRead(tid=206, to_read={3:351da1bf:::smithi18925896-6:2f=0,262144,0}, subchunks={3:351da1bf:::smithi18925896-6:2f=[0,1]}, attrs_to_read=))", 2024-09-18T10:16:16.617 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:51.508874+0000", 2024-09-18T10:16:16.617 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 25.02683588, 2024-09-18T10:16:16.617 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.27511936599999998, 2024-09-18T10:16:16.617 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:16.617 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:16:16.617 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:16.617 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.617 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:16.617 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:51.508874+0000", 2024-09-18T10:16:16.617 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:16.617 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.617 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.617 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:16.617 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:51.508874+0000", 2024-09-18T10:16:16.618 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:16.618 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.618 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.618 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:16.618 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:51.508872+0000", 2024-09-18T10:16:16.618 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999986 2024-09-18T10:16:16.618 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.618 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.618 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:16.618 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:51.508890+0000", 2024-09-18T10:16:16.618 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.8298999999999999e-05 2024-09-18T10:16:16.618 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.618 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.618 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:16.619 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:51.508930+0000", 2024-09-18T10:16:16.619 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9190000000000003e-05 2024-09-18T10:16:16.619 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.619 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.619 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:16.619 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:51.609029+0000", 2024-09-18T10:16:16.619 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100099382 2024-09-18T10:16:16.619 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.619 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.619 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:16.619 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:51.783094+0000", 2024-09-18T10:16:16.619 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.174064892 2024-09-18T10:16:16.619 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.619 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.620 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:16.620 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:51.783993+0000", 2024-09-18T10:16:16.620 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00089909300000000001 2024-09-18T10:16:16.620 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:16.620 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:16.620 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:16.620 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.620 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.620 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpRead(3.cs1 96/94 ECSubRead(tid=207, to_read={3:351da1bf:::smithi18925896-6:head=0,262144,0}, subchunks={3:351da1bf:::smithi18925896-6:head=[0,1]}, attrs_to_read=))", 2024-09-18T10:16:16.620 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:51.677052+0000", 2024-09-18T10:16:16.620 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 24.858657633, 2024-09-18T10:16:16.620 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.31682236800000002, 2024-09-18T10:16:16.620 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:16.620 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:16:16.621 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:16.621 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.621 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:16.621 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:51.677052+0000", 2024-09-18T10:16:16.621 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:16.621 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.621 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.621 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:16.621 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:51.677052+0000", 2024-09-18T10:16:16.621 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:16.621 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.621 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.621 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:16.621 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:51.677051+0000", 2024-09-18T10:16:16.622 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999995 2024-09-18T10:16:16.622 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.622 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.622 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:16.622 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:51.677069+0000", 2024-09-18T10:16:16.622 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.8182999999999999e-05 2024-09-18T10:16:16.622 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.622 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.622 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:16.622 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:51.677114+0000", 2024-09-18T10:16:16.622 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5018999999999997e-05 2024-09-18T10:16:16.622 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.622 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.622 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:16.623 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:51.777230+0000", 2024-09-18T10:16:16.623 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.100115924 2024-09-18T10:16:16.623 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.623 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.623 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:16.623 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:51.993689+0000", 2024-09-18T10:16:16.623 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.216458762 2024-09-18T10:16:16.623 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.623 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.623 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:16.623 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:51.993874+0000", 2024-09-18T10:16:16.623 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00018503000000000001 2024-09-18T10:16:16.623 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:16.623 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:16.623 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:16.624 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.624 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.624 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "MOSDECSubOpRead(3.cs1 96/94 ECSubRead(tid=208, to_read={3:351da1bf:::smithi18925896-6:2f=262144,262144,0}, subchunks={3:351da1bf:::smithi18925896-6:2f=[0,1]}, attrs_to_read=))", 2024-09-18T10:16:16.624 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-18T10:15:51.789038+0000", 2024-09-18T10:16:16.624 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 24.746671099, 2024-09-18T10:16:16.624 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20205589600000001, 2024-09-18T10:16:16.624 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-18T10:16:16.624 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-18T10:16:16.624 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-18T10:16:16.624 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.624 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-18T10:16:16.624 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:51.789038+0000", 2024-09-18T10:16:16.624 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:16.625 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.625 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.625 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-18T10:16:16.625 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:51.789038+0000", 2024-09-18T10:16:16.625 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-18T10:16:16.625 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.625 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.625 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-18T10:16:16.625 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:51.789037+0000", 2024-09-18T10:16:16.625 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4294967295.9999981 2024-09-18T10:16:16.625 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.625 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.625 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-18T10:16:16.625 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:51.789058+0000", 2024-09-18T10:16:16.626 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1498999999999999e-05 2024-09-18T10:16:16.626 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.626 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.626 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-18T10:16:16.626 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:51.789091+0000", 2024-09-18T10:16:16.626 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3030000000000001e-05 2024-09-18T10:16:16.626 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.626 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.626 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-18T10:16:16.626 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:51.789106+0000", 2024-09-18T10:16:16.626 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4566000000000001e-05 2024-09-18T10:16:16.626 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.626 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.626 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-18T10:16:16.627 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:51.990519+0000", 2024-09-18T10:16:16.627 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.20141335699999999 2024-09-18T10:16:16.627 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-18T10:16:16.627 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-18T10:16:16.627 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-18T10:16:16.627 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-18T10:15:51.991094+0000", 2024-09-18T10:16:16.627 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00057515099999999998 2024-09-18T10:16:16.627 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:16.627 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:16.627 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:16.627 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-18T10:16:16.627 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-18T10:16:16.627 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-18T10:16:16.889 INFO:teuthology.orchestra.run.smithi029.stderr:noscrub is unset 2024-09-18T10:16:16.905 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-18T10:16:17.879 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=600 2024-09-18T10:16:17.900 INFO:teuthology.orchestra.run.smithi029.stderr:nodeep-scrub is unset 2024-09-18T10:16:18.024 INFO:teuthology.orchestra.run.smithi029.stdout:mon.a: {} 2024-09-18T10:16:18.045 INFO:teuthology.orchestra.run.smithi029.stdout:mon.b: {} 2024-09-18T10:16:18.060 INFO:teuthology.orchestra.run.smithi029.stdout:mon.c: {} 2024-09-18T10:16:18.063 INFO:teuthology.orchestra.run.smithi029.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-18T10:16:18.063 INFO:teuthology.orchestra.run.smithi029.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-18T10:16:18.063 INFO:teuthology.orchestra.run.smithi029.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-18T10:16:18.080 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-18T10:16:18.903 INFO:teuthology.orchestra.run.smithi029.stderr:reweighted osd.0 to 1 (10000) 2024-09-18T10:16:18.919 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 0 1 2024-09-18T10:16:19.912 INFO:teuthology.orchestra.run.smithi029.stderr:set osd.0 primary-affinity to 1 (8655362) 2024-09-18T10:16:19.927 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-18T10:16:20.931 INFO:teuthology.orchestra.run.smithi029.stderr:reweighted osd.4 to 1 (10000) 2024-09-18T10:16:20.946 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 4 1 2024-09-18T10:16:21.939 INFO:teuthology.orchestra.run.smithi029.stderr:set osd.4 primary-affinity to 1 (8655362) 2024-09-18T10:16:21.952 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 8 1 2024-09-18T10:16:22.943 INFO:teuthology.orchestra.run.smithi029.stderr:reweighted osd.8 to 1 (10000) 2024-09-18T10:16:22.960 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 8 1 2024-09-18T10:16:23.953 INFO:teuthology.orchestra.run.smithi029.stderr:set osd.8 primary-affinity to 1 (8655362) 2024-09-18T10:16:23.968 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 12 1 2024-09-18T10:16:24.963 INFO:teuthology.orchestra.run.smithi029.stderr:reweighted osd.12 to 1 (10000) 2024-09-18T10:16:24.980 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 12 1 2024-09-18T10:16:25.972 INFO:teuthology.orchestra.run.smithi029.stderr:set osd.12 primary-affinity to 1 (8655362) 2024-09-18T10:16:25.987 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-18T10:16:26.983 INFO:teuthology.orchestra.run.smithi029.stderr:reweighted osd.1 to 1 (10000) 2024-09-18T10:16:26.997 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 1 2024-09-18T10:16:27.993 INFO:teuthology.orchestra.run.smithi029.stderr:set osd.1 primary-affinity to 1 (8655362) 2024-09-18T10:16:28.007 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-18T10:16:29.003 INFO:teuthology.orchestra.run.smithi029.stderr:reweighted osd.5 to 1 (10000) 2024-09-18T10:16:29.017 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 5 1 2024-09-18T10:16:30.014 INFO:teuthology.orchestra.run.smithi029.stderr:set osd.5 primary-affinity to 1 (8655362) 2024-09-18T10:16:30.027 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 9 1 2024-09-18T10:16:31.022 INFO:teuthology.orchestra.run.smithi029.stderr:reweighted osd.9 to 1 (10000) 2024-09-18T10:16:31.035 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 9 1 2024-09-18T10:16:32.038 INFO:teuthology.orchestra.run.smithi029.stderr:set osd.9 primary-affinity to 1 (8655362) 2024-09-18T10:16:32.051 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 13 1 2024-09-18T10:16:33.042 INFO:teuthology.orchestra.run.smithi029.stderr:reweighted osd.13 to 1 (10000) 2024-09-18T10:16:33.055 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 13 1 2024-09-18T10:16:34.055 INFO:teuthology.orchestra.run.smithi029.stderr:set osd.13 primary-affinity to 1 (8655362) 2024-09-18T10:16:34.071 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-18T10:16:35.059 INFO:teuthology.orchestra.run.smithi029.stderr:reweighted osd.2 to 1 (10000) 2024-09-18T10:16:35.074 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 1 2024-09-18T10:16:36.076 INFO:teuthology.orchestra.run.smithi029.stderr:set osd.2 primary-affinity to 1 (8655362) 2024-09-18T10:16:36.093 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-18T10:16:37.078 INFO:teuthology.orchestra.run.smithi029.stderr:reweighted osd.6 to 1 (10000) 2024-09-18T10:16:37.093 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 6 1 2024-09-18T10:16:38.095 INFO:teuthology.orchestra.run.smithi029.stderr:set osd.6 primary-affinity to 1 (8655362) 2024-09-18T10:16:38.111 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 10 1 2024-09-18T10:16:39.098 INFO:teuthology.orchestra.run.smithi029.stderr:reweighted osd.10 to 1 (10000) 2024-09-18T10:16:39.112 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 10 1 2024-09-18T10:16:40.106 INFO:teuthology.orchestra.run.smithi029.stderr:set osd.10 primary-affinity to 1 (8655362) 2024-09-18T10:16:40.119 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 14 1 2024-09-18T10:16:41.117 INFO:teuthology.orchestra.run.smithi029.stderr:reweighted osd.14 to 1 (10000) 2024-09-18T10:16:41.130 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 14 1 2024-09-18T10:16:42.126 INFO:teuthology.orchestra.run.smithi029.stderr:set osd.14 primary-affinity to 1 (8655362) 2024-09-18T10:16:42.140 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-18T10:16:43.141 INFO:teuthology.orchestra.run.smithi029.stderr:reweighted osd.3 to 1 (10000) 2024-09-18T10:16:43.155 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 3 1 2024-09-18T10:16:44.151 INFO:teuthology.orchestra.run.smithi029.stderr:set osd.3 primary-affinity to 1 (8655362) 2024-09-18T10:16:44.167 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-18T10:16:45.154 INFO:teuthology.orchestra.run.smithi029.stderr:reweighted osd.7 to 1 (10000) 2024-09-18T10:16:45.170 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 7 1 2024-09-18T10:16:46.170 INFO:teuthology.orchestra.run.smithi029.stderr:set osd.7 primary-affinity to 1 (8655362) 2024-09-18T10:16:46.184 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 11 1 2024-09-18T10:16:47.173 INFO:teuthology.orchestra.run.smithi029.stderr:reweighted osd.11 to 1 (10000) 2024-09-18T10:16:47.188 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 11 1 2024-09-18T10:16:48.185 INFO:teuthology.orchestra.run.smithi029.stderr:set osd.11 primary-affinity to 1 (8655362) 2024-09-18T10:16:48.199 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 15 1 2024-09-18T10:16:49.195 INFO:teuthology.orchestra.run.smithi029.stderr:reweighted osd.15 to 1 (10000) 2024-09-18T10:16:49.209 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 15 1 2024-09-18T10:16:50.209 INFO:teuthology.orchestra.run.smithi029.stderr:set osd.15 primary-affinity to 1 (8655362) 2024-09-18T10:16:50.223 INFO:tasks.thrashosds.thrasher:joining the do_sighup greenlet 2024-09-18T10:16:50.223 INFO:tasks.thrashosds.thrasher:joining the do_optrack_toggle greenlet 2024-09-18T10:16:50.223 INFO:tasks.thrashosds.thrasher:joining the do_dump_ops greenlet 2024-09-18T10:16:50.223 INFO:tasks.thrashosds.thrasher:joining the do_noscrub_toggle greenlet 2024-09-18T10:16:50.223 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-18T10:16:50.224 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-18T10:16:50.525 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-18T10:16:50.526 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":150,"fsid":"ab6d6beb-0627-4b7f-8cdb-4e0a44970f10","created":"2024-09-18T10:14:18.827644+0000","modified":"2024-09-18T10:16:50.188989+0000","last_up_change":"2024-09-18T10:14:30.228591+0000","last_in_change":"2024-09-18T10:15:49.554826+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-18T10:14:31.266113+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-18T10:14:33.358485+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":29,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"39cf0511-0009-433f-92f2-e5686f33d713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":22197},{"type":"v1","addr":"172.21.15.29:6809","nonce":22197}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":22197},{"type":"v1","addr":"172.21.15.29:6811","nonce":22197}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":22197},{"type":"v1","addr":"172.21.15.29:6815","nonce":22197}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":22197},{"type":"v1","addr":"172.21.15.29:6813","nonce":22197}]},"public_addr":"172.21.15.29:6809/22197","cluster_addr":"172.21.15.29:6811/22197","heartbeat_back_addr":"172.21.15.29:6815/22197","heartbeat_front_addr":"172.21.15.29:6813/22197","state":["exists","up"]},{"osd":1,"uuid":"a0bf06ab-b98e-4130-91d2-7d4283540c08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":22140},{"type":"v1","addr":"172.21.15.136:6809","nonce":22140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":22140},{"type":"v1","addr":"172.21.15.136:6811","nonce":22140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":22140},{"type":"v1","addr":"172.21.15.136:6815","nonce":22140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":22140},{"type":"v1","addr":"172.21.15.136:6813","nonce":22140}]},"public_addr":"172.21.15.136:6809/22140","cluster_addr":"172.21.15.136:6811/22140","heartbeat_back_addr":"172.21.15.136:6815/22140","heartbeat_front_addr":"172.21.15.136:6813/22140","state":["exists","up"]},{"osd":2,"uuid":"742a786e-6129-4fbc-bb5b-e2e3c03c2d77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":22095},{"type":"v1","addr":"172.21.15.154:6825","nonce":22095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":22095},{"type":"v1","addr":"172.21.15.154:6827","nonce":22095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":22095},{"type":"v1","addr":"172.21.15.154:6831","nonce":22095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":22095},{"type":"v1","addr":"172.21.15.154:6829","nonce":22095}]},"public_addr":"172.21.15.154:6825/22095","cluster_addr":"172.21.15.154:6827/22095","heartbeat_back_addr":"172.21.15.154:6831/22095","heartbeat_front_addr":"172.21.15.154:6829/22095","state":["exists","up"]},{"osd":3,"uuid":"3aa77114-24ed-49ab-86ff-a0600794cbf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":22404},{"type":"v1","addr":"172.21.15.189:6803","nonce":22404}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":22404},{"type":"v1","addr":"172.21.15.189:6805","nonce":22404}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":22404},{"type":"v1","addr":"172.21.15.189:6809","nonce":22404}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":22404},{"type":"v1","addr":"172.21.15.189:6807","nonce":22404}]},"public_addr":"172.21.15.189:6803/22404","cluster_addr":"172.21.15.189:6805/22404","heartbeat_back_addr":"172.21.15.189:6809/22404","heartbeat_front_addr":"172.21.15.189:6807/22404","state":["exists","up"]},{"osd":4,"uuid":"426567ad-b0be-41d4-afcf-dc56ec3974cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":22195},{"type":"v1","addr":"172.21.15.29:6801","nonce":22195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":22195},{"type":"v1","addr":"172.21.15.29:6803","nonce":22195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":22195},{"type":"v1","addr":"172.21.15.29:6807","nonce":22195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":22195},{"type":"v1","addr":"172.21.15.29:6805","nonce":22195}]},"public_addr":"172.21.15.29:6801/22195","cluster_addr":"172.21.15.29:6803/22195","heartbeat_back_addr":"172.21.15.29:6807/22195","heartbeat_front_addr":"172.21.15.29:6805/22195","state":["exists","up"]},{"osd":5,"uuid":"d9559e42-1f4e-4373-8392-a536a0afd871","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6817","nonce":22141},{"type":"v1","addr":"172.21.15.136:6819","nonce":22141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6821","nonce":22141},{"type":"v1","addr":"172.21.15.136:6823","nonce":22141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6829","nonce":22141},{"type":"v1","addr":"172.21.15.136:6831","nonce":22141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6825","nonce":22141},{"type":"v1","addr":"172.21.15.136:6827","nonce":22141}]},"public_addr":"172.21.15.136:6819/22141","cluster_addr":"172.21.15.136:6823/22141","heartbeat_back_addr":"172.21.15.136:6831/22141","heartbeat_front_addr":"172.21.15.136:6827/22141","state":["exists","up"]},{"osd":6,"uuid":"d50ef1e2-8bc5-4a90-9be3-365a0db2be6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":22096},{"type":"v1","addr":"172.21.15.154:6801","nonce":22096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":22096},{"type":"v1","addr":"172.21.15.154:6803","nonce":22096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":22096},{"type":"v1","addr":"172.21.15.154:6807","nonce":22096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":22096},{"type":"v1","addr":"172.21.15.154:6805","nonce":22096}]},"public_addr":"172.21.15.154:6801/22096","cluster_addr":"172.21.15.154:6803/22096","heartbeat_back_addr":"172.21.15.154:6807/22096","heartbeat_front_addr":"172.21.15.154:6805/22096","state":["exists","up"]},{"osd":7,"uuid":"bd0b85e4-cfee-4d19-90a5-79854794b48b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":22405},{"type":"v1","addr":"172.21.15.189:6819","nonce":22405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":22405},{"type":"v1","addr":"172.21.15.189:6821","nonce":22405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":22405},{"type":"v1","addr":"172.21.15.189:6825","nonce":22405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":22405},{"type":"v1","addr":"172.21.15.189:6823","nonce":22405}]},"public_addr":"172.21.15.189:6819/22405","cluster_addr":"172.21.15.189:6821/22405","heartbeat_back_addr":"172.21.15.189:6825/22405","heartbeat_front_addr":"172.21.15.189:6823/22405","state":["exists","up"]},{"osd":8,"uuid":"cd333f56-c165-45d7-8c8b-48d34bc4ddc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":22200},{"type":"v1","addr":"172.21.15.29:6825","nonce":22200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":22200},{"type":"v1","addr":"172.21.15.29:6827","nonce":22200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":22200},{"type":"v1","addr":"172.21.15.29:6831","nonce":22200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":22200},{"type":"v1","addr":"172.21.15.29:6829","nonce":22200}]},"public_addr":"172.21.15.29:6825/22200","cluster_addr":"172.21.15.29:6827/22200","heartbeat_back_addr":"172.21.15.29:6831/22200","heartbeat_front_addr":"172.21.15.29:6829/22200","state":["exists","up"]},{"osd":9,"uuid":"1e079e4c-8801-4754-9d0d-d0b68695d32e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":22142},{"type":"v1","addr":"172.21.15.136:6801","nonce":22142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":22142},{"type":"v1","addr":"172.21.15.136:6803","nonce":22142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":22142},{"type":"v1","addr":"172.21.15.136:6807","nonce":22142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":22142},{"type":"v1","addr":"172.21.15.136:6805","nonce":22142}]},"public_addr":"172.21.15.136:6801/22142","cluster_addr":"172.21.15.136:6803/22142","heartbeat_back_addr":"172.21.15.136:6807/22142","heartbeat_front_addr":"172.21.15.136:6805/22142","state":["exists","up"]},{"osd":10,"uuid":"92ee68d4-371c-4849-930f-a8fa18d64b99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":22097},{"type":"v1","addr":"172.21.15.154:6809","nonce":22097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":22097},{"type":"v1","addr":"172.21.15.154:6811","nonce":22097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":22097},{"type":"v1","addr":"172.21.15.154:6815","nonce":22097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":22097},{"type":"v1","addr":"172.21.15.154:6813","nonce":22097}]},"public_addr":"172.21.15.154:6809/22097","cluster_addr":"172.21.15.154:6811/22097","heartbeat_back_addr":"172.21.15.154:6815/22097","heartbeat_front_addr":"172.21.15.154:6813/22097","state":["exists","up"]},{"osd":11,"uuid":"634ddaf5-f7b8-4b27-a46a-04e21b73fcf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":22416},{"type":"v1","addr":"172.21.15.189:6827","nonce":22416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":22416},{"type":"v1","addr":"172.21.15.189:6829","nonce":22416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":22416},{"type":"v1","addr":"172.21.15.189:6833","nonce":22416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":22416},{"type":"v1","addr":"172.21.15.189:6831","nonce":22416}]},"public_addr":"172.21.15.189:6827/22416","cluster_addr":"172.21.15.189:6829/22416","heartbeat_back_addr":"172.21.15.189:6833/22416","heartbeat_front_addr":"172.21.15.189:6831/22416","state":["exists","up"]},{"osd":12,"uuid":"bc58137b-5948-400d-b377-193b15824d8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":22210},{"type":"v1","addr":"172.21.15.29:6817","nonce":22210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":22210},{"type":"v1","addr":"172.21.15.29:6819","nonce":22210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":22210},{"type":"v1","addr":"172.21.15.29:6823","nonce":22210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":22210},{"type":"v1","addr":"172.21.15.29:6821","nonce":22210}]},"public_addr":"172.21.15.29:6817/22210","cluster_addr":"172.21.15.29:6819/22210","heartbeat_back_addr":"172.21.15.29:6823/22210","heartbeat_front_addr":"172.21.15.29:6821/22210","state":["exists","up"]},{"osd":13,"uuid":"a8b753a7-ab69-4853-ac36-66d9b85c2e20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":22143},{"type":"v1","addr":"172.21.15.136:6818","nonce":22143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":22143},{"type":"v1","addr":"172.21.15.136:6822","nonce":22143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":22143},{"type":"v1","addr":"172.21.15.136:6830","nonce":22143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":22143},{"type":"v1","addr":"172.21.15.136:6826","nonce":22143}]},"public_addr":"172.21.15.136:6818/22143","cluster_addr":"172.21.15.136:6822/22143","heartbeat_back_addr":"172.21.15.136:6830/22143","heartbeat_front_addr":"172.21.15.136:6826/22143","state":["exists","up"]},{"osd":14,"uuid":"8ad0848f-0de4-4a75-9c63-8c4a4a7b8524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":22098},{"type":"v1","addr":"172.21.15.154:6817","nonce":22098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":22098},{"type":"v1","addr":"172.21.15.154:6819","nonce":22098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":22098},{"type":"v1","addr":"172.21.15.154:6823","nonce":22098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":22098},{"type":"v1","addr":"172.21.15.154:6821","nonce":22098}]},"public_addr":"172.21.15.154:6817/22098","cluster_addr":"172.21.15.154:6819/22098","heartbeat_back_addr":"172.21.15.154:6823/22098","heartbeat_front_addr":"172.21.15.154:6821/22098","state":["exists","up"]},{"osd":15,"uuid":"069099b3-ec1a-483b-b967-98cddbf2f678","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":22415},{"type":"v1","addr":"172.21.15.189:6811","nonce":22415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":22415},{"type":"v1","addr":"172.21.15.189:6813","nonce":22415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":22415},{"type":"v1","addr":"172.21.15.189:6817","nonce":22415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":22415},{"type":"v1","addr":"172.21.15.189:6815","nonce":22415}]},"public_addr":"172.21.15.189:6811/22415","cluster_addr":"172.21.15.189:6813/22415","heartbeat_back_addr":"172.21.15.189:6817/22415","heartbeat_front_addr":"172.21.15.189:6815/22415","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:15:56.126704+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:15:32.497046+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:15:55.189067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:15:44.298040+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:15:56.991035+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:15:51.718897+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:15:41.309784+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:16:42.140104+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:15:42.972647+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:15:36.896626+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:15:54.542105+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:15:47.318375+0000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:16:37.858675+0000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:15:36.748402+0000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:15:52.801215+0000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:15:59.018997+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"},"shecprofile":{"c":"2","crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","k":"4","m":"3","name":"shecprofile","plugin":"shec","technique":"multiple"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-18T10:16:50.540 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-18T10:16:50.540 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-18T10:16:50.838 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-18T10:16:50.838 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":150,"fsid":"ab6d6beb-0627-4b7f-8cdb-4e0a44970f10","created":"2024-09-18T10:14:18.827644+0000","modified":"2024-09-18T10:16:50.188989+0000","last_up_change":"2024-09-18T10:14:30.228591+0000","last_in_change":"2024-09-18T10:15:49.554826+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-18T10:14:31.266113+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-18T10:14:33.358485+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":29,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"39cf0511-0009-433f-92f2-e5686f33d713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":22197},{"type":"v1","addr":"172.21.15.29:6809","nonce":22197}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":22197},{"type":"v1","addr":"172.21.15.29:6811","nonce":22197}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":22197},{"type":"v1","addr":"172.21.15.29:6815","nonce":22197}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":22197},{"type":"v1","addr":"172.21.15.29:6813","nonce":22197}]},"public_addr":"172.21.15.29:6809/22197","cluster_addr":"172.21.15.29:6811/22197","heartbeat_back_addr":"172.21.15.29:6815/22197","heartbeat_front_addr":"172.21.15.29:6813/22197","state":["exists","up"]},{"osd":1,"uuid":"a0bf06ab-b98e-4130-91d2-7d4283540c08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":22140},{"type":"v1","addr":"172.21.15.136:6809","nonce":22140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":22140},{"type":"v1","addr":"172.21.15.136:6811","nonce":22140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":22140},{"type":"v1","addr":"172.21.15.136:6815","nonce":22140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":22140},{"type":"v1","addr":"172.21.15.136:6813","nonce":22140}]},"public_addr":"172.21.15.136:6809/22140","cluster_addr":"172.21.15.136:6811/22140","heartbeat_back_addr":"172.21.15.136:6815/22140","heartbeat_front_addr":"172.21.15.136:6813/22140","state":["exists","up"]},{"osd":2,"uuid":"742a786e-6129-4fbc-bb5b-e2e3c03c2d77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":22095},{"type":"v1","addr":"172.21.15.154:6825","nonce":22095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":22095},{"type":"v1","addr":"172.21.15.154:6827","nonce":22095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":22095},{"type":"v1","addr":"172.21.15.154:6831","nonce":22095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":22095},{"type":"v1","addr":"172.21.15.154:6829","nonce":22095}]},"public_addr":"172.21.15.154:6825/22095","cluster_addr":"172.21.15.154:6827/22095","heartbeat_back_addr":"172.21.15.154:6831/22095","heartbeat_front_addr":"172.21.15.154:6829/22095","state":["exists","up"]},{"osd":3,"uuid":"3aa77114-24ed-49ab-86ff-a0600794cbf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":22404},{"type":"v1","addr":"172.21.15.189:6803","nonce":22404}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":22404},{"type":"v1","addr":"172.21.15.189:6805","nonce":22404}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":22404},{"type":"v1","addr":"172.21.15.189:6809","nonce":22404}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":22404},{"type":"v1","addr":"172.21.15.189:6807","nonce":22404}]},"public_addr":"172.21.15.189:6803/22404","cluster_addr":"172.21.15.189:6805/22404","heartbeat_back_addr":"172.21.15.189:6809/22404","heartbeat_front_addr":"172.21.15.189:6807/22404","state":["exists","up"]},{"osd":4,"uuid":"426567ad-b0be-41d4-afcf-dc56ec3974cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":22195},{"type":"v1","addr":"172.21.15.29:6801","nonce":22195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":22195},{"type":"v1","addr":"172.21.15.29:6803","nonce":22195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":22195},{"type":"v1","addr":"172.21.15.29:6807","nonce":22195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":22195},{"type":"v1","addr":"172.21.15.29:6805","nonce":22195}]},"public_addr":"172.21.15.29:6801/22195","cluster_addr":"172.21.15.29:6803/22195","heartbeat_back_addr":"172.21.15.29:6807/22195","heartbeat_front_addr":"172.21.15.29:6805/22195","state":["exists","up"]},{"osd":5,"uuid":"d9559e42-1f4e-4373-8392-a536a0afd871","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6817","nonce":22141},{"type":"v1","addr":"172.21.15.136:6819","nonce":22141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6821","nonce":22141},{"type":"v1","addr":"172.21.15.136:6823","nonce":22141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6829","nonce":22141},{"type":"v1","addr":"172.21.15.136:6831","nonce":22141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6825","nonce":22141},{"type":"v1","addr":"172.21.15.136:6827","nonce":22141}]},"public_addr":"172.21.15.136:6819/22141","cluster_addr":"172.21.15.136:6823/22141","heartbeat_back_addr":"172.21.15.136:6831/22141","heartbeat_front_addr":"172.21.15.136:6827/22141","state":["exists","up"]},{"osd":6,"uuid":"d50ef1e2-8bc5-4a90-9be3-365a0db2be6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":22096},{"type":"v1","addr":"172.21.15.154:6801","nonce":22096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":22096},{"type":"v1","addr":"172.21.15.154:6803","nonce":22096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":22096},{"type":"v1","addr":"172.21.15.154:6807","nonce":22096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":22096},{"type":"v1","addr":"172.21.15.154:6805","nonce":22096}]},"public_addr":"172.21.15.154:6801/22096","cluster_addr":"172.21.15.154:6803/22096","heartbeat_back_addr":"172.21.15.154:6807/22096","heartbeat_front_addr":"172.21.15.154:6805/22096","state":["exists","up"]},{"osd":7,"uuid":"bd0b85e4-cfee-4d19-90a5-79854794b48b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":22405},{"type":"v1","addr":"172.21.15.189:6819","nonce":22405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":22405},{"type":"v1","addr":"172.21.15.189:6821","nonce":22405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":22405},{"type":"v1","addr":"172.21.15.189:6825","nonce":22405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":22405},{"type":"v1","addr":"172.21.15.189:6823","nonce":22405}]},"public_addr":"172.21.15.189:6819/22405","cluster_addr":"172.21.15.189:6821/22405","heartbeat_back_addr":"172.21.15.189:6825/22405","heartbeat_front_addr":"172.21.15.189:6823/22405","state":["exists","up"]},{"osd":8,"uuid":"cd333f56-c165-45d7-8c8b-48d34bc4ddc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":22200},{"type":"v1","addr":"172.21.15.29:6825","nonce":22200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":22200},{"type":"v1","addr":"172.21.15.29:6827","nonce":22200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":22200},{"type":"v1","addr":"172.21.15.29:6831","nonce":22200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":22200},{"type":"v1","addr":"172.21.15.29:6829","nonce":22200}]},"public_addr":"172.21.15.29:6825/22200","cluster_addr":"172.21.15.29:6827/22200","heartbeat_back_addr":"172.21.15.29:6831/22200","heartbeat_front_addr":"172.21.15.29:6829/22200","state":["exists","up"]},{"osd":9,"uuid":"1e079e4c-8801-4754-9d0d-d0b68695d32e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":22142},{"type":"v1","addr":"172.21.15.136:6801","nonce":22142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":22142},{"type":"v1","addr":"172.21.15.136:6803","nonce":22142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":22142},{"type":"v1","addr":"172.21.15.136:6807","nonce":22142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":22142},{"type":"v1","addr":"172.21.15.136:6805","nonce":22142}]},"public_addr":"172.21.15.136:6801/22142","cluster_addr":"172.21.15.136:6803/22142","heartbeat_back_addr":"172.21.15.136:6807/22142","heartbeat_front_addr":"172.21.15.136:6805/22142","state":["exists","up"]},{"osd":10,"uuid":"92ee68d4-371c-4849-930f-a8fa18d64b99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":22097},{"type":"v1","addr":"172.21.15.154:6809","nonce":22097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":22097},{"type":"v1","addr":"172.21.15.154:6811","nonce":22097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":22097},{"type":"v1","addr":"172.21.15.154:6815","nonce":22097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":22097},{"type":"v1","addr":"172.21.15.154:6813","nonce":22097}]},"public_addr":"172.21.15.154:6809/22097","cluster_addr":"172.21.15.154:6811/22097","heartbeat_back_addr":"172.21.15.154:6815/22097","heartbeat_front_addr":"172.21.15.154:6813/22097","state":["exists","up"]},{"osd":11,"uuid":"634ddaf5-f7b8-4b27-a46a-04e21b73fcf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":22416},{"type":"v1","addr":"172.21.15.189:6827","nonce":22416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":22416},{"type":"v1","addr":"172.21.15.189:6829","nonce":22416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":22416},{"type":"v1","addr":"172.21.15.189:6833","nonce":22416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":22416},{"type":"v1","addr":"172.21.15.189:6831","nonce":22416}]},"public_addr":"172.21.15.189:6827/22416","cluster_addr":"172.21.15.189:6829/22416","heartbeat_back_addr":"172.21.15.189:6833/22416","heartbeat_front_addr":"172.21.15.189:6831/22416","state":["exists","up"]},{"osd":12,"uuid":"bc58137b-5948-400d-b377-193b15824d8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":22210},{"type":"v1","addr":"172.21.15.29:6817","nonce":22210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":22210},{"type":"v1","addr":"172.21.15.29:6819","nonce":22210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":22210},{"type":"v1","addr":"172.21.15.29:6823","nonce":22210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":22210},{"type":"v1","addr":"172.21.15.29:6821","nonce":22210}]},"public_addr":"172.21.15.29:6817/22210","cluster_addr":"172.21.15.29:6819/22210","heartbeat_back_addr":"172.21.15.29:6823/22210","heartbeat_front_addr":"172.21.15.29:6821/22210","state":["exists","up"]},{"osd":13,"uuid":"a8b753a7-ab69-4853-ac36-66d9b85c2e20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":22143},{"type":"v1","addr":"172.21.15.136:6818","nonce":22143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":22143},{"type":"v1","addr":"172.21.15.136:6822","nonce":22143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":22143},{"type":"v1","addr":"172.21.15.136:6830","nonce":22143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":22143},{"type":"v1","addr":"172.21.15.136:6826","nonce":22143}]},"public_addr":"172.21.15.136:6818/22143","cluster_addr":"172.21.15.136:6822/22143","heartbeat_back_addr":"172.21.15.136:6830/22143","heartbeat_front_addr":"172.21.15.136:6826/22143","state":["exists","up"]},{"osd":14,"uuid":"8ad0848f-0de4-4a75-9c63-8c4a4a7b8524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":22098},{"type":"v1","addr":"172.21.15.154:6817","nonce":22098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":22098},{"type":"v1","addr":"172.21.15.154:6819","nonce":22098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":22098},{"type":"v1","addr":"172.21.15.154:6823","nonce":22098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":22098},{"type":"v1","addr":"172.21.15.154:6821","nonce":22098}]},"public_addr":"172.21.15.154:6817/22098","cluster_addr":"172.21.15.154:6819/22098","heartbeat_back_addr":"172.21.15.154:6823/22098","heartbeat_front_addr":"172.21.15.154:6821/22098","state":["exists","up"]},{"osd":15,"uuid":"069099b3-ec1a-483b-b967-98cddbf2f678","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":22415},{"type":"v1","addr":"172.21.15.189:6811","nonce":22415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":22415},{"type":"v1","addr":"172.21.15.189:6813","nonce":22415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":22415},{"type":"v1","addr":"172.21.15.189:6817","nonce":22415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":22415},{"type":"v1","addr":"172.21.15.189:6815","nonce":22415}]},"public_addr":"172.21.15.189:6811/22415","cluster_addr":"172.21.15.189:6813/22415","heartbeat_back_addr":"172.21.15.189:6817/22415","heartbeat_front_addr":"172.21.15.189:6815/22415","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:15:56.126704+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:15:32.497046+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:15:55.189067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:15:44.298040+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:15:56.991035+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:15:51.718897+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:15:41.309784+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:16:42.140104+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:15:42.972647+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:15:36.896626+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:15:54.542105+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:15:47.318375+0000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:16:37.858675+0000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:15:36.748402+0000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:15:52.801215+0000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:15:59.018997+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"},"shecprofile":{"c":"2","crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","k":"4","m":"3","name":"shecprofile","plugin":"shec","technique":"multiple"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-18T10:16:50.857 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-18T10:16:50.857 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-18T10:16:50.857 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-18T10:16:50.858 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-18T10:16:50.858 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-18T10:16:50.859 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-18T10:16:50.859 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-18T10:16:50.859 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-18T10:16:50.860 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.8 flush_pg_stats 2024-09-18T10:16:50.860 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.9 flush_pg_stats 2024-09-18T10:16:50.861 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.10 flush_pg_stats 2024-09-18T10:16:50.861 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.11 flush_pg_stats 2024-09-18T10:16:50.861 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.12 flush_pg_stats 2024-09-18T10:16:50.862 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.13 flush_pg_stats 2024-09-18T10:16:50.862 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.14 flush_pg_stats 2024-09-18T10:16:50.863 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.15 flush_pg_stats 2024-09-18T10:16:51.169 INFO:teuthology.orchestra.run.smithi029.stdout:94489280543 2024-09-18T10:16:51.170 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-18T10:16:51.213 INFO:teuthology.orchestra.run.smithi029.stdout:94489280543 2024-09-18T10:16:51.214 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-18T10:16:51.225 INFO:teuthology.orchestra.run.smithi029.stdout:90194313249 2024-09-18T10:16:51.225 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-18T10:16:51.228 INFO:teuthology.orchestra.run.smithi029.stdout:94489280543 2024-09-18T10:16:51.229 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-18T10:16:51.240 INFO:teuthology.orchestra.run.smithi029.stdout:94489280543 2024-09-18T10:16:51.241 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-18T10:16:51.250 INFO:teuthology.orchestra.run.smithi029.stdout:94489280544 2024-09-18T10:16:51.251 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.10 2024-09-18T10:16:51.252 INFO:teuthology.orchestra.run.smithi029.stdout:90194313247 2024-09-18T10:16:51.252 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.9 2024-09-18T10:16:51.257 INFO:teuthology.orchestra.run.smithi029.stdout:90194313247 2024-09-18T10:16:51.258 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-18T10:16:51.259 INFO:teuthology.orchestra.run.smithi029.stdout:94489280543 2024-09-18T10:16:51.260 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-18T10:16:51.263 INFO:teuthology.orchestra.run.smithi029.stdout:90194313247 2024-09-18T10:16:51.264 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.11 2024-09-18T10:16:51.265 INFO:teuthology.orchestra.run.smithi029.stdout:94489280543 2024-09-18T10:16:51.266 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-18T10:16:51.280 INFO:teuthology.orchestra.run.smithi029.stdout:94489280543 2024-09-18T10:16:51.280 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.15 2024-09-18T10:16:51.287 INFO:teuthology.orchestra.run.smithi029.stdout:94489280543 2024-09-18T10:16:51.288 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.8 2024-09-18T10:16:51.288 INFO:teuthology.orchestra.run.smithi029.stdout:90194313248 2024-09-18T10:16:51.289 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.12 2024-09-18T10:16:51.306 INFO:teuthology.orchestra.run.smithi029.stdout:94489280543 2024-09-18T10:16:51.306 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.14 2024-09-18T10:16:51.307 INFO:teuthology.orchestra.run.smithi029.stdout:94489280543 2024-09-18T10:16:51.307 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.13 2024-09-18T10:16:51.916 INFO:teuthology.orchestra.run.smithi029.stdout:94489280541 2024-09-18T10:16:51.948 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280543 got 94489280541 for osd.2 2024-09-18T10:16:52.001 INFO:teuthology.orchestra.run.smithi029.stdout:94489280541 2024-09-18T10:16:52.002 INFO:teuthology.orchestra.run.smithi029.stdout:94489280541 2024-09-18T10:16:52.024 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280543 got 94489280541 for osd.1 2024-09-18T10:16:52.043 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280543 got 94489280541 for osd.3 2024-09-18T10:16:52.070 INFO:teuthology.orchestra.run.smithi029.stdout:90194313245 2024-09-18T10:16:52.110 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313247 got 90194313245 for osd.0 2024-09-18T10:16:52.140 INFO:teuthology.orchestra.run.smithi029.stdout:94489280541 2024-09-18T10:16:52.143 INFO:teuthology.orchestra.run.smithi029.stdout:94489280542 2024-09-18T10:16:52.159 INFO:teuthology.orchestra.run.smithi029.stdout:90194313247 2024-09-18T10:16:52.160 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280543 got 94489280541 for osd.6 2024-09-18T10:16:52.160 INFO:teuthology.orchestra.run.smithi029.stdout:94489280541 2024-09-18T10:16:52.162 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280544 got 94489280542 for osd.10 2024-09-18T10:16:52.182 INFO:teuthology.orchestra.run.smithi029.stdout:94489280541 2024-09-18T10:16:52.183 INFO:teuthology.orchestra.run.smithi029.stdout:90194313245 2024-09-18T10:16:52.187 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280543 got 94489280541 for osd.15 2024-09-18T10:16:52.200 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280543 got 94489280541 for osd.7 2024-09-18T10:16:52.200 INFO:teuthology.orchestra.run.smithi029.stdout:94489280541 2024-09-18T10:16:52.202 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313249 got 90194313247 for osd.4 2024-09-18T10:16:52.203 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313247 got 90194313245 for osd.11 2024-09-18T10:16:52.217 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280543 got 94489280541 for osd.5 2024-09-18T10:16:52.243 INFO:teuthology.orchestra.run.smithi029.stdout:94489280542 2024-09-18T10:16:52.247 INFO:teuthology.orchestra.run.smithi029.stdout:90194313247 2024-09-18T10:16:52.249 INFO:teuthology.orchestra.run.smithi029.stdout:90194313247 2024-09-18T10:16:52.260 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313248 got 90194313247 for osd.12 2024-09-18T10:16:52.261 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280543 got 94489280542 for osd.8 2024-09-18T10:16:52.267 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313247 got 90194313247 for osd.9 2024-09-18T10:16:52.267 DEBUG:teuthology.parallel:result is None 2024-09-18T10:16:52.273 INFO:teuthology.orchestra.run.smithi029.stdout:94489280541 2024-09-18T10:16:52.279 INFO:teuthology.orchestra.run.smithi029.stdout:94489280542 2024-09-18T10:16:52.285 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280543 got 94489280541 for osd.14 2024-09-18T10:16:52.296 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280543 got 94489280542 for osd.13 2024-09-18T10:16:52.948 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-18T10:16:53.025 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-18T10:16:53.044 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-18T10:16:53.111 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-18T10:16:53.587 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-18T10:16:53.587 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.10 2024-09-18T10:16:53.587 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.15 2024-09-18T10:16:53.588 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-18T10:16:53.588 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-18T10:16:53.589 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.11 2024-09-18T10:16:53.589 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-18T10:16:53.589 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.12 2024-09-18T10:16:53.590 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.8 2024-09-18T10:16:53.590 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.14 2024-09-18T10:16:53.590 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.13 2024-09-18T10:16:53.592 INFO:teuthology.orchestra.run.smithi029.stdout:94489280543 2024-09-18T10:16:53.592 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280543 got 94489280543 for osd.2 2024-09-18T10:16:53.593 DEBUG:teuthology.parallel:result is None 2024-09-18T10:16:53.593 INFO:teuthology.orchestra.run.smithi029.stdout:94489280543 2024-09-18T10:16:53.594 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280543 got 94489280543 for osd.1 2024-09-18T10:16:53.594 DEBUG:teuthology.parallel:result is None 2024-09-18T10:16:53.596 INFO:teuthology.orchestra.run.smithi029.stdout:94489280543 2024-09-18T10:16:53.597 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280543 got 94489280543 for osd.3 2024-09-18T10:16:53.597 DEBUG:teuthology.parallel:result is None 2024-09-18T10:16:54.146 INFO:teuthology.orchestra.run.smithi029.stdout:90194313246 2024-09-18T10:16:54.165 INFO:teuthology.orchestra.run.smithi029.stdout:90194313247 2024-09-18T10:16:54.177 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313247 got 90194313246 for osd.11 2024-09-18T10:16:54.194 INFO:teuthology.orchestra.run.smithi029.stdout:94489280542 2024-09-18T10:16:54.194 INFO:teuthology.orchestra.run.smithi029.stdout:94489280542 2024-09-18T10:16:54.199 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313247 got 90194313247 for osd.0 2024-09-18T10:16:54.199 DEBUG:teuthology.parallel:result is None 2024-09-18T10:16:54.220 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280543 got 94489280542 for osd.7 2024-09-18T10:16:54.221 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280543 got 94489280542 for osd.15 2024-09-18T10:16:54.268 INFO:teuthology.orchestra.run.smithi029.stdout:94489280544 2024-09-18T10:16:54.270 INFO:teuthology.orchestra.run.smithi029.stdout:90194313249 2024-09-18T10:16:54.285 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280544 got 94489280544 for osd.10 2024-09-18T10:16:54.285 DEBUG:teuthology.parallel:result is None 2024-09-18T10:16:54.289 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313249 got 90194313249 for osd.4 2024-09-18T10:16:54.289 DEBUG:teuthology.parallel:result is None 2024-09-18T10:16:54.313 INFO:teuthology.orchestra.run.smithi029.stdout:94489280543 2024-09-18T10:16:54.316 INFO:teuthology.orchestra.run.smithi029.stdout:94489280543 2024-09-18T10:16:54.325 INFO:teuthology.orchestra.run.smithi029.stdout:94489280543 2024-09-18T10:16:54.330 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280543 got 94489280543 for osd.8 2024-09-18T10:16:54.331 DEBUG:teuthology.parallel:result is None 2024-09-18T10:16:54.331 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280543 got 94489280543 for osd.5 2024-09-18T10:16:54.331 DEBUG:teuthology.parallel:result is None 2024-09-18T10:16:54.334 INFO:teuthology.orchestra.run.smithi029.stdout:94489280543 2024-09-18T10:16:54.338 INFO:teuthology.orchestra.run.smithi029.stdout:90194313248 2024-09-18T10:16:54.343 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280543 got 94489280543 for osd.14 2024-09-18T10:16:54.343 DEBUG:teuthology.parallel:result is None 2024-09-18T10:16:54.347 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280543 got 94489280543 for osd.6 2024-09-18T10:16:54.347 DEBUG:teuthology.parallel:result is None 2024-09-18T10:16:54.350 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313248 got 90194313248 for osd.12 2024-09-18T10:16:54.351 DEBUG:teuthology.parallel:result is None 2024-09-18T10:16:54.369 INFO:teuthology.orchestra.run.smithi029.stdout:94489280544 2024-09-18T10:16:54.380 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280543 got 94489280544 for osd.13 2024-09-18T10:16:54.380 DEBUG:teuthology.parallel:result is None 2024-09-18T10:16:55.177 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.11 2024-09-18T10:16:55.221 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-18T10:16:55.221 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.15 2024-09-18T10:16:55.500 INFO:teuthology.orchestra.run.smithi029.stdout:90194313247 2024-09-18T10:16:55.508 INFO:teuthology.orchestra.run.smithi029.stdout:94489280543 2024-09-18T10:16:55.511 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313247 got 90194313247 for osd.11 2024-09-18T10:16:55.512 DEBUG:teuthology.parallel:result is None 2024-09-18T10:16:55.516 INFO:teuthology.orchestra.run.smithi029.stdout:94489280543 2024-09-18T10:16:55.521 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280543 got 94489280543 for osd.15 2024-09-18T10:16:55.521 DEBUG:teuthology.parallel:result is None 2024-09-18T10:16:55.529 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280543 got 94489280543 for osd.7 2024-09-18T10:16:55.529 DEBUG:teuthology.parallel:result is None 2024-09-18T10:16:55.529 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-18T10:16:55.530 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T10:16:55.867 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-18T10:16:55.870 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-09-18T10:16:55.883 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":209,"stamp":"2024-09-18T10:16:55.239416+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":147,"num_write_kb":2674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":115,"ondisk_log_size":115,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":14,"kb":1509949440,"kb_used":4865408,"kb_used_data":36952,"kb_used_omap":26,"kb_used_meta":4828197,"kb_avail":1505084032,"statfs":{"total":1546188226560,"available":1541206048768,"internally_reserved":0,"allocated":37838848,"data_stored":22477174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":27264,"internal_metadata":4944074112},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.056129"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":144,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-18T10:16:49.404525+0000","last_change":"2024-09-18T10:16:18.085265+0000","last_active":"2024-09-18T10:16:49.404525+0000","last_peered":"2024-09-18T10:16:49.404525+0000","last_clean":"2024-09-18T10:16:49.404525+0000","last_became_active":"2024-09-18T10:14:35.348943+0000","last_became_peered":"2024-09-18T10:14:35.348943+0000","last_unstale":"2024-09-18T10:16:49.404525+0000","last_undegraded":"2024-09-18T10:16:49.404525+0000","last_fullsized":"2024-09-18T10:16:49.404525+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:16:18.085027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:16:18.085027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:17:43.814969+0000","scrub_duration":0.30385945199999997,"objects_trimmed":0,"snaptrim_duration":0.00060442999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,10],"acting":[13,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":76,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-18T10:16:49.402946+0000","last_change":"2024-09-18T10:16:18.981603+0000","last_active":"2024-09-18T10:16:49.402946+0000","last_peered":"2024-09-18T10:16:49.402946+0000","last_clean":"2024-09-18T10:16:49.402946+0000","last_became_active":"2024-09-18T10:15:50.978055+0000","last_became_peered":"2024-09-18T10:15:50.978055+0000","last_unstale":"2024-09-18T10:16:49.402946+0000","last_undegraded":"2024-09-18T10:16:49.402946+0000","last_fullsized":"2024-09-18T10:16:49.402946+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:16:18.981373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:16:18.981373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:17:41.742568+0000","scrub_duration":0.40456036899999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11],"acting":[8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":140,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-18T10:16:49.303750+0000","last_change":"2024-09-18T10:15:50.883382+0000","last_active":"2024-09-18T10:16:49.303750+0000","last_peered":"2024-09-18T10:16:49.303750+0000","last_clean":"2024-09-18T10:16:49.303750+0000","last_became_active":"2024-09-18T10:14:35.329198+0000","last_became_peered":"2024-09-18T10:14:35.329198+0000","last_unstale":"2024-09-18T10:16:49.303750+0000","last_undegraded":"2024-09-18T10:16:49.303750+0000","last_fullsized":"2024-09-18T10:16:49.303750+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:15:50.883147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:15:50.883147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:17:15.846467+0000","scrub_duration":0.153222372,"objects_trimmed":0,"snaptrim_duration":0.00062415699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":142,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T10:16:48.501848+0000","last_change":"2024-09-18T10:16:02.749341+0000","last_active":"2024-09-18T10:16:48.501848+0000","last_peered":"2024-09-18T10:16:48.501848+0000","last_clean":"2024-09-18T10:16:48.501848+0000","last_became_active":"2024-09-18T10:14:35.524102+0000","last_became_peered":"2024-09-18T10:14:35.524102+0000","last_unstale":"2024-09-18T10:16:48.501848+0000","last_undegraded":"2024-09-18T10:16:48.501848+0000","last_fullsized":"2024-09-18T10:16:48.501848+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:16:02.749150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:16:02.749150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:17:04.725472+0000","scrub_duration":0.20368988499999999,"objects_trimmed":0,"snaptrim_duration":0.10077985,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"29'2","reported_seq":145,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-18T10:16:49.304395+0000","last_change":"2024-09-18T10:16:20.864837+0000","last_active":"2024-09-18T10:16:49.304395+0000","last_peered":"2024-09-18T10:16:49.304395+0000","last_clean":"2024-09-18T10:16:49.304395+0000","last_became_active":"2024-09-18T10:14:35.715374+0000","last_became_peered":"2024-09-18T10:14:35.715374+0000","last_unstale":"2024-09-18T10:16:49.304395+0000","last_undegraded":"2024-09-18T10:16:49.304395+0000","last_fullsized":"2024-09-18T10:16:49.304395+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2","last_scrub_stamp":"2024-09-18T10:16:20.864635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:16:20.864635+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:17:23.226052+0000","scrub_duration":0.20463374300000001,"objects_trimmed":0,"snaptrim_duration":0.002421224,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,12],"acting":[13,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":150,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-18T10:16:49.186308+0000","last_change":"2024-09-18T10:16:15.062108+0000","last_active":"2024-09-18T10:16:49.186308+0000","last_peered":"2024-09-18T10:16:49.186308+0000","last_clean":"2024-09-18T10:16:49.186308+0000","last_became_active":"2024-09-18T10:14:35.348262+0000","last_became_peered":"2024-09-18T10:14:35.348262+0000","last_unstale":"2024-09-18T10:16:49.186308+0000","last_undegraded":"2024-09-18T10:16:49.186308+0000","last_fullsized":"2024-09-18T10:16:49.186308+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:16:15.061892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:16:15.061892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:17:40.002913+0000","scrub_duration":0.40410323300000001,"objects_trimmed":0,"snaptrim_duration":0.00063430400000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,0],"acting":[14,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":144,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-18T10:16:52.538159+0000","last_change":"2024-09-18T10:16:15.352277+0000","last_active":"2024-09-18T10:16:52.538159+0000","last_peered":"2024-09-18T10:16:52.538159+0000","last_clean":"2024-09-18T10:16:52.538159+0000","last_became_active":"2024-09-18T10:14:35.349203+0000","last_became_peered":"2024-09-18T10:14:35.349203+0000","last_unstale":"2024-09-18T10:16:52.538159+0000","last_undegraded":"2024-09-18T10:16:52.538159+0000","last_fullsized":"2024-09-18T10:16:52.538159+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:16:15.352050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:16:15.352050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:17:45.176858+0000","scrub_duration":0.203632596,"objects_trimmed":0,"snaptrim_duration":0.00060130600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3],"acting":[9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"27'1","reported_seq":145,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-18T10:16:49.186339+0000","last_change":"2024-09-18T10:16:06.900858+0000","last_active":"2024-09-18T10:16:49.186339+0000","last_peered":"2024-09-18T10:16:49.186339+0000","last_clean":"2024-09-18T10:16:49.186339+0000","last_became_active":"2024-09-18T10:14:35.487746+0000","last_became_peered":"2024-09-18T10:14:35.487746+0000","last_unstale":"2024-09-18T10:16:49.186339+0000","last_undegraded":"2024-09-18T10:16:49.186339+0000","last_fullsized":"2024-09-18T10:16:49.186339+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'1","last_scrub_stamp":"2024-09-18T10:16:06.900601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:16:06.900601+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:17:25.247813+0000","scrub_duration":0.27078197700000001,"objects_trimmed":0,"snaptrim_duration":0.10072764100000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,5],"acting":[14,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"1.0","version":"29'112","reported_seq":275,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-18T10:16:49.187571+0000","last_change":"2024-09-18T10:16:03.315273+0000","last_active":"2024-09-18T10:16:49.187571+0000","last_peered":"2024-09-18T10:16:49.187571+0000","last_clean":"2024-09-18T10:16:49.187571+0000","last_became_active":"2024-09-18T10:14:32.508037+0000","last_became_peered":"2024-09-18T10:14:32.508037+0000","last_unstale":"2024-09-18T10:16:49.187571+0000","last_undegraded":"2024-09-18T10:16:49.187571+0000","last_fullsized":"2024-09-18T10:16:49.187571+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'112","last_scrub_stamp":"2024-09-18T10:16:03.315126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:31.267741+0000","last_clean_scrub_stamp":"2024-09-18T10:16:03.315126+0000","objects_scrubbed":2,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:17:16.808627+0000","scrub_duration":0.10318229800000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,15],"acting":[6,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":13},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":112,"ondisk_log_size":112,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":15,"up_from":22,"seq":94489280543,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304272,"kb_used_data":2752,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94067568,"statfs":{"total":96636764160,"available":96325189632,"internally_reserved":0,"allocated":2818048,"data_stored":1860517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,4,6,7,8,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:16:32 2024","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":0.83899999999999997,"15min":0.81899999999999995},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":2.0019999999999998,"5min":2.0019999999999998,"15min":2.0019999999999998},"last":1.0649999999999999},{"interface":"front","average":{"1min":0.996,"5min":5.5739999999999998,"15min":6.3369999999999997},"min":{"1min":0.72899999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.506,"5min":101.291,"15min":101.291},"last":0.97299999999999998}]},{"osd":1,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.93300000000000005},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.931,"5min":1.931,"15min":1.931},"last":1.1279999999999999}]},{"osd":2,"last update":"Wed Sep 18 10:16:32 2024","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.0169999999999999,"15min":0.995},"min":{"1min":0.79700000000000004,"5min":0.5,"15min":0.5},"max":{"1min":1.7470000000000001,"5min":1.7470000000000001,"15min":1.7470000000000001},"last":1.028},{"interface":"front","average":{"1min":1.1419999999999999,"5min":4.6589999999999998,"15min":5.2450000000000001},"min":{"1min":0.82899999999999996,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.4379999999999999,"5min":78.242999999999995,"15min":78.242999999999995},"last":1.252}]},{"osd":3,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":3.3029999999999999,"5min":3.3029999999999999,"15min":3.3029999999999999},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":41.720999999999997,"5min":41.720999999999997,"15min":41.720999999999997},"last":0.55400000000000005},{"interface":"front","average":{"1min":1.014,"5min":1.014,"15min":1.014},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.49,"5min":1.49,"15min":1.49},"last":0.89500000000000002}]},{"osd":4,"last update":"Wed Sep 18 10:16:32 2024","interfaces":[{"interface":"back","average":{"1min":1.3580000000000001,"5min":9.0879999999999992,"15min":10.375999999999999},"min":{"1min":0.747,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":8.0449999999999999,"5min":99.745999999999995,"15min":99.745999999999995},"last":1.2729999999999999},{"interface":"front","average":{"1min":1.595,"5min":9.2309999999999999,"15min":10.503},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":8.0679999999999996,"5min":99.781000000000006,"15min":99.781000000000006},"last":1.222}]},{"osd":5,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":1.0860000000000001},{"interface":"front","average":{"1min":1.7869999999999999,"5min":1.7869999999999999,"15min":1.7869999999999999},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":10.874000000000001,"5min":10.874000000000001,"15min":10.874000000000001},"last":1.163}]},{"osd":6,"last update":"Wed Sep 18 10:16:32 2024","interfaces":[{"interface":"back","average":{"1min":14.140000000000001,"5min":4.4039999999999999,"15min":2.782},"min":{"1min":0.92900000000000005,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":269.80500000000001,"5min":269.80500000000001,"15min":269.80500000000001},"last":1},{"interface":"front","average":{"1min":1.361,"5min":4.7610000000000001,"15min":5.327},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":2.012,"5min":74.156000000000006,"15min":74.156000000000006},"last":1.1919999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80400000000000005}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93000000000000005}]},{"osd":10,"last update":"Wed Sep 18 10:16:32 2024","interfaces":[{"interface":"back","average":{"1min":5.8639999999999999,"5min":6.2670000000000003,"15min":6.3339999999999996},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":93.704999999999998,"5min":93.704999999999998,"15min":93.704999999999998},"last":0.85399999999999998},{"interface":"front","average":{"1min":5.7770000000000001,"5min":6.2549999999999999,"15min":6.335},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":93.671000000000006,"5min":93.671000000000006,"15min":93.671000000000006},"last":1.121}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.254}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3680000000000001}]},{"osd":13,"last update":"Wed Sep 18 10:16:32 2024","interfaces":[{"interface":"back","average":{"1min":1.99,"5min":5.2629999999999999,"15min":5.8090000000000002},"min":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"max":{"1min":7.3179999999999996,"5min":80.197000000000003,"15min":80.197000000000003},"last":1.5029999999999999},{"interface":"front","average":{"1min":3.4830000000000001,"5min":1.8080000000000001,"15min":1.5289999999999999},"min":{"1min":0.89400000000000002,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":25.193999999999999,"5min":25.193999999999999,"15min":25.193999999999999},"last":1.4670000000000001}]},{"osd":14,"last update":"Wed Sep 18 10:16:32 2024","interfaces":[{"interface":"back","average":{"1min":10.725,"5min":3.2999999999999998,"15min":2.0619999999999998},"min":{"1min":0.95699999999999996,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":117.562,"5min":117.562,"15min":117.562},"last":1.5980000000000001},{"interface":"front","average":{"1min":4.6139999999999999,"5min":9.5229999999999997,"15min":10.340999999999999},"min":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"max":{"1min":61.567,"5min":85.006,"15min":85.006},"last":1.4390000000000001}]}]},{"osd":14,"up_from":22,"seq":94489280543,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303768,"kb_used_data":2248,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068072,"statfs":{"total":96636764160,"available":96325705728,"internally_reserved":0,"allocated":2301952,"data_stored":1341063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,5,7,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:16:32 2024","interfaces":[{"interface":"back","average":{"1min":1.0009999999999999,"5min":12.513,"15min":14.430999999999999},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.405,"5min":189.88800000000001,"15min":189.88800000000001},"last":0.59599999999999997},{"interface":"front","average":{"1min":7.0330000000000004,"5min":17.033000000000001,"15min":18.699000000000002},"min":{"1min":0.77300000000000002,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":71.900999999999996,"5min":180.09999999999999,"15min":180.09999999999999},"last":0.68700000000000006}]},{"osd":1,"last update":"Wed Sep 18 10:16:32 2024","interfaces":[{"interface":"back","average":{"1min":2.1040000000000001,"5min":7.7670000000000003,"15min":8.7110000000000003},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":21.334,"5min":79.626000000000005,"15min":79.626000000000005},"last":0.754},{"interface":"front","average":{"1min":1.1479999999999999,"5min":12.601000000000001,"15min":14.51},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.8879999999999999,"5min":189.762,"15min":189.762},"last":0.85899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.417}]},{"osd":3,"last update":"Wed Sep 18 10:16:32 2024","interfaces":[{"interface":"back","average":{"1min":3.5430000000000001,"5min":12.906000000000001,"15min":14.465999999999999},"min":{"1min":0.84799999999999998,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":44.683,"5min":180.19900000000001,"15min":180.19900000000001},"last":1.7529999999999999},{"interface":"front","average":{"1min":5.3520000000000003,"5min":18.143999999999998,"15min":20.276},"min":{"1min":0.86399999999999999,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":42.570999999999998,"5min":180.851,"15min":180.851},"last":1.1899999999999999}]},{"osd":4,"last update":"Wed Sep 18 10:16:32 2024","interfaces":[{"interface":"back","average":{"1min":6.8360000000000003,"5min":21.687000000000001,"15min":24.161999999999999},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":71.911000000000001,"5min":189.809,"15min":189.809},"last":2.0249999999999999},{"interface":"front","average":{"1min":14.032999999999999,"5min":15.031000000000001,"15min":15.196999999999999},"min":{"1min":0.76100000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":133.244,"5min":180.31700000000001,"15min":180.31700000000001},"last":1.9350000000000001}]},{"osd":5,"last update":"Wed Sep 18 10:16:32 2024","interfaces":[{"interface":"back","average":{"1min":1.831,"5min":21.507000000000001,"15min":24.786000000000001},"min":{"1min":0.66300000000000003,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":13.42,"5min":277.41199999999998,"15min":277.41199999999998},"last":1.1020000000000001},{"interface":"front","average":{"1min":1.9530000000000001,"5min":12.936999999999999,"15min":14.766999999999999},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":14.288,"5min":189.64699999999999,"15min":189.64699999999999},"last":0.91900000000000004}]},{"osd":7,"last update":"Wed Sep 18 10:16:32 2024","interfaces":[{"interface":"back","average":{"1min":6.29,"5min":13.625999999999999,"15min":14.848000000000001},"min":{"1min":0.995,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":49.378,"5min":180.268,"15min":180.268},"last":1.448},{"interface":"front","average":{"1min":3.8929999999999998,"5min":12.901,"15min":14.403},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":49.402999999999999,"5min":180.631,"15min":180.631},"last":1.494}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1619999999999999}]},{"osd":9,"last update":"Wed Sep 18 10:16:17 2024","interfaces":[{"interface":"back","average":{"1min":4,"5min":4,"15min":4},"min":{"1min":1.296,"5min":1.296,"15min":1.296},"max":{"1min":44.716000000000001,"5min":44.716000000000001,"15min":44.716000000000001},"last":1.204},{"interface":"front","average":{"1min":1.427,"5min":1.427,"15min":1.427},"min":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"max":{"1min":1.9610000000000001,"5min":1.9610000000000001,"15min":1.9610000000000001},"last":1.274}]},{"osd":10,"last update":"Wed Sep 18 10:15:49 2024","interfaces":[{"interface":"back","average":{"1min":17.039000000000001,"5min":17.039000000000001,"15min":17.039000000000001},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":180.15899999999999,"5min":180.15899999999999,"15min":180.15899999999999},"last":1},{"interface":"front","average":{"1min":31.004000000000001,"5min":31.004000000000001,"15min":31.004000000000001},"min":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":269.423,"5min":269.423,"15min":269.423},"last":1.381}]},{"osd":11,"last update":"Wed Sep 18 10:16:10 2024","interfaces":[{"interface":"back","average":{"1min":9.6890000000000001,"5min":9.6890000000000001,"15min":9.6890000000000001},"min":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":91.111999999999995,"5min":91.111999999999995,"15min":91.111999999999995},"last":1.3520000000000001},{"interface":"front","average":{"1min":8.1460000000000008,"5min":8.1460000000000008,"15min":8.1460000000000008},"min":{"1min":1.181,"5min":1.181,"15min":1.181},"max":{"1min":91.165000000000006,"5min":91.165000000000006,"15min":91.165000000000006},"last":1.1519999999999999}]},{"osd":12,"last update":"Wed Sep 18 10:16:32 2024","interfaces":[{"interface":"back","average":{"1min":10.385,"5min":14.760999999999999,"15min":15.49},"min":{"1min":1.327,"5min":0.91100000000000003,"15min":0.91100000000000003},"max":{"1min":65.189999999999998,"5min":189.84999999999999,"15min":189.84999999999999},"last":1.4079999999999999},{"interface":"front","average":{"1min":10.818,"5min":14.696,"15min":15.342000000000001},"min":{"1min":1.4890000000000001,"5min":0.91200000000000003,"15min":0.91200000000000003},"max":{"1min":65.162000000000006,"5min":180.381,"15min":180.381},"last":1.7729999999999999}]},{"osd":13,"last update":"Wed Sep 18 10:16:32 2024","interfaces":[{"interface":"back","average":{"1min":10.321999999999999,"5min":23.059000000000001,"15min":25.181999999999999},"min":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"max":{"1min":166.001,"5min":180.59100000000001,"15min":180.59100000000001},"last":1.5720000000000001},{"interface":"front","average":{"1min":1.647,"5min":20.391999999999999,"15min":23.515999999999998},"min":{"1min":0.83399999999999996,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":2.3940000000000001,"5min":189.71299999999999,"15min":189.71299999999999},"last":1.3120000000000001}]},{"osd":15,"last update":"Wed Sep 18 10:16:32 2024","interfaces":[{"interface":"back","average":{"1min":4.181,"5min":13.381,"15min":14.914},"min":{"1min":1.5780000000000001,"5min":0.99399999999999999,"15min":0.99399999999999999},"max":{"1min":44.917000000000002,"5min":180.44800000000001,"15min":180.44800000000001},"last":1.6619999999999999},{"interface":"front","average":{"1min":1.7030000000000001,"5min":15.118,"15min":17.353999999999999},"min":{"1min":0.93200000000000005,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":2.1909999999999998,"5min":180.53,"15min":180.53},"last":1.8069999999999999}]}]},{"osd":13,"up_from":22,"seq":94489280544,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307932,"kb_used_data":2252,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94063908,"statfs":{"total":96636764160,"available":96321441792,"internally_reserved":0,"allocated":2306048,"data_stored":1341082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,3,6,8,9,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:16:33 2024","interfaces":[{"interface":"back","average":{"1min":2.2599999999999998,"5min":17.045000000000002,"15min":19.509},"min":{"1min":0.65700000000000003,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":22.972999999999999,"5min":176.66800000000001,"15min":176.66800000000001},"last":0.83899999999999997},{"interface":"front","average":{"1min":4.2629999999999999,"5min":25.199000000000002,"15min":28.687999999999999},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":62.533000000000001,"5min":199.732,"15min":199.732},"last":0.90000000000000002}]},{"osd":1,"last update":"Wed Sep 18 10:15:48 2024","interfaces":[{"interface":"back","average":{"1min":1.986,"5min":1.986,"15min":1.986},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":18.471,"5min":18.471,"15min":18.471},"last":1.4219999999999999},{"interface":"front","average":{"1min":20.716999999999999,"5min":20.716999999999999,"15min":20.716999999999999},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":176.56800000000001,"5min":176.56800000000001,"15min":176.56800000000001},"last":0.86399999999999999}]},{"osd":2,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":11.984,"5min":11.984,"15min":11.984},"min":{"1min":0.751,"5min":0.751,"15min":0.751},"max":{"1min":196.59200000000001,"5min":196.59200000000001,"15min":196.59200000000001},"last":0.97699999999999998},{"interface":"front","average":{"1min":26.109999999999999,"5min":26.109999999999999,"15min":26.109999999999999},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":177.03,"5min":177.03,"15min":177.03},"last":1.2689999999999999}]},{"osd":3,"last update":"Wed Sep 18 10:16:33 2024","interfaces":[{"interface":"back","average":{"1min":9.5,"5min":22.245000000000001,"15min":24.369},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":158.74100000000001,"5min":168.27699999999999,"15min":168.27699999999999},"last":1.27},{"interface":"front","average":{"1min":1.2290000000000001,"5min":9.8819999999999997,"15min":11.324},"min":{"1min":0.84499999999999997,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.8959999999999999,"5min":196.643,"15min":196.643},"last":0.95399999999999996}]},{"osd":4,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":29.802,"5min":29.802,"15min":29.802},"min":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"max":{"1min":168.48099999999999,"5min":168.48099999999999,"15min":168.48099999999999},"last":1.347},{"interface":"front","average":{"1min":8.1780000000000008,"5min":8.1780000000000008,"15min":8.1780000000000008},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":65.188000000000002,"5min":65.188000000000002,"15min":65.188000000000002},"last":1.6379999999999999}]},{"osd":5,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":26.152999999999999,"5min":26.152999999999999,"15min":26.152999999999999},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":176.48099999999999,"5min":176.48099999999999,"15min":176.48099999999999},"last":1.49},{"interface":"front","average":{"1min":24.885999999999999,"5min":24.885999999999999,"15min":24.885999999999999},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":167.62299999999999,"5min":167.62299999999999,"15min":167.62299999999999},"last":1.528}]},{"osd":6,"last update":"Wed Sep 18 10:16:33 2024","interfaces":[{"interface":"back","average":{"1min":2.6589999999999998,"5min":24.015000000000001,"15min":27.574999999999999},"min":{"1min":1.042,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":23.131,"5min":176.81200000000001,"15min":176.81200000000001},"last":1.1080000000000001},{"interface":"front","average":{"1min":4.7169999999999996,"5min":23.227,"15min":26.312000000000001},"min":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"max":{"1min":61.874000000000002,"5min":167.947,"15min":167.947},"last":1.3340000000000001}]},{"osd":8,"last update":"Wed Sep 18 10:16:50 2024","interfaces":[{"interface":"back","average":{"1min":1.5700000000000001,"5min":1.5700000000000001,"15min":1.5700000000000001},"min":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"max":{"1min":2.117,"5min":2.117,"15min":2.117},"last":1.175},{"interface":"front","average":{"1min":1.514,"5min":1.514,"15min":1.514},"min":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"max":{"1min":2.278,"5min":2.278,"15min":2.278},"last":1.149}]},{"osd":9,"last update":"Wed Sep 18 10:16:50 2024","interfaces":[{"interface":"back","average":{"1min":2.617,"5min":17.367000000000001,"15min":19.824999999999999},"min":{"1min":0.878,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":22.934000000000001,"5min":176.73599999999999,"15min":176.73599999999999},"last":1.206},{"interface":"front","average":{"1min":1.5660000000000001,"5min":19.085000000000001,"15min":22.004999999999999},"min":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":2.448,"5min":167.84899999999999,"15min":167.84899999999999},"last":1.446}]},{"osd":10,"last update":"Wed Sep 18 10:16:39 2024","interfaces":[{"interface":"back","average":{"1min":5.1120000000000001,"5min":19.164999999999999,"15min":21.507000000000001},"min":{"1min":1.22,"5min":1.1359999999999999,"15min":1.1359999999999999},"max":{"1min":62.137999999999998,"5min":168.07900000000001,"15min":168.07900000000001},"last":1.591},{"interface":"front","average":{"1min":17.683,"5min":29.114000000000001,"15min":31.018999999999998},"min":{"1min":1.123,"5min":1.0649999999999999,"15min":1.0649999999999999},"max":{"1min":289.68700000000001,"5min":289.68700000000001,"15min":289.68700000000001},"last":1.2450000000000001}]},{"osd":11,"last update":"Wed Sep 18 10:16:50 2024","interfaces":[{"interface":"back","average":{"1min":1.6819999999999999,"5min":1.625,"15min":1.615},"min":{"1min":1.056,"5min":1.0029999999999999,"15min":1.0029999999999999},"max":{"1min":2.5699999999999998,"5min":2.5699999999999998,"15min":2.5699999999999998},"last":1.5009999999999999},{"interface":"front","average":{"1min":2.798,"5min":21.73,"15min":24.885999999999999},"min":{"1min":1.286,"5min":1.028,"15min":1.028},"max":{"1min":23.167999999999999,"5min":176.94999999999999,"15min":176.94999999999999},"last":1.377}]},{"osd":12,"last update":"Wed Sep 18 10:16:33 2024","interfaces":[{"interface":"back","average":{"1min":1.8440000000000001,"5min":1.6339999999999999,"15min":1.599},"min":{"1min":1.2490000000000001,"5min":1.0009999999999999,"15min":1.0009999999999999},"max":{"1min":2.5950000000000002,"5min":2.7879999999999998,"15min":2.7879999999999998},"last":1.413},{"interface":"front","average":{"1min":5.0570000000000004,"5min":18.542999999999999,"15min":20.791},"min":{"1min":1.3029999999999999,"5min":1.296,"15min":1.296},"max":{"1min":62.200000000000003,"5min":168.34399999999999,"15min":168.34399999999999},"last":1.5449999999999999}]},{"osd":14,"last update":"Wed Sep 18 10:16:33 2024","interfaces":[{"interface":"back","average":{"1min":1.8320000000000001,"5min":5.7210000000000001,"15min":6.3689999999999998},"min":{"1min":1.139,"5min":1.0640000000000001,"15min":1.0640000000000001},"max":{"1min":2.419,"5min":87.427000000000007,"15min":87.427000000000007},"last":1.665},{"interface":"front","average":{"1min":2.9329999999999998,"5min":21.571999999999999,"15min":24.678000000000001},"min":{"1min":1.2709999999999999,"5min":1.0980000000000001,"15min":1.0980000000000001},"max":{"1min":23.268000000000001,"5min":176.87700000000001,"15min":176.87700000000001},"last":1.468}]},{"osd":15,"last update":"Wed Sep 18 10:16:33 2024","interfaces":[{"interface":"back","average":{"1min":10.048999999999999,"5min":18.277000000000001,"15min":19.648},"min":{"1min":1.272,"5min":1.266,"15min":1.266},"max":{"1min":96.564999999999998,"5min":168.40700000000001,"15min":168.40700000000001},"last":1.7110000000000001},{"interface":"front","average":{"1min":1.891,"5min":13.459,"15min":15.387},"min":{"1min":1.1950000000000001,"5min":0.95099999999999996,"15min":0.95099999999999996},"max":{"1min":2.5169999999999999,"5min":196.512,"15min":196.512},"last":1.5609999999999999}]}]},{"osd":0,"up_from":21,"seq":90194313248,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303780,"kb_used_data":2264,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068060,"statfs":{"total":96636764160,"available":96325693440,"internally_reserved":0,"allocated":2318336,"data_stored":1350579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,4,5,6,9,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 18 10:16:39 2024","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.95699999999999996,"15min":0.95899999999999996},"min":{"1min":0.56699999999999995,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.2889999999999999,"5min":1.341,"15min":1.341},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.97899999999999998,"15min":0.98499999999999999},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.2310000000000001,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":0.625}]},{"osd":2,"last update":"Wed Sep 18 10:16:39 2024","interfaces":[{"interface":"back","average":{"1min":3.2719999999999998,"5min":1.6020000000000001,"15min":1.3240000000000001},"min":{"1min":0.76600000000000001,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":46.220999999999997,"5min":46.220999999999997,"15min":46.220999999999997},"last":1.0349999999999999},{"interface":"front","average":{"1min":0.98799999999999999,"5min":1.071,"15min":1.085},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.732,"5min":1.732,"15min":1.732},"last":0.57499999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.196}]},{"osd":4,"last update":"Wed Sep 18 10:16:30 2024","interfaces":[{"interface":"back","average":{"1min":8.4039999999999999,"5min":14,"15min":14.933},"min":{"1min":0.59699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":142.20699999999999,"5min":142.20699999999999,"15min":142.20699999999999},"last":0.49399999999999999},{"interface":"front","average":{"1min":10.48,"5min":14.896000000000001,"15min":15.632},"min":{"1min":0.63,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":132.33799999999999,"5min":132.33799999999999,"15min":132.33799999999999},"last":0.55200000000000005}]},{"osd":5,"last update":"Wed Sep 18 10:16:39 2024","interfaces":[{"interface":"back","average":{"1min":1.1739999999999999,"5min":1.24,"15min":1.2509999999999999},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":1.619,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":1.569,"5min":8.1340000000000003,"15min":9.2279999999999998},"min":{"1min":0.85599999999999998,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":10.143000000000001,"5min":130.39599999999999,"15min":130.39599999999999},"last":0.92100000000000004}]},{"osd":6,"last update":"Wed Sep 18 10:16:47 2024","interfaces":[{"interface":"back","average":{"1min":1.266,"5min":1.254,"15min":1.252},"min":{"1min":0.96099999999999997,"5min":0.94299999999999995,"15min":0.94299999999999995},"max":{"1min":1.571,"5min":1.776,"15min":1.776},"last":0.82599999999999996},{"interface":"front","average":{"1min":1.3109999999999999,"5min":2.0270000000000001,"15min":2.1469999999999998},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.7909999999999999,"5min":15.250999999999999,"15min":15.250999999999999},"last":0.66100000000000003}]},{"osd":9,"last update":"Wed Sep 18 10:16:30 2024","interfaces":[{"interface":"back","average":{"1min":1.3360000000000001,"5min":1.3919999999999999,"15min":1.4019999999999999},"min":{"1min":0.88200000000000001,"5min":0.879,"15min":0.879},"max":{"1min":1.863,"5min":1.863,"15min":1.863},"last":0.74099999999999999},{"interface":"front","average":{"1min":1.667,"5min":8.7590000000000003,"15min":9.9410000000000007},"min":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":10.180999999999999,"5min":130.441,"15min":130.441},"last":0.88400000000000001}]},{"osd":11,"last update":"Wed Sep 18 10:16:30 2024","interfaces":[{"interface":"back","average":{"1min":4.8949999999999996,"5min":2.2109999999999999,"15min":1.7629999999999999},"min":{"1min":1.1579999999999999,"5min":0.90700000000000003,"15min":0.90700000000000003},"max":{"1min":41.610999999999997,"5min":41.610999999999997,"15min":41.610999999999997},"last":1.0700000000000001},{"interface":"front","average":{"1min":1.5029999999999999,"5min":4.5620000000000003,"15min":5.0720000000000001},"min":{"1min":1.1220000000000001,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":1.9850000000000001,"5min":54.805999999999997,"15min":54.805999999999997},"last":0.87}]},{"osd":12,"last update":"Wed Sep 18 10:16:30 2024","interfaces":[{"interface":"back","average":{"1min":1.7869999999999999,"5min":9.1170000000000009,"15min":10.339},"min":{"1min":0.95999999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":10.015000000000001,"5min":130.34800000000001,"15min":130.34800000000001},"last":0.95399999999999996},{"interface":"front","average":{"1min":1.4350000000000001,"5min":1.4710000000000001,"15min":1.4770000000000001},"min":{"1min":0.85899999999999999,"5min":0.502,"15min":0.502},"max":{"1min":1.944,"5min":1.944,"15min":1.944},"last":0.85499999999999998}]},{"osd":14,"last update":"Wed Sep 18 10:16:39 2024","interfaces":[{"interface":"back","average":{"1min":3.7490000000000001,"5min":8.7810000000000006,"15min":9.6199999999999992},"min":{"1min":1.044,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":38.969999999999999,"5min":130.48500000000001,"15min":130.48500000000001},"last":0.98699999999999999},{"interface":"front","average":{"1min":6.931,"5min":2.6339999999999999,"15min":1.9179999999999999},"min":{"1min":1.2609999999999999,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":80.460999999999999,"5min":80.460999999999999,"15min":80.460999999999999},"last":1.099}]},{"osd":15,"last update":"Wed Sep 18 10:16:39 2024","interfaces":[{"interface":"back","average":{"1min":1.591,"5min":2.778,"15min":2.976},"min":{"1min":1.1699999999999999,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":2.0310000000000001,"5min":23.806999999999999,"15min":23.806999999999999},"last":0.77200000000000002},{"interface":"front","average":{"1min":6.9199999999999999,"5min":9.4809999999999999,"15min":9.9079999999999995},"min":{"1min":1.1279999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":100.925,"5min":130.52799999999999,"15min":130.52799999999999},"last":1.018}]}]},{"osd":4,"up_from":21,"seq":90194313249,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303752,"kb_used_data":2232,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068088,"statfs":{"total":96636764160,"available":96325722112,"internally_reserved":0,"allocated":2285568,"data_stored":1331589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,3,5,7,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":9.0980000000000008,"15min":10.465999999999999},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.7529999999999999,"5min":89.266999999999996,"15min":89.266999999999996},"last":0.626},{"interface":"front","average":{"1min":18.640000000000001,"5min":7.7320000000000002,"15min":5.9139999999999997},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":198.80699999999999,"5min":198.80699999999999,"15min":198.80699999999999},"last":0.66000000000000003}]},{"osd":1,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":4.0759999999999996,"5min":12.537000000000001,"15min":13.946999999999999},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":62.780000000000001,"5min":164.21199999999999,"15min":164.21199999999999},"last":0.72099999999999997},{"interface":"front","average":{"1min":2.5139999999999998,"5min":9.593,"15min":10.773},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":25.802,"5min":89.331000000000003,"15min":89.331000000000003},"last":0.77500000000000002}]},{"osd":2,"last update":"Wed Sep 18 10:15:47 2024","interfaces":[{"interface":"back","average":{"1min":23.617000000000001,"5min":23.617000000000001,"15min":23.617000000000001},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":192.136,"5min":192.136,"15min":192.136},"last":1.8740000000000001},{"interface":"front","average":{"1min":11.888999999999999,"5min":11.888999999999999,"15min":11.888999999999999},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":198.87700000000001,"5min":198.87700000000001,"15min":198.87700000000001},"last":1.347}]},{"osd":3,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":35.673000000000002,"5min":21.670999999999999,"15min":19.337},"min":{"1min":1.161,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":388.84300000000002,"5min":388.84300000000002,"15min":388.84300000000002},"last":1.462},{"interface":"front","average":{"1min":17.747,"5min":25.643999999999998,"15min":26.960000000000001},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":255.82900000000001,"5min":255.82900000000001,"15min":255.82900000000001},"last":0.997}]},{"osd":5,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":1.4910000000000001,"5min":15.449999999999999,"15min":17.776},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":2.169,"5min":189.55600000000001,"15min":189.55600000000001},"last":0.83099999999999996},{"interface":"front","average":{"1min":21.053000000000001,"5min":6.8079999999999998,"15min":4.4340000000000002},"min":{"1min":1.3640000000000001,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":199.15600000000001,"5min":199.15600000000001,"15min":199.15600000000001},"last":1.155}]},{"osd":6,"last update":"Wed Sep 18 10:15:34 2024","interfaces":[{"interface":"back","average":{"1min":32.25,"5min":32.25,"15min":32.25},"min":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":197.33199999999999,"5min":197.33199999999999,"15min":197.33199999999999},"last":1.8160000000000001},{"interface":"front","average":{"1min":18.963000000000001,"5min":18.963000000000001,"15min":18.963000000000001},"min":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"max":{"1min":179.19200000000001,"5min":179.19200000000001,"15min":179.19200000000001},"last":1.7769999999999999}]},{"osd":7,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":5.6050000000000004,"5min":23.198,"15min":26.129999999999999},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":62.704999999999998,"5min":290.036,"15min":290.036},"last":1.2689999999999999},{"interface":"front","average":{"1min":1.6639999999999999,"5min":14.273,"15min":16.375},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":2.3100000000000001,"5min":139.696,"15min":139.696},"last":0.93899999999999995}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1509999999999998}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7509999999999999}]},{"osd":10,"last update":"Wed Sep 18 10:16:49 2024","interfaces":[{"interface":"back","average":{"1min":1.653,"5min":18.559000000000001,"15min":21.376999999999999},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":2.4740000000000002,"5min":164.43899999999999,"15min":164.43899999999999},"last":0.89800000000000002},{"interface":"front","average":{"1min":4.6070000000000002,"5min":10.993,"15min":12.057},"min":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":59.408999999999999,"5min":89.442999999999998,"15min":89.442999999999998},"last":1.3049999999999999}]},{"osd":11,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":19.57,"5min":12.567,"15min":11.4},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":199.255,"5min":199.255,"15min":199.255},"last":1.3340000000000001},{"interface":"front","average":{"1min":11.393000000000001,"5min":22.213000000000001,"15min":24.016999999999999},"min":{"1min":1.0840000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"max":{"1min":155.19800000000001,"5min":164.733,"15min":164.733},"last":1.4399999999999999}]},{"osd":12,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":4.5800000000000001,"5min":10.406000000000001,"15min":11.377000000000001},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":58.856000000000002,"5min":89.400000000000006,"15min":89.400000000000006},"last":1.113},{"interface":"front","average":{"1min":31.831,"5min":12.901999999999999,"15min":9.7469999999999999},"min":{"1min":1.4530000000000001,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":199.32900000000001,"5min":199.32900000000001,"15min":199.32900000000001},"last":1.5940000000000001}]},{"osd":13,"last update":"Wed Sep 18 10:16:49 2024","interfaces":[{"interface":"back","average":{"1min":12.695,"5min":22.789999999999999,"15min":24.472000000000001},"min":{"1min":1.702,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":192.37299999999999,"5min":199.37799999999999,"15min":199.37799999999999},"last":1.706},{"interface":"front","average":{"1min":1.8029999999999999,"5min":19.068000000000001,"15min":21.946000000000002},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":2.5699999999999998,"5min":164.53,"15min":164.53},"last":1.375}]},{"osd":14,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":22.312000000000001,"5min":21.788,"15min":21.701000000000001},"min":{"1min":1.556,"5min":0.96099999999999997,"15min":0.96099999999999997},"max":{"1min":199.43000000000001,"5min":202.191,"15min":202.191},"last":1.782},{"interface":"front","average":{"1min":9.8800000000000008,"5min":17.420999999999999,"15min":18.678000000000001},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":101.492,"5min":164.35400000000001,"15min":164.35400000000001},"last":1.524}]}]},{"osd":9,"up_from":21,"seq":90194313248,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303784,"kb_used_data":2264,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068056,"statfs":{"total":96636764160,"available":96325689344,"internally_reserved":0,"allocated":2318336,"data_stored":1350579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,4,5,6,8,10,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":7.2450000000000001,"5min":7.2450000000000001,"15min":7.2450000000000001},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":127.902,"5min":127.902,"15min":127.902},"last":0.97399999999999998},{"interface":"front","average":{"1min":8.3109999999999999,"5min":8.3109999999999999,"15min":8.3109999999999999},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":96.203000000000003,"5min":96.203000000000003,"15min":96.203000000000003},"last":0.81200000000000006}]},{"osd":1,"last update":"Wed Sep 18 10:16:40 2024","interfaces":[{"interface":"back","average":{"1min":10.587,"5min":10.587,"15min":10.587},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":166.833,"5min":166.833,"15min":166.833},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.96099999999999997}]},{"osd":2,"last update":"Wed Sep 18 10:16:18 2024","interfaces":[{"interface":"back","average":{"1min":5.6420000000000003,"5min":5.6420000000000003,"15min":5.6420000000000003},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":68.049999999999997,"5min":68.049999999999997,"15min":68.049999999999997},"last":1.1579999999999999},{"interface":"front","average":{"1min":2.0310000000000001,"5min":2.0310000000000001,"15min":2.0310000000000001},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":18.553999999999998,"5min":18.553999999999998,"15min":18.553999999999998},"last":1.4119999999999999}]},{"osd":3,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":2.875,"5min":5.5549999999999997,"15min":6.0019999999999998},"min":{"1min":0.72699999999999998,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":18.431000000000001,"5min":98.888999999999996,"15min":98.888999999999996},"last":1.0389999999999999},{"interface":"front","average":{"1min":3.0169999999999999,"5min":5.5890000000000004,"15min":6.0170000000000003},"min":{"1min":0.749,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":18.472000000000001,"5min":98.947999999999993,"15min":98.947999999999993},"last":1.546}]},{"osd":4,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":8.2360000000000007,"5min":2.5840000000000001,"15min":1.6419999999999999},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":124.15300000000001,"5min":124.15300000000001,"15min":124.15300000000001},"last":1.371},{"interface":"front","average":{"1min":6.2380000000000004,"5min":8.8539999999999992,"15min":9.2899999999999991},"min":{"1min":0.81000000000000005,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":73.992000000000004,"5min":73.992000000000004,"15min":73.992000000000004},"last":1.6319999999999999}]},{"osd":5,"last update":"Wed Sep 18 10:16:18 2024","interfaces":[{"interface":"back","average":{"1min":1.821,"5min":1.821,"15min":1.821},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":14.786,"5min":14.786,"15min":14.786},"last":0.79800000000000004},{"interface":"front","average":{"1min":8.5,"5min":8.5,"15min":8.5},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":67.915999999999997,"5min":67.915999999999997,"15min":67.915999999999997},"last":1.27}]},{"osd":6,"last update":"Wed Sep 18 10:16:18 2024","interfaces":[{"interface":"back","average":{"1min":8.3840000000000003,"5min":8.3840000000000003,"15min":8.3840000000000003},"min":{"1min":0.997,"5min":0.997,"15min":0.997},"max":{"1min":70.914000000000001,"5min":70.914000000000001,"15min":70.914000000000001},"last":1.806},{"interface":"front","average":{"1min":15.705,"5min":15.705,"15min":15.705},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":136.38800000000001,"5min":136.38800000000001,"15min":136.38800000000001},"last":1.2}]},{"osd":8,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":3.0310000000000001,"5min":4.2789999999999999,"15min":4.4870000000000001},"min":{"1min":1.0509999999999999,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":30.713000000000001,"5min":64.912999999999997,"15min":64.912999999999997},"last":1.458},{"interface":"front","average":{"1min":2.0569999999999999,"5min":5.3959999999999999,"15min":5.952},"min":{"1min":0.82499999999999996,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":12.898999999999999,"5min":57.664000000000001,"15min":57.664000000000001},"last":1.857}]},{"osd":10,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":1.3400000000000001,"5min":1.3089999999999999,"15min":1.304},"min":{"1min":0.88300000000000001,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.8180000000000001,"5min":2.109,"15min":2.109},"last":1.51},{"interface":"front","average":{"1min":6.0389999999999997,"5min":4.2309999999999999,"15min":3.9300000000000002},"min":{"1min":0.97899999999999998,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":68.216999999999999,"5min":68.216999999999999,"15min":68.216999999999999},"last":1.661}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.669}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.446}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":96}]},{"osd":14,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":4.3579999999999997,"5min":8.8759999999999994,"15min":9.6289999999999996},"min":{"1min":0.86499999999999999,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":50.773000000000003,"5min":62.899000000000001,"15min":62.899000000000001},"last":1.9039999999999999},{"interface":"front","average":{"1min":4.3540000000000001,"5min":13.071,"15min":14.523999999999999},"min":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":50.709000000000003,"5min":155.61000000000001,"15min":155.61000000000001},"last":1.6950000000000001}]},{"osd":15,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":1.4630000000000001,"5min":1.492,"15min":1.4970000000000001},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.97,"5min":1.9970000000000001,"15min":1.9970000000000001},"last":1.605},{"interface":"front","average":{"1min":9.5700000000000003,"5min":5.1749999999999998,"15min":4.4429999999999996},"min":{"1min":1.0980000000000001,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":68.140000000000001,"5min":68.140000000000001,"15min":68.140000000000001},"last":1.7470000000000001}]}]},{"osd":11,"up_from":21,"seq":90194313248,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303784,"kb_used_data":2264,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068056,"statfs":{"total":96636764160,"available":96325689344,"internally_reserved":0,"allocated":2318336,"data_stored":1350579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[2,3,6,8,9,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0049999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86599999999999999}]},{"osd":2,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":8.3390000000000004,"15min":9.5719999999999992},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.3740000000000001,"5min":99.718000000000004,"15min":99.718000000000004},"last":0.88200000000000001},{"interface":"front","average":{"1min":3.6349999999999998,"5min":1.4450000000000001,"15min":1.0800000000000001},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":49.774000000000001,"5min":49.774000000000001,"15min":49.774000000000001},"last":0.93400000000000005}]},{"osd":3,"last update":"Wed Sep 18 10:16:46 2024","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":4.2210000000000001,"15min":4.7880000000000003},"min":{"1min":0.48399999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.369,"5min":85.963999999999999,"15min":85.963999999999999},"last":0.82299999999999995},{"interface":"front","average":{"1min":5.4660000000000002,"5min":19.300000000000001,"15min":21.606000000000002},"min":{"1min":0.73699999999999999,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":80.980999999999995,"5min":125.672,"15min":125.672},"last":1.073}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.444}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5980000000000001}]},{"osd":6,"last update":"Wed Sep 18 10:16:46 2024","interfaces":[{"interface":"back","average":{"1min":5.8810000000000002,"5min":2.101,"15min":1.4710000000000001},"min":{"1min":0.70599999999999996,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":86.680000000000007,"5min":86.680000000000007,"15min":86.680000000000007},"last":1.21},{"interface":"front","average":{"1min":1.1830000000000001,"5min":5.827,"15min":6.601},"min":{"1min":0.73799999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.49,"5min":94.162000000000006,"15min":94.162000000000006},"last":1.109}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.524}]},{"osd":8,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":2.7069999999999999,"5min":5.3730000000000002,"15min":5.8170000000000002},"min":{"1min":0.95299999999999996,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":27.577999999999999,"5min":85.900000000000006,"15min":85.900000000000006},"last":1.0189999999999999},{"interface":"front","average":{"1min":2.544,"5min":8.4960000000000004,"15min":9.4879999999999995},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":27.542000000000002,"5min":99.765000000000001,"15min":99.765000000000001},"last":1.1559999999999999}]},{"osd":9,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":5.8760000000000003,"5min":5.8339999999999996,"15min":5.827},"min":{"1min":0.96399999999999997,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":89.844999999999999,"5min":89.844999999999999,"15min":89.844999999999999},"last":1.1779999999999999},{"interface":"front","average":{"1min":1.351,"5min":6.8070000000000004,"15min":7.7160000000000002},"min":{"1min":0.68600000000000005,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.742,"5min":99.810000000000002,"15min":99.810000000000002},"last":1.2470000000000001}]},{"osd":10,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":4.0069999999999997,"5min":1.679,"15min":1.2909999999999999},"min":{"1min":0.98499999999999999,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":49.819000000000003,"5min":49.819000000000003,"15min":49.819000000000003},"last":1.361},{"interface":"front","average":{"1min":1.5580000000000001,"5min":6.46,"15min":7.2770000000000001},"min":{"1min":0.98699999999999999,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.976,"5min":89.290000000000006,"15min":89.290000000000006},"last":1.274}]},{"osd":12,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":5.907,"5min":2.081,"15min":1.4430000000000001},"min":{"1min":0.96599999999999997,"5min":0.498,"15min":0.498},"max":{"1min":87.463999999999999,"5min":87.463999999999999,"15min":87.463999999999999},"last":1.7270000000000001},{"interface":"front","average":{"1min":1.5660000000000001,"5min":8.6219999999999999,"15min":9.798},"min":{"1min":1.0700000000000001,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":2.0600000000000001,"5min":86.025999999999996,"15min":86.025999999999996},"last":1.494}]},{"osd":13,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":4.4210000000000003,"5min":25.207999999999998,"15min":28.672000000000001},"min":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"max":{"1min":59.841999999999999,"5min":189.25899999999999,"15min":189.25899999999999},"last":1.476},{"interface":"front","average":{"1min":7.569,"5min":2.5659999999999998,"15min":1.732},"min":{"1min":0.88900000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":71.313000000000002,"5min":71.313000000000002,"15min":71.313000000000002},"last":1.6439999999999999}]},{"osd":14,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":14.388999999999999,"5min":8.016,"15min":6.9539999999999997},"min":{"1min":1.0129999999999999,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":256.13400000000001,"5min":256.13400000000001,"15min":256.13400000000001},"last":1.45},{"interface":"front","average":{"1min":2.7650000000000001,"5min":7.7380000000000004,"15min":8.5670000000000002},"min":{"1min":0.874,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":25.791,"5min":102.625,"15min":102.625},"last":1.4330000000000001}]},{"osd":15,"last update":"Wed Sep 18 10:16:40 2024","interfaces":[{"interface":"back","average":{"1min":4.0309999999999997,"5min":4.0309999999999997,"15min":4.0309999999999997},"min":{"1min":1.153,"5min":1.153,"15min":1.153},"max":{"1min":49.927999999999997,"5min":49.927999999999997,"15min":49.927999999999997},"last":1.518},{"interface":"front","average":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"min":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"max":{"1min":2.044,"5min":2.044,"15min":2.044},"last":1.407}]}]},{"osd":12,"up_from":21,"seq":90194313249,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303784,"kb_used_data":2268,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068056,"statfs":{"total":96636764160,"available":96325689344,"internally_reserved":0,"allocated":2322432,"data_stored":1350598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[3,4,6,7,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":2.3839999999999999,"5min":2.3839999999999999,"15min":2.3839999999999999},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":32.476999999999997,"5min":32.476999999999997,"15min":32.476999999999997},"last":0.64200000000000002},{"interface":"front","average":{"1min":2.625,"5min":2.625,"15min":2.625},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":32.509,"5min":32.509,"15min":32.509},"last":0.89300000000000002}]},{"osd":1,"last update":"Wed Sep 18 10:15:32 2024","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.6399999999999999,"5min":1.6399999999999999,"15min":1.6399999999999999},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.905,"5min":1.905,"15min":1.905},"last":0.81899999999999995}]},{"osd":2,"last update":"Wed Sep 18 10:15:32 2024","interfaces":[{"interface":"back","average":{"1min":1.002,"5min":1.002,"15min":1.002},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":1.153},{"interface":"front","average":{"1min":4.1189999999999998,"5min":4.1189999999999998,"15min":4.1189999999999998},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":62.780999999999999,"5min":62.780999999999999,"15min":62.780999999999999},"last":0.96499999999999997}]},{"osd":3,"last update":"Wed Sep 18 10:16:33 2024","interfaces":[{"interface":"back","average":{"1min":5.9429999999999996,"5min":2.1040000000000001,"15min":1.464},"min":{"1min":0.627,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":74.822999999999993,"5min":74.822999999999993,"15min":74.822999999999993},"last":1.1399999999999999},{"interface":"front","average":{"1min":8.8200000000000003,"5min":2.6840000000000002,"15min":1.6619999999999999},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":94.320999999999998,"5min":94.320999999999998,"15min":94.320999999999998},"last":0.87}]},{"osd":4,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":16.058,"5min":4.0529999999999999,"15min":2.052},"min":{"1min":0.53100000000000003,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":114.184,"5min":114.184,"15min":114.184},"last":0.82199999999999995},{"interface":"front","average":{"1min":5.8920000000000003,"5min":2.1560000000000001,"15min":1.5329999999999999},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":74.698999999999998,"5min":74.698999999999998,"15min":74.698999999999998},"last":0.94299999999999995}]},{"osd":6,"last update":"Wed Sep 18 10:16:38 2024","interfaces":[{"interface":"back","average":{"1min":10.747999999999999,"5min":10.747999999999999,"15min":10.747999999999999},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":130.20699999999999,"5min":130.20699999999999,"15min":130.20699999999999},"last":1.0660000000000001},{"interface":"front","average":{"1min":14.148,"5min":14.148,"15min":14.148},"min":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":132.60900000000001,"5min":132.60900000000001,"15min":132.60900000000001},"last":1.5329999999999999}]},{"osd":7,"last update":"Wed Sep 18 10:16:38 2024","interfaces":[{"interface":"back","average":{"1min":4.774,"5min":4.774,"15min":4.774},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":90.188000000000002,"5min":90.188000000000002,"15min":90.188000000000002},"last":1.208},{"interface":"front","average":{"1min":6.0359999999999996,"5min":6.0359999999999996,"15min":6.0359999999999996},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":75.061999999999998,"5min":75.061999999999998,"15min":75.061999999999998},"last":1.3089999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":94.382000000000005}]},{"osd":9,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":8.9719999999999995,"5min":2.8079999999999998,"15min":1.7809999999999999},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":94.421000000000006,"5min":94.421000000000006,"15min":94.421000000000006},"last":1.3799999999999999},{"interface":"front","average":{"1min":5.0410000000000004,"5min":2.0179999999999998,"15min":1.514},"min":{"1min":0.70199999999999996,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":90.242999999999995,"5min":90.242999999999995,"15min":90.242999999999995},"last":1.47}]},{"osd":10,"last update":"Wed Sep 18 10:16:38 2024","interfaces":[{"interface":"back","average":{"1min":15.145,"5min":15.145,"15min":15.145},"min":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":99.251000000000005,"5min":99.251000000000005,"15min":99.251000000000005},"last":1.4359999999999999},{"interface":"front","average":{"1min":8.7080000000000002,"5min":8.7080000000000002,"15min":8.7080000000000002},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":94.468000000000004,"5min":94.468000000000004,"15min":94.468000000000004},"last":1.5700000000000001}]},{"osd":11,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":8.9269999999999996,"5min":2.8769999999999998,"15min":1.869},"min":{"1min":0.91300000000000003,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":74.888999999999996,"5min":74.888999999999996,"15min":74.888999999999996},"last":1.8899999999999999},{"interface":"front","average":{"1min":9.1590000000000007,"5min":2.9380000000000002,"15min":1.901},"min":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"max":{"1min":94.546000000000006,"5min":94.546000000000006,"15min":94.546000000000006},"last":1.798}]},{"osd":13,"last update":"Wed Sep 18 10:16:33 2024","interfaces":[{"interface":"back","average":{"1min":7.4779999999999998,"5min":2.5619999999999998,"15min":1.742},"min":{"1min":0.79900000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":143.672,"5min":143.672,"15min":143.672},"last":1.671},{"interface":"front","average":{"1min":12.677,"5min":3.6779999999999999,"15min":2.1779999999999999},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":143.619,"5min":143.619,"15min":143.619},"last":1.744}]},{"osd":14,"last update":"Wed Sep 18 10:16:33 2024","interfaces":[{"interface":"back","average":{"1min":13.363,"5min":3.8159999999999998,"15min":2.2250000000000001},"min":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"max":{"1min":94.504999999999995,"5min":94.504999999999995,"15min":94.504999999999995},"last":1.6879999999999999},{"interface":"front","average":{"1min":12.039,"5min":3.52,"15min":2.1000000000000001},"min":{"1min":0.95099999999999996,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":92.593000000000004,"5min":92.593000000000004,"15min":92.593000000000004},"last":1.615}]},{"osd":15,"last update":"Wed Sep 18 10:16:33 2024","interfaces":[{"interface":"back","average":{"1min":6.5339999999999998,"5min":2.5019999999999998,"15min":1.8300000000000001},"min":{"1min":0.751,"5min":0.751,"15min":0.751},"max":{"1min":74.966999999999999,"5min":74.966999999999999,"15min":74.966999999999999},"last":2.028},{"interface":"front","average":{"1min":14.669,"5min":4.117,"15min":2.359},"min":{"1min":0.996,"5min":0.95299999999999996,"15min":0.95299999999999996},"max":{"1min":152.11000000000001,"5min":152.11000000000001,"15min":152.11000000000001},"last":1.9219999999999999}]}]},{"osd":1,"up_from":22,"seq":94489280543,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":303748,"kb_used_data":2232,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068092,"statfs":{"total":96636764160,"available":96325726208,"internally_reserved":0,"allocated":2285568,"data_stored":1331589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,6,9,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:16:38 2024","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":2.472,"15min":2.7370000000000001},"min":{"1min":0.65800000000000003,"5min":0.629,"15min":0.629},"max":{"1min":1.1930000000000001,"5min":36.268000000000001,"15min":36.268000000000001},"last":1.127},{"interface":"front","average":{"1min":5.3659999999999997,"5min":15.244999999999999,"15min":16.890999999999998},"min":{"1min":0.73099999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":66.534999999999997,"5min":154.67599999999999,"15min":154.67599999999999},"last":0.70699999999999996}]},{"osd":2,"last update":"Wed Sep 18 10:16:38 2024","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.91400000000000003,"15min":0.90400000000000003},"min":{"1min":0.71499999999999997,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.4119999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.98999999999999999,"5min":2.5539999999999998,"15min":2.8140000000000001},"min":{"1min":0.75600000000000001,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.212,"5min":36.323999999999998,"15min":36.323999999999998},"last":0.76400000000000001}]},{"osd":3,"last update":"Wed Sep 18 10:16:38 2024","interfaces":[{"interface":"back","average":{"1min":5.548,"5min":10.811999999999999,"15min":11.689},"min":{"1min":0.94599999999999995,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":66.344999999999999,"5min":150.642,"15min":150.642},"last":0.96799999999999997},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.0649999999999999,"15min":1.0509999999999999},"min":{"1min":0.82999999999999996,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.6830000000000001,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":1.081}]},{"osd":4,"last update":"Wed Sep 18 10:15:35 2024","interfaces":[{"interface":"back","average":{"1min":3.3439999999999999,"5min":3.3439999999999999,"15min":3.3439999999999999},"min":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":40.783000000000001,"5min":40.783000000000001,"15min":40.783000000000001},"last":1.7210000000000001},{"interface":"front","average":{"1min":4.1699999999999999,"5min":4.1699999999999999,"15min":4.1699999999999999},"min":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":50.747,"5min":50.747,"15min":50.747},"last":1.518}]},{"osd":5,"last update":"Wed Sep 18 10:15:35 2024","interfaces":[{"interface":"back","average":{"1min":5.9720000000000004,"5min":5.9720000000000004,"15min":5.9720000000000004},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":52.317999999999998,"5min":52.317999999999998,"15min":52.317999999999998},"last":1.2649999999999999},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.748,"5min":1.748,"15min":1.748},"last":1.839}]},{"osd":6,"last update":"Wed Sep 18 10:16:38 2024","interfaces":[{"interface":"back","average":{"1min":9.2970000000000006,"5min":8.5380000000000003,"15min":8.4120000000000008},"min":{"1min":0.85099999999999998,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":120.85299999999999,"5min":120.85299999999999,"15min":120.85299999999999},"last":1.1779999999999999},{"interface":"front","average":{"1min":2.6040000000000001,"5min":8.8019999999999996,"15min":9.8350000000000009},"min":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"max":{"1min":20.600000000000001,"5min":85.045000000000002,"15min":85.045000000000002},"last":1.266}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":118.43600000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7789999999999999}]},{"osd":9,"last update":"Wed Sep 18 10:16:15 2024","interfaces":[{"interface":"back","average":{"1min":3.4369999999999998,"5min":3.4369999999999998,"15min":3.4369999999999998},"min":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"max":{"1min":36.506999999999998,"5min":36.506999999999998,"15min":36.506999999999998},"last":0.998},{"interface":"front","average":{"1min":11.339,"5min":11.339,"15min":11.339},"min":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"max":{"1min":165.13499999999999,"5min":165.13499999999999,"15min":165.13499999999999},"last":1.0429999999999999}]},{"osd":10,"last update":"Wed Sep 18 10:16:38 2024","interfaces":[{"interface":"back","average":{"1min":1.5209999999999999,"5min":3.048,"15min":3.302},"min":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"max":{"1min":2.0489999999999999,"5min":36.436,"15min":36.436},"last":1.3799999999999999},{"interface":"front","average":{"1min":6.0839999999999996,"5min":10.747999999999999,"15min":11.526},"min":{"1min":1.2769999999999999,"5min":0.88,"15min":0.88},"max":{"1min":66.673000000000002,"5min":100.402,"15min":100.402},"last":1.155}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.825}]},{"osd":12,"last update":"Wed Sep 18 10:16:38 2024","interfaces":[{"interface":"back","average":{"1min":1.5549999999999999,"5min":3.681,"15min":4.0350000000000001},"min":{"1min":1.101,"5min":1.0589999999999999,"15min":1.0589999999999999},"max":{"1min":2.294,"5min":49.976999999999997,"15min":49.976999999999997},"last":1.323},{"interface":"front","average":{"1min":2.0369999999999999,"5min":7.1269999999999998,"15min":7.9749999999999996},"min":{"1min":1.196,"5min":1.196,"15min":1.196},"max":{"1min":8.4049999999999994,"5min":124.78100000000001,"15min":124.78100000000001},"last":1.4419999999999999}]},{"osd":13,"last update":"Wed Sep 18 10:16:15 2024","interfaces":[{"interface":"back","average":{"1min":11.616,"5min":11.616,"15min":11.616},"min":{"1min":1.232,"5min":1.232,"15min":1.232},"max":{"1min":104.94199999999999,"5min":104.94199999999999,"15min":104.94199999999999},"last":1.2909999999999999},{"interface":"front","average":{"1min":1.5720000000000001,"5min":1.5720000000000001,"15min":1.5720000000000001},"min":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"max":{"1min":2.3809999999999998,"5min":2.3809999999999998,"15min":2.3809999999999998},"last":1.4099999999999999}]},{"osd":14,"last update":"Wed Sep 18 10:16:15 2024","interfaces":[{"interface":"back","average":{"1min":5.5910000000000002,"5min":5.5910000000000002,"15min":5.5910000000000002},"min":{"1min":1.276,"5min":1.276,"15min":1.276},"max":{"1min":66.825999999999993,"5min":66.825999999999993,"15min":66.825999999999993},"last":1.3580000000000001},{"interface":"front","average":{"1min":2.8359999999999999,"5min":2.8359999999999999,"15min":2.8359999999999999},"min":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"max":{"1min":22.16,"5min":22.16,"15min":22.16},"last":1.508}]},{"osd":15,"last update":"Wed Sep 18 10:16:15 2024","interfaces":[{"interface":"back","average":{"1min":3.677,"5min":3.677,"15min":3.677},"min":{"1min":1.165,"5min":1.165,"15min":1.165},"max":{"1min":36.575000000000003,"5min":36.575000000000003,"15min":36.575000000000003},"last":1.54},{"interface":"front","average":{"1min":5.6269999999999998,"5min":5.6269999999999998,"15min":5.6269999999999998},"min":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"max":{"1min":66.763999999999996,"5min":66.763999999999996,"15min":66.763999999999996},"last":1.468}]}]},{"osd":2,"up_from":22,"seq":94489280543,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":303764,"kb_used_data":2248,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068076,"statfs":{"total":96636764160,"available":96325709824,"internally_reserved":0,"allocated":2301952,"data_stored":1341063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,3,4,5,6,7,8,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":21.344000000000001,"15min":24.777999999999999},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.3879999999999999,"5min":419.05700000000002,"15min":419.05700000000002},"last":0.84899999999999998},{"interface":"front","average":{"1min":2.3860000000000001,"5min":4.4020000000000001,"15min":4.7380000000000004},"min":{"1min":0.57299999999999995,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":35.899999999999999,"5min":98.254000000000005,"15min":98.254000000000005},"last":0.71699999999999997}]},{"osd":3,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":5.1219999999999999,"5min":1.7529999999999999,"15min":1.1910000000000001},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":86.677000000000007,"5min":86.677000000000007,"15min":86.677000000000007},"last":0.66900000000000004},{"interface":"front","average":{"1min":3.6160000000000001,"5min":1.47,"15min":1.1120000000000001},"min":{"1min":0.68000000000000005,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":59.759,"5min":59.759,"15min":59.759},"last":0.76300000000000001}]},{"osd":4,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":7.992,"5min":3.1379999999999999,"15min":2.3290000000000002},"min":{"1min":0.55600000000000005,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":121.20099999999999,"5min":121.20099999999999,"15min":121.20099999999999},"last":0.94499999999999995},{"interface":"front","average":{"1min":6.5250000000000004,"5min":4.9359999999999999,"15min":4.6710000000000003},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":86.548000000000002,"5min":86.548000000000002,"15min":86.548000000000002},"last":0.88300000000000001}]},{"osd":5,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":12.161,"5min":3.2879999999999998,"15min":1.8089999999999999},"min":{"1min":0.72099999999999997,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":163.828,"5min":163.828,"15min":163.828},"last":0.92600000000000005},{"interface":"front","average":{"1min":2.6160000000000001,"5min":1.3660000000000001,"15min":1.1579999999999999},"min":{"1min":0.79200000000000004,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":36.119,"5min":36.119,"15min":36.119},"last":1.02}]},{"osd":6,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":5.2359999999999998,"5min":4.8529999999999998,"15min":4.7889999999999997},"min":{"1min":0.49199999999999999,"5min":0.441,"15min":0.441},"max":{"1min":86.766000000000005,"5min":86.766000000000005,"15min":86.766000000000005},"last":0.82399999999999995},{"interface":"front","average":{"1min":5.3959999999999999,"5min":4.9770000000000003,"15min":4.907},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":83.171999999999997,"5min":90.188999999999993,"15min":90.188999999999993},"last":1.038}]},{"osd":7,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":13.191000000000001,"5min":3.6019999999999999,"15min":2.004},"min":{"1min":0.84999999999999998,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":88.534000000000006,"5min":88.534000000000006,"15min":88.534000000000006},"last":1.079},{"interface":"front","average":{"1min":15.949999999999999,"5min":4.2350000000000003,"15min":2.2829999999999999},"min":{"1min":0.75600000000000001,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":88.484999999999999,"5min":88.484999999999999,"15min":88.484999999999999},"last":0.97899999999999998}]},{"osd":8,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":4.3179999999999996,"15min":4.8399999999999999},"min":{"1min":0.82899999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":2.0699999999999998,"5min":92.578999999999994,"15min":92.578999999999994},"last":1.1000000000000001},{"interface":"front","average":{"1min":2.8460000000000001,"5min":2.7799999999999998,"15min":2.7690000000000001},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":36.088000000000001,"5min":36.088000000000001,"15min":36.088000000000001},"last":1.1459999999999999}]},{"osd":12,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":2.9220000000000002,"5min":1.6719999999999999,"15min":1.464},"min":{"1min":0.872,"5min":0.872,"15min":0.872},"max":{"1min":36.014000000000003,"5min":36.014000000000003,"15min":36.014000000000003},"last":1.2290000000000001},{"interface":"front","average":{"1min":5.5140000000000002,"5min":2.7469999999999999,"15min":2.286},"min":{"1min":0.73199999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":86.832999999999998,"5min":86.832999999999998,"15min":86.832999999999998},"last":1.0589999999999999}]},{"osd":13,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":11.115,"5min":8.3970000000000002,"15min":7.944},"min":{"1min":0.97999999999999998,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":80.683000000000007,"5min":141.84200000000001,"15min":141.84200000000001},"last":1.3600000000000001},{"interface":"front","average":{"1min":2.911,"5min":4.9489999999999998,"15min":5.2889999999999997},"min":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":35.841000000000001,"5min":97.441999999999993,"15min":97.441999999999993},"last":1.2849999999999999}]},{"osd":15,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":12.472,"5min":3.6200000000000001,"15min":2.145},"min":{"1min":0.68400000000000005,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":86.980000000000004,"5min":86.980000000000004,"15min":86.980000000000004},"last":1.129},{"interface":"front","average":{"1min":3.3479999999999999,"5min":3.798,"15min":3.8730000000000002},"min":{"1min":0.96999999999999997,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":45.356999999999999,"5min":61.694000000000003,"15min":61.694000000000003},"last":1.304}]}]},{"osd":3,"up_from":22,"seq":94489280543,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303748,"kb_used_data":2232,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068092,"statfs":{"total":96636764160,"available":96325726208,"internally_reserved":0,"allocated":2285568,"data_stored":1331589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,4,6,7,9,10,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:16:37 2024","interfaces":[{"interface":"back","average":{"1min":4.173,"5min":2.048,"15min":1.694},"min":{"1min":0.64300000000000002,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":65.994,"5min":65.994,"15min":65.994},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.97899999999999998,"5min":8.8260000000000005,"15min":10.134},"min":{"1min":0.75,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.752,"5min":71.316999999999993,"15min":71.316999999999993},"last":0.76800000000000002}]},{"osd":2,"last update":"Wed Sep 18 10:16:37 2024","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":4.4379999999999997,"15min":5.0140000000000002},"min":{"1min":0.76900000000000002,"5min":0.75,"15min":0.75},"max":{"1min":1.333,"5min":56.523000000000003,"15min":56.523000000000003},"last":0.94999999999999996},{"interface":"front","average":{"1min":1.165,"5min":1.1200000000000001,"15min":1.1120000000000001},"min":{"1min":0.67400000000000004,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.639,"5min":3.327,"15min":3.327},"last":1.3300000000000001}]},{"osd":4,"last update":"Wed Sep 18 10:16:37 2024","interfaces":[{"interface":"back","average":{"1min":1.0409999999999999,"5min":5.7069999999999999,"15min":6.4850000000000003},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.6399999999999999,"5min":71.135000000000005,"15min":71.135000000000005},"last":0.91500000000000004},{"interface":"front","average":{"1min":1.635,"5min":3.8130000000000002,"15min":4.1760000000000002},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":10.039,"5min":56.253999999999998,"15min":56.253999999999998},"last":0.72799999999999998}]},{"osd":5,"last update":"Wed Sep 18 10:16:06 2024","interfaces":[{"interface":"back","average":{"1min":13.997,"5min":13.997,"15min":13.997},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":125.42400000000001,"5min":125.42400000000001,"15min":125.42400000000001},"last":1.131},{"interface":"front","average":{"1min":10.323,"5min":10.323,"15min":10.323},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":112.033,"5min":112.033,"15min":112.033},"last":0.96899999999999997}]},{"osd":6,"last update":"Wed Sep 18 10:16:37 2024","interfaces":[{"interface":"back","average":{"1min":1.9279999999999999,"5min":1.29,"15min":1.1839999999999999},"min":{"1min":0.93400000000000005,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":12.83,"5min":12.83,"15min":12.83},"last":1.0600000000000001},{"interface":"front","average":{"1min":5.0190000000000001,"5min":8.0050000000000008,"15min":8.5030000000000001},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":76.634,"5min":81.210999999999999,"15min":81.210999999999999},"last":0.998}]},{"osd":7,"last update":"Wed Sep 18 10:16:37 2024","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":13.159000000000001,"15min":15.141},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.75,"5min":129.68299999999999,"15min":129.68299999999999},"last":1.0249999999999999},{"interface":"front","average":{"1min":1.4710000000000001,"5min":11.119999999999999,"15min":12.728},"min":{"1min":0.90800000000000003,"5min":0.621,"15min":0.621},"max":{"1min":1.8500000000000001,"5min":129.726,"15min":129.726},"last":1.474}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0620000000000001}]},{"osd":9,"last update":"Wed Sep 18 10:16:37 2024","interfaces":[{"interface":"back","average":{"1min":1.5660000000000001,"5min":1.446,"15min":1.4259999999999999},"min":{"1min":0.94599999999999995,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":2.0209999999999999,"5min":2.0209999999999999,"15min":2.0209999999999999},"last":1.377},{"interface":"front","average":{"1min":6.1050000000000004,"5min":6.1079999999999997,"15min":6.1079999999999997},"min":{"1min":0.86099999999999999,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":94.546999999999997,"5min":94.546999999999997,"15min":94.546999999999997},"last":1.2969999999999999}]},{"osd":10,"last update":"Wed Sep 18 10:16:06 2024","interfaces":[{"interface":"back","average":{"1min":1.4870000000000001,"5min":1.4870000000000001,"15min":1.4870000000000001},"min":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":2.0659999999999998,"5min":2.0659999999999998,"15min":2.0659999999999998},"last":1.232},{"interface":"front","average":{"1min":4.9349999999999996,"5min":4.9349999999999996,"15min":4.9349999999999996},"min":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":71.668999999999997,"5min":71.668999999999997,"15min":71.668999999999997},"last":1.173}]},{"osd":13,"last update":"Wed Sep 18 10:16:37 2024","interfaces":[{"interface":"back","average":{"1min":9.4909999999999997,"5min":3.113,"15min":2.0499999999999998},"min":{"1min":1.04,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":91.290999999999997,"5min":91.290999999999997,"15min":91.290999999999997},"last":1.421},{"interface":"front","average":{"1min":1.5249999999999999,"5min":5.3300000000000001,"15min":5.9640000000000004},"min":{"1min":1.05,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.885,"5min":71.447000000000003,"15min":71.447000000000003},"last":1.3560000000000001}]},{"osd":14,"last update":"Wed Sep 18 10:16:37 2024","interfaces":[{"interface":"back","average":{"1min":1.6899999999999999,"5min":4.8899999999999997,"15min":5.423},"min":{"1min":1.0089999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"max":{"1min":4.625,"5min":56.664999999999999,"15min":56.664999999999999},"last":1.149},{"interface":"front","average":{"1min":1.865,"5min":2.2629999999999999,"15min":2.3290000000000002},"min":{"1min":1.1859999999999999,"5min":0.93999999999999995,"15min":0.93999999999999995},"max":{"1min":4.5910000000000002,"5min":18.523,"15min":18.523},"last":1.52}]},{"osd":15,"last update":"Wed Sep 18 10:16:48 2024","interfaces":[{"interface":"back","average":{"1min":1.6970000000000001,"5min":4.0049999999999999,"15min":4.3899999999999997},"min":{"1min":1.01,"5min":0.91500000000000004,"15min":0.91500000000000004},"max":{"1min":4.3289999999999997,"5min":56.357999999999997,"15min":56.357999999999997},"last":1.268},{"interface":"front","average":{"1min":1.7589999999999999,"5min":8.5310000000000006,"15min":9.6600000000000001},"min":{"1min":1.133,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":2.1110000000000002,"5min":122.727,"15min":122.727},"last":1.5620000000000001}]}]},{"osd":5,"up_from":22,"seq":94489280543,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303748,"kb_used_data":2232,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068092,"statfs":{"total":96636764160,"available":96325726208,"internally_reserved":0,"allocated":2285568,"data_stored":1331589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[2,3,4,6,8,9,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:15:47 2024","interfaces":[{"interface":"back","average":{"1min":7.8860000000000001,"5min":7.8860000000000001,"15min":7.8860000000000001},"min":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":84.430999999999997,"5min":84.430999999999997,"15min":84.430999999999997},"last":1.2889999999999999},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":1.353}]},{"osd":2,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":4.1360000000000001,"5min":8.6940000000000008,"15min":9.4540000000000006},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":51.091000000000001,"5min":84.566000000000003,"15min":84.566000000000003},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.98199999999999998,"5min":1.3300000000000001,"15min":1.3879999999999999},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.4690000000000001,"5min":9.5069999999999997,"15min":9.5069999999999997},"last":0.86899999999999999}]},{"osd":3,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":4.9240000000000004,"5min":7.8010000000000002,"15min":8.2810000000000006},"min":{"1min":0.69099999999999995,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":51.229999999999997,"5min":84.358999999999995,"15min":84.358999999999995},"last":0.70499999999999996},{"interface":"front","average":{"1min":8.1590000000000007,"5min":8.2100000000000009,"15min":8.218},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":74.222999999999999,"5min":97.549999999999997,"15min":97.549999999999997},"last":0.76900000000000002}]},{"osd":4,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":25.673999999999999,"15min":29.777999999999999},"min":{"1min":0.67400000000000004,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.407,"5min":266.911,"15min":266.911},"last":1.4059999999999999},{"interface":"front","average":{"1min":21.388000000000002,"5min":22.515999999999998,"15min":22.704000000000001},"min":{"1min":0.76200000000000001,"5min":0.626,"15min":0.626},"max":{"1min":171.691,"5min":260.53500000000003,"15min":260.53500000000003},"last":1.0960000000000001}]},{"osd":6,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":3.6309999999999998,"5min":8.0809999999999995,"15min":8.8230000000000004},"min":{"1min":0.70099999999999996,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":41.886000000000003,"5min":58.171999999999997,"15min":58.171999999999997},"last":0.97599999999999998},{"interface":"front","average":{"1min":5.8079999999999998,"5min":18.274999999999999,"15min":20.353000000000002},"min":{"1min":0.85699999999999998,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":54.984999999999999,"5min":227.48099999999999,"15min":227.48099999999999},"last":1.1759999999999999}]},{"osd":7,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":12.013,"5min":12.013,"15min":12.013},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":123.111,"5min":123.111,"15min":123.111},"last":1.5249999999999999},{"interface":"front","average":{"1min":13.994,"5min":13.994,"15min":13.994},"min":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"max":{"1min":84.617000000000004,"5min":84.617000000000004,"15min":84.617000000000004},"last":1.329}]},{"osd":8,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":4.4720000000000004,"5min":17.175999999999998,"15min":19.292999999999999},"min":{"1min":0.879,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":51.506,"5min":260.44200000000001,"15min":260.44200000000001},"last":1.232},{"interface":"front","average":{"1min":8.2609999999999992,"5min":7.2859999999999996,"15min":7.1239999999999997},"min":{"1min":0.83899999999999997,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":71.266000000000005,"5min":106.818,"15min":106.818},"last":1.038}]},{"osd":9,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":3.8780000000000001,"5min":5.2930000000000001,"15min":5.5289999999999999},"min":{"1min":0.91300000000000003,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":41.652000000000001,"5min":80.272000000000006,"15min":80.272000000000006},"last":1.133},{"interface":"front","average":{"1min":1.3799999999999999,"5min":1.764,"15min":1.8280000000000001},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":2.008,"5min":9.5690000000000008,"15min":9.5690000000000008},"last":1.502}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6479999999999999}]},{"osd":11,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":1.494,"5min":7.8120000000000003,"15min":8.8650000000000002},"min":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":1.825,"5min":123.158,"15min":123.158},"last":1.794},{"interface":"front","average":{"1min":8.8490000000000002,"5min":10.863,"15min":11.199},"min":{"1min":0.93200000000000005,"5min":0.86499999999999999,"15min":0.86499999999999999},"max":{"1min":68.938000000000002,"5min":84.522999999999996,"15min":84.522999999999996},"last":1.446}]},{"osd":12,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":4.0250000000000004,"5min":1.8580000000000001,"15min":1.4970000000000001},"min":{"1min":0.95199999999999996,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":41.929000000000002,"5min":41.929000000000002,"15min":41.929000000000002},"last":1.2849999999999999},{"interface":"front","average":{"1min":1.774,"5min":1.9339999999999999,"15min":1.9610000000000001},"min":{"1min":0.93999999999999995,"5min":0.88300000000000001,"15min":0.88300000000000001},"max":{"1min":4.1150000000000002,"5min":9.7710000000000008,"15min":9.7710000000000008},"last":1.5780000000000001}]},{"osd":13,"last update":"Wed Sep 18 10:16:48 2024","interfaces":[{"interface":"back","average":{"1min":1.6499999999999999,"5min":19.745999999999999,"15min":22.762},"min":{"1min":1.171,"5min":1.1120000000000001,"15min":1.1120000000000001},"max":{"1min":2.0550000000000002,"5min":140.273,"15min":140.273},"last":1.6479999999999999},{"interface":"front","average":{"1min":4.665,"5min":20.309000000000001,"15min":22.917000000000002},"min":{"1min":1.194,"5min":0.98099999999999998,"15min":0.98099999999999998},"max":{"1min":51.432000000000002,"5min":170.86000000000001,"15min":170.86000000000001},"last":1.329}]},{"osd":14,"last update":"Wed Sep 18 10:16:48 2024","interfaces":[{"interface":"back","average":{"1min":9.8930000000000007,"5min":12.584,"15min":13.032},"min":{"1min":1.385,"5min":1.0389999999999999,"15min":1.0389999999999999},"max":{"1min":93.295000000000002,"5min":101.733,"15min":101.733},"last":1.385},{"interface":"front","average":{"1min":1.79,"5min":7.9729999999999999,"15min":9.0030000000000001},"min":{"1min":1.321,"5min":0.90100000000000002,"15min":0.90100000000000002},"max":{"1min":2.2290000000000001,"5min":122.952,"15min":122.952},"last":1.7250000000000001}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8839999999999999}]}]},{"osd":6,"up_from":22,"seq":94489280543,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304280,"kb_used_data":2752,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94067560,"statfs":{"total":96636764160,"available":96325181440,"internally_reserved":0,"allocated":2818048,"data_stored":1860517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,3,4,5,7,8,9,11,12,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:15:47 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.68200000000000005}]},{"osd":1,"last update":"Wed Sep 18 10:16:48 2024","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":3.8100000000000001,"15min":4.306},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.2150000000000001,"5min":52.332999999999998,"15min":52.332999999999998},"last":0.65700000000000003},{"interface":"front","average":{"1min":4.8369999999999997,"5min":1.6990000000000001,"15min":1.1759999999999999},"min":{"1min":0.68300000000000005,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":67.763000000000005,"5min":67.763000000000005,"15min":67.763000000000005},"last":0.71199999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.343}]},{"osd":3,"last update":"Wed Sep 18 10:16:48 2024","interfaces":[{"interface":"back","average":{"1min":5.3380000000000001,"5min":1.792,"15min":1.2010000000000001},"min":{"1min":0.65500000000000003,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":76.891999999999996,"5min":76.891999999999996,"15min":76.891999999999996},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.98999999999999999,"5min":11.403,"15min":13.138999999999999},"min":{"1min":0.751,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.204,"5min":169.244,"15min":169.244},"last":1.042}]},{"osd":4,"last update":"Wed Sep 18 10:16:02 2024","interfaces":[{"interface":"back","average":{"1min":11.111000000000001,"5min":11.111000000000001,"15min":11.111000000000001},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":128.053,"5min":128.053,"15min":128.053},"last":0.92100000000000004},{"interface":"front","average":{"1min":4.6189999999999998,"5min":4.6189999999999998,"15min":4.6189999999999998},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":67.960999999999999,"5min":67.960999999999999,"15min":67.960999999999999},"last":0.86699999999999999}]},{"osd":5,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":6.0579999999999998,"5min":10.194000000000001,"15min":10.884},"min":{"1min":0.72099999999999997,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":68.025999999999996,"5min":119.032,"15min":119.032},"last":0.96799999999999997},{"interface":"front","average":{"1min":5.6760000000000002,"5min":4.0380000000000003,"15min":3.7650000000000001},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":83.858000000000004,"5min":83.858000000000004,"15min":83.858000000000004},"last":1.109}]},{"osd":7,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":4.9539999999999997,"5min":6.3959999999999999,"15min":6.6360000000000001},"min":{"1min":0.94099999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":69.171000000000006,"5min":69.171000000000006,"15min":69.171000000000006},"last":1.3120000000000001},{"interface":"front","average":{"1min":5.0960000000000001,"5min":3.4630000000000001,"15min":3.1909999999999998},"min":{"1min":1.089,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":69.239000000000004,"5min":69.239000000000004,"15min":69.239000000000004},"last":1.29}]},{"osd":8,"last update":"Wed Sep 18 10:16:51 2024","interfaces":[{"interface":"back","average":{"1min":5.3659999999999997,"5min":5.3659999999999997,"15min":5.3659999999999997},"min":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"max":{"1min":76.766000000000005,"5min":76.766000000000005,"15min":76.766000000000005},"last":1.0109999999999999},{"interface":"front","average":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"min":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"max":{"1min":1.704,"5min":1.704,"15min":1.704},"last":1.5029999999999999}]},{"osd":9,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":5.4269999999999996,"5min":7.7089999999999996,"15min":8.0890000000000004},"min":{"1min":0.95799999999999996,"5min":0.84599999999999997,"15min":0.84599999999999997},"max":{"1min":76.997,"5min":95.691999999999993,"15min":95.691999999999993},"last":1.246},{"interface":"front","average":{"1min":1.2949999999999999,"5min":10.472,"15min":12.002000000000001},"min":{"1min":0.96899999999999997,"5min":0.83099999999999996,"15min":0.83099999999999996},"max":{"1min":1.696,"5min":165.31100000000001,"15min":165.31100000000001},"last":1.4219999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.329}]},{"osd":11,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":5.0510000000000002,"5min":2.0630000000000002,"15min":1.5649999999999999},"min":{"1min":1.002,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":67.909999999999997,"5min":67.909999999999997,"15min":67.909999999999997},"last":1.1599999999999999},{"interface":"front","average":{"1min":9.1829999999999998,"5min":4.9429999999999996,"15min":4.2359999999999998},"min":{"1min":1.1870000000000001,"5min":0.92000000000000004,"15min":0.92000000000000004},"max":{"1min":77.158000000000001,"5min":77.158000000000001,"15min":77.158000000000001},"last":1.351}]},{"osd":12,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":1.3859999999999999,"5min":4.3140000000000001,"15min":4.8019999999999996},"min":{"1min":1.079,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":1.796,"5min":52.378,"15min":52.378},"last":1.204},{"interface":"front","average":{"1min":5.1109999999999998,"5min":4.5720000000000001,"15min":4.4820000000000002},"min":{"1min":0.92800000000000005,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":67.835999999999999,"5min":67.835999999999999,"15min":67.835999999999999},"last":1.3779999999999999}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2050000000000001}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2669999999999999}]},{"osd":15,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":5.1680000000000001,"5min":2.2120000000000002,"15min":1.72},"min":{"1min":1.1000000000000001,"5min":0.90300000000000002,"15min":0.90300000000000002},"max":{"1min":68.069999999999993,"5min":68.069999999999993,"15min":68.069999999999993},"last":1.468},{"interface":"front","average":{"1min":5.6200000000000001,"5min":4.3019999999999996,"15min":4.0819999999999999},"min":{"1min":1.2030000000000001,"5min":1.0429999999999999,"15min":1.0429999999999999},"max":{"1min":77.081999999999994,"5min":77.081999999999994,"15min":77.081999999999994},"last":1.452}]}]},{"osd":7,"up_from":22,"seq":94489280543,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303748,"kb_used_data":2232,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068092,"statfs":{"total":96636764160,"available":96325726208,"internally_reserved":0,"allocated":2285568,"data_stored":1331589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,4,6,8,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:16:33 2024","interfaces":[{"interface":"back","average":{"1min":3.9900000000000002,"5min":10.974,"15min":12.138},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":59.137999999999998,"5min":139.58600000000001,"15min":139.58600000000001},"last":0.75600000000000001},{"interface":"front","average":{"1min":1.121,"5min":1.0229999999999999,"15min":1.0069999999999999},"min":{"1min":0.67800000000000005,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.7390000000000001,"5min":1.7390000000000001,"15min":1.7390000000000001},"last":0.59899999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2090000000000001}]},{"osd":2,"last update":"Wed Sep 18 10:15:46 2024","interfaces":[{"interface":"back","average":{"1min":15.436,"5min":15.436,"15min":15.436},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":139.36799999999999,"5min":139.36799999999999,"15min":139.36799999999999},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.94299999999999995}]},{"osd":3,"last update":"Wed Sep 18 10:15:46 2024","interfaces":[{"interface":"back","average":{"1min":17.327999999999999,"5min":17.327999999999999,"15min":17.327999999999999},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":139.27199999999999,"5min":139.27199999999999,"15min":139.27199999999999},"last":1.3460000000000001},{"interface":"front","average":{"1min":3.964,"5min":3.964,"15min":3.964},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":23.748999999999999,"5min":23.748999999999999,"15min":23.748999999999999},"last":0.82899999999999996}]},{"osd":4,"last update":"Wed Sep 18 10:16:37 2024","interfaces":[{"interface":"back","average":{"1min":9.8670000000000009,"5min":17.291,"15min":18.527999999999999},"min":{"1min":0.68000000000000005,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":125.10299999999999,"5min":139.51599999999999,"15min":139.51599999999999},"last":0.55200000000000005},{"interface":"front","average":{"1min":3.3559999999999999,"5min":4.6699999999999999,"15min":4.8890000000000002},"min":{"1min":0.68000000000000005,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":49.241999999999997,"5min":49.241999999999997,"15min":49.241999999999997},"last":0.70199999999999996}]},{"osd":5,"last update":"Wed Sep 18 10:15:46 2024","interfaces":[{"interface":"back","average":{"1min":1.099,"5min":1.099,"15min":1.099},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":2.3620000000000001,"5min":2.3620000000000001,"15min":2.3620000000000001},"last":1.4059999999999999},{"interface":"front","average":{"1min":17.736999999999998,"5min":17.736999999999998,"15min":17.736999999999998},"min":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"max":{"1min":139.44300000000001,"5min":139.44300000000001,"15min":139.44300000000001},"last":0.78000000000000003}]},{"osd":6,"last update":"Wed Sep 18 10:16:33 2024","interfaces":[{"interface":"back","average":{"1min":6.0869999999999997,"5min":15.117000000000001,"15min":16.622},"min":{"1min":0.998,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":59.343000000000004,"5min":183.001,"15min":183.001},"last":1.024},{"interface":"front","average":{"1min":7.0830000000000002,"5min":4.8650000000000002,"15min":4.4950000000000001},"min":{"1min":0.82399999999999995,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":58.284999999999997,"5min":58.284999999999997,"15min":58.284999999999997},"last":0.98499999999999999}]},{"osd":8,"last update":"Wed Sep 18 10:16:30 2024","interfaces":[{"interface":"back","average":{"1min":1.4079999999999999,"5min":1.2809999999999999,"15min":1.26},"min":{"1min":0.84999999999999998,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.9339999999999999,"5min":2.6139999999999999,"15min":2.6139999999999999},"last":0.85699999999999998},{"interface":"front","average":{"1min":5.3029999999999999,"5min":13.785,"15min":15.199},"min":{"1min":1.0980000000000001,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":59.523000000000003,"5min":139.798,"15min":139.798},"last":0.89700000000000002}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.861}]},{"osd":10,"last update":"Wed Sep 18 10:16:30 2024","interfaces":[{"interface":"back","average":{"1min":16.035,"5min":8.4100000000000001,"15min":7.1390000000000002},"min":{"1min":1.0149999999999999,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":156.45500000000001,"5min":156.45500000000001,"15min":156.45500000000001},"last":1.2070000000000001},{"interface":"front","average":{"1min":1.4910000000000001,"5min":1.391,"15min":1.3740000000000001},"min":{"1min":0.83499999999999996,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":2.1200000000000001,"5min":2.254,"15min":2.254},"last":1.0600000000000001}]},{"osd":11,"last update":"Wed Sep 18 10:16:33 2024","interfaces":[{"interface":"back","average":{"1min":5.0460000000000003,"5min":5.4459999999999997,"15min":5.5129999999999999},"min":{"1min":1.046,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":49.154000000000003,"5min":62.356000000000002,"15min":62.356000000000002},"last":1.48},{"interface":"front","average":{"1min":2.4769999999999999,"5min":1.6240000000000001,"15min":1.482},"min":{"1min":1.0189999999999999,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":20.449999999999999,"5min":20.449999999999999,"15min":20.449999999999999},"last":0.94599999999999995}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4119999999999999}]},{"osd":13,"last update":"Wed Sep 18 10:16:33 2024","interfaces":[{"interface":"back","average":{"1min":10.028,"5min":16.707999999999998,"15min":17.821999999999999},"min":{"1min":1.1459999999999999,"5min":0.874,"15min":0.874},"max":{"1min":113.428,"5min":139.655,"15min":139.655},"last":1.349},{"interface":"front","average":{"1min":5.1970000000000001,"5min":16.942,"15min":18.899999999999999},"min":{"1min":1.077,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":49.457999999999998,"5min":173.113,"15min":173.113},"last":1.75}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1799999999999999}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2829999999999999}]}]},{"osd":8,"up_from":22,"seq":94489280543,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303768,"kb_used_data":2248,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068072,"statfs":{"total":96636764160,"available":96325705728,"internally_reserved":0,"allocated":2301952,"data_stored":1341063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[2,4,6,7,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93300000000000005}]},{"osd":1,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":1.117,"5min":1.117,"15min":1.117},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.669,"5min":1.669,"15min":1.669},"last":1.2230000000000001},{"interface":"front","average":{"1min":2.3290000000000002,"5min":2.3290000000000002,"15min":2.3290000000000002},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":29.523,"5min":29.523,"15min":29.523},"last":1.1279999999999999}]},{"osd":2,"last update":"Wed Sep 18 10:16:48 2024","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":5.5140000000000002,"15min":6.2300000000000004},"min":{"1min":0.93500000000000005,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.8,"5min":77.435000000000002,"15min":77.435000000000002},"last":0.93500000000000005},{"interface":"front","average":{"1min":1.1639999999999999,"5min":0.97099999999999997,"15min":0.93899999999999995},"min":{"1min":0.71999999999999997,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.903,"5min":1.903,"15min":1.903},"last":0.71999999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.306}]},{"osd":4,"last update":"Wed Sep 18 10:16:48 2024","interfaces":[{"interface":"back","average":{"1min":1.23,"5min":9.5879999999999992,"15min":10.981},"min":{"1min":0.71899999999999997,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":7.1559999999999997,"5min":90.438000000000002,"15min":90.438000000000002},"last":0.83899999999999997},{"interface":"front","average":{"1min":1.1699999999999999,"5min":10.691000000000001,"15min":12.278},"min":{"1min":0.66100000000000003,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":2.1309999999999998,"5min":93.939999999999998,"15min":93.939999999999998},"last":0.66100000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4199999999999999}]},{"osd":6,"last update":"Wed Sep 18 10:16:48 2024","interfaces":[{"interface":"back","average":{"1min":1.359,"5min":15.042999999999999,"15min":17.324000000000002},"min":{"1min":0.90700000000000003,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":2.0350000000000001,"5min":293.26900000000001,"15min":293.26900000000001},"last":0.90700000000000003},{"interface":"front","average":{"1min":1.2230000000000001,"5min":5.1689999999999996,"15min":5.827},"min":{"1min":0.88900000000000001,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.839,"5min":70.203000000000003,"15min":70.203000000000003},"last":1.1060000000000001}]},{"osd":7,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":7.1950000000000003,"5min":3.6070000000000002,"15min":3.0089999999999999},"min":{"1min":0.80900000000000005,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":80.635000000000005,"5min":80.635000000000005,"15min":80.635000000000005},"last":1.0580000000000001},{"interface":"front","average":{"1min":9.3490000000000002,"5min":6.726,"15min":6.2889999999999997},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":99.060000000000002,"5min":99.060000000000002,"15min":99.060000000000002},"last":1.01}]},{"osd":9,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":4.8730000000000002,"5min":3.0110000000000001,"15min":2.7010000000000001},"min":{"1min":0.92700000000000005,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":80.076999999999998,"5min":80.076999999999998,"15min":80.076999999999998},"last":1.28},{"interface":"front","average":{"1min":1.6240000000000001,"5min":7.4489999999999998,"15min":8.4199999999999999},"min":{"1min":1.196,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":2.5190000000000001,"5min":98.914000000000001,"15min":98.914000000000001},"last":1.153}]},{"osd":10,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":1.6319999999999999,"5min":5.8559999999999999,"15min":6.5599999999999996},"min":{"1min":0.99099999999999999,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":2.3679999999999999,"5min":90.751000000000005,"15min":90.751000000000005},"last":1.081},{"interface":"front","average":{"1min":1.591,"5min":7.7889999999999997,"15min":8.8219999999999992},"min":{"1min":1.0640000000000001,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":2.3340000000000001,"5min":83.795000000000002,"15min":83.795000000000002},"last":1.3839999999999999}]},{"osd":11,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":1.7450000000000001,"5min":2.669,"15min":2.823},"min":{"1min":1.3340000000000001,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":2.383,"5min":26.462,"15min":26.462},"last":1.232},{"interface":"front","average":{"1min":1.752,"5min":1.948,"15min":1.9810000000000001},"min":{"1min":1.24,"5min":0.99299999999999999,"15min":0.99299999999999999},"max":{"1min":2.3479999999999999,"5min":12.475,"15min":12.475},"last":1.202}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0760000000000001}]},{"osd":13,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":3.052,"5min":7.9329999999999998,"15min":8.7469999999999999},"min":{"1min":1.05,"5min":1.0429999999999999,"15min":1.0429999999999999},"max":{"1min":30.582999999999998,"5min":94.222999999999999,"15min":94.222999999999999},"last":1.264},{"interface":"front","average":{"1min":1.635,"5min":2.5059999999999998,"15min":2.6509999999999998},"min":{"1min":1.1180000000000001,"5min":0.93400000000000005,"15min":0.93400000000000005},"max":{"1min":2.4689999999999999,"5min":23.983000000000001,"15min":23.983000000000001},"last":1.464}]},{"osd":14,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":1.802,"5min":16.975000000000001,"15min":19.504000000000001},"min":{"1min":1.28,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":2.4500000000000002,"5min":150.93600000000001,"15min":150.93600000000001},"last":1.3240000000000001},{"interface":"front","average":{"1min":4.8170000000000002,"5min":3.8410000000000002,"15min":3.6789999999999998},"min":{"1min":1.2210000000000001,"5min":0.90200000000000002,"15min":0.90200000000000002},"max":{"1min":69.918000000000006,"5min":69.918000000000006,"15min":69.918000000000006},"last":1.3460000000000001}]},{"osd":15,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":5.9089999999999998,"5min":3.4169999999999998,"15min":3.0019999999999998},"min":{"1min":1.2549999999999999,"5min":0.93300000000000005,"15min":0.93300000000000005},"max":{"1min":98.558000000000007,"5min":98.558000000000007,"15min":98.558000000000007},"last":1.5409999999999999},{"interface":"front","average":{"1min":1.8400000000000001,"5min":2.8119999999999998,"15min":2.9740000000000002},"min":{"1min":1.3160000000000001,"5min":0.95299999999999996,"15min":0.95299999999999996},"max":{"1min":2.5859999999999999,"5min":26.414000000000001,"15min":26.414000000000001},"last":1.373}]}]},{"osd":10,"up_from":22,"seq":94489280544,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303748,"kb_used_data":2232,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068092,"statfs":{"total":96636764160,"available":96325726208,"internally_reserved":0,"allocated":2285568,"data_stored":1331589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,4,5,6,7,9,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:16:29 2024","interfaces":[{"interface":"back","average":{"1min":16.117000000000001,"5min":16.117000000000001,"15min":16.117000000000001},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":145.12799999999999,"5min":145.12799999999999,"15min":145.12799999999999},"last":1.401},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.431,"5min":1.431,"15min":1.431},"last":1.0780000000000001}]},{"osd":1,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":9.5719999999999992,"5min":10.66,"15min":10.842000000000001},"min":{"1min":0.88500000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":145.20699999999999,"5min":145.20699999999999,"15min":145.20699999999999},"last":0.876},{"interface":"front","average":{"1min":4.7510000000000003,"5min":14.728,"15min":16.390999999999998},"min":{"1min":0.754,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":38.338000000000001,"5min":99.995999999999995,"15min":99.995999999999995},"last":0.71899999999999997}]},{"osd":2,"last update":"Wed Sep 18 10:15:33 2024","interfaces":[{"interface":"back","average":{"1min":10.893000000000001,"5min":10.893000000000001,"15min":10.893000000000001},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":84.072000000000003,"5min":84.072000000000003,"15min":84.072000000000003},"last":0.99199999999999999},{"interface":"front","average":{"1min":11.550000000000001,"5min":11.550000000000001,"15min":11.550000000000001},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":99.798000000000002,"5min":99.798000000000002,"15min":99.798000000000002},"last":1.425}]},{"osd":3,"last update":"Wed Sep 18 10:16:22 2024","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":1.119,"15min":1.119},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.99,"5min":1.99,"15min":1.99},"last":1.5},{"interface":"front","average":{"1min":18.094999999999999,"5min":18.094999999999999,"15min":18.094999999999999},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":145.535,"5min":145.535,"15min":145.535},"last":1.4510000000000001}]},{"osd":4,"last update":"Wed Sep 18 10:16:49 2024","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":20.715,"15min":23.963999999999999},"min":{"1min":0.92300000000000004,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.5820000000000001,"5min":125.23099999999999,"15min":125.23099999999999},"last":1.048},{"interface":"front","average":{"1min":1.206,"5min":25.175000000000001,"15min":29.170000000000002},"min":{"1min":0.88800000000000001,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.4930000000000001,"5min":295.93200000000002,"15min":295.93200000000002},"last":1.2210000000000001}]},{"osd":5,"last update":"Wed Sep 18 10:16:49 2024","interfaces":[{"interface":"back","average":{"1min":8.9960000000000004,"5min":21.460999999999999,"15min":23.539000000000001},"min":{"1min":0.81000000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":145.24000000000001,"5min":215.51400000000001,"15min":215.51400000000001},"last":1.111},{"interface":"front","average":{"1min":2.4140000000000001,"5min":22.346,"15min":25.667999999999999},"min":{"1min":0.85399999999999998,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":20.302,"5min":252.69300000000001,"15min":252.69300000000001},"last":1.421}]},{"osd":6,"last update":"Wed Sep 18 10:16:22 2024","interfaces":[{"interface":"back","average":{"1min":12.393000000000001,"5min":12.393000000000001,"15min":12.393000000000001},"min":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"max":{"1min":111.173,"5min":111.173,"15min":111.173},"last":1.1859999999999999},{"interface":"front","average":{"1min":15.771000000000001,"5min":15.771000000000001,"15min":15.771000000000001},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":221.13900000000001,"5min":221.13900000000001,"15min":221.13900000000001},"last":0.66500000000000004}]},{"osd":7,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":1.5109999999999999,"5min":2.7490000000000001,"15min":2.9550000000000001},"min":{"1min":0.95699999999999996,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.919,"5min":34.558,"15min":34.558},"last":1.083},{"interface":"front","average":{"1min":10.101000000000001,"5min":13.186999999999999,"15min":13.701000000000001},"min":{"1min":1.0920000000000001,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":145.5,"5min":145.5,"15min":145.5},"last":1.3140000000000001}]},{"osd":9,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":8.3640000000000008,"5min":11.07,"15min":11.521000000000001},"min":{"1min":1.0349999999999999,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":119.202,"5min":119.202,"15min":119.202},"last":1.361},{"interface":"front","average":{"1min":1.629,"5min":10.364000000000001,"15min":11.82},"min":{"1min":1.073,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":2.23,"5min":97.385999999999996,"15min":97.385999999999996},"last":0.92500000000000004}]},{"osd":11,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":10.132999999999999,"5min":11.489000000000001,"15min":11.715},"min":{"1min":1.286,"5min":0.85699999999999998,"15min":0.85699999999999998},"max":{"1min":145.26900000000001,"5min":145.26900000000001,"15min":145.26900000000001},"last":1.012},{"interface":"front","average":{"1min":5.6719999999999997,"5min":19.837,"15min":22.198},"min":{"1min":1.28,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":44.121000000000002,"5min":295.70400000000001,"15min":295.70400000000001},"last":1.2729999999999999}]},{"osd":12,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":1.6559999999999999,"5min":2.8250000000000002,"15min":3.02},"min":{"1min":1.0620000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":2.2559999999999998,"5min":34.625999999999998,"15min":34.625999999999998},"last":1.4590000000000001},{"interface":"front","average":{"1min":10.140000000000001,"5min":10.986000000000001,"15min":11.127000000000001},"min":{"1min":1.198,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":145.358,"5min":145.358,"15min":145.358},"last":1.639}]},{"osd":13,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":10.433999999999999,"5min":17.994,"15min":19.254000000000001},"min":{"1min":1.1990000000000001,"5min":0.877,"15min":0.877},"max":{"1min":123.696,"5min":123.696,"15min":123.696},"last":1.5229999999999999},{"interface":"front","average":{"1min":7.6379999999999999,"5min":14.551,"15min":15.702999999999999},"min":{"1min":1.163,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":68.363,"5min":85.879999999999995,"15min":85.879999999999995},"last":1.609}]},{"osd":14,"last update":"Wed Sep 18 10:16:22 2024","interfaces":[{"interface":"back","average":{"1min":15.864000000000001,"5min":15.864000000000001,"15min":15.864000000000001},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":145.46700000000001,"5min":145.46700000000001,"15min":145.46700000000001},"last":1.4830000000000001},{"interface":"front","average":{"1min":13.93,"5min":13.93,"15min":13.93},"min":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"max":{"1min":100.092,"5min":100.092,"15min":100.092},"last":1.587}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1080000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T10:16:55.888 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T10:16:56.191 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-18T10:16:56.194 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-09-18T10:16:56.206 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":209,"stamp":"2024-09-18T10:16:55.239416+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":147,"num_write_kb":2674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":115,"ondisk_log_size":115,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":14,"kb":1509949440,"kb_used":4865408,"kb_used_data":36952,"kb_used_omap":26,"kb_used_meta":4828197,"kb_avail":1505084032,"statfs":{"total":1546188226560,"available":1541206048768,"internally_reserved":0,"allocated":37838848,"data_stored":22477174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":27264,"internal_metadata":4944074112},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.056129"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":144,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-18T10:16:49.404525+0000","last_change":"2024-09-18T10:16:18.085265+0000","last_active":"2024-09-18T10:16:49.404525+0000","last_peered":"2024-09-18T10:16:49.404525+0000","last_clean":"2024-09-18T10:16:49.404525+0000","last_became_active":"2024-09-18T10:14:35.348943+0000","last_became_peered":"2024-09-18T10:14:35.348943+0000","last_unstale":"2024-09-18T10:16:49.404525+0000","last_undegraded":"2024-09-18T10:16:49.404525+0000","last_fullsized":"2024-09-18T10:16:49.404525+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:16:18.085027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:16:18.085027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:17:43.814969+0000","scrub_duration":0.30385945199999997,"objects_trimmed":0,"snaptrim_duration":0.00060442999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,10],"acting":[13,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":76,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-18T10:16:49.402946+0000","last_change":"2024-09-18T10:16:18.981603+0000","last_active":"2024-09-18T10:16:49.402946+0000","last_peered":"2024-09-18T10:16:49.402946+0000","last_clean":"2024-09-18T10:16:49.402946+0000","last_became_active":"2024-09-18T10:15:50.978055+0000","last_became_peered":"2024-09-18T10:15:50.978055+0000","last_unstale":"2024-09-18T10:16:49.402946+0000","last_undegraded":"2024-09-18T10:16:49.402946+0000","last_fullsized":"2024-09-18T10:16:49.402946+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:16:18.981373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:16:18.981373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:17:41.742568+0000","scrub_duration":0.40456036899999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11],"acting":[8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":140,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-18T10:16:49.303750+0000","last_change":"2024-09-18T10:15:50.883382+0000","last_active":"2024-09-18T10:16:49.303750+0000","last_peered":"2024-09-18T10:16:49.303750+0000","last_clean":"2024-09-18T10:16:49.303750+0000","last_became_active":"2024-09-18T10:14:35.329198+0000","last_became_peered":"2024-09-18T10:14:35.329198+0000","last_unstale":"2024-09-18T10:16:49.303750+0000","last_undegraded":"2024-09-18T10:16:49.303750+0000","last_fullsized":"2024-09-18T10:16:49.303750+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:15:50.883147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:15:50.883147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:17:15.846467+0000","scrub_duration":0.153222372,"objects_trimmed":0,"snaptrim_duration":0.00062415699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":142,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T10:16:48.501848+0000","last_change":"2024-09-18T10:16:02.749341+0000","last_active":"2024-09-18T10:16:48.501848+0000","last_peered":"2024-09-18T10:16:48.501848+0000","last_clean":"2024-09-18T10:16:48.501848+0000","last_became_active":"2024-09-18T10:14:35.524102+0000","last_became_peered":"2024-09-18T10:14:35.524102+0000","last_unstale":"2024-09-18T10:16:48.501848+0000","last_undegraded":"2024-09-18T10:16:48.501848+0000","last_fullsized":"2024-09-18T10:16:48.501848+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:16:02.749150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:16:02.749150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:17:04.725472+0000","scrub_duration":0.20368988499999999,"objects_trimmed":0,"snaptrim_duration":0.10077985,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"29'2","reported_seq":145,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-18T10:16:49.304395+0000","last_change":"2024-09-18T10:16:20.864837+0000","last_active":"2024-09-18T10:16:49.304395+0000","last_peered":"2024-09-18T10:16:49.304395+0000","last_clean":"2024-09-18T10:16:49.304395+0000","last_became_active":"2024-09-18T10:14:35.715374+0000","last_became_peered":"2024-09-18T10:14:35.715374+0000","last_unstale":"2024-09-18T10:16:49.304395+0000","last_undegraded":"2024-09-18T10:16:49.304395+0000","last_fullsized":"2024-09-18T10:16:49.304395+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2","last_scrub_stamp":"2024-09-18T10:16:20.864635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:16:20.864635+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:17:23.226052+0000","scrub_duration":0.20463374300000001,"objects_trimmed":0,"snaptrim_duration":0.002421224,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,12],"acting":[13,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":150,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-18T10:16:49.186308+0000","last_change":"2024-09-18T10:16:15.062108+0000","last_active":"2024-09-18T10:16:49.186308+0000","last_peered":"2024-09-18T10:16:49.186308+0000","last_clean":"2024-09-18T10:16:49.186308+0000","last_became_active":"2024-09-18T10:14:35.348262+0000","last_became_peered":"2024-09-18T10:14:35.348262+0000","last_unstale":"2024-09-18T10:16:49.186308+0000","last_undegraded":"2024-09-18T10:16:49.186308+0000","last_fullsized":"2024-09-18T10:16:49.186308+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:16:15.061892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:16:15.061892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:17:40.002913+0000","scrub_duration":0.40410323300000001,"objects_trimmed":0,"snaptrim_duration":0.00063430400000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,0],"acting":[14,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":144,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-18T10:16:52.538159+0000","last_change":"2024-09-18T10:16:15.352277+0000","last_active":"2024-09-18T10:16:52.538159+0000","last_peered":"2024-09-18T10:16:52.538159+0000","last_clean":"2024-09-18T10:16:52.538159+0000","last_became_active":"2024-09-18T10:14:35.349203+0000","last_became_peered":"2024-09-18T10:14:35.349203+0000","last_unstale":"2024-09-18T10:16:52.538159+0000","last_undegraded":"2024-09-18T10:16:52.538159+0000","last_fullsized":"2024-09-18T10:16:52.538159+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:16:15.352050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:16:15.352050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:17:45.176858+0000","scrub_duration":0.203632596,"objects_trimmed":0,"snaptrim_duration":0.00060130600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3],"acting":[9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"27'1","reported_seq":145,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-18T10:16:49.186339+0000","last_change":"2024-09-18T10:16:06.900858+0000","last_active":"2024-09-18T10:16:49.186339+0000","last_peered":"2024-09-18T10:16:49.186339+0000","last_clean":"2024-09-18T10:16:49.186339+0000","last_became_active":"2024-09-18T10:14:35.487746+0000","last_became_peered":"2024-09-18T10:14:35.487746+0000","last_unstale":"2024-09-18T10:16:49.186339+0000","last_undegraded":"2024-09-18T10:16:49.186339+0000","last_fullsized":"2024-09-18T10:16:49.186339+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'1","last_scrub_stamp":"2024-09-18T10:16:06.900601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:16:06.900601+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:17:25.247813+0000","scrub_duration":0.27078197700000001,"objects_trimmed":0,"snaptrim_duration":0.10072764100000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,5],"acting":[14,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"1.0","version":"29'112","reported_seq":275,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-18T10:16:49.187571+0000","last_change":"2024-09-18T10:16:03.315273+0000","last_active":"2024-09-18T10:16:49.187571+0000","last_peered":"2024-09-18T10:16:49.187571+0000","last_clean":"2024-09-18T10:16:49.187571+0000","last_became_active":"2024-09-18T10:14:32.508037+0000","last_became_peered":"2024-09-18T10:14:32.508037+0000","last_unstale":"2024-09-18T10:16:49.187571+0000","last_undegraded":"2024-09-18T10:16:49.187571+0000","last_fullsized":"2024-09-18T10:16:49.187571+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'112","last_scrub_stamp":"2024-09-18T10:16:03.315126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:31.267741+0000","last_clean_scrub_stamp":"2024-09-18T10:16:03.315126+0000","objects_scrubbed":2,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:17:16.808627+0000","scrub_duration":0.10318229800000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,15],"acting":[6,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":13},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":112,"ondisk_log_size":112,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":15,"up_from":22,"seq":94489280543,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304272,"kb_used_data":2752,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94067568,"statfs":{"total":96636764160,"available":96325189632,"internally_reserved":0,"allocated":2818048,"data_stored":1860517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,4,6,7,8,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:16:32 2024","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":0.83899999999999997,"15min":0.81899999999999995},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":2.0019999999999998,"5min":2.0019999999999998,"15min":2.0019999999999998},"last":1.0649999999999999},{"interface":"front","average":{"1min":0.996,"5min":5.5739999999999998,"15min":6.3369999999999997},"min":{"1min":0.72899999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.506,"5min":101.291,"15min":101.291},"last":0.97299999999999998}]},{"osd":1,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.93300000000000005},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.931,"5min":1.931,"15min":1.931},"last":1.1279999999999999}]},{"osd":2,"last update":"Wed Sep 18 10:16:32 2024","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.0169999999999999,"15min":0.995},"min":{"1min":0.79700000000000004,"5min":0.5,"15min":0.5},"max":{"1min":1.7470000000000001,"5min":1.7470000000000001,"15min":1.7470000000000001},"last":1.028},{"interface":"front","average":{"1min":1.1419999999999999,"5min":4.6589999999999998,"15min":5.2450000000000001},"min":{"1min":0.82899999999999996,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.4379999999999999,"5min":78.242999999999995,"15min":78.242999999999995},"last":1.252}]},{"osd":3,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":3.3029999999999999,"5min":3.3029999999999999,"15min":3.3029999999999999},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":41.720999999999997,"5min":41.720999999999997,"15min":41.720999999999997},"last":0.55400000000000005},{"interface":"front","average":{"1min":1.014,"5min":1.014,"15min":1.014},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.49,"5min":1.49,"15min":1.49},"last":0.89500000000000002}]},{"osd":4,"last update":"Wed Sep 18 10:16:32 2024","interfaces":[{"interface":"back","average":{"1min":1.3580000000000001,"5min":9.0879999999999992,"15min":10.375999999999999},"min":{"1min":0.747,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":8.0449999999999999,"5min":99.745999999999995,"15min":99.745999999999995},"last":1.2729999999999999},{"interface":"front","average":{"1min":1.595,"5min":9.2309999999999999,"15min":10.503},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":8.0679999999999996,"5min":99.781000000000006,"15min":99.781000000000006},"last":1.222}]},{"osd":5,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":1.0860000000000001},{"interface":"front","average":{"1min":1.7869999999999999,"5min":1.7869999999999999,"15min":1.7869999999999999},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":10.874000000000001,"5min":10.874000000000001,"15min":10.874000000000001},"last":1.163}]},{"osd":6,"last update":"Wed Sep 18 10:16:32 2024","interfaces":[{"interface":"back","average":{"1min":14.140000000000001,"5min":4.4039999999999999,"15min":2.782},"min":{"1min":0.92900000000000005,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":269.80500000000001,"5min":269.80500000000001,"15min":269.80500000000001},"last":1},{"interface":"front","average":{"1min":1.361,"5min":4.7610000000000001,"15min":5.327},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":2.012,"5min":74.156000000000006,"15min":74.156000000000006},"last":1.1919999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80400000000000005}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93000000000000005}]},{"osd":10,"last update":"Wed Sep 18 10:16:32 2024","interfaces":[{"interface":"back","average":{"1min":5.8639999999999999,"5min":6.2670000000000003,"15min":6.3339999999999996},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":93.704999999999998,"5min":93.704999999999998,"15min":93.704999999999998},"last":0.85399999999999998},{"interface":"front","average":{"1min":5.7770000000000001,"5min":6.2549999999999999,"15min":6.335},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":93.671000000000006,"5min":93.671000000000006,"15min":93.671000000000006},"last":1.121}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.254}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3680000000000001}]},{"osd":13,"last update":"Wed Sep 18 10:16:32 2024","interfaces":[{"interface":"back","average":{"1min":1.99,"5min":5.2629999999999999,"15min":5.8090000000000002},"min":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"max":{"1min":7.3179999999999996,"5min":80.197000000000003,"15min":80.197000000000003},"last":1.5029999999999999},{"interface":"front","average":{"1min":3.4830000000000001,"5min":1.8080000000000001,"15min":1.5289999999999999},"min":{"1min":0.89400000000000002,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":25.193999999999999,"5min":25.193999999999999,"15min":25.193999999999999},"last":1.4670000000000001}]},{"osd":14,"last update":"Wed Sep 18 10:16:32 2024","interfaces":[{"interface":"back","average":{"1min":10.725,"5min":3.2999999999999998,"15min":2.0619999999999998},"min":{"1min":0.95699999999999996,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":117.562,"5min":117.562,"15min":117.562},"last":1.5980000000000001},{"interface":"front","average":{"1min":4.6139999999999999,"5min":9.5229999999999997,"15min":10.340999999999999},"min":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"max":{"1min":61.567,"5min":85.006,"15min":85.006},"last":1.4390000000000001}]}]},{"osd":14,"up_from":22,"seq":94489280543,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303768,"kb_used_data":2248,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068072,"statfs":{"total":96636764160,"available":96325705728,"internally_reserved":0,"allocated":2301952,"data_stored":1341063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,5,7,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:16:32 2024","interfaces":[{"interface":"back","average":{"1min":1.0009999999999999,"5min":12.513,"15min":14.430999999999999},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.405,"5min":189.88800000000001,"15min":189.88800000000001},"last":0.59599999999999997},{"interface":"front","average":{"1min":7.0330000000000004,"5min":17.033000000000001,"15min":18.699000000000002},"min":{"1min":0.77300000000000002,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":71.900999999999996,"5min":180.09999999999999,"15min":180.09999999999999},"last":0.68700000000000006}]},{"osd":1,"last update":"Wed Sep 18 10:16:32 2024","interfaces":[{"interface":"back","average":{"1min":2.1040000000000001,"5min":7.7670000000000003,"15min":8.7110000000000003},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":21.334,"5min":79.626000000000005,"15min":79.626000000000005},"last":0.754},{"interface":"front","average":{"1min":1.1479999999999999,"5min":12.601000000000001,"15min":14.51},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.8879999999999999,"5min":189.762,"15min":189.762},"last":0.85899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.417}]},{"osd":3,"last update":"Wed Sep 18 10:16:32 2024","interfaces":[{"interface":"back","average":{"1min":3.5430000000000001,"5min":12.906000000000001,"15min":14.465999999999999},"min":{"1min":0.84799999999999998,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":44.683,"5min":180.19900000000001,"15min":180.19900000000001},"last":1.7529999999999999},{"interface":"front","average":{"1min":5.3520000000000003,"5min":18.143999999999998,"15min":20.276},"min":{"1min":0.86399999999999999,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":42.570999999999998,"5min":180.851,"15min":180.851},"last":1.1899999999999999}]},{"osd":4,"last update":"Wed Sep 18 10:16:32 2024","interfaces":[{"interface":"back","average":{"1min":6.8360000000000003,"5min":21.687000000000001,"15min":24.161999999999999},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":71.911000000000001,"5min":189.809,"15min":189.809},"last":2.0249999999999999},{"interface":"front","average":{"1min":14.032999999999999,"5min":15.031000000000001,"15min":15.196999999999999},"min":{"1min":0.76100000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":133.244,"5min":180.31700000000001,"15min":180.31700000000001},"last":1.9350000000000001}]},{"osd":5,"last update":"Wed Sep 18 10:16:32 2024","interfaces":[{"interface":"back","average":{"1min":1.831,"5min":21.507000000000001,"15min":24.786000000000001},"min":{"1min":0.66300000000000003,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":13.42,"5min":277.41199999999998,"15min":277.41199999999998},"last":1.1020000000000001},{"interface":"front","average":{"1min":1.9530000000000001,"5min":12.936999999999999,"15min":14.766999999999999},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":14.288,"5min":189.64699999999999,"15min":189.64699999999999},"last":0.91900000000000004}]},{"osd":7,"last update":"Wed Sep 18 10:16:32 2024","interfaces":[{"interface":"back","average":{"1min":6.29,"5min":13.625999999999999,"15min":14.848000000000001},"min":{"1min":0.995,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":49.378,"5min":180.268,"15min":180.268},"last":1.448},{"interface":"front","average":{"1min":3.8929999999999998,"5min":12.901,"15min":14.403},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":49.402999999999999,"5min":180.631,"15min":180.631},"last":1.494}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1619999999999999}]},{"osd":9,"last update":"Wed Sep 18 10:16:17 2024","interfaces":[{"interface":"back","average":{"1min":4,"5min":4,"15min":4},"min":{"1min":1.296,"5min":1.296,"15min":1.296},"max":{"1min":44.716000000000001,"5min":44.716000000000001,"15min":44.716000000000001},"last":1.204},{"interface":"front","average":{"1min":1.427,"5min":1.427,"15min":1.427},"min":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"max":{"1min":1.9610000000000001,"5min":1.9610000000000001,"15min":1.9610000000000001},"last":1.274}]},{"osd":10,"last update":"Wed Sep 18 10:15:49 2024","interfaces":[{"interface":"back","average":{"1min":17.039000000000001,"5min":17.039000000000001,"15min":17.039000000000001},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":180.15899999999999,"5min":180.15899999999999,"15min":180.15899999999999},"last":1},{"interface":"front","average":{"1min":31.004000000000001,"5min":31.004000000000001,"15min":31.004000000000001},"min":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":269.423,"5min":269.423,"15min":269.423},"last":1.381}]},{"osd":11,"last update":"Wed Sep 18 10:16:10 2024","interfaces":[{"interface":"back","average":{"1min":9.6890000000000001,"5min":9.6890000000000001,"15min":9.6890000000000001},"min":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":91.111999999999995,"5min":91.111999999999995,"15min":91.111999999999995},"last":1.3520000000000001},{"interface":"front","average":{"1min":8.1460000000000008,"5min":8.1460000000000008,"15min":8.1460000000000008},"min":{"1min":1.181,"5min":1.181,"15min":1.181},"max":{"1min":91.165000000000006,"5min":91.165000000000006,"15min":91.165000000000006},"last":1.1519999999999999}]},{"osd":12,"last update":"Wed Sep 18 10:16:32 2024","interfaces":[{"interface":"back","average":{"1min":10.385,"5min":14.760999999999999,"15min":15.49},"min":{"1min":1.327,"5min":0.91100000000000003,"15min":0.91100000000000003},"max":{"1min":65.189999999999998,"5min":189.84999999999999,"15min":189.84999999999999},"last":1.4079999999999999},{"interface":"front","average":{"1min":10.818,"5min":14.696,"15min":15.342000000000001},"min":{"1min":1.4890000000000001,"5min":0.91200000000000003,"15min":0.91200000000000003},"max":{"1min":65.162000000000006,"5min":180.381,"15min":180.381},"last":1.7729999999999999}]},{"osd":13,"last update":"Wed Sep 18 10:16:32 2024","interfaces":[{"interface":"back","average":{"1min":10.321999999999999,"5min":23.059000000000001,"15min":25.181999999999999},"min":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"max":{"1min":166.001,"5min":180.59100000000001,"15min":180.59100000000001},"last":1.5720000000000001},{"interface":"front","average":{"1min":1.647,"5min":20.391999999999999,"15min":23.515999999999998},"min":{"1min":0.83399999999999996,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":2.3940000000000001,"5min":189.71299999999999,"15min":189.71299999999999},"last":1.3120000000000001}]},{"osd":15,"last update":"Wed Sep 18 10:16:32 2024","interfaces":[{"interface":"back","average":{"1min":4.181,"5min":13.381,"15min":14.914},"min":{"1min":1.5780000000000001,"5min":0.99399999999999999,"15min":0.99399999999999999},"max":{"1min":44.917000000000002,"5min":180.44800000000001,"15min":180.44800000000001},"last":1.6619999999999999},{"interface":"front","average":{"1min":1.7030000000000001,"5min":15.118,"15min":17.353999999999999},"min":{"1min":0.93200000000000005,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":2.1909999999999998,"5min":180.53,"15min":180.53},"last":1.8069999999999999}]}]},{"osd":13,"up_from":22,"seq":94489280544,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307932,"kb_used_data":2252,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94063908,"statfs":{"total":96636764160,"available":96321441792,"internally_reserved":0,"allocated":2306048,"data_stored":1341082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,3,6,8,9,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:16:33 2024","interfaces":[{"interface":"back","average":{"1min":2.2599999999999998,"5min":17.045000000000002,"15min":19.509},"min":{"1min":0.65700000000000003,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":22.972999999999999,"5min":176.66800000000001,"15min":176.66800000000001},"last":0.83899999999999997},{"interface":"front","average":{"1min":4.2629999999999999,"5min":25.199000000000002,"15min":28.687999999999999},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":62.533000000000001,"5min":199.732,"15min":199.732},"last":0.90000000000000002}]},{"osd":1,"last update":"Wed Sep 18 10:15:48 2024","interfaces":[{"interface":"back","average":{"1min":1.986,"5min":1.986,"15min":1.986},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":18.471,"5min":18.471,"15min":18.471},"last":1.4219999999999999},{"interface":"front","average":{"1min":20.716999999999999,"5min":20.716999999999999,"15min":20.716999999999999},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":176.56800000000001,"5min":176.56800000000001,"15min":176.56800000000001},"last":0.86399999999999999}]},{"osd":2,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":11.984,"5min":11.984,"15min":11.984},"min":{"1min":0.751,"5min":0.751,"15min":0.751},"max":{"1min":196.59200000000001,"5min":196.59200000000001,"15min":196.59200000000001},"last":0.97699999999999998},{"interface":"front","average":{"1min":26.109999999999999,"5min":26.109999999999999,"15min":26.109999999999999},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":177.03,"5min":177.03,"15min":177.03},"last":1.2689999999999999}]},{"osd":3,"last update":"Wed Sep 18 10:16:33 2024","interfaces":[{"interface":"back","average":{"1min":9.5,"5min":22.245000000000001,"15min":24.369},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":158.74100000000001,"5min":168.27699999999999,"15min":168.27699999999999},"last":1.27},{"interface":"front","average":{"1min":1.2290000000000001,"5min":9.8819999999999997,"15min":11.324},"min":{"1min":0.84499999999999997,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.8959999999999999,"5min":196.643,"15min":196.643},"last":0.95399999999999996}]},{"osd":4,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":29.802,"5min":29.802,"15min":29.802},"min":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"max":{"1min":168.48099999999999,"5min":168.48099999999999,"15min":168.48099999999999},"last":1.347},{"interface":"front","average":{"1min":8.1780000000000008,"5min":8.1780000000000008,"15min":8.1780000000000008},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":65.188000000000002,"5min":65.188000000000002,"15min":65.188000000000002},"last":1.6379999999999999}]},{"osd":5,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":26.152999999999999,"5min":26.152999999999999,"15min":26.152999999999999},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":176.48099999999999,"5min":176.48099999999999,"15min":176.48099999999999},"last":1.49},{"interface":"front","average":{"1min":24.885999999999999,"5min":24.885999999999999,"15min":24.885999999999999},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":167.62299999999999,"5min":167.62299999999999,"15min":167.62299999999999},"last":1.528}]},{"osd":6,"last update":"Wed Sep 18 10:16:33 2024","interfaces":[{"interface":"back","average":{"1min":2.6589999999999998,"5min":24.015000000000001,"15min":27.574999999999999},"min":{"1min":1.042,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":23.131,"5min":176.81200000000001,"15min":176.81200000000001},"last":1.1080000000000001},{"interface":"front","average":{"1min":4.7169999999999996,"5min":23.227,"15min":26.312000000000001},"min":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"max":{"1min":61.874000000000002,"5min":167.947,"15min":167.947},"last":1.3340000000000001}]},{"osd":8,"last update":"Wed Sep 18 10:16:50 2024","interfaces":[{"interface":"back","average":{"1min":1.5700000000000001,"5min":1.5700000000000001,"15min":1.5700000000000001},"min":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"max":{"1min":2.117,"5min":2.117,"15min":2.117},"last":1.175},{"interface":"front","average":{"1min":1.514,"5min":1.514,"15min":1.514},"min":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"max":{"1min":2.278,"5min":2.278,"15min":2.278},"last":1.149}]},{"osd":9,"last update":"Wed Sep 18 10:16:50 2024","interfaces":[{"interface":"back","average":{"1min":2.617,"5min":17.367000000000001,"15min":19.824999999999999},"min":{"1min":0.878,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":22.934000000000001,"5min":176.73599999999999,"15min":176.73599999999999},"last":1.206},{"interface":"front","average":{"1min":1.5660000000000001,"5min":19.085000000000001,"15min":22.004999999999999},"min":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":2.448,"5min":167.84899999999999,"15min":167.84899999999999},"last":1.446}]},{"osd":10,"last update":"Wed Sep 18 10:16:39 2024","interfaces":[{"interface":"back","average":{"1min":5.1120000000000001,"5min":19.164999999999999,"15min":21.507000000000001},"min":{"1min":1.22,"5min":1.1359999999999999,"15min":1.1359999999999999},"max":{"1min":62.137999999999998,"5min":168.07900000000001,"15min":168.07900000000001},"last":1.591},{"interface":"front","average":{"1min":17.683,"5min":29.114000000000001,"15min":31.018999999999998},"min":{"1min":1.123,"5min":1.0649999999999999,"15min":1.0649999999999999},"max":{"1min":289.68700000000001,"5min":289.68700000000001,"15min":289.68700000000001},"last":1.2450000000000001}]},{"osd":11,"last update":"Wed Sep 18 10:16:50 2024","interfaces":[{"interface":"back","average":{"1min":1.6819999999999999,"5min":1.625,"15min":1.615},"min":{"1min":1.056,"5min":1.0029999999999999,"15min":1.0029999999999999},"max":{"1min":2.5699999999999998,"5min":2.5699999999999998,"15min":2.5699999999999998},"last":1.5009999999999999},{"interface":"front","average":{"1min":2.798,"5min":21.73,"15min":24.885999999999999},"min":{"1min":1.286,"5min":1.028,"15min":1.028},"max":{"1min":23.167999999999999,"5min":176.94999999999999,"15min":176.94999999999999},"last":1.377}]},{"osd":12,"last update":"Wed Sep 18 10:16:33 2024","interfaces":[{"interface":"back","average":{"1min":1.8440000000000001,"5min":1.6339999999999999,"15min":1.599},"min":{"1min":1.2490000000000001,"5min":1.0009999999999999,"15min":1.0009999999999999},"max":{"1min":2.5950000000000002,"5min":2.7879999999999998,"15min":2.7879999999999998},"last":1.413},{"interface":"front","average":{"1min":5.0570000000000004,"5min":18.542999999999999,"15min":20.791},"min":{"1min":1.3029999999999999,"5min":1.296,"15min":1.296},"max":{"1min":62.200000000000003,"5min":168.34399999999999,"15min":168.34399999999999},"last":1.5449999999999999}]},{"osd":14,"last update":"Wed Sep 18 10:16:33 2024","interfaces":[{"interface":"back","average":{"1min":1.8320000000000001,"5min":5.7210000000000001,"15min":6.3689999999999998},"min":{"1min":1.139,"5min":1.0640000000000001,"15min":1.0640000000000001},"max":{"1min":2.419,"5min":87.427000000000007,"15min":87.427000000000007},"last":1.665},{"interface":"front","average":{"1min":2.9329999999999998,"5min":21.571999999999999,"15min":24.678000000000001},"min":{"1min":1.2709999999999999,"5min":1.0980000000000001,"15min":1.0980000000000001},"max":{"1min":23.268000000000001,"5min":176.87700000000001,"15min":176.87700000000001},"last":1.468}]},{"osd":15,"last update":"Wed Sep 18 10:16:33 2024","interfaces":[{"interface":"back","average":{"1min":10.048999999999999,"5min":18.277000000000001,"15min":19.648},"min":{"1min":1.272,"5min":1.266,"15min":1.266},"max":{"1min":96.564999999999998,"5min":168.40700000000001,"15min":168.40700000000001},"last":1.7110000000000001},{"interface":"front","average":{"1min":1.891,"5min":13.459,"15min":15.387},"min":{"1min":1.1950000000000001,"5min":0.95099999999999996,"15min":0.95099999999999996},"max":{"1min":2.5169999999999999,"5min":196.512,"15min":196.512},"last":1.5609999999999999}]}]},{"osd":0,"up_from":21,"seq":90194313248,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303780,"kb_used_data":2264,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068060,"statfs":{"total":96636764160,"available":96325693440,"internally_reserved":0,"allocated":2318336,"data_stored":1350579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,4,5,6,9,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 18 10:16:39 2024","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.95699999999999996,"15min":0.95899999999999996},"min":{"1min":0.56699999999999995,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.2889999999999999,"5min":1.341,"15min":1.341},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.97899999999999998,"15min":0.98499999999999999},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.2310000000000001,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":0.625}]},{"osd":2,"last update":"Wed Sep 18 10:16:39 2024","interfaces":[{"interface":"back","average":{"1min":3.2719999999999998,"5min":1.6020000000000001,"15min":1.3240000000000001},"min":{"1min":0.76600000000000001,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":46.220999999999997,"5min":46.220999999999997,"15min":46.220999999999997},"last":1.0349999999999999},{"interface":"front","average":{"1min":0.98799999999999999,"5min":1.071,"15min":1.085},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.732,"5min":1.732,"15min":1.732},"last":0.57499999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.196}]},{"osd":4,"last update":"Wed Sep 18 10:16:30 2024","interfaces":[{"interface":"back","average":{"1min":8.4039999999999999,"5min":14,"15min":14.933},"min":{"1min":0.59699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":142.20699999999999,"5min":142.20699999999999,"15min":142.20699999999999},"last":0.49399999999999999},{"interface":"front","average":{"1min":10.48,"5min":14.896000000000001,"15min":15.632},"min":{"1min":0.63,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":132.33799999999999,"5min":132.33799999999999,"15min":132.33799999999999},"last":0.55200000000000005}]},{"osd":5,"last update":"Wed Sep 18 10:16:39 2024","interfaces":[{"interface":"back","average":{"1min":1.1739999999999999,"5min":1.24,"15min":1.2509999999999999},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":1.619,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":1.569,"5min":8.1340000000000003,"15min":9.2279999999999998},"min":{"1min":0.85599999999999998,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":10.143000000000001,"5min":130.39599999999999,"15min":130.39599999999999},"last":0.92100000000000004}]},{"osd":6,"last update":"Wed Sep 18 10:16:47 2024","interfaces":[{"interface":"back","average":{"1min":1.266,"5min":1.254,"15min":1.252},"min":{"1min":0.96099999999999997,"5min":0.94299999999999995,"15min":0.94299999999999995},"max":{"1min":1.571,"5min":1.776,"15min":1.776},"last":0.82599999999999996},{"interface":"front","average":{"1min":1.3109999999999999,"5min":2.0270000000000001,"15min":2.1469999999999998},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.7909999999999999,"5min":15.250999999999999,"15min":15.250999999999999},"last":0.66100000000000003}]},{"osd":9,"last update":"Wed Sep 18 10:16:30 2024","interfaces":[{"interface":"back","average":{"1min":1.3360000000000001,"5min":1.3919999999999999,"15min":1.4019999999999999},"min":{"1min":0.88200000000000001,"5min":0.879,"15min":0.879},"max":{"1min":1.863,"5min":1.863,"15min":1.863},"last":0.74099999999999999},{"interface":"front","average":{"1min":1.667,"5min":8.7590000000000003,"15min":9.9410000000000007},"min":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":10.180999999999999,"5min":130.441,"15min":130.441},"last":0.88400000000000001}]},{"osd":11,"last update":"Wed Sep 18 10:16:30 2024","interfaces":[{"interface":"back","average":{"1min":4.8949999999999996,"5min":2.2109999999999999,"15min":1.7629999999999999},"min":{"1min":1.1579999999999999,"5min":0.90700000000000003,"15min":0.90700000000000003},"max":{"1min":41.610999999999997,"5min":41.610999999999997,"15min":41.610999999999997},"last":1.0700000000000001},{"interface":"front","average":{"1min":1.5029999999999999,"5min":4.5620000000000003,"15min":5.0720000000000001},"min":{"1min":1.1220000000000001,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":1.9850000000000001,"5min":54.805999999999997,"15min":54.805999999999997},"last":0.87}]},{"osd":12,"last update":"Wed Sep 18 10:16:30 2024","interfaces":[{"interface":"back","average":{"1min":1.7869999999999999,"5min":9.1170000000000009,"15min":10.339},"min":{"1min":0.95999999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":10.015000000000001,"5min":130.34800000000001,"15min":130.34800000000001},"last":0.95399999999999996},{"interface":"front","average":{"1min":1.4350000000000001,"5min":1.4710000000000001,"15min":1.4770000000000001},"min":{"1min":0.85899999999999999,"5min":0.502,"15min":0.502},"max":{"1min":1.944,"5min":1.944,"15min":1.944},"last":0.85499999999999998}]},{"osd":14,"last update":"Wed Sep 18 10:16:39 2024","interfaces":[{"interface":"back","average":{"1min":3.7490000000000001,"5min":8.7810000000000006,"15min":9.6199999999999992},"min":{"1min":1.044,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":38.969999999999999,"5min":130.48500000000001,"15min":130.48500000000001},"last":0.98699999999999999},{"interface":"front","average":{"1min":6.931,"5min":2.6339999999999999,"15min":1.9179999999999999},"min":{"1min":1.2609999999999999,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":80.460999999999999,"5min":80.460999999999999,"15min":80.460999999999999},"last":1.099}]},{"osd":15,"last update":"Wed Sep 18 10:16:39 2024","interfaces":[{"interface":"back","average":{"1min":1.591,"5min":2.778,"15min":2.976},"min":{"1min":1.1699999999999999,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":2.0310000000000001,"5min":23.806999999999999,"15min":23.806999999999999},"last":0.77200000000000002},{"interface":"front","average":{"1min":6.9199999999999999,"5min":9.4809999999999999,"15min":9.9079999999999995},"min":{"1min":1.1279999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":100.925,"5min":130.52799999999999,"15min":130.52799999999999},"last":1.018}]}]},{"osd":4,"up_from":21,"seq":90194313249,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303752,"kb_used_data":2232,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068088,"statfs":{"total":96636764160,"available":96325722112,"internally_reserved":0,"allocated":2285568,"data_stored":1331589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,3,5,7,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":9.0980000000000008,"15min":10.465999999999999},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.7529999999999999,"5min":89.266999999999996,"15min":89.266999999999996},"last":0.626},{"interface":"front","average":{"1min":18.640000000000001,"5min":7.7320000000000002,"15min":5.9139999999999997},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":198.80699999999999,"5min":198.80699999999999,"15min":198.80699999999999},"last":0.66000000000000003}]},{"osd":1,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":4.0759999999999996,"5min":12.537000000000001,"15min":13.946999999999999},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":62.780000000000001,"5min":164.21199999999999,"15min":164.21199999999999},"last":0.72099999999999997},{"interface":"front","average":{"1min":2.5139999999999998,"5min":9.593,"15min":10.773},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":25.802,"5min":89.331000000000003,"15min":89.331000000000003},"last":0.77500000000000002}]},{"osd":2,"last update":"Wed Sep 18 10:15:47 2024","interfaces":[{"interface":"back","average":{"1min":23.617000000000001,"5min":23.617000000000001,"15min":23.617000000000001},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":192.136,"5min":192.136,"15min":192.136},"last":1.8740000000000001},{"interface":"front","average":{"1min":11.888999999999999,"5min":11.888999999999999,"15min":11.888999999999999},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":198.87700000000001,"5min":198.87700000000001,"15min":198.87700000000001},"last":1.347}]},{"osd":3,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":35.673000000000002,"5min":21.670999999999999,"15min":19.337},"min":{"1min":1.161,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":388.84300000000002,"5min":388.84300000000002,"15min":388.84300000000002},"last":1.462},{"interface":"front","average":{"1min":17.747,"5min":25.643999999999998,"15min":26.960000000000001},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":255.82900000000001,"5min":255.82900000000001,"15min":255.82900000000001},"last":0.997}]},{"osd":5,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":1.4910000000000001,"5min":15.449999999999999,"15min":17.776},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":2.169,"5min":189.55600000000001,"15min":189.55600000000001},"last":0.83099999999999996},{"interface":"front","average":{"1min":21.053000000000001,"5min":6.8079999999999998,"15min":4.4340000000000002},"min":{"1min":1.3640000000000001,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":199.15600000000001,"5min":199.15600000000001,"15min":199.15600000000001},"last":1.155}]},{"osd":6,"last update":"Wed Sep 18 10:15:34 2024","interfaces":[{"interface":"back","average":{"1min":32.25,"5min":32.25,"15min":32.25},"min":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":197.33199999999999,"5min":197.33199999999999,"15min":197.33199999999999},"last":1.8160000000000001},{"interface":"front","average":{"1min":18.963000000000001,"5min":18.963000000000001,"15min":18.963000000000001},"min":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"max":{"1min":179.19200000000001,"5min":179.19200000000001,"15min":179.19200000000001},"last":1.7769999999999999}]},{"osd":7,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":5.6050000000000004,"5min":23.198,"15min":26.129999999999999},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":62.704999999999998,"5min":290.036,"15min":290.036},"last":1.2689999999999999},{"interface":"front","average":{"1min":1.6639999999999999,"5min":14.273,"15min":16.375},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":2.3100000000000001,"5min":139.696,"15min":139.696},"last":0.93899999999999995}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1509999999999998}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7509999999999999}]},{"osd":10,"last update":"Wed Sep 18 10:16:49 2024","interfaces":[{"interface":"back","average":{"1min":1.653,"5min":18.559000000000001,"15min":21.376999999999999},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":2.4740000000000002,"5min":164.43899999999999,"15min":164.43899999999999},"last":0.89800000000000002},{"interface":"front","average":{"1min":4.6070000000000002,"5min":10.993,"15min":12.057},"min":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":59.408999999999999,"5min":89.442999999999998,"15min":89.442999999999998},"last":1.3049999999999999}]},{"osd":11,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":19.57,"5min":12.567,"15min":11.4},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":199.255,"5min":199.255,"15min":199.255},"last":1.3340000000000001},{"interface":"front","average":{"1min":11.393000000000001,"5min":22.213000000000001,"15min":24.016999999999999},"min":{"1min":1.0840000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"max":{"1min":155.19800000000001,"5min":164.733,"15min":164.733},"last":1.4399999999999999}]},{"osd":12,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":4.5800000000000001,"5min":10.406000000000001,"15min":11.377000000000001},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":58.856000000000002,"5min":89.400000000000006,"15min":89.400000000000006},"last":1.113},{"interface":"front","average":{"1min":31.831,"5min":12.901999999999999,"15min":9.7469999999999999},"min":{"1min":1.4530000000000001,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":199.32900000000001,"5min":199.32900000000001,"15min":199.32900000000001},"last":1.5940000000000001}]},{"osd":13,"last update":"Wed Sep 18 10:16:49 2024","interfaces":[{"interface":"back","average":{"1min":12.695,"5min":22.789999999999999,"15min":24.472000000000001},"min":{"1min":1.702,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":192.37299999999999,"5min":199.37799999999999,"15min":199.37799999999999},"last":1.706},{"interface":"front","average":{"1min":1.8029999999999999,"5min":19.068000000000001,"15min":21.946000000000002},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":2.5699999999999998,"5min":164.53,"15min":164.53},"last":1.375}]},{"osd":14,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":22.312000000000001,"5min":21.788,"15min":21.701000000000001},"min":{"1min":1.556,"5min":0.96099999999999997,"15min":0.96099999999999997},"max":{"1min":199.43000000000001,"5min":202.191,"15min":202.191},"last":1.782},{"interface":"front","average":{"1min":9.8800000000000008,"5min":17.420999999999999,"15min":18.678000000000001},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":101.492,"5min":164.35400000000001,"15min":164.35400000000001},"last":1.524}]}]},{"osd":9,"up_from":21,"seq":90194313248,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303784,"kb_used_data":2264,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068056,"statfs":{"total":96636764160,"available":96325689344,"internally_reserved":0,"allocated":2318336,"data_stored":1350579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,4,5,6,8,10,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":7.2450000000000001,"5min":7.2450000000000001,"15min":7.2450000000000001},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":127.902,"5min":127.902,"15min":127.902},"last":0.97399999999999998},{"interface":"front","average":{"1min":8.3109999999999999,"5min":8.3109999999999999,"15min":8.3109999999999999},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":96.203000000000003,"5min":96.203000000000003,"15min":96.203000000000003},"last":0.81200000000000006}]},{"osd":1,"last update":"Wed Sep 18 10:16:40 2024","interfaces":[{"interface":"back","average":{"1min":10.587,"5min":10.587,"15min":10.587},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":166.833,"5min":166.833,"15min":166.833},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.96099999999999997}]},{"osd":2,"last update":"Wed Sep 18 10:16:18 2024","interfaces":[{"interface":"back","average":{"1min":5.6420000000000003,"5min":5.6420000000000003,"15min":5.6420000000000003},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":68.049999999999997,"5min":68.049999999999997,"15min":68.049999999999997},"last":1.1579999999999999},{"interface":"front","average":{"1min":2.0310000000000001,"5min":2.0310000000000001,"15min":2.0310000000000001},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":18.553999999999998,"5min":18.553999999999998,"15min":18.553999999999998},"last":1.4119999999999999}]},{"osd":3,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":2.875,"5min":5.5549999999999997,"15min":6.0019999999999998},"min":{"1min":0.72699999999999998,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":18.431000000000001,"5min":98.888999999999996,"15min":98.888999999999996},"last":1.0389999999999999},{"interface":"front","average":{"1min":3.0169999999999999,"5min":5.5890000000000004,"15min":6.0170000000000003},"min":{"1min":0.749,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":18.472000000000001,"5min":98.947999999999993,"15min":98.947999999999993},"last":1.546}]},{"osd":4,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":8.2360000000000007,"5min":2.5840000000000001,"15min":1.6419999999999999},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":124.15300000000001,"5min":124.15300000000001,"15min":124.15300000000001},"last":1.371},{"interface":"front","average":{"1min":6.2380000000000004,"5min":8.8539999999999992,"15min":9.2899999999999991},"min":{"1min":0.81000000000000005,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":73.992000000000004,"5min":73.992000000000004,"15min":73.992000000000004},"last":1.6319999999999999}]},{"osd":5,"last update":"Wed Sep 18 10:16:18 2024","interfaces":[{"interface":"back","average":{"1min":1.821,"5min":1.821,"15min":1.821},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":14.786,"5min":14.786,"15min":14.786},"last":0.79800000000000004},{"interface":"front","average":{"1min":8.5,"5min":8.5,"15min":8.5},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":67.915999999999997,"5min":67.915999999999997,"15min":67.915999999999997},"last":1.27}]},{"osd":6,"last update":"Wed Sep 18 10:16:18 2024","interfaces":[{"interface":"back","average":{"1min":8.3840000000000003,"5min":8.3840000000000003,"15min":8.3840000000000003},"min":{"1min":0.997,"5min":0.997,"15min":0.997},"max":{"1min":70.914000000000001,"5min":70.914000000000001,"15min":70.914000000000001},"last":1.806},{"interface":"front","average":{"1min":15.705,"5min":15.705,"15min":15.705},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":136.38800000000001,"5min":136.38800000000001,"15min":136.38800000000001},"last":1.2}]},{"osd":8,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":3.0310000000000001,"5min":4.2789999999999999,"15min":4.4870000000000001},"min":{"1min":1.0509999999999999,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":30.713000000000001,"5min":64.912999999999997,"15min":64.912999999999997},"last":1.458},{"interface":"front","average":{"1min":2.0569999999999999,"5min":5.3959999999999999,"15min":5.952},"min":{"1min":0.82499999999999996,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":12.898999999999999,"5min":57.664000000000001,"15min":57.664000000000001},"last":1.857}]},{"osd":10,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":1.3400000000000001,"5min":1.3089999999999999,"15min":1.304},"min":{"1min":0.88300000000000001,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.8180000000000001,"5min":2.109,"15min":2.109},"last":1.51},{"interface":"front","average":{"1min":6.0389999999999997,"5min":4.2309999999999999,"15min":3.9300000000000002},"min":{"1min":0.97899999999999998,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":68.216999999999999,"5min":68.216999999999999,"15min":68.216999999999999},"last":1.661}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.669}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.446}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":96}]},{"osd":14,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":4.3579999999999997,"5min":8.8759999999999994,"15min":9.6289999999999996},"min":{"1min":0.86499999999999999,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":50.773000000000003,"5min":62.899000000000001,"15min":62.899000000000001},"last":1.9039999999999999},{"interface":"front","average":{"1min":4.3540000000000001,"5min":13.071,"15min":14.523999999999999},"min":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":50.709000000000003,"5min":155.61000000000001,"15min":155.61000000000001},"last":1.6950000000000001}]},{"osd":15,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":1.4630000000000001,"5min":1.492,"15min":1.4970000000000001},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.97,"5min":1.9970000000000001,"15min":1.9970000000000001},"last":1.605},{"interface":"front","average":{"1min":9.5700000000000003,"5min":5.1749999999999998,"15min":4.4429999999999996},"min":{"1min":1.0980000000000001,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":68.140000000000001,"5min":68.140000000000001,"15min":68.140000000000001},"last":1.7470000000000001}]}]},{"osd":11,"up_from":21,"seq":90194313248,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303784,"kb_used_data":2264,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068056,"statfs":{"total":96636764160,"available":96325689344,"internally_reserved":0,"allocated":2318336,"data_stored":1350579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[2,3,6,8,9,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0049999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86599999999999999}]},{"osd":2,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":8.3390000000000004,"15min":9.5719999999999992},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.3740000000000001,"5min":99.718000000000004,"15min":99.718000000000004},"last":0.88200000000000001},{"interface":"front","average":{"1min":3.6349999999999998,"5min":1.4450000000000001,"15min":1.0800000000000001},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":49.774000000000001,"5min":49.774000000000001,"15min":49.774000000000001},"last":0.93400000000000005}]},{"osd":3,"last update":"Wed Sep 18 10:16:46 2024","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":4.2210000000000001,"15min":4.7880000000000003},"min":{"1min":0.48399999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.369,"5min":85.963999999999999,"15min":85.963999999999999},"last":0.82299999999999995},{"interface":"front","average":{"1min":5.4660000000000002,"5min":19.300000000000001,"15min":21.606000000000002},"min":{"1min":0.73699999999999999,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":80.980999999999995,"5min":125.672,"15min":125.672},"last":1.073}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.444}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5980000000000001}]},{"osd":6,"last update":"Wed Sep 18 10:16:46 2024","interfaces":[{"interface":"back","average":{"1min":5.8810000000000002,"5min":2.101,"15min":1.4710000000000001},"min":{"1min":0.70599999999999996,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":86.680000000000007,"5min":86.680000000000007,"15min":86.680000000000007},"last":1.21},{"interface":"front","average":{"1min":1.1830000000000001,"5min":5.827,"15min":6.601},"min":{"1min":0.73799999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.49,"5min":94.162000000000006,"15min":94.162000000000006},"last":1.109}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.524}]},{"osd":8,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":2.7069999999999999,"5min":5.3730000000000002,"15min":5.8170000000000002},"min":{"1min":0.95299999999999996,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":27.577999999999999,"5min":85.900000000000006,"15min":85.900000000000006},"last":1.0189999999999999},{"interface":"front","average":{"1min":2.544,"5min":8.4960000000000004,"15min":9.4879999999999995},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":27.542000000000002,"5min":99.765000000000001,"15min":99.765000000000001},"last":1.1559999999999999}]},{"osd":9,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":5.8760000000000003,"5min":5.8339999999999996,"15min":5.827},"min":{"1min":0.96399999999999997,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":89.844999999999999,"5min":89.844999999999999,"15min":89.844999999999999},"last":1.1779999999999999},{"interface":"front","average":{"1min":1.351,"5min":6.8070000000000004,"15min":7.7160000000000002},"min":{"1min":0.68600000000000005,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.742,"5min":99.810000000000002,"15min":99.810000000000002},"last":1.2470000000000001}]},{"osd":10,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":4.0069999999999997,"5min":1.679,"15min":1.2909999999999999},"min":{"1min":0.98499999999999999,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":49.819000000000003,"5min":49.819000000000003,"15min":49.819000000000003},"last":1.361},{"interface":"front","average":{"1min":1.5580000000000001,"5min":6.46,"15min":7.2770000000000001},"min":{"1min":0.98699999999999999,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.976,"5min":89.290000000000006,"15min":89.290000000000006},"last":1.274}]},{"osd":12,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":5.907,"5min":2.081,"15min":1.4430000000000001},"min":{"1min":0.96599999999999997,"5min":0.498,"15min":0.498},"max":{"1min":87.463999999999999,"5min":87.463999999999999,"15min":87.463999999999999},"last":1.7270000000000001},{"interface":"front","average":{"1min":1.5660000000000001,"5min":8.6219999999999999,"15min":9.798},"min":{"1min":1.0700000000000001,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":2.0600000000000001,"5min":86.025999999999996,"15min":86.025999999999996},"last":1.494}]},{"osd":13,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":4.4210000000000003,"5min":25.207999999999998,"15min":28.672000000000001},"min":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"max":{"1min":59.841999999999999,"5min":189.25899999999999,"15min":189.25899999999999},"last":1.476},{"interface":"front","average":{"1min":7.569,"5min":2.5659999999999998,"15min":1.732},"min":{"1min":0.88900000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":71.313000000000002,"5min":71.313000000000002,"15min":71.313000000000002},"last":1.6439999999999999}]},{"osd":14,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":14.388999999999999,"5min":8.016,"15min":6.9539999999999997},"min":{"1min":1.0129999999999999,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":256.13400000000001,"5min":256.13400000000001,"15min":256.13400000000001},"last":1.45},{"interface":"front","average":{"1min":2.7650000000000001,"5min":7.7380000000000004,"15min":8.5670000000000002},"min":{"1min":0.874,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":25.791,"5min":102.625,"15min":102.625},"last":1.4330000000000001}]},{"osd":15,"last update":"Wed Sep 18 10:16:40 2024","interfaces":[{"interface":"back","average":{"1min":4.0309999999999997,"5min":4.0309999999999997,"15min":4.0309999999999997},"min":{"1min":1.153,"5min":1.153,"15min":1.153},"max":{"1min":49.927999999999997,"5min":49.927999999999997,"15min":49.927999999999997},"last":1.518},{"interface":"front","average":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"min":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"max":{"1min":2.044,"5min":2.044,"15min":2.044},"last":1.407}]}]},{"osd":12,"up_from":21,"seq":90194313249,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303784,"kb_used_data":2268,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068056,"statfs":{"total":96636764160,"available":96325689344,"internally_reserved":0,"allocated":2322432,"data_stored":1350598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[3,4,6,7,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":2.3839999999999999,"5min":2.3839999999999999,"15min":2.3839999999999999},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":32.476999999999997,"5min":32.476999999999997,"15min":32.476999999999997},"last":0.64200000000000002},{"interface":"front","average":{"1min":2.625,"5min":2.625,"15min":2.625},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":32.509,"5min":32.509,"15min":32.509},"last":0.89300000000000002}]},{"osd":1,"last update":"Wed Sep 18 10:15:32 2024","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.6399999999999999,"5min":1.6399999999999999,"15min":1.6399999999999999},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.905,"5min":1.905,"15min":1.905},"last":0.81899999999999995}]},{"osd":2,"last update":"Wed Sep 18 10:15:32 2024","interfaces":[{"interface":"back","average":{"1min":1.002,"5min":1.002,"15min":1.002},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":1.153},{"interface":"front","average":{"1min":4.1189999999999998,"5min":4.1189999999999998,"15min":4.1189999999999998},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":62.780999999999999,"5min":62.780999999999999,"15min":62.780999999999999},"last":0.96499999999999997}]},{"osd":3,"last update":"Wed Sep 18 10:16:33 2024","interfaces":[{"interface":"back","average":{"1min":5.9429999999999996,"5min":2.1040000000000001,"15min":1.464},"min":{"1min":0.627,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":74.822999999999993,"5min":74.822999999999993,"15min":74.822999999999993},"last":1.1399999999999999},{"interface":"front","average":{"1min":8.8200000000000003,"5min":2.6840000000000002,"15min":1.6619999999999999},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":94.320999999999998,"5min":94.320999999999998,"15min":94.320999999999998},"last":0.87}]},{"osd":4,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":16.058,"5min":4.0529999999999999,"15min":2.052},"min":{"1min":0.53100000000000003,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":114.184,"5min":114.184,"15min":114.184},"last":0.82199999999999995},{"interface":"front","average":{"1min":5.8920000000000003,"5min":2.1560000000000001,"15min":1.5329999999999999},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":74.698999999999998,"5min":74.698999999999998,"15min":74.698999999999998},"last":0.94299999999999995}]},{"osd":6,"last update":"Wed Sep 18 10:16:38 2024","interfaces":[{"interface":"back","average":{"1min":10.747999999999999,"5min":10.747999999999999,"15min":10.747999999999999},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":130.20699999999999,"5min":130.20699999999999,"15min":130.20699999999999},"last":1.0660000000000001},{"interface":"front","average":{"1min":14.148,"5min":14.148,"15min":14.148},"min":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":132.60900000000001,"5min":132.60900000000001,"15min":132.60900000000001},"last":1.5329999999999999}]},{"osd":7,"last update":"Wed Sep 18 10:16:38 2024","interfaces":[{"interface":"back","average":{"1min":4.774,"5min":4.774,"15min":4.774},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":90.188000000000002,"5min":90.188000000000002,"15min":90.188000000000002},"last":1.208},{"interface":"front","average":{"1min":6.0359999999999996,"5min":6.0359999999999996,"15min":6.0359999999999996},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":75.061999999999998,"5min":75.061999999999998,"15min":75.061999999999998},"last":1.3089999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":94.382000000000005}]},{"osd":9,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":8.9719999999999995,"5min":2.8079999999999998,"15min":1.7809999999999999},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":94.421000000000006,"5min":94.421000000000006,"15min":94.421000000000006},"last":1.3799999999999999},{"interface":"front","average":{"1min":5.0410000000000004,"5min":2.0179999999999998,"15min":1.514},"min":{"1min":0.70199999999999996,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":90.242999999999995,"5min":90.242999999999995,"15min":90.242999999999995},"last":1.47}]},{"osd":10,"last update":"Wed Sep 18 10:16:38 2024","interfaces":[{"interface":"back","average":{"1min":15.145,"5min":15.145,"15min":15.145},"min":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":99.251000000000005,"5min":99.251000000000005,"15min":99.251000000000005},"last":1.4359999999999999},{"interface":"front","average":{"1min":8.7080000000000002,"5min":8.7080000000000002,"15min":8.7080000000000002},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":94.468000000000004,"5min":94.468000000000004,"15min":94.468000000000004},"last":1.5700000000000001}]},{"osd":11,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":8.9269999999999996,"5min":2.8769999999999998,"15min":1.869},"min":{"1min":0.91300000000000003,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":74.888999999999996,"5min":74.888999999999996,"15min":74.888999999999996},"last":1.8899999999999999},{"interface":"front","average":{"1min":9.1590000000000007,"5min":2.9380000000000002,"15min":1.901},"min":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"max":{"1min":94.546000000000006,"5min":94.546000000000006,"15min":94.546000000000006},"last":1.798}]},{"osd":13,"last update":"Wed Sep 18 10:16:33 2024","interfaces":[{"interface":"back","average":{"1min":7.4779999999999998,"5min":2.5619999999999998,"15min":1.742},"min":{"1min":0.79900000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":143.672,"5min":143.672,"15min":143.672},"last":1.671},{"interface":"front","average":{"1min":12.677,"5min":3.6779999999999999,"15min":2.1779999999999999},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":143.619,"5min":143.619,"15min":143.619},"last":1.744}]},{"osd":14,"last update":"Wed Sep 18 10:16:33 2024","interfaces":[{"interface":"back","average":{"1min":13.363,"5min":3.8159999999999998,"15min":2.2250000000000001},"min":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"max":{"1min":94.504999999999995,"5min":94.504999999999995,"15min":94.504999999999995},"last":1.6879999999999999},{"interface":"front","average":{"1min":12.039,"5min":3.52,"15min":2.1000000000000001},"min":{"1min":0.95099999999999996,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":92.593000000000004,"5min":92.593000000000004,"15min":92.593000000000004},"last":1.615}]},{"osd":15,"last update":"Wed Sep 18 10:16:33 2024","interfaces":[{"interface":"back","average":{"1min":6.5339999999999998,"5min":2.5019999999999998,"15min":1.8300000000000001},"min":{"1min":0.751,"5min":0.751,"15min":0.751},"max":{"1min":74.966999999999999,"5min":74.966999999999999,"15min":74.966999999999999},"last":2.028},{"interface":"front","average":{"1min":14.669,"5min":4.117,"15min":2.359},"min":{"1min":0.996,"5min":0.95299999999999996,"15min":0.95299999999999996},"max":{"1min":152.11000000000001,"5min":152.11000000000001,"15min":152.11000000000001},"last":1.9219999999999999}]}]},{"osd":1,"up_from":22,"seq":94489280543,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":303748,"kb_used_data":2232,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068092,"statfs":{"total":96636764160,"available":96325726208,"internally_reserved":0,"allocated":2285568,"data_stored":1331589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,6,9,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:16:38 2024","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":2.472,"15min":2.7370000000000001},"min":{"1min":0.65800000000000003,"5min":0.629,"15min":0.629},"max":{"1min":1.1930000000000001,"5min":36.268000000000001,"15min":36.268000000000001},"last":1.127},{"interface":"front","average":{"1min":5.3659999999999997,"5min":15.244999999999999,"15min":16.890999999999998},"min":{"1min":0.73099999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":66.534999999999997,"5min":154.67599999999999,"15min":154.67599999999999},"last":0.70699999999999996}]},{"osd":2,"last update":"Wed Sep 18 10:16:38 2024","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.91400000000000003,"15min":0.90400000000000003},"min":{"1min":0.71499999999999997,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.4119999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.98999999999999999,"5min":2.5539999999999998,"15min":2.8140000000000001},"min":{"1min":0.75600000000000001,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.212,"5min":36.323999999999998,"15min":36.323999999999998},"last":0.76400000000000001}]},{"osd":3,"last update":"Wed Sep 18 10:16:38 2024","interfaces":[{"interface":"back","average":{"1min":5.548,"5min":10.811999999999999,"15min":11.689},"min":{"1min":0.94599999999999995,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":66.344999999999999,"5min":150.642,"15min":150.642},"last":0.96799999999999997},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.0649999999999999,"15min":1.0509999999999999},"min":{"1min":0.82999999999999996,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.6830000000000001,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":1.081}]},{"osd":4,"last update":"Wed Sep 18 10:15:35 2024","interfaces":[{"interface":"back","average":{"1min":3.3439999999999999,"5min":3.3439999999999999,"15min":3.3439999999999999},"min":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":40.783000000000001,"5min":40.783000000000001,"15min":40.783000000000001},"last":1.7210000000000001},{"interface":"front","average":{"1min":4.1699999999999999,"5min":4.1699999999999999,"15min":4.1699999999999999},"min":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":50.747,"5min":50.747,"15min":50.747},"last":1.518}]},{"osd":5,"last update":"Wed Sep 18 10:15:35 2024","interfaces":[{"interface":"back","average":{"1min":5.9720000000000004,"5min":5.9720000000000004,"15min":5.9720000000000004},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":52.317999999999998,"5min":52.317999999999998,"15min":52.317999999999998},"last":1.2649999999999999},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.748,"5min":1.748,"15min":1.748},"last":1.839}]},{"osd":6,"last update":"Wed Sep 18 10:16:38 2024","interfaces":[{"interface":"back","average":{"1min":9.2970000000000006,"5min":8.5380000000000003,"15min":8.4120000000000008},"min":{"1min":0.85099999999999998,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":120.85299999999999,"5min":120.85299999999999,"15min":120.85299999999999},"last":1.1779999999999999},{"interface":"front","average":{"1min":2.6040000000000001,"5min":8.8019999999999996,"15min":9.8350000000000009},"min":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"max":{"1min":20.600000000000001,"5min":85.045000000000002,"15min":85.045000000000002},"last":1.266}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":118.43600000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7789999999999999}]},{"osd":9,"last update":"Wed Sep 18 10:16:15 2024","interfaces":[{"interface":"back","average":{"1min":3.4369999999999998,"5min":3.4369999999999998,"15min":3.4369999999999998},"min":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"max":{"1min":36.506999999999998,"5min":36.506999999999998,"15min":36.506999999999998},"last":0.998},{"interface":"front","average":{"1min":11.339,"5min":11.339,"15min":11.339},"min":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"max":{"1min":165.13499999999999,"5min":165.13499999999999,"15min":165.13499999999999},"last":1.0429999999999999}]},{"osd":10,"last update":"Wed Sep 18 10:16:38 2024","interfaces":[{"interface":"back","average":{"1min":1.5209999999999999,"5min":3.048,"15min":3.302},"min":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"max":{"1min":2.0489999999999999,"5min":36.436,"15min":36.436},"last":1.3799999999999999},{"interface":"front","average":{"1min":6.0839999999999996,"5min":10.747999999999999,"15min":11.526},"min":{"1min":1.2769999999999999,"5min":0.88,"15min":0.88},"max":{"1min":66.673000000000002,"5min":100.402,"15min":100.402},"last":1.155}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.825}]},{"osd":12,"last update":"Wed Sep 18 10:16:38 2024","interfaces":[{"interface":"back","average":{"1min":1.5549999999999999,"5min":3.681,"15min":4.0350000000000001},"min":{"1min":1.101,"5min":1.0589999999999999,"15min":1.0589999999999999},"max":{"1min":2.294,"5min":49.976999999999997,"15min":49.976999999999997},"last":1.323},{"interface":"front","average":{"1min":2.0369999999999999,"5min":7.1269999999999998,"15min":7.9749999999999996},"min":{"1min":1.196,"5min":1.196,"15min":1.196},"max":{"1min":8.4049999999999994,"5min":124.78100000000001,"15min":124.78100000000001},"last":1.4419999999999999}]},{"osd":13,"last update":"Wed Sep 18 10:16:15 2024","interfaces":[{"interface":"back","average":{"1min":11.616,"5min":11.616,"15min":11.616},"min":{"1min":1.232,"5min":1.232,"15min":1.232},"max":{"1min":104.94199999999999,"5min":104.94199999999999,"15min":104.94199999999999},"last":1.2909999999999999},{"interface":"front","average":{"1min":1.5720000000000001,"5min":1.5720000000000001,"15min":1.5720000000000001},"min":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"max":{"1min":2.3809999999999998,"5min":2.3809999999999998,"15min":2.3809999999999998},"last":1.4099999999999999}]},{"osd":14,"last update":"Wed Sep 18 10:16:15 2024","interfaces":[{"interface":"back","average":{"1min":5.5910000000000002,"5min":5.5910000000000002,"15min":5.5910000000000002},"min":{"1min":1.276,"5min":1.276,"15min":1.276},"max":{"1min":66.825999999999993,"5min":66.825999999999993,"15min":66.825999999999993},"last":1.3580000000000001},{"interface":"front","average":{"1min":2.8359999999999999,"5min":2.8359999999999999,"15min":2.8359999999999999},"min":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"max":{"1min":22.16,"5min":22.16,"15min":22.16},"last":1.508}]},{"osd":15,"last update":"Wed Sep 18 10:16:15 2024","interfaces":[{"interface":"back","average":{"1min":3.677,"5min":3.677,"15min":3.677},"min":{"1min":1.165,"5min":1.165,"15min":1.165},"max":{"1min":36.575000000000003,"5min":36.575000000000003,"15min":36.575000000000003},"last":1.54},{"interface":"front","average":{"1min":5.6269999999999998,"5min":5.6269999999999998,"15min":5.6269999999999998},"min":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"max":{"1min":66.763999999999996,"5min":66.763999999999996,"15min":66.763999999999996},"last":1.468}]}]},{"osd":2,"up_from":22,"seq":94489280543,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":303764,"kb_used_data":2248,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068076,"statfs":{"total":96636764160,"available":96325709824,"internally_reserved":0,"allocated":2301952,"data_stored":1341063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,3,4,5,6,7,8,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":21.344000000000001,"15min":24.777999999999999},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.3879999999999999,"5min":419.05700000000002,"15min":419.05700000000002},"last":0.84899999999999998},{"interface":"front","average":{"1min":2.3860000000000001,"5min":4.4020000000000001,"15min":4.7380000000000004},"min":{"1min":0.57299999999999995,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":35.899999999999999,"5min":98.254000000000005,"15min":98.254000000000005},"last":0.71699999999999997}]},{"osd":3,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":5.1219999999999999,"5min":1.7529999999999999,"15min":1.1910000000000001},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":86.677000000000007,"5min":86.677000000000007,"15min":86.677000000000007},"last":0.66900000000000004},{"interface":"front","average":{"1min":3.6160000000000001,"5min":1.47,"15min":1.1120000000000001},"min":{"1min":0.68000000000000005,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":59.759,"5min":59.759,"15min":59.759},"last":0.76300000000000001}]},{"osd":4,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":7.992,"5min":3.1379999999999999,"15min":2.3290000000000002},"min":{"1min":0.55600000000000005,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":121.20099999999999,"5min":121.20099999999999,"15min":121.20099999999999},"last":0.94499999999999995},{"interface":"front","average":{"1min":6.5250000000000004,"5min":4.9359999999999999,"15min":4.6710000000000003},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":86.548000000000002,"5min":86.548000000000002,"15min":86.548000000000002},"last":0.88300000000000001}]},{"osd":5,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":12.161,"5min":3.2879999999999998,"15min":1.8089999999999999},"min":{"1min":0.72099999999999997,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":163.828,"5min":163.828,"15min":163.828},"last":0.92600000000000005},{"interface":"front","average":{"1min":2.6160000000000001,"5min":1.3660000000000001,"15min":1.1579999999999999},"min":{"1min":0.79200000000000004,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":36.119,"5min":36.119,"15min":36.119},"last":1.02}]},{"osd":6,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":5.2359999999999998,"5min":4.8529999999999998,"15min":4.7889999999999997},"min":{"1min":0.49199999999999999,"5min":0.441,"15min":0.441},"max":{"1min":86.766000000000005,"5min":86.766000000000005,"15min":86.766000000000005},"last":0.82399999999999995},{"interface":"front","average":{"1min":5.3959999999999999,"5min":4.9770000000000003,"15min":4.907},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":83.171999999999997,"5min":90.188999999999993,"15min":90.188999999999993},"last":1.038}]},{"osd":7,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":13.191000000000001,"5min":3.6019999999999999,"15min":2.004},"min":{"1min":0.84999999999999998,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":88.534000000000006,"5min":88.534000000000006,"15min":88.534000000000006},"last":1.079},{"interface":"front","average":{"1min":15.949999999999999,"5min":4.2350000000000003,"15min":2.2829999999999999},"min":{"1min":0.75600000000000001,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":88.484999999999999,"5min":88.484999999999999,"15min":88.484999999999999},"last":0.97899999999999998}]},{"osd":8,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":4.3179999999999996,"15min":4.8399999999999999},"min":{"1min":0.82899999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":2.0699999999999998,"5min":92.578999999999994,"15min":92.578999999999994},"last":1.1000000000000001},{"interface":"front","average":{"1min":2.8460000000000001,"5min":2.7799999999999998,"15min":2.7690000000000001},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":36.088000000000001,"5min":36.088000000000001,"15min":36.088000000000001},"last":1.1459999999999999}]},{"osd":12,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":2.9220000000000002,"5min":1.6719999999999999,"15min":1.464},"min":{"1min":0.872,"5min":0.872,"15min":0.872},"max":{"1min":36.014000000000003,"5min":36.014000000000003,"15min":36.014000000000003},"last":1.2290000000000001},{"interface":"front","average":{"1min":5.5140000000000002,"5min":2.7469999999999999,"15min":2.286},"min":{"1min":0.73199999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":86.832999999999998,"5min":86.832999999999998,"15min":86.832999999999998},"last":1.0589999999999999}]},{"osd":13,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":11.115,"5min":8.3970000000000002,"15min":7.944},"min":{"1min":0.97999999999999998,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":80.683000000000007,"5min":141.84200000000001,"15min":141.84200000000001},"last":1.3600000000000001},{"interface":"front","average":{"1min":2.911,"5min":4.9489999999999998,"15min":5.2889999999999997},"min":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":35.841000000000001,"5min":97.441999999999993,"15min":97.441999999999993},"last":1.2849999999999999}]},{"osd":15,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":12.472,"5min":3.6200000000000001,"15min":2.145},"min":{"1min":0.68400000000000005,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":86.980000000000004,"5min":86.980000000000004,"15min":86.980000000000004},"last":1.129},{"interface":"front","average":{"1min":3.3479999999999999,"5min":3.798,"15min":3.8730000000000002},"min":{"1min":0.96999999999999997,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":45.356999999999999,"5min":61.694000000000003,"15min":61.694000000000003},"last":1.304}]}]},{"osd":3,"up_from":22,"seq":94489280543,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303748,"kb_used_data":2232,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068092,"statfs":{"total":96636764160,"available":96325726208,"internally_reserved":0,"allocated":2285568,"data_stored":1331589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,4,6,7,9,10,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:16:37 2024","interfaces":[{"interface":"back","average":{"1min":4.173,"5min":2.048,"15min":1.694},"min":{"1min":0.64300000000000002,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":65.994,"5min":65.994,"15min":65.994},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.97899999999999998,"5min":8.8260000000000005,"15min":10.134},"min":{"1min":0.75,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.752,"5min":71.316999999999993,"15min":71.316999999999993},"last":0.76800000000000002}]},{"osd":2,"last update":"Wed Sep 18 10:16:37 2024","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":4.4379999999999997,"15min":5.0140000000000002},"min":{"1min":0.76900000000000002,"5min":0.75,"15min":0.75},"max":{"1min":1.333,"5min":56.523000000000003,"15min":56.523000000000003},"last":0.94999999999999996},{"interface":"front","average":{"1min":1.165,"5min":1.1200000000000001,"15min":1.1120000000000001},"min":{"1min":0.67400000000000004,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.639,"5min":3.327,"15min":3.327},"last":1.3300000000000001}]},{"osd":4,"last update":"Wed Sep 18 10:16:37 2024","interfaces":[{"interface":"back","average":{"1min":1.0409999999999999,"5min":5.7069999999999999,"15min":6.4850000000000003},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.6399999999999999,"5min":71.135000000000005,"15min":71.135000000000005},"last":0.91500000000000004},{"interface":"front","average":{"1min":1.635,"5min":3.8130000000000002,"15min":4.1760000000000002},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":10.039,"5min":56.253999999999998,"15min":56.253999999999998},"last":0.72799999999999998}]},{"osd":5,"last update":"Wed Sep 18 10:16:06 2024","interfaces":[{"interface":"back","average":{"1min":13.997,"5min":13.997,"15min":13.997},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":125.42400000000001,"5min":125.42400000000001,"15min":125.42400000000001},"last":1.131},{"interface":"front","average":{"1min":10.323,"5min":10.323,"15min":10.323},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":112.033,"5min":112.033,"15min":112.033},"last":0.96899999999999997}]},{"osd":6,"last update":"Wed Sep 18 10:16:37 2024","interfaces":[{"interface":"back","average":{"1min":1.9279999999999999,"5min":1.29,"15min":1.1839999999999999},"min":{"1min":0.93400000000000005,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":12.83,"5min":12.83,"15min":12.83},"last":1.0600000000000001},{"interface":"front","average":{"1min":5.0190000000000001,"5min":8.0050000000000008,"15min":8.5030000000000001},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":76.634,"5min":81.210999999999999,"15min":81.210999999999999},"last":0.998}]},{"osd":7,"last update":"Wed Sep 18 10:16:37 2024","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":13.159000000000001,"15min":15.141},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.75,"5min":129.68299999999999,"15min":129.68299999999999},"last":1.0249999999999999},{"interface":"front","average":{"1min":1.4710000000000001,"5min":11.119999999999999,"15min":12.728},"min":{"1min":0.90800000000000003,"5min":0.621,"15min":0.621},"max":{"1min":1.8500000000000001,"5min":129.726,"15min":129.726},"last":1.474}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0620000000000001}]},{"osd":9,"last update":"Wed Sep 18 10:16:37 2024","interfaces":[{"interface":"back","average":{"1min":1.5660000000000001,"5min":1.446,"15min":1.4259999999999999},"min":{"1min":0.94599999999999995,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":2.0209999999999999,"5min":2.0209999999999999,"15min":2.0209999999999999},"last":1.377},{"interface":"front","average":{"1min":6.1050000000000004,"5min":6.1079999999999997,"15min":6.1079999999999997},"min":{"1min":0.86099999999999999,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":94.546999999999997,"5min":94.546999999999997,"15min":94.546999999999997},"last":1.2969999999999999}]},{"osd":10,"last update":"Wed Sep 18 10:16:06 2024","interfaces":[{"interface":"back","average":{"1min":1.4870000000000001,"5min":1.4870000000000001,"15min":1.4870000000000001},"min":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":2.0659999999999998,"5min":2.0659999999999998,"15min":2.0659999999999998},"last":1.232},{"interface":"front","average":{"1min":4.9349999999999996,"5min":4.9349999999999996,"15min":4.9349999999999996},"min":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":71.668999999999997,"5min":71.668999999999997,"15min":71.668999999999997},"last":1.173}]},{"osd":13,"last update":"Wed Sep 18 10:16:37 2024","interfaces":[{"interface":"back","average":{"1min":9.4909999999999997,"5min":3.113,"15min":2.0499999999999998},"min":{"1min":1.04,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":91.290999999999997,"5min":91.290999999999997,"15min":91.290999999999997},"last":1.421},{"interface":"front","average":{"1min":1.5249999999999999,"5min":5.3300000000000001,"15min":5.9640000000000004},"min":{"1min":1.05,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.885,"5min":71.447000000000003,"15min":71.447000000000003},"last":1.3560000000000001}]},{"osd":14,"last update":"Wed Sep 18 10:16:37 2024","interfaces":[{"interface":"back","average":{"1min":1.6899999999999999,"5min":4.8899999999999997,"15min":5.423},"min":{"1min":1.0089999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"max":{"1min":4.625,"5min":56.664999999999999,"15min":56.664999999999999},"last":1.149},{"interface":"front","average":{"1min":1.865,"5min":2.2629999999999999,"15min":2.3290000000000002},"min":{"1min":1.1859999999999999,"5min":0.93999999999999995,"15min":0.93999999999999995},"max":{"1min":4.5910000000000002,"5min":18.523,"15min":18.523},"last":1.52}]},{"osd":15,"last update":"Wed Sep 18 10:16:48 2024","interfaces":[{"interface":"back","average":{"1min":1.6970000000000001,"5min":4.0049999999999999,"15min":4.3899999999999997},"min":{"1min":1.01,"5min":0.91500000000000004,"15min":0.91500000000000004},"max":{"1min":4.3289999999999997,"5min":56.357999999999997,"15min":56.357999999999997},"last":1.268},{"interface":"front","average":{"1min":1.7589999999999999,"5min":8.5310000000000006,"15min":9.6600000000000001},"min":{"1min":1.133,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":2.1110000000000002,"5min":122.727,"15min":122.727},"last":1.5620000000000001}]}]},{"osd":5,"up_from":22,"seq":94489280543,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303748,"kb_used_data":2232,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068092,"statfs":{"total":96636764160,"available":96325726208,"internally_reserved":0,"allocated":2285568,"data_stored":1331589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[2,3,4,6,8,9,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:15:47 2024","interfaces":[{"interface":"back","average":{"1min":7.8860000000000001,"5min":7.8860000000000001,"15min":7.8860000000000001},"min":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":84.430999999999997,"5min":84.430999999999997,"15min":84.430999999999997},"last":1.2889999999999999},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":1.353}]},{"osd":2,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":4.1360000000000001,"5min":8.6940000000000008,"15min":9.4540000000000006},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":51.091000000000001,"5min":84.566000000000003,"15min":84.566000000000003},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.98199999999999998,"5min":1.3300000000000001,"15min":1.3879999999999999},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.4690000000000001,"5min":9.5069999999999997,"15min":9.5069999999999997},"last":0.86899999999999999}]},{"osd":3,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":4.9240000000000004,"5min":7.8010000000000002,"15min":8.2810000000000006},"min":{"1min":0.69099999999999995,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":51.229999999999997,"5min":84.358999999999995,"15min":84.358999999999995},"last":0.70499999999999996},{"interface":"front","average":{"1min":8.1590000000000007,"5min":8.2100000000000009,"15min":8.218},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":74.222999999999999,"5min":97.549999999999997,"15min":97.549999999999997},"last":0.76900000000000002}]},{"osd":4,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":25.673999999999999,"15min":29.777999999999999},"min":{"1min":0.67400000000000004,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.407,"5min":266.911,"15min":266.911},"last":1.4059999999999999},{"interface":"front","average":{"1min":21.388000000000002,"5min":22.515999999999998,"15min":22.704000000000001},"min":{"1min":0.76200000000000001,"5min":0.626,"15min":0.626},"max":{"1min":171.691,"5min":260.53500000000003,"15min":260.53500000000003},"last":1.0960000000000001}]},{"osd":6,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":3.6309999999999998,"5min":8.0809999999999995,"15min":8.8230000000000004},"min":{"1min":0.70099999999999996,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":41.886000000000003,"5min":58.171999999999997,"15min":58.171999999999997},"last":0.97599999999999998},{"interface":"front","average":{"1min":5.8079999999999998,"5min":18.274999999999999,"15min":20.353000000000002},"min":{"1min":0.85699999999999998,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":54.984999999999999,"5min":227.48099999999999,"15min":227.48099999999999},"last":1.1759999999999999}]},{"osd":7,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":12.013,"5min":12.013,"15min":12.013},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":123.111,"5min":123.111,"15min":123.111},"last":1.5249999999999999},{"interface":"front","average":{"1min":13.994,"5min":13.994,"15min":13.994},"min":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"max":{"1min":84.617000000000004,"5min":84.617000000000004,"15min":84.617000000000004},"last":1.329}]},{"osd":8,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":4.4720000000000004,"5min":17.175999999999998,"15min":19.292999999999999},"min":{"1min":0.879,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":51.506,"5min":260.44200000000001,"15min":260.44200000000001},"last":1.232},{"interface":"front","average":{"1min":8.2609999999999992,"5min":7.2859999999999996,"15min":7.1239999999999997},"min":{"1min":0.83899999999999997,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":71.266000000000005,"5min":106.818,"15min":106.818},"last":1.038}]},{"osd":9,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":3.8780000000000001,"5min":5.2930000000000001,"15min":5.5289999999999999},"min":{"1min":0.91300000000000003,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":41.652000000000001,"5min":80.272000000000006,"15min":80.272000000000006},"last":1.133},{"interface":"front","average":{"1min":1.3799999999999999,"5min":1.764,"15min":1.8280000000000001},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":2.008,"5min":9.5690000000000008,"15min":9.5690000000000008},"last":1.502}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6479999999999999}]},{"osd":11,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":1.494,"5min":7.8120000000000003,"15min":8.8650000000000002},"min":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":1.825,"5min":123.158,"15min":123.158},"last":1.794},{"interface":"front","average":{"1min":8.8490000000000002,"5min":10.863,"15min":11.199},"min":{"1min":0.93200000000000005,"5min":0.86499999999999999,"15min":0.86499999999999999},"max":{"1min":68.938000000000002,"5min":84.522999999999996,"15min":84.522999999999996},"last":1.446}]},{"osd":12,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":4.0250000000000004,"5min":1.8580000000000001,"15min":1.4970000000000001},"min":{"1min":0.95199999999999996,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":41.929000000000002,"5min":41.929000000000002,"15min":41.929000000000002},"last":1.2849999999999999},{"interface":"front","average":{"1min":1.774,"5min":1.9339999999999999,"15min":1.9610000000000001},"min":{"1min":0.93999999999999995,"5min":0.88300000000000001,"15min":0.88300000000000001},"max":{"1min":4.1150000000000002,"5min":9.7710000000000008,"15min":9.7710000000000008},"last":1.5780000000000001}]},{"osd":13,"last update":"Wed Sep 18 10:16:48 2024","interfaces":[{"interface":"back","average":{"1min":1.6499999999999999,"5min":19.745999999999999,"15min":22.762},"min":{"1min":1.171,"5min":1.1120000000000001,"15min":1.1120000000000001},"max":{"1min":2.0550000000000002,"5min":140.273,"15min":140.273},"last":1.6479999999999999},{"interface":"front","average":{"1min":4.665,"5min":20.309000000000001,"15min":22.917000000000002},"min":{"1min":1.194,"5min":0.98099999999999998,"15min":0.98099999999999998},"max":{"1min":51.432000000000002,"5min":170.86000000000001,"15min":170.86000000000001},"last":1.329}]},{"osd":14,"last update":"Wed Sep 18 10:16:48 2024","interfaces":[{"interface":"back","average":{"1min":9.8930000000000007,"5min":12.584,"15min":13.032},"min":{"1min":1.385,"5min":1.0389999999999999,"15min":1.0389999999999999},"max":{"1min":93.295000000000002,"5min":101.733,"15min":101.733},"last":1.385},{"interface":"front","average":{"1min":1.79,"5min":7.9729999999999999,"15min":9.0030000000000001},"min":{"1min":1.321,"5min":0.90100000000000002,"15min":0.90100000000000002},"max":{"1min":2.2290000000000001,"5min":122.952,"15min":122.952},"last":1.7250000000000001}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8839999999999999}]}]},{"osd":6,"up_from":22,"seq":94489280543,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304280,"kb_used_data":2752,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94067560,"statfs":{"total":96636764160,"available":96325181440,"internally_reserved":0,"allocated":2818048,"data_stored":1860517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,3,4,5,7,8,9,11,12,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:15:47 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.68200000000000005}]},{"osd":1,"last update":"Wed Sep 18 10:16:48 2024","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":3.8100000000000001,"15min":4.306},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.2150000000000001,"5min":52.332999999999998,"15min":52.332999999999998},"last":0.65700000000000003},{"interface":"front","average":{"1min":4.8369999999999997,"5min":1.6990000000000001,"15min":1.1759999999999999},"min":{"1min":0.68300000000000005,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":67.763000000000005,"5min":67.763000000000005,"15min":67.763000000000005},"last":0.71199999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.343}]},{"osd":3,"last update":"Wed Sep 18 10:16:48 2024","interfaces":[{"interface":"back","average":{"1min":5.3380000000000001,"5min":1.792,"15min":1.2010000000000001},"min":{"1min":0.65500000000000003,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":76.891999999999996,"5min":76.891999999999996,"15min":76.891999999999996},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.98999999999999999,"5min":11.403,"15min":13.138999999999999},"min":{"1min":0.751,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.204,"5min":169.244,"15min":169.244},"last":1.042}]},{"osd":4,"last update":"Wed Sep 18 10:16:02 2024","interfaces":[{"interface":"back","average":{"1min":11.111000000000001,"5min":11.111000000000001,"15min":11.111000000000001},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":128.053,"5min":128.053,"15min":128.053},"last":0.92100000000000004},{"interface":"front","average":{"1min":4.6189999999999998,"5min":4.6189999999999998,"15min":4.6189999999999998},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":67.960999999999999,"5min":67.960999999999999,"15min":67.960999999999999},"last":0.86699999999999999}]},{"osd":5,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":6.0579999999999998,"5min":10.194000000000001,"15min":10.884},"min":{"1min":0.72099999999999997,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":68.025999999999996,"5min":119.032,"15min":119.032},"last":0.96799999999999997},{"interface":"front","average":{"1min":5.6760000000000002,"5min":4.0380000000000003,"15min":3.7650000000000001},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":83.858000000000004,"5min":83.858000000000004,"15min":83.858000000000004},"last":1.109}]},{"osd":7,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":4.9539999999999997,"5min":6.3959999999999999,"15min":6.6360000000000001},"min":{"1min":0.94099999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":69.171000000000006,"5min":69.171000000000006,"15min":69.171000000000006},"last":1.3120000000000001},{"interface":"front","average":{"1min":5.0960000000000001,"5min":3.4630000000000001,"15min":3.1909999999999998},"min":{"1min":1.089,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":69.239000000000004,"5min":69.239000000000004,"15min":69.239000000000004},"last":1.29}]},{"osd":8,"last update":"Wed Sep 18 10:16:51 2024","interfaces":[{"interface":"back","average":{"1min":5.3659999999999997,"5min":5.3659999999999997,"15min":5.3659999999999997},"min":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"max":{"1min":76.766000000000005,"5min":76.766000000000005,"15min":76.766000000000005},"last":1.0109999999999999},{"interface":"front","average":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"min":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"max":{"1min":1.704,"5min":1.704,"15min":1.704},"last":1.5029999999999999}]},{"osd":9,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":5.4269999999999996,"5min":7.7089999999999996,"15min":8.0890000000000004},"min":{"1min":0.95799999999999996,"5min":0.84599999999999997,"15min":0.84599999999999997},"max":{"1min":76.997,"5min":95.691999999999993,"15min":95.691999999999993},"last":1.246},{"interface":"front","average":{"1min":1.2949999999999999,"5min":10.472,"15min":12.002000000000001},"min":{"1min":0.96899999999999997,"5min":0.83099999999999996,"15min":0.83099999999999996},"max":{"1min":1.696,"5min":165.31100000000001,"15min":165.31100000000001},"last":1.4219999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.329}]},{"osd":11,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":5.0510000000000002,"5min":2.0630000000000002,"15min":1.5649999999999999},"min":{"1min":1.002,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":67.909999999999997,"5min":67.909999999999997,"15min":67.909999999999997},"last":1.1599999999999999},{"interface":"front","average":{"1min":9.1829999999999998,"5min":4.9429999999999996,"15min":4.2359999999999998},"min":{"1min":1.1870000000000001,"5min":0.92000000000000004,"15min":0.92000000000000004},"max":{"1min":77.158000000000001,"5min":77.158000000000001,"15min":77.158000000000001},"last":1.351}]},{"osd":12,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":1.3859999999999999,"5min":4.3140000000000001,"15min":4.8019999999999996},"min":{"1min":1.079,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":1.796,"5min":52.378,"15min":52.378},"last":1.204},{"interface":"front","average":{"1min":5.1109999999999998,"5min":4.5720000000000001,"15min":4.4820000000000002},"min":{"1min":0.92800000000000005,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":67.835999999999999,"5min":67.835999999999999,"15min":67.835999999999999},"last":1.3779999999999999}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2050000000000001}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2669999999999999}]},{"osd":15,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":5.1680000000000001,"5min":2.2120000000000002,"15min":1.72},"min":{"1min":1.1000000000000001,"5min":0.90300000000000002,"15min":0.90300000000000002},"max":{"1min":68.069999999999993,"5min":68.069999999999993,"15min":68.069999999999993},"last":1.468},{"interface":"front","average":{"1min":5.6200000000000001,"5min":4.3019999999999996,"15min":4.0819999999999999},"min":{"1min":1.2030000000000001,"5min":1.0429999999999999,"15min":1.0429999999999999},"max":{"1min":77.081999999999994,"5min":77.081999999999994,"15min":77.081999999999994},"last":1.452}]}]},{"osd":7,"up_from":22,"seq":94489280543,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303748,"kb_used_data":2232,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068092,"statfs":{"total":96636764160,"available":96325726208,"internally_reserved":0,"allocated":2285568,"data_stored":1331589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,4,6,8,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:16:33 2024","interfaces":[{"interface":"back","average":{"1min":3.9900000000000002,"5min":10.974,"15min":12.138},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":59.137999999999998,"5min":139.58600000000001,"15min":139.58600000000001},"last":0.75600000000000001},{"interface":"front","average":{"1min":1.121,"5min":1.0229999999999999,"15min":1.0069999999999999},"min":{"1min":0.67800000000000005,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.7390000000000001,"5min":1.7390000000000001,"15min":1.7390000000000001},"last":0.59899999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2090000000000001}]},{"osd":2,"last update":"Wed Sep 18 10:15:46 2024","interfaces":[{"interface":"back","average":{"1min":15.436,"5min":15.436,"15min":15.436},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":139.36799999999999,"5min":139.36799999999999,"15min":139.36799999999999},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.94299999999999995}]},{"osd":3,"last update":"Wed Sep 18 10:15:46 2024","interfaces":[{"interface":"back","average":{"1min":17.327999999999999,"5min":17.327999999999999,"15min":17.327999999999999},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":139.27199999999999,"5min":139.27199999999999,"15min":139.27199999999999},"last":1.3460000000000001},{"interface":"front","average":{"1min":3.964,"5min":3.964,"15min":3.964},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":23.748999999999999,"5min":23.748999999999999,"15min":23.748999999999999},"last":0.82899999999999996}]},{"osd":4,"last update":"Wed Sep 18 10:16:37 2024","interfaces":[{"interface":"back","average":{"1min":9.8670000000000009,"5min":17.291,"15min":18.527999999999999},"min":{"1min":0.68000000000000005,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":125.10299999999999,"5min":139.51599999999999,"15min":139.51599999999999},"last":0.55200000000000005},{"interface":"front","average":{"1min":3.3559999999999999,"5min":4.6699999999999999,"15min":4.8890000000000002},"min":{"1min":0.68000000000000005,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":49.241999999999997,"5min":49.241999999999997,"15min":49.241999999999997},"last":0.70199999999999996}]},{"osd":5,"last update":"Wed Sep 18 10:15:46 2024","interfaces":[{"interface":"back","average":{"1min":1.099,"5min":1.099,"15min":1.099},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":2.3620000000000001,"5min":2.3620000000000001,"15min":2.3620000000000001},"last":1.4059999999999999},{"interface":"front","average":{"1min":17.736999999999998,"5min":17.736999999999998,"15min":17.736999999999998},"min":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"max":{"1min":139.44300000000001,"5min":139.44300000000001,"15min":139.44300000000001},"last":0.78000000000000003}]},{"osd":6,"last update":"Wed Sep 18 10:16:33 2024","interfaces":[{"interface":"back","average":{"1min":6.0869999999999997,"5min":15.117000000000001,"15min":16.622},"min":{"1min":0.998,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":59.343000000000004,"5min":183.001,"15min":183.001},"last":1.024},{"interface":"front","average":{"1min":7.0830000000000002,"5min":4.8650000000000002,"15min":4.4950000000000001},"min":{"1min":0.82399999999999995,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":58.284999999999997,"5min":58.284999999999997,"15min":58.284999999999997},"last":0.98499999999999999}]},{"osd":8,"last update":"Wed Sep 18 10:16:30 2024","interfaces":[{"interface":"back","average":{"1min":1.4079999999999999,"5min":1.2809999999999999,"15min":1.26},"min":{"1min":0.84999999999999998,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.9339999999999999,"5min":2.6139999999999999,"15min":2.6139999999999999},"last":0.85699999999999998},{"interface":"front","average":{"1min":5.3029999999999999,"5min":13.785,"15min":15.199},"min":{"1min":1.0980000000000001,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":59.523000000000003,"5min":139.798,"15min":139.798},"last":0.89700000000000002}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.861}]},{"osd":10,"last update":"Wed Sep 18 10:16:30 2024","interfaces":[{"interface":"back","average":{"1min":16.035,"5min":8.4100000000000001,"15min":7.1390000000000002},"min":{"1min":1.0149999999999999,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":156.45500000000001,"5min":156.45500000000001,"15min":156.45500000000001},"last":1.2070000000000001},{"interface":"front","average":{"1min":1.4910000000000001,"5min":1.391,"15min":1.3740000000000001},"min":{"1min":0.83499999999999996,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":2.1200000000000001,"5min":2.254,"15min":2.254},"last":1.0600000000000001}]},{"osd":11,"last update":"Wed Sep 18 10:16:33 2024","interfaces":[{"interface":"back","average":{"1min":5.0460000000000003,"5min":5.4459999999999997,"15min":5.5129999999999999},"min":{"1min":1.046,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":49.154000000000003,"5min":62.356000000000002,"15min":62.356000000000002},"last":1.48},{"interface":"front","average":{"1min":2.4769999999999999,"5min":1.6240000000000001,"15min":1.482},"min":{"1min":1.0189999999999999,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":20.449999999999999,"5min":20.449999999999999,"15min":20.449999999999999},"last":0.94599999999999995}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4119999999999999}]},{"osd":13,"last update":"Wed Sep 18 10:16:33 2024","interfaces":[{"interface":"back","average":{"1min":10.028,"5min":16.707999999999998,"15min":17.821999999999999},"min":{"1min":1.1459999999999999,"5min":0.874,"15min":0.874},"max":{"1min":113.428,"5min":139.655,"15min":139.655},"last":1.349},{"interface":"front","average":{"1min":5.1970000000000001,"5min":16.942,"15min":18.899999999999999},"min":{"1min":1.077,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":49.457999999999998,"5min":173.113,"15min":173.113},"last":1.75}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1799999999999999}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2829999999999999}]}]},{"osd":8,"up_from":22,"seq":94489280543,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303768,"kb_used_data":2248,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068072,"statfs":{"total":96636764160,"available":96325705728,"internally_reserved":0,"allocated":2301952,"data_stored":1341063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[2,4,6,7,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93300000000000005}]},{"osd":1,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":1.117,"5min":1.117,"15min":1.117},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.669,"5min":1.669,"15min":1.669},"last":1.2230000000000001},{"interface":"front","average":{"1min":2.3290000000000002,"5min":2.3290000000000002,"15min":2.3290000000000002},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":29.523,"5min":29.523,"15min":29.523},"last":1.1279999999999999}]},{"osd":2,"last update":"Wed Sep 18 10:16:48 2024","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":5.5140000000000002,"15min":6.2300000000000004},"min":{"1min":0.93500000000000005,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.8,"5min":77.435000000000002,"15min":77.435000000000002},"last":0.93500000000000005},{"interface":"front","average":{"1min":1.1639999999999999,"5min":0.97099999999999997,"15min":0.93899999999999995},"min":{"1min":0.71999999999999997,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.903,"5min":1.903,"15min":1.903},"last":0.71999999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.306}]},{"osd":4,"last update":"Wed Sep 18 10:16:48 2024","interfaces":[{"interface":"back","average":{"1min":1.23,"5min":9.5879999999999992,"15min":10.981},"min":{"1min":0.71899999999999997,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":7.1559999999999997,"5min":90.438000000000002,"15min":90.438000000000002},"last":0.83899999999999997},{"interface":"front","average":{"1min":1.1699999999999999,"5min":10.691000000000001,"15min":12.278},"min":{"1min":0.66100000000000003,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":2.1309999999999998,"5min":93.939999999999998,"15min":93.939999999999998},"last":0.66100000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4199999999999999}]},{"osd":6,"last update":"Wed Sep 18 10:16:48 2024","interfaces":[{"interface":"back","average":{"1min":1.359,"5min":15.042999999999999,"15min":17.324000000000002},"min":{"1min":0.90700000000000003,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":2.0350000000000001,"5min":293.26900000000001,"15min":293.26900000000001},"last":0.90700000000000003},{"interface":"front","average":{"1min":1.2230000000000001,"5min":5.1689999999999996,"15min":5.827},"min":{"1min":0.88900000000000001,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.839,"5min":70.203000000000003,"15min":70.203000000000003},"last":1.1060000000000001}]},{"osd":7,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":7.1950000000000003,"5min":3.6070000000000002,"15min":3.0089999999999999},"min":{"1min":0.80900000000000005,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":80.635000000000005,"5min":80.635000000000005,"15min":80.635000000000005},"last":1.0580000000000001},{"interface":"front","average":{"1min":9.3490000000000002,"5min":6.726,"15min":6.2889999999999997},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":99.060000000000002,"5min":99.060000000000002,"15min":99.060000000000002},"last":1.01}]},{"osd":9,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":4.8730000000000002,"5min":3.0110000000000001,"15min":2.7010000000000001},"min":{"1min":0.92700000000000005,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":80.076999999999998,"5min":80.076999999999998,"15min":80.076999999999998},"last":1.28},{"interface":"front","average":{"1min":1.6240000000000001,"5min":7.4489999999999998,"15min":8.4199999999999999},"min":{"1min":1.196,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":2.5190000000000001,"5min":98.914000000000001,"15min":98.914000000000001},"last":1.153}]},{"osd":10,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":1.6319999999999999,"5min":5.8559999999999999,"15min":6.5599999999999996},"min":{"1min":0.99099999999999999,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":2.3679999999999999,"5min":90.751000000000005,"15min":90.751000000000005},"last":1.081},{"interface":"front","average":{"1min":1.591,"5min":7.7889999999999997,"15min":8.8219999999999992},"min":{"1min":1.0640000000000001,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":2.3340000000000001,"5min":83.795000000000002,"15min":83.795000000000002},"last":1.3839999999999999}]},{"osd":11,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":1.7450000000000001,"5min":2.669,"15min":2.823},"min":{"1min":1.3340000000000001,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":2.383,"5min":26.462,"15min":26.462},"last":1.232},{"interface":"front","average":{"1min":1.752,"5min":1.948,"15min":1.9810000000000001},"min":{"1min":1.24,"5min":0.99299999999999999,"15min":0.99299999999999999},"max":{"1min":2.3479999999999999,"5min":12.475,"15min":12.475},"last":1.202}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0760000000000001}]},{"osd":13,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":3.052,"5min":7.9329999999999998,"15min":8.7469999999999999},"min":{"1min":1.05,"5min":1.0429999999999999,"15min":1.0429999999999999},"max":{"1min":30.582999999999998,"5min":94.222999999999999,"15min":94.222999999999999},"last":1.264},{"interface":"front","average":{"1min":1.635,"5min":2.5059999999999998,"15min":2.6509999999999998},"min":{"1min":1.1180000000000001,"5min":0.93400000000000005,"15min":0.93400000000000005},"max":{"1min":2.4689999999999999,"5min":23.983000000000001,"15min":23.983000000000001},"last":1.464}]},{"osd":14,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":1.802,"5min":16.975000000000001,"15min":19.504000000000001},"min":{"1min":1.28,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":2.4500000000000002,"5min":150.93600000000001,"15min":150.93600000000001},"last":1.3240000000000001},{"interface":"front","average":{"1min":4.8170000000000002,"5min":3.8410000000000002,"15min":3.6789999999999998},"min":{"1min":1.2210000000000001,"5min":0.90200000000000002,"15min":0.90200000000000002},"max":{"1min":69.918000000000006,"5min":69.918000000000006,"15min":69.918000000000006},"last":1.3460000000000001}]},{"osd":15,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":5.9089999999999998,"5min":3.4169999999999998,"15min":3.0019999999999998},"min":{"1min":1.2549999999999999,"5min":0.93300000000000005,"15min":0.93300000000000005},"max":{"1min":98.558000000000007,"5min":98.558000000000007,"15min":98.558000000000007},"last":1.5409999999999999},{"interface":"front","average":{"1min":1.8400000000000001,"5min":2.8119999999999998,"15min":2.9740000000000002},"min":{"1min":1.3160000000000001,"5min":0.95299999999999996,"15min":0.95299999999999996},"max":{"1min":2.5859999999999999,"5min":26.414000000000001,"15min":26.414000000000001},"last":1.373}]}]},{"osd":10,"up_from":22,"seq":94489280544,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303748,"kb_used_data":2232,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068092,"statfs":{"total":96636764160,"available":96325726208,"internally_reserved":0,"allocated":2285568,"data_stored":1331589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,4,5,6,7,9,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:16:29 2024","interfaces":[{"interface":"back","average":{"1min":16.117000000000001,"5min":16.117000000000001,"15min":16.117000000000001},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":145.12799999999999,"5min":145.12799999999999,"15min":145.12799999999999},"last":1.401},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.431,"5min":1.431,"15min":1.431},"last":1.0780000000000001}]},{"osd":1,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":9.5719999999999992,"5min":10.66,"15min":10.842000000000001},"min":{"1min":0.88500000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":145.20699999999999,"5min":145.20699999999999,"15min":145.20699999999999},"last":0.876},{"interface":"front","average":{"1min":4.7510000000000003,"5min":14.728,"15min":16.390999999999998},"min":{"1min":0.754,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":38.338000000000001,"5min":99.995999999999995,"15min":99.995999999999995},"last":0.71899999999999997}]},{"osd":2,"last update":"Wed Sep 18 10:15:33 2024","interfaces":[{"interface":"back","average":{"1min":10.893000000000001,"5min":10.893000000000001,"15min":10.893000000000001},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":84.072000000000003,"5min":84.072000000000003,"15min":84.072000000000003},"last":0.99199999999999999},{"interface":"front","average":{"1min":11.550000000000001,"5min":11.550000000000001,"15min":11.550000000000001},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":99.798000000000002,"5min":99.798000000000002,"15min":99.798000000000002},"last":1.425}]},{"osd":3,"last update":"Wed Sep 18 10:16:22 2024","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":1.119,"15min":1.119},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.99,"5min":1.99,"15min":1.99},"last":1.5},{"interface":"front","average":{"1min":18.094999999999999,"5min":18.094999999999999,"15min":18.094999999999999},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":145.535,"5min":145.535,"15min":145.535},"last":1.4510000000000001}]},{"osd":4,"last update":"Wed Sep 18 10:16:49 2024","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":20.715,"15min":23.963999999999999},"min":{"1min":0.92300000000000004,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.5820000000000001,"5min":125.23099999999999,"15min":125.23099999999999},"last":1.048},{"interface":"front","average":{"1min":1.206,"5min":25.175000000000001,"15min":29.170000000000002},"min":{"1min":0.88800000000000001,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.4930000000000001,"5min":295.93200000000002,"15min":295.93200000000002},"last":1.2210000000000001}]},{"osd":5,"last update":"Wed Sep 18 10:16:49 2024","interfaces":[{"interface":"back","average":{"1min":8.9960000000000004,"5min":21.460999999999999,"15min":23.539000000000001},"min":{"1min":0.81000000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":145.24000000000001,"5min":215.51400000000001,"15min":215.51400000000001},"last":1.111},{"interface":"front","average":{"1min":2.4140000000000001,"5min":22.346,"15min":25.667999999999999},"min":{"1min":0.85399999999999998,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":20.302,"5min":252.69300000000001,"15min":252.69300000000001},"last":1.421}]},{"osd":6,"last update":"Wed Sep 18 10:16:22 2024","interfaces":[{"interface":"back","average":{"1min":12.393000000000001,"5min":12.393000000000001,"15min":12.393000000000001},"min":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"max":{"1min":111.173,"5min":111.173,"15min":111.173},"last":1.1859999999999999},{"interface":"front","average":{"1min":15.771000000000001,"5min":15.771000000000001,"15min":15.771000000000001},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":221.13900000000001,"5min":221.13900000000001,"15min":221.13900000000001},"last":0.66500000000000004}]},{"osd":7,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":1.5109999999999999,"5min":2.7490000000000001,"15min":2.9550000000000001},"min":{"1min":0.95699999999999996,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.919,"5min":34.558,"15min":34.558},"last":1.083},{"interface":"front","average":{"1min":10.101000000000001,"5min":13.186999999999999,"15min":13.701000000000001},"min":{"1min":1.0920000000000001,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":145.5,"5min":145.5,"15min":145.5},"last":1.3140000000000001}]},{"osd":9,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":8.3640000000000008,"5min":11.07,"15min":11.521000000000001},"min":{"1min":1.0349999999999999,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":119.202,"5min":119.202,"15min":119.202},"last":1.361},{"interface":"front","average":{"1min":1.629,"5min":10.364000000000001,"15min":11.82},"min":{"1min":1.073,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":2.23,"5min":97.385999999999996,"15min":97.385999999999996},"last":0.92500000000000004}]},{"osd":11,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":10.132999999999999,"5min":11.489000000000001,"15min":11.715},"min":{"1min":1.286,"5min":0.85699999999999998,"15min":0.85699999999999998},"max":{"1min":145.26900000000001,"5min":145.26900000000001,"15min":145.26900000000001},"last":1.012},{"interface":"front","average":{"1min":5.6719999999999997,"5min":19.837,"15min":22.198},"min":{"1min":1.28,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":44.121000000000002,"5min":295.70400000000001,"15min":295.70400000000001},"last":1.2729999999999999}]},{"osd":12,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":1.6559999999999999,"5min":2.8250000000000002,"15min":3.02},"min":{"1min":1.0620000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":2.2559999999999998,"5min":34.625999999999998,"15min":34.625999999999998},"last":1.4590000000000001},{"interface":"front","average":{"1min":10.140000000000001,"5min":10.986000000000001,"15min":11.127000000000001},"min":{"1min":1.198,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":145.358,"5min":145.358,"15min":145.358},"last":1.639}]},{"osd":13,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":10.433999999999999,"5min":17.994,"15min":19.254000000000001},"min":{"1min":1.1990000000000001,"5min":0.877,"15min":0.877},"max":{"1min":123.696,"5min":123.696,"15min":123.696},"last":1.5229999999999999},{"interface":"front","average":{"1min":7.6379999999999999,"5min":14.551,"15min":15.702999999999999},"min":{"1min":1.163,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":68.363,"5min":85.879999999999995,"15min":85.879999999999995},"last":1.609}]},{"osd":14,"last update":"Wed Sep 18 10:16:22 2024","interfaces":[{"interface":"back","average":{"1min":15.864000000000001,"5min":15.864000000000001,"15min":15.864000000000001},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":145.46700000000001,"5min":145.46700000000001,"15min":145.46700000000001},"last":1.4830000000000001},{"interface":"front","average":{"1min":13.93,"5min":13.93,"15min":13.93},"min":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"max":{"1min":100.092,"5min":100.092,"15min":100.092},"last":1.587}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1080000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T10:16:56.211 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-18T10:16:56.211 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-18T10:16:56.220 INFO:tasks.ceph.ceph_manager.ceph:Canceling any pending splits or merges... 2024-09-18T10:16:56.220 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-18T10:16:56.510 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-18T10:16:56.510 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":151,"fsid":"ab6d6beb-0627-4b7f-8cdb-4e0a44970f10","created":"2024-09-18T10:14:18.827644+0000","modified":"2024-09-18T10:16:52.201348+0000","last_up_change":"2024-09-18T10:14:30.228591+0000","last_in_change":"2024-09-18T10:15:49.554826+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-18T10:14:31.266113+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-18T10:14:33.358485+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":29,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"39cf0511-0009-433f-92f2-e5686f33d713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":22197},{"type":"v1","addr":"172.21.15.29:6809","nonce":22197}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":22197},{"type":"v1","addr":"172.21.15.29:6811","nonce":22197}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":22197},{"type":"v1","addr":"172.21.15.29:6815","nonce":22197}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":22197},{"type":"v1","addr":"172.21.15.29:6813","nonce":22197}]},"public_addr":"172.21.15.29:6809/22197","cluster_addr":"172.21.15.29:6811/22197","heartbeat_back_addr":"172.21.15.29:6815/22197","heartbeat_front_addr":"172.21.15.29:6813/22197","state":["exists","up"]},{"osd":1,"uuid":"a0bf06ab-b98e-4130-91d2-7d4283540c08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":22140},{"type":"v1","addr":"172.21.15.136:6809","nonce":22140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":22140},{"type":"v1","addr":"172.21.15.136:6811","nonce":22140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":22140},{"type":"v1","addr":"172.21.15.136:6815","nonce":22140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":22140},{"type":"v1","addr":"172.21.15.136:6813","nonce":22140}]},"public_addr":"172.21.15.136:6809/22140","cluster_addr":"172.21.15.136:6811/22140","heartbeat_back_addr":"172.21.15.136:6815/22140","heartbeat_front_addr":"172.21.15.136:6813/22140","state":["exists","up"]},{"osd":2,"uuid":"742a786e-6129-4fbc-bb5b-e2e3c03c2d77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":22095},{"type":"v1","addr":"172.21.15.154:6825","nonce":22095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":22095},{"type":"v1","addr":"172.21.15.154:6827","nonce":22095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":22095},{"type":"v1","addr":"172.21.15.154:6831","nonce":22095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":22095},{"type":"v1","addr":"172.21.15.154:6829","nonce":22095}]},"public_addr":"172.21.15.154:6825/22095","cluster_addr":"172.21.15.154:6827/22095","heartbeat_back_addr":"172.21.15.154:6831/22095","heartbeat_front_addr":"172.21.15.154:6829/22095","state":["exists","up"]},{"osd":3,"uuid":"3aa77114-24ed-49ab-86ff-a0600794cbf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":22404},{"type":"v1","addr":"172.21.15.189:6803","nonce":22404}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":22404},{"type":"v1","addr":"172.21.15.189:6805","nonce":22404}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":22404},{"type":"v1","addr":"172.21.15.189:6809","nonce":22404}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":22404},{"type":"v1","addr":"172.21.15.189:6807","nonce":22404}]},"public_addr":"172.21.15.189:6803/22404","cluster_addr":"172.21.15.189:6805/22404","heartbeat_back_addr":"172.21.15.189:6809/22404","heartbeat_front_addr":"172.21.15.189:6807/22404","state":["exists","up"]},{"osd":4,"uuid":"426567ad-b0be-41d4-afcf-dc56ec3974cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":22195},{"type":"v1","addr":"172.21.15.29:6801","nonce":22195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":22195},{"type":"v1","addr":"172.21.15.29:6803","nonce":22195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":22195},{"type":"v1","addr":"172.21.15.29:6807","nonce":22195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":22195},{"type":"v1","addr":"172.21.15.29:6805","nonce":22195}]},"public_addr":"172.21.15.29:6801/22195","cluster_addr":"172.21.15.29:6803/22195","heartbeat_back_addr":"172.21.15.29:6807/22195","heartbeat_front_addr":"172.21.15.29:6805/22195","state":["exists","up"]},{"osd":5,"uuid":"d9559e42-1f4e-4373-8392-a536a0afd871","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6817","nonce":22141},{"type":"v1","addr":"172.21.15.136:6819","nonce":22141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6821","nonce":22141},{"type":"v1","addr":"172.21.15.136:6823","nonce":22141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6829","nonce":22141},{"type":"v1","addr":"172.21.15.136:6831","nonce":22141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6825","nonce":22141},{"type":"v1","addr":"172.21.15.136:6827","nonce":22141}]},"public_addr":"172.21.15.136:6819/22141","cluster_addr":"172.21.15.136:6823/22141","heartbeat_back_addr":"172.21.15.136:6831/22141","heartbeat_front_addr":"172.21.15.136:6827/22141","state":["exists","up"]},{"osd":6,"uuid":"d50ef1e2-8bc5-4a90-9be3-365a0db2be6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":22096},{"type":"v1","addr":"172.21.15.154:6801","nonce":22096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":22096},{"type":"v1","addr":"172.21.15.154:6803","nonce":22096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":22096},{"type":"v1","addr":"172.21.15.154:6807","nonce":22096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":22096},{"type":"v1","addr":"172.21.15.154:6805","nonce":22096}]},"public_addr":"172.21.15.154:6801/22096","cluster_addr":"172.21.15.154:6803/22096","heartbeat_back_addr":"172.21.15.154:6807/22096","heartbeat_front_addr":"172.21.15.154:6805/22096","state":["exists","up"]},{"osd":7,"uuid":"bd0b85e4-cfee-4d19-90a5-79854794b48b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":22405},{"type":"v1","addr":"172.21.15.189:6819","nonce":22405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":22405},{"type":"v1","addr":"172.21.15.189:6821","nonce":22405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":22405},{"type":"v1","addr":"172.21.15.189:6825","nonce":22405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":22405},{"type":"v1","addr":"172.21.15.189:6823","nonce":22405}]},"public_addr":"172.21.15.189:6819/22405","cluster_addr":"172.21.15.189:6821/22405","heartbeat_back_addr":"172.21.15.189:6825/22405","heartbeat_front_addr":"172.21.15.189:6823/22405","state":["exists","up"]},{"osd":8,"uuid":"cd333f56-c165-45d7-8c8b-48d34bc4ddc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":22200},{"type":"v1","addr":"172.21.15.29:6825","nonce":22200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":22200},{"type":"v1","addr":"172.21.15.29:6827","nonce":22200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":22200},{"type":"v1","addr":"172.21.15.29:6831","nonce":22200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":22200},{"type":"v1","addr":"172.21.15.29:6829","nonce":22200}]},"public_addr":"172.21.15.29:6825/22200","cluster_addr":"172.21.15.29:6827/22200","heartbeat_back_addr":"172.21.15.29:6831/22200","heartbeat_front_addr":"172.21.15.29:6829/22200","state":["exists","up"]},{"osd":9,"uuid":"1e079e4c-8801-4754-9d0d-d0b68695d32e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":22142},{"type":"v1","addr":"172.21.15.136:6801","nonce":22142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":22142},{"type":"v1","addr":"172.21.15.136:6803","nonce":22142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":22142},{"type":"v1","addr":"172.21.15.136:6807","nonce":22142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":22142},{"type":"v1","addr":"172.21.15.136:6805","nonce":22142}]},"public_addr":"172.21.15.136:6801/22142","cluster_addr":"172.21.15.136:6803/22142","heartbeat_back_addr":"172.21.15.136:6807/22142","heartbeat_front_addr":"172.21.15.136:6805/22142","state":["exists","up"]},{"osd":10,"uuid":"92ee68d4-371c-4849-930f-a8fa18d64b99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":22097},{"type":"v1","addr":"172.21.15.154:6809","nonce":22097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":22097},{"type":"v1","addr":"172.21.15.154:6811","nonce":22097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":22097},{"type":"v1","addr":"172.21.15.154:6815","nonce":22097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":22097},{"type":"v1","addr":"172.21.15.154:6813","nonce":22097}]},"public_addr":"172.21.15.154:6809/22097","cluster_addr":"172.21.15.154:6811/22097","heartbeat_back_addr":"172.21.15.154:6815/22097","heartbeat_front_addr":"172.21.15.154:6813/22097","state":["exists","up"]},{"osd":11,"uuid":"634ddaf5-f7b8-4b27-a46a-04e21b73fcf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":22416},{"type":"v1","addr":"172.21.15.189:6827","nonce":22416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":22416},{"type":"v1","addr":"172.21.15.189:6829","nonce":22416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":22416},{"type":"v1","addr":"172.21.15.189:6833","nonce":22416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":22416},{"type":"v1","addr":"172.21.15.189:6831","nonce":22416}]},"public_addr":"172.21.15.189:6827/22416","cluster_addr":"172.21.15.189:6829/22416","heartbeat_back_addr":"172.21.15.189:6833/22416","heartbeat_front_addr":"172.21.15.189:6831/22416","state":["exists","up"]},{"osd":12,"uuid":"bc58137b-5948-400d-b377-193b15824d8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":22210},{"type":"v1","addr":"172.21.15.29:6817","nonce":22210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":22210},{"type":"v1","addr":"172.21.15.29:6819","nonce":22210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":22210},{"type":"v1","addr":"172.21.15.29:6823","nonce":22210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":22210},{"type":"v1","addr":"172.21.15.29:6821","nonce":22210}]},"public_addr":"172.21.15.29:6817/22210","cluster_addr":"172.21.15.29:6819/22210","heartbeat_back_addr":"172.21.15.29:6823/22210","heartbeat_front_addr":"172.21.15.29:6821/22210","state":["exists","up"]},{"osd":13,"uuid":"a8b753a7-ab69-4853-ac36-66d9b85c2e20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":22143},{"type":"v1","addr":"172.21.15.136:6818","nonce":22143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":22143},{"type":"v1","addr":"172.21.15.136:6822","nonce":22143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":22143},{"type":"v1","addr":"172.21.15.136:6830","nonce":22143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":22143},{"type":"v1","addr":"172.21.15.136:6826","nonce":22143}]},"public_addr":"172.21.15.136:6818/22143","cluster_addr":"172.21.15.136:6822/22143","heartbeat_back_addr":"172.21.15.136:6830/22143","heartbeat_front_addr":"172.21.15.136:6826/22143","state":["exists","up"]},{"osd":14,"uuid":"8ad0848f-0de4-4a75-9c63-8c4a4a7b8524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":22098},{"type":"v1","addr":"172.21.15.154:6817","nonce":22098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":22098},{"type":"v1","addr":"172.21.15.154:6819","nonce":22098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":22098},{"type":"v1","addr":"172.21.15.154:6823","nonce":22098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":22098},{"type":"v1","addr":"172.21.15.154:6821","nonce":22098}]},"public_addr":"172.21.15.154:6817/22098","cluster_addr":"172.21.15.154:6819/22098","heartbeat_back_addr":"172.21.15.154:6823/22098","heartbeat_front_addr":"172.21.15.154:6821/22098","state":["exists","up"]},{"osd":15,"uuid":"069099b3-ec1a-483b-b967-98cddbf2f678","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":22415},{"type":"v1","addr":"172.21.15.189:6811","nonce":22415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":22415},{"type":"v1","addr":"172.21.15.189:6813","nonce":22415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":22415},{"type":"v1","addr":"172.21.15.189:6817","nonce":22415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":22415},{"type":"v1","addr":"172.21.15.189:6815","nonce":22415}]},"public_addr":"172.21.15.189:6811/22415","cluster_addr":"172.21.15.189:6813/22415","heartbeat_back_addr":"172.21.15.189:6817/22415","heartbeat_front_addr":"172.21.15.189:6815/22415","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:15:56.126704+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:15:32.497046+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:15:55.189067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:15:44.298040+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:15:56.991035+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:15:51.718897+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:15:41.309784+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:16:42.140104+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:15:42.972647+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:15:36.896626+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:15:54.542105+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:15:47.318375+0000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:16:37.858675+0000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:16:51.609110+0000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:15:52.801215+0000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:15:59.018997+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"},"shecprofile":{"c":"2","crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","k":"4","m":"3","name":"shecprofile","plugin":"shec","technique":"multiple"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-18T10:16:56.524 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-18T10:16:56.524 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T10:16:56.825 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-18T10:16:56.828 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-09-18T10:16:56.840 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":209,"stamp":"2024-09-18T10:16:55.239416+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":147,"num_write_kb":2674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":115,"ondisk_log_size":115,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":14,"kb":1509949440,"kb_used":4865408,"kb_used_data":36952,"kb_used_omap":26,"kb_used_meta":4828197,"kb_avail":1505084032,"statfs":{"total":1546188226560,"available":1541206048768,"internally_reserved":0,"allocated":37838848,"data_stored":22477174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":27264,"internal_metadata":4944074112},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.056129"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":144,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-18T10:16:49.404525+0000","last_change":"2024-09-18T10:16:18.085265+0000","last_active":"2024-09-18T10:16:49.404525+0000","last_peered":"2024-09-18T10:16:49.404525+0000","last_clean":"2024-09-18T10:16:49.404525+0000","last_became_active":"2024-09-18T10:14:35.348943+0000","last_became_peered":"2024-09-18T10:14:35.348943+0000","last_unstale":"2024-09-18T10:16:49.404525+0000","last_undegraded":"2024-09-18T10:16:49.404525+0000","last_fullsized":"2024-09-18T10:16:49.404525+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:16:18.085027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:16:18.085027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:17:43.814969+0000","scrub_duration":0.30385945199999997,"objects_trimmed":0,"snaptrim_duration":0.00060442999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,10],"acting":[13,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":76,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-18T10:16:49.402946+0000","last_change":"2024-09-18T10:16:18.981603+0000","last_active":"2024-09-18T10:16:49.402946+0000","last_peered":"2024-09-18T10:16:49.402946+0000","last_clean":"2024-09-18T10:16:49.402946+0000","last_became_active":"2024-09-18T10:15:50.978055+0000","last_became_peered":"2024-09-18T10:15:50.978055+0000","last_unstale":"2024-09-18T10:16:49.402946+0000","last_undegraded":"2024-09-18T10:16:49.402946+0000","last_fullsized":"2024-09-18T10:16:49.402946+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:16:18.981373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:16:18.981373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:17:41.742568+0000","scrub_duration":0.40456036899999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11],"acting":[8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":140,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-18T10:16:49.303750+0000","last_change":"2024-09-18T10:15:50.883382+0000","last_active":"2024-09-18T10:16:49.303750+0000","last_peered":"2024-09-18T10:16:49.303750+0000","last_clean":"2024-09-18T10:16:49.303750+0000","last_became_active":"2024-09-18T10:14:35.329198+0000","last_became_peered":"2024-09-18T10:14:35.329198+0000","last_unstale":"2024-09-18T10:16:49.303750+0000","last_undegraded":"2024-09-18T10:16:49.303750+0000","last_fullsized":"2024-09-18T10:16:49.303750+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:15:50.883147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:15:50.883147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:17:15.846467+0000","scrub_duration":0.153222372,"objects_trimmed":0,"snaptrim_duration":0.00062415699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":142,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T10:16:48.501848+0000","last_change":"2024-09-18T10:16:02.749341+0000","last_active":"2024-09-18T10:16:48.501848+0000","last_peered":"2024-09-18T10:16:48.501848+0000","last_clean":"2024-09-18T10:16:48.501848+0000","last_became_active":"2024-09-18T10:14:35.524102+0000","last_became_peered":"2024-09-18T10:14:35.524102+0000","last_unstale":"2024-09-18T10:16:48.501848+0000","last_undegraded":"2024-09-18T10:16:48.501848+0000","last_fullsized":"2024-09-18T10:16:48.501848+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:16:02.749150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:16:02.749150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:17:04.725472+0000","scrub_duration":0.20368988499999999,"objects_trimmed":0,"snaptrim_duration":0.10077985,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"29'2","reported_seq":145,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-18T10:16:49.304395+0000","last_change":"2024-09-18T10:16:20.864837+0000","last_active":"2024-09-18T10:16:49.304395+0000","last_peered":"2024-09-18T10:16:49.304395+0000","last_clean":"2024-09-18T10:16:49.304395+0000","last_became_active":"2024-09-18T10:14:35.715374+0000","last_became_peered":"2024-09-18T10:14:35.715374+0000","last_unstale":"2024-09-18T10:16:49.304395+0000","last_undegraded":"2024-09-18T10:16:49.304395+0000","last_fullsized":"2024-09-18T10:16:49.304395+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2","last_scrub_stamp":"2024-09-18T10:16:20.864635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:16:20.864635+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:17:23.226052+0000","scrub_duration":0.20463374300000001,"objects_trimmed":0,"snaptrim_duration":0.002421224,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,12],"acting":[13,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":150,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-18T10:16:49.186308+0000","last_change":"2024-09-18T10:16:15.062108+0000","last_active":"2024-09-18T10:16:49.186308+0000","last_peered":"2024-09-18T10:16:49.186308+0000","last_clean":"2024-09-18T10:16:49.186308+0000","last_became_active":"2024-09-18T10:14:35.348262+0000","last_became_peered":"2024-09-18T10:14:35.348262+0000","last_unstale":"2024-09-18T10:16:49.186308+0000","last_undegraded":"2024-09-18T10:16:49.186308+0000","last_fullsized":"2024-09-18T10:16:49.186308+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:16:15.061892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:16:15.061892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:17:40.002913+0000","scrub_duration":0.40410323300000001,"objects_trimmed":0,"snaptrim_duration":0.00063430400000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,0],"acting":[14,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":144,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-18T10:16:52.538159+0000","last_change":"2024-09-18T10:16:15.352277+0000","last_active":"2024-09-18T10:16:52.538159+0000","last_peered":"2024-09-18T10:16:52.538159+0000","last_clean":"2024-09-18T10:16:52.538159+0000","last_became_active":"2024-09-18T10:14:35.349203+0000","last_became_peered":"2024-09-18T10:14:35.349203+0000","last_unstale":"2024-09-18T10:16:52.538159+0000","last_undegraded":"2024-09-18T10:16:52.538159+0000","last_fullsized":"2024-09-18T10:16:52.538159+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:16:15.352050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:16:15.352050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:17:45.176858+0000","scrub_duration":0.203632596,"objects_trimmed":0,"snaptrim_duration":0.00060130600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3],"acting":[9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"27'1","reported_seq":145,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-18T10:16:49.186339+0000","last_change":"2024-09-18T10:16:06.900858+0000","last_active":"2024-09-18T10:16:49.186339+0000","last_peered":"2024-09-18T10:16:49.186339+0000","last_clean":"2024-09-18T10:16:49.186339+0000","last_became_active":"2024-09-18T10:14:35.487746+0000","last_became_peered":"2024-09-18T10:14:35.487746+0000","last_unstale":"2024-09-18T10:16:49.186339+0000","last_undegraded":"2024-09-18T10:16:49.186339+0000","last_fullsized":"2024-09-18T10:16:49.186339+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'1","last_scrub_stamp":"2024-09-18T10:16:06.900601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:16:06.900601+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:17:25.247813+0000","scrub_duration":0.27078197700000001,"objects_trimmed":0,"snaptrim_duration":0.10072764100000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,5],"acting":[14,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"1.0","version":"29'112","reported_seq":275,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-18T10:16:49.187571+0000","last_change":"2024-09-18T10:16:03.315273+0000","last_active":"2024-09-18T10:16:49.187571+0000","last_peered":"2024-09-18T10:16:49.187571+0000","last_clean":"2024-09-18T10:16:49.187571+0000","last_became_active":"2024-09-18T10:14:32.508037+0000","last_became_peered":"2024-09-18T10:14:32.508037+0000","last_unstale":"2024-09-18T10:16:49.187571+0000","last_undegraded":"2024-09-18T10:16:49.187571+0000","last_fullsized":"2024-09-18T10:16:49.187571+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'112","last_scrub_stamp":"2024-09-18T10:16:03.315126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:31.267741+0000","last_clean_scrub_stamp":"2024-09-18T10:16:03.315126+0000","objects_scrubbed":2,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:17:16.808627+0000","scrub_duration":0.10318229800000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,15],"acting":[6,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":13},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":112,"ondisk_log_size":112,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":15,"up_from":22,"seq":94489280543,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304272,"kb_used_data":2752,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94067568,"statfs":{"total":96636764160,"available":96325189632,"internally_reserved":0,"allocated":2818048,"data_stored":1860517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,4,6,7,8,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:16:32 2024","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":0.83899999999999997,"15min":0.81899999999999995},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":2.0019999999999998,"5min":2.0019999999999998,"15min":2.0019999999999998},"last":1.0649999999999999},{"interface":"front","average":{"1min":0.996,"5min":5.5739999999999998,"15min":6.3369999999999997},"min":{"1min":0.72899999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.506,"5min":101.291,"15min":101.291},"last":0.97299999999999998}]},{"osd":1,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.93300000000000005},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.931,"5min":1.931,"15min":1.931},"last":1.1279999999999999}]},{"osd":2,"last update":"Wed Sep 18 10:16:32 2024","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.0169999999999999,"15min":0.995},"min":{"1min":0.79700000000000004,"5min":0.5,"15min":0.5},"max":{"1min":1.7470000000000001,"5min":1.7470000000000001,"15min":1.7470000000000001},"last":1.028},{"interface":"front","average":{"1min":1.1419999999999999,"5min":4.6589999999999998,"15min":5.2450000000000001},"min":{"1min":0.82899999999999996,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.4379999999999999,"5min":78.242999999999995,"15min":78.242999999999995},"last":1.252}]},{"osd":3,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":3.3029999999999999,"5min":3.3029999999999999,"15min":3.3029999999999999},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":41.720999999999997,"5min":41.720999999999997,"15min":41.720999999999997},"last":0.55400000000000005},{"interface":"front","average":{"1min":1.014,"5min":1.014,"15min":1.014},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.49,"5min":1.49,"15min":1.49},"last":0.89500000000000002}]},{"osd":4,"last update":"Wed Sep 18 10:16:32 2024","interfaces":[{"interface":"back","average":{"1min":1.3580000000000001,"5min":9.0879999999999992,"15min":10.375999999999999},"min":{"1min":0.747,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":8.0449999999999999,"5min":99.745999999999995,"15min":99.745999999999995},"last":1.2729999999999999},{"interface":"front","average":{"1min":1.595,"5min":9.2309999999999999,"15min":10.503},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":8.0679999999999996,"5min":99.781000000000006,"15min":99.781000000000006},"last":1.222}]},{"osd":5,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":1.0860000000000001},{"interface":"front","average":{"1min":1.7869999999999999,"5min":1.7869999999999999,"15min":1.7869999999999999},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":10.874000000000001,"5min":10.874000000000001,"15min":10.874000000000001},"last":1.163}]},{"osd":6,"last update":"Wed Sep 18 10:16:32 2024","interfaces":[{"interface":"back","average":{"1min":14.140000000000001,"5min":4.4039999999999999,"15min":2.782},"min":{"1min":0.92900000000000005,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":269.80500000000001,"5min":269.80500000000001,"15min":269.80500000000001},"last":1},{"interface":"front","average":{"1min":1.361,"5min":4.7610000000000001,"15min":5.327},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":2.012,"5min":74.156000000000006,"15min":74.156000000000006},"last":1.1919999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80400000000000005}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93000000000000005}]},{"osd":10,"last update":"Wed Sep 18 10:16:32 2024","interfaces":[{"interface":"back","average":{"1min":5.8639999999999999,"5min":6.2670000000000003,"15min":6.3339999999999996},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":93.704999999999998,"5min":93.704999999999998,"15min":93.704999999999998},"last":0.85399999999999998},{"interface":"front","average":{"1min":5.7770000000000001,"5min":6.2549999999999999,"15min":6.335},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":93.671000000000006,"5min":93.671000000000006,"15min":93.671000000000006},"last":1.121}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.254}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3680000000000001}]},{"osd":13,"last update":"Wed Sep 18 10:16:32 2024","interfaces":[{"interface":"back","average":{"1min":1.99,"5min":5.2629999999999999,"15min":5.8090000000000002},"min":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"max":{"1min":7.3179999999999996,"5min":80.197000000000003,"15min":80.197000000000003},"last":1.5029999999999999},{"interface":"front","average":{"1min":3.4830000000000001,"5min":1.8080000000000001,"15min":1.5289999999999999},"min":{"1min":0.89400000000000002,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":25.193999999999999,"5min":25.193999999999999,"15min":25.193999999999999},"last":1.4670000000000001}]},{"osd":14,"last update":"Wed Sep 18 10:16:32 2024","interfaces":[{"interface":"back","average":{"1min":10.725,"5min":3.2999999999999998,"15min":2.0619999999999998},"min":{"1min":0.95699999999999996,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":117.562,"5min":117.562,"15min":117.562},"last":1.5980000000000001},{"interface":"front","average":{"1min":4.6139999999999999,"5min":9.5229999999999997,"15min":10.340999999999999},"min":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"max":{"1min":61.567,"5min":85.006,"15min":85.006},"last":1.4390000000000001}]}]},{"osd":14,"up_from":22,"seq":94489280543,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303768,"kb_used_data":2248,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068072,"statfs":{"total":96636764160,"available":96325705728,"internally_reserved":0,"allocated":2301952,"data_stored":1341063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,5,7,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:16:32 2024","interfaces":[{"interface":"back","average":{"1min":1.0009999999999999,"5min":12.513,"15min":14.430999999999999},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.405,"5min":189.88800000000001,"15min":189.88800000000001},"last":0.59599999999999997},{"interface":"front","average":{"1min":7.0330000000000004,"5min":17.033000000000001,"15min":18.699000000000002},"min":{"1min":0.77300000000000002,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":71.900999999999996,"5min":180.09999999999999,"15min":180.09999999999999},"last":0.68700000000000006}]},{"osd":1,"last update":"Wed Sep 18 10:16:32 2024","interfaces":[{"interface":"back","average":{"1min":2.1040000000000001,"5min":7.7670000000000003,"15min":8.7110000000000003},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":21.334,"5min":79.626000000000005,"15min":79.626000000000005},"last":0.754},{"interface":"front","average":{"1min":1.1479999999999999,"5min":12.601000000000001,"15min":14.51},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.8879999999999999,"5min":189.762,"15min":189.762},"last":0.85899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.417}]},{"osd":3,"last update":"Wed Sep 18 10:16:32 2024","interfaces":[{"interface":"back","average":{"1min":3.5430000000000001,"5min":12.906000000000001,"15min":14.465999999999999},"min":{"1min":0.84799999999999998,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":44.683,"5min":180.19900000000001,"15min":180.19900000000001},"last":1.7529999999999999},{"interface":"front","average":{"1min":5.3520000000000003,"5min":18.143999999999998,"15min":20.276},"min":{"1min":0.86399999999999999,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":42.570999999999998,"5min":180.851,"15min":180.851},"last":1.1899999999999999}]},{"osd":4,"last update":"Wed Sep 18 10:16:32 2024","interfaces":[{"interface":"back","average":{"1min":6.8360000000000003,"5min":21.687000000000001,"15min":24.161999999999999},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":71.911000000000001,"5min":189.809,"15min":189.809},"last":2.0249999999999999},{"interface":"front","average":{"1min":14.032999999999999,"5min":15.031000000000001,"15min":15.196999999999999},"min":{"1min":0.76100000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":133.244,"5min":180.31700000000001,"15min":180.31700000000001},"last":1.9350000000000001}]},{"osd":5,"last update":"Wed Sep 18 10:16:32 2024","interfaces":[{"interface":"back","average":{"1min":1.831,"5min":21.507000000000001,"15min":24.786000000000001},"min":{"1min":0.66300000000000003,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":13.42,"5min":277.41199999999998,"15min":277.41199999999998},"last":1.1020000000000001},{"interface":"front","average":{"1min":1.9530000000000001,"5min":12.936999999999999,"15min":14.766999999999999},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":14.288,"5min":189.64699999999999,"15min":189.64699999999999},"last":0.91900000000000004}]},{"osd":7,"last update":"Wed Sep 18 10:16:32 2024","interfaces":[{"interface":"back","average":{"1min":6.29,"5min":13.625999999999999,"15min":14.848000000000001},"min":{"1min":0.995,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":49.378,"5min":180.268,"15min":180.268},"last":1.448},{"interface":"front","average":{"1min":3.8929999999999998,"5min":12.901,"15min":14.403},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":49.402999999999999,"5min":180.631,"15min":180.631},"last":1.494}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1619999999999999}]},{"osd":9,"last update":"Wed Sep 18 10:16:17 2024","interfaces":[{"interface":"back","average":{"1min":4,"5min":4,"15min":4},"min":{"1min":1.296,"5min":1.296,"15min":1.296},"max":{"1min":44.716000000000001,"5min":44.716000000000001,"15min":44.716000000000001},"last":1.204},{"interface":"front","average":{"1min":1.427,"5min":1.427,"15min":1.427},"min":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"max":{"1min":1.9610000000000001,"5min":1.9610000000000001,"15min":1.9610000000000001},"last":1.274}]},{"osd":10,"last update":"Wed Sep 18 10:15:49 2024","interfaces":[{"interface":"back","average":{"1min":17.039000000000001,"5min":17.039000000000001,"15min":17.039000000000001},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":180.15899999999999,"5min":180.15899999999999,"15min":180.15899999999999},"last":1},{"interface":"front","average":{"1min":31.004000000000001,"5min":31.004000000000001,"15min":31.004000000000001},"min":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":269.423,"5min":269.423,"15min":269.423},"last":1.381}]},{"osd":11,"last update":"Wed Sep 18 10:16:10 2024","interfaces":[{"interface":"back","average":{"1min":9.6890000000000001,"5min":9.6890000000000001,"15min":9.6890000000000001},"min":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":91.111999999999995,"5min":91.111999999999995,"15min":91.111999999999995},"last":1.3520000000000001},{"interface":"front","average":{"1min":8.1460000000000008,"5min":8.1460000000000008,"15min":8.1460000000000008},"min":{"1min":1.181,"5min":1.181,"15min":1.181},"max":{"1min":91.165000000000006,"5min":91.165000000000006,"15min":91.165000000000006},"last":1.1519999999999999}]},{"osd":12,"last update":"Wed Sep 18 10:16:32 2024","interfaces":[{"interface":"back","average":{"1min":10.385,"5min":14.760999999999999,"15min":15.49},"min":{"1min":1.327,"5min":0.91100000000000003,"15min":0.91100000000000003},"max":{"1min":65.189999999999998,"5min":189.84999999999999,"15min":189.84999999999999},"last":1.4079999999999999},{"interface":"front","average":{"1min":10.818,"5min":14.696,"15min":15.342000000000001},"min":{"1min":1.4890000000000001,"5min":0.91200000000000003,"15min":0.91200000000000003},"max":{"1min":65.162000000000006,"5min":180.381,"15min":180.381},"last":1.7729999999999999}]},{"osd":13,"last update":"Wed Sep 18 10:16:32 2024","interfaces":[{"interface":"back","average":{"1min":10.321999999999999,"5min":23.059000000000001,"15min":25.181999999999999},"min":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"max":{"1min":166.001,"5min":180.59100000000001,"15min":180.59100000000001},"last":1.5720000000000001},{"interface":"front","average":{"1min":1.647,"5min":20.391999999999999,"15min":23.515999999999998},"min":{"1min":0.83399999999999996,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":2.3940000000000001,"5min":189.71299999999999,"15min":189.71299999999999},"last":1.3120000000000001}]},{"osd":15,"last update":"Wed Sep 18 10:16:32 2024","interfaces":[{"interface":"back","average":{"1min":4.181,"5min":13.381,"15min":14.914},"min":{"1min":1.5780000000000001,"5min":0.99399999999999999,"15min":0.99399999999999999},"max":{"1min":44.917000000000002,"5min":180.44800000000001,"15min":180.44800000000001},"last":1.6619999999999999},{"interface":"front","average":{"1min":1.7030000000000001,"5min":15.118,"15min":17.353999999999999},"min":{"1min":0.93200000000000005,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":2.1909999999999998,"5min":180.53,"15min":180.53},"last":1.8069999999999999}]}]},{"osd":13,"up_from":22,"seq":94489280544,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307932,"kb_used_data":2252,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94063908,"statfs":{"total":96636764160,"available":96321441792,"internally_reserved":0,"allocated":2306048,"data_stored":1341082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,3,6,8,9,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:16:33 2024","interfaces":[{"interface":"back","average":{"1min":2.2599999999999998,"5min":17.045000000000002,"15min":19.509},"min":{"1min":0.65700000000000003,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":22.972999999999999,"5min":176.66800000000001,"15min":176.66800000000001},"last":0.83899999999999997},{"interface":"front","average":{"1min":4.2629999999999999,"5min":25.199000000000002,"15min":28.687999999999999},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":62.533000000000001,"5min":199.732,"15min":199.732},"last":0.90000000000000002}]},{"osd":1,"last update":"Wed Sep 18 10:15:48 2024","interfaces":[{"interface":"back","average":{"1min":1.986,"5min":1.986,"15min":1.986},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":18.471,"5min":18.471,"15min":18.471},"last":1.4219999999999999},{"interface":"front","average":{"1min":20.716999999999999,"5min":20.716999999999999,"15min":20.716999999999999},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":176.56800000000001,"5min":176.56800000000001,"15min":176.56800000000001},"last":0.86399999999999999}]},{"osd":2,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":11.984,"5min":11.984,"15min":11.984},"min":{"1min":0.751,"5min":0.751,"15min":0.751},"max":{"1min":196.59200000000001,"5min":196.59200000000001,"15min":196.59200000000001},"last":0.97699999999999998},{"interface":"front","average":{"1min":26.109999999999999,"5min":26.109999999999999,"15min":26.109999999999999},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":177.03,"5min":177.03,"15min":177.03},"last":1.2689999999999999}]},{"osd":3,"last update":"Wed Sep 18 10:16:33 2024","interfaces":[{"interface":"back","average":{"1min":9.5,"5min":22.245000000000001,"15min":24.369},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":158.74100000000001,"5min":168.27699999999999,"15min":168.27699999999999},"last":1.27},{"interface":"front","average":{"1min":1.2290000000000001,"5min":9.8819999999999997,"15min":11.324},"min":{"1min":0.84499999999999997,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.8959999999999999,"5min":196.643,"15min":196.643},"last":0.95399999999999996}]},{"osd":4,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":29.802,"5min":29.802,"15min":29.802},"min":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"max":{"1min":168.48099999999999,"5min":168.48099999999999,"15min":168.48099999999999},"last":1.347},{"interface":"front","average":{"1min":8.1780000000000008,"5min":8.1780000000000008,"15min":8.1780000000000008},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":65.188000000000002,"5min":65.188000000000002,"15min":65.188000000000002},"last":1.6379999999999999}]},{"osd":5,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":26.152999999999999,"5min":26.152999999999999,"15min":26.152999999999999},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":176.48099999999999,"5min":176.48099999999999,"15min":176.48099999999999},"last":1.49},{"interface":"front","average":{"1min":24.885999999999999,"5min":24.885999999999999,"15min":24.885999999999999},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":167.62299999999999,"5min":167.62299999999999,"15min":167.62299999999999},"last":1.528}]},{"osd":6,"last update":"Wed Sep 18 10:16:33 2024","interfaces":[{"interface":"back","average":{"1min":2.6589999999999998,"5min":24.015000000000001,"15min":27.574999999999999},"min":{"1min":1.042,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":23.131,"5min":176.81200000000001,"15min":176.81200000000001},"last":1.1080000000000001},{"interface":"front","average":{"1min":4.7169999999999996,"5min":23.227,"15min":26.312000000000001},"min":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"max":{"1min":61.874000000000002,"5min":167.947,"15min":167.947},"last":1.3340000000000001}]},{"osd":8,"last update":"Wed Sep 18 10:16:50 2024","interfaces":[{"interface":"back","average":{"1min":1.5700000000000001,"5min":1.5700000000000001,"15min":1.5700000000000001},"min":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"max":{"1min":2.117,"5min":2.117,"15min":2.117},"last":1.175},{"interface":"front","average":{"1min":1.514,"5min":1.514,"15min":1.514},"min":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"max":{"1min":2.278,"5min":2.278,"15min":2.278},"last":1.149}]},{"osd":9,"last update":"Wed Sep 18 10:16:50 2024","interfaces":[{"interface":"back","average":{"1min":2.617,"5min":17.367000000000001,"15min":19.824999999999999},"min":{"1min":0.878,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":22.934000000000001,"5min":176.73599999999999,"15min":176.73599999999999},"last":1.206},{"interface":"front","average":{"1min":1.5660000000000001,"5min":19.085000000000001,"15min":22.004999999999999},"min":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":2.448,"5min":167.84899999999999,"15min":167.84899999999999},"last":1.446}]},{"osd":10,"last update":"Wed Sep 18 10:16:39 2024","interfaces":[{"interface":"back","average":{"1min":5.1120000000000001,"5min":19.164999999999999,"15min":21.507000000000001},"min":{"1min":1.22,"5min":1.1359999999999999,"15min":1.1359999999999999},"max":{"1min":62.137999999999998,"5min":168.07900000000001,"15min":168.07900000000001},"last":1.591},{"interface":"front","average":{"1min":17.683,"5min":29.114000000000001,"15min":31.018999999999998},"min":{"1min":1.123,"5min":1.0649999999999999,"15min":1.0649999999999999},"max":{"1min":289.68700000000001,"5min":289.68700000000001,"15min":289.68700000000001},"last":1.2450000000000001}]},{"osd":11,"last update":"Wed Sep 18 10:16:50 2024","interfaces":[{"interface":"back","average":{"1min":1.6819999999999999,"5min":1.625,"15min":1.615},"min":{"1min":1.056,"5min":1.0029999999999999,"15min":1.0029999999999999},"max":{"1min":2.5699999999999998,"5min":2.5699999999999998,"15min":2.5699999999999998},"last":1.5009999999999999},{"interface":"front","average":{"1min":2.798,"5min":21.73,"15min":24.885999999999999},"min":{"1min":1.286,"5min":1.028,"15min":1.028},"max":{"1min":23.167999999999999,"5min":176.94999999999999,"15min":176.94999999999999},"last":1.377}]},{"osd":12,"last update":"Wed Sep 18 10:16:33 2024","interfaces":[{"interface":"back","average":{"1min":1.8440000000000001,"5min":1.6339999999999999,"15min":1.599},"min":{"1min":1.2490000000000001,"5min":1.0009999999999999,"15min":1.0009999999999999},"max":{"1min":2.5950000000000002,"5min":2.7879999999999998,"15min":2.7879999999999998},"last":1.413},{"interface":"front","average":{"1min":5.0570000000000004,"5min":18.542999999999999,"15min":20.791},"min":{"1min":1.3029999999999999,"5min":1.296,"15min":1.296},"max":{"1min":62.200000000000003,"5min":168.34399999999999,"15min":168.34399999999999},"last":1.5449999999999999}]},{"osd":14,"last update":"Wed Sep 18 10:16:33 2024","interfaces":[{"interface":"back","average":{"1min":1.8320000000000001,"5min":5.7210000000000001,"15min":6.3689999999999998},"min":{"1min":1.139,"5min":1.0640000000000001,"15min":1.0640000000000001},"max":{"1min":2.419,"5min":87.427000000000007,"15min":87.427000000000007},"last":1.665},{"interface":"front","average":{"1min":2.9329999999999998,"5min":21.571999999999999,"15min":24.678000000000001},"min":{"1min":1.2709999999999999,"5min":1.0980000000000001,"15min":1.0980000000000001},"max":{"1min":23.268000000000001,"5min":176.87700000000001,"15min":176.87700000000001},"last":1.468}]},{"osd":15,"last update":"Wed Sep 18 10:16:33 2024","interfaces":[{"interface":"back","average":{"1min":10.048999999999999,"5min":18.277000000000001,"15min":19.648},"min":{"1min":1.272,"5min":1.266,"15min":1.266},"max":{"1min":96.564999999999998,"5min":168.40700000000001,"15min":168.40700000000001},"last":1.7110000000000001},{"interface":"front","average":{"1min":1.891,"5min":13.459,"15min":15.387},"min":{"1min":1.1950000000000001,"5min":0.95099999999999996,"15min":0.95099999999999996},"max":{"1min":2.5169999999999999,"5min":196.512,"15min":196.512},"last":1.5609999999999999}]}]},{"osd":0,"up_from":21,"seq":90194313248,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303780,"kb_used_data":2264,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068060,"statfs":{"total":96636764160,"available":96325693440,"internally_reserved":0,"allocated":2318336,"data_stored":1350579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,4,5,6,9,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 18 10:16:39 2024","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.95699999999999996,"15min":0.95899999999999996},"min":{"1min":0.56699999999999995,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.2889999999999999,"5min":1.341,"15min":1.341},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.97899999999999998,"15min":0.98499999999999999},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.2310000000000001,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":0.625}]},{"osd":2,"last update":"Wed Sep 18 10:16:39 2024","interfaces":[{"interface":"back","average":{"1min":3.2719999999999998,"5min":1.6020000000000001,"15min":1.3240000000000001},"min":{"1min":0.76600000000000001,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":46.220999999999997,"5min":46.220999999999997,"15min":46.220999999999997},"last":1.0349999999999999},{"interface":"front","average":{"1min":0.98799999999999999,"5min":1.071,"15min":1.085},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.732,"5min":1.732,"15min":1.732},"last":0.57499999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.196}]},{"osd":4,"last update":"Wed Sep 18 10:16:30 2024","interfaces":[{"interface":"back","average":{"1min":8.4039999999999999,"5min":14,"15min":14.933},"min":{"1min":0.59699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":142.20699999999999,"5min":142.20699999999999,"15min":142.20699999999999},"last":0.49399999999999999},{"interface":"front","average":{"1min":10.48,"5min":14.896000000000001,"15min":15.632},"min":{"1min":0.63,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":132.33799999999999,"5min":132.33799999999999,"15min":132.33799999999999},"last":0.55200000000000005}]},{"osd":5,"last update":"Wed Sep 18 10:16:39 2024","interfaces":[{"interface":"back","average":{"1min":1.1739999999999999,"5min":1.24,"15min":1.2509999999999999},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":1.619,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":1.569,"5min":8.1340000000000003,"15min":9.2279999999999998},"min":{"1min":0.85599999999999998,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":10.143000000000001,"5min":130.39599999999999,"15min":130.39599999999999},"last":0.92100000000000004}]},{"osd":6,"last update":"Wed Sep 18 10:16:47 2024","interfaces":[{"interface":"back","average":{"1min":1.266,"5min":1.254,"15min":1.252},"min":{"1min":0.96099999999999997,"5min":0.94299999999999995,"15min":0.94299999999999995},"max":{"1min":1.571,"5min":1.776,"15min":1.776},"last":0.82599999999999996},{"interface":"front","average":{"1min":1.3109999999999999,"5min":2.0270000000000001,"15min":2.1469999999999998},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.7909999999999999,"5min":15.250999999999999,"15min":15.250999999999999},"last":0.66100000000000003}]},{"osd":9,"last update":"Wed Sep 18 10:16:30 2024","interfaces":[{"interface":"back","average":{"1min":1.3360000000000001,"5min":1.3919999999999999,"15min":1.4019999999999999},"min":{"1min":0.88200000000000001,"5min":0.879,"15min":0.879},"max":{"1min":1.863,"5min":1.863,"15min":1.863},"last":0.74099999999999999},{"interface":"front","average":{"1min":1.667,"5min":8.7590000000000003,"15min":9.9410000000000007},"min":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":10.180999999999999,"5min":130.441,"15min":130.441},"last":0.88400000000000001}]},{"osd":11,"last update":"Wed Sep 18 10:16:30 2024","interfaces":[{"interface":"back","average":{"1min":4.8949999999999996,"5min":2.2109999999999999,"15min":1.7629999999999999},"min":{"1min":1.1579999999999999,"5min":0.90700000000000003,"15min":0.90700000000000003},"max":{"1min":41.610999999999997,"5min":41.610999999999997,"15min":41.610999999999997},"last":1.0700000000000001},{"interface":"front","average":{"1min":1.5029999999999999,"5min":4.5620000000000003,"15min":5.0720000000000001},"min":{"1min":1.1220000000000001,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":1.9850000000000001,"5min":54.805999999999997,"15min":54.805999999999997},"last":0.87}]},{"osd":12,"last update":"Wed Sep 18 10:16:30 2024","interfaces":[{"interface":"back","average":{"1min":1.7869999999999999,"5min":9.1170000000000009,"15min":10.339},"min":{"1min":0.95999999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":10.015000000000001,"5min":130.34800000000001,"15min":130.34800000000001},"last":0.95399999999999996},{"interface":"front","average":{"1min":1.4350000000000001,"5min":1.4710000000000001,"15min":1.4770000000000001},"min":{"1min":0.85899999999999999,"5min":0.502,"15min":0.502},"max":{"1min":1.944,"5min":1.944,"15min":1.944},"last":0.85499999999999998}]},{"osd":14,"last update":"Wed Sep 18 10:16:39 2024","interfaces":[{"interface":"back","average":{"1min":3.7490000000000001,"5min":8.7810000000000006,"15min":9.6199999999999992},"min":{"1min":1.044,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":38.969999999999999,"5min":130.48500000000001,"15min":130.48500000000001},"last":0.98699999999999999},{"interface":"front","average":{"1min":6.931,"5min":2.6339999999999999,"15min":1.9179999999999999},"min":{"1min":1.2609999999999999,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":80.460999999999999,"5min":80.460999999999999,"15min":80.460999999999999},"last":1.099}]},{"osd":15,"last update":"Wed Sep 18 10:16:39 2024","interfaces":[{"interface":"back","average":{"1min":1.591,"5min":2.778,"15min":2.976},"min":{"1min":1.1699999999999999,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":2.0310000000000001,"5min":23.806999999999999,"15min":23.806999999999999},"last":0.77200000000000002},{"interface":"front","average":{"1min":6.9199999999999999,"5min":9.4809999999999999,"15min":9.9079999999999995},"min":{"1min":1.1279999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":100.925,"5min":130.52799999999999,"15min":130.52799999999999},"last":1.018}]}]},{"osd":4,"up_from":21,"seq":90194313249,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303752,"kb_used_data":2232,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068088,"statfs":{"total":96636764160,"available":96325722112,"internally_reserved":0,"allocated":2285568,"data_stored":1331589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,3,5,7,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":9.0980000000000008,"15min":10.465999999999999},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.7529999999999999,"5min":89.266999999999996,"15min":89.266999999999996},"last":0.626},{"interface":"front","average":{"1min":18.640000000000001,"5min":7.7320000000000002,"15min":5.9139999999999997},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":198.80699999999999,"5min":198.80699999999999,"15min":198.80699999999999},"last":0.66000000000000003}]},{"osd":1,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":4.0759999999999996,"5min":12.537000000000001,"15min":13.946999999999999},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":62.780000000000001,"5min":164.21199999999999,"15min":164.21199999999999},"last":0.72099999999999997},{"interface":"front","average":{"1min":2.5139999999999998,"5min":9.593,"15min":10.773},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":25.802,"5min":89.331000000000003,"15min":89.331000000000003},"last":0.77500000000000002}]},{"osd":2,"last update":"Wed Sep 18 10:15:47 2024","interfaces":[{"interface":"back","average":{"1min":23.617000000000001,"5min":23.617000000000001,"15min":23.617000000000001},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":192.136,"5min":192.136,"15min":192.136},"last":1.8740000000000001},{"interface":"front","average":{"1min":11.888999999999999,"5min":11.888999999999999,"15min":11.888999999999999},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":198.87700000000001,"5min":198.87700000000001,"15min":198.87700000000001},"last":1.347}]},{"osd":3,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":35.673000000000002,"5min":21.670999999999999,"15min":19.337},"min":{"1min":1.161,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":388.84300000000002,"5min":388.84300000000002,"15min":388.84300000000002},"last":1.462},{"interface":"front","average":{"1min":17.747,"5min":25.643999999999998,"15min":26.960000000000001},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":255.82900000000001,"5min":255.82900000000001,"15min":255.82900000000001},"last":0.997}]},{"osd":5,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":1.4910000000000001,"5min":15.449999999999999,"15min":17.776},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":2.169,"5min":189.55600000000001,"15min":189.55600000000001},"last":0.83099999999999996},{"interface":"front","average":{"1min":21.053000000000001,"5min":6.8079999999999998,"15min":4.4340000000000002},"min":{"1min":1.3640000000000001,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":199.15600000000001,"5min":199.15600000000001,"15min":199.15600000000001},"last":1.155}]},{"osd":6,"last update":"Wed Sep 18 10:15:34 2024","interfaces":[{"interface":"back","average":{"1min":32.25,"5min":32.25,"15min":32.25},"min":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":197.33199999999999,"5min":197.33199999999999,"15min":197.33199999999999},"last":1.8160000000000001},{"interface":"front","average":{"1min":18.963000000000001,"5min":18.963000000000001,"15min":18.963000000000001},"min":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"max":{"1min":179.19200000000001,"5min":179.19200000000001,"15min":179.19200000000001},"last":1.7769999999999999}]},{"osd":7,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":5.6050000000000004,"5min":23.198,"15min":26.129999999999999},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":62.704999999999998,"5min":290.036,"15min":290.036},"last":1.2689999999999999},{"interface":"front","average":{"1min":1.6639999999999999,"5min":14.273,"15min":16.375},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":2.3100000000000001,"5min":139.696,"15min":139.696},"last":0.93899999999999995}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1509999999999998}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7509999999999999}]},{"osd":10,"last update":"Wed Sep 18 10:16:49 2024","interfaces":[{"interface":"back","average":{"1min":1.653,"5min":18.559000000000001,"15min":21.376999999999999},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":2.4740000000000002,"5min":164.43899999999999,"15min":164.43899999999999},"last":0.89800000000000002},{"interface":"front","average":{"1min":4.6070000000000002,"5min":10.993,"15min":12.057},"min":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":59.408999999999999,"5min":89.442999999999998,"15min":89.442999999999998},"last":1.3049999999999999}]},{"osd":11,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":19.57,"5min":12.567,"15min":11.4},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":199.255,"5min":199.255,"15min":199.255},"last":1.3340000000000001},{"interface":"front","average":{"1min":11.393000000000001,"5min":22.213000000000001,"15min":24.016999999999999},"min":{"1min":1.0840000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"max":{"1min":155.19800000000001,"5min":164.733,"15min":164.733},"last":1.4399999999999999}]},{"osd":12,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":4.5800000000000001,"5min":10.406000000000001,"15min":11.377000000000001},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":58.856000000000002,"5min":89.400000000000006,"15min":89.400000000000006},"last":1.113},{"interface":"front","average":{"1min":31.831,"5min":12.901999999999999,"15min":9.7469999999999999},"min":{"1min":1.4530000000000001,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":199.32900000000001,"5min":199.32900000000001,"15min":199.32900000000001},"last":1.5940000000000001}]},{"osd":13,"last update":"Wed Sep 18 10:16:49 2024","interfaces":[{"interface":"back","average":{"1min":12.695,"5min":22.789999999999999,"15min":24.472000000000001},"min":{"1min":1.702,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":192.37299999999999,"5min":199.37799999999999,"15min":199.37799999999999},"last":1.706},{"interface":"front","average":{"1min":1.8029999999999999,"5min":19.068000000000001,"15min":21.946000000000002},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":2.5699999999999998,"5min":164.53,"15min":164.53},"last":1.375}]},{"osd":14,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":22.312000000000001,"5min":21.788,"15min":21.701000000000001},"min":{"1min":1.556,"5min":0.96099999999999997,"15min":0.96099999999999997},"max":{"1min":199.43000000000001,"5min":202.191,"15min":202.191},"last":1.782},{"interface":"front","average":{"1min":9.8800000000000008,"5min":17.420999999999999,"15min":18.678000000000001},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":101.492,"5min":164.35400000000001,"15min":164.35400000000001},"last":1.524}]}]},{"osd":9,"up_from":21,"seq":90194313248,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303784,"kb_used_data":2264,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068056,"statfs":{"total":96636764160,"available":96325689344,"internally_reserved":0,"allocated":2318336,"data_stored":1350579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,4,5,6,8,10,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":7.2450000000000001,"5min":7.2450000000000001,"15min":7.2450000000000001},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":127.902,"5min":127.902,"15min":127.902},"last":0.97399999999999998},{"interface":"front","average":{"1min":8.3109999999999999,"5min":8.3109999999999999,"15min":8.3109999999999999},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":96.203000000000003,"5min":96.203000000000003,"15min":96.203000000000003},"last":0.81200000000000006}]},{"osd":1,"last update":"Wed Sep 18 10:16:40 2024","interfaces":[{"interface":"back","average":{"1min":10.587,"5min":10.587,"15min":10.587},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":166.833,"5min":166.833,"15min":166.833},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.96099999999999997}]},{"osd":2,"last update":"Wed Sep 18 10:16:18 2024","interfaces":[{"interface":"back","average":{"1min":5.6420000000000003,"5min":5.6420000000000003,"15min":5.6420000000000003},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":68.049999999999997,"5min":68.049999999999997,"15min":68.049999999999997},"last":1.1579999999999999},{"interface":"front","average":{"1min":2.0310000000000001,"5min":2.0310000000000001,"15min":2.0310000000000001},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":18.553999999999998,"5min":18.553999999999998,"15min":18.553999999999998},"last":1.4119999999999999}]},{"osd":3,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":2.875,"5min":5.5549999999999997,"15min":6.0019999999999998},"min":{"1min":0.72699999999999998,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":18.431000000000001,"5min":98.888999999999996,"15min":98.888999999999996},"last":1.0389999999999999},{"interface":"front","average":{"1min":3.0169999999999999,"5min":5.5890000000000004,"15min":6.0170000000000003},"min":{"1min":0.749,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":18.472000000000001,"5min":98.947999999999993,"15min":98.947999999999993},"last":1.546}]},{"osd":4,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":8.2360000000000007,"5min":2.5840000000000001,"15min":1.6419999999999999},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":124.15300000000001,"5min":124.15300000000001,"15min":124.15300000000001},"last":1.371},{"interface":"front","average":{"1min":6.2380000000000004,"5min":8.8539999999999992,"15min":9.2899999999999991},"min":{"1min":0.81000000000000005,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":73.992000000000004,"5min":73.992000000000004,"15min":73.992000000000004},"last":1.6319999999999999}]},{"osd":5,"last update":"Wed Sep 18 10:16:18 2024","interfaces":[{"interface":"back","average":{"1min":1.821,"5min":1.821,"15min":1.821},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":14.786,"5min":14.786,"15min":14.786},"last":0.79800000000000004},{"interface":"front","average":{"1min":8.5,"5min":8.5,"15min":8.5},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":67.915999999999997,"5min":67.915999999999997,"15min":67.915999999999997},"last":1.27}]},{"osd":6,"last update":"Wed Sep 18 10:16:18 2024","interfaces":[{"interface":"back","average":{"1min":8.3840000000000003,"5min":8.3840000000000003,"15min":8.3840000000000003},"min":{"1min":0.997,"5min":0.997,"15min":0.997},"max":{"1min":70.914000000000001,"5min":70.914000000000001,"15min":70.914000000000001},"last":1.806},{"interface":"front","average":{"1min":15.705,"5min":15.705,"15min":15.705},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":136.38800000000001,"5min":136.38800000000001,"15min":136.38800000000001},"last":1.2}]},{"osd":8,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":3.0310000000000001,"5min":4.2789999999999999,"15min":4.4870000000000001},"min":{"1min":1.0509999999999999,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":30.713000000000001,"5min":64.912999999999997,"15min":64.912999999999997},"last":1.458},{"interface":"front","average":{"1min":2.0569999999999999,"5min":5.3959999999999999,"15min":5.952},"min":{"1min":0.82499999999999996,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":12.898999999999999,"5min":57.664000000000001,"15min":57.664000000000001},"last":1.857}]},{"osd":10,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":1.3400000000000001,"5min":1.3089999999999999,"15min":1.304},"min":{"1min":0.88300000000000001,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.8180000000000001,"5min":2.109,"15min":2.109},"last":1.51},{"interface":"front","average":{"1min":6.0389999999999997,"5min":4.2309999999999999,"15min":3.9300000000000002},"min":{"1min":0.97899999999999998,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":68.216999999999999,"5min":68.216999999999999,"15min":68.216999999999999},"last":1.661}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.669}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.446}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":96}]},{"osd":14,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":4.3579999999999997,"5min":8.8759999999999994,"15min":9.6289999999999996},"min":{"1min":0.86499999999999999,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":50.773000000000003,"5min":62.899000000000001,"15min":62.899000000000001},"last":1.9039999999999999},{"interface":"front","average":{"1min":4.3540000000000001,"5min":13.071,"15min":14.523999999999999},"min":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":50.709000000000003,"5min":155.61000000000001,"15min":155.61000000000001},"last":1.6950000000000001}]},{"osd":15,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":1.4630000000000001,"5min":1.492,"15min":1.4970000000000001},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.97,"5min":1.9970000000000001,"15min":1.9970000000000001},"last":1.605},{"interface":"front","average":{"1min":9.5700000000000003,"5min":5.1749999999999998,"15min":4.4429999999999996},"min":{"1min":1.0980000000000001,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":68.140000000000001,"5min":68.140000000000001,"15min":68.140000000000001},"last":1.7470000000000001}]}]},{"osd":11,"up_from":21,"seq":90194313248,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303784,"kb_used_data":2264,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068056,"statfs":{"total":96636764160,"available":96325689344,"internally_reserved":0,"allocated":2318336,"data_stored":1350579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[2,3,6,8,9,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0049999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86599999999999999}]},{"osd":2,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":8.3390000000000004,"15min":9.5719999999999992},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.3740000000000001,"5min":99.718000000000004,"15min":99.718000000000004},"last":0.88200000000000001},{"interface":"front","average":{"1min":3.6349999999999998,"5min":1.4450000000000001,"15min":1.0800000000000001},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":49.774000000000001,"5min":49.774000000000001,"15min":49.774000000000001},"last":0.93400000000000005}]},{"osd":3,"last update":"Wed Sep 18 10:16:46 2024","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":4.2210000000000001,"15min":4.7880000000000003},"min":{"1min":0.48399999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.369,"5min":85.963999999999999,"15min":85.963999999999999},"last":0.82299999999999995},{"interface":"front","average":{"1min":5.4660000000000002,"5min":19.300000000000001,"15min":21.606000000000002},"min":{"1min":0.73699999999999999,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":80.980999999999995,"5min":125.672,"15min":125.672},"last":1.073}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.444}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5980000000000001}]},{"osd":6,"last update":"Wed Sep 18 10:16:46 2024","interfaces":[{"interface":"back","average":{"1min":5.8810000000000002,"5min":2.101,"15min":1.4710000000000001},"min":{"1min":0.70599999999999996,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":86.680000000000007,"5min":86.680000000000007,"15min":86.680000000000007},"last":1.21},{"interface":"front","average":{"1min":1.1830000000000001,"5min":5.827,"15min":6.601},"min":{"1min":0.73799999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.49,"5min":94.162000000000006,"15min":94.162000000000006},"last":1.109}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.524}]},{"osd":8,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":2.7069999999999999,"5min":5.3730000000000002,"15min":5.8170000000000002},"min":{"1min":0.95299999999999996,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":27.577999999999999,"5min":85.900000000000006,"15min":85.900000000000006},"last":1.0189999999999999},{"interface":"front","average":{"1min":2.544,"5min":8.4960000000000004,"15min":9.4879999999999995},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":27.542000000000002,"5min":99.765000000000001,"15min":99.765000000000001},"last":1.1559999999999999}]},{"osd":9,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":5.8760000000000003,"5min":5.8339999999999996,"15min":5.827},"min":{"1min":0.96399999999999997,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":89.844999999999999,"5min":89.844999999999999,"15min":89.844999999999999},"last":1.1779999999999999},{"interface":"front","average":{"1min":1.351,"5min":6.8070000000000004,"15min":7.7160000000000002},"min":{"1min":0.68600000000000005,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.742,"5min":99.810000000000002,"15min":99.810000000000002},"last":1.2470000000000001}]},{"osd":10,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":4.0069999999999997,"5min":1.679,"15min":1.2909999999999999},"min":{"1min":0.98499999999999999,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":49.819000000000003,"5min":49.819000000000003,"15min":49.819000000000003},"last":1.361},{"interface":"front","average":{"1min":1.5580000000000001,"5min":6.46,"15min":7.2770000000000001},"min":{"1min":0.98699999999999999,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.976,"5min":89.290000000000006,"15min":89.290000000000006},"last":1.274}]},{"osd":12,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":5.907,"5min":2.081,"15min":1.4430000000000001},"min":{"1min":0.96599999999999997,"5min":0.498,"15min":0.498},"max":{"1min":87.463999999999999,"5min":87.463999999999999,"15min":87.463999999999999},"last":1.7270000000000001},{"interface":"front","average":{"1min":1.5660000000000001,"5min":8.6219999999999999,"15min":9.798},"min":{"1min":1.0700000000000001,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":2.0600000000000001,"5min":86.025999999999996,"15min":86.025999999999996},"last":1.494}]},{"osd":13,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":4.4210000000000003,"5min":25.207999999999998,"15min":28.672000000000001},"min":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"max":{"1min":59.841999999999999,"5min":189.25899999999999,"15min":189.25899999999999},"last":1.476},{"interface":"front","average":{"1min":7.569,"5min":2.5659999999999998,"15min":1.732},"min":{"1min":0.88900000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":71.313000000000002,"5min":71.313000000000002,"15min":71.313000000000002},"last":1.6439999999999999}]},{"osd":14,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":14.388999999999999,"5min":8.016,"15min":6.9539999999999997},"min":{"1min":1.0129999999999999,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":256.13400000000001,"5min":256.13400000000001,"15min":256.13400000000001},"last":1.45},{"interface":"front","average":{"1min":2.7650000000000001,"5min":7.7380000000000004,"15min":8.5670000000000002},"min":{"1min":0.874,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":25.791,"5min":102.625,"15min":102.625},"last":1.4330000000000001}]},{"osd":15,"last update":"Wed Sep 18 10:16:40 2024","interfaces":[{"interface":"back","average":{"1min":4.0309999999999997,"5min":4.0309999999999997,"15min":4.0309999999999997},"min":{"1min":1.153,"5min":1.153,"15min":1.153},"max":{"1min":49.927999999999997,"5min":49.927999999999997,"15min":49.927999999999997},"last":1.518},{"interface":"front","average":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"min":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"max":{"1min":2.044,"5min":2.044,"15min":2.044},"last":1.407}]}]},{"osd":12,"up_from":21,"seq":90194313249,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303784,"kb_used_data":2268,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068056,"statfs":{"total":96636764160,"available":96325689344,"internally_reserved":0,"allocated":2322432,"data_stored":1350598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[3,4,6,7,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":2.3839999999999999,"5min":2.3839999999999999,"15min":2.3839999999999999},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":32.476999999999997,"5min":32.476999999999997,"15min":32.476999999999997},"last":0.64200000000000002},{"interface":"front","average":{"1min":2.625,"5min":2.625,"15min":2.625},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":32.509,"5min":32.509,"15min":32.509},"last":0.89300000000000002}]},{"osd":1,"last update":"Wed Sep 18 10:15:32 2024","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.6399999999999999,"5min":1.6399999999999999,"15min":1.6399999999999999},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.905,"5min":1.905,"15min":1.905},"last":0.81899999999999995}]},{"osd":2,"last update":"Wed Sep 18 10:15:32 2024","interfaces":[{"interface":"back","average":{"1min":1.002,"5min":1.002,"15min":1.002},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":1.153},{"interface":"front","average":{"1min":4.1189999999999998,"5min":4.1189999999999998,"15min":4.1189999999999998},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":62.780999999999999,"5min":62.780999999999999,"15min":62.780999999999999},"last":0.96499999999999997}]},{"osd":3,"last update":"Wed Sep 18 10:16:33 2024","interfaces":[{"interface":"back","average":{"1min":5.9429999999999996,"5min":2.1040000000000001,"15min":1.464},"min":{"1min":0.627,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":74.822999999999993,"5min":74.822999999999993,"15min":74.822999999999993},"last":1.1399999999999999},{"interface":"front","average":{"1min":8.8200000000000003,"5min":2.6840000000000002,"15min":1.6619999999999999},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":94.320999999999998,"5min":94.320999999999998,"15min":94.320999999999998},"last":0.87}]},{"osd":4,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":16.058,"5min":4.0529999999999999,"15min":2.052},"min":{"1min":0.53100000000000003,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":114.184,"5min":114.184,"15min":114.184},"last":0.82199999999999995},{"interface":"front","average":{"1min":5.8920000000000003,"5min":2.1560000000000001,"15min":1.5329999999999999},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":74.698999999999998,"5min":74.698999999999998,"15min":74.698999999999998},"last":0.94299999999999995}]},{"osd":6,"last update":"Wed Sep 18 10:16:38 2024","interfaces":[{"interface":"back","average":{"1min":10.747999999999999,"5min":10.747999999999999,"15min":10.747999999999999},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":130.20699999999999,"5min":130.20699999999999,"15min":130.20699999999999},"last":1.0660000000000001},{"interface":"front","average":{"1min":14.148,"5min":14.148,"15min":14.148},"min":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":132.60900000000001,"5min":132.60900000000001,"15min":132.60900000000001},"last":1.5329999999999999}]},{"osd":7,"last update":"Wed Sep 18 10:16:38 2024","interfaces":[{"interface":"back","average":{"1min":4.774,"5min":4.774,"15min":4.774},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":90.188000000000002,"5min":90.188000000000002,"15min":90.188000000000002},"last":1.208},{"interface":"front","average":{"1min":6.0359999999999996,"5min":6.0359999999999996,"15min":6.0359999999999996},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":75.061999999999998,"5min":75.061999999999998,"15min":75.061999999999998},"last":1.3089999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":94.382000000000005}]},{"osd":9,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":8.9719999999999995,"5min":2.8079999999999998,"15min":1.7809999999999999},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":94.421000000000006,"5min":94.421000000000006,"15min":94.421000000000006},"last":1.3799999999999999},{"interface":"front","average":{"1min":5.0410000000000004,"5min":2.0179999999999998,"15min":1.514},"min":{"1min":0.70199999999999996,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":90.242999999999995,"5min":90.242999999999995,"15min":90.242999999999995},"last":1.47}]},{"osd":10,"last update":"Wed Sep 18 10:16:38 2024","interfaces":[{"interface":"back","average":{"1min":15.145,"5min":15.145,"15min":15.145},"min":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":99.251000000000005,"5min":99.251000000000005,"15min":99.251000000000005},"last":1.4359999999999999},{"interface":"front","average":{"1min":8.7080000000000002,"5min":8.7080000000000002,"15min":8.7080000000000002},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":94.468000000000004,"5min":94.468000000000004,"15min":94.468000000000004},"last":1.5700000000000001}]},{"osd":11,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":8.9269999999999996,"5min":2.8769999999999998,"15min":1.869},"min":{"1min":0.91300000000000003,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":74.888999999999996,"5min":74.888999999999996,"15min":74.888999999999996},"last":1.8899999999999999},{"interface":"front","average":{"1min":9.1590000000000007,"5min":2.9380000000000002,"15min":1.901},"min":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"max":{"1min":94.546000000000006,"5min":94.546000000000006,"15min":94.546000000000006},"last":1.798}]},{"osd":13,"last update":"Wed Sep 18 10:16:33 2024","interfaces":[{"interface":"back","average":{"1min":7.4779999999999998,"5min":2.5619999999999998,"15min":1.742},"min":{"1min":0.79900000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":143.672,"5min":143.672,"15min":143.672},"last":1.671},{"interface":"front","average":{"1min":12.677,"5min":3.6779999999999999,"15min":2.1779999999999999},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":143.619,"5min":143.619,"15min":143.619},"last":1.744}]},{"osd":14,"last update":"Wed Sep 18 10:16:33 2024","interfaces":[{"interface":"back","average":{"1min":13.363,"5min":3.8159999999999998,"15min":2.2250000000000001},"min":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"max":{"1min":94.504999999999995,"5min":94.504999999999995,"15min":94.504999999999995},"last":1.6879999999999999},{"interface":"front","average":{"1min":12.039,"5min":3.52,"15min":2.1000000000000001},"min":{"1min":0.95099999999999996,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":92.593000000000004,"5min":92.593000000000004,"15min":92.593000000000004},"last":1.615}]},{"osd":15,"last update":"Wed Sep 18 10:16:33 2024","interfaces":[{"interface":"back","average":{"1min":6.5339999999999998,"5min":2.5019999999999998,"15min":1.8300000000000001},"min":{"1min":0.751,"5min":0.751,"15min":0.751},"max":{"1min":74.966999999999999,"5min":74.966999999999999,"15min":74.966999999999999},"last":2.028},{"interface":"front","average":{"1min":14.669,"5min":4.117,"15min":2.359},"min":{"1min":0.996,"5min":0.95299999999999996,"15min":0.95299999999999996},"max":{"1min":152.11000000000001,"5min":152.11000000000001,"15min":152.11000000000001},"last":1.9219999999999999}]}]},{"osd":1,"up_from":22,"seq":94489280543,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":303748,"kb_used_data":2232,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068092,"statfs":{"total":96636764160,"available":96325726208,"internally_reserved":0,"allocated":2285568,"data_stored":1331589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,6,9,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:16:38 2024","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":2.472,"15min":2.7370000000000001},"min":{"1min":0.65800000000000003,"5min":0.629,"15min":0.629},"max":{"1min":1.1930000000000001,"5min":36.268000000000001,"15min":36.268000000000001},"last":1.127},{"interface":"front","average":{"1min":5.3659999999999997,"5min":15.244999999999999,"15min":16.890999999999998},"min":{"1min":0.73099999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":66.534999999999997,"5min":154.67599999999999,"15min":154.67599999999999},"last":0.70699999999999996}]},{"osd":2,"last update":"Wed Sep 18 10:16:38 2024","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.91400000000000003,"15min":0.90400000000000003},"min":{"1min":0.71499999999999997,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.4119999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.98999999999999999,"5min":2.5539999999999998,"15min":2.8140000000000001},"min":{"1min":0.75600000000000001,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.212,"5min":36.323999999999998,"15min":36.323999999999998},"last":0.76400000000000001}]},{"osd":3,"last update":"Wed Sep 18 10:16:38 2024","interfaces":[{"interface":"back","average":{"1min":5.548,"5min":10.811999999999999,"15min":11.689},"min":{"1min":0.94599999999999995,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":66.344999999999999,"5min":150.642,"15min":150.642},"last":0.96799999999999997},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.0649999999999999,"15min":1.0509999999999999},"min":{"1min":0.82999999999999996,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.6830000000000001,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":1.081}]},{"osd":4,"last update":"Wed Sep 18 10:15:35 2024","interfaces":[{"interface":"back","average":{"1min":3.3439999999999999,"5min":3.3439999999999999,"15min":3.3439999999999999},"min":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":40.783000000000001,"5min":40.783000000000001,"15min":40.783000000000001},"last":1.7210000000000001},{"interface":"front","average":{"1min":4.1699999999999999,"5min":4.1699999999999999,"15min":4.1699999999999999},"min":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":50.747,"5min":50.747,"15min":50.747},"last":1.518}]},{"osd":5,"last update":"Wed Sep 18 10:15:35 2024","interfaces":[{"interface":"back","average":{"1min":5.9720000000000004,"5min":5.9720000000000004,"15min":5.9720000000000004},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":52.317999999999998,"5min":52.317999999999998,"15min":52.317999999999998},"last":1.2649999999999999},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.748,"5min":1.748,"15min":1.748},"last":1.839}]},{"osd":6,"last update":"Wed Sep 18 10:16:38 2024","interfaces":[{"interface":"back","average":{"1min":9.2970000000000006,"5min":8.5380000000000003,"15min":8.4120000000000008},"min":{"1min":0.85099999999999998,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":120.85299999999999,"5min":120.85299999999999,"15min":120.85299999999999},"last":1.1779999999999999},{"interface":"front","average":{"1min":2.6040000000000001,"5min":8.8019999999999996,"15min":9.8350000000000009},"min":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"max":{"1min":20.600000000000001,"5min":85.045000000000002,"15min":85.045000000000002},"last":1.266}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":118.43600000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7789999999999999}]},{"osd":9,"last update":"Wed Sep 18 10:16:15 2024","interfaces":[{"interface":"back","average":{"1min":3.4369999999999998,"5min":3.4369999999999998,"15min":3.4369999999999998},"min":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"max":{"1min":36.506999999999998,"5min":36.506999999999998,"15min":36.506999999999998},"last":0.998},{"interface":"front","average":{"1min":11.339,"5min":11.339,"15min":11.339},"min":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"max":{"1min":165.13499999999999,"5min":165.13499999999999,"15min":165.13499999999999},"last":1.0429999999999999}]},{"osd":10,"last update":"Wed Sep 18 10:16:38 2024","interfaces":[{"interface":"back","average":{"1min":1.5209999999999999,"5min":3.048,"15min":3.302},"min":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"max":{"1min":2.0489999999999999,"5min":36.436,"15min":36.436},"last":1.3799999999999999},{"interface":"front","average":{"1min":6.0839999999999996,"5min":10.747999999999999,"15min":11.526},"min":{"1min":1.2769999999999999,"5min":0.88,"15min":0.88},"max":{"1min":66.673000000000002,"5min":100.402,"15min":100.402},"last":1.155}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.825}]},{"osd":12,"last update":"Wed Sep 18 10:16:38 2024","interfaces":[{"interface":"back","average":{"1min":1.5549999999999999,"5min":3.681,"15min":4.0350000000000001},"min":{"1min":1.101,"5min":1.0589999999999999,"15min":1.0589999999999999},"max":{"1min":2.294,"5min":49.976999999999997,"15min":49.976999999999997},"last":1.323},{"interface":"front","average":{"1min":2.0369999999999999,"5min":7.1269999999999998,"15min":7.9749999999999996},"min":{"1min":1.196,"5min":1.196,"15min":1.196},"max":{"1min":8.4049999999999994,"5min":124.78100000000001,"15min":124.78100000000001},"last":1.4419999999999999}]},{"osd":13,"last update":"Wed Sep 18 10:16:15 2024","interfaces":[{"interface":"back","average":{"1min":11.616,"5min":11.616,"15min":11.616},"min":{"1min":1.232,"5min":1.232,"15min":1.232},"max":{"1min":104.94199999999999,"5min":104.94199999999999,"15min":104.94199999999999},"last":1.2909999999999999},{"interface":"front","average":{"1min":1.5720000000000001,"5min":1.5720000000000001,"15min":1.5720000000000001},"min":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"max":{"1min":2.3809999999999998,"5min":2.3809999999999998,"15min":2.3809999999999998},"last":1.4099999999999999}]},{"osd":14,"last update":"Wed Sep 18 10:16:15 2024","interfaces":[{"interface":"back","average":{"1min":5.5910000000000002,"5min":5.5910000000000002,"15min":5.5910000000000002},"min":{"1min":1.276,"5min":1.276,"15min":1.276},"max":{"1min":66.825999999999993,"5min":66.825999999999993,"15min":66.825999999999993},"last":1.3580000000000001},{"interface":"front","average":{"1min":2.8359999999999999,"5min":2.8359999999999999,"15min":2.8359999999999999},"min":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"max":{"1min":22.16,"5min":22.16,"15min":22.16},"last":1.508}]},{"osd":15,"last update":"Wed Sep 18 10:16:15 2024","interfaces":[{"interface":"back","average":{"1min":3.677,"5min":3.677,"15min":3.677},"min":{"1min":1.165,"5min":1.165,"15min":1.165},"max":{"1min":36.575000000000003,"5min":36.575000000000003,"15min":36.575000000000003},"last":1.54},{"interface":"front","average":{"1min":5.6269999999999998,"5min":5.6269999999999998,"15min":5.6269999999999998},"min":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"max":{"1min":66.763999999999996,"5min":66.763999999999996,"15min":66.763999999999996},"last":1.468}]}]},{"osd":2,"up_from":22,"seq":94489280543,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":303764,"kb_used_data":2248,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068076,"statfs":{"total":96636764160,"available":96325709824,"internally_reserved":0,"allocated":2301952,"data_stored":1341063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,3,4,5,6,7,8,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":21.344000000000001,"15min":24.777999999999999},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.3879999999999999,"5min":419.05700000000002,"15min":419.05700000000002},"last":0.84899999999999998},{"interface":"front","average":{"1min":2.3860000000000001,"5min":4.4020000000000001,"15min":4.7380000000000004},"min":{"1min":0.57299999999999995,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":35.899999999999999,"5min":98.254000000000005,"15min":98.254000000000005},"last":0.71699999999999997}]},{"osd":3,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":5.1219999999999999,"5min":1.7529999999999999,"15min":1.1910000000000001},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":86.677000000000007,"5min":86.677000000000007,"15min":86.677000000000007},"last":0.66900000000000004},{"interface":"front","average":{"1min":3.6160000000000001,"5min":1.47,"15min":1.1120000000000001},"min":{"1min":0.68000000000000005,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":59.759,"5min":59.759,"15min":59.759},"last":0.76300000000000001}]},{"osd":4,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":7.992,"5min":3.1379999999999999,"15min":2.3290000000000002},"min":{"1min":0.55600000000000005,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":121.20099999999999,"5min":121.20099999999999,"15min":121.20099999999999},"last":0.94499999999999995},{"interface":"front","average":{"1min":6.5250000000000004,"5min":4.9359999999999999,"15min":4.6710000000000003},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":86.548000000000002,"5min":86.548000000000002,"15min":86.548000000000002},"last":0.88300000000000001}]},{"osd":5,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":12.161,"5min":3.2879999999999998,"15min":1.8089999999999999},"min":{"1min":0.72099999999999997,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":163.828,"5min":163.828,"15min":163.828},"last":0.92600000000000005},{"interface":"front","average":{"1min":2.6160000000000001,"5min":1.3660000000000001,"15min":1.1579999999999999},"min":{"1min":0.79200000000000004,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":36.119,"5min":36.119,"15min":36.119},"last":1.02}]},{"osd":6,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":5.2359999999999998,"5min":4.8529999999999998,"15min":4.7889999999999997},"min":{"1min":0.49199999999999999,"5min":0.441,"15min":0.441},"max":{"1min":86.766000000000005,"5min":86.766000000000005,"15min":86.766000000000005},"last":0.82399999999999995},{"interface":"front","average":{"1min":5.3959999999999999,"5min":4.9770000000000003,"15min":4.907},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":83.171999999999997,"5min":90.188999999999993,"15min":90.188999999999993},"last":1.038}]},{"osd":7,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":13.191000000000001,"5min":3.6019999999999999,"15min":2.004},"min":{"1min":0.84999999999999998,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":88.534000000000006,"5min":88.534000000000006,"15min":88.534000000000006},"last":1.079},{"interface":"front","average":{"1min":15.949999999999999,"5min":4.2350000000000003,"15min":2.2829999999999999},"min":{"1min":0.75600000000000001,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":88.484999999999999,"5min":88.484999999999999,"15min":88.484999999999999},"last":0.97899999999999998}]},{"osd":8,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":4.3179999999999996,"15min":4.8399999999999999},"min":{"1min":0.82899999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":2.0699999999999998,"5min":92.578999999999994,"15min":92.578999999999994},"last":1.1000000000000001},{"interface":"front","average":{"1min":2.8460000000000001,"5min":2.7799999999999998,"15min":2.7690000000000001},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":36.088000000000001,"5min":36.088000000000001,"15min":36.088000000000001},"last":1.1459999999999999}]},{"osd":12,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":2.9220000000000002,"5min":1.6719999999999999,"15min":1.464},"min":{"1min":0.872,"5min":0.872,"15min":0.872},"max":{"1min":36.014000000000003,"5min":36.014000000000003,"15min":36.014000000000003},"last":1.2290000000000001},{"interface":"front","average":{"1min":5.5140000000000002,"5min":2.7469999999999999,"15min":2.286},"min":{"1min":0.73199999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":86.832999999999998,"5min":86.832999999999998,"15min":86.832999999999998},"last":1.0589999999999999}]},{"osd":13,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":11.115,"5min":8.3970000000000002,"15min":7.944},"min":{"1min":0.97999999999999998,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":80.683000000000007,"5min":141.84200000000001,"15min":141.84200000000001},"last":1.3600000000000001},{"interface":"front","average":{"1min":2.911,"5min":4.9489999999999998,"15min":5.2889999999999997},"min":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":35.841000000000001,"5min":97.441999999999993,"15min":97.441999999999993},"last":1.2849999999999999}]},{"osd":15,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":12.472,"5min":3.6200000000000001,"15min":2.145},"min":{"1min":0.68400000000000005,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":86.980000000000004,"5min":86.980000000000004,"15min":86.980000000000004},"last":1.129},{"interface":"front","average":{"1min":3.3479999999999999,"5min":3.798,"15min":3.8730000000000002},"min":{"1min":0.96999999999999997,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":45.356999999999999,"5min":61.694000000000003,"15min":61.694000000000003},"last":1.304}]}]},{"osd":3,"up_from":22,"seq":94489280543,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303748,"kb_used_data":2232,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068092,"statfs":{"total":96636764160,"available":96325726208,"internally_reserved":0,"allocated":2285568,"data_stored":1331589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,4,6,7,9,10,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:16:37 2024","interfaces":[{"interface":"back","average":{"1min":4.173,"5min":2.048,"15min":1.694},"min":{"1min":0.64300000000000002,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":65.994,"5min":65.994,"15min":65.994},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.97899999999999998,"5min":8.8260000000000005,"15min":10.134},"min":{"1min":0.75,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.752,"5min":71.316999999999993,"15min":71.316999999999993},"last":0.76800000000000002}]},{"osd":2,"last update":"Wed Sep 18 10:16:37 2024","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":4.4379999999999997,"15min":5.0140000000000002},"min":{"1min":0.76900000000000002,"5min":0.75,"15min":0.75},"max":{"1min":1.333,"5min":56.523000000000003,"15min":56.523000000000003},"last":0.94999999999999996},{"interface":"front","average":{"1min":1.165,"5min":1.1200000000000001,"15min":1.1120000000000001},"min":{"1min":0.67400000000000004,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.639,"5min":3.327,"15min":3.327},"last":1.3300000000000001}]},{"osd":4,"last update":"Wed Sep 18 10:16:37 2024","interfaces":[{"interface":"back","average":{"1min":1.0409999999999999,"5min":5.7069999999999999,"15min":6.4850000000000003},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.6399999999999999,"5min":71.135000000000005,"15min":71.135000000000005},"last":0.91500000000000004},{"interface":"front","average":{"1min":1.635,"5min":3.8130000000000002,"15min":4.1760000000000002},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":10.039,"5min":56.253999999999998,"15min":56.253999999999998},"last":0.72799999999999998}]},{"osd":5,"last update":"Wed Sep 18 10:16:06 2024","interfaces":[{"interface":"back","average":{"1min":13.997,"5min":13.997,"15min":13.997},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":125.42400000000001,"5min":125.42400000000001,"15min":125.42400000000001},"last":1.131},{"interface":"front","average":{"1min":10.323,"5min":10.323,"15min":10.323},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":112.033,"5min":112.033,"15min":112.033},"last":0.96899999999999997}]},{"osd":6,"last update":"Wed Sep 18 10:16:37 2024","interfaces":[{"interface":"back","average":{"1min":1.9279999999999999,"5min":1.29,"15min":1.1839999999999999},"min":{"1min":0.93400000000000005,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":12.83,"5min":12.83,"15min":12.83},"last":1.0600000000000001},{"interface":"front","average":{"1min":5.0190000000000001,"5min":8.0050000000000008,"15min":8.5030000000000001},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":76.634,"5min":81.210999999999999,"15min":81.210999999999999},"last":0.998}]},{"osd":7,"last update":"Wed Sep 18 10:16:37 2024","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":13.159000000000001,"15min":15.141},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.75,"5min":129.68299999999999,"15min":129.68299999999999},"last":1.0249999999999999},{"interface":"front","average":{"1min":1.4710000000000001,"5min":11.119999999999999,"15min":12.728},"min":{"1min":0.90800000000000003,"5min":0.621,"15min":0.621},"max":{"1min":1.8500000000000001,"5min":129.726,"15min":129.726},"last":1.474}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0620000000000001}]},{"osd":9,"last update":"Wed Sep 18 10:16:37 2024","interfaces":[{"interface":"back","average":{"1min":1.5660000000000001,"5min":1.446,"15min":1.4259999999999999},"min":{"1min":0.94599999999999995,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":2.0209999999999999,"5min":2.0209999999999999,"15min":2.0209999999999999},"last":1.377},{"interface":"front","average":{"1min":6.1050000000000004,"5min":6.1079999999999997,"15min":6.1079999999999997},"min":{"1min":0.86099999999999999,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":94.546999999999997,"5min":94.546999999999997,"15min":94.546999999999997},"last":1.2969999999999999}]},{"osd":10,"last update":"Wed Sep 18 10:16:06 2024","interfaces":[{"interface":"back","average":{"1min":1.4870000000000001,"5min":1.4870000000000001,"15min":1.4870000000000001},"min":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":2.0659999999999998,"5min":2.0659999999999998,"15min":2.0659999999999998},"last":1.232},{"interface":"front","average":{"1min":4.9349999999999996,"5min":4.9349999999999996,"15min":4.9349999999999996},"min":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":71.668999999999997,"5min":71.668999999999997,"15min":71.668999999999997},"last":1.173}]},{"osd":13,"last update":"Wed Sep 18 10:16:37 2024","interfaces":[{"interface":"back","average":{"1min":9.4909999999999997,"5min":3.113,"15min":2.0499999999999998},"min":{"1min":1.04,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":91.290999999999997,"5min":91.290999999999997,"15min":91.290999999999997},"last":1.421},{"interface":"front","average":{"1min":1.5249999999999999,"5min":5.3300000000000001,"15min":5.9640000000000004},"min":{"1min":1.05,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.885,"5min":71.447000000000003,"15min":71.447000000000003},"last":1.3560000000000001}]},{"osd":14,"last update":"Wed Sep 18 10:16:37 2024","interfaces":[{"interface":"back","average":{"1min":1.6899999999999999,"5min":4.8899999999999997,"15min":5.423},"min":{"1min":1.0089999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"max":{"1min":4.625,"5min":56.664999999999999,"15min":56.664999999999999},"last":1.149},{"interface":"front","average":{"1min":1.865,"5min":2.2629999999999999,"15min":2.3290000000000002},"min":{"1min":1.1859999999999999,"5min":0.93999999999999995,"15min":0.93999999999999995},"max":{"1min":4.5910000000000002,"5min":18.523,"15min":18.523},"last":1.52}]},{"osd":15,"last update":"Wed Sep 18 10:16:48 2024","interfaces":[{"interface":"back","average":{"1min":1.6970000000000001,"5min":4.0049999999999999,"15min":4.3899999999999997},"min":{"1min":1.01,"5min":0.91500000000000004,"15min":0.91500000000000004},"max":{"1min":4.3289999999999997,"5min":56.357999999999997,"15min":56.357999999999997},"last":1.268},{"interface":"front","average":{"1min":1.7589999999999999,"5min":8.5310000000000006,"15min":9.6600000000000001},"min":{"1min":1.133,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":2.1110000000000002,"5min":122.727,"15min":122.727},"last":1.5620000000000001}]}]},{"osd":5,"up_from":22,"seq":94489280543,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303748,"kb_used_data":2232,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068092,"statfs":{"total":96636764160,"available":96325726208,"internally_reserved":0,"allocated":2285568,"data_stored":1331589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[2,3,4,6,8,9,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:15:47 2024","interfaces":[{"interface":"back","average":{"1min":7.8860000000000001,"5min":7.8860000000000001,"15min":7.8860000000000001},"min":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":84.430999999999997,"5min":84.430999999999997,"15min":84.430999999999997},"last":1.2889999999999999},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":1.353}]},{"osd":2,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":4.1360000000000001,"5min":8.6940000000000008,"15min":9.4540000000000006},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":51.091000000000001,"5min":84.566000000000003,"15min":84.566000000000003},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.98199999999999998,"5min":1.3300000000000001,"15min":1.3879999999999999},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.4690000000000001,"5min":9.5069999999999997,"15min":9.5069999999999997},"last":0.86899999999999999}]},{"osd":3,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":4.9240000000000004,"5min":7.8010000000000002,"15min":8.2810000000000006},"min":{"1min":0.69099999999999995,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":51.229999999999997,"5min":84.358999999999995,"15min":84.358999999999995},"last":0.70499999999999996},{"interface":"front","average":{"1min":8.1590000000000007,"5min":8.2100000000000009,"15min":8.218},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":74.222999999999999,"5min":97.549999999999997,"15min":97.549999999999997},"last":0.76900000000000002}]},{"osd":4,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":25.673999999999999,"15min":29.777999999999999},"min":{"1min":0.67400000000000004,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.407,"5min":266.911,"15min":266.911},"last":1.4059999999999999},{"interface":"front","average":{"1min":21.388000000000002,"5min":22.515999999999998,"15min":22.704000000000001},"min":{"1min":0.76200000000000001,"5min":0.626,"15min":0.626},"max":{"1min":171.691,"5min":260.53500000000003,"15min":260.53500000000003},"last":1.0960000000000001}]},{"osd":6,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":3.6309999999999998,"5min":8.0809999999999995,"15min":8.8230000000000004},"min":{"1min":0.70099999999999996,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":41.886000000000003,"5min":58.171999999999997,"15min":58.171999999999997},"last":0.97599999999999998},{"interface":"front","average":{"1min":5.8079999999999998,"5min":18.274999999999999,"15min":20.353000000000002},"min":{"1min":0.85699999999999998,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":54.984999999999999,"5min":227.48099999999999,"15min":227.48099999999999},"last":1.1759999999999999}]},{"osd":7,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":12.013,"5min":12.013,"15min":12.013},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":123.111,"5min":123.111,"15min":123.111},"last":1.5249999999999999},{"interface":"front","average":{"1min":13.994,"5min":13.994,"15min":13.994},"min":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"max":{"1min":84.617000000000004,"5min":84.617000000000004,"15min":84.617000000000004},"last":1.329}]},{"osd":8,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":4.4720000000000004,"5min":17.175999999999998,"15min":19.292999999999999},"min":{"1min":0.879,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":51.506,"5min":260.44200000000001,"15min":260.44200000000001},"last":1.232},{"interface":"front","average":{"1min":8.2609999999999992,"5min":7.2859999999999996,"15min":7.1239999999999997},"min":{"1min":0.83899999999999997,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":71.266000000000005,"5min":106.818,"15min":106.818},"last":1.038}]},{"osd":9,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":3.8780000000000001,"5min":5.2930000000000001,"15min":5.5289999999999999},"min":{"1min":0.91300000000000003,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":41.652000000000001,"5min":80.272000000000006,"15min":80.272000000000006},"last":1.133},{"interface":"front","average":{"1min":1.3799999999999999,"5min":1.764,"15min":1.8280000000000001},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":2.008,"5min":9.5690000000000008,"15min":9.5690000000000008},"last":1.502}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6479999999999999}]},{"osd":11,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":1.494,"5min":7.8120000000000003,"15min":8.8650000000000002},"min":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":1.825,"5min":123.158,"15min":123.158},"last":1.794},{"interface":"front","average":{"1min":8.8490000000000002,"5min":10.863,"15min":11.199},"min":{"1min":0.93200000000000005,"5min":0.86499999999999999,"15min":0.86499999999999999},"max":{"1min":68.938000000000002,"5min":84.522999999999996,"15min":84.522999999999996},"last":1.446}]},{"osd":12,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":4.0250000000000004,"5min":1.8580000000000001,"15min":1.4970000000000001},"min":{"1min":0.95199999999999996,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":41.929000000000002,"5min":41.929000000000002,"15min":41.929000000000002},"last":1.2849999999999999},{"interface":"front","average":{"1min":1.774,"5min":1.9339999999999999,"15min":1.9610000000000001},"min":{"1min":0.93999999999999995,"5min":0.88300000000000001,"15min":0.88300000000000001},"max":{"1min":4.1150000000000002,"5min":9.7710000000000008,"15min":9.7710000000000008},"last":1.5780000000000001}]},{"osd":13,"last update":"Wed Sep 18 10:16:48 2024","interfaces":[{"interface":"back","average":{"1min":1.6499999999999999,"5min":19.745999999999999,"15min":22.762},"min":{"1min":1.171,"5min":1.1120000000000001,"15min":1.1120000000000001},"max":{"1min":2.0550000000000002,"5min":140.273,"15min":140.273},"last":1.6479999999999999},{"interface":"front","average":{"1min":4.665,"5min":20.309000000000001,"15min":22.917000000000002},"min":{"1min":1.194,"5min":0.98099999999999998,"15min":0.98099999999999998},"max":{"1min":51.432000000000002,"5min":170.86000000000001,"15min":170.86000000000001},"last":1.329}]},{"osd":14,"last update":"Wed Sep 18 10:16:48 2024","interfaces":[{"interface":"back","average":{"1min":9.8930000000000007,"5min":12.584,"15min":13.032},"min":{"1min":1.385,"5min":1.0389999999999999,"15min":1.0389999999999999},"max":{"1min":93.295000000000002,"5min":101.733,"15min":101.733},"last":1.385},{"interface":"front","average":{"1min":1.79,"5min":7.9729999999999999,"15min":9.0030000000000001},"min":{"1min":1.321,"5min":0.90100000000000002,"15min":0.90100000000000002},"max":{"1min":2.2290000000000001,"5min":122.952,"15min":122.952},"last":1.7250000000000001}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8839999999999999}]}]},{"osd":6,"up_from":22,"seq":94489280543,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304280,"kb_used_data":2752,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94067560,"statfs":{"total":96636764160,"available":96325181440,"internally_reserved":0,"allocated":2818048,"data_stored":1860517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,3,4,5,7,8,9,11,12,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:15:47 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.68200000000000005}]},{"osd":1,"last update":"Wed Sep 18 10:16:48 2024","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":3.8100000000000001,"15min":4.306},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.2150000000000001,"5min":52.332999999999998,"15min":52.332999999999998},"last":0.65700000000000003},{"interface":"front","average":{"1min":4.8369999999999997,"5min":1.6990000000000001,"15min":1.1759999999999999},"min":{"1min":0.68300000000000005,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":67.763000000000005,"5min":67.763000000000005,"15min":67.763000000000005},"last":0.71199999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.343}]},{"osd":3,"last update":"Wed Sep 18 10:16:48 2024","interfaces":[{"interface":"back","average":{"1min":5.3380000000000001,"5min":1.792,"15min":1.2010000000000001},"min":{"1min":0.65500000000000003,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":76.891999999999996,"5min":76.891999999999996,"15min":76.891999999999996},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.98999999999999999,"5min":11.403,"15min":13.138999999999999},"min":{"1min":0.751,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.204,"5min":169.244,"15min":169.244},"last":1.042}]},{"osd":4,"last update":"Wed Sep 18 10:16:02 2024","interfaces":[{"interface":"back","average":{"1min":11.111000000000001,"5min":11.111000000000001,"15min":11.111000000000001},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":128.053,"5min":128.053,"15min":128.053},"last":0.92100000000000004},{"interface":"front","average":{"1min":4.6189999999999998,"5min":4.6189999999999998,"15min":4.6189999999999998},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":67.960999999999999,"5min":67.960999999999999,"15min":67.960999999999999},"last":0.86699999999999999}]},{"osd":5,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":6.0579999999999998,"5min":10.194000000000001,"15min":10.884},"min":{"1min":0.72099999999999997,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":68.025999999999996,"5min":119.032,"15min":119.032},"last":0.96799999999999997},{"interface":"front","average":{"1min":5.6760000000000002,"5min":4.0380000000000003,"15min":3.7650000000000001},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":83.858000000000004,"5min":83.858000000000004,"15min":83.858000000000004},"last":1.109}]},{"osd":7,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":4.9539999999999997,"5min":6.3959999999999999,"15min":6.6360000000000001},"min":{"1min":0.94099999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":69.171000000000006,"5min":69.171000000000006,"15min":69.171000000000006},"last":1.3120000000000001},{"interface":"front","average":{"1min":5.0960000000000001,"5min":3.4630000000000001,"15min":3.1909999999999998},"min":{"1min":1.089,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":69.239000000000004,"5min":69.239000000000004,"15min":69.239000000000004},"last":1.29}]},{"osd":8,"last update":"Wed Sep 18 10:16:51 2024","interfaces":[{"interface":"back","average":{"1min":5.3659999999999997,"5min":5.3659999999999997,"15min":5.3659999999999997},"min":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"max":{"1min":76.766000000000005,"5min":76.766000000000005,"15min":76.766000000000005},"last":1.0109999999999999},{"interface":"front","average":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"min":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"max":{"1min":1.704,"5min":1.704,"15min":1.704},"last":1.5029999999999999}]},{"osd":9,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":5.4269999999999996,"5min":7.7089999999999996,"15min":8.0890000000000004},"min":{"1min":0.95799999999999996,"5min":0.84599999999999997,"15min":0.84599999999999997},"max":{"1min":76.997,"5min":95.691999999999993,"15min":95.691999999999993},"last":1.246},{"interface":"front","average":{"1min":1.2949999999999999,"5min":10.472,"15min":12.002000000000001},"min":{"1min":0.96899999999999997,"5min":0.83099999999999996,"15min":0.83099999999999996},"max":{"1min":1.696,"5min":165.31100000000001,"15min":165.31100000000001},"last":1.4219999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.329}]},{"osd":11,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":5.0510000000000002,"5min":2.0630000000000002,"15min":1.5649999999999999},"min":{"1min":1.002,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":67.909999999999997,"5min":67.909999999999997,"15min":67.909999999999997},"last":1.1599999999999999},{"interface":"front","average":{"1min":9.1829999999999998,"5min":4.9429999999999996,"15min":4.2359999999999998},"min":{"1min":1.1870000000000001,"5min":0.92000000000000004,"15min":0.92000000000000004},"max":{"1min":77.158000000000001,"5min":77.158000000000001,"15min":77.158000000000001},"last":1.351}]},{"osd":12,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":1.3859999999999999,"5min":4.3140000000000001,"15min":4.8019999999999996},"min":{"1min":1.079,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":1.796,"5min":52.378,"15min":52.378},"last":1.204},{"interface":"front","average":{"1min":5.1109999999999998,"5min":4.5720000000000001,"15min":4.4820000000000002},"min":{"1min":0.92800000000000005,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":67.835999999999999,"5min":67.835999999999999,"15min":67.835999999999999},"last":1.3779999999999999}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2050000000000001}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2669999999999999}]},{"osd":15,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":5.1680000000000001,"5min":2.2120000000000002,"15min":1.72},"min":{"1min":1.1000000000000001,"5min":0.90300000000000002,"15min":0.90300000000000002},"max":{"1min":68.069999999999993,"5min":68.069999999999993,"15min":68.069999999999993},"last":1.468},{"interface":"front","average":{"1min":5.6200000000000001,"5min":4.3019999999999996,"15min":4.0819999999999999},"min":{"1min":1.2030000000000001,"5min":1.0429999999999999,"15min":1.0429999999999999},"max":{"1min":77.081999999999994,"5min":77.081999999999994,"15min":77.081999999999994},"last":1.452}]}]},{"osd":7,"up_from":22,"seq":94489280543,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303748,"kb_used_data":2232,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068092,"statfs":{"total":96636764160,"available":96325726208,"internally_reserved":0,"allocated":2285568,"data_stored":1331589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,4,6,8,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:16:33 2024","interfaces":[{"interface":"back","average":{"1min":3.9900000000000002,"5min":10.974,"15min":12.138},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":59.137999999999998,"5min":139.58600000000001,"15min":139.58600000000001},"last":0.75600000000000001},{"interface":"front","average":{"1min":1.121,"5min":1.0229999999999999,"15min":1.0069999999999999},"min":{"1min":0.67800000000000005,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.7390000000000001,"5min":1.7390000000000001,"15min":1.7390000000000001},"last":0.59899999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2090000000000001}]},{"osd":2,"last update":"Wed Sep 18 10:15:46 2024","interfaces":[{"interface":"back","average":{"1min":15.436,"5min":15.436,"15min":15.436},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":139.36799999999999,"5min":139.36799999999999,"15min":139.36799999999999},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.94299999999999995}]},{"osd":3,"last update":"Wed Sep 18 10:15:46 2024","interfaces":[{"interface":"back","average":{"1min":17.327999999999999,"5min":17.327999999999999,"15min":17.327999999999999},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":139.27199999999999,"5min":139.27199999999999,"15min":139.27199999999999},"last":1.3460000000000001},{"interface":"front","average":{"1min":3.964,"5min":3.964,"15min":3.964},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":23.748999999999999,"5min":23.748999999999999,"15min":23.748999999999999},"last":0.82899999999999996}]},{"osd":4,"last update":"Wed Sep 18 10:16:37 2024","interfaces":[{"interface":"back","average":{"1min":9.8670000000000009,"5min":17.291,"15min":18.527999999999999},"min":{"1min":0.68000000000000005,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":125.10299999999999,"5min":139.51599999999999,"15min":139.51599999999999},"last":0.55200000000000005},{"interface":"front","average":{"1min":3.3559999999999999,"5min":4.6699999999999999,"15min":4.8890000000000002},"min":{"1min":0.68000000000000005,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":49.241999999999997,"5min":49.241999999999997,"15min":49.241999999999997},"last":0.70199999999999996}]},{"osd":5,"last update":"Wed Sep 18 10:15:46 2024","interfaces":[{"interface":"back","average":{"1min":1.099,"5min":1.099,"15min":1.099},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":2.3620000000000001,"5min":2.3620000000000001,"15min":2.3620000000000001},"last":1.4059999999999999},{"interface":"front","average":{"1min":17.736999999999998,"5min":17.736999999999998,"15min":17.736999999999998},"min":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"max":{"1min":139.44300000000001,"5min":139.44300000000001,"15min":139.44300000000001},"last":0.78000000000000003}]},{"osd":6,"last update":"Wed Sep 18 10:16:33 2024","interfaces":[{"interface":"back","average":{"1min":6.0869999999999997,"5min":15.117000000000001,"15min":16.622},"min":{"1min":0.998,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":59.343000000000004,"5min":183.001,"15min":183.001},"last":1.024},{"interface":"front","average":{"1min":7.0830000000000002,"5min":4.8650000000000002,"15min":4.4950000000000001},"min":{"1min":0.82399999999999995,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":58.284999999999997,"5min":58.284999999999997,"15min":58.284999999999997},"last":0.98499999999999999}]},{"osd":8,"last update":"Wed Sep 18 10:16:30 2024","interfaces":[{"interface":"back","average":{"1min":1.4079999999999999,"5min":1.2809999999999999,"15min":1.26},"min":{"1min":0.84999999999999998,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.9339999999999999,"5min":2.6139999999999999,"15min":2.6139999999999999},"last":0.85699999999999998},{"interface":"front","average":{"1min":5.3029999999999999,"5min":13.785,"15min":15.199},"min":{"1min":1.0980000000000001,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":59.523000000000003,"5min":139.798,"15min":139.798},"last":0.89700000000000002}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.861}]},{"osd":10,"last update":"Wed Sep 18 10:16:30 2024","interfaces":[{"interface":"back","average":{"1min":16.035,"5min":8.4100000000000001,"15min":7.1390000000000002},"min":{"1min":1.0149999999999999,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":156.45500000000001,"5min":156.45500000000001,"15min":156.45500000000001},"last":1.2070000000000001},{"interface":"front","average":{"1min":1.4910000000000001,"5min":1.391,"15min":1.3740000000000001},"min":{"1min":0.83499999999999996,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":2.1200000000000001,"5min":2.254,"15min":2.254},"last":1.0600000000000001}]},{"osd":11,"last update":"Wed Sep 18 10:16:33 2024","interfaces":[{"interface":"back","average":{"1min":5.0460000000000003,"5min":5.4459999999999997,"15min":5.5129999999999999},"min":{"1min":1.046,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":49.154000000000003,"5min":62.356000000000002,"15min":62.356000000000002},"last":1.48},{"interface":"front","average":{"1min":2.4769999999999999,"5min":1.6240000000000001,"15min":1.482},"min":{"1min":1.0189999999999999,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":20.449999999999999,"5min":20.449999999999999,"15min":20.449999999999999},"last":0.94599999999999995}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4119999999999999}]},{"osd":13,"last update":"Wed Sep 18 10:16:33 2024","interfaces":[{"interface":"back","average":{"1min":10.028,"5min":16.707999999999998,"15min":17.821999999999999},"min":{"1min":1.1459999999999999,"5min":0.874,"15min":0.874},"max":{"1min":113.428,"5min":139.655,"15min":139.655},"last":1.349},{"interface":"front","average":{"1min":5.1970000000000001,"5min":16.942,"15min":18.899999999999999},"min":{"1min":1.077,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":49.457999999999998,"5min":173.113,"15min":173.113},"last":1.75}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1799999999999999}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2829999999999999}]}]},{"osd":8,"up_from":22,"seq":94489280543,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303768,"kb_used_data":2248,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068072,"statfs":{"total":96636764160,"available":96325705728,"internally_reserved":0,"allocated":2301952,"data_stored":1341063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[2,4,6,7,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93300000000000005}]},{"osd":1,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":1.117,"5min":1.117,"15min":1.117},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.669,"5min":1.669,"15min":1.669},"last":1.2230000000000001},{"interface":"front","average":{"1min":2.3290000000000002,"5min":2.3290000000000002,"15min":2.3290000000000002},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":29.523,"5min":29.523,"15min":29.523},"last":1.1279999999999999}]},{"osd":2,"last update":"Wed Sep 18 10:16:48 2024","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":5.5140000000000002,"15min":6.2300000000000004},"min":{"1min":0.93500000000000005,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.8,"5min":77.435000000000002,"15min":77.435000000000002},"last":0.93500000000000005},{"interface":"front","average":{"1min":1.1639999999999999,"5min":0.97099999999999997,"15min":0.93899999999999995},"min":{"1min":0.71999999999999997,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.903,"5min":1.903,"15min":1.903},"last":0.71999999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.306}]},{"osd":4,"last update":"Wed Sep 18 10:16:48 2024","interfaces":[{"interface":"back","average":{"1min":1.23,"5min":9.5879999999999992,"15min":10.981},"min":{"1min":0.71899999999999997,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":7.1559999999999997,"5min":90.438000000000002,"15min":90.438000000000002},"last":0.83899999999999997},{"interface":"front","average":{"1min":1.1699999999999999,"5min":10.691000000000001,"15min":12.278},"min":{"1min":0.66100000000000003,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":2.1309999999999998,"5min":93.939999999999998,"15min":93.939999999999998},"last":0.66100000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4199999999999999}]},{"osd":6,"last update":"Wed Sep 18 10:16:48 2024","interfaces":[{"interface":"back","average":{"1min":1.359,"5min":15.042999999999999,"15min":17.324000000000002},"min":{"1min":0.90700000000000003,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":2.0350000000000001,"5min":293.26900000000001,"15min":293.26900000000001},"last":0.90700000000000003},{"interface":"front","average":{"1min":1.2230000000000001,"5min":5.1689999999999996,"15min":5.827},"min":{"1min":0.88900000000000001,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.839,"5min":70.203000000000003,"15min":70.203000000000003},"last":1.1060000000000001}]},{"osd":7,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":7.1950000000000003,"5min":3.6070000000000002,"15min":3.0089999999999999},"min":{"1min":0.80900000000000005,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":80.635000000000005,"5min":80.635000000000005,"15min":80.635000000000005},"last":1.0580000000000001},{"interface":"front","average":{"1min":9.3490000000000002,"5min":6.726,"15min":6.2889999999999997},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":99.060000000000002,"5min":99.060000000000002,"15min":99.060000000000002},"last":1.01}]},{"osd":9,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":4.8730000000000002,"5min":3.0110000000000001,"15min":2.7010000000000001},"min":{"1min":0.92700000000000005,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":80.076999999999998,"5min":80.076999999999998,"15min":80.076999999999998},"last":1.28},{"interface":"front","average":{"1min":1.6240000000000001,"5min":7.4489999999999998,"15min":8.4199999999999999},"min":{"1min":1.196,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":2.5190000000000001,"5min":98.914000000000001,"15min":98.914000000000001},"last":1.153}]},{"osd":10,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":1.6319999999999999,"5min":5.8559999999999999,"15min":6.5599999999999996},"min":{"1min":0.99099999999999999,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":2.3679999999999999,"5min":90.751000000000005,"15min":90.751000000000005},"last":1.081},{"interface":"front","average":{"1min":1.591,"5min":7.7889999999999997,"15min":8.8219999999999992},"min":{"1min":1.0640000000000001,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":2.3340000000000001,"5min":83.795000000000002,"15min":83.795000000000002},"last":1.3839999999999999}]},{"osd":11,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":1.7450000000000001,"5min":2.669,"15min":2.823},"min":{"1min":1.3340000000000001,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":2.383,"5min":26.462,"15min":26.462},"last":1.232},{"interface":"front","average":{"1min":1.752,"5min":1.948,"15min":1.9810000000000001},"min":{"1min":1.24,"5min":0.99299999999999999,"15min":0.99299999999999999},"max":{"1min":2.3479999999999999,"5min":12.475,"15min":12.475},"last":1.202}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0760000000000001}]},{"osd":13,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":3.052,"5min":7.9329999999999998,"15min":8.7469999999999999},"min":{"1min":1.05,"5min":1.0429999999999999,"15min":1.0429999999999999},"max":{"1min":30.582999999999998,"5min":94.222999999999999,"15min":94.222999999999999},"last":1.264},{"interface":"front","average":{"1min":1.635,"5min":2.5059999999999998,"15min":2.6509999999999998},"min":{"1min":1.1180000000000001,"5min":0.93400000000000005,"15min":0.93400000000000005},"max":{"1min":2.4689999999999999,"5min":23.983000000000001,"15min":23.983000000000001},"last":1.464}]},{"osd":14,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":1.802,"5min":16.975000000000001,"15min":19.504000000000001},"min":{"1min":1.28,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":2.4500000000000002,"5min":150.93600000000001,"15min":150.93600000000001},"last":1.3240000000000001},{"interface":"front","average":{"1min":4.8170000000000002,"5min":3.8410000000000002,"15min":3.6789999999999998},"min":{"1min":1.2210000000000001,"5min":0.90200000000000002,"15min":0.90200000000000002},"max":{"1min":69.918000000000006,"5min":69.918000000000006,"15min":69.918000000000006},"last":1.3460000000000001}]},{"osd":15,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":5.9089999999999998,"5min":3.4169999999999998,"15min":3.0019999999999998},"min":{"1min":1.2549999999999999,"5min":0.93300000000000005,"15min":0.93300000000000005},"max":{"1min":98.558000000000007,"5min":98.558000000000007,"15min":98.558000000000007},"last":1.5409999999999999},{"interface":"front","average":{"1min":1.8400000000000001,"5min":2.8119999999999998,"15min":2.9740000000000002},"min":{"1min":1.3160000000000001,"5min":0.95299999999999996,"15min":0.95299999999999996},"max":{"1min":2.5859999999999999,"5min":26.414000000000001,"15min":26.414000000000001},"last":1.373}]}]},{"osd":10,"up_from":22,"seq":94489280544,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303748,"kb_used_data":2232,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068092,"statfs":{"total":96636764160,"available":96325726208,"internally_reserved":0,"allocated":2285568,"data_stored":1331589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,4,5,6,7,9,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:16:29 2024","interfaces":[{"interface":"back","average":{"1min":16.117000000000001,"5min":16.117000000000001,"15min":16.117000000000001},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":145.12799999999999,"5min":145.12799999999999,"15min":145.12799999999999},"last":1.401},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.431,"5min":1.431,"15min":1.431},"last":1.0780000000000001}]},{"osd":1,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":9.5719999999999992,"5min":10.66,"15min":10.842000000000001},"min":{"1min":0.88500000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":145.20699999999999,"5min":145.20699999999999,"15min":145.20699999999999},"last":0.876},{"interface":"front","average":{"1min":4.7510000000000003,"5min":14.728,"15min":16.390999999999998},"min":{"1min":0.754,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":38.338000000000001,"5min":99.995999999999995,"15min":99.995999999999995},"last":0.71899999999999997}]},{"osd":2,"last update":"Wed Sep 18 10:15:33 2024","interfaces":[{"interface":"back","average":{"1min":10.893000000000001,"5min":10.893000000000001,"15min":10.893000000000001},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":84.072000000000003,"5min":84.072000000000003,"15min":84.072000000000003},"last":0.99199999999999999},{"interface":"front","average":{"1min":11.550000000000001,"5min":11.550000000000001,"15min":11.550000000000001},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":99.798000000000002,"5min":99.798000000000002,"15min":99.798000000000002},"last":1.425}]},{"osd":3,"last update":"Wed Sep 18 10:16:22 2024","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":1.119,"15min":1.119},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.99,"5min":1.99,"15min":1.99},"last":1.5},{"interface":"front","average":{"1min":18.094999999999999,"5min":18.094999999999999,"15min":18.094999999999999},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":145.535,"5min":145.535,"15min":145.535},"last":1.4510000000000001}]},{"osd":4,"last update":"Wed Sep 18 10:16:49 2024","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":20.715,"15min":23.963999999999999},"min":{"1min":0.92300000000000004,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.5820000000000001,"5min":125.23099999999999,"15min":125.23099999999999},"last":1.048},{"interface":"front","average":{"1min":1.206,"5min":25.175000000000001,"15min":29.170000000000002},"min":{"1min":0.88800000000000001,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.4930000000000001,"5min":295.93200000000002,"15min":295.93200000000002},"last":1.2210000000000001}]},{"osd":5,"last update":"Wed Sep 18 10:16:49 2024","interfaces":[{"interface":"back","average":{"1min":8.9960000000000004,"5min":21.460999999999999,"15min":23.539000000000001},"min":{"1min":0.81000000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":145.24000000000001,"5min":215.51400000000001,"15min":215.51400000000001},"last":1.111},{"interface":"front","average":{"1min":2.4140000000000001,"5min":22.346,"15min":25.667999999999999},"min":{"1min":0.85399999999999998,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":20.302,"5min":252.69300000000001,"15min":252.69300000000001},"last":1.421}]},{"osd":6,"last update":"Wed Sep 18 10:16:22 2024","interfaces":[{"interface":"back","average":{"1min":12.393000000000001,"5min":12.393000000000001,"15min":12.393000000000001},"min":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"max":{"1min":111.173,"5min":111.173,"15min":111.173},"last":1.1859999999999999},{"interface":"front","average":{"1min":15.771000000000001,"5min":15.771000000000001,"15min":15.771000000000001},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":221.13900000000001,"5min":221.13900000000001,"15min":221.13900000000001},"last":0.66500000000000004}]},{"osd":7,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":1.5109999999999999,"5min":2.7490000000000001,"15min":2.9550000000000001},"min":{"1min":0.95699999999999996,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.919,"5min":34.558,"15min":34.558},"last":1.083},{"interface":"front","average":{"1min":10.101000000000001,"5min":13.186999999999999,"15min":13.701000000000001},"min":{"1min":1.0920000000000001,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":145.5,"5min":145.5,"15min":145.5},"last":1.3140000000000001}]},{"osd":9,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":8.3640000000000008,"5min":11.07,"15min":11.521000000000001},"min":{"1min":1.0349999999999999,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":119.202,"5min":119.202,"15min":119.202},"last":1.361},{"interface":"front","average":{"1min":1.629,"5min":10.364000000000001,"15min":11.82},"min":{"1min":1.073,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":2.23,"5min":97.385999999999996,"15min":97.385999999999996},"last":0.92500000000000004}]},{"osd":11,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":10.132999999999999,"5min":11.489000000000001,"15min":11.715},"min":{"1min":1.286,"5min":0.85699999999999998,"15min":0.85699999999999998},"max":{"1min":145.26900000000001,"5min":145.26900000000001,"15min":145.26900000000001},"last":1.012},{"interface":"front","average":{"1min":5.6719999999999997,"5min":19.837,"15min":22.198},"min":{"1min":1.28,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":44.121000000000002,"5min":295.70400000000001,"15min":295.70400000000001},"last":1.2729999999999999}]},{"osd":12,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":1.6559999999999999,"5min":2.8250000000000002,"15min":3.02},"min":{"1min":1.0620000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":2.2559999999999998,"5min":34.625999999999998,"15min":34.625999999999998},"last":1.4590000000000001},{"interface":"front","average":{"1min":10.140000000000001,"5min":10.986000000000001,"15min":11.127000000000001},"min":{"1min":1.198,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":145.358,"5min":145.358,"15min":145.358},"last":1.639}]},{"osd":13,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":10.433999999999999,"5min":17.994,"15min":19.254000000000001},"min":{"1min":1.1990000000000001,"5min":0.877,"15min":0.877},"max":{"1min":123.696,"5min":123.696,"15min":123.696},"last":1.5229999999999999},{"interface":"front","average":{"1min":7.6379999999999999,"5min":14.551,"15min":15.702999999999999},"min":{"1min":1.163,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":68.363,"5min":85.879999999999995,"15min":85.879999999999995},"last":1.609}]},{"osd":14,"last update":"Wed Sep 18 10:16:22 2024","interfaces":[{"interface":"back","average":{"1min":15.864000000000001,"5min":15.864000000000001,"15min":15.864000000000001},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":145.46700000000001,"5min":145.46700000000001,"15min":145.46700000000001},"last":1.4830000000000001},{"interface":"front","average":{"1min":13.93,"5min":13.93,"15min":13.93},"min":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"max":{"1min":100.092,"5min":100.092,"15min":100.092},"last":1.587}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1080000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T10:16:56.845 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T10:16:57.154 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-18T10:16:57.157 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-09-18T10:16:57.169 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":209,"stamp":"2024-09-18T10:16:55.239416+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":147,"num_write_kb":2674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":115,"ondisk_log_size":115,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":14,"kb":1509949440,"kb_used":4865408,"kb_used_data":36952,"kb_used_omap":26,"kb_used_meta":4828197,"kb_avail":1505084032,"statfs":{"total":1546188226560,"available":1541206048768,"internally_reserved":0,"allocated":37838848,"data_stored":22477174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":27264,"internal_metadata":4944074112},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.056129"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":144,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-18T10:16:49.404525+0000","last_change":"2024-09-18T10:16:18.085265+0000","last_active":"2024-09-18T10:16:49.404525+0000","last_peered":"2024-09-18T10:16:49.404525+0000","last_clean":"2024-09-18T10:16:49.404525+0000","last_became_active":"2024-09-18T10:14:35.348943+0000","last_became_peered":"2024-09-18T10:14:35.348943+0000","last_unstale":"2024-09-18T10:16:49.404525+0000","last_undegraded":"2024-09-18T10:16:49.404525+0000","last_fullsized":"2024-09-18T10:16:49.404525+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:16:18.085027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:16:18.085027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:17:43.814969+0000","scrub_duration":0.30385945199999997,"objects_trimmed":0,"snaptrim_duration":0.00060442999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,10],"acting":[13,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":76,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-18T10:16:49.402946+0000","last_change":"2024-09-18T10:16:18.981603+0000","last_active":"2024-09-18T10:16:49.402946+0000","last_peered":"2024-09-18T10:16:49.402946+0000","last_clean":"2024-09-18T10:16:49.402946+0000","last_became_active":"2024-09-18T10:15:50.978055+0000","last_became_peered":"2024-09-18T10:15:50.978055+0000","last_unstale":"2024-09-18T10:16:49.402946+0000","last_undegraded":"2024-09-18T10:16:49.402946+0000","last_fullsized":"2024-09-18T10:16:49.402946+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:16:18.981373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:16:18.981373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:17:41.742568+0000","scrub_duration":0.40456036899999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11],"acting":[8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":140,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-18T10:16:49.303750+0000","last_change":"2024-09-18T10:15:50.883382+0000","last_active":"2024-09-18T10:16:49.303750+0000","last_peered":"2024-09-18T10:16:49.303750+0000","last_clean":"2024-09-18T10:16:49.303750+0000","last_became_active":"2024-09-18T10:14:35.329198+0000","last_became_peered":"2024-09-18T10:14:35.329198+0000","last_unstale":"2024-09-18T10:16:49.303750+0000","last_undegraded":"2024-09-18T10:16:49.303750+0000","last_fullsized":"2024-09-18T10:16:49.303750+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:15:50.883147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:15:50.883147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:17:15.846467+0000","scrub_duration":0.153222372,"objects_trimmed":0,"snaptrim_duration":0.00062415699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":142,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T10:16:48.501848+0000","last_change":"2024-09-18T10:16:02.749341+0000","last_active":"2024-09-18T10:16:48.501848+0000","last_peered":"2024-09-18T10:16:48.501848+0000","last_clean":"2024-09-18T10:16:48.501848+0000","last_became_active":"2024-09-18T10:14:35.524102+0000","last_became_peered":"2024-09-18T10:14:35.524102+0000","last_unstale":"2024-09-18T10:16:48.501848+0000","last_undegraded":"2024-09-18T10:16:48.501848+0000","last_fullsized":"2024-09-18T10:16:48.501848+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:16:02.749150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:16:02.749150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:17:04.725472+0000","scrub_duration":0.20368988499999999,"objects_trimmed":0,"snaptrim_duration":0.10077985,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"29'2","reported_seq":145,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-18T10:16:49.304395+0000","last_change":"2024-09-18T10:16:20.864837+0000","last_active":"2024-09-18T10:16:49.304395+0000","last_peered":"2024-09-18T10:16:49.304395+0000","last_clean":"2024-09-18T10:16:49.304395+0000","last_became_active":"2024-09-18T10:14:35.715374+0000","last_became_peered":"2024-09-18T10:14:35.715374+0000","last_unstale":"2024-09-18T10:16:49.304395+0000","last_undegraded":"2024-09-18T10:16:49.304395+0000","last_fullsized":"2024-09-18T10:16:49.304395+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2","last_scrub_stamp":"2024-09-18T10:16:20.864635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:16:20.864635+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:17:23.226052+0000","scrub_duration":0.20463374300000001,"objects_trimmed":0,"snaptrim_duration":0.002421224,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,12],"acting":[13,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":150,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-18T10:16:49.186308+0000","last_change":"2024-09-18T10:16:15.062108+0000","last_active":"2024-09-18T10:16:49.186308+0000","last_peered":"2024-09-18T10:16:49.186308+0000","last_clean":"2024-09-18T10:16:49.186308+0000","last_became_active":"2024-09-18T10:14:35.348262+0000","last_became_peered":"2024-09-18T10:14:35.348262+0000","last_unstale":"2024-09-18T10:16:49.186308+0000","last_undegraded":"2024-09-18T10:16:49.186308+0000","last_fullsized":"2024-09-18T10:16:49.186308+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:16:15.061892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:16:15.061892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:17:40.002913+0000","scrub_duration":0.40410323300000001,"objects_trimmed":0,"snaptrim_duration":0.00063430400000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,0],"acting":[14,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":144,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-18T10:16:52.538159+0000","last_change":"2024-09-18T10:16:15.352277+0000","last_active":"2024-09-18T10:16:52.538159+0000","last_peered":"2024-09-18T10:16:52.538159+0000","last_clean":"2024-09-18T10:16:52.538159+0000","last_became_active":"2024-09-18T10:14:35.349203+0000","last_became_peered":"2024-09-18T10:14:35.349203+0000","last_unstale":"2024-09-18T10:16:52.538159+0000","last_undegraded":"2024-09-18T10:16:52.538159+0000","last_fullsized":"2024-09-18T10:16:52.538159+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:16:15.352050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:16:15.352050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:17:45.176858+0000","scrub_duration":0.203632596,"objects_trimmed":0,"snaptrim_duration":0.00060130600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3],"acting":[9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"27'1","reported_seq":145,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-18T10:16:49.186339+0000","last_change":"2024-09-18T10:16:06.900858+0000","last_active":"2024-09-18T10:16:49.186339+0000","last_peered":"2024-09-18T10:16:49.186339+0000","last_clean":"2024-09-18T10:16:49.186339+0000","last_became_active":"2024-09-18T10:14:35.487746+0000","last_became_peered":"2024-09-18T10:14:35.487746+0000","last_unstale":"2024-09-18T10:16:49.186339+0000","last_undegraded":"2024-09-18T10:16:49.186339+0000","last_fullsized":"2024-09-18T10:16:49.186339+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'1","last_scrub_stamp":"2024-09-18T10:16:06.900601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:16:06.900601+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:17:25.247813+0000","scrub_duration":0.27078197700000001,"objects_trimmed":0,"snaptrim_duration":0.10072764100000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,5],"acting":[14,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"1.0","version":"29'112","reported_seq":275,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-18T10:16:49.187571+0000","last_change":"2024-09-18T10:16:03.315273+0000","last_active":"2024-09-18T10:16:49.187571+0000","last_peered":"2024-09-18T10:16:49.187571+0000","last_clean":"2024-09-18T10:16:49.187571+0000","last_became_active":"2024-09-18T10:14:32.508037+0000","last_became_peered":"2024-09-18T10:14:32.508037+0000","last_unstale":"2024-09-18T10:16:49.187571+0000","last_undegraded":"2024-09-18T10:16:49.187571+0000","last_fullsized":"2024-09-18T10:16:49.187571+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'112","last_scrub_stamp":"2024-09-18T10:16:03.315126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:31.267741+0000","last_clean_scrub_stamp":"2024-09-18T10:16:03.315126+0000","objects_scrubbed":2,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:17:16.808627+0000","scrub_duration":0.10318229800000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,15],"acting":[6,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":13},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":112,"ondisk_log_size":112,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":15,"up_from":22,"seq":94489280543,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304272,"kb_used_data":2752,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94067568,"statfs":{"total":96636764160,"available":96325189632,"internally_reserved":0,"allocated":2818048,"data_stored":1860517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,4,6,7,8,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:16:32 2024","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":0.83899999999999997,"15min":0.81899999999999995},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":2.0019999999999998,"5min":2.0019999999999998,"15min":2.0019999999999998},"last":1.0649999999999999},{"interface":"front","average":{"1min":0.996,"5min":5.5739999999999998,"15min":6.3369999999999997},"min":{"1min":0.72899999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.506,"5min":101.291,"15min":101.291},"last":0.97299999999999998}]},{"osd":1,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.93300000000000005},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.931,"5min":1.931,"15min":1.931},"last":1.1279999999999999}]},{"osd":2,"last update":"Wed Sep 18 10:16:32 2024","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.0169999999999999,"15min":0.995},"min":{"1min":0.79700000000000004,"5min":0.5,"15min":0.5},"max":{"1min":1.7470000000000001,"5min":1.7470000000000001,"15min":1.7470000000000001},"last":1.028},{"interface":"front","average":{"1min":1.1419999999999999,"5min":4.6589999999999998,"15min":5.2450000000000001},"min":{"1min":0.82899999999999996,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.4379999999999999,"5min":78.242999999999995,"15min":78.242999999999995},"last":1.252}]},{"osd":3,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":3.3029999999999999,"5min":3.3029999999999999,"15min":3.3029999999999999},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":41.720999999999997,"5min":41.720999999999997,"15min":41.720999999999997},"last":0.55400000000000005},{"interface":"front","average":{"1min":1.014,"5min":1.014,"15min":1.014},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.49,"5min":1.49,"15min":1.49},"last":0.89500000000000002}]},{"osd":4,"last update":"Wed Sep 18 10:16:32 2024","interfaces":[{"interface":"back","average":{"1min":1.3580000000000001,"5min":9.0879999999999992,"15min":10.375999999999999},"min":{"1min":0.747,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":8.0449999999999999,"5min":99.745999999999995,"15min":99.745999999999995},"last":1.2729999999999999},{"interface":"front","average":{"1min":1.595,"5min":9.2309999999999999,"15min":10.503},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":8.0679999999999996,"5min":99.781000000000006,"15min":99.781000000000006},"last":1.222}]},{"osd":5,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":1.0860000000000001},{"interface":"front","average":{"1min":1.7869999999999999,"5min":1.7869999999999999,"15min":1.7869999999999999},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":10.874000000000001,"5min":10.874000000000001,"15min":10.874000000000001},"last":1.163}]},{"osd":6,"last update":"Wed Sep 18 10:16:32 2024","interfaces":[{"interface":"back","average":{"1min":14.140000000000001,"5min":4.4039999999999999,"15min":2.782},"min":{"1min":0.92900000000000005,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":269.80500000000001,"5min":269.80500000000001,"15min":269.80500000000001},"last":1},{"interface":"front","average":{"1min":1.361,"5min":4.7610000000000001,"15min":5.327},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":2.012,"5min":74.156000000000006,"15min":74.156000000000006},"last":1.1919999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80400000000000005}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93000000000000005}]},{"osd":10,"last update":"Wed Sep 18 10:16:32 2024","interfaces":[{"interface":"back","average":{"1min":5.8639999999999999,"5min":6.2670000000000003,"15min":6.3339999999999996},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":93.704999999999998,"5min":93.704999999999998,"15min":93.704999999999998},"last":0.85399999999999998},{"interface":"front","average":{"1min":5.7770000000000001,"5min":6.2549999999999999,"15min":6.335},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":93.671000000000006,"5min":93.671000000000006,"15min":93.671000000000006},"last":1.121}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.254}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3680000000000001}]},{"osd":13,"last update":"Wed Sep 18 10:16:32 2024","interfaces":[{"interface":"back","average":{"1min":1.99,"5min":5.2629999999999999,"15min":5.8090000000000002},"min":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"max":{"1min":7.3179999999999996,"5min":80.197000000000003,"15min":80.197000000000003},"last":1.5029999999999999},{"interface":"front","average":{"1min":3.4830000000000001,"5min":1.8080000000000001,"15min":1.5289999999999999},"min":{"1min":0.89400000000000002,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":25.193999999999999,"5min":25.193999999999999,"15min":25.193999999999999},"last":1.4670000000000001}]},{"osd":14,"last update":"Wed Sep 18 10:16:32 2024","interfaces":[{"interface":"back","average":{"1min":10.725,"5min":3.2999999999999998,"15min":2.0619999999999998},"min":{"1min":0.95699999999999996,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":117.562,"5min":117.562,"15min":117.562},"last":1.5980000000000001},{"interface":"front","average":{"1min":4.6139999999999999,"5min":9.5229999999999997,"15min":10.340999999999999},"min":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"max":{"1min":61.567,"5min":85.006,"15min":85.006},"last":1.4390000000000001}]}]},{"osd":14,"up_from":22,"seq":94489280543,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303768,"kb_used_data":2248,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068072,"statfs":{"total":96636764160,"available":96325705728,"internally_reserved":0,"allocated":2301952,"data_stored":1341063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,5,7,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:16:32 2024","interfaces":[{"interface":"back","average":{"1min":1.0009999999999999,"5min":12.513,"15min":14.430999999999999},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.405,"5min":189.88800000000001,"15min":189.88800000000001},"last":0.59599999999999997},{"interface":"front","average":{"1min":7.0330000000000004,"5min":17.033000000000001,"15min":18.699000000000002},"min":{"1min":0.77300000000000002,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":71.900999999999996,"5min":180.09999999999999,"15min":180.09999999999999},"last":0.68700000000000006}]},{"osd":1,"last update":"Wed Sep 18 10:16:32 2024","interfaces":[{"interface":"back","average":{"1min":2.1040000000000001,"5min":7.7670000000000003,"15min":8.7110000000000003},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":21.334,"5min":79.626000000000005,"15min":79.626000000000005},"last":0.754},{"interface":"front","average":{"1min":1.1479999999999999,"5min":12.601000000000001,"15min":14.51},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.8879999999999999,"5min":189.762,"15min":189.762},"last":0.85899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.417}]},{"osd":3,"last update":"Wed Sep 18 10:16:32 2024","interfaces":[{"interface":"back","average":{"1min":3.5430000000000001,"5min":12.906000000000001,"15min":14.465999999999999},"min":{"1min":0.84799999999999998,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":44.683,"5min":180.19900000000001,"15min":180.19900000000001},"last":1.7529999999999999},{"interface":"front","average":{"1min":5.3520000000000003,"5min":18.143999999999998,"15min":20.276},"min":{"1min":0.86399999999999999,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":42.570999999999998,"5min":180.851,"15min":180.851},"last":1.1899999999999999}]},{"osd":4,"last update":"Wed Sep 18 10:16:32 2024","interfaces":[{"interface":"back","average":{"1min":6.8360000000000003,"5min":21.687000000000001,"15min":24.161999999999999},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":71.911000000000001,"5min":189.809,"15min":189.809},"last":2.0249999999999999},{"interface":"front","average":{"1min":14.032999999999999,"5min":15.031000000000001,"15min":15.196999999999999},"min":{"1min":0.76100000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":133.244,"5min":180.31700000000001,"15min":180.31700000000001},"last":1.9350000000000001}]},{"osd":5,"last update":"Wed Sep 18 10:16:32 2024","interfaces":[{"interface":"back","average":{"1min":1.831,"5min":21.507000000000001,"15min":24.786000000000001},"min":{"1min":0.66300000000000003,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":13.42,"5min":277.41199999999998,"15min":277.41199999999998},"last":1.1020000000000001},{"interface":"front","average":{"1min":1.9530000000000001,"5min":12.936999999999999,"15min":14.766999999999999},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":14.288,"5min":189.64699999999999,"15min":189.64699999999999},"last":0.91900000000000004}]},{"osd":7,"last update":"Wed Sep 18 10:16:32 2024","interfaces":[{"interface":"back","average":{"1min":6.29,"5min":13.625999999999999,"15min":14.848000000000001},"min":{"1min":0.995,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":49.378,"5min":180.268,"15min":180.268},"last":1.448},{"interface":"front","average":{"1min":3.8929999999999998,"5min":12.901,"15min":14.403},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":49.402999999999999,"5min":180.631,"15min":180.631},"last":1.494}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1619999999999999}]},{"osd":9,"last update":"Wed Sep 18 10:16:17 2024","interfaces":[{"interface":"back","average":{"1min":4,"5min":4,"15min":4},"min":{"1min":1.296,"5min":1.296,"15min":1.296},"max":{"1min":44.716000000000001,"5min":44.716000000000001,"15min":44.716000000000001},"last":1.204},{"interface":"front","average":{"1min":1.427,"5min":1.427,"15min":1.427},"min":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"max":{"1min":1.9610000000000001,"5min":1.9610000000000001,"15min":1.9610000000000001},"last":1.274}]},{"osd":10,"last update":"Wed Sep 18 10:15:49 2024","interfaces":[{"interface":"back","average":{"1min":17.039000000000001,"5min":17.039000000000001,"15min":17.039000000000001},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":180.15899999999999,"5min":180.15899999999999,"15min":180.15899999999999},"last":1},{"interface":"front","average":{"1min":31.004000000000001,"5min":31.004000000000001,"15min":31.004000000000001},"min":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":269.423,"5min":269.423,"15min":269.423},"last":1.381}]},{"osd":11,"last update":"Wed Sep 18 10:16:10 2024","interfaces":[{"interface":"back","average":{"1min":9.6890000000000001,"5min":9.6890000000000001,"15min":9.6890000000000001},"min":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":91.111999999999995,"5min":91.111999999999995,"15min":91.111999999999995},"last":1.3520000000000001},{"interface":"front","average":{"1min":8.1460000000000008,"5min":8.1460000000000008,"15min":8.1460000000000008},"min":{"1min":1.181,"5min":1.181,"15min":1.181},"max":{"1min":91.165000000000006,"5min":91.165000000000006,"15min":91.165000000000006},"last":1.1519999999999999}]},{"osd":12,"last update":"Wed Sep 18 10:16:32 2024","interfaces":[{"interface":"back","average":{"1min":10.385,"5min":14.760999999999999,"15min":15.49},"min":{"1min":1.327,"5min":0.91100000000000003,"15min":0.91100000000000003},"max":{"1min":65.189999999999998,"5min":189.84999999999999,"15min":189.84999999999999},"last":1.4079999999999999},{"interface":"front","average":{"1min":10.818,"5min":14.696,"15min":15.342000000000001},"min":{"1min":1.4890000000000001,"5min":0.91200000000000003,"15min":0.91200000000000003},"max":{"1min":65.162000000000006,"5min":180.381,"15min":180.381},"last":1.7729999999999999}]},{"osd":13,"last update":"Wed Sep 18 10:16:32 2024","interfaces":[{"interface":"back","average":{"1min":10.321999999999999,"5min":23.059000000000001,"15min":25.181999999999999},"min":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"max":{"1min":166.001,"5min":180.59100000000001,"15min":180.59100000000001},"last":1.5720000000000001},{"interface":"front","average":{"1min":1.647,"5min":20.391999999999999,"15min":23.515999999999998},"min":{"1min":0.83399999999999996,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":2.3940000000000001,"5min":189.71299999999999,"15min":189.71299999999999},"last":1.3120000000000001}]},{"osd":15,"last update":"Wed Sep 18 10:16:32 2024","interfaces":[{"interface":"back","average":{"1min":4.181,"5min":13.381,"15min":14.914},"min":{"1min":1.5780000000000001,"5min":0.99399999999999999,"15min":0.99399999999999999},"max":{"1min":44.917000000000002,"5min":180.44800000000001,"15min":180.44800000000001},"last":1.6619999999999999},{"interface":"front","average":{"1min":1.7030000000000001,"5min":15.118,"15min":17.353999999999999},"min":{"1min":0.93200000000000005,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":2.1909999999999998,"5min":180.53,"15min":180.53},"last":1.8069999999999999}]}]},{"osd":13,"up_from":22,"seq":94489280544,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307932,"kb_used_data":2252,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94063908,"statfs":{"total":96636764160,"available":96321441792,"internally_reserved":0,"allocated":2306048,"data_stored":1341082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,3,6,8,9,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:16:33 2024","interfaces":[{"interface":"back","average":{"1min":2.2599999999999998,"5min":17.045000000000002,"15min":19.509},"min":{"1min":0.65700000000000003,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":22.972999999999999,"5min":176.66800000000001,"15min":176.66800000000001},"last":0.83899999999999997},{"interface":"front","average":{"1min":4.2629999999999999,"5min":25.199000000000002,"15min":28.687999999999999},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":62.533000000000001,"5min":199.732,"15min":199.732},"last":0.90000000000000002}]},{"osd":1,"last update":"Wed Sep 18 10:15:48 2024","interfaces":[{"interface":"back","average":{"1min":1.986,"5min":1.986,"15min":1.986},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":18.471,"5min":18.471,"15min":18.471},"last":1.4219999999999999},{"interface":"front","average":{"1min":20.716999999999999,"5min":20.716999999999999,"15min":20.716999999999999},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":176.56800000000001,"5min":176.56800000000001,"15min":176.56800000000001},"last":0.86399999999999999}]},{"osd":2,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":11.984,"5min":11.984,"15min":11.984},"min":{"1min":0.751,"5min":0.751,"15min":0.751},"max":{"1min":196.59200000000001,"5min":196.59200000000001,"15min":196.59200000000001},"last":0.97699999999999998},{"interface":"front","average":{"1min":26.109999999999999,"5min":26.109999999999999,"15min":26.109999999999999},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":177.03,"5min":177.03,"15min":177.03},"last":1.2689999999999999}]},{"osd":3,"last update":"Wed Sep 18 10:16:33 2024","interfaces":[{"interface":"back","average":{"1min":9.5,"5min":22.245000000000001,"15min":24.369},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":158.74100000000001,"5min":168.27699999999999,"15min":168.27699999999999},"last":1.27},{"interface":"front","average":{"1min":1.2290000000000001,"5min":9.8819999999999997,"15min":11.324},"min":{"1min":0.84499999999999997,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.8959999999999999,"5min":196.643,"15min":196.643},"last":0.95399999999999996}]},{"osd":4,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":29.802,"5min":29.802,"15min":29.802},"min":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"max":{"1min":168.48099999999999,"5min":168.48099999999999,"15min":168.48099999999999},"last":1.347},{"interface":"front","average":{"1min":8.1780000000000008,"5min":8.1780000000000008,"15min":8.1780000000000008},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":65.188000000000002,"5min":65.188000000000002,"15min":65.188000000000002},"last":1.6379999999999999}]},{"osd":5,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":26.152999999999999,"5min":26.152999999999999,"15min":26.152999999999999},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":176.48099999999999,"5min":176.48099999999999,"15min":176.48099999999999},"last":1.49},{"interface":"front","average":{"1min":24.885999999999999,"5min":24.885999999999999,"15min":24.885999999999999},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":167.62299999999999,"5min":167.62299999999999,"15min":167.62299999999999},"last":1.528}]},{"osd":6,"last update":"Wed Sep 18 10:16:33 2024","interfaces":[{"interface":"back","average":{"1min":2.6589999999999998,"5min":24.015000000000001,"15min":27.574999999999999},"min":{"1min":1.042,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":23.131,"5min":176.81200000000001,"15min":176.81200000000001},"last":1.1080000000000001},{"interface":"front","average":{"1min":4.7169999999999996,"5min":23.227,"15min":26.312000000000001},"min":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"max":{"1min":61.874000000000002,"5min":167.947,"15min":167.947},"last":1.3340000000000001}]},{"osd":8,"last update":"Wed Sep 18 10:16:50 2024","interfaces":[{"interface":"back","average":{"1min":1.5700000000000001,"5min":1.5700000000000001,"15min":1.5700000000000001},"min":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"max":{"1min":2.117,"5min":2.117,"15min":2.117},"last":1.175},{"interface":"front","average":{"1min":1.514,"5min":1.514,"15min":1.514},"min":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"max":{"1min":2.278,"5min":2.278,"15min":2.278},"last":1.149}]},{"osd":9,"last update":"Wed Sep 18 10:16:50 2024","interfaces":[{"interface":"back","average":{"1min":2.617,"5min":17.367000000000001,"15min":19.824999999999999},"min":{"1min":0.878,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":22.934000000000001,"5min":176.73599999999999,"15min":176.73599999999999},"last":1.206},{"interface":"front","average":{"1min":1.5660000000000001,"5min":19.085000000000001,"15min":22.004999999999999},"min":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":2.448,"5min":167.84899999999999,"15min":167.84899999999999},"last":1.446}]},{"osd":10,"last update":"Wed Sep 18 10:16:39 2024","interfaces":[{"interface":"back","average":{"1min":5.1120000000000001,"5min":19.164999999999999,"15min":21.507000000000001},"min":{"1min":1.22,"5min":1.1359999999999999,"15min":1.1359999999999999},"max":{"1min":62.137999999999998,"5min":168.07900000000001,"15min":168.07900000000001},"last":1.591},{"interface":"front","average":{"1min":17.683,"5min":29.114000000000001,"15min":31.018999999999998},"min":{"1min":1.123,"5min":1.0649999999999999,"15min":1.0649999999999999},"max":{"1min":289.68700000000001,"5min":289.68700000000001,"15min":289.68700000000001},"last":1.2450000000000001}]},{"osd":11,"last update":"Wed Sep 18 10:16:50 2024","interfaces":[{"interface":"back","average":{"1min":1.6819999999999999,"5min":1.625,"15min":1.615},"min":{"1min":1.056,"5min":1.0029999999999999,"15min":1.0029999999999999},"max":{"1min":2.5699999999999998,"5min":2.5699999999999998,"15min":2.5699999999999998},"last":1.5009999999999999},{"interface":"front","average":{"1min":2.798,"5min":21.73,"15min":24.885999999999999},"min":{"1min":1.286,"5min":1.028,"15min":1.028},"max":{"1min":23.167999999999999,"5min":176.94999999999999,"15min":176.94999999999999},"last":1.377}]},{"osd":12,"last update":"Wed Sep 18 10:16:33 2024","interfaces":[{"interface":"back","average":{"1min":1.8440000000000001,"5min":1.6339999999999999,"15min":1.599},"min":{"1min":1.2490000000000001,"5min":1.0009999999999999,"15min":1.0009999999999999},"max":{"1min":2.5950000000000002,"5min":2.7879999999999998,"15min":2.7879999999999998},"last":1.413},{"interface":"front","average":{"1min":5.0570000000000004,"5min":18.542999999999999,"15min":20.791},"min":{"1min":1.3029999999999999,"5min":1.296,"15min":1.296},"max":{"1min":62.200000000000003,"5min":168.34399999999999,"15min":168.34399999999999},"last":1.5449999999999999}]},{"osd":14,"last update":"Wed Sep 18 10:16:33 2024","interfaces":[{"interface":"back","average":{"1min":1.8320000000000001,"5min":5.7210000000000001,"15min":6.3689999999999998},"min":{"1min":1.139,"5min":1.0640000000000001,"15min":1.0640000000000001},"max":{"1min":2.419,"5min":87.427000000000007,"15min":87.427000000000007},"last":1.665},{"interface":"front","average":{"1min":2.9329999999999998,"5min":21.571999999999999,"15min":24.678000000000001},"min":{"1min":1.2709999999999999,"5min":1.0980000000000001,"15min":1.0980000000000001},"max":{"1min":23.268000000000001,"5min":176.87700000000001,"15min":176.87700000000001},"last":1.468}]},{"osd":15,"last update":"Wed Sep 18 10:16:33 2024","interfaces":[{"interface":"back","average":{"1min":10.048999999999999,"5min":18.277000000000001,"15min":19.648},"min":{"1min":1.272,"5min":1.266,"15min":1.266},"max":{"1min":96.564999999999998,"5min":168.40700000000001,"15min":168.40700000000001},"last":1.7110000000000001},{"interface":"front","average":{"1min":1.891,"5min":13.459,"15min":15.387},"min":{"1min":1.1950000000000001,"5min":0.95099999999999996,"15min":0.95099999999999996},"max":{"1min":2.5169999999999999,"5min":196.512,"15min":196.512},"last":1.5609999999999999}]}]},{"osd":0,"up_from":21,"seq":90194313248,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303780,"kb_used_data":2264,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068060,"statfs":{"total":96636764160,"available":96325693440,"internally_reserved":0,"allocated":2318336,"data_stored":1350579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,4,5,6,9,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 18 10:16:39 2024","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.95699999999999996,"15min":0.95899999999999996},"min":{"1min":0.56699999999999995,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.2889999999999999,"5min":1.341,"15min":1.341},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.97899999999999998,"15min":0.98499999999999999},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.2310000000000001,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":0.625}]},{"osd":2,"last update":"Wed Sep 18 10:16:39 2024","interfaces":[{"interface":"back","average":{"1min":3.2719999999999998,"5min":1.6020000000000001,"15min":1.3240000000000001},"min":{"1min":0.76600000000000001,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":46.220999999999997,"5min":46.220999999999997,"15min":46.220999999999997},"last":1.0349999999999999},{"interface":"front","average":{"1min":0.98799999999999999,"5min":1.071,"15min":1.085},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.732,"5min":1.732,"15min":1.732},"last":0.57499999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.196}]},{"osd":4,"last update":"Wed Sep 18 10:16:30 2024","interfaces":[{"interface":"back","average":{"1min":8.4039999999999999,"5min":14,"15min":14.933},"min":{"1min":0.59699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":142.20699999999999,"5min":142.20699999999999,"15min":142.20699999999999},"last":0.49399999999999999},{"interface":"front","average":{"1min":10.48,"5min":14.896000000000001,"15min":15.632},"min":{"1min":0.63,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":132.33799999999999,"5min":132.33799999999999,"15min":132.33799999999999},"last":0.55200000000000005}]},{"osd":5,"last update":"Wed Sep 18 10:16:39 2024","interfaces":[{"interface":"back","average":{"1min":1.1739999999999999,"5min":1.24,"15min":1.2509999999999999},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":1.619,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":1.569,"5min":8.1340000000000003,"15min":9.2279999999999998},"min":{"1min":0.85599999999999998,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":10.143000000000001,"5min":130.39599999999999,"15min":130.39599999999999},"last":0.92100000000000004}]},{"osd":6,"last update":"Wed Sep 18 10:16:47 2024","interfaces":[{"interface":"back","average":{"1min":1.266,"5min":1.254,"15min":1.252},"min":{"1min":0.96099999999999997,"5min":0.94299999999999995,"15min":0.94299999999999995},"max":{"1min":1.571,"5min":1.776,"15min":1.776},"last":0.82599999999999996},{"interface":"front","average":{"1min":1.3109999999999999,"5min":2.0270000000000001,"15min":2.1469999999999998},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.7909999999999999,"5min":15.250999999999999,"15min":15.250999999999999},"last":0.66100000000000003}]},{"osd":9,"last update":"Wed Sep 18 10:16:30 2024","interfaces":[{"interface":"back","average":{"1min":1.3360000000000001,"5min":1.3919999999999999,"15min":1.4019999999999999},"min":{"1min":0.88200000000000001,"5min":0.879,"15min":0.879},"max":{"1min":1.863,"5min":1.863,"15min":1.863},"last":0.74099999999999999},{"interface":"front","average":{"1min":1.667,"5min":8.7590000000000003,"15min":9.9410000000000007},"min":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":10.180999999999999,"5min":130.441,"15min":130.441},"last":0.88400000000000001}]},{"osd":11,"last update":"Wed Sep 18 10:16:30 2024","interfaces":[{"interface":"back","average":{"1min":4.8949999999999996,"5min":2.2109999999999999,"15min":1.7629999999999999},"min":{"1min":1.1579999999999999,"5min":0.90700000000000003,"15min":0.90700000000000003},"max":{"1min":41.610999999999997,"5min":41.610999999999997,"15min":41.610999999999997},"last":1.0700000000000001},{"interface":"front","average":{"1min":1.5029999999999999,"5min":4.5620000000000003,"15min":5.0720000000000001},"min":{"1min":1.1220000000000001,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":1.9850000000000001,"5min":54.805999999999997,"15min":54.805999999999997},"last":0.87}]},{"osd":12,"last update":"Wed Sep 18 10:16:30 2024","interfaces":[{"interface":"back","average":{"1min":1.7869999999999999,"5min":9.1170000000000009,"15min":10.339},"min":{"1min":0.95999999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":10.015000000000001,"5min":130.34800000000001,"15min":130.34800000000001},"last":0.95399999999999996},{"interface":"front","average":{"1min":1.4350000000000001,"5min":1.4710000000000001,"15min":1.4770000000000001},"min":{"1min":0.85899999999999999,"5min":0.502,"15min":0.502},"max":{"1min":1.944,"5min":1.944,"15min":1.944},"last":0.85499999999999998}]},{"osd":14,"last update":"Wed Sep 18 10:16:39 2024","interfaces":[{"interface":"back","average":{"1min":3.7490000000000001,"5min":8.7810000000000006,"15min":9.6199999999999992},"min":{"1min":1.044,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":38.969999999999999,"5min":130.48500000000001,"15min":130.48500000000001},"last":0.98699999999999999},{"interface":"front","average":{"1min":6.931,"5min":2.6339999999999999,"15min":1.9179999999999999},"min":{"1min":1.2609999999999999,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":80.460999999999999,"5min":80.460999999999999,"15min":80.460999999999999},"last":1.099}]},{"osd":15,"last update":"Wed Sep 18 10:16:39 2024","interfaces":[{"interface":"back","average":{"1min":1.591,"5min":2.778,"15min":2.976},"min":{"1min":1.1699999999999999,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":2.0310000000000001,"5min":23.806999999999999,"15min":23.806999999999999},"last":0.77200000000000002},{"interface":"front","average":{"1min":6.9199999999999999,"5min":9.4809999999999999,"15min":9.9079999999999995},"min":{"1min":1.1279999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":100.925,"5min":130.52799999999999,"15min":130.52799999999999},"last":1.018}]}]},{"osd":4,"up_from":21,"seq":90194313249,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303752,"kb_used_data":2232,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068088,"statfs":{"total":96636764160,"available":96325722112,"internally_reserved":0,"allocated":2285568,"data_stored":1331589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,3,5,7,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":9.0980000000000008,"15min":10.465999999999999},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.7529999999999999,"5min":89.266999999999996,"15min":89.266999999999996},"last":0.626},{"interface":"front","average":{"1min":18.640000000000001,"5min":7.7320000000000002,"15min":5.9139999999999997},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":198.80699999999999,"5min":198.80699999999999,"15min":198.80699999999999},"last":0.66000000000000003}]},{"osd":1,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":4.0759999999999996,"5min":12.537000000000001,"15min":13.946999999999999},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":62.780000000000001,"5min":164.21199999999999,"15min":164.21199999999999},"last":0.72099999999999997},{"interface":"front","average":{"1min":2.5139999999999998,"5min":9.593,"15min":10.773},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":25.802,"5min":89.331000000000003,"15min":89.331000000000003},"last":0.77500000000000002}]},{"osd":2,"last update":"Wed Sep 18 10:15:47 2024","interfaces":[{"interface":"back","average":{"1min":23.617000000000001,"5min":23.617000000000001,"15min":23.617000000000001},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":192.136,"5min":192.136,"15min":192.136},"last":1.8740000000000001},{"interface":"front","average":{"1min":11.888999999999999,"5min":11.888999999999999,"15min":11.888999999999999},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":198.87700000000001,"5min":198.87700000000001,"15min":198.87700000000001},"last":1.347}]},{"osd":3,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":35.673000000000002,"5min":21.670999999999999,"15min":19.337},"min":{"1min":1.161,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":388.84300000000002,"5min":388.84300000000002,"15min":388.84300000000002},"last":1.462},{"interface":"front","average":{"1min":17.747,"5min":25.643999999999998,"15min":26.960000000000001},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":255.82900000000001,"5min":255.82900000000001,"15min":255.82900000000001},"last":0.997}]},{"osd":5,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":1.4910000000000001,"5min":15.449999999999999,"15min":17.776},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":2.169,"5min":189.55600000000001,"15min":189.55600000000001},"last":0.83099999999999996},{"interface":"front","average":{"1min":21.053000000000001,"5min":6.8079999999999998,"15min":4.4340000000000002},"min":{"1min":1.3640000000000001,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":199.15600000000001,"5min":199.15600000000001,"15min":199.15600000000001},"last":1.155}]},{"osd":6,"last update":"Wed Sep 18 10:15:34 2024","interfaces":[{"interface":"back","average":{"1min":32.25,"5min":32.25,"15min":32.25},"min":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":197.33199999999999,"5min":197.33199999999999,"15min":197.33199999999999},"last":1.8160000000000001},{"interface":"front","average":{"1min":18.963000000000001,"5min":18.963000000000001,"15min":18.963000000000001},"min":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"max":{"1min":179.19200000000001,"5min":179.19200000000001,"15min":179.19200000000001},"last":1.7769999999999999}]},{"osd":7,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":5.6050000000000004,"5min":23.198,"15min":26.129999999999999},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":62.704999999999998,"5min":290.036,"15min":290.036},"last":1.2689999999999999},{"interface":"front","average":{"1min":1.6639999999999999,"5min":14.273,"15min":16.375},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":2.3100000000000001,"5min":139.696,"15min":139.696},"last":0.93899999999999995}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1509999999999998}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7509999999999999}]},{"osd":10,"last update":"Wed Sep 18 10:16:49 2024","interfaces":[{"interface":"back","average":{"1min":1.653,"5min":18.559000000000001,"15min":21.376999999999999},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":2.4740000000000002,"5min":164.43899999999999,"15min":164.43899999999999},"last":0.89800000000000002},{"interface":"front","average":{"1min":4.6070000000000002,"5min":10.993,"15min":12.057},"min":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":59.408999999999999,"5min":89.442999999999998,"15min":89.442999999999998},"last":1.3049999999999999}]},{"osd":11,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":19.57,"5min":12.567,"15min":11.4},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":199.255,"5min":199.255,"15min":199.255},"last":1.3340000000000001},{"interface":"front","average":{"1min":11.393000000000001,"5min":22.213000000000001,"15min":24.016999999999999},"min":{"1min":1.0840000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"max":{"1min":155.19800000000001,"5min":164.733,"15min":164.733},"last":1.4399999999999999}]},{"osd":12,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":4.5800000000000001,"5min":10.406000000000001,"15min":11.377000000000001},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":58.856000000000002,"5min":89.400000000000006,"15min":89.400000000000006},"last":1.113},{"interface":"front","average":{"1min":31.831,"5min":12.901999999999999,"15min":9.7469999999999999},"min":{"1min":1.4530000000000001,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":199.32900000000001,"5min":199.32900000000001,"15min":199.32900000000001},"last":1.5940000000000001}]},{"osd":13,"last update":"Wed Sep 18 10:16:49 2024","interfaces":[{"interface":"back","average":{"1min":12.695,"5min":22.789999999999999,"15min":24.472000000000001},"min":{"1min":1.702,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":192.37299999999999,"5min":199.37799999999999,"15min":199.37799999999999},"last":1.706},{"interface":"front","average":{"1min":1.8029999999999999,"5min":19.068000000000001,"15min":21.946000000000002},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":2.5699999999999998,"5min":164.53,"15min":164.53},"last":1.375}]},{"osd":14,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":22.312000000000001,"5min":21.788,"15min":21.701000000000001},"min":{"1min":1.556,"5min":0.96099999999999997,"15min":0.96099999999999997},"max":{"1min":199.43000000000001,"5min":202.191,"15min":202.191},"last":1.782},{"interface":"front","average":{"1min":9.8800000000000008,"5min":17.420999999999999,"15min":18.678000000000001},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":101.492,"5min":164.35400000000001,"15min":164.35400000000001},"last":1.524}]}]},{"osd":9,"up_from":21,"seq":90194313248,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303784,"kb_used_data":2264,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068056,"statfs":{"total":96636764160,"available":96325689344,"internally_reserved":0,"allocated":2318336,"data_stored":1350579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,4,5,6,8,10,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":7.2450000000000001,"5min":7.2450000000000001,"15min":7.2450000000000001},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":127.902,"5min":127.902,"15min":127.902},"last":0.97399999999999998},{"interface":"front","average":{"1min":8.3109999999999999,"5min":8.3109999999999999,"15min":8.3109999999999999},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":96.203000000000003,"5min":96.203000000000003,"15min":96.203000000000003},"last":0.81200000000000006}]},{"osd":1,"last update":"Wed Sep 18 10:16:40 2024","interfaces":[{"interface":"back","average":{"1min":10.587,"5min":10.587,"15min":10.587},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":166.833,"5min":166.833,"15min":166.833},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.96099999999999997}]},{"osd":2,"last update":"Wed Sep 18 10:16:18 2024","interfaces":[{"interface":"back","average":{"1min":5.6420000000000003,"5min":5.6420000000000003,"15min":5.6420000000000003},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":68.049999999999997,"5min":68.049999999999997,"15min":68.049999999999997},"last":1.1579999999999999},{"interface":"front","average":{"1min":2.0310000000000001,"5min":2.0310000000000001,"15min":2.0310000000000001},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":18.553999999999998,"5min":18.553999999999998,"15min":18.553999999999998},"last":1.4119999999999999}]},{"osd":3,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":2.875,"5min":5.5549999999999997,"15min":6.0019999999999998},"min":{"1min":0.72699999999999998,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":18.431000000000001,"5min":98.888999999999996,"15min":98.888999999999996},"last":1.0389999999999999},{"interface":"front","average":{"1min":3.0169999999999999,"5min":5.5890000000000004,"15min":6.0170000000000003},"min":{"1min":0.749,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":18.472000000000001,"5min":98.947999999999993,"15min":98.947999999999993},"last":1.546}]},{"osd":4,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":8.2360000000000007,"5min":2.5840000000000001,"15min":1.6419999999999999},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":124.15300000000001,"5min":124.15300000000001,"15min":124.15300000000001},"last":1.371},{"interface":"front","average":{"1min":6.2380000000000004,"5min":8.8539999999999992,"15min":9.2899999999999991},"min":{"1min":0.81000000000000005,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":73.992000000000004,"5min":73.992000000000004,"15min":73.992000000000004},"last":1.6319999999999999}]},{"osd":5,"last update":"Wed Sep 18 10:16:18 2024","interfaces":[{"interface":"back","average":{"1min":1.821,"5min":1.821,"15min":1.821},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":14.786,"5min":14.786,"15min":14.786},"last":0.79800000000000004},{"interface":"front","average":{"1min":8.5,"5min":8.5,"15min":8.5},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":67.915999999999997,"5min":67.915999999999997,"15min":67.915999999999997},"last":1.27}]},{"osd":6,"last update":"Wed Sep 18 10:16:18 2024","interfaces":[{"interface":"back","average":{"1min":8.3840000000000003,"5min":8.3840000000000003,"15min":8.3840000000000003},"min":{"1min":0.997,"5min":0.997,"15min":0.997},"max":{"1min":70.914000000000001,"5min":70.914000000000001,"15min":70.914000000000001},"last":1.806},{"interface":"front","average":{"1min":15.705,"5min":15.705,"15min":15.705},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":136.38800000000001,"5min":136.38800000000001,"15min":136.38800000000001},"last":1.2}]},{"osd":8,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":3.0310000000000001,"5min":4.2789999999999999,"15min":4.4870000000000001},"min":{"1min":1.0509999999999999,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":30.713000000000001,"5min":64.912999999999997,"15min":64.912999999999997},"last":1.458},{"interface":"front","average":{"1min":2.0569999999999999,"5min":5.3959999999999999,"15min":5.952},"min":{"1min":0.82499999999999996,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":12.898999999999999,"5min":57.664000000000001,"15min":57.664000000000001},"last":1.857}]},{"osd":10,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":1.3400000000000001,"5min":1.3089999999999999,"15min":1.304},"min":{"1min":0.88300000000000001,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.8180000000000001,"5min":2.109,"15min":2.109},"last":1.51},{"interface":"front","average":{"1min":6.0389999999999997,"5min":4.2309999999999999,"15min":3.9300000000000002},"min":{"1min":0.97899999999999998,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":68.216999999999999,"5min":68.216999999999999,"15min":68.216999999999999},"last":1.661}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.669}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.446}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":96}]},{"osd":14,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":4.3579999999999997,"5min":8.8759999999999994,"15min":9.6289999999999996},"min":{"1min":0.86499999999999999,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":50.773000000000003,"5min":62.899000000000001,"15min":62.899000000000001},"last":1.9039999999999999},{"interface":"front","average":{"1min":4.3540000000000001,"5min":13.071,"15min":14.523999999999999},"min":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":50.709000000000003,"5min":155.61000000000001,"15min":155.61000000000001},"last":1.6950000000000001}]},{"osd":15,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":1.4630000000000001,"5min":1.492,"15min":1.4970000000000001},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.97,"5min":1.9970000000000001,"15min":1.9970000000000001},"last":1.605},{"interface":"front","average":{"1min":9.5700000000000003,"5min":5.1749999999999998,"15min":4.4429999999999996},"min":{"1min":1.0980000000000001,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":68.140000000000001,"5min":68.140000000000001,"15min":68.140000000000001},"last":1.7470000000000001}]}]},{"osd":11,"up_from":21,"seq":90194313248,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303784,"kb_used_data":2264,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068056,"statfs":{"total":96636764160,"available":96325689344,"internally_reserved":0,"allocated":2318336,"data_stored":1350579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[2,3,6,8,9,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0049999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86599999999999999}]},{"osd":2,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":8.3390000000000004,"15min":9.5719999999999992},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.3740000000000001,"5min":99.718000000000004,"15min":99.718000000000004},"last":0.88200000000000001},{"interface":"front","average":{"1min":3.6349999999999998,"5min":1.4450000000000001,"15min":1.0800000000000001},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":49.774000000000001,"5min":49.774000000000001,"15min":49.774000000000001},"last":0.93400000000000005}]},{"osd":3,"last update":"Wed Sep 18 10:16:46 2024","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":4.2210000000000001,"15min":4.7880000000000003},"min":{"1min":0.48399999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.369,"5min":85.963999999999999,"15min":85.963999999999999},"last":0.82299999999999995},{"interface":"front","average":{"1min":5.4660000000000002,"5min":19.300000000000001,"15min":21.606000000000002},"min":{"1min":0.73699999999999999,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":80.980999999999995,"5min":125.672,"15min":125.672},"last":1.073}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.444}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5980000000000001}]},{"osd":6,"last update":"Wed Sep 18 10:16:46 2024","interfaces":[{"interface":"back","average":{"1min":5.8810000000000002,"5min":2.101,"15min":1.4710000000000001},"min":{"1min":0.70599999999999996,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":86.680000000000007,"5min":86.680000000000007,"15min":86.680000000000007},"last":1.21},{"interface":"front","average":{"1min":1.1830000000000001,"5min":5.827,"15min":6.601},"min":{"1min":0.73799999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.49,"5min":94.162000000000006,"15min":94.162000000000006},"last":1.109}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.524}]},{"osd":8,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":2.7069999999999999,"5min":5.3730000000000002,"15min":5.8170000000000002},"min":{"1min":0.95299999999999996,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":27.577999999999999,"5min":85.900000000000006,"15min":85.900000000000006},"last":1.0189999999999999},{"interface":"front","average":{"1min":2.544,"5min":8.4960000000000004,"15min":9.4879999999999995},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":27.542000000000002,"5min":99.765000000000001,"15min":99.765000000000001},"last":1.1559999999999999}]},{"osd":9,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":5.8760000000000003,"5min":5.8339999999999996,"15min":5.827},"min":{"1min":0.96399999999999997,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":89.844999999999999,"5min":89.844999999999999,"15min":89.844999999999999},"last":1.1779999999999999},{"interface":"front","average":{"1min":1.351,"5min":6.8070000000000004,"15min":7.7160000000000002},"min":{"1min":0.68600000000000005,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.742,"5min":99.810000000000002,"15min":99.810000000000002},"last":1.2470000000000001}]},{"osd":10,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":4.0069999999999997,"5min":1.679,"15min":1.2909999999999999},"min":{"1min":0.98499999999999999,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":49.819000000000003,"5min":49.819000000000003,"15min":49.819000000000003},"last":1.361},{"interface":"front","average":{"1min":1.5580000000000001,"5min":6.46,"15min":7.2770000000000001},"min":{"1min":0.98699999999999999,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.976,"5min":89.290000000000006,"15min":89.290000000000006},"last":1.274}]},{"osd":12,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":5.907,"5min":2.081,"15min":1.4430000000000001},"min":{"1min":0.96599999999999997,"5min":0.498,"15min":0.498},"max":{"1min":87.463999999999999,"5min":87.463999999999999,"15min":87.463999999999999},"last":1.7270000000000001},{"interface":"front","average":{"1min":1.5660000000000001,"5min":8.6219999999999999,"15min":9.798},"min":{"1min":1.0700000000000001,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":2.0600000000000001,"5min":86.025999999999996,"15min":86.025999999999996},"last":1.494}]},{"osd":13,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":4.4210000000000003,"5min":25.207999999999998,"15min":28.672000000000001},"min":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"max":{"1min":59.841999999999999,"5min":189.25899999999999,"15min":189.25899999999999},"last":1.476},{"interface":"front","average":{"1min":7.569,"5min":2.5659999999999998,"15min":1.732},"min":{"1min":0.88900000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":71.313000000000002,"5min":71.313000000000002,"15min":71.313000000000002},"last":1.6439999999999999}]},{"osd":14,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":14.388999999999999,"5min":8.016,"15min":6.9539999999999997},"min":{"1min":1.0129999999999999,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":256.13400000000001,"5min":256.13400000000001,"15min":256.13400000000001},"last":1.45},{"interface":"front","average":{"1min":2.7650000000000001,"5min":7.7380000000000004,"15min":8.5670000000000002},"min":{"1min":0.874,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":25.791,"5min":102.625,"15min":102.625},"last":1.4330000000000001}]},{"osd":15,"last update":"Wed Sep 18 10:16:40 2024","interfaces":[{"interface":"back","average":{"1min":4.0309999999999997,"5min":4.0309999999999997,"15min":4.0309999999999997},"min":{"1min":1.153,"5min":1.153,"15min":1.153},"max":{"1min":49.927999999999997,"5min":49.927999999999997,"15min":49.927999999999997},"last":1.518},{"interface":"front","average":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"min":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"max":{"1min":2.044,"5min":2.044,"15min":2.044},"last":1.407}]}]},{"osd":12,"up_from":21,"seq":90194313249,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303784,"kb_used_data":2268,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068056,"statfs":{"total":96636764160,"available":96325689344,"internally_reserved":0,"allocated":2322432,"data_stored":1350598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[3,4,6,7,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":2.3839999999999999,"5min":2.3839999999999999,"15min":2.3839999999999999},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":32.476999999999997,"5min":32.476999999999997,"15min":32.476999999999997},"last":0.64200000000000002},{"interface":"front","average":{"1min":2.625,"5min":2.625,"15min":2.625},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":32.509,"5min":32.509,"15min":32.509},"last":0.89300000000000002}]},{"osd":1,"last update":"Wed Sep 18 10:15:32 2024","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.6399999999999999,"5min":1.6399999999999999,"15min":1.6399999999999999},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.905,"5min":1.905,"15min":1.905},"last":0.81899999999999995}]},{"osd":2,"last update":"Wed Sep 18 10:15:32 2024","interfaces":[{"interface":"back","average":{"1min":1.002,"5min":1.002,"15min":1.002},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":1.153},{"interface":"front","average":{"1min":4.1189999999999998,"5min":4.1189999999999998,"15min":4.1189999999999998},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":62.780999999999999,"5min":62.780999999999999,"15min":62.780999999999999},"last":0.96499999999999997}]},{"osd":3,"last update":"Wed Sep 18 10:16:33 2024","interfaces":[{"interface":"back","average":{"1min":5.9429999999999996,"5min":2.1040000000000001,"15min":1.464},"min":{"1min":0.627,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":74.822999999999993,"5min":74.822999999999993,"15min":74.822999999999993},"last":1.1399999999999999},{"interface":"front","average":{"1min":8.8200000000000003,"5min":2.6840000000000002,"15min":1.6619999999999999},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":94.320999999999998,"5min":94.320999999999998,"15min":94.320999999999998},"last":0.87}]},{"osd":4,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":16.058,"5min":4.0529999999999999,"15min":2.052},"min":{"1min":0.53100000000000003,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":114.184,"5min":114.184,"15min":114.184},"last":0.82199999999999995},{"interface":"front","average":{"1min":5.8920000000000003,"5min":2.1560000000000001,"15min":1.5329999999999999},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":74.698999999999998,"5min":74.698999999999998,"15min":74.698999999999998},"last":0.94299999999999995}]},{"osd":6,"last update":"Wed Sep 18 10:16:38 2024","interfaces":[{"interface":"back","average":{"1min":10.747999999999999,"5min":10.747999999999999,"15min":10.747999999999999},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":130.20699999999999,"5min":130.20699999999999,"15min":130.20699999999999},"last":1.0660000000000001},{"interface":"front","average":{"1min":14.148,"5min":14.148,"15min":14.148},"min":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":132.60900000000001,"5min":132.60900000000001,"15min":132.60900000000001},"last":1.5329999999999999}]},{"osd":7,"last update":"Wed Sep 18 10:16:38 2024","interfaces":[{"interface":"back","average":{"1min":4.774,"5min":4.774,"15min":4.774},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":90.188000000000002,"5min":90.188000000000002,"15min":90.188000000000002},"last":1.208},{"interface":"front","average":{"1min":6.0359999999999996,"5min":6.0359999999999996,"15min":6.0359999999999996},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":75.061999999999998,"5min":75.061999999999998,"15min":75.061999999999998},"last":1.3089999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":94.382000000000005}]},{"osd":9,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":8.9719999999999995,"5min":2.8079999999999998,"15min":1.7809999999999999},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":94.421000000000006,"5min":94.421000000000006,"15min":94.421000000000006},"last":1.3799999999999999},{"interface":"front","average":{"1min":5.0410000000000004,"5min":2.0179999999999998,"15min":1.514},"min":{"1min":0.70199999999999996,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":90.242999999999995,"5min":90.242999999999995,"15min":90.242999999999995},"last":1.47}]},{"osd":10,"last update":"Wed Sep 18 10:16:38 2024","interfaces":[{"interface":"back","average":{"1min":15.145,"5min":15.145,"15min":15.145},"min":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":99.251000000000005,"5min":99.251000000000005,"15min":99.251000000000005},"last":1.4359999999999999},{"interface":"front","average":{"1min":8.7080000000000002,"5min":8.7080000000000002,"15min":8.7080000000000002},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":94.468000000000004,"5min":94.468000000000004,"15min":94.468000000000004},"last":1.5700000000000001}]},{"osd":11,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":8.9269999999999996,"5min":2.8769999999999998,"15min":1.869},"min":{"1min":0.91300000000000003,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":74.888999999999996,"5min":74.888999999999996,"15min":74.888999999999996},"last":1.8899999999999999},{"interface":"front","average":{"1min":9.1590000000000007,"5min":2.9380000000000002,"15min":1.901},"min":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"max":{"1min":94.546000000000006,"5min":94.546000000000006,"15min":94.546000000000006},"last":1.798}]},{"osd":13,"last update":"Wed Sep 18 10:16:33 2024","interfaces":[{"interface":"back","average":{"1min":7.4779999999999998,"5min":2.5619999999999998,"15min":1.742},"min":{"1min":0.79900000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":143.672,"5min":143.672,"15min":143.672},"last":1.671},{"interface":"front","average":{"1min":12.677,"5min":3.6779999999999999,"15min":2.1779999999999999},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":143.619,"5min":143.619,"15min":143.619},"last":1.744}]},{"osd":14,"last update":"Wed Sep 18 10:16:33 2024","interfaces":[{"interface":"back","average":{"1min":13.363,"5min":3.8159999999999998,"15min":2.2250000000000001},"min":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"max":{"1min":94.504999999999995,"5min":94.504999999999995,"15min":94.504999999999995},"last":1.6879999999999999},{"interface":"front","average":{"1min":12.039,"5min":3.52,"15min":2.1000000000000001},"min":{"1min":0.95099999999999996,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":92.593000000000004,"5min":92.593000000000004,"15min":92.593000000000004},"last":1.615}]},{"osd":15,"last update":"Wed Sep 18 10:16:33 2024","interfaces":[{"interface":"back","average":{"1min":6.5339999999999998,"5min":2.5019999999999998,"15min":1.8300000000000001},"min":{"1min":0.751,"5min":0.751,"15min":0.751},"max":{"1min":74.966999999999999,"5min":74.966999999999999,"15min":74.966999999999999},"last":2.028},{"interface":"front","average":{"1min":14.669,"5min":4.117,"15min":2.359},"min":{"1min":0.996,"5min":0.95299999999999996,"15min":0.95299999999999996},"max":{"1min":152.11000000000001,"5min":152.11000000000001,"15min":152.11000000000001},"last":1.9219999999999999}]}]},{"osd":1,"up_from":22,"seq":94489280543,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":303748,"kb_used_data":2232,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068092,"statfs":{"total":96636764160,"available":96325726208,"internally_reserved":0,"allocated":2285568,"data_stored":1331589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,6,9,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:16:38 2024","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":2.472,"15min":2.7370000000000001},"min":{"1min":0.65800000000000003,"5min":0.629,"15min":0.629},"max":{"1min":1.1930000000000001,"5min":36.268000000000001,"15min":36.268000000000001},"last":1.127},{"interface":"front","average":{"1min":5.3659999999999997,"5min":15.244999999999999,"15min":16.890999999999998},"min":{"1min":0.73099999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":66.534999999999997,"5min":154.67599999999999,"15min":154.67599999999999},"last":0.70699999999999996}]},{"osd":2,"last update":"Wed Sep 18 10:16:38 2024","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.91400000000000003,"15min":0.90400000000000003},"min":{"1min":0.71499999999999997,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.4119999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.98999999999999999,"5min":2.5539999999999998,"15min":2.8140000000000001},"min":{"1min":0.75600000000000001,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.212,"5min":36.323999999999998,"15min":36.323999999999998},"last":0.76400000000000001}]},{"osd":3,"last update":"Wed Sep 18 10:16:38 2024","interfaces":[{"interface":"back","average":{"1min":5.548,"5min":10.811999999999999,"15min":11.689},"min":{"1min":0.94599999999999995,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":66.344999999999999,"5min":150.642,"15min":150.642},"last":0.96799999999999997},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.0649999999999999,"15min":1.0509999999999999},"min":{"1min":0.82999999999999996,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.6830000000000001,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":1.081}]},{"osd":4,"last update":"Wed Sep 18 10:15:35 2024","interfaces":[{"interface":"back","average":{"1min":3.3439999999999999,"5min":3.3439999999999999,"15min":3.3439999999999999},"min":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":40.783000000000001,"5min":40.783000000000001,"15min":40.783000000000001},"last":1.7210000000000001},{"interface":"front","average":{"1min":4.1699999999999999,"5min":4.1699999999999999,"15min":4.1699999999999999},"min":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":50.747,"5min":50.747,"15min":50.747},"last":1.518}]},{"osd":5,"last update":"Wed Sep 18 10:15:35 2024","interfaces":[{"interface":"back","average":{"1min":5.9720000000000004,"5min":5.9720000000000004,"15min":5.9720000000000004},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":52.317999999999998,"5min":52.317999999999998,"15min":52.317999999999998},"last":1.2649999999999999},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.748,"5min":1.748,"15min":1.748},"last":1.839}]},{"osd":6,"last update":"Wed Sep 18 10:16:38 2024","interfaces":[{"interface":"back","average":{"1min":9.2970000000000006,"5min":8.5380000000000003,"15min":8.4120000000000008},"min":{"1min":0.85099999999999998,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":120.85299999999999,"5min":120.85299999999999,"15min":120.85299999999999},"last":1.1779999999999999},{"interface":"front","average":{"1min":2.6040000000000001,"5min":8.8019999999999996,"15min":9.8350000000000009},"min":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"max":{"1min":20.600000000000001,"5min":85.045000000000002,"15min":85.045000000000002},"last":1.266}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":118.43600000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7789999999999999}]},{"osd":9,"last update":"Wed Sep 18 10:16:15 2024","interfaces":[{"interface":"back","average":{"1min":3.4369999999999998,"5min":3.4369999999999998,"15min":3.4369999999999998},"min":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"max":{"1min":36.506999999999998,"5min":36.506999999999998,"15min":36.506999999999998},"last":0.998},{"interface":"front","average":{"1min":11.339,"5min":11.339,"15min":11.339},"min":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"max":{"1min":165.13499999999999,"5min":165.13499999999999,"15min":165.13499999999999},"last":1.0429999999999999}]},{"osd":10,"last update":"Wed Sep 18 10:16:38 2024","interfaces":[{"interface":"back","average":{"1min":1.5209999999999999,"5min":3.048,"15min":3.302},"min":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"max":{"1min":2.0489999999999999,"5min":36.436,"15min":36.436},"last":1.3799999999999999},{"interface":"front","average":{"1min":6.0839999999999996,"5min":10.747999999999999,"15min":11.526},"min":{"1min":1.2769999999999999,"5min":0.88,"15min":0.88},"max":{"1min":66.673000000000002,"5min":100.402,"15min":100.402},"last":1.155}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.825}]},{"osd":12,"last update":"Wed Sep 18 10:16:38 2024","interfaces":[{"interface":"back","average":{"1min":1.5549999999999999,"5min":3.681,"15min":4.0350000000000001},"min":{"1min":1.101,"5min":1.0589999999999999,"15min":1.0589999999999999},"max":{"1min":2.294,"5min":49.976999999999997,"15min":49.976999999999997},"last":1.323},{"interface":"front","average":{"1min":2.0369999999999999,"5min":7.1269999999999998,"15min":7.9749999999999996},"min":{"1min":1.196,"5min":1.196,"15min":1.196},"max":{"1min":8.4049999999999994,"5min":124.78100000000001,"15min":124.78100000000001},"last":1.4419999999999999}]},{"osd":13,"last update":"Wed Sep 18 10:16:15 2024","interfaces":[{"interface":"back","average":{"1min":11.616,"5min":11.616,"15min":11.616},"min":{"1min":1.232,"5min":1.232,"15min":1.232},"max":{"1min":104.94199999999999,"5min":104.94199999999999,"15min":104.94199999999999},"last":1.2909999999999999},{"interface":"front","average":{"1min":1.5720000000000001,"5min":1.5720000000000001,"15min":1.5720000000000001},"min":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"max":{"1min":2.3809999999999998,"5min":2.3809999999999998,"15min":2.3809999999999998},"last":1.4099999999999999}]},{"osd":14,"last update":"Wed Sep 18 10:16:15 2024","interfaces":[{"interface":"back","average":{"1min":5.5910000000000002,"5min":5.5910000000000002,"15min":5.5910000000000002},"min":{"1min":1.276,"5min":1.276,"15min":1.276},"max":{"1min":66.825999999999993,"5min":66.825999999999993,"15min":66.825999999999993},"last":1.3580000000000001},{"interface":"front","average":{"1min":2.8359999999999999,"5min":2.8359999999999999,"15min":2.8359999999999999},"min":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"max":{"1min":22.16,"5min":22.16,"15min":22.16},"last":1.508}]},{"osd":15,"last update":"Wed Sep 18 10:16:15 2024","interfaces":[{"interface":"back","average":{"1min":3.677,"5min":3.677,"15min":3.677},"min":{"1min":1.165,"5min":1.165,"15min":1.165},"max":{"1min":36.575000000000003,"5min":36.575000000000003,"15min":36.575000000000003},"last":1.54},{"interface":"front","average":{"1min":5.6269999999999998,"5min":5.6269999999999998,"15min":5.6269999999999998},"min":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"max":{"1min":66.763999999999996,"5min":66.763999999999996,"15min":66.763999999999996},"last":1.468}]}]},{"osd":2,"up_from":22,"seq":94489280543,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":303764,"kb_used_data":2248,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068076,"statfs":{"total":96636764160,"available":96325709824,"internally_reserved":0,"allocated":2301952,"data_stored":1341063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,3,4,5,6,7,8,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":21.344000000000001,"15min":24.777999999999999},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.3879999999999999,"5min":419.05700000000002,"15min":419.05700000000002},"last":0.84899999999999998},{"interface":"front","average":{"1min":2.3860000000000001,"5min":4.4020000000000001,"15min":4.7380000000000004},"min":{"1min":0.57299999999999995,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":35.899999999999999,"5min":98.254000000000005,"15min":98.254000000000005},"last":0.71699999999999997}]},{"osd":3,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":5.1219999999999999,"5min":1.7529999999999999,"15min":1.1910000000000001},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":86.677000000000007,"5min":86.677000000000007,"15min":86.677000000000007},"last":0.66900000000000004},{"interface":"front","average":{"1min":3.6160000000000001,"5min":1.47,"15min":1.1120000000000001},"min":{"1min":0.68000000000000005,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":59.759,"5min":59.759,"15min":59.759},"last":0.76300000000000001}]},{"osd":4,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":7.992,"5min":3.1379999999999999,"15min":2.3290000000000002},"min":{"1min":0.55600000000000005,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":121.20099999999999,"5min":121.20099999999999,"15min":121.20099999999999},"last":0.94499999999999995},{"interface":"front","average":{"1min":6.5250000000000004,"5min":4.9359999999999999,"15min":4.6710000000000003},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":86.548000000000002,"5min":86.548000000000002,"15min":86.548000000000002},"last":0.88300000000000001}]},{"osd":5,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":12.161,"5min":3.2879999999999998,"15min":1.8089999999999999},"min":{"1min":0.72099999999999997,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":163.828,"5min":163.828,"15min":163.828},"last":0.92600000000000005},{"interface":"front","average":{"1min":2.6160000000000001,"5min":1.3660000000000001,"15min":1.1579999999999999},"min":{"1min":0.79200000000000004,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":36.119,"5min":36.119,"15min":36.119},"last":1.02}]},{"osd":6,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":5.2359999999999998,"5min":4.8529999999999998,"15min":4.7889999999999997},"min":{"1min":0.49199999999999999,"5min":0.441,"15min":0.441},"max":{"1min":86.766000000000005,"5min":86.766000000000005,"15min":86.766000000000005},"last":0.82399999999999995},{"interface":"front","average":{"1min":5.3959999999999999,"5min":4.9770000000000003,"15min":4.907},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":83.171999999999997,"5min":90.188999999999993,"15min":90.188999999999993},"last":1.038}]},{"osd":7,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":13.191000000000001,"5min":3.6019999999999999,"15min":2.004},"min":{"1min":0.84999999999999998,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":88.534000000000006,"5min":88.534000000000006,"15min":88.534000000000006},"last":1.079},{"interface":"front","average":{"1min":15.949999999999999,"5min":4.2350000000000003,"15min":2.2829999999999999},"min":{"1min":0.75600000000000001,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":88.484999999999999,"5min":88.484999999999999,"15min":88.484999999999999},"last":0.97899999999999998}]},{"osd":8,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":4.3179999999999996,"15min":4.8399999999999999},"min":{"1min":0.82899999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":2.0699999999999998,"5min":92.578999999999994,"15min":92.578999999999994},"last":1.1000000000000001},{"interface":"front","average":{"1min":2.8460000000000001,"5min":2.7799999999999998,"15min":2.7690000000000001},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":36.088000000000001,"5min":36.088000000000001,"15min":36.088000000000001},"last":1.1459999999999999}]},{"osd":12,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":2.9220000000000002,"5min":1.6719999999999999,"15min":1.464},"min":{"1min":0.872,"5min":0.872,"15min":0.872},"max":{"1min":36.014000000000003,"5min":36.014000000000003,"15min":36.014000000000003},"last":1.2290000000000001},{"interface":"front","average":{"1min":5.5140000000000002,"5min":2.7469999999999999,"15min":2.286},"min":{"1min":0.73199999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":86.832999999999998,"5min":86.832999999999998,"15min":86.832999999999998},"last":1.0589999999999999}]},{"osd":13,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":11.115,"5min":8.3970000000000002,"15min":7.944},"min":{"1min":0.97999999999999998,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":80.683000000000007,"5min":141.84200000000001,"15min":141.84200000000001},"last":1.3600000000000001},{"interface":"front","average":{"1min":2.911,"5min":4.9489999999999998,"15min":5.2889999999999997},"min":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":35.841000000000001,"5min":97.441999999999993,"15min":97.441999999999993},"last":1.2849999999999999}]},{"osd":15,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":12.472,"5min":3.6200000000000001,"15min":2.145},"min":{"1min":0.68400000000000005,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":86.980000000000004,"5min":86.980000000000004,"15min":86.980000000000004},"last":1.129},{"interface":"front","average":{"1min":3.3479999999999999,"5min":3.798,"15min":3.8730000000000002},"min":{"1min":0.96999999999999997,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":45.356999999999999,"5min":61.694000000000003,"15min":61.694000000000003},"last":1.304}]}]},{"osd":3,"up_from":22,"seq":94489280543,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303748,"kb_used_data":2232,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068092,"statfs":{"total":96636764160,"available":96325726208,"internally_reserved":0,"allocated":2285568,"data_stored":1331589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,4,6,7,9,10,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:16:37 2024","interfaces":[{"interface":"back","average":{"1min":4.173,"5min":2.048,"15min":1.694},"min":{"1min":0.64300000000000002,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":65.994,"5min":65.994,"15min":65.994},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.97899999999999998,"5min":8.8260000000000005,"15min":10.134},"min":{"1min":0.75,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.752,"5min":71.316999999999993,"15min":71.316999999999993},"last":0.76800000000000002}]},{"osd":2,"last update":"Wed Sep 18 10:16:37 2024","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":4.4379999999999997,"15min":5.0140000000000002},"min":{"1min":0.76900000000000002,"5min":0.75,"15min":0.75},"max":{"1min":1.333,"5min":56.523000000000003,"15min":56.523000000000003},"last":0.94999999999999996},{"interface":"front","average":{"1min":1.165,"5min":1.1200000000000001,"15min":1.1120000000000001},"min":{"1min":0.67400000000000004,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.639,"5min":3.327,"15min":3.327},"last":1.3300000000000001}]},{"osd":4,"last update":"Wed Sep 18 10:16:37 2024","interfaces":[{"interface":"back","average":{"1min":1.0409999999999999,"5min":5.7069999999999999,"15min":6.4850000000000003},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.6399999999999999,"5min":71.135000000000005,"15min":71.135000000000005},"last":0.91500000000000004},{"interface":"front","average":{"1min":1.635,"5min":3.8130000000000002,"15min":4.1760000000000002},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":10.039,"5min":56.253999999999998,"15min":56.253999999999998},"last":0.72799999999999998}]},{"osd":5,"last update":"Wed Sep 18 10:16:06 2024","interfaces":[{"interface":"back","average":{"1min":13.997,"5min":13.997,"15min":13.997},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":125.42400000000001,"5min":125.42400000000001,"15min":125.42400000000001},"last":1.131},{"interface":"front","average":{"1min":10.323,"5min":10.323,"15min":10.323},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":112.033,"5min":112.033,"15min":112.033},"last":0.96899999999999997}]},{"osd":6,"last update":"Wed Sep 18 10:16:37 2024","interfaces":[{"interface":"back","average":{"1min":1.9279999999999999,"5min":1.29,"15min":1.1839999999999999},"min":{"1min":0.93400000000000005,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":12.83,"5min":12.83,"15min":12.83},"last":1.0600000000000001},{"interface":"front","average":{"1min":5.0190000000000001,"5min":8.0050000000000008,"15min":8.5030000000000001},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":76.634,"5min":81.210999999999999,"15min":81.210999999999999},"last":0.998}]},{"osd":7,"last update":"Wed Sep 18 10:16:37 2024","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":13.159000000000001,"15min":15.141},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.75,"5min":129.68299999999999,"15min":129.68299999999999},"last":1.0249999999999999},{"interface":"front","average":{"1min":1.4710000000000001,"5min":11.119999999999999,"15min":12.728},"min":{"1min":0.90800000000000003,"5min":0.621,"15min":0.621},"max":{"1min":1.8500000000000001,"5min":129.726,"15min":129.726},"last":1.474}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0620000000000001}]},{"osd":9,"last update":"Wed Sep 18 10:16:37 2024","interfaces":[{"interface":"back","average":{"1min":1.5660000000000001,"5min":1.446,"15min":1.4259999999999999},"min":{"1min":0.94599999999999995,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":2.0209999999999999,"5min":2.0209999999999999,"15min":2.0209999999999999},"last":1.377},{"interface":"front","average":{"1min":6.1050000000000004,"5min":6.1079999999999997,"15min":6.1079999999999997},"min":{"1min":0.86099999999999999,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":94.546999999999997,"5min":94.546999999999997,"15min":94.546999999999997},"last":1.2969999999999999}]},{"osd":10,"last update":"Wed Sep 18 10:16:06 2024","interfaces":[{"interface":"back","average":{"1min":1.4870000000000001,"5min":1.4870000000000001,"15min":1.4870000000000001},"min":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":2.0659999999999998,"5min":2.0659999999999998,"15min":2.0659999999999998},"last":1.232},{"interface":"front","average":{"1min":4.9349999999999996,"5min":4.9349999999999996,"15min":4.9349999999999996},"min":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":71.668999999999997,"5min":71.668999999999997,"15min":71.668999999999997},"last":1.173}]},{"osd":13,"last update":"Wed Sep 18 10:16:37 2024","interfaces":[{"interface":"back","average":{"1min":9.4909999999999997,"5min":3.113,"15min":2.0499999999999998},"min":{"1min":1.04,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":91.290999999999997,"5min":91.290999999999997,"15min":91.290999999999997},"last":1.421},{"interface":"front","average":{"1min":1.5249999999999999,"5min":5.3300000000000001,"15min":5.9640000000000004},"min":{"1min":1.05,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.885,"5min":71.447000000000003,"15min":71.447000000000003},"last":1.3560000000000001}]},{"osd":14,"last update":"Wed Sep 18 10:16:37 2024","interfaces":[{"interface":"back","average":{"1min":1.6899999999999999,"5min":4.8899999999999997,"15min":5.423},"min":{"1min":1.0089999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"max":{"1min":4.625,"5min":56.664999999999999,"15min":56.664999999999999},"last":1.149},{"interface":"front","average":{"1min":1.865,"5min":2.2629999999999999,"15min":2.3290000000000002},"min":{"1min":1.1859999999999999,"5min":0.93999999999999995,"15min":0.93999999999999995},"max":{"1min":4.5910000000000002,"5min":18.523,"15min":18.523},"last":1.52}]},{"osd":15,"last update":"Wed Sep 18 10:16:48 2024","interfaces":[{"interface":"back","average":{"1min":1.6970000000000001,"5min":4.0049999999999999,"15min":4.3899999999999997},"min":{"1min":1.01,"5min":0.91500000000000004,"15min":0.91500000000000004},"max":{"1min":4.3289999999999997,"5min":56.357999999999997,"15min":56.357999999999997},"last":1.268},{"interface":"front","average":{"1min":1.7589999999999999,"5min":8.5310000000000006,"15min":9.6600000000000001},"min":{"1min":1.133,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":2.1110000000000002,"5min":122.727,"15min":122.727},"last":1.5620000000000001}]}]},{"osd":5,"up_from":22,"seq":94489280543,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303748,"kb_used_data":2232,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068092,"statfs":{"total":96636764160,"available":96325726208,"internally_reserved":0,"allocated":2285568,"data_stored":1331589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[2,3,4,6,8,9,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:15:47 2024","interfaces":[{"interface":"back","average":{"1min":7.8860000000000001,"5min":7.8860000000000001,"15min":7.8860000000000001},"min":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":84.430999999999997,"5min":84.430999999999997,"15min":84.430999999999997},"last":1.2889999999999999},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":1.353}]},{"osd":2,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":4.1360000000000001,"5min":8.6940000000000008,"15min":9.4540000000000006},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":51.091000000000001,"5min":84.566000000000003,"15min":84.566000000000003},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.98199999999999998,"5min":1.3300000000000001,"15min":1.3879999999999999},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.4690000000000001,"5min":9.5069999999999997,"15min":9.5069999999999997},"last":0.86899999999999999}]},{"osd":3,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":4.9240000000000004,"5min":7.8010000000000002,"15min":8.2810000000000006},"min":{"1min":0.69099999999999995,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":51.229999999999997,"5min":84.358999999999995,"15min":84.358999999999995},"last":0.70499999999999996},{"interface":"front","average":{"1min":8.1590000000000007,"5min":8.2100000000000009,"15min":8.218},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":74.222999999999999,"5min":97.549999999999997,"15min":97.549999999999997},"last":0.76900000000000002}]},{"osd":4,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":25.673999999999999,"15min":29.777999999999999},"min":{"1min":0.67400000000000004,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.407,"5min":266.911,"15min":266.911},"last":1.4059999999999999},{"interface":"front","average":{"1min":21.388000000000002,"5min":22.515999999999998,"15min":22.704000000000001},"min":{"1min":0.76200000000000001,"5min":0.626,"15min":0.626},"max":{"1min":171.691,"5min":260.53500000000003,"15min":260.53500000000003},"last":1.0960000000000001}]},{"osd":6,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":3.6309999999999998,"5min":8.0809999999999995,"15min":8.8230000000000004},"min":{"1min":0.70099999999999996,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":41.886000000000003,"5min":58.171999999999997,"15min":58.171999999999997},"last":0.97599999999999998},{"interface":"front","average":{"1min":5.8079999999999998,"5min":18.274999999999999,"15min":20.353000000000002},"min":{"1min":0.85699999999999998,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":54.984999999999999,"5min":227.48099999999999,"15min":227.48099999999999},"last":1.1759999999999999}]},{"osd":7,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":12.013,"5min":12.013,"15min":12.013},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":123.111,"5min":123.111,"15min":123.111},"last":1.5249999999999999},{"interface":"front","average":{"1min":13.994,"5min":13.994,"15min":13.994},"min":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"max":{"1min":84.617000000000004,"5min":84.617000000000004,"15min":84.617000000000004},"last":1.329}]},{"osd":8,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":4.4720000000000004,"5min":17.175999999999998,"15min":19.292999999999999},"min":{"1min":0.879,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":51.506,"5min":260.44200000000001,"15min":260.44200000000001},"last":1.232},{"interface":"front","average":{"1min":8.2609999999999992,"5min":7.2859999999999996,"15min":7.1239999999999997},"min":{"1min":0.83899999999999997,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":71.266000000000005,"5min":106.818,"15min":106.818},"last":1.038}]},{"osd":9,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":3.8780000000000001,"5min":5.2930000000000001,"15min":5.5289999999999999},"min":{"1min":0.91300000000000003,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":41.652000000000001,"5min":80.272000000000006,"15min":80.272000000000006},"last":1.133},{"interface":"front","average":{"1min":1.3799999999999999,"5min":1.764,"15min":1.8280000000000001},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":2.008,"5min":9.5690000000000008,"15min":9.5690000000000008},"last":1.502}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6479999999999999}]},{"osd":11,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":1.494,"5min":7.8120000000000003,"15min":8.8650000000000002},"min":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":1.825,"5min":123.158,"15min":123.158},"last":1.794},{"interface":"front","average":{"1min":8.8490000000000002,"5min":10.863,"15min":11.199},"min":{"1min":0.93200000000000005,"5min":0.86499999999999999,"15min":0.86499999999999999},"max":{"1min":68.938000000000002,"5min":84.522999999999996,"15min":84.522999999999996},"last":1.446}]},{"osd":12,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":4.0250000000000004,"5min":1.8580000000000001,"15min":1.4970000000000001},"min":{"1min":0.95199999999999996,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":41.929000000000002,"5min":41.929000000000002,"15min":41.929000000000002},"last":1.2849999999999999},{"interface":"front","average":{"1min":1.774,"5min":1.9339999999999999,"15min":1.9610000000000001},"min":{"1min":0.93999999999999995,"5min":0.88300000000000001,"15min":0.88300000000000001},"max":{"1min":4.1150000000000002,"5min":9.7710000000000008,"15min":9.7710000000000008},"last":1.5780000000000001}]},{"osd":13,"last update":"Wed Sep 18 10:16:48 2024","interfaces":[{"interface":"back","average":{"1min":1.6499999999999999,"5min":19.745999999999999,"15min":22.762},"min":{"1min":1.171,"5min":1.1120000000000001,"15min":1.1120000000000001},"max":{"1min":2.0550000000000002,"5min":140.273,"15min":140.273},"last":1.6479999999999999},{"interface":"front","average":{"1min":4.665,"5min":20.309000000000001,"15min":22.917000000000002},"min":{"1min":1.194,"5min":0.98099999999999998,"15min":0.98099999999999998},"max":{"1min":51.432000000000002,"5min":170.86000000000001,"15min":170.86000000000001},"last":1.329}]},{"osd":14,"last update":"Wed Sep 18 10:16:48 2024","interfaces":[{"interface":"back","average":{"1min":9.8930000000000007,"5min":12.584,"15min":13.032},"min":{"1min":1.385,"5min":1.0389999999999999,"15min":1.0389999999999999},"max":{"1min":93.295000000000002,"5min":101.733,"15min":101.733},"last":1.385},{"interface":"front","average":{"1min":1.79,"5min":7.9729999999999999,"15min":9.0030000000000001},"min":{"1min":1.321,"5min":0.90100000000000002,"15min":0.90100000000000002},"max":{"1min":2.2290000000000001,"5min":122.952,"15min":122.952},"last":1.7250000000000001}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8839999999999999}]}]},{"osd":6,"up_from":22,"seq":94489280543,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304280,"kb_used_data":2752,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94067560,"statfs":{"total":96636764160,"available":96325181440,"internally_reserved":0,"allocated":2818048,"data_stored":1860517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,3,4,5,7,8,9,11,12,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:15:47 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.68200000000000005}]},{"osd":1,"last update":"Wed Sep 18 10:16:48 2024","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":3.8100000000000001,"15min":4.306},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.2150000000000001,"5min":52.332999999999998,"15min":52.332999999999998},"last":0.65700000000000003},{"interface":"front","average":{"1min":4.8369999999999997,"5min":1.6990000000000001,"15min":1.1759999999999999},"min":{"1min":0.68300000000000005,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":67.763000000000005,"5min":67.763000000000005,"15min":67.763000000000005},"last":0.71199999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.343}]},{"osd":3,"last update":"Wed Sep 18 10:16:48 2024","interfaces":[{"interface":"back","average":{"1min":5.3380000000000001,"5min":1.792,"15min":1.2010000000000001},"min":{"1min":0.65500000000000003,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":76.891999999999996,"5min":76.891999999999996,"15min":76.891999999999996},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.98999999999999999,"5min":11.403,"15min":13.138999999999999},"min":{"1min":0.751,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.204,"5min":169.244,"15min":169.244},"last":1.042}]},{"osd":4,"last update":"Wed Sep 18 10:16:02 2024","interfaces":[{"interface":"back","average":{"1min":11.111000000000001,"5min":11.111000000000001,"15min":11.111000000000001},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":128.053,"5min":128.053,"15min":128.053},"last":0.92100000000000004},{"interface":"front","average":{"1min":4.6189999999999998,"5min":4.6189999999999998,"15min":4.6189999999999998},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":67.960999999999999,"5min":67.960999999999999,"15min":67.960999999999999},"last":0.86699999999999999}]},{"osd":5,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":6.0579999999999998,"5min":10.194000000000001,"15min":10.884},"min":{"1min":0.72099999999999997,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":68.025999999999996,"5min":119.032,"15min":119.032},"last":0.96799999999999997},{"interface":"front","average":{"1min":5.6760000000000002,"5min":4.0380000000000003,"15min":3.7650000000000001},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":83.858000000000004,"5min":83.858000000000004,"15min":83.858000000000004},"last":1.109}]},{"osd":7,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":4.9539999999999997,"5min":6.3959999999999999,"15min":6.6360000000000001},"min":{"1min":0.94099999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":69.171000000000006,"5min":69.171000000000006,"15min":69.171000000000006},"last":1.3120000000000001},{"interface":"front","average":{"1min":5.0960000000000001,"5min":3.4630000000000001,"15min":3.1909999999999998},"min":{"1min":1.089,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":69.239000000000004,"5min":69.239000000000004,"15min":69.239000000000004},"last":1.29}]},{"osd":8,"last update":"Wed Sep 18 10:16:51 2024","interfaces":[{"interface":"back","average":{"1min":5.3659999999999997,"5min":5.3659999999999997,"15min":5.3659999999999997},"min":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"max":{"1min":76.766000000000005,"5min":76.766000000000005,"15min":76.766000000000005},"last":1.0109999999999999},{"interface":"front","average":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"min":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"max":{"1min":1.704,"5min":1.704,"15min":1.704},"last":1.5029999999999999}]},{"osd":9,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":5.4269999999999996,"5min":7.7089999999999996,"15min":8.0890000000000004},"min":{"1min":0.95799999999999996,"5min":0.84599999999999997,"15min":0.84599999999999997},"max":{"1min":76.997,"5min":95.691999999999993,"15min":95.691999999999993},"last":1.246},{"interface":"front","average":{"1min":1.2949999999999999,"5min":10.472,"15min":12.002000000000001},"min":{"1min":0.96899999999999997,"5min":0.83099999999999996,"15min":0.83099999999999996},"max":{"1min":1.696,"5min":165.31100000000001,"15min":165.31100000000001},"last":1.4219999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.329}]},{"osd":11,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":5.0510000000000002,"5min":2.0630000000000002,"15min":1.5649999999999999},"min":{"1min":1.002,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":67.909999999999997,"5min":67.909999999999997,"15min":67.909999999999997},"last":1.1599999999999999},{"interface":"front","average":{"1min":9.1829999999999998,"5min":4.9429999999999996,"15min":4.2359999999999998},"min":{"1min":1.1870000000000001,"5min":0.92000000000000004,"15min":0.92000000000000004},"max":{"1min":77.158000000000001,"5min":77.158000000000001,"15min":77.158000000000001},"last":1.351}]},{"osd":12,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":1.3859999999999999,"5min":4.3140000000000001,"15min":4.8019999999999996},"min":{"1min":1.079,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":1.796,"5min":52.378,"15min":52.378},"last":1.204},{"interface":"front","average":{"1min":5.1109999999999998,"5min":4.5720000000000001,"15min":4.4820000000000002},"min":{"1min":0.92800000000000005,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":67.835999999999999,"5min":67.835999999999999,"15min":67.835999999999999},"last":1.3779999999999999}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2050000000000001}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2669999999999999}]},{"osd":15,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":5.1680000000000001,"5min":2.2120000000000002,"15min":1.72},"min":{"1min":1.1000000000000001,"5min":0.90300000000000002,"15min":0.90300000000000002},"max":{"1min":68.069999999999993,"5min":68.069999999999993,"15min":68.069999999999993},"last":1.468},{"interface":"front","average":{"1min":5.6200000000000001,"5min":4.3019999999999996,"15min":4.0819999999999999},"min":{"1min":1.2030000000000001,"5min":1.0429999999999999,"15min":1.0429999999999999},"max":{"1min":77.081999999999994,"5min":77.081999999999994,"15min":77.081999999999994},"last":1.452}]}]},{"osd":7,"up_from":22,"seq":94489280543,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303748,"kb_used_data":2232,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068092,"statfs":{"total":96636764160,"available":96325726208,"internally_reserved":0,"allocated":2285568,"data_stored":1331589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,4,6,8,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:16:33 2024","interfaces":[{"interface":"back","average":{"1min":3.9900000000000002,"5min":10.974,"15min":12.138},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":59.137999999999998,"5min":139.58600000000001,"15min":139.58600000000001},"last":0.75600000000000001},{"interface":"front","average":{"1min":1.121,"5min":1.0229999999999999,"15min":1.0069999999999999},"min":{"1min":0.67800000000000005,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.7390000000000001,"5min":1.7390000000000001,"15min":1.7390000000000001},"last":0.59899999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2090000000000001}]},{"osd":2,"last update":"Wed Sep 18 10:15:46 2024","interfaces":[{"interface":"back","average":{"1min":15.436,"5min":15.436,"15min":15.436},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":139.36799999999999,"5min":139.36799999999999,"15min":139.36799999999999},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.94299999999999995}]},{"osd":3,"last update":"Wed Sep 18 10:15:46 2024","interfaces":[{"interface":"back","average":{"1min":17.327999999999999,"5min":17.327999999999999,"15min":17.327999999999999},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":139.27199999999999,"5min":139.27199999999999,"15min":139.27199999999999},"last":1.3460000000000001},{"interface":"front","average":{"1min":3.964,"5min":3.964,"15min":3.964},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":23.748999999999999,"5min":23.748999999999999,"15min":23.748999999999999},"last":0.82899999999999996}]},{"osd":4,"last update":"Wed Sep 18 10:16:37 2024","interfaces":[{"interface":"back","average":{"1min":9.8670000000000009,"5min":17.291,"15min":18.527999999999999},"min":{"1min":0.68000000000000005,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":125.10299999999999,"5min":139.51599999999999,"15min":139.51599999999999},"last":0.55200000000000005},{"interface":"front","average":{"1min":3.3559999999999999,"5min":4.6699999999999999,"15min":4.8890000000000002},"min":{"1min":0.68000000000000005,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":49.241999999999997,"5min":49.241999999999997,"15min":49.241999999999997},"last":0.70199999999999996}]},{"osd":5,"last update":"Wed Sep 18 10:15:46 2024","interfaces":[{"interface":"back","average":{"1min":1.099,"5min":1.099,"15min":1.099},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":2.3620000000000001,"5min":2.3620000000000001,"15min":2.3620000000000001},"last":1.4059999999999999},{"interface":"front","average":{"1min":17.736999999999998,"5min":17.736999999999998,"15min":17.736999999999998},"min":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"max":{"1min":139.44300000000001,"5min":139.44300000000001,"15min":139.44300000000001},"last":0.78000000000000003}]},{"osd":6,"last update":"Wed Sep 18 10:16:33 2024","interfaces":[{"interface":"back","average":{"1min":6.0869999999999997,"5min":15.117000000000001,"15min":16.622},"min":{"1min":0.998,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":59.343000000000004,"5min":183.001,"15min":183.001},"last":1.024},{"interface":"front","average":{"1min":7.0830000000000002,"5min":4.8650000000000002,"15min":4.4950000000000001},"min":{"1min":0.82399999999999995,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":58.284999999999997,"5min":58.284999999999997,"15min":58.284999999999997},"last":0.98499999999999999}]},{"osd":8,"last update":"Wed Sep 18 10:16:30 2024","interfaces":[{"interface":"back","average":{"1min":1.4079999999999999,"5min":1.2809999999999999,"15min":1.26},"min":{"1min":0.84999999999999998,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.9339999999999999,"5min":2.6139999999999999,"15min":2.6139999999999999},"last":0.85699999999999998},{"interface":"front","average":{"1min":5.3029999999999999,"5min":13.785,"15min":15.199},"min":{"1min":1.0980000000000001,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":59.523000000000003,"5min":139.798,"15min":139.798},"last":0.89700000000000002}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.861}]},{"osd":10,"last update":"Wed Sep 18 10:16:30 2024","interfaces":[{"interface":"back","average":{"1min":16.035,"5min":8.4100000000000001,"15min":7.1390000000000002},"min":{"1min":1.0149999999999999,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":156.45500000000001,"5min":156.45500000000001,"15min":156.45500000000001},"last":1.2070000000000001},{"interface":"front","average":{"1min":1.4910000000000001,"5min":1.391,"15min":1.3740000000000001},"min":{"1min":0.83499999999999996,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":2.1200000000000001,"5min":2.254,"15min":2.254},"last":1.0600000000000001}]},{"osd":11,"last update":"Wed Sep 18 10:16:33 2024","interfaces":[{"interface":"back","average":{"1min":5.0460000000000003,"5min":5.4459999999999997,"15min":5.5129999999999999},"min":{"1min":1.046,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":49.154000000000003,"5min":62.356000000000002,"15min":62.356000000000002},"last":1.48},{"interface":"front","average":{"1min":2.4769999999999999,"5min":1.6240000000000001,"15min":1.482},"min":{"1min":1.0189999999999999,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":20.449999999999999,"5min":20.449999999999999,"15min":20.449999999999999},"last":0.94599999999999995}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4119999999999999}]},{"osd":13,"last update":"Wed Sep 18 10:16:33 2024","interfaces":[{"interface":"back","average":{"1min":10.028,"5min":16.707999999999998,"15min":17.821999999999999},"min":{"1min":1.1459999999999999,"5min":0.874,"15min":0.874},"max":{"1min":113.428,"5min":139.655,"15min":139.655},"last":1.349},{"interface":"front","average":{"1min":5.1970000000000001,"5min":16.942,"15min":18.899999999999999},"min":{"1min":1.077,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":49.457999999999998,"5min":173.113,"15min":173.113},"last":1.75}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1799999999999999}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2829999999999999}]}]},{"osd":8,"up_from":22,"seq":94489280543,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303768,"kb_used_data":2248,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068072,"statfs":{"total":96636764160,"available":96325705728,"internally_reserved":0,"allocated":2301952,"data_stored":1341063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[2,4,6,7,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93300000000000005}]},{"osd":1,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":1.117,"5min":1.117,"15min":1.117},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.669,"5min":1.669,"15min":1.669},"last":1.2230000000000001},{"interface":"front","average":{"1min":2.3290000000000002,"5min":2.3290000000000002,"15min":2.3290000000000002},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":29.523,"5min":29.523,"15min":29.523},"last":1.1279999999999999}]},{"osd":2,"last update":"Wed Sep 18 10:16:48 2024","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":5.5140000000000002,"15min":6.2300000000000004},"min":{"1min":0.93500000000000005,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.8,"5min":77.435000000000002,"15min":77.435000000000002},"last":0.93500000000000005},{"interface":"front","average":{"1min":1.1639999999999999,"5min":0.97099999999999997,"15min":0.93899999999999995},"min":{"1min":0.71999999999999997,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.903,"5min":1.903,"15min":1.903},"last":0.71999999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.306}]},{"osd":4,"last update":"Wed Sep 18 10:16:48 2024","interfaces":[{"interface":"back","average":{"1min":1.23,"5min":9.5879999999999992,"15min":10.981},"min":{"1min":0.71899999999999997,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":7.1559999999999997,"5min":90.438000000000002,"15min":90.438000000000002},"last":0.83899999999999997},{"interface":"front","average":{"1min":1.1699999999999999,"5min":10.691000000000001,"15min":12.278},"min":{"1min":0.66100000000000003,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":2.1309999999999998,"5min":93.939999999999998,"15min":93.939999999999998},"last":0.66100000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4199999999999999}]},{"osd":6,"last update":"Wed Sep 18 10:16:48 2024","interfaces":[{"interface":"back","average":{"1min":1.359,"5min":15.042999999999999,"15min":17.324000000000002},"min":{"1min":0.90700000000000003,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":2.0350000000000001,"5min":293.26900000000001,"15min":293.26900000000001},"last":0.90700000000000003},{"interface":"front","average":{"1min":1.2230000000000001,"5min":5.1689999999999996,"15min":5.827},"min":{"1min":0.88900000000000001,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.839,"5min":70.203000000000003,"15min":70.203000000000003},"last":1.1060000000000001}]},{"osd":7,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":7.1950000000000003,"5min":3.6070000000000002,"15min":3.0089999999999999},"min":{"1min":0.80900000000000005,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":80.635000000000005,"5min":80.635000000000005,"15min":80.635000000000005},"last":1.0580000000000001},{"interface":"front","average":{"1min":9.3490000000000002,"5min":6.726,"15min":6.2889999999999997},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":99.060000000000002,"5min":99.060000000000002,"15min":99.060000000000002},"last":1.01}]},{"osd":9,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":4.8730000000000002,"5min":3.0110000000000001,"15min":2.7010000000000001},"min":{"1min":0.92700000000000005,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":80.076999999999998,"5min":80.076999999999998,"15min":80.076999999999998},"last":1.28},{"interface":"front","average":{"1min":1.6240000000000001,"5min":7.4489999999999998,"15min":8.4199999999999999},"min":{"1min":1.196,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":2.5190000000000001,"5min":98.914000000000001,"15min":98.914000000000001},"last":1.153}]},{"osd":10,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":1.6319999999999999,"5min":5.8559999999999999,"15min":6.5599999999999996},"min":{"1min":0.99099999999999999,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":2.3679999999999999,"5min":90.751000000000005,"15min":90.751000000000005},"last":1.081},{"interface":"front","average":{"1min":1.591,"5min":7.7889999999999997,"15min":8.8219999999999992},"min":{"1min":1.0640000000000001,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":2.3340000000000001,"5min":83.795000000000002,"15min":83.795000000000002},"last":1.3839999999999999}]},{"osd":11,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":1.7450000000000001,"5min":2.669,"15min":2.823},"min":{"1min":1.3340000000000001,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":2.383,"5min":26.462,"15min":26.462},"last":1.232},{"interface":"front","average":{"1min":1.752,"5min":1.948,"15min":1.9810000000000001},"min":{"1min":1.24,"5min":0.99299999999999999,"15min":0.99299999999999999},"max":{"1min":2.3479999999999999,"5min":12.475,"15min":12.475},"last":1.202}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0760000000000001}]},{"osd":13,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":3.052,"5min":7.9329999999999998,"15min":8.7469999999999999},"min":{"1min":1.05,"5min":1.0429999999999999,"15min":1.0429999999999999},"max":{"1min":30.582999999999998,"5min":94.222999999999999,"15min":94.222999999999999},"last":1.264},{"interface":"front","average":{"1min":1.635,"5min":2.5059999999999998,"15min":2.6509999999999998},"min":{"1min":1.1180000000000001,"5min":0.93400000000000005,"15min":0.93400000000000005},"max":{"1min":2.4689999999999999,"5min":23.983000000000001,"15min":23.983000000000001},"last":1.464}]},{"osd":14,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":1.802,"5min":16.975000000000001,"15min":19.504000000000001},"min":{"1min":1.28,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":2.4500000000000002,"5min":150.93600000000001,"15min":150.93600000000001},"last":1.3240000000000001},{"interface":"front","average":{"1min":4.8170000000000002,"5min":3.8410000000000002,"15min":3.6789999999999998},"min":{"1min":1.2210000000000001,"5min":0.90200000000000002,"15min":0.90200000000000002},"max":{"1min":69.918000000000006,"5min":69.918000000000006,"15min":69.918000000000006},"last":1.3460000000000001}]},{"osd":15,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":5.9089999999999998,"5min":3.4169999999999998,"15min":3.0019999999999998},"min":{"1min":1.2549999999999999,"5min":0.93300000000000005,"15min":0.93300000000000005},"max":{"1min":98.558000000000007,"5min":98.558000000000007,"15min":98.558000000000007},"last":1.5409999999999999},{"interface":"front","average":{"1min":1.8400000000000001,"5min":2.8119999999999998,"15min":2.9740000000000002},"min":{"1min":1.3160000000000001,"5min":0.95299999999999996,"15min":0.95299999999999996},"max":{"1min":2.5859999999999999,"5min":26.414000000000001,"15min":26.414000000000001},"last":1.373}]}]},{"osd":10,"up_from":22,"seq":94489280544,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303748,"kb_used_data":2232,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068092,"statfs":{"total":96636764160,"available":96325726208,"internally_reserved":0,"allocated":2285568,"data_stored":1331589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,4,5,6,7,9,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:16:29 2024","interfaces":[{"interface":"back","average":{"1min":16.117000000000001,"5min":16.117000000000001,"15min":16.117000000000001},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":145.12799999999999,"5min":145.12799999999999,"15min":145.12799999999999},"last":1.401},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.431,"5min":1.431,"15min":1.431},"last":1.0780000000000001}]},{"osd":1,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":9.5719999999999992,"5min":10.66,"15min":10.842000000000001},"min":{"1min":0.88500000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":145.20699999999999,"5min":145.20699999999999,"15min":145.20699999999999},"last":0.876},{"interface":"front","average":{"1min":4.7510000000000003,"5min":14.728,"15min":16.390999999999998},"min":{"1min":0.754,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":38.338000000000001,"5min":99.995999999999995,"15min":99.995999999999995},"last":0.71899999999999997}]},{"osd":2,"last update":"Wed Sep 18 10:15:33 2024","interfaces":[{"interface":"back","average":{"1min":10.893000000000001,"5min":10.893000000000001,"15min":10.893000000000001},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":84.072000000000003,"5min":84.072000000000003,"15min":84.072000000000003},"last":0.99199999999999999},{"interface":"front","average":{"1min":11.550000000000001,"5min":11.550000000000001,"15min":11.550000000000001},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":99.798000000000002,"5min":99.798000000000002,"15min":99.798000000000002},"last":1.425}]},{"osd":3,"last update":"Wed Sep 18 10:16:22 2024","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":1.119,"15min":1.119},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.99,"5min":1.99,"15min":1.99},"last":1.5},{"interface":"front","average":{"1min":18.094999999999999,"5min":18.094999999999999,"15min":18.094999999999999},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":145.535,"5min":145.535,"15min":145.535},"last":1.4510000000000001}]},{"osd":4,"last update":"Wed Sep 18 10:16:49 2024","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":20.715,"15min":23.963999999999999},"min":{"1min":0.92300000000000004,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.5820000000000001,"5min":125.23099999999999,"15min":125.23099999999999},"last":1.048},{"interface":"front","average":{"1min":1.206,"5min":25.175000000000001,"15min":29.170000000000002},"min":{"1min":0.88800000000000001,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.4930000000000001,"5min":295.93200000000002,"15min":295.93200000000002},"last":1.2210000000000001}]},{"osd":5,"last update":"Wed Sep 18 10:16:49 2024","interfaces":[{"interface":"back","average":{"1min":8.9960000000000004,"5min":21.460999999999999,"15min":23.539000000000001},"min":{"1min":0.81000000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":145.24000000000001,"5min":215.51400000000001,"15min":215.51400000000001},"last":1.111},{"interface":"front","average":{"1min":2.4140000000000001,"5min":22.346,"15min":25.667999999999999},"min":{"1min":0.85399999999999998,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":20.302,"5min":252.69300000000001,"15min":252.69300000000001},"last":1.421}]},{"osd":6,"last update":"Wed Sep 18 10:16:22 2024","interfaces":[{"interface":"back","average":{"1min":12.393000000000001,"5min":12.393000000000001,"15min":12.393000000000001},"min":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"max":{"1min":111.173,"5min":111.173,"15min":111.173},"last":1.1859999999999999},{"interface":"front","average":{"1min":15.771000000000001,"5min":15.771000000000001,"15min":15.771000000000001},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":221.13900000000001,"5min":221.13900000000001,"15min":221.13900000000001},"last":0.66500000000000004}]},{"osd":7,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":1.5109999999999999,"5min":2.7490000000000001,"15min":2.9550000000000001},"min":{"1min":0.95699999999999996,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.919,"5min":34.558,"15min":34.558},"last":1.083},{"interface":"front","average":{"1min":10.101000000000001,"5min":13.186999999999999,"15min":13.701000000000001},"min":{"1min":1.0920000000000001,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":145.5,"5min":145.5,"15min":145.5},"last":1.3140000000000001}]},{"osd":9,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":8.3640000000000008,"5min":11.07,"15min":11.521000000000001},"min":{"1min":1.0349999999999999,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":119.202,"5min":119.202,"15min":119.202},"last":1.361},{"interface":"front","average":{"1min":1.629,"5min":10.364000000000001,"15min":11.82},"min":{"1min":1.073,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":2.23,"5min":97.385999999999996,"15min":97.385999999999996},"last":0.92500000000000004}]},{"osd":11,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":10.132999999999999,"5min":11.489000000000001,"15min":11.715},"min":{"1min":1.286,"5min":0.85699999999999998,"15min":0.85699999999999998},"max":{"1min":145.26900000000001,"5min":145.26900000000001,"15min":145.26900000000001},"last":1.012},{"interface":"front","average":{"1min":5.6719999999999997,"5min":19.837,"15min":22.198},"min":{"1min":1.28,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":44.121000000000002,"5min":295.70400000000001,"15min":295.70400000000001},"last":1.2729999999999999}]},{"osd":12,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":1.6559999999999999,"5min":2.8250000000000002,"15min":3.02},"min":{"1min":1.0620000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":2.2559999999999998,"5min":34.625999999999998,"15min":34.625999999999998},"last":1.4590000000000001},{"interface":"front","average":{"1min":10.140000000000001,"5min":10.986000000000001,"15min":11.127000000000001},"min":{"1min":1.198,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":145.358,"5min":145.358,"15min":145.358},"last":1.639}]},{"osd":13,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":10.433999999999999,"5min":17.994,"15min":19.254000000000001},"min":{"1min":1.1990000000000001,"5min":0.877,"15min":0.877},"max":{"1min":123.696,"5min":123.696,"15min":123.696},"last":1.5229999999999999},{"interface":"front","average":{"1min":7.6379999999999999,"5min":14.551,"15min":15.702999999999999},"min":{"1min":1.163,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":68.363,"5min":85.879999999999995,"15min":85.879999999999995},"last":1.609}]},{"osd":14,"last update":"Wed Sep 18 10:16:22 2024","interfaces":[{"interface":"back","average":{"1min":15.864000000000001,"5min":15.864000000000001,"15min":15.864000000000001},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":145.46700000000001,"5min":145.46700000000001,"15min":145.46700000000001},"last":1.4830000000000001},{"interface":"front","average":{"1min":13.93,"5min":13.93,"15min":13.93},"min":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"max":{"1min":100.092,"5min":100.092,"15min":100.092},"last":1.587}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1080000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T10:16:57.173 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-18T10:16:57.173 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T10:16:57.490 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-18T10:16:57.493 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-09-18T10:16:57.505 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":211,"stamp":"2024-09-18T10:16:57.239981+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":147,"num_write_kb":2674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":115,"ondisk_log_size":115,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":14,"kb":1509949440,"kb_used":4865704,"kb_used_data":37272,"kb_used_omap":26,"kb_used_meta":4828197,"kb_avail":1505083736,"statfs":{"total":1546188226560,"available":1541205745664,"internally_reserved":0,"allocated":38166528,"data_stored":22667158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":27264,"internal_metadata":4944074112},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.042083"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":145,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-18T10:16:50.837041+0000","last_change":"2024-09-18T10:16:18.085265+0000","last_active":"2024-09-18T10:16:50.837041+0000","last_peered":"2024-09-18T10:16:50.837041+0000","last_clean":"2024-09-18T10:16:50.837041+0000","last_became_active":"2024-09-18T10:14:35.348943+0000","last_became_peered":"2024-09-18T10:14:35.348943+0000","last_unstale":"2024-09-18T10:16:50.837041+0000","last_undegraded":"2024-09-18T10:16:50.837041+0000","last_fullsized":"2024-09-18T10:16:50.837041+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:16:18.085027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:16:18.085027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:17:43.814969+0000","scrub_duration":0.30385945199999997,"objects_trimmed":0,"snaptrim_duration":0.00060442999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,10],"acting":[13,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":77,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-18T10:16:50.200836+0000","last_change":"2024-09-18T10:16:18.981603+0000","last_active":"2024-09-18T10:16:50.200836+0000","last_peered":"2024-09-18T10:16:50.200836+0000","last_clean":"2024-09-18T10:16:50.200836+0000","last_became_active":"2024-09-18T10:15:50.978055+0000","last_became_peered":"2024-09-18T10:15:50.978055+0000","last_unstale":"2024-09-18T10:16:50.200836+0000","last_undegraded":"2024-09-18T10:16:50.200836+0000","last_fullsized":"2024-09-18T10:16:50.200836+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:16:18.981373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:16:18.981373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:17:41.742568+0000","scrub_duration":0.40456036899999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11],"acting":[8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":141,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-18T10:16:51.167037+0000","last_change":"2024-09-18T10:15:50.883382+0000","last_active":"2024-09-18T10:16:51.167037+0000","last_peered":"2024-09-18T10:16:51.167037+0000","last_clean":"2024-09-18T10:16:51.167037+0000","last_became_active":"2024-09-18T10:14:35.329198+0000","last_became_peered":"2024-09-18T10:14:35.329198+0000","last_unstale":"2024-09-18T10:16:51.167037+0000","last_undegraded":"2024-09-18T10:16:51.167037+0000","last_fullsized":"2024-09-18T10:16:51.167037+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:15:50.883147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:15:50.883147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:17:15.846467+0000","scrub_duration":0.153222372,"objects_trimmed":0,"snaptrim_duration":0.00062415699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":143,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-18T10:16:49.303305+0000","last_change":"2024-09-18T10:16:02.749341+0000","last_active":"2024-09-18T10:16:49.303305+0000","last_peered":"2024-09-18T10:16:49.303305+0000","last_clean":"2024-09-18T10:16:49.303305+0000","last_became_active":"2024-09-18T10:14:35.524102+0000","last_became_peered":"2024-09-18T10:14:35.524102+0000","last_unstale":"2024-09-18T10:16:49.303305+0000","last_undegraded":"2024-09-18T10:16:49.303305+0000","last_fullsized":"2024-09-18T10:16:49.303305+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:16:02.749150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:16:02.749150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:17:04.725472+0000","scrub_duration":0.20368988499999999,"objects_trimmed":0,"snaptrim_duration":0.10077985,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"29'2","reported_seq":146,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-18T10:16:50.837045+0000","last_change":"2024-09-18T10:16:20.864837+0000","last_active":"2024-09-18T10:16:50.837045+0000","last_peered":"2024-09-18T10:16:50.837045+0000","last_clean":"2024-09-18T10:16:50.837045+0000","last_became_active":"2024-09-18T10:14:35.715374+0000","last_became_peered":"2024-09-18T10:14:35.715374+0000","last_unstale":"2024-09-18T10:16:50.837045+0000","last_undegraded":"2024-09-18T10:16:50.837045+0000","last_fullsized":"2024-09-18T10:16:50.837045+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2","last_scrub_stamp":"2024-09-18T10:16:20.864635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:16:20.864635+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:17:23.226052+0000","scrub_duration":0.20463374300000001,"objects_trimmed":0,"snaptrim_duration":0.002421224,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,12],"acting":[13,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":151,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-18T10:16:50.302783+0000","last_change":"2024-09-18T10:16:15.062108+0000","last_active":"2024-09-18T10:16:50.302783+0000","last_peered":"2024-09-18T10:16:50.302783+0000","last_clean":"2024-09-18T10:16:50.302783+0000","last_became_active":"2024-09-18T10:14:35.348262+0000","last_became_peered":"2024-09-18T10:14:35.348262+0000","last_unstale":"2024-09-18T10:16:50.302783+0000","last_undegraded":"2024-09-18T10:16:50.302783+0000","last_fullsized":"2024-09-18T10:16:50.302783+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:16:15.061892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:16:15.061892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:17:40.002913+0000","scrub_duration":0.40410323300000001,"objects_trimmed":0,"snaptrim_duration":0.00063430400000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,0],"acting":[14,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":144,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-18T10:16:52.538159+0000","last_change":"2024-09-18T10:16:15.352277+0000","last_active":"2024-09-18T10:16:52.538159+0000","last_peered":"2024-09-18T10:16:52.538159+0000","last_clean":"2024-09-18T10:16:52.538159+0000","last_became_active":"2024-09-18T10:14:35.349203+0000","last_became_peered":"2024-09-18T10:14:35.349203+0000","last_unstale":"2024-09-18T10:16:52.538159+0000","last_undegraded":"2024-09-18T10:16:52.538159+0000","last_fullsized":"2024-09-18T10:16:52.538159+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:16:15.352050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:16:15.352050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:17:45.176858+0000","scrub_duration":0.203632596,"objects_trimmed":0,"snaptrim_duration":0.00060130600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3],"acting":[9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"27'1","reported_seq":146,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-18T10:16:50.202745+0000","last_change":"2024-09-18T10:16:06.900858+0000","last_active":"2024-09-18T10:16:50.202745+0000","last_peered":"2024-09-18T10:16:50.202745+0000","last_clean":"2024-09-18T10:16:50.202745+0000","last_became_active":"2024-09-18T10:14:35.487746+0000","last_became_peered":"2024-09-18T10:14:35.487746+0000","last_unstale":"2024-09-18T10:16:50.202745+0000","last_undegraded":"2024-09-18T10:16:50.202745+0000","last_fullsized":"2024-09-18T10:16:50.202745+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'1","last_scrub_stamp":"2024-09-18T10:16:06.900601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:16:06.900601+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:17:25.247813+0000","scrub_duration":0.27078197700000001,"objects_trimmed":0,"snaptrim_duration":0.10072764100000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,5],"acting":[14,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"1.0","version":"29'112","reported_seq":276,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-18T10:16:51.184803+0000","last_change":"2024-09-18T10:16:03.315273+0000","last_active":"2024-09-18T10:16:51.184803+0000","last_peered":"2024-09-18T10:16:51.184803+0000","last_clean":"2024-09-18T10:16:51.184803+0000","last_became_active":"2024-09-18T10:14:32.508037+0000","last_became_peered":"2024-09-18T10:14:32.508037+0000","last_unstale":"2024-09-18T10:16:51.184803+0000","last_undegraded":"2024-09-18T10:16:51.184803+0000","last_fullsized":"2024-09-18T10:16:51.184803+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'112","last_scrub_stamp":"2024-09-18T10:16:03.315126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:31.267741+0000","last_clean_scrub_stamp":"2024-09-18T10:16:03.315126+0000","objects_scrubbed":2,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:17:16.808627+0000","scrub_duration":0.10318229800000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,15],"acting":[6,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":13},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":112,"ondisk_log_size":112,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":15,"up_from":22,"seq":94489280544,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304300,"kb_used_data":2784,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94067540,"statfs":{"total":96636764160,"available":96325160960,"internally_reserved":0,"allocated":2850816,"data_stored":1879507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,4,6,7,8,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:16:32 2024","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":0.83899999999999997,"15min":0.81899999999999995},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":2.0019999999999998,"5min":2.0019999999999998,"15min":2.0019999999999998},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.996,"5min":5.5739999999999998,"15min":6.3369999999999997},"min":{"1min":0.72899999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.506,"5min":101.291,"15min":101.291},"last":0.80400000000000005}]},{"osd":1,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.93300000000000005},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.931,"5min":1.931,"15min":1.931},"last":1.1279999999999999}]},{"osd":2,"last update":"Wed Sep 18 10:16:32 2024","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.0169999999999999,"15min":0.995},"min":{"1min":0.79700000000000004,"5min":0.5,"15min":0.5},"max":{"1min":1.7470000000000001,"5min":1.7470000000000001,"15min":1.7470000000000001},"last":1.028},{"interface":"front","average":{"1min":1.1419999999999999,"5min":4.6589999999999998,"15min":5.2450000000000001},"min":{"1min":0.82899999999999996,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.4379999999999999,"5min":78.242999999999995,"15min":78.242999999999995},"last":1.252}]},{"osd":3,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":3.3029999999999999,"5min":3.3029999999999999,"15min":3.3029999999999999},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":41.720999999999997,"5min":41.720999999999997,"15min":41.720999999999997},"last":0.55400000000000005},{"interface":"front","average":{"1min":1.014,"5min":1.014,"15min":1.014},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.49,"5min":1.49,"15min":1.49},"last":0.89500000000000002}]},{"osd":4,"last update":"Wed Sep 18 10:16:32 2024","interfaces":[{"interface":"back","average":{"1min":1.3580000000000001,"5min":9.0879999999999992,"15min":10.375999999999999},"min":{"1min":0.747,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":8.0449999999999999,"5min":99.745999999999995,"15min":99.745999999999995},"last":0.86199999999999999},{"interface":"front","average":{"1min":1.595,"5min":9.2309999999999999,"15min":10.503},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":8.0679999999999996,"5min":99.781000000000006,"15min":99.781000000000006},"last":1.1160000000000001}]},{"osd":5,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":1.0860000000000001},{"interface":"front","average":{"1min":1.7869999999999999,"5min":1.7869999999999999,"15min":1.7869999999999999},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":10.874000000000001,"5min":10.874000000000001,"15min":10.874000000000001},"last":1.163}]},{"osd":6,"last update":"Wed Sep 18 10:16:32 2024","interfaces":[{"interface":"back","average":{"1min":14.140000000000001,"5min":4.4039999999999999,"15min":2.782},"min":{"1min":0.92900000000000005,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":269.80500000000001,"5min":269.80500000000001,"15min":269.80500000000001},"last":1.0529999999999999},{"interface":"front","average":{"1min":1.361,"5min":4.7610000000000001,"15min":5.327},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":2.012,"5min":74.156000000000006,"15min":74.156000000000006},"last":1.0820000000000001}]},{"osd":7,"last update":"Wed Sep 18 10:16:53 2024","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.304,"15min":1.304},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.9590000000000001,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":0.93300000000000005},{"interface":"front","average":{"1min":4.5819999999999999,"5min":4.5819999999999999,"15min":4.5819999999999999},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":70.611000000000004,"5min":70.611000000000004,"15min":70.611000000000004},"last":1.198}]},{"osd":8,"last update":"Wed Sep 18 10:16:53 2024","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.234,"15min":1.234},"min":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"max":{"1min":1.855,"5min":1.855,"15min":1.855},"last":0.76500000000000001},{"interface":"front","average":{"1min":1.5629999999999999,"5min":1.5629999999999999,"15min":1.5629999999999999},"min":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"max":{"1min":2.2280000000000002,"5min":2.2280000000000002,"15min":2.2280000000000002},"last":1.1759999999999999}]},{"osd":10,"last update":"Wed Sep 18 10:16:32 2024","interfaces":[{"interface":"back","average":{"1min":5.8639999999999999,"5min":6.2670000000000003,"15min":6.3339999999999996},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":93.704999999999998,"5min":93.704999999999998,"15min":93.704999999999998},"last":1.248},{"interface":"front","average":{"1min":5.7770000000000001,"5min":6.2549999999999999,"15min":6.335},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":93.671000000000006,"5min":93.671000000000006,"15min":93.671000000000006},"last":1.3169999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.361}]},{"osd":12,"last update":"Wed Sep 18 10:16:53 2024","interfaces":[{"interface":"back","average":{"1min":1.6060000000000001,"5min":1.6060000000000001,"15min":1.6060000000000001},"min":{"1min":1.077,"5min":1.077,"15min":1.077},"max":{"1min":2.145,"5min":2.145,"15min":2.145},"last":1.407},{"interface":"front","average":{"1min":1.577,"5min":1.577,"15min":1.577},"min":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"max":{"1min":2.1819999999999999,"5min":2.1819999999999999,"15min":2.1819999999999999},"last":1.1479999999999999}]},{"osd":13,"last update":"Wed Sep 18 10:16:32 2024","interfaces":[{"interface":"back","average":{"1min":1.99,"5min":5.2629999999999999,"15min":5.8090000000000002},"min":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"max":{"1min":7.3179999999999996,"5min":80.197000000000003,"15min":80.197000000000003},"last":1.27},{"interface":"front","average":{"1min":3.4830000000000001,"5min":1.8080000000000001,"15min":1.5289999999999999},"min":{"1min":0.89400000000000002,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":25.193999999999999,"5min":25.193999999999999,"15min":25.193999999999999},"last":1.4690000000000001}]},{"osd":14,"last update":"Wed Sep 18 10:16:32 2024","interfaces":[{"interface":"back","average":{"1min":10.725,"5min":3.2999999999999998,"15min":2.0619999999999998},"min":{"1min":0.95699999999999996,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":117.562,"5min":117.562,"15min":117.562},"last":1.4350000000000001},{"interface":"front","average":{"1min":4.6139999999999999,"5min":9.5229999999999997,"15min":10.340999999999999},"min":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"max":{"1min":61.567,"5min":85.006,"15min":85.006},"last":1.222}]}]},{"osd":14,"up_from":22,"seq":94489280544,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303780,"kb_used_data":2264,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068060,"statfs":{"total":96636764160,"available":96325693440,"internally_reserved":0,"allocated":2318336,"data_stored":1350579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,5,7,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:16:32 2024","interfaces":[{"interface":"back","average":{"1min":1.0009999999999999,"5min":12.513,"15min":14.430999999999999},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.405,"5min":189.88800000000001,"15min":189.88800000000001},"last":0.79100000000000004},{"interface":"front","average":{"1min":7.0330000000000004,"5min":17.033000000000001,"15min":18.699000000000002},"min":{"1min":0.77300000000000002,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":71.900999999999996,"5min":180.09999999999999,"15min":180.09999999999999},"last":0.96199999999999997}]},{"osd":1,"last update":"Wed Sep 18 10:16:32 2024","interfaces":[{"interface":"back","average":{"1min":2.1040000000000001,"5min":7.7670000000000003,"15min":8.7110000000000003},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":21.334,"5min":79.626000000000005,"15min":79.626000000000005},"last":0.89400000000000002},{"interface":"front","average":{"1min":1.1479999999999999,"5min":12.601000000000001,"15min":14.51},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.8879999999999999,"5min":189.762,"15min":189.762},"last":1.022}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.417}]},{"osd":3,"last update":"Wed Sep 18 10:16:32 2024","interfaces":[{"interface":"back","average":{"1min":3.5430000000000001,"5min":12.906000000000001,"15min":14.465999999999999},"min":{"1min":0.84799999999999998,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":44.683,"5min":180.19900000000001,"15min":180.19900000000001},"last":1.7529999999999999},{"interface":"front","average":{"1min":5.3520000000000003,"5min":18.143999999999998,"15min":20.276},"min":{"1min":0.86399999999999999,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":42.570999999999998,"5min":180.851,"15min":180.851},"last":1.1899999999999999}]},{"osd":4,"last update":"Wed Sep 18 10:16:32 2024","interfaces":[{"interface":"back","average":{"1min":6.8360000000000003,"5min":21.687000000000001,"15min":24.161999999999999},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":71.911000000000001,"5min":189.809,"15min":189.809},"last":2.0249999999999999},{"interface":"front","average":{"1min":14.032999999999999,"5min":15.031000000000001,"15min":15.196999999999999},"min":{"1min":0.76100000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":133.244,"5min":180.31700000000001,"15min":180.31700000000001},"last":1.9350000000000001}]},{"osd":5,"last update":"Wed Sep 18 10:16:32 2024","interfaces":[{"interface":"back","average":{"1min":1.831,"5min":21.507000000000001,"15min":24.786000000000001},"min":{"1min":0.66300000000000003,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":13.42,"5min":277.41199999999998,"15min":277.41199999999998},"last":1.079},{"interface":"front","average":{"1min":1.9530000000000001,"5min":12.936999999999999,"15min":14.766999999999999},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":14.288,"5min":189.64699999999999,"15min":189.64699999999999},"last":1.111}]},{"osd":7,"last update":"Wed Sep 18 10:16:32 2024","interfaces":[{"interface":"back","average":{"1min":6.29,"5min":13.625999999999999,"15min":14.848000000000001},"min":{"1min":0.995,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":49.378,"5min":180.268,"15min":180.268},"last":1.1850000000000001},{"interface":"front","average":{"1min":3.8929999999999998,"5min":12.901,"15min":14.403},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":49.402999999999999,"5min":180.631,"15min":180.631},"last":1.343}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1619999999999999}]},{"osd":9,"last update":"Wed Sep 18 10:16:17 2024","interfaces":[{"interface":"back","average":{"1min":4,"5min":4,"15min":4},"min":{"1min":1.296,"5min":1.296,"15min":1.296},"max":{"1min":44.716000000000001,"5min":44.716000000000001,"15min":44.716000000000001},"last":1.4059999999999999},{"interface":"front","average":{"1min":1.427,"5min":1.427,"15min":1.427},"min":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"max":{"1min":1.9610000000000001,"5min":1.9610000000000001,"15min":1.9610000000000001},"last":1.2609999999999999}]},{"osd":10,"last update":"Wed Sep 18 10:16:52 2024","interfaces":[{"interface":"back","average":{"1min":1.5549999999999999,"5min":13.942,"15min":16.006},"min":{"1min":0.67000000000000004,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":2.2850000000000001,"5min":180.15899999999999,"15min":180.15899999999999},"last":1.3140000000000001},{"interface":"front","average":{"1min":1.75,"5min":25.152999999999999,"15min":29.053000000000001},"min":{"1min":1.1899999999999999,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":2.4420000000000002,"5min":269.423,"15min":269.423},"last":1.4790000000000001}]},{"osd":11,"last update":"Wed Sep 18 10:16:10 2024","interfaces":[{"interface":"back","average":{"1min":9.6890000000000001,"5min":9.6890000000000001,"15min":9.6890000000000001},"min":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":91.111999999999995,"5min":91.111999999999995,"15min":91.111999999999995},"last":1.452},{"interface":"front","average":{"1min":8.1460000000000008,"5min":8.1460000000000008,"15min":8.1460000000000008},"min":{"1min":1.181,"5min":1.181,"15min":1.181},"max":{"1min":91.165000000000006,"5min":91.165000000000006,"15min":91.165000000000006},"last":1.1579999999999999}]},{"osd":12,"last update":"Wed Sep 18 10:16:32 2024","interfaces":[{"interface":"back","average":{"1min":10.385,"5min":14.760999999999999,"15min":15.49},"min":{"1min":1.327,"5min":0.91100000000000003,"15min":0.91100000000000003},"max":{"1min":65.189999999999998,"5min":189.84999999999999,"15min":189.84999999999999},"last":1.288},{"interface":"front","average":{"1min":10.818,"5min":14.696,"15min":15.342000000000001},"min":{"1min":1.4890000000000001,"5min":0.91200000000000003,"15min":0.91200000000000003},"max":{"1min":65.162000000000006,"5min":180.381,"15min":180.381},"last":1.5309999999999999}]},{"osd":13,"last update":"Wed Sep 18 10:16:32 2024","interfaces":[{"interface":"back","average":{"1min":10.321999999999999,"5min":23.059000000000001,"15min":25.181999999999999},"min":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"max":{"1min":166.001,"5min":180.59100000000001,"15min":180.59100000000001},"last":1.5489999999999999},{"interface":"front","average":{"1min":1.647,"5min":20.391999999999999,"15min":23.515999999999998},"min":{"1min":0.83399999999999996,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":2.3940000000000001,"5min":189.71299999999999,"15min":189.71299999999999},"last":1.371}]},{"osd":15,"last update":"Wed Sep 18 10:16:32 2024","interfaces":[{"interface":"back","average":{"1min":4.181,"5min":13.381,"15min":14.914},"min":{"1min":1.5780000000000001,"5min":0.99399999999999999,"15min":0.99399999999999999},"max":{"1min":44.917000000000002,"5min":180.44800000000001,"15min":180.44800000000001},"last":1.6339999999999999},{"interface":"front","average":{"1min":1.7030000000000001,"5min":15.118,"15min":17.353999999999999},"min":{"1min":0.93200000000000005,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":2.1909999999999998,"5min":180.53,"15min":180.53},"last":1.512}]}]},{"osd":13,"up_from":22,"seq":94489280545,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307944,"kb_used_data":2268,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94063896,"statfs":{"total":96636764160,"available":96321429504,"internally_reserved":0,"allocated":2322432,"data_stored":1350598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,3,6,8,9,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:16:33 2024","interfaces":[{"interface":"back","average":{"1min":2.2599999999999998,"5min":17.045000000000002,"15min":19.509},"min":{"1min":0.65700000000000003,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":22.972999999999999,"5min":176.66800000000001,"15min":176.66800000000001},"last":0.71299999999999997},{"interface":"front","average":{"1min":4.2629999999999999,"5min":25.199000000000002,"15min":28.687999999999999},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":62.533000000000001,"5min":199.732,"15min":199.732},"last":0.76200000000000001}]},{"osd":1,"last update":"Wed Sep 18 10:15:48 2024","interfaces":[{"interface":"back","average":{"1min":1.986,"5min":1.986,"15min":1.986},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":18.471,"5min":18.471,"15min":18.471},"last":1.4219999999999999},{"interface":"front","average":{"1min":20.716999999999999,"5min":20.716999999999999,"15min":20.716999999999999},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":176.56800000000001,"5min":176.56800000000001,"15min":176.56800000000001},"last":0.86399999999999999}]},{"osd":2,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":11.984,"5min":11.984,"15min":11.984},"min":{"1min":0.751,"5min":0.751,"15min":0.751},"max":{"1min":196.59200000000001,"5min":196.59200000000001,"15min":196.59200000000001},"last":0.97699999999999998},{"interface":"front","average":{"1min":26.109999999999999,"5min":26.109999999999999,"15min":26.109999999999999},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":177.03,"5min":177.03,"15min":177.03},"last":1.2689999999999999}]},{"osd":3,"last update":"Wed Sep 18 10:16:33 2024","interfaces":[{"interface":"back","average":{"1min":9.5,"5min":22.245000000000001,"15min":24.369},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":158.74100000000001,"5min":168.27699999999999,"15min":168.27699999999999},"last":1.405},{"interface":"front","average":{"1min":1.2290000000000001,"5min":9.8819999999999997,"15min":11.324},"min":{"1min":0.84499999999999997,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.8959999999999999,"5min":196.643,"15min":196.643},"last":1.1719999999999999}]},{"osd":4,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":29.802,"5min":29.802,"15min":29.802},"min":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"max":{"1min":168.48099999999999,"5min":168.48099999999999,"15min":168.48099999999999},"last":1.347},{"interface":"front","average":{"1min":8.1780000000000008,"5min":8.1780000000000008,"15min":8.1780000000000008},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":65.188000000000002,"5min":65.188000000000002,"15min":65.188000000000002},"last":1.6379999999999999}]},{"osd":5,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":26.152999999999999,"5min":26.152999999999999,"15min":26.152999999999999},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":176.48099999999999,"5min":176.48099999999999,"15min":176.48099999999999},"last":1.49},{"interface":"front","average":{"1min":24.885999999999999,"5min":24.885999999999999,"15min":24.885999999999999},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":167.62299999999999,"5min":167.62299999999999,"15min":167.62299999999999},"last":1.528}]},{"osd":6,"last update":"Wed Sep 18 10:16:33 2024","interfaces":[{"interface":"back","average":{"1min":2.6589999999999998,"5min":24.015000000000001,"15min":27.574999999999999},"min":{"1min":1.042,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":23.131,"5min":176.81200000000001,"15min":176.81200000000001},"last":0.88200000000000001},{"interface":"front","average":{"1min":4.7169999999999996,"5min":23.227,"15min":26.312000000000001},"min":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"max":{"1min":61.874000000000002,"5min":167.947,"15min":167.947},"last":1.268}]},{"osd":8,"last update":"Wed Sep 18 10:16:50 2024","interfaces":[{"interface":"back","average":{"1min":1.5700000000000001,"5min":1.5700000000000001,"15min":1.5700000000000001},"min":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"max":{"1min":2.117,"5min":2.117,"15min":2.117},"last":1.0640000000000001},{"interface":"front","average":{"1min":1.514,"5min":1.514,"15min":1.514},"min":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"max":{"1min":2.278,"5min":2.278,"15min":2.278},"last":0.82699999999999996}]},{"osd":9,"last update":"Wed Sep 18 10:16:50 2024","interfaces":[{"interface":"back","average":{"1min":2.617,"5min":17.367000000000001,"15min":19.824999999999999},"min":{"1min":0.878,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":22.934000000000001,"5min":176.73599999999999,"15min":176.73599999999999},"last":0.98799999999999999},{"interface":"front","average":{"1min":1.5660000000000001,"5min":19.085000000000001,"15min":22.004999999999999},"min":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":2.448,"5min":167.84899999999999,"15min":167.84899999999999},"last":1.5780000000000001}]},{"osd":10,"last update":"Wed Sep 18 10:16:39 2024","interfaces":[{"interface":"back","average":{"1min":5.1120000000000001,"5min":19.164999999999999,"15min":21.507000000000001},"min":{"1min":1.22,"5min":1.1359999999999999,"15min":1.1359999999999999},"max":{"1min":62.137999999999998,"5min":168.07900000000001,"15min":168.07900000000001},"last":1.5},{"interface":"front","average":{"1min":17.683,"5min":29.114000000000001,"15min":31.018999999999998},"min":{"1min":1.123,"5min":1.0649999999999999,"15min":1.0649999999999999},"max":{"1min":289.68700000000001,"5min":289.68700000000001,"15min":289.68700000000001},"last":1.4430000000000001}]},{"osd":11,"last update":"Wed Sep 18 10:16:50 2024","interfaces":[{"interface":"back","average":{"1min":1.6819999999999999,"5min":1.625,"15min":1.615},"min":{"1min":1.056,"5min":1.0029999999999999,"15min":1.0029999999999999},"max":{"1min":2.5699999999999998,"5min":2.5699999999999998,"15min":2.5699999999999998},"last":1.5329999999999999},{"interface":"front","average":{"1min":2.798,"5min":21.73,"15min":24.885999999999999},"min":{"1min":1.286,"5min":1.028,"15min":1.028},"max":{"1min":23.167999999999999,"5min":176.94999999999999,"15min":176.94999999999999},"last":1.3320000000000001}]},{"osd":12,"last update":"Wed Sep 18 10:16:33 2024","interfaces":[{"interface":"back","average":{"1min":1.8440000000000001,"5min":1.6339999999999999,"15min":1.599},"min":{"1min":1.2490000000000001,"5min":1.0009999999999999,"15min":1.0009999999999999},"max":{"1min":2.5950000000000002,"5min":2.7879999999999998,"15min":2.7879999999999998},"last":1.3080000000000001},{"interface":"front","average":{"1min":5.0570000000000004,"5min":18.542999999999999,"15min":20.791},"min":{"1min":1.3029999999999999,"5min":1.296,"15min":1.296},"max":{"1min":62.200000000000003,"5min":168.34399999999999,"15min":168.34399999999999},"last":1.6619999999999999}]},{"osd":14,"last update":"Wed Sep 18 10:16:33 2024","interfaces":[{"interface":"back","average":{"1min":1.8320000000000001,"5min":5.7210000000000001,"15min":6.3689999999999998},"min":{"1min":1.139,"5min":1.0640000000000001,"15min":1.0640000000000001},"max":{"1min":2.419,"5min":87.427000000000007,"15min":87.427000000000007},"last":1.365},{"interface":"front","average":{"1min":2.9329999999999998,"5min":21.571999999999999,"15min":24.678000000000001},"min":{"1min":1.2709999999999999,"5min":1.0980000000000001,"15min":1.0980000000000001},"max":{"1min":23.268000000000001,"5min":176.87700000000001,"15min":176.87700000000001},"last":1.212}]},{"osd":15,"last update":"Wed Sep 18 10:16:33 2024","interfaces":[{"interface":"back","average":{"1min":10.048999999999999,"5min":18.277000000000001,"15min":19.648},"min":{"1min":1.272,"5min":1.266,"15min":1.266},"max":{"1min":96.564999999999998,"5min":168.40700000000001,"15min":168.40700000000001},"last":1.7310000000000001},{"interface":"front","average":{"1min":1.891,"5min":13.459,"15min":15.387},"min":{"1min":1.1950000000000001,"5min":0.95099999999999996,"15min":0.95099999999999996},"max":{"1min":2.5169999999999999,"5min":196.512,"15min":196.512},"last":1.6080000000000001}]}]},{"osd":0,"up_from":21,"seq":90194313248,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303780,"kb_used_data":2264,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068060,"statfs":{"total":96636764160,"available":96325693440,"internally_reserved":0,"allocated":2318336,"data_stored":1350579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,4,5,6,9,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 18 10:16:39 2024","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.95699999999999996,"15min":0.95899999999999996},"min":{"1min":0.56699999999999995,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.2889999999999999,"5min":1.341,"15min":1.341},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.97899999999999998,"15min":0.98499999999999999},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.2310000000000001,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":0.625}]},{"osd":2,"last update":"Wed Sep 18 10:16:39 2024","interfaces":[{"interface":"back","average":{"1min":3.2719999999999998,"5min":1.6020000000000001,"15min":1.3240000000000001},"min":{"1min":0.76600000000000001,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":46.220999999999997,"5min":46.220999999999997,"15min":46.220999999999997},"last":1.0349999999999999},{"interface":"front","average":{"1min":0.98799999999999999,"5min":1.071,"15min":1.085},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.732,"5min":1.732,"15min":1.732},"last":0.57499999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.196}]},{"osd":4,"last update":"Wed Sep 18 10:16:30 2024","interfaces":[{"interface":"back","average":{"1min":8.4039999999999999,"5min":14,"15min":14.933},"min":{"1min":0.59699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":142.20699999999999,"5min":142.20699999999999,"15min":142.20699999999999},"last":0.49399999999999999},{"interface":"front","average":{"1min":10.48,"5min":14.896000000000001,"15min":15.632},"min":{"1min":0.63,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":132.33799999999999,"5min":132.33799999999999,"15min":132.33799999999999},"last":0.55200000000000005}]},{"osd":5,"last update":"Wed Sep 18 10:16:39 2024","interfaces":[{"interface":"back","average":{"1min":1.1739999999999999,"5min":1.24,"15min":1.2509999999999999},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":1.619,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":1.569,"5min":8.1340000000000003,"15min":9.2279999999999998},"min":{"1min":0.85599999999999998,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":10.143000000000001,"5min":130.39599999999999,"15min":130.39599999999999},"last":0.92100000000000004}]},{"osd":6,"last update":"Wed Sep 18 10:16:47 2024","interfaces":[{"interface":"back","average":{"1min":1.266,"5min":1.254,"15min":1.252},"min":{"1min":0.96099999999999997,"5min":0.94299999999999995,"15min":0.94299999999999995},"max":{"1min":1.571,"5min":1.776,"15min":1.776},"last":0.82599999999999996},{"interface":"front","average":{"1min":1.3109999999999999,"5min":2.0270000000000001,"15min":2.1469999999999998},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.7909999999999999,"5min":15.250999999999999,"15min":15.250999999999999},"last":0.66100000000000003}]},{"osd":9,"last update":"Wed Sep 18 10:16:30 2024","interfaces":[{"interface":"back","average":{"1min":1.3360000000000001,"5min":1.3919999999999999,"15min":1.4019999999999999},"min":{"1min":0.88200000000000001,"5min":0.879,"15min":0.879},"max":{"1min":1.863,"5min":1.863,"15min":1.863},"last":0.74099999999999999},{"interface":"front","average":{"1min":1.667,"5min":8.7590000000000003,"15min":9.9410000000000007},"min":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":10.180999999999999,"5min":130.441,"15min":130.441},"last":0.88400000000000001}]},{"osd":11,"last update":"Wed Sep 18 10:16:30 2024","interfaces":[{"interface":"back","average":{"1min":4.8949999999999996,"5min":2.2109999999999999,"15min":1.7629999999999999},"min":{"1min":1.1579999999999999,"5min":0.90700000000000003,"15min":0.90700000000000003},"max":{"1min":41.610999999999997,"5min":41.610999999999997,"15min":41.610999999999997},"last":1.0700000000000001},{"interface":"front","average":{"1min":1.5029999999999999,"5min":4.5620000000000003,"15min":5.0720000000000001},"min":{"1min":1.1220000000000001,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":1.9850000000000001,"5min":54.805999999999997,"15min":54.805999999999997},"last":0.87}]},{"osd":12,"last update":"Wed Sep 18 10:16:30 2024","interfaces":[{"interface":"back","average":{"1min":1.7869999999999999,"5min":9.1170000000000009,"15min":10.339},"min":{"1min":0.95999999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":10.015000000000001,"5min":130.34800000000001,"15min":130.34800000000001},"last":0.95399999999999996},{"interface":"front","average":{"1min":1.4350000000000001,"5min":1.4710000000000001,"15min":1.4770000000000001},"min":{"1min":0.85899999999999999,"5min":0.502,"15min":0.502},"max":{"1min":1.944,"5min":1.944,"15min":1.944},"last":0.85499999999999998}]},{"osd":14,"last update":"Wed Sep 18 10:16:39 2024","interfaces":[{"interface":"back","average":{"1min":3.7490000000000001,"5min":8.7810000000000006,"15min":9.6199999999999992},"min":{"1min":1.044,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":38.969999999999999,"5min":130.48500000000001,"15min":130.48500000000001},"last":0.98699999999999999},{"interface":"front","average":{"1min":6.931,"5min":2.6339999999999999,"15min":1.9179999999999999},"min":{"1min":1.2609999999999999,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":80.460999999999999,"5min":80.460999999999999,"15min":80.460999999999999},"last":1.099}]},{"osd":15,"last update":"Wed Sep 18 10:16:39 2024","interfaces":[{"interface":"back","average":{"1min":1.591,"5min":2.778,"15min":2.976},"min":{"1min":1.1699999999999999,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":2.0310000000000001,"5min":23.806999999999999,"15min":23.806999999999999},"last":0.77200000000000002},{"interface":"front","average":{"1min":6.9199999999999999,"5min":9.4809999999999999,"15min":9.9079999999999995},"min":{"1min":1.1279999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":100.925,"5min":130.52799999999999,"15min":130.52799999999999},"last":1.018}]}]},{"osd":4,"up_from":21,"seq":90194313250,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303780,"kb_used_data":2264,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068060,"statfs":{"total":96636764160,"available":96325693440,"internally_reserved":0,"allocated":2318336,"data_stored":1350579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,3,5,7,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":9.0980000000000008,"15min":10.465999999999999},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.7529999999999999,"5min":89.266999999999996,"15min":89.266999999999996},"last":0.49199999999999999},{"interface":"front","average":{"1min":18.640000000000001,"5min":7.7320000000000002,"15min":5.9139999999999997},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":198.80699999999999,"5min":198.80699999999999,"15min":198.80699999999999},"last":0.54400000000000004}]},{"osd":1,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":4.0759999999999996,"5min":12.537000000000001,"15min":13.946999999999999},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":62.780000000000001,"5min":164.21199999999999,"15min":164.21199999999999},"last":0.52700000000000002},{"interface":"front","average":{"1min":2.5139999999999998,"5min":9.593,"15min":10.773},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":25.802,"5min":89.331000000000003,"15min":89.331000000000003},"last":0.80500000000000005}]},{"osd":2,"last update":"Wed Sep 18 10:15:47 2024","interfaces":[{"interface":"back","average":{"1min":23.617000000000001,"5min":23.617000000000001,"15min":23.617000000000001},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":192.136,"5min":192.136,"15min":192.136},"last":1.8740000000000001},{"interface":"front","average":{"1min":11.888999999999999,"5min":11.888999999999999,"15min":11.888999999999999},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":198.87700000000001,"5min":198.87700000000001,"15min":198.87700000000001},"last":1.347}]},{"osd":3,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":35.673000000000002,"5min":21.670999999999999,"15min":19.337},"min":{"1min":1.161,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":388.84300000000002,"5min":388.84300000000002,"15min":388.84300000000002},"last":0.76000000000000001},{"interface":"front","average":{"1min":17.747,"5min":25.643999999999998,"15min":26.960000000000001},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":255.82900000000001,"5min":255.82900000000001,"15min":255.82900000000001},"last":0.57799999999999996}]},{"osd":5,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":1.4910000000000001,"5min":15.449999999999999,"15min":17.776},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":2.169,"5min":189.55600000000001,"15min":189.55600000000001},"last":0.70999999999999996},{"interface":"front","average":{"1min":21.053000000000001,"5min":6.8079999999999998,"15min":4.4340000000000002},"min":{"1min":1.3640000000000001,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":199.15600000000001,"5min":199.15600000000001,"15min":199.15600000000001},"last":1.127}]},{"osd":6,"last update":"Wed Sep 18 10:15:34 2024","interfaces":[{"interface":"back","average":{"1min":32.25,"5min":32.25,"15min":32.25},"min":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":197.33199999999999,"5min":197.33199999999999,"15min":197.33199999999999},"last":1.8160000000000001},{"interface":"front","average":{"1min":18.963000000000001,"5min":18.963000000000001,"15min":18.963000000000001},"min":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"max":{"1min":179.19200000000001,"5min":179.19200000000001,"15min":179.19200000000001},"last":1.7769999999999999}]},{"osd":7,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":5.6050000000000004,"5min":23.198,"15min":26.129999999999999},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":62.704999999999998,"5min":290.036,"15min":290.036},"last":0.78600000000000003},{"interface":"front","average":{"1min":1.6639999999999999,"5min":14.273,"15min":16.375},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":2.3100000000000001,"5min":139.696,"15min":139.696},"last":0.98699999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1509999999999998}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7509999999999999}]},{"osd":10,"last update":"Wed Sep 18 10:16:49 2024","interfaces":[{"interface":"back","average":{"1min":1.653,"5min":18.559000000000001,"15min":21.376999999999999},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":2.4740000000000002,"5min":164.43899999999999,"15min":164.43899999999999},"last":1.0369999999999999},{"interface":"front","average":{"1min":4.6070000000000002,"5min":10.993,"15min":12.057},"min":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":59.408999999999999,"5min":89.442999999999998,"15min":89.442999999999998},"last":1.175}]},{"osd":11,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":19.57,"5min":12.567,"15min":11.4},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":199.255,"5min":199.255,"15min":199.255},"last":0.93700000000000006},{"interface":"front","average":{"1min":11.393000000000001,"5min":22.213000000000001,"15min":24.016999999999999},"min":{"1min":1.0840000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"max":{"1min":155.19800000000001,"5min":164.733,"15min":164.733},"last":0.89100000000000001}]},{"osd":12,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":4.5800000000000001,"5min":10.406000000000001,"15min":11.377000000000001},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":58.856000000000002,"5min":89.400000000000006,"15min":89.400000000000006},"last":1.095},{"interface":"front","average":{"1min":31.831,"5min":12.901999999999999,"15min":9.7469999999999999},"min":{"1min":1.4530000000000001,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":199.32900000000001,"5min":199.32900000000001,"15min":199.32900000000001},"last":1.071}]},{"osd":13,"last update":"Wed Sep 18 10:16:49 2024","interfaces":[{"interface":"back","average":{"1min":12.695,"5min":22.789999999999999,"15min":24.472000000000001},"min":{"1min":1.702,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":192.37299999999999,"5min":199.37799999999999,"15min":199.37799999999999},"last":1.2090000000000001},{"interface":"front","average":{"1min":1.8029999999999999,"5min":19.068000000000001,"15min":21.946000000000002},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":2.5699999999999998,"5min":164.53,"15min":164.53},"last":1.149}]},{"osd":14,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":22.312000000000001,"5min":21.788,"15min":21.701000000000001},"min":{"1min":1.556,"5min":0.96099999999999997,"15min":0.96099999999999997},"max":{"1min":199.43000000000001,"5min":202.191,"15min":202.191},"last":1.246},{"interface":"front","average":{"1min":9.8800000000000008,"5min":17.420999999999999,"15min":18.678000000000001},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":101.492,"5min":164.35400000000001,"15min":164.35400000000001},"last":1.224}]}]},{"osd":9,"up_from":21,"seq":90194313248,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303784,"kb_used_data":2264,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068056,"statfs":{"total":96636764160,"available":96325689344,"internally_reserved":0,"allocated":2318336,"data_stored":1350579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,4,5,6,8,10,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":7.2450000000000001,"5min":7.2450000000000001,"15min":7.2450000000000001},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":127.902,"5min":127.902,"15min":127.902},"last":0.97399999999999998},{"interface":"front","average":{"1min":8.3109999999999999,"5min":8.3109999999999999,"15min":8.3109999999999999},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":96.203000000000003,"5min":96.203000000000003,"15min":96.203000000000003},"last":0.81200000000000006}]},{"osd":1,"last update":"Wed Sep 18 10:16:40 2024","interfaces":[{"interface":"back","average":{"1min":10.587,"5min":10.587,"15min":10.587},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":166.833,"5min":166.833,"15min":166.833},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.96099999999999997}]},{"osd":2,"last update":"Wed Sep 18 10:16:18 2024","interfaces":[{"interface":"back","average":{"1min":5.6420000000000003,"5min":5.6420000000000003,"15min":5.6420000000000003},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":68.049999999999997,"5min":68.049999999999997,"15min":68.049999999999997},"last":1.1579999999999999},{"interface":"front","average":{"1min":2.0310000000000001,"5min":2.0310000000000001,"15min":2.0310000000000001},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":18.553999999999998,"5min":18.553999999999998,"15min":18.553999999999998},"last":1.4119999999999999}]},{"osd":3,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":2.875,"5min":5.5549999999999997,"15min":6.0019999999999998},"min":{"1min":0.72699999999999998,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":18.431000000000001,"5min":98.888999999999996,"15min":98.888999999999996},"last":1.0389999999999999},{"interface":"front","average":{"1min":3.0169999999999999,"5min":5.5890000000000004,"15min":6.0170000000000003},"min":{"1min":0.749,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":18.472000000000001,"5min":98.947999999999993,"15min":98.947999999999993},"last":1.546}]},{"osd":4,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":8.2360000000000007,"5min":2.5840000000000001,"15min":1.6419999999999999},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":124.15300000000001,"5min":124.15300000000001,"15min":124.15300000000001},"last":1.371},{"interface":"front","average":{"1min":6.2380000000000004,"5min":8.8539999999999992,"15min":9.2899999999999991},"min":{"1min":0.81000000000000005,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":73.992000000000004,"5min":73.992000000000004,"15min":73.992000000000004},"last":1.6319999999999999}]},{"osd":5,"last update":"Wed Sep 18 10:16:18 2024","interfaces":[{"interface":"back","average":{"1min":1.821,"5min":1.821,"15min":1.821},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":14.786,"5min":14.786,"15min":14.786},"last":0.79800000000000004},{"interface":"front","average":{"1min":8.5,"5min":8.5,"15min":8.5},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":67.915999999999997,"5min":67.915999999999997,"15min":67.915999999999997},"last":1.27}]},{"osd":6,"last update":"Wed Sep 18 10:16:18 2024","interfaces":[{"interface":"back","average":{"1min":8.3840000000000003,"5min":8.3840000000000003,"15min":8.3840000000000003},"min":{"1min":0.997,"5min":0.997,"15min":0.997},"max":{"1min":70.914000000000001,"5min":70.914000000000001,"15min":70.914000000000001},"last":1.806},{"interface":"front","average":{"1min":15.705,"5min":15.705,"15min":15.705},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":136.38800000000001,"5min":136.38800000000001,"15min":136.38800000000001},"last":1.2}]},{"osd":8,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":3.0310000000000001,"5min":4.2789999999999999,"15min":4.4870000000000001},"min":{"1min":1.0509999999999999,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":30.713000000000001,"5min":64.912999999999997,"15min":64.912999999999997},"last":1.458},{"interface":"front","average":{"1min":2.0569999999999999,"5min":5.3959999999999999,"15min":5.952},"min":{"1min":0.82499999999999996,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":12.898999999999999,"5min":57.664000000000001,"15min":57.664000000000001},"last":1.857}]},{"osd":10,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":1.3400000000000001,"5min":1.3089999999999999,"15min":1.304},"min":{"1min":0.88300000000000001,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.8180000000000001,"5min":2.109,"15min":2.109},"last":1.51},{"interface":"front","average":{"1min":6.0389999999999997,"5min":4.2309999999999999,"15min":3.9300000000000002},"min":{"1min":0.97899999999999998,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":68.216999999999999,"5min":68.216999999999999,"15min":68.216999999999999},"last":1.661}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.669}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.446}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":96}]},{"osd":14,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":4.3579999999999997,"5min":8.8759999999999994,"15min":9.6289999999999996},"min":{"1min":0.86499999999999999,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":50.773000000000003,"5min":62.899000000000001,"15min":62.899000000000001},"last":1.9039999999999999},{"interface":"front","average":{"1min":4.3540000000000001,"5min":13.071,"15min":14.523999999999999},"min":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":50.709000000000003,"5min":155.61000000000001,"15min":155.61000000000001},"last":1.6950000000000001}]},{"osd":15,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":1.4630000000000001,"5min":1.492,"15min":1.4970000000000001},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.97,"5min":1.9970000000000001,"15min":1.9970000000000001},"last":1.605},{"interface":"front","average":{"1min":9.5700000000000003,"5min":5.1749999999999998,"15min":4.4429999999999996},"min":{"1min":1.0980000000000001,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":68.140000000000001,"5min":68.140000000000001,"15min":68.140000000000001},"last":1.7470000000000001}]}]},{"osd":11,"up_from":21,"seq":90194313248,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303784,"kb_used_data":2264,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068056,"statfs":{"total":96636764160,"available":96325689344,"internally_reserved":0,"allocated":2318336,"data_stored":1350579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[2,3,6,8,9,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0049999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86599999999999999}]},{"osd":2,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":8.3390000000000004,"15min":9.5719999999999992},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.3740000000000001,"5min":99.718000000000004,"15min":99.718000000000004},"last":0.88200000000000001},{"interface":"front","average":{"1min":3.6349999999999998,"5min":1.4450000000000001,"15min":1.0800000000000001},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":49.774000000000001,"5min":49.774000000000001,"15min":49.774000000000001},"last":0.93400000000000005}]},{"osd":3,"last update":"Wed Sep 18 10:16:46 2024","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":4.2210000000000001,"15min":4.7880000000000003},"min":{"1min":0.48399999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.369,"5min":85.963999999999999,"15min":85.963999999999999},"last":0.82299999999999995},{"interface":"front","average":{"1min":5.4660000000000002,"5min":19.300000000000001,"15min":21.606000000000002},"min":{"1min":0.73699999999999999,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":80.980999999999995,"5min":125.672,"15min":125.672},"last":1.073}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.444}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5980000000000001}]},{"osd":6,"last update":"Wed Sep 18 10:16:46 2024","interfaces":[{"interface":"back","average":{"1min":5.8810000000000002,"5min":2.101,"15min":1.4710000000000001},"min":{"1min":0.70599999999999996,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":86.680000000000007,"5min":86.680000000000007,"15min":86.680000000000007},"last":1.21},{"interface":"front","average":{"1min":1.1830000000000001,"5min":5.827,"15min":6.601},"min":{"1min":0.73799999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.49,"5min":94.162000000000006,"15min":94.162000000000006},"last":1.109}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.524}]},{"osd":8,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":2.7069999999999999,"5min":5.3730000000000002,"15min":5.8170000000000002},"min":{"1min":0.95299999999999996,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":27.577999999999999,"5min":85.900000000000006,"15min":85.900000000000006},"last":1.0189999999999999},{"interface":"front","average":{"1min":2.544,"5min":8.4960000000000004,"15min":9.4879999999999995},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":27.542000000000002,"5min":99.765000000000001,"15min":99.765000000000001},"last":1.1559999999999999}]},{"osd":9,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":5.8760000000000003,"5min":5.8339999999999996,"15min":5.827},"min":{"1min":0.96399999999999997,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":89.844999999999999,"5min":89.844999999999999,"15min":89.844999999999999},"last":1.1779999999999999},{"interface":"front","average":{"1min":1.351,"5min":6.8070000000000004,"15min":7.7160000000000002},"min":{"1min":0.68600000000000005,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.742,"5min":99.810000000000002,"15min":99.810000000000002},"last":1.2470000000000001}]},{"osd":10,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":4.0069999999999997,"5min":1.679,"15min":1.2909999999999999},"min":{"1min":0.98499999999999999,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":49.819000000000003,"5min":49.819000000000003,"15min":49.819000000000003},"last":1.361},{"interface":"front","average":{"1min":1.5580000000000001,"5min":6.46,"15min":7.2770000000000001},"min":{"1min":0.98699999999999999,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.976,"5min":89.290000000000006,"15min":89.290000000000006},"last":1.274}]},{"osd":12,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":5.907,"5min":2.081,"15min":1.4430000000000001},"min":{"1min":0.96599999999999997,"5min":0.498,"15min":0.498},"max":{"1min":87.463999999999999,"5min":87.463999999999999,"15min":87.463999999999999},"last":1.7270000000000001},{"interface":"front","average":{"1min":1.5660000000000001,"5min":8.6219999999999999,"15min":9.798},"min":{"1min":1.0700000000000001,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":2.0600000000000001,"5min":86.025999999999996,"15min":86.025999999999996},"last":1.494}]},{"osd":13,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":4.4210000000000003,"5min":25.207999999999998,"15min":28.672000000000001},"min":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"max":{"1min":59.841999999999999,"5min":189.25899999999999,"15min":189.25899999999999},"last":1.476},{"interface":"front","average":{"1min":7.569,"5min":2.5659999999999998,"15min":1.732},"min":{"1min":0.88900000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":71.313000000000002,"5min":71.313000000000002,"15min":71.313000000000002},"last":1.6439999999999999}]},{"osd":14,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":14.388999999999999,"5min":8.016,"15min":6.9539999999999997},"min":{"1min":1.0129999999999999,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":256.13400000000001,"5min":256.13400000000001,"15min":256.13400000000001},"last":1.45},{"interface":"front","average":{"1min":2.7650000000000001,"5min":7.7380000000000004,"15min":8.5670000000000002},"min":{"1min":0.874,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":25.791,"5min":102.625,"15min":102.625},"last":1.4330000000000001}]},{"osd":15,"last update":"Wed Sep 18 10:16:40 2024","interfaces":[{"interface":"back","average":{"1min":4.0309999999999997,"5min":4.0309999999999997,"15min":4.0309999999999997},"min":{"1min":1.153,"5min":1.153,"15min":1.153},"max":{"1min":49.927999999999997,"5min":49.927999999999997,"15min":49.927999999999997},"last":1.518},{"interface":"front","average":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"min":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"max":{"1min":2.044,"5min":2.044,"15min":2.044},"last":1.407}]}]},{"osd":12,"up_from":21,"seq":90194313249,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303784,"kb_used_data":2268,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068056,"statfs":{"total":96636764160,"available":96325689344,"internally_reserved":0,"allocated":2322432,"data_stored":1350598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[3,4,6,7,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":2.3839999999999999,"5min":2.3839999999999999,"15min":2.3839999999999999},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":32.476999999999997,"5min":32.476999999999997,"15min":32.476999999999997},"last":0.64200000000000002},{"interface":"front","average":{"1min":2.625,"5min":2.625,"15min":2.625},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":32.509,"5min":32.509,"15min":32.509},"last":0.89300000000000002}]},{"osd":1,"last update":"Wed Sep 18 10:15:32 2024","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.6399999999999999,"5min":1.6399999999999999,"15min":1.6399999999999999},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.905,"5min":1.905,"15min":1.905},"last":0.81899999999999995}]},{"osd":2,"last update":"Wed Sep 18 10:15:32 2024","interfaces":[{"interface":"back","average":{"1min":1.002,"5min":1.002,"15min":1.002},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":1.153},{"interface":"front","average":{"1min":4.1189999999999998,"5min":4.1189999999999998,"15min":4.1189999999999998},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":62.780999999999999,"5min":62.780999999999999,"15min":62.780999999999999},"last":0.96499999999999997}]},{"osd":3,"last update":"Wed Sep 18 10:16:33 2024","interfaces":[{"interface":"back","average":{"1min":5.9429999999999996,"5min":2.1040000000000001,"15min":1.464},"min":{"1min":0.627,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":74.822999999999993,"5min":74.822999999999993,"15min":74.822999999999993},"last":1.1399999999999999},{"interface":"front","average":{"1min":8.8200000000000003,"5min":2.6840000000000002,"15min":1.6619999999999999},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":94.320999999999998,"5min":94.320999999999998,"15min":94.320999999999998},"last":0.87}]},{"osd":4,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":16.058,"5min":4.0529999999999999,"15min":2.052},"min":{"1min":0.53100000000000003,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":114.184,"5min":114.184,"15min":114.184},"last":0.82199999999999995},{"interface":"front","average":{"1min":5.8920000000000003,"5min":2.1560000000000001,"15min":1.5329999999999999},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":74.698999999999998,"5min":74.698999999999998,"15min":74.698999999999998},"last":0.94299999999999995}]},{"osd":6,"last update":"Wed Sep 18 10:16:38 2024","interfaces":[{"interface":"back","average":{"1min":10.747999999999999,"5min":10.747999999999999,"15min":10.747999999999999},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":130.20699999999999,"5min":130.20699999999999,"15min":130.20699999999999},"last":1.0660000000000001},{"interface":"front","average":{"1min":14.148,"5min":14.148,"15min":14.148},"min":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":132.60900000000001,"5min":132.60900000000001,"15min":132.60900000000001},"last":1.5329999999999999}]},{"osd":7,"last update":"Wed Sep 18 10:16:38 2024","interfaces":[{"interface":"back","average":{"1min":4.774,"5min":4.774,"15min":4.774},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":90.188000000000002,"5min":90.188000000000002,"15min":90.188000000000002},"last":1.208},{"interface":"front","average":{"1min":6.0359999999999996,"5min":6.0359999999999996,"15min":6.0359999999999996},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":75.061999999999998,"5min":75.061999999999998,"15min":75.061999999999998},"last":1.3089999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":94.382000000000005}]},{"osd":9,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":8.9719999999999995,"5min":2.8079999999999998,"15min":1.7809999999999999},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":94.421000000000006,"5min":94.421000000000006,"15min":94.421000000000006},"last":1.3799999999999999},{"interface":"front","average":{"1min":5.0410000000000004,"5min":2.0179999999999998,"15min":1.514},"min":{"1min":0.70199999999999996,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":90.242999999999995,"5min":90.242999999999995,"15min":90.242999999999995},"last":1.47}]},{"osd":10,"last update":"Wed Sep 18 10:16:38 2024","interfaces":[{"interface":"back","average":{"1min":15.145,"5min":15.145,"15min":15.145},"min":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":99.251000000000005,"5min":99.251000000000005,"15min":99.251000000000005},"last":1.4359999999999999},{"interface":"front","average":{"1min":8.7080000000000002,"5min":8.7080000000000002,"15min":8.7080000000000002},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":94.468000000000004,"5min":94.468000000000004,"15min":94.468000000000004},"last":1.5700000000000001}]},{"osd":11,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":8.9269999999999996,"5min":2.8769999999999998,"15min":1.869},"min":{"1min":0.91300000000000003,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":74.888999999999996,"5min":74.888999999999996,"15min":74.888999999999996},"last":1.8899999999999999},{"interface":"front","average":{"1min":9.1590000000000007,"5min":2.9380000000000002,"15min":1.901},"min":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"max":{"1min":94.546000000000006,"5min":94.546000000000006,"15min":94.546000000000006},"last":1.798}]},{"osd":13,"last update":"Wed Sep 18 10:16:33 2024","interfaces":[{"interface":"back","average":{"1min":7.4779999999999998,"5min":2.5619999999999998,"15min":1.742},"min":{"1min":0.79900000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":143.672,"5min":143.672,"15min":143.672},"last":1.671},{"interface":"front","average":{"1min":12.677,"5min":3.6779999999999999,"15min":2.1779999999999999},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":143.619,"5min":143.619,"15min":143.619},"last":1.744}]},{"osd":14,"last update":"Wed Sep 18 10:16:33 2024","interfaces":[{"interface":"back","average":{"1min":13.363,"5min":3.8159999999999998,"15min":2.2250000000000001},"min":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"max":{"1min":94.504999999999995,"5min":94.504999999999995,"15min":94.504999999999995},"last":1.6879999999999999},{"interface":"front","average":{"1min":12.039,"5min":3.52,"15min":2.1000000000000001},"min":{"1min":0.95099999999999996,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":92.593000000000004,"5min":92.593000000000004,"15min":92.593000000000004},"last":1.615}]},{"osd":15,"last update":"Wed Sep 18 10:16:33 2024","interfaces":[{"interface":"back","average":{"1min":6.5339999999999998,"5min":2.5019999999999998,"15min":1.8300000000000001},"min":{"1min":0.751,"5min":0.751,"15min":0.751},"max":{"1min":74.966999999999999,"5min":74.966999999999999,"15min":74.966999999999999},"last":2.028},{"interface":"front","average":{"1min":14.669,"5min":4.117,"15min":2.359},"min":{"1min":0.996,"5min":0.95299999999999996,"15min":0.95299999999999996},"max":{"1min":152.11000000000001,"5min":152.11000000000001,"15min":152.11000000000001},"last":1.9219999999999999}]}]},{"osd":1,"up_from":22,"seq":94489280544,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":303784,"kb_used_data":2264,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068056,"statfs":{"total":96636764160,"available":96325689344,"internally_reserved":0,"allocated":2318336,"data_stored":1350579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,6,9,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:16:38 2024","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":2.472,"15min":2.7370000000000001},"min":{"1min":0.65800000000000003,"5min":0.629,"15min":0.629},"max":{"1min":1.1930000000000001,"5min":36.268000000000001,"15min":36.268000000000001},"last":0.78000000000000003},{"interface":"front","average":{"1min":5.3659999999999997,"5min":15.244999999999999,"15min":16.890999999999998},"min":{"1min":0.73099999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":66.534999999999997,"5min":154.67599999999999,"15min":154.67599999999999},"last":0.83599999999999997}]},{"osd":2,"last update":"Wed Sep 18 10:16:38 2024","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.91400000000000003,"15min":0.90400000000000003},"min":{"1min":0.71499999999999997,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.4119999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.98999999999999999,"5min":2.5539999999999998,"15min":2.8140000000000001},"min":{"1min":0.75600000000000001,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.212,"5min":36.323999999999998,"15min":36.323999999999998},"last":1.0029999999999999}]},{"osd":3,"last update":"Wed Sep 18 10:16:38 2024","interfaces":[{"interface":"back","average":{"1min":5.548,"5min":10.811999999999999,"15min":11.689},"min":{"1min":0.94599999999999995,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":66.344999999999999,"5min":150.642,"15min":150.642},"last":0.95199999999999996},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.0649999999999999,"15min":1.0509999999999999},"min":{"1min":0.82999999999999996,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.6830000000000001,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":1.157}]},{"osd":4,"last update":"Wed Sep 18 10:15:35 2024","interfaces":[{"interface":"back","average":{"1min":3.3439999999999999,"5min":3.3439999999999999,"15min":3.3439999999999999},"min":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":40.783000000000001,"5min":40.783000000000001,"15min":40.783000000000001},"last":1.7210000000000001},{"interface":"front","average":{"1min":4.1699999999999999,"5min":4.1699999999999999,"15min":4.1699999999999999},"min":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":50.747,"5min":50.747,"15min":50.747},"last":1.518}]},{"osd":5,"last update":"Wed Sep 18 10:15:35 2024","interfaces":[{"interface":"back","average":{"1min":5.9720000000000004,"5min":5.9720000000000004,"15min":5.9720000000000004},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":52.317999999999998,"5min":52.317999999999998,"15min":52.317999999999998},"last":1.2649999999999999},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.748,"5min":1.748,"15min":1.748},"last":1.839}]},{"osd":6,"last update":"Wed Sep 18 10:16:38 2024","interfaces":[{"interface":"back","average":{"1min":9.2970000000000006,"5min":8.5380000000000003,"15min":8.4120000000000008},"min":{"1min":0.85099999999999998,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":120.85299999999999,"5min":120.85299999999999,"15min":120.85299999999999},"last":1.2849999999999999},{"interface":"front","average":{"1min":2.6040000000000001,"5min":8.8019999999999996,"15min":9.8350000000000009},"min":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"max":{"1min":20.600000000000001,"5min":85.045000000000002,"15min":85.045000000000002},"last":1.115}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":118.43600000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7789999999999999}]},{"osd":9,"last update":"Wed Sep 18 10:16:15 2024","interfaces":[{"interface":"back","average":{"1min":3.4369999999999998,"5min":3.4369999999999998,"15min":3.4369999999999998},"min":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"max":{"1min":36.506999999999998,"5min":36.506999999999998,"15min":36.506999999999998},"last":1.407},{"interface":"front","average":{"1min":11.339,"5min":11.339,"15min":11.339},"min":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"max":{"1min":165.13499999999999,"5min":165.13499999999999,"15min":165.13499999999999},"last":1.198}]},{"osd":10,"last update":"Wed Sep 18 10:16:38 2024","interfaces":[{"interface":"back","average":{"1min":1.5209999999999999,"5min":3.048,"15min":3.302},"min":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"max":{"1min":2.0489999999999999,"5min":36.436,"15min":36.436},"last":1.325},{"interface":"front","average":{"1min":6.0839999999999996,"5min":10.747999999999999,"15min":11.526},"min":{"1min":1.2769999999999999,"5min":0.88,"15min":0.88},"max":{"1min":66.673000000000002,"5min":100.402,"15min":100.402},"last":1.0900000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.825}]},{"osd":12,"last update":"Wed Sep 18 10:16:38 2024","interfaces":[{"interface":"back","average":{"1min":1.5549999999999999,"5min":3.681,"15min":4.0350000000000001},"min":{"1min":1.101,"5min":1.0589999999999999,"15min":1.0589999999999999},"max":{"1min":2.294,"5min":49.976999999999997,"15min":49.976999999999997},"last":1.379},{"interface":"front","average":{"1min":2.0369999999999999,"5min":7.1269999999999998,"15min":7.9749999999999996},"min":{"1min":1.196,"5min":1.196,"15min":1.196},"max":{"1min":8.4049999999999994,"5min":124.78100000000001,"15min":124.78100000000001},"last":1.232}]},{"osd":13,"last update":"Wed Sep 18 10:16:15 2024","interfaces":[{"interface":"back","average":{"1min":11.616,"5min":11.616,"15min":11.616},"min":{"1min":1.232,"5min":1.232,"15min":1.232},"max":{"1min":104.94199999999999,"5min":104.94199999999999,"15min":104.94199999999999},"last":1.26},{"interface":"front","average":{"1min":1.5720000000000001,"5min":1.5720000000000001,"15min":1.5720000000000001},"min":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"max":{"1min":2.3809999999999998,"5min":2.3809999999999998,"15min":2.3809999999999998},"last":1.454}]},{"osd":14,"last update":"Wed Sep 18 10:16:15 2024","interfaces":[{"interface":"back","average":{"1min":5.5910000000000002,"5min":5.5910000000000002,"15min":5.5910000000000002},"min":{"1min":1.276,"5min":1.276,"15min":1.276},"max":{"1min":66.825999999999993,"5min":66.825999999999993,"15min":66.825999999999993},"last":1.3560000000000001},{"interface":"front","average":{"1min":2.8359999999999999,"5min":2.8359999999999999,"15min":2.8359999999999999},"min":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"max":{"1min":22.16,"5min":22.16,"15min":22.16},"last":1.528}]},{"osd":15,"last update":"Wed Sep 18 10:16:15 2024","interfaces":[{"interface":"back","average":{"1min":3.677,"5min":3.677,"15min":3.677},"min":{"1min":1.165,"5min":1.165,"15min":1.165},"max":{"1min":36.575000000000003,"5min":36.575000000000003,"15min":36.575000000000003},"last":1.4890000000000001},{"interface":"front","average":{"1min":5.6269999999999998,"5min":5.6269999999999998,"15min":5.6269999999999998},"min":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"max":{"1min":66.763999999999996,"5min":66.763999999999996,"15min":66.763999999999996},"last":1.4319999999999999}]}]},{"osd":2,"up_from":22,"seq":94489280544,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":303780,"kb_used_data":2264,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068060,"statfs":{"total":96636764160,"available":96325693440,"internally_reserved":0,"allocated":2318336,"data_stored":1350579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,3,4,5,6,7,8,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":21.344000000000001,"15min":24.777999999999999},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.3879999999999999,"5min":419.05700000000002,"15min":419.05700000000002},"last":0.94999999999999996},{"interface":"front","average":{"1min":2.3860000000000001,"5min":4.4020000000000001,"15min":4.7380000000000004},"min":{"1min":0.57299999999999995,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":35.899999999999999,"5min":98.254000000000005,"15min":98.254000000000005},"last":0.74299999999999999}]},{"osd":3,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":5.1219999999999999,"5min":1.7529999999999999,"15min":1.1910000000000001},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":86.677000000000007,"5min":86.677000000000007,"15min":86.677000000000007},"last":0.70299999999999996},{"interface":"front","average":{"1min":3.6160000000000001,"5min":1.47,"15min":1.1120000000000001},"min":{"1min":0.68000000000000005,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":59.759,"5min":59.759,"15min":59.759},"last":0.85799999999999998}]},{"osd":4,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":7.992,"5min":3.1379999999999999,"15min":2.3290000000000002},"min":{"1min":0.55600000000000005,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":121.20099999999999,"5min":121.20099999999999,"15min":121.20099999999999},"last":1},{"interface":"front","average":{"1min":6.5250000000000004,"5min":4.9359999999999999,"15min":4.6710000000000003},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":86.548000000000002,"5min":86.548000000000002,"15min":86.548000000000002},"last":0.90400000000000003}]},{"osd":5,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":12.161,"5min":3.2879999999999998,"15min":1.8089999999999999},"min":{"1min":0.72099999999999997,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":163.828,"5min":163.828,"15min":163.828},"last":1.0429999999999999},{"interface":"front","average":{"1min":2.6160000000000001,"5min":1.3660000000000001,"15min":1.1579999999999999},"min":{"1min":0.79200000000000004,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":36.119,"5min":36.119,"15min":36.119},"last":1.0980000000000001}]},{"osd":6,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":5.2359999999999998,"5min":4.8529999999999998,"15min":4.7889999999999997},"min":{"1min":0.49199999999999999,"5min":0.441,"15min":0.441},"max":{"1min":86.766000000000005,"5min":86.766000000000005,"15min":86.766000000000005},"last":0.97499999999999998},{"interface":"front","average":{"1min":5.3959999999999999,"5min":4.9770000000000003,"15min":4.907},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":83.171999999999997,"5min":90.188999999999993,"15min":90.188999999999993},"last":1.2090000000000001}]},{"osd":7,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":13.191000000000001,"5min":3.6019999999999999,"15min":2.004},"min":{"1min":0.84999999999999998,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":88.534000000000006,"5min":88.534000000000006,"15min":88.534000000000006},"last":1.1519999999999999},{"interface":"front","average":{"1min":15.949999999999999,"5min":4.2350000000000003,"15min":2.2829999999999999},"min":{"1min":0.75600000000000001,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":88.484999999999999,"5min":88.484999999999999,"15min":88.484999999999999},"last":1.028}]},{"osd":8,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":4.3179999999999996,"15min":4.8399999999999999},"min":{"1min":0.82899999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":2.0699999999999998,"5min":92.578999999999994,"15min":92.578999999999994},"last":1.2589999999999999},{"interface":"front","average":{"1min":2.8460000000000001,"5min":2.7799999999999998,"15min":2.7690000000000001},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":36.088000000000001,"5min":36.088000000000001,"15min":36.088000000000001},"last":1.198}]},{"osd":12,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":2.9220000000000002,"5min":1.6719999999999999,"15min":1.464},"min":{"1min":0.872,"5min":0.872,"15min":0.872},"max":{"1min":36.014000000000003,"5min":36.014000000000003,"15min":36.014000000000003},"last":1.2370000000000001},{"interface":"front","average":{"1min":5.5140000000000002,"5min":2.7469999999999999,"15min":2.286},"min":{"1min":0.73199999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":86.832999999999998,"5min":86.832999999999998,"15min":86.832999999999998},"last":1.0740000000000001}]},{"osd":13,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":11.115,"5min":8.3970000000000002,"15min":7.944},"min":{"1min":0.97999999999999998,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":80.683000000000007,"5min":141.84200000000001,"15min":141.84200000000001},"last":1.4650000000000001},{"interface":"front","average":{"1min":2.911,"5min":4.9489999999999998,"15min":5.2889999999999997},"min":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":35.841000000000001,"5min":97.441999999999993,"15min":97.441999999999993},"last":1.3779999999999999}]},{"osd":15,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":12.472,"5min":3.6200000000000001,"15min":2.145},"min":{"1min":0.68400000000000005,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":86.980000000000004,"5min":86.980000000000004,"15min":86.980000000000004},"last":1.121},{"interface":"front","average":{"1min":3.3479999999999999,"5min":3.798,"15min":3.8730000000000002},"min":{"1min":0.96999999999999997,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":45.356999999999999,"5min":61.694000000000003,"15min":61.694000000000003},"last":1.4079999999999999}]}]},{"osd":3,"up_from":22,"seq":94489280544,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303780,"kb_used_data":2264,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068060,"statfs":{"total":96636764160,"available":96325693440,"internally_reserved":0,"allocated":2318336,"data_stored":1350579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,4,6,7,9,10,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:16:37 2024","interfaces":[{"interface":"back","average":{"1min":4.173,"5min":2.048,"15min":1.694},"min":{"1min":0.64300000000000002,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":65.994,"5min":65.994,"15min":65.994},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.97899999999999998,"5min":8.8260000000000005,"15min":10.134},"min":{"1min":0.75,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.752,"5min":71.316999999999993,"15min":71.316999999999993},"last":0.67200000000000004}]},{"osd":2,"last update":"Wed Sep 18 10:16:37 2024","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":4.4379999999999997,"15min":5.0140000000000002},"min":{"1min":0.76900000000000002,"5min":0.75,"15min":0.75},"max":{"1min":1.333,"5min":56.523000000000003,"15min":56.523000000000003},"last":0.80600000000000005},{"interface":"front","average":{"1min":1.165,"5min":1.1200000000000001,"15min":1.1120000000000001},"min":{"1min":0.67400000000000004,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.639,"5min":3.327,"15min":3.327},"last":0.97499999999999998}]},{"osd":4,"last update":"Wed Sep 18 10:16:37 2024","interfaces":[{"interface":"back","average":{"1min":1.0409999999999999,"5min":5.7069999999999999,"15min":6.4850000000000003},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.6399999999999999,"5min":71.135000000000005,"15min":71.135000000000005},"last":0.88200000000000001},{"interface":"front","average":{"1min":1.635,"5min":3.8130000000000002,"15min":4.1760000000000002},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":10.039,"5min":56.253999999999998,"15min":56.253999999999998},"last":1.0669999999999999}]},{"osd":5,"last update":"Wed Sep 18 10:16:06 2024","interfaces":[{"interface":"back","average":{"1min":13.997,"5min":13.997,"15min":13.997},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":125.42400000000001,"5min":125.42400000000001,"15min":125.42400000000001},"last":1.131},{"interface":"front","average":{"1min":10.323,"5min":10.323,"15min":10.323},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":112.033,"5min":112.033,"15min":112.033},"last":0.96899999999999997}]},{"osd":6,"last update":"Wed Sep 18 10:16:37 2024","interfaces":[{"interface":"back","average":{"1min":1.9279999999999999,"5min":1.29,"15min":1.1839999999999999},"min":{"1min":0.93400000000000005,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":12.83,"5min":12.83,"15min":12.83},"last":1.115},{"interface":"front","average":{"1min":5.0190000000000001,"5min":8.0050000000000008,"15min":8.5030000000000001},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":76.634,"5min":81.210999999999999,"15min":81.210999999999999},"last":1.0449999999999999}]},{"osd":7,"last update":"Wed Sep 18 10:16:37 2024","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":13.159000000000001,"15min":15.141},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.75,"5min":129.68299999999999,"15min":129.68299999999999},"last":1.1919999999999999},{"interface":"front","average":{"1min":1.4710000000000001,"5min":11.119999999999999,"15min":12.728},"min":{"1min":0.90800000000000003,"5min":0.621,"15min":0.621},"max":{"1min":1.8500000000000001,"5min":129.726,"15min":129.726},"last":1.284}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0620000000000001}]},{"osd":9,"last update":"Wed Sep 18 10:16:37 2024","interfaces":[{"interface":"back","average":{"1min":1.5660000000000001,"5min":1.446,"15min":1.4259999999999999},"min":{"1min":0.94599999999999995,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":2.0209999999999999,"5min":2.0209999999999999,"15min":2.0209999999999999},"last":1.226},{"interface":"front","average":{"1min":6.1050000000000004,"5min":6.1079999999999997,"15min":6.1079999999999997},"min":{"1min":0.86099999999999999,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":94.546999999999997,"5min":94.546999999999997,"15min":94.546999999999997},"last":1.159}]},{"osd":10,"last update":"Wed Sep 18 10:16:06 2024","interfaces":[{"interface":"back","average":{"1min":1.4870000000000001,"5min":1.4870000000000001,"15min":1.4870000000000001},"min":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":2.0659999999999998,"5min":2.0659999999999998,"15min":2.0659999999999998},"last":1.389},{"interface":"front","average":{"1min":4.9349999999999996,"5min":4.9349999999999996,"15min":4.9349999999999996},"min":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":71.668999999999997,"5min":71.668999999999997,"15min":71.668999999999997},"last":1.246}]},{"osd":13,"last update":"Wed Sep 18 10:16:37 2024","interfaces":[{"interface":"back","average":{"1min":9.4909999999999997,"5min":3.113,"15min":2.0499999999999998},"min":{"1min":1.04,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":91.290999999999997,"5min":91.290999999999997,"15min":91.290999999999997},"last":1.4390000000000001},{"interface":"front","average":{"1min":1.5249999999999999,"5min":5.3300000000000001,"15min":5.9640000000000004},"min":{"1min":1.05,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.885,"5min":71.447000000000003,"15min":71.447000000000003},"last":1.345}]},{"osd":14,"last update":"Wed Sep 18 10:16:37 2024","interfaces":[{"interface":"back","average":{"1min":1.6899999999999999,"5min":4.8899999999999997,"15min":5.423},"min":{"1min":1.0089999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"max":{"1min":4.625,"5min":56.664999999999999,"15min":56.664999999999999},"last":1.369},{"interface":"front","average":{"1min":1.865,"5min":2.2629999999999999,"15min":2.3290000000000002},"min":{"1min":1.1859999999999999,"5min":0.93999999999999995,"15min":0.93999999999999995},"max":{"1min":4.5910000000000002,"5min":18.523,"15min":18.523},"last":1.484}]},{"osd":15,"last update":"Wed Sep 18 10:16:48 2024","interfaces":[{"interface":"back","average":{"1min":1.6970000000000001,"5min":4.0049999999999999,"15min":4.3899999999999997},"min":{"1min":1.01,"5min":0.91500000000000004,"15min":0.91500000000000004},"max":{"1min":4.3289999999999997,"5min":56.357999999999997,"15min":56.357999999999997},"last":1.3149999999999999},{"interface":"front","average":{"1min":1.7589999999999999,"5min":8.5310000000000006,"15min":9.6600000000000001},"min":{"1min":1.133,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":2.1110000000000002,"5min":122.727,"15min":122.727},"last":1.53}]}]},{"osd":5,"up_from":22,"seq":94489280544,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303780,"kb_used_data":2264,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068060,"statfs":{"total":96636764160,"available":96325693440,"internally_reserved":0,"allocated":2318336,"data_stored":1350579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[2,3,4,6,8,9,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:15:47 2024","interfaces":[{"interface":"back","average":{"1min":7.8860000000000001,"5min":7.8860000000000001,"15min":7.8860000000000001},"min":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":84.430999999999997,"5min":84.430999999999997,"15min":84.430999999999997},"last":1.2889999999999999},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":1.353}]},{"osd":2,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":4.1360000000000001,"5min":8.6940000000000008,"15min":9.4540000000000006},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":51.091000000000001,"5min":84.566000000000003,"15min":84.566000000000003},"last":1.163},{"interface":"front","average":{"1min":0.98199999999999998,"5min":1.3300000000000001,"15min":1.3879999999999999},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.4690000000000001,"5min":9.5069999999999997,"15min":9.5069999999999997},"last":1.204}]},{"osd":3,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":4.9240000000000004,"5min":7.8010000000000002,"15min":8.2810000000000006},"min":{"1min":0.69099999999999995,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":51.229999999999997,"5min":84.358999999999995,"15min":84.358999999999995},"last":1.2829999999999999},{"interface":"front","average":{"1min":8.1590000000000007,"5min":8.2100000000000009,"15min":8.218},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":74.222999999999999,"5min":97.549999999999997,"15min":97.549999999999997},"last":1.1359999999999999}]},{"osd":4,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":25.673999999999999,"15min":29.777999999999999},"min":{"1min":0.67400000000000004,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.407,"5min":266.911,"15min":266.911},"last":0.84399999999999997},{"interface":"front","average":{"1min":21.388000000000002,"5min":22.515999999999998,"15min":22.704000000000001},"min":{"1min":0.76200000000000001,"5min":0.626,"15min":0.626},"max":{"1min":171.691,"5min":260.53500000000003,"15min":260.53500000000003},"last":0.91900000000000004}]},{"osd":6,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":3.6309999999999998,"5min":8.0809999999999995,"15min":8.8230000000000004},"min":{"1min":0.70099999999999996,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":41.886000000000003,"5min":58.171999999999997,"15min":58.171999999999997},"last":1.2370000000000001},{"interface":"front","average":{"1min":5.8079999999999998,"5min":18.274999999999999,"15min":20.353000000000002},"min":{"1min":0.85699999999999998,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":54.984999999999999,"5min":227.48099999999999,"15min":227.48099999999999},"last":0.97999999999999998}]},{"osd":7,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":12.013,"5min":12.013,"15min":12.013},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":123.111,"5min":123.111,"15min":123.111},"last":1.5249999999999999},{"interface":"front","average":{"1min":13.994,"5min":13.994,"15min":13.994},"min":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"max":{"1min":84.617000000000004,"5min":84.617000000000004,"15min":84.617000000000004},"last":1.329}]},{"osd":8,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":4.4720000000000004,"5min":17.175999999999998,"15min":19.292999999999999},"min":{"1min":0.879,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":51.506,"5min":260.44200000000001,"15min":260.44200000000001},"last":0.55100000000000005},{"interface":"front","average":{"1min":8.2609999999999992,"5min":7.2859999999999996,"15min":7.1239999999999997},"min":{"1min":0.83899999999999997,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":71.266000000000005,"5min":106.818,"15min":106.818},"last":0.76300000000000001}]},{"osd":9,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":3.8780000000000001,"5min":5.2930000000000001,"15min":5.5289999999999999},"min":{"1min":0.91300000000000003,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":41.652000000000001,"5min":80.272000000000006,"15min":80.272000000000006},"last":1.4590000000000001},{"interface":"front","average":{"1min":1.3799999999999999,"5min":1.764,"15min":1.8280000000000001},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":2.008,"5min":9.5690000000000008,"15min":9.5690000000000008},"last":1.492}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6479999999999999}]},{"osd":11,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":1.494,"5min":7.8120000000000003,"15min":8.8650000000000002},"min":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":1.825,"5min":123.158,"15min":123.158},"last":1.3240000000000001},{"interface":"front","average":{"1min":8.8490000000000002,"5min":10.863,"15min":11.199},"min":{"1min":0.93200000000000005,"5min":0.86499999999999999,"15min":0.86499999999999999},"max":{"1min":68.938000000000002,"5min":84.522999999999996,"15min":84.522999999999996},"last":1.4159999999999999}]},{"osd":12,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":4.0250000000000004,"5min":1.8580000000000001,"15min":1.4970000000000001},"min":{"1min":0.95199999999999996,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":41.929000000000002,"5min":41.929000000000002,"15min":41.929000000000002},"last":1.0489999999999999},{"interface":"front","average":{"1min":1.774,"5min":1.9339999999999999,"15min":1.9610000000000001},"min":{"1min":0.93999999999999995,"5min":0.88300000000000001,"15min":0.88300000000000001},"max":{"1min":4.1150000000000002,"5min":9.7710000000000008,"15min":9.7710000000000008},"last":1.0860000000000001}]},{"osd":13,"last update":"Wed Sep 18 10:16:48 2024","interfaces":[{"interface":"back","average":{"1min":1.6499999999999999,"5min":19.745999999999999,"15min":22.762},"min":{"1min":1.171,"5min":1.1120000000000001,"15min":1.1120000000000001},"max":{"1min":2.0550000000000002,"5min":140.273,"15min":140.273},"last":1.3859999999999999},{"interface":"front","average":{"1min":4.665,"5min":20.309000000000001,"15min":22.917000000000002},"min":{"1min":1.194,"5min":0.98099999999999998,"15min":0.98099999999999998},"max":{"1min":51.432000000000002,"5min":170.86000000000001,"15min":170.86000000000001},"last":1.6339999999999999}]},{"osd":14,"last update":"Wed Sep 18 10:16:48 2024","interfaces":[{"interface":"back","average":{"1min":9.8930000000000007,"5min":12.584,"15min":13.032},"min":{"1min":1.385,"5min":1.0389999999999999,"15min":1.0389999999999999},"max":{"1min":93.295000000000002,"5min":101.733,"15min":101.733},"last":1.5189999999999999},{"interface":"front","average":{"1min":1.79,"5min":7.9729999999999999,"15min":9.0030000000000001},"min":{"1min":1.321,"5min":0.90100000000000002,"15min":0.90100000000000002},"max":{"1min":2.2290000000000001,"5min":122.952,"15min":122.952},"last":1.55}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8839999999999999}]}]},{"osd":6,"up_from":22,"seq":94489280544,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304304,"kb_used_data":2784,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94067536,"statfs":{"total":96636764160,"available":96325156864,"internally_reserved":0,"allocated":2850816,"data_stored":1879507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,3,4,5,7,8,9,11,12,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:15:47 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.68200000000000005}]},{"osd":1,"last update":"Wed Sep 18 10:16:48 2024","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":3.8100000000000001,"15min":4.306},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.2150000000000001,"5min":52.332999999999998,"15min":52.332999999999998},"last":0.624},{"interface":"front","average":{"1min":4.8369999999999997,"5min":1.6990000000000001,"15min":1.1759999999999999},"min":{"1min":0.68300000000000005,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":67.763000000000005,"5min":67.763000000000005,"15min":67.763000000000005},"last":0.94399999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.343}]},{"osd":3,"last update":"Wed Sep 18 10:16:48 2024","interfaces":[{"interface":"back","average":{"1min":5.3380000000000001,"5min":1.792,"15min":1.2010000000000001},"min":{"1min":0.65500000000000003,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":76.891999999999996,"5min":76.891999999999996,"15min":76.891999999999996},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.98999999999999999,"5min":11.403,"15min":13.138999999999999},"min":{"1min":0.751,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.204,"5min":169.244,"15min":169.244},"last":0.877}]},{"osd":4,"last update":"Wed Sep 18 10:16:02 2024","interfaces":[{"interface":"back","average":{"1min":11.111000000000001,"5min":11.111000000000001,"15min":11.111000000000001},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":128.053,"5min":128.053,"15min":128.053},"last":0.76100000000000001},{"interface":"front","average":{"1min":4.6189999999999998,"5min":4.6189999999999998,"15min":4.6189999999999998},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":67.960999999999999,"5min":67.960999999999999,"15min":67.960999999999999},"last":0.83399999999999996}]},{"osd":5,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":6.0579999999999998,"5min":10.194000000000001,"15min":10.884},"min":{"1min":0.72099999999999997,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":68.025999999999996,"5min":119.032,"15min":119.032},"last":0.68100000000000005},{"interface":"front","average":{"1min":5.6760000000000002,"5min":4.0380000000000003,"15min":3.7650000000000001},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":83.858000000000004,"5min":83.858000000000004,"15min":83.858000000000004},"last":1.0109999999999999}]},{"osd":7,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":4.9539999999999997,"5min":6.3959999999999999,"15min":6.6360000000000001},"min":{"1min":0.94099999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":69.171000000000006,"5min":69.171000000000006,"15min":69.171000000000006},"last":0.96499999999999997},{"interface":"front","average":{"1min":5.0960000000000001,"5min":3.4630000000000001,"15min":3.1909999999999998},"min":{"1min":1.089,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":69.239000000000004,"5min":69.239000000000004,"15min":69.239000000000004},"last":1.1879999999999999}]},{"osd":8,"last update":"Wed Sep 18 10:16:51 2024","interfaces":[{"interface":"back","average":{"1min":5.3659999999999997,"5min":5.3659999999999997,"15min":5.3659999999999997},"min":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"max":{"1min":76.766000000000005,"5min":76.766000000000005,"15min":76.766000000000005},"last":1.0800000000000001},{"interface":"front","average":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"min":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"max":{"1min":1.704,"5min":1.704,"15min":1.704},"last":1.238}]},{"osd":9,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":5.4269999999999996,"5min":7.7089999999999996,"15min":8.0890000000000004},"min":{"1min":0.95799999999999996,"5min":0.84599999999999997,"15min":0.84599999999999997},"max":{"1min":76.997,"5min":95.691999999999993,"15min":95.691999999999993},"last":0.91900000000000004},{"interface":"front","average":{"1min":1.2949999999999999,"5min":10.472,"15min":12.002000000000001},"min":{"1min":0.96899999999999997,"5min":0.83099999999999996,"15min":0.83099999999999996},"max":{"1min":1.696,"5min":165.31100000000001,"15min":165.31100000000001},"last":1.153}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.329}]},{"osd":11,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":5.0510000000000002,"5min":2.0630000000000002,"15min":1.5649999999999999},"min":{"1min":1.002,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":67.909999999999997,"5min":67.909999999999997,"15min":67.909999999999997},"last":0.99099999999999999},{"interface":"front","average":{"1min":9.1829999999999998,"5min":4.9429999999999996,"15min":4.2359999999999998},"min":{"1min":1.1870000000000001,"5min":0.92000000000000004,"15min":0.92000000000000004},"max":{"1min":77.158000000000001,"5min":77.158000000000001,"15min":77.158000000000001},"last":1.125}]},{"osd":12,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":1.3859999999999999,"5min":4.3140000000000001,"15min":4.8019999999999996},"min":{"1min":1.079,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":1.796,"5min":52.378,"15min":52.378},"last":1.052},{"interface":"front","average":{"1min":5.1109999999999998,"5min":4.5720000000000001,"15min":4.4820000000000002},"min":{"1min":0.92800000000000005,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":67.835999999999999,"5min":67.835999999999999,"15min":67.835999999999999},"last":1.0389999999999999}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2050000000000001}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2669999999999999}]},{"osd":15,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":5.1680000000000001,"5min":2.2120000000000002,"15min":1.72},"min":{"1min":1.1000000000000001,"5min":0.90300000000000002,"15min":0.90300000000000002},"max":{"1min":68.069999999999993,"5min":68.069999999999993,"15min":68.069999999999993},"last":1.1000000000000001},{"interface":"front","average":{"1min":5.6200000000000001,"5min":4.3019999999999996,"15min":4.0819999999999999},"min":{"1min":1.2030000000000001,"5min":1.0429999999999999,"15min":1.0429999999999999},"max":{"1min":77.081999999999994,"5min":77.081999999999994,"15min":77.081999999999994},"last":1.2110000000000001}]}]},{"osd":7,"up_from":22,"seq":94489280544,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303780,"kb_used_data":2264,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068060,"statfs":{"total":96636764160,"available":96325693440,"internally_reserved":0,"allocated":2318336,"data_stored":1350579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,4,6,8,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:16:33 2024","interfaces":[{"interface":"back","average":{"1min":3.9900000000000002,"5min":10.974,"15min":12.138},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":59.137999999999998,"5min":139.58600000000001,"15min":139.58600000000001},"last":0.91000000000000003},{"interface":"front","average":{"1min":1.121,"5min":1.0229999999999999,"15min":1.0069999999999999},"min":{"1min":0.67800000000000005,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.7390000000000001,"5min":1.7390000000000001,"15min":1.7390000000000001},"last":0.81799999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2090000000000001}]},{"osd":2,"last update":"Wed Sep 18 10:15:46 2024","interfaces":[{"interface":"back","average":{"1min":15.436,"5min":15.436,"15min":15.436},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":139.36799999999999,"5min":139.36799999999999,"15min":139.36799999999999},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.94299999999999995}]},{"osd":3,"last update":"Wed Sep 18 10:15:46 2024","interfaces":[{"interface":"back","average":{"1min":17.327999999999999,"5min":17.327999999999999,"15min":17.327999999999999},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":139.27199999999999,"5min":139.27199999999999,"15min":139.27199999999999},"last":1.3460000000000001},{"interface":"front","average":{"1min":3.964,"5min":3.964,"15min":3.964},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":23.748999999999999,"5min":23.748999999999999,"15min":23.748999999999999},"last":0.82899999999999996}]},{"osd":4,"last update":"Wed Sep 18 10:16:37 2024","interfaces":[{"interface":"back","average":{"1min":9.8670000000000009,"5min":17.291,"15min":18.527999999999999},"min":{"1min":0.68000000000000005,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":125.10299999999999,"5min":139.51599999999999,"15min":139.51599999999999},"last":0.71899999999999997},{"interface":"front","average":{"1min":3.3559999999999999,"5min":4.6699999999999999,"15min":4.8890000000000002},"min":{"1min":0.68000000000000005,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":49.241999999999997,"5min":49.241999999999997,"15min":49.241999999999997},"last":0.76200000000000001}]},{"osd":5,"last update":"Wed Sep 18 10:15:46 2024","interfaces":[{"interface":"back","average":{"1min":1.099,"5min":1.099,"15min":1.099},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":2.3620000000000001,"5min":2.3620000000000001,"15min":2.3620000000000001},"last":1.4059999999999999},{"interface":"front","average":{"1min":17.736999999999998,"5min":17.736999999999998,"15min":17.736999999999998},"min":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"max":{"1min":139.44300000000001,"5min":139.44300000000001,"15min":139.44300000000001},"last":0.78000000000000003}]},{"osd":6,"last update":"Wed Sep 18 10:16:33 2024","interfaces":[{"interface":"back","average":{"1min":6.0869999999999997,"5min":15.117000000000001,"15min":16.622},"min":{"1min":0.998,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":59.343000000000004,"5min":183.001,"15min":183.001},"last":1.2769999999999999},{"interface":"front","average":{"1min":7.0830000000000002,"5min":4.8650000000000002,"15min":4.4950000000000001},"min":{"1min":0.82399999999999995,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":58.284999999999997,"5min":58.284999999999997,"15min":58.284999999999997},"last":1.23}]},{"osd":8,"last update":"Wed Sep 18 10:16:30 2024","interfaces":[{"interface":"back","average":{"1min":1.4079999999999999,"5min":1.2809999999999999,"15min":1.26},"min":{"1min":0.84999999999999998,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.9339999999999999,"5min":2.6139999999999999,"15min":2.6139999999999999},"last":1.1599999999999999},{"interface":"front","average":{"1min":5.3029999999999999,"5min":13.785,"15min":15.199},"min":{"1min":1.0980000000000001,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":59.523000000000003,"5min":139.798,"15min":139.798},"last":1.0149999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.861}]},{"osd":10,"last update":"Wed Sep 18 10:16:30 2024","interfaces":[{"interface":"back","average":{"1min":16.035,"5min":8.4100000000000001,"15min":7.1390000000000002},"min":{"1min":1.0149999999999999,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":156.45500000000001,"5min":156.45500000000001,"15min":156.45500000000001},"last":1.3600000000000001},{"interface":"front","average":{"1min":1.4910000000000001,"5min":1.391,"15min":1.3740000000000001},"min":{"1min":0.83499999999999996,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":2.1200000000000001,"5min":2.254,"15min":2.254},"last":1.325}]},{"osd":11,"last update":"Wed Sep 18 10:16:33 2024","interfaces":[{"interface":"back","average":{"1min":5.0460000000000003,"5min":5.4459999999999997,"15min":5.5129999999999999},"min":{"1min":1.046,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":49.154000000000003,"5min":62.356000000000002,"15min":62.356000000000002},"last":1.5740000000000001},{"interface":"front","average":{"1min":2.4769999999999999,"5min":1.6240000000000001,"15min":1.482},"min":{"1min":1.0189999999999999,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":20.449999999999999,"5min":20.449999999999999,"15min":20.449999999999999},"last":1.5169999999999999}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4790000000000001}]},{"osd":13,"last update":"Wed Sep 18 10:16:33 2024","interfaces":[{"interface":"back","average":{"1min":10.028,"5min":16.707999999999998,"15min":17.821999999999999},"min":{"1min":1.1459999999999999,"5min":0.874,"15min":0.874},"max":{"1min":113.428,"5min":139.655,"15min":139.655},"last":1.55},{"interface":"front","average":{"1min":5.1970000000000001,"5min":16.942,"15min":18.899999999999999},"min":{"1min":1.077,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":49.457999999999998,"5min":173.113,"15min":173.113},"last":1.71}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.397}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6120000000000001}]}]},{"osd":8,"up_from":22,"seq":94489280544,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303780,"kb_used_data":2264,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068060,"statfs":{"total":96636764160,"available":96325693440,"internally_reserved":0,"allocated":2318336,"data_stored":1350579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[2,4,6,7,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93300000000000005}]},{"osd":1,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":1.117,"5min":1.117,"15min":1.117},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.669,"5min":1.669,"15min":1.669},"last":1.2230000000000001},{"interface":"front","average":{"1min":2.3290000000000002,"5min":2.3290000000000002,"15min":2.3290000000000002},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":29.523,"5min":29.523,"15min":29.523},"last":1.1279999999999999}]},{"osd":2,"last update":"Wed Sep 18 10:16:48 2024","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":5.5140000000000002,"15min":6.2300000000000004},"min":{"1min":0.93500000000000005,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.8,"5min":77.435000000000002,"15min":77.435000000000002},"last":0.85799999999999998},{"interface":"front","average":{"1min":1.1639999999999999,"5min":0.97099999999999997,"15min":0.93899999999999995},"min":{"1min":0.71999999999999997,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.903,"5min":1.903,"15min":1.903},"last":0.64500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.306}]},{"osd":4,"last update":"Wed Sep 18 10:16:48 2024","interfaces":[{"interface":"back","average":{"1min":1.23,"5min":9.5879999999999992,"15min":10.981},"min":{"1min":0.71899999999999997,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":7.1559999999999997,"5min":90.438000000000002,"15min":90.438000000000002},"last":0.69099999999999995},{"interface":"front","average":{"1min":1.1699999999999999,"5min":10.691000000000001,"15min":12.278},"min":{"1min":0.66100000000000003,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":2.1309999999999998,"5min":93.939999999999998,"15min":93.939999999999998},"last":0.61199999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4199999999999999}]},{"osd":6,"last update":"Wed Sep 18 10:16:48 2024","interfaces":[{"interface":"back","average":{"1min":1.359,"5min":15.042999999999999,"15min":17.324000000000002},"min":{"1min":0.90700000000000003,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":2.0350000000000001,"5min":293.26900000000001,"15min":293.26900000000001},"last":0.81100000000000005},{"interface":"front","average":{"1min":1.2230000000000001,"5min":5.1689999999999996,"15min":5.827},"min":{"1min":0.88900000000000001,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.839,"5min":70.203000000000003,"15min":70.203000000000003},"last":1.038}]},{"osd":7,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":7.1950000000000003,"5min":3.6070000000000002,"15min":3.0089999999999999},"min":{"1min":0.80900000000000005,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":80.635000000000005,"5min":80.635000000000005,"15min":80.635000000000005},"last":0.91000000000000003},{"interface":"front","average":{"1min":9.3490000000000002,"5min":6.726,"15min":6.2889999999999997},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":99.060000000000002,"5min":99.060000000000002,"15min":99.060000000000002},"last":1.0009999999999999}]},{"osd":9,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":4.8730000000000002,"5min":3.0110000000000001,"15min":2.7010000000000001},"min":{"1min":0.92700000000000005,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":80.076999999999998,"5min":80.076999999999998,"15min":80.076999999999998},"last":1.1850000000000001},{"interface":"front","average":{"1min":1.6240000000000001,"5min":7.4489999999999998,"15min":8.4199999999999999},"min":{"1min":1.196,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":2.5190000000000001,"5min":98.914000000000001,"15min":98.914000000000001},"last":0.94299999999999995}]},{"osd":10,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":1.6319999999999999,"5min":5.8559999999999999,"15min":6.5599999999999996},"min":{"1min":0.99099999999999999,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":2.3679999999999999,"5min":90.751000000000005,"15min":90.751000000000005},"last":1.1220000000000001},{"interface":"front","average":{"1min":1.591,"5min":7.7889999999999997,"15min":8.8219999999999992},"min":{"1min":1.0640000000000001,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":2.3340000000000001,"5min":83.795000000000002,"15min":83.795000000000002},"last":1.3320000000000001}]},{"osd":11,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":1.7450000000000001,"5min":2.669,"15min":2.823},"min":{"1min":1.3340000000000001,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":2.383,"5min":26.462,"15min":26.462},"last":0.96799999999999997},{"interface":"front","average":{"1min":1.752,"5min":1.948,"15min":1.9810000000000001},"min":{"1min":1.24,"5min":0.99299999999999999,"15min":0.99299999999999999},"max":{"1min":2.3479999999999999,"5min":12.475,"15min":12.475},"last":1.2370000000000001}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0760000000000001}]},{"osd":13,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":3.052,"5min":7.9329999999999998,"15min":8.7469999999999999},"min":{"1min":1.05,"5min":1.0429999999999999,"15min":1.0429999999999999},"max":{"1min":30.582999999999998,"5min":94.222999999999999,"15min":94.222999999999999},"last":1.3700000000000001},{"interface":"front","average":{"1min":1.635,"5min":2.5059999999999998,"15min":2.6509999999999998},"min":{"1min":1.1180000000000001,"5min":0.93400000000000005,"15min":0.93400000000000005},"max":{"1min":2.4689999999999999,"5min":23.983000000000001,"15min":23.983000000000001},"last":1.516}]},{"osd":14,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":1.802,"5min":16.975000000000001,"15min":19.504000000000001},"min":{"1min":1.28,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":2.4500000000000002,"5min":150.93600000000001,"15min":150.93600000000001},"last":1.085},{"interface":"front","average":{"1min":4.8170000000000002,"5min":3.8410000000000002,"15min":3.6789999999999998},"min":{"1min":1.2210000000000001,"5min":0.90200000000000002,"15min":0.90200000000000002},"max":{"1min":69.918000000000006,"5min":69.918000000000006,"15min":69.918000000000006},"last":1.4730000000000001}]},{"osd":15,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":5.9089999999999998,"5min":3.4169999999999998,"15min":3.0019999999999998},"min":{"1min":1.2549999999999999,"5min":0.93300000000000005,"15min":0.93300000000000005},"max":{"1min":98.558000000000007,"5min":98.558000000000007,"15min":98.558000000000007},"last":1.429},{"interface":"front","average":{"1min":1.8400000000000001,"5min":2.8119999999999998,"15min":2.9740000000000002},"min":{"1min":1.3160000000000001,"5min":0.95299999999999996,"15min":0.95299999999999996},"max":{"1min":2.5859999999999999,"5min":26.414000000000001,"15min":26.414000000000001},"last":1.165}]}]},{"osd":10,"up_from":22,"seq":94489280545,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303780,"kb_used_data":2264,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068060,"statfs":{"total":96636764160,"available":96325693440,"internally_reserved":0,"allocated":2318336,"data_stored":1350579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,4,5,6,7,9,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:16:29 2024","interfaces":[{"interface":"back","average":{"1min":16.117000000000001,"5min":16.117000000000001,"15min":16.117000000000001},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":145.12799999999999,"5min":145.12799999999999,"15min":145.12799999999999},"last":1.401},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.431,"5min":1.431,"15min":1.431},"last":1.0780000000000001}]},{"osd":1,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":9.5719999999999992,"5min":10.66,"15min":10.842000000000001},"min":{"1min":0.88500000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":145.20699999999999,"5min":145.20699999999999,"15min":145.20699999999999},"last":0.86699999999999999},{"interface":"front","average":{"1min":4.7510000000000003,"5min":14.728,"15min":16.390999999999998},"min":{"1min":0.754,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":38.338000000000001,"5min":99.995999999999995,"15min":99.995999999999995},"last":1.3089999999999999}]},{"osd":2,"last update":"Wed Sep 18 10:15:33 2024","interfaces":[{"interface":"back","average":{"1min":10.893000000000001,"5min":10.893000000000001,"15min":10.893000000000001},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":84.072000000000003,"5min":84.072000000000003,"15min":84.072000000000003},"last":0.99199999999999999},{"interface":"front","average":{"1min":11.550000000000001,"5min":11.550000000000001,"15min":11.550000000000001},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":99.798000000000002,"5min":99.798000000000002,"15min":99.798000000000002},"last":1.425}]},{"osd":3,"last update":"Wed Sep 18 10:16:22 2024","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":1.119,"15min":1.119},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.99,"5min":1.99,"15min":1.99},"last":1.5},{"interface":"front","average":{"1min":18.094999999999999,"5min":18.094999999999999,"15min":18.094999999999999},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":145.535,"5min":145.535,"15min":145.535},"last":1.4510000000000001}]},{"osd":4,"last update":"Wed Sep 18 10:16:49 2024","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":20.715,"15min":23.963999999999999},"min":{"1min":0.92300000000000004,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.5820000000000001,"5min":125.23099999999999,"15min":125.23099999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":1.206,"5min":25.175000000000001,"15min":29.170000000000002},"min":{"1min":0.88800000000000001,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.4930000000000001,"5min":295.93200000000002,"15min":295.93200000000002},"last":0.82399999999999995}]},{"osd":5,"last update":"Wed Sep 18 10:16:49 2024","interfaces":[{"interface":"back","average":{"1min":8.9960000000000004,"5min":21.460999999999999,"15min":23.539000000000001},"min":{"1min":0.81000000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":145.24000000000001,"5min":215.51400000000001,"15min":215.51400000000001},"last":1.0640000000000001},{"interface":"front","average":{"1min":2.4140000000000001,"5min":22.346,"15min":25.667999999999999},"min":{"1min":0.85399999999999998,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":20.302,"5min":252.69300000000001,"15min":252.69300000000001},"last":1.397}]},{"osd":6,"last update":"Wed Sep 18 10:16:22 2024","interfaces":[{"interface":"back","average":{"1min":12.393000000000001,"5min":12.393000000000001,"15min":12.393000000000001},"min":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"max":{"1min":111.173,"5min":111.173,"15min":111.173},"last":1.1899999999999999},{"interface":"front","average":{"1min":15.771000000000001,"5min":15.771000000000001,"15min":15.771000000000001},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":221.13900000000001,"5min":221.13900000000001,"15min":221.13900000000001},"last":1.024}]},{"osd":7,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":1.5109999999999999,"5min":2.7490000000000001,"15min":2.9550000000000001},"min":{"1min":0.95699999999999996,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.919,"5min":34.558,"15min":34.558},"last":1.1100000000000001},{"interface":"front","average":{"1min":10.101000000000001,"5min":13.186999999999999,"15min":13.701000000000001},"min":{"1min":1.0920000000000001,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":145.5,"5min":145.5,"15min":145.5},"last":1.2689999999999999}]},{"osd":9,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":8.3640000000000008,"5min":11.07,"15min":11.521000000000001},"min":{"1min":1.0349999999999999,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":119.202,"5min":119.202,"15min":119.202},"last":1.538},{"interface":"front","average":{"1min":1.629,"5min":10.364000000000001,"15min":11.82},"min":{"1min":1.073,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":2.23,"5min":97.385999999999996,"15min":97.385999999999996},"last":1.3480000000000001}]},{"osd":11,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":10.132999999999999,"5min":11.489000000000001,"15min":11.715},"min":{"1min":1.286,"5min":0.85699999999999998,"15min":0.85699999999999998},"max":{"1min":145.26900000000001,"5min":145.26900000000001,"15min":145.26900000000001},"last":1.774},{"interface":"front","average":{"1min":5.6719999999999997,"5min":19.837,"15min":22.198},"min":{"1min":1.28,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":44.121000000000002,"5min":295.70400000000001,"15min":295.70400000000001},"last":1.4490000000000001}]},{"osd":12,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":1.6559999999999999,"5min":2.8250000000000002,"15min":3.02},"min":{"1min":1.0620000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":2.2559999999999998,"5min":34.625999999999998,"15min":34.625999999999998},"last":1.242},{"interface":"front","average":{"1min":10.140000000000001,"5min":10.986000000000001,"15min":11.127000000000001},"min":{"1min":1.198,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":145.358,"5min":145.358,"15min":145.358},"last":1.375}]},{"osd":13,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":10.433999999999999,"5min":17.994,"15min":19.254000000000001},"min":{"1min":1.1990000000000001,"5min":0.877,"15min":0.877},"max":{"1min":123.696,"5min":123.696,"15min":123.696},"last":1.583},{"interface":"front","average":{"1min":7.6379999999999999,"5min":14.551,"15min":15.702999999999999},"min":{"1min":1.163,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":68.363,"5min":85.879999999999995,"15min":85.879999999999995},"last":1.4219999999999999}]},{"osd":14,"last update":"Wed Sep 18 10:16:22 2024","interfaces":[{"interface":"back","average":{"1min":15.864000000000001,"5min":15.864000000000001,"15min":15.864000000000001},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":145.46700000000001,"5min":145.46700000000001,"15min":145.46700000000001},"last":1.9810000000000001},{"interface":"front","average":{"1min":13.93,"5min":13.93,"15min":13.93},"min":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"max":{"1min":100.092,"5min":100.092,"15min":100.092},"last":1.4870000000000001}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1080000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T10:16:57.510 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-18T10:16:57.810 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-18T10:16:57.810 INFO:teuthology.orchestra.run.smithi029.stdout:{"epoch":152,"fsid":"ab6d6beb-0627-4b7f-8cdb-4e0a44970f10","created":"2024-09-18T10:14:18.827644+0000","modified":"2024-09-18T10:16:57.229173+0000","last_up_change":"2024-09-18T10:14:30.228591+0000","last_in_change":"2024-09-18T10:15:49.554826+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-18T10:14:31.266113+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-18T10:14:33.358485+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":29,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"39cf0511-0009-433f-92f2-e5686f33d713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":22197},{"type":"v1","addr":"172.21.15.29:6809","nonce":22197}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":22197},{"type":"v1","addr":"172.21.15.29:6811","nonce":22197}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":22197},{"type":"v1","addr":"172.21.15.29:6815","nonce":22197}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":22197},{"type":"v1","addr":"172.21.15.29:6813","nonce":22197}]},"public_addr":"172.21.15.29:6809/22197","cluster_addr":"172.21.15.29:6811/22197","heartbeat_back_addr":"172.21.15.29:6815/22197","heartbeat_front_addr":"172.21.15.29:6813/22197","state":["exists","up"]},{"osd":1,"uuid":"a0bf06ab-b98e-4130-91d2-7d4283540c08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":22140},{"type":"v1","addr":"172.21.15.136:6809","nonce":22140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":22140},{"type":"v1","addr":"172.21.15.136:6811","nonce":22140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":22140},{"type":"v1","addr":"172.21.15.136:6815","nonce":22140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":22140},{"type":"v1","addr":"172.21.15.136:6813","nonce":22140}]},"public_addr":"172.21.15.136:6809/22140","cluster_addr":"172.21.15.136:6811/22140","heartbeat_back_addr":"172.21.15.136:6815/22140","heartbeat_front_addr":"172.21.15.136:6813/22140","state":["exists","up"]},{"osd":2,"uuid":"742a786e-6129-4fbc-bb5b-e2e3c03c2d77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":22095},{"type":"v1","addr":"172.21.15.154:6825","nonce":22095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":22095},{"type":"v1","addr":"172.21.15.154:6827","nonce":22095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":22095},{"type":"v1","addr":"172.21.15.154:6831","nonce":22095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":22095},{"type":"v1","addr":"172.21.15.154:6829","nonce":22095}]},"public_addr":"172.21.15.154:6825/22095","cluster_addr":"172.21.15.154:6827/22095","heartbeat_back_addr":"172.21.15.154:6831/22095","heartbeat_front_addr":"172.21.15.154:6829/22095","state":["exists","up"]},{"osd":3,"uuid":"3aa77114-24ed-49ab-86ff-a0600794cbf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":22404},{"type":"v1","addr":"172.21.15.189:6803","nonce":22404}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":22404},{"type":"v1","addr":"172.21.15.189:6805","nonce":22404}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":22404},{"type":"v1","addr":"172.21.15.189:6809","nonce":22404}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":22404},{"type":"v1","addr":"172.21.15.189:6807","nonce":22404}]},"public_addr":"172.21.15.189:6803/22404","cluster_addr":"172.21.15.189:6805/22404","heartbeat_back_addr":"172.21.15.189:6809/22404","heartbeat_front_addr":"172.21.15.189:6807/22404","state":["exists","up"]},{"osd":4,"uuid":"426567ad-b0be-41d4-afcf-dc56ec3974cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":22195},{"type":"v1","addr":"172.21.15.29:6801","nonce":22195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":22195},{"type":"v1","addr":"172.21.15.29:6803","nonce":22195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":22195},{"type":"v1","addr":"172.21.15.29:6807","nonce":22195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":22195},{"type":"v1","addr":"172.21.15.29:6805","nonce":22195}]},"public_addr":"172.21.15.29:6801/22195","cluster_addr":"172.21.15.29:6803/22195","heartbeat_back_addr":"172.21.15.29:6807/22195","heartbeat_front_addr":"172.21.15.29:6805/22195","state":["exists","up"]},{"osd":5,"uuid":"d9559e42-1f4e-4373-8392-a536a0afd871","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6817","nonce":22141},{"type":"v1","addr":"172.21.15.136:6819","nonce":22141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6821","nonce":22141},{"type":"v1","addr":"172.21.15.136:6823","nonce":22141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6829","nonce":22141},{"type":"v1","addr":"172.21.15.136:6831","nonce":22141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6825","nonce":22141},{"type":"v1","addr":"172.21.15.136:6827","nonce":22141}]},"public_addr":"172.21.15.136:6819/22141","cluster_addr":"172.21.15.136:6823/22141","heartbeat_back_addr":"172.21.15.136:6831/22141","heartbeat_front_addr":"172.21.15.136:6827/22141","state":["exists","up"]},{"osd":6,"uuid":"d50ef1e2-8bc5-4a90-9be3-365a0db2be6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":22096},{"type":"v1","addr":"172.21.15.154:6801","nonce":22096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":22096},{"type":"v1","addr":"172.21.15.154:6803","nonce":22096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":22096},{"type":"v1","addr":"172.21.15.154:6807","nonce":22096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":22096},{"type":"v1","addr":"172.21.15.154:6805","nonce":22096}]},"public_addr":"172.21.15.154:6801/22096","cluster_addr":"172.21.15.154:6803/22096","heartbeat_back_addr":"172.21.15.154:6807/22096","heartbeat_front_addr":"172.21.15.154:6805/22096","state":["exists","up"]},{"osd":7,"uuid":"bd0b85e4-cfee-4d19-90a5-79854794b48b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":22405},{"type":"v1","addr":"172.21.15.189:6819","nonce":22405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":22405},{"type":"v1","addr":"172.21.15.189:6821","nonce":22405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":22405},{"type":"v1","addr":"172.21.15.189:6825","nonce":22405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":22405},{"type":"v1","addr":"172.21.15.189:6823","nonce":22405}]},"public_addr":"172.21.15.189:6819/22405","cluster_addr":"172.21.15.189:6821/22405","heartbeat_back_addr":"172.21.15.189:6825/22405","heartbeat_front_addr":"172.21.15.189:6823/22405","state":["exists","up"]},{"osd":8,"uuid":"cd333f56-c165-45d7-8c8b-48d34bc4ddc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":22200},{"type":"v1","addr":"172.21.15.29:6825","nonce":22200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":22200},{"type":"v1","addr":"172.21.15.29:6827","nonce":22200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":22200},{"type":"v1","addr":"172.21.15.29:6831","nonce":22200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":22200},{"type":"v1","addr":"172.21.15.29:6829","nonce":22200}]},"public_addr":"172.21.15.29:6825/22200","cluster_addr":"172.21.15.29:6827/22200","heartbeat_back_addr":"172.21.15.29:6831/22200","heartbeat_front_addr":"172.21.15.29:6829/22200","state":["exists","up"]},{"osd":9,"uuid":"1e079e4c-8801-4754-9d0d-d0b68695d32e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":22142},{"type":"v1","addr":"172.21.15.136:6801","nonce":22142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":22142},{"type":"v1","addr":"172.21.15.136:6803","nonce":22142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":22142},{"type":"v1","addr":"172.21.15.136:6807","nonce":22142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":22142},{"type":"v1","addr":"172.21.15.136:6805","nonce":22142}]},"public_addr":"172.21.15.136:6801/22142","cluster_addr":"172.21.15.136:6803/22142","heartbeat_back_addr":"172.21.15.136:6807/22142","heartbeat_front_addr":"172.21.15.136:6805/22142","state":["exists","up"]},{"osd":10,"uuid":"92ee68d4-371c-4849-930f-a8fa18d64b99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":22097},{"type":"v1","addr":"172.21.15.154:6809","nonce":22097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":22097},{"type":"v1","addr":"172.21.15.154:6811","nonce":22097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":22097},{"type":"v1","addr":"172.21.15.154:6815","nonce":22097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":22097},{"type":"v1","addr":"172.21.15.154:6813","nonce":22097}]},"public_addr":"172.21.15.154:6809/22097","cluster_addr":"172.21.15.154:6811/22097","heartbeat_back_addr":"172.21.15.154:6815/22097","heartbeat_front_addr":"172.21.15.154:6813/22097","state":["exists","up"]},{"osd":11,"uuid":"634ddaf5-f7b8-4b27-a46a-04e21b73fcf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":22416},{"type":"v1","addr":"172.21.15.189:6827","nonce":22416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":22416},{"type":"v1","addr":"172.21.15.189:6829","nonce":22416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":22416},{"type":"v1","addr":"172.21.15.189:6833","nonce":22416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":22416},{"type":"v1","addr":"172.21.15.189:6831","nonce":22416}]},"public_addr":"172.21.15.189:6827/22416","cluster_addr":"172.21.15.189:6829/22416","heartbeat_back_addr":"172.21.15.189:6833/22416","heartbeat_front_addr":"172.21.15.189:6831/22416","state":["exists","up"]},{"osd":12,"uuid":"bc58137b-5948-400d-b377-193b15824d8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":22210},{"type":"v1","addr":"172.21.15.29:6817","nonce":22210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":22210},{"type":"v1","addr":"172.21.15.29:6819","nonce":22210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":22210},{"type":"v1","addr":"172.21.15.29:6823","nonce":22210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":22210},{"type":"v1","addr":"172.21.15.29:6821","nonce":22210}]},"public_addr":"172.21.15.29:6817/22210","cluster_addr":"172.21.15.29:6819/22210","heartbeat_back_addr":"172.21.15.29:6823/22210","heartbeat_front_addr":"172.21.15.29:6821/22210","state":["exists","up"]},{"osd":13,"uuid":"a8b753a7-ab69-4853-ac36-66d9b85c2e20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":22143},{"type":"v1","addr":"172.21.15.136:6818","nonce":22143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":22143},{"type":"v1","addr":"172.21.15.136:6822","nonce":22143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":22143},{"type":"v1","addr":"172.21.15.136:6830","nonce":22143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":22143},{"type":"v1","addr":"172.21.15.136:6826","nonce":22143}]},"public_addr":"172.21.15.136:6818/22143","cluster_addr":"172.21.15.136:6822/22143","heartbeat_back_addr":"172.21.15.136:6830/22143","heartbeat_front_addr":"172.21.15.136:6826/22143","state":["exists","up"]},{"osd":14,"uuid":"8ad0848f-0de4-4a75-9c63-8c4a4a7b8524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":22098},{"type":"v1","addr":"172.21.15.154:6817","nonce":22098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":22098},{"type":"v1","addr":"172.21.15.154:6819","nonce":22098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":22098},{"type":"v1","addr":"172.21.15.154:6823","nonce":22098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":22098},{"type":"v1","addr":"172.21.15.154:6821","nonce":22098}]},"public_addr":"172.21.15.154:6817/22098","cluster_addr":"172.21.15.154:6819/22098","heartbeat_back_addr":"172.21.15.154:6823/22098","heartbeat_front_addr":"172.21.15.154:6821/22098","state":["exists","up"]},{"osd":15,"uuid":"069099b3-ec1a-483b-b967-98cddbf2f678","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":22415},{"type":"v1","addr":"172.21.15.189:6811","nonce":22415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":22415},{"type":"v1","addr":"172.21.15.189:6813","nonce":22415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":22415},{"type":"v1","addr":"172.21.15.189:6817","nonce":22415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":22415},{"type":"v1","addr":"172.21.15.189:6815","nonce":22415}]},"public_addr":"172.21.15.189:6811/22415","cluster_addr":"172.21.15.189:6813/22415","heartbeat_back_addr":"172.21.15.189:6817/22415","heartbeat_front_addr":"172.21.15.189:6815/22415","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:15:56.126704+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:15:32.497046+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:15:55.189067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:15:44.298040+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:15:56.991035+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:15:51.718897+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:16:56.237375+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:16:42.140104+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:15:42.972647+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:15:36.896626+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:15:54.542105+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:15:47.318375+0000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:16:37.858675+0000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:16:51.609110+0000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:15:52.801215+0000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:15:59.018997+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"},"shecprofile":{"c":"2","crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","k":"4","m":"3","name":"shecprofile","plugin":"shec","technique":"multiple"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-18T10:16:58.825 INFO:tasks.ceph:Scrubbing osd.0 2024-09-18T10:16:58.825 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-18T10:16:58.983 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-18T10:16:58.983 INFO:teuthology.orchestra.run.smithi029.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-18T10:16:58.983 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-18T10:16:58.997 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-18T10:16:59.304 INFO:teuthology.orchestra.run.smithi029.stderr:instructed osd(s) 0 to deep-scrub 2024-09-18T10:16:59.315 INFO:tasks.ceph:Scrubbing osd.4 2024-09-18T10:16:59.315 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-18T10:16:59.462 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-18T10:16:59.462 INFO:teuthology.orchestra.run.smithi029.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-18T10:16:59.462 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-18T10:16:59.475 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-18T10:16:59.778 INFO:teuthology.orchestra.run.smithi029.stderr:instructed osd(s) 4 to deep-scrub 2024-09-18T10:16:59.788 INFO:tasks.ceph:Scrubbing osd.8 2024-09-18T10:16:59.789 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.8 config set osd_debug_deep_scrub_sleep 0 2024-09-18T10:16:59.930 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-18T10:16:59.930 INFO:teuthology.orchestra.run.smithi029.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-18T10:16:59.930 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-18T10:16:59.943 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 8 2024-09-18T10:17:00.238 INFO:teuthology.orchestra.run.smithi029.stderr:instructed osd(s) 8 to deep-scrub 2024-09-18T10:17:00.248 INFO:tasks.ceph:Scrubbing osd.12 2024-09-18T10:17:00.249 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.12 config set osd_debug_deep_scrub_sleep 0 2024-09-18T10:17:00.395 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-18T10:17:00.395 INFO:teuthology.orchestra.run.smithi029.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-18T10:17:00.395 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-18T10:17:00.407 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 12 2024-09-18T10:17:00.715 INFO:teuthology.orchestra.run.smithi029.stderr:instructed osd(s) 12 to deep-scrub 2024-09-18T10:17:00.726 INFO:tasks.ceph:Scrubbing osd.1 2024-09-18T10:17:00.726 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-18T10:17:00.872 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-18T10:17:00.872 INFO:teuthology.orchestra.run.smithi029.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-18T10:17:00.872 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-18T10:17:00.886 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-18T10:17:01.185 INFO:teuthology.orchestra.run.smithi029.stderr:instructed osd(s) 1 to deep-scrub 2024-09-18T10:17:01.195 INFO:tasks.ceph:Scrubbing osd.5 2024-09-18T10:17:01.195 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-18T10:17:01.349 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-18T10:17:01.349 INFO:teuthology.orchestra.run.smithi029.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-18T10:17:01.349 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-18T10:17:01.362 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-18T10:17:01.672 INFO:teuthology.orchestra.run.smithi029.stderr:instructed osd(s) 5 to deep-scrub 2024-09-18T10:17:01.683 INFO:tasks.ceph:Scrubbing osd.9 2024-09-18T10:17:01.683 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.9 config set osd_debug_deep_scrub_sleep 0 2024-09-18T10:17:01.831 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-18T10:17:01.831 INFO:teuthology.orchestra.run.smithi029.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-18T10:17:01.831 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-18T10:17:01.844 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 9 2024-09-18T10:17:02.148 INFO:teuthology.orchestra.run.smithi029.stderr:instructed osd(s) 9 to deep-scrub 2024-09-18T10:17:02.166 INFO:tasks.ceph:Scrubbing osd.13 2024-09-18T10:17:02.166 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.13 config set osd_debug_deep_scrub_sleep 0 2024-09-18T10:17:02.317 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-18T10:17:02.317 INFO:teuthology.orchestra.run.smithi029.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-18T10:17:02.317 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-18T10:17:02.330 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 13 2024-09-18T10:17:02.629 INFO:teuthology.orchestra.run.smithi029.stderr:instructed osd(s) 13 to deep-scrub 2024-09-18T10:17:02.641 INFO:tasks.ceph:Scrubbing osd.2 2024-09-18T10:17:02.642 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-18T10:17:02.779 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-18T10:17:02.780 INFO:teuthology.orchestra.run.smithi029.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-18T10:17:02.780 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-18T10:17:02.792 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-18T10:17:03.093 INFO:teuthology.orchestra.run.smithi029.stderr:instructed osd(s) 2 to deep-scrub 2024-09-18T10:17:03.103 INFO:tasks.ceph:Scrubbing osd.6 2024-09-18T10:17:03.104 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-09-18T10:17:03.254 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-18T10:17:03.254 INFO:teuthology.orchestra.run.smithi029.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-18T10:17:03.254 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-18T10:17:03.267 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-09-18T10:17:03.572 INFO:teuthology.orchestra.run.smithi029.stderr:instructed osd(s) 6 to deep-scrub 2024-09-18T10:17:03.582 INFO:tasks.ceph:Scrubbing osd.10 2024-09-18T10:17:03.582 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.10 config set osd_debug_deep_scrub_sleep 0 2024-09-18T10:17:03.733 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-18T10:17:03.733 INFO:teuthology.orchestra.run.smithi029.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-18T10:17:03.734 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-18T10:17:03.746 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 10 2024-09-18T10:17:04.047 INFO:teuthology.orchestra.run.smithi029.stderr:instructed osd(s) 10 to deep-scrub 2024-09-18T10:17:04.058 INFO:tasks.ceph:Scrubbing osd.14 2024-09-18T10:17:04.058 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.14 config set osd_debug_deep_scrub_sleep 0 2024-09-18T10:17:04.209 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-18T10:17:04.209 INFO:teuthology.orchestra.run.smithi029.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-18T10:17:04.209 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-18T10:17:04.222 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 14 2024-09-18T10:17:04.523 INFO:teuthology.orchestra.run.smithi029.stderr:instructed osd(s) 14 to deep-scrub 2024-09-18T10:17:04.534 INFO:tasks.ceph:Scrubbing osd.3 2024-09-18T10:17:04.534 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-18T10:17:04.682 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-18T10:17:04.682 INFO:teuthology.orchestra.run.smithi029.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-18T10:17:04.682 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-18T10:17:04.695 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-18T10:17:04.994 INFO:teuthology.orchestra.run.smithi029.stderr:instructed osd(s) 3 to deep-scrub 2024-09-18T10:17:05.005 INFO:tasks.ceph:Scrubbing osd.7 2024-09-18T10:17:05.005 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-09-18T10:17:05.155 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-18T10:17:05.155 INFO:teuthology.orchestra.run.smithi029.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-18T10:17:05.155 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-18T10:17:05.168 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-09-18T10:17:05.473 INFO:teuthology.orchestra.run.smithi029.stderr:instructed osd(s) 7 to deep-scrub 2024-09-18T10:17:05.483 INFO:tasks.ceph:Scrubbing osd.11 2024-09-18T10:17:05.484 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.11 config set osd_debug_deep_scrub_sleep 0 2024-09-18T10:17:05.633 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-18T10:17:05.634 INFO:teuthology.orchestra.run.smithi029.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-18T10:17:05.634 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-18T10:17:05.646 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 11 2024-09-18T10:17:05.944 INFO:teuthology.orchestra.run.smithi029.stderr:instructed osd(s) 11 to deep-scrub 2024-09-18T10:17:05.954 INFO:tasks.ceph:Scrubbing osd.15 2024-09-18T10:17:05.955 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.15 config set osd_debug_deep_scrub_sleep 0 2024-09-18T10:17:06.104 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2024-09-18T10:17:06.104 INFO:teuthology.orchestra.run.smithi029.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-18T10:17:06.104 INFO:teuthology.orchestra.run.smithi029.stdout:} 2024-09-18T10:17:06.120 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 15 2024-09-18T10:17:06.437 INFO:teuthology.orchestra.run.smithi029.stderr:instructed osd(s) 15 to deep-scrub 2024-09-18T10:17:06.447 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T10:17:06.751 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-18T10:17:06.754 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-09-18T10:17:06.765 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":221,"stamp":"2024-09-18T10:17:06.300841+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":147,"num_write_kb":2674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":14,"kb":1509949440,"kb_used":4866948,"kb_used_data":38488,"kb_used_omap":26,"kb_used_meta":4828197,"kb_avail":1505082492,"statfs":{"total":1546188226560,"available":1541204471808,"internally_reserved":0,"allocated":39411712,"data_stored":23391124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":27264,"internal_metadata":4944074112},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.036543"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":147,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-18T10:16:58.392574+0000","last_change":"2024-09-18T10:16:18.085265+0000","last_active":"2024-09-18T10:16:58.392574+0000","last_peered":"2024-09-18T10:16:58.392574+0000","last_clean":"2024-09-18T10:16:58.392574+0000","last_became_active":"2024-09-18T10:14:35.348943+0000","last_became_peered":"2024-09-18T10:14:35.348943+0000","last_unstale":"2024-09-18T10:16:58.392574+0000","last_undegraded":"2024-09-18T10:16:58.392574+0000","last_fullsized":"2024-09-18T10:16:58.392574+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:16:18.085027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:16:18.085027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:17:43.814969+0000","scrub_duration":0.30385945199999997,"objects_trimmed":0,"snaptrim_duration":0.00060442999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,10],"acting":[13,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":93,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-18T10:17:02.275190+0000","last_change":"2024-09-18T10:17:00.624044+0000","last_active":"2024-09-18T10:17:02.275190+0000","last_peered":"2024-09-18T10:17:02.275190+0000","last_clean":"2024-09-18T10:17:02.275190+0000","last_became_active":"2024-09-18T10:15:50.978055+0000","last_became_peered":"2024-09-18T10:15:50.978055+0000","last_unstale":"2024-09-18T10:17:02.275190+0000","last_undegraded":"2024-09-18T10:17:02.275190+0000","last_fullsized":"2024-09-18T10:17:02.275190+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:17:00.623856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:17:00.623856+0000","last_clean_scrub_stamp":"2024-09-18T10:17:00.623856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:18:11.013972+0000","scrub_duration":0.003444309,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11],"acting":[8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":145,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-18T10:17:02.731003+0000","last_change":"2024-09-18T10:15:50.883382+0000","last_active":"2024-09-18T10:17:02.731003+0000","last_peered":"2024-09-18T10:17:02.731003+0000","last_clean":"2024-09-18T10:17:02.731003+0000","last_became_active":"2024-09-18T10:14:35.329198+0000","last_became_peered":"2024-09-18T10:14:35.329198+0000","last_unstale":"2024-09-18T10:17:02.731003+0000","last_undegraded":"2024-09-18T10:17:02.731003+0000","last_fullsized":"2024-09-18T10:17:02.731003+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:15:50.883147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:15:50.883147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:17:15.846467+0000","scrub_duration":0.153222372,"objects_trimmed":0,"snaptrim_duration":0.00062415699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":160,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-18T10:17:03.382254+0000","last_change":"2024-09-18T10:17:00.877328+0000","last_active":"2024-09-18T10:17:03.382254+0000","last_peered":"2024-09-18T10:17:03.382254+0000","last_clean":"2024-09-18T10:17:03.382254+0000","last_became_active":"2024-09-18T10:14:35.524102+0000","last_became_peered":"2024-09-18T10:14:35.524102+0000","last_unstale":"2024-09-18T10:17:03.382254+0000","last_undegraded":"2024-09-18T10:17:03.382254+0000","last_fullsized":"2024-09-18T10:17:03.382254+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:17:00.877142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:17:00.877142+0000","last_clean_scrub_stamp":"2024-09-18T10:17:00.877142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:18:19.548036+0000","scrub_duration":0.30330662200000003,"objects_trimmed":0,"snaptrim_duration":0.10077985,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"29'2","reported_seq":148,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-18T10:16:58.292451+0000","last_change":"2024-09-18T10:16:20.864837+0000","last_active":"2024-09-18T10:16:58.292451+0000","last_peered":"2024-09-18T10:16:58.292451+0000","last_clean":"2024-09-18T10:16:58.292451+0000","last_became_active":"2024-09-18T10:14:35.715374+0000","last_became_peered":"2024-09-18T10:14:35.715374+0000","last_unstale":"2024-09-18T10:16:58.292451+0000","last_undegraded":"2024-09-18T10:16:58.292451+0000","last_fullsized":"2024-09-18T10:16:58.292451+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2","last_scrub_stamp":"2024-09-18T10:16:20.864635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:16:20.864635+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:17:23.226052+0000","scrub_duration":0.20463374300000001,"objects_trimmed":0,"snaptrim_duration":0.002421224,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,12],"acting":[13,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":158,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-18T10:17:04.546732+0000","last_change":"2024-09-18T10:16:15.062108+0000","last_active":"2024-09-18T10:17:04.546732+0000","last_peered":"2024-09-18T10:17:04.546732+0000","last_clean":"2024-09-18T10:17:04.546732+0000","last_became_active":"2024-09-18T10:14:35.348262+0000","last_became_peered":"2024-09-18T10:14:35.348262+0000","last_unstale":"2024-09-18T10:17:04.546732+0000","last_undegraded":"2024-09-18T10:17:04.546732+0000","last_fullsized":"2024-09-18T10:17:04.546732+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:16:15.061892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:16:15.061892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for deep scrub","scrub_duration":0.40410323300000001,"objects_trimmed":0,"snaptrim_duration":0.00063430400000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,0],"acting":[14,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":159,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-18T10:17:03.091978+0000","last_change":"2024-09-18T10:17:03.091916+0000","last_active":"2024-09-18T10:17:03.091978+0000","last_peered":"2024-09-18T10:17:03.091978+0000","last_clean":"2024-09-18T10:17:03.091978+0000","last_became_active":"2024-09-18T10:14:35.349203+0000","last_became_peered":"2024-09-18T10:14:35.349203+0000","last_unstale":"2024-09-18T10:17:03.091978+0000","last_undegraded":"2024-09-18T10:17:03.091978+0000","last_fullsized":"2024-09-18T10:17:03.091978+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:17:03.091716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:17:03.091716+0000","last_clean_scrub_stamp":"2024-09-18T10:17:03.091716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:18:13.646830+0000","scrub_duration":0.203921877,"objects_trimmed":0,"snaptrim_duration":0.00060130600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3],"acting":[9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"27'1","reported_seq":153,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-18T10:17:04.646830+0000","last_change":"2024-09-18T10:16:06.900858+0000","last_active":"2024-09-18T10:17:04.646830+0000","last_peered":"2024-09-18T10:17:04.646830+0000","last_clean":"2024-09-18T10:17:04.646830+0000","last_became_active":"2024-09-18T10:14:35.487746+0000","last_became_peered":"2024-09-18T10:14:35.487746+0000","last_unstale":"2024-09-18T10:17:04.646830+0000","last_undegraded":"2024-09-18T10:17:04.646830+0000","last_fullsized":"2024-09-18T10:17:04.646830+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'1","last_scrub_stamp":"2024-09-18T10:16:06.900601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:14:34.288596+0000","last_clean_scrub_stamp":"2024-09-18T10:16:06.900601+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for deep scrub","scrub_duration":0.27078197700000001,"objects_trimmed":0,"snaptrim_duration":0.10072764100000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,5],"acting":[14,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"1.0","version":"156'114","reported_seq":295,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-18T10:17:04.855639+0000","last_change":"2024-09-18T10:17:04.855527+0000","last_active":"2024-09-18T10:17:04.855639+0000","last_peered":"2024-09-18T10:17:04.855639+0000","last_clean":"2024-09-18T10:17:04.855639+0000","last_became_active":"2024-09-18T10:14:32.508037+0000","last_became_peered":"2024-09-18T10:14:32.508037+0000","last_unstale":"2024-09-18T10:17:04.855639+0000","last_undegraded":"2024-09-18T10:17:04.855639+0000","last_fullsized":"2024-09-18T10:17:04.855639+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"156'114","last_scrub_stamp":"2024-09-18T10:17:04.855302+0000","last_deep_scrub":"156'114","last_deep_scrub_stamp":"2024-09-18T10:17:04.855302+0000","last_clean_scrub_stamp":"2024-09-18T10:17:04.855302+0000","objects_scrubbed":2,"log_size":114,"log_dups_size":0,"ondisk_log_size":114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:18:08.277089+0000","scrub_duration":0.71041728699999995,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,15],"acting":[6,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":13},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":114,"ondisk_log_size":114,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":15,"up_from":22,"seq":94489280546,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304380,"kb_used_data":2864,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94067460,"statfs":{"total":96636764160,"available":96325079040,"internally_reserved":0,"allocated":2932736,"data_stored":1927137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,4,6,7,8,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:16:32 2024","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":0.83899999999999997,"15min":0.81899999999999995},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":2.0019999999999998,"5min":2.0019999999999998,"15min":2.0019999999999998},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.996,"5min":5.5739999999999998,"15min":6.3369999999999997},"min":{"1min":0.72899999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.506,"5min":101.291,"15min":101.291},"last":0.83999999999999997}]},{"osd":1,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.93300000000000005},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.931,"5min":1.931,"15min":1.931},"last":1.1279999999999999}]},{"osd":2,"last update":"Wed Sep 18 10:16:32 2024","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.0169999999999999,"15min":0.995},"min":{"1min":0.79700000000000004,"5min":0.5,"15min":0.5},"max":{"1min":1.7470000000000001,"5min":1.7470000000000001,"15min":1.7470000000000001},"last":1.028},{"interface":"front","average":{"1min":1.1419999999999999,"5min":4.6589999999999998,"15min":5.2450000000000001},"min":{"1min":0.82899999999999996,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.4379999999999999,"5min":78.242999999999995,"15min":78.242999999999995},"last":1.252}]},{"osd":3,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":3.3029999999999999,"5min":3.3029999999999999,"15min":3.3029999999999999},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":41.720999999999997,"5min":41.720999999999997,"15min":41.720999999999997},"last":0.55400000000000005},{"interface":"front","average":{"1min":1.014,"5min":1.014,"15min":1.014},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.49,"5min":1.49,"15min":1.49},"last":0.89500000000000002}]},{"osd":4,"last update":"Wed Sep 18 10:16:32 2024","interfaces":[{"interface":"back","average":{"1min":1.3580000000000001,"5min":9.0879999999999992,"15min":10.375999999999999},"min":{"1min":0.747,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":8.0449999999999999,"5min":99.745999999999995,"15min":99.745999999999995},"last":0.90000000000000002},{"interface":"front","average":{"1min":1.595,"5min":9.2309999999999999,"15min":10.503},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":8.0679999999999996,"5min":99.781000000000006,"15min":99.781000000000006},"last":1.141}]},{"osd":5,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":1.0860000000000001},{"interface":"front","average":{"1min":1.7869999999999999,"5min":1.7869999999999999,"15min":1.7869999999999999},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":10.874000000000001,"5min":10.874000000000001,"15min":10.874000000000001},"last":1.163}]},{"osd":6,"last update":"Wed Sep 18 10:16:32 2024","interfaces":[{"interface":"back","average":{"1min":14.140000000000001,"5min":4.4039999999999999,"15min":2.782},"min":{"1min":0.92900000000000005,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":269.80500000000001,"5min":269.80500000000001,"15min":269.80500000000001},"last":1.234},{"interface":"front","average":{"1min":1.361,"5min":4.7610000000000001,"15min":5.327},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":2.012,"5min":74.156000000000006,"15min":74.156000000000006},"last":1.071}]},{"osd":7,"last update":"Wed Sep 18 10:16:53 2024","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.304,"15min":1.304},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.9590000000000001,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":0.98799999999999999},{"interface":"front","average":{"1min":4.5819999999999999,"5min":4.5819999999999999,"15min":4.5819999999999999},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":70.611000000000004,"5min":70.611000000000004,"15min":70.611000000000004},"last":1.202}]},{"osd":8,"last update":"Wed Sep 18 10:16:53 2024","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.234,"15min":1.234},"min":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"max":{"1min":1.855,"5min":1.855,"15min":1.855},"last":1.0309999999999999},{"interface":"front","average":{"1min":1.5629999999999999,"5min":1.5629999999999999,"15min":1.5629999999999999},"min":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"max":{"1min":2.2280000000000002,"5min":2.2280000000000002,"15min":2.2280000000000002},"last":1.3200000000000001}]},{"osd":10,"last update":"Wed Sep 18 10:16:32 2024","interfaces":[{"interface":"back","average":{"1min":5.8639999999999999,"5min":6.2670000000000003,"15min":6.3339999999999996},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":93.704999999999998,"5min":93.704999999999998,"15min":93.704999999999998},"last":1.3740000000000001},{"interface":"front","average":{"1min":5.7770000000000001,"5min":6.2549999999999999,"15min":6.335},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":93.671000000000006,"5min":93.671000000000006,"15min":93.671000000000006},"last":1.2709999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5209999999999999}]},{"osd":12,"last update":"Wed Sep 18 10:16:53 2024","interfaces":[{"interface":"back","average":{"1min":1.6060000000000001,"5min":1.6060000000000001,"15min":1.6060000000000001},"min":{"1min":1.077,"5min":1.077,"15min":1.077},"max":{"1min":2.145,"5min":2.145,"15min":2.145},"last":1.677},{"interface":"front","average":{"1min":1.577,"5min":1.577,"15min":1.577},"min":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"max":{"1min":2.1819999999999999,"5min":2.1819999999999999,"15min":2.1819999999999999},"last":1.349}]},{"osd":13,"last update":"Wed Sep 18 10:16:32 2024","interfaces":[{"interface":"back","average":{"1min":1.99,"5min":5.2629999999999999,"15min":5.8090000000000002},"min":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"max":{"1min":7.3179999999999996,"5min":80.197000000000003,"15min":80.197000000000003},"last":1.4890000000000001},{"interface":"front","average":{"1min":3.4830000000000001,"5min":1.8080000000000001,"15min":1.5289999999999999},"min":{"1min":0.89400000000000002,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":25.193999999999999,"5min":25.193999999999999,"15min":25.193999999999999},"last":1.774}]},{"osd":14,"last update":"Wed Sep 18 10:16:32 2024","interfaces":[{"interface":"back","average":{"1min":10.725,"5min":3.2999999999999998,"15min":2.0619999999999998},"min":{"1min":0.95699999999999996,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":117.562,"5min":117.562,"15min":117.562},"last":1.556},{"interface":"front","average":{"1min":4.6139999999999999,"5min":9.5229999999999997,"15min":10.340999999999999},"min":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"max":{"1min":61.567,"5min":85.006,"15min":85.006},"last":1.4279999999999999}]}]},{"osd":14,"up_from":22,"seq":94489280546,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303880,"kb_used_data":2360,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94067960,"statfs":{"total":96636764160,"available":96325591040,"internally_reserved":0,"allocated":2416640,"data_stored":1407725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,5,7,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:16:32 2024","interfaces":[{"interface":"back","average":{"1min":1.0009999999999999,"5min":12.513,"15min":14.430999999999999},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.405,"5min":189.88800000000001,"15min":189.88800000000001},"last":0.68999999999999995},{"interface":"front","average":{"1min":7.0330000000000004,"5min":17.033000000000001,"15min":18.699000000000002},"min":{"1min":0.77300000000000002,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":71.900999999999996,"5min":180.09999999999999,"15min":180.09999999999999},"last":0.76500000000000001}]},{"osd":1,"last update":"Wed Sep 18 10:16:32 2024","interfaces":[{"interface":"back","average":{"1min":2.1040000000000001,"5min":7.7670000000000003,"15min":8.7110000000000003},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":21.334,"5min":79.626000000000005,"15min":79.626000000000005},"last":1.02},{"interface":"front","average":{"1min":1.1479999999999999,"5min":12.601000000000001,"15min":14.51},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.8879999999999999,"5min":189.762,"15min":189.762},"last":0.90100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.417}]},{"osd":3,"last update":"Wed Sep 18 10:16:32 2024","interfaces":[{"interface":"back","average":{"1min":3.5430000000000001,"5min":12.906000000000001,"15min":14.465999999999999},"min":{"1min":0.84799999999999998,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":44.683,"5min":180.19900000000001,"15min":180.19900000000001},"last":1.7529999999999999},{"interface":"front","average":{"1min":5.3520000000000003,"5min":18.143999999999998,"15min":20.276},"min":{"1min":0.86399999999999999,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":42.570999999999998,"5min":180.851,"15min":180.851},"last":1.1899999999999999}]},{"osd":4,"last update":"Wed Sep 18 10:16:32 2024","interfaces":[{"interface":"back","average":{"1min":6.8360000000000003,"5min":21.687000000000001,"15min":24.161999999999999},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":71.911000000000001,"5min":189.809,"15min":189.809},"last":2.0249999999999999},{"interface":"front","average":{"1min":14.032999999999999,"5min":15.031000000000001,"15min":15.196999999999999},"min":{"1min":0.76100000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":133.244,"5min":180.31700000000001,"15min":180.31700000000001},"last":1.9350000000000001}]},{"osd":5,"last update":"Wed Sep 18 10:16:32 2024","interfaces":[{"interface":"back","average":{"1min":1.831,"5min":21.507000000000001,"15min":24.786000000000001},"min":{"1min":0.66300000000000003,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":13.42,"5min":277.41199999999998,"15min":277.41199999999998},"last":0.85599999999999998},{"interface":"front","average":{"1min":1.9530000000000001,"5min":12.936999999999999,"15min":14.766999999999999},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":14.288,"5min":189.64699999999999,"15min":189.64699999999999},"last":0.95699999999999996}]},{"osd":7,"last update":"Wed Sep 18 10:16:32 2024","interfaces":[{"interface":"back","average":{"1min":6.29,"5min":13.625999999999999,"15min":14.848000000000001},"min":{"1min":0.995,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":49.378,"5min":180.268,"15min":180.268},"last":1.3620000000000001},{"interface":"front","average":{"1min":3.8929999999999998,"5min":12.901,"15min":14.403},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":49.402999999999999,"5min":180.631,"15min":180.631},"last":1.121}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1619999999999999}]},{"osd":9,"last update":"Wed Sep 18 10:16:17 2024","interfaces":[{"interface":"back","average":{"1min":4,"5min":4,"15min":4},"min":{"1min":1.296,"5min":1.296,"15min":1.296},"max":{"1min":44.716000000000001,"5min":44.716000000000001,"15min":44.716000000000001},"last":1.4450000000000001},{"interface":"front","average":{"1min":1.427,"5min":1.427,"15min":1.427},"min":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"max":{"1min":1.9610000000000001,"5min":1.9610000000000001,"15min":1.9610000000000001},"last":1.331}]},{"osd":10,"last update":"Wed Sep 18 10:16:52 2024","interfaces":[{"interface":"back","average":{"1min":1.5549999999999999,"5min":13.942,"15min":16.006},"min":{"1min":0.67000000000000004,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":2.2850000000000001,"5min":180.15899999999999,"15min":180.15899999999999},"last":1.1679999999999999},{"interface":"front","average":{"1min":1.75,"5min":25.152999999999999,"15min":29.053000000000001},"min":{"1min":1.1899999999999999,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":2.4420000000000002,"5min":269.423,"15min":269.423},"last":1.5389999999999999}]},{"osd":11,"last update":"Wed Sep 18 10:16:10 2024","interfaces":[{"interface":"back","average":{"1min":9.6890000000000001,"5min":9.6890000000000001,"15min":9.6890000000000001},"min":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":91.111999999999995,"5min":91.111999999999995,"15min":91.111999999999995},"last":1.399},{"interface":"front","average":{"1min":8.1460000000000008,"5min":8.1460000000000008,"15min":8.1460000000000008},"min":{"1min":1.181,"5min":1.181,"15min":1.181},"max":{"1min":91.165000000000006,"5min":91.165000000000006,"15min":91.165000000000006},"last":1.145}]},{"osd":12,"last update":"Wed Sep 18 10:16:32 2024","interfaces":[{"interface":"back","average":{"1min":10.385,"5min":14.760999999999999,"15min":15.49},"min":{"1min":1.327,"5min":0.91100000000000003,"15min":0.91100000000000003},"max":{"1min":65.189999999999998,"5min":189.84999999999999,"15min":189.84999999999999},"last":1.3049999999999999},{"interface":"front","average":{"1min":10.818,"5min":14.696,"15min":15.342000000000001},"min":{"1min":1.4890000000000001,"5min":0.91200000000000003,"15min":0.91200000000000003},"max":{"1min":65.162000000000006,"5min":180.381,"15min":180.381},"last":1.6819999999999999}]},{"osd":13,"last update":"Wed Sep 18 10:16:32 2024","interfaces":[{"interface":"back","average":{"1min":10.321999999999999,"5min":23.059000000000001,"15min":25.181999999999999},"min":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"max":{"1min":166.001,"5min":180.59100000000001,"15min":180.59100000000001},"last":1.476},{"interface":"front","average":{"1min":1.647,"5min":20.391999999999999,"15min":23.515999999999998},"min":{"1min":0.83399999999999996,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":2.3940000000000001,"5min":189.71299999999999,"15min":189.71299999999999},"last":1.089}]},{"osd":15,"last update":"Wed Sep 18 10:16:32 2024","interfaces":[{"interface":"back","average":{"1min":4.181,"5min":13.381,"15min":14.914},"min":{"1min":1.5780000000000001,"5min":0.99399999999999999,"15min":0.99399999999999999},"max":{"1min":44.917000000000002,"5min":180.44800000000001,"15min":180.44800000000001},"last":1.615},{"interface":"front","average":{"1min":1.7030000000000001,"5min":15.118,"15min":17.353999999999999},"min":{"1min":0.93200000000000005,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":2.1909999999999998,"5min":180.53,"15min":180.53},"last":1.571}]}]},{"osd":13,"up_from":22,"seq":94489280546,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307980,"kb_used_data":2300,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94063860,"statfs":{"total":96636764160,"available":96321392640,"internally_reserved":0,"allocated":2355200,"data_stored":1369630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,3,6,8,9,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:16:33 2024","interfaces":[{"interface":"back","average":{"1min":2.2599999999999998,"5min":17.045000000000002,"15min":19.509},"min":{"1min":0.65700000000000003,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":22.972999999999999,"5min":176.66800000000001,"15min":176.66800000000001},"last":0.78800000000000003},{"interface":"front","average":{"1min":4.2629999999999999,"5min":25.199000000000002,"15min":28.687999999999999},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":62.533000000000001,"5min":199.732,"15min":199.732},"last":0.84699999999999998}]},{"osd":1,"last update":"Wed Sep 18 10:15:48 2024","interfaces":[{"interface":"back","average":{"1min":1.986,"5min":1.986,"15min":1.986},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":18.471,"5min":18.471,"15min":18.471},"last":1.4219999999999999},{"interface":"front","average":{"1min":20.716999999999999,"5min":20.716999999999999,"15min":20.716999999999999},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":176.56800000000001,"5min":176.56800000000001,"15min":176.56800000000001},"last":0.86399999999999999}]},{"osd":2,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":11.984,"5min":11.984,"15min":11.984},"min":{"1min":0.751,"5min":0.751,"15min":0.751},"max":{"1min":196.59200000000001,"5min":196.59200000000001,"15min":196.59200000000001},"last":0.97699999999999998},{"interface":"front","average":{"1min":26.109999999999999,"5min":26.109999999999999,"15min":26.109999999999999},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":177.03,"5min":177.03,"15min":177.03},"last":1.2689999999999999}]},{"osd":3,"last update":"Wed Sep 18 10:16:33 2024","interfaces":[{"interface":"back","average":{"1min":9.5,"5min":22.245000000000001,"15min":24.369},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":158.74100000000001,"5min":168.27699999999999,"15min":168.27699999999999},"last":1.2310000000000001},{"interface":"front","average":{"1min":1.2290000000000001,"5min":9.8819999999999997,"15min":11.324},"min":{"1min":0.84499999999999997,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.8959999999999999,"5min":196.643,"15min":196.643},"last":0.92200000000000004}]},{"osd":4,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":29.802,"5min":29.802,"15min":29.802},"min":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"max":{"1min":168.48099999999999,"5min":168.48099999999999,"15min":168.48099999999999},"last":1.347},{"interface":"front","average":{"1min":8.1780000000000008,"5min":8.1780000000000008,"15min":8.1780000000000008},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":65.188000000000002,"5min":65.188000000000002,"15min":65.188000000000002},"last":1.6379999999999999}]},{"osd":5,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":26.152999999999999,"5min":26.152999999999999,"15min":26.152999999999999},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":176.48099999999999,"5min":176.48099999999999,"15min":176.48099999999999},"last":1.49},{"interface":"front","average":{"1min":24.885999999999999,"5min":24.885999999999999,"15min":24.885999999999999},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":167.62299999999999,"5min":167.62299999999999,"15min":167.62299999999999},"last":1.528}]},{"osd":6,"last update":"Wed Sep 18 10:16:33 2024","interfaces":[{"interface":"back","average":{"1min":2.6589999999999998,"5min":24.015000000000001,"15min":27.574999999999999},"min":{"1min":1.042,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":23.131,"5min":176.81200000000001,"15min":176.81200000000001},"last":1.2110000000000001},{"interface":"front","average":{"1min":4.7169999999999996,"5min":23.227,"15min":26.312000000000001},"min":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"max":{"1min":61.874000000000002,"5min":167.947,"15min":167.947},"last":1.3260000000000001}]},{"osd":8,"last update":"Wed Sep 18 10:16:50 2024","interfaces":[{"interface":"back","average":{"1min":1.5700000000000001,"5min":1.5700000000000001,"15min":1.5700000000000001},"min":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"max":{"1min":2.117,"5min":2.117,"15min":2.117},"last":1.135},{"interface":"front","average":{"1min":1.514,"5min":1.514,"15min":1.514},"min":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"max":{"1min":2.278,"5min":2.278,"15min":2.278},"last":1.1160000000000001}]},{"osd":9,"last update":"Wed Sep 18 10:16:50 2024","interfaces":[{"interface":"back","average":{"1min":2.617,"5min":17.367000000000001,"15min":19.824999999999999},"min":{"1min":0.878,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":22.934000000000001,"5min":176.73599999999999,"15min":176.73599999999999},"last":1.0620000000000001},{"interface":"front","average":{"1min":1.5660000000000001,"5min":19.085000000000001,"15min":22.004999999999999},"min":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":2.448,"5min":167.84899999999999,"15min":167.84899999999999},"last":1.47}]},{"osd":10,"last update":"Wed Sep 18 10:16:39 2024","interfaces":[{"interface":"back","average":{"1min":5.1120000000000001,"5min":19.164999999999999,"15min":21.507000000000001},"min":{"1min":1.22,"5min":1.1359999999999999,"15min":1.1359999999999999},"max":{"1min":62.137999999999998,"5min":168.07900000000001,"15min":168.07900000000001},"last":1.6950000000000001},{"interface":"front","average":{"1min":17.683,"5min":29.114000000000001,"15min":31.018999999999998},"min":{"1min":1.123,"5min":1.0649999999999999,"15min":1.0649999999999999},"max":{"1min":289.68700000000001,"5min":289.68700000000001,"15min":289.68700000000001},"last":1.2729999999999999}]},{"osd":11,"last update":"Wed Sep 18 10:16:50 2024","interfaces":[{"interface":"back","average":{"1min":1.6819999999999999,"5min":1.625,"15min":1.615},"min":{"1min":1.056,"5min":1.0029999999999999,"15min":1.0029999999999999},"max":{"1min":2.5699999999999998,"5min":2.5699999999999998,"15min":2.5699999999999998},"last":1.175},{"interface":"front","average":{"1min":2.798,"5min":21.73,"15min":24.885999999999999},"min":{"1min":1.286,"5min":1.028,"15min":1.028},"max":{"1min":23.167999999999999,"5min":176.94999999999999,"15min":176.94999999999999},"last":1.401}]},{"osd":12,"last update":"Wed Sep 18 10:16:33 2024","interfaces":[{"interface":"back","average":{"1min":1.8440000000000001,"5min":1.6339999999999999,"15min":1.599},"min":{"1min":1.2490000000000001,"5min":1.0009999999999999,"15min":1.0009999999999999},"max":{"1min":2.5950000000000002,"5min":2.7879999999999998,"15min":2.7879999999999998},"last":1.369},{"interface":"front","average":{"1min":5.0570000000000004,"5min":18.542999999999999,"15min":20.791},"min":{"1min":1.3029999999999999,"5min":1.296,"15min":1.296},"max":{"1min":62.200000000000003,"5min":168.34399999999999,"15min":168.34399999999999},"last":1.595}]},{"osd":14,"last update":"Wed Sep 18 10:16:33 2024","interfaces":[{"interface":"back","average":{"1min":1.8320000000000001,"5min":5.7210000000000001,"15min":6.3689999999999998},"min":{"1min":1.139,"5min":1.0640000000000001,"15min":1.0640000000000001},"max":{"1min":2.419,"5min":87.427000000000007,"15min":87.427000000000007},"last":1.5409999999999999},{"interface":"front","average":{"1min":2.9329999999999998,"5min":21.571999999999999,"15min":24.678000000000001},"min":{"1min":1.2709999999999999,"5min":1.0980000000000001,"15min":1.0980000000000001},"max":{"1min":23.268000000000001,"5min":176.87700000000001,"15min":176.87700000000001},"last":1.51}]},{"osd":15,"last update":"Wed Sep 18 10:16:33 2024","interfaces":[{"interface":"back","average":{"1min":10.048999999999999,"5min":18.277000000000001,"15min":19.648},"min":{"1min":1.272,"5min":1.266,"15min":1.266},"max":{"1min":96.564999999999998,"5min":168.40700000000001,"15min":168.40700000000001},"last":1.772},{"interface":"front","average":{"1min":1.891,"5min":13.459,"15min":15.387},"min":{"1min":1.1950000000000001,"5min":0.95099999999999996,"15min":0.95099999999999996},"max":{"1min":2.5169999999999999,"5min":196.512,"15min":196.512},"last":1.45}]}]},{"osd":0,"up_from":21,"seq":90194313250,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303844,"kb_used_data":2328,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94067996,"statfs":{"total":96636764160,"available":96325627904,"internally_reserved":0,"allocated":2383872,"data_stored":1388693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,4,5,6,9,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 18 10:16:39 2024","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.95699999999999996,"15min":0.95899999999999996},"min":{"1min":0.56699999999999995,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.2889999999999999,"5min":1.341,"15min":1.341},"last":0.96299999999999997},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.97899999999999998,"15min":0.98499999999999999},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.2310000000000001,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":1.4359999999999999}]},{"osd":2,"last update":"Wed Sep 18 10:16:39 2024","interfaces":[{"interface":"back","average":{"1min":3.2719999999999998,"5min":1.6020000000000001,"15min":1.3240000000000001},"min":{"1min":0.76600000000000001,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":46.220999999999997,"5min":46.220999999999997,"15min":46.220999999999997},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.98799999999999999,"5min":1.071,"15min":1.085},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.732,"5min":1.732,"15min":1.732},"last":0.998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.196}]},{"osd":4,"last update":"Wed Sep 18 10:16:30 2024","interfaces":[{"interface":"back","average":{"1min":8.4039999999999999,"5min":14,"15min":14.933},"min":{"1min":0.59699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":142.20699999999999,"5min":142.20699999999999,"15min":142.20699999999999},"last":0.77100000000000002},{"interface":"front","average":{"1min":10.48,"5min":14.896000000000001,"15min":15.632},"min":{"1min":0.63,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":132.33799999999999,"5min":132.33799999999999,"15min":132.33799999999999},"last":1.0580000000000001}]},{"osd":5,"last update":"Wed Sep 18 10:16:39 2024","interfaces":[{"interface":"back","average":{"1min":1.1739999999999999,"5min":1.24,"15min":1.2509999999999999},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":1.619,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":1.198},{"interface":"front","average":{"1min":1.569,"5min":8.1340000000000003,"15min":9.2279999999999998},"min":{"1min":0.85599999999999998,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":10.143000000000001,"5min":130.39599999999999,"15min":130.39599999999999},"last":1.413}]},{"osd":6,"last update":"Wed Sep 18 10:16:47 2024","interfaces":[{"interface":"back","average":{"1min":1.266,"5min":1.254,"15min":1.252},"min":{"1min":0.96099999999999997,"5min":0.94299999999999995,"15min":0.94299999999999995},"max":{"1min":1.571,"5min":1.776,"15min":1.776},"last":1.3580000000000001},{"interface":"front","average":{"1min":1.3109999999999999,"5min":2.0270000000000001,"15min":2.1469999999999998},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.7909999999999999,"5min":15.250999999999999,"15min":15.250999999999999},"last":1.391}]},{"osd":9,"last update":"Wed Sep 18 10:16:30 2024","interfaces":[{"interface":"back","average":{"1min":1.3360000000000001,"5min":1.3919999999999999,"15min":1.4019999999999999},"min":{"1min":0.88200000000000001,"5min":0.879,"15min":0.879},"max":{"1min":1.863,"5min":1.863,"15min":1.863},"last":1.3029999999999999},{"interface":"front","average":{"1min":1.667,"5min":8.7590000000000003,"15min":9.9410000000000007},"min":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":10.180999999999999,"5min":130.441,"15min":130.441},"last":1.3280000000000001}]},{"osd":11,"last update":"Wed Sep 18 10:16:30 2024","interfaces":[{"interface":"back","average":{"1min":4.8949999999999996,"5min":2.2109999999999999,"15min":1.7629999999999999},"min":{"1min":1.1579999999999999,"5min":0.90700000000000003,"15min":0.90700000000000003},"max":{"1min":41.610999999999997,"5min":41.610999999999997,"15min":41.610999999999997},"last":1.2609999999999999},{"interface":"front","average":{"1min":1.5029999999999999,"5min":4.5620000000000003,"15min":5.0720000000000001},"min":{"1min":1.1220000000000001,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":1.9850000000000001,"5min":54.805999999999997,"15min":54.805999999999997},"last":1.5249999999999999}]},{"osd":12,"last update":"Wed Sep 18 10:16:30 2024","interfaces":[{"interface":"back","average":{"1min":1.7869999999999999,"5min":9.1170000000000009,"15min":10.339},"min":{"1min":0.95999999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":10.015000000000001,"5min":130.34800000000001,"15min":130.34800000000001},"last":1.123},{"interface":"front","average":{"1min":1.4350000000000001,"5min":1.4710000000000001,"15min":1.4770000000000001},"min":{"1min":0.85899999999999999,"5min":0.502,"15min":0.502},"max":{"1min":1.944,"5min":1.944,"15min":1.944},"last":1.46}]},{"osd":14,"last update":"Wed Sep 18 10:16:39 2024","interfaces":[{"interface":"back","average":{"1min":3.7490000000000001,"5min":8.7810000000000006,"15min":9.6199999999999992},"min":{"1min":1.044,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":38.969999999999999,"5min":130.48500000000001,"15min":130.48500000000001},"last":1.581},{"interface":"front","average":{"1min":6.931,"5min":2.6339999999999999,"15min":1.9179999999999999},"min":{"1min":1.2609999999999999,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":80.460999999999999,"5min":80.460999999999999,"15min":80.460999999999999},"last":1.544}]},{"osd":15,"last update":"Wed Sep 18 10:16:39 2024","interfaces":[{"interface":"back","average":{"1min":1.591,"5min":2.778,"15min":2.976},"min":{"1min":1.1699999999999999,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":2.0310000000000001,"5min":23.806999999999999,"15min":23.806999999999999},"last":1.7909999999999999},{"interface":"front","average":{"1min":6.9199999999999999,"5min":9.4809999999999999,"15min":9.9079999999999995},"min":{"1min":1.1279999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":100.925,"5min":130.52799999999999,"15min":130.52799999999999},"last":1.6679999999999999}]}]},{"osd":4,"up_from":21,"seq":90194313252,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303876,"kb_used_data":2360,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94067964,"statfs":{"total":96636764160,"available":96325595136,"internally_reserved":0,"allocated":2416640,"data_stored":1407725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,3,5,7,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":9.0980000000000008,"15min":10.465999999999999},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.7529999999999999,"5min":89.266999999999996,"15min":89.266999999999996},"last":0.47799999999999998},{"interface":"front","average":{"1min":18.640000000000001,"5min":7.7320000000000002,"15min":5.9139999999999997},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":198.80699999999999,"5min":198.80699999999999,"15min":198.80699999999999},"last":0.499}]},{"osd":1,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":4.0759999999999996,"5min":12.537000000000001,"15min":13.946999999999999},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":62.780000000000001,"5min":164.21199999999999,"15min":164.21199999999999},"last":0.95599999999999996},{"interface":"front","average":{"1min":2.5139999999999998,"5min":9.593,"15min":10.773},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":25.802,"5min":89.331000000000003,"15min":89.331000000000003},"last":0.67600000000000005}]},{"osd":2,"last update":"Wed Sep 18 10:15:47 2024","interfaces":[{"interface":"back","average":{"1min":23.617000000000001,"5min":23.617000000000001,"15min":23.617000000000001},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":192.136,"5min":192.136,"15min":192.136},"last":1.8740000000000001},{"interface":"front","average":{"1min":11.888999999999999,"5min":11.888999999999999,"15min":11.888999999999999},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":198.87700000000001,"5min":198.87700000000001,"15min":198.87700000000001},"last":1.347}]},{"osd":3,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":35.673000000000002,"5min":21.670999999999999,"15min":19.337},"min":{"1min":1.161,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":388.84300000000002,"5min":388.84300000000002,"15min":388.84300000000002},"last":0.79100000000000004},{"interface":"front","average":{"1min":17.747,"5min":25.643999999999998,"15min":26.960000000000001},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":255.82900000000001,"5min":255.82900000000001,"15min":255.82900000000001},"last":0.626}]},{"osd":5,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":1.4910000000000001,"5min":15.449999999999999,"15min":17.776},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":2.169,"5min":189.55600000000001,"15min":189.55600000000001},"last":0.88100000000000001},{"interface":"front","average":{"1min":21.053000000000001,"5min":6.8079999999999998,"15min":4.4340000000000002},"min":{"1min":1.3640000000000001,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":199.15600000000001,"5min":199.15600000000001,"15min":199.15600000000001},"last":0.64500000000000002}]},{"osd":6,"last update":"Wed Sep 18 10:15:34 2024","interfaces":[{"interface":"back","average":{"1min":32.25,"5min":32.25,"15min":32.25},"min":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":197.33199999999999,"5min":197.33199999999999,"15min":197.33199999999999},"last":1.8160000000000001},{"interface":"front","average":{"1min":18.963000000000001,"5min":18.963000000000001,"15min":18.963000000000001},"min":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"max":{"1min":179.19200000000001,"5min":179.19200000000001,"15min":179.19200000000001},"last":1.7769999999999999}]},{"osd":7,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":5.6050000000000004,"5min":23.198,"15min":26.129999999999999},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":62.704999999999998,"5min":290.036,"15min":290.036},"last":0.81899999999999995},{"interface":"front","average":{"1min":1.6639999999999999,"5min":14.273,"15min":16.375},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":2.3100000000000001,"5min":139.696,"15min":139.696},"last":0.71399999999999997}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1509999999999998}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7509999999999999}]},{"osd":10,"last update":"Wed Sep 18 10:16:49 2024","interfaces":[{"interface":"back","average":{"1min":1.653,"5min":18.559000000000001,"15min":21.376999999999999},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":2.4740000000000002,"5min":164.43899999999999,"15min":164.43899999999999},"last":0.75900000000000001},{"interface":"front","average":{"1min":4.6070000000000002,"5min":10.993,"15min":12.057},"min":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":59.408999999999999,"5min":89.442999999999998,"15min":89.442999999999998},"last":0.77200000000000002}]},{"osd":11,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":19.57,"5min":12.567,"15min":11.4},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":199.255,"5min":199.255,"15min":199.255},"last":0.94099999999999995},{"interface":"front","average":{"1min":11.393000000000001,"5min":22.213000000000001,"15min":24.016999999999999},"min":{"1min":1.0840000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"max":{"1min":155.19800000000001,"5min":164.733,"15min":164.733},"last":1.0700000000000001}]},{"osd":12,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":4.5800000000000001,"5min":10.406000000000001,"15min":11.377000000000001},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":58.856000000000002,"5min":89.400000000000006,"15min":89.400000000000006},"last":0.57099999999999995},{"interface":"front","average":{"1min":31.831,"5min":12.901999999999999,"15min":9.7469999999999999},"min":{"1min":1.4530000000000001,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":199.32900000000001,"5min":199.32900000000001,"15min":199.32900000000001},"last":0.68899999999999995}]},{"osd":13,"last update":"Wed Sep 18 10:16:49 2024","interfaces":[{"interface":"back","average":{"1min":12.695,"5min":22.789999999999999,"15min":24.472000000000001},"min":{"1min":1.702,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":192.37299999999999,"5min":199.37799999999999,"15min":199.37799999999999},"last":0.84599999999999997},{"interface":"front","average":{"1min":1.8029999999999999,"5min":19.068000000000001,"15min":21.946000000000002},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":2.5699999999999998,"5min":164.53,"15min":164.53},"last":0.86299999999999999}]},{"osd":14,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":22.312000000000001,"5min":21.788,"15min":21.701000000000001},"min":{"1min":1.556,"5min":0.96099999999999997,"15min":0.96099999999999997},"max":{"1min":199.43000000000001,"5min":202.191,"15min":202.191},"last":0.90500000000000003},{"interface":"front","average":{"1min":9.8800000000000008,"5min":17.420999999999999,"15min":18.678000000000001},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":101.492,"5min":164.35400000000001,"15min":164.35400000000001},"last":1.022}]}]},{"osd":9,"up_from":21,"seq":90194313250,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303848,"kb_used_data":2328,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94067992,"statfs":{"total":96636764160,"available":96325623808,"internally_reserved":0,"allocated":2383872,"data_stored":1388693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,4,5,6,8,10,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":7.2450000000000001,"5min":7.2450000000000001,"15min":7.2450000000000001},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":127.902,"5min":127.902,"15min":127.902},"last":0.97399999999999998},{"interface":"front","average":{"1min":8.3109999999999999,"5min":8.3109999999999999,"15min":8.3109999999999999},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":96.203000000000003,"5min":96.203000000000003,"15min":96.203000000000003},"last":0.81200000000000006}]},{"osd":1,"last update":"Wed Sep 18 10:16:40 2024","interfaces":[{"interface":"back","average":{"1min":10.587,"5min":10.587,"15min":10.587},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":166.833,"5min":166.833,"15min":166.833},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.90900000000000003}]},{"osd":2,"last update":"Wed Sep 18 10:16:18 2024","interfaces":[{"interface":"back","average":{"1min":5.6420000000000003,"5min":5.6420000000000003,"15min":5.6420000000000003},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":68.049999999999997,"5min":68.049999999999997,"15min":68.049999999999997},"last":1.028},{"interface":"front","average":{"1min":2.0310000000000001,"5min":2.0310000000000001,"15min":2.0310000000000001},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":18.553999999999998,"5min":18.553999999999998,"15min":18.553999999999998},"last":1.0569999999999999}]},{"osd":3,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":2.875,"5min":5.5549999999999997,"15min":6.0019999999999998},"min":{"1min":0.72699999999999998,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":18.431000000000001,"5min":98.888999999999996,"15min":98.888999999999996},"last":1.1499999999999999},{"interface":"front","average":{"1min":3.0169999999999999,"5min":5.5890000000000004,"15min":6.0170000000000003},"min":{"1min":0.749,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":18.472000000000001,"5min":98.947999999999993,"15min":98.947999999999993},"last":1.349}]},{"osd":4,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":8.2360000000000007,"5min":2.5840000000000001,"15min":1.6419999999999999},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":124.15300000000001,"5min":124.15300000000001,"15min":124.15300000000001},"last":1.446},{"interface":"front","average":{"1min":6.2380000000000004,"5min":8.8539999999999992,"15min":9.2899999999999991},"min":{"1min":0.81000000000000005,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":73.992000000000004,"5min":73.992000000000004,"15min":73.992000000000004},"last":1.4710000000000001}]},{"osd":5,"last update":"Wed Sep 18 10:16:18 2024","interfaces":[{"interface":"back","average":{"1min":1.821,"5min":1.821,"15min":1.821},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":14.786,"5min":14.786,"15min":14.786},"last":0.96799999999999997},{"interface":"front","average":{"1min":8.5,"5min":8.5,"15min":8.5},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":67.915999999999997,"5min":67.915999999999997,"15min":67.915999999999997},"last":1.1830000000000001}]},{"osd":6,"last update":"Wed Sep 18 10:16:18 2024","interfaces":[{"interface":"back","average":{"1min":8.3840000000000003,"5min":8.3840000000000003,"15min":8.3840000000000003},"min":{"1min":0.997,"5min":0.997,"15min":0.997},"max":{"1min":70.914000000000001,"5min":70.914000000000001,"15min":70.914000000000001},"last":1.619},{"interface":"front","average":{"1min":15.705,"5min":15.705,"15min":15.705},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":136.38800000000001,"5min":136.38800000000001,"15min":136.38800000000001},"last":1.3859999999999999}]},{"osd":8,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":3.0310000000000001,"5min":4.2789999999999999,"15min":4.4870000000000001},"min":{"1min":1.0509999999999999,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":30.713000000000001,"5min":64.912999999999997,"15min":64.912999999999997},"last":1.2869999999999999},{"interface":"front","average":{"1min":2.0569999999999999,"5min":5.3959999999999999,"15min":5.952},"min":{"1min":0.82499999999999996,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":12.898999999999999,"5min":57.664000000000001,"15min":57.664000000000001},"last":1.71}]},{"osd":10,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":1.3400000000000001,"5min":1.3089999999999999,"15min":1.304},"min":{"1min":0.88300000000000001,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.8180000000000001,"5min":2.109,"15min":2.109},"last":1.585},{"interface":"front","average":{"1min":6.0389999999999997,"5min":4.2309999999999999,"15min":3.9300000000000002},"min":{"1min":0.97899999999999998,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":68.216999999999999,"5min":68.216999999999999,"15min":68.216999999999999},"last":1.536}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.669}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.446}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":96}]},{"osd":14,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":4.3579999999999997,"5min":8.8759999999999994,"15min":9.6289999999999996},"min":{"1min":0.86499999999999999,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":50.773000000000003,"5min":62.899000000000001,"15min":62.899000000000001},"last":1.772},{"interface":"front","average":{"1min":4.3540000000000001,"5min":13.071,"15min":14.523999999999999},"min":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":50.709000000000003,"5min":155.61000000000001,"15min":155.61000000000001},"last":1.655}]},{"osd":15,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":1.4630000000000001,"5min":1.492,"15min":1.4970000000000001},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.97,"5min":1.9970000000000001,"15min":1.9970000000000001},"last":1.7450000000000001},{"interface":"front","average":{"1min":9.5700000000000003,"5min":5.1749999999999998,"15min":4.4429999999999996},"min":{"1min":1.0980000000000001,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":68.140000000000001,"5min":68.140000000000001,"15min":68.140000000000001},"last":1.679}]}]},{"osd":11,"up_from":21,"seq":90194313250,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303868,"kb_used_data":2344,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94067972,"statfs":{"total":96636764160,"available":96325603328,"internally_reserved":0,"allocated":2400256,"data_stored":1398209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[2,3,6,8,9,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0049999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86599999999999999}]},{"osd":2,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":8.3390000000000004,"15min":9.5719999999999992},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.3740000000000001,"5min":99.718000000000004,"15min":99.718000000000004},"last":1.2290000000000001},{"interface":"front","average":{"1min":3.6349999999999998,"5min":1.4450000000000001,"15min":1.0800000000000001},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":49.774000000000001,"5min":49.774000000000001,"15min":49.774000000000001},"last":0.98799999999999999}]},{"osd":3,"last update":"Wed Sep 18 10:16:46 2024","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":4.2210000000000001,"15min":4.7880000000000003},"min":{"1min":0.48399999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.369,"5min":85.963999999999999,"15min":85.963999999999999},"last":1.0580000000000001},{"interface":"front","average":{"1min":5.4660000000000002,"5min":19.300000000000001,"15min":21.606000000000002},"min":{"1min":0.73699999999999999,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":80.980999999999995,"5min":125.672,"15min":125.672},"last":0.85499999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.444}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5980000000000001}]},{"osd":6,"last update":"Wed Sep 18 10:16:46 2024","interfaces":[{"interface":"back","average":{"1min":5.8810000000000002,"5min":2.101,"15min":1.4710000000000001},"min":{"1min":0.70599999999999996,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":86.680000000000007,"5min":86.680000000000007,"15min":86.680000000000007},"last":1.1910000000000001},{"interface":"front","average":{"1min":1.1830000000000001,"5min":5.827,"15min":6.601},"min":{"1min":0.73799999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.49,"5min":94.162000000000006,"15min":94.162000000000006},"last":1.133}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.524}]},{"osd":8,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":2.7069999999999999,"5min":5.3730000000000002,"15min":5.8170000000000002},"min":{"1min":0.95299999999999996,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":27.577999999999999,"5min":85.900000000000006,"15min":85.900000000000006},"last":0.95899999999999996},{"interface":"front","average":{"1min":2.544,"5min":8.4960000000000004,"15min":9.4879999999999995},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":27.542000000000002,"5min":99.765000000000001,"15min":99.765000000000001},"last":1.3320000000000001}]},{"osd":9,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":5.8760000000000003,"5min":5.8339999999999996,"15min":5.827},"min":{"1min":0.96399999999999997,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":89.844999999999999,"5min":89.844999999999999,"15min":89.844999999999999},"last":1.5589999999999999},{"interface":"front","average":{"1min":1.351,"5min":6.8070000000000004,"15min":7.7160000000000002},"min":{"1min":0.68600000000000005,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.742,"5min":99.810000000000002,"15min":99.810000000000002},"last":1.6000000000000001}]},{"osd":10,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":4.0069999999999997,"5min":1.679,"15min":1.2909999999999999},"min":{"1min":0.98499999999999999,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":49.819000000000003,"5min":49.819000000000003,"15min":49.819000000000003},"last":1.3},{"interface":"front","average":{"1min":1.5580000000000001,"5min":6.46,"15min":7.2770000000000001},"min":{"1min":0.98699999999999999,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.976,"5min":89.290000000000006,"15min":89.290000000000006},"last":1.74}]},{"osd":12,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":5.907,"5min":2.081,"15min":1.4430000000000001},"min":{"1min":0.96599999999999997,"5min":0.498,"15min":0.498},"max":{"1min":87.463999999999999,"5min":87.463999999999999,"15min":87.463999999999999},"last":1.3979999999999999},{"interface":"front","average":{"1min":1.5660000000000001,"5min":8.6219999999999999,"15min":9.798},"min":{"1min":1.0700000000000001,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":2.0600000000000001,"5min":86.025999999999996,"15min":86.025999999999996},"last":1.988}]},{"osd":13,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":4.4210000000000003,"5min":25.207999999999998,"15min":28.672000000000001},"min":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"max":{"1min":59.841999999999999,"5min":189.25899999999999,"15min":189.25899999999999},"last":1.9359999999999999},{"interface":"front","average":{"1min":7.569,"5min":2.5659999999999998,"15min":1.732},"min":{"1min":0.88900000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":71.313000000000002,"5min":71.313000000000002,"15min":71.313000000000002},"last":1.6539999999999999}]},{"osd":14,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":14.388999999999999,"5min":8.016,"15min":6.9539999999999997},"min":{"1min":1.0129999999999999,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":256.13400000000001,"5min":256.13400000000001,"15min":256.13400000000001},"last":1.899},{"interface":"front","average":{"1min":2.7650000000000001,"5min":7.7380000000000004,"15min":8.5670000000000002},"min":{"1min":0.874,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":25.791,"5min":102.625,"15min":102.625},"last":1.8149999999999999}]},{"osd":15,"last update":"Wed Sep 18 10:16:40 2024","interfaces":[{"interface":"back","average":{"1min":4.0309999999999997,"5min":4.0309999999999997,"15min":4.0309999999999997},"min":{"1min":1.153,"5min":1.153,"15min":1.153},"max":{"1min":49.927999999999997,"5min":49.927999999999997,"15min":49.927999999999997},"last":1.7889999999999999},{"interface":"front","average":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"min":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"max":{"1min":2.044,"5min":2.044,"15min":2.044},"last":2.0800000000000001}]}]},{"osd":12,"up_from":21,"seq":90194313251,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303864,"kb_used_data":2348,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94067976,"statfs":{"total":96636764160,"available":96325607424,"internally_reserved":0,"allocated":2404352,"data_stored":1398228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[3,4,6,7,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":2.3839999999999999,"5min":2.3839999999999999,"15min":2.3839999999999999},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":32.476999999999997,"5min":32.476999999999997,"15min":32.476999999999997},"last":0.64200000000000002},{"interface":"front","average":{"1min":2.625,"5min":2.625,"15min":2.625},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":32.509,"5min":32.509,"15min":32.509},"last":0.89300000000000002}]},{"osd":1,"last update":"Wed Sep 18 10:15:32 2024","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.6399999999999999,"5min":1.6399999999999999,"15min":1.6399999999999999},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.905,"5min":1.905,"15min":1.905},"last":0.81899999999999995}]},{"osd":2,"last update":"Wed Sep 18 10:15:32 2024","interfaces":[{"interface":"back","average":{"1min":1.002,"5min":1.002,"15min":1.002},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":1.153},{"interface":"front","average":{"1min":4.1189999999999998,"5min":4.1189999999999998,"15min":4.1189999999999998},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":62.780999999999999,"5min":62.780999999999999,"15min":62.780999999999999},"last":0.96499999999999997}]},{"osd":3,"last update":"Wed Sep 18 10:16:33 2024","interfaces":[{"interface":"back","average":{"1min":5.9429999999999996,"5min":2.1040000000000001,"15min":1.464},"min":{"1min":0.627,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":74.822999999999993,"5min":74.822999999999993,"15min":74.822999999999993},"last":0.98899999999999999},{"interface":"front","average":{"1min":8.8200000000000003,"5min":2.6840000000000002,"15min":1.6619999999999999},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":94.320999999999998,"5min":94.320999999999998,"15min":94.320999999999998},"last":0.81200000000000006}]},{"osd":4,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":16.058,"5min":4.0529999999999999,"15min":2.052},"min":{"1min":0.53100000000000003,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":114.184,"5min":114.184,"15min":114.184},"last":0.85299999999999998},{"interface":"front","average":{"1min":5.8920000000000003,"5min":2.1560000000000001,"15min":1.5329999999999999},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":74.698999999999998,"5min":74.698999999999998,"15min":74.698999999999998},"last":0.77300000000000002}]},{"osd":6,"last update":"Wed Sep 18 10:16:38 2024","interfaces":[{"interface":"back","average":{"1min":10.747999999999999,"5min":10.747999999999999,"15min":10.747999999999999},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":130.20699999999999,"5min":130.20699999999999,"15min":130.20699999999999},"last":1.224},{"interface":"front","average":{"1min":14.148,"5min":14.148,"15min":14.148},"min":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":132.60900000000001,"5min":132.60900000000001,"15min":132.60900000000001},"last":1.135}]},{"osd":7,"last update":"Wed Sep 18 10:16:38 2024","interfaces":[{"interface":"back","average":{"1min":4.774,"5min":4.774,"15min":4.774},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":90.188000000000002,"5min":90.188000000000002,"15min":90.188000000000002},"last":1.0269999999999999},{"interface":"front","average":{"1min":6.0359999999999996,"5min":6.0359999999999996,"15min":6.0359999999999996},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":75.061999999999998,"5min":75.061999999999998,"15min":75.061999999999998},"last":1.0620000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":94.382000000000005}]},{"osd":9,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":8.9719999999999995,"5min":2.8079999999999998,"15min":1.7809999999999999},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":94.421000000000006,"5min":94.421000000000006,"15min":94.421000000000006},"last":1.0920000000000001},{"interface":"front","average":{"1min":5.0410000000000004,"5min":2.0179999999999998,"15min":1.514},"min":{"1min":0.70199999999999996,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":90.242999999999995,"5min":90.242999999999995,"15min":90.242999999999995},"last":1.4159999999999999}]},{"osd":10,"last update":"Wed Sep 18 10:16:38 2024","interfaces":[{"interface":"back","average":{"1min":15.145,"5min":15.145,"15min":15.145},"min":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":99.251000000000005,"5min":99.251000000000005,"15min":99.251000000000005},"last":1.262},{"interface":"front","average":{"1min":8.7080000000000002,"5min":8.7080000000000002,"15min":8.7080000000000002},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":94.468000000000004,"5min":94.468000000000004,"15min":94.468000000000004},"last":1.1799999999999999}]},{"osd":11,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":8.9269999999999996,"5min":2.8769999999999998,"15min":1.869},"min":{"1min":0.91300000000000003,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":74.888999999999996,"5min":74.888999999999996,"15min":74.888999999999996},"last":1.4730000000000001},{"interface":"front","average":{"1min":9.1590000000000007,"5min":2.9380000000000002,"15min":1.901},"min":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"max":{"1min":94.546000000000006,"5min":94.546000000000006,"15min":94.546000000000006},"last":1.5509999999999999}]},{"osd":13,"last update":"Wed Sep 18 10:16:33 2024","interfaces":[{"interface":"back","average":{"1min":7.4779999999999998,"5min":2.5619999999999998,"15min":1.742},"min":{"1min":0.79900000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":143.672,"5min":143.672,"15min":143.672},"last":1.3340000000000001},{"interface":"front","average":{"1min":12.677,"5min":3.6779999999999999,"15min":2.1779999999999999},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":143.619,"5min":143.619,"15min":143.619},"last":1.3129999999999999}]},{"osd":14,"last update":"Wed Sep 18 10:16:33 2024","interfaces":[{"interface":"back","average":{"1min":13.363,"5min":3.8159999999999998,"15min":2.2250000000000001},"min":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"max":{"1min":94.504999999999995,"5min":94.504999999999995,"15min":94.504999999999995},"last":1.288},{"interface":"front","average":{"1min":12.039,"5min":3.52,"15min":2.1000000000000001},"min":{"1min":0.95099999999999996,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":92.593000000000004,"5min":92.593000000000004,"15min":92.593000000000004},"last":1.5}]},{"osd":15,"last update":"Wed Sep 18 10:16:33 2024","interfaces":[{"interface":"back","average":{"1min":6.5339999999999998,"5min":2.5019999999999998,"15min":1.8300000000000001},"min":{"1min":0.751,"5min":0.751,"15min":0.751},"max":{"1min":74.966999999999999,"5min":74.966999999999999,"15min":74.966999999999999},"last":1.361},{"interface":"front","average":{"1min":14.669,"5min":4.117,"15min":2.359},"min":{"1min":0.996,"5min":0.95299999999999996,"15min":0.95299999999999996},"max":{"1min":152.11000000000001,"5min":152.11000000000001,"15min":152.11000000000001},"last":1.3819999999999999}]}]},{"osd":1,"up_from":22,"seq":94489280546,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":303864,"kb_used_data":2344,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94067976,"statfs":{"total":96636764160,"available":96325607424,"internally_reserved":0,"allocated":2400256,"data_stored":1398209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,6,9,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:16:38 2024","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":2.472,"15min":2.7370000000000001},"min":{"1min":0.65800000000000003,"5min":0.629,"15min":0.629},"max":{"1min":1.1930000000000001,"5min":36.268000000000001,"15min":36.268000000000001},"last":0.73499999999999999},{"interface":"front","average":{"1min":5.3659999999999997,"5min":15.244999999999999,"15min":16.890999999999998},"min":{"1min":0.73099999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":66.534999999999997,"5min":154.67599999999999,"15min":154.67599999999999},"last":0.77900000000000003}]},{"osd":2,"last update":"Wed Sep 18 10:16:38 2024","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.91400000000000003,"15min":0.90400000000000003},"min":{"1min":0.71499999999999997,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.4119999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.98999999999999999,"5min":2.5539999999999998,"15min":2.8140000000000001},"min":{"1min":0.75600000000000001,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.212,"5min":36.323999999999998,"15min":36.323999999999998},"last":0.95299999999999996}]},{"osd":3,"last update":"Wed Sep 18 10:16:38 2024","interfaces":[{"interface":"back","average":{"1min":5.548,"5min":10.811999999999999,"15min":11.689},"min":{"1min":0.94599999999999995,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":66.344999999999999,"5min":150.642,"15min":150.642},"last":1.0980000000000001},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.0649999999999999,"15min":1.0509999999999999},"min":{"1min":0.82999999999999996,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.6830000000000001,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":1.1299999999999999}]},{"osd":4,"last update":"Wed Sep 18 10:15:35 2024","interfaces":[{"interface":"back","average":{"1min":3.3439999999999999,"5min":3.3439999999999999,"15min":3.3439999999999999},"min":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":40.783000000000001,"5min":40.783000000000001,"15min":40.783000000000001},"last":1.7210000000000001},{"interface":"front","average":{"1min":4.1699999999999999,"5min":4.1699999999999999,"15min":4.1699999999999999},"min":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":50.747,"5min":50.747,"15min":50.747},"last":1.518}]},{"osd":5,"last update":"Wed Sep 18 10:15:35 2024","interfaces":[{"interface":"back","average":{"1min":5.9720000000000004,"5min":5.9720000000000004,"15min":5.9720000000000004},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":52.317999999999998,"5min":52.317999999999998,"15min":52.317999999999998},"last":1.2649999999999999},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.748,"5min":1.748,"15min":1.748},"last":1.839}]},{"osd":6,"last update":"Wed Sep 18 10:16:38 2024","interfaces":[{"interface":"back","average":{"1min":9.2970000000000006,"5min":8.5380000000000003,"15min":8.4120000000000008},"min":{"1min":0.85099999999999998,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":120.85299999999999,"5min":120.85299999999999,"15min":120.85299999999999},"last":1.28},{"interface":"front","average":{"1min":2.6040000000000001,"5min":8.8019999999999996,"15min":9.8350000000000009},"min":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"max":{"1min":20.600000000000001,"5min":85.045000000000002,"15min":85.045000000000002},"last":1.0529999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":118.43600000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7789999999999999}]},{"osd":9,"last update":"Wed Sep 18 10:16:15 2024","interfaces":[{"interface":"back","average":{"1min":3.4369999999999998,"5min":3.4369999999999998,"15min":3.4369999999999998},"min":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"max":{"1min":36.506999999999998,"5min":36.506999999999998,"15min":36.506999999999998},"last":1.2529999999999999},{"interface":"front","average":{"1min":11.339,"5min":11.339,"15min":11.339},"min":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"max":{"1min":165.13499999999999,"5min":165.13499999999999,"15min":165.13499999999999},"last":1.204}]},{"osd":10,"last update":"Wed Sep 18 10:16:38 2024","interfaces":[{"interface":"back","average":{"1min":1.5209999999999999,"5min":3.048,"15min":3.302},"min":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"max":{"1min":2.0489999999999999,"5min":36.436,"15min":36.436},"last":1.4079999999999999},{"interface":"front","average":{"1min":6.0839999999999996,"5min":10.747999999999999,"15min":11.526},"min":{"1min":1.2769999999999999,"5min":0.88,"15min":0.88},"max":{"1min":66.673000000000002,"5min":100.402,"15min":100.402},"last":1.3080000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.825}]},{"osd":12,"last update":"Wed Sep 18 10:16:38 2024","interfaces":[{"interface":"back","average":{"1min":1.5549999999999999,"5min":3.681,"15min":4.0350000000000001},"min":{"1min":1.101,"5min":1.0589999999999999,"15min":1.0589999999999999},"max":{"1min":2.294,"5min":49.976999999999997,"15min":49.976999999999997},"last":1.389},{"interface":"front","average":{"1min":2.0369999999999999,"5min":7.1269999999999998,"15min":7.9749999999999996},"min":{"1min":1.196,"5min":1.196,"15min":1.196},"max":{"1min":8.4049999999999994,"5min":124.78100000000001,"15min":124.78100000000001},"last":1.171}]},{"osd":13,"last update":"Wed Sep 18 10:16:15 2024","interfaces":[{"interface":"back","average":{"1min":11.616,"5min":11.616,"15min":11.616},"min":{"1min":1.232,"5min":1.232,"15min":1.232},"max":{"1min":104.94199999999999,"5min":104.94199999999999,"15min":104.94199999999999},"last":1.369},{"interface":"front","average":{"1min":1.5720000000000001,"5min":1.5720000000000001,"15min":1.5720000000000001},"min":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"max":{"1min":2.3809999999999998,"5min":2.3809999999999998,"15min":2.3809999999999998},"last":1.5389999999999999}]},{"osd":14,"last update":"Wed Sep 18 10:16:15 2024","interfaces":[{"interface":"back","average":{"1min":5.5910000000000002,"5min":5.5910000000000002,"15min":5.5910000000000002},"min":{"1min":1.276,"5min":1.276,"15min":1.276},"max":{"1min":66.825999999999993,"5min":66.825999999999993,"15min":66.825999999999993},"last":1.5089999999999999},{"interface":"front","average":{"1min":2.8359999999999999,"5min":2.8359999999999999,"15min":2.8359999999999999},"min":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"max":{"1min":22.16,"5min":22.16,"15min":22.16},"last":1.496}]},{"osd":15,"last update":"Wed Sep 18 10:16:15 2024","interfaces":[{"interface":"back","average":{"1min":3.677,"5min":3.677,"15min":3.677},"min":{"1min":1.165,"5min":1.165,"15min":1.165},"max":{"1min":36.575000000000003,"5min":36.575000000000003,"15min":36.575000000000003},"last":1.3400000000000001},{"interface":"front","average":{"1min":5.6269999999999998,"5min":5.6269999999999998,"15min":5.6269999999999998},"min":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"max":{"1min":66.763999999999996,"5min":66.763999999999996,"15min":66.763999999999996},"last":1.4410000000000001}]}]},{"osd":2,"up_from":22,"seq":94489280546,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":303860,"kb_used_data":2344,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94067980,"statfs":{"total":96636764160,"available":96325611520,"internally_reserved":0,"allocated":2400256,"data_stored":1398209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,3,4,5,6,7,8,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":21.344000000000001,"15min":24.777999999999999},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.3879999999999999,"5min":419.05700000000002,"15min":419.05700000000002},"last":1.097},{"interface":"front","average":{"1min":2.3860000000000001,"5min":4.4020000000000001,"15min":4.7380000000000004},"min":{"1min":0.57299999999999995,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":35.899999999999999,"5min":98.254000000000005,"15min":98.254000000000005},"last":0.69399999999999995}]},{"osd":3,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":5.1219999999999999,"5min":1.7529999999999999,"15min":1.1910000000000001},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":86.677000000000007,"5min":86.677000000000007,"15min":86.677000000000007},"last":1.0600000000000001},{"interface":"front","average":{"1min":3.6160000000000001,"5min":1.47,"15min":1.1120000000000001},"min":{"1min":0.68000000000000005,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":59.759,"5min":59.759,"15min":59.759},"last":0.98599999999999999}]},{"osd":4,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":7.992,"5min":3.1379999999999999,"15min":2.3290000000000002},"min":{"1min":0.55600000000000005,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":121.20099999999999,"5min":121.20099999999999,"15min":121.20099999999999},"last":1.0149999999999999},{"interface":"front","average":{"1min":6.5250000000000004,"5min":4.9359999999999999,"15min":4.6710000000000003},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":86.548000000000002,"5min":86.548000000000002,"15min":86.548000000000002},"last":0.95099999999999996}]},{"osd":5,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":12.161,"5min":3.2879999999999998,"15min":1.8089999999999999},"min":{"1min":0.72099999999999997,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":163.828,"5min":163.828,"15min":163.828},"last":0.61699999999999999},{"interface":"front","average":{"1min":2.6160000000000001,"5min":1.3660000000000001,"15min":1.1579999999999999},"min":{"1min":0.79200000000000004,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":36.119,"5min":36.119,"15min":36.119},"last":1.131}]},{"osd":6,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":5.2359999999999998,"5min":4.8529999999999998,"15min":4.7889999999999997},"min":{"1min":0.49199999999999999,"5min":0.441,"15min":0.441},"max":{"1min":86.766000000000005,"5min":86.766000000000005,"15min":86.766000000000005},"last":1.331},{"interface":"front","average":{"1min":5.3959999999999999,"5min":4.9770000000000003,"15min":4.907},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":83.171999999999997,"5min":90.188999999999993,"15min":90.188999999999993},"last":0.91700000000000004}]},{"osd":7,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":13.191000000000001,"5min":3.6019999999999999,"15min":2.004},"min":{"1min":0.84999999999999998,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":88.534000000000006,"5min":88.534000000000006,"15min":88.534000000000006},"last":1.242},{"interface":"front","average":{"1min":15.949999999999999,"5min":4.2350000000000003,"15min":2.2829999999999999},"min":{"1min":0.75600000000000001,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":88.484999999999999,"5min":88.484999999999999,"15min":88.484999999999999},"last":1.1679999999999999}]},{"osd":8,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":4.3179999999999996,"15min":4.8399999999999999},"min":{"1min":0.82899999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":2.0699999999999998,"5min":92.578999999999994,"15min":92.578999999999994},"last":1.214},{"interface":"front","average":{"1min":2.8460000000000001,"5min":2.7799999999999998,"15min":2.7690000000000001},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":36.088000000000001,"5min":36.088000000000001,"15min":36.088000000000001},"last":1.5840000000000001}]},{"osd":12,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":2.9220000000000002,"5min":1.6719999999999999,"15min":1.464},"min":{"1min":0.872,"5min":0.872,"15min":0.872},"max":{"1min":36.014000000000003,"5min":36.014000000000003,"15min":36.014000000000003},"last":1.4299999999999999},{"interface":"front","average":{"1min":5.5140000000000002,"5min":2.7469999999999999,"15min":2.286},"min":{"1min":0.73199999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":86.832999999999998,"5min":86.832999999999998,"15min":86.832999999999998},"last":1.2789999999999999}]},{"osd":13,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":11.115,"5min":8.3970000000000002,"15min":7.944},"min":{"1min":0.97999999999999998,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":80.683000000000007,"5min":141.84200000000001,"15min":141.84200000000001},"last":1.306},{"interface":"front","average":{"1min":2.911,"5min":4.9489999999999998,"15min":5.2889999999999997},"min":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":35.841000000000001,"5min":97.441999999999993,"15min":97.441999999999993},"last":1.5129999999999999}]},{"osd":15,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":12.472,"5min":3.6200000000000001,"15min":2.145},"min":{"1min":0.68400000000000005,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":86.980000000000004,"5min":86.980000000000004,"15min":86.980000000000004},"last":1.3720000000000001},{"interface":"front","average":{"1min":3.3479999999999999,"5min":3.798,"15min":3.8730000000000002},"min":{"1min":0.96999999999999997,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":45.356999999999999,"5min":61.694000000000003,"15min":61.694000000000003},"last":1.351}]}]},{"osd":3,"up_from":22,"seq":94489280546,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303848,"kb_used_data":2328,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94067992,"statfs":{"total":96636764160,"available":96325623808,"internally_reserved":0,"allocated":2383872,"data_stored":1388693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,4,6,7,9,10,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:16:37 2024","interfaces":[{"interface":"back","average":{"1min":4.173,"5min":2.048,"15min":1.694},"min":{"1min":0.64300000000000002,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":65.994,"5min":65.994,"15min":65.994},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.97899999999999998,"5min":8.8260000000000005,"15min":10.134},"min":{"1min":0.75,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.752,"5min":71.316999999999993,"15min":71.316999999999993},"last":0.96399999999999997}]},{"osd":2,"last update":"Wed Sep 18 10:16:37 2024","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":4.4379999999999997,"15min":5.0140000000000002},"min":{"1min":0.76900000000000002,"5min":0.75,"15min":0.75},"max":{"1min":1.333,"5min":56.523000000000003,"15min":56.523000000000003},"last":0.76600000000000001},{"interface":"front","average":{"1min":1.165,"5min":1.1200000000000001,"15min":1.1120000000000001},"min":{"1min":0.67400000000000004,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.639,"5min":3.327,"15min":3.327},"last":0.72699999999999998}]},{"osd":4,"last update":"Wed Sep 18 10:16:37 2024","interfaces":[{"interface":"back","average":{"1min":1.0409999999999999,"5min":5.7069999999999999,"15min":6.4850000000000003},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.6399999999999999,"5min":71.135000000000005,"15min":71.135000000000005},"last":0.81299999999999994},{"interface":"front","average":{"1min":1.635,"5min":3.8130000000000002,"15min":4.1760000000000002},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":10.039,"5min":56.253999999999998,"15min":56.253999999999998},"last":0.93100000000000005}]},{"osd":5,"last update":"Wed Sep 18 10:16:06 2024","interfaces":[{"interface":"back","average":{"1min":13.997,"5min":13.997,"15min":13.997},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":125.42400000000001,"5min":125.42400000000001,"15min":125.42400000000001},"last":1.131},{"interface":"front","average":{"1min":10.323,"5min":10.323,"15min":10.323},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":112.033,"5min":112.033,"15min":112.033},"last":0.96899999999999997}]},{"osd":6,"last update":"Wed Sep 18 10:16:37 2024","interfaces":[{"interface":"back","average":{"1min":1.9279999999999999,"5min":1.29,"15min":1.1839999999999999},"min":{"1min":0.93400000000000005,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":12.83,"5min":12.83,"15min":12.83},"last":1.1579999999999999},{"interface":"front","average":{"1min":5.0190000000000001,"5min":8.0050000000000008,"15min":8.5030000000000001},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":76.634,"5min":81.210999999999999,"15min":81.210999999999999},"last":1.1100000000000001}]},{"osd":7,"last update":"Wed Sep 18 10:16:37 2024","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":13.159000000000001,"15min":15.141},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.75,"5min":129.68299999999999,"15min":129.68299999999999},"last":1.0129999999999999},{"interface":"front","average":{"1min":1.4710000000000001,"5min":11.119999999999999,"15min":12.728},"min":{"1min":0.90800000000000003,"5min":0.621,"15min":0.621},"max":{"1min":1.8500000000000001,"5min":129.726,"15min":129.726},"last":1.383}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0620000000000001}]},{"osd":9,"last update":"Wed Sep 18 10:16:37 2024","interfaces":[{"interface":"back","average":{"1min":1.5660000000000001,"5min":1.446,"15min":1.4259999999999999},"min":{"1min":0.94599999999999995,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":2.0209999999999999,"5min":2.0209999999999999,"15min":2.0209999999999999},"last":1.577},{"interface":"front","average":{"1min":6.1050000000000004,"5min":6.1079999999999997,"15min":6.1079999999999997},"min":{"1min":0.86099999999999999,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":94.546999999999997,"5min":94.546999999999997,"15min":94.546999999999997},"last":1.3400000000000001}]},{"osd":10,"last update":"Wed Sep 18 10:16:06 2024","interfaces":[{"interface":"back","average":{"1min":1.4870000000000001,"5min":1.4870000000000001,"15min":1.4870000000000001},"min":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":2.0659999999999998,"5min":2.0659999999999998,"15min":2.0659999999999998},"last":1.647},{"interface":"front","average":{"1min":4.9349999999999996,"5min":4.9349999999999996,"15min":4.9349999999999996},"min":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":71.668999999999997,"5min":71.668999999999997,"15min":71.668999999999997},"last":1.4399999999999999}]},{"osd":13,"last update":"Wed Sep 18 10:16:37 2024","interfaces":[{"interface":"back","average":{"1min":9.4909999999999997,"5min":3.113,"15min":2.0499999999999998},"min":{"1min":1.04,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":91.290999999999997,"5min":91.290999999999997,"15min":91.290999999999997},"last":1.528},{"interface":"front","average":{"1min":1.5249999999999999,"5min":5.3300000000000001,"15min":5.9640000000000004},"min":{"1min":1.05,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.885,"5min":71.447000000000003,"15min":71.447000000000003},"last":1.5009999999999999}]},{"osd":14,"last update":"Wed Sep 18 10:16:37 2024","interfaces":[{"interface":"back","average":{"1min":1.6899999999999999,"5min":4.8899999999999997,"15min":5.423},"min":{"1min":1.0089999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"max":{"1min":4.625,"5min":56.664999999999999,"15min":56.664999999999999},"last":1.2230000000000001},{"interface":"front","average":{"1min":1.865,"5min":2.2629999999999999,"15min":2.3290000000000002},"min":{"1min":1.1859999999999999,"5min":0.93999999999999995,"15min":0.93999999999999995},"max":{"1min":4.5910000000000002,"5min":18.523,"15min":18.523},"last":1.47}]},{"osd":15,"last update":"Wed Sep 18 10:16:48 2024","interfaces":[{"interface":"back","average":{"1min":1.6970000000000001,"5min":4.0049999999999999,"15min":4.3899999999999997},"min":{"1min":1.01,"5min":0.91500000000000004,"15min":0.91500000000000004},"max":{"1min":4.3289999999999997,"5min":56.357999999999997,"15min":56.357999999999997},"last":1.4079999999999999},{"interface":"front","average":{"1min":1.7589999999999999,"5min":8.5310000000000006,"15min":9.6600000000000001},"min":{"1min":1.133,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":2.1110000000000002,"5min":122.727,"15min":122.727},"last":1.6970000000000001}]}]},{"osd":5,"up_from":22,"seq":94489280546,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303860,"kb_used_data":2344,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94067980,"statfs":{"total":96636764160,"available":96325611520,"internally_reserved":0,"allocated":2400256,"data_stored":1398209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[2,3,4,6,8,9,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:15:47 2024","interfaces":[{"interface":"back","average":{"1min":7.8860000000000001,"5min":7.8860000000000001,"15min":7.8860000000000001},"min":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":84.430999999999997,"5min":84.430999999999997,"15min":84.430999999999997},"last":1.2889999999999999},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":1.353}]},{"osd":2,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":4.1360000000000001,"5min":8.6940000000000008,"15min":9.4540000000000006},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":51.091000000000001,"5min":84.566000000000003,"15min":84.566000000000003},"last":0.93999999999999995},{"interface":"front","average":{"1min":0.98199999999999998,"5min":1.3300000000000001,"15min":1.3879999999999999},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.4690000000000001,"5min":9.5069999999999997,"15min":9.5069999999999997},"last":0.76900000000000002}]},{"osd":3,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":4.9240000000000004,"5min":7.8010000000000002,"15min":8.2810000000000006},"min":{"1min":0.69099999999999995,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":51.229999999999997,"5min":84.358999999999995,"15min":84.358999999999995},"last":1.097},{"interface":"front","average":{"1min":8.1590000000000007,"5min":8.2100000000000009,"15min":8.218},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":74.222999999999999,"5min":97.549999999999997,"15min":97.549999999999997},"last":1.0449999999999999}]},{"osd":4,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":25.673999999999999,"15min":29.777999999999999},"min":{"1min":0.67400000000000004,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.407,"5min":266.911,"15min":266.911},"last":1.0029999999999999},{"interface":"front","average":{"1min":21.388000000000002,"5min":22.515999999999998,"15min":22.704000000000001},"min":{"1min":0.76200000000000001,"5min":0.626,"15min":0.626},"max":{"1min":171.691,"5min":260.53500000000003,"15min":260.53500000000003},"last":0.70899999999999996}]},{"osd":6,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":3.6309999999999998,"5min":8.0809999999999995,"15min":8.8230000000000004},"min":{"1min":0.70099999999999996,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":41.886000000000003,"5min":58.171999999999997,"15min":58.171999999999997},"last":0.85099999999999998},{"interface":"front","average":{"1min":5.8079999999999998,"5min":18.274999999999999,"15min":20.353000000000002},"min":{"1min":0.85699999999999998,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":54.984999999999999,"5min":227.48099999999999,"15min":227.48099999999999},"last":1.2549999999999999}]},{"osd":7,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":12.013,"5min":12.013,"15min":12.013},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":123.111,"5min":123.111,"15min":123.111},"last":1.5249999999999999},{"interface":"front","average":{"1min":13.994,"5min":13.994,"15min":13.994},"min":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"max":{"1min":84.617000000000004,"5min":84.617000000000004,"15min":84.617000000000004},"last":1.329}]},{"osd":8,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":4.4720000000000004,"5min":17.175999999999998,"15min":19.292999999999999},"min":{"1min":0.879,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":51.506,"5min":260.44200000000001,"15min":260.44200000000001},"last":1.3169999999999999},{"interface":"front","average":{"1min":8.2609999999999992,"5min":7.2859999999999996,"15min":7.1239999999999997},"min":{"1min":0.83899999999999997,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":71.266000000000005,"5min":106.818,"15min":106.818},"last":1.2210000000000001}]},{"osd":9,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":3.8780000000000001,"5min":5.2930000000000001,"15min":5.5289999999999999},"min":{"1min":0.91300000000000003,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":41.652000000000001,"5min":80.272000000000006,"15min":80.272000000000006},"last":1.1259999999999999},{"interface":"front","average":{"1min":1.3799999999999999,"5min":1.764,"15min":1.8280000000000001},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":2.008,"5min":9.5690000000000008,"15min":9.5690000000000008},"last":1.48}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6479999999999999}]},{"osd":11,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":1.494,"5min":7.8120000000000003,"15min":8.8650000000000002},"min":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":1.825,"5min":123.158,"15min":123.158},"last":1.397},{"interface":"front","average":{"1min":8.8490000000000002,"5min":10.863,"15min":11.199},"min":{"1min":0.93200000000000005,"5min":0.86499999999999999,"15min":0.86499999999999999},"max":{"1min":68.938000000000002,"5min":84.522999999999996,"15min":84.522999999999996},"last":1.169}]},{"osd":12,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":4.0250000000000004,"5min":1.8580000000000001,"15min":1.4970000000000001},"min":{"1min":0.95199999999999996,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":41.929000000000002,"5min":41.929000000000002,"15min":41.929000000000002},"last":1.4590000000000001},{"interface":"front","average":{"1min":1.774,"5min":1.9339999999999999,"15min":1.9610000000000001},"min":{"1min":0.93999999999999995,"5min":0.88300000000000001,"15min":0.88300000000000001},"max":{"1min":4.1150000000000002,"5min":9.7710000000000008,"15min":9.7710000000000008},"last":1.5309999999999999}]},{"osd":13,"last update":"Wed Sep 18 10:16:48 2024","interfaces":[{"interface":"back","average":{"1min":1.6499999999999999,"5min":19.745999999999999,"15min":22.762},"min":{"1min":1.171,"5min":1.1120000000000001,"15min":1.1120000000000001},"max":{"1min":2.0550000000000002,"5min":140.273,"15min":140.273},"last":1.583},{"interface":"front","average":{"1min":4.665,"5min":20.309000000000001,"15min":22.917000000000002},"min":{"1min":1.194,"5min":0.98099999999999998,"15min":0.98099999999999998},"max":{"1min":51.432000000000002,"5min":170.86000000000001,"15min":170.86000000000001},"last":1.4359999999999999}]},{"osd":14,"last update":"Wed Sep 18 10:16:48 2024","interfaces":[{"interface":"back","average":{"1min":9.8930000000000007,"5min":12.584,"15min":13.032},"min":{"1min":1.385,"5min":1.0389999999999999,"15min":1.0389999999999999},"max":{"1min":93.295000000000002,"5min":101.733,"15min":101.733},"last":1.363},{"interface":"front","average":{"1min":1.79,"5min":7.9729999999999999,"15min":9.0030000000000001},"min":{"1min":1.321,"5min":0.90100000000000002,"15min":0.90100000000000002},"max":{"1min":2.2290000000000001,"5min":122.952,"15min":122.952},"last":1.627}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8839999999999999}]}]},{"osd":6,"up_from":22,"seq":94489280546,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304388,"kb_used_data":2864,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94067452,"statfs":{"total":96636764160,"available":96325070848,"internally_reserved":0,"allocated":2932736,"data_stored":1927137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,3,4,5,7,8,9,11,12,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:15:47 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.68200000000000005}]},{"osd":1,"last update":"Wed Sep 18 10:16:48 2024","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":3.8100000000000001,"15min":4.306},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.2150000000000001,"5min":52.332999999999998,"15min":52.332999999999998},"last":0.97599999999999998},{"interface":"front","average":{"1min":4.8369999999999997,"5min":1.6990000000000001,"15min":1.1759999999999999},"min":{"1min":0.68300000000000005,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":67.763000000000005,"5min":67.763000000000005,"15min":67.763000000000005},"last":0.95199999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.343}]},{"osd":3,"last update":"Wed Sep 18 10:16:48 2024","interfaces":[{"interface":"back","average":{"1min":5.3380000000000001,"5min":1.792,"15min":1.2010000000000001},"min":{"1min":0.65500000000000003,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":76.891999999999996,"5min":76.891999999999996,"15min":76.891999999999996},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.98999999999999999,"5min":11.403,"15min":13.138999999999999},"min":{"1min":0.751,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.204,"5min":169.244,"15min":169.244},"last":1.093}]},{"osd":4,"last update":"Wed Sep 18 10:16:02 2024","interfaces":[{"interface":"back","average":{"1min":11.111000000000001,"5min":11.111000000000001,"15min":11.111000000000001},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":128.053,"5min":128.053,"15min":128.053},"last":0.61699999999999999},{"interface":"front","average":{"1min":4.6189999999999998,"5min":4.6189999999999998,"15min":4.6189999999999998},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":67.960999999999999,"5min":67.960999999999999,"15min":67.960999999999999},"last":0.57399999999999995}]},{"osd":5,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":6.0579999999999998,"5min":10.194000000000001,"15min":10.884},"min":{"1min":0.72099999999999997,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":68.025999999999996,"5min":119.032,"15min":119.032},"last":0.82099999999999995},{"interface":"front","average":{"1min":5.6760000000000002,"5min":4.0380000000000003,"15min":3.7650000000000001},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":83.858000000000004,"5min":83.858000000000004,"15min":83.858000000000004},"last":0.91600000000000004}]},{"osd":7,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":4.9539999999999997,"5min":6.3959999999999999,"15min":6.6360000000000001},"min":{"1min":0.94099999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":69.171000000000006,"5min":69.171000000000006,"15min":69.171000000000006},"last":1.619},{"interface":"front","average":{"1min":5.0960000000000001,"5min":3.4630000000000001,"15min":3.1909999999999998},"min":{"1min":1.089,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":69.239000000000004,"5min":69.239000000000004,"15min":69.239000000000004},"last":1.0069999999999999}]},{"osd":8,"last update":"Wed Sep 18 10:16:51 2024","interfaces":[{"interface":"back","average":{"1min":5.3659999999999997,"5min":5.3659999999999997,"15min":5.3659999999999997},"min":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"max":{"1min":76.766000000000005,"5min":76.766000000000005,"15min":76.766000000000005},"last":1.3480000000000001},{"interface":"front","average":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"min":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"max":{"1min":1.704,"5min":1.704,"15min":1.704},"last":1.2430000000000001}]},{"osd":9,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":5.4269999999999996,"5min":7.7089999999999996,"15min":8.0890000000000004},"min":{"1min":0.95799999999999996,"5min":0.84599999999999997,"15min":0.84599999999999997},"max":{"1min":76.997,"5min":95.691999999999993,"15min":95.691999999999993},"last":1.048},{"interface":"front","average":{"1min":1.2949999999999999,"5min":10.472,"15min":12.002000000000001},"min":{"1min":0.96899999999999997,"5min":0.83099999999999996,"15min":0.83099999999999996},"max":{"1min":1.696,"5min":165.31100000000001,"15min":165.31100000000001},"last":1.4510000000000001}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.329}]},{"osd":11,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":5.0510000000000002,"5min":2.0630000000000002,"15min":1.5649999999999999},"min":{"1min":1.002,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":67.909999999999997,"5min":67.909999999999997,"15min":67.909999999999997},"last":1.3109999999999999},{"interface":"front","average":{"1min":9.1829999999999998,"5min":4.9429999999999996,"15min":4.2359999999999998},"min":{"1min":1.1870000000000001,"5min":0.92000000000000004,"15min":0.92000000000000004},"max":{"1min":77.158000000000001,"5min":77.158000000000001,"15min":77.158000000000001},"last":1.1899999999999999}]},{"osd":12,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":1.3859999999999999,"5min":4.3140000000000001,"15min":4.8019999999999996},"min":{"1min":1.079,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":1.796,"5min":52.378,"15min":52.378},"last":1.542},{"interface":"front","average":{"1min":5.1109999999999998,"5min":4.5720000000000001,"15min":4.4820000000000002},"min":{"1min":0.92800000000000005,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":67.835999999999999,"5min":67.835999999999999,"15min":67.835999999999999},"last":1.133}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2050000000000001}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2669999999999999}]},{"osd":15,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":5.1680000000000001,"5min":2.2120000000000002,"15min":1.72},"min":{"1min":1.1000000000000001,"5min":0.90300000000000002,"15min":0.90300000000000002},"max":{"1min":68.069999999999993,"5min":68.069999999999993,"15min":68.069999999999993},"last":1.387},{"interface":"front","average":{"1min":5.6200000000000001,"5min":4.3019999999999996,"15min":4.0819999999999999},"min":{"1min":1.2030000000000001,"5min":1.0429999999999999,"15min":1.0429999999999999},"max":{"1min":77.081999999999994,"5min":77.081999999999994,"15min":77.081999999999994},"last":1.5169999999999999}]}]},{"osd":7,"up_from":22,"seq":94489280546,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303860,"kb_used_data":2344,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94067980,"statfs":{"total":96636764160,"available":96325611520,"internally_reserved":0,"allocated":2400256,"data_stored":1398209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,4,6,8,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:16:33 2024","interfaces":[{"interface":"back","average":{"1min":3.9900000000000002,"5min":10.974,"15min":12.138},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":59.137999999999998,"5min":139.58600000000001,"15min":139.58600000000001},"last":0.872},{"interface":"front","average":{"1min":1.121,"5min":1.0229999999999999,"15min":1.0069999999999999},"min":{"1min":0.67800000000000005,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.7390000000000001,"5min":1.7390000000000001,"15min":1.7390000000000001},"last":0.91100000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2090000000000001}]},{"osd":2,"last update":"Wed Sep 18 10:15:46 2024","interfaces":[{"interface":"back","average":{"1min":15.436,"5min":15.436,"15min":15.436},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":139.36799999999999,"5min":139.36799999999999,"15min":139.36799999999999},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.94299999999999995}]},{"osd":3,"last update":"Wed Sep 18 10:15:46 2024","interfaces":[{"interface":"back","average":{"1min":17.327999999999999,"5min":17.327999999999999,"15min":17.327999999999999},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":139.27199999999999,"5min":139.27199999999999,"15min":139.27199999999999},"last":1.3460000000000001},{"interface":"front","average":{"1min":3.964,"5min":3.964,"15min":3.964},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":23.748999999999999,"5min":23.748999999999999,"15min":23.748999999999999},"last":0.82899999999999996}]},{"osd":4,"last update":"Wed Sep 18 10:16:37 2024","interfaces":[{"interface":"back","average":{"1min":9.8670000000000009,"5min":17.291,"15min":18.527999999999999},"min":{"1min":0.68000000000000005,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":125.10299999999999,"5min":139.51599999999999,"15min":139.51599999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":3.3559999999999999,"5min":4.6699999999999999,"15min":4.8890000000000002},"min":{"1min":0.68000000000000005,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":49.241999999999997,"5min":49.241999999999997,"15min":49.241999999999997},"last":0.64300000000000002}]},{"osd":5,"last update":"Wed Sep 18 10:15:46 2024","interfaces":[{"interface":"back","average":{"1min":1.099,"5min":1.099,"15min":1.099},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":2.3620000000000001,"5min":2.3620000000000001,"15min":2.3620000000000001},"last":1.4059999999999999},{"interface":"front","average":{"1min":17.736999999999998,"5min":17.736999999999998,"15min":17.736999999999998},"min":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"max":{"1min":139.44300000000001,"5min":139.44300000000001,"15min":139.44300000000001},"last":0.78000000000000003}]},{"osd":6,"last update":"Wed Sep 18 10:16:33 2024","interfaces":[{"interface":"back","average":{"1min":6.0869999999999997,"5min":15.117000000000001,"15min":16.622},"min":{"1min":0.998,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":59.343000000000004,"5min":183.001,"15min":183.001},"last":1.113},{"interface":"front","average":{"1min":7.0830000000000002,"5min":4.8650000000000002,"15min":4.4950000000000001},"min":{"1min":0.82399999999999995,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":58.284999999999997,"5min":58.284999999999997,"15min":58.284999999999997},"last":1.1599999999999999}]},{"osd":8,"last update":"Wed Sep 18 10:16:30 2024","interfaces":[{"interface":"back","average":{"1min":1.4079999999999999,"5min":1.2809999999999999,"15min":1.26},"min":{"1min":0.84999999999999998,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.9339999999999999,"5min":2.6139999999999999,"15min":2.6139999999999999},"last":0.78300000000000003},{"interface":"front","average":{"1min":5.3029999999999999,"5min":13.785,"15min":15.199},"min":{"1min":1.0980000000000001,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":59.523000000000003,"5min":139.798,"15min":139.798},"last":0.996}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.861}]},{"osd":10,"last update":"Wed Sep 18 10:16:30 2024","interfaces":[{"interface":"back","average":{"1min":16.035,"5min":8.4100000000000001,"15min":7.1390000000000002},"min":{"1min":1.0149999999999999,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":156.45500000000001,"5min":156.45500000000001,"15min":156.45500000000001},"last":0.82699999999999996},{"interface":"front","average":{"1min":1.4910000000000001,"5min":1.391,"15min":1.3740000000000001},"min":{"1min":0.83499999999999996,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":2.1200000000000001,"5min":2.254,"15min":2.254},"last":1.139}]},{"osd":11,"last update":"Wed Sep 18 10:16:33 2024","interfaces":[{"interface":"back","average":{"1min":5.0460000000000003,"5min":5.4459999999999997,"15min":5.5129999999999999},"min":{"1min":1.046,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":49.154000000000003,"5min":62.356000000000002,"15min":62.356000000000002},"last":1.252},{"interface":"front","average":{"1min":2.4769999999999999,"5min":1.6240000000000001,"15min":1.482},"min":{"1min":1.0189999999999999,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":20.449999999999999,"5min":20.449999999999999,"15min":20.449999999999999},"last":1.0329999999999999}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96599999999999997}]},{"osd":13,"last update":"Wed Sep 18 10:16:33 2024","interfaces":[{"interface":"back","average":{"1min":10.028,"5min":16.707999999999998,"15min":17.821999999999999},"min":{"1min":1.1459999999999999,"5min":0.874,"15min":0.874},"max":{"1min":113.428,"5min":139.655,"15min":139.655},"last":1.2010000000000001},{"interface":"front","average":{"1min":5.1970000000000001,"5min":16.942,"15min":18.899999999999999},"min":{"1min":1.077,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":49.457999999999998,"5min":173.113,"15min":173.113},"last":1.0600000000000001}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.331}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.226}]}]},{"osd":8,"up_from":22,"seq":94489280546,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303868,"kb_used_data":2344,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94067972,"statfs":{"total":96636764160,"available":96325603328,"internally_reserved":0,"allocated":2400256,"data_stored":1398209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[2,4,6,7,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93300000000000005}]},{"osd":1,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":1.117,"5min":1.117,"15min":1.117},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.669,"5min":1.669,"15min":1.669},"last":1.2230000000000001},{"interface":"front","average":{"1min":2.3290000000000002,"5min":2.3290000000000002,"15min":2.3290000000000002},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":29.523,"5min":29.523,"15min":29.523},"last":1.1279999999999999}]},{"osd":2,"last update":"Wed Sep 18 10:16:48 2024","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":5.5140000000000002,"15min":6.2300000000000004},"min":{"1min":0.93500000000000005,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.8,"5min":77.435000000000002,"15min":77.435000000000002},"last":1.0049999999999999},{"interface":"front","average":{"1min":1.1639999999999999,"5min":0.97099999999999997,"15min":0.93899999999999995},"min":{"1min":0.71999999999999997,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.903,"5min":1.903,"15min":1.903},"last":0.77000000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.306}]},{"osd":4,"last update":"Wed Sep 18 10:16:48 2024","interfaces":[{"interface":"back","average":{"1min":1.23,"5min":9.5879999999999992,"15min":10.981},"min":{"1min":0.71899999999999997,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":7.1559999999999997,"5min":90.438000000000002,"15min":90.438000000000002},"last":1.1220000000000001},{"interface":"front","average":{"1min":1.1699999999999999,"5min":10.691000000000001,"15min":12.278},"min":{"1min":0.66100000000000003,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":2.1309999999999998,"5min":93.939999999999998,"15min":93.939999999999998},"last":0.82799999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4199999999999999}]},{"osd":6,"last update":"Wed Sep 18 10:16:48 2024","interfaces":[{"interface":"back","average":{"1min":1.359,"5min":15.042999999999999,"15min":17.324000000000002},"min":{"1min":0.90700000000000003,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":2.0350000000000001,"5min":293.26900000000001,"15min":293.26900000000001},"last":0.86399999999999999},{"interface":"front","average":{"1min":1.2230000000000001,"5min":5.1689999999999996,"15min":5.827},"min":{"1min":0.88900000000000001,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.839,"5min":70.203000000000003,"15min":70.203000000000003},"last":1.038}]},{"osd":7,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":7.1950000000000003,"5min":3.6070000000000002,"15min":3.0089999999999999},"min":{"1min":0.80900000000000005,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":80.635000000000005,"5min":80.635000000000005,"15min":80.635000000000005},"last":1.0940000000000001},{"interface":"front","average":{"1min":9.3490000000000002,"5min":6.726,"15min":6.2889999999999997},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":99.060000000000002,"5min":99.060000000000002,"15min":99.060000000000002},"last":1.069}]},{"osd":9,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":4.8730000000000002,"5min":3.0110000000000001,"15min":2.7010000000000001},"min":{"1min":0.92700000000000005,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":80.076999999999998,"5min":80.076999999999998,"15min":80.076999999999998},"last":0.88500000000000001},{"interface":"front","average":{"1min":1.6240000000000001,"5min":7.4489999999999998,"15min":8.4199999999999999},"min":{"1min":1.196,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":2.5190000000000001,"5min":98.914000000000001,"15min":98.914000000000001},"last":1.1639999999999999}]},{"osd":10,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":1.6319999999999999,"5min":5.8559999999999999,"15min":6.5599999999999996},"min":{"1min":0.99099999999999999,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":2.3679999999999999,"5min":90.751000000000005,"15min":90.751000000000005},"last":0.96399999999999997},{"interface":"front","average":{"1min":1.591,"5min":7.7889999999999997,"15min":8.8219999999999992},"min":{"1min":1.0640000000000001,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":2.3340000000000001,"5min":83.795000000000002,"15min":83.795000000000002},"last":1.204}]},{"osd":11,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":1.7450000000000001,"5min":2.669,"15min":2.823},"min":{"1min":1.3340000000000001,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":2.383,"5min":26.462,"15min":26.462},"last":1.2230000000000001},{"interface":"front","average":{"1min":1.752,"5min":1.948,"15min":1.9810000000000001},"min":{"1min":1.24,"5min":0.99299999999999999,"15min":0.99299999999999999},"max":{"1min":2.3479999999999999,"5min":12.475,"15min":12.475},"last":1.3009999999999999}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0760000000000001}]},{"osd":13,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":3.052,"5min":7.9329999999999998,"15min":8.7469999999999999},"min":{"1min":1.05,"5min":1.0429999999999999,"15min":1.0429999999999999},"max":{"1min":30.582999999999998,"5min":94.222999999999999,"15min":94.222999999999999},"last":1.151},{"interface":"front","average":{"1min":1.635,"5min":2.5059999999999998,"15min":2.6509999999999998},"min":{"1min":1.1180000000000001,"5min":0.93400000000000005,"15min":0.93400000000000005},"max":{"1min":2.4689999999999999,"5min":23.983000000000001,"15min":23.983000000000001},"last":1.337}]},{"osd":14,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":1.802,"5min":16.975000000000001,"15min":19.504000000000001},"min":{"1min":1.28,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":2.4500000000000002,"5min":150.93600000000001,"15min":150.93600000000001},"last":1.486},{"interface":"front","average":{"1min":4.8170000000000002,"5min":3.8410000000000002,"15min":3.6789999999999998},"min":{"1min":1.2210000000000001,"5min":0.90200000000000002,"15min":0.90200000000000002},"max":{"1min":69.918000000000006,"5min":69.918000000000006,"15min":69.918000000000006},"last":1.3740000000000001}]},{"osd":15,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":5.9089999999999998,"5min":3.4169999999999998,"15min":3.0019999999999998},"min":{"1min":1.2549999999999999,"5min":0.93300000000000005,"15min":0.93300000000000005},"max":{"1min":98.558000000000007,"5min":98.558000000000007,"15min":98.558000000000007},"last":1.419},{"interface":"front","average":{"1min":1.8400000000000001,"5min":2.8119999999999998,"15min":2.9740000000000002},"min":{"1min":1.3160000000000001,"5min":0.95299999999999996,"15min":0.95299999999999996},"max":{"1min":2.5859999999999999,"5min":26.414000000000001,"15min":26.414000000000001},"last":1.397}]}]},{"osd":10,"up_from":22,"seq":94489280547,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303860,"kb_used_data":2344,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94067980,"statfs":{"total":96636764160,"available":96325611520,"internally_reserved":0,"allocated":2400256,"data_stored":1398209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,4,5,6,7,9,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:16:29 2024","interfaces":[{"interface":"back","average":{"1min":16.117000000000001,"5min":16.117000000000001,"15min":16.117000000000001},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":145.12799999999999,"5min":145.12799999999999,"15min":145.12799999999999},"last":1.401},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.431,"5min":1.431,"15min":1.431},"last":1.0780000000000001}]},{"osd":1,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":9.5719999999999992,"5min":10.66,"15min":10.842000000000001},"min":{"1min":0.88500000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":145.20699999999999,"5min":145.20699999999999,"15min":145.20699999999999},"last":1.093},{"interface":"front","average":{"1min":4.7510000000000003,"5min":14.728,"15min":16.390999999999998},"min":{"1min":0.754,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":38.338000000000001,"5min":99.995999999999995,"15min":99.995999999999995},"last":0.75}]},{"osd":2,"last update":"Wed Sep 18 10:15:33 2024","interfaces":[{"interface":"back","average":{"1min":10.893000000000001,"5min":10.893000000000001,"15min":10.893000000000001},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":84.072000000000003,"5min":84.072000000000003,"15min":84.072000000000003},"last":0.99199999999999999},{"interface":"front","average":{"1min":11.550000000000001,"5min":11.550000000000001,"15min":11.550000000000001},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":99.798000000000002,"5min":99.798000000000002,"15min":99.798000000000002},"last":1.425}]},{"osd":3,"last update":"Wed Sep 18 10:16:22 2024","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":1.119,"15min":1.119},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.99,"5min":1.99,"15min":1.99},"last":1.5},{"interface":"front","average":{"1min":18.094999999999999,"5min":18.094999999999999,"15min":18.094999999999999},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":145.535,"5min":145.535,"15min":145.535},"last":1.4510000000000001}]},{"osd":4,"last update":"Wed Sep 18 10:16:49 2024","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":20.715,"15min":23.963999999999999},"min":{"1min":0.92300000000000004,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.5820000000000001,"5min":125.23099999999999,"15min":125.23099999999999},"last":1.026},{"interface":"front","average":{"1min":1.206,"5min":25.175000000000001,"15min":29.170000000000002},"min":{"1min":0.88800000000000001,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.4930000000000001,"5min":295.93200000000002,"15min":295.93200000000002},"last":0.96499999999999997}]},{"osd":5,"last update":"Wed Sep 18 10:16:49 2024","interfaces":[{"interface":"back","average":{"1min":8.9960000000000004,"5min":21.460999999999999,"15min":23.539000000000001},"min":{"1min":0.81000000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":145.24000000000001,"5min":215.51400000000001,"15min":215.51400000000001},"last":0.79000000000000004},{"interface":"front","average":{"1min":2.4140000000000001,"5min":22.346,"15min":25.667999999999999},"min":{"1min":0.85399999999999998,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":20.302,"5min":252.69300000000001,"15min":252.69300000000001},"last":1.198}]},{"osd":6,"last update":"Wed Sep 18 10:16:22 2024","interfaces":[{"interface":"back","average":{"1min":12.393000000000001,"5min":12.393000000000001,"15min":12.393000000000001},"min":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"max":{"1min":111.173,"5min":111.173,"15min":111.173},"last":1.2689999999999999},{"interface":"front","average":{"1min":15.771000000000001,"5min":15.771000000000001,"15min":15.771000000000001},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":221.13900000000001,"5min":221.13900000000001,"15min":221.13900000000001},"last":1.167}]},{"osd":7,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":1.5109999999999999,"5min":2.7490000000000001,"15min":2.9550000000000001},"min":{"1min":0.95699999999999996,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.919,"5min":34.558,"15min":34.558},"last":1.224},{"interface":"front","average":{"1min":10.101000000000001,"5min":13.186999999999999,"15min":13.701000000000001},"min":{"1min":1.0920000000000001,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":145.5,"5min":145.5,"15min":145.5},"last":1.2909999999999999}]},{"osd":9,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":8.3640000000000008,"5min":11.07,"15min":11.521000000000001},"min":{"1min":1.0349999999999999,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":119.202,"5min":119.202,"15min":119.202},"last":1.3169999999999999},{"interface":"front","average":{"1min":1.629,"5min":10.364000000000001,"15min":11.82},"min":{"1min":1.073,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":2.23,"5min":97.385999999999996,"15min":97.385999999999996},"last":1.4450000000000001}]},{"osd":11,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":10.132999999999999,"5min":11.489000000000001,"15min":11.715},"min":{"1min":1.286,"5min":0.85699999999999998,"15min":0.85699999999999998},"max":{"1min":145.26900000000001,"5min":145.26900000000001,"15min":145.26900000000001},"last":1.3759999999999999},{"interface":"front","average":{"1min":5.6719999999999997,"5min":19.837,"15min":22.198},"min":{"1min":1.28,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":44.121000000000002,"5min":295.70400000000001,"15min":295.70400000000001},"last":1.399}]},{"osd":12,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":1.6559999999999999,"5min":2.8250000000000002,"15min":3.02},"min":{"1min":1.0620000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":2.2559999999999998,"5min":34.625999999999998,"15min":34.625999999999998},"last":1.341},{"interface":"front","average":{"1min":10.140000000000001,"5min":10.986000000000001,"15min":11.127000000000001},"min":{"1min":1.198,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":145.358,"5min":145.358,"15min":145.358},"last":1.423}]},{"osd":13,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":10.433999999999999,"5min":17.994,"15min":19.254000000000001},"min":{"1min":1.1990000000000001,"5min":0.877,"15min":0.877},"max":{"1min":123.696,"5min":123.696,"15min":123.696},"last":1.5629999999999999},{"interface":"front","average":{"1min":7.6379999999999999,"5min":14.551,"15min":15.702999999999999},"min":{"1min":1.163,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":68.363,"5min":85.879999999999995,"15min":85.879999999999995},"last":1.544}]},{"osd":14,"last update":"Wed Sep 18 10:16:22 2024","interfaces":[{"interface":"back","average":{"1min":15.864000000000001,"5min":15.864000000000001,"15min":15.864000000000001},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":145.46700000000001,"5min":145.46700000000001,"15min":145.46700000000001},"last":1.5209999999999999},{"interface":"front","average":{"1min":13.93,"5min":13.93,"15min":13.93},"min":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"max":{"1min":100.092,"5min":100.092,"15min":100.092},"last":1.4770000000000001}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1080000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T10:17:06.776 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-09-18T10:16:18.085027+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=10, tm_min=16, tm_sec=18, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=10, tm_min=16, tm_sec=57, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T10:17:06.777 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-09-18T10:15:50.883147+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=10, tm_min=15, tm_sec=50, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=10, tm_min=16, tm_sec=57, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T10:17:06.777 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-09-18T10:16:20.864635+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=10, tm_min=16, tm_sec=20, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=10, tm_min=16, tm_sec=57, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T10:17:06.777 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-09-18T10:16:15.061892+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=10, tm_min=16, tm_sec=15, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=10, tm_min=16, tm_sec=57, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T10:17:06.777 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-09-18T10:16:06.900601+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=10, tm_min=16, tm_sec=6, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=10, tm_min=16, tm_sec=57, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T10:17:06.777 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-18T10:17:26.778 DEBUG:teuthology.orchestra.run.smithi029:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T10:17:27.106 INFO:teuthology.orchestra.run.smithi029.stdout: 2024-09-18T10:17:27.109 INFO:teuthology.orchestra.run.smithi029.stderr:dumped all 2024-09-18T10:17:27.121 INFO:teuthology.orchestra.run.smithi029.stdout:{"pg_ready":true,"pg_map":{"version":235,"stamp":"2024-09-18T10:17:25.248316+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":147,"num_write_kb":2674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":14,"kb":1509949440,"kb_used":4868540,"kb_used_data":40088,"kb_used_omap":26,"kb_used_meta":4828197,"kb_avail":1505080900,"statfs":{"total":1546188226560,"available":1541202841600,"internally_reserved":0,"allocated":41050112,"data_stored":24343574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":27264,"internal_metadata":4944074112},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.905141"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":167,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-18T10:17:13.120424+0000","last_change":"2024-09-18T10:17:03.229649+0000","last_active":"2024-09-18T10:17:13.120424+0000","last_peered":"2024-09-18T10:17:13.120424+0000","last_clean":"2024-09-18T10:17:13.120424+0000","last_became_active":"2024-09-18T10:14:35.348943+0000","last_became_peered":"2024-09-18T10:14:35.348943+0000","last_unstale":"2024-09-18T10:17:13.120424+0000","last_undegraded":"2024-09-18T10:17:13.120424+0000","last_fullsized":"2024-09-18T10:17:13.120424+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:17:03.229435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:17:03.229435+0000","last_clean_scrub_stamp":"2024-09-18T10:17:03.229435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:18:14.550573+0000","scrub_duration":0.40426226799999998,"objects_trimmed":0,"snaptrim_duration":0.00060442999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,10],"acting":[13,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":99,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-18T10:17:11.406858+0000","last_change":"2024-09-18T10:17:00.624044+0000","last_active":"2024-09-18T10:17:11.406858+0000","last_peered":"2024-09-18T10:17:11.406858+0000","last_clean":"2024-09-18T10:17:11.406858+0000","last_became_active":"2024-09-18T10:15:50.978055+0000","last_became_peered":"2024-09-18T10:15:50.978055+0000","last_unstale":"2024-09-18T10:17:11.406858+0000","last_undegraded":"2024-09-18T10:17:11.406858+0000","last_fullsized":"2024-09-18T10:17:11.406858+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:17:00.623856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:17:00.623856+0000","last_clean_scrub_stamp":"2024-09-18T10:17:00.623856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:18:11.013972+0000","scrub_duration":0.003444309,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11],"acting":[8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":162,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-18T10:17:12.270834+0000","last_change":"2024-09-18T10:17:06.139234+0000","last_active":"2024-09-18T10:17:12.270834+0000","last_peered":"2024-09-18T10:17:12.270834+0000","last_clean":"2024-09-18T10:17:12.270834+0000","last_became_active":"2024-09-18T10:14:35.329198+0000","last_became_peered":"2024-09-18T10:14:35.329198+0000","last_unstale":"2024-09-18T10:17:12.270834+0000","last_undegraded":"2024-09-18T10:17:12.270834+0000","last_fullsized":"2024-09-18T10:17:12.270834+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:17:06.138986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:17:06.138986+0000","last_clean_scrub_stamp":"2024-09-18T10:17:06.138986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:18:21.124661+0000","scrub_duration":0.20438695300000001,"objects_trimmed":0,"snaptrim_duration":0.00062415699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":165,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-18T10:17:13.244811+0000","last_change":"2024-09-18T10:17:00.877328+0000","last_active":"2024-09-18T10:17:13.244811+0000","last_peered":"2024-09-18T10:17:13.244811+0000","last_clean":"2024-09-18T10:17:13.244811+0000","last_became_active":"2024-09-18T10:14:35.524102+0000","last_became_peered":"2024-09-18T10:14:35.524102+0000","last_unstale":"2024-09-18T10:17:13.244811+0000","last_undegraded":"2024-09-18T10:17:13.244811+0000","last_fullsized":"2024-09-18T10:17:13.244811+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:17:00.877142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:17:00.877142+0000","last_clean_scrub_stamp":"2024-09-18T10:17:00.877142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:18:19.548036+0000","scrub_duration":0.30330662200000003,"objects_trimmed":0,"snaptrim_duration":0.10077985,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"155'3","reported_seq":169,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-18T10:17:13.120401+0000","last_change":"2024-09-18T10:17:04.168794+0000","last_active":"2024-09-18T10:17:13.120401+0000","last_peered":"2024-09-18T10:17:13.120401+0000","last_clean":"2024-09-18T10:17:13.120401+0000","last_became_active":"2024-09-18T10:14:35.715374+0000","last_became_peered":"2024-09-18T10:14:35.715374+0000","last_unstale":"2024-09-18T10:17:13.120401+0000","last_undegraded":"2024-09-18T10:17:13.120401+0000","last_fullsized":"2024-09-18T10:17:13.120401+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"155'3","last_scrub_stamp":"2024-09-18T10:17:04.168606+0000","last_deep_scrub":"155'3","last_deep_scrub_stamp":"2024-09-18T10:17:04.168606+0000","last_clean_scrub_stamp":"2024-09-18T10:17:04.168606+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:18:26.959261+0000","scrub_duration":0.30750941700000001,"objects_trimmed":0,"snaptrim_duration":0.002421224,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,12],"acting":[13,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":173,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-18T10:17:19.755584+0000","last_change":"2024-09-18T10:17:19.755486+0000","last_active":"2024-09-18T10:17:19.755584+0000","last_peered":"2024-09-18T10:17:19.755584+0000","last_clean":"2024-09-18T10:17:19.755584+0000","last_became_active":"2024-09-18T10:14:35.348262+0000","last_became_peered":"2024-09-18T10:14:35.348262+0000","last_unstale":"2024-09-18T10:17:19.755584+0000","last_undegraded":"2024-09-18T10:17:19.755584+0000","last_fullsized":"2024-09-18T10:17:19.755584+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:17:19.755218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:17:19.755218+0000","last_clean_scrub_stamp":"2024-09-18T10:17:19.755218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:18:30.864058+0000","scrub_duration":0.203837711,"objects_trimmed":0,"snaptrim_duration":0.00063430400000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,0],"acting":[14,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":166,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-18T10:17:13.120199+0000","last_change":"2024-09-18T10:17:03.091916+0000","last_active":"2024-09-18T10:17:13.120199+0000","last_peered":"2024-09-18T10:17:13.120199+0000","last_clean":"2024-09-18T10:17:13.120199+0000","last_became_active":"2024-09-18T10:14:35.349203+0000","last_became_peered":"2024-09-18T10:14:35.349203+0000","last_unstale":"2024-09-18T10:17:13.120199+0000","last_undegraded":"2024-09-18T10:17:13.120199+0000","last_fullsized":"2024-09-18T10:17:13.120199+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:17:03.091716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:17:03.091716+0000","last_clean_scrub_stamp":"2024-09-18T10:17:03.091716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:18:13.646830+0000","scrub_duration":0.203921877,"objects_trimmed":0,"snaptrim_duration":0.00060130600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3],"acting":[9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"159'2","reported_seq":169,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-18T10:17:11.438271+0000","last_change":"2024-09-18T10:17:08.754080+0000","last_active":"2024-09-18T10:17:11.438271+0000","last_peered":"2024-09-18T10:17:11.438271+0000","last_clean":"2024-09-18T10:17:11.438271+0000","last_became_active":"2024-09-18T10:14:35.487746+0000","last_became_peered":"2024-09-18T10:14:35.487746+0000","last_unstale":"2024-09-18T10:17:11.438271+0000","last_undegraded":"2024-09-18T10:17:11.438271+0000","last_fullsized":"2024-09-18T10:17:11.438271+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"159'2","last_scrub_stamp":"2024-09-18T10:17:08.753880+0000","last_deep_scrub":"159'2","last_deep_scrub_stamp":"2024-09-18T10:17:08.753880+0000","last_clean_scrub_stamp":"2024-09-18T10:17:08.753880+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:18:22.795594+0000","scrub_duration":0.20914255900000001,"objects_trimmed":0,"snaptrim_duration":0.10072764100000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,5],"acting":[14,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"1.0","version":"156'114","reported_seq":300,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-18T10:17:11.438102+0000","last_change":"2024-09-18T10:17:04.855527+0000","last_active":"2024-09-18T10:17:11.438102+0000","last_peered":"2024-09-18T10:17:11.438102+0000","last_clean":"2024-09-18T10:17:11.438102+0000","last_became_active":"2024-09-18T10:14:32.508037+0000","last_became_peered":"2024-09-18T10:14:32.508037+0000","last_unstale":"2024-09-18T10:17:11.438102+0000","last_undegraded":"2024-09-18T10:17:11.438102+0000","last_fullsized":"2024-09-18T10:17:11.438102+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"156'114","last_scrub_stamp":"2024-09-18T10:17:04.855302+0000","last_deep_scrub":"156'114","last_deep_scrub_stamp":"2024-09-18T10:17:04.855302+0000","last_clean_scrub_stamp":"2024-09-18T10:17:04.855302+0000","objects_scrubbed":2,"log_size":114,"log_dups_size":0,"ondisk_log_size":114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:18:08.277089+0000","scrub_duration":0.71041728699999995,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,15],"acting":[6,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":13},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":114,"ondisk_log_size":114,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":15,"up_from":22,"seq":94489280549,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304480,"kb_used_data":2960,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94067360,"statfs":{"total":96636764160,"available":96324976640,"internally_reserved":0,"allocated":3031040,"data_stored":1984283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,4,6,7,8,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:16:32 2024","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":0.83899999999999997,"15min":0.81899999999999995},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":2.0019999999999998,"5min":2.0019999999999998,"15min":2.0019999999999998},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.996,"5min":5.5739999999999998,"15min":6.3369999999999997},"min":{"1min":0.72899999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.506,"5min":101.291,"15min":101.291},"last":1.2569999999999999}]},{"osd":1,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.93300000000000005},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.931,"5min":1.931,"15min":1.931},"last":1.1279999999999999}]},{"osd":2,"last update":"Wed Sep 18 10:16:32 2024","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.0169999999999999,"15min":0.995},"min":{"1min":0.79700000000000004,"5min":0.5,"15min":0.5},"max":{"1min":1.7470000000000001,"5min":1.7470000000000001,"15min":1.7470000000000001},"last":1.028},{"interface":"front","average":{"1min":1.1419999999999999,"5min":4.6589999999999998,"15min":5.2450000000000001},"min":{"1min":0.82899999999999996,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.4379999999999999,"5min":78.242999999999995,"15min":78.242999999999995},"last":1.252}]},{"osd":3,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":3.3029999999999999,"5min":3.3029999999999999,"15min":3.3029999999999999},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":41.720999999999997,"5min":41.720999999999997,"15min":41.720999999999997},"last":0.55400000000000005},{"interface":"front","average":{"1min":1.014,"5min":1.014,"15min":1.014},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.49,"5min":1.49,"15min":1.49},"last":0.89500000000000002}]},{"osd":4,"last update":"Wed Sep 18 10:16:32 2024","interfaces":[{"interface":"back","average":{"1min":1.3580000000000001,"5min":9.0879999999999992,"15min":10.375999999999999},"min":{"1min":0.747,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":8.0449999999999999,"5min":99.745999999999995,"15min":99.745999999999995},"last":0.81599999999999995},{"interface":"front","average":{"1min":1.595,"5min":9.2309999999999999,"15min":10.503},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":8.0679999999999996,"5min":99.781000000000006,"15min":99.781000000000006},"last":1.0169999999999999}]},{"osd":5,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":1.0860000000000001},{"interface":"front","average":{"1min":1.7869999999999999,"5min":1.7869999999999999,"15min":1.7869999999999999},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":10.874000000000001,"5min":10.874000000000001,"15min":10.874000000000001},"last":1.163}]},{"osd":6,"last update":"Wed Sep 18 10:16:32 2024","interfaces":[{"interface":"back","average":{"1min":14.140000000000001,"5min":4.4039999999999999,"15min":2.782},"min":{"1min":0.92900000000000005,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":269.80500000000001,"5min":269.80500000000001,"15min":269.80500000000001},"last":1.0760000000000001},{"interface":"front","average":{"1min":1.361,"5min":4.7610000000000001,"15min":5.327},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":2.012,"5min":74.156000000000006,"15min":74.156000000000006},"last":0.872}]},{"osd":7,"last update":"Wed Sep 18 10:16:53 2024","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.304,"15min":1.304},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.9590000000000001,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":0.93500000000000005},{"interface":"front","average":{"1min":4.5819999999999999,"5min":4.5819999999999999,"15min":4.5819999999999999},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":70.611000000000004,"5min":70.611000000000004,"15min":70.611000000000004},"last":1.1140000000000001}]},{"osd":8,"last update":"Wed Sep 18 10:16:53 2024","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.234,"15min":1.234},"min":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"max":{"1min":1.855,"5min":1.855,"15min":1.855},"last":0.996},{"interface":"front","average":{"1min":1.5629999999999999,"5min":1.5629999999999999,"15min":1.5629999999999999},"min":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"max":{"1min":2.2280000000000002,"5min":2.2280000000000002,"15min":2.2280000000000002},"last":1.2130000000000001}]},{"osd":10,"last update":"Wed Sep 18 10:16:32 2024","interfaces":[{"interface":"back","average":{"1min":5.8639999999999999,"5min":6.2670000000000003,"15min":6.3339999999999996},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":93.704999999999998,"5min":93.704999999999998,"15min":93.704999999999998},"last":1.3540000000000001},{"interface":"front","average":{"1min":5.7770000000000001,"5min":6.2549999999999999,"15min":6.335},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":93.671000000000006,"5min":93.671000000000006,"15min":93.671000000000006},"last":1.1000000000000001}]},{"osd":11,"last update":"Wed Sep 18 10:17:15 2024","interfaces":[{"interface":"back","average":{"1min":1.5049999999999999,"5min":1.5049999999999999,"15min":1.5049999999999999},"min":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"max":{"1min":1.915,"5min":1.915,"15min":1.915},"last":1.4330000000000001},{"interface":"front","average":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":2.2890000000000001,"5min":2.2890000000000001,"15min":2.2890000000000001},"last":1.0449999999999999}]},{"osd":12,"last update":"Wed Sep 18 10:16:53 2024","interfaces":[{"interface":"back","average":{"1min":1.6060000000000001,"5min":1.6060000000000001,"15min":1.6060000000000001},"min":{"1min":1.077,"5min":1.077,"15min":1.077},"max":{"1min":2.145,"5min":2.145,"15min":2.145},"last":1.5089999999999999},{"interface":"front","average":{"1min":1.577,"5min":1.577,"15min":1.577},"min":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"max":{"1min":2.1819999999999999,"5min":2.1819999999999999,"15min":2.1819999999999999},"last":1.3149999999999999}]},{"osd":13,"last update":"Wed Sep 18 10:16:32 2024","interfaces":[{"interface":"back","average":{"1min":1.99,"5min":5.2629999999999999,"15min":5.8090000000000002},"min":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"max":{"1min":7.3179999999999996,"5min":80.197000000000003,"15min":80.197000000000003},"last":1.339},{"interface":"front","average":{"1min":3.4830000000000001,"5min":1.8080000000000001,"15min":1.5289999999999999},"min":{"1min":0.89400000000000002,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":25.193999999999999,"5min":25.193999999999999,"15min":25.193999999999999},"last":1.5780000000000001}]},{"osd":14,"last update":"Wed Sep 18 10:16:32 2024","interfaces":[{"interface":"back","average":{"1min":10.725,"5min":3.2999999999999998,"15min":2.0619999999999998},"min":{"1min":0.95699999999999996,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":117.562,"5min":117.562,"15min":117.562},"last":1.387},{"interface":"front","average":{"1min":4.6139999999999999,"5min":9.5229999999999997,"15min":10.340999999999999},"min":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"max":{"1min":61.567,"5min":85.006,"15min":85.006},"last":1.2949999999999999}]}]},{"osd":14,"up_from":22,"seq":94489280549,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303952,"kb_used_data":2440,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94067888,"statfs":{"total":96636764160,"available":96325517312,"internally_reserved":0,"allocated":2498560,"data_stored":1455355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,5,7,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:16:32 2024","interfaces":[{"interface":"back","average":{"1min":1.0009999999999999,"5min":12.513,"15min":14.430999999999999},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.405,"5min":189.88800000000001,"15min":189.88800000000001},"last":0.64600000000000002},{"interface":"front","average":{"1min":7.0330000000000004,"5min":17.033000000000001,"15min":18.699000000000002},"min":{"1min":0.77300000000000002,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":71.900999999999996,"5min":180.09999999999999,"15min":180.09999999999999},"last":0.60799999999999998}]},{"osd":1,"last update":"Wed Sep 18 10:16:32 2024","interfaces":[{"interface":"back","average":{"1min":2.1040000000000001,"5min":7.7670000000000003,"15min":8.7110000000000003},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":21.334,"5min":79.626000000000005,"15min":79.626000000000005},"last":0.73699999999999999},{"interface":"front","average":{"1min":1.1479999999999999,"5min":12.601000000000001,"15min":14.51},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.8879999999999999,"5min":189.762,"15min":189.762},"last":0.89800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.417}]},{"osd":3,"last update":"Wed Sep 18 10:16:32 2024","interfaces":[{"interface":"back","average":{"1min":3.5430000000000001,"5min":12.906000000000001,"15min":14.465999999999999},"min":{"1min":0.84799999999999998,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":44.683,"5min":180.19900000000001,"15min":180.19900000000001},"last":1.7529999999999999},{"interface":"front","average":{"1min":5.3520000000000003,"5min":18.143999999999998,"15min":20.276},"min":{"1min":0.86399999999999999,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":42.570999999999998,"5min":180.851,"15min":180.851},"last":1.1899999999999999}]},{"osd":4,"last update":"Wed Sep 18 10:16:32 2024","interfaces":[{"interface":"back","average":{"1min":6.8360000000000003,"5min":21.687000000000001,"15min":24.161999999999999},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":71.911000000000001,"5min":189.809,"15min":189.809},"last":2.0249999999999999},{"interface":"front","average":{"1min":14.032999999999999,"5min":15.031000000000001,"15min":15.196999999999999},"min":{"1min":0.76100000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":133.244,"5min":180.31700000000001,"15min":180.31700000000001},"last":1.9350000000000001}]},{"osd":5,"last update":"Wed Sep 18 10:16:32 2024","interfaces":[{"interface":"back","average":{"1min":1.831,"5min":21.507000000000001,"15min":24.786000000000001},"min":{"1min":0.66300000000000003,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":13.42,"5min":277.41199999999998,"15min":277.41199999999998},"last":1.2689999999999999},{"interface":"front","average":{"1min":1.9530000000000001,"5min":12.936999999999999,"15min":14.766999999999999},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":14.288,"5min":189.64699999999999,"15min":189.64699999999999},"last":1.1439999999999999}]},{"osd":7,"last update":"Wed Sep 18 10:16:32 2024","interfaces":[{"interface":"back","average":{"1min":6.29,"5min":13.625999999999999,"15min":14.848000000000001},"min":{"1min":0.995,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":49.378,"5min":180.268,"15min":180.268},"last":0.88500000000000001},{"interface":"front","average":{"1min":3.8929999999999998,"5min":12.901,"15min":14.403},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":49.402999999999999,"5min":180.631,"15min":180.631},"last":0.95299999999999996}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1619999999999999}]},{"osd":9,"last update":"Wed Sep 18 10:17:21 2024","interfaces":[{"interface":"back","average":{"1min":1.363,"5min":3.472,"15min":3.8239999999999998},"min":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"max":{"1min":2.7040000000000002,"5min":44.716000000000001,"15min":44.716000000000001},"last":0.83899999999999997},{"interface":"front","average":{"1min":1.399,"5min":1.421,"15min":1.425},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":2.1469999999999998,"5min":2.1469999999999998,"15min":2.1469999999999998},"last":1.0960000000000001}]},{"osd":10,"last update":"Wed Sep 18 10:16:52 2024","interfaces":[{"interface":"back","average":{"1min":1.5549999999999999,"5min":13.942,"15min":16.006},"min":{"1min":0.67000000000000004,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":2.2850000000000001,"5min":180.15899999999999,"15min":180.15899999999999},"last":0.77500000000000002},{"interface":"front","average":{"1min":1.75,"5min":25.152999999999999,"15min":29.053000000000001},"min":{"1min":1.1899999999999999,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":2.4420000000000002,"5min":269.423,"15min":269.423},"last":0.92500000000000004}]},{"osd":11,"last update":"Wed Sep 18 10:17:15 2024","interfaces":[{"interface":"back","average":{"1min":1.478,"5min":8.0459999999999994,"15min":9.141},"min":{"1min":0.81599999999999995,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":2.1709999999999998,"5min":91.111999999999995,"15min":91.111999999999995},"last":1.1970000000000001},{"interface":"front","average":{"1min":1.3899999999999999,"5min":6.7939999999999996,"15min":7.6950000000000003},"min":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"max":{"1min":2.2290000000000001,"5min":91.165000000000006,"15min":91.165000000000006},"last":1.002}]},{"osd":12,"last update":"Wed Sep 18 10:16:32 2024","interfaces":[{"interface":"back","average":{"1min":10.385,"5min":14.760999999999999,"15min":15.49},"min":{"1min":1.327,"5min":0.91100000000000003,"15min":0.91100000000000003},"max":{"1min":65.189999999999998,"5min":189.84999999999999,"15min":189.84999999999999},"last":1.2270000000000001},{"interface":"front","average":{"1min":10.818,"5min":14.696,"15min":15.342000000000001},"min":{"1min":1.4890000000000001,"5min":0.91200000000000003,"15min":0.91200000000000003},"max":{"1min":65.162000000000006,"5min":180.381,"15min":180.381},"last":0.97999999999999998}]},{"osd":13,"last update":"Wed Sep 18 10:16:32 2024","interfaces":[{"interface":"back","average":{"1min":10.321999999999999,"5min":23.059000000000001,"15min":25.181999999999999},"min":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"max":{"1min":166.001,"5min":180.59100000000001,"15min":180.59100000000001},"last":1.337},{"interface":"front","average":{"1min":1.647,"5min":20.391999999999999,"15min":23.515999999999998},"min":{"1min":0.83399999999999996,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":2.3940000000000001,"5min":189.71299999999999,"15min":189.71299999999999},"last":1.298}]},{"osd":15,"last update":"Wed Sep 18 10:16:32 2024","interfaces":[{"interface":"back","average":{"1min":4.181,"5min":13.381,"15min":14.914},"min":{"1min":1.5780000000000001,"5min":0.99399999999999999,"15min":0.99399999999999999},"max":{"1min":44.917000000000002,"5min":180.44800000000001,"15min":180.44800000000001},"last":1.0700000000000001},{"interface":"front","average":{"1min":1.7030000000000001,"5min":15.118,"15min":17.353999999999999},"min":{"1min":0.93200000000000005,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":2.1909999999999998,"5min":180.53,"15min":180.53},"last":1.3859999999999999}]}]},{"osd":13,"up_from":22,"seq":94489280550,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308124,"kb_used_data":2444,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94063716,"statfs":{"total":96636764160,"available":96321245184,"internally_reserved":0,"allocated":2502656,"data_stored":1455374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,3,6,8,9,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:16:33 2024","interfaces":[{"interface":"back","average":{"1min":2.2599999999999998,"5min":17.045000000000002,"15min":19.509},"min":{"1min":0.65700000000000003,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":22.972999999999999,"5min":176.66800000000001,"15min":176.66800000000001},"last":0.73799999999999999},{"interface":"front","average":{"1min":4.2629999999999999,"5min":25.199000000000002,"15min":28.687999999999999},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":62.533000000000001,"5min":199.732,"15min":199.732},"last":1.0620000000000001}]},{"osd":1,"last update":"Wed Sep 18 10:15:48 2024","interfaces":[{"interface":"back","average":{"1min":1.986,"5min":1.986,"15min":1.986},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":18.471,"5min":18.471,"15min":18.471},"last":1.4219999999999999},{"interface":"front","average":{"1min":20.716999999999999,"5min":20.716999999999999,"15min":20.716999999999999},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":176.56800000000001,"5min":176.56800000000001,"15min":176.56800000000001},"last":0.86399999999999999}]},{"osd":2,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":11.984,"5min":11.984,"15min":11.984},"min":{"1min":0.751,"5min":0.751,"15min":0.751},"max":{"1min":196.59200000000001,"5min":196.59200000000001,"15min":196.59200000000001},"last":0.97699999999999998},{"interface":"front","average":{"1min":26.109999999999999,"5min":26.109999999999999,"15min":26.109999999999999},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":177.03,"5min":177.03,"15min":177.03},"last":1.2689999999999999}]},{"osd":3,"last update":"Wed Sep 18 10:16:33 2024","interfaces":[{"interface":"back","average":{"1min":9.5,"5min":22.245000000000001,"15min":24.369},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":158.74100000000001,"5min":168.27699999999999,"15min":168.27699999999999},"last":1.296},{"interface":"front","average":{"1min":1.2290000000000001,"5min":9.8819999999999997,"15min":11.324},"min":{"1min":0.84499999999999997,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.8959999999999999,"5min":196.643,"15min":196.643},"last":0.83299999999999996}]},{"osd":4,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":29.802,"5min":29.802,"15min":29.802},"min":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"max":{"1min":168.48099999999999,"5min":168.48099999999999,"15min":168.48099999999999},"last":1.347},{"interface":"front","average":{"1min":8.1780000000000008,"5min":8.1780000000000008,"15min":8.1780000000000008},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":65.188000000000002,"5min":65.188000000000002,"15min":65.188000000000002},"last":1.6379999999999999}]},{"osd":5,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":26.152999999999999,"5min":26.152999999999999,"15min":26.152999999999999},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":176.48099999999999,"5min":176.48099999999999,"15min":176.48099999999999},"last":1.49},{"interface":"front","average":{"1min":24.885999999999999,"5min":24.885999999999999,"15min":24.885999999999999},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":167.62299999999999,"5min":167.62299999999999,"15min":167.62299999999999},"last":1.528}]},{"osd":6,"last update":"Wed Sep 18 10:16:33 2024","interfaces":[{"interface":"back","average":{"1min":2.6589999999999998,"5min":24.015000000000001,"15min":27.574999999999999},"min":{"1min":1.042,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":23.131,"5min":176.81200000000001,"15min":176.81200000000001},"last":0.89500000000000002},{"interface":"front","average":{"1min":4.7169999999999996,"5min":23.227,"15min":26.312000000000001},"min":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"max":{"1min":61.874000000000002,"5min":167.947,"15min":167.947},"last":1.3939999999999999}]},{"osd":8,"last update":"Wed Sep 18 10:16:50 2024","interfaces":[{"interface":"back","average":{"1min":1.5700000000000001,"5min":1.5700000000000001,"15min":1.5700000000000001},"min":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"max":{"1min":2.117,"5min":2.117,"15min":2.117},"last":0.78600000000000003},{"interface":"front","average":{"1min":1.514,"5min":1.514,"15min":1.514},"min":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"max":{"1min":2.278,"5min":2.278,"15min":2.278},"last":0.997}]},{"osd":9,"last update":"Wed Sep 18 10:16:50 2024","interfaces":[{"interface":"back","average":{"1min":2.617,"5min":17.367000000000001,"15min":19.824999999999999},"min":{"1min":0.878,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":22.934000000000001,"5min":176.73599999999999,"15min":176.73599999999999},"last":0.96299999999999997},{"interface":"front","average":{"1min":1.5660000000000001,"5min":19.085000000000001,"15min":22.004999999999999},"min":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":2.448,"5min":167.84899999999999,"15min":167.84899999999999},"last":1.4710000000000001}]},{"osd":10,"last update":"Wed Sep 18 10:16:39 2024","interfaces":[{"interface":"back","average":{"1min":5.1120000000000001,"5min":19.164999999999999,"15min":21.507000000000001},"min":{"1min":1.22,"5min":1.1359999999999999,"15min":1.1359999999999999},"max":{"1min":62.137999999999998,"5min":168.07900000000001,"15min":168.07900000000001},"last":1.212},{"interface":"front","average":{"1min":17.683,"5min":29.114000000000001,"15min":31.018999999999998},"min":{"1min":1.123,"5min":1.0649999999999999,"15min":1.0649999999999999},"max":{"1min":289.68700000000001,"5min":289.68700000000001,"15min":289.68700000000001},"last":1.1699999999999999}]},{"osd":11,"last update":"Wed Sep 18 10:16:50 2024","interfaces":[{"interface":"back","average":{"1min":1.6819999999999999,"5min":1.625,"15min":1.615},"min":{"1min":1.056,"5min":1.0029999999999999,"15min":1.0029999999999999},"max":{"1min":2.5699999999999998,"5min":2.5699999999999998,"15min":2.5699999999999998},"last":1.1000000000000001},{"interface":"front","average":{"1min":2.798,"5min":21.73,"15min":24.885999999999999},"min":{"1min":1.286,"5min":1.028,"15min":1.028},"max":{"1min":23.167999999999999,"5min":176.94999999999999,"15min":176.94999999999999},"last":1.044}]},{"osd":12,"last update":"Wed Sep 18 10:16:33 2024","interfaces":[{"interface":"back","average":{"1min":1.8440000000000001,"5min":1.6339999999999999,"15min":1.599},"min":{"1min":1.2490000000000001,"5min":1.0009999999999999,"15min":1.0009999999999999},"max":{"1min":2.5950000000000002,"5min":2.7879999999999998,"15min":2.7879999999999998},"last":1.2490000000000001},{"interface":"front","average":{"1min":5.0570000000000004,"5min":18.542999999999999,"15min":20.791},"min":{"1min":1.3029999999999999,"5min":1.296,"15min":1.296},"max":{"1min":62.200000000000003,"5min":168.34399999999999,"15min":168.34399999999999},"last":1.548}]},{"osd":14,"last update":"Wed Sep 18 10:16:33 2024","interfaces":[{"interface":"back","average":{"1min":1.8320000000000001,"5min":5.7210000000000001,"15min":6.3689999999999998},"min":{"1min":1.139,"5min":1.0640000000000001,"15min":1.0640000000000001},"max":{"1min":2.419,"5min":87.427000000000007,"15min":87.427000000000007},"last":1.327},{"interface":"front","average":{"1min":2.9329999999999998,"5min":21.571999999999999,"15min":24.678000000000001},"min":{"1min":1.2709999999999999,"5min":1.0980000000000001,"15min":1.0980000000000001},"max":{"1min":23.268000000000001,"5min":176.87700000000001,"15min":176.87700000000001},"last":1.141}]},{"osd":15,"last update":"Wed Sep 18 10:16:33 2024","interfaces":[{"interface":"back","average":{"1min":10.048999999999999,"5min":18.277000000000001,"15min":19.648},"min":{"1min":1.272,"5min":1.266,"15min":1.266},"max":{"1min":96.564999999999998,"5min":168.40700000000001,"15min":168.40700000000001},"last":1.6160000000000001},{"interface":"front","average":{"1min":1.891,"5min":13.459,"15min":15.387},"min":{"1min":1.1950000000000001,"5min":0.95099999999999996,"15min":0.95099999999999996},"max":{"1min":2.5169999999999999,"5min":196.512,"15min":196.512},"last":1.373}]}]},{"osd":0,"up_from":21,"seq":90194313254,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303960,"kb_used_data":2440,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94067880,"statfs":{"total":96636764160,"available":96325509120,"internally_reserved":0,"allocated":2498560,"data_stored":1455355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,4,5,6,9,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 18 10:16:39 2024","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.95699999999999996,"15min":0.95899999999999996},"min":{"1min":0.56699999999999995,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.2889999999999999,"5min":1.341,"15min":1.341},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.97899999999999998,"15min":0.98499999999999999},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.2310000000000001,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":0.79500000000000004}]},{"osd":2,"last update":"Wed Sep 18 10:16:39 2024","interfaces":[{"interface":"back","average":{"1min":3.2719999999999998,"5min":1.6020000000000001,"15min":1.3240000000000001},"min":{"1min":0.76600000000000001,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":46.220999999999997,"5min":46.220999999999997,"15min":46.220999999999997},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.98799999999999999,"5min":1.071,"15min":1.085},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.732,"5min":1.732,"15min":1.732},"last":1.0389999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.196}]},{"osd":4,"last update":"Wed Sep 18 10:16:30 2024","interfaces":[{"interface":"back","average":{"1min":8.4039999999999999,"5min":14,"15min":14.933},"min":{"1min":0.59699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":142.20699999999999,"5min":142.20699999999999,"15min":142.20699999999999},"last":0.75},{"interface":"front","average":{"1min":10.48,"5min":14.896000000000001,"15min":15.632},"min":{"1min":0.63,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":132.33799999999999,"5min":132.33799999999999,"15min":132.33799999999999},"last":1.117}]},{"osd":5,"last update":"Wed Sep 18 10:16:39 2024","interfaces":[{"interface":"back","average":{"1min":1.1739999999999999,"5min":1.24,"15min":1.2509999999999999},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":1.619,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":1.2070000000000001},{"interface":"front","average":{"1min":1.569,"5min":8.1340000000000003,"15min":9.2279999999999998},"min":{"1min":0.85599999999999998,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":10.143000000000001,"5min":130.39599999999999,"15min":130.39599999999999},"last":0.98199999999999998}]},{"osd":6,"last update":"Wed Sep 18 10:16:47 2024","interfaces":[{"interface":"back","average":{"1min":1.266,"5min":1.254,"15min":1.252},"min":{"1min":0.96099999999999997,"5min":0.94299999999999995,"15min":0.94299999999999995},"max":{"1min":1.571,"5min":1.776,"15min":1.776},"last":1.014},{"interface":"front","average":{"1min":1.3109999999999999,"5min":2.0270000000000001,"15min":2.1469999999999998},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.7909999999999999,"5min":15.250999999999999,"15min":15.250999999999999},"last":1.3580000000000001}]},{"osd":9,"last update":"Wed Sep 18 10:16:30 2024","interfaces":[{"interface":"back","average":{"1min":1.3360000000000001,"5min":1.3919999999999999,"15min":1.4019999999999999},"min":{"1min":0.88200000000000001,"5min":0.879,"15min":0.879},"max":{"1min":1.863,"5min":1.863,"15min":1.863},"last":1.4159999999999999},{"interface":"front","average":{"1min":1.667,"5min":8.7590000000000003,"15min":9.9410000000000007},"min":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":10.180999999999999,"5min":130.441,"15min":130.441},"last":1.085}]},{"osd":11,"last update":"Wed Sep 18 10:16:30 2024","interfaces":[{"interface":"back","average":{"1min":4.8949999999999996,"5min":2.2109999999999999,"15min":1.7629999999999999},"min":{"1min":1.1579999999999999,"5min":0.90700000000000003,"15min":0.90700000000000003},"max":{"1min":41.610999999999997,"5min":41.610999999999997,"15min":41.610999999999997},"last":1.5229999999999999},{"interface":"front","average":{"1min":1.5029999999999999,"5min":4.5620000000000003,"15min":5.0720000000000001},"min":{"1min":1.1220000000000001,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":1.9850000000000001,"5min":54.805999999999997,"15min":54.805999999999997},"last":1.542}]},{"osd":12,"last update":"Wed Sep 18 10:16:30 2024","interfaces":[{"interface":"back","average":{"1min":1.7869999999999999,"5min":9.1170000000000009,"15min":10.339},"min":{"1min":0.95999999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":10.015000000000001,"5min":130.34800000000001,"15min":130.34800000000001},"last":1.25},{"interface":"front","average":{"1min":1.4350000000000001,"5min":1.4710000000000001,"15min":1.4770000000000001},"min":{"1min":0.85899999999999999,"5min":0.502,"15min":0.502},"max":{"1min":1.944,"5min":1.944,"15min":1.944},"last":1.282}]},{"osd":14,"last update":"Wed Sep 18 10:16:39 2024","interfaces":[{"interface":"back","average":{"1min":3.7490000000000001,"5min":8.7810000000000006,"15min":9.6199999999999992},"min":{"1min":1.044,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":38.969999999999999,"5min":130.48500000000001,"15min":130.48500000000001},"last":1.387},{"interface":"front","average":{"1min":6.931,"5min":2.6339999999999999,"15min":1.9179999999999999},"min":{"1min":1.2609999999999999,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":80.460999999999999,"5min":80.460999999999999,"15min":80.460999999999999},"last":1.4370000000000001}]},{"osd":15,"last update":"Wed Sep 18 10:16:39 2024","interfaces":[{"interface":"back","average":{"1min":1.591,"5min":2.778,"15min":2.976},"min":{"1min":1.1699999999999999,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":2.0310000000000001,"5min":23.806999999999999,"15min":23.806999999999999},"last":1.47},{"interface":"front","average":{"1min":6.9199999999999999,"5min":9.4809999999999999,"15min":9.9079999999999995},"min":{"1min":1.1279999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":100.925,"5min":130.52799999999999,"15min":130.52799999999999},"last":1.482}]}]},{"osd":4,"up_from":21,"seq":90194313255,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303960,"kb_used_data":2440,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94067880,"statfs":{"total":96636764160,"available":96325509120,"internally_reserved":0,"allocated":2498560,"data_stored":1455355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,3,5,7,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":9.0980000000000008,"15min":10.465999999999999},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.7529999999999999,"5min":89.266999999999996,"15min":89.266999999999996},"last":0.78700000000000003},{"interface":"front","average":{"1min":18.640000000000001,"5min":7.7320000000000002,"15min":5.9139999999999997},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":198.80699999999999,"5min":198.80699999999999,"15min":198.80699999999999},"last":0.83599999999999997}]},{"osd":1,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":4.0759999999999996,"5min":12.537000000000001,"15min":13.946999999999999},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":62.780000000000001,"5min":164.21199999999999,"15min":164.21199999999999},"last":0.92500000000000004},{"interface":"front","average":{"1min":2.5139999999999998,"5min":9.593,"15min":10.773},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":25.802,"5min":89.331000000000003,"15min":89.331000000000003},"last":1.105}]},{"osd":2,"last update":"Wed Sep 18 10:15:47 2024","interfaces":[{"interface":"back","average":{"1min":23.617000000000001,"5min":23.617000000000001,"15min":23.617000000000001},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":192.136,"5min":192.136,"15min":192.136},"last":1.8740000000000001},{"interface":"front","average":{"1min":11.888999999999999,"5min":11.888999999999999,"15min":11.888999999999999},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":198.87700000000001,"5min":198.87700000000001,"15min":198.87700000000001},"last":1.347}]},{"osd":3,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":35.673000000000002,"5min":21.670999999999999,"15min":19.337},"min":{"1min":1.161,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":388.84300000000002,"5min":388.84300000000002,"15min":388.84300000000002},"last":1.4630000000000001},{"interface":"front","average":{"1min":17.747,"5min":25.643999999999998,"15min":26.960000000000001},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":255.82900000000001,"5min":255.82900000000001,"15min":255.82900000000001},"last":1.3640000000000001}]},{"osd":5,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":1.4910000000000001,"5min":15.449999999999999,"15min":17.776},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":2.169,"5min":189.55600000000001,"15min":189.55600000000001},"last":1.0189999999999999},{"interface":"front","average":{"1min":21.053000000000001,"5min":6.8079999999999998,"15min":4.4340000000000002},"min":{"1min":1.3640000000000001,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":199.15600000000001,"5min":199.15600000000001,"15min":199.15600000000001},"last":1.2669999999999999}]},{"osd":6,"last update":"Wed Sep 18 10:15:34 2024","interfaces":[{"interface":"back","average":{"1min":32.25,"5min":32.25,"15min":32.25},"min":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":197.33199999999999,"5min":197.33199999999999,"15min":197.33199999999999},"last":1.8160000000000001},{"interface":"front","average":{"1min":18.963000000000001,"5min":18.963000000000001,"15min":18.963000000000001},"min":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"max":{"1min":179.19200000000001,"5min":179.19200000000001,"15min":179.19200000000001},"last":1.7769999999999999}]},{"osd":7,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":5.6050000000000004,"5min":23.198,"15min":26.129999999999999},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":62.704999999999998,"5min":290.036,"15min":290.036},"last":1.1759999999999999},{"interface":"front","average":{"1min":1.6639999999999999,"5min":14.273,"15min":16.375},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":2.3100000000000001,"5min":139.696,"15min":139.696},"last":1.321}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1509999999999998}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7509999999999999}]},{"osd":10,"last update":"Wed Sep 18 10:16:49 2024","interfaces":[{"interface":"back","average":{"1min":1.653,"5min":18.559000000000001,"15min":21.376999999999999},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":2.4740000000000002,"5min":164.43899999999999,"15min":164.43899999999999},"last":1.532},{"interface":"front","average":{"1min":4.6070000000000002,"5min":10.993,"15min":12.057},"min":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":59.408999999999999,"5min":89.442999999999998,"15min":89.442999999999998},"last":1.5069999999999999}]},{"osd":11,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":19.57,"5min":12.567,"15min":11.4},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":199.255,"5min":199.255,"15min":199.255},"last":1.5860000000000001},{"interface":"front","average":{"1min":11.393000000000001,"5min":22.213000000000001,"15min":24.016999999999999},"min":{"1min":1.0840000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"max":{"1min":155.19800000000001,"5min":164.733,"15min":164.733},"last":1.6579999999999999}]},{"osd":12,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":4.5800000000000001,"5min":10.406000000000001,"15min":11.377000000000001},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":58.856000000000002,"5min":89.400000000000006,"15min":89.400000000000006},"last":1.623},{"interface":"front","average":{"1min":31.831,"5min":12.901999999999999,"15min":9.7469999999999999},"min":{"1min":1.4530000000000001,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":199.32900000000001,"5min":199.32900000000001,"15min":199.32900000000001},"last":1.7050000000000001}]},{"osd":13,"last update":"Wed Sep 18 10:16:49 2024","interfaces":[{"interface":"back","average":{"1min":12.695,"5min":22.789999999999999,"15min":24.472000000000001},"min":{"1min":1.702,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":192.37299999999999,"5min":199.37799999999999,"15min":199.37799999999999},"last":1.8080000000000001},{"interface":"front","average":{"1min":1.8029999999999999,"5min":19.068000000000001,"15min":21.946000000000002},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":2.5699999999999998,"5min":164.53,"15min":164.53},"last":1.7749999999999999}]},{"osd":14,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":22.312000000000001,"5min":21.788,"15min":21.701000000000001},"min":{"1min":1.556,"5min":0.96099999999999997,"15min":0.96099999999999997},"max":{"1min":199.43000000000001,"5min":202.191,"15min":202.191},"last":1.897},{"interface":"front","average":{"1min":9.8800000000000008,"5min":17.420999999999999,"15min":18.678000000000001},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":101.492,"5min":164.35400000000001,"15min":164.35400000000001},"last":1.8600000000000001}]}]},{"osd":9,"up_from":21,"seq":90194313254,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303960,"kb_used_data":2440,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94067880,"statfs":{"total":96636764160,"available":96325509120,"internally_reserved":0,"allocated":2498560,"data_stored":1455355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,4,5,6,8,10,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":7.2450000000000001,"5min":7.2450000000000001,"15min":7.2450000000000001},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":127.902,"5min":127.902,"15min":127.902},"last":0.97399999999999998},{"interface":"front","average":{"1min":8.3109999999999999,"5min":8.3109999999999999,"15min":8.3109999999999999},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":96.203000000000003,"5min":96.203000000000003,"15min":96.203000000000003},"last":0.81200000000000006}]},{"osd":1,"last update":"Wed Sep 18 10:16:40 2024","interfaces":[{"interface":"back","average":{"1min":10.587,"5min":10.587,"15min":10.587},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":166.833,"5min":166.833,"15min":166.833},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.63100000000000001}]},{"osd":2,"last update":"Wed Sep 18 10:17:20 2024","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":4.7270000000000003,"15min":5.3369999999999997},"min":{"1min":0.83799999999999997,"5min":0.626,"15min":0.626},"max":{"1min":1.3320000000000001,"5min":68.049999999999997,"15min":68.049999999999997},"last":0.82399999999999995},{"interface":"front","average":{"1min":1.071,"5min":1.839,"15min":1.9670000000000001},"min":{"1min":0.754,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.462,"5min":18.553999999999998,"15min":18.553999999999998},"last":0.80000000000000004}]},{"osd":3,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":2.875,"5min":5.5549999999999997,"15min":6.0019999999999998},"min":{"1min":0.72699999999999998,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":18.431000000000001,"5min":98.888999999999996,"15min":98.888999999999996},"last":0.72499999999999998},{"interface":"front","average":{"1min":3.0169999999999999,"5min":5.5890000000000004,"15min":6.0170000000000003},"min":{"1min":0.749,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":18.472000000000001,"5min":98.947999999999993,"15min":98.947999999999993},"last":0.89200000000000002}]},{"osd":4,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":8.2360000000000007,"5min":2.5840000000000001,"15min":1.6419999999999999},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":124.15300000000001,"5min":124.15300000000001,"15min":124.15300000000001},"last":1.0900000000000001},{"interface":"front","average":{"1min":6.2380000000000004,"5min":8.8539999999999992,"15min":9.2899999999999991},"min":{"1min":0.81000000000000005,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":73.992000000000004,"5min":73.992000000000004,"15min":73.992000000000004},"last":1.02}]},{"osd":5,"last update":"Wed Sep 18 10:17:20 2024","interfaces":[{"interface":"back","average":{"1min":1.0329999999999999,"5min":1.663,"15min":1.768},"min":{"1min":0.79000000000000004,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.9390000000000001,"5min":14.786,"15min":14.786},"last":0.86699999999999999},{"interface":"front","average":{"1min":1.1519999999999999,"5min":7.0300000000000002,"15min":8.0099999999999998},"min":{"1min":0.71699999999999997,"5min":0.435,"15min":0.435},"max":{"1min":1.4259999999999999,"5min":67.915999999999997,"15min":67.915999999999997},"last":0.93700000000000006}]},{"osd":6,"last update":"Wed Sep 18 10:17:20 2024","interfaces":[{"interface":"back","average":{"1min":1.3500000000000001,"5min":6.9770000000000003,"15min":7.915},"min":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"max":{"1min":1.806,"5min":70.914000000000001,"15min":70.914000000000001},"last":0.96699999999999997},{"interface":"front","average":{"1min":1.347,"5min":12.833,"15min":14.747},"min":{"1min":0.96899999999999997,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":2.0289999999999999,"5min":136.38800000000001,"15min":136.38800000000001},"last":1.3520000000000001}]},{"osd":8,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":3.0310000000000001,"5min":4.2789999999999999,"15min":4.4870000000000001},"min":{"1min":1.0509999999999999,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":30.713000000000001,"5min":64.912999999999997,"15min":64.912999999999997},"last":1.4219999999999999},{"interface":"front","average":{"1min":2.0569999999999999,"5min":5.3959999999999999,"15min":5.952},"min":{"1min":0.82499999999999996,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":12.898999999999999,"5min":57.664000000000001,"15min":57.664000000000001},"last":1.1379999999999999}]},{"osd":10,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":1.3400000000000001,"5min":1.3089999999999999,"15min":1.304},"min":{"1min":0.88300000000000001,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.8180000000000001,"5min":2.109,"15min":2.109},"last":1.292},{"interface":"front","average":{"1min":6.0389999999999997,"5min":4.2309999999999999,"15min":3.9300000000000002},"min":{"1min":0.97899999999999998,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":68.216999999999999,"5min":68.216999999999999,"15min":68.216999999999999},"last":0.996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.669}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.446}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":96}]},{"osd":14,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":4.3579999999999997,"5min":8.8759999999999994,"15min":9.6289999999999996},"min":{"1min":0.86499999999999999,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":50.773000000000003,"5min":62.899000000000001,"15min":62.899000000000001},"last":1.0669999999999999},{"interface":"front","average":{"1min":4.3540000000000001,"5min":13.071,"15min":14.523999999999999},"min":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":50.709000000000003,"5min":155.61000000000001,"15min":155.61000000000001},"last":1.4690000000000001}]},{"osd":15,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":1.4630000000000001,"5min":1.492,"15min":1.4970000000000001},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.97,"5min":1.9970000000000001,"15min":1.9970000000000001},"last":1.516},{"interface":"front","average":{"1min":9.5700000000000003,"5min":5.1749999999999998,"15min":4.4429999999999996},"min":{"1min":1.0980000000000001,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":68.140000000000001,"5min":68.140000000000001,"15min":68.140000000000001},"last":1.052}]}]},{"osd":11,"up_from":21,"seq":90194313254,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303956,"kb_used_data":2440,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94067884,"statfs":{"total":96636764160,"available":96325513216,"internally_reserved":0,"allocated":2498560,"data_stored":1455355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[2,3,6,8,9,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0049999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86599999999999999}]},{"osd":2,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":8.3390000000000004,"15min":9.5719999999999992},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.3740000000000001,"5min":99.718000000000004,"15min":99.718000000000004},"last":0.90900000000000003},{"interface":"front","average":{"1min":3.6349999999999998,"5min":1.4450000000000001,"15min":1.0800000000000001},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":49.774000000000001,"5min":49.774000000000001,"15min":49.774000000000001},"last":0.81299999999999994}]},{"osd":3,"last update":"Wed Sep 18 10:16:46 2024","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":4.2210000000000001,"15min":4.7880000000000003},"min":{"1min":0.48399999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.369,"5min":85.963999999999999,"15min":85.963999999999999},"last":0.75900000000000001},{"interface":"front","average":{"1min":5.4660000000000002,"5min":19.300000000000001,"15min":21.606000000000002},"min":{"1min":0.73699999999999999,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":80.980999999999995,"5min":125.672,"15min":125.672},"last":0.70299999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.444}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5980000000000001}]},{"osd":6,"last update":"Wed Sep 18 10:16:46 2024","interfaces":[{"interface":"back","average":{"1min":5.8810000000000002,"5min":2.101,"15min":1.4710000000000001},"min":{"1min":0.70599999999999996,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":86.680000000000007,"5min":86.680000000000007,"15min":86.680000000000007},"last":0.98499999999999999},{"interface":"front","average":{"1min":1.1830000000000001,"5min":5.827,"15min":6.601},"min":{"1min":0.73799999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.49,"5min":94.162000000000006,"15min":94.162000000000006},"last":1.1479999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.524}]},{"osd":8,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":2.7069999999999999,"5min":5.3730000000000002,"15min":5.8170000000000002},"min":{"1min":0.95299999999999996,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":27.577999999999999,"5min":85.900000000000006,"15min":85.900000000000006},"last":1.276},{"interface":"front","average":{"1min":2.544,"5min":8.4960000000000004,"15min":9.4879999999999995},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":27.542000000000002,"5min":99.765000000000001,"15min":99.765000000000001},"last":1.0900000000000001}]},{"osd":9,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":5.8760000000000003,"5min":5.8339999999999996,"15min":5.827},"min":{"1min":0.96399999999999997,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":89.844999999999999,"5min":89.844999999999999,"15min":89.844999999999999},"last":1.0549999999999999},{"interface":"front","average":{"1min":1.351,"5min":6.8070000000000004,"15min":7.7160000000000002},"min":{"1min":0.68600000000000005,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.742,"5min":99.810000000000002,"15min":99.810000000000002},"last":1.006}]},{"osd":10,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":4.0069999999999997,"5min":1.679,"15min":1.2909999999999999},"min":{"1min":0.98499999999999999,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":49.819000000000003,"5min":49.819000000000003,"15min":49.819000000000003},"last":1.119},{"interface":"front","average":{"1min":1.5580000000000001,"5min":6.46,"15min":7.2770000000000001},"min":{"1min":0.98699999999999999,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.976,"5min":89.290000000000006,"15min":89.290000000000006},"last":0.95699999999999996}]},{"osd":12,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":5.907,"5min":2.081,"15min":1.4430000000000001},"min":{"1min":0.96599999999999997,"5min":0.498,"15min":0.498},"max":{"1min":87.463999999999999,"5min":87.463999999999999,"15min":87.463999999999999},"last":1.0329999999999999},{"interface":"front","average":{"1min":1.5660000000000001,"5min":8.6219999999999999,"15min":9.798},"min":{"1min":1.0700000000000001,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":2.0600000000000001,"5min":86.025999999999996,"15min":86.025999999999996},"last":1.222}]},{"osd":13,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":4.4210000000000003,"5min":25.207999999999998,"15min":28.672000000000001},"min":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"max":{"1min":59.841999999999999,"5min":189.25899999999999,"15min":189.25899999999999},"last":1.1719999999999999},{"interface":"front","average":{"1min":7.569,"5min":2.5659999999999998,"15min":1.732},"min":{"1min":0.88900000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":71.313000000000002,"5min":71.313000000000002,"15min":71.313000000000002},"last":1.2010000000000001}]},{"osd":14,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":14.388999999999999,"5min":8.016,"15min":6.9539999999999997},"min":{"1min":1.0129999999999999,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":256.13400000000001,"5min":256.13400000000001,"15min":256.13400000000001},"last":1.3819999999999999},{"interface":"front","average":{"1min":2.7650000000000001,"5min":7.7380000000000004,"15min":8.5670000000000002},"min":{"1min":0.874,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":25.791,"5min":102.625,"15min":102.625},"last":1.246}]},{"osd":15,"last update":"Wed Sep 18 10:16:40 2024","interfaces":[{"interface":"back","average":{"1min":4.0309999999999997,"5min":4.0309999999999997,"15min":4.0309999999999997},"min":{"1min":1.153,"5min":1.153,"15min":1.153},"max":{"1min":49.927999999999997,"5min":49.927999999999997,"15min":49.927999999999997},"last":1.302},{"interface":"front","average":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"min":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"max":{"1min":2.044,"5min":2.044,"15min":2.044},"last":1.3380000000000001}]}]},{"osd":12,"up_from":21,"seq":90194313255,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303960,"kb_used_data":2444,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94067880,"statfs":{"total":96636764160,"available":96325509120,"internally_reserved":0,"allocated":2502656,"data_stored":1455374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[3,4,6,7,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":2.3839999999999999,"5min":2.3839999999999999,"15min":2.3839999999999999},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":32.476999999999997,"5min":32.476999999999997,"15min":32.476999999999997},"last":0.64200000000000002},{"interface":"front","average":{"1min":2.625,"5min":2.625,"15min":2.625},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":32.509,"5min":32.509,"15min":32.509},"last":0.89300000000000002}]},{"osd":1,"last update":"Wed Sep 18 10:15:32 2024","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.6399999999999999,"5min":1.6399999999999999,"15min":1.6399999999999999},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.905,"5min":1.905,"15min":1.905},"last":0.81899999999999995}]},{"osd":2,"last update":"Wed Sep 18 10:15:32 2024","interfaces":[{"interface":"back","average":{"1min":1.002,"5min":1.002,"15min":1.002},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":1.153},{"interface":"front","average":{"1min":4.1189999999999998,"5min":4.1189999999999998,"15min":4.1189999999999998},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":62.780999999999999,"5min":62.780999999999999,"15min":62.780999999999999},"last":0.96499999999999997}]},{"osd":3,"last update":"Wed Sep 18 10:16:33 2024","interfaces":[{"interface":"back","average":{"1min":5.9429999999999996,"5min":2.1040000000000001,"15min":1.464},"min":{"1min":0.627,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":74.822999999999993,"5min":74.822999999999993,"15min":74.822999999999993},"last":0.68400000000000005},{"interface":"front","average":{"1min":8.8200000000000003,"5min":2.6840000000000002,"15min":1.6619999999999999},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":94.320999999999998,"5min":94.320999999999998,"15min":94.320999999999998},"last":0.79200000000000004}]},{"osd":4,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":16.058,"5min":4.0529999999999999,"15min":2.052},"min":{"1min":0.53100000000000003,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":114.184,"5min":114.184,"15min":114.184},"last":0.64600000000000002},{"interface":"front","average":{"1min":5.8920000000000003,"5min":2.1560000000000001,"15min":1.5329999999999999},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":74.698999999999998,"5min":74.698999999999998,"15min":74.698999999999998},"last":1.1240000000000001}]},{"osd":6,"last update":"Wed Sep 18 10:16:38 2024","interfaces":[{"interface":"back","average":{"1min":10.747999999999999,"5min":10.747999999999999,"15min":10.747999999999999},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":130.20699999999999,"5min":130.20699999999999,"15min":130.20699999999999},"last":0.95999999999999996},{"interface":"front","average":{"1min":14.148,"5min":14.148,"15min":14.148},"min":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":132.60900000000001,"5min":132.60900000000001,"15min":132.60900000000001},"last":1.2410000000000001}]},{"osd":7,"last update":"Wed Sep 18 10:16:38 2024","interfaces":[{"interface":"back","average":{"1min":4.774,"5min":4.774,"15min":4.774},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":90.188000000000002,"5min":90.188000000000002,"15min":90.188000000000002},"last":0.85599999999999998},{"interface":"front","average":{"1min":6.0359999999999996,"5min":6.0359999999999996,"15min":6.0359999999999996},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":75.061999999999998,"5min":75.061999999999998,"15min":75.061999999999998},"last":0.90700000000000003}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":94.382000000000005}]},{"osd":9,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":8.9719999999999995,"5min":2.8079999999999998,"15min":1.7809999999999999},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":94.421000000000006,"5min":94.421000000000006,"15min":94.421000000000006},"last":1.198},{"interface":"front","average":{"1min":5.0410000000000004,"5min":2.0179999999999998,"15min":1.514},"min":{"1min":0.70199999999999996,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":90.242999999999995,"5min":90.242999999999995,"15min":90.242999999999995},"last":1.0529999999999999}]},{"osd":10,"last update":"Wed Sep 18 10:16:38 2024","interfaces":[{"interface":"back","average":{"1min":15.145,"5min":15.145,"15min":15.145},"min":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":99.251000000000005,"5min":99.251000000000005,"15min":99.251000000000005},"last":1.0209999999999999},{"interface":"front","average":{"1min":8.7080000000000002,"5min":8.7080000000000002,"15min":8.7080000000000002},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":94.468000000000004,"5min":94.468000000000004,"15min":94.468000000000004},"last":1.3169999999999999}]},{"osd":11,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":8.9269999999999996,"5min":2.8769999999999998,"15min":1.869},"min":{"1min":0.91300000000000003,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":74.888999999999996,"5min":74.888999999999996,"15min":74.888999999999996},"last":1.458},{"interface":"front","average":{"1min":9.1590000000000007,"5min":2.9380000000000002,"15min":1.901},"min":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"max":{"1min":94.546000000000006,"5min":94.546000000000006,"15min":94.546000000000006},"last":0.98099999999999998}]},{"osd":13,"last update":"Wed Sep 18 10:16:33 2024","interfaces":[{"interface":"back","average":{"1min":7.4779999999999998,"5min":2.5619999999999998,"15min":1.742},"min":{"1min":0.79900000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":143.672,"5min":143.672,"15min":143.672},"last":1.165},{"interface":"front","average":{"1min":12.677,"5min":3.6779999999999999,"15min":2.1779999999999999},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":143.619,"5min":143.619,"15min":143.619},"last":1.3560000000000001}]},{"osd":14,"last update":"Wed Sep 18 10:16:33 2024","interfaces":[{"interface":"back","average":{"1min":13.363,"5min":3.8159999999999998,"15min":2.2250000000000001},"min":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"max":{"1min":94.504999999999995,"5min":94.504999999999995,"15min":94.504999999999995},"last":1.0900000000000001},{"interface":"front","average":{"1min":12.039,"5min":3.52,"15min":2.1000000000000001},"min":{"1min":0.95099999999999996,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":92.593000000000004,"5min":92.593000000000004,"15min":92.593000000000004},"last":1.276}]},{"osd":15,"last update":"Wed Sep 18 10:16:33 2024","interfaces":[{"interface":"back","average":{"1min":6.5339999999999998,"5min":2.5019999999999998,"15min":1.8300000000000001},"min":{"1min":0.751,"5min":0.751,"15min":0.751},"max":{"1min":74.966999999999999,"5min":74.966999999999999,"15min":74.966999999999999},"last":1.51},{"interface":"front","average":{"1min":14.669,"5min":4.117,"15min":2.359},"min":{"1min":0.996,"5min":0.95299999999999996,"15min":0.95299999999999996},"max":{"1min":152.11000000000001,"5min":152.11000000000001,"15min":152.11000000000001},"last":1.4279999999999999}]}]},{"osd":1,"up_from":22,"seq":94489280549,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":303960,"kb_used_data":2440,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94067880,"statfs":{"total":96636764160,"available":96325509120,"internally_reserved":0,"allocated":2498560,"data_stored":1455355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,6,9,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:16:38 2024","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":2.472,"15min":2.7370000000000001},"min":{"1min":0.65800000000000003,"5min":0.629,"15min":0.629},"max":{"1min":1.1930000000000001,"5min":36.268000000000001,"15min":36.268000000000001},"last":0.80900000000000005},{"interface":"front","average":{"1min":5.3659999999999997,"5min":15.244999999999999,"15min":16.890999999999998},"min":{"1min":0.73099999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":66.534999999999997,"5min":154.67599999999999,"15min":154.67599999999999},"last":0.93799999999999994}]},{"osd":2,"last update":"Wed Sep 18 10:16:38 2024","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.91400000000000003,"15min":0.90400000000000003},"min":{"1min":0.71499999999999997,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.4119999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":1.1919999999999999},{"interface":"front","average":{"1min":0.98999999999999999,"5min":2.5539999999999998,"15min":2.8140000000000001},"min":{"1min":0.75600000000000001,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.212,"5min":36.323999999999998,"15min":36.323999999999998},"last":0.98699999999999999}]},{"osd":3,"last update":"Wed Sep 18 10:16:38 2024","interfaces":[{"interface":"back","average":{"1min":5.548,"5min":10.811999999999999,"15min":11.689},"min":{"1min":0.94599999999999995,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":66.344999999999999,"5min":150.642,"15min":150.642},"last":1.0580000000000001},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.0649999999999999,"15min":1.0509999999999999},"min":{"1min":0.82999999999999996,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.6830000000000001,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":1.0900000000000001}]},{"osd":4,"last update":"Wed Sep 18 10:15:35 2024","interfaces":[{"interface":"back","average":{"1min":3.3439999999999999,"5min":3.3439999999999999,"15min":3.3439999999999999},"min":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":40.783000000000001,"5min":40.783000000000001,"15min":40.783000000000001},"last":1.7210000000000001},{"interface":"front","average":{"1min":4.1699999999999999,"5min":4.1699999999999999,"15min":4.1699999999999999},"min":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":50.747,"5min":50.747,"15min":50.747},"last":1.518}]},{"osd":5,"last update":"Wed Sep 18 10:15:35 2024","interfaces":[{"interface":"back","average":{"1min":5.9720000000000004,"5min":5.9720000000000004,"15min":5.9720000000000004},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":52.317999999999998,"5min":52.317999999999998,"15min":52.317999999999998},"last":1.2649999999999999},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.748,"5min":1.748,"15min":1.748},"last":1.839}]},{"osd":6,"last update":"Wed Sep 18 10:16:38 2024","interfaces":[{"interface":"back","average":{"1min":9.2970000000000006,"5min":8.5380000000000003,"15min":8.4120000000000008},"min":{"1min":0.85099999999999998,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":120.85299999999999,"5min":120.85299999999999,"15min":120.85299999999999},"last":0.88800000000000001},{"interface":"front","average":{"1min":2.6040000000000001,"5min":8.8019999999999996,"15min":9.8350000000000009},"min":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"max":{"1min":20.600000000000001,"5min":85.045000000000002,"15min":85.045000000000002},"last":1.288}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":118.43600000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7789999999999999}]},{"osd":9,"last update":"Wed Sep 18 10:17:19 2024","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":2.9980000000000002,"15min":3.29},"min":{"1min":0.998,"5min":0.89300000000000002,"15min":0.89300000000000002},"max":{"1min":1.488,"5min":36.506999999999998,"15min":36.506999999999998},"last":1.139},{"interface":"front","average":{"1min":1.3720000000000001,"5min":9.3450000000000006,"15min":10.673999999999999},"min":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"max":{"1min":1.968,"5min":165.13499999999999,"15min":165.13499999999999},"last":1.252}]},{"osd":10,"last update":"Wed Sep 18 10:16:38 2024","interfaces":[{"interface":"back","average":{"1min":1.5209999999999999,"5min":3.048,"15min":3.302},"min":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"max":{"1min":2.0489999999999999,"5min":36.436,"15min":36.436},"last":1.222},{"interface":"front","average":{"1min":6.0839999999999996,"5min":10.747999999999999,"15min":11.526},"min":{"1min":1.2769999999999999,"5min":0.88,"15min":0.88},"max":{"1min":66.673000000000002,"5min":100.402,"15min":100.402},"last":1.3440000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.825}]},{"osd":12,"last update":"Wed Sep 18 10:16:38 2024","interfaces":[{"interface":"back","average":{"1min":1.5549999999999999,"5min":3.681,"15min":4.0350000000000001},"min":{"1min":1.101,"5min":1.0589999999999999,"15min":1.0589999999999999},"max":{"1min":2.294,"5min":49.976999999999997,"15min":49.976999999999997},"last":1.3180000000000001},{"interface":"front","average":{"1min":2.0369999999999999,"5min":7.1269999999999998,"15min":7.9749999999999996},"min":{"1min":1.196,"5min":1.196,"15min":1.196},"max":{"1min":8.4049999999999994,"5min":124.78100000000001,"15min":124.78100000000001},"last":1.4650000000000001}]},{"osd":13,"last update":"Wed Sep 18 10:17:19 2024","interfaces":[{"interface":"back","average":{"1min":1.591,"5min":9.6110000000000007,"15min":10.946999999999999},"min":{"1min":1.26,"5min":1.232,"15min":1.232},"max":{"1min":2.0369999999999999,"5min":104.94199999999999,"15min":104.94199999999999},"last":1.4279999999999999},{"interface":"front","average":{"1min":1.6259999999999999,"5min":1.5820000000000001,"15min":1.575},"min":{"1min":1.381,"5min":1.2170000000000001,"15min":1.2170000000000001},"max":{"1min":2.0099999999999998,"5min":2.3809999999999998,"15min":2.3809999999999998},"last":1.403}]},{"osd":14,"last update":"Wed Sep 18 10:17:19 2024","interfaces":[{"interface":"back","average":{"1min":1.6970000000000001,"5min":4.8120000000000003,"15min":5.3310000000000004},"min":{"1min":1.3560000000000001,"5min":1.276,"15min":1.276},"max":{"1min":2.2130000000000001,"5min":66.825999999999993,"15min":66.825999999999993},"last":1.518},{"interface":"front","average":{"1min":1.6679999999999999,"5min":2.6019999999999999,"15min":2.758},"min":{"1min":1.4419999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"max":{"1min":1.9810000000000001,"5min":22.16,"15min":22.16},"last":1.488}]},{"osd":15,"last update":"Wed Sep 18 10:17:19 2024","interfaces":[{"interface":"back","average":{"1min":1.6599999999999999,"5min":3.2730000000000001,"15min":3.5419999999999998},"min":{"1min":1.3400000000000001,"5min":1.165,"15min":1.165},"max":{"1min":1.96,"5min":36.575000000000003,"15min":36.575000000000003},"last":1.5549999999999999},{"interface":"front","average":{"1min":1.7649999999999999,"5min":4.8540000000000001,"15min":5.3689999999999998},"min":{"1min":1.4319999999999999,"5min":1.3260000000000001,"15min":1.3260000000000001},"max":{"1min":2.173,"5min":66.763999999999996,"15min":66.763999999999996},"last":1.587}]}]},{"osd":2,"up_from":22,"seq":94489280549,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":303960,"kb_used_data":2440,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94067880,"statfs":{"total":96636764160,"available":96325509120,"internally_reserved":0,"allocated":2498560,"data_stored":1455355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,3,4,5,6,7,8,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":21.344000000000001,"15min":24.777999999999999},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.3879999999999999,"5min":419.05700000000002,"15min":419.05700000000002},"last":0.86299999999999999},{"interface":"front","average":{"1min":2.3860000000000001,"5min":4.4020000000000001,"15min":4.7380000000000004},"min":{"1min":0.57299999999999995,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":35.899999999999999,"5min":98.254000000000005,"15min":98.254000000000005},"last":0.74399999999999999}]},{"osd":3,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":5.1219999999999999,"5min":1.7529999999999999,"15min":1.1910000000000001},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":86.677000000000007,"5min":86.677000000000007,"15min":86.677000000000007},"last":0.83099999999999996},{"interface":"front","average":{"1min":3.6160000000000001,"5min":1.47,"15min":1.1120000000000001},"min":{"1min":0.68000000000000005,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":59.759,"5min":59.759,"15min":59.759},"last":0.80600000000000005}]},{"osd":4,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":7.992,"5min":3.1379999999999999,"15min":2.3290000000000002},"min":{"1min":0.55600000000000005,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":121.20099999999999,"5min":121.20099999999999,"15min":121.20099999999999},"last":0.999},{"interface":"front","average":{"1min":6.5250000000000004,"5min":4.9359999999999999,"15min":4.6710000000000003},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":86.548000000000002,"5min":86.548000000000002,"15min":86.548000000000002},"last":0.93700000000000006}]},{"osd":5,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":12.161,"5min":3.2879999999999998,"15min":1.8089999999999999},"min":{"1min":0.72099999999999997,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":163.828,"5min":163.828,"15min":163.828},"last":0.66400000000000003},{"interface":"front","average":{"1min":2.6160000000000001,"5min":1.3660000000000001,"15min":1.1579999999999999},"min":{"1min":0.79200000000000004,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":36.119,"5min":36.119,"15min":36.119},"last":1.107}]},{"osd":6,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":5.2359999999999998,"5min":4.8529999999999998,"15min":4.7889999999999997},"min":{"1min":0.49199999999999999,"5min":0.441,"15min":0.441},"max":{"1min":86.766000000000005,"5min":86.766000000000005,"15min":86.766000000000005},"last":0.623},{"interface":"front","average":{"1min":5.3959999999999999,"5min":4.9770000000000003,"15min":4.907},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":83.171999999999997,"5min":90.188999999999993,"15min":90.188999999999993},"last":1.1479999999999999}]},{"osd":7,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":13.191000000000001,"5min":3.6019999999999999,"15min":2.004},"min":{"1min":0.84999999999999998,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":88.534000000000006,"5min":88.534000000000006,"15min":88.534000000000006},"last":0.89300000000000002},{"interface":"front","average":{"1min":15.949999999999999,"5min":4.2350000000000003,"15min":2.2829999999999999},"min":{"1min":0.75600000000000001,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":88.484999999999999,"5min":88.484999999999999,"15min":88.484999999999999},"last":1.073}]},{"osd":8,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":4.3179999999999996,"15min":4.8399999999999999},"min":{"1min":0.82899999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":2.0699999999999998,"5min":92.578999999999994,"15min":92.578999999999994},"last":0.97499999999999998},{"interface":"front","average":{"1min":2.8460000000000001,"5min":2.7799999999999998,"15min":2.7690000000000001},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":36.088000000000001,"5min":36.088000000000001,"15min":36.088000000000001},"last":1.3169999999999999}]},{"osd":12,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":2.9220000000000002,"5min":1.6719999999999999,"15min":1.464},"min":{"1min":0.872,"5min":0.872,"15min":0.872},"max":{"1min":36.014000000000003,"5min":36.014000000000003,"15min":36.014000000000003},"last":1.214},{"interface":"front","average":{"1min":5.5140000000000002,"5min":2.7469999999999999,"15min":2.286},"min":{"1min":0.73199999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":86.832999999999998,"5min":86.832999999999998,"15min":86.832999999999998},"last":1.1879999999999999}]},{"osd":13,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":11.115,"5min":8.3970000000000002,"15min":7.944},"min":{"1min":0.97999999999999998,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":80.683000000000007,"5min":141.84200000000001,"15min":141.84200000000001},"last":1.0289999999999999},{"interface":"front","average":{"1min":2.911,"5min":4.9489999999999998,"15min":5.2889999999999997},"min":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":35.841000000000001,"5min":97.441999999999993,"15min":97.441999999999993},"last":1.379}]},{"osd":15,"last update":"Wed Sep 18 10:16:34 2024","interfaces":[{"interface":"back","average":{"1min":12.472,"5min":3.6200000000000001,"15min":2.145},"min":{"1min":0.68400000000000005,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":86.980000000000004,"5min":86.980000000000004,"15min":86.980000000000004},"last":1.2809999999999999},{"interface":"front","average":{"1min":3.3479999999999999,"5min":3.798,"15min":3.8730000000000002},"min":{"1min":0.96999999999999997,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":45.356999999999999,"5min":61.694000000000003,"15min":61.694000000000003},"last":1.248}]}]},{"osd":3,"up_from":22,"seq":94489280549,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303960,"kb_used_data":2440,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94067880,"statfs":{"total":96636764160,"available":96325509120,"internally_reserved":0,"allocated":2498560,"data_stored":1455355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,4,6,7,9,10,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:16:37 2024","interfaces":[{"interface":"back","average":{"1min":4.173,"5min":2.048,"15min":1.694},"min":{"1min":0.64300000000000002,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":65.994,"5min":65.994,"15min":65.994},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.97899999999999998,"5min":8.8260000000000005,"15min":10.134},"min":{"1min":0.75,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.752,"5min":71.316999999999993,"15min":71.316999999999993},"last":0.59799999999999998}]},{"osd":2,"last update":"Wed Sep 18 10:16:37 2024","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":4.4379999999999997,"15min":5.0140000000000002},"min":{"1min":0.76900000000000002,"5min":0.75,"15min":0.75},"max":{"1min":1.333,"5min":56.523000000000003,"15min":56.523000000000003},"last":0.99099999999999999},{"interface":"front","average":{"1min":1.165,"5min":1.1200000000000001,"15min":1.1120000000000001},"min":{"1min":0.67400000000000004,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.639,"5min":3.327,"15min":3.327},"last":0.69199999999999995}]},{"osd":4,"last update":"Wed Sep 18 10:16:37 2024","interfaces":[{"interface":"back","average":{"1min":1.0409999999999999,"5min":5.7069999999999999,"15min":6.4850000000000003},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.6399999999999999,"5min":71.135000000000005,"15min":71.135000000000005},"last":0.72099999999999997},{"interface":"front","average":{"1min":1.635,"5min":3.8130000000000002,"15min":4.1760000000000002},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":10.039,"5min":56.253999999999998,"15min":56.253999999999998},"last":0.88500000000000001}]},{"osd":5,"last update":"Wed Sep 18 10:16:06 2024","interfaces":[{"interface":"back","average":{"1min":13.997,"5min":13.997,"15min":13.997},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":125.42400000000001,"5min":125.42400000000001,"15min":125.42400000000001},"last":1.131},{"interface":"front","average":{"1min":10.323,"5min":10.323,"15min":10.323},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":112.033,"5min":112.033,"15min":112.033},"last":0.96899999999999997}]},{"osd":6,"last update":"Wed Sep 18 10:16:37 2024","interfaces":[{"interface":"back","average":{"1min":1.9279999999999999,"5min":1.29,"15min":1.1839999999999999},"min":{"1min":0.93400000000000005,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":12.83,"5min":12.83,"15min":12.83},"last":1.1719999999999999},{"interface":"front","average":{"1min":5.0190000000000001,"5min":8.0050000000000008,"15min":8.5030000000000001},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":76.634,"5min":81.210999999999999,"15min":81.210999999999999},"last":0.78500000000000003}]},{"osd":7,"last update":"Wed Sep 18 10:16:37 2024","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":13.159000000000001,"15min":15.141},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.75,"5min":129.68299999999999,"15min":129.68299999999999},"last":0.63400000000000001},{"interface":"front","average":{"1min":1.4710000000000001,"5min":11.119999999999999,"15min":12.728},"min":{"1min":0.90800000000000003,"5min":0.621,"15min":0.621},"max":{"1min":1.8500000000000001,"5min":129.726,"15min":129.726},"last":1.0189999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0620000000000001}]},{"osd":9,"last update":"Wed Sep 18 10:16:37 2024","interfaces":[{"interface":"back","average":{"1min":1.5660000000000001,"5min":1.446,"15min":1.4259999999999999},"min":{"1min":0.94599999999999995,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":2.0209999999999999,"5min":2.0209999999999999,"15min":2.0209999999999999},"last":1.2250000000000001},{"interface":"front","average":{"1min":6.1050000000000004,"5min":6.1079999999999997,"15min":6.1079999999999997},"min":{"1min":0.86099999999999999,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":94.546999999999997,"5min":94.546999999999997,"15min":94.546999999999997},"last":1.052}]},{"osd":10,"last update":"Wed Sep 18 10:17:09 2024","interfaces":[{"interface":"back","average":{"1min":1.611,"5min":1.5109999999999999,"15min":1.4950000000000001},"min":{"1min":1.1259999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":2.0350000000000001,"5min":2.0659999999999998,"15min":2.0659999999999998},"last":1.1140000000000001},{"interface":"front","average":{"1min":1.4590000000000001,"5min":4.2389999999999999,"15min":4.7030000000000003},"min":{"1min":1.1000000000000001,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":1.794,"5min":71.668999999999997,"15min":71.668999999999997},"last":0.94699999999999995}]},{"osd":13,"last update":"Wed Sep 18 10:16:37 2024","interfaces":[{"interface":"back","average":{"1min":9.4909999999999997,"5min":3.113,"15min":2.0499999999999998},"min":{"1min":1.04,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":91.290999999999997,"5min":91.290999999999997,"15min":91.290999999999997},"last":1.2669999999999999},{"interface":"front","average":{"1min":1.5249999999999999,"5min":5.3300000000000001,"15min":5.9640000000000004},"min":{"1min":1.05,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.885,"5min":71.447000000000003,"15min":71.447000000000003},"last":0.85299999999999998}]},{"osd":14,"last update":"Wed Sep 18 10:16:37 2024","interfaces":[{"interface":"back","average":{"1min":1.6899999999999999,"5min":4.8899999999999997,"15min":5.423},"min":{"1min":1.0089999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"max":{"1min":4.625,"5min":56.664999999999999,"15min":56.664999999999999},"last":1.081},{"interface":"front","average":{"1min":1.865,"5min":2.2629999999999999,"15min":2.3290000000000002},"min":{"1min":1.1859999999999999,"5min":0.93999999999999995,"15min":0.93999999999999995},"max":{"1min":4.5910000000000002,"5min":18.523,"15min":18.523},"last":1.3089999999999999}]},{"osd":15,"last update":"Wed Sep 18 10:16:48 2024","interfaces":[{"interface":"back","average":{"1min":1.6970000000000001,"5min":4.0049999999999999,"15min":4.3899999999999997},"min":{"1min":1.01,"5min":0.91500000000000004,"15min":0.91500000000000004},"max":{"1min":4.3289999999999997,"5min":56.357999999999997,"15min":56.357999999999997},"last":1.153},{"interface":"front","average":{"1min":1.7589999999999999,"5min":8.5310000000000006,"15min":9.6600000000000001},"min":{"1min":1.133,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":2.1110000000000002,"5min":122.727,"15min":122.727},"last":1.3560000000000001}]}]},{"osd":5,"up_from":22,"seq":94489280549,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303956,"kb_used_data":2440,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94067884,"statfs":{"total":96636764160,"available":96325513216,"internally_reserved":0,"allocated":2498560,"data_stored":1455355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[2,3,4,6,8,9,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:15:47 2024","interfaces":[{"interface":"back","average":{"1min":7.8860000000000001,"5min":7.8860000000000001,"15min":7.8860000000000001},"min":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":84.430999999999997,"5min":84.430999999999997,"15min":84.430999999999997},"last":1.2889999999999999},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":1.353}]},{"osd":2,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":4.1360000000000001,"5min":8.6940000000000008,"15min":9.4540000000000006},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":51.091000000000001,"5min":84.566000000000003,"15min":84.566000000000003},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.98199999999999998,"5min":1.3300000000000001,"15min":1.3879999999999999},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.4690000000000001,"5min":9.5069999999999997,"15min":9.5069999999999997},"last":0.94899999999999995}]},{"osd":3,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":4.9240000000000004,"5min":7.8010000000000002,"15min":8.2810000000000006},"min":{"1min":0.69099999999999995,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":51.229999999999997,"5min":84.358999999999995,"15min":84.358999999999995},"last":1.0980000000000001},{"interface":"front","average":{"1min":8.1590000000000007,"5min":8.2100000000000009,"15min":8.218},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":74.222999999999999,"5min":97.549999999999997,"15min":97.549999999999997},"last":1.137}]},{"osd":4,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":25.673999999999999,"15min":29.777999999999999},"min":{"1min":0.67400000000000004,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.407,"5min":266.911,"15min":266.911},"last":1.226},{"interface":"front","average":{"1min":21.388000000000002,"5min":22.515999999999998,"15min":22.704000000000001},"min":{"1min":0.76200000000000001,"5min":0.626,"15min":0.626},"max":{"1min":171.691,"5min":260.53500000000003,"15min":260.53500000000003},"last":1.194}]},{"osd":6,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":3.6309999999999998,"5min":8.0809999999999995,"15min":8.8230000000000004},"min":{"1min":0.70099999999999996,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":41.886000000000003,"5min":58.171999999999997,"15min":58.171999999999997},"last":1.004},{"interface":"front","average":{"1min":5.8079999999999998,"5min":18.274999999999999,"15min":20.353000000000002},"min":{"1min":0.85699999999999998,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":54.984999999999999,"5min":227.48099999999999,"15min":227.48099999999999},"last":1.413}]},{"osd":7,"last update":"Wed Sep 18 10:15:30 2024","interfaces":[{"interface":"back","average":{"1min":12.013,"5min":12.013,"15min":12.013},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":123.111,"5min":123.111,"15min":123.111},"last":1.5249999999999999},{"interface":"front","average":{"1min":13.994,"5min":13.994,"15min":13.994},"min":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"max":{"1min":84.617000000000004,"5min":84.617000000000004,"15min":84.617000000000004},"last":1.329}]},{"osd":8,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":4.4720000000000004,"5min":17.175999999999998,"15min":19.292999999999999},"min":{"1min":0.879,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":51.506,"5min":260.44200000000001,"15min":260.44200000000001},"last":1.302},{"interface":"front","average":{"1min":8.2609999999999992,"5min":7.2859999999999996,"15min":7.1239999999999997},"min":{"1min":0.83899999999999997,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":71.266000000000005,"5min":106.818,"15min":106.818},"last":1.514}]},{"osd":9,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":3.8780000000000001,"5min":5.2930000000000001,"15min":5.5289999999999999},"min":{"1min":0.91300000000000003,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":41.652000000000001,"5min":80.272000000000006,"15min":80.272000000000006},"last":1.355},{"interface":"front","average":{"1min":1.3799999999999999,"5min":1.764,"15min":1.8280000000000001},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":2.008,"5min":9.5690000000000008,"15min":9.5690000000000008},"last":1.49}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6479999999999999}]},{"osd":11,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":1.494,"5min":7.8120000000000003,"15min":8.8650000000000002},"min":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":1.825,"5min":123.158,"15min":123.158},"last":1.5780000000000001},{"interface":"front","average":{"1min":8.8490000000000002,"5min":10.863,"15min":11.199},"min":{"1min":0.93200000000000005,"5min":0.86499999999999999,"15min":0.86499999999999999},"max":{"1min":68.938000000000002,"5min":84.522999999999996,"15min":84.522999999999996},"last":1.554}]},{"osd":12,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":4.0250000000000004,"5min":1.8580000000000001,"15min":1.4970000000000001},"min":{"1min":0.95199999999999996,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":41.929000000000002,"5min":41.929000000000002,"15min":41.929000000000002},"last":1.6679999999999999},{"interface":"front","average":{"1min":1.774,"5min":1.9339999999999999,"15min":1.9610000000000001},"min":{"1min":0.93999999999999995,"5min":0.88300000000000001,"15min":0.88300000000000001},"max":{"1min":4.1150000000000002,"5min":9.7710000000000008,"15min":9.7710000000000008},"last":1.7050000000000001}]},{"osd":13,"last update":"Wed Sep 18 10:16:48 2024","interfaces":[{"interface":"back","average":{"1min":1.6499999999999999,"5min":19.745999999999999,"15min":22.762},"min":{"1min":1.171,"5min":1.1120000000000001,"15min":1.1120000000000001},"max":{"1min":2.0550000000000002,"5min":140.273,"15min":140.273},"last":1.641},{"interface":"front","average":{"1min":4.665,"5min":20.309000000000001,"15min":22.917000000000002},"min":{"1min":1.194,"5min":0.98099999999999998,"15min":0.98099999999999998},"max":{"1min":51.432000000000002,"5min":170.86000000000001,"15min":170.86000000000001},"last":1.45}]},{"osd":14,"last update":"Wed Sep 18 10:16:48 2024","interfaces":[{"interface":"back","average":{"1min":9.8930000000000007,"5min":12.584,"15min":13.032},"min":{"1min":1.385,"5min":1.0389999999999999,"15min":1.0389999999999999},"max":{"1min":93.295000000000002,"5min":101.733,"15min":101.733},"last":1.784},{"interface":"front","average":{"1min":1.79,"5min":7.9729999999999999,"15min":9.0030000000000001},"min":{"1min":1.321,"5min":0.90100000000000002,"15min":0.90100000000000002},"max":{"1min":2.2290000000000001,"5min":122.952,"15min":122.952},"last":1.762}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8839999999999999}]}]},{"osd":6,"up_from":22,"seq":94489280549,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304480,"kb_used_data":2960,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94067360,"statfs":{"total":96636764160,"available":96324976640,"internally_reserved":0,"allocated":3031040,"data_stored":1984283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,3,4,5,7,8,9,11,12,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:15:47 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.68200000000000005}]},{"osd":1,"last update":"Wed Sep 18 10:16:48 2024","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":3.8100000000000001,"15min":4.306},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.2150000000000001,"5min":52.332999999999998,"15min":52.332999999999998},"last":0.73799999999999999},{"interface":"front","average":{"1min":4.8369999999999997,"5min":1.6990000000000001,"15min":1.1759999999999999},"min":{"1min":0.68300000000000005,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":67.763000000000005,"5min":67.763000000000005,"15min":67.763000000000005},"last":0.81100000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.343}]},{"osd":3,"last update":"Wed Sep 18 10:16:48 2024","interfaces":[{"interface":"back","average":{"1min":5.3380000000000001,"5min":1.792,"15min":1.2010000000000001},"min":{"1min":0.65500000000000003,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":76.891999999999996,"5min":76.891999999999996,"15min":76.891999999999996},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.98999999999999999,"5min":11.403,"15min":13.138999999999999},"min":{"1min":0.751,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.204,"5min":169.244,"15min":169.244},"last":1.091}]},{"osd":4,"last update":"Wed Sep 18 10:17:05 2024","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":9.0960000000000001,"15min":10.439},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.528,"5min":128.053,"15min":128.053},"last":0.87},{"interface":"front","average":{"1min":1.077,"5min":3.9100000000000001,"15min":4.3819999999999997},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.4430000000000001,"5min":67.960999999999999,"15min":67.960999999999999},"last":1.0029999999999999}]},{"osd":5,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":6.0579999999999998,"5min":10.194000000000001,"15min":10.884},"min":{"1min":0.72099999999999997,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":68.025999999999996,"5min":119.032,"15min":119.032},"last":1.131},{"interface":"front","average":{"1min":5.6760000000000002,"5min":4.0380000000000003,"15min":3.7650000000000001},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":83.858000000000004,"5min":83.858000000000004,"15min":83.858000000000004},"last":0.89300000000000002}]},{"osd":7,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":4.9539999999999997,"5min":6.3959999999999999,"15min":6.6360000000000001},"min":{"1min":0.94099999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":69.171000000000006,"5min":69.171000000000006,"15min":69.171000000000006},"last":0.97199999999999998},{"interface":"front","average":{"1min":5.0960000000000001,"5min":3.4630000000000001,"15min":3.1909999999999998},"min":{"1min":1.089,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":69.239000000000004,"5min":69.239000000000004,"15min":69.239000000000004},"last":1.226}]},{"osd":8,"last update":"Wed Sep 18 10:16:51 2024","interfaces":[{"interface":"back","average":{"1min":5.3659999999999997,"5min":5.3659999999999997,"15min":5.3659999999999997},"min":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"max":{"1min":76.766000000000005,"5min":76.766000000000005,"15min":76.766000000000005},"last":0.94699999999999995},{"interface":"front","average":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"min":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"max":{"1min":1.704,"5min":1.704,"15min":1.704},"last":1.2010000000000001}]},{"osd":9,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":5.4269999999999996,"5min":7.7089999999999996,"15min":8.0890000000000004},"min":{"1min":0.95799999999999996,"5min":0.84599999999999997,"15min":0.84599999999999997},"max":{"1min":76.997,"5min":95.691999999999993,"15min":95.691999999999993},"last":1.0589999999999999},{"interface":"front","average":{"1min":1.2949999999999999,"5min":10.472,"15min":12.002000000000001},"min":{"1min":0.96899999999999997,"5min":0.83099999999999996,"15min":0.83099999999999996},"max":{"1min":1.696,"5min":165.31100000000001,"15min":165.31100000000001},"last":1.252}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.329}]},{"osd":11,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":5.0510000000000002,"5min":2.0630000000000002,"15min":1.5649999999999999},"min":{"1min":1.002,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":67.909999999999997,"5min":67.909999999999997,"15min":67.909999999999997},"last":1.3160000000000001},{"interface":"front","average":{"1min":9.1829999999999998,"5min":4.9429999999999996,"15min":4.2359999999999998},"min":{"1min":1.1870000000000001,"5min":0.92000000000000004,"15min":0.92000000000000004},"max":{"1min":77.158000000000001,"5min":77.158000000000001,"15min":77.158000000000001},"last":1.1819999999999999}]},{"osd":12,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":1.3859999999999999,"5min":4.3140000000000001,"15min":4.8019999999999996},"min":{"1min":1.079,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":1.796,"5min":52.378,"15min":52.378},"last":1.345},{"interface":"front","average":{"1min":5.1109999999999998,"5min":4.5720000000000001,"15min":4.4820000000000002},"min":{"1min":0.92800000000000005,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":67.835999999999999,"5min":67.835999999999999,"15min":67.835999999999999},"last":1.427}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2050000000000001}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2669999999999999}]},{"osd":15,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":5.1680000000000001,"5min":2.2120000000000002,"15min":1.72},"min":{"1min":1.1000000000000001,"5min":0.90300000000000002,"15min":0.90300000000000002},"max":{"1min":68.069999999999993,"5min":68.069999999999993,"15min":68.069999999999993},"last":1.3779999999999999},{"interface":"front","average":{"1min":5.6200000000000001,"5min":4.3019999999999996,"15min":4.0819999999999999},"min":{"1min":1.2030000000000001,"5min":1.0429999999999999,"15min":1.0429999999999999},"max":{"1min":77.081999999999994,"5min":77.081999999999994,"15min":77.081999999999994},"last":1.2729999999999999}]}]},{"osd":7,"up_from":22,"seq":94489280549,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303956,"kb_used_data":2440,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94067884,"statfs":{"total":96636764160,"available":96325513216,"internally_reserved":0,"allocated":2498560,"data_stored":1455355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,4,6,8,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:16:33 2024","interfaces":[{"interface":"back","average":{"1min":3.9900000000000002,"5min":10.974,"15min":12.138},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":59.137999999999998,"5min":139.58600000000001,"15min":139.58600000000001},"last":0.95599999999999996},{"interface":"front","average":{"1min":1.121,"5min":1.0229999999999999,"15min":1.0069999999999999},"min":{"1min":0.67800000000000005,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.7390000000000001,"5min":1.7390000000000001,"15min":1.7390000000000001},"last":1.022}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2090000000000001}]},{"osd":2,"last update":"Wed Sep 18 10:15:46 2024","interfaces":[{"interface":"back","average":{"1min":15.436,"5min":15.436,"15min":15.436},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":139.36799999999999,"5min":139.36799999999999,"15min":139.36799999999999},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.94299999999999995}]},{"osd":3,"last update":"Wed Sep 18 10:15:46 2024","interfaces":[{"interface":"back","average":{"1min":17.327999999999999,"5min":17.327999999999999,"15min":17.327999999999999},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":139.27199999999999,"5min":139.27199999999999,"15min":139.27199999999999},"last":1.3460000000000001},{"interface":"front","average":{"1min":3.964,"5min":3.964,"15min":3.964},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":23.748999999999999,"5min":23.748999999999999,"15min":23.748999999999999},"last":0.82899999999999996}]},{"osd":4,"last update":"Wed Sep 18 10:16:37 2024","interfaces":[{"interface":"back","average":{"1min":9.8670000000000009,"5min":17.291,"15min":18.527999999999999},"min":{"1min":0.68000000000000005,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":125.10299999999999,"5min":139.51599999999999,"15min":139.51599999999999},"last":0.73099999999999998},{"interface":"front","average":{"1min":3.3559999999999999,"5min":4.6699999999999999,"15min":4.8890000000000002},"min":{"1min":0.68000000000000005,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":49.241999999999997,"5min":49.241999999999997,"15min":49.241999999999997},"last":0.77000000000000002}]},{"osd":5,"last update":"Wed Sep 18 10:15:46 2024","interfaces":[{"interface":"back","average":{"1min":1.099,"5min":1.099,"15min":1.099},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":2.3620000000000001,"5min":2.3620000000000001,"15min":2.3620000000000001},"last":1.4059999999999999},{"interface":"front","average":{"1min":17.736999999999998,"5min":17.736999999999998,"15min":17.736999999999998},"min":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"max":{"1min":139.44300000000001,"5min":139.44300000000001,"15min":139.44300000000001},"last":0.78000000000000003}]},{"osd":6,"last update":"Wed Sep 18 10:16:33 2024","interfaces":[{"interface":"back","average":{"1min":6.0869999999999997,"5min":15.117000000000001,"15min":16.622},"min":{"1min":0.998,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":59.343000000000004,"5min":183.001,"15min":183.001},"last":1.0680000000000001},{"interface":"front","average":{"1min":7.0830000000000002,"5min":4.8650000000000002,"15min":4.4950000000000001},"min":{"1min":0.82399999999999995,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":58.284999999999997,"5min":58.284999999999997,"15min":58.284999999999997},"last":0.90000000000000002}]},{"osd":8,"last update":"Wed Sep 18 10:16:30 2024","interfaces":[{"interface":"back","average":{"1min":1.4079999999999999,"5min":1.2809999999999999,"15min":1.26},"min":{"1min":0.84999999999999998,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.9339999999999999,"5min":2.6139999999999999,"15min":2.6139999999999999},"last":0.82999999999999996},{"interface":"front","average":{"1min":5.3029999999999999,"5min":13.785,"15min":15.199},"min":{"1min":1.0980000000000001,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":59.523000000000003,"5min":139.798,"15min":139.798},"last":1.214}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.861}]},{"osd":10,"last update":"Wed Sep 18 10:16:30 2024","interfaces":[{"interface":"back","average":{"1min":16.035,"5min":8.4100000000000001,"15min":7.1390000000000002},"min":{"1min":1.0149999999999999,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":156.45500000000001,"5min":156.45500000000001,"15min":156.45500000000001},"last":1.1180000000000001},{"interface":"front","average":{"1min":1.4910000000000001,"5min":1.391,"15min":1.3740000000000001},"min":{"1min":0.83499999999999996,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":2.1200000000000001,"5min":2.254,"15min":2.254},"last":1.1499999999999999}]},{"osd":11,"last update":"Wed Sep 18 10:16:33 2024","interfaces":[{"interface":"back","average":{"1min":5.0460000000000003,"5min":5.4459999999999997,"15min":5.5129999999999999},"min":{"1min":1.046,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":49.154000000000003,"5min":62.356000000000002,"15min":62.356000000000002},"last":1.1899999999999999},{"interface":"front","average":{"1min":2.4769999999999999,"5min":1.6240000000000001,"15min":1.482},"min":{"1min":1.0189999999999999,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":20.449999999999999,"5min":20.449999999999999,"15min":20.449999999999999},"last":1.3029999999999999}]},{"osd":12,"last update":"Wed Sep 18 10:17:12 2024","interfaces":[{"interface":"back","average":{"1min":1.4950000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"min":{"1min":0.754,"5min":0.754,"15min":0.754},"max":{"1min":2.335,"5min":2.335,"15min":2.335},"last":1.2390000000000001},{"interface":"front","average":{"1min":1.544,"5min":1.544,"15min":1.544},"min":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"max":{"1min":2.2160000000000002,"5min":2.2160000000000002,"15min":2.2160000000000002},"last":1.4119999999999999}]},{"osd":13,"last update":"Wed Sep 18 10:16:33 2024","interfaces":[{"interface":"back","average":{"1min":10.028,"5min":16.707999999999998,"15min":17.821999999999999},"min":{"1min":1.1459999999999999,"5min":0.874,"15min":0.874},"max":{"1min":113.428,"5min":139.655,"15min":139.655},"last":1.4510000000000001},{"interface":"front","average":{"1min":5.1970000000000001,"5min":16.942,"15min":18.899999999999999},"min":{"1min":1.077,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":49.457999999999998,"5min":173.113,"15min":173.113},"last":1.4279999999999999}]},{"osd":14,"last update":"Wed Sep 18 10:17:12 2024","interfaces":[{"interface":"back","average":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"min":{"1min":1.139,"5min":1.139,"15min":1.139},"max":{"1min":2.1230000000000002,"5min":2.1230000000000002,"15min":2.1230000000000002},"last":1.335},{"interface":"front","average":{"1min":1.6919999999999999,"5min":1.6919999999999999,"15min":1.6919999999999999},"min":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"max":{"1min":2.4929999999999999,"5min":2.4929999999999999,"15min":2.4929999999999999},"last":1.3620000000000001}]},{"osd":15,"last update":"Wed Sep 18 10:17:12 2024","interfaces":[{"interface":"back","average":{"1min":1.6399999999999999,"5min":1.6399999999999999,"15min":1.6399999999999999},"min":{"1min":1.198,"5min":1.198,"15min":1.198},"max":{"1min":2.3210000000000002,"5min":2.3210000000000002,"15min":2.3210000000000002},"last":1.4670000000000001},{"interface":"front","average":{"1min":1.637,"5min":1.637,"15min":1.637},"min":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"max":{"1min":2.2010000000000001,"5min":2.2010000000000001,"15min":2.2010000000000001},"last":1.393}]}]},{"osd":8,"up_from":22,"seq":94489280549,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303956,"kb_used_data":2440,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94067884,"statfs":{"total":96636764160,"available":96325513216,"internally_reserved":0,"allocated":2498560,"data_stored":1455355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[2,4,6,7,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93300000000000005}]},{"osd":1,"last update":"Wed Sep 18 10:16:31 2024","interfaces":[{"interface":"back","average":{"1min":1.117,"5min":1.117,"15min":1.117},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.669,"5min":1.669,"15min":1.669},"last":1.2230000000000001},{"interface":"front","average":{"1min":2.3290000000000002,"5min":2.3290000000000002,"15min":2.3290000000000002},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":29.523,"5min":29.523,"15min":29.523},"last":1.1279999999999999}]},{"osd":2,"last update":"Wed Sep 18 10:16:48 2024","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":5.5140000000000002,"15min":6.2300000000000004},"min":{"1min":0.93500000000000005,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.8,"5min":77.435000000000002,"15min":77.435000000000002},"last":1.274},{"interface":"front","average":{"1min":1.1639999999999999,"5min":0.97099999999999997,"15min":0.93899999999999995},"min":{"1min":0.71999999999999997,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.903,"5min":1.903,"15min":1.903},"last":0.85299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.306}]},{"osd":4,"last update":"Wed Sep 18 10:16:48 2024","interfaces":[{"interface":"back","average":{"1min":1.23,"5min":9.5879999999999992,"15min":10.981},"min":{"1min":0.71899999999999997,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":7.1559999999999997,"5min":90.438000000000002,"15min":90.438000000000002},"last":0.89500000000000002},{"interface":"front","average":{"1min":1.1699999999999999,"5min":10.691000000000001,"15min":12.278},"min":{"1min":0.66100000000000003,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":2.1309999999999998,"5min":93.939999999999998,"15min":93.939999999999998},"last":0.93899999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4199999999999999}]},{"osd":6,"last update":"Wed Sep 18 10:16:48 2024","interfaces":[{"interface":"back","average":{"1min":1.359,"5min":15.042999999999999,"15min":17.324000000000002},"min":{"1min":0.90700000000000003,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":2.0350000000000001,"5min":293.26900000000001,"15min":293.26900000000001},"last":1.02},{"interface":"front","average":{"1min":1.2230000000000001,"5min":5.1689999999999996,"15min":5.827},"min":{"1min":0.88900000000000001,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.839,"5min":70.203000000000003,"15min":70.203000000000003},"last":1.1419999999999999}]},{"osd":7,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":7.1950000000000003,"5min":3.6070000000000002,"15min":3.0089999999999999},"min":{"1min":0.80900000000000005,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":80.635000000000005,"5min":80.635000000000005,"15min":80.635000000000005},"last":1.46},{"interface":"front","average":{"1min":9.3490000000000002,"5min":6.726,"15min":6.2889999999999997},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":99.060000000000002,"5min":99.060000000000002,"15min":99.060000000000002},"last":1.1930000000000001}]},{"osd":9,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":4.8730000000000002,"5min":3.0110000000000001,"15min":2.7010000000000001},"min":{"1min":0.92700000000000005,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":80.076999999999998,"5min":80.076999999999998,"15min":80.076999999999998},"last":1.431},{"interface":"front","average":{"1min":1.6240000000000001,"5min":7.4489999999999998,"15min":8.4199999999999999},"min":{"1min":1.196,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":2.5190000000000001,"5min":98.914000000000001,"15min":98.914000000000001},"last":1.6419999999999999}]},{"osd":10,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":1.6319999999999999,"5min":5.8559999999999999,"15min":6.5599999999999996},"min":{"1min":0.99099999999999999,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":2.3679999999999999,"5min":90.751000000000005,"15min":90.751000000000005},"last":1.248},{"interface":"front","average":{"1min":1.591,"5min":7.7889999999999997,"15min":8.8219999999999992},"min":{"1min":1.0640000000000001,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":2.3340000000000001,"5min":83.795000000000002,"15min":83.795000000000002},"last":1.3149999999999999}]},{"osd":11,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":1.7450000000000001,"5min":2.669,"15min":2.823},"min":{"1min":1.3340000000000001,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":2.383,"5min":26.462,"15min":26.462},"last":1.7290000000000001},{"interface":"front","average":{"1min":1.752,"5min":1.948,"15min":1.9810000000000001},"min":{"1min":1.24,"5min":0.99299999999999999,"15min":0.99299999999999999},"max":{"1min":2.3479999999999999,"5min":12.475,"15min":12.475},"last":1.5940000000000001}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0760000000000001}]},{"osd":13,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":3.052,"5min":7.9329999999999998,"15min":8.7469999999999999},"min":{"1min":1.05,"5min":1.0429999999999999,"15min":1.0429999999999999},"max":{"1min":30.582999999999998,"5min":94.222999999999999,"15min":94.222999999999999},"last":1.492},{"interface":"front","average":{"1min":1.635,"5min":2.5059999999999998,"15min":2.6509999999999998},"min":{"1min":1.1180000000000001,"5min":0.93400000000000005,"15min":0.93400000000000005},"max":{"1min":2.4689999999999999,"5min":23.983000000000001,"15min":23.983000000000001},"last":1.518}]},{"osd":14,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":1.802,"5min":16.975000000000001,"15min":19.504000000000001},"min":{"1min":1.28,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":2.4500000000000002,"5min":150.93600000000001,"15min":150.93600000000001},"last":1.8260000000000001},{"interface":"front","average":{"1min":4.8170000000000002,"5min":3.8410000000000002,"15min":3.6789999999999998},"min":{"1min":1.2210000000000001,"5min":0.90200000000000002,"15min":0.90200000000000002},"max":{"1min":69.918000000000006,"5min":69.918000000000006,"15min":69.918000000000006},"last":1.5509999999999999}]},{"osd":15,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":5.9089999999999998,"5min":3.4169999999999998,"15min":3.0019999999999998},"min":{"1min":1.2549999999999999,"5min":0.93300000000000005,"15min":0.93300000000000005},"max":{"1min":98.558000000000007,"5min":98.558000000000007,"15min":98.558000000000007},"last":1.619},{"interface":"front","average":{"1min":1.8400000000000001,"5min":2.8119999999999998,"15min":2.9740000000000002},"min":{"1min":1.3160000000000001,"5min":0.95299999999999996,"15min":0.95299999999999996},"max":{"1min":2.5859999999999999,"5min":26.414000000000001,"15min":26.414000000000001},"last":1.7789999999999999}]}]},{"osd":10,"up_from":22,"seq":94489280550,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303960,"kb_used_data":2440,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94067880,"statfs":{"total":96636764160,"available":96325509120,"internally_reserved":0,"allocated":2498560,"data_stored":1455355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,4,5,6,7,9,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:16:29 2024","interfaces":[{"interface":"back","average":{"1min":16.117000000000001,"5min":16.117000000000001,"15min":16.117000000000001},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":145.12799999999999,"5min":145.12799999999999,"15min":145.12799999999999},"last":1.401},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.431,"5min":1.431,"15min":1.431},"last":1.0780000000000001}]},{"osd":1,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":9.5719999999999992,"5min":10.66,"15min":10.842000000000001},"min":{"1min":0.88500000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":145.20699999999999,"5min":145.20699999999999,"15min":145.20699999999999},"last":0.84499999999999997},{"interface":"front","average":{"1min":4.7510000000000003,"5min":14.728,"15min":16.390999999999998},"min":{"1min":0.754,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":38.338000000000001,"5min":99.995999999999995,"15min":99.995999999999995},"last":0.76000000000000001}]},{"osd":2,"last update":"Wed Sep 18 10:15:33 2024","interfaces":[{"interface":"back","average":{"1min":10.893000000000001,"5min":10.893000000000001,"15min":10.893000000000001},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":84.072000000000003,"5min":84.072000000000003,"15min":84.072000000000003},"last":0.99199999999999999},{"interface":"front","average":{"1min":11.550000000000001,"5min":11.550000000000001,"15min":11.550000000000001},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":99.798000000000002,"5min":99.798000000000002,"15min":99.798000000000002},"last":1.425}]},{"osd":3,"last update":"Wed Sep 18 10:16:22 2024","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":1.119,"15min":1.119},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.99,"5min":1.99,"15min":1.99},"last":1.5},{"interface":"front","average":{"1min":18.094999999999999,"5min":18.094999999999999,"15min":18.094999999999999},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":145.535,"5min":145.535,"15min":145.535},"last":1.4510000000000001}]},{"osd":4,"last update":"Wed Sep 18 10:16:49 2024","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":20.715,"15min":23.963999999999999},"min":{"1min":0.92300000000000004,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.5820000000000001,"5min":125.23099999999999,"15min":125.23099999999999},"last":1.1879999999999999},{"interface":"front","average":{"1min":1.206,"5min":25.175000000000001,"15min":29.170000000000002},"min":{"1min":0.88800000000000001,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.4930000000000001,"5min":295.93200000000002,"15min":295.93200000000002},"last":0.79900000000000004}]},{"osd":5,"last update":"Wed Sep 18 10:16:49 2024","interfaces":[{"interface":"back","average":{"1min":8.9960000000000004,"5min":21.460999999999999,"15min":23.539000000000001},"min":{"1min":0.81000000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":145.24000000000001,"5min":215.51400000000001,"15min":215.51400000000001},"last":1.093},{"interface":"front","average":{"1min":2.4140000000000001,"5min":22.346,"15min":25.667999999999999},"min":{"1min":0.85399999999999998,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":20.302,"5min":252.69300000000001,"15min":252.69300000000001},"last":1.0449999999999999}]},{"osd":6,"last update":"Wed Sep 18 10:16:22 2024","interfaces":[{"interface":"back","average":{"1min":12.393000000000001,"5min":12.393000000000001,"15min":12.393000000000001},"min":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"max":{"1min":111.173,"5min":111.173,"15min":111.173},"last":0.95899999999999996},{"interface":"front","average":{"1min":15.771000000000001,"5min":15.771000000000001,"15min":15.771000000000001},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":221.13900000000001,"5min":221.13900000000001,"15min":221.13900000000001},"last":0.998}]},{"osd":7,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":1.5109999999999999,"5min":2.7490000000000001,"15min":2.9550000000000001},"min":{"1min":0.95699999999999996,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.919,"5min":34.558,"15min":34.558},"last":1.075},{"interface":"front","average":{"1min":10.101000000000001,"5min":13.186999999999999,"15min":13.701000000000001},"min":{"1min":1.0920000000000001,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":145.5,"5min":145.5,"15min":145.5},"last":1.3220000000000001}]},{"osd":9,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":8.3640000000000008,"5min":11.07,"15min":11.521000000000001},"min":{"1min":1.0349999999999999,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":119.202,"5min":119.202,"15min":119.202},"last":1.347},{"interface":"front","average":{"1min":1.629,"5min":10.364000000000001,"15min":11.82},"min":{"1min":1.073,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":2.23,"5min":97.385999999999996,"15min":97.385999999999996},"last":1.133}]},{"osd":11,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":10.132999999999999,"5min":11.489000000000001,"15min":11.715},"min":{"1min":1.286,"5min":0.85699999999999998,"15min":0.85699999999999998},"max":{"1min":145.26900000000001,"5min":145.26900000000001,"15min":145.26900000000001},"last":1.2569999999999999},{"interface":"front","average":{"1min":5.6719999999999997,"5min":19.837,"15min":22.198},"min":{"1min":1.28,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":44.121000000000002,"5min":295.70400000000001,"15min":295.70400000000001},"last":1.4119999999999999}]},{"osd":12,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":1.6559999999999999,"5min":2.8250000000000002,"15min":3.02},"min":{"1min":1.0620000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":2.2559999999999998,"5min":34.625999999999998,"15min":34.625999999999998},"last":1.2889999999999999},{"interface":"front","average":{"1min":10.140000000000001,"5min":10.986000000000001,"15min":11.127000000000001},"min":{"1min":1.198,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":145.358,"5min":145.358,"15min":145.358},"last":1.3759999999999999}]},{"osd":13,"last update":"Wed Sep 18 10:16:36 2024","interfaces":[{"interface":"back","average":{"1min":10.433999999999999,"5min":17.994,"15min":19.254000000000001},"min":{"1min":1.1990000000000001,"5min":0.877,"15min":0.877},"max":{"1min":123.696,"5min":123.696,"15min":123.696},"last":1.5049999999999999},{"interface":"front","average":{"1min":7.6379999999999999,"5min":14.551,"15min":15.702999999999999},"min":{"1min":1.163,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":68.363,"5min":85.879999999999995,"15min":85.879999999999995},"last":1.397}]},{"osd":14,"last update":"Wed Sep 18 10:16:22 2024","interfaces":[{"interface":"back","average":{"1min":15.864000000000001,"5min":15.864000000000001,"15min":15.864000000000001},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":145.46700000000001,"5min":145.46700000000001,"15min":145.46700000000001},"last":1.4350000000000001},{"interface":"front","average":{"1min":13.93,"5min":13.93,"15min":13.93},"min":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"max":{"1min":100.092,"5min":100.092,"15min":100.092},"last":1.46}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1080000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T10:17:27.127 DEBUG:teuthology.orchestra.run.smithi029:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-18T10:17:27.449 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-18T10:17:27.449 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-18T10:17:27.449 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-18T10:17:27.449 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T10:17:27.531 INFO:tasks.ceph.osd.0:Stopped 2024-09-18T10:17:27.532 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-18T10:17:27.532 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T10:17:27.607 INFO:tasks.ceph.osd.4:Stopped 2024-09-18T10:17:27.607 DEBUG:tasks.ceph.osd.8:waiting for process to exit 2024-09-18T10:17:27.607 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T10:17:27.701 INFO:tasks.ceph.osd.8:Stopped 2024-09-18T10:17:27.702 DEBUG:tasks.ceph.osd.12:waiting for process to exit 2024-09-18T10:17:27.702 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T10:17:27.772 INFO:tasks.ceph.osd.12:Stopped 2024-09-18T10:17:27.772 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-18T10:17:27.772 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T10:17:27.831 INFO:tasks.ceph.osd.1:Stopped 2024-09-18T10:17:27.831 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-18T10:17:27.831 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T10:17:27.900 INFO:tasks.ceph.osd.5:Stopped 2024-09-18T10:17:27.900 DEBUG:tasks.ceph.osd.9:waiting for process to exit 2024-09-18T10:17:27.900 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T10:17:27.963 INFO:tasks.ceph.osd.9:Stopped 2024-09-18T10:17:27.964 DEBUG:tasks.ceph.osd.13:waiting for process to exit 2024-09-18T10:17:27.964 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T10:17:28.020 INFO:tasks.ceph.osd.13:Stopped 2024-09-18T10:17:28.021 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-18T10:17:28.021 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T10:17:28.089 INFO:tasks.ceph.osd.2:Stopped 2024-09-18T10:17:28.089 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-09-18T10:17:28.089 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T10:17:28.171 INFO:tasks.ceph.osd.6:Stopped 2024-09-18T10:17:28.171 DEBUG:tasks.ceph.osd.10:waiting for process to exit 2024-09-18T10:17:28.171 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T10:17:28.251 INFO:tasks.ceph.osd.10:Stopped 2024-09-18T10:17:28.252 DEBUG:tasks.ceph.osd.14:waiting for process to exit 2024-09-18T10:17:28.252 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T10:17:28.340 INFO:tasks.ceph.osd.14:Stopped 2024-09-18T10:17:28.340 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-18T10:17:28.340 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T10:17:28.408 INFO:tasks.ceph.osd.3:Stopped 2024-09-18T10:17:28.409 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-09-18T10:17:28.409 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T10:17:28.499 INFO:tasks.ceph.osd.7:Stopped 2024-09-18T10:17:28.500 DEBUG:tasks.ceph.osd.11:waiting for process to exit 2024-09-18T10:17:28.500 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T10:17:28.588 INFO:tasks.ceph.osd.11:Stopped 2024-09-18T10:17:28.588 DEBUG:tasks.ceph.osd.15:waiting for process to exit 2024-09-18T10:17:28.589 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T10:17:28.662 INFO:tasks.ceph.osd.15:Stopped 2024-09-18T10:17:28.663 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-18T10:17:28.663 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-09-18T10:17:28.663 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T10:17:28.693 INFO:tasks.ceph.mgr.y:Stopped 2024-09-18T10:17:28.694 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-18T10:17:28.694 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T10:17:28.728 INFO:tasks.ceph.mgr.x:Stopped 2024-09-18T10:17:28.729 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-18T10:17:28.729 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-18T10:17:28.729 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T10:17:28.750 INFO:tasks.ceph.mon.a:Stopped 2024-09-18T10:17:28.750 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-18T10:17:28.750 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T10:17:28.768 INFO:tasks.ceph.mon.b:Stopped 2024-09-18T10:17:28.769 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-18T10:17:28.769 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T10:17:28.789 INFO:tasks.ceph.mon.c:Stopped 2024-09-18T10:17:28.789 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-18T10:17:28.789 DEBUG:teuthology.orchestra.run.smithi029:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'but it is still running' | egrep -v 'objects unfound and apparently lost' | egrep -v 'slow request' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'mons down' | egrep -v 'mon down' | egrep -v 'out of quorum' | head -n 1 2024-09-18T10:17:28.812 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi029.front.sepia.ceph.com 2024-09-18T10:17:28.812 DEBUG:teuthology.orchestra.run.smithi029:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-18T10:17:29.165 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi029.front.sepia.ceph.com 2024-09-18T10:17:29.166 DEBUG:teuthology.orchestra.run.smithi029:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-18T10:17:29.414 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-8 on ubuntu@smithi029.front.sepia.ceph.com 2024-09-18T10:17:29.415 DEBUG:teuthology.orchestra.run.smithi029:> sync && sudo umount -f /var/lib/ceph/osd/ceph-8 2024-09-18T10:17:29.564 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-12 on ubuntu@smithi029.front.sepia.ceph.com 2024-09-18T10:17:29.564 DEBUG:teuthology.orchestra.run.smithi029:> sync && sudo umount -f /var/lib/ceph/osd/ceph-12 2024-09-18T10:17:29.730 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi136.front.sepia.ceph.com 2024-09-18T10:17:29.730 DEBUG:teuthology.orchestra.run.smithi136:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-18T10:17:29.937 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi136.front.sepia.ceph.com 2024-09-18T10:17:29.937 DEBUG:teuthology.orchestra.run.smithi136:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-18T10:17:30.266 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-9 on ubuntu@smithi136.front.sepia.ceph.com 2024-09-18T10:17:30.266 DEBUG:teuthology.orchestra.run.smithi136:> sync && sudo umount -f /var/lib/ceph/osd/ceph-9 2024-09-18T10:17:30.467 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-13 on ubuntu@smithi136.front.sepia.ceph.com 2024-09-18T10:17:30.467 DEBUG:teuthology.orchestra.run.smithi136:> sync && sudo umount -f /var/lib/ceph/osd/ceph-13 2024-09-18T10:17:30.649 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi154.front.sepia.ceph.com 2024-09-18T10:17:30.649 DEBUG:teuthology.orchestra.run.smithi154:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-18T10:17:30.888 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi154.front.sepia.ceph.com 2024-09-18T10:17:30.888 DEBUG:teuthology.orchestra.run.smithi154:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-09-18T10:17:31.113 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-10 on ubuntu@smithi154.front.sepia.ceph.com 2024-09-18T10:17:31.113 DEBUG:teuthology.orchestra.run.smithi154:> sync && sudo umount -f /var/lib/ceph/osd/ceph-10 2024-09-18T10:17:31.238 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-14 on ubuntu@smithi154.front.sepia.ceph.com 2024-09-18T10:17:31.239 DEBUG:teuthology.orchestra.run.smithi154:> sync && sudo umount -f /var/lib/ceph/osd/ceph-14 2024-09-18T10:17:31.373 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi189.front.sepia.ceph.com 2024-09-18T10:17:31.373 DEBUG:teuthology.orchestra.run.smithi189:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-18T10:17:31.586 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi189.front.sepia.ceph.com 2024-09-18T10:17:31.586 DEBUG:teuthology.orchestra.run.smithi189:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-09-18T10:17:31.828 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-11 on ubuntu@smithi189.front.sepia.ceph.com 2024-09-18T10:17:31.828 DEBUG:teuthology.orchestra.run.smithi189:> sync && sudo umount -f /var/lib/ceph/osd/ceph-11 2024-09-18T10:17:32.042 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-15 on ubuntu@smithi189.front.sepia.ceph.com 2024-09-18T10:17:32.042 DEBUG:teuthology.orchestra.run.smithi189:> sync && sudo umount -f /var/lib/ceph/osd/ceph-15 2024-09-18T10:17:32.277 INFO:tasks.ceph:Archiving mon data... 2024-09-18T10:17:32.278 DEBUG:teuthology.misc:Transferring archived files from smithi029:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/skanta-2024-09-18_09:13:00-rados-wip-bharath11-testing-2024-09-18-1115-quincy-distro-default-smithi/7910845/data/mon.a.tgz 2024-09-18T10:17:32.278 DEBUG:teuthology.orchestra.run.smithi029:> mktemp 2024-09-18T10:17:32.285 INFO:teuthology.orchestra.run.smithi029.stdout:/tmp/tmp.ySt18bVnto 2024-09-18T10:17:32.286 DEBUG:teuthology.orchestra.run.smithi029:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.ySt18bVnto 2024-09-18T10:17:32.400 DEBUG:teuthology.orchestra.run.smithi029:> sudo chmod 0666 /tmp/tmp.ySt18bVnto 2024-09-18T10:17:32.421 DEBUG:teuthology.orchestra.remote:smithi029:/tmp/tmp.ySt18bVnto is 429KB 2024-09-18T10:17:32.484 DEBUG:teuthology.orchestra.run.smithi029:> rm -fr /tmp/tmp.ySt18bVnto 2024-09-18T10:17:32.491 DEBUG:teuthology.misc:Transferring archived files from smithi136:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/skanta-2024-09-18_09:13:00-rados-wip-bharath11-testing-2024-09-18-1115-quincy-distro-default-smithi/7910845/data/mon.b.tgz 2024-09-18T10:17:32.492 DEBUG:teuthology.orchestra.run.smithi136:> mktemp 2024-09-18T10:17:32.498 INFO:teuthology.orchestra.run.smithi136.stdout:/tmp/tmp.7jPhDeDvCS 2024-09-18T10:17:32.499 DEBUG:teuthology.orchestra.run.smithi136:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.7jPhDeDvCS 2024-09-18T10:17:32.646 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod 0666 /tmp/tmp.7jPhDeDvCS 2024-09-18T10:17:32.674 DEBUG:teuthology.orchestra.remote:smithi136:/tmp/tmp.7jPhDeDvCS is 620KB 2024-09-18T10:17:32.740 DEBUG:teuthology.orchestra.run.smithi136:> rm -fr /tmp/tmp.7jPhDeDvCS 2024-09-18T10:17:32.747 DEBUG:teuthology.misc:Transferring archived files from smithi154:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/skanta-2024-09-18_09:13:00-rados-wip-bharath11-testing-2024-09-18-1115-quincy-distro-default-smithi/7910845/data/mon.c.tgz 2024-09-18T10:17:32.747 DEBUG:teuthology.orchestra.run.smithi154:> mktemp 2024-09-18T10:17:32.754 INFO:teuthology.orchestra.run.smithi154.stdout:/tmp/tmp.1of1R0Rm88 2024-09-18T10:17:32.754 DEBUG:teuthology.orchestra.run.smithi154:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.1of1R0Rm88 2024-09-18T10:17:32.905 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod 0666 /tmp/tmp.1of1R0Rm88 2024-09-18T10:17:32.928 DEBUG:teuthology.orchestra.remote:smithi154:/tmp/tmp.1of1R0Rm88 is 620KB 2024-09-18T10:17:32.989 DEBUG:teuthology.orchestra.run.smithi154:> rm -fr /tmp/tmp.1of1R0Rm88 2024-09-18T10:17:32.996 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-18T10:17:32.996 DEBUG:teuthology.orchestra.run.smithi029:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-18T10:17:32.998 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-18T10:17:33.000 DEBUG:teuthology.orchestra.run.smithi154:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-18T10:17:33.003 DEBUG:teuthology.orchestra.run.smithi189:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-18T10:17:33.019 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2024-09-18T10:17:33.019 DEBUG:teuthology.orchestra.run.smithi029:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-09-18T10:17:33.021 DEBUG:teuthology.orchestra.run.smithi136:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-09-18T10:17:33.053 INFO:teuthology.orchestra.run.smithi029.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-09-18T10:17:33.055 DEBUG:teuthology.orchestra.run.smithi154:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-09-18T10:17:33.060 DEBUG:teuthology.orchestra.run.smithi189:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-09-18T10:17:33.080 INFO:teuthology.orchestra.run.smithi136.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-09-18T10:17:33.083 INFO:teuthology.orchestra.run.smithi154.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-09-18T10:17:33.087 INFO:teuthology.orchestra.run.smithi189.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-09-18T10:17:33.093 INFO:tasks.ceph:Archiving crash dumps... 2024-09-18T10:17:33.095 DEBUG:teuthology.misc:Transferring archived files from smithi029:/var/lib/ceph/crash to /home/teuthworker/archive/skanta-2024-09-18_09:13:00-rados-wip-bharath11-testing-2024-09-18-1115-quincy-distro-default-smithi/7910845/remote/smithi029/crash 2024-09-18T10:17:33.096 DEBUG:teuthology.orchestra.run.smithi029:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-18T10:17:33.123 DEBUG:teuthology.misc:Transferring archived files from smithi136:/var/lib/ceph/crash to /home/teuthworker/archive/skanta-2024-09-18_09:13:00-rados-wip-bharath11-testing-2024-09-18-1115-quincy-distro-default-smithi/7910845/remote/smithi136/crash 2024-09-18T10:17:33.124 DEBUG:teuthology.orchestra.run.smithi136:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-18T10:17:33.142 DEBUG:teuthology.misc:Transferring archived files from smithi154:/var/lib/ceph/crash to /home/teuthworker/archive/skanta-2024-09-18_09:13:00-rados-wip-bharath11-testing-2024-09-18-1115-quincy-distro-default-smithi/7910845/remote/smithi154/crash 2024-09-18T10:17:33.143 DEBUG:teuthology.orchestra.run.smithi154:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-18T10:17:33.160 DEBUG:teuthology.misc:Transferring archived files from smithi189:/var/lib/ceph/crash to /home/teuthworker/archive/skanta-2024-09-18_09:13:00-rados-wip-bharath11-testing-2024-09-18-1115-quincy-distro-default-smithi/7910845/remote/smithi189/crash 2024-09-18T10:17:33.161 DEBUG:teuthology.orchestra.run.smithi189:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-18T10:17:33.178 INFO:tasks.ceph:Compressing logs... 2024-09-18T10:17:33.178 DEBUG:teuthology.orchestra.run.smithi029:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-18T10:17:33.181 DEBUG:teuthology.orchestra.run.smithi136:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-18T10:17:33.183 DEBUG:teuthology.orchestra.run.smithi154:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-18T10:17:33.202 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-09-18T10:17:33.203 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-09-18T10:17:33.203 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-18T10:17:33.204 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.13.log 2024-09-18T10:17:33.204 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-osd.5.log: /var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.9.log 2024-09-18T10:17:33.205 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-osd.13.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-18T10:17:33.206 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-18T10:17:33.206 INFO:teuthology.orchestra.run.smithi136.stderr: 88.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-18T10:17:33.206 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32399.log 2024-09-18T10:17:33.208 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29421.log 2024-09-18T10:17:33.209 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28951.log 2024-09-18T10:17:33.209 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.32399.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34181.log 2024-09-18T10:17:33.209 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32399.log.gz 2024-09-18T10:17:33.209 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.28951.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29568.log 2024-09-18T10:17:33.209 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.29421.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28951.log.gz 2024-09-18T10:17:33.209 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29421.log.gz 2024-09-18T10:17:33.209 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.34181.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32264.log 2024-09-18T10:17:33.209 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34181.log.gz 2024-09-18T10:17:33.210 DEBUG:teuthology.orchestra.run.smithi189:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-18T10:17:33.214 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33107.log 2024-09-18T10:17:33.214 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.29568.log: gzip -5 --verbose 0.0%-- /var/log/ceph/ceph-client.admin.26661.log -- replaced with /var/log/ceph/ceph-client.admin.29568.log.gz 2024-09-18T10:17:33.214 INFO:teuthology.orchestra.run.smithi029.stderr: 2024-09-18T10:17:33.214 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.32264.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32264.log.gz 2024-09-18T10:17:33.214 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.33107.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28080.log 2024-09-18T10:17:33.215 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33107.log.gz 2024-09-18T10:17:33.215 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26161.log 2024-09-18T10:17:33.215 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.26661.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26661.log.gz 2024-09-18T10:17:33.215 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27875.log 2024-09-18T10:17:33.215 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.28080.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28080.log.gz 2024-09-18T10:17:33.215 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25557.log 2024-09-18T10:17:33.215 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.26161.log: /var/log/ceph/ceph-client.admin.27875.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26161.log.gz 2024-09-18T10:17:33.215 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27875.log.gz 2024-09-18T10:17:33.215 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26266.log 2024-09-18T10:17:33.215 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.25557.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33619.log 2024-09-18T10:17:33.215 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25557.log.gz 2024-09-18T10:17:33.216 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26250.log 2024-09-18T10:17:33.216 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.26266.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26266.log.gz 2024-09-18T10:17:33.216 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28286.log 2024-09-18T10:17:33.218 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-osd.1.log: /var/log/ceph/ceph.log: 84.6% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-18T10:17:33.225 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.10.log 2024-09-18T10:17:33.232 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-09-18T10:17:33.232 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-18T10:17:33.232 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.14.log 2024-09-18T10:17:33.232 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-mon.c.log: /var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-09-18T10:17:33.232 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-18T10:17:33.232 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-18T10:17:33.232 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-osd.14.log: /var/log/ceph/ceph.log: /var/log/ceph/ceph-osd.2.log: 88.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-18T10:17:33.235 INFO:teuthology.orchestra.run.smithi154.stderr: 84.6% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-18T10:17:33.256 INFO:teuthology.orchestra.run.smithi189.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22072.log 2024-09-18T10:17:33.256 INFO:teuthology.orchestra.run.smithi189.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22288.log 2024-09-18T10:17:33.257 INFO:teuthology.orchestra.run.smithi189.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.25896.log 2024-09-18T10:17:33.257 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/ceph-client.admin.22072.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22072.log.gz 2024-09-18T10:17:33.257 INFO:teuthology.orchestra.run.smithi189.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22316.log 2024-09-18T10:17:33.257 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/ceph-client.admin.22288.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22288.log.gz 2024-09-18T10:17:33.257 INFO:teuthology.orchestra.run.smithi189.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22153.log 2024-09-18T10:17:33.257 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/ceph-client.0.25896.log: /var/log/ceph/ceph-client.admin.22316.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22316.log.gz 2024-09-18T10:17:33.257 INFO:teuthology.orchestra.run.smithi189.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.25896.log.gz 2024-09-18T10:17:33.257 INFO:teuthology.orchestra.run.smithi189.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-18T10:17:33.257 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/ceph-client.admin.22153.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21990.log 2024-09-18T10:17:33.257 INFO:teuthology.orchestra.run.smithi189.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22153.log.gz 2024-09-18T10:17:33.257 INFO:teuthology.orchestra.run.smithi189.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22206.log 2024-09-18T10:17:33.258 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22180.log 2024-09-18T10:17:33.258 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/ceph-client.admin.21990.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21990.log.gz 2024-09-18T10:17:33.258 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/ceph-client.admin.22206.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22206.log.gz 2024-09-18T10:17:33.258 INFO:teuthology.orchestra.run.smithi189.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.11.log 2024-09-18T10:17:33.258 INFO:teuthology.orchestra.run.smithi189.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22343.log 2024-09-18T10:17:33.258 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/ceph-client.admin.22180.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22180.log.gz 2024-09-18T10:17:33.258 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21965.log 2024-09-18T10:17:33.258 INFO:teuthology.orchestra.run.smithi189.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.15.log 2024-09-18T10:17:33.258 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/ceph-client.admin.22343.log: /var/log/ceph/ceph-client.admin.21965.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22343.log.gz 2024-09-18T10:17:33.258 INFO:teuthology.orchestra.run.smithi189.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21965.log.gz 2024-09-18T10:17:33.258 INFO:teuthology.orchestra.run.smithi189.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22236.log 2024-09-18T10:17:33.259 INFO:teuthology.orchestra.run.smithi189.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22044.log 2024-09-18T10:17:33.259 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/ceph-osd.15.log: /var/log/ceph/ceph-client.admin.22236.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22236.log.gz 2024-09-18T10:17:33.259 INFO:teuthology.orchestra.run.smithi189.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22100.log 2024-09-18T10:17:33.259 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/ceph-client.admin.22044.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22044.log.gz 2024-09-18T10:17:33.259 INFO:teuthology.orchestra.run.smithi189.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22368.log 2024-09-18T10:17:33.259 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/ceph-client.admin.22100.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-09-18T10:17:33.259 INFO:teuthology.orchestra.run.smithi189.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22100.log.gz 2024-09-18T10:17:33.259 INFO:teuthology.orchestra.run.smithi189.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22126.log 2024-09-18T10:17:33.259 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/ceph-client.admin.22368.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22368.log.gz 2024-09-18T10:17:33.259 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22017.log 2024-09-18T10:17:33.259 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/ceph-client.admin.22126.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22126.log.gz 2024-09-18T10:17:33.259 INFO:teuthology.orchestra.run.smithi189.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-09-18T10:17:33.260 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/ceph-client.admin.22017.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22017.log.gz 2024-09-18T10:17:33.260 INFO:teuthology.orchestra.run.smithi189.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22263.log 2024-09-18T10:17:33.260 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/ceph-osd.7.log: /var/log/ceph/ceph-client.admin.22263.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22263.log.gz 2024-09-18T10:17:33.308 INFO:teuthology.orchestra.run.smithi189.stderr: 92.2% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-18T10:17:33.316 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.33619.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-18T10:17:33.316 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.26250.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33619.log.gz 2024-09-18T10:17:33.316 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26250.log.gz 2024-09-18T10:17:33.317 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.28286.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28286.log.gz-5 2024-09-18T10:17:33.317 INFO:teuthology.orchestra.run.smithi029.stderr:--verbose -- /var/log/ceph/ceph-client.admin.32695.log 2024-09-18T10:17:33.317 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28186.log 2024-09-18T10:17:33.317 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-mon.a.log: /var/log/ceph/ceph-client.admin.32695.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32695.log.gz 2024-09-18T10:17:33.317 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30877.log 2024-09-18T10:17:33.317 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.28186.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28186.log.gz 2024-09-18T10:17:33.317 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34397.log 2024-09-18T10:17:33.317 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31530.log 2024-09-18T10:17:33.317 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.30877.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30877.log.gz 2024-09-18T10:17:33.317 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29789.log 2024-09-18T10:17:33.317 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.34397.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34397.log.gz 2024-09-18T10:17:33.317 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.31530.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31530.log.gz-5 2024-09-18T10:17:33.317 INFO:teuthology.orchestra.run.smithi029.stderr:--verbose -- /var/log/ceph/ceph-client.admin.28215.log 2024-09-18T10:17:33.318 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.29789.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25762.log 2024-09-18T10:17:33.318 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29789.log.gz 2024-09-18T10:17:33.318 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28160.log 2024-09-18T10:17:33.318 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.28215.log: /var/log/ceph/ceph-client.admin.25762.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25762.log.gz 2024-09-18T10:17:33.318 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28215.log.gz 2024-09-18T10:17:33.318 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34289.log 2024-09-18T10:17:33.318 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.28160.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28160.log.gz 2024-09-18T10:17:33.318 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34235.log 2024-09-18T10:17:33.318 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.34289.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33609.log 2024-09-18T10:17:33.318 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34289.log.gz 2024-09-18T10:17:33.318 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28612.log 2024-09-18T10:17:33.318 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.34235.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34235.log.gz 2024-09-18T10:17:33.318 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32693.log 2024-09-18T10:17:33.319 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.33609.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33609.log.gz 2024-09-18T10:17:33.319 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.28612.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33130.log 2024-09-18T10:17:33.319 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28612.log.gz 2024-09-18T10:17:33.319 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.32693.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28998.log 2024-09-18T10:17:33.319 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32693.log.gz 2024-09-18T10:17:33.319 INFO:teuthology.orchestra.run.smithi029.stderr:gzip /var/log/ceph/ceph-client.admin.33130.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.33116.log 2024-09-18T10:17:33.319 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33130.log.gz 2024-09-18T10:17:33.319 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.28998.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32075.log 2024-09-18T10:17:33.319 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28998.log.gz 2024-09-18T10:17:33.319 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33629.log 2024-09-18T10:17:33.319 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.33116.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33116.log.gz 2024-09-18T10:17:33.319 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.32075.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34424.log 0.0% 2024-09-18T10:17:33.319 INFO:teuthology.orchestra.run.smithi029.stderr: -- replaced with /var/log/ceph/ceph-client.admin.32075.log.gz 2024-09-18T10:17:33.320 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29313.log 2024-09-18T10:17:33.320 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.33629.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33629.log.gz 2024-09-18T10:17:33.320 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28346.log 2024-09-18T10:17:33.320 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.34424.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34424.log.gz 2024-09-18T10:17:33.320 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27326.log 2024-09-18T10:17:33.320 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.29313.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29313.log.gz 2024-09-18T10:17:33.320 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32588.log 2024-09-18T10:17:33.320 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.28346.log: 0.0%/var/log/ceph/ceph-client.admin.27326.log: -- replaced with /var/log/ceph/ceph-client.admin.28346.log.gz 2024-09-18T10:17:33.320 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27326.log.gz 2024-09-18T10:17:33.320 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32691.log 2024-09-18T10:17:33.320 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.32588.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29058.log 2024-09-18T10:17:33.320 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32588.log.gz 2024-09-18T10:17:33.321 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28606.log 2024-09-18T10:17:33.321 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.32691.log: /var/log/ceph/ceph-client.admin.29058.log: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29058.log.gz 2024-09-18T10:17:33.321 INFO:teuthology.orchestra.run.smithi029.stderr: -- replaced with /var/log/ceph/ceph-client.admin.32691.log.gz 2024-09-18T10:17:33.321 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25740.log 2024-09-18T10:17:33.321 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34154.log 2024-09-18T10:17:33.321 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.28606.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28606.log.gz 2024-09-18T10:17:33.321 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34073.log 2024-09-18T10:17:33.321 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.25740.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25740.log.gz 2024-09-18T10:17:33.321 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.34154.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32129.log 2024-09-18T10:17:33.321 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34154.log.gz 2024-09-18T10:17:33.321 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34019.log 2024-09-18T10:17:33.321 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.34073.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34073.log.gz 2024-09-18T10:17:33.321 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25773.log 2024-09-18T10:17:33.321 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.32129.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32129.log.gz 2024-09-18T10:17:33.322 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.34019.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33090.log 2024-09-18T10:17:33.322 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34019.log.gz 2024-09-18T10:17:33.322 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.25773.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31385.log 2024-09-18T10:17:33.322 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25773.log.gz 2024-09-18T10:17:33.322 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.33090.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27610.log 2024-09-18T10:17:33.322 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33090.log.gz 2024-09-18T10:17:33.322 INFO:teuthology.orchestra.run.smithi029.stderr:gzip /var/log/ceph/ceph-client.admin.31385.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.34643.log 2024-09-18T10:17:33.322 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31385.log.gz 2024-09-18T10:17:33.322 INFO:teuthology.orchestra.run.smithi029.stderr:gzip /var/log/ceph/ceph-client.admin.27610.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.34724.log 2024-09-18T10:17:33.322 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27610.log.gz 2024-09-18T10:17:33.322 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31300.log 2024-09-18T10:17:33.322 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.34643.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34643.log.gz 2024-09-18T10:17:33.322 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.34724.log: gzip -5 --verbose 0.0%-- -- replaced with /var/log/ceph/ceph-client.admin.34724.log.gz/var/log/ceph/ceph-client.admin.33616.log 2024-09-18T10:17:33.322 INFO:teuthology.orchestra.run.smithi029.stderr: 2024-09-18T10:17:33.323 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.31300.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32696.log 2024-09-18T10:17:33.323 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31300.log.gz 2024-09-18T10:17:33.323 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.33616.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27380.log 2024-09-18T10:17:33.323 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33616.log.gz 2024-09-18T10:17:33.323 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32687.log 2024-09-18T10:17:33.323 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.32696.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32696.log.gz 2024-09-18T10:17:33.323 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.27380.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25775.log 2024-09-18T10:17:33.323 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27380.log.gz 2024-09-18T10:17:33.323 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28511.log 2024-09-18T10:17:33.323 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.32687.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32687.log.gz 2024-09-18T10:17:33.323 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.25775.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29704.log 2024-09-18T10:17:33.323 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25775.log.gz 2024-09-18T10:17:33.323 INFO:teuthology.orchestra.run.smithi029.stderr:gzip /var/log/ceph/ceph-client.admin.28511.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.26092.log 2024-09-18T10:17:33.324 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28511.log.gz 2024-09-18T10:17:33.324 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30647.log 2024-09-18T10:17:33.324 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.29704.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29704.log.gz 2024-09-18T10:17:33.324 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34208.log 2024-09-18T10:17:33.324 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.26092.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26092.log.gz 2024-09-18T10:17:33.324 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29843.log 2024-09-18T10:17:33.324 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.30647.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32021.log 0.0% 2024-09-18T10:17:33.324 INFO:teuthology.orchestra.run.smithi029.stderr: -- replaced with /var/log/ceph/ceph-client.admin.30647.log.gz 2024-09-18T10:17:33.324 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.34208.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34208.log.gz 2024-09-18T10:17:33.324 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29450.log 2024-09-18T10:17:33.324 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.29843.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29843.log.gz 2024-09-18T10:17:33.324 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34589.log 2024-09-18T10:17:33.324 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.32021.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32021.log.gz 2024-09-18T10:17:33.324 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.29450.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31638.log 2024-09-18T10:17:33.325 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29450.log.gz 2024-09-18T10:17:33.325 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34046.log 2024-09-18T10:17:33.325 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.34589.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34589.log.gz 2024-09-18T10:17:33.325 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.31638.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-09-18T10:17:33.325 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31638.log.gz 2024-09-18T10:17:33.325 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.34046.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34100.log 2024-09-18T10:17:33.325 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34046.log.gz 2024-09-18T10:17:33.325 INFO:teuthology.orchestra.run.smithi029.stderr:gzip /var/log/ceph/ceph-mgr.y.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.31886.log 2024-09-18T10:17:33.325 INFO:teuthology.orchestra.run.smithi029.stderr:gzip /var/log/ceph/ceph-client.admin.34100.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.33126.log 2024-09-18T10:17:33.325 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34100.log.gz 2024-09-18T10:17:33.325 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.31886.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28553.log 2024-09-18T10:17:33.325 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31886.log.gz 2024-09-18T10:17:33.325 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33126.log: /var/log/ceph/ceph-client.admin.28693.log 2024-09-18T10:17:33.326 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33126.log.gz 2024-09-18T10:17:33.326 INFO:teuthology.orchestra.run.smithi029.stderr:gzip /var/log/ceph/ceph-client.admin.28553.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.32372.log 2024-09-18T10:17:33.326 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28553.log.gz 2024-09-18T10:17:33.326 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34478.log 2024-09-18T10:17:33.326 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.28693.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28693.log.gz 2024-09-18T10:17:33.326 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.32372.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31832.log 2024-09-18T10:17:33.326 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32372.log.gz 2024-09-18T10:17:33.326 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25768.log 2024-09-18T10:17:33.326 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.34478.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34478.log.gz 2024-09-18T10:17:33.326 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.17251.log 2024-09-18T10:17:33.326 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.31832.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31832.log.gz 2024-09-18T10:17:33.326 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.25768.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30330.log 2024-09-18T10:17:33.326 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25768.log.gz 2024-09-18T10:17:33.326 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32426.log 2024-09-18T10:17:33.327 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph.tmp-client.admin.17251.log: gzip /var/log/ceph/ceph-client.admin.30330.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.33088.log 2024-09-18T10:17:33.327 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.17251.log.gz 2024-09-18T10:17:33.327 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30330.log.gz 2024-09-18T10:17:33.327 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.32426.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30157.log 2024-09-18T10:17:33.327 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32426.log.gz 2024-09-18T10:17:33.327 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.33088.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26133.log 2024-09-18T10:17:33.327 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33088.log.gz 2024-09-18T10:17:33.327 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33620.log 2024-09-18T10:17:33.327 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.30157.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30157.log.gz 2024-09-18T10:17:33.327 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30732.log 2024-09-18T10:17:33.327 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.26133.log: 93.1% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-09-18T10:17:33.327 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26133.log.gz 2024-09-18T10:17:33.327 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.33620.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25689.log 2024-09-18T10:17:33.328 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33620.log.gz 2024-09-18T10:17:33.328 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31994.log 2024-09-18T10:17:33.328 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.30732.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30732.log.gz 2024-09-18T10:17:33.328 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25609.log 2024-09-18T10:17:33.328 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.25689.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25689.log.gz 2024-09-18T10:17:33.328 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28340.log 2024-09-18T10:17:33.328 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.31994.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31994.log.gz 2024-09-18T10:17:33.328 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.25609.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26193.log 2024-09-18T10:17:33.328 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25609.log.gz 2024-09-18T10:17:33.328 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32534.log 2024-09-18T10:17:33.328 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.28340.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28340.log.gz 2024-09-18T10:17:33.328 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25766.log 2024-09-18T10:17:33.328 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.26193.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26193.log.gz 2024-09-18T10:17:33.328 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32210.log 2024-09-18T10:17:33.329 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.32534.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32534.log.gz 2024-09-18T10:17:33.329 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29931.log /var/log/ceph/ceph-client.admin.25766.log: 2024-09-18T10:17:33.329 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25766.log.gz 2024-09-18T10:17:33.329 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32694.log 2024-09-18T10:17:33.329 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.32210.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32210.log.gz 2024-09-18T10:17:33.329 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31692.log 2024-09-18T10:17:33.329 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.29931.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29931.log.gz 2024-09-18T10:17:33.329 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34805.log 2024-09-18T10:17:33.329 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.32694.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32694.log.gz 2024-09-18T10:17:33.329 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31665.log 2024-09-18T10:17:33.329 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.31692.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31692.log.gz 2024-09-18T10:17:33.329 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.34805.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-18T10:17:33.329 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34805.log.gz 2024-09-18T10:17:33.330 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30930.log 2024-09-18T10:17:33.330 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.31665.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31665.log.gz 2024-09-18T10:17:33.330 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32345.log 2024-09-18T10:17:33.330 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.30930.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34562.log 2024-09-18T10:17:33.330 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30930.log.gz 2024-09-18T10:17:33.330 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.32345.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26219.log 2024-09-18T10:17:33.330 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32345.log.gz 2024-09-18T10:17:33.330 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31327.log 2024-09-18T10:17:33.330 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.34562.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34562.log.gz 2024-09-18T10:17:33.330 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28463.log 2024-09-18T10:17:33.330 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.26219.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26219.log.gz 2024-09-18T10:17:33.330 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.31327.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34670.log 2024-09-18T10:17:33.330 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31327.log.gz 2024-09-18T10:17:33.331 INFO:teuthology.orchestra.run.smithi029.stderr: 88.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-18T10:17:33.331 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31724.log 2024-09-18T10:17:33.331 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.28463.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28463.log.gz 2024-09-18T10:17:33.331 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35021.log 2024-09-18T10:17:33.331 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.34670.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34670.log.gz 2024-09-18T10:17:33.331 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32237.log 2024-09-18T10:17:33.331 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.31724.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31724.log.gz 2024-09-18T10:17:33.331 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29149.log 2024-09-18T10:17:33.331 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.35021.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35021.log.gz 2024-09-18T10:17:33.331 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26191.log 2024-09-18T10:17:33.331 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.32237.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32237.log.gz 2024-09-18T10:17:33.331 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.29149.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26084.log 2024-09-18T10:17:33.331 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29149.log.gz 2024-09-18T10:17:33.332 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25662.log 2024-09-18T10:17:33.332 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.26191.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26191.log.gz 2024-09-18T10:17:33.332 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33104.log 2024-09-18T10:17:33.332 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.26084.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26084.log.gz 2024-09-18T10:17:33.332 INFO:teuthology.orchestra.run.smithi029.stderr:gzip /var/log/ceph/ceph-client.admin.25662.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.32183.log 2024-09-18T10:17:33.332 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25662.log.gz 2024-09-18T10:17:33.332 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34370.log 2024-09-18T10:17:33.332 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.33104.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33104.log.gz 2024-09-18T10:17:33.332 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28919.log 2024-09-18T10:17:33.332 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.32183.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32183.log.gz 2024-09-18T10:17:33.332 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29574.log 2024-09-18T10:17:33.332 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.34370.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34370.log.gz 2024-09-18T10:17:33.332 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26802.log 2024-09-18T10:17:33.333 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.28919.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28919.log.gz 2024-09-18T10:17:33.333 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25772.log 2024-09-18T10:17:33.333 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.29574.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29574.log.gz 2024-09-18T10:17:33.333 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34859.log 2024-09-18T10:17:33.333 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 /var/log/ceph/ceph-client.admin.26802.log: --verbose -- /var/log/ceph/ceph-client.admin.29509.log 2024-09-18T10:17:33.333 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.25772.log: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25772.log.gz 2024-09-18T10:17:33.333 INFO:teuthology.orchestra.run.smithi029.stderr: -- replaced with /var/log/ceph/ceph-client.admin.26802.log.gz 2024-09-18T10:17:33.333 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.34859.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26580.log 2024-09-18T10:17:33.333 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34859.log.gz 2024-09-18T10:17:33.333 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.29509.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29509.log.gz 2024-09-18T10:17:33.333 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30046.log 2024-09-18T10:17:33.333 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.26580.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26181.log 2024-09-18T10:17:33.333 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26580.log.gz 2024-09-18T10:17:33.334 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34832.log 2024-09-18T10:17:33.334 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.30046.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30046.log.gz 2024-09-18T10:17:33.334 INFO:teuthology.orchestra.run.smithi029.stderr:gzip /var/log/ceph/ceph-client.admin.26181.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.26217.log 2024-09-18T10:17:33.334 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26181.log.gz 2024-09-18T10:17:33.334 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33032.log 2024-09-18T10:17:33.334 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.34832.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34832.log.gz 2024-09-18T10:17:33.334 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.26217.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26976.log 2024-09-18T10:17:33.334 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26217.log.gz 2024-09-18T10:17:33.334 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31940.log 2024-09-18T10:17:33.334 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.33032.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33032.log.gz 2024-09-18T10:17:33.334 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.26976.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27119.log 2024-09-18T10:17:33.334 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26976.log.gz 2024-09-18T10:17:33.334 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31042.log 2024-09-18T10:17:33.335 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.31940.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31940.log.gz 2024-09-18T10:17:33.335 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32507.log 2024-09-18T10:17:33.335 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.27119.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27119.log.gz 2024-09-18T10:17:33.335 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31751.log 2024-09-18T10:17:33.335 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.31042.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31042.log.gz 2024-09-18T10:17:33.335 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31215.log 2024-09-18T10:17:33.335 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.32507.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32507.log.gz 2024-09-18T10:17:33.335 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.31751.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25764.log 2024-09-18T10:17:33.335 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31751.log.gz 2024-09-18T10:17:33.335 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.31215.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34316.log 2024-09-18T10:17:33.335 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31215.log.gz 2024-09-18T10:17:33.335 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32690.log 2024-09-18T10:17:33.336 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.25764.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25764.log.gz 2024-09-18T10:17:33.336 INFO:teuthology.orchestra.run.smithi029.stderr:gzip /var/log/ceph/ceph-client.admin.34316.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.28726.log 2024-09-18T10:17:33.336 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34316.log.gz 2024-09-18T10:17:33.336 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.32690.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26192.log 2024-09-18T10:17:33.336 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32690.log.gz 2024-09-18T10:17:33.336 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30276.log 2024-09-18T10:17:33.336 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.28726.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28726.log.gz 2024-09-18T10:17:33.336 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.12.log 2024-09-18T10:17:33.336 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.26192.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26192.log.gz 2024-09-18T10:17:33.336 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.30276.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26108.log 2024-09-18T10:17:33.336 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30276.log.gz 2024-09-18T10:17:33.336 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26803.log 2024-09-18T10:17:33.336 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-osd.12.log: /var/log/ceph/ceph-client.admin.26108.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27062.log 2024-09-18T10:17:33.336 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26108.log.gz 2024-09-18T10:17:33.337 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32686.log 2024-09-18T10:17:33.337 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.26803.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26803.log.gz 2024-09-18T10:17:33.337 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25761.log 2024-09-18T10:17:33.337 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.27062.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27062.log.gz 2024-09-18T10:17:33.337 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.32686.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26688.log 2024-09-18T10:17:33.337 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32686.log.gz 2024-09-18T10:17:33.337 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31778.log /var/log/ceph/ceph-client.admin.25761.log: 2024-09-18T10:17:33.337 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25761.log.gz 2024-09-18T10:17:33.337 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32480.log 2024-09-18T10:17:33.337 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.26688.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26688.log.gz 2024-09-18T10:17:33.337 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28580.log 2024-09-18T10:17:33.337 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.31778.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31778.log.gz 2024-09-18T10:17:33.337 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.32480.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27090.log 2024-09-18T10:17:33.338 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32480.log.gz 2024-09-18T10:17:33.338 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25771.log 2024-09-18T10:17:33.338 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.28580.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28580.log.gz 2024-09-18T10:17:33.338 INFO:teuthology.orchestra.run.smithi029.stderr:gzip /var/log/ceph/ceph-client.admin.27090.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.33157.log 2024-09-18T10:17:33.338 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27090.log.gz 2024-09-18T10:17:33.338 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33918.log 2024-09-18T10:17:33.338 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.25771.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25771.log.gz 2024-09-18T10:17:33.338 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 /var/log/ceph/ceph-client.admin.33157.log: --verbose -- /var/log/ceph/ceph-client.admin.32692.log 2024-09-18T10:17:33.338 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33157.log.gz 2024-09-18T10:17:33.338 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.33918.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33992.log 2024-09-18T10:17:33.338 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33918.log.gz 2024-09-18T10:17:33.338 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31219.log 2024-09-18T10:17:33.338 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.32692.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32692.log.gz 2024-09-18T10:17:33.339 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.33992.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26144.log 0.0% 2024-09-18T10:17:33.339 INFO:teuthology.orchestra.run.smithi029.stderr: -- replaced with /var/log/ceph/ceph-client.admin.33992.log.gz 2024-09-18T10:17:33.339 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34262.log 2024-09-18T10:17:33.339 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.31219.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31219.log.gz 2024-09-18T10:17:33.339 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34778.log 2024-09-18T10:17:33.339 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.26144.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26144.log.gz 2024-09-18T10:17:33.339 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.34262.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29365.log 2024-09-18T10:17:33.339 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34262.log.gz 2024-09-18T10:17:33.339 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34778.log: /var/log/ceph/ceph-client.admin.27499.log 2024-09-18T10:17:33.339 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34778.log.gz 2024-09-18T10:17:33.339 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34697.log 2024-09-18T10:17:33.339 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.29365.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29365.log.gz 2024-09-18T10:17:33.339 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.27499.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33622.log 2024-09-18T10:17:33.339 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27499.log.gz 2024-09-18T10:17:33.340 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32453.log 2024-09-18T10:17:33.340 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.34697.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34697.log.gz 2024-09-18T10:17:33.340 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31859.log 2024-09-18T10:17:33.340 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.33622.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33622.log.gz 2024-09-18T10:17:33.340 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.32453.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32684.log 2024-09-18T10:17:33.340 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32453.log.gz 2024-09-18T10:17:33.340 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27552.log 2024-09-18T10:17:33.340 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.31859.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31859.log.gz 2024-09-18T10:17:33.340 INFO:teuthology.orchestra.run.smithi029.stderr:gzip /var/log/ceph/ceph-client.admin.32684.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.33631.log 2024-09-18T10:17:33.340 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32684.log.gz 2024-09-18T10:17:33.340 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33517.log 2024-09-18T10:17:33.340 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.27552.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27552.log.gz 2024-09-18T10:17:33.341 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.33631.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30672.log 2024-09-18T10:17:33.341 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33631.log.gz 2024-09-18T10:17:33.341 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.33517.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30557.log 2024-09-18T10:17:33.341 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33517.log.gz 2024-09-18T10:17:33.341 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.30672.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27935.log 2024-09-18T10:17:33.341 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30672.log.gz 2024-09-18T10:17:33.341 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25774.log 2024-09-18T10:17:33.341 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.30557.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30557.log.gz 2024-09-18T10:17:33.341 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.27935.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30015.log 2024-09-18T10:17:33.341 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27935.log.gz 2024-09-18T10:17:33.341 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26177.log 2024-09-18T10:17:33.341 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.25774.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25774.log.gz 2024-09-18T10:17:33.341 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30440.log 2024-09-18T10:17:33.341 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.30015.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30015.log.gz 2024-09-18T10:17:33.342 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27216.log 2024-09-18T10:17:33.342 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.26177.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28076.log 2024-09-18T10:17:33.342 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26177.log.gz 2024-09-18T10:17:33.342 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.30440.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30440.log.gz 2024-09-18T10:17:33.342 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.27216.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28417.log 2024-09-18T10:17:33.342 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27216.log.gz 2024-09-18T10:17:33.342 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33511.log 2024-09-18T10:17:33.342 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.28076.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28076.log.gz 2024-09-18T10:17:33.342 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31913.log 2024-09-18T10:17:33.342 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.28417.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28417.log.gz 2024-09-18T10:17:33.342 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33630.log 2024-09-18T10:17:33.342 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.33511.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33511.log.gz 2024-09-18T10:17:33.342 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.31913.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34127.log 2024-09-18T10:17:33.343 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31913.log.gz 2024-09-18T10:17:33.343 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33606.log 2024-09-18T10:17:33.343 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.33630.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33630.log.gz 2024-09-18T10:17:33.343 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.34127.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27760.log 2024-09-18T10:17:33.343 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34127.log.gz 2024-09-18T10:17:33.343 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27406.log 2024-09-18T10:17:33.343 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.33606.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33606.log.gz 2024-09-18T10:17:33.343 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31128.log 2024-09-18T10:17:33.343 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.27760.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27760.log.gz 2024-09-18T10:17:33.343 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32318.log 2024-09-18T10:17:33.343 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.27406.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27406.log.gz 2024-09-18T10:17:33.343 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32699.log 2024-09-18T10:17:33.343 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.31128.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31128.log.gz 2024-09-18T10:17:33.343 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34508.log /var/log/ceph/ceph-client.admin.32318.log: 2024-09-18T10:17:33.344 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32318.log.gz 2024-09-18T10:17:33.344 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27729.log 2024-09-18T10:17:33.344 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.32699.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32699.log.gz 2024-09-18T10:17:33.344 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32689.log 2024-09-18T10:17:33.344 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.34508.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34508.log.gz 2024-09-18T10:17:33.344 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27729.log: /var/log/ceph/ceph-client.admin.34967.log 2024-09-18T10:17:33.344 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27729.log.gz 2024-09-18T10:17:33.344 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33626.log 2024-09-18T10:17:33.344 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.32689.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32689.log.gz 2024-09-18T10:17:33.344 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.34967.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33093.log 2024-09-18T10:17:33.344 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34967.log.gz 2024-09-18T10:17:33.344 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29227.log 2024-09-18T10:17:33.344 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.33626.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33626.log.gz 2024-09-18T10:17:33.345 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27004.log 2024-09-18T10:17:33.345 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.33093.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33093.log.gz 2024-09-18T10:17:33.345 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.29227.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30515.log 2024-09-18T10:17:33.345 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29227.log.gz 2024-09-18T10:17:33.345 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.27004.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29085.log 2024-09-18T10:17:33.345 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27004.log.gz 2024-09-18T10:17:33.345 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33087.log 2024-09-18T10:17:33.345 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.30515.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30515.log.gz 2024-09-18T10:17:33.345 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34886.log 2024-09-18T10:17:33.345 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.29085.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29085.log.gz 2024-09-18T10:17:33.345 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.33087.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32700.log 2024-09-18T10:17:33.345 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33087.log.gz 2024-09-18T10:17:33.345 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25739.log 2024-09-18T10:17:33.345 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.34886.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34886.log.gz 2024-09-18T10:17:33.346 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28797.log 2024-09-18T10:17:33.346 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.32700.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32700.log.gz 2024-09-18T10:17:33.346 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31611.log 2024-09-18T10:17:33.346 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.25739.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25739.log.gz 2024-09-18T10:17:33.346 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34616.log 2024-09-18T10:17:33.346 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.28797.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28797.log.gz 2024-09-18T10:17:33.346 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.31611.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30904.log 2024-09-18T10:17:33.346 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31611.log.gz 2024-09-18T10:17:33.346 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.34616.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34535.log 2024-09-18T10:17:33.346 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34616.log.gz 2024-09-18T10:17:33.346 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26800.log 2024-09-18T10:17:33.346 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.30904.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30904.log.gz 2024-09-18T10:17:33.346 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.34535.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35048.log 2024-09-18T10:17:33.346 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34535.log.gz 2024-09-18T10:17:33.347 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33154.log 2024-09-18T10:17:33.347 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.26800.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26800.log.gz 2024-09-18T10:17:33.347 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26272.log 2024-09-18T10:17:33.347 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.35048.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35048.log.gz 2024-09-18T10:17:33.347 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33911.log 2024-09-18T10:17:33.347 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.33154.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33154.log.gz 2024-09-18T10:17:33.347 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27982.log 2024-09-18T10:17:33.347 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.26272.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26272.log.gz 2024-09-18T10:17:33.347 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32561.log 2024-09-18T10:17:33.347 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.33911.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33911.log.gz 2024-09-18T10:17:33.347 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.27982.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34940.log 2024-09-18T10:17:33.347 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27982.log.gz 2024-09-18T10:17:33.347 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33117.log 2024-09-18T10:17:33.348 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.32561.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32561.log.gz 2024-09-18T10:17:33.348 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-18T10:17:33.348 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.34940.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34940.log.gz 2024-09-18T10:17:33.348 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28728.log 2024-09-18T10:17:33.348 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.33117.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33117.log.gz 2024-09-18T10:17:33.348 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34913.log 2024-09-18T10:17:33.348 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33106.log 2024-09-18T10:17:33.348 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.28728.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31583.log 2024-09-18T10:17:33.348 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.34913.log: -- replaced with /var/log/ceph/ceph-client.admin.28728.log.gz 2024-09-18T10:17:33.348 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34913.log.gz 2024-09-18T10:17:33.348 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.33106.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33610.log 2024-09-18T10:17:33.348 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33106.log.gz 2024-09-18T10:17:33.348 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30107.log 2024-09-18T10:17:33.349 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.31583.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31583.log.gz 2024-09-18T10:17:33.349 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33114.log 2024-09-18T10:17:33.349 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.33610.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33610.log.gz 2024-09-18T10:17:33.349 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.30107.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32615.log 2024-09-18T10:17:33.349 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30107.log.gz 2024-09-18T10:17:33.349 INFO:teuthology.orchestra.run.smithi029.stderr: 84.6%gzip -- replaced with /var/log/ceph/ceph.log.gz-5 2024-09-18T10:17:33.349 INFO:teuthology.orchestra.run.smithi029.stderr:--verbose -- /var/log/ceph/ceph-client.admin.34451.log 2024-09-18T10:17:33.349 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.33114.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33114.log.gz 2024-09-18T10:17:33.349 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-18T10:17:33.349 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.32615.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32615.log.gz 2024-09-18T10:17:33.349 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30393.log 2024-09-18T10:17:33.349 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.34451.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34451.log.gz 2024-09-18T10:17:33.349 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29838.log 2024-09-18T10:17:33.350 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-osd.0.log: /var/log/ceph/ceph-client.admin.30393.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31967.log 2024-09-18T10:17:33.350 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30393.log.gz 2024-09-18T10:17:33.350 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30934.log 2024-09-18T10:17:33.350 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.29838.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29838.log.gz 2024-09-18T10:17:33.350 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.31967.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34751.log 2024-09-18T10:17:33.350 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31967.log.gz 2024-09-18T10:17:33.350 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25776.log 2024-09-18T10:17:33.350 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.30934.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30934.log.gz 2024-09-18T10:17:33.350 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-09-18T10:17:33.350 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.34751.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34751.log.gz 2024-09-18T10:17:33.350 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33124.log 2024-09-18T10:17:33.350 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.25776.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25776.log.gz 2024-09-18T10:17:33.350 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26716.log 2024-09-18T10:17:33.350 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32702.log 2024-09-18T10:17:33.351 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.33124.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28241.log 2024-09-18T10:17:33.351 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.26716.log: -- replaced with /var/log/ceph/ceph-client.admin.33124.log.gz 2024-09-18T10:17:33.351 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26716.log.gz 2024-09-18T10:17:33.351 INFO:teuthology.orchestra.run.smithi029.stderr:gzip /var/log/ceph/ceph-client.admin.32702.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.29111.log 2024-09-18T10:17:33.351 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32702.log.gz 2024-09-18T10:17:33.351 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.28241.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28241.log.gz 2024-09-18T10:17:33.351 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31534.log 2024-09-18T10:17:33.351 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.29111.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30769.log 2024-09-18T10:17:33.351 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29111.log.gz 2024-09-18T10:17:33.351 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27636.log 2024-09-18T10:17:33.351 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.31534.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25767.log 2024-09-18T10:17:33.351 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31534.log.gz 2024-09-18T10:17:33.351 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.30769.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30769.log.gz 2024-09-18T10:17:33.352 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.27636.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33627.log 2024-09-18T10:17:33.352 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27636.log.gz 2024-09-18T10:17:33.352 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32697.log 2024-09-18T10:17:33.352 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.25767.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25767.log.gz 2024-09-18T10:17:33.352 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29270.log 2024-09-18T10:17:33.352 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.33627.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33627.log.gz 2024-09-18T10:17:33.352 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30388.log 2024-09-18T10:17:33.352 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.32697.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32697.log.gz 2024-09-18T10:17:33.352 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25770.log 2024-09-18T10:17:33.352 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.29270.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29270.log.gz 2024-09-18T10:17:33.352 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29127.log 2024-09-18T10:17:33.352 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.30388.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30388.log.gz 2024-09-18T10:17:33.353 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.25770.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30185.log 2024-09-18T10:17:33.353 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25770.log.gz 2024-09-18T10:17:33.353 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34343.log 2024-09-18T10:17:33.353 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.29127.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29127.log.gz 2024-09-18T10:17:33.353 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26759.log 2024-09-18T10:17:33.353 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.30185.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30185.log.gz 2024-09-18T10:17:33.353 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.34343.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32102.log 2024-09-18T10:17:33.353 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34343.log.gz 2024-09-18T10:17:33.353 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29747.log 2024-09-18T10:17:33.353 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.26759.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26759.log.gz 2024-09-18T10:17:33.353 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.32102.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22209.log 2024-09-18T10:17:33.353 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32102.log.gz 2024-09-18T10:17:33.353 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30108.log 2024-09-18T10:17:33.354 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.29747.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29747.log.gz 2024-09-18T10:17:33.354 INFO:teuthology.orchestra.run.smithi029.stderr:gzip /var/log/ceph/ceph-client.admin.22209.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.31805.log 2024-09-18T10:17:33.354 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22209.log.gz 2024-09-18T10:17:33.354 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.30108.log: gzip 0.0%-5 -- replaced with /var/log/ceph/ceph-client.admin.30108.log.gz 2024-09-18T10:17:33.354 INFO:teuthology.orchestra.run.smithi029.stderr:--verbose -- /var/log/ceph/ceph-client.admin.26634.log 2024-09-18T10:17:33.354 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.31805.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29478.log 2024-09-18T10:17:33.354 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31805.log.gz 2024-09-18T10:17:33.354 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25635.log 2024-09-18T10:17:33.354 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.26634.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26634.log.gz 2024-09-18T10:17:33.354 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.8.log 2024-09-18T10:17:33.354 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.29478.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27267.log 2024-09-18T10:17:33.354 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29478.log.gz 2024-09-18T10:17:33.355 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.25635.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25635.log.gz 2024-09-18T10:17:33.355 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31011.log 2024-09-18T10:17:33.355 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33919.log 2024-09-18T10:17:33.355 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.27267.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27267.log.gz 2024-09-18T10:17:33.355 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.31011.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31011.log.gz 2024-09-18T10:17:33.355 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27845.log 2024-09-18T10:17:33.355 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.33919.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33506.log 2024-09-18T10:17:33.355 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33919.log.gz 2024-09-18T10:17:33.355 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31462.log 2024-09-18T10:17:33.355 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.27845.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27845.log.gz 2024-09-18T10:17:33.355 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.33506.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33506.log.gz 2024-09-18T10:17:33.356 INFO:teuthology.orchestra.run.smithi154.stderr: 91.3% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-09-18T10:17:33.362 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28848.log 2024-09-18T10:17:33.362 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32156.log /var/log/ceph/ceph-client.admin.31462.log: 2024-09-18T10:17:33.362 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31462.log.gz 2024-09-18T10:17:33.362 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.28848.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35100.log 2024-09-18T10:17:33.362 INFO:teuthology.orchestra.run.smithi029.stderr: -- replaced with /var/log/ceph/ceph-client.admin.28848.log.gz 2024-09-18T10:17:33.362 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.32156.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32291.log 2024-09-18T10:17:33.362 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32156.log.gz 2024-09-18T10:17:33.362 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22057.log 2024-09-18T10:17:33.362 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.35100.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35100.log.gz 2024-09-18T10:17:33.362 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.32291.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32291.log.gz 2024-09-18T10:17:33.362 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35075.log 2024-09-18T10:17:33.363 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.22057.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25583.log 2024-09-18T10:17:33.363 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22057.log.gz 2024-09-18T10:17:33.363 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27352.log 2024-09-18T10:17:33.363 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.35075.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25763.log 2024-09-18T10:17:33.363 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.25583.log: -- replaced with /var/log/ceph/ceph-client.admin.35075.log.gz 2024-09-18T10:17:33.363 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25583.log.gz 2024-09-18T10:17:33.363 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.27352.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27352.log.gz 2024-09-18T10:17:33.363 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32688.log 2024-09-18T10:17:33.363 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.25763.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29508.log 2024-09-18T10:17:33.363 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25763.log.gz 2024-09-18T10:17:33.363 INFO:teuthology.orchestra.run.smithi029.stderr:gzip /var/log/ceph/ceph-client.admin.32688.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.31070.log 2024-09-18T10:17:33.363 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32688.log.gz 2024-09-18T10:17:33.363 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.29508.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32048.log 2024-09-18T10:17:33.364 INFO:teuthology.orchestra.run.smithi029.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29508.log.gz 2024-09-18T10:17:33.364 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.31070.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31070.log.gz 2024-09-18T10:17:33.364 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26607.log 2024-09-18T10:17:33.364 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28946.log 2024-09-18T10:17:33.364 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.32048.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32048.log.gz 2024-09-18T10:17:33.364 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.26607.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26607.log.gz 2024-09-18T10:17:33.364 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22355.log 2024-09-18T10:17:33.364 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.28946.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28946.log.gz 2024-09-18T10:17:33.364 INFO:teuthology.orchestra.run.smithi029.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34994.log 2024-09-18T10:17:33.364 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.22355.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22355.log.gz 2024-09-18T10:17:33.364 INFO:teuthology.orchestra.run.smithi029.stderr:/var/log/ceph/ceph-client.admin.34994.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34994.log.gz 2024-09-18T10:17:33.365 INFO:teuthology.orchestra.run.smithi136.stderr: 91.4% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-09-18T10:17:33.482 INFO:teuthology.orchestra.run.smithi029.stderr: 91.1% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-18T10:17:33.650 INFO:teuthology.orchestra.run.smithi029.stderr: 91.4% -- replaced with /var/log/ceph/ceph-osd.12.log.gz 2024-09-18T10:17:33.661 INFO:teuthology.orchestra.run.smithi136.stderr: 91.6% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-18T10:17:33.665 INFO:teuthology.orchestra.run.smithi136.stderr: 91.3% -- replaced with /var/log/ceph/ceph-osd.9.log.gz 2024-09-18T10:17:33.733 INFO:teuthology.orchestra.run.smithi189.stderr: 91.4% -- replaced with /var/log/ceph/ceph-osd.15.log.gz 2024-09-18T10:17:33.803 INFO:teuthology.orchestra.run.smithi029.stderr: 91.2% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-18T10:17:33.813 INFO:teuthology.orchestra.run.smithi154.stderr: 91.2% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-18T10:17:33.815 INFO:teuthology.orchestra.run.smithi136.stderr: 91.7% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-09-18T10:17:33.841 INFO:teuthology.orchestra.run.smithi189.stderr: 91.4% -- replaced with /var/log/ceph/ceph-osd.11.log.gz 2024-09-18T10:17:33.872 INFO:teuthology.orchestra.run.smithi189.stderr: 91.3% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-09-18T10:17:33.883 INFO:teuthology.orchestra.run.smithi189.stderr: 91.7% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-09-18T10:17:33.884 INFO:teuthology.orchestra.run.smithi189.stderr: 2024-09-18T10:17:33.884 INFO:teuthology.orchestra.run.smithi189.stderr:real 0m0.662s 2024-09-18T10:17:33.884 INFO:teuthology.orchestra.run.smithi189.stderr:user 0m2.337s 2024-09-18T10:17:33.884 INFO:teuthology.orchestra.run.smithi189.stderr:sys 0m0.111s 2024-09-18T10:17:33.885 INFO:teuthology.orchestra.run.smithi154.stderr: 91.7% -- replaced with /var/log/ceph/ceph-osd.10.log.gz 2024-09-18T10:17:33.928 INFO:teuthology.orchestra.run.smithi154.stderr: 91.5% -- replaced with /var/log/ceph/ceph-osd.14.log.gz 2024-09-18T10:17:33.933 INFO:teuthology.orchestra.run.smithi154.stderr: 91.4% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-09-18T10:17:33.934 INFO:teuthology.orchestra.run.smithi136.stderr: 91.5% -- replaced with /var/log/ceph/ceph-osd.13.log.gz 2024-09-18T10:17:33.935 INFO:teuthology.orchestra.run.smithi154.stderr: 2024-09-18T10:17:33.935 INFO:teuthology.orchestra.run.smithi154.stderr:real 0m0.723s 2024-09-18T10:17:33.935 INFO:teuthology.orchestra.run.smithi154.stderr:user 0m2.684s 2024-09-18T10:17:33.935 INFO:teuthology.orchestra.run.smithi154.stderr:sys 0m0.106s 2024-09-18T10:17:33.936 INFO:teuthology.orchestra.run.smithi136.stderr: 2024-09-18T10:17:33.936 INFO:teuthology.orchestra.run.smithi136.stderr:real 0m0.749s 2024-09-18T10:17:33.936 INFO:teuthology.orchestra.run.smithi136.stderr:user 0m2.356s 2024-09-18T10:17:33.936 INFO:teuthology.orchestra.run.smithi136.stderr:sys 0m0.098s 2024-09-18T10:17:33.971 INFO:teuthology.orchestra.run.smithi029.stderr: 92.1% -- replaced with /var/log/ceph/ceph-osd.8.log.gz 2024-09-18T10:17:34.195 INFO:teuthology.orchestra.run.smithi029.stderr: 92.1% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-09-18T10:17:34.196 INFO:teuthology.orchestra.run.smithi029.stderr: 2024-09-18T10:17:34.197 INFO:teuthology.orchestra.run.smithi029.stderr:real 0m1.012s 2024-09-18T10:17:34.197 INFO:teuthology.orchestra.run.smithi029.stderr:user 0m2.874s 2024-09-18T10:17:34.197 INFO:teuthology.orchestra.run.smithi029.stderr:sys 0m0.167s 2024-09-18T10:17:34.197 INFO:tasks.ceph:Archiving logs... 2024-09-18T10:17:34.197 DEBUG:teuthology.misc:Transferring archived files from smithi029:/var/log/ceph to /home/teuthworker/archive/skanta-2024-09-18_09:13:00-rados-wip-bharath11-testing-2024-09-18-1115-quincy-distro-default-smithi/7910845/remote/smithi029/log 2024-09-18T10:17:34.198 DEBUG:teuthology.orchestra.run.smithi029:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-18T10:17:34.877 DEBUG:teuthology.misc:Transferring archived files from smithi136:/var/log/ceph to /home/teuthworker/archive/skanta-2024-09-18_09:13:00-rados-wip-bharath11-testing-2024-09-18-1115-quincy-distro-default-smithi/7910845/remote/smithi136/log 2024-09-18T10:17:34.878 DEBUG:teuthology.orchestra.run.smithi136:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-18T10:17:35.299 DEBUG:teuthology.misc:Transferring archived files from smithi154:/var/log/ceph to /home/teuthworker/archive/skanta-2024-09-18_09:13:00-rados-wip-bharath11-testing-2024-09-18-1115-quincy-distro-default-smithi/7910845/remote/smithi154/log 2024-09-18T10:17:35.300 DEBUG:teuthology.orchestra.run.smithi154:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-18T10:17:35.757 DEBUG:teuthology.misc:Transferring archived files from smithi189:/var/log/ceph to /home/teuthworker/archive/skanta-2024-09-18_09:13:00-rados-wip-bharath11-testing-2024-09-18-1115-quincy-distro-default-smithi/7910845/remote/smithi189/log 2024-09-18T10:17:35.758 DEBUG:teuthology.orchestra.run.smithi189:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-18T10:17:36.197 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-18T10:17:36.206 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-18T10:17:36.206 DEBUG:teuthology.orchestra.run.smithi029:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-18T10:17:36.208 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-18T10:17:36.211 DEBUG:teuthology.orchestra.run.smithi154:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-18T10:17:36.213 DEBUG:teuthology.orchestra.run.smithi189:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-18T10:17:36.323 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-18T10:17:36.331 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-18T10:17:36.331 DEBUG:teuthology.orchestra.run.smithi029:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-18T10:17:36.334 DEBUG:teuthology.orchestra.run.smithi136:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-18T10:17:36.336 DEBUG:teuthology.orchestra.run.smithi154:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-18T10:17:36.338 DEBUG:teuthology.orchestra.run.smithi189:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-18T10:17:36.353 INFO:teuthology.orchestra.run.smithi029.stdout: remote refid st t when poll reach delay offset jitter 2024-09-18T10:17:36.353 INFO:teuthology.orchestra.run.smithi029.stdout:============================================================================== 2024-09-18T10:17:36.353 INFO:teuthology.orchestra.run.smithi029.stdout:*hv01.front.sepi 67.205.162.81 3 u 1 64 177 0.099 0.560 1.003 2024-09-18T10:17:36.353 INFO:teuthology.orchestra.run.smithi029.stdout:+hv02.front.sepi 63.231.80.2 3 u 62 64 77 0.078 -1.331 0.731 2024-09-18T10:17:36.353 INFO:teuthology.orchestra.run.smithi029.stdout:+hv03.front.sepi 74.6.168.72 3 u 60 64 77 0.071 0.861 0.777 2024-09-18T10:17:36.353 INFO:teuthology.orchestra.run.smithi029.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-18T10:17:36.357 INFO:teuthology.orchestra.run.smithi136.stdout: remote refid st t when poll reach delay offset jitter 2024-09-18T10:17:36.357 INFO:teuthology.orchestra.run.smithi136.stdout:============================================================================== 2024-09-18T10:17:36.357 INFO:teuthology.orchestra.run.smithi136.stdout:*hv01.front.sepi 67.205.162.81 3 u 3 64 177 0.064 -1.131 1.022 2024-09-18T10:17:36.357 INFO:teuthology.orchestra.run.smithi136.stdout:+hv02.front.sepi 63.231.80.2 3 u 62 64 77 0.085 -1.604 0.902 2024-09-18T10:17:36.357 INFO:teuthology.orchestra.run.smithi136.stdout:+hv03.front.sepi 74.6.168.72 3 u 63 64 77 0.089 0.996 0.909 2024-09-18T10:17:36.357 INFO:teuthology.orchestra.run.smithi136.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-18T10:17:36.359 INFO:teuthology.orchestra.run.smithi154.stdout: remote refid st t when poll reach delay offset jitter 2024-09-18T10:17:36.359 INFO:teuthology.orchestra.run.smithi154.stdout:============================================================================== 2024-09-18T10:17:36.359 INFO:teuthology.orchestra.run.smithi154.stdout:*hv01.front.sepi 67.205.162.81 3 u 61 64 77 0.083 0.176 0.776 2024-09-18T10:17:36.359 INFO:teuthology.orchestra.run.smithi154.stdout:+hv02.front.sepi 63.231.80.2 3 u 59 64 77 0.094 -1.319 0.737 2024-09-18T10:17:36.359 INFO:teuthology.orchestra.run.smithi154.stdout:+hv03.front.sepi 74.6.168.72 3 u 59 64 77 0.100 1.061 0.492 2024-09-18T10:17:36.359 INFO:teuthology.orchestra.run.smithi154.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-18T10:17:36.361 INFO:teuthology.orchestra.run.smithi189.stdout: remote refid st t when poll reach delay offset jitter 2024-09-18T10:17:36.361 INFO:teuthology.orchestra.run.smithi189.stdout:============================================================================== 2024-09-18T10:17:36.361 INFO:teuthology.orchestra.run.smithi189.stdout:+hv01.front.sepi 67.205.162.81 3 u 61 64 77 0.073 -1.145 1.014 2024-09-18T10:17:36.361 INFO:teuthology.orchestra.run.smithi189.stdout:+hv02.front.sepi 63.231.80.2 3 u 65 64 77 0.091 -2.204 0.711 2024-09-18T10:17:36.362 INFO:teuthology.orchestra.run.smithi189.stdout:*hv03.front.sepi 74.6.168.72 3 u 65 64 77 0.090 0.341 0.713 2024-09-18T10:17:36.362 INFO:teuthology.orchestra.run.smithi189.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-18T10:17:36.362 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-18T10:17:36.371 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-18T10:17:36.372 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-18T10:17:36.380 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-18T10:17:36.388 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-18T10:17:36.396 INFO:teuthology.task.internal:Duration was 746.921855 seconds 2024-09-18T10:17:36.397 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-18T10:17:36.405 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-18T10:17:36.405 DEBUG:teuthology.orchestra.run.smithi029:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-18T10:17:36.407 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-18T10:17:36.411 DEBUG:teuthology.orchestra.run.smithi154:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-18T10:17:36.413 DEBUG:teuthology.orchestra.run.smithi189:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-18T10:17:36.463 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-18T10:17:36.464 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi029.front.sepia.ceph.com 2024-09-18T10:17:36.464 DEBUG:teuthology.orchestra.run.smithi029:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-18T10:17:36.477 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi136.front.sepia.ceph.com 2024-09-18T10:17:36.477 DEBUG:teuthology.orchestra.run.smithi136:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-18T10:17:36.491 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi154.front.sepia.ceph.com 2024-09-18T10:17:36.491 DEBUG:teuthology.orchestra.run.smithi154:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-18T10:17:36.507 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi189.front.sepia.ceph.com 2024-09-18T10:17:36.507 DEBUG:teuthology.orchestra.run.smithi189:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-18T10:17:36.522 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-18T10:17:36.523 DEBUG:teuthology.orchestra.run.smithi029:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-18T10:17:36.525 DEBUG:teuthology.orchestra.run.smithi136:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-18T10:17:36.535 DEBUG:teuthology.orchestra.run.smithi154:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-18T10:17:36.537 DEBUG:teuthology.orchestra.run.smithi189:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-18T10:17:36.574 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-18T10:17:36.574 DEBUG:teuthology.orchestra.run.smithi029:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-18T10:17:36.608 DEBUG:teuthology.orchestra.run.smithi136:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-18T10:17:36.610 DEBUG:teuthology.orchestra.run.smithi154:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-18T10:17:36.617 DEBUG:teuthology.orchestra.run.smithi189:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-18T10:17:36.690 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-18T10:17:36.700 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-18T10:17:36.701 DEBUG:teuthology.orchestra.run.smithi029:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-18T10:17:36.717 DEBUG:teuthology.orchestra.run.smithi136:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-18T10:17:36.734 DEBUG:teuthology.orchestra.run.smithi154:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-18T10:17:36.750 DEBUG:teuthology.orchestra.run.smithi189:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-18T10:17:36.768 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-18T10:17:36.777 DEBUG:teuthology.orchestra.run.smithi029:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-18T10:17:36.779 DEBUG:teuthology.orchestra.run.smithi136:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-18T10:17:36.781 DEBUG:teuthology.orchestra.run.smithi154:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-18T10:17:36.783 DEBUG:teuthology.orchestra.run.smithi189:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-18T10:17:36.791 INFO:teuthology.orchestra.run.smithi029.stdout:kernel.core_pattern = core 2024-09-18T10:17:36.794 INFO:teuthology.orchestra.run.smithi136.stdout:kernel.core_pattern = core 2024-09-18T10:17:36.796 INFO:teuthology.orchestra.run.smithi154.stdout:kernel.core_pattern = core 2024-09-18T10:17:36.824 INFO:teuthology.orchestra.run.smithi189.stdout:kernel.core_pattern = core 2024-09-18T10:17:36.852 DEBUG:teuthology.orchestra.run.smithi029:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-18T10:17:36.870 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-18T10:17:36.871 DEBUG:teuthology.orchestra.run.smithi136:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-18T10:17:36.876 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-18T10:17:36.877 DEBUG:teuthology.orchestra.run.smithi154:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-18T10:17:36.882 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-18T10:17:36.883 DEBUG:teuthology.orchestra.run.smithi189:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-18T10:17:36.888 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-18T10:17:36.889 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-18T10:17:36.898 INFO:teuthology.task.internal:Transferring archived files... 2024-09-18T10:17:36.899 DEBUG:teuthology.misc:Transferring archived files from smithi029:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/skanta-2024-09-18_09:13:00-rados-wip-bharath11-testing-2024-09-18-1115-quincy-distro-default-smithi/7910845/remote/smithi029 2024-09-18T10:17:36.899 DEBUG:teuthology.orchestra.run.smithi029:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-18T10:17:36.925 DEBUG:teuthology.misc:Transferring archived files from smithi136:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/skanta-2024-09-18_09:13:00-rados-wip-bharath11-testing-2024-09-18-1115-quincy-distro-default-smithi/7910845/remote/smithi136 2024-09-18T10:17:36.926 DEBUG:teuthology.orchestra.run.smithi136:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-18T10:17:36.952 DEBUG:teuthology.misc:Transferring archived files from smithi154:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/skanta-2024-09-18_09:13:00-rados-wip-bharath11-testing-2024-09-18-1115-quincy-distro-default-smithi/7910845/remote/smithi154 2024-09-18T10:17:36.952 DEBUG:teuthology.orchestra.run.smithi154:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-18T10:17:36.977 DEBUG:teuthology.misc:Transferring archived files from smithi189:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/skanta-2024-09-18_09:13:00-rados-wip-bharath11-testing-2024-09-18-1115-quincy-distro-default-smithi/7910845/remote/smithi189 2024-09-18T10:17:36.977 DEBUG:teuthology.orchestra.run.smithi189:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-18T10:17:37.004 INFO:teuthology.task.internal:Removing archive directory... 2024-09-18T10:17:37.004 DEBUG:teuthology.orchestra.run.smithi029:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-18T10:17:37.006 DEBUG:teuthology.orchestra.run.smithi136:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-18T10:17:37.008 DEBUG:teuthology.orchestra.run.smithi154:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-18T10:17:37.016 DEBUG:teuthology.orchestra.run.smithi189:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-18T10:17:37.047 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-18T10:17:37.058 INFO:teuthology.task.internal:Not uploading archives. 2024-09-18T10:17:37.058 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-18T10:17:37.068 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-18T10:17:37.068 DEBUG:teuthology.orchestra.run.smithi029:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-18T10:17:37.070 DEBUG:teuthology.orchestra.run.smithi136:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-18T10:17:37.073 DEBUG:teuthology.orchestra.run.smithi154:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-18T10:17:37.075 DEBUG:teuthology.orchestra.run.smithi189:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-18T10:17:37.076 INFO:teuthology.orchestra.run.smithi029.stdout: 924112 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 18 10:17 /home/ubuntu/cephtest 2024-09-18T10:17:37.078 INFO:teuthology.orchestra.run.smithi136.stdout: 924112 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 18 10:17 /home/ubuntu/cephtest 2024-09-18T10:17:37.081 INFO:teuthology.orchestra.run.smithi154.stdout: 924112 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 18 10:17 /home/ubuntu/cephtest 2024-09-18T10:17:37.083 INFO:teuthology.orchestra.run.smithi189.stdout: 924112 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 18 10:17 /home/ubuntu/cephtest 2024-09-18T10:17:37.085 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-18T10:17:37.095 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-18T10:17:37.116 INFO:teuthology.run:Summary data: description: rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/osd-dispatch-delay objectstore/bluestore-bitmap rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} duration: 746.9218552112579 flavor: default owner: scheduled_skanta@teuthology success: true 2024-09-18T10:17:37.117 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-18T10:17:37.197 INFO:teuthology.run:pass